Objectives: To contribute to our understanding of the drivers of body composition during adolescence we sought to employ valid and reliable measures to investigate cross-sectional and longitudinal relationships between percentage body fat (%BF) and physical activity (PA), moderate and vigorous PA (MVPA), sedentary time (ST), total energy, sugar and fat intake. Design: Longitudinal cohort study. Methods: We measured 556 (289 male) participants at age 12.4 (SD 0.4) years, and 269 (123 males) at 16.3 (SD 0.4) years, for %BF (dual energy X-ray absorptiometry); habitual PA, MVPA, ST (accelerometry); and dietary intake (‘multi-pass’ weekday and weekend 24-h recall). Accounting for likely under-reporting of energy intake (Goldberg cut-off), general linear mixed modelling was used to generate relationships with %BF. Results: Cross-sectional analyses indicated that 10 min more MVPA per day was associated with 0.6 lower %BF (95%CI 0.4–0.9, p < 0.001), and 10 min less ST/day with 0.07 lower %BF (95%CI 0.00–0.15, p < 0.001), independently of PA. In contrast, %BF was unrelated to total energy (p = 0.4), sugar intake (p = 0.2) or fat intake (p = 0.9). Longitudinal analysis showed that if PA was increased by 3% (10,000 counts/day) over the 4 years, then %BF was reduced by 0.08 (95%CI 0.05–0.12, p = 0.06). Conclusions: The independent relationships of %BF with PA and ST, but absence of relationships with energy, sugar or fat intake, suggest that general community campaigns in a developed country directed at reducing adolescent obesity through modifications to energy intake and output would benefit from a more concerted focus on the latter.