Background: Multiparous women are at a higher risk of gestational diabetes mellitus (GDM) than primiparas. Physical activity during pregnancy has been shown to be beneficial for GDM, but there is little evidence on the association between physical activity and glycemic control among women with GDM, whether primiparas or multiparas. Thus, the objective of the present study was to examine the association between physical activity and glycemic control in women with GDM and to determine what, if any, effects result from number of parous events. Methods: A prospective cohort of 1162 women with GDM was recruited, with 604 multiparas (51.98%). The general linear model was used to calculate the risk difference and its 95% confidence interval (95%CI) to quantify the impact of parous events on glycemic control in pregnancy as well as the association between physical activity time and glycemic control. Results: Among 1162 women with GDM, the median daily activity time was 65 min (interquartile range (IQR): 45–90 min), and the abnormal plasma glucose (PG) percentage, calculated as number of abnormal PG tests divided by the total number of PG tests, was 40.00% (IQR: 22.22%–66.67%). The percentage of abnormal PG was stabilized and statistically lower with daily physical activity time exceeding 60 min among primiparas (IQR: 30.89%–44.43%) and exceeding 90 min among multiparas (ranged from 27.76% to 38.82%). After adjusting for potential confounders, primiparas tended to have a lower percentage of abnormal PG than do multiparas (rate difference = –0.39, 95%CI: –3.61 to 2.84). The same amount of physical activity time was significantly less effective for multiparas than for primiparas (trend p-value < 0.01). Conclusion: In women with GDM, being multiparous is associated with less effective glycemic control through physical activity, such that multiparas need more physical activity to achieve glycemic control at a similar level to primiparas.