Semi-arid rangelands have strong feedbacks between vegetation and abiotic factors (rainfall and soil),which manifest at the small patch/inter-patch scale. The effects of excluding grazing at the small patchscale have not been studied in these systems, despite fine-grained patch/inter-patch mosaics deter-mining landscape-scale ecosystem function and productivity. We established small (1 m × 1 m) grazingexclosures and monitored change in herbaceous vegetation cover quarterly over 2 years, in grazedand ungrazed patches (high herbaceous vegetation cover) and inter-patches (low herbaceous vegeta-tion cover), in woody encroachment and pasture sites in semi-arid eastern Australia. Prior to excludinggrazing, herbaceous groundcover in pasture patches (63–67%) was significantly greater than in woodyencroachment patches (15–16%) or inter-patches in both vegetation states (0–1%). The effect of grazingexclusion on herbaceous cover varied between patch type and vegetation state. In the absence of grazing,herbaceous cover was significantly greater in pasture patches than in woody encroachment patches atevery monitoring time. Initial differences in herbaceous cover between woody encroachment patches andpasture patches was significantly less pronounced with continued grazing pressure, indicating that graz-ing pressure can negatively influence the positive effect of a lack of woody plants on herbaceous growth inpastures. Grazed pasture patches had significantly less herbaceous cover than ungrazed pasture patchesat every monitoring time, whereas in woody encroachment, grazed patches had less herbaceous coverthan ungrazed patches on only two occasions in the second year. Inter-patches in both vegetation statesfailed to respond to grazing exclusion in the 2-year study period. Herbaceous cover change in semi-aridrangelands is a function of grazing, rainfall and woody plant incidence. If grazing pressure in pasturepatches is not carefully managed to maintain herbaceous cover, the positive effects of high rainfall andlow woody plant abundance on herbaceous cover can rapidly diminish. Improving herbaceous ground-cover overall will require an interventionist approach to overcome barriers to plant establishment ininter-patch areas such as the lack of resource retention.