Background and purpose: This study aimed to identify anatomically-localised regions where planned radiotherapy dose is associated with gastrointestinal toxicities in healthy tissues throughout the pelvic anatomy. Materials and methods: Planned dose distributions for up to 657 patients of the Trans Tasman Radiation Oncology Group 03.04 RADAR trial were deformably registered onto a single exemplar computed tomography dataset. Voxel-based multiple comparison permutation dose difference testing, Cox regression modelling and LASSO feature selection were used to identify regions where dose-increase was associated with grade ≥2 rectal bleeding (RB) or tenesmus, according to the LENT/SOMA scale. This was externally validated by registering dose distributions from the RT01 (n = 388) and CHHiP (n = 241) trials onto the same exemplar and repeating the tests on each of these data sets, and on all three datasets combined. Results: Voxel-based Cox regression and permutation dose difference testing revealed regions where increased dose was correlated with gastrointestinal toxicity. Grade ≥2 RB was associated with posteriorly extended lateral beams that manifested high doses (>55 Gy) in a small rectal volume adjacent to the clinical target volume. A correlation was found between grade ≥2 tenesmus and increased low-intermediate dose (∼25 Gy) at the posterior beam region, including the posterior rectum and perirectal fat space (PRFS). Conclusions: The serial response of the rectum with respect to RB has been demonstrated in patients with posteriorly extended lateral beams. Similarly, the parallel response of the PRFS with respect to tenesmus has been demonstrated in patients treated with the posterior beam.