Several approaches have been proposed to generate pseudo computed tomography (pCT) from MR images for radiotherapy dose calculation. Quantification of errors in pCT has been reported using global scores disregarding spatial heterogeneity. The aim of this work was to propose a population voxel-based workflow allowing the local assessment of errors in the generation of pCTs from MRI. For the voxel-wise analysis to be anatomically meaningful, a robust customized inter-patient non-rigid registration method brought the population images to the same coordinate system. To illustrate the use of this methodology, four pCT generation methods were compared: atlas-based, patch-based, and two deep learning methods. Considering global and local scores, deep learning appeared widely superior. Main source of errors were found in the cortical bones. The proposed workflow paves the way for quality control procedures within the clinical workflow.