Multimodal Segmentation Based on a Novel 3D U-Net Deep Learning Architecture

In this paper, we propose a new approach for brain image segmentation based on a novel 3D U-Net deep fusion scheme. The proposed approach takes into consideration a fusion of multiple scan modalities including FLAIR, T1, T1Gd and T2, and by using a stacked CNN based 3D U-Net architecture allows modelling of multiclass segmentation of Gliomas, an aggressive form of brain tumours. The proposed model performs well for low resource settings, and requires lesser resource requirements, and with imbalanced class distribution, and natural data augmentation, by transforming 3D volumes to 2D sequences. An extensive quantitative and qualitative experimental evaluation of the proposed model in terms of dice score and dice loss performance metrics, for two publicly available datasets, corresponding to 2018 BraTS and 2021 BraTS challenge segmentation task, shows improved performance and generalization capability of the proposed lightweight model.