Acute Pain Recognition from Facial Expression Videos using Vision Transformers

Pain assessment is significant for patients and clinicians in diagnosis and treatment injuries and disease. It could facilitate a patient’s treatment process by monitoring patients' pain levels in an accurate and regular manner. Automated detection of pain from facial expressions is a useful technique to assess pain of patients with communication disabilities. In this study, video vision transformers (ViViT) enhanced for pain recognition tasks are presented to capture spatio-temporal, facial information relevant to estimating the binary classification of pain and, thus, to provide valuable insights for automated estimation. The developed model has been trained and evaluated on two acute pain datasets, including 51 subjects using a newly collected pain intensity dataset designated as the AI4PAIN Challenge dataset, and 87 subjects from the BioVid Pain dataset. As an ablation study we used two baseline models, ResNet50 and a hybrid deep learning model based on the pretrained ResNet50+3DCNN. The results demonstrated that the proposed ViViT outperform the other models in pain detection by achieving accuracy = 66.96% for AI4PAIN dataset and accuracy = 79.95% for BioVid dataset.