Pain intensity estimation via multimodal fusion: Leveraging ternary textures of derivatives in EDA and PPG signals

In the event of pain, the autonomic nervous system reacts by affecting different physiological parameters such as blood pressure, heart rate, skin conductance, and perspiration levels, among others. This research presents an innovative approach to pain intensity recognition through a multimodal system that fuses bio-information from the skin (Electrodermal Activity or EDA) and heart (Photoplethysmograph or PPG) signals. The study involved a self-collected dataset from 61 healthy participants and encompassed two pain intensity levels (low and high) experienced at different anatomical locations (hand and forearm). Employing IIR bandpass filters, the collected EDA and PPG signals were preprocessed. A novel feature extraction method named Ternary Textures of Derivatives (TTD) is proposed, which, when fused with statistical features, exhibited robust potential as a pain intensity biomarker. Feature selection using Joint Mutual Information preceded the utilisation of an Ensemble classifier. The developed multimodal fusion-based pain recognition system outperformed the unimodal (PPG and EDA) approaches by achieving notable accuracies: 83.1%±8.8% for No Pain vs. Low Pain, 87.1%±6.7% for No Pain vs. High Pain, and 74.5%±6.8% for the No Pain vs. Low Pain vs. High Pain scenario. This approach offers an objective means of pain assessment that can furnish valuable insights to clinical teams, aiding in treatment evaluation, surgical decision-making, and overall patient care quality assessment.