Enhancing emotion classification through signal fusion and wavelet-based feature extraction
DOI:
https://doi.org/10.15407/fmmit2024.39.167Keywords:
Emotion recognition, Continuous Wavelet Transform, EEG, ECG, GSR, Multimodal Fusion, Deep Learning, Affective ComputingAbstract
Emotion recognition from physiological signals, such as EEG, ECG, and GSR, has shown promise
for various applications in affective computing. This study introduces a wavelet-based approach
that leverages continuous wavelet transform (CWT) to extract both image and numerical features
from multi-modal physiological data. The extracted features are utilized to train and compare
classification models, including a ResNet-50-based deep learning framework and traditional
machine learning models like support vector machines (SVMs). Our results demonstrate that the
combination of wavelet-based feature extraction and signal fusion significantly enhances emotion
classification performance. Notably, the numerical features derived from CWT achieve comparable
or superior accuracy to image-based features, offering insights into the effectiveness of different
feature representations. This work highlights the potential of wavelet-based methods for emotion
recognition and suggests pathways for future research in optimizing multi-modal signal processing
and classification.
References
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE transactions on affective computing, 3(1), 42-55.
Sukholeister, O., Nakonechnyi, A. (2022). Recognition of Mental Disorders from Physiological Signals Analysis. Measuring Equipment and Metrology, 83(4), 11-12
Sukholeister, O., Nakonechnyi, A. (2023). Strategies for Physiological Signal Fusion for Emotion Recognition. Measuring and Computing Devices in Technological Processes, Issue 4, 34-35.
Mohammadi, Z., Frounchi, J., & Amiri, M. (2016). Wavelet-based Emotion Recognition System Using EEG Signal. Neural Computing & Applications, 28, 1985-1990. https://doi.org/10.1007/s00521-015-2149-8
Tawsif, K., Aziz, N. A., Raja, J. E., & Hossen, J. (2022). A Systematic Review on Emotion Recognition System Using Physiological Signals: Data Acquisition and Methodology. Emerging Science Journal, 6(5), 1167-1168.
Wang, Y., Song, W., Tao, W., et al. (2022). A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances. Affective Computing Journal, 21, 1-4.
Ahmad, Z., & Khan, N. (2022). A Survey on Physiological Signal-Based Emotion Recognition. Bioengineering, 9, 688. https://doi.org/10.32920/22734278.v1
Dzedzickis, A., Kaklauskas, A., & Bucinskas, V. (2021). Human Emotion Recognition: Review of Sensors and Methods. Sensors, 20, 592. https://doi.org/10.3390/s20030592
Egger, M., Ley, M., & Hanke, S. (2021). Emotion Recognition from Physiological Signal Analysis: A Review. Electronic Notes in Theoretical Computer Science, 343, 35-55.
Addison, P. S. (2017) The Illustrated Wavelet Transform Handbook. 2nd edn. CRC Press. https://doi.org/10.1201/9781315372556
Bajaj, V., Pachori, R.B. (2015). Detection of Human Emotions Using Features Based on the Multiwavelet Transform of EEG Signals. In: Hassanien, A., Azar, A. (eds) Brain-Computer Interfaces. Intelligent Systems Reference Library, vol 74. Springer, Cham.
