Applied multimodal for emotion recognition
Corressponding author's email:
ntkhong@ute.edu.vnKeywords:
Emotion Recognition, physical – bio sensor, machine learning, CNN, SVMAbstract
Detecting and classifying emotions has currently become an important item of research and life. The more detailed and accurate emotion recognition system is due to the development of various fields such as electronics, sensors or computer engineering. Emotion recognition methods are studied using different data collection methods and one of the most popular and effective methods is physical – bio sensors. Physical – bio sensor based approaches can provide more accurate, sustainable biological information with external influences and interferences, especially when we compared with other approaches such as image processing, video processing. In this paper, a method of classifying and assessing emotions based on a combination of signals collected from physical – bio sensors, video collection and machine learning are supposed. Specifically, we will describe the platform of a physical – bio signal collection system, the process of collecting information and the information processing system used to identify how emotional behavior is characterized. We have also shown that a combination modals of physical – bio sensor acquisition systems, video processing based on machine learning methods can provide identification performance with an accuracy of 83.2%
Downloads: 0
References
Rangaraj M. Rangayyan, Biomedical Signal Analysis – A Case-Study Approach, IEEE Press 2002.
Barreto A., Heimer M., and Garcia M., Characterization of Photoplehtysmographic Blood Volume Pulse Waveforms for Exercise Evalution, Proceedings 14th Southern Biomedical Engineering Conference, Shreveport, Louisiana, April, 1995, pp. 220-223.
Christie, Israel C, Multivariate Discrimination of Emotion-specific Autonomic Nervous System Activity, Master Thesis in Science of Psychology, Virginia.
Picard R.W., Toward computers that recognize and respond to user emotion, IBM Systems Journal; Vol 39, Nos 3&4, 2000.
Healy J. and Picard R., Digital processing of Affective Signals, ICASSP 98.
Cowie R., Describing the emotional states expressed in speech, ISCA workshop on speech and emotion, Belfast 2000.
Juang B.H & Soong F.K., Hands-free Telecommunications, HSC 2001, pp.5-10; Kyoto, Japan.
Pentland A., Perceptual Intelligence, Communications of the ACM; Volume 43, Number 3 (2000), Pages 35-44.
E. Monte-Moreno, M. Chetouani, M. Faundez-Zanuy and J. SoleCasals, Maximum likelihood linear programming data fusion for speaker recognition, Speech Communication, 51(9):820–830, 2009. 68.
A. Mahdhaoui and M. Chetouani, Emotional speech classification based on multi view characterization, IAPR International Conference on Pattern Recognition, ICPR, 2010. 51.
Ammar Mahdhaoui, Analyse de Signaux Sociaux pour la Modélisation de l'interaction face à face. Traitement du signal et de l'image, Université Pierre et Marie Curie - Paris VI, 2010. French. .
Sebe, Nicu, et al, Bimodal emotion recognition. Proceedings of the 5th International Conference on Methods and Techniques in Behavioral Research. 2005.
Caifeng Shan, Shaogang Gong, Peter W. McOwan, Facial expression recognition based on Local Binary Patterns: A comprehensive study, Image and Vision Computing 27 (2009) 803–816.
E. Monte-Moreno, M. Chetouani, M. Faundez-Zanuy et J. SoleCasals, Maximum likelihood linear programming data fusion for speaker recognition, Speech Communication, 51(9):820–830, 2009. 68.
Hamza Hamdi, Plate-forme multimodale pour la reconnaissance d’émotions via l’analyse de signaux physiologiques: Application à la simulation d’entretiens d’embauche, Modeling and Simulation. Université d’Angers, 2012. French. .
R. Sharma, V.I. Pavlovic, and T.S. Huang, Toward multimodal human-computer interface. Proceedings of the IEEE, 86(5):853–869, 1998. 29, 30, 32, 167.
Kaipeng Z., Zhanpeng Z., Zhifeng L., Yu Q., Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks, IEEE Signal Processing Letters, vol. 23, pp. 1499-1503 (2019).
Vahid K., Josephine S., One Millisecond Face Alignment with an Ensemble of Regression Trees, CVPR IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867-1874 (2014).
Chen, Jun-Cheng, Vishal M. Patel, and Rama Chellappa., Unconstrained face verification using deep cnn features. IEEE winter conference on applications of computer vision (WACV). IEEE, 2016.
Hsu C.W., Lin C.J., A comparison of methods for multiclass support vector machines, IEEE Trans Neural Network 13(2):415–425 (2002).
Imen Tayari Meftah, Modélisation, détection et annotation des états émotionnels à l’aide d’un espace vectoriel multidimensionnel, Artificial Intelligence, Université Nice Sophia Antipolis, 2013. French. . .
Koné C., Tayari I.M., Le-Thanh N., Belleudy C., Multimodal Recognition of Emotions Using Physiological Signals with the Method of Decision-Level Fusion, Healthcare Applications. Inclusive Smart Cities and e-Health. ICOST 2015. Lecture Notes in Computer Science, vol 9102. Springer.
Santamaria-Granados, Luz, et al., Using deep convolutional neural network for emotion detection on a physiological signals dataset (AMIGOS). IEEE Access 7 (2018): 57-67.
Downloads
Published
How to Cite
Issue
Section
Categories
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright © JTE.
