Algorithms used for facial emotion recognition: a systematic review of the literature

Authors

DOI:

https://doi.org/10.4108/eetpht.9.4214

Keywords:

Facial emotion, computer vision, Deep Learning, Machine Learning, Algorithm

Abstract

INTRODUCTION: We currently live in a society that is constantly changing and technology has developed algorithms that allow facial emotion recognition, because facial expression transmits people's mood, feelings and state of soul. However, it is required that future research can improve the quality of emotion detection by improving the quality of the data set and the model used, for this reason, it is necessary to investigate other machine learning algorithms in the recognition of facial emotions, as they exist. identification deficiencies that limit the discrimination of extracted structural features.

OBJECTIVE: The purpose of the article was to analyze the most used algorithms for facial emotion recognition, through a systematic literature review, according to the PRISMA method.

METHOD: A search for information was carried out in articles published in open access such as: Scopus, Web of Science (WOS) and Association for Computing Machiner (ACM) in the period 2022 and 2023, totaling 38 selected articles.

RESULTS: The results obtained indicate that the algorithms most used by the authors are SVM and SoftMax with a total of 17.65% each.

CONCLUSION: It is concluded that the SVM and SoftMax algorithms are the most predominant, playing a crucial role in achieving optimal levels of precision in the training of the models. These algorithms, with their robustness and ability to deal with complex data, have proven to be fundamental pillars in the field of facial emotion recognition.

Downloads

Download data is not yet available.

References

Benitez-Quiroz CF, Srinivasan R, Martinez AM. Facial color is an efficient mechanism to visually transmit emotion. Proc Natl Acad Sci USA. 2018;115(14):3581-3586. doi:10.1073/PNAS.1716084115 DOI: https://doi.org/10.1073/pnas.1716084115

Huang Z, Chiang C, Chen J, Chen Y, Chung H, Cai Y, Hsu H. A study on computer vision for facial emotion recognition. Sci Rep. 2023;13(1). doi:10.1038/s41598-023-35446-4 DOI: https://doi.org/10.1038/s41598-023-35446-4

Gokani J. The Evolution of Banking: AI. Stanford University MS&E 238 Blog. Published August 4, 2017. https://mse238blog.stanford.edu/2017/08/jgokani/the-evolution-of-banking-ai/

Marcos JM, Gallego RE, De Alda JAGO. The interplay of prior knowledge, emotions and learning in a science experiment activity. [Conocimiento previo, emociones y aprendizaje en una actividad experimental de ciencias] Enseñanza De Las Ciencias. 2022;40(1):107-124. doi:10.5565/rev/ensciencias.3361 DOI: https://doi.org/10.5565/rev/ensciencias.3361

Ferron LM. Jumping the Gap: developing an innovative product from a Social Network Analysis perspective. AWARI 2021;2:e026-e026. https://doi.org/10.47909/awari.128 DOI: https://doi.org/10.47909/awari.128

Cáceres YMM. Management of pain reduction in mechanically ventilated care subjects. Interdisciplinary Rehabilitation / Rehabilitacion Interdisciplinaria 2023;3:59-59. https://doi.org/10.56294/ri202359 DOI: https://doi.org/10.56294/ri202359

Camargo JL, Baca LDH, Valencia ET, Aquino REA, Miranda AGR, Camargo LGL. Facial recognition proposal with the use of python. Paper presented at the Iberian Conference on Information Systems and Technologies, CISTI, 2022-June. doi:10.23919/CISTI54924.2022.9819984 DOI: https://doi.org/10.23919/CISTI54924.2022.9819984

Buongiorno M, Vaucheret E, Giacchino M, Mayoni P, Polin A, Pardo-Campos M. Facial emotion recognition in children with attention-deficit/hyperactivity disorder. [Reconocimiento de emociones faciales en niños con trastorno por déficit de atención/hiperactividad] Rev Neurol. 2020;70(4):127-133. doi:10.33588/rn.7004.2019268 DOI: https://doi.org/10.33588/rn.7004.2019268

Simhan L, Basupi G. None Deep Learning Based Analysis of Student Aptitude for Programming at College Freshman Level. Data and Metadata 2023;2:38-38. https://doi.org/10.56294/dm202338 DOI: https://doi.org/10.56294/dm202338

Zeng R, et al. CNN-Based Broad Learning for Cross-Domain Emotion Classification. Tsinghua Sci Technol. 2023;28(2):360-369. doi:10.26599/TST.2022.9010007 DOI: https://doi.org/10.26599/TST.2022.9010007

Ujjappanahalli KS, Sonawane VR, Gandhewar N. Novedosa optimización de algoritmos híbridos de selección de características para la técnica de clasificación de imágenes mediante RBFNN y MFO. Salud, Ciencia y Tecnología 2022;2:241-241. https://doi.org/10.56294/saludcyt2022241 DOI: https://doi.org/10.56294/saludcyt2022241

Kakuba S, Poulose A, Han DS. Deep Learning-Based Speech Emotion Recognition Using Multi-Level Fusion of Concurrent Features. IEEE Access. 2022;10:125538-125551. doi:10.1109/ACCESS.2022.3225684 DOI: https://doi.org/10.1109/ACCESS.2022.3225684

Coutinho KR. Digital humanities project proposal: Clipping of online and printed journals on education and institutes of education, science, and technology. Advanced Notes in Information Science 2023;3:137-55. https://doi.org/10.47909/anis.978-9916-9906-1-2.42 DOI: https://doi.org/10.47909/anis.978-9916-9906-1-2.42

Fernández CPP, Valencia JGB. Case study of the narrative displays of the self of a young Paralympic athlete: signifying the place of the body and technology from the visualization of narrative folds graphs. AWARI 2020;1:e020-e020. https://doi.org/10.47909/awari.81 DOI: https://doi.org/10.47909/awari.81

Diez RCÁ, Esparza RMV, Bañuelos-García VH, Santillán MTV, Félix BIL, Luna VA, et al. Economía plateada y emprendimiento, un área innovadora de futuro: Un marco de referencia académico, científico y empresarial para la construcción de nuevos conocimientos. Iberoamerican Journal of Science Measurement and Communication 2022;2. https://doi.org/10.47909/ijsmc.45 DOI: https://doi.org/10.47909/ijsmc.45

Kumar VP. Towards trainable man-machine interfaces: combining top-down constraints with bottom-up learning in facial analysis [Doctoral dissertation, Massachusetts Institute of Technology]. https://dspace.mit.edu/handle/1721.1/29243

Cirulli A, Godoy A. Gender, transsexuality and labor insertion. Community and Interculturality in Dialogue 2022;2:28-28. https://doi.org/10.56294/cid202228 DOI: https://doi.org/10.56294/cid202228

Dar T, Javed A, Bourouis S, Hussein HS, Alshazly H. Efficient-SwishNet Based System for Facial Emotion Recognition. IEEE Access. 2022;10:71311-71328. doi:10.1109/ACCESS.2022.3188730 DOI: https://doi.org/10.1109/ACCESS.2022.3188730

Sánchez CMC, León LAG, Yanes RCA, Oloriz MAG. Metaverse: the future of medicine in a virtual world. Metaverse Basic and Applied Research 2022;1:4-4. https://doi.org/10.56294/mr20224 DOI: https://doi.org/10.56294/mr20224

Mancilla Monsalve JL. Uso de patrones de reconocimiento de las emociones para apoyar la didáctica de enseñanza aprendizaje. Dictamen Libre. 2019;14(24):15-42. doi:10.18041/2619-4244/dl.24.5463 DOI: https://doi.org/10.18041/2619-4244/dl.24.5463

Parra AL, Escalona E, Gollo O. Estudio piloto comparativo de medidas antropométricas en bipedestación entre Tablas antropométricas y un Antropómetro Harpenden. Interdisciplinary Rehabilitation / Rehabilitacion Interdisciplinaria 2023;3:48-48. https://doi.org/10.56294/ri202348 DOI: https://doi.org/10.56294/ri202348

Artanto H, Arifin F. Emotions and gesture recognition using affective computing assessment with deep learning. https://www.researchgate.net/publication/373581660_Emotions_and_gesture_recognition_using_affective_computing_assessment_with_deep_learning

Alsemawi MRM, Mutar MH, Ahmed EH, Hanoosh HO, Abbas AH. Emotions recognition from human facial images based on fast learning network. Indonesian Journal of Electrical Engineering and Computer Science. 2023;30(3):1478-1487. doi:10.11591/ijeecs.v30.i3.pp1478-1487 DOI: https://doi.org/10.11591/ijeecs.v30.i3.pp1478-1487

Martins DL. Data science teaching and learning models: focus on the Information Science area. Advanced Notes in Information Science 2022;2:140-8. https://doi.org/10.47909/anis.978-9916-9760-3-6.100 DOI: https://doi.org/10.47909/anis.978-9916-9760-3-6.100

Hao M, Yuan F, Li J, Sun Y. Facial expression recognition based on regional adaptive correlation. IET Computer Vision. 2023;17(4):445-460. doi:10.1049/cvi2.12179 DOI: https://doi.org/10.1049/cvi2.12179

Soto IBR, Leon NSS. How artificial intelligence will shape the future of metaverse. A qualitative perspective. Metaverse Basic and Applied Research 2022;1:12-12. https://doi.org/10.56294/mr202212 DOI: https://doi.org/10.56294/mr202212

Costa W, Talavera E, Oliveira R, Figueiredo L, Teixeira JM, Lima JP, Teichrieb V. A survey on datasets for emotion recognition from vision: Limitations and in-the-wild applicability. Appl Sci (Switzerland). 2023;13(9). doi:10.3390/app13095697 DOI: https://doi.org/10.3390/app13095697

Darwin C. The Expression of Emotion in Man and Animals. Project Gutenberg. https://www.gutenberg.org/files/1227/1227-h/1227-h.htm Published 1899.

Florentin GNB. The human dimension in nursing. An approach according to Watson’s Theory. Community and Interculturality in Dialogue 2023;3:68-68. https://doi.org/10.56294/cid202368 DOI: https://doi.org/10.56294/cid202368

Valderrama B. Emociones: una taxonomía para el Desarrollo Emocional. Arandu UTIC. https://www.utic.edu.py/revista.ojs/index.php/revistas/article/view/14 Published 2021.

Elsayed Y, Elsayed A, Abdou MA. An automatic improved facial expression recognition for masked faces. Neural Comput Appl. 2023;35(20):14963-14972. doi:10.1007/s00521-023-08498-w DOI: https://doi.org/10.1007/s00521-023-08498-w

Mohammed AF, Nahi HA, Mosa AM, Kadhim I. Secure E-healthcare System Based on Biometric Approach. Data and Metadata 2023;2:56-56. https://doi.org/10.56294/dm202356 DOI: https://doi.org/10.56294/dm202356

Goleman B. Psicología Oscura 6 libros en 1: Introducción a la Psicología, Como analizar a las Personas, Manipulación, Persuasión, Secretos de la Psicología Oscura, Inteligencia Emocional y TCC, Abuso Emocional y Narcisista. Amazon. https://psicologiaymente.com/biografias/daniel-goleman Published 2021.

Matos J. Un curso de emociones. 1st ed. Ediciones Urano S.A.U.; 2020.

Samoili S, Lopez Cobo M, Gomez Gutierrez E, De Prato G, Martinez-Plumed F, Delipetrev B. AI WATCH. Defining Artificial Intelligence. European Commission. 2020. doi:10.2760/382730

Benito PV. Contemporary art and networks: Analysis of the Venus Project using the UCINET software. AWARI 2022;3. https://doi.org/10.47909/awari.166 DOI: https://doi.org/10.47909/awari.166

Alsharekh MF. Facial Emotion Recognition in Verbal Communication Based on Deep Learning. Sensors. 2022;22(16). doi:10.3390/S22166105 DOI: https://doi.org/10.3390/s22166105

Chaudhari A, Bhatt C, Krishna A, Mazzeo PL. ViTFER: Facial Emotion Recognition with Vision Transformers. Appl Syst Innov. 2022;5(4). doi:10.3390/ASI5040080 DOI: https://doi.org/10.3390/asi5040080

Andrade-Girón D, Carreño-Cisneros E, Mejía-Dominguez C, Marín-Rodriguez W, Villarreal-Torres H. Comparación de Algoritmos Machine Learning para la Predicción de Pacientes con Sospecha de COVID-19. Salud, Ciencia y Tecnología 2023;3:336-336. https://doi.org/10.56294/saludcyt2023336. DOI: https://doi.org/10.56294/saludcyt2023336

Pancholi BK, Modi PS, Chitaliya NG. Un nuevo algoritmo multiumbral para la segmentación de imágenes de resonancia magnética. Salud, Ciencia y Tecnología 2023;3:408-408. https://doi.org/10.56294/saludcyt2023408. DOI: https://doi.org/10.56294/saludcyt2023408

Bartual González R, Ignacio J, Herruzo H, Sebastia JP. Detección facial y reconocimiento anímico mediante las expresiones faciales. https://riunet.upv.es/handle/10251/85959. Publicado en 2017.

Silva LF da, Padilha RC. Digital technologies as potentiating tools in the dissemination of information in museum spaces: Impact of the Covid-19 pandemic on museums. Advanced Notes in Information Science 2023;3:156-84. https://doi.org/10.47909/anis.978-9916-9906-1-2.41 DOI: https://doi.org/10.47909/anis.978-9916-9906-1-2.41

Merchán F, Galeano S, Poveda H. Mejoras en el Entrenamiento de Esquemas de Detección de Sonrisas Basados en AdaBoost. I+D Tecnológico. 2014;10(2):17-30. https://revistas.utp.ac.pa/index.php/id-tecnologico/article/view/21/html

Liang X, Liang J, Yin T, Tang X. A lightweight method for face expression recognition based on improved MobileNetV3. IET Image Processing. 2023;17(8):2375-2384. doi:10.1049/ipr2.12798 DOI: https://doi.org/10.1049/ipr2.12798

Mustafa Hilal A, Elkamchouchi DH, Alotaibi SS, Maray M, Othman M, Abdelmageed AA, Zamani AS, Eldesouki MI. Manta Ray Foraging Optimization with Transfer Learning Driven Facial Emotion Recognition. Sustainability. 2022;14(21). doi:10.3390/SU142114308 DOI: https://doi.org/10.3390/su142114308

Araujo-Inastrilla CR, Vitón-Castillo AA. Blockchain in health sciences: Research trends in Scopus. Iberoamerican Journal of Science Measurement and Communication 2023;3. https://doi.org/10.47909/ijsmc.56 DOI: https://doi.org/10.47909/ijsmc.56

Kit NC, Ooi C, Tan W, Tan Y, Cheong S. Facial emotion recognition using deep learning detector and classifier. Int J Electr Comput Eng. 2023;13(3):3375-3383. doi:10.11591/ijece.v13i3.pp3375-3383 DOI: https://doi.org/10.11591/ijece.v13i3.pp3375-3383

Gonzalez-Argote J. Uso de la realidad virtual en la rehabilitación. Interdisciplinary Rehabilitation / Rehabilitacion Interdisciplinaria 2022;2:24-24. https://doi.org/10.56294/ri202224 DOI: https://doi.org/10.56294/ri202224

Fernández-Ríos M, Redolat R, Serra E, González-Alcaide G. Una revisión sistemática acerca del reconocimiento facial de las emociones en la Enfermedad de Alzheimer: una perspectiva evolutiva y de género. Anales de Psicología / Annals of Psychology. 2021;37(3):478-492. doi:10.6018/ANALESPS.439141 DOI: https://doi.org/10.6018/analesps.439141

Nascimento PVB do, Araújo GMD. Requirement of telematic data in Brazilian criminal investigation: Diagnosis, process flow and chain of custody supported by blockchain technology. Advanced Notes in Information Science 2023;4. https://doi.org/10.47909/anis DOI: https://doi.org/10.47909/anis.20978-9916-9906-3-6.65

Assiri B, Hossain MA. Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism. Math Biosci Eng. 2023;20(1):913-929. doi:10.3934/MBE.2023042 DOI: https://doi.org/10.3934/mbe.2023042

Montano M de las NV, Álvarez MK. The educational and pedagogical intervention in scientific research. Community and Interculturality in Dialogue 2023;3:70-70. https://doi.org/10.56294/cid202370 DOI: https://doi.org/10.56294/cid202370

Lozano E, Directores M, María, López T, Antonio B, Caballero F. Detección facial de emociones orientada a mejorar la calidad de vida y cuidado de personas mayores en ambientes inteligentes. Nature. https://doi.org/10.1038/NATURE.2012.9872. DOI: https://doi.org/10.1038/nature.2012.9872

Elsayed Y, Elsayed A, Abdou MA. An automatic improved facial expression recognition for masked faces. Neural Comput Appl. 2023;35(20). doi:10.1007/s00521-023-08498-w DOI: https://doi.org/10.1007/s00521-023-08498-w

Rivas LM, Cruz LM. Revisión de ensayos clínicos sobre la eficacia de la rehabilitación cognitiva en pacientes con lesión cerebral traumática. Interdisciplinary Rehabilitation / Rehabilitacion Interdisciplinaria 2022;2:25-25. https://doi.org/10.56294/ri202225 DOI: https://doi.org/10.56294/ri202225

Haider I, Yang H, Lee G, Kim S. Robust human face emotion classification using triplet-loss-based deep CNN features and SVM. Sensors. 2023;23(10). doi:10.3390/s23104770 DOI: https://doi.org/10.3390/s23104770

Chatterjee S, Das AK, Nayak J, Pelusi D. Improving Facial Emotion Recognition Using Residual Autoencoder Coupled Affinity Based Overlapping Reduction. Mathematics. 2022;10(3). doi:10.3390/MATH10030406 DOI: https://doi.org/10.3390/math10030406

Gupta S, Kumar P, Tekchandani RK. Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models. Multimedia Tools and Applications. 2023;82(8):11365-11394. doi:10.1007/S11042-022-13558-9 DOI: https://doi.org/10.1007/s11042-022-13558-9

Chandran R. Human-Computer Interaction in Robotics: A bibliometric evaluation using Web of Science. Metaverse Basic and Applied Research 2022;1:22-22. https://doi.org/10.56294/mr202222 DOI: https://doi.org/10.56294/mr202222

Kim S, An BS, Lee EC. Comparative analysis of AI-based facial identification and expression recognition using upper and lower facial regions. Appl Sci. 2023;13(10). doi:10.3390/app13106070 DOI: https://doi.org/10.3390/app13106070

Ran Y, Zheng W, Zong Y, Liu J. Adaptive spatio-temporal attention neural network for cross-database micro-expression recognition. Virtual Reality and Intelligent Hardware. 2023;5(2):142-156. doi:10.1016/j.vrih.2022.03.006 DOI: https://doi.org/10.1016/j.vrih.2022.03.006

Tiwari P, Chaudhary S, Majhi D, Mukherjee B. Comparing research trends through author-provided keywords with machine extracted terms: A ML algorithm approach using publications data on neurological disorders. Iberoamerican Journal of Science Measurement and Communication 2023;3. https://doi.org/10.47909/ijsmc.36 DOI: https://doi.org/10.47909/ijsmc.36

Thwaini MH. Anomaly Detection in Network Traffic using Machine Learning for Early Threat Detection. Data and Metadata 2022;1:34-34. https://doi.org/10.56294/dm202272 DOI: https://doi.org/10.56294/dm202272

Dudekula U, Purnachand N. Analysis of facial emotion recognition rate for real-time application using NVIDIA jetson nano in deep learning models. Indonesian Journal of Electrical Engineering and Computer Science. 2023;30(1):598-605. doi:10.11591/ijeecs.v30.i1.pp598-605 DOI: https://doi.org/10.11591/ijeecs.v30.i1.pp598-605

Won H, Heo YS, Kwak N. Image recommendation system based on environmental and human face information. Sensors. 2023;23(11). doi:10.3390/s23115304 DOI: https://doi.org/10.3390/s23115304

Shahzad HM, Bhatti SM, Jaffar A, Akram S, Alhajlah M, Mahmood A. Hybrid facial emotion recognition using CNN-based features. Appl Sci (Switzerland). 2023;13(9). doi:10.3390/app13095572 DOI: https://doi.org/10.3390/app13095572

Zaina RZ, Ramos VFC, Araujo GM de. Automated triage of financial intelligence reports. Advanced Notes in Information Science 2022;2:24-33. https://doi.org/10.47909/anis.978-9916-9760-3-6.115 DOI: https://doi.org/10.47909/anis.978-9916-9760-3-6.115

Rogers T, Al Madi N. On the pursuit of developer happiness: Webcam-based eye tracking and affect recognition in the IDE. Paper presented at the Eye Tracking Research and Applications Symposium (ETRA). doi:10.1145/3588015.3590129 DOI: https://doi.org/10.1145/3588015.3590129

Telmo F de A, Autran M de MM, Silva AKA da. Scientific production on open science in Information Science: a study based on the ENANCIB event. AWARI 2021;2:e027-e027. https://doi.org/10.47909/awari.127 DOI: https://doi.org/10.47909/awari.127

Singh R, Saurav S, Kumar T, Saini R, Vohra A, Singh S. Facial expression recognition in videos using hybrid CNN & ConvLSTM. Int J Inf Technol (Singapore). 2023;15(4):1819-1830. doi:10.1007/s41870-023-01183-0 DOI: https://doi.org/10.1007/s41870-023-01183-0

Han B, Hu M. The facial expression data enhancement method induced by improved StarGAN V2. Symmetry. 2023;15(4). doi:10.3390/sym15040956 DOI: https://doi.org/10.3390/sym15040956

Calcagno MRF. Independent care performed by nursing professionals in the prevention of delirium. Interdisciplinary Rehabilitation / Rehabilitacion Interdisciplinaria 2023;3:55-55. https://doi.org/10.56294/ri202355 DOI: https://doi.org/10.56294/ri202355

Pauli R, Kohls G, Tino P, Rogers JC, Baumann S, Ackermann K, De Brito SA. Machine learning classification of conduct disorder with high versus low levels of callous-unemotional traits based on facial emotion recognition abilities. Eur Child Adolesc Psychiatry. 2023;32(4):589-600. doi:10.1007/s00787-021-01893-5 DOI: https://doi.org/10.1007/s00787-021-01893-5

Montano M de las NV, Martínez M de la CG, Lemus LP. Rehabilitation of occupational stress from the perspective of Health Education. Community and Interculturality in Dialogue 2023;3:71-71. https://doi.org/10.56294/cid202371 DOI: https://doi.org/10.56294/cid202371

Ibraheem IK. Enhancing Intrusion Detection Systems using Ensemble Machine Learning Techniques. Data and Metadata 2022;1:33-33. https://doi.org/10.56294/dm202271 DOI: https://doi.org/10.56294/dm202271

Aznarte JL, Pardos MM, Lacruz López JM. On the use of facial recognition technologies in university: The UNED case. RIED-Rev Iberoam Educ Dist. 2022;25(1):261-277. doi:10.5944/ried.25.1.31533 DOI: https://doi.org/10.5944/ried.25.1.31533

Khan AR. Facial Emotion Recognition Using Conventional Machine Learning and Deep Learning Methods: Current Achievements, Analysis and Remaining Challenges. Information (Switzerland). 2022;13(6). doi:10.3390/INFO13060268 DOI: https://doi.org/10.3390/info13060268

Li Z, Zhang Y, Xing H, Chan K. Facial micro-expression recognition using double-stream 3D convolutional neural network with domain adaptation. Sensors. 2023;23(7). doi:10.3390/s23073577 DOI: https://doi.org/10.3390/s23073577

Gupta B. Understanding Blockchain Technology: How It Works and What It Can Do. Metaverse Basic and Applied Research 2022;1:18-18. https://doi.org/10.56294/mr202218 DOI: https://doi.org/10.56294/mr202218

Martínez J, Vega J. ROS System Facial Emotion Detection Using Machine Learning for a Low-Cost Robot Based on Raspberry Pi. Electronics (Switzerland). 2023;12(1). doi:10.3390/ELECTRONICS12010090 DOI: https://doi.org/10.3390/electronics12010090

Talaat FM. Real-time facial emotion recognition system among children with autism based on deep learning and IoT. Neural Comput Appl. 2023. doi:10.1007/S00521-023-08372-9 DOI: https://doi.org/10.1007/s00521-023-08372-9

Downloads

Published

24-10-2023

How to Cite

1.
Ubillús JAT, Quispe JAH, Escriba LAR, Ladera-Castañeda M, Pacherres CAA, Pacherres M Ángel A, Saavedra CLI. Algorithms used for facial emotion recognition: a systematic review of the literature. EAI Endorsed Trans Perv Health Tech [Internet]. 2023 Oct. 24 [cited 2024 Dec. 26];9. Available from: https://publications.eai.eu/index.php/phat/article/view/4214