Innovative Human Interaction System to Predict College Student Emotions Using the Extended MASK-R-CNN Algorithm
DOI:
https://doi.org/10.4108/eetiot.7874Keywords:
Object detection, emotions prediction, Deep Learning, Mask RCNN, Facial expression recognitionAbstract
There is a rising demand for emerging machines that can be self-decisive and intelligent. Machines can capture the emotions and gestures of college students to mechanise tasks and handle interactions better. Facial expressions based on emotion recognition are practices that play a substantial role in the modern fields of artificial intelligence and computer vision. Numerous manual methods for detecting emotions are focused on few basic emotions. Additionally, significant time is needed for appropriate detection. Nonetheless, these techniques are time-consuming and inefficient for obtaining better results. Therefore, an effective object detection model is needed to address such issues. To overcome these challenges, several studies have focused on object detection systems to provide effective emotion prediction. Conversely, it results in a lack of speed, precision and computational complexity. To improve object detection performance, the proposed model employs deep learning (DL)-based adaptive feature spatial anchor refinement with a mask region-based convolutional neural network (Mask RCNN). It uses the Facial Expression Recognition (FER) 2013 dataset for the evaluation process. Correspondingly, the efficacy of the projected model is calculated via various evaluation metrics, such as the recall, precision and mean average precision (mAP), to estimate the performance of the proposed DL method. It achieves 0.75298 for MAP@50, 0.70252 for precision and 0.66606 for recall. Furthermore, a comparison of existing models reveals the efficiency of the proposed DL method. The present research is intended to contribute to emerging object detection methods for enhancing real-time analysis of student emotions in various environments, such as classrooms and online education.
Downloads
References
[1] Haq, H., et al., Enhanced real-time facial expression recognition using deep learning. 2024. 3(1): p. 24-35.
[2] Russell, P.W., X. Min, and S.S. Lily, Facial recognition and emotion detection in environmental installation and social media applications, in Encyclopedia of Computer Graphics and Games. 2024, Springer. p. 694-703.
[3] Ekundayo, O. and S. Viriri, Facial Expression Recognition: A Review of Methods, Performances and Limitations.
[4] Li, S. and W. Deng, Deep facial expression recognition: A survey. IEEE transactions on affective computing, 2020. 13(3): p. 1195-1215.
[5] Ouherrou, N., et al., Comparative study on emotions analysis from facial expressions in children with and without learning disabilities in virtual learning environment. Education and Information Technologies, 2019. 24(2): p. 1777-1792.
[6] Wang, W., et al., Emotion recognition of students based on facial expressions in online education based on the perspective of computer simulation. Complexity, 2020. 2020: p. 1-9.
[7] Wang, K., et al., Region attention networks for pose and occlusion robust facial expression recognition. IEEE Transactions on Image Processing, 2020. 29: p. 4057-4069.
[8] Borgalli, R.A. and S. Surve, Deep Convolution Neural Networks for Cross-Dataset Facial Expression Recognition System.
[9] Yalew, B., Emotion Recognition from Facial Expression Using Convolutional Neural Network. 2024, St. Mary's University.
[10] Gera, D., et al., Dynamic adaptive threshold based learning for noisy annotations robust facial expression recognition. 2024. 83(16): p. 49537-49566.
[11] Lasri, I., A.R. Solh, and M. El Belkacemi, Facial Emotion Recognition of Students using Convolutional Neural Network.
[12] TS, A. and R.M.R. Guddeti, Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks. Education and information technologies, 2020. 25(2): p. 1387-1415.
[13] Pei, J. and P. Shan, A Micro-expression Recognition Algorithm for Students in Classroom Learning Based on Convolutional Neural Network. Traitement du Signal, 2019. 36(6).
[14] Roy, S. and A. Etemad, Analysis of Semi-Supervised Methods for Facial Expression Recognition. arXiv preprint arXiv:2208.00544, 2022.
[15] Ali, M.F., M. Khatun, and N.A. Turzo, Facial emotion detection using neural network. the international journal of scientific and engineering research, 2020.
[16] Deng, J. and F. Ren, Multi-label emotion detection via emotion-specified feature extraction and emotion correlation learning. IEEE Transactions on Affective Computing, 2020. 14(1): p. 475-486.
[17] Yang, K., et al., Benchmarking commercial emotion detection systems using realistic distortions of facial image datasets. The visual computer, 2021. 37: p. 1447-1466.
[18] Wu, Z.-Z., et al., Convolutional neural network based weakly supervised learning for aircraft detection from remote sensing image. IEEE Access, 2020. 8: p. 158097-158106.
[19] Yang, J., et al., Stimuli-aware visual emotion analysis. IEEE Transactions on Image Processing, 2021. 30: p. 7432-7445.
[20] Graterol, W., et al., Emotion detection for social robots based on NLP transformers and an emotion ontology. Sensors, 2021. 21(4): p. 1322.
[21] Wu, Q., et al., Improved mask R-CNN for aircraft detection in remote sensing images. Sensors, 2021. 21(8): p. 2618.
[22] Savchenko, A.V., L.V. Savchenko, and I. Makarov, Classifying emotions and engagement in online learning based on a single facial expression recognition neural network. IEEE Transactions on Affective Computing, 2022. 13(4): p. 2132-2143.
[23] Siam, A.I., et al., Deploying machine learning techniques for human emotion detection. Computational intelligence and neuroscience, 2022. 2022(1): p. 8032673.
[24] Hans, A.S.A. and S. Rao, A CNN-LSTM based deep neural networks for facial emotion detection in videos. International Journal of Advances in Signal and Image Sciences, 2021. 7(1): p. 11-20.
[25] Pabba, C. and P. Kumar, An intelligent system for monitoring students' engagement in large classroom teaching through facial expression recognition. Expert Systems, 2022. 39(1): p. e12839.
[26] Amal, V., S. Suresh, and G. Deepa. Real-time emotion recognition from facial expressions using convolutional neural network with Fer2013 dataset. in Ubiquitous Intelligent Systems: Proceedings of ICUIS 2021. 2022. Springer.
[27] Hazra, S.K., et al., Emotion recognition of human speech using deep learning method and MFCC features. Radioelectronic and Computer Systems, 2022(4): p. 161-172.
[28] Kumar, S., et al., Multilayer Neural Network Based Speech Emotion Recognition for Smart Assistance. Computers, Materials & Continua, 2023. 75(1).
[29] Wu, W., H.-S. Wong, and S. Wu, Pseudo-Siamese Teacher for Semi-Supervised Oriented Object Detection. IEEE Transactions on Geoscience and Remote Sensing, 2024.
[30] Saeed, V.A., A framework for recognition of facial expression using HOG features. International Journal of Mathematics, Statistics, and Computer Science, 2024. 2: p. 1-8.
[31] Manalu, H.V. and A.P. Rifai, Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm. Intelligent Systems with Applications, 2024. 21: p. 200339.
[32] Qazi, A.S., et al., Emotion detection using facial expression involving occlusions and tilt. Applied Sciences, 2022. 12(22): p. 11797.
[33] Meena, G., et al., Identifying emotions from facial expressions using a deep convolutional neural network-based approach. Multimedia Tools and Applications, 2024. 83(6): p. 15711-15732.
[34] Chowdary, M.K., T.N. Nguyen, and D.J. Hemanth, Deep learning-based facial emotion recognition for human–computer interaction applications. Neural Computing and Applications, 2023. 35(32): p. 23311-23328.
[35] Ganesan, P., et al., Deep Learning-based Interactive Dashboard for Enhancing Online Classroom Experience through Student Emotion Analysis. IEEE Access, 2024.
[36] Angel, J.S., et al., Faster Region Convolutional Neural Network (FRCNN) Based Facial Emotion Recognition. Computers, Materials & Continua, 2024. 79(2).
[37] Yadav, A. and E. Kumar, Object Detection on Real-Time Video with FPN and Modified Mask RCNN Based on Inception-ResNetV2. Wireless Personal Communications, 2024: p. 1-26.
[38] Imran, A., et al., FaceEngine: A Tracking-Based Framework for Real-Time Face Recognition in Video Surveillance System. SN Computer Science, 2024. 5(5): p. 609.
[39] FER2013 expression Computer Vision Project.
[40] Parambil, M.M.A., et al., Navigating the YOLO landscape: A comparative study of Object detection models for Emotion Recognition. 2024.
[41] Mukhiddinov, M., et al., Masked face emotion recognition based on facial landmarks and deep learning approaches for visually impaired people. Sensors, 2023. 23(3): p. 1080.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Dinesh P, Thailambal G
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
