A facial expression recognizer using modified ResNet-152

Authors

DOI:

https://doi.org/10.4108/eetiot.v7i28.685

Keywords:

Facial expression recognition, ResNet-152, Recognition system

Abstract

In this age of artificial intelligence, facial expression recognition is an essential pool to describe emotion and psychology. In recent studies, many researchers have not achieved satisfactory results. This paper proposed an expression recognition system based on ResNet-152. Statistical analysis showed our method achieved 96.44% accuracy. Comparative experiments show that the model is better than mainstream models. In addition, we briefly described the application of facial expression recognition technology in the IoT (Internet of things).

Downloads

Download data is not yet available.
<br data-mce-bogus="1"> <br data-mce-bogus="1">

References

J. Jeon et al., "A real-time facial expression recognizer using deep neural network," in proceedings of the 10th international conference on ubiquitous information management and communication, 2016, pp. 1-4. DOI: https://doi.org/10.1145/2857546.2857642

Y.-D. Zhang and Z.-C. Dong, "Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation," Information Fusion, vol. 64, pp. 149-187, 2020/12/01/ 2020.

B. Peng, Y.-X. Liang, J. Yang, and K. So, "Image processing methods to elucidate spatial characteristics of retinal microglia after optic nerve transection," Scientific Reports, vol. 6, 2016, Art. no. 21816.

J. Yang, "Pathological brain detection in MRI scanning via Hu moment invariants and machine learning," Journal of Experimental & Theoretical Artificial Intelligence, vol. 29, no. 2, pp. 299-312, 2017.

K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," Computer Science, 2014.

K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.

P. Ekman and W. Friesen, "Facial Action Coding System (FACS): A technique for the measurement of facial action," Rivista Di Psichiatria, vol. 47, no. 2, pp. 126-38, 1978.

H. Ali, M. Hariharan, S. Yaacob, A. H. J. J. o. M. I. Adom, and H. Informatics, "Facial Emotion Recognition Based on Higher-Order Spectra Using Support Vector Machines," 2015.

S. Lu and F. Evans, "Haar Wavelet Transform Based Facial Emotion Recognition," in 2017 7th International Conference on Education, Management, Computer and Society (EMCS 2017), 2017.

W. Yang, "Facial Emotion Recognition via Discrete Wavelet Transform , Principal Component Analysis, and Cat Swarm Optimization," Lecture Notes in Computer Science, vol. 10559, pp. 203-214, 2017.

B. Li, R. Li, and D. Lima, Facial Expression Recognition via ResNet-18. Multimedia Technology and Enhanced Learning, Third EAI International Conference, ICMTEL 2021, Virtual Event, April 8–9, 2021, Proceedings, Part II, 2021.

B. Li and D. Lima, "Facial expression recognition via ResNet-50," International Journal of Cognitive Computing in Engineering, pp. 57-64, 2021.

S. Alnefaie, A. Cherif, and S. Alshehri, "A Distributed Fog-based Access Control Architecture for IoT," KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, vol. 15, no. 12, pp. 4545-4566, DEC 31 2021.

A. A. Alaboudi, "Fifteen Deadly Cybersecurity Threats Aimed Covid-19," INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND NETWORK SECURITY, vol. 21, no. 12, pp. 123-130, DEC 30 2021.

H. M. Lu, "Facial Emotion Recognition Based on Biorthogonal Wavelet Entropy, Fuzzy Support Vector Machine, and Stratified Cross Validation," IEEE Access, vol. 4, pp. 8375-8385, 2016.

H. Bazai, E. Kargar, and M. Mehrabi, "Using an encoder-decoder convolutional neural network to predict the solid holdup patterns in a pseudo-2d fluidized bed," Chemical Engineering Science, vol. 246:Article ID.116886, 2021.

C.-F. Hsu, T.-W. Chien, and Y.-H. Yan, "An application for classifying perceptions on my health bank in Taiwan using convolutional neural networks and web-based computerized adaptive testing: A development and usability study," Medicine, vol. 100, no. 52:Article ID. e28457, 2021.

N. H. T. Nguyen, S. Perry, D. Bone, H. T. Le, and T. T. Nguyen, "Two-stage convolutional neural network for road crack detection and segmentation," Expert Systems with Applications, vol. 186, 2021.

X. Zhang, "DSSAE: Deep Stacked Sparse Autoencoder Analytical Model for COVID-19 Diagnosis by Fractional Fourier Entropy," ACM Transactions on Management Information Systems, vol. 13, no. 1, 2021, Art. no. 2.

M. A. Khan, "Pseudo Zernike Moment and Deep Stacked Sparse Autoencoder for COVID-19 Diagnosis," CMC-Computers, Materials & Continua, vol. 69, no. 3, pp. 3145–3162, 2021.

Z. Zhu, "PSCNN: PatchShuffle Convolutional Neural Network for COVID-19 Explainable Diagnosis," Frontiers in Public Health, vol. 9, 2021, Art. no. 768278.

D. Kiruithiga, V. Manikandan, and Ieee, "Time Series Load Forecasting Using Pooling Based Multitasking Deep Neural Network," presented at the 2021 IEEE POWER & ENERGY SOCIETY INNOVATIVE SMART GRID TECHNOLOGIES CONFERENCE (ISGT), 2021.

R. Ahmad, "Reviewing the relationship between machines and radiology: the application of artificial intelligence," Acta Radiologica Open, vol. 10, no. 2, Feb 2021, Art. no. 2058460121990296.

J. Huang, L. Qu, R. Jia, and B. Zhao, "O2u-net: A simple noisy label detection approach for deep neural networks," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3326-3334.

R. Blything, V. Biscione, Vankov, II, C. J. H. Ludwig, and J. S. Bowers, "The human visual system and CNNs can both support robust online translation tolerance following extreme displacements," Journal of Vision, vol. 21, no. 2, Feb 2021, Art. no. 9.

C. Evans, C. Paz, and G. Mascialino, ""Infeliz " or "Triste ": A Paradigm for Mixed Methods Exploration of Outcome Measures Adaptation Across Language Variants," Frontiers in Psychology, vol. 12, Aug 2021, Art. no. 695893.

M. Bader, L. J. Jobst, I. Zettler, B. E. Hilbig, and M. Moshagen, "Disentangling the Effects of Culture and Language on Measurement Noninvariance in Cross-Cultural Research: The Culture, Comprehension, and Translation Bias (CCT) Procedure," Psychological Assessment, vol. 33, no. 5, pp. 375-384, May 2021.

K. Bhattacharjee, A. Tiwari, M. Pant, C. W. Ahn, and S. Oh, "Multiple Instance Learning with Differential Evolutionary Pooling," Electronics, vol. 10, no. 12, Jun 2021, Art. no. [1] J. Jeon et al., "A real-time facial expression recognizer using deep neural network," in proceedings of the 10th international conference on ubiquitous information management and communication, 2016, pp. 1-4.

Y.-D. Zhang and Z.-C. Dong, "Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation," Information Fusion, vol. 64, pp. 149-187, 2020/12/01/ 2020. DOI: https://doi.org/10.1016/j.inffus.2020.07.006

B. Peng, Y.-X. Liang, J. Yang, and K. So, "Image processing methods to elucidate spatial characteristics of retinal microglia after optic nerve transection," Scientific Reports, vol. 6, 2016, Art. no. 21816. DOI: https://doi.org/10.1038/srep21816

J. Yang, "Pathological brain detection in MRI scanning via Hu moment invariants and machine learning," Journal of Experimental & Theoretical Artificial Intelligence, vol. 29, no. 2, pp. 299-312, 2017. DOI: https://doi.org/10.1080/0952813X.2015.1132274

K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," Computer Science, 2014.

K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90

P. Ekman and W. Friesen, "Facial Action Coding System (FACS): A technique for the measurement of facial action," Rivista Di Psichiatria, vol. 47, no. 2, pp. 126-38, 1978. DOI: https://doi.org/10.1037/t27734-000

H. Ali, M. Hariharan, S. Yaacob, A. H. J. J. o. M. I. Adom, and H. Informatics, "Facial Emotion Recognition Based on Higher-Order Spectra Using Support Vector Machines," 2015. DOI: https://doi.org/10.1166/jmihi.2015.1527

S. Lu and F. Evans, "Haar Wavelet Transform Based Facial Emotion Recognition," in 2017 7th International Conference on Education, Management, Computer and Society (EMCS 2017), 2017. DOI: https://doi.org/10.2991/emcs-17.2017.67

W. Yang, "Facial Emotion Recognition via Discrete Wavelet Transform , Principal Component Analysis, and Cat Swarm Optimization," Lecture Notes in Computer Science, vol. 10559, pp. 203-214, 2017. DOI: https://doi.org/10.1007/978-3-319-67777-4_18

B. Li, R. Li, and D. Lima, Facial Expression Recognition via ResNet-18. Multimedia Technology and Enhanced Learning, Third EAI International Conference, ICMTEL 2021, Virtual Event, April 8–9, 2021, Proceedings, Part II, 2021. DOI: https://doi.org/10.1007/978-3-030-82565-2_24

B. Li and D. Lima, "Facial expression recognition via ResNet-50," International Journal of Cognitive Computing in Engineering, pp. 57-64, 2021. DOI: https://doi.org/10.1016/j.ijcce.2021.02.002

S. Alnefaie, A. Cherif, and S. Alshehri, "A Distributed Fog-based Access Control Architecture for IoT," KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, vol. 15, no. 12, pp. 4545-4566, DEC 31 2021. DOI: https://doi.org/10.3837/tiis.2021.12.016

A. A. Alaboudi, "Fifteen Deadly Cybersecurity Threats Aimed Covid-19," INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND NETWORK SECURITY, vol. 21, no. 12, pp. 123-130, DEC 30 2021.

H. M. Lu, "Facial Emotion Recognition Based on Biorthogonal Wavelet Entropy, Fuzzy Support Vector Machine, and Stratified Cross Validation," IEEE Access, vol. 4, pp. 8375-8385, 2016. DOI: https://doi.org/10.1109/ACCESS.2016.2628407

H. Bazai, E. Kargar, and M. Mehrabi, "Using an encoder-decoder convolutional neural network to predict the solid holdup patterns in a pseudo-2d fluidized bed," Chemical Engineering Science, vol. 246:Article ID.116886, 2021. DOI: https://doi.org/10.1016/j.ces.2021.116886

C.-F. Hsu, T.-W. Chien, and Y.-H. Yan, "An application for classifying perceptions on my health bank in Taiwan using convolutional neural networks and web-based computerized adaptive testing: A development and usability study," Medicine, vol. 100, no. 52:Article ID. e28457, 2021. DOI: https://doi.org/10.1097/MD.0000000000028457

N. H. T. Nguyen, S. Perry, D. Bone, H. T. Le, and T. T. Nguyen, "Two-stage convolutional neural network for road crack detection and segmentation," Expert Systems with Applications, vol. 186, 2021. DOI: https://doi.org/10.1016/j.eswa.2021.115718

X. Zhang, "DSSAE: Deep Stacked Sparse Autoencoder Analytical Model for COVID-19 Diagnosis by Fractional Fourier Entropy," ACM Transactions on Management Information Systems, vol. 13, no. 1, 2021, Art. no. 2. DOI: https://doi.org/10.1145/3451357

M. A. Khan, "Pseudo Zernike Moment and Deep Stacked Sparse Autoencoder for COVID-19 Diagnosis," CMC-Computers, Materials & Continua, vol. 69, no. 3, pp. 3145–3162, 2021. DOI: https://doi.org/10.32604/cmc.2021.018040

Z. Zhu, "PSCNN: PatchShuffle Convolutional Neural Network for COVID-19 Explainable Diagnosis," Frontiers in Public Health, vol. 9, 2021, Art. no. 768278. DOI: https://doi.org/10.3389/fpubh.2021.768278

D. Kiruithiga, V. Manikandan, and Ieee, "Time Series Load Forecasting Using Pooling Based Multitasking Deep Neural Network," presented at the 2021 IEEE POWER & ENERGY SOCIETY INNOVATIVE SMART GRID TECHNOLOGIES CONFERENCE (ISGT), 2021. DOI: https://doi.org/10.1109/ISGT49243.2021.9372196

R. Ahmad, "Reviewing the relationship between machines and radiology: the application of artificial intelligence," Acta Radiologica Open, vol. 10, no. 2, Feb 2021, Art. no. 2058460121990296. DOI: https://doi.org/10.1177/2058460121990296

J. Huang, L. Qu, R. Jia, and B. Zhao, "O2u-net: A simple noisy label detection approach for deep neural networks," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3326-3334. DOI: https://doi.org/10.1109/ICCV.2019.00342

R. Blything, V. Biscione, Vankov, II, C. J. H. Ludwig, and J. S. Bowers, "The human visual system and CNNs can both support robust online translation tolerance following extreme displacements," Journal of Vision, vol. 21, no. 2, Feb 2021, Art. no. 9. DOI: https://doi.org/10.1167/jov.21.2.9

C. Evans, C. Paz, and G. Mascialino, ""Infeliz " or "Triste ": A Paradigm for Mixed Methods Exploration of Outcome Measures Adaptation Across Language Variants," Frontiers in Psychology, vol. 12, Aug 2021, Art. no. 695893. DOI: https://doi.org/10.3389/fpsyg.2021.695893

M. Bader, L. J. Jobst, I. Zettler, B. E. Hilbig, and M. Moshagen, "Disentangling the Effects of Culture and Language on Measurement Noninvariance in Cross-Cultural Research: The Culture, Comprehension, and Translation Bias (CCT) Procedure," Psychological Assessment, vol. 33, no. 5, pp. 375-384, May 2021. DOI: https://doi.org/10.1037/pas0000989

K. Bhattacharjee, A. Tiwari, M. Pant, C. W. Ahn, and S. Oh, "Multiple Instance Learning with Differential Evolutionary Pooling," Electronics, vol. 10, no. 12, Jun 2021, Art. no. 1403. DOI: https://doi.org/10.3390/electronics10121403

P. K. Nalajam and R. Varadarajan, "A Hybrid Deep Learning Model for Layer-Wise Melt Pool Temperature Forecasting in Wire-Arc Additive Manufacturing Process," IEEE ACCESS, vol. 9, pp. 100652-100664, 2021. DOI: https://doi.org/10.1109/ACCESS.2021.3097177

M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv preprint arXiv:. 2013.

X. Jiang, "Multiple Sclerosis Recognition by Biorthogonal Wavelet Features and Fitness-Scaled Adaptive Genetic Algorithm," (in English), Frontiers in Neuroscience, Original Research vol. 15, no. 1098, 2021-September-13 2021, Art. no. 737785. DOI: https://doi.org/10.3389/fnins.2021.737785

Z. Zhang and X. Zhang, "MIDCAN: A multiple input deep convolutional attention network for Covid-19 diagnosis based on chest CT and chest X-ray," Pattern Recognition Letters, vol. 150, pp. 8-16, 2021. DOI: https://doi.org/10.1016/j.patrec.2021.06.021

K. Wu, "SOSPCNN: Structurally Optimized Stochastic Pooling Convolutional Neural Network for Tetralogy of Fallot Recognition," Wireless Communications and Mobile Computing, vol. 2021, p. 5792975, 2021/07/02 2021, Art. no. 5792975.

E. Parcham, M. Ilbeygi, and M. Amini, "CBCapsNet: A novel writer-independent offline signature verification model using a CNN-based architecture and capsule neural networks," Expert Systems with Applications, vol. 185, p. 115649, 2021. DOI: https://doi.org/10.1016/j.eswa.2021.115649

C. Sitaula, T. B. Shahi, S. Aryal, and F. J. S. r. Marzbanrad, "Fusion of multi-scale bag of deep visual words features of chest X-ray images to detect COVID-19 infection," vol. 11, no. 1, pp. 1-12, 2021. DOI: https://doi.org/10.1038/s41598-021-03287-8

L. Zwingmann, M. Zedler, S. Kurzner, P. Wahl, and J.-P. J. F. i. s. Goldmann, "How Fit Are Special Operations Police Officers? A Comparison With Elite Athletes From Olympic Disciplines," Frontiers in sports active living, vol. 3, 2021. DOI: https://doi.org/10.3389/fspor.2021.742655

Y. Fushimura et al., "Orotic acid protects pancreatic 13 cell by p53 inactivation in diabetic mouse model," Biochemical biophysical research communications, vol. 585, pp. 191-195, 2021. DOI: https://doi.org/10.1016/j.bbrc.2021.10.060

E. KARADURMUŞ, G. Eda, N. TAŞKIN, and M. YÜCEER, "BROMATE REMOVAL PREDICTION IN DRINKING WATER BY USING THE LEAST SQUARES SUPPORT VECTOR MACHINE (LS-SVM)," Sigma Journal of Engineering and Natural Sciences, vol. 38, no. 4, pp. 2145-2153, 2020.

V. Patel, "Kalman-based stochastic gradient method with stop condition and insensitivity to conditioning," SIAM Journal on Optimization, vol. 26, no. 4, pp. 2620-2648, 2016. DOI: https://doi.org/10.1137/15M1048239

S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in International conference on machine learning, 2015, pp. 448-456: PMLR.

K. He, X. Zhang, S. Ren, and J. Sun, "Identity mappings in deep residual networks," in European conference on computer vision, 2016, pp. 630-645: Springer. DOI: https://doi.org/10.1007/978-3-319-46493-0_38

X. Li, C. Tang, and J. Sun, "Facial emotion recognition via stationary wavelet entropy and Biogeography-based optimization," ICST Transactions on education e-Learning, vol. 6, no. 19, p. 165702, 2020. DOI: https://doi.org/10.4108/eai.30-10-2018.165702

J. An, X. Li, Z. Zhang, W. Man, and G. Zhang, "Joint Trajectory Planning of Space Modular Reconfigurable Satellites Based on Kinematic Model," International Journal of Aerospace Engineering, vol. 2020, 2020. DOI: https://doi.org/10.1155/2020/8872788

H. Delavari and S. Naderian, "Design and HIL implementation of a new robust fractional sliding mode control of microgrids," IET Generation, Transmissionu&Distribution, vol. 14, no. 26, pp. 6690-6702, 2021. DOI: https://doi.org/10.1049/iet-gtd.2020.0865

S. Etedali, "Ranking of design scenarios of TMD for seismically excited structures using TOPSIS," Frontiers of Structural and Civil Engineering, vol. 14, no. 6, pp. 1372-1386, 2020. DOI: https://doi.org/10.1007/s11709-020-0671-y

J. F. Yang and P. Sun, "Magnetic resonance brain classification by a novel binary particle swarm optimization with mutation and time-varying acceleration coefficients," (in English), Biomedical Engineering-Biomedizinische Technik, Article vol. 61, no. 4, pp. 431-441, Aug 2016. DOI: https://doi.org/10.1515/bmt-2015-0152

X.-X. Hou, "Alcoholism detection by medical robots based on Hu moment invariants and predator-prey adaptive-inertia chaotic particle swarm optimization," Computers and Electrical Engineering, vol. 63, pp. 126-138, 2017. DOI: https://doi.org/10.1016/j.compeleceng.2017.04.009

U. Ghosh, P. Chatterjee, and S. Shetty, "Securing SDN-enabled smart power grids: SDN-enabled smart grid security," in Research Anthology on Smart Grid and Microgrid Development: IGI Global, 2022, pp. 1028-1046. DOI: https://doi.org/10.4018/978-1-6684-3666-0.ch046

A. S. Amiss et al., "Modified horseshoe crab peptides target and kill bacteria inside host cells," Cellular and Molecular Life Sciences, vol. 79, no. 1, Jan 2022. DOI: https://doi.org/10.1007/s00018-021-04041-z

S. Banerjee, M. A. Afzal, P. Chokshi, and A. S. Rathore, "Mechanistic modelling of Chinese hamster ovary cell clarification using acoustic wave separator," CHEMICAL ENGINEERING SCIENCE, vol. 246, DEC 31 2021. DOI: https://doi.org/10.1016/j.ces.2021.116894

F. Bonchi, D. Garcia-Soriano, A. Miyauchi, and C. E. Tsourakakis, "Finding densest k-connected subgraphs," DISCRETE APPLIED MATHEMATICS, vol. 305, pp. 34-47, DEC 31 2021. DOI: https://doi.org/10.1016/j.dam.2021.08.032

Downloads

Published

27-04-2022

How to Cite

[1]
W. Xu and R. S. Cloutier, “A facial expression recognizer using modified ResNet-152”, EAI Endorsed Trans IoT, vol. 7, no. 28, p. e5, Apr. 2022.

Issue

Vol. 7 No. 28 (2021): EAI Endorsed Transactions on Internet of Things

Section

Research article

License

Copyright (c) 2022 EAI Endorsed Transactions on Internet of Things

Creative Commons License

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.