Improving recognition accuracy for facial expressions using scattering wavelet

Mehdi Davari; Aryan Harooni; Afrooz Nasr; Kimia  Savoji; Masoumeh Soleimani

doi:10.4108/airo.5145

Authors

Mehdi Davari Islamic Azad University, Tehran
Aryan Harooni Kent State University
Afrooz Nasr Islamic Azad University, Tehran
Kimia Savoji Clemson University
Masoumeh Soleimani Clemson University

DOI:

https://doi.org/10.4108/airo.5145

Keywords:

Facial expressions, Scattering waves, Deep learning, Gabor Philter, Recognition Rate

Abstract

One of the most evident and meaningful feedback about people’s emotions is through facial expressions. Facial expression recognition is helpful in social networks, marketing, and intelligent education systems. The use of Deep Learning based methods in facial expression identification is widespread, but challenges such as computational complexity and low recognition rate plague these methods. Scatter Wavelet is a type of Deep Learning that extracts features from Gabor filters in a structure similar to convolutional neural networks. This paper presents a new facial expression recognition method based on wavelet scattering that identifies six states: anger, disgust, fear, happiness, sadness, and surprise. The proposed method is simulated using the JAFFE and CK+ databases. The recognition rate of the proposed method is 99.7%, which indicates the superiority of the proposed method in recognizing facial expressions.

Downloads

Download data is not yet available.

References

Samta Jain Goyal, Arvind K Upadhyay, RS Jadon, and Rajeev Goyal. Real-life facial expression recognition systems: a review. Smart computing and informatics, 77(1):311–331, 2018.

Zhihong Zeng, Maja Pantic, Glenn I Roisman, and Thomas S Huang. A survey of affect recognition methods: audio, visual and spontaneous expressions. In Proceedings of the 9th international conference on Multimodal interfaces, pages 126–133, 2007.

Keith Anderson and Peter W McOwan. A real-time automated system for the recognition of human facial expressions. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 36(1):96–105, 2006.

Beat Fasel and Juergen Luettin. Automatic facial expression analysis: a survey. Pattern recognition, 36(1):259–275, 2003.

Ira Cohen, Nicu Sebe, Ashutosh Garg, Lawrence S Chen, and Thomas S Huang. Facial expression recognition from video sequences: temporal and static modeling. Computer Vision and image understanding, 91(1-2):160– 187, 2003.

Paul Ekman. Facial expression and emotion. Facta universitatis-series: Philosophy and Sociology, 1(3):195–212, 1996.

Paul Ekman and Wallace V Friesen. Constants across cultures in the face and emotion. Journal of personality and social psychology, 17(2):124, 1971.

Xiaobai Li, Xiaopeng Hong, Antti Moilanen, Xiaohua Huang, Tomas Pfister, Guoying Zhao, and Matti Pietikäinen. Reading hidden emotions: spontaneous micro-expression spotting and recognition. arXiv preprint arXiv:1511.00423, 2(6):7, 2015.

YandanWang, John See, Yee-Hui Oh, Raphael C-WPhan, Yogachandran Rahulamathavan, Huo-Chong Ling, Su-Wei Tan, and Xujie Li. Effective recognition of facial micro-expressions with video motion magnification. Multimedia Tools and Applications, 76:21665–21690, 2017.

R Niese, A Al-Hamadi, A Farag, H Neumann, and B Michaelis. Facial expression recognition based on geometric and optical flow features in colour image sequences. IET computer vision, 6(2):79–89, 2012.

Claudio Loconsole, Catarina Runa Miranda, Gustavo Augusto, Antonio Frisoli, and Verónica Orvalho. Realtime emotion recognition novel method for geometrical facial features extraction. In 2014 International Conference on Computer Vision Theory and Applications (VISAPP), volume 1, pages 378–385. IEEE, 2014.

Anwar Saeed, Ayoub Al-Hamadi, Robert Niese, and Moftah Elzobi. Frame-based facial expression recognition using geometrical features. Advances in human-computer interaction, 2014:4–4, 2014.

Deepak Ghimire, Joonwhoan Lee, Ze-Nian Li, Sunghwan Jeong, Sang Hyun Park, and Hyo Sub Choi. Recognition of facial expressions based on tracking and selection of discriminative geometric features. International journal of multimedia and Ubiquitous Engineering, 10(3):35–44, 2015.

Amjad Rehman Khan, Majid Harouni, Sepideh Sharifi, Saeed Ali Bahaj, and Tanzila Saba. Face detection in close-up shot video events using video mining. Journal of Advances in Information Technology, 14(2), 2023.

Ata Jahangir Moshayedi, Sithembiso Khaya Sambo, and Amin Kolahdooz. Design and development of costeffective exergames for activity incrementation. In 2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE), pages 133–137. IEEE, 2022.

Deepak Ghimire, Sunghwan Jeong, Joonwhoan Lee, and San Hyun Park. Facial expression recognition based on local region specific features and support vector machines. Multimedia Tools and Applications, 76:7803–7821, 2017.

Mehdi Marani, Morteza Soltani, Mina Bahadori, Masoumeh Soleimani, and Atajahangir Moshayedi. The role of biometric in banking: A review. EAI Endorsed Transactions on AI and Robotics, 2(1), 2023.

Ge Xu, Amir Sohail Khan, Ata Jahangir Moshayedi, Xiaohong Zhang, and Yang Shuxin. The object detection, perspective and obstacles in robotic: A review. EAI Endorsed Transactions on AI and Robotics, 1(1), 2022.

Shima Alizadeh and Azar Fazel. Convolutional neural networks for facial expression recognition. arXiv preprint arXiv:1704.06756, 2017.

Mina Bahadori, Masoumeh Soleimani, Morteza Soltani, and Mehdi Davari. Application of iot in drug supply chain management (scm) during covid-19 outbreak. In Advanced IoT Technologies and Applications in the Industry 4.0 Digital Economy, pages 224–237. CRC Press.

Huma Qayyum, Muhammad Majid, Syed Muhammad Anwar, Bilal Khan, et al. Facial expression recognition using stationary wavelet transform features. Mathematical Problems in Engineering, 2017, 2017.

Ali Khansefid, Seyed Mahmoudreza Yadollahi, Gerhard Müller, and Francesca Taddei. Induced earthquake hazard by geothermal power plants: Statistical evaluation and probabilistic modeling. International Journal of Disaster Risk Science, 13(5):758–777, 2022.

Ata Jahangir Moshayedi and DC Gharpure. Development of position monitoring system for studying performance of wind tracking algorithms. In ROBOTIK 2012; 7th German Conference on Robotics, pages 1–4. VDE, 2012.

Evangelos Sariyanidi, Hatice Gunes, and Andrea Cavallaro. Learning bases of activity for facial expression recognition. IEEE Transactions on Image Processing, 26(4):1965–1978, 2017.

Usman Mehmood, Ephraim Bonah Agyekum, Salah Kamel, Hossein Shahinzadeh, and Ata Jahangir Moshayedi. Exploring the roles of renewable energy, education spending, and co2 emissions towards health spending in south asian countries. Sustainability, 14(6):3549, 2022.

AsimMunir, AyyazHussain, Sajid Ali Khan,Muhammad Nadeem, and Sadia Arshid. Illumination invariant facial expression recognition using selected merged binary patterns for real world images. Optik, 158:1016–1025, 2018.

A Jahangir Moshayedi, Ge Xu, Liefa Liao, and Amin Kolahdooz. Gentle survey on mir industrial service robots: Review & design. J. Mod. Process. Manuf. Prod, 10(1):31–50, 2021.

Nianyin Zeng, Hong Zhang, Baoye Song, Weibo Liu, Yurong Li, and Abdullah M Dobaie. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing, 273:643–649, 2018.

Amjad Rehman, Majid Harouni, Maedeh Omidiravesh, Suliman Mohamed Fati, and Saeed Ali Bahaj. Finger vein authentication based on wavelet scattering networks. Computers, Materials & Continua, 72(2), 2022.

Amjad Rehman, Majid Harouni, Negar Haghani Solati Karchegani, Tanzila Saba, Saeed Ali Bahaj, and Sudipta Roy. Identity verification using palm print microscopic images based on median robust extended local binary pattern features and k-nearest neighbor classifier. Microscopy Research and Technique, 85(4):1224–1237, 2022.

Stéphane Mallat. Group invariant scattering. Communications on Pure and Applied Mathematics, 65(10):1331–1398, 2012.

Morteza Soltani, Jeffrey P Kharoufeh, and Amin Khademi. Optimal call center staffing and pricing under qos constraints. In IIE Annual Conference. Proceedings, pages 1–6. Institute of Industrial and Systems Engineers (IISE), 2023.

Morteza Soltani, Jeffrey P Kharoufeh, and Amin Khademi. Structured replacement policies for offshore wind turbines. Probability in the Engineering and Informational Sciences, pages 1–32, 2023.

Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE transactions on pattern analysis and machine intelligence, 35(8):1872–1886, 2013.

Joan Bruna and Stéphane Mallat. Classification with scattering operators. In CVPR 2011, pages 1561–1566. IEEE, 2011.

Masoumeh Soleimani, Zahra Forouzanfar, Morteza Soltani, and Majid Jafari Harandi. Imbalancedmulticlass medical data classification based on learning automata and neural network. EAI Endorsed Transactions on AI and Robotics, 2, 2023.

Ata Jahangir Moshayedi, Saeed Safara Fard, Liefa Liao, and Seyed Ali Eftekhari. Design and development of pipe inspection robot meant for resizable pipe lines. Int. J. Robot. Control, 2(1):25, 2019.

Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba. Coding facial expressions with gabor wavelets. In Proceedings Third IEEE international conference on automatic face and gesture recognition, pages 200–205. IEEE, 1998.

Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 ieee computer society conference on computer vision and pattern recognition-workshops, pages 94–101. IEEE, 2010.

Takeo Kanade, Jeffrey F Cohn, and Yingli Tian. Comprehensive database for facial expression analysis. In Proceedings fourth IEEE international conference on automatic face and gesture recognition (cat. No. PR00580), pages 46–53. IEEE, 2000.

Mina Bahadori, Morteza Soltani, Masoumeh Soleimani, and Mahsa Bahadori. Statistical modeling in healthcare: Shaping the future of medical research and healthcare delivery. In AI and IoT-based technologies for precision medicine, pages 431–446. IGI Global, 2023.

Dilnoza Mamieva, Akmalbek Bobomirzaevich Abdusalomov, Mukhriddin Mukhiddinov, and Taeg Keun Whangbo. Improved face detection method via learning small faces on hard images based on a deep learning approach. Sensors, 23(1):502, 2023.

Shivalila Hangaragi, Tripty Singh, and N Neelima. Face detection and recognition using face mesh and deep neural network. Procedia Computer Science, 218:741–749, 2023.

Ata Jahangir Moshayedi and DC Gharpure. Evaluation of bio inspired mokhtar: Odor localization system. In 2017 18th international carpathian control conference (ICCC), pages 527–532. IEEE, 2017.

Omid Toutian Esfahani and Ata Jahangir Moshayedi. Accuracy of the positioning systems for the tracking of alzheimer’s patients-a review. International Journal of Applied Electronics in Physics & Robotics, 2(2):10–16, 2014.