EEG Emotion Recognition Based on Self-Distillation Convolutional Graph Attention Network

Hao Chao; Shuqi Feng

doi:10.4108/eetel.4974

Authors

Hao Chao Henan Polytechnic University
Shuqi Feng Henan Polytechnic University

DOI:

https://doi.org/10.4108/eetel.4974

Abstract

A convolution graph attention model based on self-distillation convolutional graph attention network (SDC-GAT) is proposed for multi-channel EEG emotion recognition. Firstly, two-dimensional feature matrix based on EEG time-domain features are constructed, and the matrix is fed into the graph attention neural network to learn the internal connections between electrical brain channels located in different brain regions. Meanwhile, the three-dimensional feature matrix is constructed according to the relative positions of the electrode channels, and the self-distillation network is employed to extract local high-level abstract features containing electrode spatial position information from the three-dimensional feature matrix. Finally, outputs of the two networks are integrated to determine the emotional states. Experiments were performed on the DEAP dataset. The experimental results show that the spatial domain information of the electrode channel and the internal connection relationship between different channels are beneficial for emotion recognition. In addition, the proposed model can effectively fuse these information to improve the performance of multi-channel EEG emotion recognition.

References

Liu S, Wang X, Zhao L, et al. Subject-independent emotion recognition of EEG signals based on dynamic empirical convolutional neural network[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2020, 18(5): 1710-1721.

Han J, Zhang Z, Pantic M, et al. Internet of emotional people: Towards continual affective computing cross cultures via audiovisual signals[J]. Future Generation Computer Systems, 2021, 114: 294-306.

Liu S, Wang X, Zhao L, et al. 3DCANN: A spatio-temporal convolution attention neural network for EEG emotion recognition[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26(11): 5321-5331.

Hu W, Zhang Z, Zhao H, et al. EEG microstate correlates of emotion dynamics and stimulation content during video watching[J]. Cerebral Cortex, 2023, 33(3): 523-542.

Dang W D, Lv D M, Li R M, et al. Multilayer network-based CNN model for emotion recognition[J]. International Journal of Bifurcation and Chaos, 2022, 32(01): 2250011.

Iyer A, Das S S, Teotia R, et al. CNN and LSTM based ensemble learning for human emotion recognition using EEG recordings[J]. Multimedia Tools and Applications, 2023, 82(4): 4883-4896.

Xin R, Miao F, Cong P, et al. Multiview Feature Fusion Attention Convolutional Recurrent Neural Networks for EEG-Based Emotion Recognition[J]. Journal of Sensors, 2023, 2023.

Li Z, Zhang G, Wang L, et al. Emotion recognition using spatial-temporal EEG features through convolutional graph attention network[J]. Journal of Neural Engineering, 2023, 20(1): 016046.

Zhang L, Bao C, Ma K. Self-distillation: Towards efficient and compact neural networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(8): 4388-4403.

Gou J, Yu B, Maybank S J, et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129: 1789-1819.

Wang X, Li Y. Harmonized dense knowledge distillation training for multi-exit architectures[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2021, 35(11): 10218-10226.

Zhang L, Bao C, Ma K. Self-distillation: Towards efficient and compact neural networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(8): 4388-4403.

Yang Z, Li Z, Shao M, et al. Masked generative distillation[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 53-69.

Song T, Zheng W, Song P, et al. EEG emotion recognition using dynamical graph convolutional neural networks[J]. IEEE Transactions on Affective Computing, 2018, 11(3): 532-541.

Yin Y, Zheng X, Hu B, et al. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM[J]. Applied Soft Computing, 2021, 100: 106954.

Ghandeharioun A, McDuff D, Czerwinski M, et al. Emma: An emotion-aware wellbeing chatbot[C]//2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2019: 1-7.

Liu Y, Fu G. Emotion recognition by deeply learned multi-channel textual and EEG features[J]. Future Generation Computer Systems, 2021, 119: 1-6.

Haj-Ali H, Anderson A K, Kron A. Comparing three models of arousal in the human brain[J]. Social Cognitive and Affective Neuroscience, 2020, 15(1): 1-11.

Houssein E H, Hammad A, Ali A A. Human emotion recognition from EEG-based brain-computer interface using machine learning: a comprehensive review[J]. Neural Computing and Applications, 2022, 34(15): 12527-12557.

Hjorth B. EEG analysis based on time domain properties[J]. Electroencephalography and clinical neurophysiology, 1970, 29(3): 306-310.

Frantzidis C A, Bratsas C, Papadelis C L, et al. Toward emotion aware computing: an integrated approach using multichannel neurophysiological recordings and affective visual stimuli[J]. IEEE transactions on Information Technology in Biomedicine,2010, 14(3): 589-597.

Cai Y, Yao Z, Dong Z, et al. Zeroq: A novel zero shot quantization framework[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 13169-13178.

Ma N, Zhang X, Zheng H T, et al. Shufflenet v2: Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 116-131.

Liu Y, Cao J, Li B, et al. Knowledge distillation via instance relationship graph[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 7096-7104.

Cheng Y, Wang D, Zhou P, et al. Model compression and acceleration for deep neural networks: The principles, progress, and challenges[J]. IEEE Signal Processing Magazine, 2018, 35(1): 126-136.

Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015.

Romero A, Ballas N, Kahou S E, et al. Fitnets: Hints for thin deep nets[J]. arXiv preprint arXiv:1412.6550, 2014.

Zagoruyko S, Komodakis N. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer[J]. arXiv preprint arXiv:1612.03928, 2016.

Yim J, Joo D, Bae J, et al. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4133-4141.

Lee S, Song B C. Graph-based knowledge distillation by multi-head attention network[J]. arXiv preprint arXiv:1907.02226, 2019.

Furlanello T, Lipton Z, Tschannen M, et al. Born again neural networks[C]//International Conference on Machine Learning. PMLR, 2018: 1607-1616.

Bagherinezhad H, Horton M, Rastegari M, et al. Label refinery: Improving imagenet classification through label progression[J]. arXiv preprint arXiv:1805.02641, 2018.

Liu Y, Chen K, Liu C, et al. Structured knowledge distillation for semantic segmentation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 2604-2613.

Gupta S, Hoffman J, Malik J. Cross modal distillation for supervision transfer[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2827-2836.

Kang M, Mun J, Han B. Towards oracle knowledge distillation with neural architecture search[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(04): 4404-4411.

Chao H and Dong L. Emotion Recognition Using Three-Dimensional Feature and Convolutional Neural Network from Multichannel EEG Signals[J]. IEEE SENSORS JOURNAL, 2021, 21(2): 2024-2034.

Wang Z, Gu T, Zhu Y et al. FLDNet: Frame-level distilling neural network for EEG emotion recognition[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2533-2544.

Joshi V `M, Ghongade R B. EEG based emotion detection using fourth order spectral moment and deep learning[J]. Biomedical Signal Processing and Control, 2021, 68: 102755.

Wang Z, Wang Y, Zhang J, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-12.

Pandey P, Seeja K R. Subject independent emotion recognition from EEG using VMD and deep learning[J]. Journal of King Saud University-Computer and Information Sciences, 2022, 34(5): 1730-1738.

Xefteris, V, TSANOUSA A, GEORGAKOPOULOU N, et al. Graph Theoretical Analysis of EEG Functional Connectivity Patterns and Fusion with Physiological Signals for Emotion Recognition. Sensors, 2022. 22(21): 8198.

Gao Y, FU X L, OUYANG T X, et al. EEG-GCN: Spatio-Temporal and Self-Adaptive Graph Convolutional Networks for Single and Multi-View EEG-Based Emotion Recognition. IEEE SIGNAL PROCESSING LETTERS, 2022. 29: 1574-1578.