EEG emotion recognition based on parallel separable convolution and label smoothing regularization

Yong ZHANG; Jikui LIU; Wenlong KE

doi:10.11959/j.issn.1000-0801.2023112

您当前的位置：

首页 >

文章列表页 >

EEG emotion recognition based on parallel separable convolution and label smoothing regularization

Research and Development | 更新时间：2024-06-05

- EEG emotion recognition based on parallel separable convolution and label smoothing regularization
- Telecommunications Science Vol. 39, Issue 5, Pages: 116-128(2023)
- 作者机构：
  
  1. 湖州师范学院信息工程学院，浙江湖州 313000
  2. 辽宁师范大学计算机与信息技术学院，辽宁大连 116081
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2023112
  CLC： TP391
- Published Online：2023-05，
  
  Published：20 May 2023
- 稿件说明：
移动端阅览
Yong ZHANG, Jikui LIU, Wenlong KE. EEG emotion recognition based on parallel separable convolution and label smoothing regularization[J]. Telecommunications science, 2023, 39(5): 116-128.
DOI：

Yong ZHANG, Jikui LIU, Wenlong KE. EEG emotion recognition based on parallel separable convolution and label smoothing regularization[J]. Telecommunications science, 2023, 39(5): 116-128. DOI： 10.11959/j.issn.1000-0801.2023112.

摘要

近年来，基于深度学习和脑电图（EEG）的情感识别方法取得了较好的效果。然而，现有方法依然存在脑电情感特征提取不够全面、受人工错误标注的情感标签影响较大等问题。对此，提出了并行可分离卷积和标签平滑正则化（PSC-LSR）网络模型。首先，通过注意力机制，赋予EEG重要时间点和重要通道更大的权重，得到 EEG 的浅层情感特征；其次，采用并行可分离卷积模块全面提取 EEG 情感信息，得到深层情感特征；最后，在优化模型参数时采用了情感标签平滑正则化方法，使模型对错误标签有更大的容错概率，增强了网络模型的泛化性和鲁棒性，提高了脑电情感识别的准确率。提出的方法在两个数据集进行了验证，其中，在DEAP数据集中，唤醒和效价两个维度的平均准确率分别达到了99.23%和99.13%；在Dreamer数据集中，唤醒和效价两个维度的平均准确率分别达到了97.33%和97.25%。

Abstract

In recent years

emotion recognition methods based on deep learning and electroencephalogram (EEG) have achieved good results.However

existing methods still have issues such as incomplete extraction of emotional features from EEG and significant impact from artificially mislabeled emotional labels.A parallel separable convolution and label smoothing regularization (PSC-LSR) network model was proposed.Firstly

through the attention mechanism

EEG important time points and important channels were given greater weight to obtain shallow emotional features of EEG.Secondly

a parallel separable convolution module was used to comprehensively extract EEG emotional information and obtain deep emotional features.Finally

the emotion label smoothing regularization method was used to optimize the model parameters

which increased model’s fault tolerance probability for incorrect labels

enhanced the generalization and robustness of the network model

and improved accuracy of EEG emotion recognition.The proposed method has been validated in two datasets

in which the average accuracy rates of arousal and valence dimensions in the DEAP dataset reaches 99.23% and 99.13%

respectively.In the Dreamer dataset

the average accuracy rates for both arousal and valence dimensions reaches 97.33% and 97.25%.

关键词

Keywords

references

GAO Z K , LI Y L , YANG Y X , et al . A GPSO-optimized convolutional neural networks for EEG-based emotion recognition [J ] . Neurocomputing , 2020 ( 380 ): 225 - 235 .

CHEN G J , ZHANG X Y , SUN Y , et al . Emotion feature analysis and recognition based on reconstructed EEG sources [J ] . IEEE Access , 2020 ( 8 ): 11907 - 11916 .

WANG Z M , ZHOU X X , WANG W L , et al . Emotion recognition using multimodal deep learning in multiple psychophysiological signals and video [J ] . International Journal of Machine Learning and Cybernetics , 2020 , 11 ( 4 ): 923 - 934 .

KESSOUS L , CASTELLANO G , CARIDAKIS G . Multimodal emotion recognition in speech-based interaction using facial expression,body gesture and acoustic analysis [J ] . Journal on Multimodal User Interfaces , 2010 , 3 ( 1 ): 33 - 48 .

ANDERSON K , MCOWAN P W . A real-time automated system for the recognition of human facial expressions [J ] . IEEE Transactions on Systems,Man,and Cybernetics Part B,Cybernetics:a Publication of the IEEE Systems,Man,and Cybernetics Society , 2006 , 36 ( 1 ): 96 - 105 .

刘晓旻 , 谭华春 , 章毓晋 . 人脸表情识别研究的新进展 [J ] . 中国图象图形学报 , 2006 , 11 ( 10 ): 1359 - 1368 .

LIU X M , TAN H C , ZHANG Y J . New research advances in facial expression recognition [J ] . Journal of Image and Graphics , 2006 , 11 ( 10 ): 1359 - 1368 .

刘玉娟 , 方富熹 . 情绪的语音交流 [J ] . 中国行为医学科学 , 2007 , 16 ( 4 ): 374 - 376 .

LIU J Y , FANG F X . Voice communication of emotions [J ] . Chinese Journal of Behavioral Medical and Brain Science , 2007 , 16 ( 4 ): 374 - 376 .

ANG J , DHILLON R , KRUPSKI A , et al . Prosody-based automatic detection of annoyance and frustration in human-computer dialog [C ] // Proceedings of 7th International Conference on Spoken Language Processing (ICSLP 2002) .[S.l.:s.n. ] , 2002 : 2037 - 2040 .

EKMAN P . An argument for basic emotions [J ] . Cognition and Emotion , 1992 , 6 ( 3/4 ): 169 - 200 .

PLUTCHIK R . The nature of emotions:human emotions have deep evolutionary roots,a fact that may explain their complexity and provide tools for clinical practice [J ] . American Scientist , 2001 , 89 ( 4 ): 344 - 350 .

RUSSELL J A . A circumplex model of affect [J ] . Journal of Personality and Social Psychology , 1980 , 39 ( 6 ): 1161 - 1178 .

MEHRABIAN A . Comparison of the PAD and PANAS as models for describing emotions and for differentiating anxiety from depression [J ] . Journal of Psychopathology and Behavioral Assessment , 1997 , 19 ( 4 ): 331 - 357 .

SHU L , XIE J Y , YANG M Y , et al . A review of emotion recognition using physiological signals [J ] . Sensors (Basel,Switzerland) , 2018 , 18 ( 7 ): 2074 .

DAI Y X , WANG X , ZHANG P B , et al . Sparsity constrained differential evolution enabled feature-channel-sample hybrid selection for daily-life EEG emotion recognition [J ] . Multimedia Tools and Applications , 2018 , 77 ( 17 ): 21967 - 21994 .

YU X Y , CHUM P , SIM K B . Analysis the effect of PCA for feature reduction in non-stationary EEG based motor imagery of BCI system [J ] . Optik , 2014 , 125 ( 3 ): 1498 - 1502 .

FAYEK H M , LECH M , CAVEDON L . Modeling subjectiveness in emotion recognition with deep neural networks:ensembles vs soft labels [C ] // Proceedings of 2016 International Joint Conference on Neural Networks (IJCNN) . Piscataway:IEEE Press , 2016 : 566 - 570 .

RINGEVAL F , EYBEN F , KROUPI E , et al . Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data [J ] . Pattern Recognition Letters , 2015 ( 66 ): 22 - 30 .

ZHONG P X , WANG D , MIAO C Y . EEG-based emotion recognition using regularized graph neural networks [J ] . IEEE Transactions on Affective Computing , 2022 , 13 ( 3 ): 1290 - 1301 .

PORBADNIGK A K , GÖRNITZ N , SANNELLI C , et al . When brain and behavior disagree:tackling systematic label noise in EEG data with machine learning [C ] // Proceedings of 2014 International Winter Workshop on Brain-Computer Interface (BCI) . Piscataway:IEEE Press , 2014 : 1 - 4 .

SZEGEDY C , VANHOUCKE V , IOFFE S , et al . Rethinking the inception architecture for computer vision [C ] // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2016 : 2818 - 2826 .

KHALILI Z , MORADI M H . Emotion recognition system using brain and peripheral signals:using correlation dimension to improve the results of EEG [C ] // Proceedings of 2009 International Joint Conference on Neural Networks . Piscataway:IEEE Press , 2009 : 1571 - 1575 .

LIN Y P , WANG C H , JUNG T P , et al . EEG-based emotion recognition in music listening [J ] . IEEE Transactions on Biomedical Engineering , 2010 , 57 ( 7 ): 1798 - 1806 .

MURUGAPPAN M , RIZON M , NAGARAJAN R , et al . Time-frequency analysis of EEG signals for human emotion detection [C ] // Proceedings of 4th Kuala Lumpur International Conference on Biomedical Engineering 2008 . Heidelberg:Springer , 2008 : 262 - 265 .

KHOSROWABADI R , QUEK H C , WAHAB A , et al . EEG-based emotion recognition using self-organizing map for boundary detection [C ] // Proceedings of 2010 20th International Conference on Pattern Recognition . Piscataway:IEEE Press , 2010 : 4242 - 4245 .

LIU W , ZHENG W L , LU B L . Emotion recognition using multimodal deep learning [C ] // Proceedings of the 23rd International Conference on Neural Information Processing Springer International Publishing . Heidelberg:Springer , 2016 : 521 - 529 .

HASSAN M M , ALAM M G R , UDDIN M Z , et al . Human emotion recognition using deep belief network architecture [J ] . Information Fusion , 2019 ( 51 ): 10 - 18 .

VIJAYAKUMAR S , FLYNN R , MURRAY N . A comparative study of machine learning techniques for emotion recognition from peripheral physiological signals [C ] // Proceedings of 2020 31st Irish Signals and Systems Conference (ISSC) . Piscataway:IEEE Press , 2020 : 1 - 6 .

MERT A , AKAN A . Emotion recognition from EEG signals by using multivariate empirical mode decomposition [J ] . Pattern Analysis and Applications , 2018 , 21 ( 1 ): 81 - 89 .

WU S Y , XU X M , SHU L , et al . Estimation of valence of emotion using two frontal EEG channels [C ] // Proceedings of 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) . Piscataway:IEEE Press , 2017 : 1127 - 1130 .

CHENG C L , WEI X W , JIAN Z . Emotion recognition algorithm based on convolution neural network [C ] // Proceedings of 2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE) . Piscataway:IEEE Press , 2018 : 1 - 5 .

KWON Y H , SHIN S B , KIM S D . Electroencephalography based fusion two-dimensional (2D)-convolution neural networks (CNN) model for emotion recognition system [J ] . Sensors , 2018 , 18 ( 5 ): 1383 .

CUI H , LIU A P , ZHANG X , et al . EEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network [J ] . Knowledge-Based Systems , 2020 ( 205 ): 106243 .

LI D D , CHAI B , WANG Z , et al . EEG emotion recognition based on 3-D feature representation and dilated fully convolutional networks [J ] . IEEE Transactions on Cognitive and Developmental Systems , 2021 , 13 ( 4 ): 885 - 897 .

LIU W , QIU J L , ZHENG W L , et al . Comparing recognition performance and robustness of multimodal deep learning models for multimodal emotion recognition [J ] . IEEE Transactions on Cognitive and Developmental Systems , 2022 , 14 ( 2 ): 715 - 729 .

LI C , LIN X J , LIU Y , et al . EEG-based emotion recognition via efficient convolutional neural network and contrastive learning [J ] . IEEE Sensors Journal , 2022 , 22 ( 20 ): 19608 - 19619 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York:ACM Press , 2017 : 6000 - 6010 .

HOWARD A G , ZHU M L , CHEN B , et al . MobileNets:efficient convolutional neural networks for mobile vision applications [J ] . arXiv preprint , 2017 ,arXiv:1704.04861.

KOELSTRA S , MUHL C , SOLEYMANI M , et al . DEAP:a database for emotion analysis; using physiological signals [J ] . IEEE Transactions on Affective Computing , 2012 , 3 ( 1 ): 18 - 31 .

KATSIGIANNIS S , RAMZAN N . DREAMER:a database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices [J ] . IEEE Journal of Biomedical and Health Informatics , 2018 , 22 ( 1 ): 98 - 107 .

SONG T F , ZHENG W M , SONG P , et al . EEG emotion recognition using dynamical graph convolutional neural networks [J ] . IEEE Transactions on Affective Computing , 2020 , 11 ( 3 ): 532 - 541 .

CHENG J , CHEN X . Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism [J ] . Computers in Biology and Medicine , 2022 ( 143 ): 105303 .

CHENG J , CHEN M Y , LI C , et al . Emotion recognition from multi-channel EEG via deep forest [J ] . IEEE Journal of Biomedical and Health Informatics , 2021 , 25 ( 2 ): 453 - 464 .

Views

527

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

GMTBLC: a deep learning-based bi-modal network traffic classification method

Research on new energy station network security assessment method based on improved LSTM network

Dynamic heterogeneous network representation learning method based on Hawkes process

Modulation recognition driven by signal enhancement

Multi-adversarial domain adaptation method based on feature correction

Related Author

WANG Xinrui

WEN Jinglong

JIANG Qinlong

WEI Debin

WANG Yao

LI Rui

LIU Shan

LIU Xingyan

Related Institution

School of Information Engineering, Dalian University

State Grid Shanxi Electric Power Company

State Grid Shanxi Electric Power Research Institute

Key Laboratory of Medical Electronics and Digital Health of Zhejiang Province， Jiaxing University

School of Computer Science and Technology (School of Artificial Intelligence), Zhejiang Normal University

⁰