Emotion recognition in conversations based on discourse parsing and graph attention network

HAO Xiulan; WEI Shaohua; CAO Qian; ZHANG Xiongtao

doi:10.11959/j.issn.1000-0801.2024149

您当前的位置：

首页 >

文章列表页 >

Emotion recognition in conversations based on discourse parsing and graph attention network

Research and Development | 更新时间：2024-08-05

- Emotion recognition in conversations based on discourse parsing and graph attention network
- Telecommunications Science Vol. 40, Issue 5, Pages: 100-111(2024)
- 作者机构：
  
  1.湖州师范学院浙江省现代农业资源智慧管理与应用研究重点实验室，浙江湖州 313002
  2.中国建设银行股份有限公司湖州分行，浙江湖州 313001
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62376094);Postgraduate Research and Innovation Project of Huzhou University(2024KYCX46;YJK24005)
- DOI：10.11959/j.issn.1000-0801.2024149
  CLC： TP391
- Received：12 January 2024，
  
  Revised：25 April 2024，
  
  Published：20 May 2024
- 稿件说明：
移动端阅览
郝秀兰,魏少华,曹乾等.基于语篇解析和图注意力网络的对话情绪识别[J].电信科学,2024,40(05):100-111.

HAO Xiulan,WEI Shaohua,CAO Qian,et al.Emotion recognition in conversations based on discourse parsing and graph attention network[J].Telecommunications Science,2024,40(05):100-111.
郝秀兰,魏少华,曹乾等.基于语篇解析和图注意力网络的对话情绪识别[J].电信科学,2024,40(05):100-111. DOI： 10.11959/j.issn.1000-0801.2024149.

HAO Xiulan,WEI Shaohua,CAO Qian,et al.Emotion recognition in conversations based on discourse parsing and graph attention network[J].Telecommunications Science,2024,40(05):100-111. DOI： 10.11959/j.issn.1000-0801.2024149.

摘要

对话情绪识别研究主要聚焦于融合对话上下文和说话者建模的相互关系。当前研究通常忽略对话内部存在的依存关系，导致对话的上下文联系不够紧密，说话者之间的关系也缺乏逻辑。因此，提出了一种基于语篇解析和图注意力网络（discourse parsing and graph attention network，DPGAT）的对话情绪识别模型，将对话内部的依存关系融入语境建模过程中，使语境信息更具有依赖性和全局性。首先，通过语篇解析获取对话内部的话语依存关系，构建语篇依存关系图和说话者关系图。随后，通过多头注意力机制将不同类型的说话者关系图进行内部融合。此外，在图注意力网络的基础上，结合依存关系进行循环学习，以达到上下文信息和说话人信息的有效融合，实现对话语境信息的外部融合。最终，通过分析内、外部融合的结果还原完整对话语境，并对说话者的情绪进行分析。通过在英文数据集MELD、EmoryNLP、DailyDialog和中文数据集M3ED上进行评估验证，F1分数分别为66.23%、40.03%、59.28%、52.77%，与主流的模型相比，所提模型具有较好的适用性，可在不同的语言场景中使用。

Abstract

The research on emotion recognition in conversations (ERC) focuses on the interrelationship between conversational context and speaker modeling. The current research usually ignores the dependency within the conversation

which leads to the weak connection between the context of the conversation and the lack of logic between the speakers. Therefore

an emotion recognition in conversations model based on discourse parsing and graph attention network (DPGAT) was proposed to integrate the inter-dependency of conversation into the context modeling to make contextual information more dependent and global. Firstly

the discourse dependency relationships within the conversation were obtained through discourse parsing

and the discourse dependency graph and the speaker relationship graph were constructed. Subsequently

different types of speaker diagrams were internally integrated by multi-head attention mechanisms. Based on the graph attention network

cyclic learning was combined with dependency relationships to achieve the effective integration of contextual information and speaker information

realizing the external integration of context-related information in conversations. Finally

by analyzing the results of internal and external integration

the complete conversation context was restored

and the speaker's emotions were analyzed. By evaluating and verifying on English dataset MELD

EmoryNLP

DailyDialog and Chinese dataset M3ED

F1 scores were 66.23%

40.03%

59.28% and 52.77%

respectively. Compared with mainstream models

the proposed model at least reaches state-of-the-art

and can be used in different language scenarios.

关键词

Keywords

references

王祥宇 , 宗成庆 . 基于监督对比学习的文本情绪类别表示 [J ] . 软件学报 , 2023 : 1 - 12 .

WANG X Y , ZONG C Q . Text emotion category representation based on supervised contrast learning [J ] . Journal of Software , 2023 : 1 - 12 .

赵妍妍 , 陆鑫 , 赵伟翔 , 等 . 情感对话技术综述 [J ] . 软件学报 , 2024 , 35 ( 3 ): 1377 - 1402 .

ZHAO Y Y , LU X , ZHAO W X , et al . Survey on emotional dialogue techniques [J ] . Journal of Software , 2024 , 35 ( 3 ): 1377 - 1402 .

FU Y , YUAN S Y , ZHANG C , et al . Emotion recognition in conversations: a survey focusing on context, speaker dependencies, and fusion methods [J ] . Electronics , 2023 , 12 ( 22 ): 4714 .

LI S M , YAN H , QIU X P . Contrast and generation make bart a good dialogue emotion recognizer [C ] // Proceedings of the 36th AAAI Conference on Artificial Intelligence . Palo Alto, CA : AAAI Press , 2022 : 11002 - 11010 .

陶涛 , 李珍 , 王冀彬 , 等 . 基于图神经网络的权益推荐技术方案研究 [J ] . 电信科学 , 2023 , 39 ( 8 ): 91 - 101 .

TAO T , LI Z , WANG J B , et al . Research on the graphical convolution neural network based benefits recommendation system strategy [J ] . Telecommunications Science , 2023 , 39 ( 8 ): 91 - 101 .

ZHANG D Z , CHEN F L , CHEN X Y . DualGATs: dual graph attention networks for emotion recognition in conversations [C ] // Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg, PA, USA : Association for Computational Linguistics , 2023 : 7395 - 7408 .

SHI Z , HUANG M . A deep sequential model for discourse parsing on multi-party dialogues [C ] // Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence . Palo Alto, CA : AAAI Press , 2019 , 33 ( 1 ): 7007 - 7014 .

VELIČKOVIĆ P , CUCURULL G , CASANOVA A , et al . Graph attention networks [EB ] . 2017 :

陈佳 , 章坚武 , 张浙亮 . 基于上下文信息与注意力特征的欺骗语音检测 [J ] . 电信科学 , 2023 , 39 ( 2 ): 92 - 102 .

CHEN J , ZHANG J W , ZHANG Z L . Spoof speech detection based on context information and attention feature [J ] . Telecommunications Science , 2023 , 39 ( 2 ): 92 - 102 .

PORIA S , CAMBRIA E , HAZARIKA D , et al . Context-dependent sentiment analysis in user-generated videos [C ] // Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: Long Papers) , Stroudsburg, PA : ACL , 2017 : 873 - 883 .

CHEN S Y , Hsu C C , KUO C C , et al . Emotionlines: An emotion corpus of multi-party conversations [C ] // Proceedings of the Eleventh International Conference on Language Resources and Evaluation . European Language Resources Association (ELRA) . rue Brillat-Savarin , Paris , 2018 : 1597 - 1601 .

PORIA S , HAZARIKA D , MAJUMDER N , et al . MELD: a multimodal multi-party dataset for emotion recognition in conversations [C ] // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg, PA, USA : Association for Computational Linguistics , 2019 : 527 - 536 .

LIU Z Y , CHEN N . Improving multi-party dialogue discourse parsing via domain integration [C ] // Proceedings of the 2nd Workshop on Computational Approaches to Discourse . Stroudsburg, PA, USA : Association for Computational Linguistics , 2021 : 122 - 127 .

ZHAO W , ZHAO Y , QIN B . MuCDN: mutual conversationaldetachment network for emotion recognition in multi-party conversations [C ] // Proceedings of the 29th International Conference on Computational Linguistics . Stroudsburg, PA : ACL , 2022 : 7020 - 7030 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . Red Hook, NY : Curran Associates, Inc. , 2017 : 1 - 15 .

王瑞琴 , 黄熠旻 , 纪其顺 , 等 . 注意力感知的边-节点交换图神经网络模型 [J ] . 电信科学 , 2024 , 40 ( 1 ): 106 - 114 .

WANG R Q , HUANG Y M , JI Q S , et al . Attention aware edge-node exchange graph neural network [J ] . Telecommunications Science , 2024 , 40 ( 1 ): 106 - 114 .

ZAHIRI S M , CHOI J D . Emotion detection on tv show transcripts with sequence-based convolutional neural networks [C ] // Workshops at the thirty-second AAAI conference on artificial intelligence . Palo Alto, CA : AAAI Press , 2018 : 44 - 51 .

LI Y R , SU H , SHEN X Y , et al . Dailydialog: a manually labelled multi-turn dialogue dataset [C ] // Proceedings of the Eighth International Joint Conference on Natural Language Processing , Volume 1 (Long Papers) ACL-IJCNLP 2017 . Stroudsburg, PA : ACL , 2017 : 986 - 995 .

ZHAO J M , ZHANG T G , HU J W , et al . M3ED: multi-modal multi-scene multi-label emotional dialogue database [C ] // Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics , Volume 1 (Long Papers) , Stroudsburg, PA : ACL , 2022 : 5699 - 5710 .

HU D , BAO Y N , WEI L W , et al . Supervised adversarial contrastive learning for emotion recognition in conversations .[C ] // Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers) , Toronto, Canada. ACL , 2023 : 10835 - 10852 .

GAN C Q , ZHENG J H , ZHU Q Y , et al . A graph neural network with context filtering and feature correction for conversational emotion recognition [J ] . Information Sciences , 2024 , 658 : 120017 .

HU J W , LIU Y C , ZHAO J M , et al . MMGCN: Multimodal fusion via deep graph convolution network for emotion recognition in conversation [C ] // Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing , Volume 1 (Long Papers) , Stroudsburg, PA : ACL , 2021 : 5666 - 5675 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰