Towards effective multimodal emotion recognition in conversations via joint multi-graph learning and self-distillation
Published in Information Sciences, 2026 IF 6.0
Abstract (excerpt). Abstract Emotion Recognition in Conversation (ERC) is a crucial component of empathetic dialogue systems, enabling conversational agents to comprehend users’ expressed emotions and respond appropriately. In recent years, ERC has increasingly evolved toward multimodal approaches, particularly graph-based ones, that leverage complementary signals to enhance effectiveness. Although these approaches show promise, they face challenges in effectively modeling relational structures, computing the contribution of each modality during fusion, and regularizing unimodal representations to prevent modality bias. In this study, we propose DiGemo, a novel multimodal ERC framework that employs Transformer-based contextual modeling and graph-based learning to jointly capture both sequential dependencies within conversations and complex inter-utterance relational structures across modalities. To …
Recommended citation: Hoang Khang Trang, Bao Quoc Nguyen, Viet Vinh Khanh Dinh, Nhut Minh Nguyen, Nhat Truong Pham, Phuong-Nam Tran, Phuong Luu Vo, Duc Ngoc Minh Dang. (2026). "Towards effective multimodal emotion recognition in conversations via joint multi-graph learning and self-distillation." Information Sciences, 124138.
Download Paper