MERSA: Multimodal Emotion Recognition with Self-Align Embedding

Abstract (excerpt). Emotions are an integral part of human communication and interaction, significantly shaping our social connections, decision-making, and overall well-being. Understanding and analyzing emotions have become essential in various fields, including psychology, human-computer interaction, marketing, and healthcare. The previous approach has indeed made significant strides in improving the accuracy of predicting emotions within speech. However, the current model’s performance still falls short when it comes to real-life applications. This limitation arises due to several factors such as lack of context, ambiguity in speech and meaning, and other contributing elements. To reduce the ambiguity of emotions within speech, this paper seeks to leverage multiple data modalities, specifically textual and acoustic information. To analyze these modalities, we propose a novel approach called MERSA which utilizes the self …

Read the full paper

Recommended citation: Quan Bao Le, Kiet Tuan Trinh, Nguyen Dinh Hung Son, Phuong-Nam Tran, Cuong Tuan Nguyen, Duc Ngoc Minh Dang. (2024). "MERSA: Multimodal Emotion Recognition with Self-Align Embedding." 2024 International Conference on Information Networking (ICOIN), 500-505.
Download Paper