Publications | Phuong-Nam Tran, Ph.D.c

denotes journal

denotes conference

2025

SwinTExCo: Exemplar-based video colorization using Swin Transformer

Duong Thanh Tran, Doan Hieu Nguyen Nguyen, Trung Thanh Pham, Phuong-Nam Tran, Thuy-Duong Thi Vu, Cuong Tuan Nguyen, Hanh Dang-Ngoc, and Duc Ngoc Minh Dang

Expert Systems with Applications, 2025

Abs HTML Code

Video colorization represents a compelling domain within the field of Computer Vision. The traditional approach in this field relies on Convolutional Neural Networks (CNNs) to extract features from each video frame and employs a recurrent network to learn information between video frames. While demonstrating considerable success in colorization, most traditional CNNs suffer from a limited receptive field size, capturing local information within a fixed-sized window. Consequently, they struggle to directly grasp long-range dependencies or pixel relationships that span large image or video frame areas. To address this limitation, recent advancements in the field have leveraged Vision Transformer (ViT) and their variants to enhance performance. This article introduces Swin Transformer Exemplar-based Video Colorization (SwinTExCo), an end-to-end model for the video colorization process that incorporates the Swin Transformer architecture as the backbone. The experimental results demonstrate that our proposed method outperforms many other state-of-the-art methods in both quantitative and qualitative metrics. The achievements of this research have significant implications for the domain of documentary and history video restoration, contributing to the broader goal of preserving cultural heritage and facilitating a deeper understanding of historical events through enhanced audiovisual materials.

2024

EAI INISCOM

Performance Comparison in Traffic Sign Recognition using Deep Learning

Huu Duc Nguyen, Nam Vo Thanh, Phuong-Nam Tran, Cuong Tuan Nguyen, and Duc Ngoc Minh Dang

In Industrial Networks and Intelligent Systems, 2024

Abs HTML

In recent years, along with the increase in private cars, traffic signs have increased in quantity, demanding greater attentiveness from drivers. Many studies have been conducted on Traffic Sign Recognition (TSR) to enhance road safety and driver assistance systems by enabling vehicles to autonomously detect and interpret traffic signs, providing crucial information to drivers in real-time. This paper examines various traffic sign detection and classification models, which give practitioners and researchers valuable insights into selecting optimal solutions tailored to the needs of real-world applications. Notably, the investigation highlights YOLOv8 as a leading detection model, displaying exceptional results with an mAP of 99.4%. YOLOv8 provides various model sizes allowing for adaptation to specific real-time processing scenarios. On the other hand, the LeNet model is a standout performer in the classification domain, consistently achieving a remarkable accuracy of 98.2% while using only 0.4 million parameters. The LeNet architecture ensures accurate and rapid traffic sign classification, making it an appealing choice for applications where resource efficiency is critical.
MERSA: Multimodal Emotion Recognition with Self-Align Embedding

Quan Bao Le, Kiet Tuan Trinh, Nhat Truong Pham, Dinh Hung Son Nguyen, Phuong-Nam Tran, Cuong Tuan Nguyen, and Duc Ngoc Minh Dang

In 2024 International Conference on Information Networking (ICOIN), 2024

Abs HTML

Emotions are an integral part of human communication and interaction, significantly shaping our social connections, decision-making, and overall well-being. Understanding and analyzing emotions have become essential in various fields, including psychology, human-computer interaction, marketing, and healthcare. The previous approach has indeed made significant strides in improving the accuracy of predicting emotions within speech. However, the current model’s performance still falls short when it comes to real-life applications. This limitation arises due to several factors such as lack of context, ambiguity in speech and meaning, and other contributing elements. To reduce the ambiguity of emotions within speech, this paper seeks to leverage multiple data modalities, specifically textual and acoustic information. To analyze these modalities, we propose a novel approach called MERSA which utilizes the self-align method to extract context features from both textual and acoustic information. By leveraging this technique, the MERSA model can effectively create fusion feature vectors of the multiple inputs, facilitating a more accurate and holistic analysis of emotions within speech. Moreover, the MERSA model has incorporated a cross-attention module into its network architecture, which enables the MERSA model to capture and leverage the interdependencies between the textual and acoustic modalities.
ICIIT

Deep Learning-Based Automated Cashier System for Bakeries

Nam Van Hai Phan, Tha Thanh Le, Tuan Phu Phan, Thu Thuy Le, Phuong-Nam Tran, Nhat Truong Pham, and Duc Ngoc Minh Dang

In Proceedings of the 2024 9th International Conference on Intelligent Information Technology, 2024

Abs HTML

The application of image recognition in the bakery business has paved the way for automatic payment systems, a significant advancement in the field of computer vision. This article delves into an exploration of advanced image recognition models to meticulously assess their effectiveness, speed, and suitability for seamless integration into specialized automatic payment systems tailored for bakeries. Specifically, YOLOX, YOLOv8, Faster R-CNN, and RetinaNet, each with different versions and backbones, are considered for evaluation based on their speed and performance. Notably, this study introduces a streamlined process for rapidly creating custom datasets for object detection research and evaluates models across these datasets. The insights and analyses derived from this study provide valuable perspectives for optimizing processes and enhancing the overall performance of automatic payment systems within bakeries.
ICIIT

Innovative Multi-Modal Control for Surveillance Spider Robot: An Integration of Voice and Hand Gesture Recognition

Dang Khoa Phan, Phuong-Nam Tran, Nhat Truong Pham, Tra Huong Thi Le, and Duc Ngoc Minh Dang

In Proceedings of the 2024 9th International Conference on Intelligent Information Technology, 2024

Abs HTML

The spider robot is designed to take on challenging tasks in hazardous conditions. It can move across challenging terrain like walls and rough surfaces, and effectively find lost objects. In this paper, an innovative multi-modal control approach was developed for the Surveillance Spider Robot (SSR) application, integrating voice recognition and hand gesture recognition as control commands. SSR, a six-legged robot, was designed using a Raspberry Pi 4B embedded device, Arduino Uno kit, RC Servo motors (MG996R), 18650 batteries, mini USB microphone (MI-350), Pi camera V1 (OV5647) and PWM generator (PCA9685). The robot can be controlled through voice or hand gesture recognition captured via camera and microphone. SSR is capable of performing ten specific tasks based on these control commands, including forward movement, backward movement, left turns, right turns, complete turns, movements with higher or lower centers of gravity, slow movement, body-hopping, and stopping. The performance evaluation of voice and hand gesture recognition suggested that SSR can be used in real-world applications with an accuracy that exceeds 90% for the ten specific tasks.
ICIIT

Vietnamese Traffic Sign Recognition Using Deep Learning

Dinh Thuan Nguyen, Minh Khanh Phan, Phuong-Nam Tran, and Duc Ngoc Minh Dang

In Proceedings of the 2024 9th International Conference on Intelligent Information Technology, 2024

Abs HTML

Recognizing the importance of road safety, traffic signs are designed in a variety of shapes, colors, and images to guide and alert drivers. Overlooking these signs, particularly the crucial ones, due to various factors, can lead to legal violations or even catastrophic accidents, impacting multiple aspects of our lives. Consequently, the use of computer systems for traffic sign recognition is urgently needed for the safety of commuters. This not only plays a pivotal role in improving our continually evolving technology but also has substantial value in academic research. Due to the complexity of terrain and the influence of weather on outcomes, Traffic Sign Recognition (TSR) remains a high-priority challenge. This article focuses on the creation of a Vietnamese traffic sign dataset comprising 46 classes. Subsequently, we implement various detection and recognition models using this dataset to assess their performance.
ICTC

Towards Real-Time Vietnamese Traffic Sign Recognition on Embedded Systems

Phuong-Nam Tran, Nhat Truong Pham, Nam Van Hai Phan, Duc Tai Phane, Cuong Tuan Nguyen, and Duc Ngoc Minh Dang

In 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), 2024

HTML
ICTC

Enhancing Speech Emotion Recognition Through Knowledge Distillation

Trung Minh Nguyen, Phuong-Nam Tran, and Duc Ngoc Minh Dang

In 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), 2024

HTML
ICTC

Voice-Based Age and Gender Recognition: A Comparative Study of LSTM, RezoNet and Hybrid CNNs-BiLSTM Architecture

Nhut Minh Nguyen, Thanh Trung Nguyen, Hua Hiep Nguyen, Phuong-Nam Tran, and Duc Ngoc Minh Dang

In 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), 2024

HTML
ICTC

Federated Learning with U-Net for Brain Tumor Segmentation: Impact of Client Numbers and Data Distribution

Thu Thuy G, Nhat Truong Pham G, Phuong-Nam Tran G, and Duc Ngoc Minh Dang

In 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), 2024

HTML

2023

Multi-modal Speech Emotion Recognition: Improving Accuracy Through Fusion of VGGish and BERT Features with Multi-head Attention

Phuong-Nam Tran, Thuy-Duong Thi Vu, Nhat Truong Pham, Duc Ngoc Minh Dang, and Anh-Khoa Tran

In Industrial Networks and Intelligent Systems, 2023

Abs HTML Code

Recent research has shown that multi-modal learning is a successful method for enhancing classification performance by mixing several forms of input, notably in speech-emotion recognition (SER) tasks. However, the difference between the modalities may affect SER performance. To overcome this problem, a novel approach for multi-modal SER called 3M-SER is proposed in this paper. The 3M-SER leverages multi-head attention to fuse information from multiple feature embeddings, including audio and text features. The 3M-SER approach is based on the SERVER approach but includes an additional fusion module that improves the integration of text and audio features, leading to improved classification performance. To further enhance the correlation between the modalities, a LayerNorm is applied to audio features prior to fusion. Our approach achieved an unweighted accuracy (UA) and weighted accuracy (WA) of 79.96% and 80.66%, respectively, on the IEMOCAP benchmark dataset. This indicates that the proposed approach is better than SERVER and recent methods with similar approaches. In addition, it highlights the effectiveness of incorporating an extra fusion module in multi-modal learning.
Comparative analysis of multi-loss functions for enhanced multi-modal speech emotion recognition

Phuong-Nam Tran, Thuy-Duong Thi Vu, Nhat Truong Pham, Hanh Dang-Ngoc, and Duc Ngoc Minh Dang

In 2023 14th International Conference on Information and Communication Technology Convergence (ICTC), 2023

Abs HTML Code

In recent years, multi-modal analysis has gained significant prominence across domains such as audio/speech processing, natural language processing, and affective computing, with a particular focus on speech emotion recognition (SER). The integration of data from diverse sources, encompassing text, audio, and images, in conjunction with classifier algorithms has led to the realization of enhanced performance in SER tasks. Traditionally, the cross-entropy loss function has been employed for the classification problem. However, it is challenging to discriminate the feature representations among classes for multi-modal classification tasks. In this study, we focus on the impact of the loss functions on multi-modal SER rather than designing the model architecture. Mainly, we evaluate the performance of multi-modal SER with different loss functions, such as cross-entropy loss, center loss, contrastive-center loss, and their combinations. Based on extensive comparative analysis, it is proven that the combination of cross-entropy loss and contrastive-center loss achieves the best performance for multi-modal SER. This combination reaches the highest accuracy of 80.27% and the highest balanced accuracy of 81.44% on the IEMOCAP dataset.
RBBA: ResNet - BERT - Bahdanau Attention for Image Caption Generator

Duc-Hieu Hoang, Anh-Khoa Tran, Duc Ngoc Minh Dang, Phuong-Nam Tran, Hanh Dang-Ngoc, and Cuong Tuan Nguyen

In 2023 14th International Conference on Information and Communication Technology Convergence (ICTC), 2023

Abs HTML

In recent years, the topic of image caption generators has gained significant attention. Several successful projects have emerged in this field, showcasing notable advancements. Image caption generators automatically generate descriptive captions for images through the encoder and decoder mechanisms. The encoder leverages computer vision models, while the decoder utilizes natural language processing models. In this study, we aim to assess a comprehensive set of seven distinct methodologies, including six existing methods from prior research and one newly proposed. These methods are trained and evaluated with bilingual evaluation (BLEU) on the Flickr8K dataset. In our experiments, the proposed ResNet50 - BERT - Bahdanau Attention model outperforms the other models in terms of the BLEU-1 score of 0.532143 and BLEU-4 score of 0.126316.
Vitexco: Exemplar-based Video Colorization using Vision Transformer

Duong Thanh Tran, Phuong-Nam Tran, Doan Hieu Nguyen Nguyen, Thuy-Duong Thi Vu, Trung Thanh Pham, and Duc Ngoc Minh Dang

In 2023 14th International Conference on Information and Communication Technology Convergence (ICTC), 2023

Abs HTML

In the field of image and video colorization, the existing research employs a CNN to extract information from each video frame. However, due to the local nature of a kernel, it is challenging for CNN to capture the relationships between each pixel and others in an image, leading to inaccurate colorization. To solve this issue, we introduce an end-to-end network called Vitexco for colorizing videos. Vitexco utilizes the power of the Vision Transformer (ViT) to capture the relationships among all pixels in a frame with each other, providing a more effective method for colorizing video frames. We evaluate our approach on DAVIS datasets and demonstrate that it outperforms the state-of-the-art methods regarding color accuracy and visual quality. Our findings suggest that using a ViT can significantly enhance the performance of video colorization.