TransFil: A Hybrid CNN–LSTM Framework for Multimodal Translation of Filipino Sign Language and Facial Expressions for Non-Signer Communication

Abstract

Filipino Sign Language (FSL) serves as the national sign language of the Philippines, yet its accessibility remains limited due to widespread unfamiliarity and the dominance of American Sign Language (ASL). Existing technologies for sign language translation often neglect crucial non-manual markers such as facial expressions, which play a key role in grammar and meaning. To address this gap, this study presents the development of an AI-powered FSL recognition system that integrates both manual gestures and facial expressions, focusing on greetings and common questions. Using a video-based dataset collected in controlled conditions, the system employs Media Pipe for landmark detection, Convolutional Neural Networks (CNNs) for spatial feature extraction, and Long Short-Term Memory (LSTM) networks for temporal modeling. Overall, The facial expression recognition (FER) model, built using a CNN architecture, was trained to identify six common emotions: angry, fear, happy, neutral, sad, and surprised. It performed strongly, achieving about 97% training accuracy and 95% validation accuracy, showing that it can generalize well to new data. While, the model demonstrated reliable performance, with precision (0.95), recall (0.94), and F1-score (0.95), making it suitable for real-world use. These results align with previous studies highlighting the strength of CNNs in capturing important facial features.