Sign Language Recognition Methods: Applications and Advances of Deep Learning Technology
DOI:
https://doi.org/10.54097/ryyycp94Keywords:
Sign Language Recognition; CNN; RNN; Transformer Models.Abstract
In the past few years, the field of sign language recognition (SLR) has attracted widespread attention. Deep learning models demonstrate their significant application potential with their outstanding performance. This paper reviews the application and the advancement of deep learning in the field of SLR, focusing on the application of models, such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory Networks. (LSTM), and Transformers in SLR. This paper systematically analyzes the key step of SLR, including data collection, preprocessing, model training, and performance evaluation. On this basis, this paper discusses the potential of MediaPipe's hand keypoint detection and the integration of CNN, 3D-CNN, and 2D-CNN fusion models, RNN-Transducer and H2SNet frameworks, as well as Transformer-based models, in improving the accuracy of SLR. The aim is to promote the development of accessible communication technologies and enhance the quality of life for those who are hearing-impaired community. This effort also seeks to provide a theoretical foundation and practical guidance for future technological advancements and societal inclusivity.
Downloads
References
[1] Azevedo R V, Coutinho T M, Ferreira J P, Gomes T L, Nascimento E R. Empowering sign language communication: Integrating sentiment and semantics for facial expression synthesis. Computers & Graphics, 2024, 124: 104065von Agris U., Zieren J., Canzler U., Bauer B., Kraiss K.-F. Recent developments in visual sign language recognition. Universal Access in the Information Society, 2007, 6(1): 123-141.
[2] Jiang, X., Sun, J., Zhang, Y., Wang, L., & Han, X. (2024). Review of Chinese Sign Language Recognition Methods and Technologies. Theoretical and Research, 2024(3), 47-58.
[3] Zuo R, Wei F, Mak B. Natural Language-Assisted Sign Language Recognition. arXiv:2303.12080v1 [cs.CV], 2023 Mar 21.
[4] von Agris, U., Zieren, J., Canzler, U., Bauer, B., & Kraiss, K.-F. (2007). Recent developments in visual sign language recognition. Transactions of China Electrotechnical Society, 19(10), 53-58.
[5] Islam M.Z., Hossain M.S., Islam R.U., Andersson K. Static Hand Gesture Recognition using Convolutional Neural Network with Data Augmentation. Transactions of China Electrotechnical Society, 2004, 19(10): 53-58.
[6] Rainio, O., Teuho, J. & Klén, R. Evaluation metrics and statistical tests for machine learning. Scientific Reports, 2024, 14: 6086.
[7] Gagandeep Singh, Aditya Raj Verma, Banavath RamJi, Karnim Meghwal, Praveen Kumar Dadheech. Enhancing Sign Language Detection through Mediapipe and Convolutional Neural Networks (CNN). Transactions of China Electrotechnical Society, 2024, 19(10): 53-58.
[8] Cheng Q.Y., Sun C.X., Zhang X.X., et al. Short-Term load forecasting model and method for power system based on complementation of neural network and fuzzy logic. Transactions of China Electrotechnical Society, 2004, 19(10): 53-58.
[9] Al-Hammadi M, Muhammad G, Abdul W, Alsulaiman M, Bencherif M A, Mekhtiche M A. Hand Gesture Recognition for Sign Language Using 3DCNN. IEEE Access, 2020, 8: 79491-79509.
[10] Gao Liqing, Li Haibo, Liu Zhijian, Liu Zekang, Wan Liang, Feng Wei. RNN-Transducer based Chinese Sign Language Recognition. Neurocomputing, 2021, 434: 45-54.
[11] Du Yao, Xie Pan, Wang Mingye, Hu Xiaohui, Zhao Zheng, Liu Jiaqi. Full transformer network with masking future for word-level sign language recognition. Neurocomputing, 2022, 500: 115-123.
[12] Cui Z, Zhang W, Li Z, Wang Z. Spatial-temporal transformer for end-to-end sign language recognition. Complex Intell Syst, 2023, 9: 4645-4656.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Highlights in Science, Engineering and Technology

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.







