A System for Automatic Translation of Punjabi Text into Indian Sign Language (ISL) Using Synthetic Animations

Authors : Gurdeep Singh*, Vishal Goyal, Lalit Goyal

Abstract

In the changing realm of technology-based education, efficient communication tools for many language groups are crucial. This work presents an innovative automated translation system for Punjabi text to Indian Sign Language (ISL) that employs synthetic animations, marking a substantial progression in the domain. At now, there is no extensive translation system available for Punjabi, since current solutions are restricted to basic dictionaries. Our system is the first rule-based automated translation for Punjabi language, created using Python and its web development framework, Flask. The system includes a bilingual dictionary that contains 3,500 words, and there are plans for future expansion. The HamNoSys framework is employed to generate synthetic animations that faithfully depict ISL signals. These animations are validated by deaf students from a number of institutions in Punjab, their instructors, and ISL experts. The system's accuracy has been improved by the incorporation of natural language processing (NLP) tools, such as a Punjabi stemmer, part-of-speech tagger, and Punjabi to English transliterator. The system has achieved an accuracy of 80% for basic sentences and 70% for complex and compound sentences. This system is intended for practical use in public spaces, by deaf school instructors, and, particularly, by parents of deaf students who are only conversant with Punjabi.

Keywords: Indian Sign language, HamNosys, Punjabi Text to ISL, Sign Language Generation System, Synthetic Animations, and Indian Sign Language, ISL bilingual Dictionary. NLP tool for Deaf, Natural Language Processing.