Published: 14th December 2017
Signing in a new future: IIT-BHU duo develop app that translates sign language into text
The app may already be one among many in the market but is the first developed for an Indian audience and will be independent of "any sort of hardware"
A student- teacher duo from IIT-BHU are working on an idea that could potentially revolutionise the way that hearing and speech impaired people communicate. The duo is developing an app that will translate sign language into text and speech.
Twenty-one-year-old Nikhil Dhaka is in his fourth year of an integrated B.Tech and M.Tech course in biomedical engineering from IIT-BHU. As a part of a project, he developed an app that recognised hand gestures. After taking a look at this project, the Head of Biomedical Engineering Department, Prof Neeraj Sharma suggested that Dhaka work on an app that could translate sign language which would make it easy to communicate between speech impaired and the general population.
Breakout idea: Nikhil Dhaka has always had a deep interest in software development
This is how Dhaka began to work on the app that could change the future of communication, "I've always had an interest in software development that's why I decided to work on the hand gesture recognition app but when my mentor Neeraj Sharma suggested the new idea, I realised how beneficial it could be. So it became really interesting to work on," he explained.
Sharma said that when he reviewed Dhaka's work it struck him that most deaf and mute people are only able to use sign language with their close family and friends who make an attempt to learn it. They are at a loss when it comes to the professional world because nobody around them makes an attempt to learn the sign language, "As long as the person is at home, they are fine but when they move out of their homes to pursue their careers they find it difficult to manage. That's when I thought that if the person has an app that can translate what they're saying and also help them understand what the other person is saying, it could solve so many problems."
Joint effort: Dhaka said that it was his mentor Prof Neeraj Sharma who first suggested the idea
The app is still under development and will be available in a year, Dhaka said. Currently, the app has the capacity to read the sign language and simultaneously type out the translation. The app will capture the gestures on live video and convert it into words. "There is still a slight delay in the translation but we're working on it as well as other small issues. We're trying to make the app as accurate as possible," Dhaka said.
The duo said there could be similar apps on the market already but they aren't available to the Indian population currently, "There are similar apps but they require some hardware, I'm trying to make this app completely independent of any sort of hardware." Sharma has more plans for the app in the future too"We are also trying to translate sign language into speech as well and not just restrict it to text. That will be the next thing we work on after this. "