This app developed by 20-year-old Mumbai students makes use of AR to help dyslexic children read better

Augmenta11y, an app developed by 20-year-old students from Mumbai uses Augmented Reality (AR) to help dyslexic children read and learn better
Augmenta11y was launched recently in March on iOS and Android smartphones and tablets and it is available for free for both platforms (Pic: Tushar)
Augmenta11y was launched recently in March on iOS and Android smartphones and tablets and it is available for free for both platforms (Pic: Tushar)

One of the most common learning disabilities prevalent in today's classrooms is Dyslexia. According to several reports, there are approximately 228,994,454 students enrolled in recognised schools, which brings the count of dyslexic children to nearly 35 million alone in India. While most parents and teachers find it difficult to deal with children with this learning disability and then eventually give up on them, it is extremely important to identify if your child is dyslexic. Because they need special care. In order to provide a viable solution to this problem, four students (all in their 20s) from the Mukesh Patel School of Technology, Management and Engineering, the engineering school of NMIMS University, Mumbai have developed an app to help dyslexic children learn and understand words better.

Tushar Gupta, Mudita Sisodia, Schezeen Fazulbhoy, and Mitali Raju created the Augmenta11y app which is a one-stop platform that aims at making life easier on a regular basis, simplifying the process of interacting with objects and text-based data (such as billboards, street signs, hard copy documents, online data) that a dyslexic child may encounter with through the course of their day.
"The four of us were quite determined that we wanted to work on accessibility for our bachelors’ thesis project. We explored various issues pertaining to this domain – such as color blindness, vision impairment, and dyslexia. Then finally, we decided to move forward with dyslexia because it was one of the disabilities where we felt that new technologies weren’t being used to their full potential. It serves as the ultimate assistant while dealing with the effects of a condition such as dyslexia," says Tushar, one of the developers of the app.

So how does it work?

The main features of the app include:

1. Real-time text detection: Augmenta11y uses an offline machine learning model stored on the user’s device to provide instant text detection using Optical Character Recognition (OCR). This is one of its distinguishing features from other apps that require the user to click a photo and wait for results.

2. Research-backed settings panel for improved readability: 
- There are options to change the font style of the text, including a special font, called Open Dyslexic designed by researchers specifically for people with dyslexia. The font helps the user focus on the words. This feature helps address some of the challenges faced by children with dyslexia, between the ages of seven to 14. These include confusion between alphabets like ‘b’-‘d’, ‘a’-‘o’, ‘u’-‘v’, jumping between lines, and losing track of the sentence being read.

- Augmenta11y also allows the user to adjust line height and letter spacing of the text, making it easier to distinguish characters from each other. 

- Users can also change the contrast ratio of the font and its background. Research has shown that a yellow-black combination is preferable over a standard black-white contrast.

3. Fully-customisable initial walkthrough: When the user opens the app for the first time, he is taken through an interactive onboarding experience where they are able to select their preferences along with a live preview. This helps them in selecting their preferred fonts, colors, line height, and letter spacing.

4. Reader mode: To facilitate the reading of paragraphs, Augmenta11y features a separate reader screen which contains additional customisations such as a ‘Read Aloud’ mode, which uses text-to-speech (TTS).

5. History mode: All pieces of the text read by the user are saved for future reference locally on their device.

The Big Launch

Augmenta11y was launched recently in March on iOS and Android smartphones and tablets and it is available for free for both platforms. Prior to the final launch, the app was in a closed alpha during Dutch Design Week in October 2018 exhibition where the team got preliminary feedback about the basic working of the app. Then in November 2018, they had launched the public beta which had over 150 testers across iOS and Android. After fixing all the beta bugs and polishing the app, the team finally launched it in March this year.
"The app uses Augmented Reality (AR) to superimpose a dyslexia-friendly version of the text on top of the text the user is pointing towards using their smartphone camera. Using AR helps in maintaining context since the coordinates/positions of the various text blocks is maintained, thus making the user feel familiar to their original environment," adds Tushar.

Making things accessible for the differently-abled

Augmenta11y is a part of Oswald Labs’s Research Division and a component of a set of smartphone apps marketed by the lab, for people with disabilities. "We approached Oswald Labs because we found out about them while researching for other products which focused on accessibility. Agastya, their web plugin helps in making the web more accessible. When we got the idea about using AR and smartphones to solve for dyslexia, I emailed them pitching our idea and they responded positively. Thus, Augmenta11y is a part of Oswald Labs’s Research Division, wherein we focus on designing and developing the app and they help us in marketing it," explains Tushar.
The app is funded by the Oswald Labs’s Research Fund, a not-for-profit fund built to support efforts in R&D and invention. Tushar adds, "As for marketing, our primary source of growth has been media coverage. Fortunately, being featured in some mainstream news publications and online podcasts has helped the app reach its target audience."

The differentiation factor

Dyslexia is not very well diagnosed, especially in India, says Tushar, further explaining that dyslexia is not a 'one size fits all' problem because the needs of every child are different. 
He continues, "This is why there are not a large number of apps specifically for children with dyslexia. Some apps have specific features like text-to-speech and font changing (for example, Google Play Books, Mozilla Pocket, and Amazon WhisperSync) but they do not have the customisation abilities of Augmenta11y. It does not impose upon one reading pattern but allows students to customise the contrast, typeface, space between letters and lines, etc., to fit their reading needs. This exclusive 'dyslexia-friendly mode' was invented by Oswald Labs two years ago and introduced in their web accessibility product, Agastya. This feature is exclusive to Augmenta11y and is not available in any other app for smartphones. Additionally, the introduction of real-time and offline text detection is something that we believe is unique to our app."

Future plans

The team aims to add more features to the app to improve accessibility for dyslexic individuals. Through user feedback and multiple rounds of testing, they have identified areas where the app's functionality can be improved further. For instance, adding support for real-time translations in an augmented reality environment will allow users to better understand their surroundings. And this feature could be particularly useful when the user is presented with unfamiliar information, such as a textbook or signage in a new language.
"Another point of improvement is the ability to customise the text-to-speech playback mode. We're looking to add a number of new voices (both male and female), and a way to control the tempo to aid children in understanding the pronunciation of certain words," concludes Tushar.

Related Stories

No stories found.