IISc researchers build atypical speech dataset for Indic languages

Researchers at the Indian Institute of Science (IISc) in Bengaluru have built a pilot dataset of atypical speech in three Indic languages to help make voice recognition technology more inclusive. Known as the Vaani Atypical Speech Corpus, the project is a collaborative initiative by IISc and ARTPARK designed to assist people with neurological, cognitive, or motor conditions that affect their speech patterns.

The project addresses a critical gap in speech technology. While Automatic Speech Recognition (ASR) systems are typically designed for standard speech, there has been a complete lack of atypical speech data for Indian languages. This makes it difficult for individuals with speech-impacting conditions to use voice-activated technologies.

Led by Prasanta Kumar Ghosh from the Department of Electrical Engineering at IISc, the pilot initiative has so far collected approximately 10 hours of atypical speech. This initial dataset includes recordings from around 40 speakers across three languages, with plans to expand the corpus in the coming days.

The pilot project was conducted in collaboration with Google’s Project Euphonia. Mr. Ghosh emphasized that this is a long-term initiative, and the team is currently open to collaborations and funding from more partners to expand the dataset to enable further research.

The atypical speech study began as an offshoot of the larger Project Vaani, which aims to build a corpus of 150,000 hours of natural speech data from about 1 million people across nearly 800 Indian districts. Under the broader project, the team has already recorded over 31,000 hours of speech data across 165 districts and 105 languages. This includes 10 to 15 districts in Karnataka, capturing languages such as Kannada, Tulu, Beary Bashe, and Marathi.

For the larger project, data is collected through image-prompted recordings. Speakers are shown generic and district-specific images and asked to describe them, capturing both audio and image data in local dialects. Each speaker describes up to 60 images, spending about 20 seconds on each. The team also works to ensure demographic balance across gender, age, socio-economic status, and education.