The documentation of endangered languages has become an increasing priority for researchers and those native speaker communities that wish to maintain and revitalize their language and culture. Estimates vary of the number of extant languages, the rate of language disappearance and cultural loss, and even the meaning of cultural loss and language death (versus change and shift, as when an Indigenous language is increasingly impacted by colonial languages and cultures). It appears certain that the majority of approximately 6,000-7,000 languages presently spoken will effectively disappear within this century. With this, the diversity of linguistic expression is diminished and the vast reservoir of human knowledge is impoverished.

Yet our ability to document endangered languages and cultures is inhibited by a "transcription bottleneck": hundreds of hours of high quality recordings can be produced in a short amount of time but take months to process. For example, in one week in the village of Yoloxochitl (Guerrero, Mexico), Jonathan Amith, a project co-PI recorded 30 hours of conversations between a dozen native speakers. An excellent native speaker linguist, Rey Castillo Garcia, would require close to a year to produce an archival quality transcription and translation of this material. To address this issue researchers have looked to automatic speech recognition (ASR), the computer-generated production of a written transcription of an audio recording. This project has explored the possibility of using ASR to achieve a high level of accuracy in the transcription of Yoloxochitl Mixtec (YM), an endangered tonal language from west-central Mexico.

The project's primary goal has been to address the "transcription bottleneck" mentioned above. This was accomplished in part by developing, under the direction of co-PI Andreas Kathol, a YM speech recognizer based on recent advances in ASR. First versions of this recognizer at the beginning of the project resulted in a word error rate (WER) of 30.1% for a held-out testset.  By project end the WER had been reduced to 19%.

A secondary set of goal has been to (1) TRAIN native speakers to write their language through a progression of increasingly challenging transcription tasks; (2) TEST native speakers on their success in learning as they progressed through increasingly difficult lessons; and (3) allow users to ANNOTATE, or correct, a transcription proposed by the ASR algorithm. To accomplish all these tasks a TTA (Training, Testing, Annotation) tool was created at SRI International and installed on a Chromebook laptop. The Training and Testing was evaluated with the participation of a native speaker, Esteban Guadalupe Sierra, who finished all lessons and then began to transcribe directly from audio. He has reached a level that is 97% in accord with the transcription of Rey Castillo Garcia, the expert native speaker linguist.

A final goal was to implement ASR (along with the TTA tool) on Chromebook computers to be used in Yoloxochitl. Amith along with Castillo and Guadalupe would work with native speakers to obtain audio recordings. The speech would then be processed locally through the ASR system on the Chromebook and the transcription would be loaded into the TAA tool to be used in Annotation mode. Then the original speaker would correct the computer-generated transcription as best as possible. Correcting mistakes in tone, nasalization, or glottal stops is still challenging for all but the best transcribers. However, speakers can be expected to reliably eliminate words that ASR has mistakenly inserted or add words that the ASR program has failed to transcribe. The transcription so corrected by the narrator would then be given to Castillo to be finalized.

We fully expect recognition accuracy to improve further with further advances in ASR technology. But even with the ASR accuracy achieved so far this project has shown the utility of an integrated approach to Training and Testing native speakers, recording and processing ASR locally, and then having the original speaker take the first step in reviewing his or her computer-generated transcription.

Last Modified: 02/26/2021
Modified by: Andreas Kathol