Learning to Generate Understandable Animations of American Sign Language

Abstract—Standardized testing has revealed that many deaf adults in the U.S. have lower levels of written English literacy; providing American Sign Language (ASL) on websites can make information and services more accessible. Unfortunately, video recordings of human signers are difficult to update when information changes, and there is no way to support just-in-time generation of web content from a query. Software is needed that can automatically synthesize understandable animations of a virtual human performing ASL, based on an easy-to-update script as input. The challenge is for this software to select the details of such animations so that they are linguistically accurate, understandable, and acceptable to users. Our research seeks models that underlie the accurate and natural movements of virtual human characters performing ASL, using the following methodology: experimental evaluation studies with native ASL signers, motion-capture data collection from signers, linguistic analysis of this data, statistical modeling techniques, and animation synthesis. Keywords—American Sign Language; animation; accessibility; technology for people who are deaf or hard-of-hearing I