Variable Frame Rate for Low Power Mobile Sign Language Communication

The MobileASL project aims to increase accessibility by en-abling Deaf people to communicate over video cell phones in their native language, American Sign Language (ASL). Real-time video over cell phones can be a computationally intensive task that quickly drains the battery, rendering the cell phone useless. Properties of conversational sign lan-guage allow us to save power and bits: namely, lower frame rates are possible when one person is not signing due to turn-taking, and signing can potentially employ a lower frame rate than fingerspelling. We conduct a user study with na-tive signers to examine the intelligibility of varying the frame rate based on activity in the video. We then describe several methods for automatically determining the activity of sign-ing or not signing from the video stream in real-time. Our results show that varying the frame rate during turn-taking is a good way to save power without sacrificing intelligibility, and that automatic activity analysis is feasible