INTEGRATION OF SPEECH & VIDEO: APPLICATIONS FOR LIP SYNCH: LIP MOVEMENT SYNTHESIS & TIME WARPING

Throughout the past several decades, much research has been done in the area of signal processing. Two of the most popular areas within this field have been applications for speech recognition and image processing. Due to these extended efforts, today there are systems that can accurately recognize and transcribe the daily television news programs that are broadcast to our homes. There are also systems that can accurately locate and track the faces within those same news programs. Recently, a new field has emerged which focuses on combining the disciplines of speech recognition and image processing in a practical way. This interest has sparked research in a broad range of new application areas, including: • Enhancement of speech recognition via visual cues • Animation of interactive talking agents to facilitate human-computer interaction • Synchronization of audio and video tracks in the film industry • Correction of lip movement for films dubbed into a foreign language and in video conferencing This paper will discuss some of the current efforts in integrating speech and video in a practical way. It will briefly discuss some image processing methods for extraction of lip coordinates and li