ESTIMATION OF SPEECH ACOUSTICS FROM VISUAL SPEECH FEATURES: A COMPARISON OF LINEAR AND NON-LINEAR MODELS.

This paper examines the degree of correlation between lip and jaw conguration and speech acoustics. The lip and jaw positions are characterised by a system of measure-ments taken from video images of the speaker's face and pro le, and the acoustics are represented using line spectral pair parameters and a measure of RMS energy. A corre-lation is found between the measured acoustic parameters and a linear estimate of the acoustics recovered from the visual data. This correlation exists despite the simplicity of the mapping and is in rough agreement with correla-tions measured in earlier work by Yehia et al. The linear estimates are also compared to estimates made using non-linear models. In particular it is shown that although per-formance of the two models is remarkably similar for static visual features, non-linear models are better able to handle dynamic features. 1