VTLN-based rapid cross-lingual adaptation for statistical parametric speech synthesis. Idiap Research Report

Abstract—Cross-lingual speaker adaptation (CLSA) has emerged as a new challenge in statistical parametric speech syn-thesis, with specific application to speech-to-speech translation. Recent research has shown that reasonable speaker similarity can be achieved in CLSA using maximum likelihood linear transformation of model parameters, but this method also has weaknesses due to the inherent mismatch caused by differing phonetic inventories of languages. In this paper, we propose that fast and effective CLSA can be made using vocal tract length normalization (VTLN), where strong constraints of the vocal tract warping function may actually help to avoid the most severe effects of the aforementioned mismatch. VTLN has a single parameter that warps spectrum. Using shifted or adapted pitch, VTLN can still achieve reasonable speaker similarity. We present our approach, VTLN-based CLSA, and evaluation results that support our proposal under the limitation that the voice identity and speaking style of a target speaker don’t diverge too far from that of the average voice model. Index Terms—vocal tract length normalization, cross-lingual speaker adaptation, rapid speaker adaptation, HMM-based speech synthesis I