LEARNING UTTERANCE LEVEL REPRESENTATION FROM SPEECH

Speaker verification (SV) is a task to verify a claimed identity from the voice signal. A well-performing SV system requires a method to transform a variable-length recording into a fixed-length representation (a.k.a. embedding vector), which compactly captures distinctive features over different speakers. There are two popular methods: i-vector and x-vector. Although i-vector is still used, x-vector outperforms i-vector in most SV tasks. However, x-vector has limitations: 1) the embedding still includes information about phones, and 2) it cannot leverage data without speaker labels. Phone information remaining in x-vector can degrade its performance in text-independent SV because utterances from the same speaker may have different embeddings due to different phone coverage, or embeddings from two utterances spoken by two different speakers could be similar because of the same sentence. To solve this issue, we propose to disentangle the phone information and speaker identity into separate latent factors using a text-to-speech (TTS) model. The results show that the TTS model added to x-vector training improves the SV results, confirming that reducing text-dependency leads to improvement in SV. Considering much more unlabeled data exists than labeled data, leveraging the unlabeled data is essential, which is not straightforward with the x-vector training. However, this is possible with the proposed TTS-based method. The experimental results show that it can leverage the unlabeled data, but it still requires some labeled data to post-process the embeddings for the final SV system. To improve the embedding learning further with unlabeled data, we adapt a self-supervised technique in computer vision to speech, distillation with no labels (DINO). DINO outperforms the TTS method in un-/semi-supervised SV while enabling the SV system development without labels. Lastly, x-vector has also been used in low-resourced speech applications, such as speech emotion recognition and Alzheimer's disease detection, by transferring the x-vector model pre-trained on large data to the applications. However, we hypothesize that the DINO embedding learned without labels is more transferable than x-vector trained for speaker classification. We observe that the DINO embedding outperforms x-vector in the transfer learning scenario, implying that DINO embedding is more transferable to speech applications other than SV