Explorations of Singing Voice Synthesis using DDSP

Machine learning based singing voice models require large datasets and lengthy training times. In this work we present a lightweight architecture, based on the Differentiable Digital Signal Processing (DDSP) library, that is able to output song-like utterances conditioned only on pitch and amplitude, after twelve hours of training using small datasets of unprocessed audio. The results are promising, as both the melody and the singer’s voice are recognizable. In addition, we explore the unused latent- vector in DDSP to improve the lyrics. Furthermore, we present two zeroconfiguration tools to train new models, including our experimental models. Our results indicate that the latent- improves both the identification of the singer as well as the comprehension of the lyrics