Hierarchical Music Source Separation Using Mixing Secret Multi-track Dataset

Music Source Separation (MSS) aims to separate the instrument sources from the mixture. Since MUSDB18 is the standard dataset for MSS, most MSS systems are developed on MUSDB18 and they can only focused on four stems, namely vocals, drums, bass, other. It is still impossible to separate more fine-grained level of sources, e.g., lead vocal, backing vocal and guitar. The limitation of MUSDB18 dataset and the lack of large multi-track datasets are the possible obstacles. In this thesis, we first focus on building a new multi-track dataset called MS21, which can be viewed as an expanded and MSS oriented version of the Mixing Secrets dataset. With a total mixture duration of 34.2 hours, MS21 features multi-track content in an unprecedented size and industrial standard. Statistics analysis shows that MS21 contains more averaged number of multi-track files (20±13), a higher percentage of vocal stem (84%) among multi-track dataset and a more diverse genre distribution among edleyDB, MUSDB18 and Slakh2100. MS21 provides MedleyDB like metadata files which capture the specially designed three-level hierarchy annotation for all 500 songs, in order to bridge between different source separation research scenarios. Hierarchical Music Source Separation (HMSS) is a new research scenario, which aims to separate the sources that share hierarchical relation, for example accompaniment - guitar - electric guitar. Multi-level HMSS system showed better performance when separating sources at more fine-grained level, compared to the singlelevel MSS system. In this thesis, to further explore the research possibility in HMSS, we proposed a theory called Mask Relation and a new mask-based method called Sequential Mask. Our HMSS experiments showed that the sequential mask-based approach provides worse results for higher-level sources, but better results for lowerlevel sources (+2dB SI-SDR) compared to a single-level MSS system, which we explain as the leaf-oriented feature of the sequential mask