Features for instrument recognition in polyphonic mixes

Automated instrument recognition is necessary to efficiently obtain instrumentation information for the existing large collections of digital music. While automated instrument recognition is possible with very high accuracy for monophonic fragments, the problem has not yet been solved for polyphonic mixes.In this work a system is designed for instrument recognition. This system is based on the popular mel frequency cepstral coefficients(MFCC) features and a Gaussian mixture model(GMM) classifier. The primary contribution of this thesis is an extension to the standard MFCC based features. While it’s known that system performance is strongly dependent on the window size used in the calculation of the MFCCs, we pose that limiting the feature extraction to a single window size limits performance. In this work we examine simultaneously using MFCC features obtained from different window sizes. Using this method we show a significant increase in performance over a baseline system using the standard MFCC features.Besides this we introduce features based on the self-similarity matrix. While these measures are normally primarily used for visualization and structure detection we show significant differences in the distributions of obtained similarity values and attempt to use this for classification. While the obtained performance doesn’t show a significant increase compared to the baseline system the class based feature distribution differences suggest the method warrants further study.Finally we examine methods of adapting GMM classifiers for the instrument recognition task along the lines of different learning strategies. Here we examine the problem as a case of multiple instance learning, semi-supervised learning and learning in the presence of label noise.Finally we give suggestions for further examination of the MFCC extension, improvements of the self-similarity features and instrument recognition dataset creation