A Semi-automatic Computer Expressive Music Performance System Using Structural Support Vector Machine

電腦合成的音樂一向被認為是僵硬、機械化而且沒有音樂表現能力。因此能夠產生具有表現能力的電腦自動演奏系統將會對音樂產業、個人化娛樂以及表驗藝術領域有重大的影響。在這篇論文中,我們藉由隱藏式馬可夫模型結構的結構性支撐向量機 (SVM-HMM) 來設計一個可以產生具有表現能力音樂的電腦自動演奏系統。我們邀請六位研究生錄製了克萊門蒂(Muzio Clementi)的小奏鳴曲集 Op.36。我們手動將這些錄音分割成樂句,並且利用程式從中抽取出音樂特徵。這些 音樂特徵藉由 SVM-HMM 訓練成數學模型後,可以利用這個數學模型來演奏訓練過程中沒有見過的樂譜(需要手動標注樂句)。此系統目前只能支援單音旋律。問卷調查的結果顯示,本系統產生的音樂尚不能達到真人的演奏水準。但是根據量化的相似度分析,本系統產生的音樂確實比無表現性的 MIDI 音樂更接近真人演奏。Computer generated music is known to be robotic and inexpressive. A computer system that can generate expressive performance potentially has significant impact on music production industry, personalized entertainment or even art. In this paper, we have designed and implemented a system that can generate expressive performance using structural support vector machine with hidden Markov model output (SVM-HMM). We recorded six sets of Muzio Clementi''s Sonatina Op.36 performed by six graduate students. The recordings and scores are manually split into phrases and had their musical features automatically extracted. Using the SVM-HMM algorithm, a mathematical model of expressive performance knowledge is learned from these features. The trained model can generate expressive performances for previously unseen scores (with user-assigned phrasings). The system currently supports monophonic music only. Subjective test shows that the computer generated performances still cannot achieve the same level of expressiveness of human performers, but quantitative similarity measures show that the computer generated performances are much similar to human performances than inexpressive MIDIs.1 Introduction 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Goal and Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Chapter Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 Previous Works 2.1 Various Goals and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Researches Classified by Methods Used . . . . . . . . . . . . . . . . . . 5 2.3 Additional Specialties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3 Proposed Method 3.1Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 3.2 A Brief Introduction to SVM-HMM . . . . . . . . . . . . . . . . . . . . 10 3.3 Learning Performance Knowledge . . . . . . . . . . . . . . . . . . . . . 16 3.3.1 Training Sample Loader . . . . . . . . . . . . . . . . . . . . . . 16 3.3.2 Features Extraction . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.3.3 SVM-HMM Learning . . . . . . . . . . . . . . . . . . . . . . . 17 3.4 Performing Expressively . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.4.1 SVM-HMM Generation . . . . . . . . . . . . . . . . . . . . . . 20 3.4.2 MIDI Generation and Synthesis . . . . . . . . . . . . . . . . . . 20 3.5 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.5.1 Score Features . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.5.2 Performance Features . . . . . . . . . . . . . . . . . . . . . . . . 23 3.5.3 Normalizing Onset Deviation . . . . . . . . . . . . . . . . . . . 24 4 Corpus Preparation 4.1 Existing Corpora . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.2 Corpus Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.3.1 Score Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.3.2 MIDI Recording . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.3.3 MIDI Cleaning and Phrase Splitting . . . . . . . . . . . . . . . . 31 4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5 Experiments 5.1 Onset Deviation Normalization . . . . . . . . . . . . . . . . . . . . . . . 36 5.2 Parameter Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.2.1 SVM-HMM-related Parameters . . . . . . . . . . . . . . . . . . 40 5.2.2 Quantization Parameter . . . . . . . . . . . . . . . . . . . . . . . 42 5.3 Human-like Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 44 6 Conclusions Bibliography 53 A Software Tools Used in This Research 6