Real-Time Score Following System and Its Applications

[[abstract]]由於電腦技術快速的發展，所以在這個時代，生活周遭的事物幾乎都可以被數位化，我們甚至也將音樂數位化和電腦結合，將音樂做更有效的處理、分析及應用。 此論文主要探討如何有效的追蹤演奏者演奏對應在樂譜上的位置，這種技術稱為樂譜對位(Score Following)，次技術可應用在自動翻譜、自動伴奏、幫助親子互動和輔助樂器初學者學習。 我們的演算法是用動態最佳化(Dynamic Programming)技術來實作，並考慮單音和多音的處理，以及容錯的機制。尤其在處理多音時，我們考慮到演奏者在彈奏多音時，理論上是要同時彈下的，但實際上這些音不一定會被同時按下，中間可能會有極微小的時間差，這會讓系統誤認為是數個單音。所以我們要將演奏的音做分群，增加系統處理多音的準確性。我們也將演奏者的一些彈奏習慣考慮到我們的演算法，像是演奏者在練習的時候常會重彈不熟的部份，或是因為不熟悉樂譜而發生漏彈、多彈或彈錯音符的現象，把這些現象考慮進我們的演算法，使系統更能正確的追蹤演奏者的演奏。 最後，我們的跟譜系統和即時音高擷取(pitch detection)模組做結合，可直接用麥克風收錄樂器的演奏，再經由音訊處理擷取出音高，就可將演奏者所彈奏的內容和樂譜做比對，並即時指出演奏者彈到樂譜的位置。[[abstract]]Our research focuses on how to make an efficient score tracking system that can track one’s musical performance on the score. This kind of technology is called score following. This technology can be used as an automatic page-turner, automatic accompaniment, helps the interaction between parents and children, and helps increase the efficiency of music instrument learners when practicing. Our algorithm applies the Dynamic Programming method, processes monophony and polyphony respectively and has a fault-tolerant function. We focus on dealing with polyphonic notes. In theory, the notes in the polyphony have to be played at the same time. But in fact, the notes in the polyphony have a tiny time difference. The system will consider the polyphonic note as several monophonic notes. So we have to group performed notes into polyphony or monophony. We also consider some performer’s habits and circumstances in our algorithm, such as repeating unfamiliar parts or playing the wrong note. Considering these cases in our algorithm, the system can more accurately track the performance. Finally, our score following system is combined with the pitch detection module. It can directly record a performance through a microphone and acquire the pitch. Then the system will align the performed notes with the score, and point to where the performer plays on the score.[[note]]碩