Sheet Music Transcription from Audio Record of a Song Played on Sopile

Sopila je tradicionalni, ručno izradeni puhački instrument, uobičajeno sviran u paru, karakterističan za Istarski poluotok u zapadnoj Hrvatskoj. Njezin prodoran zvuk, zajedno sa dvoglasnim pjevanjem u Istarskoj ljestvici sačinjenoj od šest tonova, proglašen je UNESCOvom ne materijalnom baštinom čovječanstva. Postupak pretvorbe audio zapisa u ljudski prihvatljivi oblik - note, uključuje prepoznavanje više tonova i njihove visine. Predloženo rješenje rješava spomenute probleme izvlačeći svojstva frekvencijskog spektra, koristeći tehnike strojnog učenja pod nadzorom, te dodatnom obradom nakon predvidanja. Najbolji model za predvidanje visine tona, odabran je koristeći mrežno pretraživanje za dvije ML tehnike, dodajući, po izboru, izvučena svojstva frekvencijskog spektra. Model je ostvario obečavajuću preciznost za obje postavke, monofonu i polifonu, na novostvorenom skupu podataka (namijenjenom ovoj svrsi). Cjelokupno AMT rješenje, koje se sastoji od mobilne aplikacije i poslužiteljskog API-ja, je dodatno razvijeno. Mobilna aplikacija služi snimanju, označavanju, i učitavanju zvučnih datoteka, dok poslužitelj sadrži logiku i postupke obrade datoteka u notni zapis koji se vraća kao rezultat. Na taj način je pokazano kako prikupljanje i očuvanje tradicionalne glazbe sopila, može biti jednostavno primijenjeno u stvarnoj upotrebi.Sopila is a traditional hand-made woodwind instrument, which is usually played in pair. It can be found in western parts of Croatia, along with two-part singing in the hexatonic Istrian scale. Both sopile and singing, were registered in the UNESCO Representative List of the Intangible Cultural Heritage of Humanity. This thesis serves as a insight study of automatic music transcription (AMT) for sopile instrument. In order to do that, pitch detection, along with its onsets and offsets had to be provided. This was achieved using frequency-features extraction, supervised machine learning (ML) algorithms, and postprocessing correction methods. With grid search, the best tone-prediction model was determined, optionally using frequency-feature extraction. Datasets used for training and testing were generated for the purpose of this work and performance of the proposed models, for both monophonic and polyphonic setups, were very satisfying. In order to provide insight into how proposed model could be used, proof-of-concept AMT system was developed. It consisted from mobile application for recording, tagging, and uploading audio sources, as well as back-end server that contained ML, preprocessing and postprocessing logic for creating music sheets. End result was pushed back to the mobile application, and thus was demonstrated how preserving traditional music could be done on-the-go and effortlessly