Performing Query-by-Melody on Audio Collections

Abstract. Mid-level representations are increasingly used to bridge the gap between high-level (semantic) and low-level audio representations. A mid-level representation that integrates melodic and rhythmic aspects of a music signal is introduced. The representation is formed by first performing multi-pitch detection on consecutive audio frames and then searching for dominant melodic lines within the detected pitches. Beattracking is also performed to yield a beat-synchronous representation, independent of tempo variations. The representation is used within a query-by-melody audio retrieval system. An approximate nearest neighbor search algorithm is employed to compare fragments of the mid-level representation of a query to indexed fragments of mid-level representations of songs in a collection. Symbolic queries may also be used. Results are ranked according to a weighted sum of matched fragments. Locality sensitive hashing is used for fast indexing and retrieval. Retrieval was tested on the cover song identification task, where the goal is to retrieve different interpretations of a song in a collection. Results on a collection of 2424 songs are presented