eduzhai > Applied Sciences > Engineering >

Improved Handling of Repeats and Jumps in Audio-Sheet Image Synchronization

  • king
  • (0) Download
  • 20210506
  • Save

... pages left unread,continue reading

Document pages: 8 pages

Abstract: This paper studies the problem of automatically generating piano scorefollowing videos given an audio recording and raw sheet music images. Whereasprevious works focus on synthetic sheet music where the data has been cleanedand preprocessed, we instead focus on developing a system that can cope withthe messiness of raw, unprocessed sheet music PDFs from IMSLP. We investigatehow well existing systems cope with real scanned sheet music, filler pages andunrelated pieces or movements, and discontinuities due to jumps and repeats. Wefind that a significant bottleneck in system performance is handling jumps andrepeats correctly. In particular, we find that a previously proposed Jump DTWalgorithm does not perform robustly when jump locations are unknown a priori.We propose a novel alignment algorithm called Hierarchical DTW that can handlejumps and repeats even when jump locations are not known. It first performsalignment at the feature level on each sheet music line, and then performs asecond alignment at the segment level. By operating at the segment level, it isable to encode domain knowledge about how likely a particular jump is. Throughcarefully controlled experiments on unprocessed sheet music PDFs from IMSLP, weshow that Hierarachical DTW significantly outperforms Jump DTW in handlingvarious types of jumps.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×