eduzhai > Applied Sciences > Engineering >

Unsupervised Generative Adversarial Alignment Representation for Sheet music Audio and Lyrics

  • king
  • (0) Download
  • 20210505
  • Save

... pages left unread,continue reading

Document pages: 5 pages

Abstract: Sheet music, audio, and lyrics are three main modalities during writing asong. In this paper, we propose an unsupervised generative adversarialalignment representation (UGAAR) model to learn deep discriminativerepresentations shared across three major musical modalities: sheet music,lyrics, and audio, where a deep neural network based architecture on threebranches is jointly trained. In particular, the proposed model can transfer thestrong relationship between audio and sheet music to audio-lyrics andsheet-lyrics pairs by learning the correlation in the latent shared subspace.We apply CCA components of audio and sheet music to establish new ground truth.The generative (G) model learns the correlation of two couples of transferredpairs to generate new audio-sheet pair for a fixed lyrics to challenge thediscriminative (D) model. The discriminative model aims at distinguishing theinput which is from the generative model or the ground truth. The two modelssimultaneously train in an adversarial way to enhance the ability of deepalignment representation learning. Our experimental results demonstrate thefeasibility of our proposed UGAAR for alignment representation learning amongsheet music, audio, and lyrics.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×