eduzhai > Applied Sciences > Engineering >

Ultra-light deep MIR by trimming lottery tickets

  • king
  • (0) Download
  • 20210506
  • Save

... pages left unread,continue reading

Document pages: 8 pages

Abstract: Current state-of-the-art results in Music Information Retrieval are largelydominated by deep learning approaches. These provide unprecedented accuracyacross all tasks. However, the consistently overlooked downside of these modelsis their stunningly massive complexity, which seems concomitantly crucial totheir success. In this paper, we address this issue by proposing a modelpruning method based on the lottery ticket hypothesis. We modify the originalapproach to allow for explicitly removing parameters, through structuredtrimming of entire units, instead of simply masking individual weights. Thisleads to models which are effectively lighter in terms of size, memory andnumber of operations. We show that our proposal can remove up to 90 of themodel parameters without loss of accuracy, leading to ultra-light deep MIRmodels. We confirm the surprising result that, at smaller compression ratios(removing up to 85 of a network), lighter models consistently outperform theirheavier counterparts. We exhibit these results on a large array of MIR tasksincluding audio classification, pitch recognition, chord extraction, drumtranscription and onset estimation. The resulting ultra-light deep learningmodels for MIR can run on CPU, and can even fit on embedded devices withminimal degradation of accuracy.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×