eduzhai > Applied Sciences > Engineering >

Subword Regularization An Analysis of Scalability and Generalization for End-to-End Automatic Speech Recognition

  • king
  • (0) Download
  • 20210506
  • Save

... pages left unread,continue reading

Document pages: 5 pages

Abstract: Subwords are the most widely used output units in end-to-end speechrecognition. They combine the best of two worlds by modeling the majority offrequent words directly and at the same time allow open vocabulary speechrecognition by backing off to shorter units or characters to construct wordsunseen during training. However, mapping text to subwords is ambiguous andoften multiple segmentation variants are possible. Yet, many systems aretrained using only the most likely segmentation. Recent research suggests thatsampling subword segmentations during training acts as a regularizer for neuralmachine translation and speech recognition models, leading to performanceimprovements. In this work, we conduct a principled investigation on theregularizing effect of the subword segmentation sampling method for a streamingend-to-end speech recognition task. In particular, we evaluate the subwordregularization contribution depending on the size of the training dataset. Ourresults suggest that subword regularization provides a consistent improvementof (2-8 ) relative word-error-rate reduction, even in a large-scale settingwith datasets up to a size of 20k hours. Further, we analyze the effect ofsubword regularization on recognition of unseen words and its implications onbeam diversity.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...