eduzhai > Applied Sciences > Engineering >

RT3D Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

  • king
  • (0) Download
  • 20210506
  • Save

... pages left unread,continue reading

Document pages: 9 pages

Abstract: Mobile devices are becoming an important carrier for deep learning tasks, asthey are being equipped with powerful, high-end mobile CPUs and GPUs. However,it is still a challenging task to execute 3D Convolutional Neural Networks(CNNs) targeting for real-time performance, besides high inference accuracy.The reason is more complex model structure and higher model dimensionalityoverwhelm the available computation storage resources on mobile devices. Anatural way may be turning to deep learning weight pruning techniques. However,the direct generalization of existing 2D CNN weight pruning methods to 3D CNNsis not ideal for fully exploiting mobile parallelism while achieving highinference accuracy.This paper proposes RT3D, a model compression and mobile accelerationframework for 3D CNNs, seamlessly integrating neural network weight pruning andcompiler code generation techniques. We propose and investigate two structuredsparsity schemes i.e., the vanilla structured sparsity and kernel groupstructured (KGS) sparsity that are mobile acceleration friendly. The vanillasparsity removes whole kernel groups, while KGS sparsity is a more fine-grainedstructured sparsity that enjoys higher flexibility while exploiting fullon-device parallelism. We propose a reweighted regularization pruning algorithmto achieve the proposed sparsity schemes. The inference time speedup due tosparsity is approaching the pruning rate of the whole model FLOPs (floatingpoint operations). RT3D demonstrates up to 29.1$ times$ speedup in end-to-endinference time comparing with current mobile frameworks supporting 3D CNNs,with moderate 1 -1.5 accuracy loss. The end-to-end inference time for 16 videoframes could be within 150 ms, when executing representative C3D and R(2+1)Dmodels on a cellphone. For the first time, real-time execution of 3D CNNs isachieved on off-the-shelf mobiles.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×