eduzhai > Applied Sciences > Engineering >

SpinalNet Deep Neural Network with Gradual Input

  • king
  • (0) Download
  • 20210505
  • Save

... pages left unread,continue reading

Document pages: 10 pages

Abstract: Over the past few years, deep neural networks (DNNs) have garnered remarkablesuccess in a diverse range of real-world applications. However, DNNs consider alarge number of inputs and consist of a large number of parameters, resultingin high computational demand. We study the human somatosensory system andpropose the SpinalNet to achieve higher accuracy with less computationalresources. In a typical neural network (NN) architecture, the hidden layersreceive inputs in the first layer and then transfer the intermediate outcomesto the next layer. In the proposed SpinalNet, the structure of hidden layersallocates to three sectors: 1) Input row, 2) Intermediate row, and 3) outputrow. The intermediate row of the SpinalNet contains a few neurons. The role ofinput segmentation is in enabling each hidden layer to receive a part of theinputs and outputs of the previous layer. Therefore, the number of incomingweights in a hidden layer is significantly lower than traditional DNNs. As alllayers of the SpinalNet directly contributes to the output row, the vanishinggradient problem does not exist. We also investigate the SpinalNetfully-connected layer to several well-known DNN models and perform traditionallearning and transfer learning. We observe significant error reductions withlower computational costs in most of the DNNs. We have also obtained thestate-of-the-art (SOTA) performance for QMNIST, Kuzushiji-MNIST, EMNIST(Letters, Digits, and Balanced), STL-10, Bird225, Fruits 360, and Caltech-101datasets. The scripts of the proposed SpinalNet are available with thefollowing link: this https URL

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×