eduzhai > Applied Sciences > Engineering >

Feature-Dependent Cross-Connections in Multi-Path Neural Networks

  • Save

... pages left unread,continue reading

Document pages: 8 pages

Abstract: Learning a particular task from a dataset, samples in which originate fromdiverse contexts, is challenging, and usually addressed by deepening orwidening standard neural networks. As opposed to conventional network widening,multi-path architectures restrict the quadratic increment of complexity to alinear scale. However, existing multi-column path networks or model ensemblingmethods do not consider any feature-dependent allocation of parallel resources,and therefore, tend to learn redundant features. Given a layer in a multi-pathnetwork, if we restrict each path to learn a context-specific set of featuresand introduce a mechanism to intelligently allocate incoming feature maps tosuch paths, each path can specialize in a certain context, reducing theredundancy and improving the quality of extracted features. This eventuallyleads to better-optimized usage of parallel resources. To do this, we proposeinserting feature-dependent cross-connections between parallel sets of featuremaps in successive layers. The weighting coefficients of thesecross-connections are computed from the input features of the particular layer.Our multi-path networks show improved image recognition accuracy at a similarcomplexity compared to conventional and state-of-the-art methods for deepening,widening and adaptive feature extracting, in both small and large scaledatasets.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×