eduzhai > Applied Sciences > Engineering >

Go with the Flow Adaptive Control for Neural ODEs

  • Save

... pages left unread,continue reading

Document pages: 22 pages

Abstract: Despite their elegant formulation and lightweight memory cost, neuralordinary differential equations (NODEs) suffer from known representationallimitations. In particular, the single flow learned by NODEs cannot express allhomeomorphisms from a given data space to itself, and their static weightparameterization restricts the type of functions they can learn compared todiscrete architectures with layer-dependent weights. Here, we describe a newmodule called neurally controlled ODE (N-CODE) designed to improve theexpressivity of NODEs. The parameters of N-CODE modules are dynamic variablesgoverned by a trainable map from initial or current activation state, resultingin forms of open-loop and closed-loop control, respectively. A single module issufficient for learning a distribution on non-autonomous flows that adaptivelydrive neural representations. We provide theoretical and empirical evidencethat N-CODE circumvents limitations of previous NODEs models and show howincreased model expressivity manifests in several supervised and unsupervisedlearning problems. These favorable empirical results indicate the potential ofusing data- and activity-dependent plasticity in neural networks acrossnumerous domains.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...