eduzhai > Applied Sciences > Engineering >

Modular Transfer Learning with Transition Mismatch Compensation for Excessive Disturbance Rejection

  • king
  • (0) Download
  • 20210506
  • Save

... pages left unread,continue reading

Document pages: 16 pages

Abstract: Underwater robots in shallow waters usually suffer from strong wave forces,which may frequently exceed robot s control constraints. Learning-basedcontrollers are suitable for disturbance rejection control, but the excessivedisturbances heavily affect the state transition in Markov Decision Process(MDP) or Partially Observable Markov Decision Process (POMDP). Also, purelearning procedures on targeted system may encounter damaging exploratoryactions or unpredictable system variations, and training exclusively on a priormodel usually cannot address model mismatch from the targeted system. In thispaper, we propose a transfer learning framework that adapts a control policyfor excessive disturbance rejection of an underwater robot under dynamics modelmismatch. A modular network of learning policies is applied, composed of aGeneralized Control Policy (GCP) and an Online Disturbance Identification Model(ODI). GCP is first trained over a wide array of disturbance waveforms. ODIthen learns to use past states and actions of the system to predict thedisturbance waveforms which are provided as input to GCP (along with the systemstate). A transfer reinforcement learning algorithm using Transition MismatchCompensation (TMC) is developed based on the modular architecture, that learnsan additional compensatory policy through minimizing mismatch of transitionspredicted by the two dynamics models of the source and target tasks. Wedemonstrated on a pose regulation task in simulation that TMC is able tosuccessfully reject the disturbances and stabilize the robot under an empiricalmodel of the robot system, meanwhile improve sample efficiency.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...