eduzhai > Applied Sciences > Engineering >

Exit Time Analysis for Approximations of Gradient Descent Trajectories Around Saddle Points

  • Save

... pages left unread,continue reading

Document pages: 35 pages

Abstract: This paper considers the problem of understanding the exit time fortrajectories of gradient-related first-order methods from saddle neighborhoodsunder some initial boundary conditions. Given the `flat geometry around saddlepoints, first-order methods can struggle in escaping these regions in a fastmanner due to the small magnitudes of gradients encountered. In particular,while it is known that gradient-related first-order methods escapestrict-saddle neighborhoods, existing literature does not explicitly leveragethe local geometry around saddle points in order to control behavior ofgradient trajectories. It is in this context that this paper puts forth arigorous geometric analysis of the gradient-descent method around strict-saddleneighborhoods using matrix perturbation theory. In doing so, it provides a keyresult that can be used to generate an approximate gradient trajectory for anygiven initial conditions. In addition, the analysis leads to a linear exit-timesolution for gradient-descent method under certain necessary initial conditionsfor a class of strict-saddle functions.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×