A Passivity-Based Analysis of First-Order Momentum-Based Methods

Abstract

This paper presents a discrete-time passivity-based analysis of first-order momentum-based methods for a class of functions whose gradient has lower and upper sector bounds of $0$ and $L$, respectively. Through a loop transformation, it is shown that momentum-based methods can be represented as a passive controller in negative feedback with an output strictly passive (OSP) system. The weak passivity theorem is then used to derive explicit hyperparameter conditions under which the shifted gradient asymptotically vanishes. Under an additional assumption that requires the existence of a unique stationary point and excludes arbitrarily small gradients far from that point, convergence of the iterates to the global minimizer is established.