论文标题
动量优化：动力学，控制理论和符号直观的观点
Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives
论文作者
论文摘要
我们从动态系统的角度分析了各种基于动量的优化算法的收敛速率。我们的分析利用了基本拓扑特性，例如迭代对其初始条件的连续依赖性，以简单地表征收敛速率。在许多情况下，获得将算法参数与收敛速率相关联的封闭形式表达式。分析包括离散的时间和连续时间，以及时间不变和时间变化的配方，并且不限于凸或欧几里得设置。此外，本文严格确定了为什么符合性离散方案对于基于动量的优化算法很重要，并提供了表现出加速收敛的算法的表征。
We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their initial conditions, to provide a simple characterization of convergence rates. In many cases, closed-form expressions are obtained that relate algorithm parameters to the convergence rate. The analysis encompasses discrete time and continuous time, as well as time-invariant and time-variant formulations, and is not limited to a convex or Euclidean setting. In addition, the article rigorously establishes why symplectic discretization schemes are important for momentum-based optimization algorithms, and provides a characterization of algorithms that exhibit accelerated convergence.
