论文标题
KCRL：Krasovskii约束的增强学习，并在非线性动力学系统中保证稳定
KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems
论文作者
论文摘要
学习动态系统需要稳定未知的动态，以避免状态爆炸。但是，当前的加强学习（RL）方法缺乏稳定保证，这限制了其对控制安全系统的控制。我们提出了一个具有正式稳定性的基于模型的RL框架，即Krasovskii约束RL（KCRL），该框架采用Krasovskii的Lyapunov家族作为稳定性约束。所提出的方法将系统动力学学习到使用特征表示，例如随机傅立叶功能。然后，它使用基于Krasovskii的方法来解决稳定的策略优化问题，并使用原始的偶对方法来恢复稳定策略。我们表明，KCRL可以保证在与基本未知系统的有限互动中学习稳定政策。我们还通过KCRL框架得出了用于稳定未知非线性动力学系统的样品复杂性上限。
Learning a dynamical system requires stabilizing the unknown dynamics to avoid state blow-ups. However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems. We propose a model-based RL framework with formal stability guarantees, Krasovskii Constrained RL (KCRL), that adopts Krasovskii's family of Lyapunov functions as a stability constraint. The proposed method learns the system dynamics up to a confidence interval using feature representation, e.g. Random Fourier Features. It then solves a constrained policy optimization problem with a stability constraint based on Krasovskii's method using a primal-dual approach to recover a stabilizing policy. We show that KCRL is guaranteed to learn a stabilizing policy in a finite number of interactions with the underlying unknown system. We also derive the sample complexity upper bound for stabilization of unknown nonlinear dynamical systems via the KCRL framework.
