论文标题
用于无模型风险约束的线性二次调节器的原始二元学习
Primal-dual Learning for the Model-free Risk-constrained Linear Quadratic Regulator
论文作者
论文摘要
风险感知控制虽然有望解决意外事件，但仍需要一个已知的确切动力模型。在这项工作中，我们提出了一个无模型的框架，以学习一个关注线性系统的风险感知控制器。我们将其作为一个离散时间的无限 - 摩恩LQR问题，具有状态预测差异约束。为了解决它，我们使用反馈增益对参数化策略，并利用原始偶对偶的方法来通过仅使用数据来优化它。我们首先研究拉格朗日功能的优化格局，并确定了强大的双重性，尽管其非凸性性质。除此之外，我们发现拉格朗日功能具有重要的局部梯度优势属性，然后将其利用以开发一种收敛的随机搜索算法来学习双重函数。此外，我们提出了一种具有全球融合的原始二重式算法，以学习最佳的政策 - 培养基对。最后，我们通过模拟验证结果。
Risk-aware control, though with promise to tackle unexpected events, requires a known exact dynamical model. In this work, we propose a model-free framework to learn a risk-aware controller with a focus on the linear system. We formulate it as a discrete-time infinite-horizon LQR problem with a state predictive variance constraint. To solve it, we parameterize the policy with a feedback gain pair and leverage primal-dual methods to optimize it by solely using data. We first study the optimization landscape of the Lagrangian function and establish the strong duality in spite of its non-convex nature. Alongside, we find that the Lagrangian function enjoys an important local gradient dominance property, which is then exploited to develop a convergent random search algorithm to learn the dual function. Furthermore, we propose a primal-dual algorithm with global convergence to learn the optimal policy-multiplier pair. Finally, we validate our results via simulations.
