论文标题
针对不确定系统的基于自适应观察的有效增强学习
Adaptive Observation-Based Efficient Reinforcement Learning for Uncertain Systems
论文作者
论文摘要
本文为具有不确定漂移动态的系统开发了一种基于自适应的有效增强学习（RL）方法。首先设计了一种新型的并发学习自适应扩展观察者（CL-EAO），以共同估计系统状态和参数。该观察者具有两次计算结构，不需要任何其他数值技术来计算状态导数信息。并发学习（CL）的想法被利用以使用记录的数据，这导致了可验证的激发条件，以促进参数估计的收敛性。基于CL-EAO提供的估计状态和参数，开发了基于经验的RL方案的模拟，以在线近似最佳控制策略。进行了严格的理论分析，以表明系统状态与原点的实际融合以及开发的政策与理想的最佳政策可以实现，而无需持续激发（PE）条件。最后，通过比较模拟证明了开发方法的有效性和优势。
This paper develops an adaptive observation-based efficient reinforcement learning (RL) approach for systems with uncertain drift dynamics. A novel concurrent learning adaptive extended observer (CL-AEO) is first designed to jointly estimate the system state and parameter. This observer has a two-timescale structure and doesn't require any additional numerical techniques to calculate the state derivative information. The idea of concurrent learning (CL) is leveraged to use the recorded data, which leads to a relaxed verifiable excitation condition for the convergence of parameter estimation. Based on the estimated state and parameter provided by the CL-AEO, a simulation of experience based RL scheme is developed to online approximate the optimal control policy. Rigorous theoretical analysis is given to show that the practical convergence of the system state to the origin and the developed policy to the ideal optimal policy can be achieved without the persistence of excitation (PE) condition. Finally, the effectiveness and superiority of the developed methodology are demonstrated via comparative simulations.
