论文标题
在费米·费尔（Fermi Fel）上展示的粒子加速器控制的基于模型和贝叶斯结合的基于模型的深钢筋学习
Model-free and Bayesian Ensembling Model-based Deep Reinforcement Learning for Particle Accelerator Control Demonstrated on the FERMI FEL
论文作者
论文摘要
强化学习在加速器控制中具有巨大的希望。本文的主要目的是展示如何在加速器物理问题的操作水平上使用这种方法。尽管在多个领域中没有模型的增强学习成功，但样本效率仍然是一种瓶颈，可能由基于模型的方法包含。我们将纯粹基于模型的纯粹基于模型的增强钢筋学习应用于Fermi Fel系统的强度优化。我们发现，基于模型的方法表现出更高的表示能力和样本效率，而无模型方法的渐近性能略高。基于模型的算法是使用不确定性了解模型在DYNA风格中实现的，而无模型算法基于量身定制的深Q学习。在这两种情况下，算法都是以某种方式实施的，它在加速器控制问题中无处不在，却提高了噪声稳健性。代码在https://github.com/mathphyssim/fermi_rl_paper中发布。
Reinforcement learning holds tremendous promise in accelerator controls. The primary goal of this paper is to show how this approach can be utilised on an operational level on accelerator physics problems. Despite the success of model-free reinforcement learning in several domains, sample-efficiency still is a bottle-neck, which might be encompassed by model-based methods. We compare well-suited purely model-based to model-free reinforcement learning applied to the intensity optimisation on the FERMI FEL system. We find that the model-based approach demonstrates higher representational power and sample-efficiency, while the asymptotic performance of the model-free method is slightly superior. The model-based algorithm is implemented in a DYNA-style using an uncertainty aware model, and the model-free algorithm is based on tailored deep Q-learning. In both cases, the algorithms were implemented in a way, which presents increased noise robustness as omnipresent in accelerator control problems. Code is released in https://github.com/MathPhysSim/FERMI_RL_Paper.
