论文标题
在模型不确定性下的非参数自适应鲁棒控制
Nonparametric Adaptive Robust Control Under Model Uncertainty
论文作者
论文摘要
我们考虑模型不确定性下的离散时间随机马尔可控制问题。这种不确定性不仅源于以下事实：基本随机过程的真实概率定律尚不清楚，而且真实法律所属于的概率分布的参数家族也未知。我们提出了一种非参数自适应鲁棒控制方法来解决此类问题。我们的方法取决于以下建筑概念：首先，使用自适应鲁棒范式将在线学习和不确定性降低到健壮的控制问题中；其次，通过经验分布来学习未知的概率定律，并根据经验分布周围的一系列瓦斯坦（Waserstein）球来表示不确定性的降低；第三，使用拉格朗日二重性将瓦斯坦球对的优化转换为标量优化问题，并采用机器学习技术来实现对最佳控制的有效计算。我们通过考虑实用性最大化问题来说明我们的方法论。数值比较表明，非参数自适应鲁棒控制方法比传统的鲁棒框架更可取。
We consider a discrete time stochastic Markovian control problem under model uncertainty. Such uncertainty not only comes from the fact that the true probability law of the underlying stochastic process is unknown, but the parametric family of probability distributions which the true law belongs to is also unknown. We propose a nonparametric adaptive robust control methodology to deal with such problem. Our approach hinges on the following building concepts: first, using the adaptive robust paradigm to incorporate online learning and uncertainty reduction into the robust control problem; second, learning the unknown probability law through the empirical distribution, and representing uncertainty reduction in terms of a sequence of Wasserstein balls around the empirical distribution; third, using Lagrangian duality to convert the optimization over Wasserstein balls to a scalar optimization problem, and adopting a machine learning technique to achieve efficient computation of the optimal control. We illustrate our methodology by considering a utility maximization problem. Numerical comparisons show that the nonparametric adaptive robust control approach is preferable to the traditional robust frameworks.
