论文标题
通过自我归一化的重要性加权自信地评估和选择
Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting
论文作者
论文摘要
我们考虑在上下文匪徒设置中进行违反评估，以获得强大的售出选择策略，在此策略中，根据所选策略的价值在一组提案（目标）策略中根据所选策略的价值进行评估。我们提出了一种新方法，以在上下文匪徒中进行一些记录的数据，以计算任意目标策略值的下限，以获得所需的覆盖范围。下边界围绕所谓的自称重要性加权（SN）估计器构建。它结合了半经验的Efron-Stein尾巴不等式的使用来控制偏见的浓度和新的乘法（而不是加性）控制。在许多合成和真实数据集上评估了新方法，并且在置信区间的紧密度和所选择的策略质量方面都比其主要竞争对手优越。
We consider off-policy evaluation in the contextual bandit setting for the purpose of obtaining a robust off-policy selection strategy, where the selection strategy is evaluated based on the value of the chosen policy in a set of proposal (target) policies. We propose a new method to compute a lower bound on the value of an arbitrary target policy given some logged data in contextual bandits for a desired coverage. The lower bound is built around the so-called Self-normalized Importance Weighting (SN) estimator. It combines the use of a semi-empirical Efron-Stein tail inequality to control the concentration and a new multiplicative (rather than additive) control of the bias. The new approach is evaluated on a number of synthetic and real datasets and is found to be superior to its main competitors, both in terms of tightness of the confidence intervals and the quality of the policies chosen.
