论文标题
主动学习++：使用本地模型说明合并注释者的理由
Active Learning++: Incorporating Annotator's Rationale using Local Model Explanation
论文作者
论文摘要
我们提出了一个新的主动学习（AL）框架，主动学习++，可以利用注释者的标签及其原理。注释者可以通过根据其对给定查询的重要性对输入功能进行排名来选择其选择标签的理由。为了结合此额外的意见，我们修改了委员会（QBC）采样策略的基于装袋的查询的分歧措施。我们没有平等地权衡所有委员会模型以选择下一个实例，而是将更高的权重分配给委员会模型，并与注释者的排名更高一致。具体来说，我们为每个委员会模型生成了一个基于特征的本地解释。注释者提供的功能排名与本地模型说明之间的相似性得分用于为每个相应的委员会模型分配权重。这种方法适用于使用模型 - 不合Snostic技术来生成局部解释（例如石灰）的任何类型的ML模型。通过模拟研究，我们表明我们的框架大大优于基于QBC的香草框架。
We propose a new active learning (AL) framework, Active Learning++, which can utilize an annotator's labels as well as its rationale. Annotators can provide their rationale for choosing a label by ranking input features based on their importance for a given query. To incorporate this additional input, we modified the disagreement measure for a bagging-based Query by Committee (QBC) sampling strategy. Instead of weighing all committee models equally to select the next instance, we assign higher weight to the committee model with higher agreement with the annotator's ranking. Specifically, we generated a feature importance-based local explanation for each committee model. The similarity score between feature rankings provided by the annotator and the local model explanation is used to assign a weight to each corresponding committee model. This approach is applicable to any kind of ML model using model-agnostic techniques to generate local explanation such as LIME. With a simulation study, we show that our framework significantly outperforms a QBC based vanilla AL framework.
