论文标题
对比性新颖性学习：用大语言模型期待异常值
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models
论文作者
论文摘要
在许多任务设置中，文本分类模型可能会遇到无法正确预测的新颖类中的示例。选择性预测（在低信心示例上放弃模型）提供了一个可能的解决方案，但是现有模型通常对看不见的类过于自信。为了纠正这种过度自信，我们引入了对比性的新颖性学习（CONAL），这是一种两步方法，生成了代表新阶级的OOD示例，然后训练以降低对它们的信心。首先，我们通过两次提示大型语言模型来生成OOD示例：我们提示它列举相关的新颖类，然后从每个小说类中生成匹配任务格式的示例。其次，我们培训一个具有新颖的对比目标的分类器，该分类器比训练示例更能鼓励对产生的OOD示例的信心。当接受圆锥体培训时，分类器提高了他们在精确覆盖曲线（AUAC）（AUAC）的准确性方面的检测和弃权的能力，在4个NLP数据集中的准确性平均为2.3％，没有4个NLP数据集中的AUROC，无需用于分配准确的成本。
In many task settings, text classification models are likely to encounter examples from novel classes on which they cannot predict correctly. Selective prediction, in which models abstain on low-confidence examples, provides a possible solution, but existing models are often overly confident on unseen classes. To remedy this overconfidence, we introduce Contrastive Novelty-Augmented Learning (CoNAL), a two-step method that generates OOD examples representative of novel classes, then trains to decrease confidence on them. First, we generate OOD examples by prompting a large language model twice: we prompt it to enumerate relevant novel classes, then generate examples from each novel class matching the task format. Second, we train a classifier with a novel contrastive objective that encourages lower confidence on generated OOD examples than training examples. When trained with CoNAL, classifiers improve in their ability to detect and abstain on novel class examples over prior methods by an average of 2.3% in terms of accuracy under the accuracy-coverage curve (AUAC) and 5.5% AUROC across 4 NLP datasets, with no cost to in-distribution accuracy.
