论文标题
自然语言推断具有混合效果
Natural Language Inference with Mixed Effects
论文作者
论文摘要
越来越多的证据表明，用于构建自然语言推理数据集的原始注释中分歧的普遍性使得将这些注释汇总到单个标签有问题的常见实践。我们提出了一种通用方法，该方法允许人们直接跳过聚合步骤并在原始注释上训练，而不会使模型受到注释响应偏见可能引起的不良噪声。我们证明了这种方法通过将\ textit {注释器随机效应}纳入任何现有的神经模型来概括\ textIt {混合效应模型}的概念，从而改善了不包含此类效果的模型的性能。
There is growing evidence that the prevalence of disagreement in the raw annotations used to construct natural language inference datasets makes the common practice of aggregating those annotations to a single label problematic. We propose a generic method that allows one to skip the aggregation step and train on the raw annotations directly without subjecting the model to unwanted noise that can arise from annotator response biases. We demonstrate that this method, which generalizes the notion of a \textit{mixed effects model} by incorporating \textit{annotator random effects} into any existing neural model, improves performance over models that do not incorporate such effects.
