论文标题
反事实推理：语言模型是否需要世界知识才能因果关系？
Counterfactual reasoning: Do language models need world knowledge for causal understanding?
论文作者
论文摘要
当前的预训练的语言模型已实现了下游任务的显着改进，但是很难将统计相关性的影响与更系统的逻辑推理的效果与对现实世界的理解为基础。在本文中，我们通过利用反事实条件来嘲笑这些因素，这些因素迫使语言模型根据假设命题来预测异常后果。我们介绍了一组来自心理语言实验以及大规模受控数据集的测试，以从各种流行的预训练的语言模型中探究反事实预测。我们发现，模型始终能够在反事实场景中覆盖现实世界的知识，并且在基线世界知识更强的情况下，这种效果更加可靠 - 但是，我们还发现，对于大多数模型，这种效果似乎在很大程度上是由简单的词汇提示驱动的。当我们减轻世界知识和词汇提示的影响以检验反事实的语言细微差别的知识时，我们发现只有GPT-3显示出对这些细微差别的敏感性，尽管这种敏感性也受到词汇联想因素的不利影响。
Current pre-trained language models have enabled remarkable improvements in downstream tasks, but it remains difficult to distinguish effects of statistical correlation from more systematic logical reasoning grounded on understanding of the real world. In this paper we tease these factors apart by leveraging counterfactual conditionals, which force language models to predict unusual consequences based on hypothetical propositions. We introduce a set of tests drawn from psycholinguistic experiments, as well as larger-scale controlled datasets, to probe counterfactual predictions from a variety of popular pre-trained language models. We find that models are consistently able to override real-world knowledge in counterfactual scenarios, and that this effect is more robust in case of stronger baseline world knowledge -- however, we also find that for most models this effect appears largely to be driven by simple lexical cues. When we mitigate effects of both world knowledge and lexical cues to test knowledge of linguistic nuances of counterfactuals, we find that only GPT-3 shows sensitivity to these nuances, though this sensitivity is also non-trivially impacted by lexical associative factors.
