论文标题
大型语言模型知道人类知道什么吗?
Do Large Language Models know what humans know?
论文作者
论文摘要
人类可以将信念归因于他人。但是,这种能力在多大程度上是由先天生物学的捐赠或通过儿童发育所产生的经验而导致的,尤其是对描述他人精神状态的语言的接触。我们通过评估暴露于大量人类语言的模型是否表现出对书面段落中字符的隐含知识状态的敏感性来检验语言暴露假设的生存能力。在预注册的分析中,我们向人类参与者和大型语言模型GPT-3提供了语言版本。两者都对他人的信念敏感,但是尽管语言模型大大超过了机会行为,但它的表现不如人类,也没有解释其行为的全部程度 - 尽管暴露于一生中的语言比人类更多的语言。这表明,尽管从语言暴露中学习的统计学习可能部分解释了人类如何发展推理他人心理状态的能力,但其他机制也是负责的。
Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre-registered analyses, we present a linguistic version of the False Belief Task to both human participants and a Large Language Model, GPT-3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans, nor does it explain the full extent of their behavior -- despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible.