论文标题
错误的错误，但刻板印象更多？模型大小对性别偏差的影响
Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias
论文作者
论文摘要
预期模型的大小正在增加，它们在各种NLP任务上的性能也在增加。但是，随着记忆能力的增长，他们可能会增加更多的社会偏见。在这项工作中，我们检查了模型大小及其性别偏见之间的联系（特别是职业性别偏见）。我们在两个设置中测量了三个蒙版语言模型家族（Roberta，Deberta和T5）中的偏见：直接使用基于提示的方法，并使用下游任务（Winogender）。一方面，我们发现较大的模型在以前的任务上获得了更高的偏差分数，但是当对后者进行评估时，它们会造成更少的性别错误。为了检查这些潜在的矛盾结果，我们仔细研究了Winogender不同模型的行为。我们发现，尽管较大的模型的表现要比较小的模型，但其错误是由性别偏见引起的概率。此外，我们发现，与抗疾病的型号相比，刻板印象误差的比例随模型大小而生长。我们的发现突出了增加模型大小可能引起的潜在风险。
The size of pretrained models is increasing, and so is their performance on a variety of NLP tasks. However, as their memorization capacity grows, they might pick up more social biases. In this work, we examine the connection between model size and its gender bias (specifically, occupational gender bias). We measure bias in three masked language model families (RoBERTa, DeBERTa, and T5) in two setups: directly using prompt based method, and using a downstream task (Winogender). We find on the one hand that larger models receive higher bias scores on the former task, but when evaluated on the latter, they make fewer gender errors. To examine these potentially conflicting results, we carefully investigate the behavior of the different models on Winogender. We find that while larger models outperform smaller ones, the probability that their mistakes are caused by gender bias is higher. Moreover, we find that the proportion of stereotypical errors compared to anti-stereotypical ones grows with the model size. Our findings highlight the potential risks that can arise from increasing model size.
