论文标题
通过贝叶斯推断，更好的同伴分级
Better Peer Grading through Bayesian Inference
论文作者
论文摘要
同行评分系统汇总了来自多个学生的嘈杂报告，以尽可能接近真实的成绩。大多数当前系统要么以报告等级的平均值或中位数为中位数；其他人则旨在估计学生在概率模型下的评分准确性。本文以三种关键方式以后一种方法扩展了最新技术：（1）认识到学生可以策略性地行事（例如，在不做工作的情况下，报告成绩接近班级平均水平）； （2）适当处理由离散值分级标题产生的审查数据； （3）使用混合整数编程来提高分配给学生的成绩的解释性。我们在此模型中展示了如何使贝叶斯推论实用，并通过在四个大型类中使用我们实现的系统来评估我们对合成和现实数据的方法。这些广泛的实验表明，使用我们的模型的等级聚合准确地估计了真正的成绩，学生提交非信息成绩的可能性以及其固有的分级错误的变化；我们还表征了模型的鲁棒性。
Peer grading systems aggregate noisy reports from multiple students to approximate a true grade as closely as possible. Most current systems either take the mean or median of reported grades; others aim to estimate students' grading accuracy under a probabilistic model. This paper extends the state of the art in the latter approach in three key ways: (1) recognizing that students can behave strategically (e.g., reporting grades close to the class average without doing the work); (2) appropriately handling censored data that arises from discrete-valued grading rubrics; and (3) using mixed integer programming to improve the interpretability of the grades assigned to students. We show how to make Bayesian inference practical in this model and evaluate our approach on both synthetic and real-world data obtained by using our implemented system in four large classes. These extensive experiments show that grade aggregation using our model accurately estimates true grades, students' likelihood of submitting uninformative grades, and the variation in their inherent grading error; we also characterize our models' robustness.
