论文标题
开放域多文档摘要：检索下模型的综合研究
Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval
论文作者
论文摘要
多文件摘要（MDS）假设提供了一组与主题相关的文档作为输入。实际上，此文档集并非总是可用的；考虑到信息需要，即问题或主题语句，我们将其配音为“开放域” MDS，需要检索它。我们通过使用现有数据集，猎犬和摘要器将任务进行正式化并引导它来研究这种更具挑战性的设置。 Via extensive automatic and human evaluation, we determine: (1) state-of-the-art summarizers suffer large reductions in performance when applied to open-domain MDS, (2) additional training in the open-domain setting can reduce this sensitivity to imperfect retrieval, and (3) summarizers are insensitive to the retrieval of duplicate documents and the order of retrieved documents, but highly sensitive to other errors, like the retrieval of无关的文件。根据我们的结果，我们提供了实用的指南，以实现未来的开放域MDS工作，例如如何选择要汇总的文档的数量。我们的结果表明，对于在开放域环境中的进一步进步，必须采用新的检索和摘要方法以及用于培训和评估的注释资源。
Multi-document summarization (MDS) assumes a set of topic-related documents are provided as input. In practice, this document set is not always available; it would need to be retrieved given an information need, i.e. a question or topic statement, a setting we dub "open-domain" MDS. We study this more challenging setting by formalizing the task and bootstrapping it using existing datasets, retrievers and summarizers. Via extensive automatic and human evaluation, we determine: (1) state-of-the-art summarizers suffer large reductions in performance when applied to open-domain MDS, (2) additional training in the open-domain setting can reduce this sensitivity to imperfect retrieval, and (3) summarizers are insensitive to the retrieval of duplicate documents and the order of retrieved documents, but highly sensitive to other errors, like the retrieval of irrelevant documents. Based on our results, we provide practical guidelines to enable future work on open-domain MDS, e.g. how to choose the number of retrieved documents to summarize. Our results suggest that new retrieval and summarization methods and annotated resources for training and evaluation are necessary for further progress in the open-domain setting.
