论文标题
预验证的变压器的任务不合时宜的数据增加有效？
How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers?
论文作者
论文摘要
事实证明，即使在验证的模型上，任务不合时宜的数据增强形式在计算机视觉中也广泛有效。在NLP中，相似的结果是针对低数据制度，非预测模型或预审前模型的情况最常见的。在本文中，我们询问将这些技术应用于预告片的变压器时的真正有效性。我们使用两种流行的任务数据增强（不适合任何特定任务），易于数据增强（Wei和Zou，2019年）和反向翻译（Sennrichet al。，2015），我们对5个分类任务的效果进行系统的检查，6个数据集，6个数据集和3个变种的现代化prefrated Pretrained Fresheraverrained prefrained Truncers，包括Bera，xln，xln，x.我们观察到一个负面的结果，发现以前报道的非预告模型的强大改进的技术也无法始终提高预验证的变压器的性能，即使训练数据受到限制。我们希望这种经验分析有助于告知从业人员数据增强技术可以提供改进的地方。
Task-agnostic forms of data augmentation have proven widely effective in computer vision, even on pretrained models. In NLP similar results are reported most commonly for low data regimes, non-pretrained models, or situationally for pretrained models. In this paper we ask how effective these techniques really are when applied to pretrained transformers. Using two popular varieties of task-agnostic data augmentation (not tailored to any particular task), Easy Data Augmentation (Wei and Zou, 2019) and Back-Translation (Sennrichet al., 2015), we conduct a systematic examination of their effects across 5 classification tasks, 6 datasets, and 3 variants of modern pretrained transformers, including BERT, XLNet, and RoBERTa. We observe a negative result, finding that techniques which previously reported strong improvements for non-pretrained models fail to consistently improve performance for pretrained transformers, even when training data is limited. We hope this empirical analysis helps inform practitioners where data augmentation techniques may confer improvements.
