论文标题
VIT5：越南语言的预估计的文本到文本变压器
ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation
论文作者
论文摘要
我们提出了VIT5，这是一种基于变压器的基于变压器的编码器模型，用于越南语言。有了T5风格的自我监督预处理，VIT5接受了大量高质量和多样化的越南文本的训练。我们基于两个下游文本生成任务，抽象性文本摘要和命名实体识别基础VIT5。尽管由于其丰富而大的数据来源，已经对英语的抽象性文本摘要进行了广泛的研究，但对越南人的同一任务的研究很少，这是一种较低的资源语言。在这项工作中，我们对越南抽象的摘要和命名实体识别进行了详尽的实验，从而验证了VIT5的性能对许多其他基于变压器的编码器模型。我们的实验表明，VIT5显着优于现有模型，并在越南文本摘要上实现最先进的结果。在命名实体识别的任务下，VIT5与基于编码器的变压器模型的先前最佳结果具有竞争力。进一步的分析表明，在自我监督预处理期间，上下文长度在不同环境的下游性能上的重要性。
We present ViT5, a pretrained Transformer-based encoder-decoder model for the Vietnamese language. With T5-style self-supervised pretraining, ViT5 is trained on a large corpus of high-quality and diverse Vietnamese texts. We benchmark ViT5 on two downstream text generation tasks, Abstractive Text Summarization and Named Entity Recognition. Although Abstractive Text Summarization has been widely studied for the English language thanks to its rich and large source of data, there has been minimal research into the same task in Vietnamese, a much lower resource language. In this work, we perform exhaustive experiments on both Vietnamese Abstractive Summarization and Named Entity Recognition, validating the performance of ViT5 against many other pretrained Transformer-based encoder-decoder models. Our experiments show that ViT5 significantly outperforms existing models and achieves state-of-the-art results on Vietnamese Text Summarization. On the task of Named Entity Recognition, ViT5 is competitive against previous best results from pretrained encoder-based Transformer models. Further analysis shows the importance of context length during the self-supervised pretraining on downstream performance across different settings.
