论文标题
持续学习中任务订购的效果
The Effect of Task Ordering in Continual Learning
论文作者
论文摘要
我们研究任务排序对持续学习绩效的影响。我们对合成和自然主义数据集进行了一系列广泛的经验实验，并表明重新排序的任务显着影响灾难性遗忘的数量。连接到课程学习领域，我们表明可以利用任务排序的效果来修改持续的学习绩效，并提出一种简单的方法。我们的方法计算所有任务对之间的距离，其中将距离定义为朝目标任务的梯度步骤的源任务曲率。使用统计上严格的方法和声音实验设计，我们表明任务订购是持续学习的重要方面，可以修改以提高性能。
We investigate the effect of task ordering on continual learning performance. We conduct an extensive series of empirical experiments on synthetic and naturalistic datasets and show that reordering tasks significantly affects the amount of catastrophic forgetting. Connecting to the field of curriculum learning, we show that the effect of task ordering can be exploited to modify continual learning performance, and present a simple approach for doing so. Our method computes the distance between all pairs of tasks, where distance is defined as the source task curvature of a gradient step toward the target task. Using statistically rigorous methods and sound experimental design, we show that task ordering is an important aspect of continual learning that can be modified for improved performance.
