论文标题
用于持续学习的元合并
Meta-Consolidation for Continual Learning
论文作者
论文摘要
在不忽视已经获得的知识的情况下，不断学习和适应新任务的能力是生物学习系统的标志，当前深度学习系统无法实现。在这项工作中，我们提出了一种持续学习的新方法，称为Merlin：持续学习的元合并。 我们假设神经网络的权重$ \boldsymbolψ$用于解决任务$ \ boldsymbol t $，来自元分布$ p（\ boldsymbol {ψ| t}）$。该元分布是逐步学习和巩固的。我们在具有挑战性的在线持续学习环境中运作，其中仅一次模型可以看到数据点。 我们通过MNIST，CIFAR-10，CIFAR-100和MINI-IMAGENET数据集的持续学习基准进行的实验表现出比五个基线的一致改进，包括最近的最新技术，证实了Merlin的承诺。
The ability to continuously learn and adapt itself to new tasks, without losing grasp of already acquired knowledge is a hallmark of biological learning systems, which current deep learning systems fall short of. In this work, we present a novel methodology for continual learning called MERLIN: Meta-Consolidation for Continual Learning. We assume that weights of a neural network $\boldsymbol ψ$, for solving task $\boldsymbol t$, come from a meta-distribution $p(\boldsymbol{ψ|t})$. This meta-distribution is learned and consolidated incrementally. We operate in the challenging online continual learning setting, where a data point is seen by the model only once. Our experiments with continual learning benchmarks of MNIST, CIFAR-10, CIFAR-100 and Mini-ImageNet datasets show consistent improvement over five baselines, including a recent state-of-the-art, corroborating the promise of MERLIN.
