论文标题
防止预测编码网络中的分类精度恶化
Preventing Deterioration of Classification Accuracy in Predictive Coding Networks
论文作者
论文摘要
预测性编码网络（PCN）旨在学习世界的生成模型。给定观察结果，可以倒入该生成模型以推断这些观察结果的原因。但是，当训练PCNS时，通常会观察到明显的病理学，而推理精度峰值峰值，然后通过进一步的训练下降。这不能通过过度拟合来解释，因为训练和测试准确性同时降低。在这里，我们提供了对这一现象的彻底研究，并表明它是由PCN层次融合的速度之间的不平衡引起的。我们证明，可以通过在每一层的重量矩阵正规化：限制矩阵奇异值的相对大小来预防这一点，我们允许重量矩阵改变，但限制了一层可以对其邻居产生的整体影响。我们还证明，通过更具生物学上的简单方案，仅限制重量，可以实现类似的效果。
Predictive Coding Networks (PCNs) aim to learn a generative model of the world. Given observations, this generative model can then be inverted to infer the causes of those observations. However, when training PCNs, a noticeable pathology is often observed where inference accuracy peaks and then declines with further training. This cannot be explained by overfitting since both training and test accuracy decrease simultaneously. Here we provide a thorough investigation of this phenomenon and show that it is caused by an imbalance between the speeds at which the various layers of the PCN converge. We demonstrate that this can be prevented by regularising the weight matrices at each layer: by restricting the relative size of matrix singular values, we allow the weight matrix to change but restrict the overall impact which a layer can have on its neighbours. We also demonstrate that a similar effect can be achieved through a more biologically plausible and simple scheme of just capping the weights.
