论文标题
通过赛车培训进行数据驱动的正规化，用于概括神经网络
Data-driven Regularization via Racecar Training for Generalizing Neural Networks
论文作者
论文摘要
我们提出了一种新的培训方法，用于改善神经网络中的概括。我们表明，与正交性的常规约束相反，我们的方法代表{\ em数据依赖性}正交性约束，并且与权重矩阵的单数值分解密切相关。我们还展示了如何通过反向通行证在实用网络体系结构中易于实现的表述，该票旨在重建网络内部状态的完整顺序。尽管是一个简单的变化，但我们证明了这种前向训练方法（我们称为{\ em赛车}训练，它导致从给定数据集中提取的更多通用功能。通过我们的方法训练的网络在所有层中显示了输入和输出之间的更平衡的共同信息，可提高解释性，并在各种任务和任务转移方面提高了性能。
We propose a novel training approach for improving the generalization in neural networks. We show that in contrast to regular constraints for orthogonality, our approach represents a {\em data-dependent} orthogonality constraint, and is closely related to singular value decompositions of the weight matrices. We also show how our formulation is easy to realize in practical network architectures via a reverse pass, which aims for reconstructing the full sequence of internal states of the network. Despite being a surprisingly simple change, we demonstrate that this forward-backward training approach, which we refer to as {\em racecar} training, leads to significantly more generic features being extracted from a given data set. Networks trained with our approach show more balanced mutual information between input and output throughout all layers, yield improved explainability and, exhibit improved performance for a variety of tasks and task transfers.
