论文标题
深网的分离和集中
Separation and Concentration in Deep Networks
论文作者
论文摘要
数值实验表明，深度神经网络分类器围绕其平均值逐渐分开分布，从而在训练集上实现线性可分离性，并增加了Fisher判别比率。我们用两种类型的操作员来解释这种机制。我们证明，没有偏见的整流器适用于签名的紧密框架可以分开类平均值并增加Fisher比率。相反，紧密框架上的软阈值可以降低类内的变化，同时保留类手段。高斯混合模型证明了差异界限。对于图像分类，我们表明，可以通过未学习的整流小波框架来实现类平均值的分离。它定义了散射变换。学习$ 1 \ times 1 $沿散射频道卷积紧身框架，并应用软势率降低了类内部的变化。所得的散射网络达到了RESNET-18在CIFAR-10和Imagenet上的分类准确性，层较少，没有学习的偏见。
Numerical experiments demonstrate that deep neural network classifiers progressively separate class distributions around their mean, achieving linear separability on the training set, and increasing the Fisher discriminant ratio. We explain this mechanism with two types of operators. We prove that a rectifier without biases applied to sign-invariant tight frames can separate class means and increase Fisher ratios. On the opposite, a soft-thresholding on tight frames can reduce within-class variabilities while preserving class means. Variance reduction bounds are proved for Gaussian mixture models. For image classification, we show that separation of class means can be achieved with rectified wavelet tight frames that are not learned. It defines a scattering transform. Learning $1 \times 1$ convolutional tight frames along scattering channels and applying a soft-thresholding reduces within-class variabilities. The resulting scattering network reaches the classification accuracy of ResNet-18 on CIFAR-10 and ImageNet, with fewer layers and no learned biases.
