论文标题
一种用于培训多层进发神经网络的有效有效的初始化计划
An Effective and Efficient Initialization Scheme for Training Multi-layer Feedforward Neural Networks
论文作者
论文摘要
网络初始化是训练神经网络的第一步骤和关键步骤。在本文中，我们根据著名的斯坦的身份提出了一种新颖的网络初始化计划。通过将多层馈电神经网络视为多指数模型的级联，对第一个隐藏层的投影权重可以使用输入的二阶得分函数和响应之间的跨音量矩阵的特征向量初始化。然后将输入数据转发到下一层，并可以重复此类过程，直到所有隐藏的图层初始化为止。最后，通过广义线性建模初始化输出层的权重。通过广泛的数值结果显示，这种提出的SteingLM方法比训练神经网络常用的其他流行方法更快，更准确。
Network initialization is the first and critical step for training neural networks. In this paper, we propose a novel network initialization scheme based on the celebrated Stein's identity. By viewing multi-layer feedforward neural networks as cascades of multi-index models, the projection weights to the first hidden layer are initialized using eigenvectors of the cross-moment matrix between the input's second-order score function and the response. The input data is then forward propagated to the next layer and such a procedure can be repeated until all the hidden layers are initialized. Finally, the weights for the output layer are initialized by generalized linear modeling. Such a proposed SteinGLM method is shown through extensive numerical results to be much faster and more accurate than other popular methods commonly used for training neural networks.
