论文标题
一个改善神经网络性能的浮雕层
A ReLU Dense Layer to Improve the Performance of Neural Networks
论文作者
论文摘要
我们提出了一种简单且低复杂的方法，以提高训练有素的神经网络的性能。我们使用随机权重和整流线性单元（Relu）激活函数的组合来在训练有素的神经网络中添加一个恢复密度（还原）层，以便它可以实现较低的训练损失。 Relu的无损流属性（LFP）是实现较低训练损失的关键，同时保持概括误差较小。由于具有浅层结构，培训中的梯度问题消失了。我们在实验上表明，还可以改善具有不同优化损失和激活功能的各种神经网络体系结构的训练和测试性能。最后，我们测试了一些最先进的体系结构的还原，并显示基准数据集的性能提高。
We propose ReDense as a simple and low complexity way to improve the performance of trained neural networks. We use a combination of random weights and rectified linear unit (ReLU) activation function to add a ReLU dense (ReDense) layer to the trained neural network such that it can achieve a lower training loss. The lossless flow property (LFP) of ReLU is the key to achieve the lower training loss while keeping the generalization error small. ReDense does not suffer from vanishing gradient problem in the training due to having a shallow structure. We experimentally show that ReDense can improve the training and testing performance of various neural network architectures with different optimization loss and activation functions. Finally, we test ReDense on some of the state-of-the-art architectures and show the performance improvement on benchmark datasets.
