论文标题
改善神经网络的Levenberg-Marquardt算法
Improving Levenberg-Marquardt Algorithm for Neural Networks
论文作者
论文摘要
我们探讨了Levenberg-Marquardt（LM）算法在神经网络中的回归（非线性最小二乘）和分类（通用高斯 - 纽顿方法）任务的用法。我们将LM方法的性能与其他流行的一阶算法（例如SGD和ADAM）以及其他二阶算法（例如L-BFG，无Hessian-free和KFAC）进行了比较。我们通过使用自适应动量，学习率线搜索和上坡步骤接受进一步加快LM方法的速度。
We explore the usage of the Levenberg-Marquardt (LM) algorithm for regression (non-linear least squares) and classification (generalized Gauss-Newton methods) tasks in neural networks. We compare the performance of the LM method with other popular first-order algorithms such as SGD and Adam, as well as other second-order algorithms such as L-BFGS , Hessian-Free and KFAC. We further speed up the LM method by using adaptive momentum, learning rate line search, and uphill step acceptance.
