论文标题
限制性玻尔兹曼机器的三个学习阶段和准确性效率的权衡
Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines
论文作者
论文摘要
受限的玻尔兹曼机器（RBMS）为无监督的机器学习提供了一种多功能体系结构，原则上可以以任意准确的速度近似任何目标概率分布。但是，RBM模型通常由于其计算复杂性而无法直接访问，并调用Markov-Chain采样来分析学习概率分布。因此，对于培训和最终应用，希望拥有既准确又有效的采样器。我们强调，这两个目标通常相互竞争，无法同时实现。更具体地说，我们确定并定量地表征了RBM学习的三个制度：独立学习，在这种情况下，精度提高而不会失去效率；相关学习，较高的精度需要较低的效率；和退化，精度和效率都不再改善甚至恶化。这些发现基于数值实验和启发式论证。
Restricted Boltzmann Machines (RBMs) offer a versatile architecture for unsupervised machine learning that can in principle approximate any target probability distribution with arbitrary accuracy. However, the RBM model is usually not directly accessible due to its computational complexity, and Markov-chain sampling is invoked to analyze the learned probability distribution. For training and eventual applications, it is thus desirable to have a sampler that is both accurate and efficient. We highlight that these two goals generally compete with each other and cannot be achieved simultaneously. More specifically, we identify and quantitatively characterize three regimes of RBM learning: independent learning, where the accuracy improves without losing efficiency; correlation learning, where higher accuracy entails lower efficiency; and degradation, where both accuracy and efficiency no longer improve or even deteriorate. These findings are based on numerical experiments and heuristic arguments.
