论文标题
当地的可塑性规则可以使用自我监督的对比预测来学习深层的表示
Local plasticity rules can learn deep representations using self-supervised contrastive predictions
论文作者
论文摘要
在大脑中的学习知之甚少，学习规则，尊重生物学约束但产生深层层次表示的规则仍然未知。在这里，我们提出了一个学习规则，该规则从神经科学和自学深度学习方面的最新进展中获得灵感。学习可以最大程度地减少简单的特定损失函数，并且不需要在层之间或之间的误差信号后退。取而代之的是，重量更新遵循局部的Hebbian学习规则，仅取决于前和突触后神经元活动，预测性树突状输入以及广播广播的调制因子，这些因子与大型神经元相同。学习规则将对比的预测学习应用于使用扫视的因果生物学设置（即凝视方向的快速变化）。我们发现接受了这种自我监管和本地规则培训的网络建立了图像，语音和视频的深层层次结构表示。
Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. Learning minimizes a simple layer-specific loss function and does not need to back-propagate error signals within or between layers. Instead, weight updates follow a local, Hebbian, learning rule that only depends on pre- and post-synaptic neuronal activity, predictive dendritic input and widely broadcasted modulation factors which are identical for large groups of neurons. The learning rule applies contrastive predictive learning to a causal, biological setting using saccades (i.e. rapid shifts in gaze direction). We find that networks trained with this self-supervised and local rule build deep hierarchical representations of images, speech and video.
