论文标题
神经编码具有视觉关注
Neural encoding with visual attention
论文作者
论文摘要
视觉感知受到关注重点的严重影响。由于资源有限，众所周知，神经表现形式有利于参加活动的位置。使用来自大量人类受试者观看电影的同时观看电影的同时引人注目的磁共振成像（fMRI）录音，我们首先证明，以注意力掩盖的形式利用凝视信息可以显着提高神经编码模型中的大脑响应预测准确性。接下来，我们提出了一种新颖的神经编码方法，包括一个可训练的软感注意模块。使用我们的新方法，我们证明，只有在fMRI响应数据上，而不依赖任何眼神跟踪，可以通过端到端的学习来学习视觉注意力策略。有趣的是，尽管没有明确的监督，但模型对独立数据估算的注意位置与相应的眼睛固定模式非常吻合。总之，这些发现表明，注意模块可以在视觉刺激的神经编码模型中发挥作用。
Visual perception is critically influenced by the focus of attention. Due to limited resources, it is well known that neural representations are biased in favor of attended locations. Using concurrent eye-tracking and functional Magnetic Resonance Imaging (fMRI) recordings from a large cohort of human subjects watching movies, we first demonstrate that leveraging gaze information, in the form of attentional masking, can significantly improve brain response prediction accuracy in a neural encoding model. Next, we propose a novel approach to neural encoding by including a trainable soft-attention module. Using our new approach, we demonstrate that it is possible to learn visual attention policies by end-to-end learning merely on fMRI response data, and without relying on any eye-tracking. Interestingly, we find that attention locations estimated by the model on independent data agree well with the corresponding eye fixation patterns, despite no explicit supervision to do so. Together, these findings suggest that attention modules can be instrumental in neural encoding models of visual stimuli.
