论文标题
通过有效的注意力桥梁融合通过极化驱动的语义分割
Polarization-driven Semantic Segmentation via Efficient Attention-bridged Fusion
论文作者
论文摘要
语义细分（SS）对于在自动驾驶汽车，辅助导航等安全性临界应用中的室外现场感知很有希望。但是，传统SS主要基于RGB图像，该图像限制了SS在复杂的室外场景中的可靠性，其中RGB图像缺乏必要的信息维度来完全感知不受约束的环境。作为初步研究，我们在意外的障碍检测方案中检查了SS，这证明了多模式融合的必要性。因此，在这项工作中，我们提出了Eafnet，这是一个有效的注意力桥接融合网络，可利用来自不同光学传感器的互补信息。具体而言，我们考虑到其光学特性以符合多种材料的稳健表示，我们结合了极化传感以获取补充信息。通过使用单发偏振传感器，我们构建了第一个RGB-P数据集，该数据集由394个注释的像素对齐的RGB偏振图像组成。全面的实验显示了EAFNET对融合极化和RGB信息的有效性，以及适应其他传感器组合方案的灵活性。
Semantic Segmentation (SS) is promising for outdoor scene perception in safety-critical applications like autonomous vehicles, assisted navigation and so on. However, traditional SS is primarily based on RGB images, which limits the reliability of SS in complex outdoor scenes, where RGB images lack necessary information dimensions to fully perceive unconstrained environments. As preliminary investigation, we examine SS in an unexpected obstacle detection scenario, which demonstrates the necessity of multimodal fusion. Thereby, in this work, we present EAFNet, an Efficient Attention-bridged Fusion Network to exploit complementary information coming from different optical sensors. Specifically, we incorporate polarization sensing to obtain supplementary information, considering its optical characteristics for robust representation of diverse materials. By using a single-shot polarization sensor, we build the first RGB-P dataset which consists of 394 annotated pixel-aligned RGB-Polarization images. A comprehensive variety of experiments shows the effectiveness of EAFNet to fuse polarization and RGB information, as well as the flexibility to be adapted to other sensor combination scenarios.
