论文标题
动态对象去除和时空的RGB-D通过几何学吸引对抗性学习
Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning
论文作者
论文摘要
动态对象对机器人对环境的看法有重大影响，这会降低本地化和映射等基本任务的性能。在这项工作中，我们通过在动态对象遮挡的区域中合成合理的颜色，纹理和几何形状来解决此问题。我们提出了新的几何形状感知的Dynafill架构，该结构遵循粗到美的拓扑结构，并结合了我们的封闭式反复反馈机制，以适应来自以前的时间段的融合信息。我们使用对抗性训练来优化架构，以合成精美的逼真的纹理，使其能够以空间和时间连贯的方式在线遮挡的区域幻觉，而无需依靠未来的帧信息。我们的模型将我们作为图像对图像翻译任务的介绍为图像到图像翻译任务，还纠正了与场景中的动态对象（例如阴影或反射）相关的区域。我们介绍了带有RGB-D图像，语义分割标签，相机姿势以及遮挡区域的地面RGB-D信息的大规模高现实数据集。广泛的定量和定性评估表明，即使在充满挑战的天气条件下，我们的方法也可以达到最先进的表现。此外，我们通过综合图像展示了我们方法的实用性，为基于检索的视觉定位提供了结果。
Dynamic objects have a significant impact on the robot's perception of the environment which degrades the performance of essential tasks such as localization and mapping. In this work, we address this problem by synthesizing plausible color, texture and geometry in regions occluded by dynamic objects. We propose the novel geometry-aware DynaFill architecture that follows a coarse-to-fine topology and incorporates our gated recurrent feedback mechanism to adaptively fuse information from previous timesteps. We optimize our architecture using adversarial training to synthesize fine realistic textures which enables it to hallucinate color and depth structure in occluded regions online in a spatially and temporally coherent manner, without relying on future frame information. Casting our inpainting problem as an image-to-image translation task, our model also corrects regions correlated with the presence of dynamic objects in the scene, such as shadows or reflections. We introduce a large-scale hyperrealistic dataset with RGB-D images, semantic segmentation labels, camera poses as well as groundtruth RGB-D information of occluded regions. Extensive quantitative and qualitative evaluations show that our approach achieves state-of-the-art performance, even in challenging weather conditions. Furthermore, we present results for retrieval-based visual localization with the synthesized images that demonstrate the utility of our approach.
