论文标题
深度引导的稀疏结构从电影和电视节目中
Depth-Guided Sparse Structure-from-Motion for Movies and TV Shows
论文作者
论文摘要
现有的运动方法（SFM）产生令人印象深刻的3-D重建结果，尤其是在使用具有大差异的图像时。但是，为了在电影和电视节目中创建引人入胜的视频含义，拍摄特定拍摄时可以移动相机的数量通常受到限制。视频框架之间产生的小动物视差使基于标准几何的SFM方法对电影和电视节目不那么有效。为了应对这一挑战，我们提出了一种简单而有效的方法，该方法使用了从预审计的网络获得的单帧深度优点，以显着改善基于几何形状的SFM。为此，我们首先使用检测到的关键点的深度估计值来重建点云和摄像头，以进行初始的两视频重建。然后，我们执行深度调查优化以注册新图像并在增量重建过程中进行三角测量。为了全面评估我们的方法，我们介绍了一个新的数据集（StudioSFM），该数据集（StudioSFM）由130张镜头组成，并带有15个工作室制作的视频的21k帧，这些视频由专业CG工作室手动注释。我们证明了我们的方法：（a）显着提高了小帕拉克斯环境的3-D重建质量，（b）不会导致具有较大parallax的数据的任何降解，并且（c）保持基于几何基于几何的稀疏稀疏SFM的概括性和可伸缩性。我们的数据集可在https://github.com/amazon-research/small-baseline-camera-tracking上获取。
Existing approaches for Structure from Motion (SfM) produce impressive 3-D reconstruction results especially when using imagery captured with large parallax. However, to create engaging video-content in movies and TV shows, the amount by which a camera can be moved while filming a particular shot is often limited. The resulting small-motion parallax between video frames makes standard geometry-based SfM approaches not as effective for movies and TV shows. To address this challenge, we propose a simple yet effective approach that uses single-frame depth-prior obtained from a pretrained network to significantly improve geometry-based SfM for our small-parallax setting. To this end, we first use the depth-estimates of the detected keypoints to reconstruct the point cloud and camera-pose for initial two-view reconstruction. We then perform depth-regularized optimization to register new images and triangulate the new points during incremental reconstruction. To comprehensively evaluate our approach, we introduce a new dataset (StudioSfM) consisting of 130 shots with 21K frames from 15 studio-produced videos that are manually annotated by a professional CG studio. We demonstrate that our approach: (a) significantly improves the quality of 3-D reconstruction for our small-parallax setting, (b) does not cause any degradation for data with large-parallax, and (c) maintains the generalizability and scalability of geometry-based sparse SfM. Our dataset can be obtained at https://github.com/amazon-research/small-baseline-camera-tracking.
