论文标题
天空中的城堡：视频中动态的天空替代与协调
Castle in the Sky: Dynamic Sky Replacement and Harmonization in Videos
论文作者
论文摘要
本文提出了一种基于视觉天空替代和协调的基于视觉的方法，该方法可以在具有可控样式的视频中自动生成逼真而戏剧性的天空背景。与以前的Sky编辑方法不同，该方法要么关注静态照片，要么需要在智能手机上集成在拍摄视频中的惯性测量单元，我们的方法纯粹是基于视觉的，而没有对捕获设备的任何要求，并且可以很好地应用于在线或离线处理方案。我们的方法实时运行，并且没有用户交互。我们将这个艺术创作过程分解为几个代理任务，包括天空垫，运动估算和图像混合。实验是在通过手持智能手机和破折号摄像机在野外捕获的视频上进行的，并在视觉质量和照明/运动动力学方面表现出高忠诚度和良好的概括。我们的代码和动画结果可在\ url {https://jiupinjia.github.io/skyar/}上获得。
This paper proposes a vision-based method for video sky replacement and harmonization, which can automatically generate realistic and dramatic sky backgrounds in videos with controllable styles. Different from previous sky editing methods that either focus on static photos or require inertial measurement units integrated in smartphones on shooting videos, our method is purely vision-based, without any requirements on the capturing devices, and can be well applied to either online or offline processing scenarios. Our method runs in real-time and is free of user interactions. We decompose this artistic creation process into a couple of proxy tasks including sky matting, motion estimation, and image blending. Experiments are conducted on videos diversely captured in the wild by handheld smartphones and dash cameras, and show high fidelity and good generalization of our method in both visual quality and lighting/motion dynamics. Our code and animated results are available at \url{https://jiupinjia.github.io/skyar/}.
