论文标题
PS-NERV：视频的贴剂风格化神经表示
PS-NeRV: Patch-wise Stylized Neural Representations for Videos
论文作者
论文摘要
我们研究如何代表具有隐式神经表示（INRS）的视频。经典INRS方法通常利用MLP将输入坐标映射到输出像素。虽然最近的一些作品试图将整个图像直接用CNN重建。但是，我们认为，以上像素和图像策略都不利于视频数据。取而代之的是，我们提出了一个贴片解决方案PS-NERV，该解决方案将视频表示为贴片的函数和相应的补丁坐标。它自然继承了图像方法的优势，并以快速解码速度实现了出色的重建性能。整个方法包括常规模块，例如位置嵌入，MLP和CNN，同时还引入了ADAIN以增强中间特征。这些简单而基本的更改可以帮助网络轻松拟合高频细节。广泛的实验证明了它在几个与视频有关的任务中的有效性，例如视频压缩和视频介绍。
We study how to represent a video with implicit neural representations (INRs). Classical INRs methods generally utilize MLPs to map input coordinates to output pixels. While some recent works have tried to directly reconstruct the whole image with CNNs. However, we argue that both the above pixel-wise and image-wise strategies are not favorable to video data. Instead, we propose a patch-wise solution, PS-NeRV, which represents videos as a function of patches and the corresponding patch coordinate. It naturally inherits the advantages of image-wise methods, and achieves excellent reconstruction performance with fast decoding speed. The whole method includes conventional modules, like positional embedding, MLPs and CNNs, while also introduces AdaIN to enhance intermediate features. These simple yet essential changes could help the network easily fit high-frequency details. Extensive experiments have demonstrated its effectiveness in several video-related tasks, such as video compression and video inpainting.
