论文标题
通过明智的选择学习一个弱监督的视频演员分割模型
Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection
论文作者
论文摘要
我们讨论了弱监督的视频行动分段（VAAS），该视频分段（VAAS）扩展了一般视频对象细分（VOS），以便考虑参与者的动作标签。 VOS上最成功的方法合成了伪注销（PAS），然后迭代地改进它们。但是，他们面临着如何从大量PA的高品质高质量中选择的挑战，如何为弱监督训练设定适当的停止条件，以及如何初始化与VAA有关的PA。为了克服这些挑战，我们提出了一个普遍的弱监督框架，并通过明智的训练样本和模型评估标准（WS^2）进行选择。 WS^2不是盲目信任的质量不合时宜的PA，而是采用基于学习的选择来选择有效的PA，并作为新颖的区域完整性标准作为弱监督培训的停止条件。此外，设计了一个3D-CONV GCAM，以适应VAAS任务。广泛的实验表明，WS^2在弱监督的VOS和VAAS任务上都达到了最新的性能，并且与VAAS上最好的全面监督方法相当。
We address weakly-supervised video actor-action segmentation (VAAS), which extends general video object segmentation (VOS) to additionally consider action labels of the actors. The most successful methods on VOS synthesize a pool of pseudo-annotations (PAs) and then refine them iteratively. However, they face challenges as to how to select from a massive amount of PAs high-quality ones, how to set an appropriate stop condition for weakly-supervised training, and how to initialize PAs pertaining to VAAS. To overcome these challenges, we propose a general Weakly-Supervised framework with a Wise Selection of training samples and model evaluation criterion (WS^2). Instead of blindly trusting quality-inconsistent PAs, WS^2 employs a learning-based selection to select effective PAs and a novel region integrity criterion as a stopping condition for weakly-supervised training. In addition, a 3D-Conv GCAM is devised to adapt to the VAAS task. Extensive experiments show that WS^2 achieves state-of-the-art performance on both weakly-supervised VOS and VAAS tasks and is on par with the best fully-supervised method on VAAS.
