论文标题
通过检测行动边界来减轻过度细分错误
Alleviating Over-segmentation Errors by Detecting Action Boundaries
论文作者
论文摘要
我们为时间动作分割任务提出了一个有效的框架，即动作细分细分框架（ASRF）。我们的模型架构由一个长期特征提取器和两个分支组成：动作分割分支（ASB）和边界回归分支（BRB）。长期特征提取器为两个分支提供了宽阔的时间接受场的共享特征。 ASB将视频帧与动作类别分类，而BRB会回归动作边界概率。 BRB预测的动作边界可改善ASB的输出，从而显着提高了性能。我们的贡献是三个方面：（i）我们提出了一个时间动作分割的框架，即ASRF，将时间动作分割分为框架的动作分类和动作边界回归。我们的框架优化了使用预测的动作边界的动作类别框架级别的假设。 （ii）我们提出了一种损失函数，以平滑作用概率的过渡，并分析各种损失函数的时间分割。 （iii）我们的框架在三个具有挑战性的数据集上优于最先进的方法，就节段性的编辑距离而言，提高了13.7％的提高，就节段性F1分数而言，提高了13.7％。我们的代码即将公开可用。
We propose an effective framework for the temporal action segmentation task, namely an Action Segment Refinement Framework (ASRF). Our model architecture consists of a long-term feature extractor and two branches: the Action Segmentation Branch (ASB) and the Boundary Regression Branch (BRB). The long-term feature extractor provides shared features for the two branches with a wide temporal receptive field. The ASB classifies video frames with action classes, while the BRB regresses the action boundary probabilities. The action boundaries predicted by the BRB refine the output from the ASB, which results in a significant performance improvement. Our contributions are three-fold: (i) We propose a framework for temporal action segmentation, the ASRF, which divides temporal action segmentation into frame-wise action classification and action boundary regression. Our framework refines frame-level hypotheses of action classes using predicted action boundaries. (ii) We propose a loss function for smoothing the transition of action probabilities, and analyze combinations of various loss functions for temporal action segmentation. (iii) Our framework outperforms state-of-the-art methods on three challenging datasets, offering an improvement of up to 13.7% in terms of segmental edit distance and up to 16.1% in terms of segmental F1 score. Our code will be publicly available soon.
