论文标题
虚假视频检测的生理信号的视觉表示
Visual Representations of Physiological Signals for Fake Video Detection
论文作者
论文摘要
鉴于我们不断增加的在线形象和信息摄入，现实的虚假视频是传播有害错误信息的潜在工具。本文提出了一种基于多模式学习的方法，用于检测真实和虚假视频。该方法结合了来自三种模式的信息 - 音频，视频和生理学。我们通过将视频与生理学的信息相结合，或通过新颖地学习这两种方式与所提出的图形卷积网络体系结构的融合来研究两种结合视频和生理方式的策略。将两种方式结合的两种策略都取决于一种新的方法来产生生理信号的视觉表示。然后，对真实视频和虚假视频的检测基于音频和修改的视频方式之间的差异。在两个基准数据集上评估了所提出的方法，与以前的方法相比，结果显示出检测性能的显着增加。
Realistic fake videos are a potential tool for spreading harmful misinformation given our increasing online presence and information intake. This paper presents a multimodal learning-based method for detection of real and fake videos. The method combines information from three modalities - audio, video, and physiology. We investigate two strategies for combining the video and physiology modalities, either by augmenting the video with information from the physiology or by novelly learning the fusion of those two modalities with a proposed Graph Convolutional Network architecture. Both strategies for combining the two modalities rely on a novel method for generation of visual representations of physiological signals. The detection of real and fake videos is then based on the dissimilarity between the audio and modified video modalities. The proposed method is evaluated on two benchmark datasets and the results show significant increase in detection performance compared to previous methods.
