论文标题
基于转移学习的虚拟现实耳机的部分阻塞下的面部表情识别
Facial Expression Recognition Under Partial Occlusion from Virtual Reality Headsets based on Transfer Learning
论文作者
论文摘要
情感的面部表情是我们日常交流中的主要渠道，并且近年来一直进行了激烈的研究。为了自动推断面部表情，卷积神经网络的方法因其对面部表达识别（FER）任务的可靠性而被广泛采用。另一方面虚拟现实（VR）作为沉浸式多媒体平台获得了知名度，而FER可以提供丰富的媒体体验。但是，由于面部的上半部被完全遮住，戴上头部载有VR耳机的面部表情是一项艰巨的任务。在本文中，我们试图克服这些问题，并在存在严重阻塞的情况下专注于面部表情识别，因为用户在VR设置中戴着头部安装的显示。我们提出了一个几何模型，以模拟可以将可以应用于现有的FER数据集的三星齿轮VR耳机产生的遮挡。然后，我们采用了一种转移学习方法，从两个验证的网络（即VGG和Resnet）开始。我们进一步微调了FER+和RAF-DB数据集上的网络。实验结果表明，我们的方法在三个修改的基准数据集上训练，这些方法与戴上商品VR耳机构成的逼真的遮挡相当。本文的代码可在以下网址获得：https：//github.com/bita-github/mrp-fer
Facial expressions of emotion are a major channel in our daily communications, and it has been subject of intense research in recent years. To automatically infer facial expressions, convolutional neural network based approaches has become widely adopted due to their proven applicability to Facial Expression Recognition (FER) task.On the other hand Virtual Reality (VR) has gained popularity as an immersive multimedia platform, where FER can provide enriched media experiences. However, recognizing facial expression while wearing a head-mounted VR headset is a challenging task due to the upper half of the face being completely occluded. In this paper we attempt to overcome these issues and focus on facial expression recognition in presence of a severe occlusion where the user is wearing a head-mounted display in a VR setting. We propose a geometric model to simulate occlusion resulting from a Samsung Gear VR headset that can be applied to existing FER datasets. Then, we adopt a transfer learning approach, starting from two pretrained networks, namely VGG and ResNet. We further fine-tune the networks on FER+ and RAF-DB datasets. Experimental results show that our approach achieves comparable results to existing methods while training on three modified benchmark datasets that adhere to realistic occlusion resulting from wearing a commodity VR headset. Code for this paper is available at: https://github.com/bita-github/MRP-FER
