论文标题
通过知识蒸馏规避自动仪的异常值
Circumventing Outliers of AutoAugment with Knowledge Distillation
论文作者
论文摘要
自动仪是一种强大的算法，可以提高许多视觉任务的准确性，但它对操作员空间以及超参数敏感，不正确的设置可能会退化网络优化。本文深入研究了工作机制，并揭示了自动说明可以从培训图像中删除一部分判别信息，因此坚持地面真实标签不再是最佳选择。为了减轻监督的不准确性，我们利用知识蒸馏是指教师模型的输出来指导网络培训。实验是在标准图像分类基准中进行的，并证明了我们方法在抑制数据增强噪声和稳定训练方面的有效性。在知识蒸馏和自动宣传的合作之后，我们声称对Imagenet分类的最新最先进，前1位准确性为85.8％。
AutoAugment has been a powerful algorithm that improves the accuracy of many vision tasks, yet it is sensitive to the operator space as well as hyper-parameters, and an improper setting may degenerate network optimization. This paper delves deep into the working mechanism, and reveals that AutoAugment may remove part of discriminative information from the training image and so insisting on the ground-truth label is no longer the best option. To relieve the inaccuracy of supervision, we make use of knowledge distillation that refers to the output of a teacher model to guide network training. Experiments are performed in standard image classification benchmarks, and demonstrate the effectiveness of our approach in suppressing noise of data augmentation and stabilizing training. Upon the cooperation of knowledge distillation and AutoAugment, we claim the new state-of-the-art on ImageNet classification with a top-1 accuracy of 85.8%.
