论文标题
神经网络合奏的特征空间粒子推断
Feature Space Particle Inference for Neural Network Ensembles
论文作者
论文摘要
深度神经网络的合奏表明，单个模型的性能提高了。从贝叶斯的角度来看，为了增强合奏成员的多样性，基于粒子的推理方法为一种有希望的方法提供了一种有希望的方法。但是，将这些方法应用于神经网络的最佳方法仍不清楚：由于过度参数化问题，从重量空间后部寻求未效率低下的样本，而直接从功能空间后部寻求样品通常会导致严重的不适当。在这项研究中，我们提出在特征空间中优化颗粒，在特征空间中，特定中间层的激活所在以解决上述困难。我们的方法鼓励每个成员捕获不同的特征，这有望改善集成预测的鲁棒性。对现实世界数据集的广泛评估表明，我们的模型在各种指标（包括准确性，校准和鲁棒性）上的金标准深度合奏极大地胜过。代码可在https://github.com/densoitlab/featurepi上找到。
Ensembles of deep neural networks demonstrate improved performance over single models. For enhancing the diversity of ensemble members while keeping their performance, particle-based inference methods offer a promising approach from a Bayesian perspective. However, the best way to apply these methods to neural networks is still unclear: seeking samples from the weight-space posterior suffers from inefficiency due to the over-parameterization issues, while seeking samples directly from the function-space posterior often results in serious underfitting. In this study, we propose optimizing particles in the feature space where the activation of a specific intermediate layer lies to address the above-mentioned difficulties. Our method encourages each member to capture distinct features, which is expected to improve ensemble prediction robustness. Extensive evaluation on real-world datasets shows that our model significantly outperforms the gold-standard Deep Ensembles on various metrics, including accuracy, calibration, and robustness. Code is available at https://github.com/DensoITLab/featurePI .
