论文标题
GraphFM：通过功能动量改善大规模GNN训练
GraphFM: Improving Large-Scale GNN Training via Feature Momentum
论文作者
论文摘要
对大规模淋巴结分类的图形神经网络（GNNS）培训具有挑战性。关键困难在于在避免邻里爆炸问题的同时获得准确的隐藏节点表示。在这里，我们提出了一种名为“特征动量”（FM）的新技术，该技术在更新功能表示时使用动量步骤来合并历史嵌入。我们开发了两种特定的算法，即GraphFM-ib和GraphFM-OB，它们分别考虑了内部和隔离外数据。 GraphFM-IB将FM应用于批处理采样数据，而GraphFM-OB将FM应用于隔离数据的口气外数据，这是批处理数据的1跳社区。我们为GraphFM-ib提供了收敛分析，并为GraphFM-OB提供了一些理论见解。从经验上讲，我们观察到GraphFM-IB可以有效缓解现有方法的邻里爆炸问题。此外，GraphFM-OB在多个大型图形数据集上实现了有希望的性能。
Training of graph neural networks (GNNs) for large-scale node classification is challenging. A key difficulty lies in obtaining accurate hidden node representations while avoiding the neighborhood explosion problem. Here, we propose a new technique, named feature momentum (FM), that uses a momentum step to incorporate historical embeddings when updating feature representations. We develop two specific algorithms, known as GraphFM-IB and GraphFM-OB, that consider in-batch and out-of-batch data, respectively. GraphFM-IB applies FM to in-batch sampled data, while GraphFM-OB applies FM to out-of-batch data that are 1-hop neighborhood of in-batch data. We provide a convergence analysis for GraphFM-IB and some theoretical insight for GraphFM-OB. Empirically, we observe that GraphFM-IB can effectively alleviate the neighborhood explosion problem of existing methods. In addition, GraphFM-OB achieves promising performance on multiple large-scale graph datasets.
