论文标题
带有加权图形步行自动机的变压器的桥接图形位置编码
Bridging Graph Position Encodings for Transformers with Weighted Graph-Walking Automata
论文作者
论文摘要
图形神经网络文献中的当前目标是使变压器能够在语言和视觉任务上成功地在图形结构数据上运行。由于变压器的原始正弦位置编码（PES）不适用于图形，因此最近的工作重点是开发图形PE，它植根于光谱图理论或图形的各种空间特征。在这项工作中，我们基于加权绘制的步行自动机（Graphing Walking Automata的新型扩展），引入了一个新的图PE，图形自动机PE（GAPE）。我们将GAPE的性能与机器翻译和图形结构任务上的其他PE方案进行了比较，并表明它概括了其他几个PES。这项研究的另一个贡献是对图形变压器中许多最新PE的理论和受控的实验比较，与边缘特征的使用无关。
A current goal in the graph neural network literature is to enable transformers to operate on graph-structured data, given their success on language and vision tasks. Since the transformer's original sinusoidal positional encodings (PEs) are not applicable to graphs, recent work has focused on developing graph PEs, rooted in spectral graph theory or various spatial features of a graph. In this work, we introduce a new graph PE, Graph Automaton PE (GAPE), based on weighted graph-walking automata (a novel extension of graph-walking automata). We compare the performance of GAPE with other PE schemes on both machine translation and graph-structured tasks, and we show that it generalizes several other PEs. An additional contribution of this study is a theoretical and controlled experimental comparison of many recent PEs in graph transformers, independent of the use of edge features.
