论文标题
符号学习以优化：迈向可解释性和可伸缩性
Symbolic Learning to Optimize: Towards Interpretability and Scalability
论文作者
论文摘要
关于优化的学习（L2O）的最新研究提出了自动化和加速复杂任务的优化程序的有希望的途径。现有的L2O模型通过神经网络参数化优化规则，并通过元训练来学习这些数字规则。但是，他们面临两个常见的陷阱：（1）可伸缩性：神经网络代表的数值规则为应用L2O模型创建了额外的内存开销，并将其适用性限制为优化较大的任务； （2）解释性：目前尚不清楚L2O模型在其黑盒优化规则中学到了什么，也不直接以可解释的方式比较不同的L2O模型。为了避免这两个陷阱，本文通过将符号回归的强大工具引入L2O来证明我们可以“用一块石头杀死两只鸟”的概念。在本文中，我们为L2O建立了整体符号表示和分析框架，该框架为可学习的优化者提供了一系列见解。利用我们的发现，我们进一步提出了一个轻巧的L2O模型，该模型可以在大规模问题上进行元训练，并且表现优于人类设计和调整的优化器。我们的工作将为L2O研究提供全新的观点。代码可在以下网址获得：https：//github.com/vita-group/symbolic-learning-to-optimize。
Recent studies on Learning to Optimize (L2O) suggest a promising path to automating and accelerating the optimization procedure for complicated tasks. Existing L2O models parameterize optimization rules by neural networks, and learn those numerical rules via meta-training. However, they face two common pitfalls: (1) scalability: the numerical rules represented by neural networks create extra memory overhead for applying L2O models, and limit their applicability to optimizing larger tasks; (2) interpretability: it is unclear what an L2O model has learned in its black-box optimization rule, nor is it straightforward to compare different L2O models in an explainable way. To avoid both pitfalls, this paper proves the concept that we can "kill two birds by one stone", by introducing the powerful tool of symbolic regression to L2O. In this paper, we establish a holistic symbolic representation and analysis framework for L2O, which yields a series of insights for learnable optimizers. Leveraging our findings, we further propose a lightweight L2O model that can be meta-trained on large-scale problems and outperformed human-designed and tuned optimizers. Our work is set to supply a brand-new perspective to L2O research. Codes are available at: https://github.com/VITA-Group/Symbolic-Learning-To-Optimize.
