论文标题
信号处理和机器学习中的零订单优化的底漆
A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning
论文作者
论文摘要
零阶（ZO）优化是在许多信号处理和机器学习应用中出现的无梯度优化的子集。它用于解决与基于梯度的方法类似的优化问题。但是，仅使用功能评估，它不需要梯度。具体而言，ZO优化迭代执行三个主要步骤：梯度估计，下降方向计算和解决方案更新。在本文中，我们对ZO优化进行了全面的综述，重点是展示基本的直觉，优化原则和收敛分析的最新进展。此外，我们展示了ZO优化的有希望的应用，例如评估鲁棒性并从黑盒深度学习模型中产生解释以及有效的在线传感器管理。
Zeroth-order (ZO) optimization is a subset of gradient-free optimization that emerges in many signal processing and machine learning applications. It is used for solving optimization problems similarly to gradient-based methods. However, it does not require the gradient, using only function evaluations. Specifically, ZO optimization iteratively performs three major steps: gradient estimation, descent direction computation, and solution update. In this paper, we provide a comprehensive review of ZO optimization, with an emphasis on showing the underlying intuition, optimization principles and recent advances in convergence analysis. Moreover, we demonstrate promising applications of ZO optimization, such as evaluating robustness and generating explanations from black-box deep learning models, and efficient online sensor management.
