论文标题
使SGD参数不含
Making SGD Parameter-Free
论文作者
论文摘要
我们开发了一种用于无参数随机凸优化（SCO）的算法，该算法的收敛速率仅是相应已知参数设置的双重载体因子大于最佳速率。相比之下，最佳的无参数SCO率基于在线无参数后悔界限，与已知参数相比，其不可避免的多余的对数项。我们的算法在概念上很简单，具有高概率的保证，并且也部分适应未知的梯度规范，平滑度和强大的凸度。我们结果的核心是SGD步骤尺寸选择的新型无参数证书，并且具有均匀的浓度结果，该结果假定SGD迭代的A-Priori界限。
We develop an algorithm for parameter-free stochastic convex optimization (SCO) whose rate of convergence is only a double-logarithmic factor larger than the optimal rate for the corresponding known-parameter setting. In contrast, the best previously known rates for parameter-free SCO are based on online parameter-free regret bounds, which contain unavoidable excess logarithmic terms compared to their known-parameter counterparts. Our algorithm is conceptually simple, has high-probability guarantees, and is also partially adaptive to unknown gradient norms, smoothness, and strong convexity. At the heart of our results is a novel parameter-free certificate for SGD step size choice, and a time-uniform concentration result that assumes no a-priori bounds on SGD iterates.
