论文标题
通过顺序均匀设计优化超参数
Hyperparameter Optimization via Sequential Uniform Designs
论文作者
论文摘要
超参数优化（HPO）在自动化机器学习（AUTOML）中起着核心作用。这是一项具有挑战性的任务，因为超参数的响应表面通常是未知的，因此本质上是一个全球优化问题。本文将HPO重新定义为计算机实验，并提出了具有三倍优势的新型顺序统一设计（SECD）策略：a）使用均匀分布的设计点对超参数空间进行自适应探索，而无需昂贵的元模型和获取优化； b）逐批批量设计点是通过并行处理支持依次生成的； c）开发了一种新的增强统一设计算法，可为实时生成后续设计点的有效生成。对全球优化任务和HPO应用程序进行了广泛的实验。数值结果表明，所提出的SECD策略的表现优于基准HPO方法，因此，它可以是现有汽车工具的有前途和竞争性的替代方法。
Hyperparameter optimization (HPO) plays a central role in the automated machine learning (AutoML). It is a challenging task as the response surfaces of hyperparameters are generally unknown, hence essentially a global optimization problem. This paper reformulates HPO as a computer experiment and proposes a novel sequential uniform design (SeqUD) strategy with three-fold advantages: a) the hyperparameter space is adaptively explored with evenly spread design points, without the need of expensive meta-modeling and acquisition optimization; b) the batch-by-batch design points are sequentially generated with parallel processing support; c) a new augmented uniform design algorithm is developed for the efficient real-time generation of follow-up design points. Extensive experiments are conducted on both global optimization tasks and HPO applications. The numerical results show that the proposed SeqUD strategy outperforms benchmark HPO methods, and it can be therefore a promising and competitive alternative to existing AutoML tools.
