论文标题
Autohas：高效的超参数和体系结构搜索
AutoHAS: Efficient Hyperparameter and Architecture Search
论文作者
论文摘要
有效的高参数或体系结构搜索方法显示出了显着的结果，但是每个方法仅适用于搜索超参数（HPS）或体系结构。在这项工作中，我们提出了一条统一的管道Autohas，以有效地搜索体系结构和超参数。 Autohas学会了交替更新共享的网络权重和增强学习（RL）控制器，该控制器学习了候选候选者和HP候选者的概率分布。引入了临时权重以存储所选HPS（由控制器）的更新重量，并且基于此临时权重的验证精度可作为更新控制器的奖励。在实验中，我们表明Autohas有效且可以推广到不同的搜索空间，基准和数据集。特别是，在CIFAR-10/100，Imagenet和其他四个数据集上，Autohas可以提高对流行网络体系结构的准确性，例如Resnet和ExcilityNet。
Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architectures. In this work, we propose a unified pipeline, AutoHAS, to efficiently search for both architectures and hyperparameters. AutoHAS learns to alternately update the shared network weights and a reinforcement learning (RL) controller, which learns the probability distribution for the architecture candidates and HP candidates. A temporary weight is introduced to store the updated weight from the selected HPs (by the controller), and a validation accuracy based on this temporary weight serves as a reward to update the controller. In experiments, we show AutoHAS is efficient and generalizable to different search spaces, baselines and datasets. In particular, AutoHAS can improve the accuracy over popular network architectures, such as ResNet and EfficientNet, on CIFAR-10/100, ImageNet, and four more other datasets.
