论文标题
Automlbench：对自动化机器学习框架的全面实验评估
AutoMLBench: A Comprehensive Experimental Evaluation of Automated Machine Learning Frameworks
论文作者
论文摘要
随着对机器学习应用的蓬勃发展的需求，人们已经认识到，知识渊博的数据科学家无法随着我们数字世界中的数据量和应用需求的增长而扩展。为了应对这一需求，已经开发了几个自动化的机器学习（AUTOML）框架，以通过自动化建筑机器学习管道的过程来填补人类专业知识的空白。每个框架都有不同的基于启发式的设计决策。在这项研究中，我们在已建立的AutoMl Benchmark Marksk套件的100个数据集中，对六个流行的AutoMl框架的性能特征进行了全面评估和比较。我们的实验评估考虑了不同方面的比较，包括几项设计决策的性能影响，包括时间预算，搜索空间的规模，元学习和集合构建。我们的研究结果揭示了各种有趣的见解，可以显着指导和影响汽车框架的设计。
With the booming demand for machine learning applications, it has been recognized that the number of knowledgeable data scientists can not scale with the growing data volumes and application needs in our digital world. In response to this demand, several automated machine learning (AutoML) frameworks have been developed to fill the gap of human expertise by automating the process of building machine learning pipelines. Each framework comes with different heuristics-based design decisions. In this study, we present a comprehensive evaluation and comparison of the performance characteristics of six popular AutoML frameworks, namely, AutoWeka, AutoSKlearn, TPOT, Recipe, ATM, and SmartML, across 100 data sets from established AutoML benchmark suites. Our experimental evaluation considers different aspects for its comparison, including the performance impact of several design decisions, including time budget, size of search space, meta-learning, and ensemble construction. The results of our study reveal various interesting insights that can significantly guide and impact the design of AutoML frameworks.
