James Bergstra
University of Waterloo

Dan Yamins
Massachusetts Institute of Technology

David D. Cox
Harvard University

Abstract

Sequential model-based optimization (also known as Bayesian optimization) is one of the most efficient methods (per function evaluation) of function minimization. This efficiency makes it appropriate for optimizing the hyperparameters of machine learning algorithms that are slow to train. The Hyperopt library provides algorithms and parallelization infrastructure for performing hyperparameter optimization (model selection) in Python. This paper presents an introductory tutorial on the usage of the Hyperopt library, including the description of search spaces, minimization (in serial and parallel), and the analysis of the results collected in the course of minimization. The paper closes with some discussion of ongoing and future work.

Keywords

Bayesian optimization, hyperparameter optimization, model selection

DOI

10.25080/Majora-8b375195-003

Bibtex entry

Full text PDF

Copyright © 2013 James Bergstra et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.