Brent Komer
Centre for Theoretical Neuroscience, University of Waterloo

James Bergstra
Centre for Theoretical Neuroscience, University of Waterloo

Chris Eliasmith
Centre for Theoretical Neuroscience, University of Waterloo

Abstract

Hyperopt-sklearn is a new software project that provides automatic algorithm configuration of the Scikit-learn machine learning library. Following Auto-Weka, we take the view that the choice of classifier and even the choice of preprocessing module can be taken together to represent a single large hyperparameter optimization problem. We use Hyperopt to define a search space that encompasses many standard components (e.g. SVM, RF, KNN, PCA, TFIDF) and common patterns of composing them together. We demonstrate, using search algorithms in Hyperopt and standard benchmarking data sets (MNIST, 20-Newsgroups, Convex Shapes), that searching this space is practical and effective. In particular, we improve on best-known scores for the model space for both MNIST and Convex Shapes.

Keywords

bayesian optimization, model selection, hyperparameter optimization, scikit-learn

DOI

10.25080/Majora-14bd3278-006

Bibtex entry

Full text PDF

Copyright © 2014 Brent Komer et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.