Klaus Greff
Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA)
Università della Svizzera italiana (USI)
Scuola universitaria professionale della Svizzera italiana (SUPSI)

Aaron Klein
University of Freiburg

Martin Chovanec
Czech Technical University in Prague

Frank Hutter
University of Freiburg

Jürgen Schmidhuber
Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA)
Università della Svizzera italiana (USI)
Scuola universitaria professionale della Svizzera italiana (SUPSI)

Abstract

We present a toolchain for computational research consisting of Sacred and two supporting tools. Sacred is an open source Python framework which aims to provide basic infrastructure for running computational experiments independent of the methods and libraries used. Instead, it focuses on solving universal everyday problems, such as managing configurations, reproducing results, and bookkeeping. Moreover, it provides an extensible basis for other tools, two of which we present here: Labwatch helps with tuning hyperparameters, and Sacredboard provides a web-dashboard for organizing and analyzing runs and results.

Keywords

reproducible research, Python, machine learning, database, hyperparameter optimization

DOI

10.25080/shinma-7f4c6e7-008

Bibtex entry

Full text PDF

Copyright © 2017 Klaus Greff et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.