Dorota Jarecka
Massachusetts Institute of Technology, Cambridge, MA, USA

Mathias Goncalves
Stanford University, Stanford, CA, USA
Massachusetts Institute of Technology, Cambridge, MA, USA

Christopher J. Markiewicz
Stanford University, Stanford, CA, USA

Oscar Esteban
Stanford University, Stanford, CA, USA

Nicole Lo
Massachusetts Institute of Technology, Cambridge, MA, USA

Jakub Kaczmarzyk
Stony Brook University School of Medicine, Stony Brook, NY, USA
Massachusetts Institute of Technology, Cambridge, MA, USA

Satrajit Ghosh
Massachusetts Institute of Technology, Cambridge, MA, USA

Abstract

This paper presents a new lightweight dataflow engine written in Python: Pydra. Pydra is developed as an open-source project in the neuroimaging community, but it is designed as a general-purpose dataflow engine to support any scientific domain. The paper describes the architecture of the software, as well as several useful features, that make Pydra a customizable and powerful dataflow engine. Two examples are presented to demonstrate the syntax and properties of the package.

Keywords

dataflow engine, scientific workflows, reproducibility

DOI

10.25080/Majora-342d178e-012

Bibtex entry

Full text PDF

Copyright © 2020 Dorota Jarecka et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.