Shin-Rong Tsai
National Taiwan University, Department of Physics
University of Illinois at Urbana-Champaign, School of Information Sciences

Hsi-Yu Schive
National Taiwan University, Department of Physics
National Taiwan University, Institute of Astrophysics
National Taiwan University, Center for Theoretical Physics
National Center for Theoretical Sciences, Physics Division

Matthew J. Turk
University of Illinois at Urbana-Champaign, School of Information Sciences

Abstract

In the era of exascale computing, storage and analysis of large scale data have become more important and difficult. We present libyt, an open source C++ library, that allows researchers to analyze and visualize data using yt or other Python packages in parallel during simulation runtime. We describe the code method for organizing adaptive mesh refinement grid data structure and simulation data, handling data transition between Python and simulation with minimal memory overhead, and conducting analysis with no additional time penalty using Python C API and NumPy C API. We demonstrate how it solves the problem in astrophysical simulations and increases disk usage efficiency. Finally, we conclude it with discussions about libyt.

Keywords

astronomy data analysis, astronomy data visualization, in situ analysis, open source software

DOI

10.25080/gerudo-f2bc6f59-011

Bibtex entry

Full text PDF

Copyright © 2023 Shin-Rong Tsai et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.