The Advanced Scientific Data Format (ASDF): An Update
Perry Greenfield
Edward Slavich
William Jamieson
Nadia Dencheva
We report on progress in developing and extending the new (ASDF) format
we have developed for the data from the James Webb and Nancy Grace Roman
Space Telescopes since we reported on it at a previous Scipy. While the
format was developed as a replacement for the long-standard FITS format
used in astronomy, it is quite generic and not restricted to use with
astronomical data. We will briefly review the format, and extensions and
changes made to the standard itself, as well as to the reference Python
implementation we
have developed to support it. The standard itself has been clarified in
a number of respects.
Recent improvements to the Python implementation include an improved
framework for conversion between complex Python objects and ASDF,
better control of the configuration of
extensions supported and versioning of extensions, tools for display and
searching of the structured metadata, better developer documentation, tutorials,
and a more maintainable and flexible schema system. This has
included a reorganization of the components to make the standard free
from astronomical assumptions. A important motivator for the format was
the ability to support serializing functional transforms in multiple dimensions as well as expressions built out of such transforms, which
has now been implemented. More generalized compression schemes are now
enabled. We are currently working on adding chunking support and will
discuss our plan for further enhancements.
data formats, standards, world coordinate systems, yaml
DOI10.25080/majora-212e5952-000