| Title: |
pyfive: A pure-Python HDF5 reader |
| Authors: |
Lawrence, Bryan N.; Cimadevilla, Ezequiel; Nolf, Wout De; Hassell, David; Helmus, Jonathan; Hodel, Benjamin; Maranville, Brian; Mühlbauer, Kai; Predoi, Valeriu |
| Publication Year: |
2026 |
| Collection: |
CentAUR: Central Archive at the University of Reading |
| Description: |
pyfive is an open-source and thread-safe pure Python package for reading data stored in HDF5. While it is not a complete implementation of all the specifications and capabilities of HDF5, it includes all the core functionality necessary to read gridded datasets, whether stored contiguously or with chunks (with or without standard compression options). All data access is fully lazy as the data is only read from storage when the numpy data arrays are manipulated. Originally developed some years ago, the package has recently been expanded to support lazy data access, and to add missing features necessary for handling all the HDF5-based environmental data known to the authors. It is now a realistic option for production data access in environmental science and more broadly across other domains. The API is based on that of h5py (https://github.com/h5py/h5py, a Python shimmy over the HDF5 C-library which itself is not thread-safe), with some API extensions to help optimise remote access. With these extensions, coupled with thread safety, many of the limitations precluding the efficient use of HDF5 (and netCDF4) on cloud storage have been removed. |
| Document Type: |
article in journal/newspaper |
| File Description: |
text |
| Language: |
English |
| ISSN: |
2475-9066 |
| Relation: |
https://centaur.reading.ac.uk/128494/1/10.21105.joss.09688.pdf; Lawrence, B. N. orcid:0000-0001-9262-7860 , Cimadevilla, E. orcid:0000-0002-8437-2068 , Nolf, W. D. orcid:0000-0003-2258-9402 , Hassell, D. orcid:0000-0002-5312-4950 , Helmus, J., Hodel, B., Maranville, B. orcid:0000-0002-6105-8789 , Mühlbauer, K. orcid:0000-0001-6599-1034 and Predoi, V. orcid:0000-0002-9729-6578 (2026) pyfive: A pure-Python HDF5 reader. Journal of Open Source Software, 11 (118). 9688. ISSN 2475-9066 doi:10.21105/joss.09688 |
| Availability: |
https://centaur.reading.ac.uk/128494/; https://centaur.reading.ac.uk/128494/1/10.21105.joss.09688.pdf |
| Rights: |
cc_by_4 |
| Accession Number: |
edsbas.B1190F4D |
| Database: |
BASE |