-
Notifications
You must be signed in to change notification settings - Fork 45
Open
Labels
KerchunkRelating to the kerchunk library / specification itselfRelating to the kerchunk library / specification itselfenhancementNew feature or requestNew feature or requestzarr-pythonRelevant to zarr-python upstreamRelevant to zarr-python upstream
Milestone
Description
@rabernat made the interesting point to me that uncompressed files (e.g. netCDF3 files) have no specific chunking, as you can start reading bytes from any point in the file immediately, with no minimum unit of decompression to do first.
I'm not totally sure what this implies for VirtualiZarr generating references from netCDF3 files, as it's still meaningful to talk about one chunk per file concatenated together in a manifest.
Perhaps this is something that should ultimately be taken account of in zarr readers: that reading bytes from an uncompressed array does not require loading an entire chunk into memory first.
Metadata
Metadata
Assignees
Labels
KerchunkRelating to the kerchunk library / specification itselfRelating to the kerchunk library / specification itselfenhancementNew feature or requestNew feature or requestzarr-pythonRelevant to zarr-python upstreamRelevant to zarr-python upstream