Skip to content

Refactor Derecho Work into virtualization template #211

@jbusecke

Description

@jbusecke

Based on #196 and the discussion with the story team just now I propose some follow up actions:

  • Refactor test scripts into templates (notebook+scripts)
    • General documentation -> Virtualizarr (more general explanation of how users can virtualize datasets with virtualizarr, obstore, lithops, and icechunk, with NASA data as example?
    • Earthdata/NASA specific version to NASA-IMPACT (docs?) that potentially uses earthaccess convenience wrappers (earthaccess.open_virtual_mfdataset and the proposed earthaccess.open_icechunk ) and would work towards "virtualization on demand" (e.g. users creating virtual stores), but could later be adopted for more upstream virtual store creation.
  • Test notebook agains small instances. I have only run them on a fairly large machine so far. I want to at least ensure that things run smoothly on a small 8GB RAM machine (or 16GB see Add derecho story to support VEDA Stories veda-docs#217 (comment)).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions