Skip to content

Feature request: a function that would export all datasets as netcdf files and then will re-create the database file for all those datasets but without containing the raw data #7212

@astafan8

Description

@astafan8

Typically when running measurements with qcodes, the users have a database file, and the raw measured data is stored in that database file as well. In addition, many users prefer to have automatic export to NetCDF files enables since that is a much more useful format for sharing data with others. And then as a result users start having duplicate data on their machines: raw data inside the database file, and that same data in the form of the exported NetCDF files.

In this issue, I would like to request a very simply function that given a database file, it would first make sure that all the datasets have been exported to NetCDF files (to a location specified in the qcodes configuration file, or to one provided by the user), and then it will create a new database file that would only include the metadata of these exported datasets and NOT the raw data (which can be done by loading these exported netcdf datasets into a database file, there's a function for it). The function has to try it's best to keep the order of the datasets in the runs table as in the original database file, that is, run_id should ideally remain the same. If along the way, some datasets are not being able to get exported to NetCDF file, than those datasets should be transferred as-is into the new database file.

This functionality should be covered with tests, unit tests and a few integration tests (idelly using pytest.mark.parametrize or hypothesis).

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions