Add test for anemoi-datasets loader using CERRA test dataset hosted on EWC#1
Add test for anemoi-datasets loader using CERRA test dataset hosted on EWC#1
Conversation
|
Currently the test fails, I need to find out if that is because either:
|
|
Hi @leifdenby from my own tests with using the validator with the mxalign loaders I have noticed that some of the attributes (long_name, units, ..) are not currently not set correctly or not provided at all by the loaders. This is probably causing the validation to fail |
|
I've (temporarily) integrated Given that the loaded dataset now passes both from the perspective of |

I have added a test that applies the
anemoi-datasetsloader and then usesmlwp-data-specsvalidator to validate the loadedxr.Dataset.To implement this I also uploaded
cerra-rr-an-oper-0001-mars-5p5km-2017-2017-6h-v3-testing.zarrfrom the anemoi dataset registry to the EWC (European Weather Cloud) object store at https://object-store.os-api.cci2.ecmwf.int/mlwp-sample-datasets/anemoi-datasets/cerra-rr-an-oper-0001-mars-5p5km-2017-2017-6h-v3-testing.zarr/The point of this PR is to adjust
mlwp-data-specsand theanemoi-datasetsloader here inmlwp-data-loadersuntil the validation of the loadedxr.Datasetpasses.Currently this installs
mlwp-data-specsdirectly frommain(i.e. whatever commit is onHEAD) of https://github.com/mlwp-tools/mlwp-data-specs. Eventually we should tag and publish a release ofmlwp-data-specsonce this data-loader (and one foranemoi-inferencedatasets?`) pass the validation.@mpvginde your thoughts would be much appreciated :)