Skip to content

Fix order of available projects in test_recipe#3023

Open
valeriupredoi wants to merge 7 commits intomainfrom
fix_test_recipe
Open

Fix order of available projects in test_recipe#3023
valeriupredoi wants to merge 7 commits intomainfrom
fix_test_recipe

Conversation

@valeriupredoi
Copy link
Contributor

@valeriupredoi valeriupredoi commented Mar 20, 2026

Description

Order got mangled for some reason - you know why @bouweandela ?


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.


To help with the number pull requests:

@bouweandela
Copy link
Member

Based on which tests are failing here: https://app.circleci.com/pipelines/github/ESMValGroup/ESMValCore?branch=main, it looks like something changed in a dependency. That test is a pain anyway, I've had to update it several times already. Maybe you could change it so it doesn't depend on the order of the projects and doesn't fail again when we add a another project?

@valeriupredoi
Copy link
Contributor Author

Based on which tests are failing here: https://app.circleci.com/pipelines/github/ESMValGroup/ESMValCore?branch=main, it looks like something changed in a dependency. That test is a pain anyway, I've had to update it several times already. Maybe you could change it so it doesn't depend on the order of the projects and doesn't fail again when we add a another project?

ah! Good call - I was suspecting it's a dependency, since we've not changed anything since yesterday 😁 Yes, let me have a closer look at it then

@valeriupredoi
Copy link
Contributor Author

@bouweandela ut's blithering Dask!

  - dask         2026.3.0  pyhc364b38_0  conda-forge     Cached
  + dask         2026.1.2  pyhcf101f3_1  conda-forge     Cached
  - dask-core    2026.3.0  pyhc364b38_0  conda-forge     Cached
  + dask-core    2026.1.2  pyhcf101f3_0  conda-forge     Cached
  - distributed  2026.3.0  pyhc364b38_0  conda-forge     Cached
  + distributed  2026.1.2  pyhcf101f3_1  conda-forge     Cached

aghh!

@valeriupredoi
Copy link
Contributor Author

OK @bouweandela the test is now dask-agnostic 🍺

@codecov
Copy link

codecov bot commented Mar 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.70%. Comparing base (69f2019) to head (5376610).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3023   +/-   ##
=======================================
  Coverage   95.70%   95.70%           
=======================================
  Files         267      267           
  Lines       15768    15768           
=======================================
  Hits        15090    15090           
  Misses        678      678           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Member

@bouweandela bouweandela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks V! Some suggestions to make this more future-proof

Comment on lines +3891 to +3893
"The following tables are available: CMIP3, CMIP5, CMIP6, CMIP7, "
"CORDEX, obs4MIPs, ana4MIPs, OBS, OBS6, native6, ACCESS, CESM, EMAC, "
'ICON, IPSLCM."'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be really nice to make this independent of which projects have been configured exactly, because now this test needs to be updated every time you add support for a new project. How about something like?

Suggested change
"The following tables are available: CMIP3, CMIP5, CMIP6, CMIP7, "
"CORDEX, obs4MIPs, ana4MIPs, OBS, OBS6, native6, ACCESS, CESM, EMAC, "
'ICON, IPSLCM."'
"The following tables are available: .*."'

followed by

assert re.match(msg, exc.value.failed_tasks[0].message)

and as an additional sanity check, test that one or two expected projects are in the message:

assert "CMIP7" in exc.value.failed_tasks[0].message

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a different test approach, bud - my idea was this test to be restrictive ie when a project is no longer supported, or another becomes supported, it should fail and one of us should edit that list. What you are proposing is to make the test rather loose - if you think that's better then I'll pop in the suggestions, but I like it restrictive TBF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants