Sgh ensemble generator template#942
Open
alexolinhager wants to merge 15 commits intoMPAS-Dev:mainfrom
Open
Conversation
Generalize ensemble generator to support multiple model configurations used for different studies. * Introduced a new configuration module to handle model configurations for ensemble generation. * Updated `BranchRun` and `EnsembleMember` classes to accept a `resource_module` parameter for dynamic configuration loading. * Created default configuration files for branch and spinup ensembles, including necessary namelists and stream definitions. * Modified the main ensemble generator configuration to streamline the setup process and improve clarity. * Enhanced error handling for missing configuration sections and options. * Updated the `SpinupEnsemble` class to utilize the new configuration methods for improved modularity and maintainability.
Variations on the word configuration are already too widespread so this change should reduce confusion.
The primary new functionality is the ability to support any namelist option as a parameter rather than only pre-defined parameters. The refactor also simplifies how parameter values are specified and puts all parameters in a dedicated cfg section. More details on the format are included in the updated docs.
Remove unnecessary extra section level, Separate spinup_ensemble options from options general to the whole test group.
This function was flagged as too complex, so Copilot helped me break it into smaller functions.
Update ensemble generator parsing and docs based on outdated but still relevant Copilot review feedback from PR MPAS-Dev#940. - Sanitize multiline option parsing in spinup_ensemble._split_entries to remove continuation backslashes before tokenization. - Use importlib.resources.as_file() when handling optional albany_input.yaml in ensemble_member setup. - Clarify in developer docs that albany_input.yaml is copied only when present for Albany-based configurations. - Fix users-guide cfg block indentation and multiline .option_name examples to match ConfigParser behavior.
Still need to set up branch ensemble This commit also makes basal melt and TF input files optional to be more compatible with SGH configurations.
Introduces new test cases within the ensemble generator: sgh_analysis - Tests for steady state and validates runs against specularity content. Creates json files with steady-state/validation metric for each run sgh_results - Identifies runs that have not yet reached steady state and creates a new enemble command to restart these runs. Functionality still needs testing. Eventually both of these test cases could be moved within the ensemble manager to perform automatically.
alexolinhager
added
in progress
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
api.rst) has any new or modified class, method and/or functions listedE3SM-Projectsubmodule has been updated with relevant E3SM changesMALI-Devsubmodule has been updated with relevant MALI changesTestingin this PR) any testing that was used to verify the changes