Add full unit testing suite and CI ecosystem support - #269
Draft
grey-nearing wants to merge 11 commits into
Draft
Conversation
- Add comprehensive unit tests for all modules across googlehydrology (evaluation metrics, losses, models, dataset registry, Caravan loading, CLI runners, memory, LSTM initialization, and logging utilities). - Fix UnboundLocalError in samplingutils.py during error message formatting. - Fix chunking on object-dtype attribute columns in datasetzoo/caravan.py. - Clean up invalid TiedFrequencyMSERegularization reference in training/__init__.py. - Fix mock assertion sequence in test_multimet.py to match scaler save-before-scale ordering. - Fix CMAL deterministic quantile clipping assertion in test_uncertainty.py. - Add environments/environment_cpu.yml and update GitHub Actions CI workflow to Python 3.12 with PyTorch CPU and coverage reporting. - Configure pyproject.toml with test markers and warning filters.
…Guide - Add 'main' branch to pull_request and push triggers in .github/workflows/pytest-ci.yml. - Standardize import grouping into distinct standard library, third-party, and first-party blocks across test files. - Enforce 80-character maximum line length across all test suites, docstrings, comments, and fixtures. - Verify all 204 unit tests pass cleanly.
…te flags, and vectorize math - Break up long implied embedding assignments and error messages in samplingutils.py. - Wrap condition and error formatting within 80 characters in test_uncertainty.py. - Remove unused imports across conftest.py, test_caravan.py, test_logging_utils.py, test_losses.py, test_mfdata_loader.py, test_models.py, and test_samplingutils.py. - Restore original root logger handlers upon test exit in test_logging_utils.py. - Use absl flagsaver in test_mfdata_loader.py to prevent global flag state leakage. - Vectorize CMAL quantile monotonicity assertions in test_cmal_math.py. - Assert total dispatcher call count in test_cli.py.
- Add 'permissions: read-all' block to pytest-ci.yml. - Pin actions/checkout to immutable commit hash 11bd71901bbe5b1630ceea73d27597364c9af683 (# v4.2.2). - Pin conda-incubator/setup-miniconda to immutable commit hash d2e6a045a86077fb6cad6f5adf368e9076ddaa8d (# v3.1.0).
…ing and path resolution
… clear cached dataset file handles
… datasets in unit tests
…failure on Windows
…is absent on Windows and set compile: False in test config
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
1. Comprehensive Unit Test Suite (204 Tests, 0 Failures)
Adds dedicated unit test coverage across previously untested core modules:
evaluation/metrics.py(test/test_metrics.py): Unit tests for all 14 hydrological metrics (NSE,MSE,RMSE,KGE,Alpha-NSE,Beta-NSE,Beta-KGE,Pearson-r,FHV,FMS,FLV,Peak-Timing,Missed-Peaks,Peak-MAPE), edge cases (all-NaN, empty slices, shape mismatches), and metric dispatchers.training/loss.py®ularization.py(test/test_losses.py): Tests forMaskedMSELoss,MaskedRMSELoss,MaskedNSELoss,MaskedCMALLoss, multi-frequency masking, and multi-target loss weighting.utils/cmal_deterministic.py(test/test_cmal_math.py): Mathematical validation for CMAL PDF/CDF distributions, PPF inverse-CDF calculations, and vectorized quantile monotonicity.utils/samplingutils.py(test/test_samplingutils.py): Tests for point prediction sampling (sample_cmal,sample_cmal_deterministic,sample_mcd), zero-threshold normalization, and negative value handling modes.modelzoo/(test/test_models.py): Tests forFCMLP layers,PositionalEncoding,Regression/CMALprediction heads, andBaseModellifecycle.training/(test/test_trainer.py,test/test_logger.py): Tests for optimizers, loss/regularization initializers,Loggertraining/validation lifecycles, and artifact image generation.datasetzoo/caravan.py& registry (test/test_caravan.py,test/test_datasetregistry.py,test/test_mfdata_loader.py): Tests for static attribute loading, CSV timeseries parsing, dataset registry dynamic dispatch, and CLI data loader.utils/(test/test_logging_utils.py,test/test_lstm_utils.py,test/test_memory.py,test/test_plots.py): Tests for memory management, LSTM weight initialization, log filters, and hydrograph plotting.test/test_cli.py): Argument parsing and mode dispatching forrun.pyandrun_scheduler.py.2. Bug Fixes & Code Improvements
googlehydrology/utils/samplingutils.py: FixedUnboundLocalErrorduring error message formatting when embedding dropouts are uninitialized.googlehydrology/datasetzoo/caravan.py: FixedNotImplementedErrorwith Dask.chunk('auto')when attribute tables contain string/object columns by specifyingchunk({'basin': -1}).googlehydrology/training/__init__.py: Cleaned up obsolete reference to non-existentTiedFrequencyMSERegularization.test/test_multimet.py: Corrected mock assertion sequence to reflect scaler save-before-scale ordering.test/test_uncertainty.py: Guarded negative sample clipping check for deterministic CMAL quantile outputs.3. Google Python Style Guide & Readability Compliance
test_logging_utils.pyand isolated globalFLAGSusingabsl.testing.flagsaverintest_mfdata_loader.py.4. CI Ecosystem Modernization & Security Hardening
environments/environment_cpu.ymlfor fast, lightweight CI test execution on Python 3.12.mainbranch to CIpull_requestandpushtriggers in.github/workflows/pytest-ci.yml.zizmor):permissions: read-all.actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2,conda-incubator/setup-miniconda@d2e6a045a86077fb6cad6f5adf368e9076ddaa8d # v3.1.0).pyproject.tomlwith test markers (unit,integration,slow,gpu) and warning filters.Verification
googlehydrology.evaluation,googlehydrology.training,googlehydrology.utils,googlehydrology.modelzoo) >95% covered.