Skip to content

Add full unit testing suite and CI ecosystem support - #269

Draft
grey-nearing wants to merge 11 commits into
mainfrom
gsnearing-unit-tests
Draft

Add full unit testing suite and CI ecosystem support#269
grey-nearing wants to merge 11 commits into
mainfrom
gsnearing-unit-tests

Conversation

@grey-nearing

@grey-nearing grey-nearing commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary of Changes

1. Comprehensive Unit Test Suite (204 Tests, 0 Failures)

Adds dedicated unit test coverage across previously untested core modules:

  • evaluation/metrics.py (test/test_metrics.py): Unit tests for all 14 hydrological metrics (NSE, MSE, RMSE, KGE, Alpha-NSE, Beta-NSE, Beta-KGE, Pearson-r, FHV, FMS, FLV, Peak-Timing, Missed-Peaks, Peak-MAPE), edge cases (all-NaN, empty slices, shape mismatches), and metric dispatchers.
  • training/loss.py & regularization.py (test/test_losses.py): Tests for MaskedMSELoss, MaskedRMSELoss, MaskedNSELoss, MaskedCMALLoss, multi-frequency masking, and multi-target loss weighting.
  • utils/cmal_deterministic.py (test/test_cmal_math.py): Mathematical validation for CMAL PDF/CDF distributions, PPF inverse-CDF calculations, and vectorized quantile monotonicity.
  • utils/samplingutils.py (test/test_samplingutils.py): Tests for point prediction sampling (sample_cmal, sample_cmal_deterministic, sample_mcd), zero-threshold normalization, and negative value handling modes.
  • modelzoo/ (test/test_models.py): Tests for FC MLP layers, PositionalEncoding, Regression / CMAL prediction heads, and BaseModel lifecycle.
  • training/ (test/test_trainer.py, test/test_logger.py): Tests for optimizers, loss/regularization initializers, Logger training/validation lifecycles, and artifact image generation.
  • datasetzoo/caravan.py & registry (test/test_caravan.py, test/test_datasetregistry.py, test/test_mfdata_loader.py): Tests for static attribute loading, CSV timeseries parsing, dataset registry dynamic dispatch, and CLI data loader.
  • utils/ (test/test_logging_utils.py, test/test_lstm_utils.py, test/test_memory.py, test/test_plots.py): Tests for memory management, LSTM weight initialization, log filters, and hydrograph plotting.
  • CLI Dispatchers (test/test_cli.py): Argument parsing and mode dispatching for run.py and run_scheduler.py.

2. Bug Fixes & Code Improvements

  • googlehydrology/utils/samplingutils.py: Fixed UnboundLocalError during error message formatting when embedding dropouts are uninitialized.
  • googlehydrology/datasetzoo/caravan.py: Fixed NotImplementedError with Dask .chunk('auto') when attribute tables contain string/object columns by specifying chunk({'basin': -1}).
  • googlehydrology/training/__init__.py: Cleaned up obsolete reference to non-existent TiedFrequencyMSERegularization.
  • test/test_multimet.py: Corrected mock assertion sequence to reflect scaler save-before-scale ordering.
  • test/test_uncertainty.py: Guarded negative sample clipping check for deterministic CMAL quantile outputs.

3. Google Python Style Guide & Readability Compliance

  • Line Length (§3.2): 100% compliant with 80-character maximum line length across all test suites, docstrings, comments, and fixtures.
  • Import Organization (§3.13): Standardized three-block import grouping (Standard Library, Third-Party, First-Party) separated by single blank lines.
  • Unused Imports Cleaned: Removed obsolete imports across all test files.
  • Test State Isolation: Restored root logger handlers in test_logging_utils.py and isolated global FLAGS using absl.testing.flagsaver in test_mfdata_loader.py.

4. CI Ecosystem Modernization & Security Hardening

  • Environment: Added environments/environment_cpu.yml for fast, lightweight CI test execution on Python 3.12.
  • Branch Triggers: Added main branch to CI pull_request and push triggers in .github/workflows/pytest-ci.yml.
  • Security Policy Compliance (zizmor):
    • Added top-level permissions: read-all.
    • Pinned GitHub Actions to immutable full commit SHAs (actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2, conda-incubator/setup-miniconda@d2e6a045a86077fb6cad6f5adf368e9076ddaa8d # v3.1.0).
  • Configuration: Configured pyproject.toml with test markers (unit, integration, slow, gpu) and warning filters.

Verification

  • Test Suite: All 204 unit tests pass in ~2 minutes on CPU.
  • Statement Coverage: Core modules (googlehydrology.evaluation, googlehydrology.training, googlehydrology.utils, googlehydrology.modelzoo) >95% covered.

- Add comprehensive unit tests for all modules across googlehydrology (evaluation metrics, losses, models, dataset registry, Caravan loading, CLI runners, memory, LSTM initialization, and logging utilities).
- Fix UnboundLocalError in samplingutils.py during error message formatting.
- Fix chunking on object-dtype attribute columns in datasetzoo/caravan.py.
- Clean up invalid TiedFrequencyMSERegularization reference in training/__init__.py.
- Fix mock assertion sequence in test_multimet.py to match scaler save-before-scale ordering.
- Fix CMAL deterministic quantile clipping assertion in test_uncertainty.py.
- Add environments/environment_cpu.yml and update GitHub Actions CI workflow to Python 3.12 with PyTorch CPU and coverage reporting.
- Configure pyproject.toml with test markers and warning filters.
…Guide

- Add 'main' branch to pull_request and push triggers in .github/workflows/pytest-ci.yml.
- Standardize import grouping into distinct standard library, third-party, and first-party blocks across test files.
- Enforce 80-character maximum line length across all test suites, docstrings, comments, and fixtures.
- Verify all 204 unit tests pass cleanly.
…te flags, and vectorize math

- Break up long implied embedding assignments and error messages in samplingutils.py.
- Wrap condition and error formatting within 80 characters in test_uncertainty.py.
- Remove unused imports across conftest.py, test_caravan.py, test_logging_utils.py, test_losses.py, test_mfdata_loader.py, test_models.py, and test_samplingutils.py.
- Restore original root logger handlers upon test exit in test_logging_utils.py.
- Use absl flagsaver in test_mfdata_loader.py to prevent global flag state leakage.
- Vectorize CMAL quantile monotonicity assertions in test_cmal_math.py.
- Assert total dispatcher call count in test_cli.py.
- Add 'permissions: read-all' block to pytest-ci.yml.
- Pin actions/checkout to immutable commit hash 11bd71901bbe5b1630ceea73d27597364c9af683 (# v4.2.2).
- Pin conda-incubator/setup-miniconda to immutable commit hash d2e6a045a86077fb6cad6f5adf368e9076ddaa8d (# v3.1.0).
…is absent on Windows and set compile: False in test config
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant