Skip to content

Harden prediction load and multiclass CV against invalid/empty input#391

Merged
gbeane merged 1 commit into
feature/multiclassfrom
fix/multiclass-prediction-cv-robustness
Jun 3, 2026
Merged

Harden prediction load and multiclass CV against invalid/empty input#391
gbeane merged 1 commit into
feature/multiclassfrom
fix/multiclass-prediction-cv-robustness

Conversation

@gbeane
Copy link
Copy Markdown
Collaborator

@gbeane gbeane commented Jun 3, 2026

Addresses two robustness review comments from the feature/multiclassmain PR.

1. prediction_manager — invalid/corrupted prediction records

io.load() raises ValueError when a saved record has an invalid schema (e.g. class_names present but probabilities are 2-D instead of (n_id, n_frames, n_classes)). _load_prediction_record only caught KeyError/FileNotFoundError, so load_multiclass_predictions() would propagate the error instead of falling back to empty outputs.

  • io.load is now wrapped so a ValueError (corrupt/invalid record) returns empty outputs, consistent with the missing-file behavior.
  • The deliberate mode-mismatch guard (multi-class record requested as binary or vice-versa) still raises — that's a logic error, not a corrupt file — and now runs after the load rather than inside the same catch.
  • Fixed test_load_multiclass_predictions_invalid_shape_returns_empty, which previously passed for the wrong reason: it wrote the group as __multiclass__ while the loader looks it up via to_safe_name(...) = multiclass, so it hit a KeyError and never exercised the invalid-shape path. It now writes the correct safe group name and genuinely exercises the ValueError fallback.

2. cross_validation — empty labels_by_behavior

In multi-class mode _prepare_cv_labels called merge_labels(...) unconditionally; merge_labels raises on an empty dict (no labeled frames yet), so CV crashed instead of being skipped like the binary path.

  • _prepare_cv_labels now short-circuits on empty labels_by_behavior, returning an empty label array so the caller finds no valid splits and skips CV gracefully.

Tests

  • New test_multiclass_cv_skips_when_no_labels (CV skipped, no raise).
  • Fixed/strengthened the invalid-shape prediction test.
  • tests/classifier + tests/project (298) pass; ruff clean.

@gbeane gbeane merged commit 6d68929 into feature/multiclass Jun 3, 2026
2 checks passed
@gbeane gbeane deleted the fix/multiclass-prediction-cv-robustness branch June 3, 2026 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant