Feature/multiclass#390
Open
gbeane wants to merge 141 commits into
Open
Conversation
…l classes required to consider a group)
…ted in all splits
…it has all classes above threshold
…boost is not importable
… if the mode from the project settings file doesn't match a known mode then log a warning indicating the default mode is being used
…setting Add classifier mode selection to project settings dialog
Add MultiClassClassifier implementation
…xclusivity in multi-class labeling
Change button label to 'None' in multi-class mode
…timeline widgets - add VideoLabels.build_multiclass_label_array and iter_behavior_labels - add make_behavior_color_map and build_multiclass_color_lut in colors.py - refactor timeline widgets (ManualLabelWidget, TimelineLabelWidget) to hold a dynamic _color_lut and accept pre-normalized LUT-index ndarrays instead of TrackLabels; callers now own the normalization step - add track_labels_to_lut_indices and binary_predictions_to_lut_indices helpers in label_overview_util, exported from stacked_timeline_widget package - central_widget.py normalizes labels and predictions to LUT indices before passing to StackedTimelineWidget, keeping raw arrays available for player widget - delete TimelinePredictionWidget (now identical to TimelineLabelWidget) - add set_color_lut forwarding on LabelOverviewWidget and PredictionOverviewWidget
…ding in timeline overview bar
…ure-extraction docstring
Cleanup pass on feature/multiclass: dedupe and per-mode strategies
…lassifier into feature/multiclass # Conflicts: # src/jabs/feature_extraction/features.py # src/jabs/scripts/classify.py
…or-desync Fix rename_behavior desync in multi-class projects
…abel Label multi-class classifier mode as a preview feature
…ABS-behavior-classifier into fix/multiclass-prediction-type-safety # Conflicts: # src/jabs/classifier/multi_class_classifier.py
…pe-safety Fix/multiclass prediction type safety
…ticlass-save-predictions-shape # Conflicts: # tests/project/test_project.py
…ons-shape Derive prediction probability shape from class_names in save_predictions
Multi-class beta polish: label summary wording + preview docs
Contributor
There was a problem hiding this comment.
Pull request overview
This PR merges the previously reviewed feature/multiclass branch into main, introducing a preview multi-class classifier mode alongside existing binary mode. It updates training/classification pipelines, prediction persistence/schema, timeline visualization, and user/CLI documentation to support multi-class workflows.
Changes:
- Added multi-class mode support end-to-end (settings, training, classification, prediction load/save, UI timeline/overlay rendering).
- Extended prediction and training export schemas to carry multi-class metadata (notably
class_namesand 3-D probability tensors). - Added/expanded test coverage across UI threads, prediction manager I/O, training export/readback, and cross-validation behavior.
Reviewed changes
Copilot reviewed 88 out of 88 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/ui/test_training_thread.py | Verifies TrainingThread dispatches correctly for binary vs multi-class. |
| tests/ui/test_settings_dialog.py | Tests overlap-check thread + classifier mode settings group roundtrip. |
| tests/ui/test_label_count_widget.py | Tests dynamic relabeling of summary row headers. |
| tests/ui/test_colors.py | Tests deterministic multi-class color generation + LUT construction. |
| tests/ui/test_classification_thread.py | Verifies ClassifyThread branching (postprocessing vs none, class_names persistence). |
| tests/ui/test_central_widget_multiclass.py | Tests multiclass prediction decomposition/fallback behavior. |
| tests/ui/test_central_widget_mode.py | Tests CentralWidget per-mode helper dispatch logic. |
| tests/ui/_fakes.py | Adds shared fakes for UI thread tests. |
| tests/ui/init.py | Declares UI tests as a package for shared helpers. |
| tests/project/test_video_labels.py | Adds tests for multi-class label array construction and validation. |
| tests/project/test_prediction_manager.py | Adds multi-class prediction read/write/validation tests. |
| tests/project/test_parallel_workers.py | Tests multiclass labeled-feature collection worker. |
| tests/project/test_export_training_multiclass.py | Tests multiclass training export + reader + CLI branching. |
| tests/feature_extraction/test_identity_features.py | Regression test for window recompute after flattened cache hit. |
| tests/classifier/test_training_report.py | Tests multi-class CV reporting + JSON serialization. |
| tests/classifier/test_cross_validation.py | Tests CV behavior for no splits + multiclass settings reuse. |
| tests/classifier/test_classifier.py | Updates classifier metadata expectations (optional file/hash) + feature_names roundtrip. |
| src/jabs/ui/training_thread.py | Refactors TrainingThread to use per-mode strategies. |
| src/jabs/ui/training_strategy.py | Implements binary vs multi-class training strategy hooks. |
| src/jabs/ui/stacked_timeline_widget/label_overview_widget/timeline_prediction_widget.py | Removes legacy stacked timeline prediction widget implementation. |
| src/jabs/ui/stacked_timeline_widget/label_overview_widget/timeline_label_widget.py | Removes legacy stacked timeline label widget implementation. |
| src/jabs/ui/stacked_timeline_widget/label_overview_widget/prediction_overview_widget.py | Removes legacy prediction overview widget. |
| src/jabs/ui/stacked_timeline_widget/label_overview_widget/predicted_label_widget.py | Removes legacy predicted label widget. |
| src/jabs/ui/stacked_timeline_widget/label_overview_widget/init.py | Removes legacy exports for stacked timeline widgets. |
| src/jabs/ui/stacked_timeline_widget/init.py | Removes legacy stacked timeline package export. |
| src/jabs/ui/settings_dialog/settings_dialog.py | Adds classifier mode group + overlap validation flow on save. |
| src/jabs/ui/settings_dialog/classifier_mode_settings_group.py | Adds UI control/docs for classifier mode selection. |
| src/jabs/ui/player_widget/player_widget.py | Adds API to set a label-color LUT for overlays. |
| src/jabs/ui/player_widget/overlays/label_overlay.py | Uses LUT-based colors when provided (multi-class overlay support). |
| src/jabs/ui/player_widget/frame_with_overlays.py | Stores/updates per-class LUT for overlay rendering. |
| src/jabs/ui/main_window/menu_handlers.py | Updates timeline wiring + export-training branching by classifier mode; adds multiclass layout actions. |
| src/jabs/ui/main_window/menu_builder.py | Adds multiclass layout toggle actions to Timeline menu. |
| src/jabs/ui/main_window/main_window.py | Ensures classifier mode UI/layout updates apply safely on settings change. |
| src/jabs/ui/main_window/central_widget_mode.py | Adds per-mode helper functions for predictions + labeling operations. |
| src/jabs/ui/main_control_widget/main_control_widget.py | Adds classifier mode awareness (repurpose “Not Behavior” as “None”). |
| src/jabs/ui/main_control_widget/label_count_widget.py | Makes label count headers retitleable for multi-class. |
| src/jabs/ui/export_training_thread.py | Exports binary vs multi-class training files based on classifier mode. |
| src/jabs/ui/dialogs/user_guide_dialog.py | Adds multi-class user guide page link. |
| src/jabs/ui/dialogs/progress_dialog.py | Exposes maximum setters/getters used by updated progress flows. |
| src/jabs/ui/colors.py | Adds multi-class palette generation + LUT builder. |
| src/jabs/ui/classify_strategy.py | Adds binary vs multi-class classification strategy hooks. |
| src/jabs/ui/classification_thread.py | Refactors ClassifyThread to use per-mode strategies and persist class_names. |
| src/jabs/ui/behavior_timeline/track_widgets/timeline_util.py | Adds LUT-index converters; improves search-hit rendering precision. |
| src/jabs/ui/behavior_timeline/track_widgets/prediction_track_widget.py | New prediction track widget using probability-aware detail bars. |
| src/jabs/ui/behavior_timeline/track_widgets/per_class_prediction_track_widget.py | New per-class probability track widget. |
| src/jabs/ui/behavior_timeline/track_widgets/label_track_widget.py | Updates label track widget contract to accept LUT-index arrays + collapsible detail bar. |
| src/jabs/ui/behavior_timeline/track_widgets/class_prediction_detail_bar.py | New probability alpha-blended prediction detail bar. |
| src/jabs/ui/behavior_timeline/track_widgets/behavior_probability_detail_bar.py | New heatmap-style per-class probability detail bar. |
| src/jabs/ui/behavior_timeline/track_widgets/init.py | Exposes track widgets package. |
| src/jabs/ui/behavior_timeline/frame_ruler_widget.py | Renames frame ruler widget and updates docs/comments. |
| src/jabs/ui/behavior_timeline/init.py | Exposes BehaviorTimelineWidget + LUT conversion helpers. |
| src/jabs/scripts/cli/cli.py | Makes export-training behavior conditional on classifier mode. |
| src/jabs/resources/docs/user_guide/multi-class.md | Adds multi-class preview user guide (embedded resources). |
| src/jabs/resources/docs/user_guide/classifier-types.md | Clarifies classifier type vs classifier mode; links to multi-class page. |
| src/jabs/project/video_labels.py | Adds multi-class label array builder + behavior-iteration helper. |
| src/jabs/project/settings_manager.py | Adds classifier_mode setting accessor with validation/logging. |
| src/jabs/project/read_training.py | Adds reader for multi-class exported training files. |
| src/jabs/project/prediction_manager.py | Adds multi-class prediction key + class_names support + mode-guarded loading. |
| src/jabs/project/export_training.py | Adds multi-class training export writer. |
| src/jabs/project/init.py | Exports new multi-class training export/load APIs. |
| src/jabs/feature_extraction/features.py | Fixes window feature recompute when per-frame features only available in flattened form. |
| src/jabs/classifier/factories.py | Adds mode-aware classifier factory registry + CatBoost multiclass setup. |
| src/jabs/classifier/cross_validation.py | Adds multi-class CV metrics generation + shared CV helpers. |
| src/jabs/classifier/classifier_utils.py | Adds LOGO split validation helpers + multiclass label merging. |
| src/jabs/classifier/init.py | Re-exports MultiClassClassifier and CV result types. |
| packages/jabs-io/tests/internal/prediction/test_hdf5.py | Tests HDF5 adapter for multiclass prediction + optional classifier metadata. |
| packages/jabs-io/tests/internal/dataclass/test_hdf5.py | Updates dataclass HDF5 roundtrip expectations for new schema. |
| packages/jabs-io/src/jabs/io/internal/prediction/hdf5.py | Persists class_names + optional classifier metadata; robust dataset overwrite logic. |
| packages/jabs-core/tests/test_prediction_types.py | Adds unit tests for multi-class probability tensor validation. |
| packages/jabs-core/src/jabs/core/types/prediction.py | Extends prediction types to support class_names + 3-D probabilities; makes classifier metadata optional. |
| docs/user-guide/multi-class.md | Adds multi-class preview documentation (standalone docs). |
| docs/user-guide/classifier-types.md | Clarifies classifier type vs mode; links multi-class docs. |
| CLAUDE.md | Updates coding standards (spelling + avoid EN DASH). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…-robustness Harden prediction load and multiclass CV against invalid/empty input
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
merge of the feature/multiclass branch
This PR merges the feature/multiclass branch into main. While this consists of thousands of lines of change, all the changes were already reviewed over a series of a dozen or more pull requests onto the feature branch.
LLM Summary:
This pull request introduces support for multi-class classification as a preview feature in JABS, updates the documentation to explain this new mode, and refines the prediction data structures to accommodate multi-class predictions.
Multi-class classification support and documentation:
multi-class.md) describing the multi-class classification preview feature, its usage, limitations, and how it differs from binary classification.Prediction data structure updates:
BehaviorPredictionclass to support multi-class predictions by adding an optionalclass_namesfield, updating the expected shape for probability arrays, and adjusting validation logic to handle both binary and multi-class cases. [1] [2] [3] [4]ClassifierMetadatato allowclassifier_fileandclassifier_hashto be optional, reflecting cases where this information may not be available.Developer tooling:
preview_multiclass_timeline.py) that generates fake multi-class and binary prediction data, and provides an interactive UI for visualizing stacked timelines under different classifier modes and identity views. This aids in development and testing of the multi-class timeline widget.Documentation and standards:
CLAUDE.md, including enforcing American English spelling, and added a note to avoid ambiguous EN DASH characters in docstrings to prevent Ruff linter errors.