Send image embedding-plot selection as geometry

[ikondrat]

What has changed and why?

Image embedding-plot lasso/rectangle selections are now stored and sent to the backend as region geometry (embedding_region) instead of a resolved list of sample_ids. The backend resolves the polygon to sample ids server-side, so the request body stays small and constant-size regardless of how many items are selected.

Key pieces:

• New shared plumbing — useEmbeddingPlotSelection (a per-collection selection-count store: setPlotSelectionCount / clearPlotSelectionCount / getPlotSelectionCount) and useRegionFilterVisibility, which drives the sidebar chip's count/visibility from the plot-propagated count rather than a resolved id list. Clearing the filter removes the region entirely (geometry has no separate "hidden"/restore state).
• useImageFilters — adds updateEmbeddingRegion(...) to set/clear only the region in the existing filter params, and threads embedding_region into the derived imageFilter sent to the API.
• Request/type wiring — embedding_region is added to NormalModeFilters, buildRequestBody, and the external-filter sync (syncFilterParams) so the region is excluded from base-param comparison and carried forward across collections alongside sample_ids.
• useEmbeddingFilterForImages — switched from the sample_ids-based useFilterVisibility to the new useRegionFilterVisibility.

How has it been tested?

Unit tests updated/added:

• useEmbeddingPlotSelection.test.ts — per-collection count isolation, defaults, set/clear.
• useEmbeddingFilter.test.ts — chip visibility/count now driven by propagated plot-selection count; setVisibility(false) / clearFilter clear the region and call setRangeSelection.
• useImagesInfinite.test.ts — embedding_region geometry is propagated to sample_filter, and omitted when no region is selected.
• syncFilterParams.test.ts — embedding_region treated as an external filter and carried forward across collections.

Did you update CHANGELOG.md?

• Yes
• Not needed (internal change)

Summary by CodeRabbit

• New Features

  • Lasso and region selections now work consistently across images and annotations, with selection state preserved as highlighted regions.
  • Selection counts are now tracked per collection, improving visibility of active filters.

• Bug Fixes

  • Fixed selection handling so clearing or switching collections resets the correct region state.
  • Improved filter syncing so region-based selections remain accurate when navigating or reloading.
  • Select-all behavior now respects annotation regions instead of only item lists.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 499b2850-65f0-419e-8ef1-bdea20bddca6

📥 Commits

Reviewing files that changed from the base of the PR and between dd81371 and ff1d2f3.

📒 Files selected for processing (22)

• lightly_studio_view/src/lib/components/AnnotationsGrid/AnnotationsGrid.svelte
• lightly_studio_view/src/lib/components/Images/syncFilterParams.test.ts
• lightly_studio_view/src/lib/components/Images/syncFilterParams.ts
• lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.svelte
• lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.test.ts
• lightly_studio_view/src/lib/components/PlotPanel/usePlotData/usePlotData.test.ts
• lightly_studio_view/src/lib/components/PlotPanel/usePlotData/usePlotData.ts
• lightly_studio_view/src/lib/hooks/useAnnotationsInfinite/createAnnotationsInfiniteOptions.test.ts
• lightly_studio_view/src/lib/hooks/useAnnotationsInfinite/createAnnotationsInfiniteOptions.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilter.test.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForAnnotations.test.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForAnnotations.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForImages.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingPlotSelection.test.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingPlotSelection.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useRegionFilterVisibility.ts
• lightly_studio_view/src/lib/hooks/useImageFilters/useImageFilters.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/buildRequestBody.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/types.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/useImagesInfinite.test.ts
• lightly_studio_view/src/lib/hooks/useSelectAll/useSelectAll.ts
• lightly_studio_view/src/routes/datasets/[dataset_id]/[collection_type]/[collection_id]/+layout.svelte

✅ Files skipped from review due to trivial changes (1)

• lightly_studio_view/src/lib/hooks/useImagesInfinite/buildRequestBody.ts

🚧 Files skipped from review as they are similar to previous changes (21)

• lightly_studio_view/src/routes/datasets/[dataset_id]/[collection_type]/[collection_id]/+layout.svelte
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingPlotSelection.test.ts
• lightly_studio_view/src/lib/hooks/useAnnotationsInfinite/createAnnotationsInfiniteOptions.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/useImagesInfinite.test.ts
• lightly_studio_view/src/lib/components/Images/syncFilterParams.ts
• lightly_studio_view/src/lib/components/Images/syncFilterParams.test.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/types.ts
• lightly_studio_view/src/lib/components/AnnotationsGrid/AnnotationsGrid.svelte
• lightly_studio_view/src/lib/hooks/useAnnotationsInfinite/createAnnotationsInfiniteOptions.test.ts
• lightly_studio_view/src/lib/components/PlotPanel/usePlotData/usePlotData.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useRegionFilterVisibility.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingPlotSelection.ts
• lightly_studio_view/src/lib/components/PlotPanel/usePlotData/usePlotData.test.ts
• lightly_studio_view/src/lib/hooks/useImageFilters/useImageFilters.ts
• lightly_studio_view/src/lib/hooks/useSelectAll/useSelectAll.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilter.test.ts
• lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.svelte
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForImages.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForAnnotations.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForAnnotations.test.ts
• lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.test.ts

---

📝 Walkthrough

Walkthrough

Frontend embedding-plot selection is reworked to persist and consume lasso/rectangle region geometry (embedding_region) instead of resolved sample-id lists, for both images and annotations. New selection-count and region-visibility hooks drive chip visibility, and PlotPanel, AnnotationsGrid, filter hooks, query builders, and layout cleanup are updated accordingly.

Changes

Embedding region filter feature (frontend)

Layer / File(s)	Summary
Plot selection count store and region visibility hook useEmbeddingFilter/useEmbeddingPlotSelection.ts, useEmbeddingFilter/useEmbeddingPlotSelection.test.ts, useEmbeddingFilter/useRegionFilterVisibility.ts	New per-collection selection-count store (set/get/clear) and a useRegionFilterVisibility hook deriving visibility from that count and clearing region/backend selection.
Annotation and image filter hooks switched to region state useEmbeddingFilterForAnnotations.ts, useEmbeddingFilterForAnnotations.test.ts, useEmbeddingFilterForImages.ts, useEmbeddingFilter.test.ts	Annotation selection state moves from sample-id arrays to annotationPlotRegion/saveRegion; clearAnnotationPlotSelection now requires collectionId; both image and annotation filters now use useRegionFilterVisibility.
Image query params and filter merge wired to embedding_region useImageFilters.ts, useImagesInfinite/types.ts, useImagesInfinite/buildRequestBody.ts, useImagesInfinite/useImagesInfinite.test.ts, Images/syncFilterParams.ts, Images/syncFilterParams.test.ts	updateEmbeddingRegion added to useImageFilters; NormalModeFilters picks up embedding_region; syncFilterParams excludes/merges embedding_region like other external filters.
Annotations infinite query wired to embedding_region createAnnotationsInfiniteOptions.ts, createAnnotationsInfiniteOptions.test.ts	embedding_region added to the queryKey and request body sent for annotations.
PlotPanel region-based selection commit/clear flow PlotPanel/PlotPanel.svelte, PlotPanel/PlotPanel.test.ts, usePlotData/usePlotData.ts, usePlotData/usePlotData.test.ts	PlotPanel commits/clears region geometry for images/annotations via new saveRegion/commitRegion/clearRegion helpers, derives committedHighlightRegion, and passes highlightRegion into usePlotData for category highlighting.
AnnotationsGrid, select-all, and layout cleanup using region AnnotationsGrid.svelte, useSelectAll.ts, [collection_id]/+layout.svelte	AnnotationsGrid queries and loader identifiers use embedding_region instead of sample ids; useSelectAll stamps annotation snapshots with embedding_region; layout clears annotation plot selection scoped by collection.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

• lightly-ai/lightly-studio#1585: Backend addition of SampleFilter.embedding_region and server-side resolution directly matches this PR's frontend switch to sending region geometry.
• lightly-ai/lightly-studio#1520: Touches the same annotations embedding-plot flow (AnnotationsGrid/PlotPanel selection wiring) that this PR converts from sample_ids to region geometry.
• lightly-ai/lightly-studio#829: Modifies the same PlotPanel embeddings query filter construction (sample_ids handling) that this PR reworks around embedding_region.

Suggested reviewers: LeonardoRosaa

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: sending image embedding-plot selection as geometry.
Description check	✅ Passed	The description follows the required template and covers the change, testing, and changelog status.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch kondrat/lig-9903-4-images-frontend

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a37412f2d4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ikondrat]

/review

[ikondrat]

/review

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

lightly_studio_view/src/lib/hooks/useSelectAll/useSelectAll.ts (1)

84-97: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Keep filter_type out of the raw annotation fetch filter.

Line 92 stamps filter_type into filter, and Line 97 then passes that same object to fetchSampleIdsForAnnotations. The function-level comment says only snapshotFilter should receive the discriminant, so keep the raw fetch filter to annotation predicates only.

Suggested fix

-                const filter =
+                const filter =
                     plotRegion !== null
                         ? {
                               ...get(annotationFilter),
-                              filter_type: GRID_FILTER_TYPE.annotations,
                               embedding_region: plotRegion
                           }
                         : get(annotationFilter);

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lightly_studio_view/src/lib/hooks/useSelectAll/useSelectAll.ts` around lines
84 - 97, The raw annotation fetch filter in useSelectAll should not be stamped
with GRID_FILTER_TYPE.annotations; keep filter as only the annotation predicates
and reserve the discriminant for snapshotFilter. Update the logic in
useSelectAll so fetchIds still calls fetchSampleIdsForAnnotations with the
unmodified annotation filter plus embedding_region, while snapshotFilter is the
only object that adds filter_type. Locate the change around
get(annotationFilter), plotRegion, and the returned snapshotFilter/fetchIds
pair.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lightly_studio_view/src/lib/components/Images/syncFilterParams.ts`:
- Around line 18-49: `mergeExternalFilters` is carrying `embedding_region`
across a collection switch without updating the dependent plot-selection count,
so the active filter can survive while `useRegionFilterVisibility` hides its
chip. Update the merge logic in `syncFilterParams.ts` so when
`currentParams.filters.embedding_region` is preserved into `baseParams`, the
corresponding selection count is also seeded or recomputed for the new
collection, keeping `getPlotSelectionCount()` non-zero whenever that region
filter is still active.

---

Outside diff comments:
In `@lightly_studio_view/src/lib/hooks/useSelectAll/useSelectAll.ts`:
- Around line 84-97: The raw annotation fetch filter in useSelectAll should not
be stamped with GRID_FILTER_TYPE.annotations; keep filter as only the annotation
predicates and reserve the discriminant for snapshotFilter. Update the logic in
useSelectAll so fetchIds still calls fetchSampleIdsForAnnotations with the
unmodified annotation filter plus embedding_region, while snapshotFilter is the
only object that adds filter_type. Locate the change around
get(annotationFilter), plotRegion, and the returned snapshotFilter/fetchIds
pair.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 58e0a5f5-9213-418b-a732-c3834ed08e99

📥 Commits

Reviewing files that changed from the base of the PR and between c531a30 and dd81371.

📒 Files selected for processing (29)

• lightly_studio/src/lightly_studio/api/routes/api/annotation.py
• lightly_studio/src/lightly_studio/resolvers/annotation_resolver/get_all_with_payload.py
• lightly_studio/src/lightly_studio/resolvers/annotation_resolver/get_sample_ids.py
• lightly_studio/src/lightly_studio/resolvers/annotations/annotations_filter.py
• lightly_studio/src/lightly_studio/resolvers/sample_resolver/sample_filter.py
• lightly_studio/tests/resolvers/annotations/test_annotations_filters.py
• lightly_studio/tests/resolvers/sample_resolver/test_sample_filter.py
• lightly_studio_view/src/lib/components/AnnotationsGrid/AnnotationsGrid.svelte
• lightly_studio_view/src/lib/components/Images/syncFilterParams.test.ts
• lightly_studio_view/src/lib/components/Images/syncFilterParams.ts
• lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.svelte
• lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.test.ts
• lightly_studio_view/src/lib/components/PlotPanel/usePlotData/usePlotData.test.ts
• lightly_studio_view/src/lib/components/PlotPanel/usePlotData/usePlotData.ts
• lightly_studio_view/src/lib/hooks/useAnnotationsInfinite/createAnnotationsInfiniteOptions.test.ts
• lightly_studio_view/src/lib/hooks/useAnnotationsInfinite/createAnnotationsInfiniteOptions.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilter.test.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForAnnotations.test.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForAnnotations.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingFilterForImages.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingPlotSelection.test.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useEmbeddingPlotSelection.ts
• lightly_studio_view/src/lib/hooks/useEmbeddingFilter/useRegionFilterVisibility.ts
• lightly_studio_view/src/lib/hooks/useImageFilters/useImageFilters.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/buildRequestBody.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/types.ts
• lightly_studio_view/src/lib/hooks/useImagesInfinite/useImagesInfinite.test.ts
• lightly_studio_view/src/lib/hooks/useSelectAll/useSelectAll.ts
• lightly_studio_view/src/routes/datasets/[dataset_id]/[collection_type]/[collection_id]/+layout.svelte