Skip to content

Lig 9903 2 fe images#1583

Closed
ikondrat wants to merge 2 commits into
mainfrom
lig-9903-2-fe-images
Closed

Lig 9903 2 fe images#1583
ikondrat wants to merge 2 commits into
mainfrom
lig-9903-2-fe-images

Conversation

@ikondrat

@ikondrat ikondrat commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

What has changed and why?

(Delete this: Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.)

How has it been tested?

(Delete this: Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.)

Did you update CHANGELOG.md?

  • Yes
  • Not needed (internal change)

Summary by CodeRabbit

  • New Features

    • Added support for selecting samples by drawing a region on the embedding plot.
    • Region selections are now preserved in filters and sent through the app.
  • Bug Fixes

    • Improved selection behavior so highlighted regions map to the correct samples.
    • Empty region matches now return no results instead of partial selections.
  • Tests

    • Added coverage for region-based selection, request payloads, and clearing behavior.

@ikondrat ikondrat requested a review from a team as a code owner July 6, 2026 12:44
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 5346a633-bfa3-4c04-b294-e8ff5544fcb9

📥 Commits

Reviewing files that changed from the base of the PR and between 054c7d4 and 5a3fcb0.

📒 Files selected for processing (14)
  • lightly_studio/src/lightly_studio/models/embedding_region.py
  • lightly_studio/src/lightly_studio/resolvers/embedding_region_resolver.py
  • lightly_studio/src/lightly_studio/resolvers/image_resolver/count_image_annotations_by_collection.py
  • lightly_studio/src/lightly_studio/resolvers/image_resolver/get_all_by_collection_id.py
  • lightly_studio/src/lightly_studio/resolvers/image_resolver/get_sample_ids.py
  • lightly_studio/src/lightly_studio/resolvers/sample_resolver/sample_filter.py
  • lightly_studio/tests/resolvers/sample_resolver/test_sample_filter.py
  • lightly_studio/tests/resolvers/test_embedding_region_resolver.py
  • lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.svelte
  • lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.test.ts
  • lightly_studio_view/src/lib/hooks/useImageFilters/useImageFilters.ts
  • lightly_studio_view/src/lib/hooks/useImagesInfinite/buildRequestBody.ts
  • lightly_studio_view/src/lib/hooks/useImagesInfinite/types.ts
  • lightly_studio_view/src/lib/hooks/useImagesInfinite/useImagesInfinite.test.ts

📝 Walkthrough

Walkthrough

Adds backend support for embedding-plot region selections: a new EmbeddingRegion/Point2D model, a point-in-polygon resolver wired into image/sample filters, and SampleFilter support for resolved region sample IDs. Frontend PlotPanel, filter hooks, and request-building types are updated to send and commit polygon-based selections instead of raw sample IDs.

Changes

Backend region resolution

Layer / File(s) Summary
EmbeddingRegion and Point2D data model
lightly_studio/src/lightly_studio/models/embedding_region.py
New Pydantic models Point2D (x, y) and EmbeddingRegion (polygon), with a validator requiring at least three vertices.
Region resolver and point-in-polygon algorithm
lightly_studio/src/lightly_studio/resolvers/embedding_region_resolver.py, lightly_studio/tests/resolvers/test_embedding_region_resolver.py
resolve_embedding_region, get_sample_ids_in_region, and a vectorized _points_in_polygon ray-casting test convert a polygon into sample IDs using cached 2D embeddings; covered by new tests.
SampleFilter embedding_region field and apply() integration
lightly_studio/src/lightly_studio/resolvers/sample_resolver/sample_filter.py, lightly_studio/tests/resolvers/sample_resolver/test_sample_filter.py
Adds embedding_region field, set_resolved_region_sample_ids, and _apply_embedding_region_filter (raises if unresolved, matches nothing if empty, else filters by ID), tested.
Wiring into image resolvers
lightly_studio/src/lightly_studio/resolvers/image_resolver/count_image_annotations_by_collection.py, .../get_all_by_collection_id.py, .../get_sample_ids.py
Calls resolve_embedding_region before existing counting/query logic in each resolver.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Frontend region selection UI and request wiring

Layer / File(s) Summary
PlotPanel region-mode selection commit and clearing
lightly_studio_view/src/lib/components/PlotPanel/PlotPanel.svelte, PlotPanel.test.ts
Adds region-mode derivation, committed polygon highlighting, mouse-up handling that commits/clears embedding_region instead of sample IDs, and updated clear/active-selection logic, with test coverage.
useImageFilters embedding_region field and updater
lightly_studio_view/src/lib/hooks/useImageFilters/useImageFilters.ts
Includes embedding_region in derived sampleFilter and adds/exposes updateEmbeddingRegion.
Request body and type propagation
lightly_studio_view/src/lib/hooks/useImagesInfinite/buildRequestBody.ts, types.ts, useImagesInfinite.test.ts
NormalModeFilters and buildRequestBody now include sample_filter.embedding_region, with new unit tests.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant PlotPanel
  participant useImageFilters
  participant ImageResolver
  participant embedding_region_resolver
  participant SampleFilter

  User->>PlotPanel: draw lasso/rectangle and mouse up
  PlotPanel->>useImageFilters: updateEmbeddingRegion(polygon)
  useImageFilters->>useImageFilters: set filters.embedding_region
  PlotPanel->>ImageResolver: request images (sample_filter.embedding_region)
  ImageResolver->>embedding_region_resolver: resolve_embedding_region(session, collection_id, filters)
  embedding_region_resolver->>embedding_region_resolver: get_sample_ids_in_region (point-in-polygon)
  embedding_region_resolver->>SampleFilter: set_resolved_region_sample_ids(ids)
  ImageResolver->>SampleFilter: apply(query)
  SampleFilter-->>ImageResolver: filtered results
  ImageResolver-->>PlotPanel: matching samples
Loading

Possibly related PRs

  • lightly-ai/lightly-studio#829: Both PRs modify the same SampleFilter class in sample_filter.py, one removing collection_id filtering, this one adding embedding_region filtering.
  • lightly-ai/lightly-studio#890: Both PRs modify PlotPanel.svelte/PlotPanel.test.ts selection commit/clear logic for embedding-plot selections.
  • lightly-ai/lightly-studio#1143: Both PRs touch the buildRequestBody.ts normal-mode filter construction logic (withNormalFilters).

Suggested reviewers: LeonardoRosaa

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title looks like a branch/reference label and does not clearly describe the actual feature change. Rename it to a short, descriptive sentence that summarizes the main change, such as embedding-region selection support.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch lig-9903-2-fe-images

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a3fcb015c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

...params,
filters: {
...params.filters,
embedding_region: embeddingRegion ?? undefined

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve embedding_region during image filter sync

When a user commits an image lasso, this writes the selection into filterParams.filters.embedding_region, but the Images.svelte sync path still treats only sample_ids and confusion_cell as externally managed filters (syncFilterParams.ts omits/merges only those keys). On the next reactive sync, the current params differ from the base grid params and mergeExternalFilters rewrites filterParams without embedding_region, so the new region selection is cleared before it can reliably drive the grid. Include embedding_region in that external-filter omit/merge path.

Useful? React with 👍 / 👎.

Comment on lines +88 to +91
if self._resolved_region_sample_ids is None:
raise RuntimeError(
"embedding_region must be resolved with set_resolved_region_sample_ids() "
"before the filter is applied."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Resolve embedding regions before adjacent image queries

With an image lasso active, useAdjacentImages sends the current ImageFilter to /samples/{sample_id}/adjacents, but the image adjacency resolvers call filters.apply(...) directly in both keyset and window paths without calling resolve_embedding_region. This new guard therefore raises for any unresolved embedding_region, causing image detail navigation/adjacent lookups after a lasso selection to fail with a server error instead of returning neighbors. Resolve or strip the region in the adjacency path before applying the filter.

Useful? React with 👍 / 👎.

Comment on lines +83 to +85
const embeddingRegion = $filterParams.filters?.embedding_region;
if (embeddingRegion) {
sampleFilter.embedding_region = embeddingRegion;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Forward embedding regions to annotation counts

After this change image plot selections live in sample_filter.embedding_region rather than sample_ids, but the layout still builds image annotation-count filters from $imageFilterFromHook?.sample_filter?.sample_ids only before calling useImageAnnotationCounts. In the scenario where an image lasso is active, the grid is filtered by the backend-resolved region while the annotation filter rows/counts continue to reflect the whole grid; pass the region through the count-filter builder as well.

Useful? React with 👍 / 👎.

@ikondrat ikondrat closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant