Skip to content

feat(engine): compact context crops and stable per-event detection crops#379

Merged
MateoLostanlen merged 11 commits into
developfrom
feat/stable-context-crops
Jun 16, 2026
Merged

feat(engine): compact context crops and stable per-event detection crops#379
MateoLostanlen merged 11 commits into
developfrom
feat/stable-context-crops

Conversation

@MateoLostanlen

@MateoLostanlen MateoLostanlen commented Jun 12, 2026

Copy link
Copy Markdown
Member

This PR addresses two issues with the 4K detection crops:

  • Reduce RAM usage: the full decoded 4K frame (~23 MB) was kept in the sliding window and in the alert queue (up to 100 entries) only to cut 224x224 crops at send time. We now keep a small JPEG context crop around the detections, with its position in the frame, and cut the final crops from it at send time. Measured on real 4K frames, a stored entry drops from ~23 MB to ~0.14 MB on average, which frees ~0.5 GB in steady state on a 4-pose setup and removes a ~2.3 GB worst case when the alert queue fills during an API outage.
  • Stabilize crops across the alert sequence: crops were re-centered on each frame's bbox, so they moved with bbox jitter. Crop boxes are now computed from the union of tracked bboxes, squared and padded, then frozen for the whole event, so every frame of the sequence is cropped at the same location.
  • Multi-detection: overlapping bboxes are clustered and each cluster gets its own frozen crop box. A detection that outgrows its box re-anchors to a new one once.

Store a JPEG-encoded region around the raw predictions (3x the union
size, min 1024px) with its offset instead of the full decoded 4K frame
(~24MB each) in the sliding window and alert queue.

At staging, cluster the tracked bboxes and freeze one square crop box
per cluster until the event ends, so the 224x224 crops stay centered
on the same spot instead of following bbox jitter. Crops are cut from
the stored context crop at send time.
Size both context-crop axes on the largest side of the preds union so
the square detection crop always fits, re-anchor the frozen box when a
bbox grows mostly outside it, and re-square clipped crops to avoid
aspect distortion.
@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.15789% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.83%. Comparing base (438493e) to head (e29027d).

Files with missing lines Patch % Lines
pyroengine/engine.py 93.15% 13 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #379      +/-   ##
===========================================
+ Coverage    75.32%   78.83%   +3.50%     
===========================================
  Files            6        6              
  Lines          685      841     +156     
===========================================
+ Hits           516      663     +147     
- Misses         169      178       +9     
Flag Coverage Δ
unittests 78.83% <93.15%> (+3.50%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Size the stored region from the final crop box plus a fixed jitter
margin instead of 3x the bbox, so a very large detection cannot grow
the in-RAM region toward the full frame.
… view

Restore a wide field of view (3x the preds-union side) so the frozen
crop box stays inside the stored region even when smoke drifts with the
wind. Bound RAM by downscaling the stored pixels above CONTEXT_MAX_SIDE
instead of narrowing the view: small regions keep full resolution, large
ones are downscaled. Track the region box in full-frame coords so crop
extraction maps coordinates through the downscale factor.
Assign a frozen crop box only when it covers the bbox by at least
MIN_BBOX_COVERAGE, so a drifted backfilled pred is no longer cropped on
a box it merely grazes. Dedupe boxes by IoU before appending so an
oversized cluster (uncoverable by a square crop) cannot grow
event_crop_boxes on every frame of an event.
…t-crops

# Conflicts:
#	pyroengine/engine.py
#	tests/test_engine.py
A frame with no detection inside an ongoing alert no longer produces a
blank/placeholder frame: it builds a context crop of the current frame
around the known fire location (the frozen crop boxes) and carries the
previous bbox forward at conf 0, so the alert sequence keeps a crop at
the same spot instead of a gap.
During an ongoing alert _build_context_crop now folds the frozen event
boxes into the covered region, so a frame whose preds are elsewhere (or
absent) still stores a crop containing the fire location. Without this a
carried-forward bbox could be cropped from the wrong region. Also lower
CONTEXT_MAX_SIDE 1536 -> 1024 (still well above the ~560px floor needed
to avoid upscaling the 224 crop), roughly halving worst-case crop RAM.
When a new alert starts while the window still holds staged frames from
a previous event, carry-forward could seed the old fire location onto
gap frames of the new event. _end_event now clears staged frames' bboxes
(keeping unstaged lead-up frames), so a previous event's location cannot
bleed into the next event's carry-forward or tracked set.
@MateoLostanlen MateoLostanlen merged commit 8958a05 into develop Jun 16, 2026
11 checks passed
@MateoLostanlen MateoLostanlen deleted the feat/stable-context-crops branch June 16, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant