feat(engine): compact context crops and stable per-event detection crops#379
Merged
Conversation
Store a JPEG-encoded region around the raw predictions (3x the union size, min 1024px) with its offset instead of the full decoded 4K frame (~24MB each) in the sliding window and alert queue. At staging, cluster the tracked bboxes and freeze one square crop box per cluster until the event ends, so the 224x224 crops stay centered on the same spot instead of following bbox jitter. Crops are cut from the stored context crop at send time.
Size both context-crop axes on the largest side of the preds union so the square detection crop always fits, re-anchor the frozen box when a bbox grows mostly outside it, and re-square clipped crops to avoid aspect distortion.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #379 +/- ##
===========================================
+ Coverage 75.32% 78.83% +3.50%
===========================================
Files 6 6
Lines 685 841 +156
===========================================
+ Hits 516 663 +147
- Misses 169 178 +9
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Size the stored region from the final crop box plus a fixed jitter margin instead of 3x the bbox, so a very large detection cannot grow the in-RAM region toward the full frame.
… view Restore a wide field of view (3x the preds-union side) so the frozen crop box stays inside the stored region even when smoke drifts with the wind. Bound RAM by downscaling the stored pixels above CONTEXT_MAX_SIDE instead of narrowing the view: small regions keep full resolution, large ones are downscaled. Track the region box in full-frame coords so crop extraction maps coordinates through the downscale factor.
Assign a frozen crop box only when it covers the bbox by at least MIN_BBOX_COVERAGE, so a drifted backfilled pred is no longer cropped on a box it merely grazes. Dedupe boxes by IoU before appending so an oversized cluster (uncoverable by a square crop) cannot grow event_crop_boxes on every frame of an event.
…t-crops # Conflicts: # pyroengine/engine.py # tests/test_engine.py
A frame with no detection inside an ongoing alert no longer produces a blank/placeholder frame: it builds a context crop of the current frame around the known fire location (the frozen crop boxes) and carries the previous bbox forward at conf 0, so the alert sequence keeps a crop at the same spot instead of a gap.
During an ongoing alert _build_context_crop now folds the frozen event boxes into the covered region, so a frame whose preds are elsewhere (or absent) still stores a crop containing the fire location. Without this a carried-forward bbox could be cropped from the wrong region. Also lower CONTEXT_MAX_SIDE 1536 -> 1024 (still well above the ~560px floor needed to avoid upscaling the 224 crop), roughly halving worst-case crop RAM.
When a new alert starts while the window still holds staged frames from a previous event, carry-forward could seed the old fire location onto gap frames of the new event. _end_event now clears staged frames' bboxes (keeping unstaged lead-up frames), so a previous event's location cannot bleed into the next event's carry-forward or tracked set.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR addresses two issues with the 4K detection crops: