Fingerprint frame layer previews

[BryonLewis]

Modifies #397 to not have a LayerStyle connection and rely only the fingerprint and layer foreign key for indentifying a preview.

New Models

• RasterFramePreview: caches one rendered PNG per (layer_frame, style_fingerprint). Fields include style_fingerprint (sha256 of raster_style_params), raster_style_params (render snapshot), status (creating / regenerating / complete / failed), S3 image, plus width / height / bounds. Unique constraint on (layer_frame, style_fingerprint). Deleting a row also deletes its S3 image.
• LayerStyle.raster_style_params: JSONField holding the client-computed django-large-image style query used for tiles and previews.
• Layer helpers: raster_frames() and is_multiframe_raster() (previews only apply when a layer has more than one raster frame).
• Migration: 0025_raster_frame_preview.py.

Frame Preview Workflow

Previews are content-addressed by a fingerprint of raster_style_params (params_fingerprint / style_fingerprint).

1. Enqueue via invalidate_and_enqueue_layer_previews:
   • Skip non-multiframe layers.
   • Skip if every frame already has a complete preview for that fingerprint.
   • Skip if a frame_preview TaskResult is already in flight for (layer_id, fingerprint).
   • Otherwise upsert preview rows (creating / regenerating), clear old images, create a TaskResult (task_type="frame_preview"), and run generate_frame_previews.
2. Generation (generate_frame_previews): render a styled PNG thumbnail per frame with large-image, store on the preview row, mark complete (or failed), then complete the TaskResult (WebSocket notify).
3. Lookup / API status: preview_status is "ready" only when every raster frame has a complete image for the fingerprint; otherwise "notready". multiframe_previews is only serialized when ready.

How new styles update items

On style create/update, the client sends raster_style_params (write-only). The serializer saves the style, then calls invalidate_and_enqueue_previews(instance):

• New fingerprint → new/regenerating preview rows for that style’s params.
• Existing fingerprints are left alone (no supersede); uniqueness is by fingerprint.
• Response returns preview_status: "notready" and omits multiframe_previews until generation finishes.
  Also triggered after:
  • Dataset conversion — empty-params ({}) previews for each multiframe layer.
  • Flood simulation — sets style raster_style_params and regenerates sync so outputs are ready before the flood task completes.

REST Endpoint Updates

• Prefetch helper layer_queryset_with_previews() loads raster frames + their preview rows in one go.
• LayerViewSet / dataset layers: use that queryset; LayerSerializer exposes preview_status and multiframe_previews (default fingerprint: default style params, else {}).
• LayerStyleViewSet: uses LayerStyleWithPreviewsSerializer with the same fields keyed to the style’s fingerprint; create/update accept raster_style_params and enqueue regeneration.
• Null preview fields are omitted from responses.

Task Updates

Notification suppression

suppress_task_notifications() (ContextVar) silences TaskResult WebSocket pushes during sync contexts (ingest, sync conversion, sync preview generation) where no client is listening. Failed pushes are also best-effort (logged, never abort save()).

asynchronous=True → run_mode (Ruff / boolean-trap)

Boolean default args for async vs sync were replaced with an explicit enum.:

• TaskRunMode.ASYNC / TaskRunMode.SYNC ("async" / "sync")
• Call sites updated: Dataset.spawn_conversion_task / convert_dataset, create_layers_and_frames, preview enqueue helpers, flood/geoai analytics, ingest sample scripts.
  Async preview jobs enqueue with transaction.on_commit(...delay). Sync jobs run under suppress_task_notifications() via .apply(), matching the surrounding conversion/ingest mode (no Celery worker required for ingest).

Client Updates — How the Client Gets Information

1. Initial load: layer/style API payloads include preview_status and (when ready) multiframe_previews (url, width, height, bounds). Selecting a layer copies those onto the active style and prefetches image URLs.
2. Style save: client posts raster_style_params, immediately marks local state notready, clears stale previews, and dismisses the map overlay so real tiles show while regenerating.
3. Completion via WebSocket:
   • Project-scoped regenerations → analytics WS (analysis store).
   • Conversion-time default previews (often no project) → conversion WS (conversion store).
   • Both call framePreviewStore.onPreviewTaskComplete, which re-fetches the layer or style and reattaches previews to selected copies still using that style.
4. Map UX (framePreview store + framePreviewLayer utils): show the current-frame PNG as a MapLibre image overlay, prefetch adjacent frames for scrubbing, then fade to real tiles once the tile source loads. Layers/legend panels indicate when a preview overlay is active. Style editing dismisses previews until edit mode ends.

[cloudflare-workers-and-pages]

Deploying geodatalytics with    Cloudflare Pages

Latest commit:	1427287
Status:	✅  Deploy successful!
Preview URL:	https://69a4a023.geodatalytics.pages.dev
Branch Preview URL:	https://fingerprint-frame-layer-prev.geodatalytics.pages.dev

View logs

[annehaley]

As with #397, I'll probably do a few rounds of review on this. My first pass is more focused on server-side, I'll test it locally and take a closer look at client-side on my next pass.

Could you rebase this and ensure that your commit messages have semantic prefixes? Without them, a release won't be made upon merge.

[annehaley]

If a frame preview generation task occurs as a consequence of a dataset conversion, we should run the generation synchronously within the conversion task so the conversion only appears complete once the previews are ready too (the same way you did for the flood simulation task).

[BryonLewis]

I don't think this is the correct referenced lines for this comment. But I made it so that any preview generation task started by the core/tasts/dataset.py and specifically create_layers_and_frames will default to using run_mode=Task.SYNC inside of the invalidate_and_enqueue_layer_previews so it will run inside of the current task instead of starting a new task.

I also fixed some bad merge history in the _dispatch_frame_preview_task function that was duplicating some functions.
Done in : 9b32957

[annehaley]

I had left this comment here because running the preview generation task synchronously within the dataset conversion task means that we don't need to pick an arbitrary project to associate with the task. The dataset conversion tasks are intentionally not associated with any project. If we spawn a preview generation task because a style object is updated, we can use the style's project.

Is there some other case wherein we need to pick an arbitrary project for the task result object? If not, I think _resolve_preview_task_project can be removed entirely and replaced with layer_style.project || project, which is allowed to be None.

[BryonLewis]

Merging #458 improves/changes the following:

• Preview first loading - previews are loaded before the raster image to make sure that even when raster tiles a long time to come up a preview is shown relatively quickly
• updateLayerStyles now applies the new style_spec to all loaded frames, previously it was only the current frame so you would get a cached preview of the other layer style from the raster view (not the preview). This was seen before as a flash of an old style right after the preview loaded and the raster tiles first displayed
• clearPreviewsForStyleChange - clears map preview overlays and invalidate cached multiframe_previews until the API reports that they are ready again. This should make it so frame previews are all invalidated when a style is changed and relies on the raster tile until the previews are loaded again
• Frame Scrubbing Debounce - large lists would attempt to load the raster tiles immediately. This has updated to give a 1000ms delay while scrubbing until it attempts to load the raster tile.
• Preview Indicator - solid blue for looking at a frame preview, flashing gray for generating previews, corresponding tooltip text has been updated as well
• backfill_frame_previews Management Command - a command that will create frame previews for any multiframe rasters that exist in the system. It will first generate the None style previews and then generate any custom styles. It does have a local raster_style_params generator so it doesn't need the client to do this
  • Helper clear_frame_previews management command that will clear all frame previews and reset the raster_style_params on any styles to be None. Useful for testing the process for the backfill_frame_previews