Skip to content

feat(audience): add DiskStore and EventQueue for persistent event batching (SDK-127)#690

Merged
ImmutableJeffrey merged 3 commits into
mainfrom
feat/sdk-127-event-queue
Apr 17, 2026
Merged

feat(audience): add DiskStore and EventQueue for persistent event batching (SDK-127)#690
ImmutableJeffrey merged 3 commits into
mainfrom
feat/sdk-127-event-queue

Conversation

@nattb8
Copy link
Copy Markdown
Collaborator

@nattb8 nattb8 commented Apr 15, 2026

Summary

  • Adds DiskStore (Runtime/Transport/DiskStore.cs): file-per-event persistent store writing {ticks}_{uuid}.json files atomically to imtbl_audience/queue/; supports ReadBatch (oldest-first, max 100, stale-event pruning after 30 days), Delete, and Count; crash-safe (survives restarts by scanning the queue directory on init)
  • Adds EventQueue (Runtime/Transport/EventQueue.cs): wraps ConcurrentQueue<string> with a background drain thread; flushes to DiskStore every FlushIntervalSeconds or when FlushSize events accumulate; FlushAsync() for on-demand drain; Shutdown()/Dispose() for clean exit with final flush
  • Adds DiskStoreTests and EventQueueTests covering write, batch read, stale pruning, crash recovery, size-triggered flush, interval flush, shutdown, and post-shutdown enqueue

Test plan

  • DiskStoreTests: write creates file, contents match JSON, batch returns oldest first, max size respected, clamped to MaxBatchSize, stale files excluded and deleted, Delete removes files, Count tracks file count, empty/zero cases, crash recovery
  • EventQueueTests: enqueue + FlushAsync persists, multiple events, FlushSize auto-drain, Shutdown flushes remainder, double-shutdown safe, enqueue-after-shutdown ignored, interval auto-flush, Dispose flushes

🤖 Generated with Claude Code


Note

Medium Risk
Introduces new background-thread and on-disk persistence behavior that can affect event delivery reliability and performance (timing, race conditions, and file I/O edge cases), though changes are additive and covered by tests.

Overview
Adds a disk-backed batching layer to the Audience SDK by introducing DiskStore (file-per-event persistence under imtbl_audience/queue/, batched reads capped to Constants.MaxBatchSize, and automatic pruning of events older than Constants.StaleEventDays) and EventQueue (thread-safe in-memory enqueue with a background drain thread that flushes by size threshold or time interval, plus Shutdown/Dispose for final flush).

Includes new NUnit runtime tests covering atomic writes, ordering and batch sizing, stale-file cleanup and crash recovery for DiskStore, plus flush triggers, interval draining, and shutdown/dispose behavior for EventQueue.

Reviewed by Cursor Bugbot for commit 14b57a1. Bugbot is set up for automated code reviews on this repo. Configure here.

@nattb8 nattb8 requested review from a team as code owners April 15, 2026 00:27
@nattb8 nattb8 force-pushed the feat/sdk-127-event-queue branch from a3a7892 to 427b8d6 Compare April 16, 2026 05:39
…ching (SDK-127)

Implements file-per-event disk persistence under imtbl_audience/queue/ with
atomic writes, stale-event pruning (30d), and crash recovery. EventQueue
wraps a ConcurrentQueue with a background drain thread that flushes to
DiskStore on a time interval or when FlushSize is reached; Shutdown/Dispose
guarantee a final flush.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread src/Packages/Audience/Runtime/Transport/EventQueue.cs Outdated
Comment thread src/Packages/Audience/Runtime/Transport/EventQueue.cs
ImmutableJeffrey and others added 2 commits April 17, 2026 11:25
Addresses cursor[bot] review comment on EventQueue.cs:29.

- Mark _disposed as volatile so the shutdown signal is visible across
  threads. On weakly-ordered architectures (ARM — Apple Silicon, etc.)
  the C# memory model does not guarantee a non-volatile write is
  eventually observed by other threads without a fence.

- Reorder Shutdown: set _disposed=true first, before canceling the drain
  thread. The shutdown flag being the first observable state change is a
  safer ordering and makes the lifecycle easier to reason about.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Addresses cursor[bot] review comment on EventQueue.cs:67.

The method blocks synchronously and returns void — the Async suffix is
misleading in C# where it conventionally signals a Task-returning,
awaitable method. Renaming to FlushSync makes the blocking semantics
explicit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ImmutableJeffrey ImmutableJeffrey force-pushed the feat/sdk-127-event-queue branch from ececaa8 to 14b57a1 Compare April 17, 2026 01:53
@ImmutableJeffrey ImmutableJeffrey merged commit 06a5d0a into main Apr 17, 2026
19 checks passed
@ImmutableJeffrey ImmutableJeffrey deleted the feat/sdk-127-event-queue branch April 17, 2026 02:03
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 14b57a1. Configure here.

internal EventQueue(DiskStore store, int flushIntervalSeconds, int flushSize)
{
_store = store ?? throw new ArgumentNullException(nameof(store));
_flushIntervalMs = Math.Max(1, flushIntervalSeconds) * 1000;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Integer overflow in flush interval kills drain thread

Medium Severity

Math.Max(1, flushIntervalSeconds) * 1000 performs unchecked int multiplication that silently overflows for flushIntervalSeconds values ≥ 2,147,484 (~24.8 days), producing a negative _flushIntervalMs. When the drain thread calls _flushGate.Wait(_flushIntervalMs) with that negative value, it throws ArgumentOutOfRangeException (only -1 is valid for negative). Since DrainLoop has no try-catch, the background thread dies silently and events are never persisted to disk. A user setting Int32.MaxValue to disable interval flushing would trigger this immediately.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 14b57a1. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

2 participants