Skip to content

bug(acp): observer replay and broadcast retention lack a byte budget #7987

Description

@shanemhamilton

Describe the bug

The process-local ACP observer retains raw payloads in two count-limited stores before the relay publisher trims or byte-bounds them. OBSERVER_BUFFER_CAP = 1_000 bounds event count, but not retained payload bytes. Large, valid ACP messages can consequently retain gigabytes of JSON while the downstream relay queue remains within its 4 MiB budget.

This is a source-confirmed retention gap, not a measured production RSS result. The code remains present on upstream main at d7a35afa9859a0c666de93ab1cd4f8a433a1cfe8.

Steps to reproduce / proposed regression

  1. Create an in-process ObserverHandle and retain a subscriber that does not drain immediately.
  2. Emit 1,000 observer events containing 1 MiB ASCII string payloads.
  3. Inspect the replay snapshot's serialized byte total. The current buffer keeps all 1,000 events because it evicts only when the event count reaches 1,000; the retained string data alone totals 1,000 MiB.
  4. A lagging subscriber's count-limited broadcast ring also retains raw event payloads.

This reproduction has not been run here; the observations above follow from the current retention paths. A regression should assert both a byte bound and bounded handling of a single event larger than that bound.

Expected behavior

Observer replay and broadcast retention should have a defined byte budget in addition to the count limit. Oversized events should become a bounded representation before retention/broadcast, or be explicitly dropped with loss accounting. Eviction should preserve ordering and recent events. The regression should exercise ObserverHandle::emit and snapshot, including large events and a stalled subscriber.

Version and platform

  • Installed environment: Buzz Desktop 0.5.25, macOS.
  • Source inspection: upstream main at d7a35afa9859a0c666de93ab1cd4f8a433a1cfe8.
  • The retention implementation is platform independent; other platforms have not been exercised.

Logs / additional context

Relevant production paths:

Suggested contribution: add byte accounting to replay retention and bound the representation placed in the broadcast ring, with deterministic tests for many large events, a single oversized event, eviction order, and subscriber lag. Bounding only the replay VecDeque would leave the broadcast ring retaining the same large payloads.

Duplicate searches covered observer replay/buffer/memory/byte retention and the OBSERVER_BUFFER_CAP symbol. Closest related work: #6830 (publish cadence), #5718 (Desktop transcript rebuilding), #5220 (Unix event sink), and #5578/#7086 (NIP-44 plaintext limit). None found addressing this process-local raw retention gap.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions