Skip to content

Inter-worker message bus and per-function call sampling - #1604

Open
elopez wants to merge 2 commits into
masterfrom
feat/interworker-bus
Open

Inter-worker message bus and per-function call sampling#1604
elopez wants to merge 2 commits into
masterfrom
feat/interworker-bus

Conversation

@elopez

@elopez elopez commented Aug 14, 2026

Copy link
Copy Markdown
Member

Why

Workers can currently communicate in exactly one direction: each pushes events
onto the campaign's event queue, which the UI and the listener read. Nothing can
talk back to a running worker, and no worker can tell another one what it just
learned. Every remaining piece of #1502 needs that channel, so it lands here
together with the first thing that uses it.

What

feat: sample per-function call statistics

A worker can be asked to watch a handful of functions and keep, for each, how
often it was called, how often the call did not return successfully, the range
its return value spanned, and a short tail of revert summaries — what an outside
observer needs to answer "is this function reachable, and what does it do when
it is" without reading the corpus.

The state hangs off WorkerState.sampledFunctions, keyed by canonical
signature. The per-event algebra is factored out of the worker into
Types/Campaign.hs (applySampleEvent, abiCompare, mergeSampleStats) so it
is pure and unit-testable; updateSampleStats in Worker/Sequence.hs is left
with just decode-and-dispatch. Tests.Sample covers all three, 25 cases.

feat: add an inter-worker message bus

Env gains a broadcast STM channel alongside the event queue. Readers take
their own dupTChan, so a message reaches every reader instead of being raced
for by whoever gets there first. Echidna.Types.InterWorker holds the protocol.

Two users: callseq broadcasts NewCoverageInfo when a sequence finds
coverage, carrying the sequence and whether it came from replaying the corpus
(the new argument threaded through callseq's callers); and a fuzzing worker
checks the bus once per loop iteration for commands addressed to it, which is
how sampling is turned on and off.

Notes for review

  • Nothing enables sampling yet, and nothing listens for NewCoverageInfo
    yet.
    Both clients are the MCP server, two rungs up. Tests.Sample is the
    entry point for the sampling algebra in the meantime. The two commands land
    here rather than with their sender because they are what makes the bus worth
    having.
  • The rest of the upstream protocol is left out. Symbolic commands,
    request/response, and the other broadcasts have neither a sender nor a reader
    on dev-agents-3; each comes back with the code that uses it. (WorkerStopped
    also collided with Types.Worker.WorkerEvent's constructor of the same name.)
  • checkMessages drains its queue rather than handling one message per
    iteration.
    Every broadcast is copied into every worker's duplicate, so with
    N workers a worker receives roughly N messages per iteration; consuming one
    would let the queue grow without bound during the early phase of a campaign,
    when almost every sequence finds new coverage. Commands originate outside the
    campaign at human pace, so draining cannot starve fuzzing.
  • The broadcast's sender is resolved through workerIDToType, the same way
    pushWorkerEvent decides which worker type to attribute an event to. The
    symbolic worker calls callseq too, and it is not a fuzzer.
  • The strictness in SampleStats is load-bearing and was not free. These
    fields are updated on the fuzzing hot path and only read when a client asks,
    so take n (new : old) would keep old alive through its unevaluated tail
    and old its predecessor — the list capped at five entries would have
    retained the entire revert history. Hence takeStrict, and the forced tuple
    components in the range update.
  • The command handler lives in its own module rather than another where-block
    inside runFuzzWorker, matching what the preceding decomposition was aiming
    for and giving the handlers in later rungs a home.

Verification

  • cabal build all and cabal build tests clean under -Wall -Wunused-packages
  • cabal run tests — all 190 pass, symbolic tests included, so both a normal
    fuzzing run and a symbolic run still terminate
  • hlint lib src — clean (one pre-existing hint in SourceMapping.hs, untouched
    here)
  • Driven end-to-end with a throwaway probe standing in for the MCP client: a
    ToFuzzer 0 (EnableSampling "bump(uint256)") written by any worker reached
    worker 0 through its duplicate, workers 1–3 correctly ignored it, and worker 0
    reported sampleCalls = 2526, sampleReverts = 1603 with a widened return
    range and exactly maxRecentReverts summaries retained.

@elopez
elopez force-pushed the feat/interworker-bus branch from ac2386b to d1c7e87 Compare August 14, 2026 16:02
Base automatically changed from refactor/agent-typeclass to master August 14, 2026 18:36
elopez and others added 2 commits August 14, 2026 15:36
A worker can now be asked to watch a handful of functions and keep, for
each of them, how often it was called, how often the call did not return
successfully, the range its return value spanned, and a short tail of
revert summaries. This is what an outside observer needs to answer "is
this function reachable, and what does it do when it is" without reading
the corpus.

The state hangs off WorkerState.sampledFunctions, keyed by canonical
signature. Nothing populates that map yet -- the command that enables
sampling arrives with the inter-worker bus -- so callseq's cost is a
Map.null check per sequence, and the per-call work only happens for
functions someone asked about.

The interesting part is the per-event algebra, so it is factored out of
the worker: applySampleEvent folds one call result into a SampleStats,
abiCompare gives the partial ordering the range tracking needs, and
mergeSampleStats combines two workers' views. All three are pure and
covered by Tests.Sample; updateSampleStats in Worker/Sequence.hs is left
with just the decode-and-dispatch.

Every SampleStats field is strict, and the revert list is built with a
take that forces its spine and elements. These are updated on the fuzzing
hot path and only read when a client asks, so a lazy field would quietly
retain the entire call history: a plain `take n (new : old)` keeps `old`
alive through its unevaluated tail, and `old` keeps the one before it.

Co-authored-by: gustavo-grieco <gustavo.grieco+github@gmail.com>
Workers have so far been able to communicate in exactly one direction:
each pushes events onto the campaign's event queue, which the UI and the
listener read. Nothing can talk back to a running worker, and a worker
cannot tell another one what it just learned.

Add a bus to Env alongside the event queue: a broadcast STM channel, so
readers take their own dupTChan of it and every reader sees every
message rather than racing for it. Echidna.Types.InterWorker holds the
protocol -- a message, its sender, and the commands a fuzzing worker
accepts.

Two things use it. callseq broadcasts NewCoverageInfo whenever a
sequence finds coverage, carrying the sequence and whether it came from
replaying the corpus, which is the new argument threaded through
callseq's callers. And a fuzzing worker checks the bus once per loop
iteration for commands addressed to it, which is how sampling gets
turned on and off.

The two commands land here rather than with a sender because they are
what makes the bus worth having; the client that issues them, and the
listener for NewCoverageInfo, arrive with the MCP server. Everything
else in the upstream protocol -- symbolic commands, request/response,
the other broadcasts -- has neither a sender nor a reader, so it is left
out until it has both.

The command handler lives in its own module rather than in another
where-block inside runFuzzWorker, which is the shape the preceding
decomposition was aiming for and the natural home for the handlers that
follow.

Two details worth calling out:

checkMessages drains its queue rather than taking one message per
iteration. Every broadcast is copied to every worker's duplicate, so
with N workers finding coverage a worker receives roughly N messages per
iteration; consuming one would let the queue grow without bound during
the early phase of a campaign, when almost every sequence is new
coverage. Commands come from outside the campaign at human pace, so
there is nothing to starve fuzzing with.

The broadcast's sender is resolved through workerIDToType, the same way
pushWorkerEvent decides which worker type to attribute an event to. The
symbolic worker calls callseq too, and it is not a fuzzer.

Co-authored-by: gustavo-grieco <gustavo.grieco+github@gmail.com>
@elopez
elopez force-pushed the feat/interworker-bus branch from 2e06348 to 7b32c00 Compare August 14, 2026 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant