Skip to content

transport stage B: carriers over the host's I/O (tcp, stdio), the connector drives every connection through a carrier - #502

Open
MattJackson wants to merge 46 commits into
predevfrom
p2-transport-carrier
Open

MattJackson wants to merge 46 commits into
predevfrom
p2-transport-carrier

Conversation

@MattJackson

Copy link
Copy Markdown
Collaborator

Stage B of the transport stack: every connection rides a carrier over the host's I/O.

What changes

  • Host I/O table (io.*) — new memory-ABI table in busbar-contract/src/abi/host/io.rs: open, listen, accept, read, write, ready, shut, close, spawn and ends. The host owns every OS handle, and a plugin holds only an opaque handle that it owns. The layout golden and the C header are regenerated, and the trait-to-slot coverage test is extended to carriers and framers.
  • Carriers — tcp and stdio are carriers over io.*, both in their own repos and in-tree. Framers stay sans-IO. The connector builds a stack from a carrier, the optional TLS in connsec, and an optional framer, and drives the carrier through its slots the same way whether the carrier is linked or dropped in. TLS stays in the connector, and upgrades still move the byte stream.
  • Destination guard — the connector admits exactly the pinned address, bind or program for each ticket. Anything else is refused by the host before any system call.
  • Address carrier — the first carrier, in declared order, whose claim serves a port. Declared order is the build's linked rows, then the dropped-in doors in plugins-directory sorted order. There is no process-wide slot: each wire holds a resolver that the root hands it. The choice is pinned by a registry test, and the dropped-in tcp leg asserts that every data worker accepts through the dropped-in carrier, also with the neutral framer declared first.
  • Root listeners — the data and admin listeners accept through the address carrier and hand the socket up to the kernel.
  • Conformance — the suite's carrier script runs on tcp and stdio both ways: linked against dlopen, compared byte for byte, exact crossing counts, red arm kept. The interim "suite refuses a CARRIER role" guard is removed, and the loader's own transport both-ways test now drives the shipped carrier through the same script.
  • Source-scan test — busbar-core-connector/tests/no_io_outside_hostio.rs: no connector file outside the host's I/O opens, binds, accepts, spawns or shuts an OS handle. The red arm is kept. tls/engine.rs and udp/dtls are named exceptions, and both live in the connector.
  • neutral_frame_door — the fixture now declares itself a framer, so it can never be picked as an address carrier.

Proof

Debug builds on Latchkey.

  • This head:
    • clippy (--workspace --all-targets -D warnings) and fmt --check are green; the C header is unchanged after regeneration.
    • Tests of contract, connector, loader, stdio, busbar, kernel and the ws subject are green, except ledger_identity, which needs the oracle engine cloned from GitHub (runner limitation).
    • The dropped-in tcp and dropped-in stdio legs are green.
    • Single plane llm fails two tests: declared_codes_do_not_collide_with_the_registry and export_request_log_file_1_5_5. Neither file is touched by this PR; whether predev fails the same way on this leg is not yet confirmed.
  • Before the last predev merge (6b0c0a3):
    • Workspace tests: 13033 passed. The 11 failures are xtask tests that need .git (runner limitation) and the loader-naming test, fixed since in 4ff76aa.
    • The timing, duplex-ws, loader pack, contract test-seal, voice runtime, admin, oauth2 and webhook legs are green.
  • Gate rows (gate --all): the FAIL rows on the merged head are the same set as predev c162c71's.
  • Oracle: the route., wire. and stdio cells, recorded and replayed against golden/1.5.5, give rows identical to predev, and all 9 wire cells PASS.

…3-seam-ledger + t-h2host)

# Conflicts:
#	crates/busbar-contract/src/abi/host/hook.rs
#	crates/busbar-contract/src/abi/tests/host_hook_tests.rs
#	crates/busbar-core-connector/src/compose.rs
#	crates/busbar-core-connector/src/lib.rs
#	crates/busbar-core-connector/src/socket.rs
#	crates/plugin-loader/src/tests/hook_door_tests.rs
#	crates/plugin-loader/tests/fixtures/hook_door_plugin.rs
… role is stated; no transport names another

- http one entry claiming http + sse (busbar-transport-http#11), ws one entry (busbar-transport-ws#13),
  tcp ROLE_CARRIER (busbar-transport-tcp#13), stdio ROLE_CARRIER in tree; root rows http/ws on the
  transport-door axis, the sse row gone (a claim of the http entry); the seal registers every scheme a
  door's Statement claims (plugin-loader linked_facts, root doors::claims_of, LinkedTransport.claims).
- ARCHITECT Q128 U7: the connector reads the tail's role where it read composes_over (framed(),
  program/member needs, HostWire's held-message drive); the compose-by-list branch leaves (HostWire::
  composed/over, RootWire's lower/listen/accept/adopt through a layer); a door naming a layer is
  refused where it would be served.
- tests: the fold and session fixtures, the composed-over consistency (no wire built over another),
  the ws composition battery removed (composition left; its upgrade cells return with stage B).
…s (UNIT0_UPGRADE); ws re-pinned at its claim-row fix; the dropped-in wire proof under U7

- ARCHITECT Q128 U7 (approved): the data door's upgrade lines are every linked claim whose row states
  unit0_trigger = UNIT0_UPGRADE (loader TransportFacts.upgrades; root doors::upgrades_of,
  LinkedTransport.upgrades), never a layer list; RED at the loader (trigger 0 => no line), the real
  tree's lines pinned at the root. ws pinned at 364b14eb44 (busbar-transport-ws#13: its claim row
  states UNIT0_UPGRADE, as meta.rs does).
- transport_dropped_in_serves: no layer waits on an unlinked key now, so that leg serves without the
  wire; its RED is a second copy of the dropped-in wire refused as one key twice.
- the no-plugins and proto-deletion gates' notes on transport-tcp re-read under U7 (features unchanged).
… no kind is pending or skipped (ARCHITECT ruling W4B-Q1; RUN.md:119 "read pinned_exemplars, never skip")

Port of lane-w4c 7452fe6578 onto the P5 extraction line.

A plugin crate busbar pulls at a pinned rev (the git deps the root manifest pins: transport-tcp,
-http, -ws, secret-env/-file, the four export sinks, auth-oidc, auth-admin-tokens) is read from its
cargo checkout, laid over the tree at crates/<package> beneath any overlay
(xtask/src/gates/kind_isolation/pinned.rs). A pinned package is mounted only where no crate on disk
carries its name, so the in-tree store-memory, hooks-ranking and auth crates are read where they sit.
BUSBAR-1.6.0.md §9: every plugin lives in its own repo, so a kind whose crates are all extracted is
live through its pins, never dead and never skipped.

- PENDING_KINDS (secret, export) is gone with the skip it held; the dead-kind rule scores every kind,
  live while a crate of it resolves in the tree or at a pinned checkout. Its selftest arm is replaced
  by two: pins gone => dead-kind `secret`; a plant into the mounted secret-env crate is read by :vocab.
- A mounted crate carries its git+ source (CrateInfo.pinned) and owes no [workspace.members] line.
- root -> secret is granted (ARCHITECTURE_ALLOWED, SPEC_ALLOWED_MANIFEST): the root links secret-env
  and secret-file on its secret axis; the grant was absent only because the census could not see them.
- :wires: the both-ways witness row naming the cdylib twin (busbar-transport-tcp-plugin) also covers
  the logic crate it packages (busbar-transport-tcp); a green arm proves it, and the unnamed-wire arm
  names the tcp twin as the real table does.
- qa/kind-isolation.toml: 10 [[dep]] rows for edges that existed at the merge-base and were unmeasured
  while the census read only this tree (kernel -> secret-env/-file test-support stand-ins, shipped;
  kernel -> export-prometheus, kernel -> auth-oidc, core-admin -> export-prometheus, loader ->
  export-otlp/-prometheus and transport-tcp, test, not-allowed; loader -> secret-env test, tcb;
  busbar -> auth-oidc test, allowed). Appended at the end of the file so no existing finding's line moves.

Adapted, not copied: store-memory and hooks-ranking are still in-tree here, so the selftest plants
lane-w4c re-pointed to crates/busbar-store-memory / busbar-hook-ranking keep naming
crates/store-memory / busbar-hooks-ranking, and the kernel -> store-memory / -> hook-ranking rows
are not added (their edges were measured all along). The second-registration arm keeps this tree's
stdio plant and finding.

The pinned transport checkouts make `http` and `sse` needles on the unplanted tree, so
busbar-plane-llm × transport is a measured cell before any plant and the debt-free selftest subject
hides a rise in it: the two id-prefix RED cases ("a real transport `ws` reference in the same
dialect module", "an id-prefix literal outside a dialect module") move to the cell, as the loader's
auth-ABI mask already is, in matrix::tests::the_dialect_id_prefix_literal_is_masked_and_a_real_ws_still_counts
(over the pinned mount: masked prefix +0, real `ws` +1, prefix outside a dialect module +1).
… framer is the kind's entry file, transport.rs)
… meta.rs states the kind's tail and each claim's key)
… CARRIER is refused by name (interim guard until p2-transport-carrier), an unknown role fails; RED arms in role_tests
…rpc#6 merged into dev: the transport kind's skeleton; its busbar pin 7008c5f)
…e had re-selected windows-sys for four unrelated packages; cargo metadata --locked resolves this lock)
…s, tcp, grpc) and ws's dropped tokio/futures edges; nothing else re-resolved (cargo metadata --locked)
…p twin's role); the loader's TransportFacts carry the claims that hold a session; universal_needs finds its request/response framer by role and session and serves the neutral carrier beside it (no composes_over finders); a seam-ledger root test reads predev's store row (Q-STORE (B): no default flag)
… ws f8ba71d259 (#13), tcp 4af9e2ca8b (#13); grpc a5140eee3e (#6) already
…) and its SDK; HostTables.io; carrier frame bits on write/read; the loader's io slots, inline tickets and Plugin::call_inline
… (listen/accept, dial, exchange, a pending read the host's wake resumes, the host's guard as the RED arm); the interim carrier guard is gone; the SDK's host-buffer io calls
…les; every connection rides a carrier's slots (carrier.rs: two inline sides, the destination guard admitting exactly what it pinned); compose, listen, program and the legacy wire seam carry through it; stdio is a carrier over io.* (WIP: tests)
…rry), install the host's I/O, the root's own binds listen through the address carrier; loader states a carrier's ported claim (WIP)
… framer-knobbed test entries are framers over it; host I/O and carrier witnesses; a handed-up stream half-closes through the host
…-tcp p2-transport-carrier: tcp is a carrier over io.*); stdio's lock drops tokio and futures
… kept); the trait-to-kind-op coverage of the memory ABI (RED arm kept); the layout golden and the header generator learn the host I/O table
…table, HostTables.io and the carrier's frame bits (cargo xtask abi-header --write)
…h the published suite's carrier script (linked vs dropped in, exact counts, RED arms kept); contract docs cite the design in words; universal_needs serves the test carrier as the address carrier
… rows then the dropped-in doors (plugins dir sorted order), the first carrier serving a port is the address carrier (pinned by a registry test); the neutral frame door states FRAMER; the dropped-in tcp leg asserts every data worker accepts through the dropped-in carrier, also with the neutral framer declared first
@MattJackson
MattJackson enabled auto-merge October 6, 2026 21:49
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

promote into predev: BOARD @aeb7232c1: 1 failing test row(s), 12 DENY row(s)

Failing tests (1)

crate test step first panic
`` nextest xtask::cli::selftest_runs_every_registered_gates_red_proof test:workspace

DENY rows (12)

gate row detail
construction one-pick-site 3 production call site(s) of 'pick_among(' (ceiling 2): crates/busbar-kernel-egress/src/walk.rs:298; crates/busbar-llm/src/engine/exhaustion/fallback.rs:120; crates/busbar-llm/src/engine/pipeline.rs:8
kind-isolation kind-isolation:deps 4 finding(s), 101 shipped edge instance(s) over 33 class(es), 101 declaration(s); 63 '[[dep]]' row(s), 0 question(s): unlisted-dep-edge cleanliness -> plugin-tooling busbar-core-admin -> busbar-plugin
kind-isolation kind-isolation:test-deps 5 finding(s), 46 test edge instance(s) over 25 class(es), 46 declaration(s); 32 '[[dep]]' row(s), 0 question(s): unlisted-dep-edge cleanliness -> legacy busbar-core-admin -> busbar-a2a is a test edge
kind-isolation-ship kind-isolation:deps 18 finding(s) over 101 shipped edge(s): ship-edge cleanliness -> plugin-tooling busbar-core-admin -> busbar-plugin-loader is 'not-allowed': the architecture grants no cleanliness -> plugin-tooling edg
kind-isolation-ship kind-isolation:test-deps 22 finding(s) over 46 test edge(s): ship-edge cleanliness -> export busbar-core-admin -> busbar-export-prometheus is 'not-allowed': the architecture grants no cleanliness -> export edge, and the ship
kind-isolation-ship kind-isolation:faces 4 finding(s) over 43 crate(s): foreign-entry crates/busbar busbar is kind 'root' and implements 'Store' 2 time(s) in shipped source — the entry face of kind 'store'. A trait implementation is a claim
kind-isolation-ship kind-isolation:testkit 16 finding(s) over 24 crate(s): no-battery kind:export no battery for kind export — none of its 4 crate(s) runs a shared conformance battery (no 'testkit' dev-dependency, no tests/conformance.rs). C
kind-isolation-ship kind-isolation:legacy-drain 5 finding(s): transitional-live qa/kind-isolation.toml 'busbar-a2a -> busbar-core-admin' (legacy drain: the retiring A2A engine's tests drive the admin surface that drained into the cleanliness tier)
kind-isolation-ship kind-isolation:control-path 74 finding(s) over 3 control surface(s): upstream crates/busbar-core-admin/src/admin_codec/meta.rs:84 busbar-core-admin names 'egress' — a control surface has no upstream to reach, so the vocabulary o
ship-ready ship-ready:ship-twin 'kind-isolation-ship' is not green: kind-isolation:deps (a dependency the architecture does not grant is still in the graph); kind-isolation:test-deps (a dependency the architecture does not grant is
instance-noun-neutrality instance-noun-neutrality:voice tracked known-debt census — 8: crates/busbar-plane-llm/src/codec/gemini/handler.rs×7 [cross-plugin] | crates/busbar-plane-llm/src/codec/gemini/tests/handler_tests.rs×2 [cross-plugin] | crates/busbar
structure-lint structure-lint:plane-dup:unledgered 24 finding(s): PLANE-DUPLICATE (module): 'config.rs' — a2a:crates/busbar-a2a/src/a2a/config.rs decisions:crates/busbar-plane-decisions/src/config.rs mcp:crates/busbar-mcp/src/mcp/config.rs voice:crate

Judged against base dc60715e8: 0 new red, 0 worse, 7 standing (excused).

tests passed: 24687, failed: 0. Run: https://github.com/GetBusbar/busbar/actions/runs/37586187974 . Artifact verdict-aeb7232c1bbea1e7330e5ee865d5ef1e9a28cead (failures.json, junit.xml, raw.log; 90 days).

…rmance_suite!) as a live battery entry, so a plugin that runs busbar's suite is not 'battery-ignored' (stdio, decisions); linked-dropped-features' unified-deps plant gives the plugin a registry dep of its own first (the stdio carrier has none to share)
@MattJackson
MattJackson added this pull request to the merge queue Oct 6, 2026
…-end case: a whole construction run and a whole kind-isolation-ship run, on one thread; 9 294 units on 5172cd4, predev 8 415-8 493), allowed at the declared slack
@MattJackson
MattJackson removed this pull request from the merge queue due to a manual request Oct 6, 2026
@MattJackson
MattJackson added this pull request to the merge queue Oct 6, 2026
# Conflicts:
#	crates/busbar-contract/include/busbar_plugin.h
#	crates/busbar/src/root/tests/serve.rs
#	crates/busbar/src/root/tests/serve_door.rs
@MattJackson
MattJackson removed this pull request from the merge queue due to a manual request Oct 7, 2026
… (file, webhook) carry [[dep]] rows like otlp's and prometheus's (measured once the census reads pinned checkouts); test-deps back to predev's 5
@MattJackson
MattJackson enabled auto-merge October 7, 2026 02:06
@MattJackson
MattJackson added this pull request to the merge queue Oct 7, 2026
# Conflicts:
#	crates/busbar-transport-stdio/tests/conformance.rs
@MattJackson
MattJackson removed this pull request from the merge queue due to a manual request Oct 7, 2026
@MattJackson
MattJackson enabled auto-merge October 7, 2026 04:09
@MattJackson
MattJackson added this pull request to the merge queue Oct 7, 2026
# Conflicts:
#	crates/busbar-core-connector/src/lib.rs
#	crates/busbar/src/root/tests/serve_door.rs
#	crates/plugin-loader/src/dispatch/worker.rs
@MattJackson
MattJackson removed this pull request from the merge queue due to a manual request Oct 7, 2026
@MattJackson
MattJackson enabled auto-merge October 7, 2026 06:04
…ned (Q-P4-5: "read resumed" pins 0, its op's first invocation is "read pending"); kind-isolation verdict: with the pinned checkouts in the census (W4B-Q1) the plant's busbar_transport_http path is the third undeclared crate path
…ransport-carrier: DoorFacts and TransportFacts carry both the carrier's port and the claims' status rows; a write after the far side's frames ended goes through put (the framer's, per 4l); the 4l framed-stream tests state the framer role; DATA_CARRIER stays for the framed stream only
…sport-carrier: the one dispatcher call keeps a kind's flag bits and sets only the resume bit; the decisions linked-entry row is #497's, the sse wire row stays struck (one entry per transport); predev's services() passes the host's io table; C header regenerated over the merged ABI
MattJackson added a commit that referenced this pull request Oct 7, 2026
Conflict by hand: connector tests/support.rs imports (#502's DEST_PROGRAM and READ_END_OF_FRAME
beside PIECE_END). The carrier-as-itself reads (wire.rs, compose.rs) are frames, never a stream's
end (end: false); the empty-message test dials through the support via (#502's dial signature).
MattJackson added a commit that referenced this pull request Oct 7, 2026
… states FRAMER), and the dialled empty-messages exchange is framed over a carrier entry (#502: a dial is carried)
MattJackson added a commit that referenced this pull request Oct 7, 2026
…aker-neutral-ts: the tree is #502's with this lane's own fault commits re-laid

Every conflict takes #502's side, and the hunks this branch's merge commits brought in
(p3-suite-ext, p2-transport-stack 56f1edf in 0f755e9) give way to the versions predev
carries. Re-laid on top, hunk for hunk: 00259bd and b7360c0 (the breaker names no plane
and no header), 0484f87 (the kernel-breaker x plane KI cell struck), 6f1cce5 (the breaker
reads the transport's fault reading; the bands leave classify.rs and port.rs), the WireFault
re-export of 62e1dcc, 8a16ac2 (FaultRow, TransportTail::fault_rows, FramePiece::fault,
check_fault_rows/check_fault_cover with their RED arms, layout golden), 487b926 (the walk
records from the reading), 8b24168 (the program carrier states no reading), 8c8bbf7
(a zero byte reads FAULT_NONE). 0c61fbd is already on predev as #480's own form.

#502's own literals gain the field: the carrier-as-itself Got in compose.rs and wire.rs
states FAULT_NONE, the stdio carrier's tail states an empty fault table, and the framed-stream
test framer and neutral frame door fill fault where the padding was. C header regenerated.
MattJackson added a commit that referenced this pull request Oct 7, 2026
…ins and the fault table (http 6a10f17207, ws e00327bdc0, grpc 694f1849af); tcp stays at #502's carrier pin eb334dd6e9

Cargo.lock: transport-http's dev names no socket-capable crate, so futures, hyper-util, tokio
and tokio-util leave its dependency list (nothing re-resolved; the same four edges 154a0bb dropped).
MattJackson added a commit that referenced this pull request Oct 7, 2026
…=128) into p2-breaker-neutral-ts: the transport check reads PIECE_END beside the fault reading

The one conflict is check.rs's import list (FaultRow, FAULT_CALLER, FAULT_HARD beside PIECE_END).
sdk_hyper's fill states PIECE_END after PIECE_STREAM_FAILED and the fault byte from fault_of, and
the carrier-as-itself Got literals carry both fault: FAULT_NONE and end: false. C header
regenerated.
MattJackson added a commit that referenced this pull request Oct 7, 2026
…IECE_END (tcp b3047ce874, ws 2fd38617db, http 0ff0e022e8, grpc 3430de60cf: each branch p2-breaker-neutral-ts-pin = dev + p2-piece-end-b, repinned to busbar 2add33a)

tcp is #502's carrier (dev 198a609) with its tail's empty fault table; ws/http/grpc are dev (fault
table, SEAM-4l) merged with the PIECE-END lane's framer commits. No dependency changes: the lock
moves five sources only.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant