Skip to content

feat(host): add the agent-device host front-end - #3271

Open
vkuprin wants to merge 3 commits into
callstack:mainfrom
vkuprin:feat/host-front-end
Open

vkuprin wants to merge 3 commits into
callstack:mainfrom
vkuprin:feat/host-front-end

Conversation

@vkuprin

@vkuprin vkuprin commented Oct 7, 2026 •

Copy link
Copy Markdown

Summary

Adds agent-device host, the Host front-end from ADR 0021 §3. It runs as its own process, starts or reuses the local HTTP daemon, and serves it to remote verification workers through @agent-device/proxy. The proxy command is untouched.

agent-device host --host 0.0.0.0 --port 8443 --tls-cert ./cert.pem --tls-key ./key.pem

Workers authenticate with one service credential at <state dir>/host/service-credential.json (directory 0700, file 0600). Host writes it once it is serving, prints the token that one time, and reuses it after restarts.

Host refuses to start, with a typed reason and before any daemon starts, when:

  • the credential is malformed, a link, or open to group or others;
  • a TLS file is unreadable, or the certificate and key don't load as a pair;
  • it would serve plain HTTP off loopback.

A wildcard bind advertises the machine's hostname; startup prints the exact worker command.

Closes #3265. 22 files, 991 gross lines, with chore(gates) last.

Validation

Tested commit 025fb41b9:

  • pnpm check:affected --run: all pass except a flaky affected-selector test (ENOTEMPTY on temp cleanup; unchanged from main). The 17 checks after it pass when run separately.
  • Tests cover:
    • hostCommand end to end against a stub daemon: public health, 401 for a wrong token, and an authenticated request reaching the daemon with the daemon token, across a restart that prints the token only once;
    • 404 on unserved routes;
    • each startup refusal, before any daemon starts;
    • an HTTPS round-trip.

No device-facing change.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files

Reply to a comment to ask cubic a question or push back. It learns from your replies.

Turn on auto-fix | Re-trigger cubic

Comment thread src/commands/schema/cli-help.ts Outdated
Comment thread src/cli/host/local-daemon.ts
Comment thread src/cli/commands/host.ts Outdated
Comment thread scripts/__tests__/help-conformance-topic-coverage.test.ts Outdated
Comment thread scripts/integration-progress-model.ts Outdated
Comment thread website/docs/docs/remote-proxy.md Outdated
Comment thread src/cli/commands/host.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread src/cli/commands/host.ts
Comment thread src/commands/schema/cli-help-topics.test.ts Outdated
@thymikee

thymikee commented Oct 7, 2026

Copy link
Copy Markdown
Member

The code looks good at 328366c, but I cannot call it fully proven yet because no test or run drives hostCommand end to end against a real daemon. The 401, 404, restart and HTTPS tests call createHostServer with a fake upstream, and host.test.ts covers only the refusal paths. The wiring from ensureLocalHttpDaemon to createHostServer to listenOnTcp holds when traced, but nothing executes it. This is not a device-facing change, so a live run is not required. One short test that starts hostCommand against a real or stub daemon and checks GET /health and an authenticated route would close the gap. CI is green, with 1 check and none failing at this head, and no conflicts were reported.

Not blocking, and you can take or leave these: readHostTlsMaterial only checks that the files are readable, so a malformed PEM or a cert/key mismatch throws from https.createServer after ensureLocalHttpDaemon has already started or reused the daemon. If it called tls.createSecureContext({cert,key}) and mapped a failure to a typed reason such as host-tls-invalid, every TLS refusal would run before the daemon starts. Also, 'host' is added by hand to three command lists in src/cli.ts, next to 'proxy'. A declared trait in packages/command-registry/src/registry.ts for local front-end commands that never route remotely could drive all three checks.

Is there a smaller design? I looked and found none. Host reuses createDaemonProxy, createDaemonProxyRequestListener, publishDurableFileSync and openVerifiedFileForRead, and it moves proxy's startup helpers into one shared module. The only optional trim is to inline the 28-line createHostServer into host.ts, or let createDaemonProxyServer accept TLS options.

On the open threads: the cubic P3 thread on the help text assertion still applies, since the test would stay green if (401 without it) or the 404 wording were removed. The cubic P2 thread on the TLS key readability check does not apply. #3265 only requires host-tls-unreadable for files Host cannot read, and common layouts such as the group-readable 0640 ssl-cert setup would break if Host refused them. You can resolve that one.

Nothing blocks merge once you decide whether to add the end-to-end check.

@vkuprin
vkuprin force-pushed the feat/host-front-end branch from 328366c to caaa3a9 Compare October 7, 2026 03:38
Add `agent-device host`, the Host front-end from ADR 0021 §3. It runs as
its own process, starts or reuses the local HTTP daemon, and serves it to
remote verification workers through the daemon proxy.

Workers authenticate with one persistent service credential. Host creates
it on first start at <state dir>/host/service-credential.json (directory
0700, file 0600) and reuses it after every restart. A malformed or
group/other-readable file stops startup with a typed reason, and so does a
TLS file Host cannot read. Both checks run before any daemon starts.
--tls-cert and --tls-key serve HTTPS. A wildcard bind advertises the
machine's hostname, since workers cannot dial 0.0.0.0.

The proxy command behaves as before. Its daemon startup and listen helpers
move into a module both commands use.

Closes callstack#3265
- Host checks that the TLS certificate and key load together, and that
  the key is mode 0600, before any daemon starts. A bind other than
  loopback without TLS is refused (host-tls-required), so the service
  token never travels in cleartext.
- A new credential reaches disk only once Host is serving, so a start
  that fails earlier never hides the token from the next one. A
  credential another start wrote first is refused (host-credential-raced).
- A credential that is a link or unreadable gets a typed reason, and
  platforms without POSIX ownership skip the mode check.
- The advertised URL keeps the host name the operator bound to, and the
  worker command in the startup output names the real URL and token.
- The proxy command keeps its original code. Host's daemon and listen
  helpers live in src/cli/host/local-daemon.ts.
- The host help topic moves into its own module.
Add `host` to the reviewed device-claim policy set, give the
--tls-cert/--tls-key flags their own Host bucket in the integration
progress model, list `hostCommand` with the dynamically loaded CLI handlers
in the fallow production exemptions, and waive the operator-facing `host`
help topic from the help benchmark.
@vkuprin
vkuprin force-pushed the feat/host-front-end branch from caaa3a9 to 025fb41 Compare October 7, 2026 05:26
@vkuprin

vkuprin commented Oct 7, 2026 •

Copy link
Copy Markdown
Author

Thanks for the review!

One short test that starts hostCommand against a real or stub daemon and checks GET /health and an authenticated route would close the gap.

Added it in host.test.ts. It runs hostCommand against a stub daemon and checks public /health, a 401 for a wrong token, and an authenticated RPC that reaches the daemon with the daemon token. It also restarts Host on the same state dir and checks the token is printed only the first time. After that the 401 and restart tests in host-server.test.ts didn't add anything, so I removed them.

If it called tls.createSecureContext({cert,key}) and mapped a failure to a typed reason such as host-tls-invalid, every TLS refusal would run before the daemon starts.

Done. A bad PEM or a cert/key mismatch now fails with host-tls-invalid before any daemon starts.

A declared trait in packages/command-registry/src/registry.ts for local front-end commands that never route remotely could drive all three checks.

Agree it's nicer, but the 3 lists hold different sets of commands (connect, daemon, plugins, device, session and a few more), so the trait would have to cover all of them. I'd rather do that as a separate refactor than grow this PR.

The only optional trim is to inline the 28-line createHostServer into host.ts, or let createDaemonProxyServer accept TLS options.

Kept it separate. #3272 wraps it with the route policy, and its tests call it directly.

the cubic P3 thread on the help text assertion still applies

Fixed, the test matches the whole sentence now, (401 without it) and the 404 part included.

The cubic P2 thread on the TLS key readability check does not apply.

Agreed. I'd added a 0600 check because of that thread, now it's gone, so 0640 ssl-cert keys work.

Also rebased on main after #3262, so Host imports only from @agent-device/proxy now.

@thymikee

thymikee commented Oct 7, 2026

Copy link
Copy Markdown
Member

Thanks for the update. Most of the earlier review is now fixed, but one problem remains at 025fb41. The delta reverted proxy.ts to its main version, so src/cli/host/local-daemon.ts now repeats proxy.ts's private helpers. formatHostForUrl, formatOutputValue and waitForever are verbatim copies. resolveLocalDaemonBaseUrl, listenOnTcp and resolveLocalHttpDaemonSettings differ only by a command-name parameter. Proxy and host both front the local HTTP daemon, so each now owns its own copy of the loopback upstream URL, the env mask, bind and listen, and URL formatting. A fix to one, such as the wildcard-advertise fix this PR made for host, will silently miss the other. I approved the earlier head because these helpers were shared. The rule is that each local-daemon front-end helper has one definition and every front-end command imports it. Please restore the 328366c shape. proxy.ts should import resolveLocalHttpDaemonSettings, ensureLocalHttpDaemon, listenOnTcp, formatHostForUrl, formatOutputValue and waitForever from src/cli/host/local-daemon.ts, or from a neutral src/cli/commands location, and its private copies should go. That is an import-only change for proxy, and its existing tests should cover it. Please also drop "proxy is untouched" from the PR body. Not blocking: the ensureDaemon stub at host.test.ts:129 goes through as never instead of a typed partial result, and the 49cfee5 commit message still claims a 0600 key-mode check that the head does not have, so take or leave typing the stub and fixing the message when you squash.

CI is green at 025fb41, with one check and none failing. I did not run host.test.ts locally, and I judged the duplication by reading the code, not by running a mutation. Before merge, proxy.ts needs to import the shared helpers again.

On the other open threads, the wildcard-bind advertise thread, the credential-before-daemon-start thread, the host-tls-unreadable thread, the help text threads, the docs thread, the options bucket thread and the topic-coverage waiver thread are all fixed at this head. The key-mode thread is benign, since #3265 needs only host-tls-unreadable and 0640 ssl-cert key layouts must keep working. You can resolve all of them. Cubic has not reviewed this head, so this covers threads from earlier heads only.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Host: agent-device host front-end command and persistent service credential

2 participants