Skip to content

Boiler blate code to enable getLedgerEntries, simulateTransaction, sendTransaction on v2 (#884). Wiring comes in (#889) - #903

Merged
karthikiyer56 merged 8 commits into
feature/full-historyfrom
karthik/884-captive-core-endpoints
Jul 31, 2026
Merged

Boiler blate code to enable getLedgerEntries, simulateTransaction, sendTransaction on v2 (#884). Wiring comes in (#889)#903
karthikiyer56 merged 8 commits into
feature/full-historyfrom
karthik/884-captive-core-endpoints

Conversation

@karthikiyer56

@karthikiyer56 karthikiyer56 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

TL;DR

  • Nothing becomes callable here. v2 still binds no port; serveReads is a no-op stub. This PR builds the plumbing the three captive-core endpoints need. Registering them is Wire the v2 binary's JSON-RPC handlers #889, and their handlers also need the router-backed store.LedgerReader (LedgerReader + LedgerReaderTx adapter #885EventReader adapter #887).
  • v2 gets a real host.Daemon over its own captive core (new internal/rpcv2/corestate), so Wire the v2 binary's JSON-RPC handlers #889 can run v1's existing internal/preflight and internal/ledgerentries code unmodified.
  • host.Daemon.GetCore() *ledgerbackend.CaptiveStellarCore narrows to CoreVersion() string — the whole SDK core object was being handed out for one version string.
  • Config grows three method tables, six [ingestion] core-HTTP keys, and [service.preflight] — including the flag machinery's first bool.
  • ⚠️ This diverges from v1 on purpose. v1 rejects a captive-core file that disagrees about core's HTTP settings; v2 overrides it and warns. Details immediately below.

⚠️ Divergence from v1: core's HTTP settings

v1 and v2 both take these four settings from RPC config and write them into the captive-core toml. They differ in what happens when the operator's captive-core file also names them.

v1 v2 (this PR)
File silent RPC's value is used RPC's value is used
File agrees with RPC config starts starts
File disagrees startup fails RPC config wins, warning logged
Who enforces it the SDK, inside NewCaptiveCoreTomlFromFile this daemon, before the SDK sees the params
  • v1's rejection is toml.go's validate: HTTP_PORT in captive core config file: %d does not match passed configuration (%d). It is never a silent override.
  • Why v2 differs: core_http_query_thread_pool_size defaults to NumCPU(). Under a conflict check, a core file with QUERY_THREAD_POOL_SIZE = 8 and no [ingestion] value starts on an 8-CPU host and refuses to start on a 16-CPU host — the operator set one value, and a default disagreed with them. Only this key is machine-dependent; the other three compare against fixed defaults.
  • Second reason: the SDK's three query-server mismatch branches format their error with params.PeerPort, which this daemon never sets, so a disagreeing file panics rather than reporting the conflict. v2 passes no HTTP params at all, which makes those branches unreachable.
  • What operators must know: a value in the captive-core file no longer has any effect on these four keys. Carrying a v1 file with HTTP_PORT = 11625 moves core to 11626, so firewall rules or monitoring scrapes pointed at the old port need updating. The warning names both values:
captive_core_config sets HTTP_PORT = 11625, but [ingestion].core_http_port owns core's
HTTP settings — running core with 11626. Remove HTTP_PORT from the captive-core file
to silence this.
  • NETWORK_PASSPHRASE is unchanged and still owned by the captive-core file — the ownership split is deliberate, not blanket.

What a running daemon actually does differently

Change Operational effect
Core's HTTP ports are written into the live core's toml The captive core this daemon launches binds 11626 (admin) and 11628 (query). Both must be free on the host.
[ingestion] overrides the captive-core file A file setting HTTP_PORT, HTTP_QUERY_PORT, QUERY_THREAD_POOL_SIZE or QUERY_SNAPSHOT_LEDGERS is overridden, with a warning naming both values. Startup is not blocked.
New config validation Bad ports, a sub-1ms core_request_timeout, a malformed core_url, a local core_url that doesn't match core_http_port (or uses https — the local core speaks plain HTTP), a core_url port past 65535, or a zero preflight worker count all fail startup.
corestate.New Builds two HTTP clients and reads stellar-core version once, under a 5s timeout tied to the daemon's context (plus a 1s WaitDelay, so a wrapper's child holding stdout can't stall past the kill).
Preflight worker pool Starts worker_count idle goroutines and registers 5 Prometheus collectors.

Inert until #889 — the request paths. Nothing calls CoreClient(), FastCoreClient(), LedgerEntryGetter() or pool.GetPreflight(), so the clients never issue a request and the pool never gets a job.

Why a Daemon and not three handlers

Endpoint Path to core What core does
sendTransaction CoreClient → admin port validation + flooding; the only real submit path
getLedgerEntries FastCoreClient → query port bucket-list entry reads
simulateTransaction preflight pool → Rust libpreflight in-process the Rust host fetches each entry back through FastCoreClient

All three already sit behind internal/host's interfaces. This PR supplies the missing implementation; #889 assembles the method table over #885's reader.

New config keys (15 keys, all with v1 counterparts)

[service.methods.<method>] — three new tables, same MethodConfig shape:

Key Default v1 flag
sendTransaction.queue_limit 500 request-backlog-send-transaction-queue-limit
sendTransaction.max_execution_duration "15s" max-send-transaction-execution-duration
simulateTransaction.queue_limit 100 request-backlog-simulate-transaction-queue-limit
simulateTransaction.max_execution_duration "15s" max-simulate-transaction-execution-duration
getLedgerEntries.queue_limit 1000 request-backlog-get-ledger-entries-queue-limit
getLedgerEntries.max_execution_duration "5s" max-get_ledger-entries-execution-duration

[ingestion] — core's HTTP surface, placed with the process it configures:

Key Default v1 flag
core_http_port 11626 stellar-captive-core-http-port
core_url http://localhost:{core_http_port} stellar-core-url
core_request_timeout "2s" stellar-core-timeout
core_http_query_port 11628 stellar-captive-core-http-query-port
core_http_query_thread_pool_size NumCPU …-http-query-thread-pool-size
core_http_query_snapshot_ledgers 4 …-http-query-snapshot-ledgers

[service.preflight] — serving policy, like [service.fee_stats]:

Key Default v1 flag
worker_count NumCPU preflight-worker-count
worker_queue_size NumCPU preflight-worker-queue-size
enable_debug true preflight-enable-debug
  • Every key is also a --dotted.path CLI flag, free from the reflection-derived flag set.
  • core_request_timeout joins the ≥1ms nanosecond-typo guard next to [backfill.bsb].retry_wait.
  • core_url is derived after flags are overlaid, so --ingestion.core_http_port=31626 alone moves the submission URL with it.
  • The sample toml sets enable_debug = false — a deliberate departure from the true default, since the sample is a production starting point.
What changed where
Area Change
internal/host/host.go, noOpDaemon.go GetCore()CoreVersion() string; the ledgerbackend dependency leaves the interface
internal/methods/get_version_info.go reads daemon.CoreVersion() inside the handler closure
internal/rpcv1/daemon/metrics.go one-line adapter to d.core.GetCoreVersion()
internal/rpcv2/corestate/ (new) Daemon: two *stellarcore.Clients at the two ports, registry/namespace, version string. LedgerEntryGetter(store.LedgerReader) is the seam #889 calls, pinning entry reads to the ledger v2 last committed
internal/rpcv2/config/config.go three method tables, [service.preflight], six [ingestion] core keys, shared NumCPU() helper
internal/rpcv2/config/flags.go kindBool in leafKind/BindFlags/setLeaf, GetBool on FlagOverrides
internal/rpcv2/config_validate.go new validateIngestion + validatePreflight; the three methods join validateService
internal/rpcv2/daemon.go resolveCore returns resolvedCore{live, backfill, networkPassphrase, binaryPath}; both openers come from ONE read of the captive-core file; corestate + preflight pool built once, outside the supervised loop
Makefile, docker/Dockerfile.rpcv2 build-rpc-v2: build-libs; the rust stage's now-wrong "not needed yet" comments are gone
sample toml + design doc all new keys documented; "NOT here (by design)" is down to friendbot_url
Why core's HTTP ports land on the live core only
  • runDaemonWith hands one CoreOpener to two consumers: the live ingestion loop, and (with no lake) captiveSource, which starts a fresh bounded-replay core per chunk, in parallel.
  • Ports in that shared toml would have every backfill core contend on binding them. Offline/catchup mode zeroes HTTP_PORT but not HTTP_QUERY_PORT, so the collision is real.
  • Hence two openers: applyHTTPServers sets all four settings on the live toml, disableHTTPServers clears them on the backfill toml.
  • Neither is passed through CaptiveCoreTomlParams. The SDK treats params as a cross-check rather than an override, and its three query-server mismatch branches format the error with params.PeerPort — always nil here — so they panic instead of reporting. Passing no HTTP params leaves those branches unreachable.
Design decisions
Decision Reasoning
[ingestion] overrides the captive-core file's four HTTP keys, with a warning, rather than rejecting the config • one owner per setting, so operators don't reconcile two files
core_http_query_thread_pool_size defaults to NumCPU, so a conflict check would refuse to start on a host with a different CPU count than the one the config was written on
• the warning keeps the override visible: a v1 file with HTTP_PORT = 11625 may have firewall rules pointed at 11625
Ports must be 1..65535 and must differ; v1's "0 disables core's HTTP server" is rejected • v2 always serves sendTransaction off the admin port and getLedgerEntries off the query port
• a disabled server is a broken deployment, not a mode
A local core_url must name core_http_port • this daemon starts the core it submits to, so any other local port is not that core — often another daemon's captive core, which would accept the transaction
• non-local core_urls are left alone; that's the reason to set one
getVersionInfo calls daemon.CoreVersion() per request • the version is only known once core starts, which is after handlers are built
Plain stellarcore.Clients; v1's CoreClientWithMetrics txsub summaries not reproduced • that wrapper lives in internal/rpcv1/daemon, not a shared package
TODO(#889) at the client, since #889 is what wires sendTransaction
A failed stellar-core version lookup logs and yields "" • cosmetic response field; mirrors the SDK's own setCoreVersion
• bounded by exec.CommandContext + 5s, so a wrapper script or stalled mount can't freeze startup before anything is logged
resolveCore returns two openers instead of widening CoreOpener • the interface stays one method, so all three existing test fakes are untouched
corestate + preflight pool are locals in runDaemonWith with defer Close() • "construction only" per the issue
• built outside supervise, since re-registering the pool's collectors panics
Two http.Clients, one per core server • not pool isolation — both share http.DefaultTransport, whose pool already keys connections by host:port
• kept separate so #889 can wrap the submission client alone for v1's txsub metrics
enable_debug is an ordinary pflag bool --service.preflight.enable_debug=false turns it off; a bare flag sets true
• the flag's own false default never leaks in, since ApplyFlags reads only flags the user set
Fixes from the branch code review
Finding Verdict Resolution
Passing HTTPQueryServerParams makes an SDK path reachable that panics on a nil *params.PeerPort CONFIRMED — real SIGSEGV at ledgerbackend/toml.go:728 with a core file declaring HTTP_QUERY_PORT = 11111 The HTTP settings are no longer passed as params at all; they are written onto each generated toml, leaving those branches unreachable
The backfill opener was only portless because the fixture declared no ports: the SDK keeps a file-declared HTTP_QUERY_PORT, and CatchupToml() clears HTTP_PORT but never the query port CONFIRMED — the generated catchup toml contained HTTP_QUERY_PORT disableHTTPServers clears all four fields, the same mechanism CatchupToml uses. Test asserts on the marshalled toml, direct and catchup
core_url had no form validation, so core.internal:11626 started fine and failed per request while health looked green CONFIRMED validateIngestion requires an http/https scheme and a host, and rejects a loopback core_url whose port isn't core_http_port
The conflict check compared against post-default values, so QUERY_THREAD_POOL_SIZE in a core file could start on one host and fail on another purely by CPU count CONFIRMED Replaced with override-plus-warning; no default is compared against anything
readCoreVersion ran stellar-core version with no context or timeout, on the startup path before any log line CONFIRMED exec.CommandContext with a 5s cap, bound to the daemon's context
"Separate http.Clients so a slow submission cannot tie up a connection the query path needs" — both share http.DefaultTransport, so that isn't what the split buys CONFIRMED Comment and test rationale corrected to what actually holds
v1's two txsub metrics aren't published by the unwrapped submission client Real, out of scope TODO(#889) — needs v1's wrapper lifted out of internal/rpcv1/daemon first
Fixes from the Copilot review
Finding Resolution
CommandContext kills only the direct process; a wrapper's child holding stdout keeps Output waiting past the 5s timeout cmd.WaitDelay = time.Second forces the pipes closed after the kill
The hanging-binary test cancelled its context before New, so the script never started and the timeout was never exercised Live 100ms deadline so the script starts and is killed; a second test covers a wrapper whose backgrounded child holds stdout
isLoopbackHost missed LOCALHOST and localhost. Case-insensitive compare, trailing root dot stripped
https://localhost:11626 passed validation but the local core speaks plain HTTP, so every submission would fail its TLS handshake Rejected: https to a loopback core_url is a startup error
An explicit remote port past 65535 (http://core.internal:70000) parsed fine and failed only at dial time Explicit core_url ports are range-checked 1–65535

Testing

What Result
go build ./... passes
go test ./cmd/stellar-rpc/... (incl. the ~3min goleak-guarded rpcv2 suite) passes
golangci-lint run ./cmd/stellar-rpc/... clean, bar one local-only artifact: my 2.12.2's gosec no longer flags int → uint, so nolintlint calls the //nolint:gosec on uint(runtime.NumCPU()) unused. CI's pinned 2.11.3 needs the directive; v1 carries the identical line at rpcv1/config/options.go:40 with the same skew
make build-rpc-v2 links; nm stellar-rpc-v2 shows _preflight_invoke_hf_op, so the rust bindings are really in the binary

New tests:

  • corestate — client URLs/timeouts per port, registry identity, incomplete-config rejection, version read from a fake stellar-core version binary, empty on a missing binary, a live deadline proving a hanging binary is killed rather than waited out, and a wrapper whose backgrounded child holds stdout open (the WaitDelay case). An httptest server proves LedgerEntryGetter hits the query port asking for the daemon's own latest ledger.
  • rpcv2 — the live-vs-backfill port split (including when the core file declares ports itself), four override cases asserting the live toml's value plus the warning, no warning when the file already agrees, injected-opener-serves-both-roles, and the [ingestion]/[service.preflight]/core_url rejection cases.
  • config — defaults and cascade participation for the three new tables, preflight defaults incl. an explicit false, core-HTTP defaults, core_url derivation, and the bool flag override.

)

These three endpoints read CURRENT ledger state, which lives in captive core's
bucket list rather than in v2's stores. The existing implementations
(internal/preflight, internal/ledgerentries) are already written against
internal/host's interfaces, so v2 needs one concrete Daemon over its own captive
core — not reimplemented handlers.

host.Daemon.GetCore() handed out the whole SDK core object for the sake of one
version string, which v2's stream deliberately hides (it runs a fresh core per
supervised run). Narrowing it to CoreVersion() lets both daemons answer from the
configured binary path alone.

Core's HTTP ports go on the LIVE core's toml only. A no-lake backfill starts one
bounded-replay core per chunk in parallel, and offline mode drops HTTP_PORT but
not HTTP_QUERY_PORT — so a shared toml would have those cores contending on the
query port.
Setting HTTPQueryServerParams for the first time made an SDK path reachable that
PANICS instead of reporting a conflict: its query-server mismatch branches format
the error with params.PeerPort, which this daemon never sets. Peek the four
core-HTTP keys out of the captive-core file and reject a disagreement here, with
a message naming both sides, before the SDK ever compares them.

The backfill toml was only portless because no test fixture declared ports. The
SDK keeps a file-declared HTTP_QUERY_PORT, and the bounded-replay config clears
only HTTP_PORT — so parallel replays would have contended on the query port after
all. Clear the fields outright rather than relying on absent params.

core_url gains form validation: a missing scheme otherwise started fine and then
failed on every submission while the daemon still reported healthy.
The design doc's config tables had drifted into implementation notes — flag
machinery internals, an SDK panic, validation rules — the opposite of what that
doc is for. It now lists the keys and the one design decision behind them: core's
HTTP ports go on the live core only.

Code comments say the same things in fewer, plainer words. Config struct fields
keep their explanations, since the schema is what needs describing; test
functions lose docstrings their names already carry.
#772 is the epic; the read server is #889's work. Markers that attributed the
work to #772 sent a reader to a ticket that does not describe it. References to
the v1-to-v2 cutover itself still point at #772, which is what owns that.
v2 serves nothing today, and #889 alone does not change that: the handlers it
registers need the router-backed store readers, so serving arrives across the
epic's sub-tasks rather than at one ticket.
@karthikiyer56 karthikiyer56 changed the title Serve getLedgerEntries, simulateTransaction, sendTransaction on v2 (#884) Boiler blate code to enable getLedgerEntries, simulateTransaction, sendTransaction on v2 (#884). Wiring comes in (#889) Jul 28, 2026
@karthikiyer56
karthikiyer56 requested a review from a team July 28, 2026 19:31
The captive-core file's HTTP_PORT, HTTP_QUERY_PORT, QUERY_THREAD_POOL_SIZE
and QUERY_SNAPSHOT_LEDGERS are now overridden with a warning rather than
refused. A conflict check compared post-default values, so a core file with
QUERY_THREAD_POOL_SIZE set and no [ingestion] value started on one host and
failed on another purely by CPU count. The four settings are written onto
each generated toml instead of passed as CaptiveCoreTomlParams, which also
keeps the SDK's nil-PeerPort panic branches unreachable.

Also:
- reject a loopback core_url whose port is not core_http_port; it can only
  reach some other process, often another daemon's captive core
- bound the stellar-core version read with a context and a 5s timeout
- turn enable_debug off in the sample config

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds v2 plumbing for captive-core-backed RPC methods without exposing the handlers yet.

Changes:

  • Adds v2 core clients, preflight workers, and live/backfill core configuration.
  • Extends configuration, validation, flags, documentation, and tests.
  • Narrows the host daemon interface and enables Rust preflight linkage.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
Makefile Links v2 with Rust libraries.
design-docs/full-history-streaming-workflow.md Documents new configuration.
cmd/stellar-rpc/rpcv2/sample_config_test.go Tests sample defaults.
cmd/stellar-rpc/rpcv2/rpc-v2-sample-config.toml Adds example settings.
cmd/stellar-rpc/internal/rpcv2/startup.go Updates server references.
cmd/stellar-rpc/internal/rpcv2/helpers_test.go Adds log capture helper.
cmd/stellar-rpc/internal/rpcv2/health.go Updates server references.
cmd/stellar-rpc/internal/rpcv2/daemon.go Wires core state and preflight.
cmd/stellar-rpc/internal/rpcv2/daemon_test.go Tests core opener behavior.
cmd/stellar-rpc/internal/rpcv2/corestate/corestate.go Implements v2 host daemon.
cmd/stellar-rpc/internal/rpcv2/corestate/corestate_test.go Tests core-state integration.
cmd/stellar-rpc/internal/rpcv2/config/flags.go Adds Boolean flags.
cmd/stellar-rpc/internal/rpcv2/config/flags_test.go Tests new flags.
cmd/stellar-rpc/internal/rpcv2/config/config.go Defines new configuration.
cmd/stellar-rpc/internal/rpcv2/config/config_test.go Tests defaults and cascading.
cmd/stellar-rpc/internal/rpcv2/config_validate.go Validates core and preflight settings.
cmd/stellar-rpc/internal/rpcv2/config_validate_test.go Tests validation rules.
cmd/stellar-rpc/internal/rpcv1/daemon/metrics.go Adapts v1 core version access.
cmd/stellar-rpc/internal/methods/get_version_info.go Uses narrowed daemon API.
cmd/stellar-rpc/internal/host/noOpDaemon.go Updates no-op implementation.
cmd/stellar-rpc/internal/host/host.go Narrows the daemon interface.
cmd/stellar-rpc/docker/Dockerfile.rpcv2 Documents required Rust tooling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +163 to +164
if host == "localhost" {
return true

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done as of ef7a0da

ctx, cancel := context.WithTimeout(ctx, coreVersionTimeout)
defer cancel()

out, err := exec.CommandContext(ctx, binaryPath, "version").Output()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done as of ef7a0da

Comment on lines +146 to +156
// This daemon starts the core it submits to, and that core binds
// core_http_port, so a local core_url on any other port reaches something
// else — often a second daemon's captive core, which would accept the
// transaction — with neither startup nor getHealth reporting a problem. A
// non-local core_url is left alone: that is the reason to set it at all.
if isLoopbackHost(u.Hostname()) && u.Port() != strconv.FormatUint(uint64(*ing.CoreHTTPPort), 10) {
return fmt.Errorf("[ingestion].core_url %q points at this host but not at %s (%d) — "+
"the captive core this daemon starts listens on that port. Leave core_url unset to "+
"derive it, or give it a non-local address if core's admin server really is elsewhere",
ing.CoreURL, keyCoreHTTPPort, *ing.CoreHTTPPort)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done as of ef7a0da

Comment on lines +136 to +139
ctx, cancel := context.WithCancel(context.Background())
cancel()

d, err := New(ctx, cfg)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done as of ef7a0da

Comment thread cmd/stellar-rpc/internal/rpcv2/config_validate.go Outdated

@tamirms tamirms left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the copilot findings are worth reviewing. But apart from that the PR looks good!

@karthikiyer56

Copy link
Copy Markdown
Contributor Author

I think the copilot findings are worth reviewing. But apart from that the PR looks good!

done.
adressed all copilot suggestions

@karthikiyer56
karthikiyer56 merged commit f98bb55 into feature/full-history Jul 31, 2026
14 checks passed
@karthikiyer56
karthikiyer56 deleted the karthik/884-captive-core-endpoints branch July 31, 2026 01:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Serve getLedgerEntries, simulateTransaction, sendTransaction on v2

3 participants