Add RPC database backfill performance evaluation leg

[cjonas9]

What

Adds a new backfill leg atop the coordinator architecture that spawns an EC2 instance and runs backfill. Metrics for the performance of this backfill are logged through the GHA coordinator alongside metrics for the apply-load ingestion load test.

The core components of this addition are organized canonically in-line with the precendent of the apply-load ingestion test. In particular, this is:

• perf-eval/backfill-test/: contains EC2 box script + a runner whose instantiate renders a default pubnet config with a datalake (testdata/backfill-pubnet.toml.tmpl), starts the daemon's backfill, detects the completion sentinel, and publishes the elapsed/ledger-count result to S3
• load-test-coordinator.yml: contains a roster entry in the coordinator so the leg "Backfill ingestion" fans out through the same matrix as apply-load

Why

See epic #706. This is one of ~4 jobs that will be completed as part of the automated RPC release evaluation framework, and it is necessary for the completion of the leg that evaluates endpoint performance using stellar-rpc-blaster.

Known limitations

Currently, this test measures a backfill of one day of ledgers (=17280 ledgers). This is because Github Actions has a hard-cap of 6 hours per job, and a one-week backfill exceeds this time constraint. Backfill optimizations should be made to allow backfill to execute a one-week backfill in a shorter amount of time.

[Copilot]

Pull request overview

Adds a new “Backfill ingestion” performance-evaluation leg to the existing perf-eval coordinator architecture, running a backfill on an ephemeral EC2 instance and reporting metrics through the same S3-backed harness used by the ingestion load test.

Changes:

• Adds a new backfill-test leg (EC2 user-data script, runner, and a pubnet+datalake config template) that runs stellar-rpc backfill and publishes elapsed/ledger-range results to S3.
• Extends the shared perf-eval gatherer outputs to include verdict and S3 result location (bucket/key), including on timeout paths.
• Updates the load-test coordinator workflow to fan out the new backfill leg via the existing leg matrix (with an explicit time budget).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
cmd/stellar-rpc/internal/integrationtest/infrastructure/perf-eval/harness/gather.go	Adds additional step outputs (verdict + S3 location) for downstream reporting/debugging, including timeout cases.
cmd/stellar-rpc/internal/integrationtest/infrastructure/perf-eval/backfill-test/testdata/backfill-pubnet.toml.tmpl	Introduces a rendered TOML template for running backfill against pubnet ledgers served from a public S3 datastore.
cmd/stellar-rpc/internal/integrationtest/infrastructure/perf-eval/backfill-test/runner/main.go	Adds a leg runner entrypoint wired into the shared perf-eval harness.
cmd/stellar-rpc/internal/integrationtest/infrastructure/perf-eval/backfill-test/runner/instantiate.go	Implements the on-instance workflow: build stellar-rpc, render config, run backfill, and publish results.
cmd/stellar-rpc/internal/integrationtest/infrastructure/perf-eval/backfill-test/run-backfill.sh	Adds the EC2 leg script that bootstraps the box and runs the backfill runner.
.github/workflows/load-test-coordinator.yml	Adds “Backfill ingestion” to the coordinator leg roster and passes per-leg budget to ec2-leg.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tamirms]

Thorough pass over the backfill leg. It mirrors the apply-load precedent cleanly, the new Go code vets, and the branch's own end-to-end run is green, so nothing here is a functional blocker on the happy path. The one must-fix before merge is the temporary backfill-test push trigger. The rest are a few worth-raising items (unused gather outputs, the --base main change), some nits, and two design and measurement observations, including whether the backfill could be measured in-process the way the apply-load leg already is. Details inline.

[Shaptic]

Coupla minor notes but otherwise this looks really good! Love how clean it is to add a new job leg.