Skip to content

feat(filter): outbound chain binding with authority-bound deferred credentials - #1126

Merged
leseb merged 3 commits into
praxis-proxy:mainfrom
leseb:leseb/implement-issue-1074
Sep 11, 2026
Merged

leseb merged 3 commits into
praxis-proxy:mainfrom
leseb:leseb/implement-issue-1074

Conversation

@leseb

@leseb leseb commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Implements issue #1074: lets a filter bind a nested "outbound" filter chain (named or inline) that runs against the resolved upstream, exposing a small public FilteredSubrequestExecutor::run() API that returns either a buffered or a streaming response. Bound chains are held to the same validation top-level chains face — filter-count cap, empty-predicate rejection, inline-cluster SSRF/insecure-TLS gating, branch-chain constraints (max-iterations ceiling, nesting depth, per-filter and configuration-wide total-branch caps), and rejection of terminal/TCP filters — all of which they would otherwise bypass by never appearing in Config::filter_chains, with every scan traversing branch sub-chains. Credentials are staged during request processing and materialized only into a request bound for the canonical logical authority they were issued for, after header sanitization, and only when actually injected, so a secret cannot reach a shared-vhost endpoint under a different Host. Bound outbound chains participate fully in the config-reload contract, being rebuilt and re-injected with runtime resources (KV stores, session stores, subrequest client) including into nested and branch-embedded pipelines. No IRR or AI-client migration is included.

Which issue(s) does this relate to?

Fixes #1074

Checklist

  • Signed off all commits (git commit -s)
  • Tests added or updated
  • Documentation updated (if applicable)
  • make lint && make test && make test-integration passes locally

Does this introduce a breaking change?

No.

@leseb
leseb requested a review from a team September 9, 2026 16:32
@leseb
leseb requested a review from shaneutt as a code owner September 9, 2026 16:32
@praxis-bot-app

praxis-bot-app Bot commented Sep 9, 2026

Copy link
Copy Markdown

PR too large: 5334 lines added (limit: 500, excludes Cargo files, tests, docs, examples, and benchmarks). Please split into smaller PRs.

@shaneutt shaneutt moved this from Next to Review in Core Proxy Sep 9, 2026
@shaneutt shaneutt added this to the v0.6.0 milestone Sep 9, 2026

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: Outbound chain binding with authority-bound deferred credentials

Adds a chain-binding mechanism that lets application filters (e.g. AI callouts) own a prebuilt outbound FilterPipeline resolved at construction time, plus an authority-bound deferred credential system that materializes secrets only after destination resolution. Bound chains face the same validation gates as top-level chains (cardinality, SSRF, branch limits, terminal/TCP filter rejection, ordering). Budgets are shared across binding boundaries so the ceiling cannot be evaded.

Assessment

This is a well-structured, security-conscious PR with thorough build-time validation and extensive test coverage across all modules. The credential binding model is sound -- secrets are sealed until the resolved authority matches, and injection happens after sanitization with Host pinning to prevent vhost-targeted exfiltration. The build-time budget sharing (materialization instances and branch definitions) correctly prevents fan-out evasion across binding boundaries. Two medium-severity test coverage gaps are noted below.

Findings

Severity File Finding
Medium credentials.rs IPv6 authority parsing paths lack test coverage
Medium credentials.rs inject_canonical silently swallows invariant violation

Comment thread crates/filter/src/credentials.rs
Comment thread crates/filter/src/credentials.rs Outdated
@leseb
leseb force-pushed the leseb/implement-issue-1074 branch from 7c9257f to 5533b01 Compare September 10, 2026 12:33
@leseb
leseb requested a review from a team September 10, 2026 12:33
@twghu

twghu commented Sep 11, 2026 •

Copy link
Copy Markdown
Collaborator

CalloutStreamingBody: a response-ceiling breach leaves the stream runnable

The defect

CalloutStreamingBody::checked (crates/filter/src/filtered_subrequest/streaming.rs) returns Err when a chunk would push the stream past max_response_bytes, but leaves the body in a runnable state — finished stays false and inner stays Some. Tracing next_chunk: a caller that polls again finds pending empty, deferred_error None, finished false and inner live, so it pulls the next upstream chunk and returns it if it fits.

Two things compound:

  • The rejected chunk is never emitted, so emitted_bytes never advances — the return Err at the ceiling check precedes self.emitted_bytes = total. Later chunks are therefore measured against a pre-breach count.
  • The breach is recorded nowhere, so nothing after the error knows the body is no longer contiguous.

Net effect: the oversized chunk is dropped out of the middle of the body while smaller subsequent chunks keep flowing. The consumer receives a body with a hole in it rather than an error-terminated stream — for a callout response (an SSE stream, a JSON document) a worse outcome than overshooting the ceiling would have been.

The deferred_error arm twelve lines below already does the right thing (finished = true; inner = None). That inconsistency is what suggests the omission at the ceiling check is an oversight rather than a deliberate "callers must stop on Err" assumption.

Evidence

Added run_streaming_ceiling_breach_ends_the_stream: 20-byte upstream body, 15-byte ceiling, test_terminal_event in the chain so a 14-byte data: [DONE] completion event follows EOF. Because the rejected 20-byte chunk never advances the counter, that 14-byte event still fits under 15. Against the unpatched code:

thread '...run_streaming_ceiling_breach_ends_the_stream' panicked at
crates/filter/src/filtered_subrequest/tests.rs:1432:5:
a ceiling breach must end the stream; polling again must not resume it,
got resumed with 14 more bytes

That is the completion event being delivered after a fatal error.

The change

Split checked into a teardown wrapper plus an account helper, so every error path out of the accounting — the ceiling breach and the checked_add overflow, which had the same problem — sets finished = true and clears pending before the error propagates. The next poll then hits the existing if self.finished { return Ok(None) } arm.

One deliberate divergence from the deferred_error arm: inner is left in place. That arm can null it safely because drain_completion has already populated held_extensions. At the ceiling check it has not, and checked is synchronous — so taking inner there would drop the upstream body without awaiting SubResponseBody::cancel(), and would strand the parent extensions where swap_extensions cannot reach them (it falls back to held_extensions only when inner is None). Leaving inner alive keeps a later cancel()/suppress() able to cancel the upstream asynchronously and recover the extensions. finished = true is what stops the stream; dropping inner was never the part doing the work.

The test's pull loop tolerates the upstream splitting those 20 bytes across several chunks, since spawn_raw_backend issues a single write_all and the test shouldn't rest on non-coalescing.

Verification

  • cargo test -p praxis-proxy-filter --lib filtered_subrequest::tests::run_streaming — 7/7 pass
  • cargo clippy -p praxis-proxy-filter --all-targets --all-features -- -D warnings — clean
  • RUSTDOCFLAGS=-Dwarnings cargo doc -p praxis-proxy-filter --no-deps --document-private-items — clean
  • The same test fails as shown above with the streaming.rs hunk reverted and the test retained

Not run: cargo +nightly fmt --all -- --check (no nightly toolchain locally; rustfmt.toml relies on nightly-only options, so a stable run would be a false pass).

Patches:

git --no-pager diff -- crates/filter/src/filtered_subrequest/streaming.rs
diff --git a/crates/filter/src/filtered_subrequest/streaming.rs b/crates/filter/src/filtered_subrequest/streaming.rs
index 7bb94737..d7b42f52 100644
--- a/crates/filter/src/filtered_subrequest/streaming.rs
+++ b/crates/filter/src/filtered_subrequest/streaming.rs
@@ -349,8 +349,34 @@ impl CalloutStreamingBody {
         }
     }
 
-    /// Account one outgoing chunk against the response byte ceiling.
+    /// Account one outgoing chunk against the response byte ceiling, ending the
+    /// stream if it does not fit.
+    ///
+    /// A breach is *terminal*. The rejected chunk is never emitted, so
+    /// `emitted_bytes` does not advance; without marking the stream finished a
+    /// caller that polled again would resume around the hole and keep delivering
+    /// later chunks against the stale count, handing the consumer a body with a
+    /// gap in it instead of an error-terminated stream. Marking it finished and
+    /// dropping queued chunks makes the next poll report end-of-stream, matching
+    /// the `deferred_error` arm of [`next_chunk`](StreamingResponseBody::next_chunk).
+    ///
+    /// The inner body is deliberately left in place: tearing it down here would
+    /// have to drop the upstream synchronously, whereas a subsequent
+    /// [`cancel`](StreamingResponseBody::cancel) or
+    /// [`suppress`](StreamingResponseBody::suppress) can still cancel it
+    /// asynchronously and recover the parent extensions.
     fn checked(&mut self, chunk: Bytes) -> Result<Option<Bytes>, FilterError> {
+        let outcome = self.account(chunk);
+        if outcome.is_err() {
+            self.finished = true;
+            self.pending.clear();
+        }
+        outcome
+    }
+
+    /// Add `chunk` to the emitted-byte total, rejecting a counter overflow or a
+    /// breach of the response ceiling.
+    fn account(&mut self, chunk: Bytes) -> Result<Option<Bytes>, FilterError> {
         let total = self
             .emitted_bytes
             .checked_add(chunk.len())

git --no-pager diff -- crates/filter/src/filtered_subrequest/tests.rs
diff --git a/crates/filter/src/filtered_subrequest/tests.rs b/crates/filter/src/filtered_subrequest/tests.rs
index 9d99ac70..d5c023d1 100644
--- a/crates/filter/src/filtered_subrequest/tests.rs
+++ b/crates/filter/src/filtered_subrequest/tests.rs
@@ -1360,6 +1360,81 @@ async fn run_streaming_enforces_response_byte_ceiling() {
     );
 }
 
+#[tokio::test]
+#[expect(clippy::large_futures, reason = "drives the full executor future in a test")]
+async fn run_streaming_ceiling_breach_ends_the_stream() {
+    use std::{
+        sync::Arc,
+        time::{Duration, Instant},
+    };
+
+    // A 20-byte upstream body (hex chunk length 14), behind a chain whose
+    // completion hook emits a 14-byte terminal event at EOF.
+    let (addr, backend) = spawn_raw_backend(
+        "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n14\r\nAAAAAAAAAAAAAAAAAAAA\r\n0\r\n\r\n",
+    )
+    .await;
+    let registry = callout_registry();
+    let mut entries: Vec<crate::FilterEntry> =
+        serde_yaml::from_str(&routed_chain_yaml(addr, "- filter: test_terminal_event\n")).unwrap();
+    let pipeline = Arc::new(crate::FilterPipeline::build(&mut entries, &registry).unwrap());
+
+    // 20 bytes of body cannot fit under a 15-byte ceiling however the upstream
+    // frames it, so some chunk must be rejected. A rejected chunk is never
+    // emitted and so never advances `emitted_bytes` — leaving room for the
+    // 14-byte completion event to pass the check afterwards. A stream that kept
+    // running would therefore resume around the hole and hand the caller a body
+    // with a gap in it instead of an error-terminated one.
+    let executor = streaming_executor(15);
+    let request = crate::SubRequest {
+        method: http::Method::GET,
+        uri: http::Uri::from_static("/"),
+        headers: HeaderMap::new(),
+        body: bytes::Bytes::new(),
+    };
+    let deadline = Instant::now() + Duration::from_secs(5);
+
+    let mut body = match executor
+        .run(&pipeline, &request, crate::RequestExtensions::default(), deadline)
+        .await
+        .expect("streaming callout should open")
+    {
+        crate::CalloutResponse::Streaming { body, .. } => body,
+        crate::CalloutResponse::Buffered(_) => panic!("the chain selected streaming mode"),
+    };
+
+    // Pull until the ceiling rejects a chunk, whatever framing the upstream used.
+    let mut breach = None;
+    for _ in 0_u8..32 {
+        match body.next_chunk().await {
+            Ok(Some(_)) => {},
+            Ok(None) => break,
+            Err(error) => {
+                breach = Some(error);
+                break;
+            },
+        }
+    }
+    let breach = breach.expect("20 bytes of body must breach a 15-byte ceiling");
+    assert!(
+        breach.to_string().contains("exceeds configured body limit"),
+        "the breach must be the body-limit error: {breach}"
+    );
+
+    let resumed = body.next_chunk().await;
+    backend.abort();
+
+    let observed = match &resumed {
+        Ok(Some(chunk)) => format!("resumed with {} more bytes", chunk.len()),
+        Ok(None) => "end of stream".to_owned(),
+        Err(error) => format!("error: {error}"),
+    };
+    assert!(
+        matches!(resumed, Ok(None)),
+        "a ceiling breach must end the stream; polling again must not resume it, got {observed}"
+    );
+}
+
 #[tokio::test]
 #[expect(clippy::large_futures, reason = "drives the full executor future in a test")]
 async fn run_streaming_surfaces_unhandled_upstream_termination() {

@twghu twghu left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A response-ceiling breach in CalloutStreamingBody::checked doesn't end the stream, so polling again resumes around the rejected chunk and hands the caller a body with a hole in it instead of an error — fix and failing-test evidence in the previous comment.

…edentials

Implement issue praxis-proxy#1074: a filter can bind a nested "outbound" filter chain
that runs against the resolved upstream, injecting credentials that are
staged during request processing and materialized only once the upstream
authority is known.

Chain binding: ChainBindingContext resolves a ChainRef (named or inline)
into a FilterPipeline at pipeline-build time, bounded by a maximum nesting
depth. Bound chains are held to the same validation top-level chains face,
which they would otherwise bypass by never appearing in
Config::filter_chains: the per-chain filter-count cap via
validate_chain_entries_cardinality, empty-predicate condition rejection —
top-level and inside inline branch sub-chains — via
validate_chain_entries_conditions, inline-cluster SSRF and insecure-TLS
gating via validate_chain_entries_inline_clusters, and the core branch-chain
constraints (the re-entrant max_iterations ceiling, nesting depth, per-
filter and total branch caps, filter-name uniqueness, chain-reference
resolution) via validate_chain_entries_branch_chains. The total-branch cap
is a configuration-wide budget: it is seeded with the whole configuration's
branch count (the listener's own branches plus every named chain) and
accumulated across every bound inline chain in the build, so bindings cannot
each stay under the ceiling while exceeding it collectively — nor start
counting from zero and ignore branches already present elsewhere in the
configuration. A named outbound chain is a top-level filter chain the
whole-config validate_branch_chains pass already counted once toward the
ceiling, so it is validated when bound but not re-accumulated; only inline
outbound chains, invisible to that pass, accumulate. Seeding config-wide
(not listener-scoped) keeps a named chain the listener never references —
which still materializes when bound — in the baseline that inline bindings
add on top of, so the named pool and the inline pool cannot each sit under
the ceiling while exceeding it together. A bound chain that
contains a terminal filter or a TCP-level filter — at the top level or
nested inside a branch sub-chain — is rejected at build time because the
HTTP filtered sub-request executor forwards to a resolved upstream and can
run neither. Terminal filters are detected by the
HttpFilter::produces_terminal_response capability rather than a hard-coded
builtin name list, so a custom filter that returns a terminal action cannot
escape the check. build_with_chains now threads InsecureOptions so a bound
chain cannot bypass those gates.

Branch-recursion completeness: the pipeline-wide scans that enforce these
contracts all traverse branch sub-chains rather than only top-level
entries, so a filter buried in a branch is treated exactly like a top-level
one instead of silently escaping validation or lifecycle wiring. This
covers non-HTTP and terminal-filter detection, the failure_mode: open
security guardrail (check_open_security_filters, which errors or — under
insecure_options.allow_open_security_filters — warns, consistent with the
SkipTo and Terminal branch-bypass checks), referenced-file discovery for
reload, body-limit and insecure-option propagation, and session-store
injection.

Deferred credentials: PendingCredentials/DeferredCredential stage secrets
keyed to a canonical logical authority. The subrequest executor
materializes them only into a request bound for the authority each
credential was issued for, and only after header sanitization so the
credential header is never stripped as hop-by-hop or framing. A deferred
credential may not carry Host, content-length, or a hop-by-hop header, so
an injected credential cannot retarget the request after its authority was
authorized.

Logical authority vs transport: credentials bind to the operator-configured
authority override (the logical HTTP authority), not the transport endpoint
bytes travel to. The outbound subrequest also sends that authority as its
Host header, mirroring the normal proxy path's authority override and
honoring the documented Upstream.authority contract; the transport endpoint
is used as the Host only when no override is configured. The sent Host is
pinned to the authorized authority only when a credential is actually
injected, keeping a secret from reaching a shared-vhost endpoint under a
different Host than it was scoped to; a staged credential that matches
nothing injects no secret and leaves a caller- or step-set Host untouched,
so it cannot silently change virtual-host routing.

Reload contract: bound outbound chains are rebuilt and re-injected with
runtime resources (KV stores, session stores, subrequest client) on config
reload. Resource injection recurses into nested pipelines — both filter-
embedded outbound chains and branch sub-chains — so bound chains observe
the parent pipeline's stores.

Signed-off-by: Sébastien Han <seb@redhat.com>
…auth

Address the two review findings on the authority-bound deferred
credential system:

- inject_canonical no longer silently swallows a re-validation failure.
  When a pre-validated credential value fails HeaderValue::from_str at
  injection time, warn! and drop it (still a no-op, no header written).
  The diagnostic omits the secret value and the offending bytes.
- Add IPv6 authority coverage: bracketed and unbracketed literals for
  both exact authority bindings and host wildcards, plus the
  port-required rejection paths.
- Add a capture_logs test helper and a regression test proving the
  invariant-violation path warns, writes no header, and never leaks the
  secret.

Signed-off-by: Sébastien Han <seb@redhat.com>
@twghu

twghu commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Once the fix is considered this PR can be approved.

@leseb
leseb force-pushed the leseb/implement-issue-1074 branch from 5533b01 to d0b470f Compare September 11, 2026 07:49
CalloutStreamingBody::checked returned an error on a ceiling breach (or
emitted-byte counter overflow) but left the stream runnable: finished
stayed false and pending/inner were untouched. A caller that polled again
after the rejected chunk resumed around it, receiving a body with a hole
instead of an error-terminated stream.

Split checked into a teardown wrapper that, on rejection, sets
finished = true and clears pending so a re-poll cannot resume, plus an
account helper holding the original byte-accounting logic. inner is left
in place deliberately so async cancel/suppress and extension recovery
still work.

Reported by maintainer @twghu.

Signed-off-by: Sébastien Han <seb@redhat.com>
@leseb

leseb commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator Author

A response-ceiling breach in CalloutStreamingBody::checked doesn't end the stream, so polling again resumes around the rejected chunk and hands the caller a body with a hole in it instead of an error — fix and failing-test evidence in the previous comment.

Addressed in 5013eb9 - PTAL :)

@leseb
leseb requested a review from twghu September 11, 2026 08:01
@leseb
leseb enabled auto-merge (squash) September 11, 2026 11:45
@leseb
leseb disabled auto-merge September 11, 2026 11:45
@leseb
leseb merged commit f96954d into praxis-proxy:main Sep 11, 2026
18 of 20 checks passed
@github-project-automation github-project-automation Bot moved this from Review to Done in Core Proxy Sep 11, 2026
@leseb
leseb deleted the leseb/implement-issue-1074 branch September 11, 2026 11:46
@shaneutt shaneutt modified the milestones: v0.6.0, v0.5.5 Sep 15, 2026
shaneutt added a commit that referenced this pull request Sep 19, 2026
…binding

PendingCredentials and DeferredCredential are the credential half of the
outbound-callout mechanism (#1126), so they now compile only under the
experimental chain-binding feature alongside register_chain_binding. The
filtered-subrequest credential drain is gated with them; default builds
drop the unused authority-bound-credential surface.

Signed-off-by: Shane Utt <shaneutt@linux.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

feat(filter): configure and validate outbound subrequest filter chains

4 participants