Proxied JSON payloads are re-serialized with alphabetized object keys, breaking order-enforcing structured-output backends

[himorishige]

Summary

When a request passes through the router, every JSON object in the payload is re-serialized with its keys sorted alphabetically (serde_json's default Map is BTreeMap-backed; the workspace does not enable the preserve_order feature). Array order is preserved; only object key order changes.

For most fields this is cosmetic. For response_format.json_schema.schema it is semantic: order-enforcing structured-output backends (vLLM with its default xgrammar backend) use the properties declaration order as the generation order. Reordering the schema therefore changes what the model is forced to generate.

I'm opening a companion PR alongside this issue — the fix is a one-line Cargo.toml change — so the details below double as its rationale.

Reproduction (model-independent)

1. Configure a passthrough route to any vLLM target started with --enable-log-requests.

2. Send a chat completion with a json_schema response format whose properties are deliberately non-alphabetical:

   "properties": {
     "zebra_field": {"type": "string"},
     "apple_field": {"type": "string"},
     "mango_field": {"type": "string"}
   },
   "required": ["zebra_field", "apple_field", "mango_field"]

3. vLLM's request log shows what the router actually forwarded:

   StructuredOutputsParams(json={'additionalProperties': False, 'properties':
   {'apple_field': ..., 'mango_field': ..., 'zebra_field': ...},
   'required': ['zebra_field', 'apple_field', 'mango_field'], ...})
   

   The properties map is alphabetized while the required array keeps its order, and the model's output follows the imposed alphabetical order.

Observed on the v0.2.0 release image; the serialization path is unchanged on current main.

Impact we measured

We run a LoRA fine-tuned capability judge (Nemotron 3.5 Lightning) as an llm_classifier target on a local vLLM. The model was fine-tuned to emit its verdict keys in a fixed order ending with p_solve. Behind the router, the alphabetized schema forces a different key order, and in 36/200 judgments (18%) the model stalled after the third key and emitted whitespace until max_tokens (the classic grammar/EOS deadlock), producing an invalid verdict and a fail-open on every occurrence. Interposing a 30-line proxy that only restores the original key order dropped this to 0/100 with fully stable decisions, which isolates key order as the cause. Even without a stall, forcing capability_boundary to be generated before crux measurably changed verdicts.

A generalist judge model is robust to the reordering, which is presumably why this has gone unnoticed.

Fix (companion PR)

The PR enables serde_json's preserve_order feature at the workspace level, so the router forwards object keys as the client sent them. It is one line in Cargo.toml plus the lockfile (indexmap is already in the dependency tree), and cargo test --workspace passes. If you'd rather avoid the workspace-wide feature — e.g., carry passthrough JSON as RawValue so the router never re-serializes user payloads — I'm happy to rework in that direction.

Environment

• switchyard-server 0.2.0 (release image), config: llm_classifier + passthrough routes
• Backend: vLLM v0.27.1 (--served-model-name judge), default structured outputs backend (xgrammar), DGX Spark (aarch64)