Context
Child of #2118. The Inference Server was restructured after the v1.2.0 code freeze in #1813, #1820, #1833, #1834, and #1861, with platform/runtime follow-ups in #1895, #1957, and #2053.
The current Fern page still imports the removed InferenceModelConfig and presents Ray Serve as the only backend. Its copy-paste examples fail against current main.
Required coverage
- Replace
InferenceModelConfig with RayServeModelConfig and RayServeServerConfig.
- Document
DynamoVLLMModelConfig, DynamoServerConfig, DynamoRoleConfig, and DynamoRouterConfig.
- Provide runnable Ray Serve, Dynamo aggregated, and Dynamo disaggregated examples.
- Explain model/server config type validation, multi-model rules, replica and tensor-parallel placement, and single-node constraints for disaggregated roles.
- Document router modes, KV event behavior, per-role engine overrides, runtime environment merging, subprocess environment propagation, lifecycle, health checks, and cleanup.
- Explain HAProxy ingress behavior and the final tested dependency stack, including architecture restrictions.
- Include a before/after migration from
InferenceModelConfig.
- Incorporate the multimodal, hybrid-KV, subprocess-environment, and false-CLI-flag corrections from #2053.
Target pages
fern/versions/main/pages/curate-text/synthetic/inference-server.mdx
- Related LLM-client, NDD, installation, and infrastructure pages where they repeat the obsolete API
Acceptance criteria
Related PRs: #1813, #1820, #1833, #1834, #1861, #1895, #1957, #2053
Context
Child of #2118. The Inference Server was restructured after the v1.2.0 code freeze in #1813, #1820, #1833, #1834, and #1861, with platform/runtime follow-ups in #1895, #1957, and #2053.
The current Fern page still imports the removed
InferenceModelConfigand presents Ray Serve as the only backend. Its copy-paste examples fail against currentmain.Required coverage
InferenceModelConfigwithRayServeModelConfigandRayServeServerConfig.DynamoVLLMModelConfig,DynamoServerConfig,DynamoRoleConfig, andDynamoRouterConfig.InferenceModelConfig.Target pages
fern/versions/main/pages/curate-text/synthetic/inference-server.mdxAcceptance criteria
InferenceModelConfigRelated PRs: #1813, #1820, #1833, #1834, #1861, #1895, #1957, #2053