Skip to content

[Docs] Rewrite InferenceServer docs for typed Ray Serve and Dynamo backends #2146

Description

@lbliii

Context

Child of #2118. The Inference Server was restructured after the v1.2.0 code freeze in #1813, #1820, #1833, #1834, and #1861, with platform/runtime follow-ups in #1895, #1957, and #2053.

The current Fern page still imports the removed InferenceModelConfig and presents Ray Serve as the only backend. Its copy-paste examples fail against current main.

Required coverage

  • Replace InferenceModelConfig with RayServeModelConfig and RayServeServerConfig.
  • Document DynamoVLLMModelConfig, DynamoServerConfig, DynamoRoleConfig, and DynamoRouterConfig.
  • Provide runnable Ray Serve, Dynamo aggregated, and Dynamo disaggregated examples.
  • Explain model/server config type validation, multi-model rules, replica and tensor-parallel placement, and single-node constraints for disaggregated roles.
  • Document router modes, KV event behavior, per-role engine overrides, runtime environment merging, subprocess environment propagation, lifecycle, health checks, and cleanup.
  • Explain HAProxy ingress behavior and the final tested dependency stack, including architecture restrictions.
  • Include a before/after migration from InferenceModelConfig.
  • Incorporate the multimodal, hybrid-KV, subprocess-environment, and false-CLI-flag corrections from #2053.

Target pages

  • fern/versions/main/pages/curate-text/synthetic/inference-server.mdx
  • Related LLM-client, NDD, installation, and infrastructure pages where they repeat the obsolete API

Acceptance criteria

  • No published example imports or references InferenceModelConfig
  • Ray Serve quickstart runs with the typed config API
  • Dynamo aggregated and disaggregated examples match current constructors and validation rules
  • Routing, placement, runtime-env, lifecycle, and cleanup behavior are documented
  • CUDA/Ray/PyTorch/vLLM/ai-dynamo requirements and x86_64 restrictions are explicit
  • Migration guidance explains the breaking typed-config change
  • Related Fern pages are audited for stale examples
  • Fern validation and link checks pass

Related PRs: #1813, #1820, #1833, #1834, #1861, #1895, #1957, #2053

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentation

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions