Skip to content

[Feature] Add serving contract support for lib-serving decoupling #752

Description

@bumble0918

Summary

Add a pypto-lib owned serving contract layer so pypto-serving can consume model kernel ABI details through structured contracts instead of duplicating lib internals.

Motivation / Use Case

The serving-side architecture work is tracked in hw-native-sys/pypto-serving#65. To complete the decoupling, pypto-lib should become the source of truth for model-specific serving ABI details, including kernel loading, host-side JIT wrapper signatures, compile/runtime argument builders, weight layout preparation, and compatibility validation.

This keeps pypto-serving focused on runtime orchestration while allowing Qwen3-14B, DeepSeek-V4 Flash/Pro, and future models to expose their serving requirements through lib-owned contracts.

Proposed API / Behavior

Introduce pypto-lib support for:

  • a serving contract registry that selects a ModelServingContract from explicit family/variant or structured model config metadata;
  • ModelServingContract metadata for model identity, capabilities, limits, KV/cache layout, execution graph, kernel specs, and weight metadata;
  • lib-owned kernel loaders and validators;
  • lib-owned host-side JIT wrappers, compile argument builders, runtime argument builders, and weight layout preparation hooks;
  • structured compatibility errors including schema version, ABI fingerprint, incompatible field, and suggested action.

Detailed design reference: hw-native-sys/pypto-serving#65

Alternatives Considered

No response

Additional Context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions