Skip to content

Please clarify structured output support #544

@MRYingLEE

Description

@MRYingLEE

Deepspeed supports lm-format-enforcer(https://github.com/noamgat/lm-format-enforcer) through prefix_allowed_tokens_fn.
An example is https://github.com/noamgat/lm-format-enforcer?tab=readme-ov-file#basic-tutorial. So, I am sure DeepSpeed supports structured output.

So far, I didn't find such a kind of example with DeepSpeed-MII.

Can someone clarify whether DeepSpeed-MII has structured output support built-in?

Thank you in advance,

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions