Currently, the platform supports a limited set of proprietary model providers. However, there's a growing need to integrate open-source models(e.g., Mistral, LLaMA 3, Mixtral, Falcon, etc.) and Amazon Bedrock to offer more flexible, cost-effective, and customizable deployment options.

Describe the solution you'd like

It would be great to have:

• Native support for popular open-source models (via APIs or local runtime).
• Integration with Amazon Bedrock for easy access to foundation models from Anthropic, Meta, Cohere, etc.
• A configuration layer to switch between model providers dynamically based on use case or cost optimization.

Additional context

Support for these models would significantly improve flexibility for devs who want to:

• Keep infrastructure self-hosted
• Avoid vendor lock-in
• Optimize for data privacy and compliance

Assignee

Assigning this task to myself. I’ll begin scoping out the architecture and integration paths for both open-source model support and Amazon Bedrock. Happy to share progress updates or open a draft PR soon.