[OPIK-2982] [P SDK] [BE] [FE] ChatPrompt support #3987

alexkuzmik · 2025-11-07T16:46:35Z

Details

Implemented ChatPrompt feature.
Added it to:

Python SDK: API objects, integrated into existing prompt-related features and APIs, added new client functions to create/get/search chat prompts.
BE: added new filterable column to prompt database - template_structure: "text" or "chat")
FE: prompt library (view, create, update + Python code snippets), prompt playground (import/export).

https://pr-3987.dev.comet.com/

Demo script for new SDK API:

"""
Example: Chat Prompt Management with Opik SDK

This example demonstrates the new chat prompt functionality including:
- Creating chat prompts with multi-role messages
- Searching for prompts (both string and chat)
- Formatting prompts with variables
"""

import opik

# Initialize Opik client
client = opik.Opik()

# ============================================================================
# 1. Create a chat prompt with multiple roles
# ============================================================================
print("Creating chat prompt...")

chat_prompt = client.create_chat_prompt(  # can also be created via opik.ChatPrompt(...)
    name="customer-support-agent",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful customer support agent. Be friendly and professional and answer the customer's question.",
        },
        {"role": "user", "content": "Answer me: {{customer_question}}"},
    ],
)

print(f"✓ Created: {chat_prompt.name} (commit: {chat_prompt.commit[:8]})")

# ============================================================================
# 2. Search for prompts (returns both string and chat prompts)
# ============================================================================
print("\nSearching for prompts...")

# Get all prompts
all_prompts = client.search_prompts()
print(f"✓ Found {len(all_prompts)} total prompts")

# Filter by type using OQL
chat_prompts = client.search_prompts(filter_string='template_structure = "chat"')
string_prompts = client.search_prompts(filter_string='template_structure = "text"')

print(f"  - {len(chat_prompts)} chat prompts")
print(f"  - {len(string_prompts)} string prompts")

# ============================================================================
# 3. Retrieve and format a chat prompt
# ============================================================================
print("\nRetrieving and formatting prompt...")

retrieved = client.get_chat_prompt(name="customer-support-agent")

# Format with variables
formatted_messages = retrieved.format(
    variables={"customer_question": "How do I reset my password?"}
)

print(f"✓ Formatted {len(formatted_messages)} messages:")
for msg in formatted_messages:
    print(f"  [{msg['role']}]: {msg['content'][:50]}...")

# ============================================================================
# 4. Type-safe prompt handling
# ============================================================================
print("\nType-safe prompt handling...")

# search_prompts() returns Union[Prompt, ChatPrompt]
for prompt in all_prompts[:3]:  # Show first 3
    if isinstance(prompt, opik.ChatPrompt):
        print(f"  • {prompt.name} (chat, {len(prompt.template)} messages)")
    elif isinstance(prompt, opik.Prompt):
        print(f"  • {prompt.name} (string)")

print("\n✅ Example completed!")

https://github.com/user-attachments/assets/dce53a8e-2175-4755-a4dc-7f34f30dd372

Change checklist

User facing
Documentation update

Issues

Resolves #
OPIK-2982

Testing

Added unit tests and e2e tests to the SDK. Tested FE manually. Added new tests to prompt resource in the BE.

Documentation

Will be added in a separate PR.

- SDK: Add ChatPrompt class using ChatPromptTemplate, serialize to JSON with template_structure='chat' - Backend: Add template_structure field to PromptVersion model, create migration script - Frontend: Add template_structure column to prompts table, implement chat prompt creation UI with message-by-message editor, add ChatPromptView component for displaying chat prompts - Tests: Add unit and e2e tests for ChatPrompt functionality

…ructure field

- Improve ChatPromptView with modern chat interface design - Add circular avatar icons with role-specific colors - Enhance hover effects and visual hierarchy - Update PromptVersion type to include template_structure field - Add create_chat_prompt_example.py script demonstrating ChatPrompt usage

…nd property consistency - Add verify_chat_prompt_version() function to verify ChatPrompt properties - Verify template_structure field is set to 'chat' for ChatPrompt instances - Update test_chat_prompt.py to use the new verifier - Initialize internal backend properties to None in both Prompt and ChatPrompt __init__ - Import ChatPrompt in verifiers module

- Remove template_structure parameter requirement from verify_chat_prompt_version - Update all 10 chat prompt tests to use verify_chat_prompt_version - template_structure is now always verified to be 'chat' (no need to pass it) - Replace manual assertions with verifier calls for consistency

…tion logic - Remove temporary inference logic for template_structure field - Simplify comparison logic to directly use template_structure from API response - Backend now properly returns template_structure after regenerating OpenAPI code

…pt.View.Public.class) to PromptVersion fields

…xity - Remove latest_version subqueries from PromptDAO find() and findById() - Remove conditional logic from PromptVersionColumnMapper - Update retrievePromptVersion to fetch latest version directly from prompt_versions table - Update frontend to use versions[0] instead of latest_version.id This simplifies the backend and improves query performance by eliminating unnecessary JSON object construction in SQL queries.

- Add template_structure field to Prompt model in OpenAPI spec - Regenerate Python SDK REST API client code with Fern - Regenerate TypeScript SDK REST API client code with Fern This includes all the type definitions, serialization, and client methods needed to support the template_structure field in both SDKs.

@JSONVIEW

- Add TemplateStructure enum (string/chat) in backend API - Add template_structure field to Prompt model with proper @JSONVIEW annotations - Update PromptVersion to hide template_structure from Public view (internal field) - Add TemplateStructureArgumentFactory for JDBI enum handling - Add Liquibase migration to add template_structure column to prompts table This establishes the core backend support for differentiating between string and chat prompts at the Prompt level (not PromptVersion level).

…on SDK - Add template_structure property to Prompt class (always 'string') - Implement ChatPrompt class with messages array support - Update PromptClient to send template_structure when creating prompts - Add JSON comparison for chat prompts to avoid formatting differences - Add verify_chat_prompt_version() to E2E test verifiers The ChatPrompt class serializes messages to JSON for backend storage and sends template_structure='chat' during creation.

- Add template_structure field to Prompt TypeScript interface - Display template_structure column in prompts list (Chat/String) - Add type column with proper formatting and styling The frontend now displays whether each prompt is a 'Chat' or 'String' type based on the template_structure field from the backend.

…mat() - Remove supported_modalities parameter from ChatPrompt.format() method - Update unit tests to remove supported_modalities usage - Update e2e tests to remove supported_modalities usage - Rename test functions to reflect they test multimodal content, not modalities This simplifies the ChatPrompt API by removing the unused supported_modalities parameter.

- Remove create_chat_prompt_example.py from git (should not be committed) - File remains locally but is no longer tracked by git

github-actions · 2025-11-07T16:46:54Z

📋 PR Linter Failed

❌ Incomplete Details Section. The ## Details section cannot be empty.

❌ Incomplete Issues Section. You must reference at least one GitHub issue (#xxxx), Jira ticket (OPIK-xxxx), (CUST-xxxx) or DEV ticket (DEV-xxxx) under the ## Issues section.

Copilot

Pull Request Overview

This PR introduces comprehensive support for chat-style prompts across the Opik platform. The implementation adds a new "chat" template structure alongside the existing "string" structure, enabling users to create prompts with structured message arrays (system, user, assistant roles) instead of single text templates. The change spans backend schema updates, SDK implementations in both Python and TypeScript, and frontend UI enhancements.

Key changes:

Added template_structure field (enum: "string" | "chat") to prompts and prompt versions, immutable after creation
Implemented ChatPrompt class in Python SDK with JSON serialization of message arrays
Enhanced frontend with chat prompt creation/editing UI, message role visualization, and playground integration
Backend validation ensures template_structure consistency across prompt versions

Reviewed Changes

Copilot reviewed 86 out of 87 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
sdks/python/src/opik/api_objects/prompt/chat_prompt.py	New ChatPrompt class for chat-style prompts with message array support
sdks/python/tests/e2e/test_chat_prompt.py	Comprehensive E2E tests for ChatPrompt functionality
apps/opik-backend/.../TemplateStructure.java	New enum defining template_structure types
apps/opik-backend/.../PromptService.java	Backend logic for template_structure validation and enforcement
apps/opik-frontend/.../AddEditPromptDialog.tsx	UI for creating/editing chat prompts with message builder
apps/opik-frontend/.../ChatPromptView.tsx	Component for displaying chat messages with role-based styling
apps/opik-frontend/.../useLoadPlayground.ts	Playground integration for chat prompt message parsing
sdks/typescript/src/opik/rest_api/.../PromptTemplateStructure.ts	TypeScript SDK types for template_structure

Files not reviewed (1)

apps/opik-frontend/package-lock.json: Language not supported

sdks/python/src/opik/api_objects/prompt/client.py

apps/opik-backend/src/main/java/com/comet/opik/domain/PromptService.java

apps/opik-frontend/src/components/pages/PromptsPage/AddEditPromptDialog.tsx

apps/opik-frontend/src/components/pages/PromptPage/PromptTab/EditPromptVersionDialog.tsx

.../opik-frontend/src/components/pages-shared/llm/LLMPromptMessages/LLMPromptMessageActions.tsx

sdks/python/tests/e2e/verifiers.py

github-actions · 2025-11-10T11:04:37Z

🌿 Preview your docs: https://opik-preview-b903024f-e561-4dd4-b526-306d6632409f.docs.buildwithfern.com/docs/opik

No broken links found

…ering to save dialog

github-actions · 2025-11-10T11:57:08Z

📋 PR Linter Failed

❌ Incomplete Details Section. The ## Details section cannot be empty.

❌ Incomplete Issues Section. You must reference at least one GitHub issue (#xxxx), Jira ticket (OPIK-xxxx), (CUST-xxxx) or DEV ticket (DEV-xxxx) under the ## Issues section.

github-actions · 2025-11-26T16:38:44Z

🔄 Test environment deployment started

Building images for PR #3987...

You can monitor the build progress here.

CometActions · 2025-11-26T16:47:31Z

✅ Test environment is now available!

Access Information

URL: https://pr-3987.dev.comet.com
Cluster: comet-ml-development
Namespace: pr-3987
Version: 1.9.32-3987-merge-547
Application logs: View in Grafana

The deployment has completed successfully and the version has been verified.

github-actions · 2025-11-27T11:27:14Z

🌿 Preview your docs: https://opik-preview-e37073bd-0ac7-46e0-9df0-9d32f9ae88a3.docs.buildwithfern.com/docs/opik

No broken links found

aadereiko

Amazing job! Thanks for addressing comments that I have left! An extremely useful feature

We've discussed one issue that can be fixed later - the jumps between loaded prompts for chats and for a message. But it doesn't seem critical now :)

Again, nice!

aadereiko · 2025-11-27T13:17:14Z

apps/opik-frontend/src/components/pages/PromptPage/PromptTab/EditPromptVersionDialog.tsx

+  const templateHasChanges = isChatPrompt
+    ? JSON.stringify(
+        messages.map((m) => ({ role: m.role, content: m.content })),
+      ) !== promptTemplate
+    : template !== promptTemplate;


I would useMemo it in the future

aadereiko · 2025-11-27T13:18:54Z

apps/opik-frontend/src/components/pages/PromptsPage/AddEditPromptDialog.tsx

+  const editPrompt = () => {
    updateMutate({
      prompt: {
        id: defaultPrompt?.id,
        name,
        ...(description ? { description } : {}),
      },
    });
    setOpen(false);
-  }, [updateMutate, defaultPrompt?.id, name, description, setOpen]);
+  };


Why there is no useCallback here anymore?

github-actions · 2025-11-27T17:02:35Z

🔄 Test environment deployment started

Building images for PR #3987...

You can monitor the build progress here.

CometActions · 2025-11-27T17:11:55Z

✅ Test environment is now available!

Access Information

URL: https://pr-3987.dev.comet.com
Cluster: comet-ml-development
Namespace: pr-3987
Version: 1.9.32-3987-merge-563
Application logs: View in Grafana

The deployment has completed successfully and the version has been verified.

github-actions · 2025-11-28T09:30:43Z

🌿 Preview your docs: https://opik-preview-7ec70000-5888-413b-af9f-4800e3de7461.docs.buildwithfern.com/docs/opik

No broken links found

github-actions · 2025-12-02T11:45:55Z

🌿 Preview your docs: https://opik-preview-e0e77f62-6082-488c-b585-57a280fdfdb7.docs.buildwithfern.com/docs/opik

The following broken links where found:

Page:
❌ Broken link: {} ()

Page:
❌ Broken link: ] ()

Page:
❌ Broken link: {} ()

Page:
❌ Broken link: ] ()

Page:
❌ Broken link: {} ()

Page:
❌ Broken link: ] ()

Page:
❌ Broken link: {} ()

Page:
❌ Broken link: ] ()

github-actions · 2025-12-02T12:17:06Z

🌿 Preview your docs: https://opik-preview-65455004-84a5-442a-9bf9-1155088c1bb7.docs.buildwithfern.com/docs/opik

No broken links found

github-actions · 2025-12-02T12:20:16Z

🔄 Test environment deployment started

Building images for PR #3987...

You can monitor the build progress here.

CometActions · 2025-12-02T12:30:46Z

✅ Test environment is now available!

Access Information

URL: https://pr-3987.dev.comet.com
Cluster: comet-ml-development
Namespace: pr-3987
Version: 1.9.35-3987-merge-615
Application logs: View in Grafana

The deployment has completed successfully and the version has been verified.

alexkuzmik added 17 commits November 6, 2025 18:15

Revision 2: Update Fern-generated OpenAPI client code for template_st…

fc339d8

…ructure field

Refactor e2e tests for chat prompt

706ec0c

Revision 3: Fix frontend display of chat prompts - add @JSONVIEW(Prom…

8026905

…pt.View.Public.class) to PromptVersion fields

Revision 8: Remove internal_sandbox file from git tracking

6f9e4d7

- Remove create_chat_prompt_example.py from git (should not be committed) - File remains locally but is no longer tracked by git

Refactor ChatPrompt and tests for it, fix python linter

2533cdd

Add import to prompt playground

2d75e34

Copilot AI review requested due to automatic review settings November 7, 2025 16:46

alexkuzmik requested review from a team as code owners November 7, 2025 16:46

alexkuzmik marked this pull request as draft November 7, 2025 16:46

github-actions bot assigned alexkuzmik Nov 7, 2025

Copilot AI reviewed Nov 7, 2025

View reviewed changes

alexkuzmik and others added 2 commits November 10, 2025 11:43

Remove obsolete local migration

115256c

Merge branch 'main' into OPIK-2982-sdk-be-fe-implement-chat-prompt

3e33f4c

alexkuzmik added 2 commits November 10, 2025 12:56

Revision 6: Fix save chat prompt icon and add template structure filt…

b21aa25

…ering to save dialog

Revision 7: Add dynamic search placeholders for prompt dropdowns

21498ae

alexkuzmik added the test-environment Deploy Opik adhoc environment label Nov 26, 2025

Merge branch 'main' into OPIK-2982-sdk-be-fe-implement-chat-prompt

7a0b18d

aadereiko previously approved these changes Nov 27, 2025

View reviewed changes

alexkuzmik added test-environment Deploy Opik adhoc environment and removed test-environment Deploy Opik adhoc environment labels Nov 27, 2025

Merge branch 'main' into OPIK-2982-sdk-be-fe-implement-chat-prompt

2ccdb70

github-actions bot assigned andrescrz Nov 28, 2025

Merge branch 'main' into OPIK-2982-sdk-be-fe-implement-chat-prompt

90d6110

alexkuzmik dismissed aadereiko’s stale review via 90d6110 December 2, 2025 11:44

alexkuzmik added 2 commits December 2, 2025 13:01

Renamed migration scripts

ec51e48

Re-generate openapi spec

a926ac9

alexkuzmik removed the test-environment Deploy Opik adhoc environment label Dec 2, 2025

alexkuzmik added the test-environment Deploy Opik adhoc environment label Dec 2, 2025

[OPIK-2982] [P SDK] [BE] [FE] ChatPrompt support #3987

Are you sure you want to change the base?

[OPIK-2982] [P SDK] [BE] [FE] ChatPrompt support #3987

Uh oh!

Conversation

alexkuzmik commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Change checklist

Issues

Testing

Documentation

Uh oh!

github-actions bot commented Nov 7, 2025

📋 PR Linter Failed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 10, 2025

Uh oh!

github-actions bot commented Nov 10, 2025

📋 PR Linter Failed

Uh oh!

github-actions bot commented Nov 26, 2025

Uh oh!

CometActions commented Nov 26, 2025

Access Information

Uh oh!

github-actions bot commented Nov 27, 2025

Uh oh!

aadereiko left a comment

Choose a reason for hiding this comment

Uh oh!

aadereiko Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

aadereiko Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 27, 2025

Uh oh!

CometActions commented Nov 27, 2025

Access Information

Uh oh!

github-actions bot commented Nov 28, 2025

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

CometActions commented Dec 2, 2025

Access Information

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

alexkuzmik commented Nov 7, 2025 •

edited

Loading