Convert all previous examples for client tool call to use client effect; add a real client tool call example where the output matters (#39)

* location_select_mode is now a client effect instead of a client tool call

* Use client effects for fire-and-forget data pushes from server to client

* update metro map to have a real example of a client tool call

* updated readmes

Author: jiwon-oai

README.md

@@ -36,17 +36,18 @@ You can run the following examples:
   - `show_line_selector` presents the user a multiple-choice question using a widget.
   - Route-planning replies attach entity sources for the stations in the suggested path as annotations.
 
-### Client tool calls that mutate UI state
+### Client tool calls that mutate or fetch UI state
+
+- **Metro Map**:
+  - Client tool `get_selected_stations` pulls the currently selected nodes from the canvas so the agent can use client-side state in its response ([ChatKitPanel.tsx](examples/metro-map/frontend/src/components/ChatKitPanel.tsx), [metro_map_agent.py](examples/metro-map/backend/app/agents/metro_map_agent.py)).
+
+### Fire-and-forget client effects
 
 - **Cat Lounge**:
-  - Client tool `update_cat_status` is invoked by server tools `feed_cat`, `play_with_cat`, `clean_cat`, and `speak_as_cat` to sync UI state.
-  - When invoked, it is handled client-side with the `handleClientToolCall` callback in [ChatKitPanel.tsx](examples/cat-lounge/frontend/src/components/ChatKitPanel.tsx).
-- **News Guide**:
-  - The `open_article` widget action triggers client-side navigation to the selected story and forwards the action back to the server via `sendCustomAction` to follow up with context-aware prompts ([ChatKitPanel.tsx](examples/news-guide/frontend/src/components/ChatKitPanel.tsx)).
+  - Client effects `update_cat_status` and `cat_say` are invoked by server tools to sync UI state and surface speech bubbles; handled via `onEffect` in [ChatKitPanel.tsx](examples/cat-lounge/frontend/src/components/ChatKitPanel.tsx).
 - **Metro Map**:
-  - Client tools `add_station` (sent after the server adds a stop) and `location_select_mode` (sent after a line is chosen) update the metro map canvas ([ChatKitPanel.tsx](examples/metro-map/frontend/src/components/ChatKitPanel.tsx)).
-  - Inference is deliberately skipped after the `location_select_mode` client tool call output is sent to the server; the `respond` method on the server early returns when the last item in the thread is the `location_select_mode` client tool call ([server.py](examples/news-guide/backend/app/server.py)).
-  - The `location_select_mode` client tool call is streamed within the server action handler ([server.py](examples/news-guide/backend/app/server.py)).
+  - Client effect `location_select_mode` is streamed within the server action handler ([server.py](examples/metro-map/backend/app/server.py)) after a line is chosen and updates the metro map canvas ([ChatKitPanel.tsx](examples/metro-map/frontend/src/components/ChatKitPanel.tsx)).
+  - Client effect `add_station` is streamed by the agent after map updates to immediately sync the canvas and focus the newly created stop ([metro_map_agent.py](examples/metro-map/backend/app/agents/metro_map_agent.py), [ChatKitPanel.tsx](examples/metro-map/frontend/src/components/ChatKitPanel.tsx)).
 
 ### Page-aware model responses
 
@@ -60,7 +61,12 @@ You can run the following examples:
   - Retrieval tools stream `ProgressUpdateEvent` messages while searching tags, authors, keywords, exact text, or loading the current page so the UI surfaces “Searching…”/“Loading…” states ([news_agent.py](examples/news-guide/backend/app/agents/news_agent.py)).
   - The event finder emits progress as it scans dates, days of week, or keywords to keep users informed during longer lookups ([event_finder_agent.py](examples/news-guide/backend/app/agents/event_finder_agent.py)).
 - **Metro Map**:
-  - The metro agent emits a quick sync update when it loads the line data via `get_map` ([metro_map_agent.py](examples/metro-map/backend/app/agents/metro_map_agent.py)).
+  - The metro agent emits a progress update while retrieving map information in `get_map`; it also emits a progress update while waiting for a client tool call to complete in `get_selected_stations` ([metro_map_agent.py](examples/metro-map/backend/app/agents/metro_map_agent.py)).
+
+### Response lifecycle UI state
+
+- **Metro Map**:
+  - The client locks map interaction at response start and unlocks when the stream ends so canvas state doesn’t drift during agent updates by adding `onResponseStart` and `onResponseEnd` handlers ([ChatKitPanel.tsx](examples/metro-map/frontend/src/components/ChatKitPanel.tsx)).
 
 ### Widgets without actions
 
@@ -86,7 +92,7 @@ You can run the following examples:
 - **News Guide**:
   - The `view_event_details` action is processed server-side to update the timeline widget with expanded descriptions without a round trip to the model ([server.py](examples/news-guide/backend/app/server.py)).
 - **Metro Map**:
-  - The `line.select` action is handled server-side to stream an updated widget, add a `<LINE_SELECTED>` hidden context item to thread, stream an assistant message to ask the user whether to add the station at the line’s start or end, and trigger the `location_select_mode` client tool call for the UI to sync ([server.py](examples/metro-map/backend/app/server.py)).
+  - The `line.select` action is handled server-side to stream an updated widget, add a `<LINE_SELECTED>` hidden context item to thread, stream an assistant message to ask the user whether to add the station at the line’s start or end, and trigger the `location_select_mode` client effect for the UI to sync ([server.py](examples/metro-map/backend/app/server.py)).
 
 ### Annotations
 

examples/cat-lounge/README.md

@@ -21,6 +21,6 @@ Virtual cat caretaker demo built with ChatKit (FastAPI backend + Vite/React fron
 - Server tools to read and mutate per-thread cat state: `get_cat_status`, `feed_cat`, `play_with_cat`, `clean_cat`, `set_cat_name`, `speak_as_cat`.
 - Name suggestion workflow with a selectable widget and client-handled actions (`cats.select_name`, `cats.more_names`) plus server reconciliation for chosen names.
 - Profile card widget (`show_cat_profile`) streamed from the server with presentation-only content.
-- Client tool call `update_cat_status` keeps the UI stats in sync after each server tool invocation.
+- One-way client effects (`update_cat_status`, `cat_say`) are streamed from the server to keep the UI stats in sync and surface speech bubbles after each server tool invocation.
 - Hidden context tags track recent actions (<FED_CAT>, <PLAYED_WITH_CAT>, <CLEANED_CAT>, <CAT_NAME_SELECTED>) so the agent remembers what already happened.
 - Quick actions call `chatkit.sendUserMessage` to send canned requests without typing ([App.tsx](frontend/src/App.tsx)).

examples/cat-lounge/backend/README.md

@@ -5,10 +5,11 @@
 from typing import Annotated, Any, Callable
 
 from agents import Agent, RunContextWrapper, StopAtTools, function_tool
-from chatkit.agents import AgentContext, ClientToolCall
+from chatkit.agents import AgentContext
 from chatkit.types import (
     AssistantMessageContent,
     AssistantMessageItem,
+    ClientEffectEvent,
     HiddenContextItem,
     ThreadItemDoneEvent,
 )
@@ -94,12 +95,14 @@ async def _sync_status(
     state: CatState,
     flash: str | None = None,
 ) -> None:
-    ctx.context.client_tool_call = ClientToolCall(
-        name="update_cat_status",
-        arguments={
-            "state": state.to_payload(ctx.context.thread.id),
-            "flash": flash,
-        },
+    await ctx.context.stream(
+        ClientEffectEvent(
+            name="update_cat_status",
+            data={
+                "state": state.to_payload(ctx.context.thread.id),
+                "flash": flash,
+            },
+        )
     )
 
 
@@ -297,13 +300,22 @@ async def speak_as_cat(
     if not message:
         raise ValueError("A line is required for the cat to speak.")
     state = await _get_state(ctx)
-    ctx.context.client_tool_call = ClientToolCall(
-        name="cat_say",
-        arguments={
-            "state": state.to_payload(ctx.context.thread.id),
-            "message": message,
-        },
+    await ctx.context.stream(
+        ClientEffectEvent(
+            name="cat_say",
+            data={
+                "state": state.to_payload(ctx.context.thread.id),
+                "message": message,
+            },
+        )
     )
+    # ctx.context.client_tool_call = ClientToolCall(
+    #     name="cat_say",
+    #     arguments={
+    #         "state": state.to_payload(ctx.context.thread.id),
+    #         "message": message,
+    #     },
+    # )
 
 
 @function_tool(
@@ -362,27 +374,21 @@ async def suggest_cat_names(
         get_cat_status,
         # Produces a simple widget output.
         show_cat_profile,
-        # Invokes a simple client tool call to make the cat speak.
+        # Invokes a client effect to make the cat speak.
         speak_as_cat,
-        # Mutates state then invokes a client tool call to sync client state.
+        # Mutates state then invokes a client effect to sync client state.
         feed_cat,
         play_with_cat,
         clean_cat,
-        # Mutates both cat state and thread state then invokes a client tool call
+        # Mutates both cat state and thread state then invokes a client effect
         # to sync client state.
         set_cat_name,
         # Outputs interactive widget output with partially agent-generated content.
         suggest_cat_names,
     ],
-    # Stop inference after client tool calls or tool calls with widget outputs
-    # to prevent repetition.
+    # Stop inference after tool calls with widget outputs to prevent repetition.
     tool_use_behavior=StopAtTools(
         stop_at_tool_names=[
-            feed_cat.name,
-            play_with_cat.name,
-            clean_cat.name,
-            speak_as_cat.name,
-            set_cat_name.name,
             suggest_cat_names.name,
             show_cat_profile.name,
         ]

examples/cat-lounge/backend/app/cat_agent.py

@@ -17,6 +17,7 @@
     AssistantMessageItem,
     Attachment,
     HiddenContextItem,
+    StreamOptions,
     ThreadItemDoneEvent,
     ThreadItemReplacedEvent,
     ThreadMetadata,
@@ -110,6 +111,10 @@ async def respond(
             yield event
         return
 
+    def get_stream_options(self, thread: ThreadMetadata, context: dict[str, Any]) -> StreamOptions:
+        # Don't allow stream cancellation because most cat-lounge interactions update the cat's state.
+        return StreamOptions(allow_cancel=False)
+
     async def to_message_content(self, _input: Attachment) -> ResponseInputContentParam:
         raise RuntimeError("File attachments are not supported in this demo.")
 

examples/cat-lounge/backend/app/server.py

@@ -83,29 +83,23 @@ export function ChatKitPanel({
     [refresh, handleStatusUpdate, activeThread]
   );
 
-  const handleClientToolCall = useCallback((toolCall: {
+  const handleClientEffect = useCallback(({name, data}: {
     name: string;
-    params: Record<string, unknown>;
+    data: Record<string, unknown>;
   }) => {
-      if (toolCall.name === "update_cat_status") {
-        const data = toolCall.params.state as CatStatePayload | undefined;
-        if (data) {
-          handleStatusUpdate(data, toolCall.params.flash as string | undefined);
+      if (name === "update_cat_status") {
+        const catState = data.state as CatStatePayload | undefined;
+        if (catState) {
+          handleStatusUpdate(catState, data.flash as string | undefined);
         }
-        return { success: true };
       }
 
-      if (toolCall.name === "cat_say") {
-        const message = String(toolCall.params.message ?? "");
+      if (name === "cat_say") {
+        const message = String(data.message ?? "");
         if (message) {
-          setSpeech({
-            message,
-            mood: toolCall.params.mood as string | undefined,
-          });
+          setSpeech({message});
         }
-        return { success: true };
       }
-      return { success: false };
     }, [])
 
   const chatkit = useChatKit({
@@ -139,7 +133,6 @@ export function ChatKitPanel({
     widgets: {
       onAction: handleWidgetAction,
     },
-    onClientTool: handleClientToolCall,
     onThreadChange: ({ threadId }) => setThreadId(threadId),
     onError: ({ error }) => {
       // ChatKit handles displaying the error to the user
@@ -148,6 +141,7 @@ export function ChatKitPanel({
     onReady: () => {
       onChatKitReady?.(chatkit);
     },
+    onEffect: handleClientEffect,
   });
   chatkitRef.current = chatkit;
 

examples/cat-lounge/frontend/src/components/ChatKitPanel.tsx

@@ -13,13 +13,15 @@ Chat-driven GUI updates for a metro map using a React Flow canvas that lets the
 - "Add a new station named Aurora" (line picker widget will appear)
 - "Plan a route from Titan Border to Lyra Verge."
 - "Tell me about @Cinderia station." (@-mention stations; need to type @ manually, copy paste won't work)
+- "Tell me about the stations I've selected." (lasso some stations on the canvas first)
 
 ## Features
 
 - Map sync + lookup tools: `get_map`, `list_lines`, `list_stations`, `get_line_route`, `get_station` keep the agent grounded in the latest network data.
+- Selection-aware replies: the agent calls the `get_selected_stations` client tool to pull the user’s current canvas selection before continuing a response, handled in `onClientTool` ([ChatKitPanel.tsx](frontend/src/components/ChatKitPanel.tsx), [metro_map_agent.py](backend/app/agents/metro_map_agent.py)).
 - Plan-a-route responses attach entity sources for each station in the recommended path so ChatKit can keep the canvas focused on the stops being discussed.
 - Station creation flow: `show_line_selector` streams a clickable `line.select` widget, the server stashes `<LINE_SELECTED>`, and `add_station` triggers a widget update and a client tool call to refresh the canvas and focus the new stop.
-- Location placement helper: after a line is chosen, a `location_select_mode` client tool call flips the UI into placement mode so users pick start/end of line for insertion.
+- Location placement helper: after a line is chosen, a `location_select_mode` client effect flips the UI into placement mode so users pick start/end of line for insertion.
 - Progress updates: initial map fetch streams a quick progress event while loading line data.
 - Entity tags: station @-mentions in the composer add `<STATION_TAG>` content for the agent and can be clicked to focus the station on the canvas.
 - Custom header action: a right-side icon toggles dark/light themes in the ChatKit header.

examples/metro-map/README.md

@@ -10,6 +10,7 @@
     Annotation,
     AssistantMessageContent,
     AssistantMessageItem,
+    ClientEffectEvent,
     EntitySource,
     ProgressUpdateEvent,
     ThreadItemDoneEvent,
@@ -68,6 +69,8 @@
       When this is the latest message, acknowledge the selection.
     - <STATION_TAG>...</STATION_TAG> - contains full station details (id, name, description, coordinates, and served lines with ids/colors/orientations).
       Use the data inside the tag directly; do not call `get_station` just to resolve a tagged station.
+
+    When the user mentions "selected stations" or asks about the current selection, call `get_selected_stations` to fetch the station ids from the client.
 """
 
 
@@ -100,6 +103,10 @@ class StationDetailResult(BaseModel):
     lines: list[Line]
 
 
+class SelectedStationsResult(BaseModel):
+    station_ids: list[str]
+
+
 @function_tool(
     description_override=(
         "Show a clickable widget listing metro lines.\n"
@@ -229,12 +236,14 @@ async def add_station(
     await ctx.context.stream(ProgressUpdateEvent(text="Adding station..."))
     try:
         updated_map, new_station = ctx.context.metro.add_station(station_name, line_id, append)
-        ctx.context.client_tool_call = ClientToolCall(
-            name="add_station",
-            arguments={
-                "stationId": new_station.id,
-                "map": updated_map.model_dump(mode="json"),
-            },
+        await ctx.context.stream(
+            ClientEffectEvent(
+                name="add_station",
+                data={
+                    "stationId": new_station.id,
+                    "map": updated_map.model_dump(mode="json"),
+                },
+            )
         )
         return MapResult(map=updated_map)
     except Exception as e:
@@ -256,6 +265,24 @@ async def add_station(
         raise
 
 
+@function_tool(
+    description_override=(
+        "Fetch the ids of the currently selected stations from the client UI. No parameters."
+    )
+)
+async def get_selected_stations(
+    ctx: RunContextWrapper[MetroAgentContext],
+) -> SelectedStationsResult:
+    logger.info("[TOOL CALL] get_selected_stations")
+    # This progress update will persist while waiting for the client tool output to be send back to the server.
+    await ctx.context.stream(ProgressUpdateEvent(text="Fetching selected stations from the map..."))
+    ctx.context.client_tool_call = ClientToolCall(
+        name="get_selected_stations",
+        arguments={},
+    )
+    return SelectedStationsResult(station_ids=[])
+
+
 metro_map_agent = Agent[MetroAgentContext](
     name="metro_map",
     instructions=INSTRUCTIONS,
@@ -272,13 +299,15 @@ async def add_station(
         show_line_selector,
         # Update the metro map
         add_station,
+        # Request client selection
+        get_selected_stations,
     ],
     # Stop inference after client tool call or widget output
     tool_use_behavior=StopAtTools(
         stop_at_tool_names=[
-            add_station.name,
             plan_route.name,
             show_line_selector.name,
+            get_selected_stations.name,
         ]
     ),
 )