Skip to content

Commit 3751521

Browse files
committed
Document how a handler deals with cancellation
Add a Cancellation page under "Inside your handler". When the client gives up on a call the SDK cancels the handler, and the page shows the two cases that need code, using public API only: an `async def` tool cleaning up in a `finally` with a shielded, time-limited scope, and a plain `def` tool calling `anyio.from_thread.check_cancelled()` between units of work. It also names the two Streamable HTTP options under which the handler is not cancelled. The page is linked from the section index, the Progress page and the `async def` section of Tools. The tests run both tutorials against a client that gives up, in process, over a JSON-RPC stream and over Streamable HTTP. Fixes #3389
1 parent ebf6e5a commit 3751521

9 files changed

Lines changed: 327 additions & 2 deletions

File tree

‎docs/handlers/cancellation.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Cancellation
2+
3+
A client can give up on a call: the user pressed stop, or a timeout ran out.
4+
5+
When it does, the SDK **cancels your handler**. The `await` it is waiting on raises, the function unwinds, and nothing it returns is sent. Most handlers need to do nothing about that.
6+
7+
Two kinds do: a handler with something to clean up, and a handler that is a plain `def`.
8+
9+
## Clean up in an `async def` tool
10+
11+
Put the cleanup in a `finally`:
12+
13+
```python title="server.py" hl_lines="23 26-28"
14+
--8<-- "docs_src/cancellation/tutorial001.py"
15+
```
16+
17+
* The `finally` runs however the tool ends: it returned, it raised, or it was cancelled.
18+
* Cleanup that has to `await` needs `shield=True`. In a cancelled handler every further `await` raises too, so without the shield `release_hold` would stop at its first line.
19+
* Nothing can cancel a shielded block, so give it a time limit. Here that is `5` seconds.
20+
21+
!!! tip
22+
Reach for `finally`, not `except`. The cancellation has to keep travelling up once your cleanup
23+
is done, and a `finally` lets it.
24+
25+
## Stop early in a plain `def` tool
26+
27+
A plain `def` tool runs in a thread, and nothing can interrupt a thread from outside. The tool has to ask:
28+
29+
```python title="server.py" hl_lines="18"
30+
--8<-- "docs_src/cancellation/tutorial002.py"
31+
```
32+
33+
* `anyio.from_thread.check_cancelled()` does nothing while the call is live, and raises once it has been cancelled. Call it between units of work.
34+
* A `def` tool that never asks runs to the end, and its result is thrown away.
35+
36+
## Where it applies
37+
38+
Prompt and resource functions are cancelled exactly like tools.
39+
40+
It works the same over stdio and Streamable HTTP. With this SDK's `Client`, giving up means cancelling the task that awaits `call_tool`, or letting its `read_timeout_seconds` run out.
41+
42+
!!! warning
43+
Two Streamable HTTP options keep the news from your handler: `json_response=True` on a
44+
`2026-07-28` connection, and `stateless_http=True` on a legacy one. There the handler runs to
45+
the end whatever the client did.
46+
47+
## Recap
48+
49+
* When the client gives up on a call, the SDK cancels the handler: tool, prompt or resource.
50+
* `async def`: clean up in a `finally`, and put cleanup that awaits inside `anyio.move_on_after(seconds, shield=True)`.
51+
* Plain `def`: call `anyio.from_thread.check_cancelled()` between units of work, or the tool runs to the end.
52+
* `json_response=True` (modern connections) and `stateless_http=True` (legacy ones) switch cancellation off.
53+
54+
Progress and cancellation are between a running tool and its *caller*. The lines it logs for *you*, the person operating the server, are a different channel: **[Logging](logging.md)**.

‎docs/handlers/index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ What it can do while it runs:
2222
**[Sampling and roots](sampling-and-roots.md)**, deprecated but still
2323
served.
2424
* Report **[Progress](progress.md)** on something slow.
25+
* Clean up, or stop early, when the client gives up on the call, with
26+
**[Cancellation](cancellation.md)**.
2527
* Write logs (to standard error, for whoever operates the server) with
2628
**[Logging](logging.md)**.
2729
* Tell subscribed clients that something changed with

‎docs/handlers/progress.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,4 +115,4 @@ The callback receives `total=None`. A client can still show *activity* ("3 impor
115115
* No callback on the call means `report_progress` does nothing. Report unconditionally.
116116
* Omit `total` when you don't know it; the callback gets `None`.
117117

118-
Progress is what a running tool shows the *user*. The lines it logs for *you*, the person operating the server, are a different channel: **[Logging](logging.md)**.
118+
Progress is for a client that is still waiting. What your tool sees when the client stops waiting is **[Cancellation](cancellation.md)**.

‎docs/servers/tools.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ You can mix and match: plain parameters next to model parameters, nested models,
136136

137137
If a tool does I/O (calls an API, reads a file, queries a database), declare it `async def` and `await` inside it. The SDK awaits it.
138138

139-
A plain `def` tool works too: the SDK runs it in a thread so it never blocks the server.
139+
A plain `def` tool works too: the SDK runs it in a thread so it never blocks the server. A long one can check whether the client is still waiting; see **[Cancellation](../handlers/cancellation.md)**.
140140

141141
There is nothing else to configure.
142142

‎docs_src/cancellation/__init__.py‎

Whitespace-only changes.
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import anyio
2+
3+
from mcp.server import MCPServer
4+
5+
mcp = MCPServer("Bookshop")
6+
7+
holds: set[str] = set()
8+
9+
10+
async def take_payment(title: str) -> None:
11+
await anyio.sleep(30) # the customer is typing a card number
12+
13+
14+
async def release_hold(title: str) -> None:
15+
await anyio.sleep(0.1) # a round trip to the stock system
16+
holds.discard(title)
17+
18+
19+
@mcp.tool()
20+
async def order_book(title: str) -> str:
21+
"""Hold a copy of a book while the customer pays for it."""
22+
holds.add(title)
23+
try:
24+
await take_payment(title)
25+
return f"Ordered {title!r}."
26+
finally:
27+
with anyio.move_on_after(5, shield=True):
28+
await release_hold(title)
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import time
2+
3+
import anyio.from_thread
4+
5+
from mcp.server import MCPServer
6+
7+
mcp = MCPServer("Bookshop")
8+
9+
10+
def index_book(title: str) -> None:
11+
time.sleep(1) # slow work with nothing to await
12+
13+
14+
@mcp.tool()
15+
def rebuild_index(titles: list[str]) -> str:
16+
"""Rebuild the search index, one book at a time."""
17+
for title in titles:
18+
anyio.from_thread.check_cancelled()
19+
index_book(title)
20+
return f"Indexed {len(titles)} books."

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ nav:
4040
- Multi-round-trip requests: handlers/multi-round-trip.md
4141
- Sampling and roots: handlers/sampling-and-roots.md
4242
- Progress: handlers/progress.md
43+
- Cancellation: handlers/cancellation.md
4344
- Logging: handlers/logging.md
4445
- Subscriptions: handlers/subscriptions.md
4546
- Running your server:
Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
"""`docs/handlers/cancellation.md`: every claim the page makes, proved against the real SDK."""
2+
3+
import threading
4+
from collections.abc import Awaitable, Callable
5+
6+
import anyio
7+
import anyio.from_thread
8+
import pytest
9+
from mcp_types import REQUEST_TIMEOUT
10+
11+
from docs_src.cancellation import tutorial001, tutorial002
12+
from mcp import Client, MCPError
13+
from mcp.client.streamable_http import streamable_http_client
14+
from mcp.server import MCPServer
15+
from tests.interaction._connect import BASE_URL, mounted_app
16+
17+
# See test_index.py for why this is a per-module mark and not a conftest hook.
18+
pytestmark = [pytest.mark.anyio, pytest.mark.filterwarnings("error::mcp.MCPDeprecationWarning")]
19+
20+
# "auto" dispatches in process; "legacy" puts a JSON-RPC stream, and so a cancellation message, in between.
21+
both_connections = pytest.mark.parametrize("mode", ["auto", "legacy"])
22+
23+
24+
async def abandon(
25+
call: Callable[[], Awaitable[object]], started: anyio.Event, then: Callable[[], object] = lambda: None
26+
) -> None:
27+
"""Start `call`, cancel the task awaiting it once `started` is set, let the server settle, then run `then`."""
28+
scope = anyio.CancelScope()
29+
30+
async def doomed() -> None:
31+
with scope:
32+
await call()
33+
raise NotImplementedError # unreachable: the call never resolves
34+
35+
async with anyio.create_task_group() as tg:
36+
tg.start_soon(doomed)
37+
await started.wait()
38+
scope.cancel()
39+
await anyio.wait_all_tasks_blocked()
40+
then()
41+
42+
43+
@pytest.fixture
44+
def payment_started(monkeypatch: pytest.MonkeyPatch) -> anyio.Event:
45+
"""Replace tutorial001's `take_payment` with one that says the tool reached it and then never finishes."""
46+
started = anyio.Event()
47+
48+
async def take_payment(title: str) -> None:
49+
assert title in tutorial001.holds
50+
started.set()
51+
await anyio.sleep_forever()
52+
53+
monkeypatch.setattr(tutorial001, "take_payment", take_payment)
54+
return started
55+
56+
57+
@pytest.fixture
58+
def hold_released(monkeypatch: pytest.MonkeyPatch) -> anyio.Event:
59+
"""Wrap tutorial001's own `release_hold` so the test can wait for it to reach its last line."""
60+
released = anyio.Event()
61+
release_hold = tutorial001.release_hold
62+
63+
async def announcing_release_hold(title: str) -> None:
64+
await release_hold(title)
65+
released.set()
66+
67+
monkeypatch.setattr(tutorial001, "release_hold", announcing_release_hold)
68+
return released
69+
70+
71+
@both_connections
72+
async def test_abandoning_the_call_runs_the_shielded_cleanup_to_the_end(
73+
mode: str, payment_started: anyio.Event, hold_released: anyio.Event
74+
) -> None:
75+
"""tutorial001: the client gives up mid-payment, and the `finally` still awaits `release_hold` to completion."""
76+
with anyio.fail_after(5):
77+
async with Client(tutorial001.mcp, mode=mode) as client:
78+
await abandon(lambda: client.call_tool("order_book", {"title": "Dune"}), payment_started)
79+
await hold_released.wait()
80+
assert tutorial001.holds == set()
81+
82+
83+
@pytest.mark.parametrize("mode", ["2026-07-28", "legacy"])
84+
async def test_abandoning_the_call_over_streamable_http_runs_the_cleanup_too(
85+
mode: str, payment_started: anyio.Event, hold_released: anyio.Event
86+
) -> None:
87+
"""The last section: with the default options, either era's way of cancelling over HTTP reaches tutorial001."""
88+
with anyio.fail_after(5):
89+
async with (
90+
mounted_app(tutorial001.mcp) as (http, _),
91+
Client(streamable_http_client(f"{BASE_URL}/mcp", http_client=http), mode=mode) as client,
92+
):
93+
await abandon(lambda: client.call_tool("order_book", {"title": "Dune"}), payment_started)
94+
await hold_released.wait()
95+
# Let the legacy transport's late answer to the abandoned call land while the client is still open.
96+
await anyio.wait_all_tasks_blocked()
97+
assert tutorial001.holds == set()
98+
99+
100+
@both_connections
101+
async def test_a_client_timeout_cancels_the_tool_the_same_way(
102+
mode: str, payment_started: anyio.Event, hold_released: anyio.Event
103+
) -> None:
104+
"""The last section: `read_timeout_seconds` running out is the other way this SDK's client gives up."""
105+
with anyio.fail_after(5):
106+
async with Client(tutorial001.mcp, mode=mode) as client:
107+
with pytest.raises(MCPError) as exc_info:
108+
# The tool never answers, so any positive timeout expires; this one adds no wall-clock time.
109+
await client.call_tool("order_book", {"title": "Dune"}, read_timeout_seconds=0.000001)
110+
await hold_released.wait()
111+
assert exc_info.value.error.code == REQUEST_TIMEOUT
112+
assert payment_started.is_set()
113+
assert tutorial001.holds == set()
114+
115+
116+
@both_connections
117+
async def test_check_cancelled_stops_a_def_tool_at_its_next_check(mode: str, monkeypatch: pytest.MonkeyPatch) -> None:
118+
"""tutorial002: cancelled during the first book, the loop raises at its next check and indexes no more."""
119+
started = anyio.Event()
120+
resume = threading.Event()
121+
indexed: list[str] = []
122+
123+
def index_book(title: str) -> None:
124+
indexed.append(title)
125+
anyio.from_thread.run_sync(started.set)
126+
assert resume.wait(5)
127+
128+
monkeypatch.setattr(tutorial002, "index_book", index_book)
129+
titles = ["Dune", "Emma", "Ulysses"]
130+
with anyio.fail_after(5):
131+
# Leaving the block waits for the tool's thread, so `indexed` is final after it.
132+
async with Client(tutorial002.mcp, mode=mode) as client:
133+
await abandon(lambda: client.call_tool("rebuild_index", {"titles": titles}), started, then=resume.set)
134+
assert indexed == ["Dune"]
135+
136+
137+
async def test_a_def_tool_that_never_checks_runs_to_the_end() -> None:
138+
"""The `def` section's last bullet: nothing interrupts the thread, so the tool outlives its own cancellation."""
139+
started = anyio.Event()
140+
resume = threading.Event()
141+
finished: list[str] = []
142+
mcp = MCPServer("Bookshop")
143+
144+
@mcp.tool()
145+
def rebuild_index() -> str:
146+
anyio.from_thread.run_sync(started.set)
147+
assert resume.wait(5)
148+
finished.append("rebuild_index")
149+
return "Indexed 3 books."
150+
151+
with anyio.fail_after(5):
152+
async with Client(mcp, mode="legacy") as client:
153+
await abandon(lambda: client.call_tool("rebuild_index", {}), started, then=resume.set)
154+
assert finished == ["rebuild_index"]
155+
156+
157+
async def test_prompt_and_resource_functions_are_cancelled_like_tools() -> None:
158+
"""The last section: a prompt or a resource function parked on an `await` is cancelled when the client gives up."""
159+
started = {"blurb": anyio.Event(), "stock": anyio.Event()}
160+
cancelled = {"blurb": anyio.Event(), "stock": anyio.Event()}
161+
mcp = MCPServer("Bookshop")
162+
163+
async def park(name: str) -> str:
164+
started[name].set()
165+
try:
166+
await anyio.sleep_forever()
167+
finally:
168+
cancelled[name].set()
169+
raise NotImplementedError # unreachable: only cancellation ends the sleep
170+
171+
@mcp.prompt()
172+
async def blurb() -> str:
173+
return await park("blurb")
174+
175+
@mcp.resource("stock://all")
176+
async def stock() -> str:
177+
return await park("stock")
178+
179+
with anyio.fail_after(5):
180+
async with Client(mcp, mode="legacy") as client:
181+
await abandon(lambda: client.get_prompt("blurb"), started["blurb"])
182+
await cancelled["blurb"].wait()
183+
await abandon(lambda: client.read_resource("stock://all"), started["stock"])
184+
await cancelled["stock"].wait()
185+
186+
187+
@pytest.mark.parametrize(
188+
("json_response", "stateless_http", "mode"),
189+
[(True, False, "2026-07-28"), (False, True, "legacy")],
190+
ids=["json_response-modern", "stateless_http-legacy"],
191+
)
192+
async def test_two_http_options_keep_the_cancellation_from_the_handler(
193+
json_response: bool, stateless_http: bool, mode: str
194+
) -> None:
195+
"""The `!!! warning`: on these two pairings the abandoned tool is still there once everything has settled.
196+
197+
Pins known gaps. A stateless legacy server has no session in which to find the request that
198+
`notifications/cancelled` names. In JSON-response mode the 2026-07-28 entry does not watch for
199+
the disconnect that is that revision's cancellation signal; if that arm starts timing out, the
200+
gap was closed and the warning should lose that half.
201+
"""
202+
started = anyio.Event()
203+
resume = anyio.Event()
204+
finished = anyio.Event()
205+
mcp = MCPServer("Bookshop")
206+
207+
@mcp.tool()
208+
async def order_book() -> str:
209+
started.set()
210+
await resume.wait()
211+
finished.set()
212+
return "Ordered."
213+
214+
with anyio.fail_after(5):
215+
async with (
216+
mounted_app(mcp, json_response=json_response, stateless_http=stateless_http) as (http, _),
217+
Client(streamable_http_client(f"{BASE_URL}/mcp", http_client=http), mode=mode) as client,
218+
):
219+
await abandon(lambda: client.call_tool("order_book", {}), started, then=resume.set)
220+
await finished.wait()

0 commit comments

Comments
 (0)