Skip to content

ChatGPT: keep the map open next to the chat and let the model draw on it - #333

Open
zmofei wants to merge 7 commits into
mainfrom
feat/chatgpt-thread-entrypoint
Open

zmofei wants to merge 7 commits into
mainfrom
feat/chatgpt-thread-entrypoint

Conversation

@zmofei

@zmofei zmofei commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

In ChatGPT, render_map_tool opens its map as a tab next to the chat, where it stays open and works with the conversation. Hosts without the OpenAI MCP Extensions ignore the new metadata.

  • Open it yourself: render_map_tool is a thread entrypoint. "+" → "More tools" → Mapbox Map opens an empty map.
    • 2026-11-16 15 04 16
  • Map to chat: the open map shares its view through ui/update-model-context, and the selected place as a titled item in the composer, so "what's around here?" and "what is this?" work. A place can be a pin, a search result, or a label of the Standard base map: points of interest, place names, landmark icons and indoor labels. A selected label is highlighted; a second click deselects it.
    • 2026-11-16 15 07 48
  • Chat to map (WebMCP): the map registers redraw_map, search_map, add_places, move_map and read_map, so follow-ups draw on the open map instead of opening another. redraw_map takes render_map_tool's arguments.
    • 2026-11-16 15 12 55
  • The server instructions, the render_map_tool description, the render hint and the map's shared context name these tools, each saying it applies in ChatGPT, so other clients aren't sent looking for tools they don't have. In ChatGPT, the result of a call that opens a map names them too.

Verified in ChatGPT Desktop with a local dev server: the map opens next to the chat and from the thread tab bar; the model answers from the shared view; in Codex, follow-ups change the open map; clicking base map labels selects, switches and deselects them, and the selection stays after a message is sent. With the tools named directly (no read_map call first), "show coffee shops on it" drew on the open map through search_map, without a second render_map_tool call. Unit tests, typecheck and lint pass.

Known limits:

  • Road, water and address labels can't be selected: Standard doesn't offer them as featuresets.
  • ChatGPT sends the same context-cleared notification when a message is sent and when the user removes the item, so removing the item doesn't deselect the place on the map.
  • The WebMCP tools work only in ChatGPT Desktop. With several maps open, the model picks the map from the conversation, not the tab in view: after switching to another map's tab, "show me hotels" still drew on the map discussed last. Naming the map ("on the first map") works. ChatGPT on the web (Work and classic) doesn't support WebMCP, so there each follow-up opens another map tab; the map still opens next to the chat and shares its view and the selected place.
  • Once, the model said the map's tools weren't available.
  • Once, the map tab showed "Tab content couldn't render" after a message was sent; the cause isn't known yet.
  • On a phone there is no room beside the chat, so ChatGPT shows a button in the reply instead of the map. Tapping it opens the map fullscreen, and the chat continues from there. The map is one tap further away than the inline map it replaces.

@zmofei
zmofei force-pushed the feat/chatgpt-sidebar-app branch from e963e07 to 9c9b9d5 Compare November 11, 2026 10:58
zmofei added a commit that referenced this pull request Nov 13, 2026
…l draw on it

Ports the map experience of #333 onto main without the sidebar home page
(#316): the thread entrypoint and explorer, shared view and selected place,
base map POI selection, and the WebMCP tools that act on the open map.
@zmofei
zmofei force-pushed the feat/chatgpt-thread-entrypoint branch from 8f395ae to 1f121f5 Compare November 13, 2026 11:21
@zmofei zmofei changed the title Spike: open Mapbox Maps next to a chat and let the model draw on it ChatGPT: keep the map open next to the chat and let the model draw on it Nov 13, 2026
@zmofei
zmofei changed the base branch from feat/chatgpt-sidebar-app to main November 13, 2026 11:22
@zmofei
zmofei force-pushed the feat/chatgpt-thread-entrypoint branch 2 times, most recently from 115f55e to 7da00fc Compare November 13, 2026 12:46
…l draw on it

In ChatGPT, render_map_tool opens its map as a tab next to the chat that
stays open:

- A thread entrypoint opens an empty map.
- The map shares its view and the selected place (a pin, a search result
  or a Standard base map label) as model context.
- WebMCP tools (redraw_map, search_map, add_places, move_map, read_map) let
  the model change the open map instead of opening another.
- The server instructions, tool description and results point the model at
  those tools.
@zmofei
zmofei force-pushed the feat/chatgpt-thread-entrypoint branch from 7da00fc to 5634bda Compare November 13, 2026 13:00
…apAppHtml.ts

mapAppHtml.ts had grown past 3,300 lines. The new map page code now lives in
three files whose scripts renderMapAppHtml splices into the page's single
<script>. The generated page's code is unchanged.
ChatGPT on the web offers document.modelContext but doesn't give the
registered tools to the model, so a map that listed them in its context had
the model explain that the tools it was told about weren't callable. The
context no longer lists them, and the server texts say ChatGPT Desktop.
…ender_map_tool an icon

On a phone, ChatGPT's title bar covered the map's zoom buttons and summary.
The map now places its controls, labels and cards inside
hostContext.safeAreaInsets, while the map itself runs underneath, and
inline it asks for no more than containerDimensions.maxHeight. A square
Mapbox mark icon replaces the wide server logo in ChatGPT's "+" menu.
@zmofei
zmofei marked this pull request as ready for review November 16, 2026 13:44
@zmofei
zmofei requested a review from a team as a code owner November 16, 2026 13:44

@mattpodwysocki mattpodwysocki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this, keeping the map open next to the chat is a big step up. I ran a review pass and left six comments. The first one (an empty map for any call with no drawable data, which also drops the camera) is the one I'd fix before merging. The rest are smaller or questions.

A few minor things too, not commented inline: size-changed can go out before ui/notifications/initialized (applyHostContext in mapAppHtml.ts), selecting the card that's already active doesn't clear dismissedPlaceKey, the Satellite toggle loses the selected label's feature state, and placesOnMap / pinnedPlace skip coord() rounding so placeKey can differ from labels.

// ChatGPT sends the same null when the user removes the item and when
// a sent message uses it up, so the map keeps its selection either way.
dismissedPlaceKey = placeKey(selectedPlace());
lastSharedContext = '';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question more than a bug: after a sent message uses up the context, nothing shares the view again until the user pans or zooms. So "what's around here?" works, but a follow-up like "and restaurants here?" goes without the view unless the map moved. I get that re-sharing right away would put back a chip the user just removed, since ChatGPT sends the same null for both. Is the once-per-pan behavior what we want, or is there a way to tell the two apart (or re-share just the view, without the place)?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this in Codex on Desktop. After a send uses up the context, a follow-up still reaches the open app instance: the model can read the open map or the earlier turns in the history. With a pin selected, I sent "what neighborhood is this?" (context attached), then, without touching the map, "what's the nearest metro station to here?" with no context attached, and the answer was still about the selected pin. So I'd keep the once-per-pan behavior, since re-sharing right after a send would put back a chip the user may have just removed. Not tested on ChatGPT web.

Comment thread src/resources/ui-apps/mapAppToolsJs.ts Resolved
tool.execute = function(input) {
return Promise.resolve(input).then(run);
};
try { document.modelContext.registerTool(tool); } catch (_) { /* already registered */ }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With more than one map in the conversation (an older inline one plus the one open next to the chat), which one ends up with these tools? If ChatGPT keeps the first registration and this catch swallows the rest, "show coffee shops on it" could draw on a map the user scrolled past. Could we register only from maps that stay open (explorerMode or fullscreen), or unregister when a map closes?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this in Codex on Desktop with two map tabs open (Helsinki, then Paris). Each map keeps its own tools: "show coffee shops on it" drew on Paris, and "show me hotels on the first map" drew on Helsinki, both keeping their earlier pins. So the first registration doesn't win, and I'd leave the registration as is.

One thing it did show: the model picks the map from the conversation, not the tab in view. After I switched to the Helsinki tab, "show me hotels" still went to Paris. Updated the known limits to say that.

Not tested: an older inline map next to an open one.

- render_map_tool opens the empty map only for a call with no arguments.
  A camera alone or empty payload_refs is an error again, as on main.
- A context clear from the host that arrives while a share request is in
  flight is applied once the request settles, instead of being dropped.
- search_map no longer adds a "Search center" pin, and a search that finds
  nothing leaves the map as it was. It also hides the error a failed
  search left behind.
- The hint in a ChatGPT result no longer assumes the map is open next to
  the chat or that its tools exist.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants