Chat input becomes very laggy while the LLM response is streaming (fine during tool calls, worst late in a session)

[Broccolito]

Summary

Typing in the chat box becomes very laggy while the API is streaming a response from the LLM. Keystrokes render with visible delay, making it hard to compose the next message during a running turn.

Symptom profile

• During tool calls: typing is totally fine — no lag.
• During API streaming (tokens arriving): typing becomes very laggy.
• Worst near the end of an agent session, when the API is heavily streaming and/or receiving many tool results back-to-back. The longer the conversation, the worse the lag.

Expected behavior

The chat input should stay responsive regardless of what the stream is doing. Streaming render updates should never compete with keystroke latency.

Likely cause (from a quick code read)

• ui/desktop/src/hooks/chatStreamStore.tsx notifies subscribers synchronously on every incoming stream event (notify() per message update), with no throttling, debouncing, or rAF batching anywhere in the store or in useChatStream.ts.
• BaseChat.tsx (~2,100 lines) subscribes via useSyncExternalStore and re-renders on every notification; ChatInput.tsx (~1,850 lines) lives inside that tree, so each streamed token forces a re-render of the entire chat view — including the input the user is typing into.
• This matches the symptom profile: tool-call phases emit few events (no lag), token streaming emits dozens of notifications per second (lag), and late in a session the message list is largest so each re-render is most expensive (worst lag).

Plausible fixes: batch/throttle stream notifications (e.g. one flush per animation frame), and/or isolate ChatInput from message-list re-renders (memoization / separate store slice) and virtualize the transcript so render cost stops growing with conversation length.

Environment

• BioRouter desktop app, v1.88.3 (also observed on the current release line)
• macOS