fix(slack): Stop an empty Seer mention before it runs an agent turn - #122941
Merged
sehr-m merged 1 commit intoAug 28, 2026
Conversation
`_handle_seer_prompt` rejects empty `text`, but it checks before the bot mention is stripped — that happens later, in `process_mention_for_slack` via `extract_prompt`. A bare "@seer" has truthy text `"<@U0BOT>"`, passes the guard, and reaches Seer as `query=""`. Seer then runs a full turn on nothing: an ExplorerIndex query, four context engine stages, three Vertex embedding calls and an LLM call, only to answer "It looks like your message came through empty!". The embedding calls 400 with `The text content is empty.` — 393 events in 30 days (SEER-8GR, plus LLM-PROXY-S on the proxy path), all of them `explorer.category_key:slack_thread`. Guard at the webhook instead, before `set_thread_status("is thinking...")` so no spinner is left running. The check is `has_prompt_content`, which asks whether anything remains besides mentions and whitespace, rather than reusing `extract_prompt`. `extract_prompt` needs the bot's user id, which comes from the event's `authorizations` — Slack omits that in some cases, and the existing `test_app_mention_dispatches_task_no_authorizations` covers exactly that. With no id the regex degrades to `<@>` and strips nothing, so a bare mention would have slipped through as the non-empty prompt `"<@U0BOT>"`. Another user's mention still counts as context whenever real text accompanies it; only a mention-only message is empty. Halting silently would have been a regression — the user used to get an answer, however wasteful. `send_empty_prompt_message` sends a fixed ephemeral nudge, mirroring `send_not_org_member_message`: no link button, send failures logged rather than raised so a Slack outage cannot turn the webhook into a 500. No model, no agent run, no Vertex call. Uses a distinct `EMPTY_PROMPT` halt reason rather than MISSING_EVENT_DATA, which is documented as meaning the event was malformed — a bare mention is a normal user action, and conflating them would make that metric misleading.
sehr-m
marked this pull request as ready for review
August 28, 2026 16:33
sehr-m
deleted the
sehrmoosabhoy/slack-empty-prompt-after-mention-strip
branch
August 28, 2026 16:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_handle_seer_promptalready rejects emptytext, but it checks before the bot mention is stripped — that happens later, inprocess_mention_for_slackviaextract_prompt. A bare@Seerhas truthy text"<@U0BOT>", so it passes the guard and reaches Seer asquery="".Seer then runs a full turn on nothing — an ExplorerIndex query, four context engine stages, three Vertex embedding calls and an LLM call — only to answer "It looks like your message came through empty!".
The embedding calls 400 with
The text content is empty.:ClientError: Empty instances./embeddingsVertex 400: "The text content is empty."All 393 are
explorer.category_key:slack_threadandexplorer_main_task. Slack is the only entrypoint affected — the Explorer chat endpoint already enforcesallow_blank=False, and every other caller templates its prompt, soqueryis structurally non-blank there.Fix
Guard at the webhook, before
set_thread_status("is thinking...")so no spinner is left running.The check is a new
has_prompt_content— is there anything besides mentions and whitespace — rather than reusingextract_prompt.extract_promptneeds the bot's user id, which comes from the event'sauthorizations; Slack omits that in some cases, andtest_app_mention_dispatches_task_no_authorizationsalready covers it. With no id the regex degrades to<@>and strips nothing, so a bare mention would have slipped through as the non-empty prompt"<@U0BOT>".Another user's mention still counts as context whenever real text accompanies it; only a mention-only message is empty.
Halting silently would have been a regression — the user used to get an answer, however wasteful.
send_empty_prompt_messagesends a fixed ephemeral nudge, mirroringsend_not_org_member_message: no link button, and send failures are logged rather than raised so a Slack outage can't turn the webhook into a 500. No model, no agent run, no Vertex call.Uses a distinct
EMPTY_PROMPThalt reason rather thanMISSING_EVENT_DATA, which is documented as meaning the event was malformed — a bare mention is a normal user action, and conflating them would make that metric misleading.Test
tests/sentry/integrations/slack/webhooks/events/+tests/sentry/seer/entrypoints/slack/— 201 passed. mypy and pre-commit clean.New coverage: bare mention halts and sends the nudge; halts with no
authorizations; mention-only-with-other-users halts; a mention plus real text still dispatches; a failing nudge still returns 200 without dispatching.Notes for review
process_mention_for_slackre-derives the prompt andtrigger_agentforwards it, neither re-checks — so invoking the task directly still bypasses it.authorizationsis missing, the<@U0BOT>token still rides along into the query as prompt noise. Storing the bot user id on the integration would close that; it's a prompt-quality issue, not correctness, now that this guard doesn't depend on the id.🤖 Generated with Claude Code