Skip to content

docs: browser-use 1.0 judges every run (result.judgement, judge, judge_context) - #421

Open
mathisdittrich wants to merge 8 commits into
mainfrom
hannes/docs-oss-judge
Open

mathisdittrich wants to merge 8 commits into
mainfrom
hannes/docs-oss-judge

Conversation

@mathisdittrich

@mathisdittrich mathisdittrich commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

browser-use/browser-use#6398 (merged 2026-11-14) brings the judge back in the open-source library: after each run that took a step, one low-effort request to the agent's own model fills result.judgement (plus one text-only retry if the model rejects the images with an HTTP 4xx, per #6403). The open-source docs still said the judge was removed (use_judge ignored, ground_truth / judgement() → result.success).

Changes, all facts from the #6398 diff (MIGRATING.md, README.md, agent/service.py, agent/judge.py, agent/views.py, legacy.py, telemetry/):

  • all-parameters: new judge (default True) and judge_context (default None) rows; use_judge removed from the ignored list (it is now an alias, popped from kwargs, no longer in IGNORED).
  • output: judgement field, cost_usd includes the judge, new "Judge" section with the Judgement fields.
  • migrating: use_judge / ground_truth kept as judge / judge_context; judge_llm dropped; judgement() is now the result.judgement attribute; browser_use.agent.judge JudgementResult / SimpleJudgeResult point to Judgement.
  • observability / telemetry: cost covers the judge; agent_event carries the scrubbed judgement (schema 5), only whether judge_context is set is sent as configuration; the verdict is scrubbed text, so short or partial quotes of the context can remain.

⚠️ Merge after the next browser-use release. #6398 merged after 1.0.4 (PyPI 2026-11-13 20:56 UTC), and the 1.0.4 wheel has neither judge nor judge_context, so pip install browser-use users would see params that do not exist yet. (The docs-oss-params CI installs browser-use from GitHub main, so it already passes.)

Not documented (unsure): the exact warning text a failed judge logs, and whether the CLI/MCP server expose the judgement.

🤖 Generated with Claude Code


Summary by cubic

Updates the open-source docs to reflect that browser-use 1.0 judges every run that took a step, filling result.judgement with the verdict.

  • Documents judge (default True) and judge_context, plus result.judgement and its fields; gotchas are also written on success when a site rejected, errored or blocked the agent, and if the model rejects images with an HTTP 4xx error that mentions image, the judge retries once with text alone.
  • use_judge and ground_truth stay as aliases, judge_llm is dropped, and judgement() becomes the result.judgement attribute.
  • The judge reads the run as redacted text capped at 120k characters, plus up to 4 screenshots (the run's last distinct ones and the final page) when vision is on; a failed judge request leaves result.judgement None and never changes the run's own result.
  • Notes the judge's cost in cost_usd and the scrubbed verdict in telemetry, with the 300-character caps that apply per field; judge_context itself is never sent, though partial quotes of it can survive in the scrubbed verdict.

Merge after the next browser-use release. browser-use/browser-use#6398 merged after 1.0.4, and PyPI 1.0.4 has neither judge nor judge_context; the parameter check CI installs from main and passes.

Written for commit fc46e1d. Summary will update on new commits.

View guided diff Turn on auto-fix

…e_context)

Source: browser-use/browser-use#6398.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
browser-use 🟢 Ready View Preview Nov 14, 2026, 9:21 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 5 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/open-source/telemetry.mdx Resolved
Comment thread docs/open-source/output.mdx Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… case

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/open-source/telemetry.mdx Resolved
…ss (browser-use#6403)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/open-source/output.mdx Resolved
HannesCC and others added 2 commits November 14, 2026 00:23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/open-source/all-parameters.mdx Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
staging - docs — fc46e1d4 Deployed Nov 14, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant