Skip to content

🤖 Update Mock Benchmark Results#5828

Merged
thomhurst merged 1 commit intomainfrom
automated-mock-benchmarks-update
May 4, 2026
Merged

🤖 Update Mock Benchmark Results#5828
thomhurst merged 1 commit intomainfrom
automated-mock-benchmarks-update

Conversation

@thomhurst
Copy link
Copy Markdown
Owner

Automated Mock Benchmark Update

This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • mock-benchmark-MockCreation
  • mock-benchmark-Setup
  • mock-benchmark-Invocation
  • mock-benchmark-Verification
  • mock-benchmark-Callback
  • mock-benchmark-CombinedWorkflow
  • mock-benchmark-summary (aggregated overview)

Libraries Compared

Libraries are discovered dynamically from the benchmark output. See the generated summary.json for the full list.

Changes

  • Updated mock benchmark data in docs/static/benchmarks/mocks/latest.json
  • Regenerated mock benchmark documentation in docs/docs/benchmarks/mocks/
  • Updated mock benchmark summary in docs/static/benchmarks/mocks/summary.json

Workflow Run

  • Run ID: 25299172670
  • Triggered: schedule

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels May 4, 2026
@thomhurst thomhurst temporarily deployed to Pull Requests May 4, 2026 03:27 — with GitHub Actions Inactive
@thomhurst thomhurst temporarily deployed to Pull Requests May 4, 2026 03:27 — with GitHub Actions Inactive
@thomhurst thomhurst temporarily deployed to Pull Requests May 4, 2026 03:27 — with GitHub Actions Inactive
@thomhurst thomhurst merged commit 830ff54 into main May 4, 2026
12 of 13 checks passed
@thomhurst thomhurst deleted the automated-mock-benchmarks-update branch May 4, 2026 03:27
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an automated benchmark data update — no source code changes. A few observations:

Overall: LGTM for an automated data PR.

Structural Change Worth Noting

lost the Median field from all entries in this update. All other JSON files retain it. This schema inconsistency could cause issues if anything consuming this JSON expects Median to be present on all benchmark results.

If the benchmark runner conditionally emits Median (e.g., only when enough samples are collected, or only for certain run configs), the automation workflow should either:

  • Normalize the output schema before committing, or
  • Document that Median is optional and ensure all consumers handle its absence gracefully.

Benchmark Results

Numbers look like routine variance from a nightly run. The +30% regression in Moq's CombinedWorkflow (310 → 403 μs) is notable but consistent with Moq's typical benchmarked behavior — TUnit.Mocks continues to show ~250x better performance in that category (1.64 μs vs 403 μs).

No Issues With the Automation Itself

The PR description is well-structured and the file set is consistent with the workflow intent. The docs/ scope is correctly bounded.

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an automated benchmark data update — no source code changes.

Overall: LGTM for an automated data PR.

Structural Change Worth Noting

docs/static/benchmarks/mocks/Setup.json lost the Median field from all entries in this update. All other JSON files retain it. This schema inconsistency could cause issues if anything consuming this JSON expects Median to be present on all benchmark results.

If the benchmark runner conditionally emits Median, the automation workflow should either:

  • Normalize the output schema before committing, or
  • Document that Median is optional and ensure all consumers handle its absence gracefully.

Benchmark Results

Numbers look like routine variance from a nightly run. The +30% regression in Moq CombinedWorkflow (310 → 403 μs) is notable but consistent with expected behavior — TUnit.Mocks continues to show ~250x better performance in that category (1.64 μs vs 403 μs).

No Issues With the Automation

The PR description is well-structured and the file set is consistent with the workflow intent. The docs/ scope is correctly bounded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant