Skip to content

🤖 Update Benchmark Results#5170

Merged
thomhurst merged 1 commit intomainfrom
automated-benchmarks-update
Mar 16, 2026
Merged

🤖 Update Benchmark Results#5170
thomhurst merged 1 commit intomainfrom
automated-benchmarks-update

Conversation

@thomhurst
Copy link
Owner

Automated Benchmark Update

This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • benchmark-DataDrivenTests
  • benchmark-AsyncTests
  • benchmark-ScaleTests
  • benchmark-MatrixTests
  • benchmark-MassiveParallelTests
  • benchmark-SetupTeardownTests
  • benchmark-BuildTime
  • benchmark-summary (aggregated overview)

Runtime Benchmarks

  • AsyncTests
  • DataDrivenTests
  • MassiveParallelTests
  • MatrixTests
  • ScaleTests
  • SetupTeardownTests

Build Benchmarks

  • BuildTime
  • 2026-03-16
  • Ubuntu Latest • .NET SDK 10.0.201

Changes

  • Updated benchmark data in docs/static/benchmarks/latest.json
  • Updated historical trends in docs/static/benchmarks/historical.json
  • Regenerated benchmark documentation in docs/docs/benchmarks/index.md
  • Updated benchmark summary in docs/static/benchmarks/summary.json

Workflow Run

  • Run ID: 23123040484
  • Triggered: schedule
  • Date:

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels Mar 16, 2026
@thomhurst thomhurst merged commit d87be50 into main Mar 16, 2026
9 of 12 checks passed
@thomhurst thomhurst deleted the automated-benchmarks-update branch March 16, 2026 00:40
Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Benchmark Update Review

This PR is a fully automated benchmark documentation update — all 19 changed files are confined to docs/docs/benchmarks/ and docs/static/benchmarks/, covering Markdown pages and JSON data files. There are no source code, engine, or test changes.

Assessment

The changes are straightforward and appropriate:

  • Markdown files (AsyncTests.md, BuildTime.md, DataDrivenTests.md, etc.) are refreshed with updated dates, timing numbers, and regenerated Mermaid charts.
  • JSON data files (latest.json, historical.json, summary.json, and per-benchmark JSONs) are updated with the new run's measurements.
  • All changes are consistent with what an automated CI workflow ("Speed Comparison") would produce on a scheduled run (2026-03-16).

Notable Benchmark Observations

TUnit AOT performance remains exceptionally strong (e.g., AsyncTests AOT at ~123 ms vs. ~556 ms for JIT TUnit and ~780 ms for xUnit3), which continues to be a compelling differentiator.

Slight variations in JIT timings across all frameworks day-to-day are expected on shared CI runners and do not indicate regressions.

No Concerns

  • No source code, public API, snapshot tests, or engine logic is touched.
  • The automated PR workflow and merge process look correct.
  • No .received.txt files were committed (as required).

This is a clean, low-risk automated update. Nothing to change here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant