Skip to content

Batch Writes and Add Backpressure in Local Tx Log - #3894

Open
alexanderkiel wants to merge 1 commit into
mainfrom
claude/issue-3886-tzjsyf
Open

Batch Writes and Add Backpressure in Local Tx Log#3894
alexanderkiel wants to merge 1 commit into
mainfrom
claude/issue-3886-tzjsyf

Conversation

@alexanderkiel

Copy link
Copy Markdown
Member

Closes: #3886

@alexanderkiel alexanderkiel self-assigned this Jul 17, 2026
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.95918% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.45%. Comparing base (2d27a50) to head (180a490).

Files with missing lines Patch % Lines
modules/db/src/blaze/db/tx_log/local.clj 97.24% 2 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3894      +/-   ##
==========================================
+ Coverage   96.43%   96.45%   +0.01%     
==========================================
  Files         437      437              
  Lines       27452    27538      +86     
  Branches      631      634       +3     
==========================================
+ Hits        26474    26561      +87     
- Misses        471      472       +1     
+ Partials      507      505       -2     
Files with missing lines Coverage Δ
modules/db/src/blaze/db/node.clj 99.27% <100.00%> (+0.05%) ⬆️
modules/db/src/blaze/db/tx_log/local.clj 97.18% <97.24%> (-1.66%) ⬇️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexanderkiel
alexanderkiel force-pushed the claude/issue-3886-tzjsyf branch 5 times, most recently from 88f2de2 to 1b603b4 Compare July 30, 2026 10:59
@alexanderkiel

alexanderkiel commented Jul 30, 2026

Copy link
Copy Markdown
Member Author

Load test results, baseline (main, 03855d9b1) vs. this branch, both with DB_RESOURCE_STORE_KV_THREADS=64:

System VUs Req/s before Req/s after Speedup q99 before q99 after
A5N46 32 145.7 1000.0 6.9× 693 ms 42 ms
A5N46 64 162.6 2064.3 12.7× 1781 ms 42 ms
LEA47 32 443.6 1392.5 3.1× 218 ms 31 ms
LEA47 64 452.2 1370.3 3.0× 434 ms 68 ms
LEA79 64 5617.3 5381.2 0.96× 15.6 ms 13.0 ms

The gain tracks fsync latency: on A5N46 (5.4 ms syncs) the baseline serialized one tx-log fsync per transaction, while batching amortizes it across everything buffered in flight — the median holds at ~30 ms from 4 to 64 clients and throughput still doubles with each doubling of clients at 64, i.e. unsaturated. LEA79 syncs in 8 µs, was never fsync-bound, and stays at parity (−4% at 64 VUs, within noise).

@alexanderkiel
alexanderkiel force-pushed the claude/issue-3886-tzjsyf branch 5 times, most recently from 3b4c00c to 01e5009 Compare August 5, 2026 15:33
@alexanderkiel
alexanderkiel force-pushed the claude/issue-3886-tzjsyf branch from 01e5009 to 3bf310a Compare August 6, 2026 14:17
Closes: #3886
Signed-off-by: Alexander Kiel <alexanderkiel@gmx.net>
@alexanderkiel
alexanderkiel force-pushed the claude/issue-3886-tzjsyf branch from 3bf310a to 180a490 Compare August 7, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Batch Writes and Add Backpressure in Local Tx Log

1 participant