perf(precomputed): re-enable multithreading for local file reads/writes

[nkemnitz]

With seung-lab/cloud-files#126 compression/decompression running GIL-free, preserving the ThreadPool also makes sense for local precomputed layers. (And it would for mem and LRU cache if we ever decide to allow compressed data there, too)

[william-silversmith]

I think this would need careful benchmarking. The advantage of turning off threads was very large.

[nkemnitz]

🤷‍♂️

Looks like there is 20ms overhead for creating the ThreadedQueue. Claude found a way to reeduce it a little bit by disabling the progress bar. I added a gate based on chunk count. The gzip speedup with this PR alone won't be visible, yet. https://github.com/dcwatson/deflate already pushed the changes to main, but hasn't pushed a new release, yet. For other libs, such as libjpeg-turbo, this change already helps.

🤖 Benchmark results

GCP c4-standard-16 (Xeon 8581C, 8 physical cores + SMT), native Ubuntu 24.04, Python 3.13, warm page cache. Local file:// precomputed layer, 134 MB, 128×128×64 chunks. Min-of-5 for throughput, median-of-9 for latency. Two deflate builds: PyPI 0.8.1 vs main @ b55ef31 (the Py_BEGIN_ALLOW_THREADS change).

Ratio = threads OFF (master) / threads ON (this PR); >1 means the PR helps.

image raw+gzip, 134 MB	deflate 0.8.1	deflate main
read	1.06x	3.86x
write	0.95x	4.87x

Confirms the premise: nothing without the GIL release, ~4-5x with it. Single-thread throughput is identical between the two builds, so this is purely scaling. Separately, jpeg layers already get ~7x in both builds — libjpeg-turbo releases the GIL today, so that part of the win is available regardless of what deflate does.

The case that needed guarding

Threading isn't free: there's a fixed ~20 ms cost per batch, which is more than a small cutout spends decompressing. Read latency vs chunk count:

chunks	MB	OFF	ON	ratio
1	1.0	4.17 ms	4.12 ms	1.01x
2	2.1	7.48 ms	24.75 ms	0.30x
4	4.2	11.54 ms	26.06 ms	0.44x
8	8.4	23.02 ms	27.68 ms	0.83x
16	16.8	45.20 ms	28.15 ms	1.61x
32	33.6	91.74 ms	29.01 ms	3.16x
128	134.2	386.93 ms	78.70 ms	4.92x
512	536.9	1648.13 ms	278.88 ms	5.91x

The 1-chunk case is safe only because schedule_jobs short-circuits it. This is the regression the original file:// exclusion was protecting against, and it's a function of chunk count, not protocol.

About half that fixed cost is pure waste: ThreadedQueue.wait() polls at time.sleep(0.015) solely to advance a tqdm bar that is disabled on this path, while join() right below it already waits and reraises. Skipping the poll when there's no bar to draw:

chunks	OFF	ON	ON + poll fix
2	7.48	24.75	15.20
4	11.54	26.06	15.90
8	23.02	27.68	17.26
16	45.20	28.15	20.83
512	1648.13	278.88	275.87

Break-even moves from ~16 chunks to ~8, and large reads are unaffected.

Small writes never regress (1.00x at 1 chunk, 1.03x at 2, 1.91x at 4, 3.32x at 8) — compression is ~5.5x more expensive per byte than decompression, so the batch cost is amortized immediately. The write path needs no threshold.

Pushed in response

Rebased onto master (12.14.4), plus:

1. Chunk-count gate — MIN_THREADED_DISK_READS = 16 on local reads instead of excluding file:// outright. Keeps the full large-read win, removes the small-read regression, with margin for slower disks and fewer cores.
2. ThreadedQueue.wait() poll fix — as above.
3. Cache hits now thread. locations['local'] was hardcoded concurrency=0, so a cached remote layer running the identical file+gzip workload got none of this. gs:// source, 128 chunks, 117 MB served entirely from local disk cache: 487.7 ms → 142.7 ms (3.42x).
4. Greenlets stay serial — they can't overlap C-level (de)compression, so they got no benefit and paid the ~20 ms anyway (457 ms → 473 ms on a 128-chunk read). Measured on reads; applied to writes on the same reasoning.

test/test_cloudvolume.py gives identical results before and after (142 passed / 62 failed / 62 errors, all pre-existing on master in the same environment).

Caveat

The gzip win needs an unreleased deflate — PyPI latest is 0.8.1 (2025-07-18) and has no GIL release. Merged today, most users see ~1.0x on gzip layers; the jpeg win and the cache-hit fix land immediately either way.

[william-silversmith]

One note: skipping the sleep cycle means that errors are not raised until all elements are processed, meaning it can be a long time to see an error.

[william-silversmith]

This was some very early threading work I did. Probably it should be using a condition variable or something to signal errors/task completion.

[william-silversmith]

What do you think of this approach for the sleepiness?

#703

[nkemnitz]

Commented on #703 and added a commit that addresses the delayed error issue

[nkemnitz]

deflate 0.9.0 is now on PyPI 🎉

[william-silversmith]

I wonder if we should gate this functionality behind whether the python version is free threaded or not