Repository navigation
Performance regression for ByteArrayMemory on Java 25 #224
Description
Activity
Yeah that doesn't seem right. The access mode shouldn't change across accesses and should be established on the first call at most.
@andreas-karlsson I have let an agent do the heavy comparisons and benchmarks:
Follow-up on the JDK 23+ atomics regression (#224): I prototyped the two backing stores the JDK
recommends instead ofbyte[]views and measured them against the current memories.What was tested. Two full
Memoryimplementations: directByteBufferpages with
byteBufferViewVarHandleatomics (Java 9+), and heaplong[]pages throughMemorySegment
(Java 22+). Both keep volatile/CAS lock-free on JDK 21 and 25, verified. Compared against
ByteArrayMemoryandByteBufferMemoryon the threaded sieve and on five real libraries
(quickjs4j, sqlite4j, jq4j, pglite4j, prism), JMH, Temurin 21.0.11 and 25.0.3. All workloads
returned identical results across memory types.Sieve, 4 threads, compiled, ops/s (higher is better):
Memory JDK 21 JDK 25 ByteArrayMemory (main) 47.6 14.4 ByteBufferMemory 13.8 14.7 direct ByteBuffer 41.5 44.8 long[] MemorySegment n/a 40.3 Real workloads, JDK 25, time per op (lower is better); last column is how much slower (+) or faster (-) direct is than byte[]:
Workload ByteArrayMemory ByteBufferMemory direct ByteBuffer direct vs byte[] quickjs4j, 1 ms script 165 ms 186 ms 187 ms +13% quickjs4j, 1+232.6 us 41.1 us 40.0 us +22% sqlite4j, insert 1000 rows 12.7 ms 14.4 ms 14.2 ms +12% sqlite4j, select 10k rows 61.6 ms 63.8 ms 65.5 ms +6% sqlite4j, open connection 6.5 ms 6.7 ms 6.6 ms +2% prism, parse 130 KB 115 ms 117 ms 119 ms +4% pglite4j, SELECT 11.23 ms 1.35 ms 1.26 ms +2% pglite4j, create instance (167 MB) 71 ms 70 ms 69 ms -3% jq4j, filter 100 KB JSON (shared memory) 477 ms 206 ms 191 ms -60% JDK 21 shows the same shape with more noise: direct is 3 to 35% slower than
byte[]on plain
workloads and equal on the sieve.Takeaways.
byte[]stays fastest for plain access, by 2 to 15% on JDK 25 across every library. That is the
common case.- Direct buffers win only where atomics are involved: 3x on the sieve on JDK 25, and 2.3x on jq4j
even single-threaded (its pthread build takes a lock on every allocation). - Off-heap has a cost beyond speed. Opening one 32 MB sqlite connection per op on JDK 21 produced
iterations of 1.6 s and 4.3 s instead of 10 ms: direct pages are reclaimed only by GC, so
Bits.reserveMemoryhit the direct-memory limit and ranSystem.gc()with sleep backoff. Pages
also count againstMaxDirectMemorySizeand grow RSS past-Xmx, and there is no deterministic
free on Java 11. - Separately: on JDK 25 any atomic op on a non-shared
ByteArrayMemorytraps as out-of-bounds
(monitor(addr)dereferences the nullwaitStatesmap and the NPE is rethrown as a bounds trap).
Still present on main;ByteBufferMemory.lockalready handles the non-shared case.
Caveat: the benchmark machine was thermally throttled (1.7 to 2.6 GHz), so compare within a JDK
column, not across JDKs.Interesting results! I would guess that the atomic operations are cheaper against direct memory because the JVM does not need to be at a safe point. Heap data structures need to be pinned in memory for the duration of the atomic swap, while direct memory is already pinned by definition.
I'm not sure I understand your results, though. Is one of these a VarHandle + byte[] implementation? If the VarHandle is slower than any other on-heap variant, that should probably be reported to folks on the compiler team. Would be interesting to look at the assembly it's emitting.
Never mind, now I see ByteArrayMemory is the VarHandle version. And if I'm reading right it is the fastest overall, ignoring small gains from atomic operations against DirectByteBuffer?
Reacted by Andrea Peruffo@andreaTP Thanks a lot for the thorough investigation, and for testing against real libraries.
I'll open a PR for the non-shared locking bug (
monitor()dereferencing the nullwaitStates). While looking at it I noticed a related issue in the same fallback: the lock is chosen by the exact start address of the access, so overlapping atomics of different sizes, e.g. ani32at 0 and ani16at 2, take different locks and aren't atomic relative to each other. Since wasm atomics are naturally aligned and at most 8 bytes, locking on the aligned 8-byte word should fix that. I will open a PR for that as well.Some context from our production use of Endive. We have a highly concurrent server workload, but it also requires sharing memory, so we can't pool single-threaded instances, and hence we use a lot of atomics.
For that workload I built our own mmapped memory, similar to Redline's
NativeMemory. Initial results look very promising, roughly 2.5–2.8× the throughput ofByteArrayMemoryon JDK 21 (2.5× single-threaded, 2.8× at 10 threads, Apple M1 Max). I think the interesting part I can contribute to your numbers is how big a role function size plays for JVM inlining. A rough breakdown of the performance increase:- mmapped memory with FFM access on JDK 25: about the same as
ByteArrayMemoryon JDK 21 - bypassing the FFM access path for plain loads/stores (one explicit bounds check, then raw
Unsafeaccess): +45%. I'm not suggestingUnsafefor Endive, but it shows how much the FFM access path costs in large generated methods. - raising C2's inlining limits (mainly
-XX:-ClipInliningand raised-XX:MaxTrivialSize): about +35–40%. - shrinking the memory accessors and atomics so they fit within those limits: another +35%
Since QuickJS, SQLite and Prism have very large functions too, it would be interesting to see whether
-XX:-ClipInlining -XX:MaxTrivialSize=40changes your numbers, possibly for all memory types.To be clear, we're not blocked by any of this, since we can keep using our own memory implementation, but I'd be happy to help move things forward. A few thoughts:
- Your results suggest picking the memory based on whether it's shared:
byte[]is fastest for plain access, and atomics only matter for shared memories. SoByteArrayMemoryfor non-shared memories and a native memory for shared ones. - For the shared case I think a native mmapped memory is the best option, given the GC,
MaxDirectMemorySizeand RSS issues with direct buffers you describe. It also commits pages lazily and can be freed explicitly. With the hardening in Fix Redline native memory growth, atomics, wait/notify and traps #232, Redline'sNativeMemorylooks close to that already, so once it's production-ready I don't see a reason not to also use it with the bytecode compilers. - Getting the same speed without JVM flags would take careful function-size work in the compiler, so that memory helpers stay within C2's default inlining limits. That may never be worth the complexity, but if anyone goes in that direction, we'd first need a better benchmark suite to guide it (Improved performance benchmarking #225).
- mmapped memory with FFM access on JDK 25: about the same as
Since QuickJS, SQLite and Prism have very large functions too, it would be interesting to see whether -XX:-ClipInlining -XX:MaxTrivialSize=40 changes your numbers, possibly for all memory types.
I am familiar with these flags and intrigued that you are seeing good performance gains using them. I'll try to test that against prism as well.
We currently deploy prism using the native AOT compiled version from cranelift, and serialized access to the parser runtime (parallel loading is not very common in Ruby apps) but on platforms without AOT support we must fall back on the WASM runtime. Until there's an effective mechanism for splitting large functions, this remains slower than we would like.
Reacted by andreas.karlsson@headius Nice. You might need to adjust the limit. I reached the best performance after also adjusting the the memory access functions so they resulted in smaller bytecode fitting below the 40 byte limit.
If running the multithreaded BenchmarkSievePrimes bench on JDK 25, one can witness a radical performance breakdown. On my local machine it runs 220 times slower than on JDK 21. The reason is primarily that it calls
isAccessModeSupportedon every atomic read/write. On JDK 25 this throws aNoSuchMethodErrorinternally before returning false. AFAIK there is no supported way of doing atomics on byte[] in JDK 25 and in this scenario ByteArrayMemory should fallback to the slow lock path just like ByteBufferMemory.After fixing this, there still won't be a performant memory on JDK 25. I propose the Redline NativeMemory (or a hardened version thereof) is promoted to a neutral supported module. It still needs its own module if runtime is to remain on JDK 11. My own experimentation has shown that the native mapped memory can also give a good performance boost when using the bytecode compilers.