Repository navigation
x86-64: compute the safe hash index with a select, not a mask - #42
Merged
Merged
Conversation
In the branchless hash (N >= 64 or MaxKeyLen > 16), the safe index was `idx & -has`. Once the map is constant-folded, idx is a constant. When it is not a valid x86 address scale (1, 2, 4, 8), GCC builds the mask with `seta %al; shl $63; sar $63` and no preceding `xor`. The byte write merges into the register's previous value -- in a lookup loop, the previous lookup's result -- so successive lookups no longer overlap. On x86-64, compute the index as `idx < klen ? idx : 0`, which compiles to a cmov. Other targets keep the mask: AArch64 has no partial-register writes, and its code is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The branchless hash (used when N >= 64 or MaxKeyLen > 16) computes a safe index for each position as
idx & -has, withhas = idx < klen. Once the map's constants are folded into the caller (the usual case for a globalconstexprmap),idxis a constant. For 1, 2, 4 or 8, GCC uses the 0/1 flag as a scaled index, which is fine. For other positions, such as byte 26 on the Java FQCN set, it builds the mask withwith no
xorbefore theseta. The byte write merges into%rax, and%raxlast held the value loaded at the end of the previous lookup. Each lookup then waits for the previous one: FQCN hits take 23 cycles instead of 10.Change
On x86-64 only, compute the safe index as
idx < klen ? idx : 0, which compiles to acmov. Other targets keepidx & -has. AArch64 has no partial-register writes: there it compiles tocmp+csetm, and its code is unchanged by this PR (I checked: identical machine code for all tenbench_protocolsets and forbench_tickers, with Apple clang 17 and GCC 15 on an M4).Measurements
Emerald Rapids (Xeon Gold 6548N), GCC 16.2,
bench_protocol --filter make_perfect_map, best of 3 runs alternatingmainand this branch, pinned to one core. "Specialized" means the build where GCC folds the map's constants into the benchmark loop (--param=ipa-cp-unit-growth=200 --param=ipa-cp-eval-threshold=1). In the default build, GCC does that for seven of the ten sets but not for JavaScript, Counters and FQCN (a separate benchmark issue, to be fixed in its own PR).mainhits / misses / mixed (ns)The cost is S&P 100. Its positions are bytes 0, 1, 2 and the last byte, where
main's code was already optimal, and GCC compiles the select into a branch there (fractional instruction counts). Even so, S&P 100 stays well ahead of the best competitor on this machine: ankerl 8.46 ns on hits, gperf 3.31 ns on misses, ankerl 11.22 ns on mixed.All 82 tests pass, and
bench_protocol --verifypasses on all ten sets, in both builds.Alternatives measured and not used
idx - klen(sub+sar): same FQCN gain, but S&P 100 +11 % and HTTP Headers 50 +8 %, from extra instructions on every position.idx * has: GCC compiles it to exactly the same code asmain.min(idx, klen - 1): same FQCN gain, S&P 100 +10 %, Counters +4 %.