Skip to content

x86-64: compute the safe hash index with a select, not a mask - #42

Merged
lemire merged 2 commits into
mainfrom
lemire/x86-hash-select
Nov 14, 2026
Merged

lemire merged 2 commits into
mainfrom
lemire/x86-hash-select

Conversation

@lemire

@lemire lemire commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Problem

The branchless hash (used when N >= 64 or MaxKeyLen > 16) computes a safe index for each position as idx & -has, with has = idx < klen. Once the map's constants are folded into the caller (the usual case for a global constexpr map), idx is a constant. For 1, 2, 4 or 8, GCC uses the 0/1 flag as a scaled index, which is fine. For other positions, such as byte 26 on the Java FQCN set, it builds the mask with

cmp  $0x1a,%rcx
seta %al          ; writes only the low byte of rax
shl  $0x3f,%rax
sar  $0x3f,%rax

with no xor before the seta. The byte write merges into %rax, and %rax last held the value loaded at the end of the previous lookup. Each lookup then waits for the previous one: FQCN hits take 23 cycles instead of 10.

Change

On x86-64 only, compute the safe index as idx < klen ? idx : 0, which compiles to a cmov. Other targets keep idx & -has. AArch64 has no partial-register writes: there it compiles to cmp + csetm, and its code is unchanged by this PR (I checked: identical machine code for all ten bench_protocol sets and for bench_tickers, with Apple clang 17 and GCC 15 on an M4).

Measurements

Emerald Rapids (Xeon Gold 6548N), GCC 16.2, bench_protocol --filter make_perfect_map, best of 3 runs alternating main and this branch, pinned to one core. "Specialized" means the build where GCC folds the map's constants into the benchmark loop (--param=ipa-cp-unit-growth=200 --param=ipa-cp-eval-threshold=1). In the default build, GCC does that for seven of the ten sets but not for JavaScript, Counters and FQCN (a separate benchmark issue, to be fixed in its own PR).

Set Build main hits / misses / mixed (ns) This PR (ns)
Java FQCNs specialized 6.67 / 5.20 / 10.21 2.95 / 2.93 / 7.69
HTTP Headers 50 specialized 2.02 / 1.86 / 4.35 1.89 / 1.67 / 4.19
S&P 100 both 2.45 / 2.24 / 4.42 3.88 / 2.44 / 5.52 (slower)
Counters default 3.35 / 3.11 / 10.65 3.40 / 3.15 / 10.73 (+1 %)
other 6 sets both within 1 %

The cost is S&P 100. Its positions are bytes 0, 1, 2 and the last byte, where main's code was already optimal, and GCC compiles the select into a branch there (fractional instruction counts). Even so, S&P 100 stays well ahead of the best competitor on this machine: ankerl 8.46 ns on hits, gperf 3.31 ns on misses, ankerl 11.22 ns on mixed.

All 82 tests pass, and bench_protocol --verify passes on all ten sets, in both builds.

Alternatives measured and not used

  • Mask from the sign of idx - klen (sub + sar): same FQCN gain, but S&P 100 +11 % and HTTP Headers 50 +8 %, from extra instructions on every position.
  • Select on both the index and the value: GCC turns them into branches, up to +70 % on Counters.
  • idx * has: GCC compiles it to exactly the same code as main.
  • Clamp, min(idx, klen - 1): same FQCN gain, S&P 100 +10 %, Counters +4 %.

In the branchless hash (N >= 64 or MaxKeyLen > 16), the safe index was
`idx & -has`. Once the map is constant-folded, idx is a constant. When it
is not a valid x86 address scale (1, 2, 4, 8), GCC builds the mask with
`seta %al; shl $63; sar $63` and no preceding `xor`. The byte write merges
into the register's previous value -- in a lookup loop, the previous
lookup's result -- so successive lookups no longer overlap.

On x86-64, compute the index as `idx < klen ? idx : 0`, which compiles to a
cmov. Other targets keep the mask: AArch64 has no partial-register writes,
and its code is unchanged.
@lemire
lemire merged commit 04a4994 into main Nov 14, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant