Repository navigation
Benchmarks: make the gperf baseline a tuned map - #41
Merged
Merged
Conversation
Every gperf lane is now a map, like the other methods: %struct-type with an int value (the declaration index), and the wrapper returns entry->value. Two lanes did not measure gperf: - JavaScript reserved words ran a std::unordered_map lookup after every gperf hit (22.3 -> 7.2 ns on hits, Apple M3 Max). - Counters was a std::unordered_set, not gperf (13.9 -> 7.7 ns). Options, measured per set (M3 Max, clang 22, min of two runs): - %compare-lengths on MIME, letters, counters and FQCN, where it wins (FQCN hits 12.2 -> 4.3 ns, MIME 9.0 -> 6.4 ns). Kept %compare-strncmp on the short-key sets, where the length table costs more (S&P 100 misses 1.8 -> 4.0 ns with it). - %switch=1 was slower on every set (up to 2.6x on hits), and -m 10000 made no consistent difference, so neither is used. All tables are regenerated with GNU gperf 3.3 by one script, benchmarks/gperf/regenerate.sh. The letters, headers50 and FQCN inputs move into benchmarks/gperf/ with the others. Verification now checks gperf's values, and bench_tickers verifies its two gperf tables. The ticker lanes and the short-key sets are unchanged within noise.
lemire
approved these changes
Nov 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #29.
#29 asked for the gperf baseline to be a map with tuned options. Checking it turned up two lanes that weren't measuring gperf at all:
std::unordered_maplookup after every gperf hit.std::unordered_set; there was no gperf table for it.Changes
%struct-typewith anintvalue (the key's declaration index), plus%readonly-tablesand%enum. The wrappers returnentry->value.benchmarks/gperf/counters.gperf; the JS lane no longer touchesstd::unordered_map.%compare-lengthsis used for MIME types, letters, Counters and Java FQCNs;%compare-strncmpis kept for the short-key sets and the S&P 500 / Nasdaq tickers, where the length table costs more than it saves.benchmarks/gperf/regenerate.sh [path/to/gperf]. The letters, HTTP Headers 50 and FQCN inputs move intobenchmarks/gperf/with the others.gen_symbol_headers.pywrites the ticker inputs in the same form.bench_tickersverifies its two gperf tables.benchmarks/gperf/README.mddocuments the options and the measurements behind them.Options measured and not used
%switch=1: slower on every set, up to 2.6x on hits.-m 10000: shrank some tables but made no consistent difference to lookup time (it already saturates around-m 100).%compare-lengthseverywhere: big wins on long or similar keys (FQCN hits 12.2 → 4.3 ns) but it hurts short ones (S&P 100 misses 1.8 → 4.0 ns), hence the per-set choice.Results
gperf lane, ns per lookup,
main→ this branch (Apple M3 Max, clang 22, best of two runs). The last column ismake_perfect_mapfor reference.The short-key sets are unchanged within noise, and so are the S&P 500 and Nasdaq gperf lanes in
bench_tickers. The README headline gperf number (URL Protocols) is unaffected. MIME misses get slower with%compare-lengthswhile hits and mixed get faster; I kept it there because the mixed workload improves.Not measured: x86-64 / glibc. The per-set choice depends on how the C library's
memcmpcompares with itsstrncmp, so it may come out differently there.CI doesn't build the benchmarks, so I checked locally:
bench_protocolandbench_tickers, and all 10 protocol sets and the 4 ticker checks verify;-Wall -Wextraand no warnings, and every key (6,411 in all) returns its value. The full benchmarks don't build with Homebrew GCC on macOS, because its libstdc++ doesn't match the SDK headers that abseil includes; that's unrelated to this change.