Skip to content

fix: signed-overflow UB in top-bucket value-range math (hdr_max/percentile/iterator) - #148

Open
filipecosta90 wants to merge 4 commits into
mainfrom
fix/top-bucket-value-range-overflow
Open

filipecosta90 wants to merge 4 commits into
mainfrom
fix/top-bucket-value-range-overflow

Conversation

@filipecosta90

@filipecosta90 filipecosta90 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

In-contract int64 overflow in top-bucket value-range math, found by adversarial review during the ClusterFuzzLite effort

For a histogram whose highest_trackable_value is near INT64_MAX, the top bucket's lowest_equivalent_value + size_of_equivalent_value_range exceeds INT64_MAX → signed-overflow UB. Two arithmetic sites:

  • hdr_next_non_equivalent_value (:294) and highest_equivalent_value (:307) → poisons hdr_max, hdr_value_at_percentile, and the value-range lines of hdr_percentiles_print.
  • move_next (:918, the all-values iterator, which visits every bucket regardless of counts) → the iterator's highest_equivalent_value.

In-contract (hdr_init accepts up to INT64_MAX) and reachable from an untrusted decoded log (decode → hdr_reset_internal_countershighest_equivalent_value; and log_reader_fuzzer calls hdr_max/hdr_value_at_percentile on every decoded histogram).

Fix

Saturate all three value-range computations at INT64_MAX instead of overflowing. highest_equivalent_value gets its own overflow-aware clamp (not next_non_equivalent - 1), so hdr_max of a histogram that recorded INT64_MAX correctly returns INT64_MAX and agrees with the iterator path.

Scope (precise)

This PR fixes the value-range additions only. The related hdr_mean count * value overflow (which also feeds hdr_stddev and the hdr_percentiles_print CLASSIC footer) is a separate overflow class fixed in the companion #147. Both are needed for a fully overflow-free read path on near-INT64_MAX histograms. The linear/log iterator reporting-level overflows are addressed separately (follow-up PR).

Verification

Under ASan+UBSan: hdr_max / percentiles / value-range iteration no longer trap and return representable values (INT64_MAX); normal histograms are byte-identical (guard only triggers on genuine top-bucket overflow). Regression test added and pins the saturated values. Full suite green.

The ASan/UBSan ctest CI job that makes these regression tests deterministic guards is in #145.

🤖 Generated with Claude Code

Found by adversarial review during the ClusterFuzzLite effort. For a histogram
whose highest_trackable_value is near INT64_MAX, the top bucket's
lowest_equivalent_value + size_of_equivalent_value_range exceeds INT64_MAX,
which is signed-overflow UB. This poisoned hdr_next_non_equivalent_value ->
highest_equivalent_value (hence hdr_max, hdr_value_at_percentile,
hdr_percentiles_print) and the move_next iterator step (hence hdr_stddev and
any full iteration -- the all-values iterator visits the top bucket regardless
of counts). All in-contract and reachable via the read path on a decoded
histogram.

Saturate both additions at INT64_MAX instead of overflowing. Verified under
ASan+UBSan: hdr_init(1, INT64_MAX, 3) + record(INT64_MAX) then hdr_max /
percentile / stddev / full iteration / percentiles_print no longer trap and
return representable values. Added a regression test.

Found-by: adversarial review during the ClusterFuzzLite fuzzing effort
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…variant)

Adversarial review of the initial saturation found that deriving
highest_equivalent_value as hdr_next_non_equivalent_value(v) - 1 yielded
INT64_MAX-1 for the top bucket, so hdr_max of a histogram that recorded
INT64_MAX returned a value BELOW it (violating value <= highest_equivalent),
and disagreed with move_next() which saturates the iterator's
highest_equivalent_value to INT64_MAX. Give highest_equivalent_value its own
overflow-aware computation clamping to INT64_MAX so hdr_max / value_at_percentile
and the iterator agree. Strengthen the regression test to pin the saturated
values (hdr_max == INT64_MAX, hdr_next_non_equivalent_value == INT64_MAX -- the
latter distinguishes the fix without a sanitizer, since the buggy code wraps to
INT64_MIN).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@fcostaoliveira
fcostaoliveira force-pushed the fix/top-bucket-value-range-overflow branch from cedc46a to 92a219f Compare August 28, 2026 09:54
fcostaoliveira added a commit that referenced this pull request Aug 28, 2026
Found by adversarial review during the ClusterFuzzLite effort. For a histogram
whose highest_trackable_value is near INT64_MAX, the linear and logarithmic
iterators advanced their reporting level with 'next += value_units_per_bucket'
(iter_linear_next) and 'next *= (int64_t)log_base' (log_iter_next), both of
which overflow int64 -> signed-overflow UB, then a negative left-shift when the
wrapped value flows into lowest_equivalent_value.

Saturate the reporting level at INT64_MAX when the next step would overflow.
Crucially, also pin next_value_reporting_level_lowest_equivalent to INT64_MAX
in that case: no bucket value ever equals INT64_MAX, so the 'iter->value >=
level' emit test can no longer fire and the iterator drains its remaining
buckets and terminates rather than re-reporting the saturated level forever.

Verified under ASan+UBSan: linear (value_units_per_bucket=2^62) and log (base 2)
over an INT64_MAX-range histogram no longer trap and terminate (2 and 64 steps);
normal-range iteration is unchanged. Regression test added.

Stacked on #148 (whose move_next fix the iterators depend on).

Found-by: adversarial review during the ClusterFuzzLite fuzzing effort
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

🤖 Automated first-pass review — a human maintainer's review is still required before merge.

The arithmetic checks out. For every config hdr_calculate_bucket_config accepts, the top bucket's low + size is exactly 2^63 — buckets_needed_to_cover_value always stops on the first power of two above INT64_MAX / 2, and low + size is sub_bucket_count << (unit_magnitude + bucket), so it can't land anywhere else. That means low + size - 1 is exactly INT64_MAX and both new clamps are lossless, not approximations. hdr_median_equivalent_value and the iterator's median_equivalent_value are low + size/2, which stays under 2^63, so leaving those alone is correct and the stated scope holds. No struct layout or signature change, so nothing for the HDR_SOVERSION_* process here, and highest_equivalent_value still does the same two underlying calls it did before, so no regression against the reuse work in #94.

The one point I'd want a human look at is hdr_next_non_equivalent_value, which is public (include/hdr/hdr_histogram.h:502) and is the only caller-visible behaviour change here: for the top bucket it now returns a value that is in the same equivalence range as its argument, so hdr_values_are_equivalent(h, v, hdr_next_non_equivalent_value(h, v)) becomes true, and an external caller stepping ranges with v = hdr_next_non_equivalent_value(h, v) stops advancing rather than terminating on the old wrapped-negative value — an infinite loop where there used to be UB, which is the failure shape of #36. Nothing in-tree does that (after this change highest_equivalent_value no longer calls it, and it has no other internal caller), so this is a contract question, not an in-tree bug. Against the Java reference: highestEquivalentValue there double-wraps and lands on Long.MAX_VALUE, which is what this PR's highest_equivalent_value now returns by construction, so that half agrees; nextNonEquivalentValue wraps rather than saturating, so that half is a deliberate divergence. I couldn't run the Java side to confirm either, so that's worth checking against AbstractHistogram rather than taking my word for it. Clamping only in highest_equivalent_value and leaving hdr_next_non_equivalent_value untouched would preserve the public contract but keep the UB, so it's a real trade-off, not an obvious mistake.

Two smaller notes on the test. iter.highest_equivalent_value >= 0 could pin INT64_MAX on the final step instead of just non-negativity — as written it passes on the unfixed code, which the comment does honestly say. And isfinite(hdr_stddev(h)) passes here only because a single top-bucket sample keeps count * median under INT64_MAX inside hdr_mean; it isn't exercising this change, and it's one extra hdr_record_value(h, INT64_MAX) away from tripping the separate overflow class #147 covers.

Worth stating plainly for whoever merges this: the per-PR ClusterFuzzLite job runs ASan only, so signed-overflow UB of this kind won't be regression-guarded there (only the weekly batch runs UBSan). The added ctest case under the ASan+UBSan job from #145, now merged, is what actually pins this.

@fcostaoliveira fcostaoliveira changed the title fix: int64 overflow in top-bucket value-range math (hdr_max/percentile/stddev) fix: signed-overflow UB in top-bucket value-range math (hdr_max/percentile/iterator) Sep 2, 2026
# Conflicts:
#	test/hdr_histogram_test.c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants