Skip to content

fix: int64 overflow in linear/log iterator reporting level (near INT64_MAX) - #149

Open
filipecosta90 wants to merge 5 commits into
fix/top-bucket-value-range-overflowfrom
fix/iterator-reporting-level-overflow
Open

fix: int64 overflow in linear/log iterator reporting level (near INT64_MAX)#149
filipecosta90 wants to merge 5 commits into
fix/top-bucket-value-range-overflowfrom
fix/iterator-reporting-level-overflow

Conversation

@filipecosta90

Copy link
Copy Markdown
Contributor

Stacked on #148 (the iterators call move_next, whose value-range overflow #148 fixes). Base this PR on #148; retarget to main once #148 merges.

Bug (found by adversarial review during the ClusterFuzzLite effort)

For a histogram with highest_trackable_value near INT64_MAX, the linear and log iterators overflow int64 when advancing the reporting level:

  • iter_linear_next (:1162): next_value_reporting_level += value_units_per_bucket
  • log_iter_next (:1219): next_value_reporting_level *= (int64_t) log_base

Both are in-contract (public iterators) and reachable via any full linear/log iteration of such a histogram; the wrapped negative level then triggers a negative left-shift in lowest_equivalent_value.

Fix

Saturate the reporting level at INT64_MAX when the next step would overflow. Termination: naive saturation would re-report the top level forever (the emit branch doesn't advance counts_index). Since no bucket value ever equals INT64_MAX, pinning next_value_reporting_level_lowest_equivalent to INT64_MAX as well makes the iter->value >= level test stop firing, so the iterator drains remaining buckets and terminates.

Verification (ASan+UBSan)

  • Linear (units=2^62) and log (base 2) over an INT64_MAX histogram: no trap, terminate (2 and 64 steps). Normal-range iteration unchanged. Regression test added; full suite green.

🤖 Generated with Claude Code

@filipecosta90
filipecosta90 requested review from giltene and mikeb01 July 24, 2026 09:09
@fcostaoliveira
fcostaoliveira force-pushed the fix/top-bucket-value-range-overflow branch from cedc46a to 92a219f Compare August 28, 2026 09:54
@fcostaoliveira
fcostaoliveira force-pushed the fix/iterator-reporting-level-overflow branch from 49b2366 to 655ad34 Compare August 28, 2026 10:31
fcostaoliveira and others added 2 commits August 28, 2026 15:38
Found by adversarial review during the ClusterFuzzLite effort. For a histogram
whose highest_trackable_value is near INT64_MAX, the linear and logarithmic
iterators advanced their reporting level with 'next += value_units_per_bucket'
(iter_linear_next) and 'next *= (int64_t)log_base' (log_iter_next), both of
which overflow int64 -> signed-overflow UB, then a negative left-shift when the
wrapped value flows into lowest_equivalent_value.

Saturate the reporting level at INT64_MAX when the next step would overflow.
Crucially, also pin next_value_reporting_level_lowest_equivalent to INT64_MAX
in that case: no bucket value ever equals INT64_MAX, so the 'iter->value >=
level' emit test can no longer fire and the iterator drains its remaining
buckets and terminates rather than re-reporting the saturated level forever.

Verified under ASan+UBSan: linear (value_units_per_bucket=2^62) and log (base 2)
over an INT64_MAX-range histogram no longer trap and terminate (2 and 64 steps);
normal-range iteration is unchanged. Regression test added.

Stacked on #148 (whose move_next fix the iterators depend on).

Found-by: adversarial review during the ClusterFuzzLite fuzzing effort
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adversarial review of the reporting-level saturation found two more iterator
issues in the same functions:

- next_value_greater_than_reporting_level_upper_bound peeked
  hdr_value_at_index(h, counts_index + 1); at the last bucket that reads one
  past counts[], and value_from_index shifts into the sign bit -> signed-shift
  UB for a near-INT64_MAX range. It was masked before this PR (the reporting
  level overflowed first); saturating that unmasked it. Require counts_index+1
  to be in range before peeking.

- Degenerate iterator parameters looped forever: value_units_per_bucket <= 0
  (level never advances) and log_base <= 1 (cast to 0/1, *= never advances).
  Pin the level (and its lowest-equivalent) to INT64_MAX in those cases too so
  the emit test cannot re-fire and the iterator drains and terminates. The
  degenerate checks are ordered first so the guard arithmetic (INT64_MAX - vpb,
  INT64_MAX / base) can't itself overflow or divide by zero.

Verified under ASan+UBSan: the peek repro (hdr_init(1, 1<<62, 1), record
1<<62, linear vpb=INT64_MAX-1 / log base 2) and the degenerate cases (vpb=0,
log_base 1.0/0.5/1.5) all terminate with no trap; normal iteration unchanged.
Regression tests extended.

Found-by: adversarial review during the ClusterFuzzLite fuzzing effort
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@fcostaoliveira
fcostaoliveira force-pushed the fix/iterator-reporting-level-overflow branch from 655ad34 to e478997 Compare August 28, 2026 14:41
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

🤖 Automated first-pass review — a human maintainer's review is still required before merge.

The core overflow fix reads correctly to me, and the diff stays on the safe side of the ABI line: no change to struct hdr_histogram, struct hdr_iter, or any public signature, so there's no HDR_SOVERSION_* question here. Three things worth a look:

  1. The degenerate-parameter handling is a new, undocumented public-API contract. hdr_iter_linear_init with value_units_per_bucket <= 0, and hdr_iter_log_init with log_base <= 1 or a non-positive first bucket, now quietly produce a draining iteration instead of hanging. Better than an infinite loop, certainly — but my recollection is that Java's LinearIterator/LogarithmicIterator reject those arguments up front rather than degrading, and I haven't actually checked that against the current upstream source, so I'd want that confirmed before settling on saturation as the C semantics. Either way, include/hdr/hdr_histogram.h currently says nothing about the valid range for these parameters and this PR doesn't add it; both init functions return void, so the doc comment is the only place that contract can live.

  2. peek_next_value_from_index duplicates hdr_value_at_index's decomposition. The index→(bucket_index, sub_bucket_index) math now exists in two places and will drift. Saturating inside value_from_index, or adding a saturating variant next to it, keeps it in one place.

  3. The second commit's message says the peek "reads one past counts[]." It doesn't — hdr_value_at_index is pure arithmetic and never touches the array. The signed-shift UB is real; the out-of-bounds-read framing isn't, and that's the text that ends up in git log for a fix people will go read later.

On coverage: the per-PR ClusterFuzzLite job runs ASan only — UBSan is weekly-batch only — so this whole class of signed-overflow/shift UB isn't caught by the per-PR gate. The added unit test is the real regression gate here, and the 1023-step tail assertion is a good thing to have after the peek-guard revision, given that revision was itself fixing an over-eager guard.

Finally, the description still says to base this on #148 and retarget to main once that merges — worth confirming that's actually done. The last three commits also have empty bodies, unlike the first two, which are unusually good.

Comment thread test/hdr_histogram_test.c
Comment on lines +619 to +622
while (hdr_iter_next(&iter))
{
mu_assert("linear iterator must terminate", ++steps < 1000000);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like iter.value_iterated_to remains at the previous reporting level after saturation (instead of being updated to INT64_MAX)
Should this test also assert that the final reported value is INT64_MAX?

If so, it should also apply for the log iterator test too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants