Skip to content

Chunked scans miss EXACT/NOT_EXACT matches that straddle a 256 MiB chunk boundary (overlap only applied to str) #85

Description

@krishna3554

Summary

In process/scanning.py, iter_search_results reads bufflength - 1 extra bytes past a chunk boundary only when pytype is str. But EXACT_VALUE and NOT_EXACT_VALUE scans are routed through scan_memory_for_exact_value, which locates the target bytes with bytes.find at any byte offset — not just the bufflength-aligned offsets a stepping numeric scan visits. For any region larger than DEFAULT_MAX_REGION_CHUNK (256 MiB), an exact match whose bytes cross an internal chunk boundary is found by neither chunk and is silently dropped from the results.

Location

  • File: PyMemoryEditor/process/scanning.py
  • Function: iter_search_results
  • Relevant code:
str_overlap = bufflength - 1 if pytype is str else 0
...
is_last_chunk = chunk_offset + chunk_size >= size
read_size = chunk_size + (0 if is_last_chunk else str_overlap)

with the routing decision a few lines above:

if scan_type in (ScanTypesEnum.EXACT_VALUE, ScanTypesEnum.NOT_EXACT_VALUE):
    searching_method = scan_memory_for_exact_value

Problem

The overlap logic assumes non-string scans step in bufflength strides, so a typed numeric scan can never have a value straddle a boundary — the rationale documented on util.scan.iter_region_chunks. That holds for the comparison scans driven by scan_memory (BIGGER_THAN, VALUE_BETWEEN, ...), which iterate aligned windows inside each chunk. It does not hold for the two scan types that bypass scan_memory:

  1. EXACT_VALUEscan_memory_for_exact_value runs data.find(target) from every offset of the buffer it receives. Chunk N ends mid-value and chunk N+1 starts after the value began, so no chunk ever sees the complete byte sequence → missed result.
  2. NOT_EXACT_VALUE — the candidate range per chunk is range(0, read_size - bufflength + 1, step). Windows straddling a boundary fall outside both neighboring chunks' ranges, so offsets whose value differs from the target are silently omitted there as well.

Since EXACT_VALUE is the default scan_type of search_by_value, this affects the library's primary code path whenever a single region exceeds 256 MiB — large heaps (browsers, JVMs) being exactly the case the 256 MiB cap exists for.

Trigger / Reproduction

Static-analysis finding (not executed); the trace follows the code paths above.

  • A region larger than 256 MiB, split by iter_region_chunks into aligned chunks.
  • An int32 (or any-width) target whose bytes sit across a chunk boundary, e.g. starting at chunk_size - 2.
  • search_by_value(int, value=..., scan_type=ScanTypesEnum.EXACT_VALUE) over that region reports nothing for that address; the same scan on an unchunked region finds it.

The existing test tests/scan/test_chunking_integration.py::test_scan_memory_across_chunked_region_finds_all_matches plants targets at offsets 100 / 5000 / 60000 in each 64 KiB chunk — all safely ≥ 4 bytes from the end — so the straddling case is currently untested.

Expected Behavior

Every exact match present in the region should be reported regardless of where it falls relative to internal read chunks; NOT_EXACT should evaluate every candidate window exactly once.

Actual Behavior

Matches/windows that straddle an interior chunk boundary of a >256 MiB region are skipped without any error or warning.

Impact

Silent false negatives in memory scans — a particularly bad failure mode for a cheat-engine-style workflow, because the user concludes the value is not present rather than suspecting the tool. The larger the target process's regions, the more likely a boundary lands mid-value.

Suggested Direction

Widen the overlap condition to cover every scan mode that can match at a non-aligned offset, e.g.

needs_overlap = pytype is str or scan_type in (
    ScanTypesEnum.EXACT_VALUE,
    ScanTypesEnum.NOT_EXACT_VALUE,
)
overlap = bufflength - 1 if needs_overlap else 0

The existing clamp (if offset >= chunk_size: continue) already prevents double-emission of overlap-region matches, and the stepping comparisons driven by scan_memory keep their current no-overlap behavior unchanged. A regression test planting a target at chunk_size - bufflength // 2 would pin this.

Evidence

  • process/scanning.py gates the extra read solely on pytype is str, while the routing block sends EXACT/NOT_EXACT to the offset-arbitrary scan_memory_for_exact_value.
  • util/scan.py::scan_memory_for_exact_value yields every data.find hit with stride 1 — alignment is not considered.
  • util/scan.py::iter_region_chunks docstring documents the aligned-stride assumption that EXACT scanning does not follow.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions