Skip to content

Change SpillPartitionId::kMaxSpillLevel to 7 #15400

New issue
New issue
Open
#15401
Open
Change SpillPartitionId::kMaxSpillLevel to 7#15400
#15401
Labels
enhancementNew feature or request
@wForget

Description

@wForget
wForget
opened on Nov 5, 2025

Description

Forwarded issue from: apache/incubator-gluten#11018

I am getting the Spill level 4 exceeds max spill level 3 error in Gluten 1.5.0. and I can't increase this limit value by setting max_spill_level.

Based on the design of encodedId_, spillLevel can support up to 7 level (3 bits). Could we change SpillPartitionId::kMaxSpillLevel from 3 to 7?

velox/velox/exec/Spill.h

Lines 325 to 335 in ba52907

// Encoded hirachical spill partition id. Below shows the layout from the low
// bits.
// <LSB>
// (0 ~ 2 bits): Represents the partition number at the 1st level.
// (3 ~ 5 bits): Represents the partition number at the 2nd level.
// (6 ~ 8 bits): Represents the partition number at the 3rd level.
// (9 ~ 11 bits): Represents the partition number at the 4th level.
// (12 ~ 28 bits): Unused
// (29 ~ 31 bits): Represents the current spill level.
// <MSB>
uint32_t encodedId_{kInvalidEncodedId};

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions