Skip to content

WIP: Boolean mask backed RowSelection #8902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

WIP: Boolean mask backed RowSelection #8902

wants to merge 3 commits into from
+915 −63

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Nov 21, 2025 •
edited
Loading

Which issue does this PR close?

TODOs

  • Make a PR to extract the RowSelection backed code into its own module

Rationale for this change

Make the parquet predicate evaluation faster by not converting back/forth between BooleanArray and RowSelection as much

What changes are included in this PR?

  1. Change RowSelection to have two possible backings
  2. Add a BooleanArray backed implementation, based on @XiangpengHao 's code from [Parquet] Add BooleanArray based row selection #6624

Are these changes tested?

TBD

Are there any user-facing changes?

Internal notes for myself

  • Get it compiling, not actually routing through BooleanSelection
  • Start routing through BooleanSelection

@github-actions github-actions bot added the parquet Changes to the parquet crate label Nov 21, 2025
@alamb alamb mentioned this pull request Nov 21, 2025
Draft
@alamb alamb mentioned this pull request Nov 21, 2025
Draft
alamb
alamb commented Nov 25, 2025
View reviewed changes
parquet/src/arrow/arrow_reader/selection/mask.rs
/// Optimized version of `boolean_buffer_and_then` using BMI2 PDEP instructions.
/// This function performs the same operation but uses bit manipulation instructions
/// for better performance on supported x86_64 CPUs.
pub fn boolean_buffer_and_then(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {
Copy link
Contributor Author

@alamb alamb Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@XiangpengHao did this it in liquid cache:

I will copy that new version over here in this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

parquet Changes to the parquet crate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Parquet] Avoid Mask --> RowSelection --> Mask conversion to improve predicate pushdown performance

1 participant

@alamb