fix: preserve per-node UNION vs UNION ALL when unparsing - #24498
Open
shinzoxD wants to merge 1 commit into
Open
Conversation
QueryBuilder::distinct_union was a sticky query-level flag, so a nested Distinct(Union) rewrote every UNION in the statement as DISTINCT. Consume the flag per UNION node and unparse each operand in its own QueryBuilder so mixed ALL vs DISTINCT (and parentheses) are preserved.
Contributor
|
we seem to already have a PR for this |
Contributor
|
for anyone looking at this PR please see my comment here: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
Unparsing mixed
UNION/UNION ALLtrees was incorrect.a UNION ALL (b UNION a)was emitted as a flat, all-distincta UNION b UNION a.QueryBuilder::distinct_unionwas a single query-level boolean: walking the innerDistinct(Union)set that flag, and every later UNION in the same statement then unparsed as distinct. Set quantifiers are a per-node property, so a sticky statement-wide flag cannot represent mixed ALL vs DISTINCT.What changes are included in this PR?
distinct_unionon the UNION node that it belongs to (take_distinct_union), so the flag cannot leak to a parent or sibling union.QueryBuilder. Nested distinct unions (and operand-scopedORDER BY/LIMIT) stay isolated from the enclosing set operation.SetOperations, so this is required to keepALLvs distinct precedence.Are these changes tested?
Yes. The issue reproduction is in
datafusion/sql/tests/cases/plan_to_sql.rsas both a plan round-trip and a snapshot of the unparsed SQL. The snapshot asserts the outerUNION ALLand the parenthesized innerUNION. Without this change the snapshot fails because the unparser emits a flat distinct UNION.Are there any user-facing changes?
Unparsed SQL for mixed
UNION/UNION ALLnow preserves each node's set quantifier and the parentheses needed for that meaning. This is a bug fix forplan_to_sql; there is no public API break.