Repository navigation
fix: validate leaf extraction alias suffixes - #26969
Open
mattsu2020 wants to merge 2 commits into
Open
mattsu2020 wants to merge 2 commits into
mattsu2020 wants to merge 2 commits into
Conversation
mattsu2020
marked this pull request as ready for review
October 31, 2026 11:07
Contributor
Author
|
I also reviewed the AI-generated code myself. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Partially addresses #25449; this does not close it. This is a narrower follow-up to closed #26769, focused on validating extraction alias suffixes. The typed-marker design remains open.
Rationale for this change
A user alias such as
__datafusion_extracted_manualcan make the optimizer move an ordinary expression below a filter. For example, this query fails with division by zero even though the filter excludes zero:With this change it returns
10. Zero and zero-padded suffixes, such as_0and_01, have the same problem and are covered by this fix as well.What changes are included in this PR?
Recognize extraction aliases only when the prefix is followed by an underscore and an ASCII decimal ID in
1..usize::MAX, without leading zeros. Share that check between extraction recognition and counter seeding. Update the existing extraction fixture to use a valid ID.Exclude
usize::MAXbecause counter seeding adds one to the parsed ID and would panic with overflow checks enabled. This is a local input guard, not a complete solution to generator exhaustion: the generator can still wrap if advanced near the integer limit. That broader counter behavior is outside this PR.What is the testing strategy for this PR?
The new cases in
projection_pushdown.sltcover the bare prefix and the suffixes_manual,_1x,_+1,_0, and_01. All reproduce division by zero without the fix and return the expected result with it. Existing plan expectations are unchanged.Rust tests cover suffix boundaries, including leading zeros, signs, non-ASCII digits,
usize::MAX, and out-of-range values. A plan-level regression checks that invalid IDs do not advance the generator or cause the maximum-value panic. Integer limits are derived from the target'susizewidth.The leaf-extraction tests,
projection_pushdown.slt, workspace extended test suite, and Clippy with all targets and features passed.Are there any user-facing changes?
Aliases that merely share the extraction prefix no longer cause ordinary expressions to be treated as internal extractions. Public Rust APIs and protobuf formats are unchanged.
Canonical names such as
__datafusion_extracted_1remain reserved and ambiguous. Resolving that ambiguity requires the separate marker design discussed in #25449.This draft was prepared with AI assistance.