Skip to content

fix: validate leaf extraction alias suffixes - #26969

Open
mattsu2020 wants to merge 2 commits into
apache:mainfrom
mattsu2020:fix/leaf-extraction-alias-format
Open

mattsu2020 wants to merge 2 commits into
apache:mainfrom
mattsu2020:fix/leaf-extraction-alias-format

Conversation

@mattsu2020

@mattsu2020 mattsu2020 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Partially addresses #25449; this does not close it. This is a narrower follow-up to closed #26769, focused on validating extraction alias suffixes. The typed-marker design remains open.

Rationale for this change

A user alias such as __datafusion_extracted_manual can make the optimizer move an ordinary expression below a filter. For example, this query fails with division by zero even though the filter excludes zero:

CREATE TABLE t AS SELECT * FROM (VALUES (0), (1)) AS v(id);
SELECT 10 / id AS __datafusion_extracted_manual FROM t WHERE id > 0;

With this change it returns 10. Zero and zero-padded suffixes, such as _0 and _01, have the same problem and are covered by this fix as well.

What changes are included in this PR?

Recognize extraction aliases only when the prefix is followed by an underscore and an ASCII decimal ID in 1..usize::MAX, without leading zeros. Share that check between extraction recognition and counter seeding. Update the existing extraction fixture to use a valid ID.

Exclude usize::MAX because counter seeding adds one to the parsed ID and would panic with overflow checks enabled. This is a local input guard, not a complete solution to generator exhaustion: the generator can still wrap if advanced near the integer limit. That broader counter behavior is outside this PR.

What is the testing strategy for this PR?

The new cases in projection_pushdown.slt cover the bare prefix and the suffixes _manual, _1x, _+1, _0, and _01. All reproduce division by zero without the fix and return the expected result with it. Existing plan expectations are unchanged.

Rust tests cover suffix boundaries, including leading zeros, signs, non-ASCII digits, usize::MAX, and out-of-range values. A plan-level regression checks that invalid IDs do not advance the generator or cause the maximum-value panic. Integer limits are derived from the target's usize width.

The leaf-extraction tests, projection_pushdown.slt, workspace extended test suite, and Clippy with all targets and features passed.

Are there any user-facing changes?

Aliases that merely share the extraction prefix no longer cause ordinary expressions to be treated as internal extractions. Public Rust APIs and protobuf formats are unchanged.

Canonical names such as __datafusion_extracted_1 remain reserved and ambiguous. Resolving that ambiguity requires the separate marker design discussed in #25449.

This draft was prepared with AI assistance.

@github-actions github-actions Bot added optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Oct 31, 2026
@mattsu2020 mattsu2020 changed the title fix: require numeric suffixes for leaf extraction aliases fix: validate leaf extraction alias suffixes Oct 31, 2026
@mattsu2020
mattsu2020 marked this pull request as ready for review October 31, 2026 11:07
@mattsu2020

Copy link
Copy Markdown
Contributor Author

I also reviewed the AI-generated code myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant