Skip to content

Push decimal comparisons, between and fills through Widen - #11003

Draft
connortsui20 wants to merge 1 commit into
ct/decimal-widenfrom
ct/decimal-widen-compare
Draft

connortsui20 wants to merge 1 commit into
ct/decimal-widenfrom
ct/decimal-widen-compare

Conversation

@connortsui20

@connortsui20 connortsui20 commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Tracking Issue: #10701

Stacked on #10946. #11004 builds on it. No release gate.

Summary

Without kernels, a compare, between or fill null on a decimal Widen first copies every value into the canonical width. These now run on the narrow child.

Changes

A constant within the child precision is cast to the child dtype. A constant outside the child precision answers the compare without reading the values, because a decimal precision range is symmetric about zero. Two decimal Widens compare at the larger child precision. between already rewrites into two compares, so it uses the same kernel. A fill value outside the child precision declines, because only the values can pick a wider child.

These kernels decide from the child dtype alone, so a child precision that misses a valid value gives wrong answers. The Widen docs now state that rule.

Local benchmark results

cargo bench -p vortex-array --bench widen_decimal, Apple M4, medians. The numbers were measured at the top of this stack before the split, so they cover the whole stack, not this layer alone. The baseline is the canonical input, and the candidate is the same values as a Widen (child precision 5 under precision 10, 9 under 18, and 18 under 38). CodSpeed on this PR measures the layer itself.

bench p10 canonical p10 Widen p18 canonical p18 Widen p38 canonical p38 Widen
compare_constant 4.166 µs 5.041 µs 4.124 µs 5.041 µs 10.7 µs 4.207 µs
compare_constant_out_of_range 3.791 µs 374.5 ns 3.79 µs 374.5 ns 9.874 µs 397.9 ns

API Changes

None.

Without kernels, a compare, between or fill null on a decimal Widen
first copies every value into the logical width. Run them on the
narrower child instead. A constant within the child precision is cast
to the child dtype, a constant outside it answers a compare without
reading values, and two decimal Widens compare at the larger child
precision. A fill value outside the child precision declines, because
only the values can pick a wider child.

These kernels decide from the child dtype alone, so document that the
child precision must hold every valid value.

Signed-off-by: Connor Tsui <[email protected]>
@codspeed

codspeed Bot commented Oct 29, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 6 improved benchmarks
❌ 1 regressed benchmark
✅ 2353 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation filter_powerlaw_by_random[10000] 418 ns 1,236.4 ns -66.2%
⚡ CPU Simulation compare_constant_out_of_range[p38_childp18_widen] 1,217.5 µs 52.9 µs ×23
⚡ CPU Simulation compare_constant_out_of_range[p18_childp9_widen] 503.6 µs 52.8 µs ×9.5
⚡ CPU Simulation compare_constant_out_of_range[p10_childp5_widen] 503.6 µs 52.9 µs ×9.5
⚡ CPU Simulation compare_constant[p38_childp18_widen] 1,217.5 µs 323.4 µs ×3.8
⚡ CPU Simulation compare_constant[p18_childp9_widen] 503.5 µs 214.6 µs ×2.3
⚡ CPU Simulation compare_constant[p10_childp5_widen] 503.6 µs 214.9 µs ×2.3

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/decimal-widen-compare (3f13e37) with ct/decimal-widen (fdf9ac2)

Open in CodSpeed

@connortsui20 connortsui20 added the changelog/performance A performance improvement label Oct 29, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant