[connortsui20]

Tracking Issue: #10701

Stacked on #10905. Merge this right after #10905, which itself waits for 0.91.0 to publish the CD2 compat fixtures. Without #10905 underneath, an i128 array whose values fit i64 would get no lower parts and decode at the MSP width.

Summary

The btrblocks decimal scheme copied each array into narrowed_decimal, the narrowest width that holds every slot (null slots included), and split it by that stored width. A large value in a null slot could then force lower parts that the lower-part bound of #10905 rejects. The narrow DecimalArray that the copy builds also becomes illegal once every DecimalArray must have the canonical width of its dtype. The scheme now starts from the canonical array and chooses the lower parts from the valid values only.

Changes

• lower_part_count returns how many lower parts the valid values need: 0 when they fit i64, 1 when they fit i128, and 3 otherwise. It uses valid_range instead of the decimal min_max aggregate, because that aggregate builds decimal scalars, which panic on a value outside the precision.
• split_decimal takes that count, so a wide array whose values fit i64 copies into an i64 MSP, and an i256 array whose values fit i128 splits into one lower part.
• In v1 mode, an array that needs lower parts stays canonical. Without lower parts, only the MSP narrows, through the deprecated WidenArray::encode_values (with a TODO(connor)[Widen] on whether it should become a Widen).
• narrowed_decimal is removed, since the scheme was its only caller.

The first commit adds golden entries on the old scheme, so the snapshot diff shows exactly what changes on disk. Only two entries change, and both are accepted:

• decimal_wide_p40: since the decimal is no longer narrowed, v1 mode writes precision 39 to 76 with values that need 65 to 128 bits as a canonical i256 array instead of an i128 one.
• decimal_null_slot_values: large values in null slots no longer widen the MSP, which goes from i64 to i32.

API Changes

Adds vortex_decimal_byte_parts::lower_part_count. split_decimal takes the lower-part count and returns an error for a count that a valid value does not fit, or for a shape it does not support. Removes vortex_array::arrays::decimal::narrowed_decimal.

[codspeed]

Merging this PR will improve performance by 25.32%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 7 improved benchmarks
✅ 2358 untouched benchmarks

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	CPU Simulation	filter_powerlaw_by_random[10000]	700.5 ns	384.8 ns	+82.01%
⚡	CPU Simulation	bench_compare_varbin[(50000, 2)]	161.4 µs	132.5 µs	+21.87%
⚡	CPU Simulation	bench_compare_varbin[(50000, 64)]	163.6 µs	134.2 µs	+21.86%
⚡	CPU Simulation	encode_primitives[u8, (2000, 4)]	163.8 µs	135.1 µs	+21.25%
⚡	CPU Simulation	bench_compare_sliced_dict_varbinview[(1000, 10000)]	89.2 µs	77.8 µs	+14.56%
⚡	CPU Simulation	bench_compare_sliced_dict_varbinview[(7500, 10000)]	227.1 µs	199.4 µs	+13.9%
⚡	CPU Simulation	bench_compare_sliced_dict_varbinview[(10000, 10000)]	233.3 µs	205.6 µs	+13.52%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/dbp-scheme-canonical (53dd750) with ct/dbp-canonical-decode (8d7e3ee)}