[connortsui20]

Tracking Issue: #10701

Stacked on #11057. #10946 builds on it. No release gate.

Summary

Widen has no wire form yet (#10733), but IPC and writes without a compressor still need to store it. A writer plugin makes that work through the same session API used by direct metadata callers.

Changes

An integer Widen writes its canonical primitive form. Recursive canonicalization makes lazy child validity storable, while keeping the Widen as the serialization source preserves statistics in its logical dtype. Restricted contexts need only the existing primitive wire ID. Decimal writes follow in #10946.

API Changes

Adds public vortex_array::arrays::widen::WidenPlugin to the default ArraySession. Empty array sessions can register it explicitly. The plugin registers no reader because vortex.widen has no wire form.

[codspeed]

Merging this PR will degrade performance by 7.89%

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 7 improved benchmarks
❌ 6 regressed benchmarks
✅ 2305 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	CPU Simulation	filter_powerlaw_by_random[10000]	358.1 ns	999.1 ns	-64.15%
❌	Walltime	threshold_neon[64]	2.6 µs	3.8 µs	-32.65%
❌	Walltime	threshold_neon[80]	3.4 µs	4.4 µs	-22.69%
❌	Walltime	filter_neon[80]	2.5 µs	3.1 µs	-20.87%
❌	Walltime	filter_neon[64]	1.8 µs	2.2 µs	-16.45%
❌	Walltime	scalar_subtract_neon	10.5 µs	12 µs	-12.01%
⚡	CPU Simulation	bench_compare_varbin[(50000, 2)]	162.8 µs	132.7 µs	+22.67%
⚡	CPU Simulation	bench_compare_varbin[(50000, 64)]	164.7 µs	134.5 µs	+22.38%
⚡	CPU Simulation	encode_primitives[u8, (2000, 4)]	163.3 µs	135 µs	+21.01%
⚡	CPU Simulation	take_filter_primitive_small_uncached_random_mask_random_indices[4096, 10]	24.4 µs	20.9 µs	+16.83%
⚡	CPU Simulation	bench_compare_sliced_dict_varbinview[(1000, 10000)]	88.4 µs	77.2 µs	+14.47%
⚡	CPU Simulation	bench_compare_sliced_dict_varbinview[(7500, 10000)]	228 µs	199.4 µs	+14.33%
⚡	CPU Simulation	bench_compare_sliced_dict_varbinview[(10000, 10000)]	234.2 µs	205.5 µs	+13.96%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/widen-serialize-canonical (72749e5) with ct/array-plugin-read-write-ids (9556b96)}

[connortsui20]

Here is an open question: instead of doing this, do we want to have a more generalized version of this fix?

Not saying the PR should be blocked on this (as I do need to make progress on this stack), but there exist other encodings that we only have in-memory that we would want to ensure do not get serialized to disk (either because of an edition, or because we do not want them to be serialized ever).

Some examples:

• Filter
• Slice
• Patched (the LLM is even telling me that this is a bug, and Patched needs a similar change as this one)
• ScalarFn (which we serialize now but imo we might want a different name for serialized scalar functions, see Tracking Issue: Encodings vs Scalar Functions #9156)
• Any encoding that might easily be created in memory but we disallow via editions