[555aaditya]

Rationale for this change

dataset.scanner(fragment_readahead=0).to_table() never returns.

GH-15264 (#29185) made the scan node read fragments with a concatenated generator when fragment_readahead is 0 or 1. That change only covered require_sequenced_output. The unsequenced path, which ToTable and ScanBatchesUnordered use, still calls MakeMergedGenerator(..., fragment_readahead). With 0, the merged generator never subscribes to a fragment, so the scan waits forever.

ScannerBuilder::FragmentReadahead accepts 0, so 0 is a valid value.

What changes are included in this PR?

MakeScanNode now uses the concatenated generator for fragment_readahead <= 1 on both paths. For fragment_readahead > 1 nothing changes.

Are these changes tested?

Yes.

• C++: added TestScanner.ScanWithoutFragmentReadahead. It runs ordered and unordered scans with fragment_readahead 0 and 1. Without the fix it hangs on the first case. With the fix all 4 parameterizations pass.
• Python: added test_scanner_no_fragment_readahead in test_dataset.py, which reproduces the snippet from the issue.
• ctest -R "arrow-dataset|arrow-acero" passes locally (26 tests, macOS arm64, debug build).

Are there any user-facing changes?

Scans with fragment_readahead=0 no longer hang.

Was AI used for this PR?

In accordance to the AI generation guidelines, please disclose below whether and how AI was used in this PR.

PR code and description written by:

• Human
• AI

Reviewed before submission by:

• Human
• AI
• Not reviewed

• GitHub Issue: [Python][C++] scanner(fragment_readahead=0).to_table() hangs indefinitely. #48111

[github-actions]

⚠️ GitHub issue #48111 has been automatically assigned in GitHub to PR creator.

[555aaditya]

For context: #51543 tried to fix this earlier and was closed. This PR takes a slightly different route. Instead of keeping the merged generator and forcing it to one subscription, it uses the concatenated generator for fragment_readahead <= 1 on both paths, which is what the sequenced path has done since GH-15264. I reproduced the hang locally, checked that both new tests hang without the fix, and also checked empty and multi-file datasets with use_threads on and off.