Skip to content

Fix run-tests.sh and run.sh - #110

Open
iHiD wants to merge 3 commits into
remove-tbbfrom
runner-script-fixes
Open

iHiD wants to merge 3 commits into
remove-tbbfrom
runner-script-fixes

Conversation

@iHiD

@iHiD iHiD commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

This description was written by Claude (Claude Code), not by iHiD.

This pulls the bug fixes out of #106 so they can be reviewed on their own. It is stacked on #109.

  • The test suite could never fail. bin/run-tests.sh compares each test's results with its expected results. It checked whether that comparison had failed only after deleting the temporary files, so it was really checking whether the delete worked. A test with the wrong results showed a diff in the log, but the run still passed.
  • Line numbers were never ignored. Before comparing, the script is meant to replace line numbers with a placeholder, so that moving code around in a test doesn't break the comparison. The pattern it used (\d) isn't something sed understands, so nothing was ever replaced.
  • One test depended on the exact compiler version. Once the suite could fail, tests/example-empty-file did. Its expected results hold the linker's exact error message, recorded with GCC 12.2.1 in 2023. The image now has GCC 15.2.0, whose message has a different path and an extra offset. Build errors are now compared on their status only, so compiler upgrades don't break the suite.
  • bin/run.sh used bash features while declaring sh. Alpine's sh happens to accept them, which is why it works in the Docker image, but any sh that sticks to the standard would fail on them.
  • include/helpers.h used std::ptr_fun, which was removed from the language in C++17. GCC still accepts it, but clang with libc++ does not. It now uses a lambda, which does the same thing.

🤖 Generated with Claude Code

allsey87 and others added 2 commits October 22, 2026 13:22
run-tests.sh checked $? after the rm that cleaned up the temporary files, so
it saw the exit status of rm and the suite never failed on a mismatch. It now
checks the result of diff directly.

The line-number pattern used \d, which sed does not understand, so line
numbers were never normalised. It now uses [0-9]\{1,\}.

std::ptr_fun and std::not1 were removed in C++17, so helpers.h now uses a
lambda instead.

Co-authored-by: Michael Allwright <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@iHiD
iHiD requested a review from a team as a code owner October 22, 2026 04:25
@iHiD
iHiD added this pull request to stack #111 October 22, 2026 04:26
The expected results stored the exact compiler output, which changes
with every compiler upgrade. For example, example-empty-file was
recorded with GCC 12.2.1 in 2023 and the image now has GCC 15.2.0.
Build errors are now compared by their status, so the suite no longer
breaks when the compiler changes.

Co-authored-by: Michael Allwright <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants