Repository navigation
test: hold every edit mutate makes across the repository - #39
Merged
Merged
Conversation
Mutation testing showed these edits change nothing a caller sees: skipJSONValue only meets opening delimiters at a value's position and can consume an object's keys like its values; orderedObjectKeys' empty input already fails the first token; ociAnnotationID's missing or empty annotation already fails to unmarshal; filepath.Localize only returns local paths, so the IsLocal check after it refuses nothing; every caller of downloadFeatureTarball passes a context; and the replacer's capacity hint is unobservable. The 512 MiB download cap is exempt, as only a 512 MiB response could show it move. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A mutation run over these files left edits no test refused: the argument wiring of templates metadata, features info, generate-docs and features test, features test's flag conflicts, the template metadata failures, createTarArchive's entry names, empty user names and footer, extracted file modes, the 2xx-only tarball download and the per-user Linux cache folder. Each test asserts what a caller sees, without Docker, reusing readRegistry and httptest servers. Co-Authored-By: Claude Opus 5.5 <[email protected]>
read-configuration --cache-key is documented to hash the committed features lockfile, but computeCacheKey only used the lockfile when ReadLockfile's bool was true, and that bool reports the empty marker file, which comes back with a nil lock. A lockfile with content was therefore never hashed, so changing the pinned digests kept the key. Use the lockfile whenever ReadLockfile returns one. Keys for configurations without a lockfile, or with an empty one, are unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A mutation run over outdated.go, features_test_runner.go, secrets.go and validate.go left edits no test refused. Add tests for what callers see: upgrade's flag checks, lockfile versions and write location against the httptest registry; features test's Feature discovery, global scenarios, exit codes and container cleanup through a docker on PATH; the id-label, remote-env, mount and terminal validators; build secrets. Two small source changes make the rest testable or remove dead code: the `<self> exec` command is built by testScriptCommand, leaving execTestInContainer to the parity lane that runs it (features.test-single-scenario-success), and upgrade drops a lockfile path fallback for an empty ConfigFilePath, which a loaded configuration never has. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A whole-repository mutate run left 66 edits in open, stop, check, exec, up, build and gpu that no test refused. The new tests check what a caller depends on, without Docker: the positional [path] of open, stop, down and build, open's editor launch and config, check's exit code, saved state and JSON, the up/build host hint on a real pseudo-terminal, the lifecycle marker scripts run by the host's sh, exec's Compose container lookup against the fake engine, and the GPU runtime probe. Two edits nobody can observe carry a mutate-exempt instead: the tabwriter's minwidth, tabwidth and flags in the check table (moved to a const so the padding stays mutated and tested), and the signal branch of execExitCode, where ExitCode -1 always comes with a signaled WaitStatus. Co-Authored-By: Claude Opus 5.5 <[email protected]>
cmd/devtool had no test, so go tool mutate reported every edit in it as untested. Its commands now live in devtools.Main, which takes the arguments and standard streams and returns the exit code; main only hands it the process's. Tests drive each command through Main with fake sudo, docker, go, sh and node first on PATH and throwaway git repositories, so nothing touches the host, and assert the output, exit codes, calls and file modes the Taskfile and workflows depend on. ci-prepare-runner's decision is split into prepareRunner(free) so both disk branches are tested. Command lines, output and exit codes are unchanged: an old and a new build gave identical results for every command run locally. readLines drops its 4 MiB scanner buffer: git prints paths, which PATH_MAX keeps far below the default 64 KiB token limit. Co-Authored-By: Claude Opus 5.5 <[email protected]>
go tool mutate left six edits no test refused. Pct of a package with no statements is now tested (0, not NaN), and so is a cli file that is not a test and a test outside internal/cli, which both run the full parity matrix. ParseFloat's bitSize of 64 is exempt: any value but 32 means 64. The exec helper process now gets a GOCOVERDIR, since a binary built with coverage, as mutate builds it, warned on the helper's stderr and failed TestOSRunnerProcessContract before any edit; that hid the nil-stream edit in OSRunner.Run. A test with a single Env entry holds the len > 0 guard that the extra entry would otherwise leave untested. Co-Authored-By: Claude Opus 5.5 <[email protected]>
aledbf
force-pushed
the
test/mutation-repo
branch
from
October 25, 2026 22:04
bc280a5 to
946343d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A whole-repository
go tool mutaterun (every function, not only a diff) after #37 left 171 edits no test refused and 146 edits incmd/devtool, which had no tests at all. This PR answers each one, so the repository holds againstmutateeverywhere, not only where a pull request touches.Rebased onto main after #37 merged.
What changed
docker.APIandexec.Runnerfakes, httptest registries, fake executables onPATH). No new test-only seam.cmd/devtool.main.gois a one-line dispatcher; the commands moved tointernal/devtools/command.go, tested throughdevtools.Mainwith fakesudo,docker,goandnodeonPATH. Taskfile tasks and their output are unchanged; old and new binaries were compared on the same inputs.os.GetwdfallbackresolvePath("")already does, guards a decoder already enforces).mutate-laneonly on wiring a Docker-backed lane runs, naming the parity cases.mutate-exemptonly where both values mean the same to every caller, each with its reason;mutate -stale-exemptsin lint keeps them honest.Bugs found
read-configuration --cache-keynever folded the features lockfile into the key: the code tested the "lockfile is the empty marker" boolean, under which the lockfile is always nil.TestComputeCacheKeyFoldsInTheLockfilefails before the fix.internal/exec's process tests failed whenever the test binary was built with coverage (asmutatebuilds it), because the child printed "GOCOVERDIR not set".Left for follow-ups (found, not changed here)
upgrade --feature/--target-versionvalidates its flags and then ignores them; TS rewrites the Feature's version in devcontainer.json first.--cache-key.Validation
go tool mutate -base refactor/parity-cleanup, Docker unreachable (as on CI)mutateruns of each group over its whole filesgo test ./cmd/... ./internal/...(hermetic)task lint(golangci, testquality,mutate -stale-exempts, deadgate, taskgate, sfw)🤖 Generated with Claude Code