Skip to content

ci: fix build cache, drop duplicated test run - #1385

Open
marcleblanc2 wants to merge 3 commits into
mainfrom
marc/ci-go-cache
Open

marcleblanc2 wants to merge 3 commits into
mainfrom
marc/ci-go-cache

Conversation

@marcleblanc2

@marcleblanc2 marcleblanc2 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Why

CI compiled the whole dependency tree from scratch on every run, on all three OSes: go-test took 5–9 min, go-lint 3 min. Three causes:

  • The Go build cache was never populated. goreleaser-check saved an empty cache under the same key the test and lint jobs use, so they always "hit" that empty cache and never saved their own.
  • go-ci ran the test suite twice: once with -race, then again without, which recompiles everything. The second pass only existed because the upload tests in internal/batches/service were excluded from -race (Turn off race detection for upload test #1082).
  • The Windows job was silently failing under -race (the Windows upload test was never excluded), but the non-race second pass masked it.

What

Why application code changed

Running the upload tests under -race exposed a real race in the test mock, not in the tests themselves. The mock recorded the request body as a bare *io.PipeReader; when testify printed the recorded arguments to compare them, it walked into the pipe's internals while the upload goroutine was still writing to it. internal/api/mock now wraps the body in a small RequestBody struct so testify prints a pointer instead of walking the pipe. With that fixed, the !race build tag and the separate non-race test pass are no longer needed.

While there, uploadFile now starts its writer goroutine only after the request is built; before, an error building the request leaked a goroutine blocked on a pipe nobody reads.

Measured on this branch

job before (#1383) cold (first push) warm (workflow_dispatch rerun)
go-lint 2m58s 3m10s 18s
go-test ubuntu 5m15s 3m23s 30s
go-test macos 6m06s 3m28s 31s
go-test windows 9m06s 5m45s 1m50s (74 s of it is the cache untar)
goreleaser check 7s 5s 5s

Caches on main appear after this merges; until then every branch's first push is cold.

Amp thread

https://ampcode.com/threads/T-01a18b5c-c5bb-7497-82f1-3e9333c8b7e1

Every CI run compiled the whole dependency tree from scratch on all three
OSes, because:

- goreleaser-check ran setup-go without ever building anything, so it
  finished first and saved a 7 KB cache under the key that go-lint and
  go-test on Linux then restored ('cache hit', never re-saved).
- setup-go's built-in cache has no restore-keys, so any go.sum change
  (every renovate PR) started cold on macOS and Windows too.
- go-ci ran 'go test -race' and then 'go test' again, recompiling
  everything without race instrumentation to rerun the same tests.

Now each job owns its cache key (job, OS, arch, Go version, go.sum hash)
with a prefix fallback, goreleaser-check no longer installs Go, and the
test job runs the race pass only.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18b5c-c5bb-7497-82f1-3e9333c8b7e1
Co-authored-by: Amp <[email protected]>
@github-actions

Copy link
Copy Markdown

The upload tests were tagged //go:build !race in #1082 because testify
fmt-prints every recorded mock argument, and printing a bare
*io.PipeReader reflects into the pipe's internals while the producer
goroutine writes to it. The Windows variant never got the tag, so
'go test -race' has been failing on Windows the whole time, hidden
because the step runs under pwsh, which does not fail on a non-zero
exit in the middle of a multi-line script; the second non-race pass
then went green.

The mock client now wraps the body in a RequestBody struct, so fmt
prints the pointer address instead of walking the pipe. That removes
the need for the !race tag and for the second test pass.

uploadFile also starts the writer goroutine only after NewHTTPRequest
succeeds; before, an error there leaked a goroutine blocked forever on
a pipe nobody reads.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18b5c-c5bb-7497-82f1-3e9333c8b7e1
Co-authored-by: Amp <[email protected]>
@marcleblanc2 marcleblanc2 changed the title ci: cache Go modules and build output per job, drop redundant test pass ci: fix build cache, drop redundant test pass Oct 30, 2026
@marcleblanc2
marcleblanc2 enabled auto-merge (squash) October 30, 2026 06:33
@marcleblanc2 marcleblanc2 changed the title ci: fix build cache, drop redundant test pass ci: fix build cache, drop duplicate test run Oct 30, 2026
@marcleblanc2 marcleblanc2 changed the title ci: fix build cache, drop duplicate test run ci: fix build cache, drop duplicated test run Oct 30, 2026
@marcleblanc2
marcleblanc2 requested a review from a team October 30, 2026 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant