Skip to content

[RFC] Slim maintenance-prs: tests + security weekly, capped, ready for review, announced after green CI - #9023

Draft
isaacroldan wants to merge 2 commits into
mainfrom
gardener/rfc-slim-maintenance-prs
Draft

isaacroldan wants to merge 2 commits into
mainfrom
gardener/rfc-slim-maintenance-prs

Conversation

@isaacroldan

@isaacroldan isaacroldan commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

RFC — do not merge. For discussion in the gardener-unification work; maintenance-prs stays as-is until the team agrees.

WHY are these changes introduced?

maintenance-prs produces PRs that almost nobody picks up, and right now it doesn't run at all. Numbers are from a read-only inventory collected on 2026-10-27:

WHAT is this pull request doing?

Changes to .github/workflows/maintenance-prs.yml:

  1. Tests and security only, once a week. The matrix is now task: [tests, security], scheduled for Mondays at 00:00 UTC (0 0 * * 1). The Monday-only gate step is gone. This PR also deletes .agents/automated-tasks/refactor.md and performance.md; nothing else referenced them.
  2. Cap of 2 open PRs per task. The first step counts open PRs whose head branch starts with <task>-maintenance-. If there are 2 or more, it logs Skipping: N open <task> PRs (cap 2) and every later step is skipped. Drafts count too: today there are 20 open tests- and 5 open security- maintenance PRs, so both tasks would skip until that backlog is triaged.
  3. A token that triggers CI. Checkout (so pushes use the token), claude-code-action's github_token, and GH_TOKEN now use secrets.SHOPIFY_GH_ACCESS_TOKEN. This is the pattern shopify-app-js (main-release.yml, release-candidate.yml) and graphql-codegen (release.yml) already use: their PRs are authored by app/shopify-github-actions-access and CI runs on them (for example shopify-app-js#3494 got 13 checks). That secret does not exist on Shopify/cli yet, so a new early step fails the run with a clear error when it's empty. The workflow's own GITHUB_TOKEN is now read-only in the maintenance job.
  4. Ready for review. The prompt now says to use gh pr create without --draft. It still says "Never merge or approve" and no longer says "never mark ready". CODEOWNERS (* @shopify/dev_experience) then requests the team automatically, which is the queue Mission Control's gardener hub reads. The workflow doesn't add the devtools-gardener label, because in this repo that label makes gardener-notify-* post a second, bare Slack message. The prompt also tells Claude not to add labels or request reviewers. tests.md and security.md don't mention drafts or "ready", so they are unchanged.
  5. Announce only after required CI passes. The Slack post moves to a new announce job (needs: maintenance, 90-minute timeout).
    • Finding the PR: the job runs the same task matrix and finds its PR by the branch name both jobs compute, <task>-maintenance-<run_id>. It doesn't use job outputs, because outputs from matrix legs share names and the last leg to finish overwrites the others.
    • Waiting for checks: it polls for up to 10 minutes until required checks register. It then waits up to 70 minutes for every workflow run on the PR's head commit to finish, and decides with gh pr checks --required.
      • I didn't use gh pr checks --required --watch because it reports green too early. The required Unit tests check comes from a gate job with needs: unit-tests, and that job only registers once the unit-test matrix is done. In 40484089545 the other jobs were created at 09:52:21 and Unit tests at 10:02:41. On Remove unused Gardener - Investigate Issue workflow #9022, while its PR run was still in progress, gh pr checks --required listed 6 required checks (no Unit tests) and exited 0.
    • When CI is green: it posts one line to C0ARV62K59C with the existing SLACK_GARDENER_BOT_TOKEN / slack-github-action step, with the title still HTML-escaped: CLI maintenance · <url|[Tests] …> · CI green · Needs: review + merge from the Dev Exp gardener. The task prompts already put [Tests] or [Security] in the PR title.
    • Otherwise (failed checks, checks that never register, or a timeout): no Slack post. The job closes the PR with a short comment that links this run and lists the failing required checks, and deletes the branch.
    • Permissions: this job uses GITHUB_TOKEN with explicit permissions (actions, checks and statuses read; contents and pull-requests write). None of its actions need to trigger other workflows.
  6. Bump anthropics/claude-code-action from 36a69b6 (v1.0.134) to 8041f1d17e0e7d2f2cfe35002962e9f26237c0bb (the commit behind the annotated tag v1.0.263, which bundles Claude Code 2.1.311). --model claude-opus-latest stays. Every input this workflow uses exists in v1.0.263's action.yml.
  7. Fix a false positive in packages/cli/src/cli/repo-health.test.ts. The action-pinning test matched the uses: inside the announce job's statuses: read and reported read as an unpinned action. It now matches uses: only at a word boundary. Across all 16 workflows, the only match that changes is that false positive.

Open questions

  1. Token. Should we add SHOPIFY_GH_ACCESS_TOKEN to Shopify/cli? It needs to be able to push branches and open PRs, plus read access to Actions and issues, because the tests task reviews recent CI failures through gh. The alternative is a dedicated GitHub App, with a token minted per run by actions/create-github-app-token (pinned by SHA). Commits are still authored as github-actions[bot]; we could switch that to the App's bot identity if we prefer.
  2. Failure handling. Should a red or timed-out PR be closed and its branch deleted (as proposed), or converted back to a draft? Converting keeps the work, but drafts are how 52 open PRs piled up.
  3. Prompt ownership. Who owns .agents/automated-tasks/tests.md and security.md, and who reviews changes to them?

Not in this PR

How to manually test your changes?

Please don't dispatch this workflow before open question #1 is settled, because it opens real PRs and posts to Slack. I only checked the workflow statically: actionlint 1.7.12 reports nothing, the YAML parses, and every run: script passes bash -n. Once a token exists, dispatch maintenance-prs on this branch and check that:

  • each task skips while it has 2 or more open PRs;
  • a new PR is ready for review and requests @shopify/dev_experience;
  • CI starts without needing approval;
  • Slack gets a message only after the required checks pass.

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is user-facing — I've identified the correct bump type (patch for bug fixes · minor for new features · major for breaking changes) and added a changeset with pnpm changeset add

Run tests and security weekly, cap open PRs per task at 2, open ready-for-review PRs with a non-GITHUB_TOKEN token so CI runs, announce them only after required CI passes, and bump claude-code-action to v1.0.263. Remove the refactor and performance task prompts.

Co-authored-by: Claude Code <[email protected]>
@github-actions github-actions Bot added the no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. label Oct 27, 2026
The pinning test matched the 'uses:' inside 'statuses: read' and reported 'read' as an unpinned action. Match 'uses:' only at a word boundary.

Co-authored-by: Claude Code <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant