Skip to content

Enforce Repo Assist output consistency before applying safe outputs - #406

Closed
dsyme wants to merge 1 commit into
mainfrom
fix/repo-assist-output-consistency
Closed

dsyme wants to merge 1 commit into
mainfrom
fix/repo-assist-output-consistency

Conversation

@dsyme

@dsyme dsyme commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add deterministic safe-output validation so a green run cannot silently deliver contradictory noop/mutation output or omit required monthly reporting after substantive non-command work.

The fleet audit found fsprojects/Paket labelling an issue then emitting noop, and fsprojects/FSharpx.Collections posting a substantive comment without its monthly update.

  • Validate the output batch before any safe-output writes.
  • Reject noop together with labels, comments, issue/PR changes, branch pushes, or report_incomplete—including in command mode.
  • Require substantive non-command work to create/update the current month's activity issue with this run's linked heading inside its Run History section; closing an old tracker or updating an unrelated issue does not qualify.
  • Preserve genuine noops, memory-only checks and command-mode reporting exemptions.
  • Permit truthful report_incomplete outcomes rather than requiring a success-shaped noop for partial work. Exempt runs do not depend on a monthly-issue API lookup; required lookups fail explicitly.
  • Clarify requested outputs versus downstream delivery, and validated memory versus the later memory commit. Do not claim full completion when obligations remain.
  • Add 20 dependency-free tests, including the observed audit regressions, and run them in CI.

This branch is independently based on main, not stacked on the separate PR-cap/maintenance change. No memory migration or schema redesign is included.

Validation

  • python3 -m unittest discover -s tests -p 'test_repo_assist_*.py' -v: 20 passed.
  • gh aw compile workflows/repo-assist.md --approve: passed with gh-aw v0.93.3; freshness reports Yes.
  • Shellcheck passed for the added run script.
  • Checked generated YAML: agent remains read-only; validation runs before process_safe_outputs without continue-on-error.
  • git diff --check: passed.
  • No agent workflow was manually dispatched; actual hosted agent execution remains untested. Hosted regression-test results are below.

The compiler reports ten warnings in its generated scaffolding (including undeclared admission outputs and persisted-checkout credentials). They are not suppressed here. Sample workflows/*.lock.yml files are intentionally ignored in this repository, so the regenerated lock is validated locally rather than force-added. Shared action-cache churn was excluded.

Compiler compatibility / security review

Compilation with the current release required removing the unsupported native Copilot web-fetch entry and normalizing request_review to request-review. Existing allowed networking and shell tools remain available; no agent write permission is added.

The new authored action use is actions/download-artifact@v8, resolved by compilation to 9000827ccba6bdab643e8b6fd33ac0654aef8333 (v8.0.2), whose tag was verified. The generated setup pin is gh-aw-actions v0.93.3 (e89a5670dc4c790b37ade361215b8d72d8ce8942), also verified. No new secrets or redirect changes. The existing GitHub token is used only for read-only API calls by this validator. Artifact JSON and issue titles are parsed as data, never executed. The memory validator is unchanged; the current compiler's newly recorded validator approval was reviewed.

Hosted CI result

CI run 40114442057 ran all 20 new regression tests successfully. The subsequent repository-wide compile failed on unchanged .github/workflows/daily-repo-goals.md and .github/workflows/link-checker.md, whose native tools.web-fetch entries are rejected by the latest compiler. Neither file is modified by this PR. The updated Repo Assist template compiles successfully in focused local validation. This unrelated compatibility upgrade is intentionally not bundled here.

Companion independent PR: #407.

Reject noop plus mutations before delivery and require substantive non-command work to include a current-run monthly report, while preserving truthful incomplete runs and reporting exemptions.

Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant