Skip to content

pstack orch: stale-lock takeover can be lost (unlink-then-create TOCTOU); frontier SHAs use local branches #730

Open

Description

@BowmanStephen

Summary

acquireLock in the pstack orchestrator store takes over a stale lock by unlink-then-create, which has a TOCTOU window: two processes that both see the same dead holder can end up both holding the lock.

  • Version: pstack 0.15.28 (harness/pstack/claude-code)
  • File: skills/poteto-mode/scripts/orch/store.ts, acquireLock (~L376–405)
const takeOver = async (): Promise<void> => {
  await unlink(path);          // (1) both processes can reach here
  try {
    await create();            // (2) one wins; the loser's unlink may land later
  } catch (retryError) { ... }
};

Interleaving that loses mutual exclusion

step P1 P2
1 reads holder, decides it is dead
2 reads holder, decides it is dead
3 unlink(path)
4 create() → holds lock
5 unlink(path) → deletes P1's fresh lock
6 create() → also "holds" lock

Both processes then proceed to write the store, so one writer's update is lost.

Suggested fix

Make takeover atomic rather than unlink-then-create. Either:

  1. Lock directory — mkdir the lock path (atomic; EEXIST is the contention signal) and rmdir on release. Changes the lock artifact from a file to a directory, and the reader of holderIsDead needs to read the pid from a file inside it.
  2. Rename-based takeover — rename(path, path + ".stale." + pid): only one process can win the rename; then read the renamed file to report the previous holder, and create() the new lock. On EEXIST, report the current holder.

A two-process test (spawn two children that both observe a dead pid and assert exactly one acquireLock resolves) would pin the behaviour.

Related finding in the same file: frontier SHAs come from local branches

skills/poteto-mode/scripts/orch/store.ts (~L1229–1235):

return chain.map((row) => ({
  pr: row.pr,
  branches: row.branch,
  sha: branchSha({ branch: row.branch, repo }),   // local branch, not PR head
  state: row.state,
}));

The frontier's verification identity is taken from the local branch, while the thing being verified is the PR head. If a local branch is stale or missing, the recorded SHA silently describes a commit that is not under review — a false verification identity rather than an error.

Suggested fix: source the SHA from the PR head OID already present in the snapshot, and represent "no local branch" as an explicit unknown instead of substituting a branch tip.


Happy to send PRs for either; filing first in case the lock artifact shape (file vs directory) is a compatibility constraint I cannot see.

Activity

  1. deanrie commented on Oct 20, 2026

    @deanrie

    Hey @BowmanStephen, confirmed on main: create() is an atomic wx open, but takeOver() is unlink then create, so two processes that both read the same dead holder can interleave exactly as in your table, and the second unlink removes the first process's fresh lock. The existing test only covers one taker. Tracked internally with a fix sketch: serialize the takeover with a second wx file (<lock>.takeover), so the loser falls back to create(), hits EEXIST, re-reads the holder and gets held by pid <winner>, plus a two-taker test. PR to follow unless the pstack owners prefer the rename-and-verify variant.

  2. deanrie commented on Oct 28, 2026

    @deanrie

    This can be closed: #1043 removed scripts/orch/ entirely, so the lock and its takeover path are gone from main. #733 is closed for the same reason. The same race was re-reported against the pre-#1043 commit as #1055.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions