Skip to content

fix(graphql-server): drop a queued request whose client hung up instead of leaking its concurrency slot - #1924

Open
pyramation wants to merge 1 commit into
mainfrom
fix/admission-queued-abort-leak
Open

pyramation wants to merge 1 commit into
mainfrom
fix/admission-queued-abort-leak

Conversation

@pyramation

Copy link
Copy Markdown
Contributor

Summary

Bug. In createAdmissionControlMiddleware, the res.on('close') / res.on('finish') listeners that release a concurrency slot were attached only after await concurrency.acquire(...) resolved. acquire can wait up to maxQueueWaitMs in the queue, so a client that hung up while queued emitted close before any listener existed. When its turn came the dead request was handed the slot, next() ran it against a closed socket, and nothing ever called lease.release() — the per-database budget shrank by one for the life of the process. Reachable anonymously: a few hundred connect-queue-abort cycles starve the endpoint.

Fix.

// packages/express-context/src/admission.ts
interface AdmissionRequest { …; signal?: AbortSignal }          // new
type AdmissionRefusal = 'queue_full' | 'queue_timeout' | 'aborted' // new member
// acquire(): an aborted waiter leaves the queue immediately and resolves
// REFUSED('aborted'); already-aborted callers are refused without queueing.

// graphql/server/src/middleware/admission-control.ts
const gone = new AbortController();
res.once('close', () => gone.abort());            // watched from BEFORE the wait
lease = await concurrency.acquire(databaseId, { …, signal: gone.signal });
res.off('close', …);
if (gone.signal.aborted) { lease.release(); return; }  // grant that raced the hang-up

The existing post-grant close/finish release listeners are unchanged; release stays idempotent.

Tests

  • admission-control.test.ts — does not leak the slot when the client hangs up while queued: slot held, second request queued, queued response emits close, holder finishes; the queued request must not start and a third caller (maxQueueWaitMs: 0) must be admitted immediately.
  • admission.test.ts (ConcurrencyLimiter) — an aborted waiter leaves the queue with refusal 'aborted' and the next arrival gets the slot; a pre-aborted signal is refused without queueing.

Before (main + these tests):

✕ does not leak the slot when the client hangs up while queued
    Expected number of calls: 0   // queued.next — the dead request was run
    Received number of calls: 1
Tests: 1 failed
packages/express-context admission.test.ts: does not compile (`signal` is not an AdmissionRequest field)

After: admission-control.test.ts 18 passed, admission.test.ts 28 passed; eslint clean on the four touched files.

Link to Devin session: https://app.devin.ai/sessions/959661f0d69640789bd6b38550d379b7
Open in Devin Desktop: https://app.devin.ai/desktop/session/959661f0d69640789bd6b38550d379b7?variant=devin
Requested by: @pyramation

…ad of leaking its concurrency slot

The release listeners on the response were attached only after
ConcurrencyLimiter.acquire() resolved, so a client that closed the
connection while still waiting in the queue was missed: when its turn came
it was handed the slot, nothing ever released it, and the per-database
budget shrank by one for the life of the process.

Watch 'close' from before the wait and pass an AbortSignal into acquire():
an aborted waiter leaves the queue at once (refusal 'aborted') and a grant
that races the hang-up is released immediately instead of running the
request.
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@tenki-reviewer

tenki-reviewer Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Review complete. No issues found — approved ✅.


This PR wires request abort signals into the admission control path: the middleware listens for connection close and aborts queued waiters, and the shared ConcurrencyLimiter learns to settle aborted waiters and evict idle tenant state. Tests cover pre-abort, mid-queue abort, and granted-then-aborted paths.

Files Change
packages/express-context/src/admission.ts Limiter support for abort-driven settling, waiter removal, and idle-state eviction.
graphql/server/src/middleware/admission-control.ts Middleware attaches close/abort handling around acquire/release with listener cleanup.
packages/express-context/__tests__/admission.test.ts Limiter unit tests for abort and eviction semantics.
graphql/server/src/middleware/__tests__/admission-control.test.ts Middleware tests for hang-up during queue wait and after grant.

Reviewed commit: 9648a44

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant