Skip to content

fix: preserve native cleanup callbacks during interpreter shutdown - #2397

Open
emme1t wants to merge 2 commits into
abetlen:mainfrom
emme1t:fix/shutdown-cleanup-1998
Open

emme1t wants to merge 2 commits into
abetlen:mainfrom
emme1t:fix/shutdown-cleanup-1998

Conversation

@emme1t

@emme1t emme1t commented Nov 1, 2026

Copy link
Copy Markdown

Model finalizers can run after Python clears module globals, causing the cleanup callbacks to raise TypeError: 'NoneType' object is not callable or AttributeError while releasing native resources.

Capture the native release functions when registering the ExitStack callbacks for models, contexts, batches, samplers (including clones/custom samplers), and LoRA adapters. The callbacks retain their existing release order and idempotent close behavior through module teardown.

Fixes #1998.
Fixes #2002.

Validation on Windows CPU, Python 3.12.13, with the native library built from the pinned llama.cpp revision 0c1e57098 using LLVM-MinGW/Ninja (GGML_NATIVE=OFF, GGML_OPENMP=OFF, _WIN32_WINNT=0x0A00):

  • Full python -m pytest tests -q: 32 passed, including the Qwen3.5, Mamba, Falcon-H1, tiny Llama, and embedding-model tests.
  • Six new subprocess cases cover explicit/repeated close and garbage collection with intact globals, cleared function bindings, and cleared module references. They verify exactly-once release, context-before-model ordering, and reverse removal of custom samplers. Baseline: 4 failed / 2 passed; fixed: 6 passed. Model/context/batch/sampler resources are real native allocations; the optional LoRA allocation/release is stubbed.
  • Real stories260K.gguf inference generated eight tokens before explicit close and garbage collection with the bindings module reference cleared; both completed without unraisable exceptions.
  • Repository-wide Ruff lint and formatting checks passed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant