[gpshead]

Since gh-83065, the first import of pkg.mod kept pkg's module lock until pkg/mod.py had finished executing. This shipped in 3.15.0. Threads needing only the finished pkg were blocked for that long, and concurrent first imports could raise _DeadlockError or a spurious "partially initialized module" ImportError, or hang, with no import cycle involved.

Parent packages are now imported, or waited for, before the submodule's lock is taken, and only that lock is held while it executes. The original gh-83065 deadlock stays fixed.

importlib.import_module() now returns the partially initialized module in a cross-thread circular import, as the import statement already did, instead of raising _DeadlockError.

Two threads, step by step

The scenario from test_concurrent_first_imports_of_sibling_packages: nothing is imported yet, a/leaf.py does from b.leaf import X and b/__init__.py does import a. The dependencies are a.leaf → b.leaf → b → a, so there is no import cycle. Thread 2 starts while thread 1 is still inside a/__init__.py.

Before (3.15.0)

After (this PR)

sequenceDiagram participant T1 as Thread 1<br/>import a.leaf participant T2 as Thread 2<br/>import b.leaf Note over T1: 1. acquire locks a and a.leaf<br/>run a/__init__.py Note over T2: 2. acquire locks b and b.leaf<br/>run b/__init__.py (import a) T2-->>T1: 3. wait for lock a Note over T1: 4. a/__init__.py returns<br/>lock a is NOT released<br/>run a/leaf.py (from b.leaf import X) T1--xT2: 5. needs lock b.leaf, but T2 is waiting on T1 Note over T1: 6. _DeadlockError Loading

sequenceDiagram participant T1 as Thread 1<br/>import a.leaf participant T2 as Thread 2<br/>import b.leaf Note over T1: 1. acquire lock a<br/>run a/__init__.py Note over T2: 2. acquire lock b<br/>run b/__init__.py (import a) T2-->>T1: 3. wait for lock a Note over T1: 4. a/__init__.py returns<br/>release lock a Note over T1: 5. acquire lock a.leaf<br/>run a/leaf.py (from b.leaf import X) Note over T2: 6. a is ready, b/__init__.py returns, release lock b<br/>acquire lock b.leaf, run b/leaf.py T1-->>T2: 7. wait for lock b.leaf Note over T2: 8. release lock b.leaf<br/>import b.leaf done Note over T1: 9. X imported, release lock a.leaf<br/>import a.leaf done Loading

Before, thread 1 keeps lock a after a/__init__.py has returned, so thread 2, which needs only the finished a, stays blocked while holding the locks thread 1 needs next. After, each thread holds one module lock at a time and both imports complete. Steps 5 to 8 of the right-hand side can interleave differently (thread 1 may load b.leaf itself); either way neither thread waits on a lock while holding one the other needs.

---

authored using Claude Fable

• Issue: 3.15 regression: concurrent first imports can fail or hang without any import cycle #159963

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #35310738 | 📁 Comparing 5d2a312 against main (80040b6)

  🔍 Preview build  

45 files changed · + 1 added · ± 44 modified

+ Added

• c-api/pyexpat.html

± Modified

• c-api/code.html
• c-api/conversion.html
• c-api/dict.html
• c-api/exceptions.html
• c-api/file.html
• c-api/function.html
• c-api/hash.html
• c-api/import.html
• c-api/intro.html
• c-api/iterator.html
• and 34 more...

[gpshead]

@emmatyping volunteered as review tribute at the sprint 😅