Skip to content

Fix data race in registering instances - #6215

Merged
rwgk merged 2 commits into
pybind:masterfrom
YukaSadaoka:fix-init-instance-race
Oct 27, 2026
Merged

rwgk merged 2 commits into
pybind:masterfrom
YukaSadaoka:fix-init-instance-race

Conversation

@YukaSadaoka

@YukaSadaoka YukaSadaoka commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Description

  • class_::init_instance() added the new instance to registered_instances before setting its status flags and constructing its holder. On free-threaded builds another thread can find the instance in the map and read those flags (e.g. simple_layout in value_and_holder) while they are still being written. TSan reports this as a data race when several threads cast the same C++ object.
  • This change only moves registration for the holders that can't run user code during the construction, to the end of both init_instance() overloads (classic and smart_holder). The instance map lock then orders every write to the instance before any reader that finds it there. There are no new locks and no layout or ABI changes.
  • For the holder type that could run user code during the construction, the data race is not fixed in this PR because this reordering caused regression (see the comment). I'll address it separately.
  • test_concurrent_cast_of_shared_cpp_object in test_thread.py covers both overloads. Under TSan on 3.14t it fails without this change and passes with it. It won't fail on a regular build, since the race only shows up under TSan.
  • There's a related but separate case that I'd like to handle in a follow-up PR. The smart_holder cast paths (smart_holder_from_unique_ptr / smart_holder_from_shared_ptr) call keep_alive_impl() after init_instance(), and that writes has_patients after the instance is already visible.

Suggested changelog entry:

  • Fixed a data race on free-threaded builds where another thread could find a newly created instance with non-custom holder types before its holder and status flags were set.

📚 Documentation preview 📚: https://pybind11--6215.org.readthedocs.build/

@rwgk rwgk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

codex GPT-6.1-Sol ultra review:

Found one P2 regression at pybind11.h:1735: custom-holder construction loses Python virtual dispatch.

An intrusive holder can invoke a virtual callback while acquiring a reference. Moving init_holder() before registration prevents get_override() from finding the Python instance, so it silently calls the C++ fallback. I reproduced this on ordinary CPython: the Python callback executes at the base revision but is skipped at the PR head. Preserve reentrant instance lookup during holder initialization and add coverage for this case.

Reproducer:

@YukaSadaoka

Copy link
Copy Markdown
Contributor Author

Thanks for catching the regression @rwgk! You are right, the reordering broke the lookups. I updated the change; the reordering only applies to holders that can't run user code during the construction. I also added the regression repro test. This PR keeps the data race with the holder that could run user code as a result. Fixing it for the type seems to require more thinking so I'd prefer to handle that in a separate PR if that's ok. Please let me know what you think!

Note the 5 failing CI jobs appear to be failing due to gitlab related issues.

@rwgk rwgk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and codex GPT-6.1-Sol ultra.

For a moment I was worried that test_holder_ctor_calls_python_override might be flaky in free-threaded builds. Codex explained that the callback completes synchronously before the assertion, and the test doesn't expose the instance to another thread during initialization. It also passed 10,000 repetitions with the GIL disabled.

@rwgk
rwgk merged commit 2a33aba into pybind:master Oct 27, 2026
206 of 217 checks passed
@github-actions github-actions Bot added the needs changelog Possibly needs a changelog entry label Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs changelog Possibly needs a changelog entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants