Skip to content

gh-143620: Fix race condition in test_traceback_when_child_process_terminates_abruptly#143621

Open
a12k wants to merge 1 commit into
python:mainfrom
a12k:flaky-process-pool-test
Open

gh-143620: Fix race condition in test_traceback_when_child_process_terminates_abruptly#143621
a12k wants to merge 1 commit into
python:mainfrom
a12k:flaky-process-pool-test

Conversation

@a12k

@a12k a12k commented Jan 9, 2026

Copy link
Copy Markdown
Contributor

Fixes gh-143620

There is a race condition in test_concurrent_futures.test_process_pool. The test asserts that a BrokenProcessPool exception has a __cause__ immediately after future.result() raises.

In concurrent.futures.process, the _terminate_broken method sets the exception on the future before it attaches the _RemoteTraceback to the __cause__ attr. Sometimes the test thread wakes up and inspects the exception while __cause__ is still None, which leads to a test failure. I was able to repro (see issue for more details) by injecting a delay in _terminate_broken between the set_exception() call and the __cause__ assignment. This causes a 100% failure rate with AssertionError: None is not an instance of _RemoteTraceback.

My proposed fix is to update the test to use support.sleeping_retry to wait for the __cause__ to be populated. This seems to be the pattern elsewhere in the test suite.

@WYSIATI

WYSIATI commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

I think the race condition analysis here may be slightly off. Looking at _terminate_broken(), bpe.__cause__ is assigned on line 493 before set_exception() on line 498, so the test thread shouldn't be able to see __cause__ as None due to ordering.

The actual issue seems to be that when cause is None (the os._exit path), p.exitcode can still return None at line 487 because waitpid() hasn't collected the exit status yet — even though the process sentinel has already fired. This means errors stays empty, cause_str stays None, and bpe.__cause__ is never set at all.

A sleeping_retry in the test won't help here because nobody goes back to set __cause__ after the fact — once _terminate_broken() finishes, the exception is final.

A fix in _terminate_broken() itself might work better — calling p.join(timeout=3) before reading p.exitcode would ensure waitpid() has collected the exit status:

for p in self.processes.values():
    p.join(timeout=3)
    if p.exitcode is not None and p.exitcode != 0:
        errors.append(...)

The sentinel has already fired at this point so join() should return almost immediately. This way p.exitcode is reliably available when we read it.

@github-actions

github-actions Bot commented May 4, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review stale Stale PR or inactive for long period of time. tests Tests in the Lib/test dir

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition in test_traceback_when_child_process_terminates_abruptly

2 participants