Pickler can call a freed reducer after dispatch-table replacement

[Nievesjyl]

Description

When no private dispatch table is configured, _pickle retrieves a borrowed reducer from the public copyreg.dispatch_table and increments it separately. Concurrent replacement can remove the dictionary's last reference to that reducer between lookup and ownership promotion.

Observed Behavior

On a free-threaded build, equivalent reducer replacements race with pickle.dumps() and reproducibly terminate with SIGSEGV in under 0.30 seconds. The same executable completes a ten-second compatibility-GIL control normally.

Affected Version

CPython 3.14.7 at commit 823f0323ee6ec1402088b73bce1a38473cac36dc, using the free-threaded build with PYTHON_GIL=0.

Reproduction

Run:

PYTHON_GIL=0 python3.14t poc/reproduce.py 10

PoC Source Code

poc/reproduce.py:

#!/usr/bin/env python3
"""Race copyreg reducer replacement with C Pickler dispatch lookup."""

import copyreg
import pickle
import sys
import threading
import time


class Target:
    pass


class Reducer:
    def __call__(self, obj):
        return Target, ()


copyreg.dispatch_table[Target] = Reducer()
STOP = threading.Event()
FAILURES = []


def mutate():
    while not STOP.is_set():
        copyreg.dispatch_table[Target] = Reducer()


def pickle_worker(deadline):
    count = 0
    try:
        while time.monotonic() < deadline and not STOP.is_set():
            pickle.dumps(Target(), protocol=4)
            count += 1
    except BaseException as exc:
        FAILURES.append((type(exc).__name__, repr(exc), count))
        STOP.set()


def main():
    seconds = float(sys.argv[1]) if len(sys.argv) > 1 else 10.0
    deadline = time.monotonic() + seconds
    threads = [threading.Thread(target=mutate) for _ in range(2)]
    threads += [threading.Thread(target=pickle_worker, args=(deadline,)) for _ in range(12)]
    for thread in threads:
        thread.start()
    for thread in threads[2:]:
        thread.join()
    STOP.set()
    for thread in threads[:2]:
        thread.join()
    print("gil=", sys._is_gil_enabled(), "failures=", FAILURES)
    return bool(FAILURES)


raise SystemExit(main())

[ZeroIntensity]

Refer to #157124 (comment)