marshal.dumps() can use invalid entries during concurrent container mutation

[Nievesjyl]

Description

marshal.dumps() traverses exact lists and dictionaries with unchecked borrowed access, without holding the container's critical section or first creating an owned snapshot. Concurrent clear, resize, or replacement can invalidate list slots, dictionary positions, keys, or values before recursive serialization.

Observed Behavior

On a free-threaded ASan build, the dictionary reproducer crashes after about 0.25 seconds through a stale value used at Python/marshal.c:614; the list reproducer crashes after about 0.22 seconds at Python/marshal.c:603. Both modes complete normally with the compatibility GIL enabled.

Affected Version

CPython 3.14.7 at commit 823f0323ee6ec1402088b73bce1a38473cac36dc, built with ASan and free-threading support.

Reproduction

Run both modes:

ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 PYTHON_GIL=0 python3.14 poc/reproduce.py 10 dict
ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 PYTHON_GIL=0 python3.14 poc/reproduce.py 10 list

For comparison, run the same commands with PYTHON_GIL=1.

PoC Source Code

poc/reproduce.py:

import marshal
import sys
import threading
import time


duration = float(sys.argv[1]) if len(sys.argv) > 1 else 10.0
mode = sys.argv[2] if len(sys.argv) > 2 else "dict"
stop = threading.Event()

if mode == "dict":
    original = {f"key_{i:04d}": f"value_{i:04d}" for i in range(512)}
    shared = dict(original)

    def mutate():
        while not stop.is_set():
            shared.clear()
            shared.update({f"key_{i:04d}": f"value_{i:04d}" for i in range(512)})
            shared.update(original)
elif mode == "list":
    original = [f"value_{i:04d}" for i in range(512)]
    shared = list(original)

    def mutate():
        while not stop.is_set():
            shared.clear()
            shared.extend(f"value_{i:04d}" for i in range(512))
            shared[:] = original
else:
    raise SystemExit("mode must be dict or list")


def exercise():
    while not stop.is_set():
        try:
            marshal.dumps(shared)
        except (TypeError, ValueError):
            pass


threads = [threading.Thread(target=mutate)]
threads.extend(threading.Thread(target=exercise) for _ in range(3))
for thread in threads:
    thread.start()
time.sleep(duration)
stop.set()
for thread in threads:
    thread.join()
print("completed", mode)

[ZeroIntensity]

Refer to #157124 (comment)