Element child readers can use freed storage during concurrent mutation

[Nievesjyl]

Description

Multiple _elementtree.Element read APIs separately access extra, child length, and the child array before promoting a raw child pointer. Concurrent clear() can detach and free ElementObjectExtra and its heap storage, while refill publishes a different array and length. A reader can consequently load through stale representation state or increment a reclaimed child. Affected paths include find/findtext/findall, integer and slice subscription, shallow/deep copy, __getstate__, and tree iteration.

Observed Behavior

All seven independently selected modes in the reproducer (find, getitem, slice, copy, deepcopy, getstate, and iter) terminated with an ASan SEGV in their corresponding Element read path. The same executable completed every mode with the GIL enabled.

Affected Version

CPython 3.14.7 at commit 823f0323ee6ec1402088b73bce1a38473cac36dc, built with --disable-gil --with-address-sanitizer.

Reproduction

ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 PYTHON_GIL=0 python3.14 poc/reproduce.py --mode find --seconds 10

Valid modes are find, getitem, slice, copy, deepcopy, getstate, and iter. The schedule is probabilistic, so repeat a clean free-threaded run. Setting PYTHON_GIL=1 provides the GIL-enabled control.

PoC Source Code

poc/reproduce.py

#!/usr/bin/env python3
"""Stress Element child readers against clear/refill on one shared Element."""

import argparse
import copy
import threading
import time
import xml.etree.ElementTree as ET


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--seconds", type=float, default=10.0)
    parser.add_argument("--readers", type=int, default=8)
    parser.add_argument(
        "--mode",
        choices=(
            "find", "getitem", "slice", "copy", "deepcopy", "getstate", "iter"
        ),
        default="find",
    )
    args = parser.parse_args()

    root = ET.Element("root")
    root.extend(ET.Element("needle", value=str(i)) for i in range(128))
    stop = threading.Event()
    counts = [0] * (args.readers + 1)

    def mutate() -> None:
        generation = 0
        while not stop.is_set():
            root.clear()
            root.extend(
                ET.Element("needle", value=f"{generation}:{i}")
                for i in range(128)
            )
            generation += 1
            counts[-1] += 1

    def read(slot: int) -> None:
        while not stop.is_set():
            try:
                if args.mode == "find":
                    root.find("needle")
                    root.findtext("needle")
                    root.findall("needle")
                elif args.mode == "getitem":
                    if len(root):
                        root[0]
                elif args.mode == "slice":
                    root[:]
                elif args.mode == "copy":
                    copy.copy(root)
                elif args.mode == "deepcopy":
                    copy.deepcopy(root)
                elif args.mode == "getstate":
                    root.__getstate__()
                else:
                    list(root.iter())
            except (IndexError, RuntimeError):
                pass
            counts[slot] += 1

    workers = [threading.Thread(target=mutate)]
    workers.extend(
        threading.Thread(target=read, args=(i,)) for i in range(args.readers)
    )
    for worker in workers:
        worker.start()
    time.sleep(args.seconds)
    stop.set()
    for worker in workers:
        worker.join()
    print(
        {"mode": args.mode, "reads": sum(counts[:-1]), "mutations": counts[-1]}
    )


if __name__ == "__main__":
    main()

[ZeroIntensity]

Refer to #157124 (comment)