Element structural writers can corrupt child storage during mutation

[Nievesjyl]

Description

An _elementtree.Element stores extra, child length, capacity, and the child array as one mutable representation, but its structural writers do not serialize their transitions. insert(), remove(), item or slice assignment, and __setstate__() retain raw destinations or old children across stores, moves, resizing, and decrements. Concurrent clear() and refill can free or replace the storage those operations are using, causing stale writes, invalid decrements, and inconsistent ownership.

Observed Behavior

All five reproducer modes failed on the free-threaded ASan build. insert and remove reached invalid child decrements in dealloc_extra; setitem failed while decrementing in element_setitem; slice triggered a mimalloc heap assertion; and setstate reached an invalid decrement in element_setstate_from_attributes. All modes completed with the same executable under the GIL.

Affected Version

CPython 3.14.7 at commit 823f0323ee6ec1402088b73bce1a38473cac36dc, built with --disable-gil --with-address-sanitizer.

Reproduction

ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 PYTHON_GIL=0 python3.14 poc/reproduce.py --mode setitem --seconds 8

Valid modes are insert, remove, setitem, slice, and setstate. Setting PYTHON_GIL=1 provides the GIL-enabled control.

PoC Source Code

poc/reproduce.py

#!/usr/bin/env python3
"""Stress Element structural writers against clear/refill."""

import argparse
import threading
import time
import xml.etree.ElementTree as ET


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--seconds", type=float, default=8.0)
    parser.add_argument("--writers", type=int, default=6)
    parser.add_argument(
        "--mode",
        choices=("insert", "remove", "setitem", "slice", "setstate"),
        default="insert",
    )
    args = parser.parse_args()

    root = ET.Element("root")
    victim = ET.Element("victim")
    root.extend([victim] * 128)
    replacement = [ET.Element("new") for _ in range(128)]
    state = {
        "tag": "root",
        "_children": replacement,
        "attrib": {},
        "text": None,
        "tail": None,
    }
    stop = threading.Event()
    counts = [0] * (args.writers + 1)

    def mutate() -> None:
        while not stop.is_set():
            root.clear()
            root.extend([victim] * 128)
            counts[-1] += 1

    def write(slot: int) -> None:
        while not stop.is_set():
            try:
                if args.mode == "insert":
                    root.insert(0, victim)
                elif args.mode == "remove":
                    root.remove(victim)
                elif args.mode == "setitem":
                    if len(root):
                        root[0] = victim
                elif args.mode == "slice":
                    root[:] = replacement
                else:
                    root.__setstate__(state)
            except (IndexError, ValueError):
                pass
            counts[slot] += 1

    threads = [threading.Thread(target=mutate)]
    threads.extend(
        threading.Thread(target=write, args=(i,)) for i in range(args.writers)
    )
    for thread in threads:
        thread.start()
    time.sleep(args.seconds)
    stop.set()
    for thread in threads:
        thread.join()
    print(
        {"mode": args.mode, "writes": sum(counts[:-1]), "mutations": counts[-1]}
    )


if __name__ == "__main__":
    main()

[ZeroIntensity]

Refer to #157124 (comment)