gh-156939: Fix struct.pack('0p', bytes)

[vstinner]

If the Pascal string is empty (size=0), do not write the size prefix. Previously, a NUL byte was written outsize the buffer (buffer overflow).

• Issue: C API: Detect buffer overflow in PyBytesWriter #156939

[vstinner]

In Python 3.14 and older, struct.pack('0p', b'abc') writes a NUL byte after the bytes contents. But bytes objects allocates an extra byte for a trailing NUL byte, so the buffer overflow doesn't write arbitrary memory. It's just a silent write which doesn't corrupt anything.

On Python 3.15 and newer, struct.pack() uses PyBytesWriter which doesn't allocate an extra trailing NUL byte for small strings (up to 256 bytes). And so a buffer overflow can write outsize PyBytesWriter.small_buffer. In practice, it should write a NUL byte in PyBytesWriter.obj which is a NULL pointer, so it should not corrupt arbitrary memory.

Well, it's better to avoid a buffer overflow anyway :-)

[vstinner]

Ah, I forgot to mention that the 0p format always produces an empty bytes string:

>>> struct.pack('0p', b'abc')
b''

[vstinner]

In Python 3.14 and older, struct.pack('0p', b'abc') writes a NUL byte after the bytes contents. But bytes objects allocates an extra byte for a trailing NUL byte, so the buffer overflow doesn't write arbitrary memory.

Oh, I forgot that Python 3.13 and 3.14 uses the old internal _PyBytesWriter API. struct.pack('0p', b'abc') uses a small buffer in the writer API. The small buffer is between 10 and 512 bytes, so writing a NUL byte a position 0 doesn't corrupt memory. It's just a silent (ignored) write.

[miss-islington-app]

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

[bedevere-app]

GH-157129 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-157130 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-157131 is a backport of this pull request to the 3.13 branch.

[vstinner]

If the struct.pack() output size is 512 bytes, this bug can lead to a buffer overflow writing one NUL byte into the stack memory. Example of code triggering the overflow:

import struct
size = 512
res = struct.pack(f'{size}s0p', b'x' * size, b'ignored')
print(len(res))