gh-156939: Fix memory corruption check in fcntl

[vstinner]

For "large" buffer using PyBytesWriter, copy guard bytes rather than just checking the last trailing byte.

• Issue: C API: Detect buffer overflow in PyBytesWriter #156939

[vstinner]

Python 3.13 and 3.14 are not affected. They fail with an error if the buffer size is larger than 1024 bytes.

[vstinner]

With this change, I'm not sure that it's still needed to have a "fast path" (is it really faster?) for buffer smaller than or equal to 1024 bytes (the code path which doesn't use PyBytesWriter).

            // Truncate the trailing guard bytes
            return PyBytesWriter_FinishWithSize(writer, len);

The updated code now allocates extra 8 bytes and then resize the bytes object to truncate the last 8 bytes. It's less efficient than the previous code which relies on the fact that bytes objects always end with a trailing NUL byte. IMO it's better to have better check for memory overflow, than paying attention of the performance of such short operation.

[vstinner]

Ubuntu (installed) / build, install and test

On a CI building Python in release mode, test_ioctl does crash :-(

0:04:01 load avg: 4.14 mem: 49.8 MiB [1/2/1] test_ioctl worker non-zero exit code (Exit code -6 (SIGABRT))
Re-running test_ioctl in verbose mode
test_bad_fd (test.test_ioctl.IoctlTestsPty.test_bad_fd) ... *** buffer overflow detected ***: terminated
Fatal Python error: Aborted

[vstinner]

On a CI building Python in release mode, test_ioctl does crash :-(

Ah, I just found and fixed a typo.

[vstinner]

Currently, the code is correct. It relies on the fact that bytes objects always end with a NUL byte.

This change is mostly needed if we decide to implement the check to detect buffer overflow in PyBytesWriter: PR gh-156943.