Skip to content

gh-160781: Fix BytesIO overflow handling for large sizes - #160783

Merged
StanFromIreland merged 3 commits into
python:mainfrom
cmaloney:bio_1
Oct 29, 2026
Merged

StanFromIreland merged 3 commits into
python:mainfrom
cmaloney:bio_1

Conversation

@cmaloney

@cmaloney cmaloney commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

BytesIO.write() at a position near sys.maxsize raised SystemError ("Negative size passed to PyBytes_FromStringAndSize") because the overallocated buffer size could exceed PY_SSIZE_T_MAX. Check the computed allocation, not only the requested size.

BytesIO.truncate() converted its argument with PyLong_AsLong(), which rejects sizes above 2**31 - 1 on Windows where C long is 32-bit. Use PyNumber_AsSsize_t() like the Py_ssize_t Argument Clinic converter it used before.

BytesIO.write() at a position near sys.maxsize raised SystemError
("Negative size passed to PyBytes_FromStringAndSize") because the
overallocated buffer size could exceed PY_SSIZE_T_MAX.  Check the
computed allocation, not only the requested size.

BytesIO.truncate() converted its argument with PyLong_AsLong(), which
rejects sizes above 2**31 - 1 on Windows where C long is 32-bit.  Use
PyNumber_AsSsize_t() like the Py_ssize_t Argument Clinic converter it
used before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@cmaloney cmaloney added 3.14 bugs and security fixes 3.15 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes and removed 3.14 bugs and security fixes 3.15 bugs and security fixes labels Oct 27, 2026
@@ -0,0 +1,2 @@
Fix :class:`io.BytesIO` implementation of :meth:`io.IOBase.truncate` rejecting

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future, it's fine to handle this with StringIO and put them in the same news entry.

@StanFromIreland
StanFromIreland merged commit 7b727ab into python:main Oct 29, 2026
60 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @cmaloney for the PR, and @StanFromIreland for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Oct 29, 2026

Copy link
Copy Markdown

GH-160891 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 29, 2026
@bedevere-app

bedevere-app Bot commented Oct 29, 2026

Copy link
Copy Markdown

GH-160892 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Oct 29, 2026
StanFromIreland pushed a commit that referenced this pull request Oct 29, 2026
@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86 Debian Installed with X 3.14 (tierless) has failed when building commit daccca0.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1793/builds/2045) and take a look at the build logs.
  4. Check if the failure is related to this commit (daccca0) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1793/builds/2045

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File �[35m"/buildbot/buildarea/3.14.ware-debian-x86.installed/build/target/lib/python3.14/test/test_memoryio.py"�[0m, line �[35m571�[0m, in �[35mtest_issue141311�[0m
    self.assertEqual(�[31mmemio.getvalue�[0m�[1;31m()�[0m, b"")
                     �[31m~~~~~~~~~~~~~~�[0m�[1;31m^^�[0m
  File �[35m"/buildbot/buildarea/3.14.ware-debian-x86.installed/build/target/lib/python3.14/_pyio.py"�[0m, line �[35m901�[0m, in �[35mgetvalue�[0m
    return bytes(self._buffer)
�[1;35mOverflowError�[0m: �[35mbyte string is too large�[0m

@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM Raspbian 3.14 (tier-3) has failed when building commit daccca0.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1689/builds/2102) and take a look at the build logs.
  4. Check if the failure is related to this commit (daccca0) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1689/builds/2102

Failed tests:

  • test_memoryio

Failed subtests:

  • test_issue141311 - test.test_memoryio.PyBytesIOTest.test_issue141311

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.14.gps-raspbian.nondebug/build/Lib/test/test_memoryio.py", line 571, in test_issue141311
    self.assertEqual(memio.getvalue(), b"")
                     ~~~~~~~~~~~~~~^^
  File "/var/lib/buildbot/workers/3.14.gps-raspbian.nondebug/build/Lib/_pyio.py", line 901, in getvalue
    return bytes(self._buffer)
OverflowError: byte string is too large

@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86 Debian Non-Debug with X 3.14 (tierless) has failed when building commit daccca0.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1687/builds/1999) and take a look at the build logs.
  4. Check if the failure is related to this commit (daccca0) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1687/builds/1999

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File �[35m"/buildbot/buildarea/3.14.ware-debian-x86.nondebug/build/Lib/test/test_memoryio.py"�[0m, line �[35m571�[0m, in �[35mtest_issue141311�[0m
    self.assertEqual(�[31mmemio.getvalue�[0m�[1;31m()�[0m, b"")
                     �[31m~~~~~~~~~~~~~~�[0m�[1;31m^^�[0m
  File �[35m"/buildbot/buildarea/3.14.ware-debian-x86.nondebug/build/Lib/_pyio.py"�[0m, line �[35m901�[0m, in �[35mgetvalue�[0m
    return bytes(self._buffer)
�[1;35mOverflowError�[0m: �[35mbyte string is too large�[0m

@StanFromIreland

Copy link
Copy Markdown
Member

Fix for the test in #160903.

@cmaloney
cmaloney deleted the bio_1 branch October 29, 2026 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants