gh-129409: Fix Integer overflow - SEGV while writing data more than 2GB in CSV file #129413
Conversation
srinivasreddy
commented
Jan 28, 2025
•
- Issue: CSV write has SEGV when trying to write data 2GB or larger #129409
Lib/test/test_csv.py
Outdated
| @@ -1560,6 +1560,20 @@ def test_ordered_dict_reader(self): | |||
| OrderedDict([('fname', 'John'), ('lname', 'Cleese')]), | |||
| ]) | |||
|
|
|||
| class HugeDataTest(unittest.TestCase): | |||
| def test_write_huge_data(self): | |||
There was a problem hiding this comment.
This test should probably be guarded with support.requires("largefile", ...) so that it only runs when the resource is enabled via python -m test -u largefile
There was a problem hiding this comment.
I would prefer not to test this, 2GB files are too much for this kind of change.
There was a problem hiding this comment.
Not disagreeing, but would you mind elaborating on why we shouldn't test this, just for future reference? 🙂
There was a problem hiding this comment.
Tests are rather hard to reason about. I would put it this way: since it is a very slow and heavy test that tests an integer overflow. It is not specific to any arch / os / runtime / etc.
There was a problem hiding this comment.
Makes sense, thanks for the explanation :)
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
| Fixed a segmentation fault in the :mod:`csv` module when writing data fields | ||
| larger than 2GB due to integer overflow. |
There was a problem hiding this comment.
A bit more concise:
| Fixed a segmentation fault in the :mod:`csv` module when writing data fields | |
| larger than 2GB due to integer overflow. | |
| Fix an integer overflow in the :mod:`csv` module when writing data | |
| larger than 2GB. |
|
Thanks @srinivasreddy for the PR, and @sobolevn for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13. |
…than 2GB in CSV file (pythonGH-129413) (cherry picked from commit 97b0ef0) Co-authored-by: Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్ రెడ్డి) <thatiparthysreenivas@gmail.com>
|
GH-129436 is a backport of this pull request to the 3.13 branch. |
…than 2GB in CSV file (pythonGH-129413) (cherry picked from commit 97b0ef0) Co-authored-by: Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్ రెడ్డి) <thatiparthysreenivas@gmail.com>
|
GH-129437 is a backport of this pull request to the 3.12 branch. |