-
-
Notifications
You must be signed in to change notification settings - Fork 32.1k
csv: Inconsistency re QUOTE_NONNUMERIC #74232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
A csv.writer with quoting=csv.QUOTE_NONNUMERIC does not quote boolean values, which makes a csv.reader with the same quoting behaviour fail on that value: -------- csv.py ---------- import csv
import io
f = io.StringIO()
writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
writer.writerow(['asdf', 1, True])
f.seek(0)
reader = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
for row in reader:
print(row) $ python3 csvbug.py
Traceback (most recent call last):
File "csvbug.py", line 12, in <module>
for row in reader:
ValueError: could not convert string to float: 'True' I'd consider this inconsistency a bug, but in any case something that needs documenting. |
boolean is not quoted since in Python it's a subclass of int so True and False are numeric. This is also the case with numeric objects defining __int__ or __float__ but doesn't get a corresponding string representation. Since QUOTE_NONNUMERIC will converts data to float when reading, I think we may force the converting even when writing so the inconsistency would disappear. Or document this limitation. |
This issue is not easy, it needs a thoughtful design before starting coding. I agree with Xiang's analysis and proposed solutions. But if just convert numbers to float we can get an overflow for large integers or lost precision in case of Decimal. The consumer of the CSV file may be not Python and it may support larger precision than Python float. And it is not clear what would be better solution for enums. Should they be serialized by name or by value? |
I would like to solve this issue. |
Oh, I read the Serhiy Storchaka 's comment just right now. |
I don't think it is reasonable to change the type system and this isn't a type bug perse, more of a difference in type expectations. I've added the note to the docs but this is my first doc fix in CPython so not sure if I did it correctly. |
Other example is Fraction. We need a parameter for the reader to control how to convert non-quoted fields to numeric. Something like We may add also add parameters for the reader to control how to convert numerics and other types to strings. |
…SV (pythonGH-134963) (cherry picked from commit e814f43) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
…SV (pythonGH-134963) (cherry picked from commit e814f43) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Uh oh!
There was an error while loading. Please reload this page.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs
The text was updated successfully, but these errors were encountered: