Skip to content

csv: Inconsistency re QUOTE_NONNUMERIC #74232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tlotze mannequin opened this issue Apr 11, 2017 · 7 comments
Open

csv: Inconsistency re QUOTE_NONNUMERIC #74232

tlotze mannequin opened this issue Apr 11, 2017 · 7 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@tlotze
Copy link
Mannequin

tlotze mannequin commented Apr 11, 2017

BPO 30046
Nosy @serhiy-storchaka, @zhangyangyu, @tlotze, @corona10

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-04-11.20:41:55.773>
labels = ['type-bug', 'library']
title = 'csv: Inconsistency re QUOTE_NONNUMERIC'
updated_at = <Date 2017-04-19.07:11:22.021>
user = 'https://github.com/tlotze'

bugs.python.org fields:

activity = <Date 2017-04-19.07:11:22.021>
actor = 'corona10'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-04-11.20:41:55.773>
creator = 'tlotze'
dependencies = []
files = []
hgrepos = []
issue_num = 30046
keywords = []
message_count = 5.0
messages = ['291516', '291590', '291862', '291864', '291865']
nosy_count = 4.0
nosy_names = ['serhiy.storchaka', 'xiang.zhang', 'tlotze', 'corona10']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30046'
versions = []

Linked PRs

@tlotze
Copy link
Mannequin Author

tlotze mannequin commented Apr 11, 2017

A csv.writer with quoting=csv.QUOTE_NONNUMERIC does not quote boolean values, which makes a csv.reader with the same quoting behaviour fail on that value:

-------- csv.py ----------

import csv
import io


f = io.StringIO()

writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
writer.writerow(['asdf', 1, True])

f.seek(0)
reader = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
for row in reader:
    print(row)

$ python3 csvbug.py 
Traceback (most recent call last):
  File "csvbug.py", line 12, in <module>
    for row in reader:
ValueError: could not convert string to float: 'True'

I'd consider this inconsistency a bug, but in any case something that needs documenting.

@tlotze tlotze mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Apr 11, 2017
@zhangyangyu
Copy link
Member

boolean is not quoted since in Python it's a subclass of int so True and False are numeric. This is also the case with numeric objects defining __int__ or __float__ but doesn't get a corresponding string representation.

Since QUOTE_NONNUMERIC will converts data to float when reading, I think we may force the converting even when writing so the inconsistency would disappear. Or document this limitation.

@serhiy-storchaka
Copy link
Member

This issue is not easy, it needs a thoughtful design before starting coding. I agree with Xiang's analysis and proposed solutions. But if just convert numbers to float we can get an overflow for large integers or lost precision in case of Decimal. The consumer of the CSV file may be not Python and it may support larger precision than Python float.

And it is not clear what would be better solution for enums. Should they be serialized by name or by value?

@corona10
Copy link
Member

I would like to solve this issue.
Is there any other way than casting specifically for the bool object?
(e.g For general ways?)

@corona10
Copy link
Member

Oh, I read the Serhiy Storchaka 's comment just right now.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@aterrel
Copy link

aterrel commented May 21, 2025

I don't think it is reasonable to change the type system and this isn't a type bug perse, more of a difference in type expectations.

I've added the note to the docs but this is my first doc fix in CPython so not sure if I did it correctly.

@serhiy-storchaka
Copy link
Member

Other example is Fraction. 1/2 cannot be converted to float.

We need a parameter for the reader to control how to convert non-quoted fields to numeric. Something like parse_float in json, but it will work not only with floats.

We may add also add parameters for the reader to control how to convert numerics and other types to strings.

miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jun 2, 2025
…SV (pythonGH-134963)

(cherry picked from commit e814f43)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jun 2, 2025
…SV (pythonGH-134963)

(cherry picked from commit e814f43)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit that referenced this issue Jun 2, 2025
…CSV (GH-134963) (GH-135049)

(cherry picked from commit e814f43)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit that referenced this issue Jun 2, 2025
…CSV (GH-134963) (GH-135048)

(cherry picked from commit e814f43)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Status: No status
Development

No branches or pull requests

4 participants