Skip to content

escapechar=',' Causes Double Commas in Output (from pandas) #123109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wooseogchoi opened this issue Aug 17, 2024 · 1 comment
Closed

escapechar=',' Causes Double Commas in Output (from pandas) #123109

wooseogchoi opened this issue Aug 17, 2024 · 1 comment
Labels
type-bug An unexpected behavior, bug, or error

Comments

@wooseogchoi
Copy link

wooseogchoi commented Aug 17, 2024 •
edited by github-actions bot
Loading

Bug report

Bug description:

This is related to the issue in pandas.

pandas-dev/pandas#59454

The pandas relies on the writer in cpython's csv to save DataFrame to the given csv file.
When the escapechar is given to pandas, it passes it to writer object as escapedchar.
The quoting is set to the default value, csv.QUOTE_MINIMAL.
The expectation is that data surrounded the given escaped char or delimiter are surronded by quotes. Instead, csv writer just added escaped char after escapted char.
Please see the example below.

# Add a code block here, if required
import csv

data = [
    ['Name', 'Branch', 'Year', 'CGPA'],
    ['Nik,hil', 'COE', 2, 9.0],
    ['San,chit', 'COE', 2, 9.1],
    ['Adi,tya', 'IT', 2, 9.3],
    ['Sa,gar', 'SE', 1, 9.5],
    ['Pra,teek', 'MCE', 3, 7.8],
    ['Sa,hil', 'EP', 2, 9.1]
]

with open('university_records.csv', 'w', newline='') as csvfile:
    writer = csv.writer(csvfile, escapechar=',')
    writer.writerows(data)

Name,Branch,Year,CGPA
Nik,,hil,COE,2,9.0
San,,chit,COE,2,9.1
Adi,,tya,IT,2,9.3
Sa,,gar,SE,1,9.5
Pra,,teek,MCE,3,7.8
Sa,,hil,EP,2,9.1

# Add a code block here, if required
import csv

data = [
    ['Name', 'Branch', 'Year', 'CGPA'],
    ['Nik,hil', 'COE', 2, 9.0],
    ['San,chit', 'COE', 2, 9.1],
    ['Adi,tya', 'IT', 2, 9.3],
    ['Sa,gar', 'SE', 1, 9.5],
    ['Pra,teek', 'MCE', 3, 7.8],
    ['Sa,hil', 'EP', 2, 9.1]
]

with open('university_records.csv', 'w', newline='') as csvfile:
    writer = csv.writer(csvfile, escapechar='.')
    writer.writerows(data)

Name,Branch,Year,CGPA
"Nik,hil",COE,2,9..0
"San,chit",COE,2,9..1
"Adi,tya",IT,2,9..3
"Sa,gar",SE,1,9..5
"Pra,teek",MCE,3,7..8
"Sa,hil",EP,2,9..1

CPython versions tested on:

3.11

Operating systems tested on:

Windows
@wooseogchoi wooseogchoi added the type-bug An unexpected behavior, bug, or error label Aug 17, 2024
@wooseogchoi wooseogchoi mentioned this issue Aug 17, 2024
Closed
3 tasks
@serhiy-storchaka
Copy link
Member

This was fixed in #113796. Now it is an error to specify the same escapechar and delimiter.

@serhiy-storchaka serhiy-storchaka closed this as not planned Won't fix, can't repro, duplicate, stale Aug 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-bug An unexpected behavior, bug, or error
Projects
CSV issues
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@serhiy-storchaka @wooseogchoi