Skip to content
This repository was archived by the owner on Apr 20, 2023. It is now read-only.

Encode s3 upload data as utf8 #121

Merged
merged 1 commit into from
Dec 18, 2017
Merged

Encode s3 upload data as utf8 #121

merged 1 commit into from
Dec 18, 2017

Conversation

JrGoodle
Copy link
Contributor

Resolves #85

I ran into this issue recently on Circle CI. After making the fix mentioned in the issue comments in a branch on my fork, I was able to successfully upload my test results to codecov.

Before: https://circleci.com/gh/JrGoodle/clowder/97
After: https://circleci.com/gh/JrGoodle/clowder/98

The output before and after the change can be seen at the bottom of the "upload code coverage results" build phase.

If I need to add any tests, just let me know how it would be preferable to test this.

@stevepeak stevepeak merged commit bc70732 into codecov:master Dec 18, 2017
@chfast
Copy link
Contributor

chfast commented Jan 3, 2018

I'm still having this issue in https://circleci.com/gh/ethereum/cpp-ethereum/2049. v2.0.11.

@stevepeak
Copy link
Contributor

This patch is live in v2.0.11. But It appears @chfast is still having issues with it...

@JrGoodle
Copy link
Contributor Author

JrGoodle commented Jan 3, 2018

This error message is different than the one I was seeing.

My error message:

Error: 'latin-1' codec can't encode character '\u2019' in position 139609: Body ('’') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.

This error message:

Error: 'ascii' codec can't decode byte 0xe2 in position 12947300: ordinal not in range(128)

This SO post seems relevant: https://stackoverflow.com/questions/21129020/how-to-fix-unicodedecodeerror-ascii-codec-cant-decode-byte

I'm not an expert in this area, though. I just applied the change suggested here and it fixed the issue I was running into.

@chfast
Copy link
Contributor

chfast commented Jan 3, 2018

Some additional information about bug #130.

  1. It is happening only on macOS, Linux nad Windows seem to be ok. However, Linux is using v2.0.10, macOS v2.0.11.
  2. It started happening recently.

@chfast
Copy link
Contributor

chfast commented Jan 5, 2018

I can confirm it's v2.0.11 regression. After Linux auto-upgraded to v2.0.11 I have the same issue there too.

@blueyed
Copy link
Contributor

blueyed commented Jan 5, 2018

It looks like this PR created the issue #130.

Not sure where the Use body.encode('utf-8') if you want to send it encoded in UTF-8. msg/hint is coming from that this tries to fix - but it does not refer to the line likely where it was applied.

@@ -719,7 +719,7 @@ def main(*argv, **kwargs):
result, upload_url = res[0], res[1]

write(' Uploading to S3...')
s3 = requests.put(upload_url, data=reports,
s3 = requests.put(upload_url, data=reports.encode('utf-8'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is bad because reports is bytes in python2 and string in python3. Only string must be encoded to bytes.

Alternatively would be to make reports a string also in python2 (there it is called unicode). You can use u'' prefix for that.

@JrGoodle
Copy link
Contributor Author

JrGoodle commented Jan 6, 2018

@chfast I made a PR with a proposed fix: #133

Let me know if you think that approach looks good

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot upload test results that contain characters outside of latin-1
4 participants