Skip to content

Tries to perform "utf8" decoding when encoding set to "base64" on file commit #427

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
JonBoyleCoding opened this issue Feb 5, 2018 · 2 comments
Closed

Tries to perform "utf8" decoding when encoding set to "base64" on file commit #427

JonBoyleCoding opened this issue Feb 5, 2018 · 2 comments
Labels
docs enhancement
Milestone
v1.4

Comments

@JonBoyleCoding
Copy link

When I'm trying to commit a binary file into a repository and setting the encoding to "base64" (as specified in https://docs.gitlab.com/ce/api/repository_files.html#create-new-file-in-repository), I'm getting the following error:

Traceback (most recent call last):
File "create_submission_point_for_group.py", line 261, in
project.files.create(blob)
File "/usr/local/lib/python2.7/dist-packages/gitlab/cli.py", line 42, in wrapped_f
return f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/exceptions.py", line 247, in wrapped_f
return f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/v4/objects.py", line 1806, in create
server_data = self.gitlab.http_post(path, post_data=new_data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/init.py", line 792, in http_post
post_data=post_data, files=files, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/init.py", line 678, in http_request
prepped = self.session.prepare_request(req)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 394, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/usr/lib/python2.7/dist-packages/requests/models.py", line 298, in prepare
self.prepare_body(data, files, json)
File "/usr/lib/python2.7/dist-packages/requests/models.py", line 424, in prepare_body
body = complexjson.dumps(json)
File "/usr/lib/python2.7/dist-packages/simplejson/init.py", line 380, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 291, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 373, in iterencode
return _iterencode(o, 0)

This makes it seem like the python api is ignoring the encoding option. I've ensured the encoding is set to base64 in the 'blob' that I've passed to project.files.create(). Other ordinary files without the encoding commit perfectly fine.
@JonBoyleCoding
Copy link
Author

JonBoyleCoding commented Feb 21, 2018 •
edited
Loading

After a long look around I found that you actually have to use the package base64 to encode the file, something like below

encoding = "text"
content = open(f,"r").read()

if IsBinaryFile(f): # Uses pip package binaryornot to determine if it's a binary file
	encoding = "base64"
	content = base64.b64encode(content)

# Add to the data blob of all files being added
data["actions"].append({
	"action": "create",
	"file_path": filename,
	"encoding": encoding,
	"content": content
})

I might suggest that something should be added to commits documentation describing how to handle the encoding, as it's not particularly clear for someone that's new to how the API works. Here's the page I mean:

http://python-gitlab.readthedocs.io/en/stable/gl_objects/commits.html

@gpocentek
Copy link
Contributor

Hi @JonBoyleCoding

Sorry for my late answer, and thanks a lot for your feedback on this issue. I'll update the documentation to add the information you provided.

@gpocentek gpocentek added this to the v1.4 milestone Feb 22, 2018
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
docs enhancement
Projects
None yet
Milestone
v1.4
Development

No branches or pull requests
2 participants
@JonBoyleCoding @gpocentek