Skip to content

Strip base64 padding characters in Datastore's (to|from)_legacy_urlsafe #3560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 28, 2017

Conversation

michaelenglo
Copy link

Hi,

I am proposing a change to the Google Cloud Platform Datastore's Python client library, specifically the Key module. @dhermes and I have discussed about removing the padding characters ("=") from regular b64encoded urlsafe to match the version that is used in Datastore (which does not have the padding character "="). This is done to ensure consistency in using and reproducing Datastore's convention for urlsafe key.

Please let me know if there is any other steps I may not have included and are needed to do to improve the changes.

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jun 28, 2017
@michaelenglo
Copy link
Author

michaelenglo commented Jun 28, 2017

@dhermes Ignore the last PR (#3559). I just made a new exact same commit using my personal account (I was using my work account when committing the earlier one, which is not associated to this Github account and not CLA signed).

@dhermes
Copy link
Contributor

dhermes commented Jun 28, 2017

Thanks for using your personal account. Did you have a chance to see my comments on #3559?

@michaelenglo
Copy link
Author

Hi @dhermes , thanks for your helpful comments! I made another commit to my master to follow your comments. Let me know if it's already good to go or not!

@@ -324,7 +325,8 @@ def from_legacy_urlsafe(cls, urlsafe):
This is intended to work with the "legacy" representation of a
datastore "Key" used within Google App Engine (a so-called
"Reference"). This assumes that ``urlsafe`` was created within an App
Engine app via something like ``ndb.Key(...).urlsafe()``.
Engine app via something like ``ndb.Key(...).urlsafe()``. The base64
encoded values will have padding removed.

This comment was marked as spam.

@@ -333,6 +335,9 @@ def from_legacy_urlsafe(cls, urlsafe):
:rtype: :class:`~google.cloud.datastore.key.Key`.
:returns: The key corresponding to ``urlsafe``.
"""
padding = b'=' * (-len(urlsafe) % 4)
urlsafe += padding

This comment was marked as spam.

Michael Englo added 2 commits June 28, 2017 10:03
Since both stripped and non-stripped padding should be acceptable.
@dhermes dhermes added the api: datastore Issues related to the Datastore API. label Jun 28, 2017
@googlebot
Copy link

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that they're okay with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

@googlebot googlebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Jun 28, 2017
@dhermes
Copy link
Contributor

dhermes commented Jun 28, 2017

@michaelenglo I've added some new tests to actually make sure the padding gets added/stripped.

This LGTM and I'll merge one CircleCI is green.

Thanks a lot!

@dhermes dhermes merged commit eb03366 into googleapis:master Jun 28, 2017
@dhermes dhermes changed the title strip padding characters from urlsafe Strip base64 padding characters from urlsafe in Datastore's (to|from)_legacy_urlsafe Jun 28, 2017
@dhermes dhermes changed the title Strip base64 padding characters from urlsafe in Datastore's (to|from)_legacy_urlsafe Strip base64 padding characters in Datastore's (to|from)_legacy_urlsafe Jun 28, 2017
@michaelenglo
Copy link
Author

michaelenglo commented Jun 28, 2017

Thanks a lot @dhermes ! I appreciate your responsiveness and helpfulness since day one. Hope to get more chance to learn and contribute to the library!

tseaver pushed a commit that referenced this pull request Jul 10, 2017
…_legacy_urlsafe (#3560)

Also

* add padding characters in `from_legacy_urlsafe` if needed
* add an extra example in the unit tests that actually requires base64 padding
landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 21, 2017
…_legacy_urlsafe (googleapis#3560)

Also

* add padding characters in `from_legacy_urlsafe` if needed
* add an extra example in the unit tests that actually requires base64 padding
landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 22, 2017
…_legacy_urlsafe (googleapis#3560)

Also

* add padding characters in `from_legacy_urlsafe` if needed
* add an extra example in the unit tests that actually requires base64 padding
landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 22, 2017
…_legacy_urlsafe (googleapis#3560)

Also

* add padding characters in `from_legacy_urlsafe` if needed
* add an extra example in the unit tests that actually requires base64 padding
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: datastore Issues related to the Datastore API. cla: no This human has *not* signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants