implement S3 v3 pre-signed handler #8918
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thrau
approved these changes
Aug 16, 2023
There was a problem hiding this comment.
LGTM! modifying the headers this way seems fine (we allowed for request header mutability for exactly these use cases). the impact on the handler chain is minimal since we're only executing this for specific services, which is good.
the encapsulation in the handler class is 👌
great set of changes overall! 🥇
Comment on lines
+8
to
+12
| class ParsePreSignedUrlRequest(Handler): | ||
| def __init__(self): | ||
| self.pre_signed_handlers: dict[str, Handler] = { | ||
| "s3": S3PreSignedURLRequestHandler(), | ||
| } |
There was a problem hiding this comment.
nice 👍 maybe at some point in the future we can consolidate this with ServiceResponseHandlers
3ed03e0 to
153493c
Compare
626bc96 to
555135b
Compare
555135b to
2c1a505
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Move the s3 pre-signed handler earlier in the handler chain, so that we can modify the request in the
RequestContextbefore actually parsing it: this allows us to fully use the ASF parser.When pre-signing a request, the S3 client will move some of the request parameters into the query string, when those parameters would be in the headers. Now, the pre-signed handler will add back some headers that we know should be in there, and the parser will properly pick it up.
Changes
Create a general pre-signed handler in the handler chain, which will be able to manage more kind of pre-signed URL than S3. Only S3 is implemented now. Some logic had to be changed in the handler, because we were using the previously parsed
Bucketfrom the request. We now need to parse it ourselves following the same logic as the RequestParser (especially a weird case where we had the sanitized path, and we needed the raw one).