Skip to content

fix APIGW path converter with prepended multiple slashes #11268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 29, 2024

Conversation

bentsku
Copy link
Contributor

@bentsku bentsku commented Jul 25, 2024

Motivation

As reported in #11253, our Edge Router had issue with path parameter starting with multiple slashes, as it would return a redirect which we shouldn't do.

This PR introduces a new converter in the default ROUTER, which is the one we already use internally for the OP Router for AWS, and also internally for the new generation APIGW.

This PR introduces the change and uses the new converter for APIGW routes, and fixes the behavior around it, with added integration test cases.

Changes

  • migrate the GreedyPathConverter to edge.py as we need to use it in our default router, and refactor its usages
  • add new tests for the router with the greedy path converter
  • fix APIGW NG behavior with the new greedy path
  • fix the legacy APIGW, which was a bit more tricky, I'll run tests from upstream as well to be sure we're not breaking behavior

Note: failing tests are unrelated, and I'm running upstream full test suite to validate the changes which is green. Failing tests will be addressed with #11261

fixes #11253

@bentsku bentsku added aws:apigateway Amazon API Gateway area: asf semver: patch Non-breaking changes which can be included in patch releases labels Jul 25, 2024
@bentsku bentsku self-assigned this Jul 25, 2024
Copy link

github-actions bot commented Jul 25, 2024

S3 Image Test Results (AMD64 / ARM64)

  2 files    2 suites   3m 32s ⏱️
405 tests 353 ✅  52 💤 0 ❌
810 runs  706 ✅ 104 💤 0 ❌

Results for commit 767c095.

♻️ This comment has been updated with latest results.

@bentsku bentsku added semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases and removed semver: patch Non-breaking changes which can be included in patch releases labels Jul 25, 2024
Copy link

github-actions bot commented Jul 25, 2024

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 34m 50s ⏱️ +3s
3 303 tests ±0  2 909 ✅ ±0  394 💤 ±0  0 ❌ ±0 
3 305 runs  ±0  2 909 ✅ ±0  396 💤 ±0  0 ❌ ±0 

Results for commit 767c095. ± Comparison against base commit 1aad84d.

♻️ This comment has been updated with latest results.

@bentsku bentsku marked this pull request as ready for review July 26, 2024 16:02
Copy link
Member

@thrau thrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't say much about the APIGW changes, just would like the converter to move somewhere else :-)

StrictMethodRule,
path_param_regex,
post_process_arg_name,
transform_path_params_to_rule_vars,
)
from localstack.http import Request
from localstack.http.request import get_raw_path
from localstack.services.edge import GreedyPathConverter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know i said "just add the greedy match router + converter to edge.py?", but really meant either copy&pasting it here, or move it to a more generic place and import it from there :-)

i don't like the localstack.services.edge import here.

i'd suggest moving the GreedyPathConverter to localstack.http.router for now, and perhaps also introduce it in rolo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is much nicer, thanks 👌 I didn't like it either, it felt very out of place. This looks better!



ROUTER: Router[Handler] = Router(
dispatcher=handler_dispatcher(), converters={"greedy_path": GreedyPathConverter}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding this here is 👌

Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice drill down, and a clean and well-tested change in API GW! 🏅

I agree with @thrau that the GreedyPathConverter shouldn't move into localstack.services.edge but rather into localstack.http.router.
Besides that I only had a question on a small change in a helper (see comment).
So as soon as the question is answered, the converter is moved, and most importantly all tests are green (this is a tricky change which can break something totally unexpected), this is looking good to get merged! 🚀

@bentsku bentsku force-pushed the apigw-slash-path branch from f7e9117 to 767c095 Compare July 29, 2024 09:21
@bentsku
Copy link
Contributor Author

bentsku commented Jul 29, 2024

So as soon as the question is answered, the converter is moved, and most importantly all tests are green (this is a tricky change which can break something totally unexpected), this is looking good to get merged! 🚀

@alexrashed I'll re-run this repo tests after rebasing, which should fix the flakes, but the upstream pipeline is already fully green, that should be good!

Copy link
Contributor

@cloutierMat cloutierMat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Super neat fix 🚀

Glad to see how simple it was to support with Next Gen also!

@bentsku bentsku merged commit aa7fc40 into master Jul 29, 2024
40 checks passed
@bentsku bentsku deleted the apigw-slash-path branch July 29, 2024 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: asf aws:apigateway Amazon API Gateway semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: API Gateway V1 (targeted to Lambda) gives 500 error with // in path
4 participants