Resolve non-subdomain host prefixes to LocalStack #12653
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Test Results - Alternative Providers597 tests 419 ✅ 13m 57s ⏱️ Results for commit e5ee939. ♻️ This comment has been updated with latest results. |
Test Results (amd64) - Integration, Bootstrap 5 files 5 suites 2h 22m 15s ⏱️ Results for commit e5ee939. ♻️ This comment has been updated with latest results. |
cb5aed9 to
d8751a2
Compare
# Conflicts: # tests/aws/services/lambda_/test_lambda.snapshot.json
649d180 to
27cc631
Compare
Likely updated related to this change (just a guess): https://github.com/localstack/localstack/pull/12645/files#diff-1d20d60454c412d95e42d6c9d2626a4389b249eb1cc22a8c1a815cf81b1893f5L62 Couldn't find any operations that use the `discovery-` prefix anymore. ```py discovery_prefixes = df[df['hostPrefix'].str.startswith('discovery-', na=False)] ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends: on public DNS entries for the dash-prefixed domains deployed (e.g.,
data-localhost.localstack.cloud)Motivation
The botocore specs include 562 operations with prefixed endpoints such as
api.ordata-and non-subdomains (i.e.,-dash-prefixed operations) require special consideration.These operations are defined in the botocore specs with a
hostPrefix. Example for DiscoverInstances in CloudMap:"endpoint":{"hostPrefix":"data-"}Example of a failing AWS CLI command due to missing DNS resolution:
aws --profile localstack servicediscovery discover-instances --namespace-name my-namespace --service-name my-service Could not connect to the endpoint URL: "https://data-localhost.localstack.cloud:4566/"To fix this problem, we need to:
a) deploy public DNS entries for these currently 11 prefixes
b) add them to
NAME_PATTERNS_POINTING_TO_LOCALSTACKto ensure they get resolved to LocalStack by our DNS serverChanges
NAME_PATTERNS_POINTING_TO_LOCALSTACKsuch that these are resolved to LocalStack.test_resolve_localstack_hostto cover thehostPrefixAPI operation scenariohostPrefixoperation for Lambda intest_lambda_host_prefix_api_operationDiscussion
Why a static list?
I think it's clearer (and faster) to maintain a static list such that we have full control and easier visibility into which name patterns get resolved to LocalStack. It also helps detecting changes, which require manual action by deploying a new DNS entry into our public DNS.