Skip to content

Replicator: support generating specific VPC ids #11772

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 13, 2024
Merged

Conversation

simonrw
Copy link
Contributor

@simonrw simonrw commented Nov 1, 2024

Motivation

For the Replicator project, we need the ability to create resources with fixed IDs. VPCs have a random ID that is not an input parameter on creation. E.g. `vpc-abcdefg0123456789'. We need the ability to generate knwon ids using the ID store.

Changes

  • add ability to generate known IDs, where the CIDR block is used as a unique key along with the account id and region
    • handle this case if a custom ID is not provided via tags
  • Create a unit test for this behaviour

@simonrw simonrw added the semver: patch Non-breaking changes which can be included in patch releases label Nov 1, 2024
@simonrw simonrw self-assigned this Nov 1, 2024
Copy link

github-actions bot commented Nov 1, 2024

LocalStack Community integration with Pro

 2 files   2 suites   22s ⏱️
22 tests 20 ✅ 2 💤 0 ❌
24 runs  20 ✅ 4 💤 0 ❌

Results for commit 732e393.

♻️ This comment has been updated with latest results.

@simonrw simonrw marked this pull request as ready for review November 1, 2024 17:30
@simonrw simonrw requested review from pinzon and cloutierMat November 1, 2024 17:30
Copy link
Contributor

@cloutierMat cloutierMat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks pretty good. Glad you were able to make it work! 🚀

I think you are missing out on the power of the generate method. It does all of the conditional stuff for you!

The fix would probably be better if done in moto, modifying create_vpc and random_vpc_id directly. That would allow us to remove a lot of dirty patch work! 🤣 But this can also be tackled in batches once we have progressed more on a service rather than a resource by resource pr. Should we keep a tally?

custom_ids = [tag["Value"] for tag in tags if tag["Key"] == TAG_KEY_CUSTOM_ID]
custom_id = custom_ids[0] if len(custom_ids) > 0 else None

# try to generate a custom id from the id store
if not custom_id:
resource_identifier = VpcIdentifier(self.account_id, self.region_name, cidr_block)
id = localstack_id_manager.get_custom_id(resource_identifier)
if id:
custom_id = id

# Check if custom id is unique
if custom_id and custom_id in self.vpcs:
raise InvalidVpcDuplicateCustomIdError(custom_id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the generate method will do all the work for you! It also looks at the TAG_KEY_CUSTOM_ID, and duplicate ids. It will not raise though 🤔 and instead will generate a new id... so maybe you will want to leave that part out?

Suggested change
custom_ids = [tag["Value"] for tag in tags if tag["Key"] == TAG_KEY_CUSTOM_ID]
custom_id = custom_ids[0] if len(custom_ids) > 0 else None
# try to generate a custom id from the id store
if not custom_id:
resource_identifier = VpcIdentifier(self.account_id, self.region_name, cidr_block)
id = localstack_id_manager.get_custom_id(resource_identifier)
if id:
custom_id = id
# Check if custom id is unique
if custom_id and custom_id in self.vpcs:
raise InvalidVpcDuplicateCustomIdError(custom_id)
resource_identifier = VpcIdentifier(self.account_id, self.region_name, cidr_block)
custom_id = resource_identifier.generate(tags=tags, existing_ids=self.vpcs.keys())

Comment on lines 32 to 33
suffix = "".join(random.choice(string.ascii_letters) for _ in range(17))
return f"vpc-{suffix}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we could return "". And maybe add a comment that moto does the generation?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think @cloutierMat merged recently a PR so moto resource can have the custom id generation. I think this utility is for our LocalStack store resources.

Copy link
Member

@pinzon pinzon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but moto now has its own way to generate a custom Id. I think 🤔

@simonrw simonrw force-pushed the feat/replicator/vpc branch from 9d566b9 to 2fc8407 Compare November 12, 2024 17:54
@simonrw simonrw added this to the 4.1 milestone Nov 13, 2024
@simonrw simonrw force-pushed the feat/replicator/vpc branch from 1eaec09 to 732e393 Compare November 13, 2024 17:29
@simonrw simonrw requested a review from cloutierMat November 13, 2024 17:29
Copy link
Contributor

@cloutierMat cloutierMat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments. 🚀

@simonrw simonrw merged commit 1b3239d into master Nov 13, 2024
32 checks passed
@simonrw simonrw deleted the feat/replicator/vpc branch November 13, 2024 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants