CloudFormation V2 Engine: Support for Pseudo Parameter References #12595

MEPalma · 2025-05-07T16:04:25Z

Motivation

The introduction of the CloudFormation v2 engine laid the foundation for a redesigned engine capable of accurately determining update requirements between CloudFormation deployments, while also enabling parallel execution during updates. However, the current implementation still lacks support for Pseudo Parameter References

Changes

Add support for Pseudo parameters
Add base ChangeSet lifecycle test
Un-skip v1 test for apigateway
Moved the responsibility of change set description to the provider
Other minors

localstack-bot · 2025-05-07T16:04:39Z

Currently, only patch changes are allowed on master. Your PR labels (semver: minor) indicate that it cannot be merged into the master at this time.

github-actions · 2025-05-07T16:21:09Z

LocalStack Community integration with Pro

2 files ± 0 2 suites ±0 21m 39s ⏱️ - 1h 19m 54s
491 tests - 3 960 315 ✅ - 3 752 176 💤 - 208 0 ❌ ±0
493 runs - 3 960 315 ✅ - 3 752 178 💤 - 208 0 ❌ ±0

Results for commit 87bc478. ± Comparison against base commit 5f9ae76.

This pull request removes 3961 and adds 1 tests. Note that renamed tests count towards both.

tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_lambda_dynamodb
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_opensearch_crud
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_search_books
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_setup
tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_destination_sns
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_infra
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_prefill_dynamodb_table
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input0-SUCCEEDED]
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input1-SUCCEEDED]
…

tests.aws.services.cloudformation.v2.test_change_set_ref.TestChangeSetRef ‑ test_supported_pseudo_parameter

This pull request removes 209 skipped tests and adds 1 skipped test. Note that renamed tests count towards both.

tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input4-FAILED]
tests.aws.scenario.mythical_mysfits.test_mythical_misfits.TestMythicalMisfitsScenario ‑ test_deployed_infra_state
tests.aws.scenario.mythical_mysfits.test_mythical_misfits.TestMythicalMisfitsScenario ‑ test_populate_data
tests.aws.scenario.mythical_mysfits.test_mythical_misfits.TestMythicalMisfitsScenario ‑ test_user_clicks_are_stored
tests.aws.services.apigateway.test_apigateway_api.TestApiGatewayApiRestApi ‑ test_get_api_case_insensitive
tests.aws.services.apigateway.test_apigateway_api.TestApigatewayIntegration ‑ test_put_integration_request_parameter_bool_type
tests.aws.services.apigateway.test_apigateway_basic.TestAPIGateway ‑ test_api_gateway_authorizer_crud
tests.aws.services.apigateway.test_apigateway_basic.TestAPIGateway ‑ test_api_gateway_http_integration_with_path_request_parameter
tests.aws.services.apigateway.test_apigateway_basic.TestAPIGateway ‑ test_api_gateway_lambda_proxy_integration[/lambda/foo1]
…

tests.aws.services.cloudformation.v2.test_change_set_ref.TestChangeSetRef ‑ test_supported_pseudo_parameter

♻️ This comment has been updated with latest results.

simonrw

A great set of changes and a nice pr thanks!

simonrw · 2025-05-08T19:11:10Z

localstack-core/localstack/services/cloudformation/engine/v2/change_set_model_preproc.py

+        # TODO: review required behaviour of before and after, are pseudo parameter values constant during
+        #       change set lifecycles?


As we discussed, yes they are

Suggested change

# TODO: review required behaviour of before and after, are pseudo parameter values constant during

# change set lifecycles?

simonrw · 2025-05-08T19:12:02Z

localstack-core/localstack/services/cloudformation/engine/v2/change_set_model_preproc.py

+        #       change set lifecycles?
+        match pseudo_parameter_name:
+            case "AWS::Partition":
+                after = "aws"


What about our other partition support like govcloud? Maybe leave a todo. What does the v1 engine do here?

Thanks, indeed we can use the get_partition utility about the region name as the v1 provider does

simonrw · 2025-05-08T19:13:33Z

tests/aws/services/cloudformation/v2/ported_from_v1/resources/test_apigateway.py

@@ -318,6 +318,8 @@ def test_cfn_deploy_apigateway_integration(deploy_cfn_template, snapshot, aws_cl
        "$.get-stage.lastUpdatedDate",
        "$.get-stage.methodSettings",
        "$.get-stage.tags",
+        "$..endpointConfiguration.ipAddressType",
+        "$..binaryMediaTypes",


Maybe rebase onto master, this is now in that branch

simonrw · 2025-05-08T19:15:04Z

tests/aws/services/cloudformation/v2/test_change_set_ref.py

+        template_1 = {
+            "Resources": {
+                "Topic": {"Type": "AWS::SNS::Topic", "Properties": {"TopicName": topic_name}},
+            }
+        }


Is there a reason we create an sns topic in template one and use a completely different resource in template two?

No, there is no particular reason other than needing an update with a resource listing a number of supported pseudo parameters. I changed it to only use SNS topics.

simonrw · 2025-05-08T19:17:05Z

tests/aws/services/cloudformation/v2/test_change_set_ref.py

+                            {"Key": "AccountId", "Value": {"Ref": "AWS::AccountId"}},
+                            {"Key": "Region", "Value": {"Ref": "AWS::Region"}},
+                            {"Key": "StackName", "Value": {"Ref": "AWS::StackName"}},
+                            {"Key": "StackId", "Value": {"Ref": "AWS::StackId"}},


We don't cover urlsuffix here. I realise that by definition it will be different from AWS to LocalStack but at least it would be execute the code path

simonrw · 2025-05-08T19:18:03Z

localstack-core/localstack/services/cloudformation/v2/provider.py

@@ -285,6 +290,31 @@ def _run(*args):

        return ExecuteChangeSetOutput()

+    def _describe_change_set(


Great I'm glad it's in the provider.

simonrw · 2025-05-08T19:18:38Z

localstack-core/localstack/services/cloudformation/v2/provider.py

+            StackId=change_set.stack.stack_id,
+            StackName=change_set.stack.stack_name,
+            CreationTime=change_set.creation_time,
+            Parameters=[


Can we keep the todo about masking ?

MEPalma added 3 commits May 7, 2025 14:35

parity improvements

2106678

cleanup deployed props retrieval

6d1d6c2

add support for pseudo parameters

850d0f5

MEPalma self-assigned this May 7, 2025

MEPalma added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label May 7, 2025

MEPalma added this to the Playground milestone May 7, 2025

MEPalma changed the base branch from master to MEP-CFN-parity_b0 May 8, 2025 11:26

Base automatically changed from MEP-CFN-parity_b0 to master May 8, 2025 13:04

MEPalma added 2 commits May 8, 2025 15:47

conflicts

e487368

move describe responsabilities

02fc9b6

MEPalma marked this pull request as ready for review May 8, 2025 14:33

MEPalma requested review from dominikschubert, pinzon and simonrw as code owners May 8, 2025 14:33

MEPalma removed request for dominikschubert and pinzon May 8, 2025 14:33

MEPalma added 3 commits May 8, 2025 16:36

minor

971a8a7

cleanup

af622e2

cleanup

ae9dc0f

MEPalma modified the milestones: Playground, 4.5 May 8, 2025

MEPalma added semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases and removed semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases labels May 8, 2025

simonrw approved these changes May 8, 2025

View reviewed changes

conflicts, pr items

87bc478

MEPalma merged commit fa0bbcd into master May 19, 2025
32 checks passed

MEPalma deleted the MEP-CFN-pseudo_parameters branch May 19, 2025 11:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

CloudFormation V2 Engine: Support for Pseudo Parameter References #12595

CloudFormation V2 Engine: Support for Pseudo Parameter References #12595

Uh oh!

MEPalma commented May 7, 2025 •

edited

Loading

Uh oh!

localstack-bot commented May 7, 2025

Uh oh!

github-actions bot commented May 7, 2025 •

edited

Loading

Uh oh!

simonrw left a comment

Uh oh!

simonrw May 8, 2025

Uh oh!

simonrw May 8, 2025

Uh oh!

MEPalma May 19, 2025

Uh oh!

simonrw May 8, 2025

Uh oh!

simonrw May 8, 2025

Uh oh!

MEPalma May 19, 2025

Uh oh!

simonrw May 8, 2025

Uh oh!

simonrw May 8, 2025

Uh oh!

simonrw May 8, 2025

Uh oh!

Uh oh!

Uh oh!

		# TODO: review required behaviour of before and after, are pseudo parameter values constant during
		# change set lifecycles?

		@@ -285,6 +290,31 @@ def _run(*args):

		return ExecuteChangeSetOutput()

		def _describe_change_set(

Uh oh!

CloudFormation V2 Engine: Support for Pseudo Parameter References #12595

CloudFormation V2 Engine: Support for Pseudo Parameter References #12595

Uh oh!

Conversation

MEPalma commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Uh oh!

localstack-bot commented May 7, 2025

Uh oh!

github-actions bot commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack Community integration with Pro

Uh oh!

simonrw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MEPalma commented May 7, 2025 •

edited

Loading

github-actions bot commented May 7, 2025 •

edited

Loading