Skip to content

add support for Fn::Tranform in CFnV2 #12966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Aug 14, 2025
Merged

add support for Fn::Tranform in CFnV2 #12966

merged 32 commits into from
Aug 14, 2025

Conversation

pinzon
Copy link
Member

@pinzon pinzon commented Aug 6, 2025

Motivation

This PR implements the capability to resolve FnTransform in the new CFnV2 engine in multiple locations of a template.

Changes

  • Add properties (parent, jsonpath) to Scope class
  • NodeResources and NodeProperties define a possible fn::transformation
  • New NodeIntrinsicFunctionFnTransform class to model a Fn::Transform
  • _visit_intrinsic_function can return a NodeIntrinsicFunctionFnTransform
  • Refactor _excute_macro and AWS::Include into transformer
  • Add events when a transformation fails
  • Add template body character limit assertion
  • Transformer adds some extra overrides due to missing resources during transformations

Testing

  • Removed skip_if_v2_provider in related tests

Notes

This PR only implements the resolution of the transformations in specific locations. More locations are going to be needed in the future

@pinzon pinzon added the semver: patch Non-breaking changes which can be included in patch releases label Aug 6, 2025
Copy link

github-actions bot commented Aug 6, 2025

Test Results - Preflight, Unit

22 107 tests  ±0   20 372 ✅ ±0   6m 20s ⏱️ -14s
     1 suites ±0    1 735 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Aug 6, 2025

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 6s ⏱️ -8s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Aug 6, 2025

Test Results (amd64) - Integration, Bootstrap

    5 files  ±0      5 suites  ±0   2h 20m 42s ⏱️ - 1m 10s
4 987 tests ±0  4 395 ✅ ±0  592 💤 ±0  0 ❌ ±0 
4 993 runs  ±0  4 395 ✅ ±0  598 💤 ±0  0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Aug 6, 2025

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 45m 8s ⏱️ + 2m 14s
4 628 tests ±0  4 188 ✅ ±0  440 💤 ±0  0 ❌ ±0 
4 630 runs  ±0  4 188 ✅ ±0  442 💤 ±0  0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

@pinzon pinzon force-pushed the cp/cfn/v2/transform branch from a85bc97 to aae7310 Compare August 7, 2025 19:40
@pinzon pinzon force-pushed the cp/cfn/v2/transform branch 3 times, most recently from 8703ae2 to b22a894 Compare August 12, 2025 20:36
@pinzon pinzon requested a review from simonrw August 12, 2025 22:08
@pinzon pinzon force-pushed the cp/cfn/v2/transform branch from daa7743 to c962e2f Compare August 12, 2025 22:18
@pinzon pinzon marked this pull request as ready for review August 12, 2025 22:24
Comment on lines +307 to +314
except FailedTransformationException as e:
change_set.status = ChangeSetStatus.FAILED
change_set.status_reason = e.message
change_set.stack.set_stack_status(
status=StackStatus.ROLLBACK_IN_PROGRESS, reason=e.message
)
change_set.stack.set_stack_status(status=StackStatus.CREATE_FAILED)
return
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonrw This is the section I'm not liking that much

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right, it's not general enough (e.g. it hard codes CREATE_FAILED) however I think as we get more testing around transforms in general this will work itself out.

@pinzon pinzon force-pushed the cp/cfn/v2/transform branch from e078fad to 3409251 Compare August 13, 2025 21:38
pinzon and others added 3 commits August 13, 2025 19:58
Copy link
Contributor

@simonrw simonrw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing stuff, this was a really hard problem and it's solution is actually pretty clean. I am still not 100% on board with using the scope to manipulate the template structure however it is still quite a neat solution, and it prevents us from needing to add checks for Fn::Transform in parent nodes as well 👍

Comment on lines +479 to +480
before_siblings: list[Any],
after_siblings: list[Any],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: I'm slightly confused why the arguments can be a ChangeSetEntity but the siblings have to be separate values. When I try to change it I end up re-visiting the transform and hitting the "Invalid: Fn::Transforms cannot be nested inside another Fn::Transform" case. I understand why (the Fn::Transform is included in the siblings) and I don't think it's worth it but perhaps we can try to switch to this type (i.e. make these a single siblings: ChangeSetEntity property) in a follow up

"Invalid: Fn::Transforms cannot be nested inside another Fn::Transform"
)

path = "$" + ".".join(scope.split("/")[:-1])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: could this be replaced with

Suggested change
path = "$" + ".".join(scope.split("/")[:-1])
path = scope.parent.jsonpath

Comment on lines +307 to +314
except FailedTransformationException as e:
change_set.status = ChangeSetStatus.FAILED
change_set.status_reason = e.message
change_set.stack.set_stack_status(
status=StackStatus.ROLLBACK_IN_PROGRESS, reason=e.message
)
change_set.stack.set_stack_status(status=StackStatus.CREATE_FAILED)
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right, it's not general enough (e.g. it hard codes CREATE_FAILED) however I think as we get more testing around transforms in general this will work itself out.


from localstack.aws.api.lambda_ import Runtime
from localstack.testing.pytest import markers
from localstack.utils.strings import short_uid


@pytest.mark.skip(reason="Not implemented with either provider")
@skip_if_v1_provider("change sets")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@@ -830,7 +825,7 @@ def test_attribute_uses_macro(self, deploy_cfn_template, create_lambda_function,
assert "test-" in resulting_value

@markers.aws.validated
@pytest.mark.skip(reason="Fn::Transform does not support array of transformations")
@skip_if_v1_provider("V1 is unable to resolve fn::transform with lists")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@simonrw simonrw merged commit c417544 into main Aug 14, 2025
43 of 44 checks passed
@simonrw simonrw deleted the cp/cfn/v2/transform branch August 14, 2025 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants