Skip to content

Conversation

pinzon
Copy link
Member

@pinzon pinzon commented Aug 6, 2025

Motivation

This PR implements the capability to resolve FnTransform in the new CFnV2 engine in multiple locations of a template.

Changes

  • Add properties (parent, jsonpath) to Scope class
  • NodeResources and NodeProperties define a possible fn::transformation
  • New NodeIntrinsicFunctionFnTransform class to model a Fn::Transform
  • _visit_intrinsic_function can return a NodeIntrinsicFunctionFnTransform
  • Refactor _excute_macro and AWS::Include into transformer
  • Add events when a transformation fails
  • Add template body character limit assertion
  • Transformer adds some extra overrides due to missing resources during transformations

Testing

  • Removed skip_if_v2_provider in related tests

Notes

This PR only implements the resolution of the transformations in specific locations. More locations are going to be needed in the future

@pinzon pinzon added the semver: patch Non-breaking changes which can be included in patch releases label Aug 6, 2025
Copy link

github-actions bot commented Aug 6, 2025

Test Results - Preflight, Unit

22 107 tests  ±0   20 372 ✅ ±0   6m 20s ⏱️ -14s
     1 suites ±0    1 735 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Aug 6, 2025

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 6s ⏱️ -8s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Aug 6, 2025

Test Results (amd64) - Integration, Bootstrap

    5 files  ±0      5 suites  ±0   2h 20m 42s ⏱️ - 1m 10s
4 987 tests ±0  4 395 ✅ ±0  592 💤 ±0  0 ❌ ±0 
4 993 runs  ±0  4 395 ✅ ±0  598 💤 ±0  0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Aug 6, 2025

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 45m 8s ⏱️ + 2m 14s
4 628 tests ±0  4 188 ✅ ±0  440 💤 ±0  0 ❌ ±0 
4 630 runs  ±0  4 188 ✅ ±0  442 💤 ±0  0 ❌ ±0 

Results for commit 1b4f91b. ± Comparison against base commit 5a920cb.

♻️ This comment has been updated with latest results.

@pinzon pinzon force-pushed the cp/cfn/v2/transform branch from a85bc97 to aae7310 Compare August 7, 2025 19:40
@pinzon pinzon force-pushed the cp/cfn/v2/transform branch 3 times, most recently from 8703ae2 to b22a894 Compare August 12, 2025 20:36
@pinzon pinzon requested a review from simonrw August 12, 2025 22:08
@pinzon pinzon force-pushed the cp/cfn/v2/transform branch from daa7743 to c962e2f Compare August 12, 2025 22:18
@pinzon pinzon marked this pull request as ready for review August 12, 2025 22:24
Comment on lines +307 to +314
except FailedTransformationException as e:
change_set.status = ChangeSetStatus.FAILED
change_set.status_reason = e.message
change_set.stack.set_stack_status(
status=StackStatus.ROLLBACK_IN_PROGRESS, reason=e.message
)
change_set.stack.set_stack_status(status=StackStatus.CREATE_FAILED)
return
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonrw This is the section I'm not liking that much

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right, it's not general enough (e.g. it hard codes CREATE_FAILED) however I think as we get more testing around transforms in general this will work itself out.

@pinzon pinzon force-pushed the cp/cfn/v2/transform branch from e078fad to 3409251 Compare August 13, 2025 21:38
pinzon and others added 3 commits August 13, 2025 19:58
Copy link
Contributor

@simonrw simonrw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing stuff, this was a really hard problem and it's solution is actually pretty clean. I am still not 100% on board with using the scope to manipulate the template structure however it is still quite a neat solution, and it prevents us from needing to add checks for Fn::Transform in parent nodes as well 👍

Comment on lines +479 to +480
before_siblings: list[Any],
after_siblings: list[Any],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: I'm slightly confused why the arguments can be a ChangeSetEntity but the siblings have to be separate values. When I try to change it I end up re-visiting the transform and hitting the "Invalid: Fn::Transforms cannot be nested inside another Fn::Transform" case. I understand why (the Fn::Transform is included in the siblings) and I don't think it's worth it but perhaps we can try to switch to this type (i.e. make these a single siblings: ChangeSetEntity property) in a follow up

"Invalid: Fn::Transforms cannot be nested inside another Fn::Transform"
)

path = "$" + ".".join(scope.split("/")[:-1])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: could this be replaced with

Suggested change
path = "$" + ".".join(scope.split("/")[:-1])
path = scope.parent.jsonpath

Comment on lines +307 to +314
except FailedTransformationException as e:
change_set.status = ChangeSetStatus.FAILED
change_set.status_reason = e.message
change_set.stack.set_stack_status(
status=StackStatus.ROLLBACK_IN_PROGRESS, reason=e.message
)
change_set.stack.set_stack_status(status=StackStatus.CREATE_FAILED)
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right, it's not general enough (e.g. it hard codes CREATE_FAILED) however I think as we get more testing around transforms in general this will work itself out.


from localstack.aws.api.lambda_ import Runtime
from localstack.testing.pytest import markers
from localstack.utils.strings import short_uid


@pytest.mark.skip(reason="Not implemented with either provider")
@skip_if_v1_provider("change sets")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@@ -830,7 +825,7 @@ def test_attribute_uses_macro(self, deploy_cfn_template, create_lambda_function,
assert "test-" in resulting_value

@markers.aws.validated
@pytest.mark.skip(reason="Fn::Transform does not support array of transformations")
@skip_if_v1_provider("V1 is unable to resolve fn::transform with lists")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@simonrw simonrw merged commit c417544 into main Aug 14, 2025
43 of 44 checks passed
@simonrw simonrw deleted the cp/cfn/v2/transform branch August 14, 2025 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants