Step Functions: Extended Support for Escape Sequences in String Literals #12222
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
LocalStack Community integration with Pro 2 files ± 0 2 suites ±0 34m 32s ⏱️ - 1h 46m 2s Results for commit acb958e. ± Comparison against base commit 84b2f5f. This pull request removes 2656 and adds 6 tests. Note that renamed tests count towards both.This pull request removes 244 skipped tests and adds 2 skipped tests. Note that renamed tests count towards both. |
joe4dev
approved these changes
Feb 4, 2025
| @markers.aws.validated | ||
| @pytest.mark.skip( | ||
| reason=( | ||
| "Lack of generalisable approach to escape sequences support " |
There was a problem hiding this comment.
Thank you for this clarification 👍
| @@ -1183,8 +1183,8 @@ def visitPayload_value_null(self, ctx: ASLParser.Payload_value_nullContext) -> P | |||
| return PayloadValueNull() | |||
|
|
|||
| def visitPayload_value_str(self, ctx: ASLParser.Payload_value_strContext) -> PayloadValueStr: | |||
| str_val = self._inner_string_of(parser_rule_context=ctx.string_literal()) | |||
| return PayloadValueStr(val=str_val) | |||
| string_literal: StringLiteral = self.visitString_literal(ctx=ctx.string_literal()) | |||
There was a problem hiding this comment.
literal string abstraction sounds clearer than "innver_string" 👍
| "IntrinsicEscape": { | ||
| "Type": "Pass", | ||
| "Parameters": { | ||
| "parsed.$": "States.StringToJson('{\"text\":\"An \\\"escaped double quote\\\" here\"}')" |
There was a problem hiding this comment.
I love the descriptive message clarifying the test intention 💯 ✨
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
The SFN v2 interpreter currently does not consistently handle escape sequences in string literals. This can lead to issues when users declare strings with escape sequences, potentially causing failures in string comparison logic.
Previous changes in #12072 introduced a strategy to correctly interpret string literals containing escape sequences, but this fix was limited to JSONata expression literals. However, escape sequence handling issues persist in other areas, such as baseline string literals, intrinsic functions, and JSONPath strings.
During this investigation, all string literal types were analyzed and tested. However, after extensive research into AWS’s behavior, we found that only strict string literals could be reliably aligned with AWS’s handling of escape sequences. Additionally, issues related to parsing string literals as JSONata expressions in Output and Assign blocks were identified and resolved.
That said, achieving full parity with AWS’s handling of escape sequences in JSONPath and intrinsic functions proved impractical. Specifically, we could not limit LocalStack’s escape sequence support to match AWS’s restrictions without introducing significant limitations. As a result, LocalStack continues to allow escape sequences in more scenarios than AWS.
It's worth noting that with the recent shift towards JSONata expressions, state machines are expected to standardize around just two types of string literals: String Literals and JSONata Expressions. Given this transition, precisely mirroring AWS’s handling of escape sequences in other string types is not an immediate priority.
* Please refer to the details included in the backlog item associated to these changes for more insights.
Changes