StepFunctions: Support for Escape Sequences in JSONata String Literals #12072
Conversation
LocalStack Community integration with Pro 2 files 2 suites 33m 29s ⏱️ Results for commit 4698d8b. ♻️ This comment has been updated with latest results. |
| def from_string_literal(parser_rule_context: ParserRuleContext) -> Optional[str]: | ||
| string_literal = parser_rule_context.getText() | ||
| if string_literal.startswith('"') and string_literal.endswith('"'): | ||
| string_literal = string_literal[1:-1] |
There was a problem hiding this comment.
Probably an edge-case but I got a failure when trying to compare the single quote string '"' where the resulting value was not escaped correctly.
Snippet of step function (FYI I snapshot tested this case against AWS):
...
"Pass": {
"Type": "Pass",
"Output": {
"str_value": "\"",
},
"Next": "Choice"
},
"Choice": {
"Type": "Choice",
"Choices": [
{
"Next": "Success",
"Condition": "{% $states.input.str_value = '\"' %}"
}
],
"Default": "Fail"
},
...Result from test:
�[32m(+)�[0m /events/[6]/executionFailedEventDetails ( {} )
�[33m(~)�[0m /events/[3]/stateEnteredEventDetails/input/str_value '"' → '\\"' ... (expected → actual)
�[33m(~)�[0m /events/[5]/stateEnteredEventDetails/input/str_value '"' → '\\"' ... (expected → actual)
�[33m(~)�[0m /events/[5]/stateEnteredEventDetails/name 'Success' → 'Fail' ... (expected → actual)
�[33m(~)�[0m /events/[2]/stateExitedEventDetails/output/str_value '"' → '\\"' ... (expected → actual)
�[33m(~)�[0m /events/[4]/stateExitedEventDetails/output/str_value '"' → '\\"' ... (expected → actual)
�[31m(-)�[0m /events/[6]/stateExitedEventDetails ( {'name': 'Success', 'output': {'str_value': '"'}, 'outputDetails': {'truncated': False}} )
�[33m(~)�[0m /events/[5]/type 'SucceedStateEntered' → 'FailStateEntered' ... (expected → actual)
�[33m(~)�[0m /events/[6]/type 'SucceedStateExited' → 'ExecutionFailed' ... (expected → actual)
�[31m(-)�[0m /events/[7] ( {'executionSucceededEventDetails': {'output': {'str_value': '"'}, 'outputDetails': {'truncated': False}}, 'id': 8, 'previousEventId': 7, 'timestamp': 'timestamp', 'type': 'ExecutionSucceeded'} )
�[32m(+)�[0m /events/[7] ( not present )
Perhaps an issue with ast.literal_eval(f'"{string_literal}"')?
There was a problem hiding this comment.
Thank you @gregfurman!
Indeed, this PR limits the fix to JSONata expressions, however similar parsing issues may still occur for other string literals. In your example the issue is actually the parsing of Output.str_value.
We should definately address this, but in future changes, as updating the parsing logic to all string literals will have many repercussions on other areas of the interpreter, such as intrinsic functions (which are parsed separately).
Motivation
The SFN v2 interpreter currently preserves escape sequences (like \n, \t) as raw character sequences in string literals instead of converting them to actual characters, which leads to errors such as in #12061. When these strings are passed to the Java library through Python's string representation, the escape sequences get double-escaped (
"Hello\nWorld" -> "Hello\\nWorld"). This affects JSONata expression evaluation where escaped characters are expected to be properly interpreted.Changes