Motivation

Currently the SFN internpreter v2 represents string literals though a keyword_or_string sub-production or through specialised terminal token types. This approach lacked an intermediary representation for string carrying executable expressions (jsonpaths, context objects, intrinsic functions, jsonata expressions, variable sampling), leading to some logic duplication as each declaration often having to implement the evaluation logic of these strings in varying context. These changes introduce the string_expression sub-production and a few logic child productions. Most importantly, all its sub productions are preprocessed as the expected evaluation component for the specific string expression they represent. The result is a more delcarative grammar, which ensures the correct string expressions are placed in the intended productions, with the productions seemingly having to just delegate the string expression provided, or apply the relevant logic overrides.

Changes

• introduced the string_expression concept
• introduced reusable string expression evaluation component
• updated the grammar to be more delcarative in the expected string expressions for each sub-production
• applied relevant changes for this migration