Skip to content

Add MessageAttributes to SQS messages when executed via Step Function #11405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 22, 2024
Merged

Add MessageAttributes to SQS messages when executed via Step Function #11405

merged 1 commit into from
Aug 22, 2024

Conversation

gregfurman
Copy link
Contributor

@gregfurman gregfurman commented Aug 22, 2024 •
edited
Loading

Motivation

  • The Step Functions' SQS provider did not fully support the MessageAttributes field during a SendMessage since the initial implementation used the incorrect field name provided by the AWS docs.
  • This PR fixes the fieldname and adds a parity test to ensure a MessageAttributes JSON object and Md5OfMessageAttributes hash is correctly returned.
  • Raised by bug: stepfunction sends sqs Message without MessageAttributes #11109

Changes

  • Renames "MessageAttribute" => "MessageAttributes"
  • Normalises field "MD5OfMessageAttributes" => "Md5OfMessageAttributes" to ensure PascalCase for SFN output
  • Add new stepfunction template for SQS message with attributes

Testing

@gregfurman gregfurman force-pushed the add/sfn-sqs-message-attributes branch from 1674aff to f4e79be Compare August 22, 2024 10:49
@gregfurman gregfurman self-assigned this Aug 22, 2024
@gregfurman gregfurman added type: bug Bug report aws:stepfunctions AWS Step Functions semver: patch Non-breaking changes which can be included in patch releases labels Aug 22, 2024
@gregfurman gregfurman added this to the 3.7 milestone Aug 22, 2024
@gregfurman gregfurman marked this pull request as ready for review August 22, 2024 10:50
@github-actions GitHub Actions
Copy link

LocalStack Community integration with Pro

    2 files      2 suites   1h 36m 59s ⏱️
3 326 tests 2 932 ✅ 394 💤 0 ❌
3 328 runs  2 932 ✅ 396 💤 0 ❌

Results for commit f4e79be.

MEPalma
MEPalma approved these changes Aug 22, 2024
View reviewed changes
Copy link
Contributor

@MEPalma MEPalma left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gregfurman, LGTM!

Aside from the misleading docs which lead to this error, an underlying problem appears to be our generic API Action key normaliser (upstream of where you added this normalisation step): here it is not able to recognise MD5 is an acronym.
We should now incorporate this ad-hoc change, as it has already been applied to other scenarios in the SFN interpreter. This decision is further supported by the frequent occurrence of such issues. However, moving forward, we can stay mindful of this limitation and, with future evidence, consider developing a generalisable solution.

@gregfurman gregfurman merged commit f27739f into master Aug 22, 2024
41 of 42 checks passed
@gregfurman gregfurman deleted the add/sfn-sqs-message-attributes branch August 22, 2024 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joe4dev joe4dev Awaiting requested review from joe4dev

@dominikschubert dominikschubert Awaiting requested review from dominikschubert

@steffyP steffyP Awaiting requested review from steffyP

1 more reviewer

@MEPalma MEPalma MEPalma approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees

@gregfurman gregfurman

Labels
aws:stepfunctions AWS Step Functions semver: patch Non-breaking changes which can be included in patch releases type: bug Bug report
Projects
None yet
Milestone
3.7
Development

Successfully merging this pull request may close these issues.
2 participants
@gregfurman @MEPalma