Skip to content

[SFN] Improve responsiveness on shutdown #11596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 16, 2024
Merged

[SFN] Improve responsiveness on shutdown #11596

merged 3 commits into from
Oct 16, 2024

Conversation

MEPalma
Copy link
Contributor

@MEPalma MEPalma commented Sep 27, 2024

Motivation

I recently noticed that LocalStack is unable to terminate responsively to user input when Step Functions workflows are evaluating callback tasks. Upon closer inspection, I found that some daemon threads were not properly configured. I also took the opportunity to introduce more conservative behaviour for certain infinite threads, explicitly syncing their lifetimes to that of the state machine. This last change could also improve the responsiveness of stop API actions.

Changes

  • Ensure proper configuration of daemon threads and TMP_THREADS where relevant
  • Modify infinite loops to terminate when the associated state machine halts

@MEPalma MEPalma added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label Sep 27, 2024
@MEPalma MEPalma added this to the 3.8 milestone Sep 27, 2024
@MEPalma MEPalma self-assigned this Sep 27, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 27, 2024 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ±    0    2 suites  ±0   29m 33s ⏱️ - 1h 10m 56s
1 048 tests  - 2 444  890 ✅  - 2 189  158 💤  - 255  0 ❌ ±0 
1 050 runs   - 2 444  890 ✅  - 2 189  160 💤  - 255  0 ❌ ±0 

Results for commit e5ba884. ± Comparison against base commit 67f5698.

This pull request removes 2444 tests. 
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_lambda_dynamodb
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_opensearch_crud
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_search_books
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_setup
tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_destination_sns
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_infra
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_prefill_dynamodb_table
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input0-SUCCEEDED]
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input1-SUCCEEDED]
…

♻️ This comment has been updated with latest results.

@MEPalma MEPalma modified the milestones: 3.8, 4.0 Oct 1, 2024
@MEPalma MEPalma added semver: patch Non-breaking changes which can be included in patch releases semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases and removed semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases semver: patch Non-breaking changes which can be included in patch releases labels Oct 7, 2024
@localstack-bot
Copy link
Collaborator

Currently, only patch changes are allowed on master. Your PR labels (semver: minor) indicate that it cannot be merged into the master at this time.

@MEPalma MEPalma added semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases and removed semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases labels Oct 9, 2024
gregfurman
gregfurman approved these changes Oct 15, 2024
View reviewed changes
Copy link
Contributor

@gregfurman gregfurman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! One nit (that can be done in a follow-up) and a clarification question on program state.

@MEPalma MEPalma merged commit ed6f7ec into master Oct 16, 2024
34 checks passed
@MEPalma MEPalma deleted the MEP-sfn-fix_daemon_threads branch October 16, 2024 07:59
macnev2013 pushed a commit that referenced this pull request Oct 22, 2024
@MEPalma @macnev2013
Step Functions: Improve responsiveness on shutdown (#11596)
16000db
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gregfurman gregfurman gregfurman approved these changes

@joe4dev joe4dev Awaiting requested review from joe4dev joe4dev is a code owner

@dominikschubert dominikschubert Awaiting requested review from dominikschubert

Assignees

@MEPalma MEPalma

Labels
semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Milestone
4.0
Development

Successfully merging this pull request may close these issues.
3 participants
@MEPalma @localstack-bot @gregfurman