Skip to content

fix flaky CLI tests after startup time improved #7996

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 29, 2023
Merged

fix flaky CLI tests after startup time improved #7996

merged 1 commit into from
Mar 29, 2023

Conversation

alexrashed
Copy link
Member

We recently experienced a flake with a CLI test on Python 3.11 in a downstream repo.
The recent changes with the init system further improved the startup time, so I guess the time has come, @thrau! 🚀 🥳

    def test_wait_timeout_raises_exception(self, runner, container_client):
        result = runner.invoke(cli, ["start", "-d"])
        assert result.exit_code == 0
        assert "starting LocalStack" in result.output
    
        result = runner.invoke(cli, ["wait", "-t", "0.5"])
        # one day this test will surely fail ;-)
>       assert result.exit_code != 0
E       assert 0 != 0
E        +  where 0 = <Result okay>.exit_code

I verified this locally on my machine with a pre-pulled community image (which is the image being used currently for the CLI tests):

$ localstack start -d && localstack wait -t 0.5 && echo $?

     __                     _______ __             __
    / /   ____  _________ _/ / ___// /_____ ______/ /__
   / /   / __ \/ ___/ __ `/ /\__ \/ __/ __ `/ ___/ //_/
  / /___/ /_/ / /__/ /_/ / /___/ / /_/ /_/ / /__/ ,<
 /_____/\____/\___/\__,_/_//____/\__/\__,_/\___/_/|_|

 💻 LocalStack CLI 1.4.1.dev20230215233822

[10:12:39] starting LocalStack in Docker mode 🐳                                                                                                                                                                                                                         
   ...
[10:12:40] detaching                                                                                                                                                                                                                                                      bootstrap.py:694
0

@alexrashed alexrashed requested a review from thrau March 29, 2023 08:22
@github-actions
Copy link

LocalStack integration with Pro

1 866 tests  ±0   1 675 ✔️ ±0   1h 36m 18s ⏱️ + 15m 15s
       1 suites ±0      191 💤 ±0 
       1 files   ±0          0 ❌ ±0 

Results for commit aa2c0f1. ± Comparison against base commit 5211924.

thrau
thrau approved these changes Mar 29, 2023
View reviewed changes
Copy link
Member

@thrau thrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't believe we reached this milestone 👯

We should maybe add to the backlog a task to add a proper test so we can enure that localstack wait actually works as expected and doesn't return immediately when localstack is started. we could do this by mounting a boot/startup init script into the container that adds a time.sleep somewhere.

@alexrashed alexrashed merged commit bcf01f3 into master Mar 29, 2023
@alexrashed alexrashed deleted the fix-cli-flake branch March 29, 2023 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thrau thrau thrau approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@alexrashed @thrau