swap out timeout for flaky (GoogleCloudPlatform#6539)

leahecole · web-flow · commit 9987c42547f1 · 2021-08-09T20:54:27.000Z
## Description We were seeing flaky test failures that I believe are just a quirk of having a long running service. Given that we have an overall test timeout in kokoro of 30 min, I felt that removing this timeout was appropriate and adding a flaky check is a better way to ensure these tests passing. Fixes GoogleCloudPlatform#6478 Fixes GoogleCloudPlatform#6469 Note: It's a good idea to open an issue first for discussion. ## Checklist - [x] I have followed [Sample Guidelines from AUTHORING_GUIDE.MD](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md) - [x] README is updated to include [all relevant information](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#readme-file) - [ ] **Tests** pass: `nox -s py-3.6` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup)) - [x] **Lint** pass: `nox -s lint` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup)) - [ ] These samples need a new **API enabled** in testing projects to pass (let us know which ones) - [ ] These samples need a new/updated **env vars** in testing projects set to pass (let us know which ones) - [x] Please **merge** this PR for me once it is approved. - [ ] This sample adds a new sample directory, and I updated the [CODEOWNERS file](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/.github/CODEOWNERS) with the codeowners for this sample
diff --git a/data-science-onramp/vertex-ai/requirements-test.txt b/data-science-onramp/vertex-ai/requirements-test.txt
@@ -1,2 +1,2 @@
 pytest==6.2.4
-pytest-timeout==1.4.2
+flaky==3.7.0
diff --git a/data-science-onramp/vertex-ai/sklearn_test.py b/data-science-onramp/vertex-ai/sklearn_test.py
@@ -82,7 +82,7 @@ def setup_teardown(
     aip_job_client.delete_custom_job(name=shared_state["model_name"]).result()
 
 
-@pytest.mark.timeout(1800)
+@pytest.mark.flaky(max_runs=3, min_passes=1)
 def test_sklearn(
     setup_teardown: Tuple[storage.bucket.Bucket, aip.JobServiceClient],
     shared_state: dict,
diff --git a/data-science-onramp/vertex-ai/tfkeras_test.py b/data-science-onramp/vertex-ai/tfkeras_test.py
@@ -83,7 +83,7 @@ def setup_teardown(
     aip_job_client.delete_custom_job(name=shared_state["model_name"]).result()
 
 
-@pytest.mark.timeout(1800)
+@pytest.mark.flaky(max_runs=3, min_passes=1)
 def test_tfkeras(
     setup_teardown: Tuple[storage.bucket.Bucket, aip.JobServiceClient],
     shared_state: dict,

Original file line number	Diff line number	Diff line change
`@@ -1,2 +1,2 @@`
`1`	`1`	`pytest==6.2.4`
`2`		`-pytest-timeout==1.4.2`
	`2`	`+flaky==3.7.0`