Skip to content

Commit 9211f70

Browse files
billyjacobsonbillyjacobson
authored
Merge branch 'master' into bigtable-scaler
2 parents 566a046 + 49fbc9c commit 9211f70

File tree

171 files changed

+746
-3058
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

171 files changed

+746
-3058
lines changed

‎.github/CODEOWNERS

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
/bigquery_storage/ @shollyman @GoogleCloudPlatform/python-samples-owners
1616
/bigtable/ @billyjacobson @GoogleCloudPlatform/python-samples-owners
1717
/blog/ @GoogleCloudPlatform/python-samples-owners
18-
/cdn/ @GoogleCloudPlatform/python-samples-owners
18+
/cdn/ @mpwarres @elithrar @GoogleCloudPlatform/python-samples-owners
1919
/cloud-sql/ @kvg @GoogleCloudPlatform/python-samples-owners
2020
/codelabs/ @GoogleCloudPlatform/python-samples-owners
2121
/composer/ @leahecole @GoogleCloudPlatform/python-samples-owners

‎.kokoro/python3.6/periodic.cfg

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,6 @@
1414

1515
# Format: //devtools/kokoro/config/proto/build.proto
1616

17-
timeout_mins: 420 # 7 hours
18-
1917
# Tell the trampoline which build file to use.
2018
env_vars: {
2119
key: "TRAMPOLINE_BUILD_FILE"

‎.kokoro/python3.7/periodic.cfg

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,6 @@
1414

1515
# Format: //devtools/kokoro/config/proto/build.proto
1616

17-
timeout_mins: 420 # 7 hours
18-
1917
# Tell the trampoline which build file to use.
2018
env_vars: {
2119
key: "TRAMPOLINE_BUILD_FILE"

‎.kokoro/python3.8/periodic.cfg

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,6 @@
1414

1515
# Format: //devtools/kokoro/config/proto/build.proto
1616

17-
timeout_mins: 420 # 7 hours
18-
1917
# Tell the trampoline which build file to use.
2018
env_vars: {
2119
key: "TRAMPOLINE_BUILD_FILE"

‎AUTHORING_GUIDE.md

Lines changed: 100 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,7 @@ SQLAlchemy==1.3.12
240240
If a sample has testing requirements that differ from its runtime requirements
241241
(such as dependencies on [pytest](http://pytest.org/en/latest/) or other
242242
testing libraries), the testing requirements may be listed in a separate
243-
`requirements-testing.txt` file instead of the main `requirements.txt` file.
243+
`requirements-test.txt` file instead of the main `requirements.txt` file.
244244

245245
### Region Tags
246246

@@ -285,6 +285,10 @@ for system testing, as shown in [this
285285
example](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/appengine/standard/localtesting/datastore_test.py).
286286
* All tests should be independent of one another and order-independent.
287287
* We use parallel processing for tests, so tests should be capable of running in parallel with one another.
288+
* Use pytest's fixture for resource setup and teardown, instead of
289+
having them in the test itself.
290+
* Avoid infinite loops.
291+
* Retry RPCs
288292

289293
### Arrange, Act, Assert
290294

@@ -320,10 +324,27 @@ that embodies assumptions about the behavior of the APIs.
320324
When tests need temporary resources (such as a temp file or folder), they
321325
should create reasonable names for these resources with a UUID attached to
322326
assure uniqueness. Use the Python ```uuid``` package from the standard
323-
library to generate UUIDs for resource names.
327+
library to generate UUIDs for resource names. For example:
324328

325-
All temporary resources should be explicitly deleted when
326-
testing is complete.
329+
```python
330+
glossary_id = 'test-glossary-{}'.format(uuid.uuid4())
331+
```
332+
333+
or:
334+
335+
```python
336+
# If full uuid4 is too long, use its hex representation.
337+
encrypted_disk_name = 'test-disk-{}'.format(uuid.uuid4().hex)
338+
```
339+
340+
```python
341+
# If the hex representation is also too long, slice it.
342+
encrypted_disk_name = 'test-disk-{}'.format(uuid.uuid4().hex[:5]
343+
```
344+
345+
All temporary resources should be explicitly deleted when testing is
346+
complete. Use pytest's fixture for cleaning up these resouces instead
347+
of doing it in test itself.
327348

328349
### Console Output
329350

@@ -333,16 +354,89 @@ is expected. Strive to verify the content of the output rather than the syntax.
333354
For example, the test might verify that a string is included in the output,
334355
without taking a dependency on where that string occurs in the output.
335356

336-
### Running tests
357+
### Avoid infinite loops
358+
359+
Never put potential infinite loops in the test code path. A typical
360+
example is about gRPC's LongRunningOperations. Make sure you pass the
361+
timeout parameter to the `result()` call.
362+
363+
Good:
364+
365+
```python
366+
# will raise google.api_core.GoogleAPICallError after 60 seconds
367+
operation.result(60)
368+
```
369+
370+
Bad:
371+
372+
```python
373+
operation.result() # this could wait forever.
374+
```
375+
376+
We recommend the timeout parameter to be around the number that gives
377+
you more than 90% success rate. Don't put too long a timeout.
378+
379+
Now this test is inevitably flaky, so consider marking the test as
380+
`flaky` as follows:
381+
382+
```python
383+
384+
@pytest.mark.flaky(max_runs=3, min_passes=1)
385+
def my_flaky_test():
386+
# test that involves LRO poling with the timeout
387+
```
388+
389+
This combination will give you very high success rate with fixed test
390+
execution time (0.999 success rate and 180 seconds operation wait time
391+
in the worst case in this example).
337392

393+
### Retry RPCs
394+
395+
All the RPCs are inevitably flaky. It can fail for many reasons. The
396+
`google-cloud` Python client retries requests automatically for most
397+
cases.
398+
399+
The old api-client doesn't retry automatically, so consider using
400+
[`backoff`](https://pypi.org/project/backoff/) for retrying. Here is a
401+
simple example:
402+
403+
```python
404+
405+
import backoff
406+
from googleapiclient.errors import HttpError
407+
408+
@pytest.fixture(scope='module')
409+
def test_resource():
410+
@backoff.on_exception(backoff.expo, HttpError, max_time=60)
411+
def create_resource():
412+
try:
413+
return client.projects().imaginaryResource().create(
414+
name=resource_id, body=body).execute()
415+
except HttpError as e:
416+
if '409' in str(e):
417+
# Ignore this case and get the existing one.
418+
return client.projects().imaginaryResource().get(
419+
name=resource_id).execute()
420+
else:
421+
raise
422+
423+
resource = create_resource()
424+
425+
yield resource
426+
427+
# cleanup
428+
...
429+
```
430+
431+
### Running tests
338432

339433
Automated testing for samples in `python-docs-samples` is managed by
340434
[nox](https://nox.readthedocs.io). Nox allows us to run a variety of tests,
341435
including the flake8 linter, Python 2.7, Python 3.x, and App Engine tests,
342436
as well as automated README generation.
343437

344438
__Note:__ As a temporary workaround, each project currently uses first
345-
`noxfile-template.py` found in a parent folder above the current sample. In
439+
`noxfile-template.py` found in a parent folder above the current sample. In
346440
order to simulate this locally, you need to copy + rename the parent
347441
`noxfile-template.py` as `noxfile.py` in the folder of the project you want to
348442
run tests.
@@ -373,7 +467,6 @@ To run a specific test from a specific following:
373467
nox -s py-3.7 -- snippets_test.py:test_list_blobs
374468
```
375469

376-
377470
### Test Environment Setup
378471

379472
Because all tests are system tests that use live resources, running tests

‎appengine/flexible/scipy/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@ Flask==1.1.2
22
gunicorn==20.0.4
33
imageio==2.8.0
44
numpy==1.18.3
5-
pillow==7.1.1
5+
pillow==7.1.2
66
scipy==1.4.1

‎appengine/flexible/storage/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
Flask==1.1.2
2-
google-cloud-storage==1.26.0
2+
google-cloud-storage==1.28.0
33
gunicorn==20.0.4; python_version > '3.0'
44
gunicorn==19.10.0; python_version < '3.0'

‎appengine/standard/firebase/firenotes/backend/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
Flask==1.1.2
22
pyjwt==1.7.1
33
flask-cors==3.0.8
4-
google-auth==1.14.0
4+
google-auth==1.14.1
55
requests==2.23.0
66
requests-toolbelt==0.9.1

‎appengine/standard/firebase/firetactoe/requirements-test.txt

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,2 @@
11
pytest==4.6.9
2-
gcp-devrel-py-tools==0.0.15
32
WebTest==2.0.34

‎appengine/standard/pubsub/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
Flask==1.1.2
2-
google-api-python-client==1.8.0
2+
google-api-python-client==1.8.2

0 commit comments

Comments
 (0)