Skip to content

feat: add python 3.11 testing #8872

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 9, 2023
Merged

feat: add python 3.11 testing #8872

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8dc7382
feat: add python 3.11 testing
parthea Jan 6, 2023
18c9080
update dependency scipy to 1.10.0
parthea Jan 6, 2023
3d32fa9
update dependency psycopg2-binary to 2.9.5
parthea Jan 6, 2023
42619db
exclude python 3.11 for appengine/standard_python3/bigquery
parthea Jan 6, 2023
0ebc55c
exclude python 3.11 for tests that use google-cloud-bigquery==2.34.4
parthea Jan 6, 2023
da4baff
Merge branch 'main' into add-python-311
parthea Jan 9, 2023
d3bbf29
update copyright year
parthea Jan 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .kokoro/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ RUN set -ex \
&& export GNUPGHOME="$(mktemp -d)" \
&& echo "disable-ipv6" >> "${GNUPGHOME}/dirmngr.conf" \
&& /tmp/fetch_gpg_keys.sh \
&& for PYTHON_VERSION in 2.7.18 3.7.12 3.8.13 3.9.13 3.10.5 3.11.1; do \
&& for PYTHON_VERSION in 2.7.18 3.7.16 3.8.16 3.9.16 3.10.9 3.11.1; do \
wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" \
&& wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" \
&& gpg --batch --verify python-${PYTHON_VERSION}.tar.xz.asc python-${PYTHON_VERSION}.tar.xz \
Expand Down
66 changes: 66 additions & 0 deletions .kokoro/python3.11/common.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Format: //devtools/kokoro/config/proto/build.proto

timeout_mins: 300

# Configure the docker image for kokoro-trampoline.
env_vars: {
key: "TRAMPOLINE_IMAGE"
value: "gcr.io/cloud-devrel-kokoro-resources/python-samples-testing-docker"
}

# Download trampoline resources.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-docs-samples/.kokoro/trampoline_v2.sh"

# Download secrets from Cloud Storage.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"

# Access btlr binaries used in the tests
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/btlr"

# Copy results for Resultstore
action {
define_artifacts {
regex: "**/*sponge_log.xml"
}
}

# Specify which tests to run
env_vars: {
key: "RUN_TESTS_SESSION"
value: "py-3.11"
}

# Declare build specific Cloud project. It still uses the common one,
# but we'll update the value once we have more Cloud projects.
env_vars: {
key: "BUILD_SPECIFIC_GCLOUD_PROJECT"
value: "python-docs-samples-tests-311"
}

# Number of test workers.
env_vars: {
key: "NUM_TEST_WORKERS"
value: "10"
}

env_vars: {
key: "TRAMPOLINE_DOCKERFILE"
value: ".kokoro/docker/Dockerfile"
}
21 changes: 21 additions & 0 deletions .kokoro/python3.11/continuous.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Format: //devtools/kokoro/config/proto/build.proto

# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: ".kokoro/tests/run_tests_diff_head.sh"
}
32 changes: 32 additions & 0 deletions .kokoro/python3.11/periodic.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Format: //devtools/kokoro/config/proto/build.proto

# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: ".kokoro/tests/run_tests.sh"
}

env_vars: {
key: "REPORT_TO_BUILD_COP_BOT"
value: "true"
}

# Tell Trampoline to upload the Docker image after successfull build.
env_vars: {
key: "TRAMPOLINE_IMAGE_UPLOAD"
value: "true"
}
21 changes: 21 additions & 0 deletions .kokoro/python3.11/presubmit.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Format: //devtools/kokoro/config/proto/build.proto

# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: ".kokoro/tests/run_tests_diff_main.sh"
}
20 changes: 12 additions & 8 deletions MAC_SETUP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,21 +76,25 @@ test their code.
```console
$ pyenv install 3.6.13
```
* 3.7.13 (latest 3.7.x release)
* 3.7.16 (latest 3.7.x release)
```console
$ pyenv install 3.7.13
$ pyenv install 3.7.16
```
* 3.8.13 (latest 3.8.x release)
* 3.8.16 (latest 3.8.x release)
```console
$ pyenv install 3.8.13
$ pyenv install 3.8.16
```
* 3.9.12 (latest 3.9.x release)
* 3.9.16 (latest 3.9.x release)
```console
$ pyenv install 3.9.12
$ pyenv install 3.9.16
```
* 3.10.4 (latest 3.10.x release)
* 3.10.9 (latest 3.10.x release)
```console
$ pyenv install 3.10.4
$ pyenv install 3.10.9
```
* 3.11.1 (latest 3.11.x release)
```console
$ pyenv install 3.11.1
```

> ℹ️ *Note*: If you are using an M1 Mac,
Expand Down
4 changes: 3 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Python samples for [Google Cloud Platform products][cloud].

[![Build Status][py-2.7-shield]][py-2.7-link] [![Build Status][py-3.7-shield]][py-3.7-link] [![Build Status][py-3.8-shield]][py-3.8-link] [![Build Status][py-3.9-shield]][py-3.9-link] [![Build Status][py-3.10-shield]][py-3.10-link]
[![Build Status][py-2.7-shield]][py-2.7-link] [![Build Status][py-3.7-shield]][py-3.7-link] [![Build Status][py-3.8-shield]][py-3.8-link] [![Build Status][py-3.9-shield]][py-3.9-link] [![Build Status][py-3.10-shield]][py-3.10-link] [![Build Status][py-3.11-shield]][py-3.11-link]

## Google Cloud Samples

Expand Down Expand Up @@ -77,3 +77,5 @@ Contributions welcome! See the [Contributing Guide](CONTRIBUTING.md).
[py-3.9-link]: https://storage.googleapis.com/cloud-devrel-public/python-docs-samples/badges/py-3.9.html
[py-3.10-shield]: https://storage.googleapis.com/cloud-devrel-public/python-docs-samples/badges/py-310.svg
[py-3.10-link]: https://storage.googleapis.com/cloud-devrel-public/python-docs-samples/badges/py-3.10.html
[py-3.11-shield]: https://storage.googleapis.com/cloud-devrel-public/python-docs-samples/badges/py-311.svg
[py-3.11-link]: https://storage.googleapis.com/cloud-devrel-public/python-docs-samples/badges/py-3.11.html
2 changes: 1 addition & 1 deletion appengine/flexible/django_cloudsql/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Django==4.0.4; python_version >= '3.8'
Django==3.2.14; python_version < '3.8'
gunicorn==20.1.0
psycopg2-binary==2.9.4
psycopg2-binary==2.9.5
django-environ==0.9.0
google-cloud-secret-manager==2.12.0
django-storages[google]==1.13
2 changes: 1 addition & 1 deletion appengine/flexible/scipy/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.6"],
"ignored_versions": ["2.7", "3.6", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion appengine/flexible/scipy/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ imageio==2.14.0
numpy==1.22.0; python_version > "3.7"
numpy==1.21.4; python_version <= "3.7"
pillow==9.2.0
scipy==1.8.0; python_version > "3.7"
scipy==1.10.0; python_version > "3.7"
scipy==1.7.3; python_version <= "3.7"
4 changes: 2 additions & 2 deletions appengine/standard/noxfile-template.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,10 @@ def get_pytest_env_vars():

# DO NOT EDIT - automatically generated.
# All versions used to tested samples.
ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8", "3.9", "3.10"]
ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]

# Any default versions that should be ignored.
IGNORED_VERSIONS = ["3.6", "3.7", "3.8", "3.9", "3.10"]
IGNORED_VERSIONS = ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]

TESTED_VERSIONS = sorted([v for v in ALL_VERSIONS if v not in IGNORED_VERSIONS])

Expand Down
2 changes: 1 addition & 1 deletion appengine/standard_python3/bigquery/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
# There's no google-cloud-bigquery package for Python 3.9.
"ignored_versions": ["2.7", "3.6", "3.9"],
"ignored_versions": ["2.7", "3.6", "3.9", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion appengine/standard_python3/cloud_debugger/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.6", "3.10"],
"ignored_versions": ["2.7", "3.6", "3.10", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion appengine/standard_python3/cloudsql/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
flask==2.1.0
# psycopg2==2.8.4
psycopg2-binary==2.9.4 # you will need either the binary or the regular - for more info see http://initd.org/psycopg/docs/install.html
psycopg2-binary==2.9.5 # you will need either the binary or the regular - for more info see http://initd.org/psycopg/docs/install.html
PyMySQL==1.0.2
SQLAlchemy==1.4.38
2 changes: 1 addition & 1 deletion appengine/standard_python3/django/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Django==4.0.4; python_version >= '3.8'
Django==3.2.14; python_version < '3.8'
django-environ==0.9.0
psycopg2-binary==2.9.4
psycopg2-binary==2.9.5
google-cloud-secret-manager==2.12.0
2 changes: 1 addition & 1 deletion bigquery/bqml/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
# Skipping for Python 3.9 due to pyarrow compilation failure.
"ignored_versions": ["2.7", "3.6", "3.9"],
"ignored_versions": ["2.7", "3.6", "3.9", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion bigquery/datalab-migration/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
# Skipping 3.6 and 3.7 to avoid needing a vulnerable version of IPython
"ignored_versions": ["2.7", "3.6", "3.7"],
"ignored_versions": ["2.7", "3.6", "3.7", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion bigquery/pandas-gbq-migration/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
# Skipping for Python 3.9 due to pyarrow compilation failure.
"ignored_versions": ["2.7", "3.6", "3.9"],
"ignored_versions": ["2.7", "3.6", "3.9", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion cloud-sql/mysql/client-side-encryption/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.6", "3.10"],
"ignored_versions": ["2.7", "3.6", "3.10", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": True,
Expand Down
2 changes: 1 addition & 1 deletion cloud-sql/postgres/client-side-encryption/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.6", "3.10"],
"ignored_versions": ["2.7", "3.6", "3.10", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": True,
Expand Down
2 changes: 1 addition & 1 deletion cloud-sql/sql-server/client-side-encryption/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.6", "3.10"],
"ignored_versions": ["2.7", "3.6", "3.10", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion composer/2022_airflow_summit/constraints.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ prompt-toolkit==3.0.30
proto-plus==1.19.6
protobuf==3.20.0
psutil==5.9.1
psycopg2-binary==2.9.3
psycopg2-binary==2.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pure-sasl==0.6.2
Expand Down
2 changes: 1 addition & 1 deletion composer/2022_airflow_summit/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
'ignored_versions': ["2.7", "3.6", "3.7", "3.9", "3.10"], # Composer w/ Airflow 2 only supports Python 3.8
'ignored_versions': ["2.7", "3.6", "3.7", "3.9", "3.10", "3.11"], # Composer w/ Airflow 2 only supports Python 3.8
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion composer/airflow_1_samples/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
# Skipping for Python 3.9 due to numpy compilation failure.
"ignored_versions": ["2.7", "3.9", "3.10"],
"ignored_versions": ["2.7", "3.9", "3.10", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion composer/blog/gcp-tech-blog/data-orchestration-with-composer/constraints.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ prompt-toolkit==3.0.30
proto-plus==1.19.6
protobuf==3.20.0
psutil==5.9.1
psycopg2-binary==2.9.3
psycopg2-binary==2.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pure-sasl==0.6.2
Expand Down
3 changes: 2 additions & 1 deletion composer/blog/gcp-tech-blog/data-orchestration-with-composer/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@
"3.6",
"3.7",
"3.9",
"3.10"
"3.10",
"3.11",
], # Composer w/ Airflow 2 only supports Python 3.8
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
Expand Down
2 changes: 1 addition & 1 deletion composer/blog/gcp-tech-blog/unit-test-dags-cloud-build/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
# Skipping for Python 3.9 due to numpy compilation failure.
"ignored_versions": ["2.7", "3.9", "3.10"],
"ignored_versions": ["2.7", "3.9", "3.10", "3.11"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": False,
Expand Down
2 changes: 1 addition & 1 deletion composer/cicd_sample/constraints.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ prompt-toolkit==3.0.30
proto-plus==1.19.6
protobuf==3.20.0
psutil==5.9.1
psycopg2-binary==2.9.3
psycopg2-binary==2.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pure-sasl==0.6.2
Expand Down
1 change: 1 addition & 0 deletions composer/cicd_sample/noxfile_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"3.7",
"3.9",
"3.10",
"3.11",
], # Composer w/ Airflow 2 only supports Python 3.8
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
Expand Down
Loading