Skip to content

DOC: clean up automated tests section of workflow docs #27696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ commands:
export git_log=$(git log --max-count=1 --pretty=format:"%B" | tr "\n" " ")
echo "Got commit message:"
echo "${git_log}"
if [[ -v CIRCLE_PULL_REQUEST ]] && ([[ "$git_log" == *"[skip circle]"* ]] || [[ "$git_log" == *"[circle skip]"* ]]); then
if [[ -v CIRCLE_PULL_REQUEST ]] && [[ $git_log =~ (\[skip circle\]|\[circle skip\]|\[skip doc\]|\[doc skip\]) ]]; then
echo "Skip detected, exiting job ${CIRCLE_JOB} for PR ${CIRCLE_PULL_REQUEST}."
circleci-agent step halt;
fi
Expand Down
2 changes: 2 additions & 0 deletions doc/devel/coding_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ consistency, and maintainability of the code base.
If adding new features, changing behavior or function signatures, or removing
public interfaces, please consult the :ref:`api_changes`.

.. _code-style:

PEP8, as enforced by flake8
===========================

Expand Down
116 changes: 77 additions & 39 deletions doc/devel/development_workflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -452,48 +452,86 @@ Automated tests
Whenever a pull request is created or updated, various automated test tools
will run on all supported platforms and versions of Python.

* Make sure the Linting, GitHub Actions, AppVeyor, CircleCI, and Azure
pipelines are passing before merging (All checks are listed at the bottom of
the GitHub page of your pull request). Here are some tips for finding the
cause of the test failure:

- If *Linting* fails, you have a code style issue, which will be listed
as annotations on the pull request's diff.
- If *Mypy* or *Stubtest* fails, you have inconsistency in type hints, which
will be listed as annotations in the diff.
- If a GitHub Actions or AppVeyor run fails, search the log for ``FAILURES``.
The subsequent section will contain information on the failed tests.
- If CircleCI fails, likely you have some reStructuredText style issue in
the docs. Search the CircleCI log for ``WARNING``.
- If Azure pipelines fail with an image comparison error, you can find the
images as *artifacts* of the Azure job:

- Click *Details* on the check on the GitHub PR page.
- Click *View more details on Azure Pipelines* to go to Azure.
- On the overview page *artifacts* are listed in the section *Related*.


* Codecov and CodeQL are currently for information only. Their failure is not
necessarily a blocker.

* tox_ is not used in the automated testing. It is supported for testing
locally.

.. _tox: https://tox.readthedocs.io/

* If you know only a subset of CIs need to be run, this can be controlled on
individual commits by including the following substrings in commit messages:

- ``[ci doc]``: restrict the CI to documentation checks. For when you only
changed documentation (this skip is automatic if the changes are only under
``doc/`` or ``galleries/``).
- ``[skip circle]``: skip the documentation build check. For when you didn't
change documentation.
- Unit tests can be turned off for individual platforms with

- ``[skip actions]``: GitHub Actions
- ``[skip appveyor]`` (must be in the first line of the commit): AppVeyor
- ``[skip azp]``: Azure Pipelines
* Codecov and CodeQL are currently for information only. Their failure is not
necessarily a blocker.

- ``[skip ci]``: skip all CIs. Use this only if you know your changes do not
need to be tested at all, which is very rare.
Make sure the Linting, GitHub Actions, AppVeyor, CircleCI, and Azure pipelines are
passing before merging. All checks are listed at the bottom of the GitHub page of your
pull request.

.. list-table::
:header-rows: 1
:stub-columns: 1
:widths: 20 20 60

* - Name
- Check
- Tips for finding cause of failure
* - Linting
- :ref:`code style <code-style>`
- Errors are displayed as annotations on the pull request diff.
* - | Mypy
| Stubtest
- :ref:`static type hints <type-hints>`
- Errors are displayed as annotations on the pull request diff.
* - CircleCI
- :ref:`documentation build <writing-rest-pages>`
- Search the CircleCI log for ``WARNING``.
* - | GitHub Actions
| AppVeyor
| Azure pipelines
- :ref:`tests <testing>`
- | Search the log for ``FAILURES``. Subsequent section should contain information
on failed tests.
|
| On Azure, find the images as *artifacts* of the Azure job:
| 1. Click *Details* on the check on the GitHub PR page.
| 2. Click *View more details on Azure Pipelines* to go to Azure.
| 3. On the overview page *artifacts* are listed in the section *Related*.

Skip CI checks
--------------

If you know only a subset of CI checks need to be run, you can skip unneeded CI checks
on individual commits by including the following strings in the commit message:

.. list-table::
:header-rows: 1
:stub-columns: 1
:widths: 25 20 55

* - String
- Effect
- Notes
* - ``[ci doc]``
- Only run documentation checks.
- | For when you have only changed documentation.
| ``[ci doc]`` is applied automatically when the changes are only to files in
``doc/**/`` or ``galleries/**/``
* - ``[skip doc]``
- Skip documentation checks.
- For when you didn't change documentation.
* - ``[skip appveyor]``
- Skip AppVeyor run.
- Substring must be in first line of commit message.
* - ``[skip azp]``
- Skip Azure Pipelines.
-
* - ``[skip actions]``
- Skip GitHub Actions.
-
* - ``[skip ci]``
- Skip all CI checks.
- Use only for changes where documentation checks and unit tests do not apply.


``[skip actions]`` and ``[skip ci]`` only skip Github Actions CI workflows that are
triggered on ``on: push`` and ``on: pull_request`` events. For more information,
see `Skipping workflow runs`_.

.. _`Skipping workflow runs`: https://docs.github.com/en/actions/managing-workflow-runs/skipping-workflow-runs