Skip to content

Ci/apply concurrency to workflow #1200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 34 additions & 35 deletions .github/workflows/cicd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ jobs:
validate:
uses: ./.github/workflows/validate.yml
needs: eval-changes
concurrency:
group: ${{ github.workflow }}-validate-${{ github.ref_name }}
cancel-in-progress: true
with:
python-versions-linux: '["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]'
python-versions-windows: '["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]'
Expand All @@ -81,10 +84,13 @@ jobs:
release:
name: Semantic Release
runs-on: ubuntu-latest
concurrency: push
needs: validate
if: ${{ needs.validate.outputs.new-release-detected == 'true' }}

concurrency:
group: ${{ github.workflow }}-release-${{ github.ref_name }}
cancel-in-progress: false

permissions:
contents: write

Expand All @@ -93,18 +99,21 @@ jobs:
GITHUB_ACTIONS_AUTHOR_EMAIL: actions@users.noreply.github.com

steps:
# Note: we need to checkout the repository at the workflow sha in case during the workflow
# the branch was updated. To keep PSR working with the configured release branches,
# we force a checkout of the desired release branch but at the workflow sha HEAD.
- name: Setup | Checkout Repository at workflow sha
# Note: We checkout the repository at the branch that triggered the workflow
# with the entire history to ensure to match PSR's release branch detection
# and history evaluation.
# However, we forcefully reset the branch to the workflow sha because it is
# possible that the branch was updated while the workflow was running. This
# prevents accidentally releasing un-evaluated changes.
- name: Setup | Checkout Repository on Release Branch
uses: actions/checkout@v4
with:
ref: ${{ github.ref_name }}
fetch-depth: 0
ref: ${{ github.sha }}

- name: Setup | Force correct release branch on workflow sha
- name: Setup | Force release branch to be at workflow sha
run: |
git checkout -B ${{ github.ref_name }}
git reset --hard ${{ github.sha }}

- name: Setup | Download Build Artifacts
uses: actions/download-artifact@v4
Expand All @@ -113,32 +122,32 @@ jobs:
name: ${{ needs.validate.outputs.distribution-artifacts }}
path: dist

- name: Evaluate | Determine Next Version
id: version
uses: ./
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
root_options: "-v --noop"

- name: Release | Bump Version in Docs
if: steps.version.outputs.released == 'true' && steps.version.outputs.is_prerelease == 'false'
if: needs.validate.outputs.new-release-is-prerelease == 'false'
env:
NEW_VERSION: ${{ steps.version.outputs.version }}
NEW_RELEASE_TAG: ${{ steps.version.outputs.tag }}
NEW_VERSION: ${{ needs.validate.outputs.new-release-version }}
NEW_RELEASE_TAG: ${{ needs.validate.outputs.new-release-tag }}
run: |
python -m scripts.bump_version_in_docs
git add docs/*

- name: Evaluate | Verify upstream has NOT changed
# Last chance to abort before causing an error as another PR/push was applied to the upstream branch
# while this workflow was running. This is important because we are committing a version change
shell: bash
run: bash .github/workflows/verify_upstream.sh

- name: Release | Python Semantic Release
id: release
uses: ./
uses: python-semantic-release/python-semantic-release@v9.20.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
root_options: "-v"
build: false

- name: Release | Add distribution artifacts to GitHub Release Assets
uses: python-semantic-release/publish-action@v9.20.0
if: steps.release.outputs.released == 'true'
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
tag: ${{ steps.release.outputs.tag }}
Expand Down Expand Up @@ -166,8 +175,9 @@ jobs:
git push -u origin "$MAJOR_VERSION_TAG" --force

outputs:
released: ${{ steps.release.outputs.released }}
tag: ${{ steps.release.outputs.tag }}
released: ${{ steps.release.outputs.released || 'false' }}
new-release-version: ${{ steps.release.outputs.version }}
new-release-tag: ${{ steps.release.outputs.tag }}


deploy:
Expand All @@ -187,19 +197,6 @@ jobs:
id-token: write # needed for PyPI upload

steps:
# Note: we need to checkout the repository at the workflow sha in case during the workflow
# the branch was updated. To keep PSR working with the configured release branches,
# we force a checkout of the desired release branch but at the workflow sha HEAD.
- name: Setup | Checkout Repository at workflow sha
uses: actions/checkout@v4
with:
fetch-depth: 1
ref: ${{ github.sha }}

- name: Setup | Force correct release branch on workflow sha
run: |
git checkout -B ${{ github.ref_name }}

- name: Setup | Download Build Artifacts
uses: actions/download-artifact@v4
id: artifact-download
Expand All @@ -210,6 +207,8 @@ jobs:
# see https://docs.pypi.org/trusted-publishers/
- name: Publish package distributions to PyPI
id: pypi-publish
uses: pypa/gh-action-pypi-publish@release/v1
uses: pypa/gh-action-pypi-publish@v1.12.4
with:
packages-dir: dist
print-hash: true
verbose: true
46 changes: 36 additions & 10 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,15 @@ on:
new-release-detected:
description: Boolean string result for if new release is available
value: ${{ jobs.build.outputs.new-release-detected }}
new-release-version:
description: Version string for the new release
value: ${{ jobs.build.outputs.new-release-version }}
new-release-tag:
description: Tag string for the new release
value: ${{ jobs.build.outputs.new-release-tag }}
new-release-is-prerelease:
description: Boolean string result for if new release is a pre-release
value: ${{ jobs.build.outputs.new-release-is-prerelease }}
distribution-artifacts:
description: Artifact Download name for the distribution artifacts
value: ${{ jobs.build.outputs.distribution-artifacts }}
Expand Down Expand Up @@ -91,17 +100,31 @@ jobs:
python -m pip install --upgrade pip setuptools wheel
pip install -e .[build]

- name: Build | Create the distribution artifacts
- name: Build | Build next version artifacts
id: version
uses: python-semantic-release/python-semantic-release@v9.20.0
with:
github_token: ""
root_options: "-v"
build: true
changelog: true
commit: false
push: false
tag: false
vcs_release: false

- name: Build | Annotate next version
if: steps.version.outputs.released == 'true'
run: |
printf '%s\n' "::notice::Next release will be '${{ steps.version.outputs.tag }}'"

- name: Build | Create non-versioned distribution artifact
if: steps.version.outputs.released == 'false'
run: python -m build .

- name: Build | Set distribution artifact variables
id: build
run: |
if new_version="$(semantic-release --strict version --print)"; then
printf '%s\n' "::notice::Next version will be '$new_version'"
printf '%s\n' "new_release_detected=true" >> $GITHUB_OUTPUT
semantic-release version --changelog --no-commit --no-tag
else
printf '%s\n' "new_release_detected=false" >> $GITHUB_OUTPUT
python -m build .
fi
printf '%s\n' "dist_dir=dist/*" >> $GITHUB_OUTPUT
printf '%s\n' "artifacts_name=dist" >> $GITHUB_OUTPUT

Expand All @@ -114,7 +137,10 @@ jobs:
retention-days: 2

outputs:
new-release-detected: ${{ steps.build.outputs.new_release_detected }}
new-release-detected: ${{ steps.version.outputs.released }}
new-release-version: ${{ steps.version.outputs.version }}
new-release-tag: ${{ steps.version.outputs.tag }}
new-release-is-prerelease: ${{ steps.version.outputs.is_prerelease }}
distribution-artifacts: ${{ steps.build.outputs.artifacts_name }}


Expand Down
33 changes: 33 additions & 0 deletions .github/workflows/verify_upstream.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/bash

set -eu +o pipefail

# Example output of `git status -sb`:
# ## master...origin/master [behind 1]
# M .github/workflows/verify_upstream.sh
UPSTREAM_BRANCH_NAME="$(git status -sb | head -n 1 | cut -d' ' -f2 | grep -E '\.{3}' | cut -d'.' -f4)"
printf '%s\n' "Upstream branch name: $UPSTREAM_BRANCH_NAME"

set -o pipefail

if [ -z "$UPSTREAM_BRANCH_NAME" ]; then
printf >&2 '%s\n' "::error::Unable to determine upstream branch name!"
exit 1
fi

git fetch "${UPSTREAM_BRANCH_NAME%%/*}"

if ! UPSTREAM_SHA="$(git rev-parse "$UPSTREAM_BRANCH_NAME")"; then
printf >&2 '%s\n' "::error::Unable to determine upstream branch sha!"
exit 1
fi

HEAD_SHA="$(git rev-parse HEAD)"

if [ "$HEAD_SHA" != "$UPSTREAM_SHA" ]; then
printf >&2 '%s\n' "[HEAD SHA] $HEAD_SHA != $UPSTREAM_SHA [UPSTREAM SHA]"
printf >&2 '%s\n' "::error::Upstream has changed, aborting release..."
exit 1
fi

printf '%s\n' "Verified upstream branch has not changed, continuing with release..."
68 changes: 59 additions & 9 deletions docs/automatic-releases/github-actions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -643,7 +643,7 @@ Examples
Common Workflow Example
-----------------------

The following is a common workflow example that uses both the Python Semantic Release Action
The following is a simple common workflow example that uses both the Python Semantic Release Action
and the Python Semantic Release Publish Action. This workflow will run on every push to the
``main`` branch and will create a new release upon a successful version determination. If a
version is released, the workflow will then publish the package to PyPI and upload the package
Expand All @@ -661,25 +661,69 @@ to the GitHub Release Assets as well.
jobs:
release:
runs-on: ubuntu-latest
concurrency: release
concurrency:
group: ${{ github.workflow }}-release-${{ github.ref_name }}
cancel-in-progress: false

permissions:
id-token: write
contents: write

steps:
# Note: we need to checkout the repository at the workflow sha in case during the workflow
# the branch was updated. To keep PSR working with the configured release branches,
# we force a checkout of the desired release branch but at the workflow sha HEAD.
- name: Setup | Checkout Repository at workflow sha
# Note: We checkout the repository at the branch that triggered the workflow
# with the entire history to ensure to match PSR's release branch detection
# and history evaluation.
# However, we forcefully reset the branch to the workflow sha because it is
# possible that the branch was updated while the workflow was running. This
# prevents accidentally releasing un-evaluated changes.
- name: Setup | Checkout Repository on Release Branch
uses: actions/checkout@v4
with:
ref: ${{ github.ref_name }}
fetch-depth: 0
ref: ${{ github.sha }}

- name: Setup | Force correct release branch on workflow sha
- name: Setup | Force release branch to be at workflow sha
run: |
git checkout -B ${{ github.ref_name }} ${{ github.sha }}
git reset --hard ${{ github.sha }}

- name: Evaluate | Verify upstream has NOT changed
# Last chance to abort before causing an error as another PR/push was applied to
# the upstream branch while this workflow was running. This is important
# because we are committing a version change (--commit). You may omit this step
# if you have 'commit: false' in your configuration.
#
# You may consider moving this to a repo script and call it from this step instead
# of writing it in-line.
shell: bash
run: |
set +o pipefail

UPSTREAM_BRANCH_NAME="$(git status -sb | head -n 1 | cut -d' ' -f2 | grep -E '\.{3}' | cut -d'.' -f4)"
printf '%s\n' "Upstream branch name: $UPSTREAM_BRANCH_NAME"

set -o pipefail

if [ -z "$UPSTREAM_BRANCH_NAME" ]; then
printf >&2 '%s\n' "::error::Unable to determine upstream branch name!"
exit 1
fi

git fetch "${UPSTREAM_BRANCH_NAME%%/*}"

if ! UPSTREAM_SHA="$(git rev-parse "$UPSTREAM_BRANCH_NAME")"; then
printf >&2 '%s\n' "::error::Unable to determine upstream branch sha!"
exit 1
fi

HEAD_SHA="$(git rev-parse HEAD)"

if [ "$HEAD_SHA" != "$UPSTREAM_SHA" ]; then
printf >&2 '%s\n' "[HEAD SHA] $HEAD_SHA != $UPSTREAM_SHA [UPSTREAM SHA]"
printf >&2 '%s\n' "::error::Upstream has changed, aborting release..."
exit 1
fi

printf '%s\n' "Verified upstream branch has not changed, continuing with release..."

- name: Action | Semantic Version Release
id: release
Expand All @@ -706,6 +750,11 @@ to the GitHub Release Assets as well.
one release job in the case if there are multiple pushes to ``main`` in a short period
of time.

Secondly the *Evaluate | Verify upstream has NOT changed* step is used to ensure that the
upstream branch has not changed while the workflow was running. This is important because
we are committing a version change (``commit: true``) and there might be a push collision
that would cause undesired behavior. Review Issue `#1201`_ for more detailed information.

.. warning::
You must set ``fetch-depth`` to 0 when using ``actions/checkout@v4``, since
Python Semantic Release needs access to the full history to build a changelog
Expand All @@ -721,6 +770,7 @@ to the GitHub Release Assets as well.
case, you will also need to pass the new token to ``actions/checkout`` (as
the ``token`` input) in order to gain push access.

.. _#1201: https://github.com/python-semantic-release/python-semantic-release/issues/1201
.. _concurrency: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idconcurrency

Version Overrides Example
Expand Down
Loading