Skip to content

[FEATURE] Automated PyPI Release #4318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Bibo-Joshi opened this issue Jun 22, 2024 · 1 comment · Fixed by #4364
Closed

[FEATURE] Automated PyPI Release #4318

Bibo-Joshi opened this issue Jun 22, 2024 · 1 comment · Fixed by #4364

Comments

@Bibo-Joshi
Copy link
Member

What kind of feature are you missing? Where do you notice a shortcoming of PTB?

The release process currently requires several manual steps, including

  • building the source & whl distributions
  • signing them & computing digests
  • uploading the artifacts to pypi
  • creating a GH release & uploading the artifacts there

Even though these steps are a comparatiely small part of the release process, I'd like to automate them.

Benefits that I see

  • reduces time required to do a release, which allows for more frequent releases (e.g. faster release after API changes are incorprated)
  • an automated build & deploy process doesn't require setup on a local machine such that in case of a maintainer switch, the new maintainer has less to set up
  • the time I currently spent on doing releases, I can then use for other work on PTB (or for a nap :))

Describe the solution you'd like

I would like to automate the steps mentioned above using PyPI’s trusted publishing via GitHub Actions as described in the official python packaging documentation. Recently I've tried this out with aiorem and I'm positively suprised about how well and easy that works.

let me list a few insights/comments I had:

  • this approach uses GitHub environments, which are a kind of restricted version of Actions. In particular, you can (and IISC must for usage with pypi) specify that the run must be manually approved by an admin such that you don't make releases by accident
  • the environment has a name that must be set in the pypi settings, making sure that only trusted environments can make releases
  • The Python Packaging Authority recommends to use Sigstore to sign binaries. Sigstore is used also for signing CPython and endorsed by github. the python sigstore even has a special subcommand to verify signatures generated via GitHub Actions. Using sigstore means that we don't even have to take care about our own signing keys anymore. Note that PyPI doesn't accept signatures anymore for quite some time, so the signatures would instead go to the GH releases page as they currently already do.
  • a bit care is needed in terms of setting up the triggers for the workflows. For one, pypi and test-pypi don't accept multiple uploads with the same version number so one should take not to run into that even for test uploads. Moreover, if you reject the manual approval of an environment run, it's marked as failed and that at least doesn't look nice in the commit status :D

Describe alternatives you've considered

One could use a manual approach to authenticate at PyPI and/or use manual GPG signatures as done so far, but both would require a custom approach that is likely to be less safe & less frequently updated than the GH Actions tools provided by PyPA and sigstore.

Additional context

To address the more time consuming part of the releases - namely the creation of changelogs - I'll try to open a second thread soonish. My goal is that ideally (almost) the complete release & deply process can be automated, maximising the benefits I lsited above :)

@harshil21
Copy link
Member

I approve of this. I'm all for automation!

@Bibo-Joshi Bibo-Joshi mentioned this issue Jul 9, 2024
6 tasks
@github-actions github-actions bot locked and limited conversation to collaborators Jul 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants