Skip to content

Also upload a subset of nightly wheels #21637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Conversation

ianhi
Copy link
Contributor

@ianhi ianhi commented Nov 15, 2021

PR Summary

Add a second artifact upload to the wheels that build on every merge to main. This is a subset of the wheels upload but should be significantly smaller to enable fast downloading for 3rd party tests in CI.

Closes: #21635

@ianhi
Copy link
Contributor Author

ianhi commented Nov 15, 2021

ooooh or better would be something like pip install -i https://pypi.anaconda.org/scipy-wheels-nightly/simple matplotlib as in #21634

Copy link
Member

@QuLogic QuLogic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be uploading to SciPy's nightly channel instead. I think that's waiting on @tacaswell for some keys.

@tacaswell
Copy link
Member

Thank you for prodding me to make progress on this. Current state: I have sorted out who to ask and have (just) provided them with my anaconda.org user name. Once I have that set up I'll sort out which secrets I need to put where...

@greglucas
Copy link
Contributor

@ianhi, I like how you're thinking with all these PRs :)

I think you can add an on: schedule: cron to this file, and then on your new upload to scipy nightly step put in something like: if: ${{ github.event_name == 'schedule' }}

@tacaswell, It looks like most other projects don't upload wheels every night, so you might want to check and see what an acceptable/preferred cadence is there too.

@ianhi
Copy link
Contributor Author

ianhi commented Nov 16, 2021

@tacaswell, It looks like most other projects don't upload wheels every night, so you might want to check and see what an acceptable/preferred cadence is there too.

Yeah, something like weekly would also be totally fine that's still much faster than the mpl release cadence :). It would be handy if it would be well documented when this upload happened so that cron jobs that depend on that upload can happen a bit after.

e.g. this upload will always happen at ___ on sunday

@tacaswell
Copy link
Member

Update: I have the login/access, but have not generated the secrets yet (on my to-do for tomorrow). Consensus is nightly is OK.

@tacaswell tacaswell added this to the v3.6.0 milestone Nov 17, 2021
@tacaswell
Copy link
Member

There is now a token ANACONDA_ORG_UPLOAD_TOKEN which will be available on non-PRs in the GHA secrets.

@tacaswell
Copy link
Member

https://github.com/h5py/h5py/blob/master/ci/upload_wheels.sh <- this is how h5py is doing the upload.

@greglucas
Copy link
Contributor

We should also probably add some documentation about where these wheels can be found somewhere too once we upload them. Then this PR will close #9994

@jklymak jklymak marked this pull request as draft November 22, 2021 09:20
@jklymak
Copy link
Member

jklymak commented Nov 22, 2021

@ianhi, feel free to pop out of draft when ready...

@matthewfeickert
Copy link
Contributor

@ianhi 👋 Thanks for getting this PR going and for all the work that you've done on moving nightly wheels forward in general. 🙏

Now that @tacaswell has the ANACONDA_ORG_UPLOAD_TOKEN available as a repository secret and h5py has an upload recipe that can be adopted to a GHA workflow, do you think this is something that could move forward soon?

I can fully appreciate that you most probably have other works that is more pressing at the moment. So I would be happy to help get a GHA workflow for this.

(Apologies in advance to you and the maintainers for the noise if this is currently on hold for other reasons not documented in this PR.)

@ianhi
Copy link
Contributor Author

ianhi commented Mar 30, 2022

if this is currently on hold for other reasons not documented in this PR.

I think this is on hold because I forgot about it and then was traveling when jody sent a reminder and so forgot once more :(

So I would be happy to help get a GHA workflow for this.

by all means have at it :) - I'm actually traveling again on Friday and we know how that impacted things last time....

@matthewfeickert
Copy link
Contributor

I think this is on hold because I forgot about it and then was traveling when jody sent a reminder and so forgot once more :(

100% understand. Been there many many times. :)

by all means have at it :) - I'm actually traveling again on Friday and we know how that impacted things last time....

SGTM. 👍 I'll try to get something that looks reasonable on a branch on my fork going and then discuss it back on Issue #21635 before adding something else into the mix. Safe travels!

@ianhi
Copy link
Contributor Author

ianhi commented Mar 31, 2022

Closing in favor of #22733

@ianhi ianhi closed this Mar 31, 2022
@ianhi ianhi deleted the patch-3 branch March 31, 2022 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ENH]: Add a nightly wheel build
6 participants