The root cause of the problem seems to be described here: https://travis-ci.community/t/builds-hang-with-output-truncated-mid-line/7611/9 (currently reading the discussion).

Ok apparently my workaround works, for some reason I do not really understand...

Testing wheels used to take 961.99s before tuning parallelism using joblib.cpu_count(). Let's see if the last commit can improve upon this.

Having a look to the logs, the issue is apparently a stalled test, is it not?

Having a look to the logs, the issue is apparently a stalled test, is it not?

It could be but we cannot be sure because the std output buffer is truncated. And it often happens that the stalling happens when cibuildwheel is running basic pip install commands prior to running the tests themselves.

Others suspect in https://travis-ci.community/t/builds-hang-with-output-truncated-mid-line/7611/9 that the buffering of std output on travis linux/arm64 workers is actually the cause of the problem: because travis does not see output, it detects the build as stalled even though it is not.

I think parallelism is well tuned now. ~7 min for each build and they mostly happen in parallel. This is much faster than the circleci sequential build.

Let me re-enable the other CIs.

I am not sure which change helps the most to fix the original problem. I think the first time I got it to work prior to rebasing was after moving the pip install inside the test script. I am not sure if disabling Python buffering and removing the set -e mode in bash scripts is actually needed or not.

But anyways it seems to work and more importantly it is fast!

When squash-merging this PR, it would be nice to put [cd build] in the commit message so has to trigger a fake weekly build for linux/arm64 on the main branch and get the README.md badge green again.

The failure is unrelated and being addressed in #20963.

Just a question, have you tried the new CPU_COUNT without the || travis_terminate sentences?

I think I had one that worked with a lower value of CPU_COUNT. Unfortunately I rebased and it's no longer in the history of this PR.

Merged to make it easier to move forward with the release process.

@adrinjalali you might want to backport this to the 1.0.X branch to get linux/arm64 wheels generated in fast and robust way (hopefully).

tagging #20965