Could you please trigger a second build in this PR. I would like to check if the ccache statistics displayed at the end of the build show that ccache is effective with a high hit rate. If it's not the case it would be worth investigating.

@jjerphan do you recall why you needed to use mamba/conda-forge instead of pip/pypi.org to make it work for #20254? Was it a problem with the openblas version installed by pip?

@jjerphan do you recall why you needed to use mamba/conda-forge instead of pip/pypi.org to make it work for #20254? Was it a problem with the openblas version installed by pip?

Yes, it was (#20254 (comment)).

If we build in editable mode, we should use --no-build-isolation to avoid re-downloading the build dependencies.

But I think @glemaitre's point was to build / install without editable, then cd /tmp && pytest --pyargs sklearn to launch the test on the result of the installation.

@cmarmo: Thanks for your comment.

To give the context on this PR: patches (which have been extracted from #20254) were meant to use conda-forge via mamba instead of PyPi via pip to be able access the last OpenBLAS version for ARM64 because previous ones had problems for this architecture (#20254 (comment)).

But do we always want to use conda-forge instead of PyPi?

I also updated the PR description accordingly.

To give the context on this PR: patches (which have been extracted from #20254) were meant to use conda-forge via mamba instead of PyPi via pip to be able access the last OpenBLAS version for ARM64 because previous ones had problems for this architecture (#20254 (comment)).

Thanks @jjerphan. I've seen the references. Python packages are also all installed via mamba (conda) in this PR, not only OpenBLAS. I guess this way, installing from PyPi is never tested for arm64? Well, probably on travis, but travis will be discarded at some point?

The last Circle CI log show better but still low cache hit-rate:

+ ccache -s --verbose
Summary:
  Cache directory:    /home/circleci/.cache/ccache
  Primary config:     /home/circleci/.config/ccache/ccache.conf
  Secondary config:   /home/circleci/miniconda/envs/testenv/etc/ccache.conf
  Stats updated:      Thu Sep 30 14:09:40 2021
  Hits:                 15 /  126 (11.90 %)
    Direct:              6 /  126 (4.76 %)
    Preprocessed:        9 /  120 (7.50 %)
  Misses:              111
    Direct:            120
    Preprocessed:      111
  Uncacheable:         124
Primary storage:
  Hits:                 21 /  252 (8.33 %)
  Misses:              231
  Cache size (GB):    0.01 / 0.00
  Files:               231
Uncacheable:
  Called for linking:  118
  No input file:         6

I don't know what's wrong with the setup.

What ccache stats do you get if you rebuild scikit-learn from a new git clone in the same folder locally?

A bigger cache is populated and comes with a higher cache hit rate:

cache directory                     /home/jjerphan/.cache/ccache
primary config                      /home/jjerphan/.config/ccache/ccache.conf
secondary config (readonly)         /etc/ccache.conf
stats updated                       Fri Oct  1 12:05:25 2021
cache hit (direct)                   111
cache hit (preprocessed)               0
cache miss                           105
cache hit rate                     51.39 %
called for link                      120
no input file                          5
cleanups performed                     0
files in cache                       210
cache size                           6.5 MB

@thomasjpfan's proposal with some supplementary caching (b0572b7) solved the problem.

+ ccache -s --verbose
Summary:
  Cache directory:    /home/circleci/.cache/ccache
  Primary config:     /home/circleci/.config/ccache/ccache.conf
  Secondary config:   /home/circleci/miniconda/envs/testenv/etc/ccache.conf
  Stats updated:      Thu Oct  7 11:41:05 2021
  Hits:                 65 /  138 (47.10 %)
    Direct:             65 /  139 (46.76 %)
    Preprocessed:        0 /   74 (0.00 %)
  Misses:               73
    Direct:             74
    Preprocessed:       74
  Uncacheable:         123
Primary storage:
  Hits:                130 /  278 (46.76 %)
  Misses:              148
  Cache size (GB):    0.01 / 0.00
  Files:               146
Uncacheable:
  Called for linking:  118
  Compilation failed:    1
  No input file:         4

Build are faster by about 5 minutes (it took 18m 35s before but takes 13m 25s now).

@thomasjpfan merge?