It would be nice to find a solution to array creation that can take advantage of NEP 18.

This is being looked at in NEP 35 numpy/numpy#14715

Thanks @thomasjpfan I'll take a look at that NEP

I think the failure is unrelated (SpectralCoclustering)

I think the failure is unrelated (SpectralCoclustering)

It should be fixed in master.

shape for empty_like requires nump 1.17, is this an issue?
https://docs.scipy.org/doc/numpy/reference/generated/numpy.empty_like.html

That will be a problem since our minimum requirements for numpy are 1.11.0

With #15106 it will move to 1.13.3.

Hmm, this is a blocker to move forward with nep13/18 support. Do you have any suggestions to be future compatible with numpy >= 1.17, for instance, try np.empty_like, but provide a fallback to np.empy for earlier versions. Is this generally accepted in sklearn?

Maybe @amueller has some ideas as well, I believe he has some interest in nep13/nep18 support.

I did an attempt to implement empty_like in fixes.py, it is less simple than I hoped. It works on numpy < 1.17 by forwarding to np.empy or np.empty_like depending on the shape parameter and the type of array. For >= 1.17, it simply forwards to np.empty_like.

The only loose end is order='K', but one could argue not to use that, and always specify 'C' or 'F' and maybe allow 'A'.

Are there _like for everything we need? And do we want to adopt that or wait what happens with numpy/numpy#14715 and NEP 37?

We will need dispatching for zeros_like and ones_like. There is no arange_like, right? Are there others that are missing? For random arrays we have the separate fix of allowing more general random states, but that requires some extra effort by the user, right?

From https://docs.scipy.org/doc/numpy/reference/routines.array-creation.html I can see
4: (zeros|full|ones|empty)_like.

I thought linspace could be an issue, but grep -n linspace sklearn/**/*.py | grep -v "test" revealed:

• QuantileTransformer: uses it for the quantiles, this shouldn't hurt too much.
• _hist_gradient_boosting/binning.py:73 the percentiles, should also be small.

I guess np.arange would be most common.

Apart from whatever NEP will be the best way forward, do you think it makes sense to put array creation functions in a separate module (maybe not fixed.py)?

If we adopt towards whatever NEP will succeed, maybe we should have a separate file just for helping implement the NEP, including array creation and other things (like scipy dispatch).

Though right now it looks like we mostly do a backport, which is somewhat different from writing helpers.

maybe we should have a separate file just for helping implement the NEP, including array creation and other things

so, for instance, a stripped-down version of empty_like in this PR, plus some others, possible a range_like, etc, is that what you have in mind?

grep -n order sklearn/**/*.py | grep -v "test" | grep 'K seems to confirm we don't need to support order='K'.

like scipy dispatch

Never heard of this, does scipy have it's own dispatch mechanism?

Never heard of this, does scipy have it's own dispatch mechanism?

Sorry for being cryptic. No, it does not. But we often use scipy.linalg and we need to figure out in what cases we can use the __array_function__ machinery instead and I assume we'll need some helpers around that.

But we often use scipy.linalg and we need to figure out in what cases we can use the __array_function__ machinery instead and I assume we'll need some helpers around that.

Indeed, I rely on monkey patching now, but it would be nice to for instance to try to use the NEP18 protocol for this.

I gave it another refactor, a separate module for array creation function (implemented (empty|zeros|ones)_like) which has a numpy like API.
I made the KBinsDiscretizer _BaseEncoder and PolynomialFeatures use these.

The downside of this approach is that it introduces another layer of indirection, which does not make it easier to read the source code. Using an API with the same name and similar behavior should help digest it I think.

Regarding the *_like functions, I'd just like to point out that I finally got numpy/numpy#16935 to implement NEP-35, which will add a like= argument to all NumPy array creation functions with similar behavior to that of empty_like (and derived functions), using __array_function__ as the dispatching mechanism. The PR only supports a few functions now, but hopefully it can be extended to most or all array creation functions in the coming weeks if there's no major setback in the implementation that would cause it to be rejected by NumPy maintainers and the community.

@ogrisel @betatim I wondering if this PR is still relevant or with the work of the array-API, we will use an alternative to use the proper namespace to create the array?

So closing as superseded by array API work.