Skip to content

API: remove undocumented use of __array__(dtype, context) #15118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 16, 2020

Conversation

mattip
Copy link
Member

@mattip mattip commented Dec 16, 2019

In 59bd342 the possibility to add a second argument to __array__ was added, with no documentation and no clear use case. Over time the code was refactored, but that extra argument remained so that the added test would pass. I think we should remove it. I could not find use of this argument in dask, scipy, pandas, cython, or astropy.

On first glance, it might be prudent to completely remove context from PyArray_FromArrayAttr since it is no longer used. Unfortunately, the context argument percolates out of PyArray_FromArrayAttr to PyArray_GetArrayParamsFromObject to PyArray_FromAny, all of which are API functions so we cannot remove it.

On the other hand, that means I can use it for np.allow_object xref gh-15083

@mattip mattip added 03 - Maintenance 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes and removed 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes labels Dec 16, 2019
@seberg seberg added the 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. label Dec 16, 2019
@mattip
Copy link
Member Author

mattip commented Dec 16, 2019

I think there is another simililar thing in the argument parsing used by normal ufunc calls (currently inside PyUfunc_GenericFunction

Removed that too

@mattip
Copy link
Member Author

mattip commented Dec 16, 2019

Note that __array_wrap__ uses context and it is documented

@eric-wieser
Copy link
Member

eric-wieser commented Dec 16, 2019

I worry that this perhaps should go through a deprecation cycle. I agree we don't need it though, since __array_ufunc__ should cover the same functionality.

Either way, this will need to hit the mailing list, in case there really are users out there. We should probably tag the commit API not MAINT

@mattip mattip changed the title MAINT: remove undocumented use of __array__(dtype, context) API: remove undocumented use of __array__(dtype, context) Dec 16, 2019
@mattip mattip added the 63 - C API Changes or additions to the C API. Mailing list should usually be notified. label Dec 16, 2019
@mattip
Copy link
Member Author

mattip commented Dec 16, 2019

Mailing list link

@seberg
Copy link
Member

seberg commented Dec 23, 2019

Hmmm, I have been silly, right. I would be fine with not going through a deprecation here. But we can actually go through it, right? We can try with the context passed, and if that succeeds give a deprecation warning?

It is too bad, as it seems like just dead overhead to even build the context, but... should we go with a single release deprecation here?

@mattip mattip force-pushed the cleanup-array-call branch from 3073c37 to 4b1fc04 Compare January 13, 2020 19:13
@mattip mattip added the triage review Issue/PR to be discussed at the next triage meeting label Jan 13, 2020
@mattip mattip requested a review from seberg January 14, 2020 21:39
Copy link
Member

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, possibly the release notes should note that context is now ignored in PyArray_FromAny, and actually, this means that the documentation needs adapting:

The context argument is passed to the __array__ method of op and is only used if the array is constructed that way. Almost always this parameter is NULL.

Should be replaced with void *reserved the input is currently ignored (maybe with a versionchanged tag?), or just keep the name, but write that the input is not passed on anymore.

Same for: PyArray_FromArrayAttr, PyArray_HasArrayInterfaceType (probably no change needed) and PyArray_CheckFromAny (if you change the name) and PyArray_GetArrayParamsFromObject.

In any case, I am in favor of just putting it in, nobody ever answered on the list and things are just simpler if we give it a shot. Lets mention it very briefly on wednesday in case someone disagrees?

@mattip
Copy link
Member Author

mattip commented Jan 14, 2020

adjusting documentation.

@mattip
Copy link
Member Author

mattip commented Jan 15, 2020

We should make any use of context in the c-api an error.

EDIT: Other then that, I think we were fine with going ahead with error/changing the API (in the 15.01.2020 triage meeting call)

@seberg seberg added triaged Issue/PR that was discussed in a triage meeting and removed triage review Issue/PR to be discussed at the next triage meeting labels Jan 15, 2020
@mattip mattip force-pushed the cleanup-array-call branch from d3c95c0 to f38504c Compare January 15, 2020 21:24
@mattip
Copy link
Member Author

mattip commented Jan 15, 2020

Squashed to a single commit just in case we need to revert. Also actively check that the C-API use is always NULL, which exposed some calls in datetime_busday.c and datetime_busdaycal.c.

@seberg
Copy link
Member

seberg commented Jan 16, 2020

Thanks Matti, hopefully nobody will notice. If anyone does and finds this, please do comment, we are prepared to revert if necessary.

@jgillis
Copy link

jgillis commented Jun 24, 2020

👋 breakage of rather old and hacky code in casadi project here
Deprecation cycle would have been nice.
Let's see if this __array_ufunc__ saves the day..

https://xkcd.com/1172/

@eric-wieser
Copy link
Member

If it's a big deal, I suppose we could change this to a DeprecationWarning for 1.19.1

@mattip
Copy link
Member Author

mattip commented Jun 24, 2020

it seems my check for using context is somehow failing, since the issue over in casadi is not that the call to __array__ is raising, rather that it used to have a context argument and now no longer does.

@seberg
Copy link
Member

seberg commented Jun 24, 2020

Well, we stopped providing context in some cases (specifically ufunc calls), it would be good to know if the context was actually useful here for some reason. The Deprecation will only be on the part of others providing a context manually...

What is the return np.array([np.nan]) for, and how come you need __array_ufunc__ to fix this (as opposed to actually clean it up a lot)? The reason we removed it, is that it never really worked. For example I am pretty sure the mentioned code will fail for np.add(a, b, out=a) even though it should be identical to a += b.

@mattip mattip deleted the cleanup-array-call branch November 2, 2020 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
03 - Maintenance 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. 63 - C API Changes or additions to the C API. Mailing list should usually be notified. triaged Issue/PR that was discussed in a triage meeting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants