Writing something like arr+1 never has anything to do with offsets in
numpy. + always means addition, and what's happening here is that emp+1
is adding 1 to each of the values in the emp array, and storing the
results in a temporary array. Then this temporary array gets overwritten by
the call to copyto, and then its reference count drops to zero and it
gets discarded.

Compare to

new_arr = emp + 1
np.copyto(new_arr, np.arange(3))

This code is exactly the same as far as numpy can tell - the same
operations are performed on the same arrays - but it makes perfect sense
now.

The only thing I can think of is, we could teach np.copyto to issue a
warning "assignment to temporary value" if its argument has a reference
count of 1, meaning it will be deallocated as soon as copyto returns? I
suppose that would be useful. Such assignments aren't necessarily harmful,
but they're never useful.

It looks like we could do the same for regular slice assignment too. I was
worried about the temporaries that python sometimes generates for in-place
calls triggering the warning accidentally, but it looks like the python
interpreter holds references to those, just like if they were real
variables. (As it has to I guess, to avoid the generic func call code
destroying the temporaries before returning.)
On 10 Dec 2012 00:05, "erg" notifications@github.com wrote:

I know that I should be using slice notation instead of np.copyto, but I
also feel like case #1 #1 should
throw an error.

The case that works:

import numpy as np
emp = np.empty(3)

emp is array([ 1.72723371e-077, 1.72723371e-077, 2.19630579e-314])

np.copyto(emp, np.arange(3))

emp is array([ 0., 1., 2.]) # great, just like I expect

Case #1:

import numpy as np
emp = np.empty(3)

emp is array([ 1.72723371e-077, 1.72723371e-077, 2.19630579e-314])

np.copyto(emp+1, np.arange(3)) # want an error here, acts like no-op or worse

emp is array([ 1.72723371e-077, 1.72723371e-077, 2.19630579e-314])

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2802.

Of course, views complicate this picture... At a minimum we should check
that the array owns its data before issuing any warnings. A more clever
implementation would check the reference count on .base as well, and issue
a warning for arrays which didn't own their data but for which all the
reference counts on the .base chain were also 1.
On 10 Dec 2012 00:57, njs@pobox.com wrote:

Writing something like arr+1 never has anything to do with offsets in
numpy. + always means addition, and what's happening here is that emp+1
is adding 1 to each of the values in the emp array, and storing the
results in a temporary array. Then this temporary array gets overwritten by
the call to copyto, and then its reference count drops to zero and it
gets discarded.

Compare to

new_arr = emp + 1
np.copyto(new_arr, np.arange(3))

This code is exactly the same as far as numpy can tell - the same
operations are performed on the same arrays - but it makes perfect sense
now.

The only thing I can think of is, we could teach np.copyto to issue a
warning "assignment to temporary value" if its argument has a reference
count of 1, meaning it will be deallocated as soon as copyto returns? I
suppose that would be useful. Such assignments aren't necessarily harmful,
but they're never useful.

It looks like we could do the same for regular slice assignment too. I was
worried about the temporaries that python sometimes generates for in-place
calls triggering the warning accidentally, but it looks like the python
interpreter holds references to those, just like if they were real
variables. (As it has to I guess, to avoid the generic func call code
destroying the temporaries before returning.)
On 10 Dec 2012 00:05, "erg" notifications@github.com wrote:

I know that I should be using slice notation instead of np.copyto, but I
also feel like case #1 #1 should
throw an error.

The case that works:

import numpy as np
emp = np.empty(3)

emp is array([ 1.72723371e-077, 1.72723371e-077, 2.19630579e-314])

np.copyto(emp, np.arange(3))

emp is array([ 0., 1., 2.]) # great, just like I expect

Case #1:

import numpy as np
emp = np.empty(3)

emp is array([ 1.72723371e-077, 1.72723371e-077, 2.19630579e-314])

np.copyto(emp+1, np.arange(3)) # want an error here, acts like no-op or worse

emp is array([ 1.72723371e-077, 1.72723371e-077, 2.19630579e-314])

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2802.

I forgot about slicing notation and was trying to find an api call to make it work and got confused. Maybe the docs for np.copyto should mention slice notation as an alternative or maybe I just need to step up my python game... Making this syntax throw an error does seem like a hard problem and probably isn't worth the effort. I can close this issue if you want.

Well, I think there is a real issue here, so I edited the bug description to describe a more general solution; perhaps someone will see it and be inspired to fix it :-). (The new description also has another plausible example of how this same basic issue can bite people in nasty ways, even if they aren't otherwise confused. And anyway I like software that I can successfully use even when I'm confused, since everyone gets confused sometimes...)

Just to note, the functions I can think of right now that could use this warning are:

• array assignments as in njsmith's example
• copyto and its friends: putmask, put and place
• array.sort() method
• np.random.shuffle

Since most functions in numpy (ie. ufuncs) return the output array, no warning is possible there.

Doh, of course, that's a good point about out= arguments. And it'd be nice
if we could avoid adding this quirky refcount-checking function to the
public multiarray API, so if the ufunc library doesn't need it then that's
nice...

Still trying to work out in my mind how this actually works. Like you say,
it's possible to pass a temporary as an out= argument and have it mean
something, but it still seems rather odd, yes? You need something like:

c = np.add(a, b, out=np.empty(10))

I'm not sure why someone would write this; is there any situation where
it's useful? it's exactly equivalent to

c = np.add(a, b, dtype=np.float64)

so arguably a warning is still a good idea. But it might still be a useful
construct for auto-generated code or something like that for some reason
I'm not quite seeing right now?

On Fri, Dec 14, 2012 at 1:50 PM, seberg notifications@github.com wrote:

Just to note, the functions I can think of right now that could use this
warning are:

• array assignments as in njsmith's example
• copyto and its friends: putmask, put and place
• array.sort() method
• np.random.shuffle

Since most functions in numpy (ie. ufuncs) return the output array, no
warning is possible there.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2802#issuecomment-11376744.

@njsmith didn't think enough about it maybe. It seems to me you are right, unless you want some very quirky memory order of the result (that cannot be specified with just order), I cannot see any reason to use that design, too. So it is probably so rare that it is actually wanted behaviour that numpy could warn anyway. Maybe the only way to find out is to see if people complain when you add warning to a development version :).

Can we close this? It seems there are edge cases to the edge cases in the logic necessary to detect assigning to temporary variables

No further response, closing. Feel free to reopen, but seems like this is not a useful issue anymore.

This still strikes me as a handy feature to implement - and since we were able to implement elision of temporaries using refcounts, it certainly seems possible that we could pull off the same trick here to emit warnings.

I'd argue we should reopen this, but it was certainly not a good first issue.