Skip to content

PEP 3141 numbers should be considered scalars #9691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 24, 2017
Merged

Conversation

jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Sep 15, 2017

The function numpy.isscalar() should recognize all PEP 3141 numbers as scalars.

The underlying issue is that the upgrade to SciPy 0.19.1 broke lil_matrix() with Sage Integers as input, because SciPy now uses numpy.isscalar() to validate certain inputs.

Downstream: https://trac.sagemath.org/ticket/23867

@eric-wieser
Copy link
Member

I'd argue that isscalar is generally broken, and np.ndim(x) == 0 is normally the correct test

@mhvk
Copy link
Contributor

mhvk commented Sep 15, 2017

@eric-wieser - since it is used in the wild, we still might want to ensure it works as advertised. Possibly, we can add np.ndim(x) == 0 as a final stanza in the if statement?

@jdemeyer - I think the PR as stands is useful already - but could you add a test case?
Also, in the example, I think one should use an actual Fraction rather than one with no arguments.

@eric-wieser
Copy link
Member

I guess my point is that I would consider it to be a bug in scipy that isscalar is being used (or at least, could construct a failing tests case due to its use).

@mhvk
Copy link
Contributor

mhvk commented Sep 15, 2017

@eric-wieser - but where do we document this? Doesn't it make more sense that isscalar works as advertised? (Or, more specifically, that it solves the problem scipy has - I'm not 100% sure I know what that is, though!)

Also, more practically, this PR would seem to be an improvement no matter what...

@jdemeyer
Copy link
Contributor Author

@jdemeyer - I think the PR as stands is useful already - but could you add a test case?

What do you mean with a test case? Isn't the Fraction() test sufficient?

@jdemeyer
Copy link
Contributor Author

...or do you mean that I should implement a new class based on numbers.Number for testing?

@jdemeyer
Copy link
Contributor Author

Tests changed as requested.

@mhvk
Copy link
Contributor

mhvk commented Sep 15, 2017

@jdemeyer - yes, I meant a test with Fraction, but written in the corresponding test file - core/tests/test_numeric.py. I should note that this is sufficiently old that it doesn't actually test isscalar at all... So, it would need a new class; I think it is fine to just add the same tests as you currently have in the docstring; no need to have a tiny PR lead to a major test writing excercise!

p.s. Sadly, tests in the docs are actually not run in numpy...

@jdemeyer
Copy link
Contributor Author

p.s. Sadly, tests in the docs are actually not run in numpy...

OK, that explains your earlier comment.

@charris
Copy link
Member

charris commented Sep 17, 2017

@jdemeyer You should do your work in a branch rather than master.

@jdemeyer
Copy link
Contributor Author

You should do your work in a branch rather than master.

You are certainly right in principle, but for a one-off pull request like this, that doesn't matter.

Are the tests OK now?

@kiwifb
Copy link
Contributor

kiwifb commented Oct 18, 2017

Anything else preventing this to be merged?

@eric-wieser
Copy link
Member

I have a feeling this might change the behaviour of np.linspace for the worse, since there the use of isscalar is actually "is this a builtin scalar that behaves predictably, and won't lose information by casting".

I'd be happier if we could remove all internal uses of isscalar before merging this.

@charris
Copy link
Member

charris commented Oct 21, 2017

This seems reasonable to me. @mhvk I don't think this causes a problem with linspace. Is it the case that numbers with units are not considered Numbers?

@mhvk
Copy link
Contributor

mhvk commented Oct 21, 2017

@charris - our Quantity still would never cause a True, so I don't expect any problems from that side. Overall, as said before, I think this is an improvement, and I think we should have it even if eventually we might like to deprecate isscalar.

@eric-wieser
Copy link
Member

My question here was whether np.linspace ends up behaving worse for the sage types this set out to fix

@eric-wieser
Copy link
Member

eric-wieser commented Oct 21, 2017

My primary objection here is that if we're going to change behavior of np.isscalar, we may as well do it just once, and change the implementation to

def isscalar(x):
    return np.ndim(x) == 0

or perhaps with less breakage,

def isscalar(x):
    # still refuses to consider 0d arrays scalars
    return not isinstance(x, np.ndarray) and np.asarray(x).ndim == 0

There are other bugs (#9898) that this would fix at the same time

@mhvk
Copy link
Contributor

mhvk commented Oct 21, 2017

Your simplest example would definitely break things for Quantity - the second might well be OK, but would probably be quite a bit slower.

Maybe more relevantly, I don't think this has to hold up this particular PR: it just allows for classes that have specifically stated they are number-like by registering as such -- which I think is hard to argue against.

Anyway, my suggestion would be (1) merge this PR; (2) open new issue that we should look carefully in our own code base what isscalar is actually used for (changing to ndim == 0 wherever appropriate); (3) afterwards, decide whether we need isscalar at all, how it should behave, and deprecate if need be.

@eric-wieser
Copy link
Member

look carefully in our own code base what isscalar is actually used for

The answer is:

  • That case in np.linspace
  • np.piecewise, which is probably easy to fix
  • np.histogramdd, which might be a little more work
  • A poorly-behaved np.ma.apply_along_axis (MAINT: make np.ma.apply_along_axis consistent with np.apply_along_axis #8511)
  • Code that we're only keeping around for compatibility anyway (best to leave?)
    • What is probably just an optimization in operator overloads in np.poly1d
    • Broken stuff in defmatrix that we probably can't change now anyway:
      >>> a[:,0]
      matrix([[ 1.],
              [ 0.]])
      >>> a[:,np.array(0)]
      matrix([[ 1.,  0.]])
      

@mhvk
Copy link
Contributor

mhvk commented Oct 21, 2017

well, it sounds like step (2) would be relatively easy! (In the meantime, this PR still fixes a real bug... even if it is in scipy.)

@eric-wieser
Copy link
Member

If @kiwifb can confirm that linspace still does the right thing on sage integers after this patch, then I suppose I'm +0 on this

@charris
Copy link
Member

charris commented Oct 24, 2017

Haven't heard back from @kiwifb . I am going to put this in, we can fix it later if things go wrong.

@charris charris merged commit 3e61cb4 into numpy:master Oct 24, 2017
@charris
Copy link
Member

charris commented Oct 24, 2017

Thanks @jdemeyer .

@kiwifb
Copy link
Contributor

kiwifb commented Oct 24, 2017

Sorry, was a bit busy. We have been using this patch (@jdemeyer wrote it for sage after all) for the last month or so and we haven't seen side effects.

@eric-wieser
Copy link
Member

eric-wieser commented Oct 24, 2017

So np.linspace(sage_int, sage_int) returns the same type as before? I actually meant to tag @jdemeyer there, you just made it sound like this was your patch in that comment!

@kiwifb
Copy link
Contributor

kiwifb commented Oct 24, 2017

I don't see a difference before and after.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants