Skip to content

Conversation

MaanasArora
Copy link
Contributor

Fixes #29182.

Adds a promoter for first argument PyArray_PyLongDType to ldexp. It casts straight to double, not sure if that is more arbitrary than needed.

Thanks for reviewing!

@mattip
Copy link
Member

mattip commented Aug 20, 2025

What will happen with complex input?

@MaanasArora
Copy link
Contributor Author

@mattip this only promotes integer inputs (Python scalar integer, actually), so the rest would stay the same?

@seberg
Copy link
Member

seberg commented Aug 26, 2025

Thanks a for looking at this, this is very cool to see this use!

I am curious if we can push this further and actually use the promoter always. That is, we:

  • Set the first and last dtype to PyArray_CommonDType(first_dtype, AbstractFloatDType):
    • If the last was forced, we just set the first to that instead.
    • Yes, we ignore the second (integer) argument here considering how it is currently defined.
  • Set the second dtype to PyArray_CommonDType(second_dtype, int_dtype). That just seems like it always works in practice, so I don't really feel a need to do something more complicated.

@seberg seberg added the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Aug 26, 2025
@MaanasArora
Copy link
Contributor Author

MaanasArora commented Aug 27, 2025

Thanks, that extension makes a lot of sense!

I tried implementing this, but it causes a segmentation fault. It seems to be due to the fact that common_dtype is not defined on FloatPyArray_FloatAbstractDType (in the dtypemeta declaration). I'm wondering if the abstract classes support the common dtype operation yet? (I assume the python integers won't be the ones casting up to floats)

Edit: that does seem to be it; reverting the operation for the first dtype and setting the others in this way doesn't crash (though of course the promotion doesn't work properly)

@seberg
Copy link
Member

seberg commented Aug 27, 2025

Oh, hmmm, the pyfloat one should work ok probably, although the abstract version woukd be nicer.

@MaanasArora
Copy link
Contributor Author

PyFloat doesn't seem to work well oddly, but Float does. I've pushed that, but yeah it's not as nice. It could be interesting to look into supporting common_dtype more fully for the abstract types if you'd like!

@seberg
Copy link
Member

seberg commented Aug 28, 2025

Making the abstract and PyFloat do the same thing probably makes sense, but the fact that it doesn't work here seems like a bug somewhere that I (or we) need track down to make this work (and a couple of other cases, because we should use this for other float only functions).

@MaanasArora
Copy link
Contributor Author

MaanasArora commented Aug 29, 2025

To be clear it doesn't crash, it just doesn't promote further up than float16 so causes overflow.

In line 223-226 of abstractdtypes.c we do in float_common_dtype:

    else if (other == &PyArray_PyLongDType) {
        Py_INCREF(cls);
        return cls;
    }

This is equivalent to doing no promotion as we already know it's a float, but looking at NEP 50, this actually seems correct behavior - Python ints can promote to the lowest float (given we need a float) right? So for ldexp if we don't consider argument values, we would have to just use something that can fit all values?

@MaanasArora
Copy link
Contributor Author

MaanasArora commented Aug 29, 2025

Yes, we promote to DoubleDType when the other dtype is any legacy integer, but do not promote (keep the same class) when it is PyLong. That seems a bit inconsistent, so it is probably buggy in at least one case? Unless I'm missing something

if (NPY_DT_is_legacy(other) && other->type_num < NPY_NTYPES_LEGACY) {
if (other->type_num == NPY_BOOL || PyTypeNum_ISINTEGER(other->type_num)) {
/* Use the default integer for bools and ints: */
return NPY_DT_NewRef(&PyArray_DoubleDType);
}
}
else if (other == &PyArray_PyLongDType) {
Py_INCREF(cls);
return cls;
}

@seberg
Copy link
Member

seberg commented Aug 29, 2025

Ah, that makes sense (if I am getting the right picture). We are getting the abstract version as a result. In other paths that is OK because we only need a concrete dtype instance (via default_descr), but here we need to convert the abstract one to the concrete Float64 one.
I actually don't think I needed that before, it seems like a missing piece for generic promotion!
We could do it via the default_descr type, which not be so bad in practice, although it's reversed...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
00 - Bug 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: incorrect promotion for Python integer inputs in ldexp
3 participants