BUG: Fix uint->timedelta promotion to raise TypeError #16639
Conversation
| @@ -166,6 +166,9 @@ numpy/core/src/umath/simd.inc | |||
| numpy/core/src/umath/struct_ufunc_test.c | |||
| numpy/core/src/umath/test_rational.c | |||
| numpy/core/src/umath/umath_tests.c | |||
| numpy/core/src/umath/_umath_tests.dispatch.avx2.c | |||
| numpy/core/src/umath/_umath_tests.dispatch.h | |||
| numpy/core/src/umath/_umath_tests.dispatch.sse41.c | |||
There was a problem hiding this comment.
added a few files to gitignore, maybe got added as part of #13516 ?
| ``np.promote_types("uint64", "m8")`` aligns with | ||
| ``np.promote_types("m8", "uint64")`` now and both raise a TypeError. | ||
| Previously, ``np.promote_types("uint64", "m8")`` returned ``"m8"`` which | ||
| was considered a bug. |
There was a problem hiding this comment.
please let me know if this should go into a seperate file 16639.compatibility.rst
There was a problem hiding this comment.
Thanks, i think both of this and the above gitignore additions are fine here.
This affects not just uint64, but all unsigned integers, with only signed integers remaining to promote (for the time being).
There was a problem hiding this comment.
so uint32 works fine, since it already finds the casting mapping in _npy_type_promotion_table .
>>> np.promote_types("uint32", "m8")
dtype('<m8')
>>> np.promote_types("m8", "uint32")
dtype('<m8')
>>> np.promote_types("m8", "uint64")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: invalid type promotion
>>> np.promote_types("uint64", "m8")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: invalid type promotion
>>>
My understanding was that it should work fine for uint32, it should only fail for large uints since it maybe too big in value to be cast accurately ?
There was a problem hiding this comment.
Oh, ok, right, thanks for the explanation. Yes, I think it is fine here then.
I do think we probably should go all the way, and outright deprecate all integer+timedelta promotions. However, that is better for a new PR. We should also make the casting safety of these at least unsafe, probably even no-cast unless it has generic units. But both of those are different issues.
There was a problem hiding this comment.
I do think we probably should go all the way, and outright deprecate all integer+timedelta promotions.
Maybe i am missing something here, but why should this "not be allowed" if it can be cast accurately to timedelta ? Even if we had non generic units we can always convert the integer to the timedelta of that many units right ?
There was a problem hiding this comment.
Hmmm, I keep going in circles. I think the promotion should definitely not be allowed, because an integer and a timedelta represent completely different things. If we concatenate a timedelta64 and an integer array, then the result being a timedelta may be a misrepresentation of the integers. I think we can expect users to do the cast manually?
Casting safety is a bit of a different issue. Because they are completely different things, the casting should possibly not be considered "safe". This is what astropy.units do as far as I know. They refuse to cast, unless the value is unit-less.
On the other hand, there was the opinion that it could be considered "safe" if it will definitely round-trip correctly (which it does). Which is a good point. Possibly we need to add new casting safety parameters...
There was a problem hiding this comment.
Anyway, thanks for this! I will put this in, since it does not hurt for now and the asymmetric definition is not helpful to begin with.
Whether the other promotion cases should also be deprecated is a bit of a different issue, I am pretty sure I want that in the long run. And I am starting to be more and more happy with decoupling that from the type-safety discussion.
There was a problem hiding this comment.
Thanks Sebastian for the explanation and the merge !
Whether the other promotion cases should also be deprecated is a bit of a different issue, I am pretty sure I want that in the long run. And I am starting to be more and more happy with decoupling that from the type-safety discussion.
The reason I don't quite understand how we will be able to decouple type-safety from promotion is because the docs mention the following : """Returns the data type with the smallest size and smallest scalar kind to which both type1 and type2 may be safely cast.""" so, atleast according to documentation promotion depends on casting safety to a common type.
There was a problem hiding this comment.
Yeah, its the current rule, and it has its arguments. I am not quite sure on it yet.
I think I prefer the coupling version, with these type of casts just not being designated "safe".
There is things like strings. We currently promote string+number to string, which seems plain wrong to me. But we currently also consider it a safe cast.
The "safe" cast part could at least be motivated by the fact that you can convert any number into a string, and it will roundtrip back correctly. But I do not see how you can possibly motivate promotion to work. There is no operation which works the same on strings and integers, so storing both into a string array (e.g. concatenation) is just wrong. And I think for promotion to make sense, it should not be wrong.
Long story short: For numbers that rule is clean and clear. I am not really convinced it was ever quite thought when the rule was generalized to non-numbers.
Please see: #16592 (comment)