Is there any way for the repr of a bytes or Unicode scalar to include the actual size?

Is there any way for the repr of a bytes or Unicode scalar to include the actual size?

There is no point right now:

a = np.array(["1"], dtype="S100")
a[()].dtype

is still "S1", the length is not currently preserved when creating a scalar. If we had, that I guess we would also have np.bytes_("a", length=100).

Good point, thanks.

One can create instances with padded zeros directly:

In [23]: s = np.str_('abc\0\0')

In [24]: s.dtype
Out[24]: dtype('<U5')

In [25]: s
Out[25]: 'abc'

In that case, it would be nice for the repr to show the actual storage size, but I don't know if that possibility is enough incentive to do so. Perhaps it could be displayed only when there are padded zeros. E.g. repr(s) is "np.str_('abc', length=5)", but for a = np.str_('abc'), repr(a) is "np.str_('abc')". (Of course, then it would also be nice for str_ to actually accept a length argument.)

Will this affect the repr of a structured data type?

E.g.

In [66]: dt = np.dtype([('x', np.float32), ('code', 'S4'), ('flag', np.bool_)])

In [67]: a = np.array((1.25, 'ABC', False), dtype=dt)[()]

In [68]: a
Out[68]: (1.25, b'ABC', False)

Would repr(a) change to "(np.float32(1.25), np.bytes_(b'ABC'), np.False_)"? I'm not implying that it should; I'm just interested in understanding the extent of the proposed changes.

Like @WarrenWeckesser, curious about np.record -- ideally, it no longer pretends to be a tuple!

I like the NEP otherwise; do include the strings too.

Hmm, I had forgotten about structured, after thinking we print np.void. My first thought would be to print them as:

np.void((x, z, y), dtype=...)

since that also gives the field names. Do you have a different suggestion? We would have to add that as a valid constructor, but that should not be hard.

About strings, I had not thought about the fact that it is possible to create larger strings at all. I suppose that might be something to change: If the string is longer, include the length as length=8 or dtype="S8", users would see that practically never right now.

Yes, I like the suggested np.void((x, z, y), dtype=...). And it would be good to be able to construct those directly. For reference, right now the docstring is:

Init signature: np.void(self, /, *args, **kwargs)
Docstring:     
Either an opaque sequence of bytes, or a structure.

>>> np.void(b'abcd')
void(b'\x61\x62\x63\x64')

Structured `void` scalars can only be constructed via extraction from :ref:`structured_arrays`:

>>> arr = np.array((1, 2), dtype=[('x', np.int8), ('y', np.int8)])
>>> arr[()]
(1, 2)  # looks like a tuple, but is `np.void`

:Character code: ``'V'``

Is there any way the name void can be changed to something more descriptive? Maybe struct, or record (at least for the case where the type actually is a structured data type). void has always seemed like a bad name. Perhaps split void into two distinct things: blob (or something like that) for the opaque bytes case, and struct for the structured data type. (I haven't searched, but I wouldn't be surprised if this has been discussed before.)

This is getting a bit unwieldy. I think right now things are fine, because I believe Python always prints str and repr for its numbers the same (so we can use str instead of repr in many case).

I am hoping to discuss this in the call on wednesday a bit, maybe you can come?

For the question of structured dtypes, my current angle would be that I am not sure how salvageable np.void is. Rather the real solution to me is probably to create a new "structured dtype" (in NumPy proper). That could clean-slate fix some things and also consider whether structured_arr == structured_arr should (for example) not return a boolean array with the structure preserved.
But, given that, the question is if tweaking the representation a bit is even worthwhile if we have to think about it a lot...

@seberg - indeed, let the perfect not be the enemy of the good. For the NEP, maybe the most important thing is to mention structured types and postponing a decision on those...

I thought this would be a relatively straight forward nut to start cracking on 🤦... But, in some cases we just use the repr to print arrays, and if we bloat the repr then we bloat array printing and need new special cases.

This is half thinking out loud for how to extend the NEP: I have to extend the NEP to include a new way to do repr for arrays, there are simply three ways of printing:

• scalar.__str__ for raw strings of the scalar
• scalar.__repr__ an (ideally) round-trippable representation of the scalar that has some indication of type (since it is not exactly settled what representation).
• item_repr(): The representation used by the array. This is a repr that does not need to have type information! This is because we already have the dtype information. The naive approach prints np.array([np.str_("a")], dtype="S1").
  • Normally (or maybe always) the scalar.__repr__ would just be the same as the array __repr__ with the additional type information.

Now, that new item_repr is something we could special case for all our dtypes. But, I would rather move it onto the DType, because otherwise we just make things worse for new user DTypes probably...

And, unfortunately, that means designing that API first. Probably not too bad, but I suspect we should for example move the print option to a contextvar while at it.

Had a look just to see what the status was -- it looks good to me!

Thanks for the reviews!