Would it make sense for the __class_getitem__ implementation on np.dtype to return the actual dtype class instances in np.types? If for some reason that breaks the typing support, would it be possible to change the repr for the GenericAlias instance you get back right now to at least point runtime users to the correct place to get the type they want?

In [2]: np.dtype[str]
Out[2]: numpy.dtype[str]

In [3]: type(_)
Out[3]: types.GenericAlias

Just to be clear, are you suggesting that as a replacement or as additional to this? I had that originally even, but we pulled it out for GenericAlias.

I have not digged in whether making __class_getitem__ work for both typing and this can work. I.e. why not return concrete instance (when it exists!) and otherwise the GenericAlias?! Maybe that would even be better for typing?

I think it could work as a replacement, although I like the nice explicit names you've given the dtype classes here, so I guess weakly as an addition to this.

I don't know enough about numpy's typing support or python typing in general to say whether or not the concrete types would break anything, but I think eliminating the use of GenericAlias and instead returning concrete dtype classes would be best if that's possible.

@BvB93 do you know if adapting __class_getitem__ to return the concrete classes would work? I guess np.dtype[Any] must keep returning the GenericAlias though at the very least?!

I don't hate it as a replacement, although the current class __name__ is also annoying (since it is not a proper name)! So additionally to np.dtype[...] this does give them a nicer name (although not much nicer to read, but one that makes sense as a variable name).

OK, I think the answer for modifying np.dtype[np.float64] at runtime is a naaaah: https://mail.python.org/archives/list/typing-sig@python.org/thread/LRDYM2MR3ZNXG6IUFTDCDJBOOVDT2U72/

The only alternative then would be if we want np.dtypes or so as a namespace, and the only argument for that I have is that we might at some point want more functionality similar to np.promote_types (e.g. it is probably a good addition to have np.cast_dtype).

@ngoldbaum (and others), whats your take on this? How can we come to a decision whether to just do this or what alternatives to explore?

Given the reaction you got on typing-sig, I think this PR is probably the way to go.

@rgommers and @mattip should we go ahead with this?

When reviewing NEP 52, both @mhvk and @stefanv commented about the choice for types as a submodule name as not ideal, thinking it was a choice that that NEP made. It's not, the choice is being made here - so I'll xref this PR and let Stefan and Marten comment here for themselves.

Hmm, I had seen this PR vaguely and thought it was all about typing... But now see it is about storing dtypes somewhere. Copying my comment:

I think that makes sense, though would love to stop using type as equivalent to a data type as much as possible... Since this is a new module, there is an opportunity in naming it. data_types or datatypes are probably too long; is dtypes an option? Or too confusing with np.dtype?
Or could it be np.dt? One could then think of pre-defining some dtype so that one could do np.array(..., dtype=np.dt.f8).

I actually like my last suggestion, of having the dtypes defined by the abbreviations. This is similar to what we do for astropy.units, where the module contains the long and short names of all the units as pre-defined instances (well, one could in principle have __getattr__ define them as needed...)

Of course, one could also encourage a standard pattern like import numpy.dtypes as dt - that allows a longer name.

Anyway, I think the name is important and types is a bad one! I just implies classes to me...

Reading #23537 (comment), I'm a bit less sure about having f8 etc as instances; maybe float64 is better and clearer. Though ideally float64 and co would then be instances of dtype classes, not classes of scalars. Would this be a right time to do that? (In the new module at least)

My gut feeling aligns with Marten's:

• If we use numpy.types to suggest an equivalence between "numpy types" and "numpy data type", that's an unnecessary (and for new users counter-intuitive) jump.
• If we use numpy.types to suggest that we will only store actual types in the submodule, that is unnecessarily restrictive.

Sure, the DTypes may be types, but that is not relevant to how we group the functionality here, and so I would also prefer np.dtypes (np.dt could work, but may be a bit cryptic to new users).

Personally, I don't like dt because it is relatively common to do import datetime as dt

(including internal to numpy source, as well as at least pandas and xarray)

On further thought, I agree that it should not be np.dt - better to have np.dtypes (or perhaps np.datatypes). Users can then decide themselves whether or not to do a import numpy.d[ata]types as dt (datetime cannot handle leap seconds so is pretty useless in astronomy...)

The question is a bit what we want. I don't disagree with np.dtypes or so as a name. The question is whether np.dtypes.f8 for example is a useful alias for Float64DType (the class) or not for the singleton instance np.dtype("f8"). (This is not always clear, because semantics can differ in subtle ways and I am not actually sure what we want typically: asarray(asarray([1.], ">f8"), dtype=Float64DType) might preserve the endianess. OTOH, the dtype/signature in ufuncs accepts the instance but really cares more about the class).

The follow-up question then would be: If np.dtypes.f8 are not then hiding them in np.types seems OK to me again. OTOH, maybe it should be the classes and if you are worried about possible funny byte-order preserving, you should write dtype=dt.f8().

I can see more dtype utility functions, for example, we are missing a way to spell new_dtype = dtype.astype(new_dtype_or_Class) (which could be a method as here, though). The drop metadata hanging PR is also one (could be a method as well).

The question is a bit what we want. I don't disagree with np.dtypes or so as a name. The question is whether np.dtypes.f8

I think these are two separate questions. The one on the submodule name (types, dtypes, or other) clearly needs a decision. Introducing new aliases like f8 is unrelated to that, and I'm personally -1 on that given that we already have way too many aliases floating around. Use of scalars should be discouraged, so new short aliases are going in the wrong direction.

My thought was: If the purpose here is to house the DType classes for niche use-cases, I am not sure putting them into a np.dtypes namespace that may also house convenience aliases like np.dtypes.f8 if all that useful.

I suggest to forget about f8 and focus on the main comments of Marten and Stefan, who both don't like the types name.

If the purpose here is to house the DType classes for niche use-cases,

That is not the sole stated purpose in the docs. From types.py in this PR: Similar to the builtin types module, this submodule defines types (classes) that are not widely used directly.

DType classes are a subset of that, so I'm personally fine with the types name. And dtypes seems more restrictive. What if we'd want to move other classes here, like return types for objects like what np.finfo returns?

The point of the original suggestion was to have a place to put types/classes that users rarely/never access. DTypes are most of those (there could be more, like np.ufunc, ArrayFunctionDispatcher).

I don't care about the name, the suggestion was because Python has types as a catch-all for classes that users rarely access directly. The point I wanted to make still stands: If we don't expect it to house any conveniences but only rarely used classes, I am not sure I mind the name. If we want it to house more dtypes related stuff, then that is a very clear argument against the name.

That doesn't mean I care much either way, I don't.

Agreed to leave out f8, etc. They were not obviously a good idea.

On the name: I came very much at this from the NEP write-up, so had no idea about the larger idea of types becoming a collection of rarely used but exposed classes. I can see that logic, but wonder if it goes too much in the direction of just having another lib or utils. It also does not look much like python's types module, which does indeed have a set of classes, but they are mostly the building blocks, not collections of related classes like the data types. If one were to stick with np.types, it might be more logical to define a np.types.dtypes submodule.

But with all the work to define the data types, it seems OK to me for them to have their own space even if it is not used much -- and it may get used more than you think once it becomes easy to define user dtype in python rather than in C! So, from that perspective, I think it still makes sense to have np.dtypes as a place to hold everything related to how dtypes are defined.

That's fair. At least abstract DTypes should happen pretty soon, and unlike these classes may be useful a bit more often.

The first/main "create DType in Python" thing, that I currently think would be nice would be to create a custom structured DTypes in a manner similar to dataclasses.

OK, I updated this to np.dtypes. I am assuming we will at least put the open PR for drop_metadata there (we can always deprecate+replace).

Can we move this forward? If we have it, we can use it to fix-up and put in the "drop_meatadata" function.

Thanks @seberg