In pairwise_distances, check_pairwise_arrays is not run if metric is a callable or a metric in PAIRWISE_DISTANCE_FUNCTIONS.

For metrics in PAIRWISE_DISTANCE_FUNCTIONS, it delegates the check to the metric functions since they already do it. Otherwise it would result in a double validation.

Most PAIRWISE_DISTANCE_FUNCTIONS metrics run check_pairwise_arrays themselves but the following do not:
haversine_distances (this is implemented in Cython but does seem to check that X is 2D)
cosine_distances

cosine_distances does because it calls cosine_similarity which calls check_pairwise_arrays.

haversine_distances doesn't indeed. The cython implementation calls check array separately. To me the python haversine_distances function should run the validation as well as all other public functions or classes directly calling DistanceMetrics, but that goes beyond just the haversine distances then and I don't know how many functions are concerned. The current state is probably good enough and don't seem to have been problematic so far.

check_array is also not performed for user provided metric callables, but maybe this is not 'needed' ?

Yes, in case of a callable, all validations are left to the callable. It's done on purpose to allow for use cases that we don't have within sklearn but may be useful more generally like distances between strings.

To summarize, I think the status of check_pairwise_arrays in pairwise_distances is in accordance to our expectations and that we can close this.

Sure but we could at least leave a comment in the code about:

Yes, in case of a callable, all validations are left to the callable.

Indeed adding a comment would not hurt :)

Yes, in case of a callable, all validations are left to the callable. It's done on purpose to allow for use cases that we don't have within sklearn but may be useful more generally like distances between strings.

That was wrong. We still call check_pairwise_arrays (in _pairwise_callable). We just don't force 2D arrays here.

So in the end the use of check_pairwise_arrays is pretty consistent. There's only haversine that is special and implements its own validation but I think it's good enough.

Yeah I saw that later too. Agreed, it's enough