Let's be explicit about dtypes and let's actually test on the device returned by the fixture:

the rest of the test will need to be updated accordingly (along with the code, I think).

You can use https://gist.github.com/EdAbati/ff3bdc06bafeb92452b3740686cc8d7c to launch this test on a machine with pytorch running on a non-CPU device.

What do you think about adding integer dtype_names in yield_namespace_device_dtype_combinations? Right now there are many asserts that do not depend on dtype_name, thus they won't check anything new. E.g. assert for empty_int32_labels with xp=torch, device=cpu, dtype_name=float64 and xp=torch, device=cpu, dtype_name=float32.

Adding integer dtypes could help remove such repetitions.
For example, this test could be rewritten as

@pytest.mark.parametrize(
    "array_namespace, device, dtype_name",
    yield_namespace_device_dtype_combinations(yield_integers=True),
)
def test_entropy_array_api(array_namespace, device, dtype_name):
    xp = _array_api_for_tests(array_namespace, device)
    labels = xp.asarray(np.asarray([0, 0, 42.0], dtype=dtype_name), device=device)
    empty_labels = xp.asarray(np.asarray([], dtype=dtype_name), device=device)
    constant_labels = xp.asarray(np.asarray([1, 1, 1, 1], dtype=dtype_name), device=device)
    with config_context(array_api_dispatch=True):
        assert_almost_equal(entropy(labels), 0.6365141, 5)
        assert entropy(empty_labels) == 1
        assert entropy(constant_labels) == 0

If it makes sense I can open separate issue for discussion.

I don't want to have a combinatory explosion of test cases where most cases are not likely to yield interesting things to test.

For tests that require integer dtypes, we can just use either xp.int32 or xp.int64 that should be enough for most scikit-learn use cases and are supported by all known platforms.

For floating values, it's interesting to test with xp.float32 and xp.float64 (when possible) because most GPUs work best in xp.float32 and sometimes do not support xp.float64 which is the default dtype of numpy, hence the special need to test for both when possible, conditionally on the choice of namespace and device.

Nice code simplification by the way :)