This sample invariance test is currently applied to the ranking tests, and that seems strange.
For ranking the scores should have a different shape and be integer (right now y_true and y_pred are binary so it's valid scores but it's not a great test).

Also, the same should be tested for multi-class.
Via #10481.