You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Not sure what should be the output namespace / device in case we output an array, e.g. roc_auc_score with average=None on multiclass problems...
Currently all regression/classification metrics that support array API and multiclass or multioutput, all output an array in the same namespace and device as the input (checked code and manually). Summary of these metrics :
Looking at the metrics code, if we wanted to support a list of scalars, we'd generally have to do extra processing to convert an array (often output of an xp.** function) to a list of scalars.
Once we arrive at a consensus we should update the array API documentation and update the check_array_api_metric in tests such that when the output is array/list - we check that the output type etc is correct.
Clarification of how we should handle array output type when a metric outputs several values (i.e. accepts multiclass or multioutput input).
The issue was summarised succinctly in #30439 (comment):
Currently all regression/classification metrics that support array API and multiclass or multioutput, all output an array in the same namespace and device as the input (checked code and manually). Summary of these metrics :
Regression metrics
Returns array in same namespace/device:
Classification metrics
Returns array in same namespace/device:
Looking at the metrics code, if we wanted to support a list of scalars, we'd generally have to do extra processing to convert an array (often output of an xp.** function) to a list of scalars.
Once we arrive at a consensus we should update the array API documentation and update the
check_array_api_metric
in tests such that when the output is array/list - we check that the output type etc is correct.cc @ogrisel @betatim
The text was updated successfully, but these errors were encountered: