Also, multiple outputs are currently handled by flattening 2d arrays and view them as 1d array. This corresponds to micro averaging. For my application, I would prefer macro averaging (averaging over classes). For example, for the R^2 score that would be: np.mean([r2_score(Y_true[:, k], Y_pred[:, k]) for k in xrange(Y_true.shape[1])])

Is it really micro-averaged r2 score ? Just a small experiment

In [1]: import numpy as np
In [2]: y_true = np.random.rand(5, 3)
In [3]: y_pred = np.random.rand(5, 3)
In [4]: from sklearn.metrics import r2_score

# Current multi-output r2_score
In [5]: r2_score(y_true, y_pred)
Out[5]: -1.2018060998146924

 # It would be micro-r2 score
In [6]: r2_score(y_true.ravel(), y_pred.ravel())
Out[6]: -1.1395845816752996

In [7]: from sklearn.metrics import explained_variance_score

# Check that it's equal to r2_score in this case
In [8]: explained_variance_score(y_true.ravel(), y_pred.ravel()) 
Out[8]: -1.132385768714816

# r2-score with no averaging
In [9]: r2 = [r2_score(y_true[:, i], y_pred[:, i]) for i in range(y_true.shape[1])] 
In [10]: r2
Out[10]: [-1.0513131617660676, -1.2263410810199482, -1.2582117503263115]

# It would be macro-r2 score
In [11]: np.mean(r2) 
Out[11]: -1.178621997704109

# For reproducibility
In [12]: y_true
Out[12]: 
array([[ 0.28481499,  0.34159449,  0.89364091],
       [ 0.08516499,  0.24426185,  0.58491767],
       [ 0.65374035,  0.78358486,  0.84892285],
       [ 0.12355558,  0.32354626,  0.02966046],
       [ 0.65858239,  0.59705347,  0.00573082]])

In [13]: y_pred
Out[13]: 
array([[ 0.32639174,  0.87657742,  0.23203866],
       [ 0.66826156,  0.06449232,  0.21180403],
       [ 0.19938095,  0.65445628,  0.13731781],
       [ 0.19451816,  0.10242323,  0.50932089],
       [ 0.95501124,  0.33805111,  0.61441609]])

Uh oh!

option to return an array in metrics if multi-output #2200

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions