Thanks a lot! You're on fire :)

1. Awesome! See Evaluation metrics for multi label classifiers #558.
2. I'd actually prefer to have zero_one_loss and zero_one_score. They should definitely end with loss and score! We can talk about whether both or one of them should be called accuracy.
3. Because it was easier to implement than to import? Feel free to switch to zero_one_score (needs a lazy import to avoid import loops).

Currently the clustering metrics are documented in the clustering module. I think I would prefer them to be documented in the metrics module. Any opinions on that @arjoly @ogrisel @GaelVaroquaux @robertlayton ?

By the way, there is two metrics that ends with _error with mean_absolute_error and mean_square_error.

@amueller I have to leave, I will think about it tomorrow.

Edit: yes indeed mean_squared_error.

Lol I see we will become good friends :)

Maybe for regression the losses are called errors? I am not sure how much thought went into a naming scheme.

I think being explicit is important, which is why I don't like zero_one.

Being consistent is nice, too, but we shouldn't overdo it. Is the rest consistent? Is it worth renaming to mean_absolute_loss and mean_squared_loss?

Or could we try to have separate naming schemes for classification, regression and clustering metrics?

OT mean_square_error is deprecated and now mean_squared_error.

Any opinions on that @arjoly @ogrisel @GaelVaroquaux @robertlayton ?

No strong opinion as long as there are some frequent and highly visible
links.

G

I think its ok to keep the _error suffix instead of _loss when this is a very common name such as mean_squared_error. mean_squared_loss would be too confusing and mean_squared_error_loss quite redundant in my opinion.

+1 for being more explicit by deprecating zero_one and adding an explicit name such as zero_one_loss and accuracy_score (one being 1 minus the other).

+1 for having accuracy_score. Generally, if the thing has an explicit name, we should use that. In cases where the same metric has multiple names, this runs into problems.

As a thought, how do people feel about having aliases? i.e. after the definition of mean_absolute_error, have the line: mean_squared_error = mean_absolute_error.
This would make searching for functions significantly easier, both on the webpage and by doing something like:
print [d for d in dir(sklearn) if "squared" in d]

Oh, and put all metric documentation in the metrics part. Makes sense, that is where I would have expected them to be. With the clustering documentation update I'm (slowly) getting through, the documentation page may get significantly longer.

As a thought, how do people feel about having aliases?

Not too excited: there should be one and only one prefered way of
achieving a result, elsewhere people are going to wonder what the
differences are.

This would make searching for functions significantly easier, both on the
webpage and by doing something like:
print [d for d in dir(sklearn) if "squared" in d]

I'd suggest to put disambiguation in the docstring and recommend the use
of 'numpy.lookfor'.

If we are lucky then people can google the docs ;)
Btw mean_squared_error and mean_absolute_error are different ;)

Btw mean_squared_error and mean_absolute_error are different ;)

That's what I thought, but I didn't dare mention it :)

Btw mean_squared_error and mean_absolute_error are different ;)

Of course they are. In my defence, I only just had my first coffee of the day.

Currently the clustering metrics are documented in the clustering module. I think I would prefer them to be documented in the metrics module.

It would be nice to have everything in one place and a link from the clustering documentation to the module evaluation part.

Oh, and put all metric documentation in the metrics part. Makes sense, that is where I would have expected them to be. With the clustering documentation update I'm (slowly) getting through, the documentation page may get significantly longer.

Will you make it or I perform the change?

I will initiate name changes (accuracy_score and zero_one_loss). I suppose that I have to add deprecation warning for 0.15, remove previous name from classes.rst, perform change in the narrative doc and the __init__.py

I will initiate name changes (accuracy_score and zero_one_loss). I suppose that I have to add deprecation warning for 0.15, remove previous name from classes.rst, perform change in the narrative doc and the init.py

Yes.

And update the API change of the whats_new.rst file when ready to merge.

+1 for being more explicit by deprecating zero_one and adding an explicit name such as zero_one_loss and accuracy_score (one being 1 minus the other).

Currently zero_one_loss is the sum of the zero one loss over the samples. So accuracy_score is not 1 - zero_one_loss.

I think there is still some typo to hunt down, but what I wanted to achieve is there.

So reviews and comments are welcome. :-)

Alright for keeping zero_one_loss as the sum and making accuracy_score(true, pred) == np.mean(true != pred).

you meant accuracy_score(true, pred) == np.mean(true == pred) ?

I think that is what he meant.

Indeed... sorry for the confusion.

I have rebased on top of master to take into account #1534.

@GaelVaroquaux The deprecated decorator is now used.

To summarize what are the remaining questions / remarks to take into account

1. The explained variance question of @amueller.
2. The import consistency question

For

1. I don't know this metric enough. It is pretty new to me.
2. This might be solve latter.

Have you changed the default behavior of the renamed zero_one? could you please test that the old name has the old default behavior and the new name has the new?

Have you changed the default behavior of the renamed zero_one?

No.

could you please test that the old name has the old default behavior and the new name has the new?

See the reference.

Does it answer your question?

Taken into account your question, I have also clarified the api changed. Good catch! Thanks.

Explained variance can be negative for completely random or anti-correlated predictions:

>>> from sklearn.metrics import explained_variance_score
>>> explained_variance_score([0, 1, 2], [1, 2, 0])
-2.0
>>> explained_variance_score([0, 1, 2], [1, 2, 3])
1.0
>>> explained_variance_score([0, 1, 2], [1, 2, 2])
0.66666666666666663

Usually for a real life, non dummy model it's positive though (if the signal is not itself completely random).

@arjoly Thanks for the pointers and sorry for being lazy. Having a busy day. Apart from my comment on the deprecation warnings I think this is good to go.

Thanks @ogrisel for your lights on this subject.

@amueller Don't worry, we all have our busy day.

@amueller Warnings are catched!

thanks :) I guess you are good to know. feel free to merge using you preferred method ;)

Arnaud Joly notifications@github.com schrieb:

Warnings are catched!

---

Reply to this email directly or view it on GitHub:
#1512 (comment)

Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

Merge by rebase! :-)

Thanks all for your time and review!!!

A total of 1,306 additions and 215 deletions. That's a huge contribution! Thanks heaps!

+1 :)

Mathieu Blondel notifications@github.com schrieb:

A total of 1,306 additions and 215 deletions. That's a huge
contribution! Thanks heaps!

---

Reply to this email directly or view it on GitHub:
#1512 (comment)