[MRG] replace 'f1' scorer by explicit variants #2676
Conversation
This makes the averaging options clearer for users It entails a deprecation procedure for scorers.
| @@ -93,5 +93,7 @@ | |||
| 'v_measure_score', | |||
| 'consensus_score', | |||
| 'zero_one_loss', | |||
| 'get_scorer', | |||
| 'list_scorers', | |||
| 'make_scorer', | |||
| 'SCORERS'] | |||
There was a problem hiding this comment.
Should I remove SCORERS from sklearn.metrics package import?
|
I think we can even deprecate the "weighted" scheme and remove it in two releases. |
|
Well it's been the default for some time. But you're right that without this sort of patch we couldn't even consider deprecating weighted. |
|
Yes, I think it's a good opportunity. I completely agree that people should not report results in the literature with this option! |
|
Quick question: do f1_macro and f1_micro reduce to f1_binary when there are only 2 classes in y_true? |
|
No, because |
|
I forgot to add an option here for |
|
Merged into #2679 |
This makes the averaging options for P/R/F scorers clearer for users, and avoids users getting binary behaviour when they shouldn't (cf. #2094 where
scoringisn't used). I think this is extra important because "weighted" F1 isn't especially common in the literature, and having people report it without realising that's what it is is unhelpful to the applied ML community. This helps, IMO, towards a more explicit and robust API for binary classification metrics (cf. #2610).It also entails a deprecation procedure for scorers, and more API there:
get_scorerandlist_scorers