Also, using the term 'ovr' in the binary case (which after #11859 will be everywhere) is very confusing.

I'll take this on.

Awesome; thanks for letting me know!

I agree with the assessment of the code being opaque. Are there any more actionable insights except for better naming of "ovr" in the binary case?

Trying to improve some things a bit for linear models, I got stuck in the swamp of _logistic.py and test_logistic.py. TBO, I can't and won't maintain that code! And this despite the fact that given so much functionality, the code seems quite compact. (IMHO, the HGBT code, for instance, is long but very understandable and maintainable).

If I were to rewrite it without keeping legacy, I would do:

• Use penalty parametrization as in ElasticNet
• Remove "ovr". If a solver supports the full multinomial loss, there is no reason to use OvR, is there? And if a solver does not support multinomial, there is no choice anyway.
  Alternative option: new class for OvR classifiers. And we already have OneVsRestClassifier
  DEP deprecate multi_class in LogisticRegression #28703
• Use a single best penalty for LogisticRegressionCV not one per class (which might stem from the OvR legacy)
• Get rid of (most of) joblib parallelism
• Big overhaul of the tests
• ...

@scikit-learn/core-devs ping for discussion.

Is this one of those things like what we did with model_selection that it'd be easier to implement in a new module somehow?

I like the ideas you have here, but I'm not sure why the parallelism needs to go away or why it's not helpful.

I think it'd be nice to have a maintainable code there.

@lorentzenchr The changes that you are proposing seem so significant that, should we go in this direction, it is probably better to ramp up a new object to ensure smooth transition to our users (rather than an abrupt break of backward compatibility).

However, I feel that most of the changes that you propose are net loss of useful functionality for the user.

With regards to the penalization parametrization, I clearly agree with you, and there are several places in scikit-learn where I would like to change how we parametrize our penalties (for instance, in the ridge, it should account for a norm of the design matrix, in order to have good defaults). One question is: how to do this smoothly? It can be addressed, but I am not sure that it will simplify the code.

With regards to ovr, I think that it is a actually an important aspect of the objects, in particular the CV one. In my experience, in multiclass one often needs different penalty per class. The reason is for instance that the prevalence of the classes vary widely. One could argue that the use of the metaclassifier OneVsRestClassifier means that the corresponding functionality can be implemented via a pipeline. But it's much harder to implement. Having good patterns easy to implement for the user seems like a priority.

With regards to parallelism, I actually think that parallelism is something very important, and that we should be working on having more parallelism in scikit-learn. Modern computers have many cores (my laptop has 8, many compute nodes have much more).