Again, votes for or against the more magical #8374 are welcome

tests not passing ;)

I generally prefer meta-estimator solutions because they are easy for us and pretty safe and super flexible. @agramfort and @GaelVaroquaux tend not to like them because they feel they are hard to use / inspect.

The main use-case of freezing is inside a meta-estimator, so here we're adding another level of indirection.
One could argue that people are not interested to go that deep? If I already fit a model and then freeze it and put it in a CalibratedClassifierCV, do I really need to access it programmatically afterwards?

API question: should we freeze the estimator we get if prefit=True, or deprecate prefit=True and expect the user to freeze? I don't see anything wrong with supporting prefit=True and it would be easier for users.

I think this is my preferred solution.

There is no benefit that I can see in deprecating cv=predict in
calibration, but we should certainly deprecate prefit in SelectFromModel.

Fair.

I'm now not altogether sure whether freezing is the right solution for transfer learning and semi-supervision... The problem is that you can't search over the parameters of a frozen estimator (to optimise some target supervised prediction accuracy, for instance), something which I assume is desirable. Rather you want some kind of FixedDataWrapper(estimator, X, y, memory='/tmp/blah') (i.e. fix the training data for the estimator, but allow parameters to vary; cache the models).

But that doesn't solve the problem of making a pre-trained model appear as part of something. Do we have good examples that motivate freezing???

You mean search parameters that would change training? That would be quite different, yes.
My motivating examples are still things that have prefit and putting them in pipelines / cloning them.

Is there a reason this implementation doesn't mirror all attributes?
If we provide a freeze function, we can set all the attributes etc on the instance match the base estimator, right?

Another idea that might be possible would be to create new frozen variants classes based on the estimator we get. That should allow us to cleanly overwrite the fit methods, but also mirror all other functionality, allow isinstance etc. Also, the class would have a name that's specific to the estimator but also clearly shows that it's frozen.
Though if we pre-define them we can't freeze 3rd party estimators, and if we try to create them on the fly, I'm not sure if we could get them to pickle.

I think if we go with this solution we should copy over the dict.
@GaelVaroquaux is offline for the next couple of weeks, and I think we should include him in the discussion.

Frozen variant classes is not a reasonable option as far as I'm concerned.

Can you elaborate?

Closing for now