Related to #5811, #22272

I have an estimator, which prints to standard output in fit(), e.g. it logs how a training loss changes over iterations. For instance, I use XGBRegressor, which is reportedly sklearn-compliant. I plug this estimator into cross_validate or GridSearchCV or anything similar that uses joblib.Parellel to run jobs in parallel. Ideally, I would like to see contiguous chunks of output produced by each clone when the call to cross_validate() finishes. Of course, this is not what happens.

• If I use the default backend="loky", then I see no output on the first call of cross_validate(). A subsequent call may flush output from the previous call.
• If I use backend="multiprocessing" or backend="threading", then I see messy output, as it is being printed by concurrent clones.

I do this in a Jupyter notebook, but I think I had the same problem when running a script.

1. Is there a practicable way to collect printed output by an estimator in cross_validate or similar functions?
2. More generally, is there a way to collect training progress metrics from cross_validate? (I know that verbose parameter can print something after each training job is complete, but I want the stuff that is generated during the training)
3. Do stdout and stderr behave similarly in this context? I saw stderr being discussed separately, and I did not quite understand why.

Thanks.