FYI right now numpy arrays are converted to lists for printing. If there are very large numpy arrays in the repr, this could be problematic. It also doesn't allow the user to distinguish numpy arrays from lists in the arguments, so that's maybe non-optimal. But doing something else shouldn't be that hard.

Code is horrible right now and could be simplified.
Logic between pipeline (which is special cased) and estimator is duplicated, that should probably be simplified. We could also think about special casing BaseSearchCV but it actually looks fine right now.
Feature union might need a similar special case.

Also, this doesn't detect terminal etc so that's broken, but also fixable.

I'll have to think about whether there's a way to do that kind of layout without special-casing pipeline, which doesn't seem right

As mentioned IRL, I think that the functionality is really great (I haven't looked at the code so far)!

+1 for exploring it further

@jnothman depends on what you want. You can comment it out and it will still work. But I want to control where the line-break is manually, and I want the indentation of the tuples to be different.

I thought about this for the html quite a bit. I think it's worth special-casing pipeline because it's important to show this nicely and it won't be as nice without explicit treatment.
I want the indentation of steps to be such that lines up with the "steps" string. But In general I want to indent by 4 whenever I do a line-break. If I would always align with the thing in the line before, you'd get to the end of the line way too quickly.

without special treatment:

I could always just indent lists by one, but if you have a line that doesn't start with the list, it looks odd. We could change the indentation based on whether the line starts with the list/tuple or not, though.

Do you think we should also be ordering these parameters in order of the signature, rather than sorting, where possible (i.e. for those with positional arg names)?? I think this would greatly improve readability too as there is often topicalisation (i.e. most important first) and coherence (most related together) in the ordering. On the other hand, sorting allows a user to find the parameter they want...

Yeah, the code needs a major cleanup, sorry to waste your time. I should really put a comparison image to master in here to show how much better it actually is ^^. But this is not my highest priority right now.

And yes, definitely the special casing should be extended to the other composite estimators. And sorting vs not sorting is a good question.

The specs would be how the formatting should look like?
And yes, it totally makes sense to open this up.

I like this to be the spec:

The problem is that it's very hard to write down a simple spec that works for everything. In particular we're limited in line-length in jupyter, but we're not limited in nesting depth in sklearn. So if you write RFE(RFE(RFE(RFE(RFE(RFE(RFE(LogisticRegresssion)))))), I have no idea how to format that. It's unlikely someone will write something like this. Basically fixed indentation increase + limited width + arbitrary nesting depth is not solvable. I guess we can use ... once the indented line-length becomes too small, or so small that we can't fit the next thing onto it. Or just for a given nesting level? That will create possibly much longer strings than now, though. Right now we limit the string-length and cut off totally arbitrarily. I don't have a good idea how to do that better.

I think it should be something like this:

• Indentation should increase in steps of 4 and be nested (indentation in master is entirely broken).
• For composed estimators, make the estimator lists indented by only 1 to match up with opening ( or [.
• for composed estimators put each estimator name and each estimator on a separate line.
• use (...) for nesting level 5 (?).

master for comparison:

This would be plain text (maybe with terminal colors?). So all estimators and all contexts until we have better ones for fancier contexts ;) The current __repr__/pprint thing has obvious bugs that can't be fixed without doing something like this PR.

Well:

RFE(RFE(RFE(RFE(RFE(RFE(RFE(LogisticRegresssion))))))

could be written as:

RFE(rfe1)
where rf1 = RFE(rfe2)
where rfe2 = RFE(rfe3)
where rfe3 = RFE(rfe4)
where rfe4 = RFE(rfe5)
where rfe5 = RFE(rfe6)
where rfe6 = RFE(LogisticRegresssion)

Perhaps this is a bit extreme, but we could extract and define separately nested estimators that are too long (vertically or horizontally) for the current indent...

Once this simple modification is in, I'll try to review this PR and aim for merge. I think that it is so terribly useful and will bring a lot of value to users.

Is this still valid with 'help wanted'? I could have a go through the discussion (of related PR's as well) and may be try to get it in, in case this still needs help from a contributor.

Putting this one back on my stack... let's see...

@amueller : I think that we should not be too ambitious with this PR, and just implement my suggested change: "allow ourselves to extend beyond the line limit when the useful fraction of the line becomes too small", add a few tests, and merge. This is very useful to users already.

Do you have the bandwidth to do it, or should we open it for contributions during the sprint?

@GaelVaroquaux I agree. I'm not sure about extending the line length. I have to think about that. But yes, we should keep this one small.
And we can totally open it up during the sprint.

In the first place, I don't think 79 characters is an especially reasonable width when code consistency is not the concern. But if the max width becomes a parameter, I suppose we'd better keep to it.

replaced by #11705