I'm not sure about functions. They are mutable, but I guess they are fine, right?

Should wait for #4713, then it will work.
I'm not sure if it is intentional that the memory objects are mutable defaults. @ogrisel @agramfort @GaelVaroquaux ?

I'm not sure if it is intentional that the memory objects are mutable
defaults. @ogrisel https://github.com/ogrisel @agramfort
https://github.com/agramfort @GaelVaroquaux
https://github.com/GaelVaroquaux ?

it's not supposed to be changed in the functions so I think it's fine...

ok, should pass now.

Looks good.

Merging as minor then.

I just ran into this, because I created a custom estimator that takes an object of a custom class as a default.

I wonder whether this should be tested for at all. I understand that the original motivation was to remove mutable default arguments from scikit-learn, but that has been achieved.

Note that the test allows functions and Memory objects, which are mutable, because their state is not expected to change. Likewise, the state of the custom objects I use will not change after initialisation, but the test obviously cannot know about that.

Indeed, while we can check it internally in scikit-learn, I agree that it could be something you might want to opt out in contrib projects.

See #13969 and #14381 for a future possible workaround

@oulenz A possibly overkill but straightforward solution would be to introduce an estimator tag allow_mutable_default that disables this test.

I guess it depends how fine-grained you envision the system of estimator tags. Alternatively, perhaps an opt-out list of tests for individual estimators could be a more general solution.

Or just not including this one in the non-strict mode as @rth suggested.

I think depending on names of tests is bad because multiple tests test the same thing or rely on similar assumptions and we'll get in a real mess if we try that.

We should encode our assumptions about the estimator, not what specific tests to run.