Why do you want to remove it? Just because nothing in scikit-learn uses it now, doesn't mean people from the future can't/shouldn't use it. Or, if there is a reason to not use it I think it would be good to record it here so those people from the future have an easier time figuring out why :D

I think this is more an accident of history that Memory was allowed as a default argument value but you are right that it could in principle be used in third-party estimators and so that this change would potentially cause them not to pass check_estimator ... all in all I think this is the kind of thing which is not worth spending too much time over.

I'll leave the PR open for a few days for other opinions but I am fine closing it.

For further reference, the result of a code archaeology session:

• the test to check that mutable arguments were not used was done in [MRG + 1] TST test that all default arguments are not mutable #4379. At this time some estimators used a Memory instance as default argument e.g.:

  scikit-learn/sklearn/cluster/hierarchical.py

  Lines 684 to 688 in 90d5ef1

  	def __init__(self, n_clusters=2, affinity="euclidean",
  	memory=Memory(cachedir=None, verbose=0),
  	connectivity=None, n_components=None,
  	compute_full_tree='auto', linkage='ward',
  	pooling_func=np.mean):

  and so Memory was deemed fine [MRG + 1] TST test that all default arguments are not mutable #4379 (comment)
• in [MRG+1] EHN/DOC Make error msg more informative in Pipeline #8865 all the memory arguments were changed to default to None but the test checking that no mutable argument was not updated.

Closing in favour of #25498