See #10977 and #10978. Well we should probably test that fitting an estimator twice gives the same result