We can think of our estimators' parameters as two different types:

• model hyper-parameters which influence the end result, accuracy, complexity, etc., e.g. SVM's nu or the number of trees in a random forest.
• config parameters which control other aspects of an estimator such as verbosity, copy_X, return_xyz, etc.

We could split these two types of parameters and have model hyperparams as __init__ args and the rest as set_params or a new configure args. This would also allow us to introduce new configuration parameters to all estimators w/o having to add another arg to __init__. An example would be the routing parameter required for sample props implementation, or potentially controlling log level at the estimator level.

My proposal would be to have a config_params estimator tag which includes the names of those parameters, and have clone, get_params and set_params understand them, and potentially introduce a new configure method to set those parameters.

This also would mean partially relaxing the set only __init__ params in __init__ requirement since __init__ would be allowed to set those extra params.

cc: @scikit-learn/core-devs

also related to: #17441 (comment)