An alternative would be to change the prediction function of the Classifier/Regressor.

I would be more in favour of this and it would be easy for somebody to create an estimator that does such transform:

class TransformPredictionRegressor(BaseEstimator):
    def __init__(self, estimator=None, transform=np.log):
        self.estimator = estimator
        self.transform = transform
    def fit(self, X, y):
        # TODO: validate the parameters
        self.estimator.fit(X, y)
        return self
    def predict(self, X):
        return transform(self.estimator.predict(X))

Okay, agreed. The wish to apply such transform in the explainers will raise as soon as we get interaction importance measues like Friedman's H. There, it will be essential to quickly switch to the "raw" level of predictions.

I don't get that suggestion. y in fit and the return value of predict must
be on the same scale, or metrics don't work.

To my limited knowledge, I agree. I thought that we develop the TransformedTargetRegressor, especially to avoid such evaluation in practice.

I don't get that suggestion. y in fit and the return value of predict must
be on the same scale, or metrics don't work.

To my limited knowledge, I agree. I thought that we develop the TransformedTargetRegressor, especially to avoid such evaluation in practice.

For permutation importance, being able to transform the scale is less important than for partial dependence (and many other interpretability methods hopefully up to come). We could thus focus on partial dependence (and keep it in mind for upcoming extensions of inspection).

To make the argument explicit, let us assume a PoissonRegressor (similiar for LogisticRegression) that predicts exp(a + b + c). On the "raw"/"log" scale, there are no interactions, i.e. a + b + c, whereas on the scale of y there is a 3-fold interaction exp(a) * exp(b) * exp(c). Therefore, it would be very convenient for a user, if she or he could switch the analysis scale for partial dependence plots.

TransformedTargetRegressor is a bit overkill. We don't want to fit at all, but apply a transformation to predict of an already fitted model, i.e. a FunctionTransformer applied to the outcome of predict. But FunctionTransformer(lambda x: np.log(reg.predict(X))) doesn't work as it has no predict method.

class PredictionTransformRegressor(BaseEstimator):
    """Apply transformation to an already fitted estimator."""
    def __init__(self, estimator=None, transform=np.log):
        self.estimator = estimator
        self.transform = transform
    def fit(self, X, y):
        # TODO: validate the parameters
        # This should be a no-op, i.e. we do not fit anything.
        return self
    def predict(self, X):
        # We do not need to check if self was fitted.
        return transform(self.estimator.predict(X))  # will call check_is_fitted(estimator)

reg = PoissonRegressor().fit(X, y)
transformed_reg = PredictionTransformRegressor(reg, transform=np.log)
partial_dependence(transformed_reg, X=X, features=...)

I wonder if a simple option in partial_dependence would be more user friendly.

I still think this is very important for usability. To be more explicit, we are discussing
sklearn.inspection.partial_dependence and sklearn.inspection.PartialDependenceDisplay.

Options are:

1. Add an argument like transform_to_link_space="auto" | "identity" | "log" | "logit" | callable (todo: better name!!!)
2. Extend allowed values of method, e.g. method="_linear_predictor" for PoissonRegressor.
   • advantage: Could be faster
   • disadvantage: its either a private method (we could change that) or one needs to add such a function to an estimator (how does that impact pickle`).

This would be extremely useful. In the currently available inspection tools, its indeed only partial dependence that would need to be touched. (Later also H-statistics drafted in this PR).

In this jupyter notebook, we use a custom LogRegressor class to see that certain ICE curves are parallel in a Poisson boosted trees model with interaction constraints.

@glemaitre @adrinjalali @NicolasHug @thomasjpfan Do you have preferences or other proposals? See #18309 (comment).

For PoissonRegressor, is it enough to define a decision_function that outputs the raw scale? Then we can use response_method="decision_function" with partial_dependence to get the desired plots.