Description

Compute regression error characteristic [1] curve, which is essentially the CDF of the regression errors. Its function is similar to that of ROC curves - allows comparing performance profiles of regressors beyond just one summary statistic, such as RMSE or MAE.

Examples

Used like this:

from sklearn.metrics import RecCurveDisplay

lr_estimator = LinearRegression()
lr_estimator.fit(X_train, y_train)

RecCurveDisplay.from_estimator(lr_estimator, X_test, y_test, name="Linear regression")

The result is like this:

It allows comparing one regressor to a constant prediction baseline (by default - the median of the test samples).

We can also compare several regressors:

fig, ax = plt.subplots()

RecCurveDisplay.from_predictions(
    y_test,
    pred_lr,
    ax=ax,
    name=f"Linear regression (MAE={mean_absolute_error(pred_lr, y_test):.2f})",
    plot_const_predictor=False,
)

RecCurveDisplay.from_predictions(
    y_test,
    pred_knn,
    ax=ax,
    name=f"KNN (MAE={mean_absolute_error(pred_lr, y_test):.2f})",
    plot_const_predictor=False,
)

# this one will plot the constant predictor for reference - note that the plot_const_predictor is not set to False!
RecCurveDisplay.from_predictions(
    y_test,
    pred_hgbr,
    ax=ax,
    name=f"Gradient Boosting Regressor (MAE={mean_absolute_error(pred_lr, y_test):.2f})",
)

fig.show()

This will plot something like this:

You can see here a clear domination of regressors.

Sometimes the curves will cross - in this case there is no clear domination, and the performance profiles of both regressors differs. Some are better for smaller error tolerance, whereas others may be better for larger error tolerance.

P.S - my first ever contribution to this phenomenal library. I hope I haven't missed something important.

References

[1]: Bi, J. and Bennett, K.P., 2003. Regression error characteristic curves. In Proceedings of the 20th international conference on machine learning (ICML-03) (pp. 43-50).