Skip to content

Add the baseline corrected accuracy score for (multi-class) classification to sklearn.metrics #31244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Carl-McBride-Ellis opened this issue Apr 24, 2025 · 1 comment

Comments

@Carl-McBride-Ellis
Copy link

Describe the workflow you want to enable

Would it be possible to add a new score to sklearn.metrics, namely the baseline corrected accuracy score (BCAS) (DOI:10.5281/zenodo.15262049). The proposed metric quantifies the model improvement w.r.t. the baseline, and represents a direct evaluation of classifier performance. See the proposed code below, which is label agnostic, and is suitable for both binary and multi-class classification.

Describe your proposed solution

import numpy as np

def BCAS(y_true, y_pred):
    """Baseline corrected accuracy score (BCAS).

    Parameters
    ----------
    y_true : Ground truth (correct) labels.

    y_pred : Predicted labels.

    Returns
    -------
    score : float
    """
    label, count = np.unique(y_true, return_counts=True)
    most_frequent_class = label[np.argmax(count)]
    y_baseline = np.full(len(y_true), most_frequent_class)
    as_baseline = np.mean(y_true == y_baseline)
    as_predicted = np.mean(y_true == y_pred)
    return (as_predicted - as_baseline)

Describe alternatives you've considered, if relevant

No response

Additional context

No response

@ogrisel
Copy link
Member

ogrisel commented Apr 25, 2025

Thanks for your suggestion. However, new feature in scikit-learn are subject to the following inclusion criterion:

As far as I can see, what you propose is not standard enough in the ML community to be included as such the project's code base.

@ogrisel ogrisel closed this as not planned Won't fix, can't repro, duplicate, stale Apr 25, 2025
@ogrisel ogrisel removed the Needs Triage Issue requires triage label Apr 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants