Skip to content

Tests for sample order invariance in estimator_checks #8695

Closed
MLH-Fellowship/scikit-learn
#1
@jnothman

Description

@jnothman

While sample and feature order can have subtle effects on the model fit by an estimator, I think we should have common tests to ensure that reordering or subsampling X in predict or transform or score_samples or predict_proba or decision_function does not change the sample-wise output. That is:

idx = np.random.randint(X.shape[0], size=X.shape[0] // 2)
assert_array_equal(method(X)[idx], method(X[idx]))

Apologies if we already have such tests, but I can't see them (which is also an issue: we don't actually have a clear list of what is asserted by estimator_checks)

Metadata

Metadata

Assignees

No one assigned

    Labels

    EasyWell-defined and straightforward way to resolve

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions