Skip to content

Test - sklearn/utils/estimator_checks.py::check_supervised_y_2d does not clone estimator #10977

Closed
@omtinez

Description

@omtinez

Description

Test function check_supervised_y_2d reuses an already fitted estimator which could leave potential side effects. For example, if an estimator uses a flag to only set random state once, the call to set_random_state(estimator) will have no effect and that can lead to a false negative (failure) in this test.

Steps/Code to Reproduce

import numpy
import warnings
from sklearn.base import BaseEstimator
from sklearn.exceptions import DataConversionWarning
from sklearn.utils.estimator_checks import check_supervised_y_2d

class RoundRobinClassifier(BaseEstimator):
    def __init__(self, random_state=0):
        self.random_state = random_state
        
    def fit(self, X, y):
        if len(y.shape[1:]) >= 1:
            warnings.warn(
                'A column-vector y was passed when a 1d array was expected',
                DataConversionWarning)
        if not hasattr(self, 'idx'):
            self.idx = int(self.random_state)
        self.classes_ = numpy.unique(y)
        return self
    
    def predict(self, X):
        pred = []
        for i, _ in enumerate(X):
            pred.append(self.classes_[self.idx % len(self.classes_)])
            self.idx = (self.idx + 1) % len(self.classes_)
        return numpy.asarray(pred)

classif = RoundRobinClassifier()
check_supervised_y_2d(classif.__class__.__name__, classif)

Expected Results

Test passes, since the required warning is thrown.

Actual Results

AssertionError:
Not equal to tolerance rtol=1e-07, atol=0

(mismatch 100.0%)
x: array([0, 1, 2, 0, 1, 2, 0, 1, 2, 0])
y: array([1, 2, 0, 1, 2, 0, 1, 2, 0, 1])

Versions

Platform info:

  • Darwin-17.4.0-x86_64-i386-64bit
  • Python 3.6.3 |Intel Corporation| (default, Oct 16 2017, 10:30:26)
  • [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)]
  • NumPy 1.14.2
  • SciPy 1.0.1

Tested in the following versions:

  • Scikit-Learn 0.19.1
  • Scikit-Learn 0.20.dev0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions