Skip to content

[0.20.2] test_non_meta_estimators fails on Powerpc 64 bit little endian #13051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
olebole opened this issue Jan 27, 2019 · 6 comments
Closed
Milestone

Comments

@olebole
Copy link

olebole commented Jan 27, 2019

Splitting #13036:
Test failure for test_non_meta_estimators[KernelPCA-KernelPCA-check_pipeline_consistency]:

name = 'KernelPCA'
Estimator = <class 'sklearn.decomposition.kernel_pca.KernelPCA'>
check = <function check_pipeline_consistency at 0x3fff8b056b18>

    @pytest.mark.parametrize(
            "name, Estimator, check",
            _generate_checks_per_estimator(_yield_all_checks,
                                           _tested_non_meta_estimators()),
            ids=_rename_partial
    )
    def test_non_meta_estimators(name, Estimator, check):
        # Common tests for non-meta estimators
        with ignore_warnings(category=(DeprecationWarning, ConvergenceWarning,
                                       UserWarning, FutureWarning)):
            estimator = Estimator()
            set_checking_parameters(estimator)
>           check(name, estimator)

sklearn/tests/test_common.py:101: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
sklearn/utils/testing.py:350: in wrapper
    return fn(*args, **kwargs)
sklearn/utils/estimator_checks.py:1048: in check_pipeline_consistency
    assert_allclose_dense_sparse(result, result_pipe)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

x = array([[ 9.17543358e-01,  4.04410079e-02,  2.21585378e-02,
        -2.69563828...,  2.47000231e-11,
         2.81623715e-09,  1.36863809e-09,  1.28803077e-09]])
y = array([[ 9.17543358e-01,  4.04410079e-02,  2.21585378e-02,
        -2.69560026..., -6.90266524e-10,
        -8.33997668e-11,  1.37018888e-09,  2.03087265e-09]])
rtol = 1e-07, atol = 1e-09, err_msg = ''

    def assert_allclose_dense_sparse(x, y, rtol=1e-07, atol=1e-9, err_msg=''):
        """[...]"""
        if sp.sparse.issparse(x) and sp.sparse.issparse(y):
            x = x.tocsr()
            y = y.tocsr()
            x.sum_duplicates()
            y.sum_duplicates()
            assert_array_equal(x.indices, y.indices, err_msg=err_msg)
            assert_array_equal(x.indptr, y.indptr, err_msg=err_msg)
            assert_allclose(x.data, y.data, rtol=rtol, atol=atol, err_msg=err_msg)
        elif not sp.sparse.issparse(x) and not sp.sparse.issparse(y):
            # both dense
>           assert_allclose(x, y, rtol=rtol, atol=atol, err_msg=err_msg)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=1e-09
E           
E           (mismatch 13.7777777778%)
E            x: array([[ 9.175434e-01,  4.044101e-02,  2.215854e-02, -2.695638e-08,
E                    3.128852e-08,  0.000000e+00,  0.000000e+00,  0.000000e+00,
E                    0.000000e+00,  0.000000e+00,  0.000000e+00,  0.000000e+00,...
E            y: array([[ 9.175434e-01,  4.044101e-02,  2.215854e-02, -2.695600e-08,
E                    3.128926e-08,  0.000000e+00,  0.000000e+00,  0.000000e+00,
E                    0.000000e+00,  0.000000e+00,  0.000000e+00,  0.000000e+00,...

sklearn/utils/testing.py:464: AssertionError

Similarly for test_non_meta_estimators[LocallyLinearEmbedding-LocallyLinearEmbedding-check_pipeline_consistency]:

>           assert_allclose(x, y, rtol=rtol, atol=atol, err_msg=err_msg)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=1e-09
E           
E           (mismatch 50.0%)
E            x: array([[-2.011707e-01,  2.825862e-01],
E                  [-1.618549e-01,  1.102424e-12],
E                  [-1.618549e-01,  3.172043e-11],...
E            y: array([[-1.753929e-01,  2.825862e-01],
E                  [-1.894835e-01, -1.533793e-11],
E                  [-1.894835e-01,  3.785336e-11],...

and for test_non_meta_estimators[LocallyLinearEmbedding-LocallyLinearEmbedding-check_transformer_data_not_an_array]:

>           assert_allclose(x, y, rtol=rtol, atol=atol, err_msg=err_msg)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=0.01
E           fit_transform and transform outcomes not consistent in LocallyLinearEmbedding(eigen_solver='auto', hessian_tol=0.0001, max_iter=5,
E                       method='standard', modified_tol=1e-12, n_components=2,
E                       n_jobs=None, n_neighbors=5, neighbors_algorithm='auto',
E                       random_state=0, reg=0.001, tol=1e-06)
E           (mismatch 50.0%)
E            x: array([[-8.953081e-02,  2.983824e-01],
E                  [-2.421795e-01,  4.555523e-13],
E                  [-2.421795e-01,  3.751804e-12],...
E            y: array([[-1.897519e-01,  2.981428e-01],
E                  [-1.751025e-01,  6.989894e-12],
E                  [-1.751025e-01,  9.856396e-12],...

and for test_non_meta_estimators[LocallyLinearEmbedding-LocallyLinearEmbedding-check_transformer_general]:

>           assert_allclose(x, y, rtol=rtol, atol=atol, err_msg=err_msg)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=0.01
E           fit_transform and transform outcomes not consistent in LocallyLinearEmbedding(eigen_solver='auto', hessian_tol=0.0001, max_iter=5,
E                       method='standard', modified_tol=1e-12, n_components=2,
E                       n_jobs=None, n_neighbors=5, neighbors_algorithm='auto',
E                       random_state=0, reg=0.001, tol=1e-06)
E           (mismatch 50.0%)
E            x: array([[ 4.698100e-02, -2.983824e-01],
E                  [ 2.538887e-01, -1.711825e-12],
E                  [ 2.538887e-01, -8.653966e-12],...
E            y: array([[-2.546916e-01, -2.981428e-01],
E                  [ 4.241318e-02, -6.757535e-12],
E                  [ 4.241318e-02, -2.107003e-11],...
@rth rth added this to the 0.20.3 milestone Jan 27, 2019
@olebole
Copy link
Author

olebole commented Jan 27, 2019

BTW, I just found that a similar issue is #12448.

@jnothman
Copy link
Member

We long ago made exceptions for CCA, LocallyLinearEmbedding and KernelPCA saying that they are not deterministic on 32-bit platforms. This happened in #3363 when introducing Windows CI. But it seems that wasn't quite the right criteria for exclusion.

@rth
Copy link
Member

rth commented Feb 26, 2019

With the latest version on master we have the non_deterministic estimator tag. We could set it to true for 32 bit arch or powerpc. How could we reliably detect it?

import platform
platform.machine.lower().startswith('powerpc')

looking at CPython config.sub but I'm not sure if those actually match platform.machine..

@rth
Copy link
Member

rth commented Feb 26, 2019

From #12448 (comment),

machine: Linux-4.9.0-8-powerpc64le-ppc64le-with-debian-buster-sid

Unless platform.machine is "ppc64le" hmm.

@rth
Copy link
Member

rth commented Feb 26, 2019

Actually, for KernelPCA this may have been fixed in #13241 -- not sure..

@rth
Copy link
Member

rth commented Feb 28, 2019

Made at attempt to skip the non deterministic tests in #13323

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants