Skip to content

test_uniform_grid[barnes_hut] can fail on aarch64 #15821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hmaarrfk opened this issue Dec 7, 2019 · 4 comments
Open

test_uniform_grid[barnes_hut] can fail on aarch64 #15821

hmaarrfk opened this issue Dec 7, 2019 · 4 comments
Labels
arch:arm ARM related issues Bug help wanted module:test-suite everything related to our tests

Comments

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Dec 7, 2019

________________________ test_uniform_grid[barnes_hut] _________________________

method = 'barnes_hut'

    @pytest.mark.parametrize('method', ['barnes_hut', 'exact'])
    def test_uniform_grid(method):
        """Make sure that TSNE can approximately recover a uniform 2D grid
    
        Due to ties in distances between point in X_2d_grid, this test is platform
        dependent for ``method='barnes_hut'`` due to numerical imprecision.
    
        Also, t-SNE is not assured to converge to the right solution because bad
        initialization can lead to convergence to bad local minimum (the
        optimization problem is non-convex). To avoid breaking the test too often,
        we re-run t-SNE from the final point when the convergence is not good
        enough.
        """
        seeds = [0, 1, 2]
        n_iter = 500
        for seed in seeds:
            tsne = TSNE(n_components=2, init='random', random_state=seed,
                        perplexity=20, n_iter=n_iter, method=method)
            Y = tsne.fit_transform(X_2d_grid)
    
            try_name = "{}_{}".format(method, seed)
            try:
>               assert_uniform_grid(Y, try_name)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:784: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

Y = array([[ 52.326397  , -15.92225   ],
       [ 46.679527  , -20.175953  ],
       [ 40.870537  , -24.181147  ],
       ...[-35.291374  ,  22.122814  ],
       [-42.2738    ,  18.793724  ],
       [-48.922283  ,  15.606232  ]], dtype=float32)
try_name = 'barnes_hut_1'

    def assert_uniform_grid(Y, try_name=None):
        # Ensure that the resulting embedding leads to approximately
        # uniformly spaced points: the distance to the closest neighbors
        # should be non-zero and approximately constant.
        nn = NearestNeighbors(n_neighbors=1).fit(Y)
        dist_to_nn = nn.kneighbors(return_distance=True)[0].ravel()
        assert dist_to_nn.min() > 0.1
    
        smallest_to_mean = dist_to_nn.min() / np.mean(dist_to_nn)
        largest_to_mean = dist_to_nn.max() / np.mean(dist_to_nn)
    
        assert smallest_to_mean > .5, try_name
>       assert largest_to_mean < 2, try_name
E       AssertionError: barnes_hut_1
E       assert 6.67359409617653 < 2

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:807: AssertionError

During handling of the above exception, another exception occurred:

method = 'barnes_hut'

    @pytest.mark.parametrize('method', ['barnes_hut', 'exact'])
    def test_uniform_grid(method):
        """Make sure that TSNE can approximately recover a uniform 2D grid
    
        Due to ties in distances between point in X_2d_grid, this test is platform
        dependent for ``method='barnes_hut'`` due to numerical imprecision.
    
        Also, t-SNE is not assured to converge to the right solution because bad
        initialization can lead to convergence to bad local minimum (the
        optimization problem is non-convex). To avoid breaking the test too often,
        we re-run t-SNE from the final point when the convergence is not good
        enough.
        """
        seeds = [0, 1, 2]
        n_iter = 500
        for seed in seeds:
            tsne = TSNE(n_components=2, init='random', random_state=seed,
                        perplexity=20, n_iter=n_iter, method=method)
            Y = tsne.fit_transform(X_2d_grid)
    
            try_name = "{}_{}".format(method, seed)
            try:
                assert_uniform_grid(Y, try_name)
            except AssertionError:
                # If the test fails a first time, re-run with init=Y to see if
                # this was caused by a bad initialization. Note that this will
                # also run an early_exaggeration step.
                try_name += ":rerun"
                tsne.init = Y
                Y = tsne.fit_transform(X_2d_grid)
>               assert_uniform_grid(Y, try_name)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:792: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

Y = array([[-18.169476  ,   6.0802336 ],
       [-18.278513  ,   2.8822129 ],
       [-18.671782  ,  -0.4646889 ],
       ...[ 22.550077  ,  19.698557  ],
       [ 21.399723  ,  22.933178  ],
       [ 16.22136   ,  28.22955   ]], dtype=float32)
try_name = 'barnes_hut_1:rerun'

    def assert_uniform_grid(Y, try_name=None):
        # Ensure that the resulting embedding leads to approximately
        # uniformly spaced points: the distance to the closest neighbors
        # should be non-zero and approximately constant.
        nn = NearestNeighbors(n_neighbors=1).fit(Y)
        dist_to_nn = nn.kneighbors(return_distance=True)[0].ravel()
        assert dist_to_nn.min() > 0.1
    
        smallest_to_mean = dist_to_nn.min() / np.mean(dist_to_nn)
        largest_to_mean = dist_to_nn.max() / np.mean(dist_to_nn)
    
        assert smallest_to_mean > .5, try_name
>       assert largest_to_mean < 2, try_name
E       AssertionError: barnes_hut_1:rerun
E       assert 2.145051767903112 < 2

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:807: AssertionError

The following NEW packages will be INSTALLED:

    apipkg:                      1.5-py_0               conda-forge
    attrs:                       19.3.0-py_0            conda-forge
    binutils_impl_linux-aarch64: 2.29.1-hc862510_0      c4aarch64  
    ca-certificates:             2019.11.28-hecc5488_0  conda-forge
    certifi:                     2019.11.28-py38_0      conda-forge
    cython:                      0.29.14-py38he1b5a44_0 conda-forge
    execnet:                     1.7.1-py_0             conda-forge
    importlib_metadata:          1.2.0-py38_0           conda-forge
    joblib:                      0.14.0-py_0            conda-forge
    libblas:                     3.8.0-14_openblas      conda-forge
    libcblas:                    3.8.0-14_openblas      conda-forge
    libffi:                      3.2.1-h4c5d2ac_1006    conda-forge
    libgcc-ng:                   7.3.0-h5c90dd9_0       c4aarch64  
    libgfortran-ng:              7.3.0-h6bc79d0_0       c4aarch64  
    liblapack:                   3.8.0-14_openblas      conda-forge
    libopenblas:                 0.3.7-h5ec1e0e_4       conda-forge
    libstdcxx-ng:                7.3.0-h5c90dd9_0       c4aarch64  
    more-itertools:              8.0.2-py_0             conda-forge
    ncurses:                     6.1-hf484d3e_1002      conda-forge
    numpy:                       1.17.3-py38h91f3968_0  conda-forge
    openssl:                     1.1.1d-h516909a_0      conda-forge
    packaging:                   19.2-py_0              conda-forge
    pluggy:                      0.13.1-py38_0          conda-forge
    py:                          1.8.0-py_0             conda-forge
    pyparsing:                   2.4.5-py_0             conda-forge
    pytest:                      5.3.1-py38_0           conda-forge
    pytest-forked:               1.1.2-py_0             conda-forge
    pytest-sugar:                0.9.2-py_0             conda-forge
    pytest-timeout:              1.3.3-py_0             conda-forge
    pytest-xdist:                1.30.0-py_0            conda-forge
    python:                      3.8.0-heaf0f07_5       conda-forge
    readline:                    8.0-h75b48e3_0         conda-forge
    scikit-learn:                0.22-py38h1971d64_0    local      
    scipy:                       1.3.2-py38hb5cb654_0   conda-forge
    setuptools:                  42.0.2-py38_0          conda-forge
    six:                         1.13.0-py38_0          conda-forge
    sqlite:                      3.30.1-h283c62a_0      conda-forge
    termcolor:                   1.1.0-py_2             conda-forge
    tk:                          8.6.10-hed695b0_0      conda-forge
    wcwidth:                     0.1.7-py_1             conda-forge
    xz:                          5.2.4-hda93590_1001    conda-forge
    zipp:                        0.6.0-py_0             conda-forge
    zlib:                        1.2.11-h516909a_1006   conda-forge

https://cloud.drone.io/conda-forge/scikit-learn-feedstock/32/3/2

@ogrisel
Copy link
Member

ogrisel commented Mar 3, 2020

For reference this also fails for ppc64le architectures.

@ogrisel
Copy link
Member

ogrisel commented Mar 3, 2020

@ogrisel
Copy link
Member

ogrisel commented Mar 3, 2020

I could reproduce a similar failures by trying more random seeds on my Linux amd64 laptop.

@sandrotosi
Copy link

this also fails on arm64

raspbian-autopush pushed a commit to raspbian-packages/scikit-learn that referenced this issue May 7, 2020
Origin: upstream, scikit-learn/scikit-learn#15821

Gbp-Pq: Name Fix-for-test_uniform_grid.patch
@cmarmo cmarmo added arch:arm ARM related issues module:test-suite everything related to our tests Bug help wanted and removed Build / CI labels Sep 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch:arm ARM related issues Bug help wanted module:test-suite everything related to our tests
Projects
None yet
Development

No branches or pull requests

4 participants