FIX array_api support for non-integer n_components in PCA #27431

ogrisel · 2023-09-21T12:50:57Z

Our array api tests did not check for cases such as PCA(n_components=.2) and PCA(n_components="mle") and therefore did not check some helper functions such as stable_cumsum and _assess_dimension.

Here is a start to do so but:

we would need Add support for computing the cumulative sum to the standard data-apis/array-api#597
stable_cumsum tries to force the use of xp.float64 which is not available on MPS devices for PyTorch.
we would also need RFC: add searchsorted to the specification data-apis/array-api#688

In the long term, I think we might want to deprecate stable_cumsum and try to use a regular cumsum with the input data dtype instead.

EDIT: In the mean time, we can just do the cumsum/searchsorted calls on a local numpy copy, this is not an expensive operation in the case of the PCA (only depends on n_features).

github-actions · 2023-09-21T12:52:47Z

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 23f2c57. Link to the linter CI: here}

sklearn/decomposition/tests/test_pca.py

…ends do not support float64

ogrisel · 2023-09-27T18:10:55Z

I converting this PR back to draft because xp.searchsorted does not work as expected for cupy:

============================================================================ FAILURES ============================================================================
_______________ test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-cupy-None-None] _______________

estimator = PCA(n_components=0.1, svd_solver='full', whiten=True), check = <function check_array_api_input_and_values at 0x7f1dca8adda0>, array_namespace = 'cupy'
device = None, dtype = None

    @pytest.mark.parametrize(
        "array_namespace, device, dtype", yield_namespace_device_dtype_combinations()
    )
    @pytest.mark.parametrize(
        "check",
        [check_array_api_input_and_values, check_array_api_get_precision],
        ids=_get_check_estimator_ids,
    )
    @pytest.mark.parametrize(
        "estimator",
        [
            PCA(n_components=2, svd_solver="full"),
            PCA(n_components=0.1, svd_solver="full", whiten=True),
            PCA(
                n_components=2,
                svd_solver="randomized",
                power_iteration_normalizer="QR",
                random_state=0,  # how to use global_random_seed here?
            ),
        ],
        ids=_get_check_estimator_ids,
    )
    def test_pca_array_api_compliance(estimator, check, array_namespace, device, dtype):
        name = estimator.__class__.__name__
        xp, device, dtype = _array_api_for_tests(array_namespace, device, dtype)
        if not isinstance(estimator.n_components, int) and not hasattr(xp, "cumsum"):
            # Our code anticipates the implementation of xp.cumsum that should be
            # standardized at some point, see:
            # https://github.com/data-apis/array-api/issues/597
            pytest.xfail(
                f"Array API namespace {array_namespace} does not support cumsum yet."
            )
        if not isinstance(estimator.n_components, int) and not hasattr(xp, "searchsorted"):
            # Our code anticipates the implementation of xp.searchsorted that
            # should be standardized at some point, see:
            # https://github.com/data-apis/array-api/issues/688
            pytest.xfail(
                f"Array API namespace {array_namespace} does not support searchsorted yet."
            )
>       check(name, estimator, array_namespace, device=device, dtype=dtype)

array_namespace = 'cupy'
check      = <function check_array_api_input_and_values at 0x7f1dca8adda0>
device     = None
dtype      = None
estimator  = PCA(n_components=0.1, svd_solver='full', whiten=True)
name       = 'PCA'
xp         = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>

sklearn/decomposition/tests/test_pca.py:773: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
sklearn/utils/estimator_checks.py:1002: in check_array_api_input_and_values
    return check_array_api_input(
        array_namespace = 'cupy'
        device     = None
        dtype      = None
        estimator_orig = PCA(n_components=0.1, svd_solver='full', whiten=True)
        name       = 'PCA'
sklearn/utils/estimator_checks.py:889: in check_array_api_input
    est_xp.fit(X_xp, y_xp)
        X          = array([[-2.02514259,  0.0291022 , -0.47494531, ..., -0.33450124,
         0.86575519, -1.20029641],
       [ 1.6137112...623, -1.06511366],
       [-0.00607091,  1.30857636, -0.17495976, ...,  0.99204235,
         0.32169781, -0.66809045]])
        X_xp       = array([[-2.02514259,  0.0291022 , -0.47494531, ..., -0.33450124,
         0.86575519, -1.20029641],
       [ 1.6137112...623, -1.06511366],
       [-0.00607091,  1.30857636, -0.17495976, ...,  0.99204235,
         0.32169781, -0.66809045]])
        array_attributes = {'components_': array([[-0.03484641, -0.60455227,  0.09228081,  0.09317968, -0.02118701,
         0.46225127, -0.03672..., -0.03647842,  0.00852413,  0.13682879,
       -0.03120609,  0.05840072,  0.05582498,  0.12556161, -0.03976961]), ...}
        array_namespace = 'cupy'
        check_values = True
        device     = None
        dtype      = None
        est        = PCA(n_components=0.1, svd_solver='full', whiten=True)
        est_xp     = PCA(n_components=0.1, svd_solver='full', whiten=True)
        estimator_orig = PCA(n_components=0.1, svd_solver='full', whiten=True)
        name       = 'PCA'
        xp         = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>
        y          = array([0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,
       0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0,...1,
       0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
       1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0])
        y_xp       = array([0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,
       0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0,...1,
       0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
       1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0])
sklearn/base.py:1215: in wrapper
    return fit_method(estimator, *args, **kwargs)
        args       = (array([[-2.02514259,  0.0291022 , -0.47494531, ..., -0.33450124,
         0.86575519, -1.20029641],
       [ 1.613711...,
       0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
       1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0]))
        estimator  = PCA(n_components=0.1, svd_solver='full', whiten=True)
        fit_method = <function PCA.fit at 0x7f1dcac16c00>
        global_skip_validation = False
        kwargs     = {}
        partial_fit_and_fitted = False
        prefer_skip_nested_validation = True
sklearn/decomposition/_pca.py:438: in fit
    self._fit(X)
        X          = array([[-2.02514259,  0.0291022 , -0.47494531, ..., -0.33450124,
         0.86575519, -1.20029641],
       [ 1.6137112...623, -1.06511366],
       [-0.00607091,  1.30857636, -0.17495976, ...,  0.99204235,
         0.32169781, -0.66809045]])
        self       = PCA(n_components=0.1, svd_solver='full', whiten=True)
        y          = array([0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,
       0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0,...1,
       0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
       1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0])
sklearn/decomposition/_pca.py:520: in _fit
    return self._fit_full(X, n_components)
        X          = array([[-2.21502293, -0.00923   , -0.55143331, ..., -0.39032622,
         0.74019358, -1.1605268 ],
       [ 1.4238309...785, -1.02534406],
       [-0.19595126,  1.27024416, -0.25144776, ...,  0.93621737,
         0.19613619, -0.62832085]])
        is_array_api_compliant = True
        n_components = 0.1
        self       = PCA(n_components=0.1, svd_solver='full', whiten=True)
        xp         = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>
sklearn/decomposition/_pca.py:585: in _fit_full
    n_components = xp.searchsorted(ratio_cumsum, n_components, side="right") + 1
        S          = array([2.02981634e+01, 1.42856603e+01, 1.28046793e+01, 1.21903333e+01,
       1.18505054e+01, 1.16648157e+01, 1.109279...8.04206402e+00, 7.80207531e+00, 7.50569464e+00,
       7.01282418e+00, 6.55356018e+00, 2.57592758e-15, 8.36741292e-16])
        U          = array([[ 0.00893647,  0.05215268, -0.09820667, ..., -0.0008873 ,
         0.12132781,  0.01039382],
       [-0.0242668...911,  0.03320314],
       [-0.05989941, -0.06672174, -0.08371833, ...,  0.04763651,
        -0.03077404,  0.01670712]])
        Vt         = array([[-3.48464090e-02, -6.04552271e-01,  9.22808056e-02,
         9.31796838e-02, -2.11870097e-02,  4.62251267e-01,
...-4.62021233e-17,
         6.93044994e-01,  5.18349048e-17,  1.48737107e-17,
         2.97854026e-17, -2.14399161e-17]])
        X          = array([[-2.21502293, -0.00923   , -0.55143331, ..., -0.39032622,
         0.74019358, -1.1605268 ],
       [ 1.4238309...785, -1.02534406],
       [-0.19595126,  1.27024416, -0.25144776, ...,  0.93621737,
         0.19613619, -0.62832085]])
        components_ = array([[-3.48464090e-02, -6.04552271e-01,  9.22808056e-02,
         9.31796838e-02, -2.11870097e-02,  4.62251267e-01,
...-4.62021233e-17,
         6.93044994e-01,  5.18349048e-17,  1.48737107e-17,
         2.97854026e-17, -2.14399161e-17]])
        explained_variance_ = array([4.16177210e+00, 2.06141506e+00, 1.65615972e+00, 1.50105278e+00,
       1.41853008e+00, 1.37442348e+00, 1.242930...6.53280745e-01, 6.14872516e-01, 5.69044969e-01,
       4.96764677e-01, 4.33829808e-01, 6.70242715e-32, 7.07208071e-33])
        explained_variance_ratio_ = array([1.92326985e-01, 9.52636840e-02, 7.65357155e-02, 6.93677955e-02,
       6.55541933e-02, 6.35159055e-02, 5.743924...3.01899078e-02, 2.84149574e-02, 2.62971399e-02,
       2.29568679e-02, 2.00484738e-02, 3.09737673e-33, 3.26820385e-34])
        is_array_api_compliant = True
        n_components = 0.1
        n_features = 20
        n_samples  = 100
        ratio_cumsum = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
       0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
       0.95699466, 0.97995153, 1.        , 1.        , 1.        ])
        self       = PCA(n_components=0.1, svd_solver='full', whiten=True)
        singular_values_ = array([2.02981634e+01, 1.42856603e+01, 1.28046793e+01, 1.21903333e+01,
       1.18505054e+01, 1.16648157e+01, 1.109279...8.04206402e+00, 7.80207531e+00, 7.50569464e+00,
       7.01282418e+00, 6.55356018e+00, 2.57592758e-15, 8.36741292e-16])
        total_var  = array(21.63904407)
        xp         = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>
/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/cupy/_sorting/search.py:422: in searchsorted
    return _searchsorted(a, v, side, sorter, True)
        a          = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
       0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
       0.95699466, 0.97995153, 1.        , 1.        , 1.        ])
        side       = 'right'
        sorter     = None
        v          = 0.1
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

a = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
       0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
       0.95699466, 0.97995153, 1.        , 1.        , 1.        ])
v = 0.1, side = 'right', sorter = None, assume_increasing = True

    def _searchsorted(a, v, side, sorter, assume_increasing):
        """`assume_increasing` is used in the kernel to
        skip monotonically increasing or decreasing verification
        inside the cuda kernel.
        """
        if not isinstance(a, cupy.ndarray):
            raise NotImplementedError('Only int or ndarray are supported for a')
    
        if not isinstance(v, cupy.ndarray):
>           raise NotImplementedError('Only int or ndarray are supported for v')
E           NotImplementedError: Only int or ndarray are supported for v

a          = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
       0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
       0.95699466, 0.97995153, 1.        , 1.        , 1.        ])
assume_increasing = True
side       = 'right'
sorter     = None
v          = 0.1

/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/cupy/_sorting/search.py:434: NotImplementedError

ogrisel · 2023-09-28T07:44:53Z

Ok now it seems to work on all supported hardware/library combinations:

with a cuda GPU with PyTorch and CuPy:

(pytorch) [ogrisel@margpu009 scikit-learn]$ pytest -vl sklearn/decomposition/tests/test_pca.py  -k "array_api" -x
============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.11.5, pytest-7.4.2, pluggy-1.3.0 -- /data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/bin/python3.11
cachedir: .pytest_cache
rootdir: /home/soda/ogrisel/code/scikit-learn
configfile: setup.cfg
collected 193 items / 120 deselected / 73 selected                                                                                                                                                               

sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-numpy-None-None] PASSED                                      [  1%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-numpy.array_api-None-None] PASSED                            [  2%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-cupy-None-None] PASSED                                       [  4%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-cupy.array_api-None-None] PASSED                             [  5%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-torch-cpu-float64] PASSED                                    [  6%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-torch-cpu-float32] PASSED                                    [  8%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-torch-cuda-float64] PASSED                                   [  9%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-torch-cuda-float32] PASSED                                   [ 10%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-torch-mps-float32] SKIPPED (Skipping MPS device test bec...) [ 12%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-numpy-None-None] PASSED                                         [ 13%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-numpy.array_api-None-None] PASSED                               [ 15%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-cupy-None-None] PASSED                                          [ 16%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-cupy.array_api-None-None] PASSED                                [ 17%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-torch-cpu-float64] PASSED                                       [ 19%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-torch-cpu-float32] PASSED                                       [ 20%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-torch-cuda-float64] PASSED                                      [ 21%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-torch-cuda-float32] PASSED                                      [ 23%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-torch-mps-float32] SKIPPED (Skipping MPS device test becaus...) [ 24%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-numpy-None-None] PASSED                        [ 26%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-numpy.array_api-None-None] PASSED              [ 27%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-cupy-None-None] PASSED                         [ 28%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-cupy.array_api-None-None] PASSED               [ 30%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-torch-cpu-float64] PASSED                      [ 31%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-torch-cpu-float32] PASSED                      [ 32%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-torch-cuda-float64] PASSED                     [ 34%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-torch-cuda-float32] PASSED                     [ 35%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-torch-mps-float32] SKIPPED (Skipping MPS d...) [ 36%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-numpy-None-None] PASSED                           [ 38%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-numpy.array_api-None-None] PASSED                 [ 39%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-cupy-None-None] PASSED                            [ 41%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-cupy.array_api-None-None] PASSED                  [ 42%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-torch-cpu-float64] PASSED                         [ 43%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-torch-cpu-float32] PASSED                         [ 45%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-torch-cuda-float64] PASSED                        [ 46%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-torch-cuda-float32] PASSED                        [ 47%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-torch-mps-float32] SKIPPED (Skipping MPS devi...) [ 49%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-numpy-None-None] PASSED [ 50%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-numpy.array_api-None-None] PASSED [ 52%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-cupy-None-None] PASSED [ 53%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-cupy.array_api-None-None] PASSED [ 54%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-torch-cpu-float64] PASSED [ 56%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-torch-cpu-float32] PASSED [ 57%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-torch-cuda-float64] PASSED [ 58%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-torch-cuda-float32] PASSED [ 60%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-torch-mps-float32] SKIPPED [ 61%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-numpy-None-None] PASSED [ 63%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-numpy.array_api-None-None] PASSED [ 64%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-cupy-None-None] PASSED [ 65%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-cupy.array_api-None-None] PASSED [ 67%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-torch-cpu-float64] PASSED [ 68%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-torch-cpu-float32] PASSED [ 69%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-torch-cuda-float64] PASSED [ 71%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-torch-cuda-float32] PASSED [ 72%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-torch-mps-float32] SKIPPED [ 73%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-numpy-None-None] PASSED                                         [ 75%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-numpy.array_api-None-None] PASSED                               [ 76%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-cupy-None-None] PASSED                                          [ 78%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-cupy.array_api-None-None] PASSED                                [ 79%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-torch-cpu-float64] PASSED                                       [ 80%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-torch-cpu-float32] PASSED                                       [ 82%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-torch-cuda-float64] PASSED                                      [ 83%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-torch-cuda-float32] PASSED                                      [ 84%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-torch-mps-float32] SKIPPED (Skipping MPS device test becaus...) [ 86%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-numpy-None-None] PASSED                                 [ 87%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-numpy.array_api-None-None] PASSED                       [ 89%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-cupy-None-None] PASSED                                  [ 90%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-cupy.array_api-None-None] PASSED                        [ 91%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-torch-cpu-float64] PASSED                               [ 93%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-torch-cpu-float32] PASSED                               [ 94%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-torch-cuda-float64] PASSED                              [ 95%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-torch-cuda-float32] PASSED                              [ 97%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-torch-mps-float32] SKIPPED (Skipping MPS device tes...) [ 98%]
sklearn/decomposition/tests/test_pca.py::test_array_api_error_and_warnings_on_unsupported_params PASSED                                                                                                    [100%]

=========================================================================== 65 passed, 8 skipped, 120 deselected, 33 warnings in 6.29s ===========================================================================

on Apple M1 with PyTorch and the MPS device:

dev ❯ PYTORCH_ENABLE_MPS_FALLBACK=1 pytest -vl sklearn/decomposition/tests/test_pca.py  -k "array_api and mps"
============================================================================================== test session starts ===============================================================================================
platform darwin -- Python 3.11.0, pytest-7.4.0, pluggy-1.0.0 -- /Users/ogrisel/mambaforge/envs/dev/bin/python3.11
cachedir: .pytest_cache
Using --randomly-seed=3465581367
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/ogrisel/code/scikit-learn
configfile: setup.cfg
plugins: snapshot-0.9.0, memray-1.4.0, xdist-3.1.0, line-profiler-0.2.1, timeout-2.1.0, mock-3.10.0, pytest_notebook-0.8.1, clarity-1.0.1, randomly-3.12.0, profiling-1.7.0, cov-4.0.0, repeat-0.9.1, anyio-3.6.2, benchmark-4.0.0
collected 193 items / 185 deselected / 8 selected                                                                                                                                                                

sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_input-torch-mps-float32] PASSED                                       [ 12%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_get_precision-torch-mps-float32] PASSED                         [ 25%]
sklearn/decomposition/tests/test_pca.py::test_pca_mle_array_api_compliance[PCA(n_components='mle',svd_solver='full')-check_array_api_get_precision-torch-mps-float32] PASSED                               [ 37%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-torch-mps-float32] PASSED [ 50%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_get_precision-torch-mps-float32] PASSED                                       [ 62%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-torch-mps-float32] PASSED [ 75%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-torch-mps-float32] PASSED                      [ 87%]
sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full')-check_array_api_input_and_values-torch-mps-float32] PASSED                                    [100%]

================================================================================== 8 passed, 185 deselected, 1 warning in 4.00s ==================================================================================

ogrisel · 2023-09-28T08:25:25Z

/cc @betatim @thomasjpfan

sklearn/utils/extmath.py

betatim · 2023-10-13T13:21:35Z

LGTM

thomasjpfan

LGTM

…rn#27431)

FIX array_api support for non-integer n_components in PCA

eceb63e

github-actions bot added module:decomposition module:utils labels Sep 21, 2023

ogrisel marked this pull request as draft September 21, 2023 12:52

TST test (and fix) for n_components='mle'

cc4784b

ogrisel commented Sep 21, 2023

View reviewed changes

sklearn/decomposition/tests/test_pca.py Outdated Show resolved Hide resolved

Typo

0c41916

ogrisel added the Array API label Sep 22, 2023

ogrisel added 4 commits September 27, 2023 19:17

Do not make stable_cumsum pretend to work with array api as some back…

3569edc

…ends do not support float64

Revert to a simpler version of numpy-specific stable_cumsum

209fa47

Even simpler version of numpy-specific stable_cumsum

34ff427

Document change in the changelog

ff3a93c

ogrisel marked this pull request as ready for review September 27, 2023 18:00

Merge branch 'main' into follow-up-pca-array_api

621258b

ogrisel marked this pull request as draft September 27, 2023 18:10

ogrisel added 2 commits September 28, 2023 09:28

Always perform cumsum and searchsorted using numpy for now

723d19d

Restore local variable to avoid complex multi-line formatting

68d0b9c

ogrisel marked this pull request as ready for review September 28, 2023 07:45

Merge branch 'main' into follow-up-pca-array_api

62ff348

ogrisel mentioned this pull request Sep 28, 2023

[Array API] stable_cumsum uses np.float64 rather than xp.float64 #27427

Closed

ogrisel added the Waiting for Reviewer label Sep 28, 2023

betatim reviewed Sep 29, 2023

View reviewed changes

sklearn/utils/extmath.py Show resolved Hide resolved

betatim approved these changes Oct 13, 2023

View reviewed changes

ogrisel added the Waiting for Second Reviewer First reviewer is done, need a second one! label Oct 18, 2023

Merge branch 'main' into follow-up-pca-array_api

23f2c57

thomasjpfan approved these changes Nov 15, 2023

View reviewed changes

thomasjpfan merged commit 5c09163 into scikit-learn:main Nov 15, 2023

REDVM pushed a commit to REDVM/scikit-learn that referenced this pull request Nov 16, 2023

FIX array_api support for non-integer n_components in PCA (scikit-lea…

3363080

…rn#27431)

ogrisel deleted the follow-up-pca-array_api branch November 23, 2023 17:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FIX array_api support for non-integer n_components in PCA #27431

FIX array_api support for non-integer n_components in PCA #27431

ogrisel commented Sep 21, 2023 •

edited

Loading

github-actions bot commented Sep 21, 2023 •

edited

Loading

ogrisel commented Sep 27, 2023 •

edited

Loading

ogrisel commented Sep 28, 2023

ogrisel commented Sep 28, 2023

betatim commented Oct 13, 2023

thomasjpfan left a comment

FIX array_api support for non-integer n_components in PCA #27431

FIX array_api support for non-integer n_components in PCA #27431

Conversation

ogrisel commented Sep 21, 2023 • edited Loading

github-actions bot commented Sep 21, 2023 • edited Loading

✔️ Linting Passed

ogrisel commented Sep 27, 2023 • edited Loading

ogrisel commented Sep 28, 2023

ogrisel commented Sep 28, 2023

betatim commented Oct 13, 2023

thomasjpfan left a comment

Choose a reason for hiding this comment

ogrisel commented Sep 21, 2023 •

edited

Loading

github-actions bot commented Sep 21, 2023 •

edited

Loading

ogrisel commented Sep 27, 2023 •

edited

Loading