sparse
support in scikit-learn
#29064
hameerabbasi
started this conversation in
Ideas
Replies: 1 comment
-
I am not sure yet which approach makes more sense:
The latter option means that we can benefit from scikit-learn's custom Cython code to handle sparse inputs, but it also means that scikit-learn will never delegate computation to the pydata/sparse library in cases it performs better than its scipy sparse counter part. I see limited interest in option b. though: it adds yet another dependency to test again on the scikit-learn CI + complexity in updating the docstrings. Instead the users could directly convert pydata/sparse to scipy sparse by themselves prior to calling scikit-learn. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello, scikit-learn maintainers!
We're making efforts to better integrate
sparse
into the PyData ecosystem. For those unaware,sparse
provides N-dimensional sparse arrays and aspires to comply as closely as possible to the Array API.We've already made significant progress in this direction:
sparse
intoscipy.sparse
:pydata/sparse
input for csgraph module scipy/scipy#19796expm
andexpm_multiply
topydata/sparse
input scipy/scipy#20485The next step for us seems to be to target other upstream libraries which would benefit from sparsity, and scikit-learn is at the top of that list. To this end, we'd like to start by adding an upstream CI job for testing
sparse
, and we'll also make efforts on our end to squash any issues (on either side) that may arise, as ecosystem integration is one of our highest priorities.#29031 was a step in that direction, but we understand that a path forward needs to be agreed upon with the scikit-learn maintainers before anything lands.
cc @ogrisel @betatim as contributors leading the Array API work in scikit-learn.
xref #26724 as it will likely become relevant
Beta Was this translation helpful? Give feedback.
All reactions