Enable config setting sparse_interface to control sparray and spmatrix creation
#31177
Conversation
|
Another note: We can work around it for now with e.g. The recent features for sparse by version are:
This info might help us decide when to support which versions. I think the construction functions are all that is currently needed. If/when we start using indexing code for both sparse and dense, we will likely want 1.15. If/when we want nD sparse we will need v1.16, and broadcasting binary operations in v1.17. But for now, 1.8 leaves out only construction functions from current code. |
e2b7d8b to
251e2e2
Compare
|
It sounds like the community has decided to convert to SciPy sparray using the config parameter. This PR is a start toward that. I've removed the "Draft status" from the PR. I am ready to implement this approach in other parts of the library. I have two questions:
|
| if _as_sparse(X_csr) is X_csr: | ||
| assert X_transform is X_csr | ||
| else: | ||
| assert X_transform is not X_csr |
There was a problem hiding this comment.
Although the object identity is not the same, is the underlying data/indices/indexes pointing to the same data? If so, we can check the data?
There was a problem hiding this comment.
Yes -- the underlying data is the same. Good idea. I've added a check for indptr.
sklearn/utils/_sparse.py
Outdated
| from .._config import get_config | ||
|
|
||
|
|
||
| def _as_sparse(X_sparse): |
There was a problem hiding this comment.
Do you think it'll be simpler to have _as_sparse be _select_interface_if_sparse?
There was a problem hiding this comment.
I guess the difference between functions is whether we know already that the input is sparse. We usually know whether it is sparse. But there are some cases where the input could be dense or sparse.
Looking at this again now, the cost of a simpler one-function approach is small. We always check issparse so we could forego the exception and just pass-through anything that is not sparse.
Do you have a suggestion for the name of the single function? _as_sparse suggests it converts everything to sparse. But _select_interface_if_sparse is a long name. :) Maybe _align_api_if_sparse? Or _align_sparse_api?
There was a problem hiding this comment.
I'm okay with _align_api_if_sparse.
86b2de7 to
61d6dc0
Compare
…nd 3 example modules
5e3d3cd to
511de76
Compare
|
I've updated the name and example code -- and I added a commit that implements (with a These changes are largely orthogonal to the return sparse interface issue we've been focusing on here. But it is a needed step that would be easier to get feedback on here before this gets any bigger. If you'd prefer this in a different PR let me know. And if it'd be good to put further changes in a separate PR let me know. The further changes are mostly switching Do my choices for function names that bridge the old versions of SciPy look ok? |
This PR sets up a config parameter
sparse_interfaceto indicate "sparray" or "spmatrix" outputs, as suggested in #26418.The first commit sets everything up and implements the system for a few modules. Please take a look and provide feedback for whether this is the way to proceed. The next commit(s) will implement these same style of changes throughout the library. If you would prefer they be in separate PRs let me know. (I'll keep Draft status until there is feedback and the full library is convered.)
More specifically, this PR does the following:
sparse_interfaceto the config parameters. (I think this name is better than sparse_format because "format" means csr/coo/lil, etc in the sparse world.) The values it can hold are "sparray" or "spmatrix". Update config tests accordingly.utils._sparse.pywith (private) helper functions. The difference is how much checking is done. Tests added too._as_sparse(x_sparse), raises unless sparse input. converts to interface chosen by config._select_interface_if_sparse(x), allows dense input with no action, sparse input uses_as_sparse._convert_from_spmatrix_to_sparray(x)and_convert_from_sparray_to_spmatrix(x)sklearn/feature_selection/text.pyand adapts tests.sklearn/linear_model/_coordinate_descent.pyno tests change needed.sklearn/manifold/_locally_linear.pyno tests change needed.@thomasjpfan can you see if this does what you had in mind? I tried to pick modules that cover returning sparse, setting estimators to hold sparse, and transforming to sparse, so you can see how this would work.
Let me know if you think
_as_sparseshould be a public function, and if my approach aligns with how you want it. The next steps for this PR are to repeat these type of changes throughout the library.