Skip to content

[MRG-1] ENH: Allow float32 to pass through with copy #5932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

jseabold
Copy link
Contributor

This is the easy path to fixing this.

I was a bit surprised to see the float64 default and the copies elsewhere in the sparse functions. Are there rules for consistency as to float32 vs. float64 in the Cython code?

@GaelVaroquaux
Copy link
Member

In general the good looks good, but I would like to know the exact problem that you are trying to solve. Ideally, this problem should be illustrated by a test that fails without your patch and passes with it. The idea being that it will guide later refactoring.

In the long run, we want to tackle the issue of multiple types in Cython using fused types.

@GaelVaroquaux GaelVaroquaux changed the title ENH: Allow float32 to pass through with copy [MRG+1] ENH: Allow float32 to pass through with copy Nov 29, 2015
@jseabold
Copy link
Contributor Author

I found that computing cosine distance fails on 32-bit floating point sparse arrays. Added regression tests.

I added some clarifying documentation on the assign_rows_csr but perhaps this should fail given it's supposed to be no-copy?

Fused types would be nice.

@MechCoder
Copy link
Member

This fix should work, but I'm afraid that this is equivalent to changing the dtype of the sparse matrix prior and passing it to the function as it produces a copy.

It would be great if you could add fused type support so that we can do this without copying ! Just for this function would be a great start.

@springcoil
Copy link

I had a quick look at this today - the code looks fine to me, and the fix works. I would recommend merging it anyway - but fused types needs done too.

np.ndarray[DOUBLE, ndim=1] data = X.data
# might copy
np.ndarray[DOUBLE, ndim=1] data = np.asarray(X.data,
dtype=np.float64)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change to assign_rows_csr is not tested by the new regression test.

@ogrisel
Copy link
Member

ogrisel commented Feb 16, 2016

inplace_csr_row_normalize_l1 and inplace_csr_row_normalize_l2 are supposed to be inplace operations, therefore they cannot by design make a copy of the input data.

The only correct fix for those 2 functions is to use cython fused types for np.float32 and np.float64 (or directly cython.floating that should work for this case AFAIK).

Therefore I am -1 on this part of the PR.

+0 for the change to assign_rows_csr if a test is added, but I think it would be worth using fused types directly for this case as well.

@ogrisel ogrisel changed the title [MRG+1] ENH: Allow float32 to pass through with copy [MRG+1-1] ENH: Allow float32 to pass through with copy Feb 16, 2016
@GaelVaroquaux
Copy link
Member

inplace_csr_row_normalize_l1 and inplace_csr_row_normalize_l2 are
supposed to be inplace operations, therefore they cannot by design make
a copy of the input data.

Therefore I am -1 on this part of the PR.

Agreed with the analysis. Sorry for my previous 👍, it would have been
an error to have "in-place" functions copy data.

The only correct fix for those 2 functions is to use cython fused types for
np.float32 and np.float64 (or directly cython.floating that should work for
this case AFAIK).

That would be good.

@GaelVaroquaux GaelVaroquaux changed the title [MRG+1-1] ENH: Allow float32 to pass through with copy [MRG-1] ENH: Allow float32 to pass through with copy Feb 16, 2016
@jnothman
Copy link
Member

Can we close this as the incorrect patch, while waiting for a fix as part of @yenchenlin1994's GSoC?

@jnothman
Copy link
Member

That's what I'm doing anyway...

@jnothman jnothman closed this Apr 26, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants