Skip to content

BLD: Merge the npysort library into multiarray #17167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 7, 2020

Conversation

ewmoore
Copy link
Contributor

@ewmoore ewmoore commented Aug 26, 2020

This will no longer build npysort as library that is linked into
multiarray. This library was not installed and was not linked to
anything else.

This PR was extracted from #16942 as suggested there to make reviewing easier.

@eric-wieser
Copy link
Member

So to be clear, the npysort library was already completely useless?

@ewmoore
Copy link
Contributor Author

ewmoore commented Aug 26, 2020

Well, perhaps unused would be better. It was being built as a library that was then linked into multiarray. This wasn't used anywhere else and wasn't installed for other people to link against. Doing it this way did force the code in npysort to not depend on anything in multiarray.

I can't speak to why it was done this way, but perhaps @charris can.

@eric-wieser
Copy link
Member

I think the goal is similar to the intent behind the npymath library, which thankfully got a quick description at https://numpy.org/doc/stable/reference/c-api/coremath.html.

What's the argument for making this change?

@eric-wieser
Copy link
Member

#89 seems relevant, notably e4d790f which opted to make it not installed. As you remark, @charris looks like the person to decide on this.

@ewmoore
Copy link
Contributor Author

ewmoore commented Aug 26, 2020

In #16942, from where this change was extracted, this is needed so that the dtype transfer mechanism can be used in searchsorted so that full copies don't have to be made. Now, I suppose that if the decision is that searchsorted should be made into a gufunc, then this is not needed, since the optimization I'm trying to make is not possible within the current gufunc framework.

@@ -941,7 +931,7 @@ def generate_umath_c(ext, build_dir):
],
depends=deps + multiarray_deps + umath_deps +
common_deps,
libraries=['npymath', 'npysort'],
libraries=['npymath'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#17127 shows that npymath is also not needed here, and in fact contains symbols that are already in the other object files which leads to build failures with strict linker settings.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know that we can just get rid of npymath, though, it gets installed to numpy/core/lib/ for other projects to link against. I'm not sure how common that is, but at least scipy is using it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't stop npymath from being built, it just stops it being passed to the linker since it does not contain any symbols that are not already available through the object files created from the compiled C files (and in fact contains many identically named functions).

@mattip
Copy link
Member

mattip commented Aug 30, 2020

@charris any thoughts?

@charris
Copy link
Member

charris commented Aug 31, 2020

I don't have a problem with merging this. The original idea was to remove the original sort module that initialized the sort function pointers when it was loaded, so this just completes that removal. The reason it was made a library was the thought of making it available downstream as already mentioned here, but as there have been no requests for that functionality it is probably better to merge it with the _umath_multiarray module build. My recollection of the library option was that code would be copied from the library and built as part of multiarray only if needed, but that probably means the whole of the archived source file comes with it, so not much different than what is here. The functions and prototypes should also be marked NPY_NO_EXPORT so they are not visible.

This will no longer build npysort as library that is linked into
multiarray. This library was not installed and was not linked to
anything else.
@ewmoore
Copy link
Contributor Author

ewmoore commented Sep 3, 2020

Updated to mark things with NPY_NO_EXPORT.

@seberg
Copy link
Member

seberg commented Sep 3, 2020

Looks good to me. Does this need a release note?

@ewmoore
Copy link
Contributor Author

ewmoore commented Sep 3, 2020

I don't think it needs a release note. The npysort library wasn't installed, so this should be a totally invisible change.

@charris charris merged commit dc56a3f into numpy:master Sep 7, 2020
@charris
Copy link
Member

charris commented Sep 7, 2020

Thanks @ewmoore . Hmm, looks like NPY_NO_EXPORT has been renamed NPY_VISIBILITY_HIDDEN, I'll open an issue to do that rename in all the many places it is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants