-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
MAINT: Translate binsearch.c.src to C++ using templates. #20489
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MAINT: Translate binsearch.c.src to C++ using templates. #20489
Conversation
The Pypy error seems unrelated :-/ |
xref #20491, probably picking up an unstable nightly build |
Could |
@scratchmex that's indeed the goal, see #19713 |
numpy/core/src/npysort/binsearch.cpp
Outdated
const char *sort, char *ret, | ||
npy_intp arr_len, npy_intp key_len, | ||
npy_intp arr_str, npy_intp key_str, | ||
npy_intp sort_str, npy_intp ret_str, | ||
PyArrayObject *NOT_USED) | ||
Cmp cmp) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cmp cmp) |
static pointers require at least optimization level 2 in order to be inlined, it's better to use lambda or just pass meta-type as template arg holds operator ()
contains the scalar operation. for example:
template<typename Tag>
struct cmp_lt {
template <typename T>
bool operator () (T a, T b)
{ return a < b; }
};
template<>
struct cmp_lt<half_tag> {
bool operator () (npy_half a, npy_half b)
{ return HALF_LT(a, b); }
};
// specialize the rest float/cfloat
// ...
to pass it
argbinsearch_<half_tag, cmp_lt>(...);
to initialize it
template<class Tag, template<typename> class Cmp, class T = typename Tag::type>
static int argbinsearch_(...)
{
Cmp<Tag> cmp;
}
edit: fix template arg
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is actually very compiler specific, see https://godbolt.org/z/7TGKvM8o3
I've yet to see a case where -O1 is used in a place where optimization really matters, so I'm not a big fan of this argument. at -O2 all compiler inline the function pointer correctly, and I tend to prefer readability over optimizing for -O1. What are the thoughts of the other reviewers on that point?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can see -O1
being used for debugging. -O3
breaks occsionally, but -O2
usually fixes it. That said, having unstated assumptions floating around could be dangerous. Is it the case that inlining is not required for the code to run correctly? If so, might add a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't change the smeantic of the code, it only changes inlining, i.e. execution speed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is actually very compiler specific, see https://godbolt.org/z/7TGKvM8o3
on GCC -O1
https://godbolt.org/z/4b6f3hxcM isn't going to inline static pointers.
I tend to prefer readability over optimizing
passing a function pointer for internal use that can be handled in compile time is 100% against readability, 100% against the idea behind the existence of C++. The above suggestion is a common pattern we tend to use it since C++ 98 and it can be replaced with lambda.
I can see -O1 being used for debugging
we use -O0
for debugging, distros may build NumPy with -O1
.
It doesn't change the smeantic of the code, it only changes inlining, i.e. execution speed.
giving a chance for the compiler to avoid inlining a function with one line expression that can be executed within a loop is in general a sign of bad design.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought BOOL_LT
was used else where, which is why I reused existing symbol: to prevent duplication. it turns out it's not the case, so I'll move the definition to the tags.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's actually used in heap sort. To avoid duplication, I'll keep the current implementation until it's not needed elsewhere. Small steps!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
adding extern C
to npysort_common.h will fix the duplication between C and C++ symbols without the need of wraps around via static pointers.
eaffe98
to
6bc519c
Compare
Folks, any thoughts on this new version? |
6bc519c
to
d1c9c58
Compare
(rebased on main) |
d1c9c58
to
2db8bfb
Compare
numpy/core/src/npysort/binsearch.cpp
Outdated
template<arg_t arg> | ||
struct binsearch_base; | ||
|
||
template<> | ||
struct binsearch_base<arg> { | ||
using function_type = PyArray_ArgBinSearchFunc*; | ||
struct value_type { | ||
int typenum; | ||
function_type binsearch[NPY_NSEARCHSIDES]; | ||
}; | ||
template<class... Tags> | ||
static constexpr std::array<value_type, sizeof...(Tags)> make_binsearch_map(npy::taglist<Tags...>) { | ||
return std::array<value_type, sizeof...(Tags)>{value_type{Tags::type_value, { (function_type)&argbinsearch<Tags, left>,(function_type)argbinsearch<Tags, right>}}...}; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice !, would you please sort out the code style here?
It looks way better than the previous one, easier to read, no more C macros. Good job!. |
Side effects: - make each type tag aware of how to compare them - add a few __cplusplus guard in numpy header - handle confict between some static and non static symbols with the same name
2db8bfb
to
b358604
Compare
@seiko2plus updated with formatting and usage of wrapper instead of function pointers |
I wonder why coverage thinks some of the code paths are not covered. Is that a false-positive? |
@mattip I think it indicates that binsearch is not tested for all scalar types - the comparison functions are used elsewhere in the C code base so their coverage is 100%, but the C++ version are (currently) only used in binsearch. Two paths here: either we test all dtypes for binsearch, or we wait until I convert other sort algorithms. Your call ;-) |
I think we can wait for the rest of sort. @seiko2plus any reason not to merge? |
Do we want to run clang-format on the files? We have the needed |
For test coverage, there are likely many numpy tests that could be improved by testing over more dtypes. |
@mattip, I have some minor reservations, not sure if it deserves another round of review. In general, the new code looks good to me and we should merge it. |
Thanks @serge-sans-paille |
Side effects: