-
-
Notifications
You must be signed in to change notification settings - Fork 11.3k
Description
We talked about this a few years ago in #22710, but I think we're at the point where we have broad consensus we should start working on moving long C macros and .c.src
templates to C++ templates.
Reasons to do this:
- C macros are hard to debug and hard to read
- The
.c.src
templates are a thing any new contributor has to learn. They also make it substantially harder to grep the codebase for strings that show up in tracebacks. - C++ templates have much better editor and LSP tooling, with broad support for C++ features in popular tooling that universally has zero support for our custom
.c.src
format.
C++ templates aren't perfect, of course, and we should strive to avoid overusing them in ways that will harm binary size or compile time. That will require review on a case-by-case basis.
Currently we have the following .c.src
templates in the codebase:
numpy/_core/src/_simd/_simd.dispatch.c.src
numpy/_core/src/_simd/_simd_data.inc.src
numpy/_core/src/multiarray/_multiarray_tests.c.src
numpy/_core/src/multiarray/argfunc.dispatch.c.src
numpy/_core/src/multiarray/arraytypes.c.src
numpy/_core/src/multiarray/einsum.c.src
numpy/_core/src/multiarray/einsum_sumprod.c.src
numpy/_core/src/multiarray/lowlevel_strided_loops.c.src
numpy/_core/src/multiarray/nditer_templ.c.src
numpy/_core/src/multiarray/scalartypes.c.src
numpy/_core/src/npymath/ieee754.c.src
numpy/_core/src/npymath/npy_math_complex.c.src
numpy/_core/src/umath/_umath_tests.c.src
numpy/_core/src/umath/funcs.inc.src
numpy/_core/src/umath/loops.c.src
numpy/_core/src/umath/loops_arithm_fp.dispatch.c.src
numpy/_core/src/umath/loops_arithmetic.dispatch.c.src
numpy/_core/src/umath/loops_autovec.dispatch.c.src
numpy/_core/src/umath/loops_comparison.dispatch.c.src
numpy/_core/src/umath/loops_exponent_log.dispatch.c.src
numpy/_core/src/umath/loops_half.dispatch.c.src
numpy/_core/src/umath/loops_minmax.dispatch.c.src
numpy/_core/src/umath/loops_modulo.dispatch.c.src
numpy/_core/src/umath/loops_umath_fp.dispatch.c.src
numpy/_core/src/umath/loops_unary.dispatch.c.src
numpy/_core/src/umath/loops_unary_complex.dispatch.c.src
numpy/_core/src/umath/loops_unary_fp.dispatch.c.src
numpy/_core/src/umath/loops_unary_fp_le.dispatch.c.src
numpy/_core/src/umath/matmul.c.src
numpy/_core/src/umath/scalarmath.c.src
I know that @seiko2plus has a long overdue patch to replace all the SIMD loops with C++ templates, and we should work on getting that in and not try to duplicate.
Separately I think good candidates to look at first include matmul.c.src
, einsum.c.src
, and einsum_sumprod.c.src
.
I wouldn't worry too much about the usage in internal-only tests, since those files don't get touched often and changes there won't help us with code that users actually run.
See Ralf's comment below for npymath, it should be left alone.
I think scalarmath.c.src
, scalartypes.c.src
, nditer_templ.c.src
, and arraytypes.c.src
might be trickier since that would require dealing with a lot of internal headers that never anticipated being used inside C++ code.