-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
ENH: add support for array-likes in polybase __call__ #18639
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Needs rebase and release note. |
Specifying color in RGB in the guidelines for the new NumPy logo. Fixes #gh-398 numpy/numpy.org#398
Removing the unofficial names of the logo colors.
This reduces the number of cases when floating point precision makes the argument to `log` become `0`
Use log1p(-x) instead of log(1 - x) Seperate legacy version from current closes #17020
Improve the exception when low is 0 in case the single input form was used. closes #14333
I found these convenient to have on hand while reading
When creating the `bad_commands` dictionary it is useless to declare commands that are already set with a default message in the 329th line's for loop
This is a fast argument parser (an original version also supported dictionary unpacking (for args, kwargs call style) which supports the new FASTCALL and VECTORCALL convention of CPython. Fastcall is supported by all Python versions we support. This allows todrastically reduces the overhead of methods when keyword arguments are passed.
Array methods are the easiest target for the new parser. They do not require any larger improvements and most functions in multiarray are wrapped. Similarly the printing functions are not dispatched through `__array_function__` and thus have the most to gain.
While introducing the buffer fixed the in-place problem years ago, running valgrind (and masked arrays) pointed out to me that without the additional `...` NumPy will unpack and repack objects leading to slightly incorrect results. MAINT: Warn about shuffle bug instead of fixing it in old random API
This changes all casting loops to get passed in a new `context` which holds some data such as the descriptors (and methods). For the most part, this is unnecessary information, but it allows us to pass additional information to ufuncs while also aligning ufunc and casting signatures (and definition). To achieve this, we have to fill and store this context everywhere, which leads to some rework. That means that this PR changes some of the chained loops to use fewer specialized loops. There should be no typical speed effect for larger arrays aside possibly a small one for rarer cases such as byteswapped or unaligned arrays (although we currently consider strings unaligned much too often). However, there will be a small increase in overheads, timing it, it appears to be <10% in the most extreme cases. While not neglible there are bigger optimizations possible (even moving the buffer-info data to the array struct was a bigger speedup in most cases!). Below are individual (squashed) commit while working on this, since the change had to be mostly complete to compile/test its fairly long (leaving it just for fun) Individual commit ----------------- Further changes Fixup the main multi-step cast functions Restructure field transfers (partially?) continuation WIP: This is tricky, need to redo some stuff (this is the start of that redo) WIP: more small changes And more work... Clean out/simplify legacy multi-step casts (mainly datetime) Fixup strided one to n field transfer (and decref partially) Fixup n-to-n cast for new methods Fix subarray casts for the new calling convention Finish fixing up the structured transfers and decref start with cleaning up masked casts BUG: Fixup struct access typo Fixup any-to-object Finish all/most of masked casting, this brings me to debugging... Fixup object copy and move functions Code now compiles without errors (of course segfaults immediately :)) Finish fixup of array_assign_* Pass pointer to cast_info_xfree (to be able to NULL func and error check) Also silence a few warnings Bug: Make sure datetime casts have copy/copyswap only path This is necessary to ensure the recursive calls don't recurse forever :). Small fixes, mostly make sure we use PyMem_* for all allocations/frees BUG: Fix memset in multistep cast Seems that code was little enough tested, that most test runs would simply not notice the issue (i.e. requires debug mode usually probably). Now it is used a lot and things came down hard. Bug: Fix stride order typo affecting strided masked indexing BUG: Finish npy-iter copy fixup (apparently it was not...) BUG: Fix scalar assignment mixup Fix bug in field bookkeeping and rename decsrcref to decref Its called "decref" everywhere, the "src" is just confusing especially since now we don't have the `func(dst, src)` calling convention anymore. Final fixes and cleanup of comments e.g. removed some comments alluring to a future that is unlikely to ever come... DOC: Change casting error message to be closer to other messages Fixup rebase: masked transfer function initialize got lost Fixup: remove print added for debugging into recfunctions
These is probably fairly common, so optimize the lookup in this extremely simple case. More optimizations should probably done eventually, e.g. we could add a fast-list for all builtin ones, or have an "intern" mechanism so that casts for any interned dtype will be stored in a list, with the assumption that an interned dtype can e.g. never be deleted and that there should not be too many of them.
This could maybe be done a bit nicer, for now a small (local) helper seemed convenient. This is only needed here.
There were some paths that manipulated the unused data. This was fixed in the big refactor (or earlier), I expect there were some fast paths for equal sized dtypes that kicked in before
As noted by Matti and Ross, these comments were incorrect. Co-authored-by: Matti Picus <matti.picus@gmail.com> Co-authored-by: Ross Barnowski <rossbar@berkeley.edu>
Not the prettiest tests, but should ensure that casts involving byteorder changes within datetime/timedelta and between datetimes or timedeltas and strings are handled correctly.
It seemst he unicode didn't actually help coverage, but here we go.
This is utterly confusing, but adds coverage to another uncovered pass. I honestly do not really understand right now, why "writemasked" and "buffered" combination can bypass the buffering here. I guess there some special logic about that in the `where=` ufunc code that uses the masked versions here?
* Added chain exception in _bits_of func * Added chain exception * Added chain exception in unixccompiler.py * Added chain exception in config.py * Added chain exception in fcompiler __init__.py * Added chain exception in compaq.py * Added chain exception in format.py * Updated raise chain exception * STY: Break long line. Co-authored-by: Charles Harris <charlesr.harris@gmail.com>
Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
* Added chain exception in _pocketfft.py * Added chain exception in format.py * Added chain exception in make_lite.py * Added chain exception in mrecords.py * added from e for exceptions * Minor update for _read_array_header exception msg * Removed \n from excp msg and e from msg format. Co-authored-by: Eric Wieser <wieser.eric@gmail.com> * Update numpy/linalg/lapack_lite/make_lite.py Co-authored-by: Eric Wieser <wieser.eric@gmail.com> Co-authored-by: Eric Wieser <wieser.eric@gmail.com> Co-authored-by: Charles Harris <charlesr.harris@gmail.com>
As per NEP 29, Numpy < 1.15 is dead, so there is no need to mention it in the README.
Add missing type to cdef statement Small spelling fix
* BUG: Fix small valgrind-found issues This should be backportable. There was at least one that I could not reproduce when running the tests again. And the new random-shuffle tests give false-positives (which is just slightly annoying, considering that we are very close to almost only "longdouble" related false-positives) * BUG: Add missing decref in user-dtype fallback paths The missing decref here only leaks references and can never leak actual memory fortunately. * MAINT,TST: Simplify the "refcount logic" in the dispatch tests again Using SETREF can be nice, but was just overcomplicating thing here...
The array creation functions have the most to gain: 1. np.asarray is 4 times faster and commonly used. 2. Other functions are wrapped using __array_function__ in Python making it more difficult This commit (unfortunatly) has to do a few things: * Modify __array_function__ C-side dispatching to accomodate the fastcall argument convention. * Move asarray, etc. to C after removing all "fast paths" from np.array (simplifying the code) * Fixup imports, since asarray was imported directly in a few places * Replace some places where `np.array` was probably used for speed instead of np.asarray or similar. (or by accident in 1 or 2 places)
The argument name for the python side defined function was always `a` and not `object` so fix it and add a test. Also add a test to ensure the error paths are taken for all of the function. The missing NULL returns cannot be tested easily (but thanks to code coverage its nice to look at the uncovered stuff closer...)
I had forgotten that the docs for these functions where updated when rebasing.
On the C-side binops only guarantee that one of the operands is an instance of `PyArrayObject *`, but we do not know which one. Typing the first as `PyArrayObject *` is just misleading for almost no actual payoff
Co-authored-by: Matti Picus <matti.picus@gmail.com>
This can currently only be passed positionally, and this is undocumented.
Including this directory is painful for cross-compiling, see gh-14980 and gh-13280. Removing this directory fixes the following build failure: ``` gcc: numpy/core/src/common/numpyos.c In file included from numpy/core/src/common/numpyos.c:23: /usr/include/xlocale.h:27:16: error: redefinition of ‘struct __locale_struct’ 27 | typedef struct __locale_struct ``` This error also shows up in various other build issues outside of the NumPy issue tracker. Compilers normally always include this path, so this shouldn't break anything. The default include paths for the compiler can be checked, e.g. for gcc with `cpp -v`. That will typically have /usr/include last. In case this breaks something for a nonstandard compiler, that can be worked around via a site.cfg file in the root of the repo (or equivalently, `~/numpy-site.cfg`) containing: ``` [DEFAULT] include_dirs = /usr/include ``` The same principle should apply to `/usr/lib`. I will make that change in a separate commit, because the failure mode for that will be different (and I'm not running into it right now).
- 3D Axes are created via add_subplot(projection='3d') - There is now a `stairs()` function that's specifically designed for showing histogram curves - Labels should be passed as keyword arguments to the plot functions instead of to `legend()`, which reduces the risk of mixing them up. - ensure equal axis scaling in the meshgrid example
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ENH: add suport for array-likes in polybase
__call__
Closes #17949.
Adds an
asarray()
call in the polybase__call__
method so that evaluation works with array-likes too. See #17949 for the initial reproduced code example; after the above change, evaluation works as expected: