Skip to content

Navigation Menu

Explore
By company size
By use case
By industry
View all solutions
Topics
- AI
- DevOps
- Security
- Software Development
- View all
Explore
- GitHub Sponsors
  Fund open source developers
- The ReadME Project
  GitHub community articles
Repositories
- Enterprise platform
  AI-powered developer platform
Available add-ons
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Sign up

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

numpy / numpy Public

Notifications You must be signed in to change notification settings
Fork 10.8k
Star 29.4k

Code
Issues 2.1k
Pull requests 231
Actions
Projects 3
Wiki
Security
Insights

Additional navigation options

Code
Issues
Pull requests
Actions
Projects
Wiki
Security
Insights

Missingdata - add NA mask, multitude of other improvements#141

Closed

mwiebe wants to merge 167 commits intonumpy:masternumpy/numpy:masterfrom

mwiebe:missingdatamwiebe/numpy:missingdataCopy head branch name to clipboard

Conversation Commits167 (167)Checks Files changed

Commits

Commits on Aug 26, 2011

ENH: missingdata: Add the NA mask members to PyArrayObject

Mark Wiebe
authored and
mwiebe
committed
ENH: Create NA singleton file

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Add boilerplate for NA singleton type

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Write NA repr function

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Make comparisons with NA return NA, raise on 'if np.NA: ...'

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Define the standard Python arithmetic operations for NA

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Use NPY_NA_NOPAYLOAD instead of constant 255 everywhere

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Add (untested) functions for creating the NA mask

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Have some basic assignment and indexing with NA working

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Really simple printing with NA works in some cases now too

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Clean up object dtype detection to work with NAs properly

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Get printing of NAs to work a little bit better

Mark Wiebe
authored and
mwiebe
committed
BLD: missingdata: Signature of arraydescr_short_construction_repr changed

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Part way through supporting the NPY_ITER_USE_MASKNA flag

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: More progress towards NPY_ITER_USE_MASKNA flag support

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: In progress exposing USE_MASKNA to Python numpy.nditer

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Finish the initial implementation of numpy.isna

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: In progress making slicing work with NA masks

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Get the NA mask working with slice indexing

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Rewrote boolean indexing to support NA masks

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Implemented boolean assignment, working with NA masks

Mark Wiebe
authored and
mwiebe
committed
BUG: ma: Fix a bug in numpy.ma hardmasks, exposed by the boolean indexing rewrite

Mark Wiebe
authored and
mwiebe
committed
TST: core: Regression test of boolean indexing was invalid, fixed it

Mark Wiebe
authored and
mwiebe
committed
ENH: core: Revert optimistic use of PyNumber_Index for indexing

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Write NA mask support into PyArray_CopyInto

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Fill in view function, add many tests for NA view functions

Mark Wiebe
authored and
mwiebe
committed
BUG: missingdata: Add deallocation of the NA mask

Mark Wiebe
authored and
mwiebe
committed
TST: missingdata: Cover some more unmasking assignment cases

Mark Wiebe
authored and
mwiebe
committed
TST: missingdata: Move maskna array tests to their own file

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Fill in buffered NAMASK nditer, add maskna= to zeros, empty, and empty_like

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Some tests for MASKNA iteration in Python

Mark Wiebe
authored and
mwiebe
committed
TST: missingdata: Add test for buffered MASKNA iteration

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Make the nditer USE_MASKNA mode work with buffering

Mark Wiebe
authored and
mwiebe
committed
ENH: nditer: Allow a virtual mask of ones for non-masked USE_MASKNA operands
mwiebe
committed
ENH: missingdata: Got masked element-wise ufuncs working in preliminary fashion
mwiebe
committed
ENH: missingdata: Progress towards supporting ufunc reduce with NA masks
mwiebe
committed
ENH: missingdata: Improve some raw iteration methods
mwiebe
committed
ENH: missingdata: Write function for reducing the NA mask array

Mark Wiebe
authored and
mwiebe
committed
ENH: ufunc: Split Reduce and Accumulate apart so that adding NA support is easier

Mark Wiebe
authored and
mwiebe
committed
API: nditer: Rename NpyIter_GetMaskNAIndices to NpyIter_GetMaskNAIndexArray

Mark Wiebe
authored and
mwiebe
committed
ENH: ufunc: Rewrite PyUFunc_Reduce to be more general and easier to adapt to NA masks

Mark Wiebe
authored and
mwiebe
committed
TST: ufunc: Tweak ma test to have a good output parameter, disable crashing NA test temporarily

Mark Wiebe
authored and
mwiebe
committed
ENH: umath: Make sum, prod, any, or functions use the .reduce method directly

Mark Wiebe
authored and
mwiebe
committed
ENH: ufunc: Add tests for zero-sized array reductions

Mark Wiebe
authored and
mwiebe
committed
DOC: ufunc: Document the 'axis=' improvments to reduction functions

Mark Wiebe
authored and
mwiebe
committed
ENH: Work in progress on arr.reshape, other misc changes

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Fix up more NA mask indexing, make things work in 2D
mwiebe
committed
ENH: missingdata: Change boolean indexing to broadcast to the left manually

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Change things to help scipy pass its tests

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Fix the remaining scipy errors

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Fix an iterator MASKNA bug, fill in more missing stuff

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Get ufunc reduce skipna=False working better

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Some tests/fixes for reduce method with skipna=True

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Make reduction with skipna=True work better in some cases

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Fix construction of maskna array with all NAs

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Make the ndarray.take function work with NA masks

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Try to get basic NA printing to be ok

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Another NA array formatting tweak

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Add skipna parameters to sum, prod, etc

Mark Wiebe
authored and
mwiebe
committed
BUG: missingdata: Was adding MASKNA even when not desired currently

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Make NA ufuncs work with output parameters properly

Mark Wiebe
authored and
mwiebe
committed
BUG: missingdata: Negative strides bug in USE_MASKNA nditer mode

Mark Wiebe
authored and
mwiebe
committed
ENH: missingdata: Adding NA support to various methods
mwiebe
committed
WIP: fixing reduce NA bug
mwiebe
committed
ENH: missingdata: Writing some low level general array assignment routines
mwiebe
committed
ENH: missingdata: Implement wheremask scalar assignment
mwiebe
committed
ENH: missingata: Move the alignment check out of the assignment functions
mwiebe
committed
ENH: missingdata: Use a static buffer in scalar assignment for small dtypes
mwiebe
committed
ENH: missingdata: Implement wheremasked scalar assignment with overwritena=True
mwiebe
committed
ENH: missingdata: Finish implementation of scalar to array assignment
mwiebe
committed
ENH: missingdata: Fix remaining issues in scalar -> array assignment function
mwiebe
committed
ENH: missingdata: Change FillWithZero/One to AssignZero/One, add parameters
mwiebe
committed
ENH: core: Rewrite PyArray_FillWithScalar to use array_assign_scalar
mwiebe
committed
ENH: core: Make 1-byte low level loops use memset
mwiebe
committed
ENH: missingdata: Implement routine for array to array assignment
mwiebe
committed
ENH: core: Make the array assignment routine handle overlapping arrays
mwiebe
committed
ENH: core: Make PyArray_CopyInto and PyArray_MoveInto be calls to array_assign_array
mwiebe
committed
ENH: core: Add tests for copyto function with new array_assign_array features
mwiebe
committed
ENH: umath: Fix reduce with NAs for ufuncs that have no unit
mwiebe
committed
BUG: missingdata: Fleshing things out, tracking down a memory corruption
mwiebe
committed
DOC: missingdata: Add some NA mask info to the documentation
mwiebe
committed
ENH: missingdata: Make arr.item() and arr.itemset() work with NA masks
mwiebe
committed
ENH: missingdata: Move the new MultiIndex Get/Set Item functions into item_selection.c
mwiebe
committed
BUG: missingdata: Fix src_itemsize in USE_MASKNA copy to buffer
mwiebe
committed
ENH: ufunc: Move the default SAME_KIND casting rule out of the ufunc code
mwiebe
committed
ENH: missingdata: Add NA support to np.diagonal, change np.diagonal to always return a view
mwiebe
committed
ENH: missingdata: Rewrite PyArray_Concatenate to work with NA masks
mwiebe
committed
BUG: missingdata: The ndmin parameter to np.array wasn't respecting NA masks
mwiebe
committed
BUG: missingdata: Fix mask usage in PyArray_TakeFrom, add tests for it
mwiebe
committed
ENH: missingdata: Add nastr= parameter to np.set_printoptions()
mwiebe
committed
ENH: missingdata: trying some more functions to see how they treat NAs
mwiebe
committed
ENH: missingdata: Change default to create NA-mask when NAs are in lists
mwiebe
committed
ENH: missingdata: Move some of the refactored reduction code into the API
mwiebe
committed
ENH: missingdata: Towards making count_nonzero a full-featured reduction operation
mwiebe
committed
ENH: missingdata: Finish count_nonzero as a full-fledged reduction operation
mwiebe
committed
ENH: missingdata: Move the Reduce boilerplate into a function PyArray_ReduceWrapper
mwiebe
committed
ENH: missingdata: Create count_reduce_items function
mwiebe
committed
ENH: missingdata: Support 'skipna=' parameter in np.mean
mwiebe
committed
ENH: missingdata: Implement skipna= support for np.std and np.var
mwiebe
committed
ENH: missingdata: Implement tests for np.std, add skipna= and keepdims= parameters to more functions
mwiebe
committed
BUG: nditer: NA masks in arrays with leading 1 dimensions had an issue
mwiebe
committed
TST: missingdata: Finish up NA mask tests for np.std and np.var
mwiebe
committed
BUG: Some bugs in squeeze and concatenate found by testing SciPy
mwiebe
committed
DOC: Small tweak to release notes
mwiebe
committed
BUG: nditer: The nditer was reusing the reduce loop inappropriately (#1938)
mwiebe
committed
BLD: Failure in single file build mode because of a static function in two separate files
mwiebe
committed
TST: datetime: Change pytz warning message about skipping pytz tests
mwiebe
committed
BLD: missingdata: Fixes for Python 3
mwiebe
committed
DOC: Tweak to the release notes
mwiebe
committed
BLD: keep VC compiler happy by moving the threading macros after variable definition

87
authored and
mwiebe
committed
BLD: keep VC happy by moving inlined variable definitions to top

87
authored and
mwiebe
committed
BUG: fix crash in test_datetime_as_string

87
authored and
mwiebe
committed
BUG: ufunc: Fix bug in multi-dimensional reduction without a unit
mwiebe
committed
DOC: nditer: Improve the comment doc about the new NpyIter_IsFirstVisit function
mwiebe
committed
NEP: missingdata: Some fixes and updates to the NEP
mwiebe
committed
BUG: ufunc: Missed a small update for the unitless reduction case
mwiebe
committed
STY: core: Move some misc converters into conversion_utils.c to clean things up
mwiebe
committed
ENH: missingdata: Add the maskna= parameter to np.ones and np.ones_like
mwiebe
committed
ENH: missingdata: Add maskna= parameter to np.linspace and np.logspace
mwiebe
committed
DOC: nditer: Document NpyIter_IsFirstVisit function
mwiebe
committed
ENH: missingdata: Change NA repr to use strings like the dtype repr does
mwiebe
committed
BUG: repr: Make NA line up in the float array repr like inf and nan
mwiebe
committed
ENH: missingdata: Improve error message when assigning NA to non-NA-masked arrays
mwiebe
committed
BUG: missingdata: np.isna function wasn't accepting object arrays
mwiebe
committed
BUG: missingdata: Fix long double printing of NAs
mwiebe
committed
ENH: ufunc: Separate type resolution from loop selection
mwiebe
committed
ENH: umath: Add checking for reorderable ufuncs, add PyArray_ReduceWrapper to API
mwiebe
committed
ENH: umath: Switch PyUFunc_Reduce to call PyArray_ReduceWrapper to simplify code
mwiebe
committed
ENH: ufunc: Add a mask dtype parameter to the masked ufunc loop selector
mwiebe
committed
DOC: Add info to the release notes about the full boolean indexing change
mwiebe
committed
ENH: missingdata: Add maskna= parameter to np.copy and ndarray.copy
mwiebe
committed
ENH: missingdata: Make ndarray.view validate the maskna= parameter better
mwiebe
committed
BUG: dtype: Mitigate crash bug for some kinds of dtype inputs
mwiebe
committed
ENH: ufunc: Remove CreateReduceResult and InitializeReduceResult from the API
mwiebe
committed
ENH: missingdata: Add wheremask to PyArray_ContainsNA
mwiebe
committed
ENH: missingdata: Move ReduceMaskNAArray out of the public API
mwiebe
committed
ENH: missingdata: Future-proof AssignNA and AssignMaskNA for later multi-NA support
mwiebe
committed
ENH: missingdata: Add support for identity-less skipna reductions with ndim > 1
mwiebe
committed
ENH: missingdata: Add skipna=, keepdims= parameters to methods
mwiebe
committed
ENH: core: Add static caching of the callables for C to core._method calls
mwiebe
committed
BUG: umath: Make the ufunc follow the actual priority for __r<op>__
mwiebe
committed
ENH: missingdata: Add maskna= flag to np.eye constructor
mwiebe
committed
ENH: missingdata: Add maskna= flag to np.identity constructor
mwiebe
committed
ENH: missingdata: Add maskna= and ownmaskna= parameters to np.asarray and friends
mwiebe
committed
BUG: missingdata: Add support for NA masks to the order='K' case of np.ravel
mwiebe
committed
ENH: missingdata: Rename na_singleton.[ch] to na_object.[ch]
mwiebe
committed
ENH: missingdata: Expose Npy_NA as a global singleton, same as Py_None
mwiebe
committed
ENH: missingdata: Finish adding C-API access to the NpyNA object
mwiebe
committed
DOC: missingdata: Documenting C API for NA-masked arrays
mwiebe
committed
DOC: missingdata: Add example of a C-API function supporting NA masks
mwiebe
committed
DOC: missingdata: Some tweaks to the NA mask documentation
mwiebe
committed
ENH: core: Rename PyArrayObject_fieldaccess to PyArrayObject_fields
mwiebe
committed
DOC: missingdata: Add introductory documentation for NA-masked arrays
mwiebe
committed
DOC: missingdata: Also show what assigning a non-NA value does in each case
mwiebe
committed
ENH: missingdata: Make numpy.all follow the NA || True == True rule
mwiebe
committed
ENH: missingdata: Make numpy.all follow the NA && False == False rule
mwiebe
committed
TST: missingdata: Write some tests for the np.any and np.all NA behavior
mwiebe
committed
STY: Remove trailing whitespace
mwiebe
committed
TST: dtype: Adjust void dtype test to pass without raising a zero-size exception
mwiebe
committed
DOC: Mention the update to np.all and np.any in the release notes
mwiebe
committed
BLD: core: onefile build fix and Python3 compatibility change
mwiebe
committed
ENH: missingdata: Make comparisons with NA return NA(dtype='bool')
mwiebe
committed
ENH: nditer: Change the Python nditer exposure to automatically add NPY_ITER_USE_MASKNA
mwiebe
committed
DOC: missingdata: Updates based on pull request feedback
mwiebe
committed
DOC: missingdata: Updates from pull request feedback
mwiebe
committed
DOC: missingdata: Add a mention of the design NEP, and masks vs bitpatterns
mwiebe
committed
ENH: missingdata: Make PyArray_Converter and PyArray_OutputConverter safer for legacy code
mwiebe
committed

Footer

© 2025 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Docs
Contact

You can’t perform that action at this time.