Skip to content

DOC: Update 1.14 notes #10159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 7, 2017
Merged

DOC: Update 1.14 notes #10159

merged 2 commits into from
Dec 7, 2017

Conversation

charris
Copy link
Member

@charris charris commented Dec 4, 2017

Also forward ports the 1.13.3 release notes and changelog. I suggest viewing the notes as a rendered file as there are a number of rearrangements that clutter up the diff.

@charris charris added this to the 1.14.0 release milestone Dec 4, 2017
@charris charris changed the title DOC: Update 1.14 notes WIP, DOC: Update 1.14 notes Dec 4, 2017
@@ -342,6 +364,23 @@ This new default changes the float output relative to numpy 1.13. The old
behavior can be obtained in 1.13 "legacy" printing mode, see compatibility
notes above.

''hermitiean`` option added to``np.linalg.matrix_rank``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo

computing dtype equality. For example, with the dtypes ::

`x = dtype({'names': ['A', 'B'], 'formats': ['i4', 'f4'], 'offsets': [0, 4]})`
`y = dtype({'names': ['B', 'A'], 'formats': ['f4', 'i4'], 'offsets': [4, 0]})`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backticks no longer needed?

them more clear::

>>> np.polynomial.Polynomial(range(4))
Polynomial([ 0., 1., 2., 3.], domain=[-1, 1], window=[-1, 1])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs updating with the new repr

@jakirkham
Copy link
Contributor

Did PR ( #5580 ) get added to the changelog?

@charris
Copy link
Member Author

charris commented Dec 4, 2017

@jakirkham The changelog is script generated before the release. It will be in the 1.14.0-changelog when it is made.

@jakirkham
Copy link
Contributor

Sure. Didn't know if it needed further mention since it changes behavior.

@charris
Copy link
Member Author

charris commented Dec 4, 2017

@jakirkham It does probably need a release note. If you post an entry here I'll put it in, probably under Compatibility Notes.


* Calling ``np.fromstring`` with the default value of the ``sep`` argument is
deprecated. When that argument is not provided, a broken version of
`np.frombuffer` is used that silently accepts unicode strings and -- after
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

double rather than single backticks for np.frombuffer

@@ -342,6 +364,28 @@ This new default changes the float output relative to numpy 1.13. The old
behavior can be obtained in 1.13 "legacy" printing mode, see compatibility
notes above.

''hermitian`` option added to``np.linalg.matrix_rank``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

incorrect ticks instead of backticks here

Support for PGI flang compiler on Windows
-----------------------------------------
The PGI flang compiler is a Fortran front end for LLVM released by NVIDIA under
the Apache 2 license.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to add the magic incantation here, something like (did not check, just an example):

python setup.py config --compiler=clang --fcompiler=flang install

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or links to another doc that gives this command

@charris
Copy link
Member Author

charris commented Dec 5, 2017

@jakirkham I added an entry for #5580.

@jakirkham
Copy link
Contributor

Thanks @charris.

lines. If there is more than 1 dimension, the array attributes are now
printed in a new "left-justified" printing style.
* ``NaT`` values in datetime arrays are now properly aligned.
* Arrays and scalars of ``np.void`` datatype are now print using hex notation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we remove "are" here? or print -> printed.

@@ -248,7 +282,8 @@ C API changes
---------------------------------------------------------------------

The ``UPDATEIFCOPY`` flag is deprecated, and replaced by a new flag
``WRITEBACKIFCOPY``. Use of the flag requires a call to ``PyArray_ResolveWritebackIfCopy``.
``WRITEBACKIFCOPY``. Use of the flag requires a call to
``PyArray_ResolveWritebackIfCopy``.

Code using ``UPDATEIFCOPY`` must be updated to account for changed semantics, as
follows:. When ndarrays are created with either the deprecated ``UPDATEIFCOPY``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove period after :

@charris
Copy link
Member Author

charris commented Dec 6, 2017

@mattip The PyArray_ResolveWritebackIfCopy function is not documented in c-api.array.rst, could you fix that?

@charris
Copy link
Member Author

charris commented Dec 6, 2017

@mattip Just to be sure, the way I would like things to be for 1.14 (and later) is that nobody will notice the updateifcopy work unless they specifically intend to support PyPy. I don't want downstream projects to need concern themselves otherwise. So for this work I think deprecate is too strong a word. To go along with that, I think the documentation in c-info.how-to-extend.rst should have a separate section on PyPy compatiility, and the other bits should probably be unchanged, as writing to the new interface is not backwards compatible.

@mattip
Copy link
Member

mattip commented Dec 6, 2017

I am working on a documentation pull request.

Currently (on master) calling PyArrayObject constructor functions with NPY_ARRAY_UPDATEIFCOPY will emit a deprecation, advising to use NPY_ARRAY_WRITEBACKIFCOPY + PyArray_ResolveWritebackIfCopy. Is there consensus that this (and the test) should be reverted for the foreseeable future?

@mattip
Copy link
Member

mattip commented Dec 6, 2017

writing to the new interface is not backwards compatible

Using the NPY_WRITEBACKIFCOPY flag, without calling PyArray_ResolveWritebackIfCopy will produce correct results and raise a DeprecationWarning. Using NPY_UPDATEIFCOPY will emit a DeprecationWarning when creating the array but not when deallocating it.

I am not sure I understood what you meant.

@charris
Copy link
Member Author

charris commented Dec 6, 2017

@mattip The basic policy is that we cannot break the C-API, that is, extension modules compiled against earlier versions of Numpy should still work. Now there are a few api functions that are deprecated because of changes to the python api, mostly due to the transition to 64 bits, but those c-api functions are still there after all this time, they just call the newer functions internally. The current changes are a bit iffier, as they are only needed for PyPy compatibility (sorry ;), so I would actually prefer a whole new iterator rather than changing the behavior of the old one. I suspect that there is a lot of code that could be used by both the old and new iterators with small modifications. I am not familiar enough with the current state of things to make specific suggestions, so am just trying so set out the general philosophy. Perhaps this needs some more discussion, or maybe I just need some instruction. @eric-wieser, @ahaldane, @pv, @njsmith Thoughts?

@mattip
Copy link
Member

mattip commented Dec 6, 2017

I would actually prefer a whole new iterator ...

nditer and NpyIter pull request #9998 was not merged, the UPDATEIFCOPY pull request #9639 which deprecates the UPDATEIFCOPY flag in favor of WRITEBACKIFCOPY + PyArray_ResolveWritebackIfCopy was. There is agreement that #9998 is not ready. Is there a place where I jumped the gun with documentation, or the documentation is unclear?

@charris
Copy link
Member Author

charris commented Dec 6, 2017

Is there a place where I jumped the gun with documentation, or the documentation is unclear?

Well, I don't know, that's what I am trying to find out :) I Found the documentation in the release notes somewhat confusing, so what I have now (not committed) is

PyPy compatible alternative to ``UPDATEIFCOPY`` arrays
------------------------------------------------------
``UPDATEIFCOPY`` arrays are contiguous copies of existing arrays, possibly with
different dimensions, whose contents are copied back to the original array when
their refcount goes to zero and they are deallocated. Because PyPy does not use
refcounts, they do not function correctly with PyPy. NumPy is in the process of
eliminating their use internally and two new C-API functions,
``PyArray_SetWritebackIfCopyBase`` and ``PyArray_ResolveWritebackIfCopy``, have
been added together with a complimentary flag, ``WRITEBACKIFCOPY``. If PyPy
compatibility is not a concern, these new functions can be ignored. If you do
wish to pursue PyPy compatibility, more information on these functions may be
found at <missing link>```

@mattip
Copy link
Member

mattip commented Dec 6, 2017

@charris see PR #10166 for PyArray_ResolveWritebackIfCopy. If you reach the conclusion UPDATEIFCOPY should not be deprecated, this and other places should be changed, as well as C code and a deprecation test

@mattip
Copy link
Member

mattip commented Dec 6, 2017

Alternate wording for the release notes, pending a decision on deprecation/non-deprecation:

``UPDATEIFCOPY`` semantics deprecated in favor of ``WRITEBACKIFCOPY``
---------------------------------------------------------------------
``UPDATEIFCOPY`` arrays are contiguous copies of existing arrays, possibly with
different memory layout, whose contents are copied back to the original array when done.
The trigger for the copy was deallocation. To accommodate PyPy's garbage collection and
to properly separate functionality from memory management in  the ``array_dealloc``
function, this flag has been deprecated in favor of a new ``WRITEBACKIFCOPY`` and a
new API function ``PyArray_ResolveWritebackIfCopy`` added. 
``PyArray_SetUpdateIfCopyBase`` has likewise
been deprecated in favor of ``PyArray_SetWritebackIfCopyBase``. If an array is created
with copied data, either through use of flags or ``PyArray_Set...Base`` functions,
``PyArray_ResolveWritebackIfCopy`` should be called to copy the data back. An additional
function ``PyArray_DiscardWritebackIfCopy`` may be used instead to throw the copied data
away without copying it back

@charris
Copy link
Member Author

charris commented Dec 6, 2017

I think the tutorial parts of the alternative belong elsewhere. Release notes are "what" and maybe some "why", but not so much "how" ;)

@mattip
Copy link
Member

mattip commented Dec 6, 2017

``UPDATEIFCOPY`` semantics deprecated in favor of ``WRITEBACKIFCOPY``
---------------------------------------------------------------------
``UPDATEIFCOPY`` arrays are contiguous copies of existing arrays, possibly with
different memory layout, whose contents are copied back to the original array when done.
The trigger for the copy was deallocation. To accommodate PyPy's garbage collection and
to properly separate functionality from memory management in  the ``array_dealloc``
function, this flag has been deprecated in favor of a new ``WRITEBACKIFCOPY`` flag.
Use of the flag requires a call to the new ``PyArray_ResolveWritebackIfCopy`` function.
The new ``PyArray_DiscardWritebackIfCopy`` may be used instead.

``PyArray_SetUpdateIfCopyBase`` has likewise
been deprecated in favor of ``PyArray_SetWritebackIfCopyBase``.

Calls to ``PyArray_XDECREF_ERR`` have been deprecated and
should be replaced by ``PyArray_DiscardWritebackIfCopy`` + ``Py_XDECREF```

``NPY_ARRAY_INOUT_ARRAY`` and
``NPY_ARRAY_INOUT_FARRAY`` flags should be replaced by
``NPY_ARRAY_INOUT_ARRAY2`` and ``NPY_ARRAY_INOUT_FARRAY2``, which behave
similarly but require use of the new ``WRITEBACKIFCOPY`` semantics.

@mattip
Copy link
Member

mattip commented Dec 6, 2017

but I should stop now, documentation is not my forte

That release was not documented in the master branch.

[ci skip]
@charris charris changed the title WIP, DOC: Update 1.14 notes DOC: Update 1.14 notes Dec 7, 2017
@charris charris merged commit 8daad29 into numpy:master Dec 7, 2017
@charris charris deleted the update-1.14-notes branch December 7, 2017 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants