Skip to content

BUG: Clear error before and after PyErr_WarnEx for 3118 buffers #9348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

BUG: Clear error before and after PyErr_WarnEx for 3118 buffers #9348

wants to merge 2 commits into from

Conversation

tynn
Copy link
Contributor

@tynn tynn commented Jul 2, 2017

Fixes #6741

The PyErr_WarnEx seems to be the reason of the PyObject_Call: Assertion failure.

Here I assumed that PyErr_WarnEx is primarily used to print the warning message and suppress all errors. Thus PyErr_Clear should be safe to be used to clear the error which is turned into the warning and clear all errors possibly set by the warning.

From PyErr_WarnEx

They normally print a warning message to sys.stderr; however, it is also possible that the user has specified that warnings are to be turned into errors, and in that case they will raise an exception. It is also possible that the functions raise an exception because of a problem with the warning machinery.

The tests relate to #6214

@njsmith
Copy link
Member

njsmith commented Jul 2, 2017

From a quick glance at the patch, the use of PyErr_Clear here seems like it must be wrong. If something is setting an error then we almost always should exit and propagate the exception, not ignore the error and continue. And in the rare cases where PyErr_Clearis the right thing to use, it should be called immediately after the potentially-failing operation, with a comment explaining why we want to ignore errors there.

@tynn
Copy link
Contributor Author

tynn commented Jul 2, 2017

The usage of PyErr_WarnEx seems wrong here. Also _array_from_buffer_3118 seems to not follow some Python conventions. It returns -1, but doesn't set an error. With the usage of this function, errors are ignored anyway. And I'm not sure if the warnings should ever make the function fail.

The issue itself arises after _descriptor_from_pep3118_format fails and sets an error which was never handled. Would you accept it if PyErr_Clear is called just before PyBytes_FromFormat, or would you prefer rewriting this part completely?

@seberg
Copy link
Member

seberg commented Jul 2, 2017

Why does it give a warning and not just error out at all? Not following python conventions is maybe not nice, but not so bad for an internal function, but I think the problem here may be that the function should be able to signal three and not two possible results:

  1. Everything is good
  2. This is not a buffer at all (some error may have happened, but we should have cleared it)
  3. It seems to be an invalid buffer

And it seems for 3. it actually sets a warning in a pretty nasty way, since that warning can also easily be an exception. The second clear seemes to be more likely correct to do something of the logic if PyErr_Occurred(): goto fail to be honest. I am not sure the warnings should be warnings at all and not errors though. Can it happen that we want to warn, then continue and create an object array or so?! We may need a deprecation to turn it to an unconditional error?

EDIT: Or maybe split it into two functions, one that can't error and decides whether to try or not, the other that can error and the error will always be shown (possibly after deprecation).

if (!PyBytes_Check(op) && !PyUnicode_Check(op)) {
if (_array_from_buffer_3118(op, (PyObject **)out_arr) == 0) {
if (writeable &&
PyArray_FailUnlessWriteable(*out_arr, "PEP 3118 buffer") < 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look safe if out_arr == NULL...

@eric-wieser
Copy link
Member

Looks like _descriptor_from_pep3118_format needs a fair bit of work here too...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants