TST: Suppressed warnings #7099

seberg · 2016-01-23T02:28:49Z

OK, this is a start, a context manager to suppress warnings a little better (nothing is prefect, though maybe I am missing something). And trying to remove all those "ignore" stuff everywhere mostly.

A few quite general warnings are still suppressed globally (where possible, sometimes needs local duplication), that may be fine. The switch to only show warnings in release mode is missing, and since some things never showed warnings for me, I sometimes removed it completely. This is not beta or rc, but I think you can get the idea.

matthew-brett · 2016-01-23T03:07:51Z

numpy/testing/utils.py

+    of setting a warning to "ignore", it is possible to add a suppression.
+    This has the advantage of ensuring a clean warning registry.
+
+    Suppressions are inserted either immediatly, or upon entering the


typo 'immediatly'

charris · 2016-01-23T03:22:28Z

The switch to only show warnings in release mode is missing

What do you mean? Currently it depends on the version string.

seberg · 2016-01-23T03:24:18Z

Hmmm, python 3.4 and later actually should invalidate the warning registry it appears, so the dance could be unnecessary there.... Not sure what that means.

@charris I mean in this version I broke it currently.

seberg · 2016-01-23T12:31:49Z

Ok, this is getting somewhere. The biggest issue for me is that I bet scipy will explode in dev mode at least. I also added stacklevel to most warnings (that I found quickly with a grep), no tests for that sorry, but that would seem insane :).

There are no tests for the new warning context manager yet, but I will probably add them later. Comments?!

seberg · 2016-01-23T12:32:38Z

The solution is currently to keep some rather broad global suppressions, but I think that is somewhat fine, since those could be tried to be narrowed down later.

seberg · 2016-01-23T12:50:06Z

Hmmm, anyone has an idea why the benchmarks are not successful? Is it an asv problem?!

seberg · 2016-01-23T14:48:04Z

OK, I guess adding stacklevel is nonsense. Undid it. I avoided the ASV problem now.

seberg · 2016-01-23T15:01:58Z

Hmmm, I readded the stacklevel stuff. I guess we say it is right, I am not quite sure, also it does change things slightly, since the warnings.filterwarnings(module=...) will now be the caller instead of us.

seberg · 2016-01-23T15:39:32Z

@charris since AppVeyor is so slow, if you would go to teams there and add "GitHub teams", might that give us the option to abort test runs manually?

charris · 2016-01-23T15:59:58Z

I can't get anywhere on AppVeyor, their interface is crap. I'll probably need to delete all the cookies and fool around to figure out how to do anything, but IIRC, you need to have an AppVeyor account and then I can add you to the team on AppVeyor. AppVeyor really needs someone who can do interfaces.

seberg · 2016-01-23T16:41:08Z

OK, this is actually in a state where it can be reviewed now.

rgommers · 2016-01-23T20:36:55Z

numpy/testing/utils.py

+
+    Note that all of this is only necessary for Python versions before
+    python (about) 3.4. After that "ingnore" can be used safely within
+    the catch_warnings context.


Maybe I missed some issue where this was discussed in detail, but I don't really get the point of why this is really needed. Apparently there's a bug in Python 2.7 in the warnings module, which we cannot work around and therefore we need to replace catch_warnings with a new context manager?

This is all about the __warningregistry__ and avoiding getting stuff into it, which catch_warnings does not really help with at all. On newer pythons, this is solved (the registry is invalidated automatically).

seberg · 2016-01-24T10:52:38Z

OK, I have thought about it some more. The new context manager might be mostly not needed. There are three reasons why it is nice (I have to check whether I actually use the second one):

Printing the warning is actually possible. Without this context manager, we have to filter out all warnings on the outer level in release mode. So it is either hide everything or raise as an error.
It is possible to record more convenient:

with suppressed_warnings(record=True) as sup:
    sup.append(FutureWarning, "I can ignore this")
    warnings.filterwarnings("always", category=DeprecationWarning,
                                          message="This is interesting!")

    command_that_gives_both_warnings()
    assert_(len(sup.log) == 1)

If the futurewarning would not occur every time, counting things would get really annoying. I admit there may be only a single occurance of this (plus the deprecation tests could be made nicer with this, there I needed this kind of behaviour to make the test specific to a single deprecation).

If you use it as the inner context manager, the outer suppressions are still active (without recording only), so applying more global suppressions can be far more convenient, and that might actually be used a few times (it is hard to tell without trying what happens when you remove the global ones).

In summery, I still don't care too much about this context manager, but would like to get somewhere with this, because this is just needed to test warnings reliably.
My take is this, if we say we are content with either raising warnings or hiding them and you are unhappy about this rather complicated context manager, then I can get rid of it again and work around it where it is annoying. There is also this clear_and_catch_warnings thing that can do the second thing as well with a little more precision needed.

EDIT: added third reason ;)

seberg · 2016-08-31T21:40:43Z

I think I got the last comments. Had some doc fixups and added a note to the release note. Can squash a little when the time comes.

charris · 2016-09-01T22:34:05Z

I can put this in as is. If you can easily squash the reversions and some of the fixups, that would be fine also, but don't put too much effort into it.

Also modify the corresponding test to suppress the non Deprecation warnings created to test specificity.

This means that warnings of different origin then the one tested for behave normally. If the normal behaviour is to igonre them this might decrease specificity in rare cases. In most cases the specificity will be slightly higher.

Printing of datetime arrays used to cause warning due to comparison warnings in NaT. This is circumvented by using views to integer values. Part of this should be simplified when the deprecation is over. Also fixes a bug with non-native byteorder.

Comment mentions a speedup, but it seems unsure why it should be there. Instead use an error state in divide_by_count. Some extra complex warnings had to be ignored (but those seemed correct)

In some places, just remove aparently unnecessary filters. After this, all cases of ignore filters should be removed from the tests, making testing (even multiple runs) normally fully predictable.

Making the outer context manager a suppress warnings gives good control to print warnings only once in release mode and suppress some specific warnings which cannot be easily avoided otherwise.

Also silences a spurious warning during tests (the multiplication could give a warning).

These are warnings, which when raised as an error for one reason or another are already silenced.

The PIPE in the tests caused a ResourceWarning during testing in python 3.

they used to be called `..._w_...` and `..._wo_...`.

…nfuncs

seberg · 2016-09-02T08:12:16Z

OK, got rid of the two REVs (and put that doc thing to the part where its implemented). Should be good.

charris · 2016-09-02T14:12:55Z

In it goes, one more step on the way to 1.12. Thanks Sebastian.

seberg · 2016-09-02T14:59:10Z

Thanks for the persistent reviewing :).

seberg · 2016-09-02T15:58:56Z

Ahhh, just rebased my oindex branch on this, and I get a nice list of all the tests that need modification to be future proof with it :).

matthew-brett reviewed Jan 23, 2016
View reviewed changes

seberg force-pushed the suppressed_warnings branch 3 times, most recently from 43892e5 to 4320148 Compare January 23, 2016 11:47

seberg force-pushed the suppressed_warnings branch 5 times, most recently from 7d26647 to 2fa8be6 Compare January 23, 2016 14:46

seberg force-pushed the suppressed_warnings branch from c321359 to 6cb04ac Compare January 23, 2016 15:12

seberg force-pushed the suppressed_warnings branch 2 times, most recently from f7d2d9b to d839868 Compare January 23, 2016 17:36

charris added 05 - Testing component: numpy.testing labels Jan 23, 2016

rgommers reviewed Jan 23, 2016
View reviewed changes

seberg force-pushed the suppressed_warnings branch from a3ee53d to d20e2dd Compare January 26, 2016 22:43

seberg force-pushed the suppressed_warnings branch from 44731c3 to af23313 Compare August 31, 2016 21:39

seberg force-pushed the suppressed_warnings branch from af23313 to c116a18 Compare September 1, 2016 08:02

seberg added 13 commits September 2, 2016 10:10

MAINT: Simplify deprecation test decorator

29a45ef

Also modify the corresponding test to suppress the non Deprecation warnings created to test specificity.

ENH: Make warning testing context managers more specific

968507b

This means that warnings of different origin then the one tested for behave normally. If the normal behaviour is to igonre them this might decrease specificity in rare cases. In most cases the specificity will be slightly higher.

BUG: Suppress common NaT warnings

86b0a5e

Printing of datetime arrays used to cause warning due to comparison warnings in NaT. This is circumvented by using views to integer values. Part of this should be simplified when the deprecation is over. Also fixes a bug with non-native byteorder.

MAINT: Add missing teardown in deprecation test case

78d7cc4

ENH: Remove warning ignoring from nanfuncs

c1ddf84

Comment mentions a speedup, but it seems unsure why it should be there. Instead use an error state in divide_by_count. Some extra complex warnings had to be ignored (but those seemed correct)

TST: Use new warnings context manager in all tests

308161c

In some places, just remove aparently unnecessary filters. After this, all cases of ignore filters should be removed from the tests, making testing (even multiple runs) normally fully predictable.

ENH: Use new context manager for testing

9bf7d14

Making the outer context manager a suppress warnings gives good control to print warnings only once in release mode and suppress some specific warnings which cannot be easily avoided otherwise.

ENH: Masked array simplify copying back of unchanged values

f078cb4

Also silences a spurious warning during tests (the multiplication could give a warning).

TST: Silence warnings otherwise shown in release mode

2e86117

These are warnings, which when raised as an error for one reason or another are already silenced.

MAINT: Close some filehandlers and PIPEs after being done

b831444

The PIPE in the tests caused a ResourceWarning during testing in python 3.

MAINT: Spell out with and without in ma test functions

61694be

they used to be called `..._w_...` and `..._wo_...`.

TST: Replace catch_warnings when recording is not enforced in test_na…

20ea3a2

…nfuncs

TST: Use ComplexWarning suppression only where needed

514d136

seberg force-pushed the suppressed_warnings branch from c116a18 to 514d136 Compare September 2, 2016 08:11

charris merged commit 9164f23 into numpy:master Sep 2, 2016

seberg deleted the suppressed_warnings branch September 2, 2016 14:59

charris changed the title ~~Suppressed warnings~~ TST: Suppressed warnings Nov 16, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TST: Suppressed warnings #7099

TST: Suppressed warnings #7099

seberg commented Jan 23, 2016

matthew-brett Jan 23, 2016

charris commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

charris commented Jan 23, 2016

seberg commented Jan 23, 2016

rgommers Jan 23, 2016

seberg Jan 23, 2016

seberg commented Jan 24, 2016

seberg commented Aug 31, 2016

charris commented Sep 1, 2016

seberg commented Sep 2, 2016

charris commented Sep 2, 2016

seberg commented Sep 2, 2016

seberg commented Sep 2, 2016

TST: Suppressed warnings #7099

TST: Suppressed warnings #7099

Conversation

seberg commented Jan 23, 2016

matthew-brett Jan 23, 2016

Choose a reason for hiding this comment

charris commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

seberg commented Jan 23, 2016

charris commented Jan 23, 2016

seberg commented Jan 23, 2016

rgommers Jan 23, 2016

Choose a reason for hiding this comment

seberg Jan 23, 2016

Choose a reason for hiding this comment

seberg commented Jan 24, 2016

seberg commented Aug 31, 2016

charris commented Sep 1, 2016

seberg commented Sep 2, 2016

charris commented Sep 2, 2016

seberg commented Sep 2, 2016

seberg commented Sep 2, 2016