that's also the fix I've used in #11570 now btw. That contains fixes for all deprecation warnings in the tests.

I pushed a fix having to do with a slightly dangerous caveat of ignore_warnings: if you pass the warning class as the first argument, then the test is not run:

from sklearn.utils.testing import ignore_warnings

@ignore_warnings(UserWarning)
def test():
    1/0

❯ pytest /tmp/test.py
======================================================== test session starts ========================================================
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /tmp, inifile:
plugins: cov-2.5.1, hypothesis-3.56.0
collected 0 items                                                                                                                   

=================================================== no tests ran in 0.14 seconds ====================================================

The reason is that the first argument is the test callable. Maybe that could be fixed not sure.

Sorry @amueller for not checking your work carefully.
Thanks @lesteve (I don't have time to check it tonight but I definitely trust you). I should have used recommended pytest.mark.filterwarnings. Maybe you should also post your comment to #11570.

I pushed a fix having to do with a slightly dangerous caveat of ignore_warnings:

Yes, that is very tricky, I would have missed that in review as well. I agree, we should try to switch to pytest.mark.filterwarnings if possible.

The limitation with the pytest.mark.filterwarnings can only be used at a test level. There is no way to use a context manager, which sometimes is convenient when you just want to ignore a warning for a few lines.

Resolved in #11570