Skip to content

[TST]: Appveyor Qt tests failing #24394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ksunden opened this issue Nov 7, 2022 · 10 comments · Fixed by #24397
Closed

[TST]: Appveyor Qt tests failing #24394

ksunden opened this issue Nov 7, 2022 · 10 comments · Fixed by #24397

Comments

@ksunden
Copy link
Member

ksunden commented Nov 7, 2022

Summary

Appveyor Qt sigint tests appear to be consistently failing due DLL loading

Relevant log portion
__________________________ test_sigint[show-kwargs0] __________________________
[gw0] win32 -- Python 3.8.13 C:\Miniconda3-x64\envs\test-environment\python.exe
target = 'show', kwargs = {'block': True}
    @pytest.mark.backend('QtAgg', skip_on_importerror=True)
    @pytest.mark.parametrize("target, kwargs", [
        ('show', {'block': True}),
        ('pause', {'interval': 10})
    ])
    def test_sigint(target, kwargs):
        backend = plt.get_backend()
        proc = WaitForStringPopen(
            [sys.executable, "-c",
             inspect.getsource(_test_sigint_impl) +
             f"\n_test_sigint_impl({backend!r}, {target!r}, {kwargs!r})"])
        try:
            proc.wait_for('DRAW')
>           stdout, _ = proc.communicate(timeout=_test_timeout)
lib\matplotlib\tests\test_backend_qt.py:130: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Miniconda3-x64\envs\test-environment\lib\subprocess.py:1028: in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <matplotlib.tests.test_backend_qt.WaitForStringPopen object at 0x0000004E17E65CD0>
input = None, endtime = 526.437, orig_timeout = 60
    def _communicate(self, input, endtime, orig_timeout):
        # Start reader threads feeding into a list hanging off of this
        # object, unless they've already been started.
        if self.stdout and not hasattr(self, "_stdout_buff"):
            self._stdout_buff = []
            self.stdout_thread = \
                    threading.Thread(target=self._readerthread,
                                     args=(self.stdout, self._stdout_buff))
            self.stdout_thread.daemon = True
            self.stdout_thread.start()
        if self.stderr and not hasattr(self, "_stderr_buff"):
            self._stderr_buff = []
            self.stderr_thread = \
                    threading.Thread(target=self._readerthread,
                                     args=(self.stderr, self._stderr_buff))
            self.stderr_thread.daemon = True
            self.stderr_thread.start()
    
        if self.stdin:
            self._stdin_write(input)
    
        # Wait for the reader threads, or time out.  If we time out, the
        # threads remain reading and the fds left open in case the user
        # calls communicate again.
        if self.stdout is not None:
            self.stdout_thread.join(self._remaining_time(endtime))
            if self.stdout_thread.is_alive():
>               raise TimeoutExpired(self.args, orig_timeout)
E               subprocess.TimeoutExpired: Command '['C:\\Miniconda3-x64\\envs\\test-environment\\python.exe', '-c', "def _test_sigint_impl(backend, target_name, kwargs):\n    import sys\n    import matplotlib.pyplot as plt\n    import os\n    import threading\n\n    plt.switch_backend(backend)\n    from matplotlib.backends.qt_compat import QtCore  # noqa\n\n    def interrupter():\n        if sys.platform == 'win32':\n            import win32api\n            win32api.GenerateConsoleCtrlEvent(0, 0)\n        else:\n            import signal\n            os.kill(os.getpid(), signal.SIGINT)\n\n    target = getattr(plt, target_name)\n    timer = threading.Timer(1, interrupter)\n    fig = plt.figure()\n    fig.canvas.mpl_connect(\n        'draw_event',\n        lambda *args: print('DRAW', flush=True)\n    )\n    fig.canvas.mpl_connect(\n        'draw_event',\n        lambda *args: timer.start()\n    )\n    try:\n        target(**kwargs)\n    except KeyboardInterrupt:\n        print('SUCCESS', flush=True)\n\n_test_sigint_impl('QtAgg', 'show', {'block': True})"]' timed out after 60 seconds
C:\Miniconda3-x64\envs\test-environment\lib\subprocess.py:1401: TimeoutExpired
----------------------------- Captured log setup ------------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend QtAgg version 5.15.7.
---------------------------- Captured stderr call -----------------------------
Windows fatal exception: code 0xc0000139
Current thread 0x00000900 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1166 in create_module
  File "<frozen importlib._bootstrap>", line 556 in module_from_spec
  File "<frozen importlib._bootstrap>", line 657 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<string>", line 12 in interrupter
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254 in run
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932 in _bootstrap_inner
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 890 in _bootstrap
Thread 0x00000344 (most recent call first):
  File "c:\projects\matplotlib\lib\matplotlib\backends\qt_compat.py", line 168 in _exec
  File "c:\projects\matplotlib\lib\matplotlib\backends\backend_qt.py", line 1022 in mainloop
  File "c:\projects\matplotlib\lib\matplotlib\backend_bases.py", line 3554 in show
  File "c:\projects\matplotlib\lib\matplotlib\pyplot.py", line 421 in show
  File "<string>", line 30 in _test_sigint_impl
  File "<string>", line 34 in <module>
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254, in run
    self.function(*self.args, **self.kwargs)
  File "<string>", line 12, in interrupter
ImportError: DLL load failed while importing win32api: The specified procedure could not be found.
---------------------------- Captured log teardown ----------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend agg version v2.2.
_________________________ test_sigint[pause-kwargs1] __________________________
[gw0] win32 -- Python 3.8.13 C:\Miniconda3-x64\envs\test-environment\python.exe
target = 'pause', kwargs = {'interval': 10}
    @pytest.mark.backend('QtAgg', skip_on_importerror=True)
    @pytest.mark.parametrize("target, kwargs", [
        ('show', {'block': True}),
        ('pause', {'interval': 10})
    ])
    def test_sigint(target, kwargs):
        backend = plt.get_backend()
        proc = WaitForStringPopen(
            [sys.executable, "-c",
             inspect.getsource(_test_sigint_impl) +
             f"\n_test_sigint_impl({backend!r}, {target!r}, {kwargs!r})"])
        try:
            proc.wait_for('DRAW')
            stdout, _ = proc.communicate(timeout=_test_timeout)
        except:
            proc.kill()
            stdout, _ = proc.communicate()
            raise
        print(stdout)
>       assert 'SUCCESS' in stdout
E       AssertionError: assert 'SUCCESS' in '\n'
lib\matplotlib\tests\test_backend_qt.py:136: AssertionError
----------------------------- Captured log setup ------------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend QtAgg version 5.15.7.
---------------------------- Captured stdout call -----------------------------
---------------------------- Captured stderr call -----------------------------
Windows fatal exception: code 0xc0000139
Current thread 0x00000664 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1166 in create_module
  File "<frozen importlib._bootstrap>", line 556 in module_from_spec
  File "<frozen importlib._bootstrap>", line 657 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "<string>", line 12 in interrupter
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254 in run
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932 in _bootstrap_inner
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 890 in _bootstrap
Thread 0x00000538 (most recent call first):
  File "c:\projects\matplotlib\lib\matplotlib\backends\qt_compat.py", line 168 in _exec
  File "c:\projects\matplotlib\lib\matplotlib\backends\backend_qt.py", line 408 in start_event_loop
  File "c:\projects\matplotlib\lib\matplotlib\pyplot.py", line 557 in pause
  File "<string>", line 30 in _test_sigint_impl
  File "<string>", line 34 in <module>
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Miniconda3-x64\envs\test-environment\lib\threading.py", line 1254, in run
    self.function(*self.args, **self.kwargs)
  File "<string>", line 12, in interrupter
ImportError: DLL load failed while importing win32api: The specified procedure could not be found.
---------------------------- Captured log teardown ----------------------------
DEBUG    matplotlib.pyplot:pyplot.py:316 Loaded backend agg version v2.2.

Proposed fix

No response

@ksunden
Copy link
Member Author

ksunden commented Nov 7, 2022

@tacaswell
Copy link
Member

Neither Qt or PyQt had releases on conda-forge in the last few days which was my first guess at what is going wrong.

Sorry I miss-read these logs on #24380 .

@ksunden
Copy link
Member Author

ksunden commented Nov 7, 2022

https://www.appveyor.com/updates/2022/11/07/

Probably related to the appveyor platform updates that rolled out today

@ksunden
Copy link
Member Author

ksunden commented Nov 7, 2022

Possibly also related:

matplotlib/.appveyor.yml

Lines 57 to 59 in 9f96205

# pull pywin32 from conda because on py38 there is something wrong with finding
# the dlls when installed from pip
- conda install -c conda-forge pywin32

Could be that the issue wasn't "pip vs conda", but rather some other factor... not sure what, though

@ksunden
Copy link
Member Author

ksunden commented Nov 7, 2022

https://ci.appveyor.com/project/ksunden/appveyor-test/build/job/akqg7r4vfncf862d

Replicated with a stripped down version of the tests (didn't install mpl, but installed all requirements still)

@ksunden
Copy link
Member Author

ksunden commented Nov 7, 2022

Okay, I know what is happening now:

Pywin32 released yesterday and so is on version 305 on pypi, but version 304 on conda forge

We attempt to install via conda because of the reasons indicated by the comment above. However, since the pypi version is higher, the conda install gets overridden with the PyPI version since pywin32 is specified in requirements/testing/extra.txt and it is installed with --upgrade

The pip version is broken for the conda install.

Our options are:

  • wait for conda forge to get 305, and just ignore it (maybe add some more comments)
  • remove pywin32 from requirements/testing/extra.txt (and special case it for azure pipelines if we care to include it there... it is already special cased for appveyor, and only applies to Windows CI, so not github actions)
  • See if reordering the conda install fixes the problem (testing this now on my test repo) (not a huge fan of doing conda package installs after pip package installs, but if it works...)

@ksunden
Copy link
Member Author

ksunden commented Nov 7, 2022

Reordering works with the caveat that it still shows up as the pypi version in conda list, and so is likely to be confusing if anybody is looking at a similar problem in the future...

@tacaswell
Copy link
Member

The mixing of conda and pip on CI is not great. Is it plausible to go pure conda on appveyor?

Another option is "does appveyor actually add coverage we do not get from azure + gha?" If not we should just drop the whole CI platform....

@ksunden
Copy link
Member Author

ksunden commented Nov 8, 2022

We do get windows from azure (GHA is not currently configured for windows, could be added, but haven't looked at usage caps/simultaneous run caps or anything).

We do not have any other tests which use conda, if that is important to us. (in particular, conda can be a little slow when it comes to updating qt dependencies... I know over at pyqtgraph we made explicitly sure to work with the version of qt available via conda, and CI as much)

As for going full conda, no reason why it couldn't be done, just would be an additional place to update deps as it can't use the requirements txt files since package names are sometimes different.

@timhoffm
Copy link
Member

timhoffm commented Nov 8, 2022

We have an environment.yml. I think that could be used for appveyor.

ksunden added a commit to ksunden/matplotlib that referenced this issue Nov 8, 2022
ksunden added a commit to ksunden/matplotlib that referenced this issue Nov 8, 2022
ksunden added a commit to ksunden/matplotlib that referenced this issue Nov 8, 2022
ksunden added a commit to ksunden/matplotlib that referenced this issue Nov 8, 2022
ksunden added a commit to ksunden/matplotlib that referenced this issue Nov 8, 2022
ksunden added a commit to ksunden/matplotlib that referenced this issue Nov 10, 2022
Closes matplotlib#24394

Adjust environment.yml

skip fontconfig private method test on windows

skip fontconfig private method test on windows

revert adding fontconfig explicitly to environment

Add wxpython back

Skip some wx tests

Upgrade to gtk4 in environment.yml

ignore private modules in getattr test

_backend_gtk would fail because it expects one of gtk4 or gtk3 to be imported first

Clean up appveyor.yml comments

skip memleak test for wxagg

conda activate does not, in fact, work

revert to simply activate
@QuLogic QuLogic added this to the v3.7.0 milestone Nov 14, 2022
melissawm pushed a commit to melissawm/matplotlib that referenced this issue Dec 19, 2022
Closes matplotlib#24394

Adjust environment.yml

skip fontconfig private method test on windows

skip fontconfig private method test on windows

revert adding fontconfig explicitly to environment

Add wxpython back

Skip some wx tests

Upgrade to gtk4 in environment.yml

ignore private modules in getattr test

_backend_gtk would fail because it expects one of gtk4 or gtk3 to be imported first

Clean up appveyor.yml comments

skip memleak test for wxagg

conda activate does not, in fact, work

revert to simply activate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants