Figure equality-based tests. #11408
Figure equality-based tests. #11408
Conversation
|
I like this! |
2eec7e8 to
91354a1
Compare
|
Seems useful. However, the new test is failing 😉 |
|
Yes, and I can't reproduce the failure locally... Can anyone else give it a try? |
|
@anntzer I just pulled your PR branch and ran Python 3.6.4 (from conda) |
|
yeah, sorry, works on my machine as well (well I skip svg) |
|
Interestingly my machine is a mac and it passes on my machine, and this passes the travis Mac tests. But not the linux ones. No idea whats going on here. |
91354a1 to
3e16090
Compare
Implement a `check_figures_equal` decorator, which allows tests where both the reference and the test image are generated. The idea is to allow tests of the form "this feature should be equivalent to that (usually more complex) way of achieving the same thing" without further bloating the test image directory. The saved images are properly created in the `result_images` folder, but cannot be "accepted" or "rejected" using the triage_tests UI (as there is indeed no reference image to be saved in the repo). Includes an example use case.
3e16090 to
9cd3fbe
Compare
|
Almost certain that the failure actually already existed previously on svg and pdf, except that we didn't actually test these two formats (only png) before; moreover that's likely due to old versions of inkscape and gs on travis. The new failure on Py3.5 is likely unrelated too. |
Implement a
check_figures_equaldecorator, which allows tests whereboth the reference and the test image are generated. The idea is to
allow tests of the form "this feature should be equivalent to that
(usually more complex) way of achieving the same thing" without further
bloating the test image directory.
The saved images are properly created in the
result_imagesfolder, butcannot be "accepted" or "rejected" using the triage_tests UI (as there
is indeed no reference image to be saved in the repo).
Includes an example use case. Other PRs that could use this feature
include #9426 and #11407.
PR Summary
PR Checklist