Skip to content

Sketch seed #26050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Sketch seed #26050

wants to merge 1 commit into from

Conversation

AryanSheka
Copy link

@AryanSheka AryanSheka commented Jun 2, 2023

PR summary

The pr is related to #25796

closes #13479

rebased from @oscargus

PR checklist

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for opening your first PR into Matplotlib!

If you have not heard from us in a week or so, please leave a new comment below and that should bring it to our attention. Most of our reviewers are volunteers and sometimes things fall through the cracks.

You can also join us on gitter for real-time discussion.

For details on testing, writing docs, and our review process, please see the developer guide

We strive to be a welcoming and open project. Please follow our Code of Conduct.

@rcomer
Copy link
Member

rcomer commented Jun 2, 2023

Thank you for your contribution @AryanSheka! Please put some information in the PR summary to tell us about the problem you are solving, with links to the issue being addressed and to the PR this replaces.

You will also need to rebase your branch to fix a conflict in a file that has changed both here and on our main branch.

If you need help, feel free to ask questions here. Or you may prefer to ask them in our incubator gitter room.

@AryanSheka
Copy link
Author

Hi,
The pr is related to #25796

I'm sorry I'm very new to this so I'm very confused about everything.

@story645
Copy link
Member

story645 commented Jun 3, 2023

Awesome, thanks for pulling this together! No need to apologize, we'd be happy to help you out w/ whatever you're confused about if you tell us what's tripping you up. We also have a new contributors meeting next tuesday that may be of help https://scientific-python.org/calendars/

@AryanSheka
Copy link
Author

AryanSheka commented Jun 3, 2023

I have tried to fix the code as much as possible, but still some tests are failing and I do not know how to fix it. Can someone please
help me??

@story645
Copy link
Member

story645 commented Jun 4, 2023

So the two tests that are failing are the xkcd image comparison tests, which means that this PR is probably messing with the default sketch. You can compare the expected and actual figures visually, see the image comparison test docs for how to find them https://matplotlib.org/devdocs/devel/testing.html#writing-an-image-comparison-test

@AryanSheka
Copy link
Author

AryanSheka commented Jun 4, 2023

So basically the image comparision test compares the expected image to the one we get from plotting. But if we see the main issue #13479 we are changing the sketch transformation of xkcd to make it give us a randomized pattern result rather than the repeating pattern result like the expected image.
So isn't it natural for the image comparision test to fail??

@rcomer
Copy link
Member

rcomer commented Jun 4, 2023

If I have understood the code comment, the pattern will be different than it was before, but it will still be deterministic because it just adds one to the previous seed value

/* handle the special-case value -1 which means: use the
previous seed plus one. If not a special value, only
remember it as the last one and return it.

So I think it should be enough to replace the baseline image with the result image that you now get?

@AryanSheka
Copy link
Author

AryanSheka commented Jun 5, 2023

Everytime I try to run the pytest in order to generate the images, I am always hit with the following error:

ImportError: cannot import name '_c_internal_utils' from partially initialized module 'matplotlib' (most likely due to a circular import) (/workspaces/matplotlib/lib/matplotlib/init.py)

I have tried fixing it but was unsuccessful.

@rcomer
Copy link
Member

rcomer commented Jun 5, 2023

Did you rebase since you installed Matplotlib? If so I think you will need to install it again as the C code will have to be re-compiled.

@AryanSheka
Copy link
Author

Well I have run into an issue, the expected image of xkcd and xkcd markers are:
xkcd-expected
xkcd_marker-expected

But we are getting :
xkcd-failed-diff

xkcd_marker-failed-diff

@story645
Copy link
Member

story645 commented Jun 6, 2023

@AryanSheka are you sure what you're showing isn't the diff (expected-actual) image?

@AryanSheka
Copy link
Author

AryanSheka commented Jun 6, 2023

I'm sorry my bad, I thought they were the same thing, I have updated the baseline images and am still getting an image_comparision error.
Here are the images I recieved when I ran the tests after updating the baseline_images.

xkcd-expected
xkcd-expected

xkcd-recieved
xkcd

xkcd-failed-diff
xkcd-failed-diff

It doesnt matter how many times I replace the baseline_images, the new one produced is always slightly (not very noticable) different from the old one and the test fails. Even if you reinstall matplotlib after replacing the images, and then test it , they fail.

@AryanSheka AryanSheka force-pushed the sketch-seed branch 3 times, most recently from 32087b7 to 0b03de1 Compare June 7, 2023 20:28
@story645
Copy link
Member

story645 commented Jun 7, 2023

@AryanSheka can you do me a favor and just try to generate the image twice not as part of the test suite and check if they're the same?

@AryanSheka
Copy link
Author

AryanSheka commented Jun 7, 2023

@AryanSheka can you do me a favor and just try to generate the image twice not as part of the test suite and check if they're the same?

Yep here are 2 images generated together with the exact same parameters.

a1
a2

An online image comparision shows a small difference in the marked area even though the parameters were same.
marked

@eudoxos
Copy link

eudoxos commented Jun 9, 2023

@AryanSheka well done, I think now someone only needs to figure out where to allocate Sketch::previous_seed (currently in path_converters.cpp) and how to integrate that into the build process in a non-hacky way.

@AryanSheka AryanSheka force-pushed the sketch-seed branch 3 times, most recently from f200920 to 170fddf Compare June 14, 2023 14:11
@AryanSheka
Copy link
Author

AryanSheka commented Jun 14, 2023

I have tried searching for other ways to allocate value to Sketch::previous_seed from the past few days, I could not find any other way to do this other than the way it is done in the code ( ie create path_converters.cpp and allocate value to Sketch::previous_seed in it). At this point I do not know what to do further.

@eudoxos
Copy link

eudoxos commented Jun 14, 2023

I could not find any other way to do this other than the way it is done in the code ( ie create path_converters.cpp and allocate value to Sketch::previous_seed in it). At this point I do not know what to do further.

I tried and also did not succeed, it needs to be allocated both in _backend_agg and _path. So as you have it, it is probably the best we can do.

Copy link
Member

@oscargus oscargus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments on the documentation (and a minor on the code).

@AryanSheka
Copy link
Author

AryanSheka commented Jul 19, 2023

I think the biggest thing is in trying to implement my specific suggestions (which j appreciate the effort) it's losing sight of the big picture "what do you need to test to make sure this is implemented correctly?"

Well I have added tests, I will tell the significance of each of them:

This tests whether passing a seed to the Artist.set_sketch actually changes the global rcParam

def test_artist_seed_update():

This tests the same but for xkcd

def test_xkcd_seed_update():

This tests xkcd with default parameters (seed==0). This indirectly also tests whether passing a seed to xkcd impacts the sketch.

This tests whether setting a seed through artist impacts the sketch (it should)

def test_xkcd_artist():

This tests whether changing the rcParam impacts the sketch (it should).

def test_xkcd_seed():

Overall these tests cover all the changes made in this pr(including seed rolling) and can ensure that future commits do not break the changes the made in this pr.

Note: The image comparision tests also test the rolling charecteristic of seed

@melissawm
Copy link
Member

The relevant failures here are

FAILED lib/matplotlib/tests/test_path.py::test_xkcd_artist[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 20.469):
FAILED lib/matplotlib/tests/test_path.py::test_xkcd_seed[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 20.341):

Corresponding diffs are:
xkcd_artist-failed-diff
xkcd_seed-failed-diff

@AryanSheka AryanSheka force-pushed the sketch-seed branch 3 times, most recently from d81bcae to 6a1d20e Compare July 25, 2023 19:50
Copy link
Member

@story645 story645 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the reason I'm kind of advocating using https://matplotlib.org/devdocs/api/testing_api.html#matplotlib.testing.decorators.check_figures_equal after verifying that the seed isn't 0 is b/c there's not a great here to verify that the image is using the passed in seed.

@story645 story645 requested a review from oscargus July 31, 2023 22:27
Copy link
Member

@story645 story645 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big thing is I'd like to see verification that the non-patch artists participate in the rolling behavior.

@@ -823,7 +823,8 @@ def draw(self, renderer):
gc.set_foreground(ec_rgba, isRGBA=True)
if self.get_sketch_params() is not None:
scale, length, randomness = self.get_sketch_params()
gc.set_sketch_params(scale/2, length/2, 2*randomness)
seed = self._sketch_seed
gc.set_sketch_params(scale/2, length/2, 2*randomness, seed)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this isn't incremented, are you sure that the rolling is correctly implemented?
like xkcd(seed=1234)
ax1.plot([1,2,3])
ax2.plot([1,2,3])

should have lines that look different - and yes please test if you haven't

Comment on lines +1021 to +1022
m_rand(0),
m_seed(seed)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused what's going on here

@AryanSheka
Copy link
Author

AryanSheka commented Aug 17, 2023

Apologies for the delay in making the changes. Have been quite busy the past few week and also will be busy for some time in the future (possibly next few weeks), hence I wont be able to work on this pr during that time. In the mean time anyone can feel free to finish this.
If it is still open later, I will work on it.

Seed can be manually set for path.sketch by modifying the value of rcParams path.sketch_seed or by passing a seed value to xkcd or Artist.set_sketch . Seed will also have a rolling(auto incrementing) behaviour.

Co-Authored-By: Oscar Gustafsson <8114497+oscargus@users.noreply.github.com>
Co-Authored-By: eudoxos <1029876+eudoxos@users.noreply.github.com>
@story645
Copy link
Member

story645 commented Aug 18, 2023

Thanks for all the work so far, I rebased it to make it a little bit easier to review.

@story645
Copy link
Member

story645 commented Nov 8, 2023

Hey @AryanSheka, just wanted to check about your availability for finishing this? otherwise I'll write finish this out.

@AryanSheka
Copy link
Author

Hey @story645, I don't think so I will be able to contribute to this pr further. Please feel free to move ahead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Needs review
Development

Successfully merging this pull request may close these issues.

path randomness is repetitive (sketch RNG seeded to 0 for each path anew)
8 participants