Skip to content

Add new operation alpha_resize meant for downsampling images with alpha #1675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

alon-ne
Copy link
Contributor

@alon-ne alon-ne commented Jun 8, 2020

Add new operation alpha_resize meant for downsampling images with alpha. New tests in test_resample demonstrate cases that it doesn't add artifacts added by the existing premultiply/resize/unpremultiply procedure.

…ha. New tests in test_resample demonstrate cases that it doesn't add artifacts added by the existing premultiply/resize/unpremultiply procedure.
@jcupitt
Copy link
Member

jcupitt commented Jun 8, 2020

Hello @alon-ne,

resize should handle images with alpha correctly -- it's a bug if it doesn't.

There was a fix a few weeks ago for a dumb error in alpha upsizing:

ba0dea0

Do you see issues with resize using git master?

@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 8, 2020

Hello @alon-ne,

resize should handle images with alpha correctly -- it's a bug if it doesn't.

There was a fix a few weeks ago for a dumb error in alpha upsizing:

ba0dea0

Do you see issues with resize using git master?

Yes, this branch was based on master and when resizing the two test images added here (logo2.png and logo3.png) I see artifacts are added - unless I'm missing something of course :)

@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 8, 2020

Hello @alon-ne,
resize should handle images with alpha correctly -- it's a bug if it doesn't.
There was a fix a few weeks ago for a dumb error in alpha upsizing:
ba0dea0
Do you see issues with resize using git master?

Yes, this branch was based on master and when resizing the two test images added here (logo2.png and logo3.png) I see artifacts are added - unless I'm missing something of course :)

p.s. I did notice the fix for upsizing, here I'm talking about downsizing

@jcupitt
Copy link
Member

jcupitt commented Jun 8, 2020

Could you explain the motivation here? What are the artifacts, what causes them, how this patch fixes it, etc.

Do you have a link with some background?

@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 8, 2020

So the story is, in Wix we get some users' images that are actually logos in PNG format with an alpha channel. Specifically on some images we see some artifacts, I guess I could describe them as "jagged stripes". Here are two examples:

https://storage.googleapis.com/tempvid/alpha-resize/alpha-resize2a.html
https://storage.googleapis.com/tempvid/alpha-resize/alpha-resize3.html

On the left side you see the results of the existing implementation, on the right side you see the new implementation.
The top row is the original image, the second is resized, and the third is resized and then sharpened.
In the "El Taquito" image, there are some weird stripes on the Q of the logo, and they become more pronounced after sharpening. In the other image, the stripes appear on pretty much all the letters.

You'll probably have to zoom to about 250% in your browser to really see that, my html skills are not good enough to make the page do that on its own :)

I tried the same resizes in ImageMagick and saw that there these artifacts didn't show, so I followed the ImageMagick implementation and adapted it as best I could to libvips.

This is the image Magick code I followed: https://github.com/ImageMagick/ImageMagick/blob/8a36f4e4187f3ccf44e839923f3c9544afb548b4/MagickCore/resize.c

What causes the artifact - somehow (not sure exactly) premultiply and unpremultiply. I suspect some information is lost in those steps. The new implementation blends the alpha as part of the resize process which seems to help.
The new implementation also uses a polynomial approximation of the lanczos filter, and it re-calculates the kernel coefficients for every destination pixel (terribly inefficient, I know) and always does all calculations with floating point.

Feel free to treat the new implementation as a kind of reference, it really probably shouldn't be a separate operation at the end of the day.

@jcupitt
Copy link
Member

jcupitt commented Jun 8, 2020

I think you might have found a bug in thumbnail. With this original:

logo3

If I run:

vipsthumbnail logo3.png -s 328 -o logo3-thumbnail.png

I get:

logo3-thumbnail

But with:

vips resize logo3.png logo3-resize.png 0.138

I get:

logo3-resize

They should be identical (I think), but they are not. I'll investigate.

@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 8, 2020

I think the difference is that vips resize doesn't do premultiplication

@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 8, 2020

I think the difference is that vips resize doesn't do premultiplication

Also note that in the test pages I sent, in the left column, when I did resize I also did premultiply before and unpremultiply after. These files were generated by this test code:
https://github.com/wix-playground/libvips/blob/970bc67d3c8f5eb06226ae90fe6365db94ba4045/test/test-suite/test_resample.py#L131

jcupitt added a commit that referenced this pull request Jun 8, 2020
We were clipping alpha very aggressively. With over- and under-shoot
ringing on edges, this could introduce extra fringes.

see #1675
@jcupitt
Copy link
Member

jcupitt commented Jun 8, 2020

I think I might have fixed it in git master, would you have time to test?

@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 9, 2020

Absolutely, thanks!

@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 9, 2020

Looks good to me

@jcupitt
Copy link
Member

jcupitt commented Jun 9, 2020

Ah fantastic, thanks for testing, and thanks for pointing out this dumbness. This will be in 8.10.

The revised unpremultiply is even slightly quicker.

@jcupitt jcupitt closed this Jun 9, 2020
@alon-ne
Copy link
Contributor Author

alon-ne commented Jun 9, 2020

Thanks a million!

jcupitt added a commit that referenced this pull request Feb 27, 2021
We were not detecting division by zero carefully enough, nor clipping
the alpha range sufficiently in unpremultiply.

see #1941

also see #1675 for another
difficult test case
@jcupitt
Copy link
Member

jcupitt commented Feb 27, 2021

Hello, I've had to revise this code again :( I hope I've not broken your case. Would you have time to quickly double-check this? It's this branch:

https://github.com/libvips/libvips/tree/revise-premultiply

There's a test for this this too, so hopefully it won't break again.

@jcupitt jcupitt mentioned this pull request Feb 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants