Add jxl chunked save #4174

jcupitt · 2024-09-29T14:28:50Z

This PR adds support for the new libjxl chunked save API.

We were preparing a whole frame of pixels then writing in one shot, but that's obviously not great for large images. This adds a new save path using JxlEncoderAddChunkedFrame(). With a 30k x 30k sample image I see:

$ /usr/bin/time -f %M:%e vips copy st-francis.jpg x.jxl
5719092:586.74
$ /usr/bin/time -f %M:%e vips copy st-francis.jpg x.jxl[chunked]
854360:573.48

It's very slightly faster, since it's able to overlap JPEG decode with JXL encode, and needs 7x less memory.

This is version 3 of this PR I think :( and is quite simple. The NOTES file outlines earlier (and crazily complicated) schemes I tried.

TODO:

this uses the iteration in JxlEncoderAddChunkedFrame() to drive libvips evaluation (and not vips_sink_*()), so it's missing things like preeval, eval etc. -- we should add these
the old sequential save path can go once testing is over
we'll need to bump the min libjxl version to at least 0.11
add chunked load as well
add extra channel support (more than 4 bands)

wrote some NOTES on what we need to do next

instead, try to drive libvips from data_at it should be A LOT simpler

same speed, about half the memory use for a 10k x 10k pixel image still missing the signals you'd expect for a sink, eg. "preeval", "eval" etc.

Add preeval, eval, posteval, minimise signals. We have to move vips_image_preeval(), vips_image_eval(), vips_image_posteval() to the public API.

jcupitt · 2024-09-29T17:45:14Z

It seems (from #libjxl on discord) that we'll need to make a threadpool and generate libjxl tiles in parallel. We can't rely on _data_at() being executed in parallel.

…to add-jxl-chunked-save2

experiment with various caching strategies ... but it looks like none will work, since we will always throw away the cache on the original jpegload on minimise we're going to have to use the super-complicated sink_disc plan

jcupitt · 2024-10-01T13:47:35Z

We are doing parallel pixel generation by running a libvips pipeline for each 2k x 2k JXL tile. But this means that we emit a minimise signal at the end of each crop, and this drops the cache from the sequential attached to the jpegload at the end of each tile.

This PR works, but only by making sequential() persist over minimise. This will cause huge memory use.

We should probably bite the bullet and implement the incredibly complicated jxlsave outlined in NOTES.

jcupitt · 2024-10-01T14:58:55Z

Ah, a better fix! Don't emit minimise for sub-evaluations, just as we don't emit preeval, eval, etc.

ready for chunked load to go in

jcupitt · 2024-10-05T10:23:56Z

Sample JXL HDR image for testing:

https://github.com/immich-app/immich/files/14755028/JPEG.XL.HDR.Test.zip

libvips loads this as a ushort, it looks fine, I guess the HDR element is in the transfer function. There's a synthetic profile attached, but it only supports perceptual intent (libvips is relative by default).

$ vipsheader -f icc-profile-data JPEG_XL_HDR.jxl | base64 -d > hdr.icc
$ file hdr.icc 
hdr.icc: ColorSync color profile 4.4, type jxl, RGB/Lab-mntr device by jxl, 4344 bytes, 1-12-2019, 0xaa027a64eabe3514 MD5 'RGB_D65_DCI_Per_PeQ'

You can import to make a LAB image:

$ vips icc_import JPEG_XL_HDR.jxl x.v --intent perceptual
$ vipsheader x.v
x.v: 6000x4000 float, 3 bands, lab, jxlload

It looks nice:

But I can't see any out of range brightnesses:

$ vips stats x.v .csv
-75.852142333984375	94.802780151367188	2672661272.8074112	136308783384.3548	37.120295455658486	22.699366058239452	5923	3900	2379	2770
0	94.802780151367188	1401669949.0775206	89046135071.592117	58.402914544896689	17.301884669527556	4779	3376	2379	2770
-75.852142333984375	76.128402709960938	430761739.44445068	11741448443.999996	17.948405810185445	12.926010766460045	5923	3900	5390	3426
-58.478599548339844	89.867706298828125	840229584.28544009	35521199868.76268	35.009566011893334	15.949303834018592	4781	3346	2876	3245

(the second column in the second row is peak L*, 94.8 here, I'd expect to see values over 100 for specular highlights)

You can get the LAB->sRGB conversion automatically like this:

$ vips icc_import JPEG_XL_HDR.jxl x.jpg --intent perceptual

Making this image:

That will use the libvips LAB->sRGB conversion. You could also write:

$ vips icc_transform JPEG_XL_HDR.jxl x.jpg srgb --intent perceptual

That will convert to sRGB using LCMS and the srgb profile bundled with libvips.

jcupitt · 2024-10-05T10:36:12Z

We'll should probably make libvips ICC import fall back to perceptual if relative is not supported.

jcupitt · 2024-10-06T12:06:03Z

Useful JXL HDR test images:

https://sneyers.info/hdrtest/

jcupitt · 2025-04-30T17:48:43Z

With libjxl 0.11.1 in release mode and the current version of this PR I see:

$ /usr/bin/time -f %M:%e vips copy st-francis.jpg x.jxl
memory: high-water mark 3.99 GB
5475176:59.19
$ /usr/bin/time -f %M:%e vips copy st-francis.jpg x.jxl[chunked]
memory: high-water mark 1.69 GB
2151556:66.88

So it's still better, though less than it used to be thanks to libjxl improvements.

I bump the min libjxl version requirement to 0.11, which rules out a lot of current linuxes.

I'll give it another once-over and see if I can can improve it a bit more.

kleisauke

g_mutex_{init,clear}() still needs to be done for mutexes that are part of dynamically allocated structures.

libvips/foreign/jxlsave.c

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>

kleisauke · 2025-05-05T10:08:32Z

libvips/foreign/jxlsave.c

+		if (jxl->delay &&
+			n < jxl->delay_length)
+			header.duration = jxl->delay[n];
+		else
+			header.duration = jxl->gif_delay * 10;


Incorporates the changes from 94502d8 and 2672bb6.

Suggested change

if (jxl->delay &&

n < jxl->delay_length)

header.duration = jxl->delay[n];

else

header.duration = jxl->gif_delay * 10;

if (!jxl->is_animated)

header.duration = 0xffffffff;

else

header.duration =

vips_foreign_save_jxl_get_delay(jxl, n);

kleisauke · 2025-05-05T10:12:15Z

libvips/foreign/jxlsave.c

-				 */
-				memcpy(exif_data + 4, data, length);
+	if (jxl->info.have_animation) {
+		JxlFrameHeader header = { 0 };


Nice! It looks like the memset() in vips_foreign_save_jxl_add_frame() can also be replaced with zero initialization now (= { 0 }).

kleisauke · 2025-05-05T10:17:46Z

macOS CI failures looks real, but I couldn't reproduce that locally (on Fedora 42 with libjxl 0.11.1).

kleisauke · 2025-05-05T10:22:44Z

the old sequential save path can go once testing is over

We could also consider keeping it behind a oneshot flag (like jp2kload), but I'm not sure if that's worth it.

jcupitt added 11 commits September 16, 2024 10:30

start adding chunked load for jxl

c36aa02

Merge branch 'master' into add-jxl-chunked-load

0dea771

update todo

150b5d3

sort-of works

6104da7

wrote some NOTES on what we need to do next

Merge branch 'master' into add-jxl-chunked-save

cc37d71

sync

a598131

Merge branch 'add-jxl-chunked-load' into add-jxl-chunked-save

a7e7168

sync

f208181

sync before direction change

4533358

instead, try to drive libvips from data_at it should be A LOT simpler

switch back to crop

145bf9b

works!

7751da2

same speed, about half the memory use for a 10k x 10k pixel image still missing the signals you'd expect for a sink, eg. "preeval", "eval" etc.

jcupitt marked this pull request as draft September 29, 2024 14:28

jcupitt added 3 commits September 29, 2024 16:19

add signals for jxlsave

c5a71de

Add preeval, eval, posteval, minimise signals. We have to move vips_image_preeval(), vips_image_eval(), vips_image_posteval() to the public API.

Merge branch 'master' into add-jxl-chunked-save2

70a4b82

fix jxlsave DEBUG mode

c3ae7c0

jcupitt added 3 commits September 29, 2024 22:40

Merge branch 'add-jxl-chunked-save2' of github.com:libvips/libvips in…

c64b352

…to add-jxl-chunked-save2

use crop() for paralle demand

9a13a10

experiment with various caching strategies ... but it looks like none will work, since we will always throw away the cache on the original jpegload on minimise we're going to have to use the super-complicated sink_disc plan

works, sort of

e244055

jcupitt added 2 commits October 1, 2024 15:48

don't emit "minimise" for sub-evaluations

4cb4728

small cleanups

404c5d0

jcupitt added 4 commits October 1, 2024 16:10

Merge branch 'master' into add-jxl-chunked-save2

5f3a045

reformat jxlload

f5f5d7f

ready for chunked load to go in

Merge branch 'master' into add-jxl-chunked-save2

a121b7c

update jxlload ICC profile load API

a206dad

Merge branch 'master' into add-jxl-chunked-save2

dda21ff

jcupitt added 5 commits October 6, 2024 13:07

Merge branch 'master' into add-jxl-chunked-save2

4eebe3a

experiment with JxlDecoderSetPreferredColorProfile

3095582

Merge branch 'master' into add-jxl-chunked-save2

323ff2e

Merge branch 'master' into add-jxl-chunked-save2

4e9c45b

fix gmutes use

0588876

kleisauke reviewed Apr 30, 2025

View reviewed changes

libvips/foreign/jxlsave.c Show resolved Hide resolved

libvips/foreign/jxlsave.c Show resolved Hide resolved

Update libvips/foreign/jxlsave.c

6b22ef1

Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>

kleisauke reviewed May 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add jxl chunked save #4174

Add jxl chunked save #4174

jcupitt commented Sep 29, 2024 •

edited

Loading

jcupitt commented Sep 29, 2024

jcupitt commented Oct 1, 2024

jcupitt commented Oct 1, 2024

jcupitt commented Oct 5, 2024 •

edited

Loading

jcupitt commented Oct 5, 2024

jcupitt commented Oct 6, 2024

jcupitt commented Apr 30, 2025

kleisauke left a comment

kleisauke May 5, 2025

kleisauke May 5, 2025

kleisauke commented May 5, 2025

kleisauke commented May 5, 2025

Add jxl chunked save #4174

Are you sure you want to change the base?

Add jxl chunked save #4174

Conversation

jcupitt commented Sep 29, 2024 • edited Loading

jcupitt commented Sep 29, 2024

jcupitt commented Oct 1, 2024

jcupitt commented Oct 1, 2024

jcupitt commented Oct 5, 2024 • edited Loading

jcupitt commented Oct 5, 2024

jcupitt commented Oct 6, 2024

jcupitt commented Apr 30, 2025

kleisauke left a comment

Choose a reason for hiding this comment

kleisauke May 5, 2025

Choose a reason for hiding this comment

kleisauke May 5, 2025

Choose a reason for hiding this comment

kleisauke commented May 5, 2025

kleisauke commented May 5, 2025

jcupitt commented Sep 29, 2024 •

edited

Loading

jcupitt commented Oct 5, 2024 •

edited

Loading