C Libvips implementation - Code review and some questions #4374

jstkdng · 2025-02-01T05:25:26Z

jstkdng
Feb 1, 2025

Hello, I've rewritten my project from vips-cpp to C libvips and I found that it's memory usage goes from 8m (without loading any image) to 1gb after many images are loaded/unloaded.

Is that memory usage normal?
Is my memory management correct after vips operations?
Why doesn't vips_thumbnail accept an access parameter
Is vips_thumbnail faster than vips_thumbnail_image with VIPS_ACCESS_SEQUENTIAL?
How can I get a read only pointer to the image data contained in VipsImage?

This is the main file containing the implementation and the header file. I appreciate any help and thank you for building this awesome library.

#include "image/vips.hpp"
#include "base/image.hpp"
#include "util/result.hpp"

#include <spdlog/spdlog.h>
#include <vips/vips.h>

#include <ranges>
#include <unordered_set>
#include <utility>

namespace upp
{

LibvipsImage::LibvipsImage(std::string output) :
    output(std::move(output))
{
}

LibvipsImage::~LibvipsImage()
{
    g_object_unref(image);
}

auto LibvipsImage::can_load(const std::string &file_path) -> bool
{
    return vips_foreign_find_load(file_path.c_str()) != nullptr;
}

auto LibvipsImage::load(ImageProps props) -> Result<void>
{
    this->props = std::move(props);
    return read_image().and_then([this]() -> Result<void> {
        resize_image();
        process_image();
        logger->info("loaded image {}", this->props.file_path);
        return {};
    });
}

auto LibvipsImage::read_image() -> Result<void>
{
    image = vips_image_new_from_file(props.file_path.c_str(), "access", VIPS_ACCESS_SEQUENTIAL, nullptr);
    if (image == nullptr) {
        return Err("failed to load image");
    }
    return {};
}

void LibvipsImage::process_image()
{
    const std::unordered_set<std::string_view> bgra_outputs = {"x11", "chafa", "wayland"};
    if (bgra_outputs.contains(output)) {
        // alpha channel required
        if (vips_image_hasalpha(image) == FALSE) {
            vips_addalpha(image, &image_out, nullptr);
            g_object_unref(image);
            image = image_out;
        }

        // convert from RGB to BGR
        std::vector<VipsImage *> bands(num_channels());
        for (auto [idx, band] : std::views::enumerate(bands)) {
            vips_extract_band(image, &band, static_cast<int>(idx), nullptr);
        }
        std::swap(bands[0], bands[2]);

        vips_bandjoin(bands.data(), &image_out, static_cast<int>(bands.size()), nullptr);
        g_object_unref(image);
        image = image_out;

        for (auto *band : bands) {
            g_object_unref(band);
        }

    } else if (output == "sixel") {
        // sixel expects RGB888
        if (vips_image_hasalpha(image) == TRUE) {
            vips_flatten(image, &image_out, nullptr);
        }
        g_object_unref(image);
        image = image_out;
    }

    image_buffer.reset(static_cast<unsigned char *>(vips_image_write_to_memory(image, nullptr)));
}

void LibvipsImage::resize_image()
{
    if (props.scaler == "contain") {
        contain_scaler();
    }
}

auto LibvipsImage::num_channels() -> int
{
    return vips_image_get_bands(image);
}

void LibvipsImage::contain_scaler()
{
    auto [new_width, new_height] =
        contain_sizes({.width = props.width, .height = props.height, .image_width = width(), .image_height = height()});

    logger->debug("resizing image to {}x{}", new_width, new_height);

    vips_thumbnail(props.file_path.c_str(), &image_out, new_width, "height", new_height, nullptr);
    g_object_unref(image);
    image = image_out;

    vips_colourspace(image, &image_out, VIPS_INTERPRETATION_sRGB, nullptr);
    g_object_unref(image);
    image = image_out;
}

auto LibvipsImage::data() -> unsigned char *
{
    return image_buffer.get();
}

auto LibvipsImage::width() -> int
{
    return vips_image_get_width(image);
}

auto LibvipsImage::height() -> int
{
    return vips_image_get_height(image);
}

} // namespace upp

jcupitt · 2025-02-01T11:16:22Z

jcupitt
Feb 1, 2025
Maintainer

Hello @jstkdng, your project looks interesting!

Is that memory usage normal? Is my memory management correct after vips operations?

You might have a reference leak. I would add vips_shutdown() just before you terminate, then if you run your program with VIPS_LEAK set it'll output a list of leaked references.

$ VIPS_LEAK=1 ./a.out
....

Why doesn't vips_thumbnail accept an access parameter

It always picks the best access mode for you (usually sequential, but not always).

Is vips_thumbnail faster than vips_thumbnail_image with VIPS_ACCESS_SEQUENTIAL?

It depends on the image format, number of pixels, size of thumbnail, etc., but yes, it can be many times faster. For example, with a 6k x 4k JPEG I see:

$ time vips thumbnail theo.jpg x.jpg 512
memory: high-water mark 15.20 MB

real	0m0.164s
user	0m0.250s
sys	0m0.059s
$ time vips thumbnail_image theo.jpg x.jpg 512
memory: high-water mark 140.97 MB

real	0m0.254s
user	0m0.640s
sys	0m0.144s

thumbnail_image needs 3x more CPU and 10x more memory.

How can I get a read only pointer to the image data contained in VipsImage?

Most images don't really exist and have no data, they are just nodes in a computation graph.

You can force an image into memory and get a pointer with:

    if (vips_image_wio_input(image))
        error_handling();

    // get a pointer to the pixel at this coordinate
    unsigned char *p = VIPS_IMAGE_ADDR(image, 128, 2568);

But this will (of course) take a lot of time and memory.

0 replies

jstkdng · 2025-02-01T20:55:25Z

jstkdng
Feb 1, 2025
Author

I tried using VIPS_LEAK=1 but how can I interpret the output?

vips_threadset_free: peak of 17 threads
70 objects alive:
0) VipsImage (0x7945c00c1480), count=3
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb, jpegload
1) VipsColourspace (0x7945c00b5db0), count=1
VipsColourspace (colourspace), convert to a new colorspace, colourspace in=((VipsImage*) 0x7945c00b4710) out=((VipsImage*) 0x7945c00b5070) space=((VipsInterpretation) VIPS_INTERPRETATION_sRGB) -
2) VipsCopy (0x7945c00dc220), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00c3180) out=((VipsImage*) 0x7945c00dc950) -
3) VipsExtractBand (0x7945c00e17e0), count=1
VipsExtractBand (extract_band), extract band from an image, extract_band in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00a9a90) band=0 -
4) VipsSequential (0x7945c00cf950), count=1
VipsSequential (sequential), check sequential access, sequential in=((VipsImage*) 0x7945c00ceaa0) out=((VipsImage*) 0x7945c00cfd40) tile-height=10 -
5) VipsCopy (0x7945c00ed4c0), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00a9a90) out=((VipsImage*) 0x7945c00edcf0) -
6) VipsImage (0x7945c00af9b0), count=3
VipsImage (image), image class, 1269x841 uchar, 3 bands, srgb, jpegload
7) VipsImage (0x7945c00d3950), count=2
VipsImage (image), image class, 1269x841 uchar, 3 bands, srgb, jpegload
8) VipsImage (0x794590100de0), count=2
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb
9) VipsImage (0x7945c00cc1e0), count=2
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb, jpegload
10) VipsCopy (0x7945c00eec60), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00e6980) out=((VipsImage*) 0x7945c00ef7c0) -
11) VipsImage (0x7945c00e4f80), count=3
VipsImage (image), image class, 1269x841 uchar, 4 bands, srgb, jpegload
12) VipsExtractBand (0x7945c00e2110), count=1
VipsExtractBand (extract_band), extract band from an image, extract_band in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00aabd0) band=1 -
13) VipsAddAlpha (0x7945c00a8990), count=1
VipsAddAlpha (addalpha), append an alpha channel, addalpha in=((VipsImage*) 0x7945c00a8760) out=((VipsImage*) 0x7945c00a9050) -
14) VipsEmbed (0x7945c00dd950), count=1
VipsEmbed (embed), embed an image in a larger image, embed in=((VipsImage*) 0x7945c00dc950) out=((VipsImage*) 0x7945c00de170) x=7 y=0 width=3102 height=841 extend=((VipsExtend) VIPS_EXTEND_COPY) -
15) VipsCopy (0x7945c00ade10), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00ac450) -
16) VipsCopy (0x7945c00c0d90), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00b5070) out=((VipsImage*) 0x7945c00c1480) -
17) VipsImage (0x7945c00b4710) 1080 bytes, count=3
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb, jpegload
18) VipsCopy (0x7945c00a9ff0), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00a8760) out=((VipsImage*) 0x7945c00af9b0) -
19) VipsImage (0x7945900feeb0), count=1
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb
20) VipsReduceh (0x7945c00d2ea0) 9750 bytes, count=1
VipsReduceh (reduceh), shrink an image horizontally, reduceh in=((VipsImage*) 0x7945c00c3180) out=((VipsImage*) 0x7945c00d3950) hshrink=2.432818 kernel=((VipsKernel) VIPS_KERNEL_LANCZOS3) gap=2.000000 -
21) VipsResize (0x7945c00bfaf0), count=1
VipsResize (resize), resize an image, resize in=((VipsImage*) 0x7945c00b5070) out=((VipsImage*) 0x7945c00c0310) vscale=0.411046 scale=0.411046 -
22) VipsImage (0x7945c00ead30), count=3
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
23) VipsImage (0x7945c00cfd40), count=2
VipsImage (image), image class, 3087x841 uchar, 3 bands, srgb, jpegload
24) VipsImage (0x7945c00ac450), count=3
VipsImage (image), image class, 1269x841 uchar, 4 bands, srgb, jpegload
25) VipsCopy (0x7945c00e47f0), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00e4f80) -
26) VipsThumbnailFile (0x7945c00b1da0), count=1
VipsThumbnailFile (thumbnail), generate thumbnail from file, thumbnail filename="/home/jk/.config/variety/Downloaded/nasa_apod/2P_Encke_2023_08_24JuneLake_California_USA_DEBartlett.jpg" out=((VipsImage*) 0x7945c00dfea0) width=1269 height=841 -
27) VipsImage (0x7945c00ceaa0) 256 bytes, count=3
VipsImage (image), image class, 3087x841 uchar, 3 bands, srgb, jpegload
28) VipsImage (0x7945c00e97a0), count=1
VipsImage (image), image class, 1269x841 uchar, 4 bands, srgb, jpegload
29) VipsEmbed (0x7945c00cd150), count=1
VipsEmbed (embed), embed an image in a larger image, embed in=((VipsImage*) 0x7945c00cc1e0) out=((VipsImage*) 0x7945c00cd970) x=0 y=7 width=3087 height=2061 extend=((VipsExtend) VIPS_EXTEND_COPY) -
30) VipsImage (0x7945c00de170), count=1
VipsImage (image), image class, 3102x841 uchar, 3 bands, srgb, jpegload
31) VipsImage (0x7945900d8210) 1080 bytes, count=1
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb
32) VipsSource (0x7945c00b6020), count=3
VipsSource (source), input source, 
33) VipsImage (0x7945c00b5070), count=3
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb, jpegload
34) VipsImage (0x7945c00c3180), count=3
VipsImage (image), image class, 3087x841 uchar, 3 bands, srgb, jpegload
35) VipsCopy (0x7945c00a9c20), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00ab240) -
36) VipsImage (0x7945900fdbd0), count=2
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb
37) VipsImage (0x7945c00aabd0), count=2
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
38) VipsImage (0x7945c00c0310), count=2
VipsImage (image), image class, 1269x841 uchar, 3 bands, srgb, jpegload
39) VipsLineCache (0x7945c00d0880), count=1
VipsLineCache (linecache), cache an image as a set of lines, linecache in=((VipsImage*) 0x7945c00ceaa0) out=((VipsImage*) 0x7945c00d0ec0) tile-height=10 access=((VipsAccess) VIPS_ACCESS_SEQUENTIAL) -
40) VipsImage (0x7945c00cd970), count=1
VipsImage (image), image class, 3087x2061 uchar, 3 bands, srgb, jpegload
41) VipsCopy (0x7945c00ebd70), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00aabd0) out=((VipsImage*) 0x7945c00ec550) -
42) VipsBandjoin (0x7945c00e9510), count=1
VipsBandjoin (bandjoin), bandwise join a set of images, bandjoin in=((VipsArrayImage*) 0x7945c00e9440) out=((VipsImage*) 0x7945c00e97a0) -
43) VipsCopy (0x7945c00cb9a0), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00c1480) out=((VipsImage*) 0x7945c00cc1e0) -
44) VipsImage (0x7945c00d0ec0), count=1
VipsImage (image), image class, 3087x841 uchar, 3 bands, srgb, jpegload
45) VipsImage (0x7945c00a8760), count=3
VipsImage (image), image class, 1269x841 uchar, 3 bands, srgb, jpegload
46) VipsImage (0x7945c00e7e40), count=3
VipsImage (image), image class, 1269x841 uchar, 4 bands, srgb, jpegload
47) VipsImage (0x7945c00ab240), count=3
VipsImage (image), image class, 1269x841 uchar, 4 bands, srgb, jpegload
48) VipsSequential (0x7945900fd760), count=1
VipsSequential (sequential), check sequential access, sequential in=((VipsImage*) 0x7945900daef0) out=((VipsImage*) 0x7945900fdbd0) tile-height=8 -
49) VipsImage (0x7945c00dc950), count=2
VipsImage (image), image class, 3087x841 uchar, 3 bands, srgb, jpegload
50) VipsExtractArea (0x7945901008f0), count=1
VipsExtractArea (extract_area), extract an area from an image, extract_area input=((VipsImage*) 0x7945900fdbd0) out=((VipsImage*) 0x794590100de0) left=0 top=0 width=3087 height=2046 -
51) VipsColourspace (0x7945c00a7cb0), count=1
VipsColourspace (colourspace), convert to a new colorspace, colourspace in=((VipsImage*) 0x7945c00dfea0) out=((VipsImage*) 0x7945c00a8760) space=((VipsInterpretation) VIPS_INTERPRETATION_sRGB) -
52) VipsImage (0x7945c00ef7c0), count=3
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
53) VipsImage (0x7945c00e3b80), count=2
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
54) VipsCopy (0x7945c00e76b0), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00e7e40) -
55) VipsImage (0x7945c00dfea0), count=2
VipsImage (image), image class, 1269x841 uchar, 3 bands, srgb, jpegload
56) VipsImage (0x7945900daef0), count=3
VipsImage (image), image class, 3087x2046 uchar, 3 bands, srgb
57) VipsImage (0x7945c00e6980), count=2
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
58) VipsImage (0x7945c00ad160), count=2
VipsImage (image), image class, 1269x841 uchar, 4 bands, srgb, jpegload
59) VipsCopy (0x7945c00ea550), count=1
VipsCopy (copy), copy an image, copy in=((VipsImage*) 0x7945c00e3b80) out=((VipsImage*) 0x7945c00ead30) -
60) VipsLineCache (0x7945900fe810), count=1
VipsLineCache (linecache), cache an image as a set of lines, linecache in=((VipsImage*) 0x7945900daef0) out=((VipsImage*) 0x7945900feeb0) tile-height=8 access=((VipsAccess) VIPS_ACCESS_SEQUENTIAL) -
61) VipsImage (0x7945c00ec550), count=3
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
62) VipsImage (0x7945c00a9a90), count=2
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
63) VipsImage (0x7945c00a9050), count=8
VipsImage (image), image class, 1269x841 uchar, 4 bands, srgb, jpegload
64) VipsBandjoinConst (0x7945c00b0430), count=1
VipsBandjoinConst (bandjoin_const), append a constant band to an image, bandjoin_const in=((VipsImage*) 0x7945c00a8760) out=((VipsImage*) 0x7945c00ad160) c=255  -
65) VipsForeignLoadJpegFile (0x7945c00b67c0), count=1
VipsForeignLoadJpegFile (jpegload), load jpeg from file (.jpg, .jpeg, .jpe, .jfif), priority=50, is_a, get_flags, get_flags_filename, header, load, jpegload filename="/home/jk/.config/variety/Downloaded/nasa_apod/2P_Encke_2023_08_24JuneLake_California_USA_DEBartlett.jpg" out=((VipsImage*) 0x7945c00b4710) shrink=1 flags=((VipsForeignFlags) VIPS_FOREIGN_SEQUENTIAL) access=((VipsAccess) VIPS_ACCESS_SEQUENTIAL) fail-on=((VipsFailOn) VIPS_FAIL_ON_NONE) -
66) VipsReducev (0x7945c00c2970) 9750 bytes, count=1
VipsReducev (reducev), shrink an image vertically, reducev in=((VipsImage*) 0x7945c00c1480) out=((VipsImage*) 0x7945c00c3180) vshrink=2.432818 kernel=((VipsKernel) VIPS_KERNEL_LANCZOS3) gap=2.000000 -
67) VipsImage (0x7945c00edcf0), count=3
VipsImage (image), image class, 1269x841 uchar, 1 band, srgb, jpegload
68) VipsExtractBand (0x7945c00e6620), count=1
VipsExtractBand (extract_band), extract band from an image, extract_band in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00e6980) band=3 -
69) VipsExtractBand (0x7945c00e3850), count=1
VipsExtractBand (extract_band), extract band from an image, extract_band in=((VipsImage*) 0x7945c00a9050) out=((VipsImage*) 0x7945c00e3b80) band=2 -
13 VipsArea alive
	0x7945900e2d10 count = 6, bytes = 3024
	0x7945900d84c0 count = 6, bytes = 2660
	0x7945900d8510 count = 6, bytes = 5
	0x7945900db360 count = 6, bytes = 2
	0x7945c00e9440 count = 1, bytes = 32
	0x7945c00a91e0 count = 1, bytes = 8
	0x7945c00d3d80 count = 1, bytes = 8
	0x7945c00c35b0 count = 1, bytes = 8
	0x7945c00b2e90 count = 30, bytes = 3024
	0x7945c00b64f0 count = 30, bytes = 2660
	0x7945c00b56e0 count = 30, bytes = 5
	0x7945c00b2720 count = 30, bytes = 2
	0x7945c00b4aa0 count = 31, bytes = 8
memory: high-water mark 64.91 MB

I see a lot of copies (?

thumbnail_image needs 3x more CPU and 10x more memory

awesome, i'll keep using thumbnail_image

Most images don't really exist and have no data, they are just nodes in a computation graph.

rip, i'll keep using write_to_memory then

1 reply

jcupitt Feb 12, 2025
Maintainer

awesome, i'll keep using thumbnail_image

Don't use thumbnail_image! It is often very slow, and will often need a lot of memory.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

C Libvips implementation - Code review and some questions #4374

{{title}}

Replies: 2 comments 1 reply

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

C Libvips implementation - Code review and some questions #4374

jstkdng Feb 1, 2025

Replies: 2 comments · 1 reply

jcupitt Feb 1, 2025 Maintainer

jstkdng Feb 1, 2025 Author

jcupitt Feb 12, 2025 Maintainer

jstkdng
Feb 1, 2025

Replies: 2 comments 1 reply

jcupitt
Feb 1, 2025
Maintainer

jstkdng
Feb 1, 2025
Author

jcupitt Feb 12, 2025
Maintainer