Greetings, libvips is such a great image library that we used it to produce lots of images with text in it, but recently we noticed that core(SIGSEGV) has been happening for a long time since we used it (about half of year ago, it happened in a quite odd rhythm, and it did't affect our production, hence we found out this problem this late), and out of curiousty and future safety concern, we decided to take a look at it, here is the info we found out:

• system envirinment:

ubuntu 20.04.1 LTS (Focal Fossa) in a docker container

• vips version:

libvips-8.14.3, relase version (we did't compile it by ourself, instead downloaded it from open resource that can not be traced anymore)

• runtime system infomation when cores:

cpu at average 20%-40% with 8Cs
memory at average around 40- 60% with 8G, later we upgraded the memory to 32G, memory stays around 10%-20%.

• core frequency:

ussually one time per approxmitely one million images processed, put it in time, will happen once every 20-30hours at 10 qps, but some time occassionly, it cores after several days.

• is reproducible?

not reproducible, every time we can locate the exact input that caused the core, but under new environment, it can be processed smoothly without any problem. we even collected all the inputs that was being procesed before, during, and after the core, still did't reproduce.

• core trace

All the core traces we met boils down with the same end, libc's malloc free (free invalid pointer).

- our code example:

//internal data object that has its own render material and common render functions
std::vector<* INTERNAL_OBJ> reqs;

try
{
    std::vector<vips::VImage> tmpImages;
    for (int i = 0; i < reqs.size(); i++)
    {
        tmpImages.push_back((reqs.at(i))->process());
    }

    vips::VImage resultImage = tmpImages[0].copy();
    for (int i = 1; i < reqs.size(); i++)
    {
        resultImage = resultImage.join(tmpImages[i], VipsDirection::VIPS_DIRECTION_VERTICAL, vips::VImage::option()->set("expand", true)->set("align", VipsAlign::VIPS_ALIGN_LOW));
    }

    void *buffer = nullptr;
    size_t len = 0;
    if (!reqFormat.compare("jpg") || !reqFormat.compare("JPG"))
    {
        resultImage.write_to_buffer(".jpg", &buffer, &len, vips::VImage::option()->set("Q", 95));
    }
    else
    {
        resultImage.write_to_buffer(".png", &buffer, &len, vips::VImage::option()->set("compression", 1)->set("filter", VIPS_FOREIGN_PNG_FILTER_ALL));
    }

    while (!reqs.empty())
    {
        delete reqs.back();
        reqs.pop_back();
    }

}
catch (const vips::VError &err)
{
    LOG(ERROR) << err.what();
    LOG(ERROR) << "vips process err request id is : " << requestId << std::endl;
}

this try block is in a warper of rpc server, it gets to be called in paralle, so there could be more than a dozen of threads running this part of code.
core happened when the try block finisnes and the tmpImages auto-destroys itselves, the decode part should prove that tmpImages' content is alright.

• what we tried

We tried to compile libvips-8.14.3 in debug mode, and managed to get the debug info library in the production, but the image process became extremely slow, that we can not reach 10 qps, and no core was spotted.
so there are no detailed debug trace of the destruct function of VImage.

Without the knowledge of what Vimage's being processed and destroyed, our current guess it that it may be a memory double free problem caused by data race that only happened with certain parallel condition.

So we were hoping that anyone who might encount this problem before or who might have the sufficient knowledge of Vimage would throw us a pice of information or instructions that what might happen here and what should we do next to find our more infomation.
anything would be much appreciated.

btw
we did find and try out issue 401(lovell/sharp#401) 's method, but it did't help out.