Skip to content

CMake portable build #7232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 74 commits into
base: main
Choose a base branch
from

Conversation

mmomtchev
Copy link
Contributor

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the existing coding patterns and practices as demonstrated in the repository.

This is the CMake portable build that was started by @MarcoMartins86 and then maintained by @Cyriuz.

This build is mainly targeted at portable software projects that include ImageMagick as a library and are distributed as source.

It is still not capable of fully replacing the existing autotools build on UNIX for application users - though it might be one day. It will never replace the existing application build on Windows which also includes GUI elements - though it might become one day one of its components.

It is currently in a much more advanced state and comes with:

  • unified build system that supports (most) of the original delegates in an identical way across all platforms
  • optional conan integration on all platforms allowing for a fully self-contained and fully reproducible build on all platforms
  • supports pkg-config packages without conan
  • supports cross-compilation and comes with a WASM build
  • unit-tested on all supported platforms and configrations (CMake portable build)
  • fully integrated debug builds which include all dependencies
  • generates native build files on each platform but can also use non-standard compilers and runners (clang on Linux and Windows, or ninja on all platforms)

You can see a brief overview of the build process here: https://github.com/mmomtchev/ImageMagick/blob/cmake-portable-build/README.CMake.md

@urban-warrior
Copy link
Member

Thank you for submitting your pull request. However, after reviewing it, we have decided not to merge it as it is incomplete and would disrupt the current build environment. Our recommendation is to package the build so a user can download and unarchive it into the ImageMagick source folder for a CMake build. In the mean-time, after successfully testing the build, we can reference it on the ImageMagick build page as an alternative build. We will continue to test and fix any issues with the CMake build and may revisit the possibility of replacing the ImageMagick autoconf/automake build in the future with a CMake build.

@mmomtchev
Copy link
Contributor Author

Can you provide more details on what is incomplete and what would disrupt the environment, I am willing to fix it and to maintain it.
Everyone can check this out from my branch.

@Cyriuz
Copy link

Cyriuz commented Apr 8, 2024

We've been very close getting this into conan which would be a great first step I think. @valgur did a huge job trying to get it to work (Cyriuz#2 & conan-io/conan-center-index#21699) but I think both of us didn't have more time to spend on it yet unfortunately.

@urban-warrior
Copy link
Member

We need to ensure the robustness and stability of the CMake build before we can contemplate merging it, and we're not keen on maintaining both builds within the same source distribution. Instead, we propose offering users the option to download the CMake files and integrate them into the ImageMagick source distribution. We'll then rely on user feedback to identify any potential build issues, missing delegate support, or other concerns. Once we confirm that the build is not only complete but also particularly robust, we'll consider replacing the current autoconf/automake build. It's important to note that the Azure pipelines currently rely on the autoconf/automake build, so we'll need time to test the pipeline's dependencies with CMake.

Could you post instructions for the CMake build in the discussion section and mark it as an announcement? The build should be hosted on GitHub so that users can submit pull requests to encourage collaboration. The ImageMagick development team will initiate local builds and assess the Azure build pipelines. Once we are assured that CMake effectively replaces and enhances all the functionalities of the autoconf/automake builds, we will reconsider merging.

@valgur
Copy link
Contributor

valgur commented Apr 9, 2024

My version of the CMake-based build system is still missing some major parts for general-purpose use:

  1. Dependency handling without Conan. I.e. creating Find*.cmake modules for dependencies that don't install a CMake config file for find_package(). This is not going to be an easy task for a project with this magnitude of dependencies.
  2. Generating and installing a CMake config for ImageMagick itself. Should not be too difficult.

Another aspect is whether CMake is the build system of choice that you want to go with. Meson has been popular a popular choice in many OSS projects recently, especially ones that have previously relied on Autotools, such as the whole GNOME ecosystem. I'm thinking of creating an alternative Meson-based implementation in addition to the CMake one to see how it compares.

@mmomtchev
Copy link
Contributor Author

mmomtchev commented Apr 9, 2024

Ok, I am not very sure that I understand how exactly robustness and stability apply to a build, and I will continue to maintain this build. This PR will always stay current.

If someone else thinks that meson should be used, he is obviously free to make a meson build.

I also intend to officially question and doubt the actual motives for those decisions.

@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from 312cca2 to 9b41d7d Compare April 27, 2024 18:18
@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from 9cecaf7 to 5b82da0 Compare May 6, 2024 08:09
@mmomtchev mmomtchev force-pushed the cmake-portable-build branch 3 times, most recently from 7678915 to b31881a Compare July 4, 2024 16:33
@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from b31881a to b8b6fd2 Compare July 19, 2024 21:34
@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from 1f10c87 to e8e8365 Compare July 29, 2024 09:58
@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from abbf991 to 8ba8ef6 Compare October 10, 2024 10:31
@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from 8ba8ef6 to 965266d Compare January 11, 2025 10:52
@lunacd
Copy link

lunacd commented Feb 2, 2025

It seems that @urban-warrior would want some user testing before cutting over to CMake, which is perfectly reasonable.

I would personally love to see this thing merged and then ImageMagick made available via Conan Center. Thank you greatly ImageMagick devs for this exceptional project and people who worked on getting it into Conan, and of course, @mmomtchev for making this patch and keeping it up to date for this long

To move things forward here, @mmomtchev, how hard would it be it provide experimental binary builds from the fork? We could, as @urban-warrior proposed, post an announcement in discussions. That announcement could link to the alternative CMake builds that people could try out. I, for one, would be more than happy to test Windows and Linux builds and provide feedback/attest to its quality.

I'd be happy to help tweak whatever CI to make the experimental build :)

@mmomtchev
Copy link
Contributor Author

mmomtchev commented Feb 2, 2025

This build is mostly geared towards building the library. Its strength lies in the fact that it can be included in other cross-platform projects distributed as source code. For example, the PHP bindings might be interested.

ImageMagick on Windows includes a GUI tool which this build does not include. On Windows it would provide only the CLI tool.

I can create a workflow that produces statically linked binaries - as well as a statically linked library. This workflow will be launched manually at each new release. The only problem would be where to upload them. I can create releases in my fork. However I wonder if anyone actually needs such binaries and why.

@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from e08f77e to 2b787a3 Compare February 2, 2025 18:18
@lunacd
Copy link

lunacd commented Feb 2, 2025

This build is mostly geared towards building the library. Its strength lies in the fact that it can be included in other cross-platform projects distributed as source code. For example, the PHP bindings might be interested.

Yes, I'm completely onboard with why we are making the switch. My personal desire for this patch is being able to access it from Conan, which is not exactly what you are describing but close enough.

ImageMagick on Windows includes a GUI tool which this build does not include. On Windows it would provide only the CLI tool.

It seems that ImageMagick devs are hesitant with having two build systems in the source tree and prefers completely cutting over to CMake when it fully replaces autotools, judging from #7232 (comment). If this were my project, I would be okay as long as people volunteer to maintain that extra build system, but this isn't my project and I can completely understand their hesitance.

Then this seems like the gap? I can find some time to fill that gap and give you a PR, if that's okay with you

@mmomtchev
Copy link
Contributor Author

I am not very familiar with the GUI tool on Windows, but I think that it is a .NET project. CMake can easily replace autotools, but replacing the Visual Studio build for a .NET project is another matter. Maybe it is possible - it should be - but I prefer that someone who knows very well .NET, Visual Studio and CMake on Windows to give his opinion.

@lunacd
Copy link

lunacd commented Feb 2, 2025

Ah I probably should take a closer look at the repo and all of its components. I imagine the goal here is for CMake to completely replace autotools, but not necessarily all build systems. If that GUI tool is not built with autotools to begin with, perhaps @urban-warrior and other devs might be okay with them staying as is?

Again I should go read the codebase :)

@urban-warrior
Copy link
Member

urban-warrior commented Feb 2, 2025

We have no objections to using CMake. However, any delay in its adoption is due to the significant effort required to transition from Autotools. This includes evaluating whether CMake can match the coverage and capabilities of Autotools, ensuring adoption by the user community, installing new toolsets to accommodate CMake, and retooling our GitHub Actions to build the binaries.

To address these concerns, we recommend the following steps:

  1. Ensure Comprehensive Coverage: First, verify that the CMake configuration covers all or most of the build options and capabilities provided by Autotools.
  2. Package and Document: Package the CMake configurations and include thorough documentation. This should allow users to unpack the configurations into the ImageMagick source distribution and build a binary.
  3. User Adoption: Once the CMake package is ready, we will include a link to it on the ImageMagick website. This will facilitate user adoption.
  4. Dual Builds: After gaining sufficient user adoption, we can include the CMake configuration directly in the ImageMagick source distribution, allowing for dual builds using either CMake or Autotools.
  5. Modify GitHub Actions: Update our GitHub Actions to use CMake instead of Autotools.
  6. Deprecate Autotools: Eventually, we will deprecate Autotools while still supporting dual builds to accommodate our user base and Linux RPM package builds that depend on Autotools.
  7. Final Transition: Finally, we can consider distributing only the CMake configuration with the ImageMagick source, while providing an Autotools package for those unable to make the switch.

ImageMagick is an open-source project, but it operates with the reliability and stability expected of an enterprise package. Any disruptions can lead to significant backlash from the community. Therefore, we must proceed cautiously to ensure a smooth transition.

@lunacd
Copy link

lunacd commented Feb 2, 2025

@urban-warrior Thank you for your very detailed response!

Just to align everyone's expectations here, what stage (1-7) do you expect this PR this reach before we could proceed to merge this PR? To rephrase, would you rather we finish stage 1, merge, then come to stage 2 with another PR, or would you rather, for example, for this PR to complete all the way to stage 5 before we could merge this PR?

@urban-warrior
Copy link
Member

We'd like to see a cmake configuration build package with instructions that users can download and drop-into the ImageMagick source distribution and build a binary with cmake in place of configure. Once its ready, announce the package in the ImageMagick Github forums so that the user community can provide feedback and inform the future evolution of the package. Once this effort stabilizes, we can then consider next steps.

@lunacd
Copy link

lunacd commented Feb 2, 2025

Gotcha! I'll put together some instructions when I find the time and hopefully get user testing kickstarted :)

@urban-warrior
Copy link
Member

We added a 'cmake build' label for this specific effort. Be sure to label your discussions with cmake build.

@mmomtchev
Copy link
Contributor Author

mmomtchev commented Feb 2, 2025

Coverage:

  • currently CMake is missing tested support for a number of somewhat exotic libraries that are not present on Conan
    • the most important of these is librsvg which is now implemented in Rust and it is not very trivial to add to Conan
    • pango was not ported to Conan 2, but now can be added
    • Ghostscript is a huge problem and will have to be installed by other means
    • the rest is autotrace, FLIF, Flashpix, libgvc (???), libraqm, WMF, Liquid Rescale and libumem
  • the dynamic module loading is still not working

mmomtchev and others added 29 commits July 18, 2025 19:11
* ability to disable select delegates

* fix the conan logic

* fix fallback to pkgconfig when cmake module is found but does not have the target

* handle X11 on Windows
)

* disable jemalloc by default and test disabling/enabling

* add the CUSTOM_DELEGATES var to the CMake command

* replace the expected delegates
* use POSIX file structure on Windows

* restore line ident

* restore formatting

* add missing config defines

---------

Co-authored-by: Momtchil Momtchev <momtchil@motmchev.com>
* link X11 statically when building a static library

* fix libpng pkg-config name

* pick libdeflate manually and add more pkgconfig names

* the gnu linker needs a link group for the static libraries

* implement a 3 stage dependency resolution

use the CMake find modules only as a last resort

* enumerate properly all X11 targets

* disable tiff when building with the system libraries on ubuntu

* do not use pkg-config for X11

* fix a typo

* restore the additional X11 targets

* use an invalid name for X11 pkg-config

* drop the obsolete openjp2_static target

* disable zip on macos/system

* correct cmake package name of ZIP is libzip

* remove the shunt

* refactor to eliminate the dulication

* add a todo
* try adding macos-arm

* do not run the autotools build at each push

* use the python action

* do not include openmp on mac/arm/debug
* use a separate option for static libs

* get jxl from conan and add it to the required delegates

* dedupe highway

* use the latest version and allow conan to resolve the deps

* experiment with NO_CMAKE_INSTALL_PREFIX

* compatibility with both jxl as one package and as two packages

* include the _cms version too (if present)

* workaround for jxl on linux/conan/shared

* do not expect jxl

* solve the question of life, the universe and everything

* fix the info message

* update the readme

* MAGICK_PREFER_STATIC_LIBS is not usable without conan

* upgrade the cache actions
* MinGW support

* add urlmon to MagickCore
* add LIBRARY_ABSOLUTE_PATH

* fix the copy&paste
* is this still needed?

* drop the link groups
* upgrade OpenEXR

* upgrade libheif

* set the C++ standard on the command line

* use C++20

* use C++20 everywhere

* ImageMagick now requires C++17

* use /std:clatest for [[fallthrough]]

* set it only for C

* fix typo

* fix the conan cache

* verbose build

* set the C standard on the target

* try setting on the target
* don't depend on bleeding edge language features

* restore identiation
* Add pango and pangocairo from conan

* drop pango + fontconfig on Windows

* use the correct pkgconfig name

* the delegates test uses C++17
* add more Ubuntu libs

* add graphviz-dev package

* add dejavu

* add lqr

* fix LQR
* experiment with a solution for gslib

* rename the find module and enable debug

* rename

* rename again

* set the CMake target name

* use a manual configuration

* drop the debug
* macOS fonts

* dynamically construct the installed files list

* add Linux fonts

* fix typo

* update the doc
* no need to copy the sources

* test the configurations

* fix typos

* more typos

* build the delegates

* fix rpath and config on Linux and test the installed CLI

* use the correct path

* fix the rpath on macOS

* test running the CLI from the build tree

* fix path

* add the Windows path

* Windows DLLs require PATH
@mmomtchev mmomtchev force-pushed the cmake-portable-build branch from 8814df0 to 84a2a6e Compare July 18, 2025 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

6 participants