Improved framework for writing find-modules

Hello,

I’m sorry if you are discussing about this topic somewhere else. Please point me if there are some plans about what I’m asking. The question is if you are considering writing an improved framework that makes it trivial to write a module that supports minimum common features (eg. _LIBRARIES, _INCLUDE_DIR, _FOUND variables, support debug/optimized libraries for windows, targets support, support external override for libraries, …), but also flexible enough to add any desired custom behavior. To me the degree of freedom for writing modules it’s the biggest remaining weaknesses of CMake: modules can or can not support common conventions or lack support for debug/optimized libraries, or can not support manually specifying libraries (which is useful on Windows). For example FindLibXml2 or FindFontconfig don’t support finding different debug/optimized libraries. Or FindPNG is still bugged after couple of years after I reported a simple bug (and also provided the solution). But I’m not complaining here if bugs are not fixed or features are not added to modules, because one can always find a workaround for such issues. What I would like to see in the medium/long term is to make it possible to write modules with no boiler plate code, so in the end modules can be converted to this framework, adding features and fixing common bugs at the same time.

Using result variables is really not recommended, find modules should instead provide imported targets.

I’m not sure what such a framework would look like – do you want some sort of declarative format where you just specify the library name and header names, and this tool would generate a FindFoo.cmake file for you?

There’s really no way getting around specifying the details for a library at some place in your code. Find modules do contain some boilerplate, but if you have a good template to work from you can basically just replace the library and header names, maybe some other trivial details, but it’s really not that bad IMHO.

I actually began writing this message some time ago, and never sent it, at a time where using variables was more common. While I consider myself a proficient user of CMake, and also use lot of new features that were added after 3.20, for some reasons I discovered targets just recently, but really the basic concept of my message applies to targets as well: it’s not trivial to write modules that have a basic set of common features that users expect from find-modules. Also many modules don’t support targets, and this strongly supports my point.

No. I’m asking for better CMake functions to directly write the find-module and describe the process of finding the library on better “rails”, getting the features I mentioned above with virtually zero boilerplate.

I tend to disagree with your “it’s really not that bad”: some CMake distributed modules are in a sad state and in general the feature level among modules is very inconsistent.

I strongly disagree with your “there’s no way around”: there’s always a better solution and never say something is impossible to do. Of course it’s not a framework easy to design: a lot of considerations needs to be done about naming conventions of libraries, prefixes/suffixes, versions, etc… But something better than current level of anarchy is definitely doable and once I also began sketching some ideas about such framework during a short meeting with colleagues. Unfortunately I didn’t take notes.

I agree with this. CMake-shipped find modules could really use some refactors and modernization.

So how do you imagine a find module to look after these new features have been implemented? There could be a single function that finds the library and creates imported targets for you, but you still need to specify the library name, header name(s), build configs to search for, etc.

You could also have a go at writing such an all-in-one function yourself.

First thing: I am sure CMake developers are aware of the strengths and the weakness of their software. I’m here primary to ask if my concern is shared among them and CMake users and if there are plans for the future. This a actually an area of interest where Meson taleb…eeehhrrr developers :grin: may come up with some good ideas, at some point. Last time I checked no.

Write the function probably not, but I could write some virtual examples of use to show what I have in mind. Maybe later on this weekend.

Does that mean checking _FOUND is also discouraged, and instead we should use if(TARGET ...)?

One of the weaknesses of variables, I’m finding, is that if the same find-module is used from several locations, and it has an include_guard(GLOBAL), the variables will only be set for the first directory that uses the module. Before the best practice became to vend imported targets, were we supposed to write find-modules that were intended to be repeatedly included (thus, most likely, repeating a lot of work)?

Is it still best practice to use find_package_handle_standard_args?

No. The find_package() documentation makes it very clear that the <PackageName>_FOUND variable is the canonical way to tell if a package was deemed to be found:

Regardless of the mode used, a <PackageName>_FOUND variable will be set to indicate whether the package was found. When the package is found, package-specific information may be provided through other variables and Imported Targets documented by the package itself.

The “mode used” here refers to whether a Find module or a <packageName>-config.cmake file was used to fulfil the request.

That’s a bug in the Find module. It should not use include_guard().

If you’re writing a Find module, then find_package_handle_standard_args() can be used, but think of it as something that may be useful as a convenience. I see many people misuse it to make decisions after having already created imported targets, but you should only create imported targets if you already know your Find module will be returning a “success” result. With today’s target-centric approaches, find_package_handle_standard_args() isn’t really buying you much.

Is there any chance of receiving some further comments on the topic of my post?

Ah, I missed that your original post was asking about something else. Let’s steer things back to that original question. In short, I’m not aware of any plans to expand or enhance the support for Find modules. These days, they are more the fallback option for when a project doesn’t provide its own config package files. Config files are generally far more reliable, since they are normally distributed as part of the package itself. Find modules are usually distributed separately and rely on heuristics to work out details about the package based on the knowledge of the Find module author when the module was written. Find modules frequently fall behind as the package they are associated with releases more versions and evolve their package contents.

CMake packages used to rely solely on setting some variables, which projects then had to pass around and use in fairly fragile ways. Providing targets is far more robust, easier to work with on the consumer side, and much more aligned with current best practice. Both config package files and Find modules can provide both targets and variables, but consumer code should always prefer to use just the targets if that’s enough for them to do what they need. The only variable that is still a must-define is the <packageName>_FOUND variable, which indicates the find_package() call was successful, and the find_package() command will ensure that is set unless the config file or Find module explicitly sets it to false.

Personally, I don’t see providing some kind of framework or expanded support for Find modules is consistent with the direction CMake is heading. Over time, we’d actually like to see fewer Find modules as projects provide config files instead. Find modules still have their place when a project doesn’t want to provide such config files for CMake (which happens for some projects where the maintainers are not interested in CMake support), but I’d normally recommend such Find modules only create targets, not variables. Only Find modules that need to preserve some sort of backward compatibility should set variables for consumers to find things the package provides.

Also note that Find modules and package config files are not intended to be customizable by the consumer. Apart from a few basic things like selecting package components and the package version, the Find module or config package file’s job is fixed: provide a clear, predictable set of targets (ideally) and variables (if you really have to) for the specific version of the package that was found. They might also define commands associated with the package, but that’s less common. They might also have side effects (e.g. pulling in other dependencies), but that’s getting outside the scope of this discussion.

CMake’s own Find modules are a fairly poor example to follow. They have a long history and many are quite old. Some have not had much attention for a long time, others are more actively updated where there is a community of users interested in maintaining it. These CMake-provided Find modules are constrained by needing to continue providing backward compatibility, which is why most of them still define variables. A project writing a new Find module from scratch should ideally define just targets, not variables.

For the aspect of your question about providing better “rails” for finding libraries within a Find module, that’s kinda difficult. Projects frequently do their own thing and don’t follow conventions. Different platforms have different conventions too. The Find module author has to use their knowledge of the package they are writing for and tailor the logic to that package’s quirks. There are not really any rails here other than look at the history of that package, work out all the different things it has done over its lifetime and try to write heuristics to cover all the different things they did. If you’re lucky, the package is predictable and follows conventions, in which case the heuristics might be quite simple. Quite often though, you are not that lucky and you’re essentially in “bespoke logic” land.

Probably not the answer you were hoping for, but that’s my honest assessment. In short, avoid writing Find modules at all if you can, the time is often better spent working with the upstream project to add CMake config package support there instead (if they are receptive).

Seems to me if the target exists and the module is only setting the _FOUND variable and declaring targets, after setting the _FOUND variable there’s nothing left to do and it should exit. What am I missing?

With today’s target-centric approaches, find_package_handle_standard_args() isn’t really buying you much

Hmm, looking through its arguments, except for the deprecated FOUND_VAR they all look useful to me. What is it about a target-centric approach that tends to make this function less valuable?

The only variable that is still a must-define is the <packageName>_FOUND variable, which indicates the find_package() call was successful, and the find_package() command will ensure that is set unless the config file or Find module explicitly sets it to false.

Unless FetchContent_MakeAvailable is setting that variable, that is not consistent with the results I’m seeing. I am setting up CMAKE_MODULE_PATH to point at a repository of Find modules—used across my suite of projects—that mostly just FetchContent_Declare and FetchContent_MakeAvailable (and maybe apply some workarounds that haven’t been upstreamed yet). Then I just use find_package and it works without complaint.

In short, avoid writing Find modules at all if you can

I’ve been planning to create my own “rails,” but this and other statements above makes me think I’m heading off in the wrong direction. That said, using FindModules is the only clean solution I’ve found to the problems outlined in this post.