CMake project review and suggestions (closed)

I’m working on my first CMake project, a C library.
I’ll ask for help and suggestions as I make progress (like an “art review”, but for CMake)
I’ve read plenty of material on the subject (documentation included, of course, it’s pretty good), but nevertheless I keep discovering better more modern alternatives everywhere, or I can’t find material at all, so I came to this forum to build a solid understanding of what I need (and why I need it) and very useful “how I would have done that, and why” from experts.

The project layout I came up with is the following:

visualt
│   CMakeLists.txt (1)
│   Doxyfile
│   Doxyfile.in // info such as current version are provided by CMake
│
├───src
│       CMakeLists.txt (2)
│       version.h // i found very ugly to put the generated header in the build directory, i'd rather put it in the same directory, and use it normally. i don't get why should it be in other places.
│       version.h.in // info such as version and build date are provided by CMake
│       visualt.c
│
├───include
│   └───visualt
│           visualt.h
│           visualtUnprefixed.h // alternative header, a wrapper around "visualt.h"
│
├───res // diagnostic and assets for development
│       CMakeLists.txt (3) // target for test.c
│       test.c // not a "test" in the classical meaning, it's just a diagnostic program
│       ...
│
└───examples // small programs that uses the library (with it's own "target_link_libraries()")
    │   CMakeLists.txt (4) // adds every subdirectory
    │
    ├───01
    │   │   CMakeLists.txt (6) // target for example01.c
    │   │   example01 // the target's RUNTIME_OUTPUT_DIRECTORY is set as CMAKE_CURRENT_LIST_DIR, because the executable might require files like "cat.obj"
    │   │   example01.c
    │   │
    │   └───res
    │           cat.obj
    │           cat.txt
    │
    ├───... // the same folder structure is repeated for every example
    │
    └───getch
            CMakeLists.txt (5) // an INTERFACE library (header only) used by examples
            getch.h

I’m stuck on: Installing

Here’s CMakeLists.txt (2), still incomplete:

# set headers
set(private_headers_path "${VisualT_SOURCE_DIR}/src")
set(public_headers_path "${VisualT_SOURCE_DIR}/include")

set(private_headers
    "${private_headers_path}/version.h")
set(public_headers
    "${public_headers_path}/visualt/visualt.h"
    "${public_headers_path}/visualt/visualtUnprefixed.h")

# set source files
set(srcs "visualt.c")

add_library(visualt ${private_headers} ${public_headers} ${srcs})
target_include_directories(visualt
                           PRIVATE ${private_headers_path}
                           PUBLIC ${public_headers_path})
set_target_properties(visualt PROPERTIES
                      PRIVATE_HEADER "${private_headers}"
                      PUBLIC_HEADER "${public_headers}")
target_compile_features(visualt PUBLIC c_std_99)

here’s CMakeLists.txt (1):

# Works with 3.11 and tested through 3.16
cmake_minimum_required(VERSION 3.11...3.16)

# Project name and a few useful settings. Other commands can pick up the results
project(VisualT
        VERSION 3.2.0
        DESCRIPTION "a text-based graphic library"
        LANGUAGES C)

if("${CMAKE_PROJECT_NAME}" STREQUAL "VisualT")
    string(TIMESTAMP VisualT_BUILD_DATE "%d %B %Y" UTC)
    configure_file(
            "${VisualT_SOURCE_DIR}/src/version.h.in"
            "${VisualT_SOURCE_DIR}/src/version.h"
            @ONLY
    )
    configure_file(
            "${VisualT_SOURCE_DIR}/Doxyfile.in"
            "${VisualT_SOURCE_DIR}/Doxyfile"
            @ONLY
    )
endif()

# library code is here
add_subdirectory(src)

# examples are here
add_subdirectory(examples)

# tests and resources are here
add_subdirectory(res)
if(${CMAKE_})
install(TARGETS visualt
        EXPORT VisualT
        RUNTIME DESTINATION bin #temporary
        LIBRARY DESTINATION lib #temporary
        ARCHIVE DESTINATION lib #temporary
        PUBLIC_HEADER DESTINATION include)

install(EXPORT my_library DESTINATION "${lib_dest}")
  1. As you can see I made an attempt to detect if the project is being compiled as the “main project”, or as a subProject. I have no idea if this is the right way to do that. Perhaps I should use CMAKE_BUILD_TYPE to discern?

  2. I’ve used PRIVATE_HEADER and PUBLIC_HEADER properties to avoid using install(FILE). But the main problem remains: I have no idea where I should put the files. I’ve read many discussion about this and I decided to discard the idea of having “one path for all” systems, and I decided instead to make it os-dependent.(picked from here) I guess I should use an if over CMAKE_SYSTEM_NAME's value?

  3. I’d like to use components to modularize a bit the installation (like for the examples, and make them OPTIONAL), but I don’t know how they correlate with export groups. What are the differences?

  4. I’ve seen the INCLUDES DESTINATION but I didn’t get it. What are the differences with target_include_directories()? I understood that they are applied only on the exported target? how? why?

That’s all for now, I hope to untangle everything step by step. In the meantime I’ll keep reading. Thank you!

1 Like

Files written to during the build should go to the build directory. If the header has per-build information, writing back to the source tree means your version.h is wrong for build trees sharing the same source tree with different contents.

This embeds your configure time into the header. If I configure today and build tomorrow, this value is “wrong” now.

It looks fine. CMAKE_BUILD_TYPE has nothing to do with the main project. It also doesn’t exist for multi-config generators anyways.

Use include(GNUInstallDirs) and use a path relative to CMAKE_INSTALL_INCLUDEDIR in your install(PUBLIC_HEADERS DESTINATION) arguments.

The paths documented in the page you linked are for config.cmake files, not headers.

Export sets each get their own targets.cmake file you need to include from your config.cmake file. How/when you include them depends on your own logic for COMPONENTS in your config.cmake file. They don’t need to correlate (but it’d be easier for you if they did).

INCLUDES DESTINATION seems to be a shortcut for target_include_directories(INTERFACE $<INSTALL_INTERFACE:>). I don’t know why it was added. Convenience maybe?

1 Like

thank you for your time!

Alright, I’ll take that, no need to reinvent the wheel.

Oof, that’s embarrassing. I totally missed that. I found an alternative here, but that’s not viable with the Doxyfile for example: the file is too large. Is there a common way to add those info (cersion and build date) with CMake or should I look elsewhere?

But I need to make the project cross-platform (at least Linux and Windows), shouldn’t I need to set the installation path accordingly anyway? (with an if over CMAKE_SYSTEM_NAME 's value maybe?).
I’m not sure where GNUInstallDirs would point to under windows. I assume it doesn’t use CMAKE_INSTALL_PREFIX because the various destinations aren’t even under a common folder, like they would be on windows.

I see. I think I’d prefer to have one export group, and many components. I’ll think about that again when I’ll be ready to think about installation customization

ok, sounds good to me. I can’t yet figure the need of a “export-only” include directory, but that’s probably because I don’t need that. (i hope)

Generate a file like this at configure time:

set(some_var_used "value")
# and so on for other variables (logic can be used)
configure_file(path/to/Doxyfile.in path/to/Doxyfile @ONLY)

then schedule to run that file during the build:

add_custom_command(
  OUTPUT path/to/Doxyfile
  DEPENDS path/to/script.cmake path/to/Doxyfile.in
  COMMAND "${CMAKE_COMMAND}" -Danother_var=value -P path/to/script.cmake
  COMMENT "Configuring Doxyfile")

The install prefix still makes sense there; projects rarely splay files everywhere (and if they do…they’re probably not nice to uninstall either). Most of the paths are pretty much the same between the platforms. Any difference can be handled by just modifying how some destinations are built up (typically where config.cmake files go, plugins, config files). The only difference is whether you have multiple architectures in a single install (I recommend deferring that until you have a single-arch install working).

The install will use your install tree, but your build will use your build tree. Since includes are (usually) very different between the two, you’ll likely need something like it eventually.

1 Like

This is the current root CMakeLists.txt (1)

# Works with 3.11 and tested through 3.16
cmake_minimum_required(VERSION 3.11...3.16)
include(GNUInstallDirs)

# Project name and a few useful settings. Other commands can pick up the results
project(VisualT
        VERSION 3.2.0
        DESCRIPTION "a text-based graphic library"
        LANGUAGES C)

macro(generate_configure_script)
    file(WRITE "${configure_script_path}" "
        configure_file(
                \"${version_in_path}\"
                \"${version_configured_path}\"
                @ONLY
        )
        configure_file(
                \"${doxyfile_in_path}\"
                \"${doxyfile_configured_path}\"
                @ONLY
        )
    ")
endmacro()

if("${CMAKE_PROJECT_NAME}" STREQUAL "VisualT")
    string(TIMESTAMP VisualT_BUILD_DATE "%d %B %Y" UTC)
    set(configure_script_path "${VisualT_BINARY_DIR}/configureScript.cmake")
    set(version_in_path "${VisualT_SOURCE_DIR}/src/version.h.in")
    set(doxyfile_in_path "${VisualT_SOURCE_DIR}/Doxyfile.in")
    set(version_configured_path "${VisualT_BINARY_DIR}/src/version.h")
    set(doxyfile_configured_path "${VisualT_BINARY_DIR}/Doxyfile")

    generate_configure_script()
    add_custom_command(
            OUTPUT "${version_configured_path}" "${doxyfile_configured_path}"
            DEPENDS "${configure_script_path}" "${version_in_path}" "${doxyfile_in_path}"
            COMMAND "${CMAKE_COMMAND}" -P "${configure_script_path}"
            COMMENT "configuring files")
endif()

# library code is here
add_subdirectory(src)
# examples are here
add_subdirectory(examples)
# tests and resources are here
add_subdirectory(res)

install(TARGETS visualt
        EXPORT VisualT
        RUNTIME DESTINATION "${CMAKE_INSTALL_LIBDIR}"
        LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
        ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
        PUBLIC_HEADER DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}")

install(EXPORT visualt DESTINATION "${CMAKE_INSTALL_LIBDIR}")

I’ve added the script generation code, and the custom command. I’ve read some documentation but I didn’t quite understand when the command is supposed to be ran:

A target created in the same directory ( CMakeLists.txt file) that specifies any output of the custom command as a source file is given a rule to generate the file using the command at build time.

Well, I’m generating a header and a doxyfile, so this could make sense only for the header.
Anyhow, the visualt target is not in the same CMakeLists (and nor it should be), so, as now, I can’t configure correctly because of the add_library(visualt ...version.h...) in CMakeLists.txt (2).
version.h doesn’t exist yet, as the command hasn’t been executed.

Additional doubts:
I couldn’t find how GNUInstallDirs is supposed to behave under windows. What I understand from the replies and the documentation is that all the locations lead to some subdirectory under CMAKE_INSTALL_PREFIX, named with the word between round brackets in the documentation.
For example, when I read

BINDIR

user executables ( bin )

That means that the variable leads to “CMAKE_INSTALL_PREFIX/bin” (/usr/local/bin for example).
The only exception seems to be OLDINCLUDEDIR, so I won’t use that one.

That said, if I change CMAKE_INSTALL_PREFIX to %ProgramFiles%/VisualT when in CMAKE_HOST_WIN32, everything should install nicely under that path.

sorry for the inactivity, i’ve been struggling to find spare time these weeks.

I’ve deepened a bit my understanding of dependencies, custom commands and custom targets, but I got stuck trying to make the generated source files work.
The main problem is that source files aren’t flagged globally as “generated”, so CMake doesn’t know it has to skip them when configuring.
I’ve read here that that’s how things currently work, fair enough, but they mention a workaround:

For now one can work around the problem by adding the generated property in the top-level directory:

set_property(SOURCE ${CMAKE_CURRENT_BINARY_DIR}/bar/bar.cpp PROPERTY GENERATED 1)

That in my case becames:
CMakeLists.txt (1)

set_property(SOURCE "${version_configured_path}" PROPERTY GENERATED 1)
set_property(SOURCE "${doxyfile_configured_path}" PROPERTY GENERATED 1)

That doesn’t work. The lines are in the root CMakeLists.txt (1), before the add_subdirectory() calls, but add_library(visualt ${private_headers} ${public_headers} ${sources}) still fails.

Even if I put them in src/CMakeLists.txt, the project configures, but at build time CMake still doesn’t know what to do to generate the files (run the command) and the build fails.

I’ve also tried creating a custom target:

add_custom_target(configured_files ALL
                  COMMAND "${CMAKE_COMMAND}" -P "${configure_script_path}"
                  BYPRODUCTS "${version_configured_path}" "${doxyfile_configured_path}"
                  DEPENDS "${configure_script_path}" #"${version_in_path}" "${doxyfile_in_path}"
                  COMMENT "configuring files")

But the situation didn’t really improve, but maybe it’s nicer because it’s a target.

Another problem arises, previously I had an if("${CMAKE_PROJECT_NAME}" STREQUAL "VisualT") that decided if to re-generate the files or not. With the completely out of source build I can’t do that anymore, I must generate them anyway. How do I handle the build date then? I have to store it somewhere, so I can write it back when the library isn’t build as the main project.

Thank You

I got this working as part of sprokit years ago. I suggest following the logic from the file I linked. version.h is the file to track. Sorry, I don’t know how better to explain it; I end up going back and cribbing myself most of the time when I need this myself :slight_smile: .

1 Like

I’ve seen the file, but from what I understood it does a different thing: that script fetches the extra info from git (the git hash).
I can’t do that, the build date is some string I have eventually to store somewhere, somehow. I can’t think of a completely out of source layout that could do that.

snippet
# ...
elseif (GIT_FOUND)
  set(configure_code "
  # ...
  execute_process(
    COMMAND           \"${GIT_EXECUTABLE}\"
                      rev-parse
                      HEAD
    WORKING_DIRECTORY \"${sprokit_source_dir}\"
    RESULT_VARIABLE   git_return
    OUTPUT_VARIABLE   sprokit_git_hash)
  # ...
  message(STATUS \"version: \${sprokit_version}\")
  message(STATUS \"git hash: \${sprokit_git_hash}\")
  message(STATUS \"git short hash: \${sprokit_git_hash_short}\")
  message(STATUS \"git dirty: \${sprokit_git_dirty}\")
endif ()
")
else ()
  set(sprokit_git_hash       "<unknown>")
  set(sprokit_git_hash_short "<unknown>")
  set(sprokit_git_dirty      "<unknown>")
endif ()

sprokit_configure_file_always(version.h
  "${CMAKE_CURRENT_SOURCE_DIR}/version.h.in"
  "${CMAKE_CURRENT_BINARY_DIR}/version.h"
  sprokit_version_major
  sprokit_version_minor
  sprokit_version_patch
  sprokit_version
  SPROKIT_BUILT_FROM_GIT
  sprokit_git_hash
  sprokit_git_hash_short
  sprokit_git_dirty)
# ...

You could just as easily fetch the date in your code. It’s more about the pattern of fetching data at build time and only updating things if they need to.

1 Like

The date should itself be generated:

If I’d decided to hard-code the date, and triggered an out of source update, eventually that updated date would have had to be hard-coded back in the source again, to be fetched by subsequent builds.
To be clear, this is the reasoning behind:

  • the library is being built as the main project? that probably means someone is working on it and making main modifications: update the build date
  • the library is not being built as the main project? that probably means it’s just being compiled to be used in another project, leave the build date as it is.

I’m beginning to suspect this is not how I should approach the problem, as I’m not seeing anyone doing or mentioning something like this. How is that done?

Trigger the difference based on CMAKE_PROJECT_NAME rather than a $Format:$ replacement.

1 Like

I’m sorry I don’t understand

Something like:

if (CMAKE_PROJECT_NAME STREQUAL "thislib")
  # get the date at build time
else ()
  set(staticdate "20200416")
endif ()
1 Like

The options I know are:

  1. update the set(staticdate "20200416") line manually every time (I don’t want to do that)
  2. make an exception of the out of source rule for version.h

edit: I think the point that didn’t get through is that I’d like to do that repeatedly, not just once. The updates have to be every time permanent to make sense.

edit 2: Your hints brought me to the conclusion that there’s no right way to do that, it’s an inherently “not out of source” thing to do. Like you were implying, I should focus on having the updated date available only in the built executable. After all, I’ve read somewhere that the date-in-the-source thing isn’t recommended anymore.

I’ll continue with the cmaking

Yes, that’s what the infrastructure for sprokit_configure_file does in the project I linked before does. It does configure_file at build time. The difference is that you need to tell it what variables to send from the configure step across. There’s also support for running code in that script to make variables through logic.

I’m back from the dead!
i hope

I finally got the project back in hand, and I started scraping from where I left off last time: the file dependency system doesn’t work across folders. I’ve taken another look at sprokit, but I couldn’t quite track what exactly does the trick.
Could it be that it does the configure_file both at configure time and build time? That would avoid CMake’s complains about missing sources at configure time, and that’s what I’m currently doing. But that’s very ugly to see.
(Still better than set_property(SOURCE "..." PROPERTY GENERATED 1), that’s a big no no for me)
Maybe there’s some optimization I could do to reduce code redundancy?

macro(generate_configure_script)
    file(WRITE "${configure_script_path}" "
        configure_file(
                \"${version_in_path}\"
                \"${version_configured_path}\"
                @ONLY
        )
        configure_file(
                \"${doxyfile_in_path}\"
                \"${doxyfile_configured_path}\"
                @ONLY
        )
    ")
endmacro()

configure_file( # there's no add_subdirectory() equivalent for single files :/
        "${version_in_path}"
        "${version_configured_path}"
        @ONLY
)
configure_file(
        "${doxyfile_in_path}"
        "${doxyfile_configured_path}"
        @ONLY
)

if("${CMAKE_PROJECT_NAME}" STREQUAL "VisualT")
    generate_configure_script()
    add_custom_target(configured_files ALL
                      COMMAND "${CMAKE_COMMAND}" -P "${configure_script_path}"
                      BYPRODUCTS "${version_configured_path}" "${doxyfile_configured_path}"
                      DEPENDS "${configure_script_path}" "${version_in_path}" "${doxyfile_in_path}"
                      COMMENT "configuring files")
else()
    add_custom_target(configured_files ALL) # empty :|
endif()

As you can see I’m using a custom target, not a custom command, as the custom command doesn’t work across folders and doesn’t work for multiple target dependencies. I find that the custom command has a pretty narrow use cases after all considered. (basically same-folder, one-target, automated file dependency resolution)

I improved the last solution to this:

# macros
macro(add_configurable_file in_path configured_path)
    file(APPEND "${configure_script_path}" "
        configure_file(
                \"${in_path}\"
                \"${configured_path}\"
                @ONLY
        )
    ")
endmacro()

# variables
set(configure_script_path "${VisualT_BINARY_DIR}/conf/configureScript.cmake")
string(TIMESTAMP VisualT_BUILD_DATE "%d %B %Y" UTC)

# register configurable files
file(REMOVE "${configure_script_path}")
add_configurable_file("${VisualT_SOURCE_DIR}/src/version.h.in" "${VisualT_BINARY_DIR}/src/version.h")
add_configurable_file("${VisualT_SOURCE_DIR}/doxyfile.in" "${VisualT_BINARY_DIR}/doxyfile")

# build-time configuration if main project
if("${CMAKE_PROJECT_NAME}" STREQUAL "VisualT")
    add_custom_target(configured_files ALL
                      COMMAND "${CMAKE_COMMAND}" -P "${configure_script_path}"
                      COMMENT "running configure script")
else()
    add_custom_target(configured_files ALL)
endif()

# configure-time configuration to avoid missing file warnings
execute_process(
        COMMAND "${CMAKE_COMMAND}" -P "${configure_script_path}")

It still has some ugly “code generation” but I currently don’t know a proper way to avoid that:
For example, I could create a configurator.cmake.in with only

configure_file(
                @configurable@
                @configured@)

And create a new configurator-xxx.cmake for every configurable file. Then run all of them as separate scripts. That would get rid of the “code-in-a-string”, but it would cost too much in performance

Edit 1

goddammit I didn’t know include() could include standard scripts too. Now I can replace

simply with

include("${configure_script_path}")

much better.

I had just finished basting the documentation generation (doxygen+sphinx+breathe), and I had begun removing all the unnecessary double quotes around variables I kept putting everywhere. (there’s a lot of confusion online on the matter, perhaps the documentation can be a bit misinterpreted, I had to test it by first hand). I wanted to proceed figuring out the installation and config-file packages, but I ran into this weird behaviour:

message("PRIVATE_HEADER=" ${private_headers} "\n" "PUBLIC_HEADER=" ${public_headers})
set_target_properties(visualt PROPERTIES
                      PRIVATE_HEADER ${private_headers}
                      PUBLIC_HEADER ${public_headers})
get_target_property(private_headers visualt PRIVATE_HEADER)
get_target_property(public_headers visualt PUBLIC_HEADER)

gives (long paths shortened with ...):

PRIVATE_HEADER=/mnt/d/.../VisualT/cmake-build-debug-wsl/src/version.h
PUBLIC_HEADER=/mnt/d/.../VisualT/include/visualt/visualt.h/mnt/d/.../VisualT/include/visualt/visualtUnprefixed.h
CMake Error at src/CMakeLists.txt:18 (set_target_properties):
  set_target_properties called with incorrect number of arguments.

But if I write instead

set_target_properties(visualt PROPERTIES
                      PRIVATE_HEADER ${private_headers}
                      PUBLIC_HEADER "${public_headers}") # this probably won't work

Alternatively, two set_property() work alright.

How is that?
From PUBLIC_HEADER's documentation: This property may be set to a list of header files, is set_target_properties() unable to handle lists? why?

The signature of set_target_properties is

set_target_properties(target1 target2 ...
                      PROPERTIES prop1 value1
                      prop2 value2 ...)

When I write

set(list_of_values "a" "b" "c" "d" "e")
set_target_properties(myTarget PROPERTIES CUSTOM_PROP ${list_of_values})

it gets interpreted as:

prop1 = CUSTOM_PROP
value1 = a
prop2 = b
value2 = c
prop3 = d
value3 = e

If I change that code to set(list_of_values "a" "b" "c" "d") (even number of elements), there is an argument passed for prop3, but not for value3 anymore, so CMake reports an error.

Now, if I write:

set(list_of_values "a" "b" "c" "d")
set_target_properties(myTarget PROPERTIES CUSTOM_PROP "${list_of_values}")

it gets interpreted as:

prop1 = CUSTOM_PROP
value1 = a;b;c;d

So you should quote both ${private_headers} and ${public_headers} when passing them to set_target_properties.

1 Like

image


So that’s because of how functions are designed, right? Not because lists get expanded (like a macro), and then they are interpreted as multiple arguments.
How am I supposed to write CMake code that doesn’t break if some paths have semicolons in them?
I mean, that’s a quite common character, I can’t just leave it out.
Is there a systematic way to escape them?
Else I’ll just write a escape semicolons() function, but I would have to use it for every variable reference that might contain paths. Aaah!