Custom language with single-step compilation, dyndep sources only

Usage
1

I’m trying to capture a toolchain, which as its CLI takes a folder, and processes all JSONs in it which match a glob expression. All the JSONs (which reference a few external files) are then compiled into a singular binary file.

I saw that unfortunately the messages of the build steps are fairly hardcoded, and I would want to avoid having “Linking MYLANG static library” et al. printed to the console, because it has nothing to do with linking anything. It’s a compilation step.

What I came up with now is abusing the OBJECT library mechanism, my target creates a stamp file at configuration time which is used as the of the target, but it does nothing really, other than serving as an anchor to a timestamp. Then a command comes and evaluates the GLOB on the folder to create the file and finally the actual compiler invocation. The only problems are that installing OBJECT libraries are problematic and controlling the location of their artifacts isn’t simple either.

Is there a better way to capture such a language and toolchain?

2

I managed to get everything working for Ninja, but am struggling with MSBuild to trigger the custom language build steps. To elaborate a bit further, this is the definition of the language:

CMakeDeterminePCCompiler.cmake

find_program(CMAKE_PC_COMPILER NAMES "pcc")
mark_as_advanced(CMAKE_PC_COMPILER)

set(CMAKE_PC_SOURCE_FILE_EXTENSIONS "")
set(CMAKE_PC_OUTPUT_EXTENSION .bin)
set(CMAKE_PC_COMPILER_ENV_VAR "PCC")

configure_file(
    "${CMAKE_CURRENT_LIST_DIR}/CMakePCCompiler.cmake.in"
    "${CMAKE_PLATFORM_INFO_DIR}/CMakePCCompiler.cmake"
)

CMakePCInformation.cmake

# Generate the binary
set(CMAKE_PC_COMPILE_OBJECT
    "${CMAKE_COMMAND} -D PATH_ARG:PATH=<INCLUDES> -D OUT_ARG:FILEPATH=<OBJECT> -D DEP_FILE:FILEPATH=<DEP_FILE> <FLAGS> -P ${SOME_HARDCODED_PATH_FOR_THE_MOMENT}\\PCCDepfileGenerator.cmake"
    "<CMAKE_PC_COMPILER> <FLAGS> --path <INCLUDES> --out <OBJECT>"
)

set(CMAKE_PC_INFORMATION_LOADED 1)

PCCDepfileGenerator.cmake

Uninteresting for the purpose of the question. Runs a file(GLOB) on the <INCLUDES> folder, searching for JSON files and writes them into <DEP_FILE> along with some extra files that the JSONs reference.

Then the user is supposed to use this language as such (along with vanilla C code that I’ll omit here):

CMakeLists.txt

cmake_minimum_required(VERSION 3.22...4.0)

project(TestPC LANGUAGES PC)

file(CONFIGURE
  OUTPUT ${PROJECT_NAME}
  CONTENT "stamp file use to trigger rebuilds of target ${PROJECT_NAME}"
)
set_source_files_properties("${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}" PROPERTIES
  LANGUAGE PC
)
add_library(${PROJECT_NAME} OBJECT "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}")
set_target_properties(${PROJECT_NAME} PROPERTIES
  LINKER_LANGUAGE PC
)

install(FILES $<TARGET_OBJECTS:${PROJECT_NAME}>
  DESTINATION "${CMAKE_INSTALL_DATADIR}"
)

Unfortunately MSBuild does nothing, when I invoke the TestPC target. The generated TestPC.vcxproj only has timestamp generation-related custom rules. I also tried defining CMAKE_PC_CREATE_SHARED_LIBRARY, CMAKE_PC_CREATE_SHARED_MODULE, CMAKE_PC_CREATE_STATIC_LIBRARY, CMAKE_PC_ARCHIVE_APPEND, CMAKE_PC_ARCHIVE_CREATE, CMAKE_PC_ARCHIVE_FINISH to the very same commands as above, because I saw that OBJECT libraries with MSVC ersult in .lib files, not an .obj file as one would assume, but still nothing happens. Can MSBuild execute custom languages at all?

3

AFAIK The generated Visual Studio projects let VS do the compilation. If you use that generator, the compiler and language needs to be supported by VS itself.

Why don’t you simply create an add_custom_command?

4

Ultimately I may just do that (and descend to custom commands). The thing that you lose with custom commands (AFAICT) is most of the generator expression machinery. You can’t change target properties “after the fact” you defined them, you can’t set compiler flags using familiar syntax like target_compile_options(), etc. It just all dissolves into custom logic and interfaces. (Plus, custom commands really mess up the output of MSBuild builds. They become super noisy and null-info with all the noise introduced by the custom rule trigger notices.)

5

You can use generator expressions in custom commands.
I usually create a corresponding custom target, where I store my properties.

6

How do you refer to the $<TARGET_FILE:tgt> of a custom target?

7

That’s not possible directly, because a custom target doesn’t have a corresponding file on disk, so TARGET_FILE doesn’t make sense for it in the general case. However, if your custom targets have something like a “final output file,” you can easily create your own property for this and set it on the custom target. Something like this:

add_custom_target(MyTarget ALL ...)
set_property(TARGET MyTarget PROPERTY PC_TARGET_FILE ${CMAKE_CURRENT_BINARY_DIR}/whatever)

The above can of course be wrapped in a suitable function like add_pc_target(...).

You can then reference such a property in generator expressions normally:

$<TARGET_PROPERTY:MyTarget,PC_TARGET_FILE>
1 Like
8

@Angew How do you make this work in multi-config scenarios? If I have my custom_target expose a property like you said, its content is going to have $ inside, but then when I try to use it, the $ part won’t get expanded, because it’s not a multi-level expression.

target_compile_definitions(${PROJECT_NAME}
    PRIVATE $<BUILD_INTERFACE:PC_BINARY="$<PATH:RELATIVE_PATH,$<TARGET_PROPERTY:pc_target,PC_TARGET_FILE>,$<TARGET_FILE_DIR:myapp>>">
)

In the executable expands to "../$<CONFIG>/pipeline_cache.bin". If I, in a loop create config-specific properties to forego having a nested generator expression, then it just simply evaluates to an empty string. (I guess the property name itself can’t be a mixture of literal and genexpr evaluation.) And $<GENEX_EVAL> don’t seem to help in this case.

(@jtxa This is what I had in mind when I said that one loses many of the generator expression facilities when going the custom command way. E.g. having a single generator expression that expands properly in single- and multi-config scenarios.)

9

There are reasons against using a language:

  • Compilers and languages are not meant to be defined by users, they are part of CMake. The API / needed variables are not public.
  • You want to use a native VS project, but the language and compiler does not seem to be supported by VS.
  • You don’t seem to have a source file in the first place. At least not in a similar way to any other programming language.

I guess one of the reasons for having GenEx support on the OUTPUT of add_custom_command (since v3.20) is to support things like $<CONFIG>. So I would expect that everything related works.
Putting the output file name to a property, and accessing it like this should work: $<GENEX_EVAL:$<TARGET_PROPERTY:pc_target,PC_TARGET_FILE>>.