# Generating linker command line arguments to a file

**URL:** https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117
**Category:** Code
**Created:** [April 10, 2021, 9:32pm UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117 "2021-04-10T21:32:51Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![amp](https://discourse.cmake.org/user_avatar/discourse.cmake.org/amp/32/1357_2.png) [@amp](https://discourse.cmake.org/u/amp)
#### Post date: [April 10, 2021, 9:32pm UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117/1 "2021-04-10T21:32:51Z")

</div>

I need to run a non-cmake sub-build (setuptools based in this case) in a cmake project and pass the linker and compiler command lines to that sub-build. My current implementation writes the command line arguments to a file using lots of generator expressions; the sub-build reads this file in. It is at [katana/KatanaPythonSetupSubdirectory.cmake at 4f418f0aeab539c05fd296f3b28c5b7616dc747f · KatanaGraph/katana · GitHub](https://github.com/KatanaGraph/katana/blob/4f418f0aeab539c05fd296f3b28c5b7616dc747f/cmake/Modules/KatanaPythonSetupSubdirectory.cmake#L97-L122). However, it fails when the new(-ish) cmake because the `HOST_LINK` generator expression is used and my code doesn’t handle the `LINKER:` and `SHELL:` magic supported by linker options. I encountered both because cmake 3.20 seems to use those features in FindMPI.cmake and generates them internally, so if the project imports MPI on a machine with CUDA my cmake generation failes with an error: `$<HOST_LINK:...> may only be used with binary targets to specify link options.`

Questions:

- How do I generate a file in a context that correctly handles `$<HOST_LINK:...>`? I have tried using the `TARGET` option with a dummy executable target with `file(GENERATE...)` but it doesn’t seem to make any difference. (probably because the generator expressions are not processed in the special “linker option” context.)
- Is there a generator expression or other technique that will decode the `LINKER:` and `SHELL:` magic, so I don’t have to do it on the python side?

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [April 12, 2021, 11:51am UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117/2 "2021-04-12T11:51:10Z")

</div>

The `LINKER:` and `SHELL:` expansions depend on the shell (e.g., cmd vs sh) and the linker wrapper used (e.g., icc vs gcc). Without knowing what is used, they’ll have to be decoded manually. I don’t know if there’s a way around the `$<HOST_LINK>` thing either.

For ParaView, [this tool](https://gitlab.kitware.com/paraview/paraview/-/blob/master/Clients/CommandLineExecutables/paraview-config.in) is used to extract the link and compilation flags needed to consume ParaView by generating a project that does what is wanted then diffing the command lines with and without the ParaView targets of interest. Something like this is probably the most robust way because CMake generates the flags as it would then they are extracted out regardless of how they end up getting generated.

---

<div class="post-metadata">

### Author: ![amp](https://discourse.cmake.org/user_avatar/discourse.cmake.org/amp/32/1357_2.png) [@amp](https://discourse.cmake.org/u/amp)
#### Post date: [April 13, 2021, 1:32pm UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117/3 "2021-04-13T13:32:40Z")

</div>

Thanks for the information. I’m bummed there is no ability to extract this information cleanly.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [April 13, 2021, 3:30pm UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117/4 "2021-04-13T15:30:50Z")

</div>

Making an interface for it is probably very hard to pin down. The caller needs to provide the desired shell and linker context in which the flag is to be used. Right now, the only way to do that is have CMake generate a project and then interrogate the build system what commands it has. `compile_commands.json` might work for compile command extraction, but that spec doesn’t have space for linking commands.

---

<div class="post-metadata">

### Author: ![amp](https://discourse.cmake.org/user_avatar/discourse.cmake.org/amp/32/1357_2.png) [@amp](https://discourse.cmake.org/u/amp)
#### Post date: [April 13, 2021, 4:35pm UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117/5 "2021-04-13T16:35:38Z")

</div>

> [@ben.boeckel](#):
>
> `compile_commands.json` might work for compile command extraction, but that spec doesn’t have space for linking commands.

A `link_commands.json` file would solve my problem. I could create a dummy target that doesn’t actually get built at the CMake level (`EXCLUDE_FROM_ALL`), but is configured with all the correct dependencies and options. Then I could grab the options from `compile_commands.json` and `link_commands.json` based on the dummy source file and taget/output file names.

That said, my ideal would be a set of generator expressions (which expand to cmake lists, to avoid space safety issues, which appear to be a minor problem with `compile_commands.json`):

- `$<TARGET_COMPILER_COMMAND:target,source_file>`: the compiler command with no options, e.g., `gcc` or `ccache;gcc`
- `$<TARGET_COMPILER_OPTIONS:target,source_file>`: the command-line arguments to the compiler, including `-D`, `-o`, and input file names.
- `$<TARGET_LINKER_COMMAND:target>`: the linker command with no options, e.g., `gcc` or `gcc;-shared`.
- `$<TARGET_LINKER_OPTIONS:target>`: the command line arguments to the linker.

I would love to split the `_OPTIONS` expressions into `FLAGS`, `INPUTS`, and `OUTPUTS`, but I suspect that may not be possible in all cases since flags and inputs/outputs may need to be interspersed because argument order matters.

EDIT: My assumption is that the context you mention (shell and linker) can be extracted from the main generator in use and the target specified in the generator expressions. This information must be available at generation time. Clearly it’s not possible to compute this stuff at main script execution time in the cmake architecture.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [April 13, 2021, 4:45pm UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117/6 "2021-04-13T16:45:44Z")

</div>

Please file [an issue](https://gitlab.kitware.com/cmake/cmake/-/issues) with these new proposed genexes.

A `link_commands.json` has been requested before, but we’re just following [Clang’s specification](https://clang.llvm.org/docs/JSONCompilationDatabase.html). Rather than making something up from scratch (at least that isn’t CMake-specific such as `CMakePresets.json`), I’d like to see other stakeholders’ input (interested IDE developers, other build systems/tools) on any format updates/changes for such things.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [April 14, 2021, 1:51pm UTC](https://discourse.cmake.org/t/generating-linker-command-line-arguments-to-a-file/3117/7 "2021-04-14T13:51:07Z")

</div>

The issue has been filed [here](https://gitlab.kitware.com/cmake/cmake/-/issues/22057).