add_custom_command, problem with quotes and spaces

Code
,
1

If I build up the COMMAND for add_custom_command in a variable, and that variable’s value has spaces in it (as it might if the string combines args like “x y”, or “-o outfile.txt”), then the content of that variable gets quotes (") put around it where it appears in the generated build.ninja. The problem is this results in an ill-formed actual command, because there’s already a separate quotation started with “cd…”:

cmd.exe /C “cd /D C:\source\antlr\use-antlr4-vcpkg\cmake-build-dbg-mgw\test_rig && “x y””

If I type the same literal text in under COMMAND in add_custom_command, it doesn’t get extra quotes. This second behavior is the desired one, but I also wouldn’t have expected a difference. I was able to replicate the first condition by typing “x y” instead of x y. If I have x y without the quotes, I get:

cmd.exe /C “cd /D C:\source\antlr\use-antlr4-vcpkg\cmake-build-dbg-mgw\test_rig && x y”

I think I understand that the difference in using variables to build the command versus putting the raw text of the command after COMMAND must have to do with how arguments are expanded in CMake. I guess to prevent a single arg from getting splatted across multiple CMake function args just because it happens to have a space in it, it gets quotes added.

However it seems like quite the footgun in this case. You can build up a string(s) for the command in stages using variables, but the moment a space is involved it breaks. Is it a bug that this results in ill-formed code in build.ninja? If this is intended behavior, there should be a big fat warning in the CMake add_custom_command documentation. Also would be handy if CMake could warn about this.

How even would you properly escape the inner quotes of an arg unless cmd.exe supports quote escape sequences within the argument following /C? I’m thinking of the case of what if you had to pass a command a path with spaces in it as a single arg. It seems like this implementation breaks building from a path with spaces in it.

Maybe the bug is the generated build.ninja SHOULDN’T be quoting the whole command after the /C in the first place. As I understand it, cmd.exe /C foo bar should work without outer quotes around “foo bar” so the default form of the command is including spurious quotes on the outside of the “cd …”.

My implied question in posting this is, am I overlooking something obvious or am I correct in these conclusions? I find it hard to believe what must be as basic a feature as add_custom_command could be completely broken (at least on Windows) if your command path includes spaces and contain such a footgun as the fact you can’t build your whole COMMAND string in a variable (must be passed as separate args) yet this isn’t warned about in the documentation.

2

I don’t know if it may resolve your problem, but maybe you should read this :

1 Like
3

add_custom_command() command expects a list for its arguments, so to get what you expect, you have to build a list, not a string:

set(MY_CMD x)
list(APPEND MY_CMD y)

add_custom_command(OUTPUT ...
                                       COMMAND ${MY_CMD})

And you have an argument with spaces, use quotes during insertion in the list:

set(MY_CMD x)
list(APPEND MY_CMD y)
list(APPEND MY_CMD "path with space")

add_custom_command(OUTPUT ...
                                       COMMAND ${MY_CMD})
1 Like
4

Marc’s observation that the argument following COMMAND is a list solves the actual problem I had. I just need to build up my command variable as a list with semicolons instead of a string with spaces.

The Crascit article Oodini shared explains why it works. “Whitespace argument separators are found before variable evaluation. Semicolon argument separators are found after variable evaluation.”

I didn’t have luck with Marc’s second example appending a “path with spaces” to a list. I’ll detail that in a separate reply.

5

A path with spaces doesn’t work correctly on Windows whether it’s in a list or delivered to add_custom_command in any other form. The problem remains that there’s (I believe, superfluous) quotes already around the entire command in build.ninja, so any inner quotes accidentally end the outer quotation instead of nesting.

I don’t think it’s really solvable unless CMake stops generating extraneous quotes around the commands in build.ninja, or unless cmd.exe supported some additional escape sequences for quotes different from both Ninja and the Windows command line.

If cmd.exe (or a custom program - hmmm) supported a special syntax such as, “U+xxxx”, like this:

# Note the lack of ' " ' literals, and where literal space is and isn't 
set(MY_CMD U+0022path with spaceU+0022)
# Results in list (3):  U+0022path;with;spaceU+0022

That is using an escape sequence that can survive CMake, Ninja, and the Windows command terminal without confusing anything in between, and then get converted by cmd.exe (or some custom cmd.exe replacement) into literal quotes when the process is executed. That’s what would be required to make this idiot-proof, IMO.

(“U+xxxx” is not a good choice for this syntax; it’s sort of the dumbest thing that could possibly work. I just picked it to make it clear it’s something that won’t trigger special behavior at any of the in-between layers. In contrast “"e;” won’t work at all for this because & gets interpreted as something special by the shell.)

It would look like this in build.ninja:

  COMMAND = cmd.exe /C "cd /D C:\source\antlr\use-antlr4-vcpkg\cmake-build-dbg-mgw\test_rig && U+0022path with spaceU+0022"

Then, as I said, either cmd.exe or a custom executable would have to turn the U+0022 back into " when it exec’s the args.

P.S. I was playing around just now with cmd.exe /C in a Windows Command Prompt, and actually it’s quite robust with respect to re-entrant double quotes. For example, this works:

C:\Users\dferr>cmd.exe /C "echo hello > "a file.txt" && type "a file.txt""
hello

C:\Users\dferr>type "a file.txt"
hello

While I still maintain CMake should try not generating the extra quotes, I’m now wondering if this is actually a bug in Ninja, that it doesn’t handle the re-entrant quotation when it parses “COMMAND = …” out of build.ninja, and not a limitation of the Windows shell itself. I can’t yet rule out the succesful example above isn’t actually some special magic the Window Command Prompt window itself is doing with my user-typed text that doesn’t happen when a process is launched programmatically…