add_test() assumes test executable is first string after COMMAND

chriskot870
Fri, 15 Aug 2025 05:03:20 +0000
Code

I am using cmake through vscode to cross-compile from a Ubuntu host to a Raspberry Pi Zero 2W target. I need to run the tests on the remote host. I created the following add_test:

add_test(
  NAME UnitsPressureTests
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
  COMMAND ${CMAKE_SOURCE_DIR}/cmake/run_remote_test.sh units_pressure_test
  --gtest_output=json:units_pressure_test_results.json
  --gtest_filter=UnitsPressureTest.*)

Since I need to run the test on a remote host the COMMAND is ${CMAKE_SOURCE_DIR}/cmake/run_remote_test.sh and it simply copies the test executable to the remote and then runs it on the remote:

#!/bin/bash
REMOTE_HOST="chrisk@qwtest.local"
REMOTE_PATH="/usr/local/qw/tests"
LOCAL_BINARY="$1" # TestExecutable
shift
# Transfer the binary
scp "${LOCAL_BINARY}" "${REMOTE_HOST}:${REMOTE_PATH}/"
# Execute remotely and capture result
ssh "${REMOTE_HOST}" "cd ${REMOTE_PATH} && ./${LOCAL_BINARY} $@"
exit $?

When I run a test it works fine. The script is called and the test gets copied to the remote target and it gets executed. Everything passes and all is well.
In order to debug the test I need to create a configuration in launch.json that will copy the test to the remote host and start the gdbserver on the remote host. The launcher also needs to know where the test executable is in order to load the symbol table. The program field in the launch.json configuration defines this.
In vscode I have a launch.json configuration to debug the tests as below:

{
            "name": "Debug Tests",
            "cwd": "${cmake.testWorkingDirectory}",
            "request": "launch",
            "program": "${cmake.testProgram}",
            "args": ["${cmake.testArguments}"],
            "type": "cppdbg",
            "MIMode": "gdb",
            "miDebuggerPath": "/usr/bin/gdb-multiarch",
            "miDebuggerArgs": "",
            "miDebuggerServerAddress": "192.168.50.6:4711",
            "targetArchitecture": "arm64",
            "stopAtEntry": false,
            "preLaunchTask": "StartUnitTestGDBServer",
            "postDebugTask": "StopUnitTestGDBServer"
        }

The ${cmake.testProgram} is the suggested value in order to allow this one configuration to work for many different tests. When I try to debug the test from vscode’s Test explorer I get the following error:

GDB failed with message: "/home/chrisk/Projects/RaspberryPi/WS/cmake/run_remote_test.sh":not in executable format:file format not recognized.

Note that ${cmake.testProgram} resolves to run_remote_test.sh. This is the first string in the add_test() COMMAND property. It is the script I have to run to copy the executable over and run on the remote host when running the test.
I really want that value to be the second string units_pressure_test. That is the real test executable. But I can’t call it as the first string of the COMMAND because it needs to run on the remote host, not locally.

I have not been able to find a way to specify the program field in launch.json in a generic way that will work for all the tests.

Maybe there should be a way of separating the executable from the COMMAND field so they can be defined independently. Maybe add an EXECUTABLE option to add_test that overrides the rule of using the first string of COMMAND:

add_test(
  NAME UnitsPressureTests
  EXECUTABLE {CMAKE_CURRENT_BINARY_DIR}/units_pressure_test
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
  COMMAND ${CMAKE_SOURCE_DIR}/cmake/run_remote_test.sh units_pressure_test
  --gtest_output=json:units_pressure_test_results.json
  --gtest_filter=UnitsPressureTest.*)

The EXECUTABLE value can define a ${cmake.testExecutable} that can be used in place of ${cmake.testProgram} as the launch.json program field when the first string after COMMAND is not the correct thing to use.

To verify things I made the units_pressure_test the first string after COMMAND:

add_test(
  NAME UnitsTemperatureTests
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
  #COMMAND ${CMAKE_SOURCE_DIR}/cmake/run_remote_test.sh units_pressure_test
  COMMAND ${CMAKE_CURRENT_BINARY_DIR}/units_pressure_test
  --gtest_output=json:units_pressure_test_results.json
  --gtest_filter=UnitsPressureTest.*)

I was then able to run the debugger and it correctly copied the file to the remote host, started up gdbserver on the rtemote host and ran gdb to connect to it just fine. i was able to debug just fine. But, of course I can’t actually run the test because it will try to execute units_pressure_test locally which will fail because it is for the wrong architecture.

Here are all the software versions I am using
I am using cmake version:

$ cmake --version
cmake version 3.28.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

I am using it within Vscode. That about output shows:

Version: 1.103.1
Commit: 360a4e4fd251bfce169a4ddf857c7d25d1ad40da
Date: 2025-08-12T16:25:40.542Z
Electron: 37.2.3
ElectronBuildId: 12035395
Chromium: 138.0.7204.100
Node.js: 22.17.0
V8: 13.8.500258-electron.0
OS: Linux x64 6.8.0-71-generic snap

I am running on a Linux host with the following os-release output

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.3 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo

6 posts - 3 participants