Attempting to write a CMake function that does testing with an interactive prompt.

Usage
1 
# https://stackoverflow.com/a/54675911/19504610
FUNCTION(command_line_test test_executable input_arguments expected_output
         expected_error)
    EXECUTE_PROCESS(
        COMMAND
            "echo \"${input_arguments}\" | ${CMAKE_BINARY_DIR}/${test_executable} 1>&1 2>&1"
        RESULT_VARIABLE result
        OUTPUT_VARIABLE output
        ERROR_VARIABLE err_output COMMAND_ECHO STDOUT)

    # Find if ${expected_output} is in ${output}; if cannot be found, found =
    # "-1"
    MESSAGE(NOTICE output=${output};expected_output=${expected_output})
    STRING(FIND ${output} ${expected_output} found)
    MESSAGE(found ${found})
    IF((NOT (${result} STREQUAL "0")) OR (${found} STREQUAL "-1"))
        MESSAGE(
            FATAL_ERROR
                "Test failed with return value '${result}'; output '${output}'; expected output '${expected_output}' and found '${found}'";
        )
    ENDIF()
ENDFUNCTION()

COMMAND_LINE_TEST(gcd "5 20" "GCD of 5 and 20 is 5" "")

So what it does is it attempts to see if the output of piping arguments "5 20" into <path-to-executable>" has within it "GCD of 5 and 20 is 5".

The COMMAND "echo \"${input_arguments}\" | ${CMAKE_BINARY_DIR}/${test_executable} 1>&1 2>&1" becomes echo "5 20" | /root/c_projects/c_template_two/build/release/test/gcd 1>&1 2>&1 and when called on a separate shell, the output is Enter two integers: GCD of 5 and 20 is 5.

However, when running the test,

The line MESSAGE(NOTICE output=${output};expected_output=${expected_output}), prints output=expected_output=GCD of 5 and 20 is 5'echo "5 20" | /root/c_projects/c_template_two/build/release/test/gcd 1>&1 2>&1'.

This means that the output to STDOUT was not captured into the output variable.

Any best practices to fixing this?

2

From: https://cmake.org/cmake/help/latest/command/execute_process.html :

Options:

COMMAND

A child process command line.

CMake executes the child process using operating system APIs directly:

  • On POSIX platforms, the command line is passed to the child process in an argv[] style array.
  • On Windows platforms, the command line is encoded as a string such that child processes using CommandLineToArgvW will decode the original arguments.

No intermediate shell is used, so shell operators such as > are treated as normal arguments. (Use the INPUT_*, OUTPUT_*, and ERROR_* options to redirect stdin, stdout, and stderr.)

For sequential execution of multiple commands use multiple execute_process calls each with a single COMMAND argument.

3

Not sure what you are implying as I have read the documentation which you linked a couple of times when attempting to debug my cmake codes.

Anyway, I managed to get a working solution, not perfect but working.

4

What I’m implying is that your COMMAND parameter is wrong, and you should use the INPUT_FILE or (although I find it ugly) call the shell directly.
Most probably the execute_process() reports execution errors back, but you don’t check those.

1 Like
5

Thank you for your prompt reply.

In the end, I guess I went for your second approach which is to call the shell directly, specifically bash - assuming it is in the build system.

You’re right - I should have also checked if there was any output piped to STDERR - could have been a much faster debugging I guess. :joy:

1 Like
6

Now I have another problem.

Suppose I have this ${ROOT_DIR}/cmake/tests.cmake file which supposes to only declare two functions, shown below:

In ${ROOT_DIR}/cmake/tests.cmake

# https://stackoverflow.com/a/54675911/19504610
# https://stackoverflow.com/questions/70539001/how-to-ctest-an-interactive-command-line-utility
# https://discourse.cmake.org/t/attempting-to-write-a-cmake-function-that-does-testing-with-an-interactive-prompt/11190

FUNCTION(interactive_command_line_should_pass test_executable input_arguments
      expected_output)
    EXECUTE_PROCESS(
        COMMAND
            bash -c
            "echo \"${input_arguments}\" | ${CMAKE_BINARY_DIR}/${test_executable}"
        RESULT_VARIABLE result
        OUTPUT_VARIABLE output)

    # Find if ${expected_output} is in ${output}; if cannot be found, found =
    # "-1"
    STRING(FIND ${output} ${expected_output} found)
    IF((NOT (${result} STREQUAL "0")) OR (${found} STREQUAL "-1"))
        MESSAGE(
            FATAL_ERROR
                "Test failed with return value '${result}'; output '${output}'; expected output '${expected_output}' and found '${found}'";
        )
    ENDIF()
ENDFUNCTION()

FUNCTION(interactive_command_line_should_fail test_executable input_arguments
      expected_error_output)

    EXECUTE_PROCESS(
        COMMAND
            bash -c
            "echo \"${input_arguments}\" | ${CTEST_BINARY_DIR}/${test_executable}"
        RESULT_VARIABLE result
        ERROR_VARIABLE output COMMAND_ECHO STDERR)

    # Find if ${expected_output} is in ${output}; if cannot be found, found =
    # "-1"
    MESSAGE(
        NOTICE
        "output=${output}; expected_error_output=${expected_error_output}; found=${found}"
    )
    STRING(FIND ${output} ${expected_error_output} found)
    IF((NOT (${result} STREQUAL "1")) OR (${found} STREQUAL "-1"))
        MESSAGE(
            FATAL_ERROR
                "Test failed with return value '${result}'; output '${output}'; expected_error_output '${expected_error_output}' and found '${found}'";
        )
    ENDIF()
ENDFUNCTION()

Now in ${ROOT_DIR}/test/CMakeList.txt, I attempt to call the functions like so:

INCLUDE(${CMAKE_SOURCE_DIR}/cmake/tests.cmake)

INTERACTIVE_COMMAND_LINE_SHOULD_PASS(gcd "5 20" "GCD of 5 and 20 is 5" "")
INTERACTIVE_COMMAND_LINE_SHOULD_FAIL(gcd "51 20" "GCD of 51 and 20 is 1" "")

The problem is that the moment INCLUDE(${CMAKE_SOURCE_DIR}/cmake/tests.cmake) is called, the function named STRING inside the two functions named INTERACTIVE_COMMAND_LINE_SHOULD_PASS and INTERACTIVE_COMMAND_LINE_SHOULD_FAIL is executed immediately, causing an error.

The question is: how do I ‘import’ a function without ‘executing’ whatever that is within?

7

The import itself does not execute the body of any function.
What does the error message say?

8

I apologize for being confused.

It didn’t execute the function during the INCLUDE(...) line was executed. It was actually executed after when I called it myself with INTERACTIVE_COMMAND_LINE_SHOULD_PASS(gcd "5 20" "GCD of 5 and 20 is 5" "").