unable to submit tests to cdash.org

Usage
1

It may not be CDash problem but with my CMake configurations itself.

I’m unable to submit the tests to the cdash.org, for the command ctest --submit

I get the error: Although i have added tests through add_test

No tests were found!!!

CMakeLists.txt

option(NANOJSON_BUILD_TESTS "Build tests" ON) # tests
if (NANOJSON_BUILD_TESTS)
    enable_testing()
    include(CTest)
    set(CTEST_OUTPUT_ON_FAILURE ON)

    add_subdirectory(test) # 
    add_subdirectory(example)
endif ()

test/CMakeLists.txt

add_executable(test_object test_object.c)
target_link_libraries(test_object PRIVATE nanojsonc)
target_include_directories(test_object PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/../include)
add_test(NAME TestObject COMMAND test_object)

CTestConfig.cmake

set(CTEST_PROJECT_NAME nanoJSONc)
set(CTEST_NIGHTLY_START_TIME 01:00:00 UTC)

set(CTEST_DROP_SITE "my.cdash.org")
set(CTEST_DROP_LOCATION "/submit.php?project=projectname")

set(CTEST_DROP_SITE_CDASH TRUE)

If I run them from cmake-build-debug directory ctest

The following tests FAILED:
1 - TestObject (Not Run)

Please advise on what’s the right way to run and submit tests.

2

This looks fine to me. Did you build the project? The error at the end doesn’t align with the “No tests were found” error at the beginning but instead indicates that the tests were run before building.

One other (minor) thing I see is:

This is not read as a variable, but is expected to be an environment variable used at test time.

3

Thanks, Ben, for looking into this. I removed the unneeded variable.

I’m able to run tests in my IDE (CLion → All CTest Debug), and it can see all the tests run successfully.

  1. I cannot do so with the command line from the root directory.
    ctest --submit
    output: No tests were found!!!

  2. Therefore, I assume the results of my tests don’t get submitted to cdash.org

Maybe the problem is with my command?

4

I would suggest using a CTest script to perform CDash-related actions. There is some state that CTest keeps around for submissions that’s easier to track if it’s all done as part of a script. These are the relevant commands:

ctest_start()
ctest_configure()
ctest_build()
ctest_test()
ctest_submit()
5

I created a separate file CTestScript.cmake and executed it, here is the output.

/Users/user/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --debug-server-port 63212 --debug-token-path /private/var/folders/5v/9cyndhqn10d__qsf4pwsdbd80000gn/T/cmake-debugtoken -S "/Users/user/Library/Mobile Documents/com~apple~CloudDocs/Projects/project" -B "/Users/user/Library/Mobile Documents/com~apple~CloudDocs/Projects/project/cmake-build-debug" -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_MAKE_PROGRAM=/Users/user/Applications/CLion.app/Contents/bin/ninja/mac/ninja
-- Configuring done (0.1s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/user/Library/Mobile Documents/com~apple~CloudDocs/Projects/project/cmake-build-debug
Process terminated with exit code 0

research a bit and then on the command line, I tried

ctest --test-dir cmake-build-debug

it worked and ran all the tests, but then this didn’t work, I don’t see submissions on cdash.org

ctest --test-dir cmake-build-debug --submit

6

ctest runs scripts using -S. See the ctest_*.cmake scripts here.

7

ctest -S CTestScript.cmake --test-dir cmake-build-debug

Still no success with executing tests or submission to cdash.

8

Can you share the script?

9

Do you mean CTestScript? I used your code

ctest_start()
ctest_configure()
ctest_build()
ctest_test()
ctest_submit()

my other scripts are mentioned up in the question.

10

Oh, sorry, I should have been clearer that it was skeleton code. You’ll need to set up arguments to those calls based on their docs. CTest also wants things like CTEST_SOURCE_DIRECTORY and CTEST_BINARY_DIRECTORY set so that it knows what directories to look at.

11

I see. I am unfamiliar with that. Can I get a complete script?

Here’s the structure

- cmake-build-debug
- include
- src
- test

both src and test directories have their own CMakeLists.txt referenced in the root CMakeLists.txt

12

I linked to CMake’s CI scripts here. They are separate, but you can also perform the steps all together (after factoring the prep each does at the top of its file).