It may not be CDash problem but with my CMake configurations itself.
I’m unable to submit the tests to the cdash.org , for the command ctest --submit
I get the error: Although i have added tests through add_test
No tests were found!!!
CMakeLists.txt
option(NANOJSON_BUILD_TESTS "Build tests" ON) # tests
if (NANOJSON_BUILD_TESTS)
enable_testing()
include(CTest)
set(CTEST_OUTPUT_ON_FAILURE ON)
add_subdirectory(test) #
add_subdirectory(example)
endif ()
test/CMakeLists.txt
add_executable(test_object test_object.c)
target_link_libraries(test_object PRIVATE nanojsonc)
target_include_directories(test_object PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/../include)
add_test(NAME TestObject COMMAND test_object)
CTestConfig.cmake
set(CTEST_PROJECT_NAME nanoJSONc)
set(CTEST_NIGHTLY_START_TIME 01:00:00 UTC)
set(CTEST_DROP_SITE "my.cdash.org")
set(CTEST_DROP_LOCATION "/submit.php?project=projectname")
set(CTEST_DROP_SITE_CDASH TRUE)
If I run them from cmake-build-debug
directory ctest
The following tests FAILED:
1 - TestObject (Not Run)
Please advise on what’s the right way to run and submit tests.
ben.boeckel
(Ben Boeckel (Kitware))
November 19, 2023, 2:36am
2
This looks fine to me. Did you build the project? The error at the end doesn’t align with the “No tests were found” error at the beginning but instead indicates that the tests were run before building.
One other (minor) thing I see is:
This is not read as a variable, but is expected to be an environment variable used at test time.
saadmuizz
(Saad)
November 20, 2023, 10:29am
3
Thanks, Ben, for looking into this. I removed the unneeded variable.
I’m able to run tests in my IDE (CLion → All CTest Debug), and it can see all the tests run successfully.
I cannot do so with the command line from the root directory.
ctest --submit
output: No tests were found!!!
Therefore, I assume the results of my tests don’t get submitted to cdash.org
Maybe the problem is with my command?
ben.boeckel
(Ben Boeckel (Kitware))
November 20, 2023, 12:25pm
4
I would suggest using a CTest script to perform CDash-related actions. There is some state that CTest keeps around for submissions that’s easier to track if it’s all done as part of a script. These are the relevant commands:
ctest_start()
ctest_configure()
ctest_build()
ctest_test()
ctest_submit()
I created a separate file CTestScript.cmake and executed it, here is the output.
/Users/user/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --debug-server-port 63212 --debug-token-path /private/var/folders/5v/9cyndhqn10d__qsf4pwsdbd80000gn/T/cmake-debugtoken -S "/Users/user/Library/Mobile Documents/com~apple~CloudDocs/Projects/project" -B "/Users/user/Library/Mobile Documents/com~apple~CloudDocs/Projects/project/cmake-build-debug" -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_MAKE_PROGRAM=/Users/user/Applications/CLion.app/Contents/bin/ninja/mac/ninja
-- Configuring done (0.1s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/user/Library/Mobile Documents/com~apple~CloudDocs/Projects/project/cmake-build-debug
Process terminated with exit code 0
research a bit and then on the command line, I tried
ctest --test-dir cmake-build-debug
it worked and ran all the tests, but then this didn’t work, I don’t see submissions on cdash.org
ctest --test-dir cmake-build-debug --submit
ben.boeckel
(Ben Boeckel (Kitware))
November 20, 2023, 8:14pm
6
ctest
runs scripts using -S
. See the ctest_*.cmake
scripts here .
ctest -S CTestScript.cmake --test-dir cmake-build-debug
Still no success with executing tests or submission to cdash.
ben.boeckel
(Ben Boeckel (Kitware))
November 21, 2023, 1:16pm
8
Can you share the script?
Do you mean CTestScript? I used your code
ctest_start()
ctest_configure()
ctest_build()
ctest_test()
ctest_submit()
my other scripts are mentioned up in the question.
ben.boeckel
(Ben Boeckel (Kitware))
November 21, 2023, 1:34pm
10
Oh, sorry, I should have been clearer that it was skeleton code. You’ll need to set up arguments to those calls based on their docs. CTest also wants things like CTEST_SOURCE_DIRECTORY
and CTEST_BINARY_DIRECTORY
set so that it knows what directories to look at.
saadmuizz
(Saad)
November 21, 2023, 1:52pm
11
I see. I am unfamiliar with that. Can I get a complete script?
Here’s the structure
- cmake-build-debug
- include
- src
- test
both src and test directories have their own CMakeLists.txt referenced in the root CMakeLists.txt
ben.boeckel
(Ben Boeckel (Kitware))
November 22, 2023, 2:21pm
12
I linked to CMake’s CI scripts here . They are separate, but you can also perform the steps all together (after factoring the prep each does at the top of its file).