We run the VTK build on one machine but would like to run the tests on different machines.
First attempt is to do an out-of-source build, zip the build directory, copy to the test machine, and run ctest from within that build directory. Unfortunately we have discovered that somewhere (cmake cache files?) the full path to all the test executables is stored, and it is not possible to replicate that tree structure on the test machine. All tests fail.
We looked for ctest options to override the build directory, but failed to find any.
We got it to work by using a custom script to first rewrite the paths inside *.cmake, *.tcl, and *.recipe files.
Option A: Create a package on the build machine and install that package on the test machine, then run your tests against that.
Option B: Build all your executables and shared libraries with the install RPATH, and base those on relative locations using $ORIGIN. This assumes the relative directory structure of the binaries in your build tree matches your installed structure, which frequently isn’t the case.
I guess I don’t understand what you mean by “create a package” for option A. In one sense, we created a package using “zip”
We elected to do a static build; so the executables run fine on the test machine – it’s not an rpath problem. Rather, it is the “ctest” runner that is failing to find the executable files – because it is searching using the build-machine path, which is different from the test-machine path.
Example output :
00:05:38.615 554/1569 Test #554: VTK::FiltersPointsCxx-UnitTestKernels ........................................................***Not Run 0.00 sec
00:05:38.615 Start 555: VTK::FiltersPointsCxx-TestSPHKernels
00:05:38.615 Could not find executable C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/vtkFiltersPointsCxxTests.exe
00:05:38.615 Looked in the following places:
00:05:38.615 C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/vtkFiltersPointsCxxTests.exe
00:05:38.615 C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/vtkFiltersPointsCxxTests.exe.exe
00:05:38.615 C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/Release/vtkFiltersPointsCxxTests.exe
00:05:38.615 C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/Release/vtkFiltersPointsCxxTests.exe.exe
00:05:38.615 Release/C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/vtkFiltersPointsCxxTests.exe
00:05:38.615 Release/C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/vtkFiltersPointsCxxTests.exe.exe
00:05:38.616 Unable to find executable: C:/Jenkins/workspace/COTS/vtk/bld/bin/Release/vtkFiltersPointsCxxTests.exe
The “C:/Jenkins/workspace/…” path is the build server path. On the test machine, we unpack the build folder somewhere on drive D. Was hoping there is some way to tell ctest where its build folder is.
Ah, I see. You’re copy the build tree, including the generated files that ctest uses. That’s not going to work. Those files will potentially contain some absolute paths that only make sense on the build machine, as you’ve observed. You need to generate those ctest files on the machine on which you’re going to run it. That essentially means you need to re-run the CMake configure step on the test machine. That might or might not be possible in your case though, depending on whether your test machine has the same environment and capabilities as the build machine. You’d have to discard the CMakeCache.txt file too, since CMake doesn’t allow you to change the source or build directory of an existing build without discarding the cache.
Interesting. I hadn’t considered rebuilding the cache files. We have included the full cmake distribution in the zip file that is transported to the test machine. However, the test machine has no compilers, for example. I don’t know what ctest would make of that.
Alternatively: is there any way to tell cmake/ctest at build time to use relative paths in the generated files?
There have been various back-and-forth discussions and changes relating to absolute versus relative paths in the build project files (Makefiles, build.ninja, etc.). I don’t know what the current situation is, but my read is basically don’t assume you have one or the other. I don’t know if the same sort of back-and-forth applies to the files written out for ctest, but I suspect it is in a similar situation. Basically, these are internal implementation details and you shouldn’t depend on those things.
If you really must go down this path, I guess a recursive search-and-replace for the build machine paths to transform them to the test machine paths could get you what you want. You’d have to know all the files that are involved. Maybe start with the various CTest*.cmake files and see how far that gets you. If your project makes use of any other files that may contain paths, you’d have to include those in your search-and-replace.
We do this for VTK’s CI. We just force the machines to all agree on a build path for a given build configuration so that the absolute paths continue to work.
However, VTK also has a mechanism to build the test suite against an already-built VTK (build or install tree). I’d recommend an install tree just so that all build paths can be punted. You can use the Testing/External subdirectory of VTK to run its test suite in this way.
And finally in ${PROJECT_SOURCE_DIR}/test/CTestTestfile.cmake:
file(GLOB _testfiles CTestTestfile.*.cmake)
foreach(_tf IN LISTS _testfiles)
include(${_tf})
endforeach()
So I’m basically writing CTest files by myself, create a separate install component for the tests, and that’s it. Maybe not the most elegant solution but works in my case. All you need on the test machine is “XXXSERVER_BIN” environment variable (but you could write it differently, in my case this was needed anyway)
