No Qt tests done for windows external test ci phase

Hi,

I was checking why neither gitlab CI nor nightly cdash is running Qt AUTOGEN tests for the t:windows-vs2022-x64 / nightly-cmake-windows_vs2022_x64

https://gitlab.kitware.com/cmake/cmake/-/jobs/8645764
https://open.cdash.org/build/8965503

It appears to be due to Qt missing in path when configuring tests for those jobs.

Specifically the qt env is not sourced for .cmake_test_windows_external: as it is done for regular tests.
https://gitlab.kitware.com/cmake/cmake/-/blob/master/.gitlab/os-windows.yml#L372

The same is not the case for macos for example, which includes qt for external tests with xcode as well.

https://gitlab.kitware.com/cmake/cmake/-/blob/master/.gitlab/os-macos.yml#L167
https://gitlab.kitware.com/cmake/cmake/-/blob/master/.gitlab/os-macos.yml#L199
https://gitlab.kitware.com/cmake/cmake/-/jobs/8645763

Is there a reason for that? What’s the distinction between external and non external tests?

The “external” CI jobs configure the CMake test suite to run without actually building CMake, but instead point at an externally-provided CMake, which comes from an earlier job in the pipeline. The name comes from the CMake_TEST_EXTERNAL_CMAKE option, though nowadays we use CMake_TEST_HOST_CMAKE for this in the CI jobs, which configures the test suite to run the tests using the cmake that was itself used to configure the build tree.

The external test infrastructure on Windows uses a minimal initial environment because it is used for driving all kinds of special compiler environments. We do load .gitlab/ci/env.ps1 to prepare per-job environments. The same should probably be done on macOS, but there currently are not enough job variants to bother.

The t:windows-vs2022-x64 job currently leaves out Qt testing. I’ll look at adding it so that we can get CI coverage of autogen with the VS generator.

1 Like

See CMake MR 8771

1 Like

I think the name comes from VTK’s usage of it for doing builds against install trees of VTK itself. That was likely inspired by the Testing/External subdirectory VTK uses for its own “external” testing. Eventually VTK would like to contract test other projects (e.g., ParaView) against its own install tree; these would also use the external name I expect.

1 Like