stderr mismatch error iresults in test failures while creating cmake 3.25.1 debian package in Ubuntu 20.04

Hi all,

I am trying to create cmake 3.25.1 debain package for ubuntu 20.04 as it is not available for ubuntu 20.04. I was able to install all dependencies except python3-sphinxcontrib.qthelp:native as it is not available for ubuntu 20.04. I hope it is fine to skip it as it is used only for cmake documentation and not required for cmake binary.(I have commented out dh_sphinxdoc -pcmake-doc in rules file in debian folder). I also downgraded Jinja2 to 3.0.0 from 3.1.2 due to below error:

ImportError: Error importing plugin "sphinx.testing.fixtures": cannot import name 'contextfunction' from 'jinja2' (/usr/local/lib/python3.8/dist-packages/jinja2/__init__.py)

While packaging CMake (dpkg-buildpackage -rfakeroot -uc -b), I see that lot of tests fail because stderr does not match that expected. I don’t see any mismatch. Only difference I see is the regular expression in the expected text.

eg:-

Expected stderr to match:

   expect-err> ^CMake Deprecation Warning at CMakeLists.txt:1 \(cmake_minimum_required\):
   expect-err>   Compatibility with CMake < 2.8.12 will be removed from a future version of
   expect-err>   CMake.
   expect-err> 
   expect-err>   Update the VERSION argument <min> value or use a ...<max> suffix to tell
   expect-err>   CMake that the project does not need compatibility with older versions.
   expect-err> +
   expect-err> CMake Warning \(dev\) in CMakeLists.txt:
   expect-err>   Policy CMP0019 is not set: Do not re-expand variables in include and link
   expect-err>   information.  Run "cmake --help-policy CMP0019" for policy details.  Use
   expect-err>   the cmake_policy command to set the policy and suppress this warning.
   expect-err> 
   expect-err>   The following variable evaluations were encountered:
   expect-err> 
   expect-err>   Evaluated directory INCLUDE_DIRECTORIES
   expect-err> 
   expect-err>     /usr/include/\${VAR_INCLUDE};/usr/include/normal
   expect-err> 
   expect-err>   as
   expect-err> 
   expect-err>     /usr/include/VAL_INCLUDE;/usr/include/normal
   expect-err> 
   expect-err>   Evaluated target some_target INCLUDE_DIRECTORIES
   expect-err> 
   expect-err>     /usr/include/\${VAR_INCLUDE};/usr/include/normal
   expect-err> 
   expect-err>   as
   expect-err> 
   expect-err>     /usr/include/VAL_INCLUDE;/usr/include/normal
   expect-err> 
   expect-err>   Evaluated link directories
   expect-err> 
   expect-err>     /usr/lib/\${VAR_LINK_DIRS};/usr/lib/normal
   expect-err> 
   expect-err>   as
   expect-err> 
   expect-err>     /usr/lib/VAL_LINK_DIRS;/usr/lib/normal
   expect-err> 
   expect-err>   Evaluated link library
   expect-err> 
   expect-err>     \${VAR_LINK_LIBS}
   expect-err> 
   expect-err>   as
   expect-err> 
   expect-err>     VAL_LINK_LIBS
   expect-err> 
   expect-err> This warning is for project developers.  Use -Wno-dev to suppress it.$

  Actual stderr:

   actual-err> CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
   actual-err>   Compatibility with CMake < 2.8.12 will be removed from a future version of
   actual-err>   CMake.
   actual-err> 
   actual-err>   Update the VERSION argument <min> value or use a ...<max> suffix to tell
   actual-err>   CMake that the project does not need compatibility with older versions.
   actual-err> 
   actual-err> 
   actual-err> CMake Warning (dev) in CMakeLists.txt:
   actual-err>   Policy CMP0019 is not set: Do not re-expand variables in include and link
   actual-err>   information.  Run "cmake --help-policy CMP0019" for policy details.  Use
   actual-err>   the cmake_policy command to set the policy and suppress this warning.
   actual-err> 

What can be the problem?

Thanks in advance!

What test is that failure from?

And yes, AFAIK, Sphinx is doc-only.

Cc: @brad.king

It also occurs to me that https://apt.kitware.com/ may have what you want already. (@kyle.edwards)

Thanks for your reply.
Here is the list of failed tests

75% tests passed, 172 tests failed out of 688

Label Time Summary:
CMake      = 712.89 sec*proc (283 tests)
CUDA       =  55.57 sec*proc (11 tests)
HIP        =  11.10 sec*proc (6 tests)
ISPC       =  36.29 sec*proc (6 tests)
Label1     =   0.05 sec*proc (1 test)
Label2     =   0.05 sec*proc (1 test)
Qt5        = 419.96 sec*proc (43 tests)
command    =   5.11 sec*proc (26 tests)
policy     =  48.16 sec*proc (40 tests)
run        = 707.78 sec*proc (257 tests)

Total Test time (real) = 1717.88 sec

The following tests FAILED:
        335 - RunCMake.CMP0019 (Failed)
        336 - RunCMake.CMP0022 (Failed)
        337 - RunCMake.CMP0026 (Failed)
        339 - RunCMake.CMP0037 (Failed)
        340 - RunCMake.CMP0038 (Failed)
        341 - RunCMake.CMP0039 (Failed)
        342 - RunCMake.CMP0040 (Failed)
        343 - RunCMake.CMP0041 (Failed)
        344 - RunCMake.CMP0043 (Failed)
        345 - RunCMake.CMP0045 (Failed)
        346 - RunCMake.CMP0046 (Failed)
        347 - RunCMake.CMP0049 (Failed)
        348 - RunCMake.CMP0050 (Failed)
        349 - RunCMake.CMP0051 (Failed)
        350 - RunCMake.CMP0053 (Failed)
        351 - RunCMake.CMP0054 (Failed)
        352 - RunCMake.CMP0055 (Failed)
        353 - RunCMake.CMP0057 (Failed)
        354 - RunCMake.CMP0059 (Failed)
        355 - RunCMake.CMP0060 (Failed)
        356 - RunCMake.CMP0064 (Failed)
        357 - RunCMake.CMP0069 (Failed)
        358 - RunCMake.CMP0081 (Failed)
        359 - RunCMake.CMP0102 (Failed)
        360 - RunCMake.CMP0106 (Failed)
        361 - RunCMake.CMP0111 (Failed)
        362 - RunCMake.CMP0115 (Failed)
        363 - RunCMake.CMP0118 (Failed)
        365 - RunCMake.CMP0121 (Failed)
        366 - RunCMake.CMP0125 (Failed)
        369 - RunCMake.CMP0135 (Failed)
        370 - RunCMake.CMP0139 (Failed)
        375 - RunCMake.ABI (Failed)
        377 - RunCMake.AutogenQt5 (Failed)
        379 - RunCMake.BuildDepends (Failed)
        381 - RunCMake.CMakeDependentOption (Failed)
        385 - RunCMake.CompilerChange (Failed)
        386 - RunCMake.CompilerNotFound (Failed)
        388 - RunCMake.Configure (Failed)
        389 - RunCMake.DisallowedCommands (Failed)
        393 - RunCMake.ExternalData (Failed)
        395 - RunCMake.FPHSA (Failed)
        397 - RunCMake.FindBoost (Failed)
        399 - RunCMake.FindOpenGL (Failed)
        404 - RunCMake.GenEx-LINK_LANGUAGE (Failed)
        405 - RunCMake.GenEx-LINK_LANG_AND_ID (Failed)
        408 - RunCMake.GenEx-LINK_LIBRARY (Failed)
        409 - RunCMake.GenEx-LINK_GROUP (Failed)
        410 - RunCMake.GenEx-TARGET_FILE (Failed)
        411 - RunCMake.GenEx-GENEX_EVAL (Failed)
        414 - RunCMake.GenEx-PATH_EQUAL (Failed)
        415 - RunCMake.GeneratorExpression (Failed)
        416 - RunCMake.GeneratorInstance (Failed)
        417 - RunCMake.GeneratorPlatform (Failed)
        418 - RunCMake.GeneratorToolset (Failed)
        419 - RunCMake.GetPrerequisites (Failed)
        420 - RunCMake.GNUInstallDirs (Failed)
        423 - RunCMake.TargetPropertyGeneratorExpressions (Failed)
        424 - RunCMake.Languages (Failed)
        425 - RunCMake.LinkItemValidation (Failed)
        427 - RunCMake.ObjectLibrary (Failed)
        430 - RunCMake.RuntimePath (Failed)
        432 - RunCMake.Swift (Failed)
        433 - RunCMake.TargetObjects (Failed)
        434 - RunCMake.TargetProperties (Failed)
        435 - RunCMake.ToolchainFile (Failed)
        437 - RunCMake.CompileDefinitions (Failed)
        439 - RunCMake.CompileFeatures (Failed)
        441 - RunCMake.PolicyScope (Failed)
        443 - RunCMake.WriteCompilerDetectionHeader (Failed)
        446 - RunCMake.VisibilityPreset (Failed)
        447 - RunCMake.CompatibleInterface (Failed)
        448 - RunCMake.Syntax (Failed)
        450 - RunCMake.MaxRecursionDepth (Failed)
        451 - RunCMake.add_custom_command (Failed)
        452 - RunCMake.add_custom_target (Failed)
        454 - RunCMake.add_executable (Failed)
        455 - RunCMake.add_library (Failed)
        456 - RunCMake.add_subdirectory (Failed)
        458 - RunCMake.build_command (Failed)
        459 - RunCMake.execute_process (Failed)
        460 - RunCMake.export (Failed)
        461 - RunCMake.cmake_host_system_information (Failed)
        462 - RunCMake.cmake_language (Failed)
        463 - RunCMake.cmake_minimum_required (Failed)
        464 - RunCMake.cmake_parse_arguments (Failed)
        480 - RunCMake.define_property (Failed)
        481 - RunCMake.file (Failed)
        482 - RunCMake.file-CHMOD (Failed)
        483 - RunCMake.file-RPATH (Failed)
        484 - RunCMake.find_file (Failed)
        485 - RunCMake.find_library (Failed)
        486 - RunCMake.find_package (Failed)
        487 - RunCMake.find_path (Failed)
        488 - RunCMake.find_program (Failed)
        489 - RunCMake.foreach (Failed)
        492 - RunCMake.get_filename_component (Failed)
        493 - RunCMake.get_property (Failed)
        494 - RunCMake.if (Failed)
        495 - RunCMake.include (Failed)
        496 - RunCMake.include_directories (Failed)
        498 - RunCMake.list (Failed)
        499 - RunCMake.load_cache (Failed)
        500 - RunCMake.math (Failed)
        501 - RunCMake.message (Failed)
        502 - RunCMake.option (Failed)
        504 - RunCMake.project (Failed)
        505 - RunCMake.project_injected (Failed)
        508 - RunCMake.separate_arguments (Failed)
        509 - RunCMake.set_property (Failed)
        510 - RunCMake.string (Failed)
        512 - RunCMake.BundleUtilities (Failed)
        513 - RunCMake.try_compile (Failed)
        514 - RunCMake.try_run (Failed)
        515 - RunCMake.set (Failed)
        516 - RunCMake.variable_watch (Failed)
        517 - RunCMake.while (Failed)
        520 - RunCMake.alias_targets (Failed)
        521 - RunCMake.InterfaceLibrary (Failed)
        523 - RunCMake.configure_file (Failed)
        527 - RunCMake.CXXModules (Failed)
        530 - RunCMake.File_Archive (Failed)
        531 - RunCMake.File_Configure (Failed)
        532 - RunCMake.File_Generate (Failed)
        533 - RunCMake.ExportWithoutLanguage (Failed)
        535 - RunCMake.target_link_libraries (Failed)
        539 - RunCMake.target_link_libraries-LINK_LIBRARY (Failed)
        540 - RunCMake.target_link_libraries-LINK_GROUP (Failed)
        544 - RunCMake.target_compile_features (Failed)
        545 - RunCMake.target_compile_options (Failed)
        547 - RunCMake.target_sources (Failed)
        549 - RunCMake.CheckSourceCompiles (Failed)
        550 - RunCMake.CheckSourceRuns (Failed)
        551 - RunCMake.CheckModules (Failed)
        552 - RunCMake.CheckIPOSupported (Failed)
        554 - RunCMake.CommandLine (Failed)
        556 - RunCMake.install (Failed)
        557 - RunCMake.file-GET_RUNTIME_DEPENDENCIES (Failed)
        559 - RunCMake.CPackConfig (Failed)
        561 - RunCMake.ExternalProject (Failed)
        564 - RunCMake.CTestCommandLine (Failed)
        567 - RunCMake.IfacePaths_INCDIRS (Failed)
        580 - RunCMake.CPack_DEB.CUSTOM_NAMES (Failed)
        581 - RunCMake.CPack_DEB.DEBUGINFO (Failed)
        582 - RunCMake.CPack_DEB.DEFAULT_PERMISSIONS (Failed)
        583 - RunCMake.CPack_DEB.DEPENDENCIES (Failed)
        584 - RunCMake.CPack_DEB.EMPTY_DIR (Failed)
        585 - RunCMake.CPack_DEB.VERSION (Failed)
        587 - RunCMake.CPack_DEB.GENERATE_SHLIBS (Failed)
        588 - RunCMake.CPack_DEB.GENERATE_SHLIBS_LDCONFIG (Failed)
        589 - RunCMake.CPack_DEB.LONG_FILENAMES (Failed)
        590 - RunCMake.CPack_DEB.MINIMAL (Failed)
        591 - RunCMake.CPack_DEB.PER_COMPONENT_FIELDS (Failed)
        592 - RunCMake.CPack_DEB.TIMESTAMPS (Failed)
        593 - RunCMake.CPack_DEB.MD5SUMS (Failed)
        594 - RunCMake.CPack_DEB.DEB_PACKAGE_VERSION_BACK_COMPATIBILITY (Failed)
        595 - RunCMake.CPack_DEB.DEB_DESCRIPTION (Failed)
        596 - RunCMake.CPack_DEB.PROJECT_META (Failed)
        597 - RunCMake.CPack_7Z (Failed)
        598 - RunCMake.CPack_TBZ2 (Failed)
        599 - RunCMake.CPack_TGZ (Failed)
        600 - RunCMake.CPack_TXZ (Failed)
        601 - RunCMake.CPack_TZ (Failed)
        602 - RunCMake.CPack_ZIP (Failed)
        603 - RunCMake.CPack_STGZ (Failed)
        604 - RunCMake.CPack_External (Failed)
        609 - RunCMake.UnityBuild (Failed)
        610 - RunCMake.CMakePresets (Failed)
        611 - RunCMake.CMakePresetsBuild (Failed)
        613 - RunCMake.CMakePresetsPackage (Failed)
        614 - RunCMake.CMakePresetsWorkflow (Failed)
        615 - RunCMake.VerifyHeaderSets (Failed)
Errors while running CTest
make[2]: *** [Makefile:94: test] Error 8

Good to now. Thanks a lot!

That seems quite odd. Do you mind uploading results of the test suite to CDash?