CPackRPM Installing Unnecessary Files

Hi,

I’ve been trying to understand the following issue for the last several hours, but now it’s time to call in the cavalry… :thinking:

In the project that I’m working on at the moment (atlas / atlasexternals · GitLab) we build a bunch “external projects” that our “main project” would eventually depend on. The way we do this is a bit complicated, because for historical reasons we needed to ensure that in our build directory we would produce binaries with the same layout that they would eventually have after being installed. (Long story…)

When using ExternalProject_Add, the logic we/I came up with is that:

  • The ExternalProject_Add(...) command installs the particular “external” into its own dedicated directory inside of the build directory;
  • From there I copy everything into the “common directory” of the build directory as part of the build;
  • Using install(DIRECTORY …) the contents of the “custom directory” are installed into “.”.

See for instance: External/CLHEP/CMakeLists.txt · master · atlas / atlasexternals · GitLab

Note that for the “second step” of the previous description we use a custom “buildinstall” step like: External/CLHEP/CMakeLists.txt · master · atlas / atlasexternals · GitLab

To complicate things even further, we also override the default behaviour of CPack normally, but that’s a long story of its own. The thing is, with that “overridden behaviour” we’ve been using CPack happily for years for producing RPM and TGZ packages from our project(s). But (again because of a complicated set of reasons) now I’m trying to make our build work with a “more default” setup of CPack. (The latest code changes that I’m working on right now, are in: CPack Cleanup, master branch (2023.06.02.) (!1015) · Merge requests · atlas / atlasexternals · GitLab) And I bumped into a behaviour that I can’t manage to understand. :frowning_face:

When generating an RPM package, I end up with a directory structure like the following:

[bash][pcadp04]:AthSimulationExternals > tree -L 4 _CPack_Packages/Linux/RPM/AthSimulationExternals_24.0.6_x86_64-centos
9-gcc11-opt/
_CPack_Packages/Linux/RPM/AthSimulationExternals_24.0.6_x86_64-centos9-gcc11-opt/
├── mnt
│   └── hdd1
│       └── krasznaa
│           └── projects
└── usr
    └── AthSimulationExternals
        └── 24.0.6
            └── InstallArea

8 directories, 0 files
[bash][pcadp04]:AthSimulationExternals >

While I get a much more reasonable layout while I generate a TGZ package.

[bash][pcadp04]:AthSimulationExternals > tree -L 4 _CPack_Packages/Linux/TGZ/AthSimulationExternals_24.0.6_x86_64-centos9-gcc11-opt
_CPack_Packages/Linux/TGZ/AthSimulationExternals_24.0.6_x86_64-centos9-gcc11-opt
└── usr
    └── AthSimulationExternals
        └── 24.0.6
            └── InstallArea

4 directories, 0 files
[bash][pcadp04]:AthSimulationExternals >

The “unnecessary directories” that I get with the RPM generation are copies of the “separate installation directories” that the “buildinstall” steps and the install(DIRECTORY ...) calls use as sources. And I can just not figure out why this behaviour is happening. :frowning_face:

Note that I only see this for the externals where the “buildinstall machinery” is at play. For Makefile based projects like External/Gdb/CMakeLists.txt · master · atlas / atlasexternals · GitLab, where this behaviour is achieved simply by defining a multi-step installation command, I don’t get any weird behaviour with CPackRPM. :confused: So it’s somehow the custom step in ExternalProject_Add(...) that makes CPackRPM’s installation step behave in a weird way. But I can just not figure out why… :sob:

I know that our code is a bit too complicated to look at without a lot more context. I’m mainly just hoping that somebody may have an idea why custom steps with ExternalProject_Add(...) may lead to a weird behaviour with CPackRPM… :thinking:

Cheers,
Attila

I think tracing the execution of cmake_install.cmake would be the best way to get more information here. There’s some code that is installing what you’re looking to exclude…finding it may help point what needs to change to fully exclude it.

Hi Ben,

Let’s take the following package: External/CLHEP/CMakeLists.txt · master · atlas / atlasexternals · GitLab

If I put CPack into trace mode with cpack -G RPM --trace, I find the following output about this package in it:

/srv/build_athgenext/cmake_install.cmake(151):  if(NOT CMAKE_INSTALL_LOCAL_ONLY )
/srv/build_athgenext/cmake_install.cmake(153):  include(/srv/build_athgenext/External/CLHEP/cmake_install.cmake )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(4):  if(NOT DEFINED CMAKE_INSTALL_PREFIX )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(7):  string(REGEX REPLACE /$  CMAKE_INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX} )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(10):  if(NOT DEFINED CMAKE_INSTALL_CONFIG_NAME )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(21):  if(NOT CMAKE_INSTALL_COMPONENT )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(22):  if(COMPONENT )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(25):  else()
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(26):  set(CMAKE_INSTALL_COMPONENT )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(31):  if(NOT DEFINED CMAKE_INSTALL_SO_NO_EXE )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(36):  if(NOT DEFINED CMAKE_CROSSCOMPILING )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(41):  if(NOT DEFINED CMAKE_OBJDUMP )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(45):  if(CMAKE_INSTALL_COMPONENT STREQUAL Main OR NOT CMAKE_INSTALL_COMPONENT )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(46):  file(INSTALL DESTINATION ${CMAKE_INSTALL_PREFIX}/src/External/CLHEP TYPE DIRECTORY FILES /srv/atlasexternals/External/CLHEP/ USE_SOURCE_PERMISSIONS REGEX /\\.svn$ EXCLUDE REGEX /\\.git$ EXCLUDE REGEX /[^/]*\\~$ EXCLUDE )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(49):  if(CMAKE_INSTALL_COMPONENT STREQUAL Main OR NOT CMAKE_INSTALL_COMPONENT )
/srv/build_athgenext/External/CLHEP/cmake_install.cmake(50):  file(INSTALL DESTINATION ${CMAKE_INSTALL_PREFIX}/. TYPE DIRECTORY OPTIONAL FILES /srv/build_athgenext/External/CLHEP/CMakeFiles/CLHEPBuild/ USE_SOURCE_PERMISSIONS )

After this the next time I see it mentioned is in the warnings about files being in non-relocatable directories during the RPM building. :thinking:

Since I was building the RPM on a spinning disk right now, I noticed that all the “unnecessary” files get installed “at the following stage”:

CPack: Create package using RPM
CPack: Install projects
CPack: - Run preinstall target for: AthGenerationExternals

But I don’t know what code I should look at to see why these files are picked up as “preinstall targets”. :frowning_face: Since the cmake_install.cmake script for the package in question for instance is not all too suspicious in my mind.

Singularity> more External/CLHEP/cmake_install.cmake 
# Install script for directory: /srv/atlasexternals/External/CLHEP

# Set the install prefix
if(NOT DEFINED CMAKE_INSTALL_PREFIX)
  set(CMAKE_INSTALL_PREFIX "/AthGenerationExternals/22.0.0/InstallArea/x86_64-centos7-gcc11-opt")
endif()
string(REGEX REPLACE "/$" "" CMAKE_INSTALL_PREFIX "${CMAKE_INSTALL_PREFIX}")

# Set the install configuration name.
if(NOT DEFINED CMAKE_INSTALL_CONFIG_NAME)
  if(BUILD_TYPE)
    string(REGEX REPLACE "^[^A-Za-z0-9_]+" ""
           CMAKE_INSTALL_CONFIG_NAME "${BUILD_TYPE}")
  else()
    set(CMAKE_INSTALL_CONFIG_NAME "Release")
  endif()
  message(STATUS "Install configuration: \"${CMAKE_INSTALL_CONFIG_NAME}\"")
endif()

# Set the component getting installed.
if(NOT CMAKE_INSTALL_COMPONENT)
  if(COMPONENT)
    message(STATUS "Install component: \"${COMPONENT}\"")
    set(CMAKE_INSTALL_COMPONENT "${COMPONENT}")
  else()
    set(CMAKE_INSTALL_COMPONENT)
  endif()
endif()

# Install shared libraries without execute permission?
if(NOT DEFINED CMAKE_INSTALL_SO_NO_EXE)
  set(CMAKE_INSTALL_SO_NO_EXE "0")
endif()

# Is this installation the result of a crosscompile?
if(NOT DEFINED CMAKE_CROSSCOMPILING)
  set(CMAKE_CROSSCOMPILING "FALSE")
endif()

# Set default install directory permissions.
if(NOT DEFINED CMAKE_OBJDUMP)
  set(CMAKE_OBJDUMP "/cvmfs/sft.cern.ch/lcg/releases/binutils/2.37-355ed/x86_64-centos7/bin/objdump")
endif()

if(CMAKE_INSTALL_COMPONENT STREQUAL "Main" OR NOT CMAKE_INSTALL_COMPONENT)
  file(INSTALL DESTINATION "${CMAKE_INSTALL_PREFIX}/src/External/CLHEP" TYPE DIRECTORY FILES "/srv/atlasexternals/External/CLHEP/" 
USE_SOURCE_PERMISSIONS REGEX "/\\.svn$" EXCLUDE REGEX "/\\.git$" EXCLUDE REGEX "/[^/]*\\~$" EXCLUDE)
endif()

if(CMAKE_INSTALL_COMPONENT STREQUAL "Main" OR NOT CMAKE_INSTALL_COMPONENT)
  file(INSTALL DESTINATION "${CMAKE_INSTALL_PREFIX}/." TYPE DIRECTORY OPTIONAL FILES "/srv/build_athgenext/External/CLHEP/CMakeFile
s/CLHEPBuild/" USE_SOURCE_PERMISSIONS)
endif()

Singularity>

(As a reminder, “bad” files get installed into directories like _CPack_Packages/Linux/RPM/AthGenerationExternals_22.0.0_x86_64-centos7-gcc11-opt/srv/build_athgenext/External/CLHEP/.)

As before, any help/idea is very welcome.

Cheers,
Attila

It looks like the build part is picking up on some CPack environment and being taken along on a ride (probably DESTDIR). I don’t know if there’s a way to tell CPack “don’t run the build” off-hand (I don’t see any obvious variables at least).

Argh… I set out to create a standalone reproducer for the issue, and managed to identify the thing triggering this unwanted behaviour…

I can reproduce this behaviour with the following code:

# Set up the project.
cmake_minimum_required(VERSION 3.10)
project(CPackWithExternalProject VERSION 1.0.0 LANGUAGES C)

# CMake include(s).
include(CPack)
include(ExternalProject)

# Build GoogleTest as an external package.
ExternalProject_Add(GoogleTest
   INSTALL_DIR "${CMAKE_BINARY_DIR}/gtest"
   URL "https://github.com/google/googletest/archive/refs/tags/release-1.11.0.tar.gz"
   URL_MD5 "e8a8df240b6938bb6384155d4c37d937"
   CMAKE_CACHE_ARGS
   -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
   -DBUILD_GTEST:BOOL=ON
   -DBUILD_GMOCK:BOOL=ON
   -DBUILD_SHARED_LIBS:BOOL=ON)
ExternalProject_Add_Step(GoogleTest forceconfigure
   COMMAND ${CMAKE_COMMAND} -E remove -f "<BINARY_DIR>/CMakeCache.txt"
   COMMENT "Forcing the configuration of GoogleTest"
   DEPENDEES update
   DEPENDERS configure
   ALWAYS 1)

Since our externals depend on each other in some non-trivial ways, in order to make incremental builds sort-of-kind-of functional, I defined these sort of forceconfigure steps for the builds.

This does not actually interfere with the TGZ generator. The example shown above generates an empty TGZ file just fine. :stuck_out_tongue:

Singularity> cpack -G TGZ
CPack: Create package using TGZ
CPack: Install projects
CPack: - Run preinstall target for: CPackWithExternalProject
CPack: - Install project: CPackWithExternalProject []
CPack: Create package
CPack: - package: /srv/build/CPackWithExternalProject-1.0.0-Linux.tar.gz generated.
Singularity> ls -l CPackWithExternalProject-1.0.0-Linux.tar.gz
-rw-rw-r-- 1 krasznaa krasznaa 29 Jun 20 17:27 CPackWithExternalProject-1.0.0-Linux.tar.gz
Singularity>

But when generating an RPM file, all hell breaks lose.

Singularity> cpack -G RPM
CPack: Create package using RPM
CPack: Install projects
CPack: - Run preinstall target for: CPackWithExternalProject
CPack: - Install project: CPackWithExternalProject []
CPack: Create package
CMake Warning (dev) at /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:188 (message):
  CPackRPM:Warning: Path /srv/build/gtest/include/gmock/gmock-actions.h is
  not on one of the relocatable paths! Package will be partially relocatable.
Call Stack (most recent call first):
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1056 (cpack_rpm_prepare_relocation_paths)
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1968 (cpack_rpm_generate_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:188 (message):
  CPackRPM:Warning: Path /srv/build/gtest/include/gmock/gmock-cardinalities.h
  is not on one of the relocatable paths! Package will be partially
  relocatable.
Call Stack (most recent call first):
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1056 (cpack_rpm_prepare_relocation_paths)
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1968 (cpack_rpm_generate_package)
This warning is for project developers.  Use -Wno-dev to suppress it.
...
CMake Warning (dev) at /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:188 (message):
  CPackRPM:Warning: Path /srv/build/gtest/lib64/pkgconfig/gtest_main.pc is
  not on one of the relocatable paths! Package will be partially relocatable.
Call Stack (most recent call first):
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1056 (cpack_rpm_prepare_relocation_paths)
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1968 (cpack_rpm_generate_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

CPackRPM: Will use GENERATED spec file: /srv/build/_CPack_Packages/Linux/RPM/SPECS/cpackwithexternalproject.spec
CPack: - package: /srv/build/CPackWithExternalProject-1.0.0-Linux.rpm generated.
Singularity> ls -l CPackWithExternalProject-1.0.0-Linux.rpm
-rw-rw-r-- 1 krasznaa krasznaa 535552 Jun 20 17:28 CPackWithExternalProject-1.0.0-Linux.rpm
Singularity>

:thinking: Is this forceconfigure step really so evil? Or is this rather a legitimate bug in CPackRPM? :thinking:

I would see if something in GoogleTest cares about DESTDIR (or something else that CPack sets up during an installation) during configure for some reason. If it does and it is from its code, I’d go look at fixing it in GoogleTest. If it’s some cmake.git code, we can look at how to fix it.

:confused: It’s definitely not something specific to GoogleTest. I’ve seen the same behaviour with every CMake based project that we build using ExternalProject_Add.

I believe the issue is that since a build is always triggered by this setup, one is started by the preinstall step of CPack. My educated guess is that DESTDIR is already set as an environment variable when the preinstall happens with CPackRPM. But it’s not (at least not in this step…) when a TGZ is generated.

I think this is the thing causing my problems: https://gitlab.kitware.com/cmake/cmake/-/blob/master/Source/CPack/cmCPackRPMGenerator.cxx#L26-28

I guess that’s just a bit too early for “forcing on” the DESTDIR usage. :thinking:

To simplify things further, let’s take the following, really basic setup:

# Set up the project.
cmake_minimum_required(VERSION 3.10)
project(CPackWithExternalProject VERSION 1.0.0 LANGUAGES C)

# CMake include(s).
include(CPack)
include(ExternalProject)

# Build GoogleTest as an external package.
ExternalProject_Add(Eigen
   INSTALL_DIR "${CMAKE_BINARY_DIR}/eigen"
   URL "https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.tar.gz"
   URL_MD5 "9e30f67e8531477de4117506fe44669b"
   CMAKE_CACHE_ARGS
   -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
   -DBUILD_TESTING:BOOL=FALSE
   -DEIGEN_BUILD_DOC:BOOL=FALSE
   -DEIGEN_TEST_NOQT:BOOL=TRUE)

(This time with Eigen, just for some variety.)

If I build it “normally”, things work as they should.

Singularity> cmake ../source/
-- The C compiler identification is GNU 11.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin/gcc - skipped
...
Singularity> make
[ 12%] Creating directories for 'Eigen'
[ 25%] Performing download step (download, verify and extract) for 'Eigen'
-- Downloading...
   dst='/srv/build/Eigen-prefix/src/eigen-3.3.7.tar.gz'
   timeout='none'
...
[100%] Completed 'Eigen'
[100%] Built target Eigen
Singularity> cpack -G RPM
CPack: Create package using RPM
CPack: Install projects
CPack: - Run preinstall target for: CPackWithExternalProject
CPack: - Install project: CPackWithExternalProject []
CPack: Create package
CPackRPM: Will use GENERATED spec file: /srv/build/_CPack_Packages/Linux/RPM/SPECS/cpackwithexternalproject.spec
CPack: - package: /srv/build/CPackWithExternalProject-1.0.0-Linux.rpm generated.
Singularity> ls -l CPackWithExternalProject-1.0.0-Linux.rpm
-rw-rw-r-- 1 krasznaa krasznaa 1812 Jun 20 20:19 CPackWithExternalProject-1.0.0-Linux.rpm
Singularity>

But if I execute cpack without first building the project, I get this:

Singularity> cmake ../source/
-- The C compiler identification is GNU 11.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin/gcc - skipped
...
-- Configuring done
-- Generating done
-- Build files have been written to: /srv/build
Singularity> cpack -G RPM
CPack: Create package using RPM
CPack: Install projects
CPack: - Run preinstall target for: CPackWithExternalProject
CPack: - Install project: CPackWithExternalProject []
CPack: Create package
CMake Warning (dev) at /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:188 (message):
  CPackRPM:Warning: Path /srv/build/eigen/include/eigen3/Eigen/Cholesky is
  not on one of the relocatable paths! Package will be partially relocatable.
Call Stack (most recent call first):
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1056 (cpack_rpm_prepare_relocation_paths)
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1968 (cpack_rpm_generate_package)
This warning is for project developers.  Use -Wno-dev to suppress it.
...
CMake Warning (dev) at /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:188 (message):
  CPackRPM:Warning: Path /srv/build/eigen/share/pkgconfig/eigen3.pc is not on
  one of the relocatable paths! Package will be partially relocatable.
Call Stack (most recent call first):
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1056 (cpack_rpm_prepare_relocation_paths)
  /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/share/cmake-3.24/Modules/Internal/CPack/CPackRPM.cmake:1968 (cpack_rpm_generate_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

CPackRPM: Will use GENERATED spec file: /srv/build/_CPack_Packages/Linux/RPM/SPECS/cpackwithexternalproject.spec
CPack: - package: /srv/build/CPackWithExternalProject-1.0.0-Linux.rpm generated.
Singularity> ls -l CPackWithExternalProject-1.0.0-Linux.rpm
-rw-rw-r-- 1 krasznaa krasznaa 1014476 Jun 20 20:22 CPackWithExternalProject-1.0.0-Linux.rpm
Singularity>

This is not by design… right? :thinking: