Static analyzers as seperate targets

Leon0402 · September 12, 2021, 11:33am

Hi,

I currently use clang-tidy, cppcheck and incldue-what-you-use cmake integrations, but there are a few problems:

It’s somewhat annoying when just prototyping because of all the output
It’s slow as hell when recompiling
It will need a recompilation of everything to produce the errors. So building twice will not reprint previous warnings from the tool. A clean is needed (which is especially annoying as I use FetchContent and don’t need all deps to be rebuilt again)

I thought about it and actually I would much more prefer separate targets for static analyzers. So once I’m happy with my code, everything compiles, I can run static analyzers.
This would be especially nice with CMakePresets:

        {
            "name": "analyse",
            "inherits": "build-ninja-clang-debug",
            "configurePreset": "default",
            "targets": [
                "clang-tidy"
            ]
        }

The problem is that the tools need a list of source files or I believe, in case of include what you use, even a complete compilation.
The built in variables like CMAKE_CXX_CPPCHECK make this handy, as they feed the source files to the tools during build.

So what I currently do is controlling CMAKE_CXX_CPPCHECK and others with variables. And usually I have them OFF, but then make them ON. Which is better, but still not nice:

It recompiles everything including dependencies
It still not produces the same output on multiple runs → Once it is compiled, warnings will obviously not show up again
It might be uncessary for things like clang-tidy and cppcheck to actually compile. I think these work without it

Any ideas how I could do it better? As I said having some targets, which run the tools on all source files and header files (and just them, not dependencies) and always produce the same warnings no matter how often you run the tool without changing anything would be ideal.

marc.chevrier · September 12, 2021, 1:57pm

You can do different CMake configurations in different build directories.

Leon0402 · September 12, 2021, 2:01pm

I don’t quite though see how that fixes my issue.

The output of analyzers will still show only for the files I recompile. So in order to get output I usually have to clean and then everything will be recompiled.

marc.chevrier · September 12, 2021, 2:14pm

Anyway, because compilation is required, having a specific target will not solve the problem.

Having different build directories enable you to work in an optimum environment (i.e. no useless recompilation, speed) for the development and a dedicated environment for the various checks. For these environments, a clean + compilation make sense for full check…

Leon0402 · September 12, 2021, 2:48pm

How would I solve the problem with the dependencies? I think this would really be the missing piece for this to work.
I use FetchContent a lot and always building all dependencies is not good.

craig.scott · September 12, 2021, 11:07pm

It would be nice to have a separate target that only executed the rules for static analysis (where this is a separate tool invocation from invoking the compiler for real). For comparison, Qt has this for its QML CMake API. For each QML module (aka target), you also get a ${target}_qmllint target, and a global all_qmllint target which depends on all the individual ${target}_qmllint targets.

Regarding how to do a partial clean, what I tend to do is delete the part of the build tree that corresponds to the targets I want to rebuild. I then re-run CMake which ensures everything can build again and the next build will build just the bits I removed instead of everything. This relies on having your build structured well, but hopefully if you’re using FetchContent to bring in your dependencies, this is already likely to be the case (since your dependencies will be in their own separate directories automatically).

Leon0402 · October 3, 2021, 12:41pm

@craig.scott I made a WIP implementation of individual targets for cppcheck, clang-tidy and include-what-you-use:

option(ENABLE_CPPCHECK "Enable static analysis with cppcheck" OFF)
option(ENABLE_CLANG_TIDY "Enable static analysis with clang-tidy" OFF)
option(ENABLE_INCLUDE_WHAT_YOU_USE "Enable static analysis with include-what-you-use" OFF)

function(add_target_static_analyers target)
  get_target_property(TARGET_SOURCES ${target} SOURCES)
  # Include only cpp files
  list(FILTER TARGET_SOURCES INCLUDE REGEX .*\.cpp)

  if(ENABLE_INCLUDE_WHAT_YOU_USE)
    find_program(INCLUDE_WHAT_YOU_USE include-what-you-use REQUIRED)

    add_custom_target(${target}_iwyu
      COMMAND ${CMAKE_SOURCE_DIR}/tools/iwyu_tool.py
        -p ${CMAKE_SOURCE_DIR}/compile_commands.json
        ${TARGET_SOURCES}
        -- 
        -Xiwyu --quoted_includes_first
        -Xiwyu --cxx17ns
        -Xiwyu --no_fwd_decls
        -Xiwyu --mapping_file=${CMAKE_SOURCE_DIR}/iwyu_mapping.imp
      WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
      USES_TERMINAL
    )
  endif()

  if(ENABLE_CPPCHECK)
    find_program(CPPCHECK cppcheck REQUIRED)

    # TO DO: Analyzes all files from compilation database rather than just the TARGET_SOURCES
    add_custom_target(${target}_cppcheck 
      COMMAND ${CPPCHECK}
        ${TARGET_SOURCES}
        --inline-suppr
        --enable=warning,style,information,missingInclude
        --project=${CMAKE_SOURCE_DIR}/compile_commands.json
        -i${CMAKE_BINARY_DIR}/_deps/
      USES_TERMINAL
    )
  endif()

  if(ENABLE_CLANG_TIDY)
    find_program(CLANGTIDY clang-tidy REQUIRED)

    get_target_property(TARGET_SOURCES ${target} SOURCES)

    add_custom_target(${target}_clangtidy 
      COMMAND ${CMAKE_SOURCE_DIR}/tools/run-clang-tidy.py 
        ${TARGET_SOURCES} 
        -header-filter=.* 
      WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
      USES_TERMINAL
    )
  endif()
endfunction()

Some Notes:

For all tools a compilation database is needed, this can be exported via cmake. In the example it’s assumed that the compilation database is in the root folder (${CMAKE_SOURCE_DIRECTORY}).

I also used some python helper scripts from the official tools repos, I put them under the original name in ${CMAKE_SOURCE_DIRECTORY}/tools/<helper_file_name>

For clang tidy, I took the python script from here:

github.com

llvm/llvm-project/blob/main/clang-tools-extra/clang-tidy/tool/run-clang-tidy.py

#!/usr/bin/env python
#
#===- run-clang-tidy.py - Parallel clang-tidy runner --------*- python -*--===#
#
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
#
#===-----------------------------------------------------------------------===#
# FIXME: Integrate with clang-tidy-diff.py


"""
Parallel clang-tidy runner
==========================

Runs clang-tidy over all files in a compilation database. Requires clang-tidy
and clang-apply-replacements in $PATH.

Example invocations.

This file has been truncated. show original

=> Needed so clang tidy is actually efficient. This will run clang-tidy in parallel and such stuff. It also works without it, but clang-tidy is then slow as hell.

For include what you use, I took the python script from here:

github.com

include-what-you-use/include-what-you-use/blob/master/iwyu_tool.py

#!/usr/bin/env python

##===--- iwyu_tool.py -----------------------------------------------------===##
#
#                     The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
#
##===----------------------------------------------------------------------===##

""" Driver to consume a Clang compilation database and invoke IWYU.

Example usage with CMake:

  # Unix systems
  $ mkdir build && cd build
  $ CC="clang" CXX="clang++" cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ...
  $ iwyu_tool.py -p .

This file has been truncated. show original

=> Needed to use the “manual” mode with compilation database rather than the cmake integration

Some pain points currently:

Cppcheck doesn’t yet work fully. It processes all files from the compilation database rather than only the files of the target. I couldn’t yet figure out the command line parameter
The additional scripts → Need python and probably I should execute them in a more cross plattform way
Getting the source files. Currently I use the target property, so this function needs to be called after all target_sources calls have been made!! I also tried it with generator expression → Then you can call the function at any time, which is much better, but the generator property SOURCES contains also sources from target linked libraries, which is bad
They probably need some also some DEPENDS and such stuff. Need to do more testing here.

Also one question I have: Can I create a target for these scenerarios:

Executes all three analyzers for one target
Execute one analyzer on all targets (-> So for all add_target_static_analyers calls)
Execute all analyzers on all targets

If you have any suggestion on how to imrove this (expecially regarding the pain points) I would be happy to hear them.

The current script is even with the pain points quite helpful already and more convenient for me than the partial build.

I would also be interested if something like this could be baked into CMake directly perhaps?

craig.scott · October 3, 2021, 8:56pm

Take a look at the $<FILTER:...> generator expression, available since CMake 3.15. That’s probably the piece you were missing.

Leon0402 · October 4, 2021, 7:54am

Hmm I’m not quite sure how I could use filter here. SOURCES contains all public SOURCES of it’s dependencies. I guess with some assumptions it’s possible to filter out some sources (-> For example filter out sources in _deps).
But I don’t see how to this in a general way.

For instance say I have:

Foo1.cpp
Foo2.cpp
CMakeLists.txt

I declare two libraries, one with Foo1.cpp and one with Foo2.cpp. Foo1 links to Foo2 and get’s also the sources, so the SOURCES containts then:
Foo1.cpp
Foo2.cpp

How could I possibly know in this case that Foo2.cpp is not part of Foo1, but Foo1.cpp is part of Foo1?

Even iterating over Foo1 dependencies and subtracting Foo2 Sources from Foo1 Sources won’t work in special cases (When Foo1 and Foo2 in fact both declare one source).
Do I miss anything?