Projects/Subprojects in cdash

I work on a project (call it super) that has several subprojects (call them sub1 and sub2) and I have a few issues (I tried to bold the the issues). Also, I had to remove several images because I am a new user…

In CDash, I went through the process of adding the project super:

had to remove image (i’m a new user…)

and then attaching sub1 and sub2.

The first issue is as follows - when I click on the link for the super project it takes me to the following page:

had to remove image (i’m a new user…)

The above page collects build jobs for all sub projects which is ok but too much detail. What I would rather see is the following:

Note that in the above webpage I can navigate to either the super project or the subprojects. The only way I can get to this page is by click on the drop down item SubProjects as shown below:

had to remove image (i’m a new user…)

Furthermore, when I click on the subprojects the header contains the name of the super project rather than the subproject which can be confusing.

Beyond the navigation issue described above, I also have a problem with coverage submissions under this project/subproject model. That is, the coverage reports are not published under the dashboard for the subproject rather they wind up on the project page. Continuous, Nightly, Experimental build jobs all go to the subproject (and super project) as one would expect.

I am currently on version 2.6.0. The behavior that I am after we have in a previous version of cdash (i.e. 2.0.2), but it seems to have gone away in more recent versions.

Thoughts? Thanks!

The default view in CDash (index.php) shows a list of builds for a given testing day. I’m glad you like the subproject-by-subproject view. It’s been on our backlog to allow users to customize their default landing page, but we haven’t gotten around to implementing this feature yet.

Good suggestion. We’ll look into adding the SubProject name to the navigation header.

We definitely support splitting coverage results up by SubProject. Is the CDash instance you’re submitting to publicly visible? If so I’ll take a closer look to see if I can figure out what’s going wrong.

Unfortunately, it is not a public project. We might be able to do some debugging nonetheless. For example, I generate a CTestConfiguration.ini file that contains all of the information about the project. Some of the relevant variables that I set I just learned are considered legacy:

  • DropLocation
  • DropMethod
  • DropSite

I see that I should instead use 'SubmitURL`.

I also use:

  • LabelsForSubprojects

The variables listed above are he only ones I see that might impact the submission step. Below I capture the relevant CI steps invoked:

ctest -M Experimental -T Start -- --track Experimental
ctest -M Experimental -T Configure -- --track Experimental
ctest -M Experimental -T Build -- --track Experimental
ctest -M Experimental -T Test -- --track Experimental
ctest -M Experimental -T Coverage -- --track Experimental
ctest -M Experimental -T Submit -- --track Experimental

As an aside, the redundant specification of Experimental is a result of using a generic CI script where the option following --track is a variable e.g. ${CTEST_MODE} and the reason we do that is due to the fact that:

ctest -M Continuous -T Start
ctest -M Continuous -T Configure
ctest -M Continuous -T Build
ctest -M Continuous -T Test

won’t actually trigger any of the steps in the context of CI - I believe that this is due to the fact that in the context of CI/git it doesn’t look like anything changed so there is no reason configure, build, test, etc. so we use Experimental and then send everything to the appropriate track or in CDash speak group.

I should be able to whip up a quick example demonstrating code coverage split up by subproject.

With CTest & CDash, there’s two separate ways to do subproject builds:

  1. “one at a time”. Build/test each subproject separately.
  2. “all at once”. Build your whole project at the same time and use CTest/CDash to split up the results on a subproject-by-subproject basis.

Which approach sounds more appropriate for your situation?

Thanks Zack - I am using the one at a time method.

I was out of the office this week but I am back and I discovered that using a ctest script seems to resolve the issue of the coverage jobs going to the project dashboard and NOT going to the subproject. I don’t understand why yet, but ctest scripts seem to be the solution.

@craig.scott recommended that I move to ctest scripts as opposed to explicit ctest dashboard commands for another usage issue and I am now convinced.

OK so I am perplexed. As I pointed out earlier, by moving things into a ctest script, I was able to get things reporting to the dashboard where I would expect. I borrowed an old script from another team member and ran it as is and it worked. I then started stripping things out and found that again the coverage reports were only going to the super project again!!! After applying the Sherlock Holmes method I found that the key was the following:

set_property(GLOBAL PROPERTY SubProject "${SUBPROJECT}")

I can’t find this property anywhere on the cmake properties documentation page…

In other words, I think the ctest script hypothesis was wrong, but I still prefer it to what I was doing before it actually cleaned up my CI.

I’ve filed CMake Issue 20357 for this.