# Sending more than pass/fail data to CDash - what is the data format CDash expects?

**URL:** <https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392>\
**Category:** Code\
**Created:** [December 17, 2020, 3:34pm UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392 "2020-12-17T15:34:06Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jverdicchio](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/j/3da27b/32.png) [@jverdicchio](https://discourse.cmake.org/u/jverdicchio)\
**Post date:** [December 17, 2020, 3:34pm UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/1 "2020-12-17T15:34:06Z")

</div>

At my previous employer, we used CDash to display statistics about the tests that were run. When the code was run in “test” mode, additional statistics were recorded about the code and saved to an XML file. These statistics were then available in CDash to plot as history files. Unfortunately I didn’t set the CDash up - I only ever used it. I’m now attempting to set up CTest with CDash with my new employer.

My question is : what is the data format the CDash is expecting such that it can display all this data?  
The data is going to be something like:  
testname: simpletest  
statistic1: “numberOfSteps”  
staticistic type: int  
statistic value: 10  
statistic2: “maximumQuality”  
staticistic type: float  
statistic value: 0.1  
etc.

I know that one could compare these statistics from one test to the next run using a baseline set of test-data. Again, never wrote this, only know that it happened.  
Anyone have any ideas how one might go about this? If this requires some scripting to occur post-testing, then fine.

---

<div class="post-metadata">

**Author:** ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)\
**Post date:** [December 17, 2020, 11:51pm UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/2 "2020-12-17T23:51:31Z")

</div>

Is the `MEASUREMENT` test property what you are looking for? Note that it is a static value determined at configure time, not a value computed as part of running the test.

If you want to have the test compute a value to be reported to CDash, there’s an undocumented capability for that. It’s been on my todo list to document it for longer than I want to admit, but you can find a starting point for basic details and links [here](https://gitlab.kitware.com/cmake/cmake/-/issues/18079).

---

<div class="post-metadata">

**Author:** ![jverdicchio](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/j/3da27b/32.png) [@jverdicchio](https://discourse.cmake.org/u/jverdicchio)\
**Post date:** [December 18, 2020, 9:20am UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/3 "2020-12-18T09:20:36Z")

</div>

These are going to be values calculated from the code during the test. We have instrumented our code to record statistics about what it is doing. At the end of a run if a certain environment variable is set the code exports the statistics to a JSON format file. Our team may add or delete these statistics from the code but we don’t want to edit the cmake setup.

---

<div class="post-metadata">

**Author:** ![jverdicchio](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/j/3da27b/32.png) [@jverdicchio](https://discourse.cmake.org/u/jverdicchio)\
**Post date:** [December 18, 2020, 9:28am UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/4 "2020-12-18T09:28:32Z")

</div>

I think you documentation problem is best solved by an example. What I’ve found so far in dealing with CMake/CTest is the documentation is rather like the linux man pages. Great if you are looking for some fine detail about a command you already know the existence of but not so good if you don’t know what the command is or even how a thing may be performed. I’ve been dealing with CMake on and off for around 6 years and I still feel like a novice. Novices like examples to follow and edit.

---

<div class="post-metadata">

**Author:** ![smokris](https://discourse.cmake.org/user_avatar/discourse.cmake.org/smokris/32/108_2.png) [@smokris](https://discourse.cmake.org/u/smokris)\
**Post date:** [December 24, 2020, 1:09am UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/5 "2020-12-24T01:09:18Z")

</div>

> [@craig.scott](#):
>
> you can find a starting point for basic details and links here

Thanks for the breadcrumbs; here’s what I found:

[VTK’s tests write text to `stdout`](https://github.com/Kitware/VTK/blob/4192fb4f8a0239c0c843aaefa76a1342bf5ce871/Testing/Rendering/vtkTesting.cxx#L344) containing an XML fragment:

```xml
<DartMeasurement name="WallTime" type="numeric/double">42</DartMeasurement>

```

[CTest scans the output for those XML fragments](https://gitlab.kitware.com/cmake/cmake/-/blob/v3.19.2/Source/CTest/cmCTestRunTest.cxx#L667), transforms them, and outputs them in the `Testing/**/Test.xml` document in this format:

```xml
<NamedMeasurement name="WallTime" type="numeric/double">
    <Value>42</Value>
</NamedMeasurement>

```

---

<div class="post-metadata">

**Author:** ![jverdicchio](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/j/3da27b/32.png) [@jverdicchio](https://discourse.cmake.org/u/jverdicchio)\
**Post date:** [January 5, 2021, 10:20am UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/6 "2021-01-05T10:20:17Z")

</div>

This is how I’ve now got this to work. Assume I have 4 pieces of additional data I want stored in CDash:

- numberSides - integer , 6
- areaSides - double , 6.0
- wasItASquare - boolean, true
- nameOfTheResult - string , “TheUltimateAnswer”

Your program must write to std::out.  
The output for my 4 pieces of data would look like this sent to std::out:

```auto
<DartMeasurement name="numberSides" type=numeric/integer">6</DartMeasurement>
<DartMeasurement name="areaSides" type=numeric/double">6.0</DartMeasurement>
<DartMeasurement name="wasItASquare" type=bool>"true"</DartMeasurement>
<DartMeasurement name="nameOfTheResult" type=string>"TheUltimateAnswer"</DartMeasurement>

```

CTest does all the hard work and packages these extra data sets up and sends to CDash. The data can be viewed and plotted in CDash.

---

<div class="post-metadata">

**Author:** ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)\
**Post date:** [January 5, 2021, 11:34am UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/7 "2021-01-05T11:34:36Z")

</div>

@jverdicchio I edited your post to fix the formatting. Feel free to take a look, but it’s basically the usual Markdown formatting.

---

<div class="post-metadata">

**Author:** ![jverdicchio](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/j/3da27b/32.png) [@jverdicchio](https://discourse.cmake.org/u/jverdicchio)\
**Post date:** [January 5, 2021, 11:37am UTC](https://discourse.cmake.org/t/sending-more-than-pass-fail-data-to-cdash-what-is-the-data-format-cdash-expects/2392/8 "2021-01-05T11:37:01Z")

</div>

Craig, thank you very much.