@kyle.edwards I’m wondering if it might be wise to somehow incorporate a protocol or version number in the JSON file used for test resource allocation. If we ever need to change the information given in that file, it would be difficult to do with the current format definition. Should we require that the first element be some sort of protocol or version number to provide ourselves with the opportunity for that in the future if it is needed?
@craig.scott thanks. We were very careful about versioning for the fileapi and ctest --show-only=json-v1 features. In both of those cases the input queries have “v1” somewhere in the file path or command-line argument name, thus reserving the option of making v2 not even a JSON format. To do that in this case we’d have to spell the option ctest --resource-spec-file v1:$file or something like that.
@craig.scott Good idea. I would like to name the command-line argument --resource-spec-file-v1, and name the ctest_test() argument RESOURCE_SPEC_FILE_V1. Do you have any objections to this?
Hmmm… can’t say I’m enthusiastic about encoding the version number into the command line argument name or the command keyword. It doesn’t feel natural and it’s not a common practice (in CMake or elsewhere). Realistically, I think we are unlikely to change the file format in the short or medium term. For that, we can embed the version in the JSON file itself, which makes usage pretty clean. It also means you can substitute in different resource spec files of different versions and the rest of your workflow doesn’t have to care. If we ever do need to change the format to something other than JSON, perhaps Brad’s suggestion can be a way to deal with that (prefix the file name with a version, like v5:somefile.foo or similar). An absence of any such version prefix can be understood to mean that the JSON file itself defines the version as its first element (must be the first to make it independent of any other arbitrary format/protocol changes).
Another reason to avoid embedding version numbers into command line options and CMake command keywords is that it may not be as friendly to option/keyword completion tooling.
When v2 happens, how does a tool specify that it wants v1 (because v2 isn’t supported yet)? If there’s no way to specify it today, they’re stuck until such a flag is supported and then have to specify it to not get v2 (otherwise v2 is going to have to have some other mechanism to get it and v1 is our default forever).
I’ve opened CMake Issue 20007 to get this addressed for 3.16.0. See that issue for my proposed approach.