I think I’ve solved the problem of how to build and run Swift-based XCTests, on all platforms where Swift is supported, using CMake. My code is here. It seems like some part of this work ought to be lifted into CMake, so I’m opening this issue to discuss if/how that should be done. Additionally I have some thoughts about how CMake’s existing FindXCTest is structured, which maybe warrant discussion here as well.
Do you think this is best done as a module like FindGTest and its CMake API or something built-in for CMake (i.e., C++ code)? We do now have an experimental feature gate mechanism to aid in deploying this without committing to a stable interface if that would help get more feedback.
I’m not all that well versed in these areas of CMake, but a separate topic might be best to avoid them cluttering each other’s discussions.
Definitely no need to write new C++ code in this case! As noted here I view this mostly as a modernization of the existing FindXCTest so I think FindXCTest (or FindSwiftXCTest if my code can’t be made to work with Objective-C on macOS—I haven’t tried that) would be appropriate.
The one complication I see is that support for non-macOS platforms involves building and running a code generator written in Swift. I don’t know if there’s any precedent for including something like that in CMake… but ideally the whole thing belongs there.
Can we find_program this and it be maintained separately from CMake?
We can do anything. I just don’t know CMake-isms, culture, etc., well enough to know what’s most appropriate, which is part of why I started this discussion rather than just, say, opening a PR.
Ok, that sounds good to me then. Yes, discussion for proposed features starts here; we can open issues as concrete tasks arise.
Since it sounds like maintaining the codegen separately from CMake is an option (it seems to be in Swift because the stdlib contains a Swift parser?), that sounds preferable to me to avoid:
- new toolchain for CMake packages
- more development environment needed for CMake developers
- coupling to CMake’s release schedule
Since it sounds like maintaining the codegen separately from CMake is an option (it seems to be in Swift because the stdlib contains a Swift parser?), that sounds preferable to me
It’s not in the standard library, but it’s a kind of (mostly?) official auxilliary package.
What would you suggest my next steps are then?
Bump. I’ve been pushing this work forward myself but I am reaching the limits of what’s practical without some guidance. For what it’s worth, I think it could be in Kitware’s interest to upstream this work, which is currently 157 lines of cmake code:
- Almost every Swift developer uses XCTest.
- OOTB CMake is not currently doing what Swift devs need for XCTest support on any platform.
- Apple’s own Swift Package Manager has numerous problems that make CMake adoption attractive once projects grow beyond a very limited scope.
And if y’all aren’t interested, I’d like to know that too so I can stop asking 
I get the sense that you’re now the number 1 or 2 person about the module 
. If there are comprehensive tests for it, that should ease integration.
Thanks for your reply, but I’m not sure what to make of the fact that I’m the number 1 or 2 person about the module. In fact, I assume by ”the module” you mean FindXCTest but my code is currently structured as a separate module, FindSwiftXCTest, because for Swift clients on macOS there’s an additional library dependency that Objective-C clients don’t need… and I assume creating that dependency for existing ObjC clients would be unacceptable. The cost of avoiding this dependency is complexity that hurts usability somewhat. That’s exactly the kind of tradeoff I would like guidance on. I don’t want to invest more effort in preparing a submission only to be told the approach is wrong.
There are tests that are passing in GitHub CI on the 3 platforms supported by Swift. Whether you consider them comprehensive is I suppose another matter. Looking at some of the tests in the CMake sources that exercise FindXCTest is a bit daunting. It’s not obvious to me what one should do to test this new code in that context.
@ben.boeckel so,
- did you have a look?
- Are the tests adequate for integration? If not, what do I need to do?
- Is a separate
FindSwiftXCTestmodule appropriate? I suppose it’s possible to have that dependency on the Swift-only XCTest layer be generated only when there are Swift files in the test, so I could think about modifyingFindXCTestitself.
I’d like to avoid a separate FindSwiftXCTest if possible. We have language-specific conditional behavior in many find modules.
OK, well in case anyone is monitoring the state of this work, I’ve separated my FindSwiftXCTest from the generator executable and to-the-best-of-my ability maximally organized/cleaned the code. I’ll try to prepare a PR integrating it into FindXCTest at some point soonish.
I’m not sure where that should go; does using find_program conflict with my personal goal of building all the parts as dependencies of my project?
find_program will skip running if set somewhere else. You could set it to $<TARGET_FILE:target_name>.
OK, thanks. That appears to be undocumented behavior? https://gitlab.kitware.com/cmake/cmake/-/issues/25844
From the docs:
Emphasis added. I suppose it isn’t clear that it doesn’t care about the cache state, just the value not being “falsey”.
No, what’s undocumented is that if the variable is already set, then no search will ever be performed. Because of the use of the word “repeated,” the doc only tells us that if the search has already happened once it won’t happen again.
Ah…the implicit knowledge here is that the cache is the only form of memory CMake has, so if a variable is set, it’s not possible to know if it was from a previous find_* call, given explicitly via -D, or a set() call. More wordsmithing needed…
Oh, I wasn’t even thinking about memory across CMake invocations. In a single CMake run if I force-set the _FOUND variable and do a find_program, presumably it is a no-op. The docs don’t say that is the case. For example, it would be consistent with the current docs for CMake to use another variable or property to remember whether a search has been performed, and when that variable is false perform the search regardless of the _FOUND variable state.