ctest enabling colored output on windows?

Usage
1

Hello guys,

using ctest on Linux and the test output appeared according to the colors being set by the test. This does not happen on Windows, getting the color codes as plain text.
I can see here: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/2577
that Sylvain Joubert writing on this same issue:
I have been conservative and disabled color on Windows as I can't test on that platform. Are ANSI code supported on Windows or is there another way to output color to the console?

Is this feature disabled on windows?

Thanks,
Serge

2

Looks like the current code still doesn’t support coloured output on Windows:

bool cmCTest::ColoredOutputSupportedByConsole()
{
#if defined(_WIN32)
  // Not supported on Windows
  return false;
#else
  // On UNIX we need a non-dumb tty.
  // ...<implementation omitted>
  return ConsoleIsNotDumb();
#endif
}
1 Like
3

Part of this is likely that (AFAIK), colors are in-band on VT-alikes and out-of-band on Windows terminals. So color can be managed directly by the data stream on VT but windows requires calls to the terminal handle. Of course, maybe the new terminal understands ANSI color sequences, but I’ve not encountered the need yet either.

1 Like
4

Thanks Ben for the clarifications,
are there any plan to support Windows and can you think on any possible workaround to use for now?

Thanks,
Serge

5

Answering to myself :slight_smile:
Also thanking Craig as well for pointing to the code culprit.

Using mingw console on Windows and I can see the colors back, so this will be our workaround for now.
Also it surprised me to see that even though Windows and despite the above disabling code it still can work and show colors.

Thanks,
Serge

6

It used to be true that color escape sequences weren’t supported by Windows, but this hasn’t been the case for a few years:

Console Virtual Terminal Sequences - Windows Console | Microsoft Learn matches the common unix escape sequences, and this is now Microsoft’s recommended approach:

  • the documentation for the old out-of-band method - SetConsoleTextAttributesA()- explicitly deprecates it, and recommends escape sequences instead. No link because this discourse gives me a 2-link limit.
  • a longer Microsoft article notes the same with much more detail

It’s possible to check support at runtime by seeing if GetConsoleMode() flags include both ENABLE_PROCESSED_OUTPUT and ENABLE_VIRTUAL_TERMINAL_PROCESSING

For example, ctest works fine in Microsoft’s new-ish ‘Windows Terminal’ app if I set $Env:CLICOLOR_FORCE=1

1 Like
7

Hi,

I think in general there’s probably two things that need to be done here to enable color output for Windows shells (including the venerable CMD shell). As Fred mentioned it’s possible to use GetConsoleMode() to retrieve flags for the stdout handle, which can be retrieved using GetStdHandle(STD_OUTPUT_HANDLE). Testing the flags can be done in cmCTest::ColoredOutputSupportedByConsole() to determine if virtual terminal sequences are supported so we can have our nice color text printed.

The problem is that for Windows CMD, virtual terminal sequence processing is disabled by default, so a call to SetConsoleMode with ENABLE_VIRTUAL_TERMINAL_PROCESSING as one of the flags is necessary (maybe when CMake starts up?) in order to enable processing.

For exposition, here’s one way to modify cmCTest::ColoredOutputSupportedByConsole():

// note: minimal error checking; call GetLastError() for error code.
// or we could just print warnings to stderr, that works too?
bool cmCTest::ColoredOutputSupportedByConsole()
{
  // ... env checking
#if defined(_WIN32)
  // stdout handle
  auto out_hnd = GetStdHandle(STD_OUTPUT_HANDLE);
  // if no stdout or error then not supported
  if (!out_hnd || out_hnd == INVALID_HANDLE_VALUE)
    return false;
  // get flags + check
  DWORD out_flags;
  if (!GetConsoleMode(out_hnd, &out_flags))
    return false;
  return (out_flags & ENABLE_PROCESSED_OUTPUT) &&
    (out_flags & ENABLE_VIRTUAL_TERMINAL_PROCESSING);
#else
  // On UNIX we need a non-dumb tty.
  return ConsoleIsNotDumb();
#endif  // !defined_WIN32)
}

For the SetConsoleMode() call to enable virtual terminal sequences it’s possible to try and use some RAII wrapper or just a pair of routines for this.

8

Edit: Forgot about calling Instance() before CLICOLOR_FORCE is checked.

Well, I tried making some changes after checking out the CMake source at 8733dcd115, creating a cmStdOutHandle class that attempts to enable virtual terminal sequences on construction (via a Meyer’s singleton-esque Instance() static function). The instance stores a HANDLE retrieved via GetStdHandle(STD_OUTPUT_HANDLE) and DWORD flags from GetConsoleMode().

So the only real changes to cmCTest::ColoredOutputSupportedByConsole() are:

bool cmCTest::ColoredOutputSupportedByConsole()
{
  // check if VTS is enabled on Windows (attempts to enable)
#ifdef _WIN32
  bool vts_enabled = cmStdOutHandle::Instance().VtsEnabled();
#endif  // _WIN32
  // ... env checking
#if defined(_WIN32)
  return vts_enabled;
#else
  // On UNIX we need a non-dumb tty.
  return ConsoleIsNotDumb();
#endif  // !defined(_WIN32)
}

There should be a way to allow this kind of functionality to be enabled for cmake itself, not just ctest, although I don’t know enough about the code base to know where to put it. Maybe in a Source/cmStdOutHandle.h header (with accompanying .cxx file)?