execute_process buffering

Baruch_Burstein · March 1, 2023, 11:03pm

I am using execute_process to run a process that can take some time (specifically - docker build). I want to see the output from the process as it is generated since it lets me know where it is up to and how much longer it will take, but execute_process is buffering the output and only dumping it all once the process is finished. How can I get it to show the output as it is generated, without buffering?

craig.scott · March 2, 2023, 8:50pm

Are you using OUTPUT_VARIABLE and/or ERROR_VARIABLE to capture the output? If so, try ECHO_OUTPUT_VARIABLE and ECHO_ERROR_VARIABLE instead (requires CMake 3.18 or later).

Baruch_Burstein · March 9, 2023, 6:59pm

I am not using either of those. The only thing I am using is OUTPUT_STRIP_TRAILING_WHITESPACE

scivision · March 10, 2023, 3:45am

This little experiment shows that execute_process() doesn’t print till the process ends or stdout pipe is flushed.
If you run the Python script directly without CMake, the printing occurs “real time” even when not flushed thought that could be a bit platform-dependent.

gist.github.com

https://gist.github.com/scivision/62e822cdf6e35b6858933676e5b98cd5

execute_process_stdout.cmake

# Observe for Python flush=False, the printing is buffered till the first flush=True

cmake_minimum_required(VERSION 3.19)

find_package(Python COMPONENTS Interpreter REQUIRED)

set(CMAKE_EXECUTE_PROCESS_COMMAND_ECHO STDOUT)

execute_process(COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_LIST_DIR}/sleepy_print.py
RESULT_VARIABLE ret

This file has been truncated. show original

sleepy_print.py

#!/usr/bin/env python3
"""
print() without flush and with flush

https://docs.python.org/3/library/functions.html#print
"""

from time import sleep

N = 12

This file has been truncated. show original