protobuf_generate_cpp: version in repo won't generate grpc files

Code
1
  • I want to use CMake to use protoc to generate the RPC files for cpp
  • I am generating make files
  • I enable various tracing outputs
  • I can see the CMake flags --grpc_out and --plugin in the CMake putput
  • The CMake generated build.make file does not contain these flags
  • Below are relevant snippets from CMake and FindProtobuf.cmake
  • I made a project specific copy of FindProtobuf.cmake because the one
    in the kitware repo contains a way to enable --grpc_out.
  • I need both the generated gRPC client and server code

Does anyone have any suggestions?

// CMakeLists.txt
# =============================================================
# Protobuf
# =============================================================
list(APPEND CMAKE_PREFIX_PATH "/usr/local/protobuf-3.3.0/")


set(Protobuf_USE_STATIC_LIBS ON)
set(Protobuf_DEBUG 1)
set(protobuf_generate_PLUGIN "${GRPC_CPP_PLUGIN_EXECUTABLE}")
include(cmake/FindProtobuf.cmake)
FIND_PACKAGE(Protobuf 3.3.0 REQUIRED)

set(GENERATED_PROTOS_DIR ${CMAKE_CURRENT_BINARY_DIR}/generated)
file(MAKE_DIRECTORY ${GENERATED_PROTOS_DIR})
include_directories(${GENERATED_PROTOS_DIR})

protobuf_generate_cpp( GEN_PROTO_SRCS_CPP GEN_PROTO_HDRS_CPP 
                       IMPORT_DIRS ${PROTO_PATHS} 
                       PROTOS ${PROTO_FILE}
                     )

add_library(${PROJECT_NAME} SHARED
            ${GEN_PROTO_SRCS_CPP} ${GEN_PROTO_HDRS_CPP}
            src/FooInterface.cpp
            )

// FindProtobuf.cmake found here
// https://github.com/Kitware/CMake/blob/master/Modules/FindProtobuf.cmake

// Latest from the repo contains
  if(protobuf_generate_PLUGIN)
    set(_plugin "--plugin=${protobuf_generate_PLUGIN}")
  endif()
// then
    add_custom_command(
      OUTPUT ${_generated_srcs}
      COMMAND  protobuf::protoc
      ARGS --${protobuf_generate_LANGUAGE}_out ${_dll_export_decl}${protobuf_generate_PROTOC_OUT_DIR} ${_plugin} ${_dll_desc_out} ${_protobuf_include_path} ${_abs_file}
      DEPENDS ${_abs_file} protobuf::protoc
      COMMENT "Running ${protobuf_generate_LANGUAGE} protocol buffer compiler on ${_proto}"
      VERBATIM )

// running cmake3 output

-- OUTPUT   FooServices.grpc.pb.cc FooServices.grpc.pb.h

-- COMMAND /usr/local/protobuf-3.3.0/bin/protoc --grpc_out /user/me/ProjectFoo/build/default        --cpp_out /user/me/ProjectFoo/build/default -I /user/me/ProjectFoo/src        --plugin=protoc-gen-grpc=/usr/local/bin/grpc_cpp_plugin
-- DEPENDS /user/me/ProjectFoo/src/FooServices.proto
-- GEN_PROTO_SRCS_CPP=/user/me/ProjectFoo/build/default/FooServices.pb.cc GEN_PROTO_SRCS_PY=
-- UTIL_LIB_PATH=/build
-- Protobuf_LIBRARIES=/usr/local/protobuf-3.3.0/lib/libprotobuf.a;-lpthread

// running make output 
[ 25%] Running cpp protocol buffer compiler on /user/me/ProjectFoo/foo-interface/src/FooServices.proto
/usr/local/protobuf-3.3.0/bin/protoc --cpp_out /user/me/ProjectFoo/foo-interface/build/default -I /user/me/ProjectFoo/foo-interface/src /user/me/ProjectFoo/foo-interface/src/FooServices.proto


// from generated build.make
CMakeFiles/foo-interface.dir/build.make:	/usr/local/protobuf-3.3.0/bin/protoc --cpp_out /user/me/ProjectFoo/foo-interface/build/default -I /user/me/ProjectFoo/foo-interface/src /user/me/ProjectFoo/foo-interface/src/FooServices.proto
2

Did you add the generated files in GEN_PROTO_SRCS_CPP variable to any target?

3

For an unknown reason my edits were flagged to be reviewed by an administrator.
Therefore my posting wasn’t visible for a while and I could not edit it further until now.

@hsattler Yes GEN_PROTO_SRCS_CPP was later reference in add_library()

I have fully resolved all of my issues.

  • I was calling Find_Package(Protobuf 3.3.0 REQUIRED) which includes the standard FindProtobuf.cmake
  • The include of my own custom FindProtobuf.cmake occurred after the call to Find_Package()
  • The custom protobuf_generate* functions were being overridden by the standard versions
  • I reversed the order so that Find_Package() call is first.
  • The newer FindProtobuf.cmake from which I based my custom version does not support the --grpc_out flag which is required for the --plugin=protoc-gen-grpc to perform the gRPC generation
  • I temporarily hardcoded the --grpc_out parameter in my custom FindProtobuf.cmake

Additional

  • The protobuf_generate() function has a PLUGIN parameter.
  • The protobuf_generate_cpp() function does NOT have a PLUGIN parameter.
  • The way function parameter are implemented is not clear to me so it may take me some time to figure out a more generic solution.
4

I wish this could be fixed earlier.

To @JRHeisey I think what you said is not true here. The implementation has protobuf_generate_cpp_UNPARSED_ARGUMENTS after calling cmake_parse_argument and passed into the protobuf_generate function as ${_proto_files}. So if you have PLUGIN ... there, it should still work as expected.

The real issue is that we have no way to add --grpc_out to the ARGS in the add_custom_command. The implementation doesn’t seem to have a way to tell (in a generic way) what kind of plugin is provided and whether it would be correct to add extra flags to the protoc command.

Perhaps checking protobuf_generate_PLUGIN would be a way

  set(_plugin)
  if(protobuf_generate_PLUGIN)
    LIST(APPEND _plugin "--plugin=${protobuf_generate_PLUGIN}")
    string(FIND "${protobuf_generate_PLUGIN}" "protoc-gen-grpc" _grpc_found)
    if(_grpc_found GREATER -1)
      list(APPEND _plugin "--grpc_out" "${protobuf_generate_PROTOC_OUT_DIR}")
    endif()
  endif()

then the user can just call

protobuf_generate_cpp(
  helloworld_proto_srcs
  helloworld_proto_hdrs
  helloworld.proto
  PLUGIN "protoc-gen-grpc=${_GRPC_CPP_PLUGIN_EXECUTABLE}"
  GENERATE_EXTENSIONS .pb.h .pb.cc .grpc.pb.h .grpc.pb.cc
)