Reproducing your example:
cmake_minimum_required(VERSION 3.20 FATAL_ERROR)
project(Example LANGUAGES C CXX)
add_library(target SHARED test.c test.cpp)
set(MY_WARNING_FLAGS -Wall -Wextra $<$<COMPILE_LANGUAGE:CXX>:-Wold-style-cast> -pedantic)
target_compile_options(target PRIVATE ${MY_WARNING_FLAGS})
Has everything work as expected locally. I would first verify locally that the above example works for you.