# Maintaining list of Python source files without file(GLOB\_RECURSE ...)
**URL:** https://discourse.cmake.org/t/maintaining-list-of-python-source-files-without-file-glob-recurse/1404
**Category:** Usage
**Created:** [June 18, 2020, 7:29pm UTC](https://discourse.cmake.org/t/maintaining-list-of-python-source-files-without-file-glob-recurse/1404 "2020-06-18T19:29:20Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@praetorian](https://discourse.cmake.org/u/praetorian)
#### Post date: [June 18, 2020, 7:29pm UTC](https://discourse.cmake.org/t/maintaining-list-of-python-source-files-without-file-glob-recurse/1404/1 "2020-06-18T19:29:20Z")
Hi everyone,
I have a large project that uses CMake (3.6) & ninja for building. The code is mostly C++ but it has a few Python components as well. The build process auto-generates some Python files in `${CMAKE_BINARY_DIR}/py`. I need to copy some hand-written Python files into the same directory for running unit tests and creating an sdist.
The question is about best practices for maintaining the list of these hand-written Python files. Say I have the following directory structure that needs to be copied to `${CMAKE_BINARY_DIR}/py`:
```auto
a
├── b
│ ├── c
│ │ ├── CMakeLists.txt
│ │ └── __init__.py
│ ├── CMakeLists.txt
│ └── __init__.py
├── CMakeLists.txt
└── __init__.py
```
This is how I currently create a list of all Python source files
_a.CMakeLists.txt_
```auto
add_subdirectory(b)
set(A_PYTHON_SOURCES ${A_PYTHON_SOURCES}
${CMAKE_CURRENT_SOURCE_DIR}/ __init__.py
)
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/a.stamp
DEPENDS ${A_PYTHON_SOURCES}
COMMAND rsync -avz ...
COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_BINARY_DIR}/a.stamp
)
add_custom_target(
build_a ALL
DEPENDS ${CMAKE_BINARY_DIR}/a.stamp
)
```
_b.CMakeLists.txt_
```auto
add_subdirectory(c)
set(A_PYTHON_SOURCES ${A_PYTHON_SOURCES}
${CMAKE_CURRENT_SOURCE_DIR}/ __init__.py
PARENT_SCOPE
)
```
_c.CMakeLists.txt_
```auto
set(A_PYTHON_SOURCES ${A_PYTHON_SOURCES}
${CMAKE_CURRENT_SOURCE_DIR}/ __init__.py
PARENT_SCOPE
)
```
This works, but the building of `A_PYTHON_SOURCES` using `PARENT_SCOPE` feels a bit dirty. Another option I know of is to `include(...)` all the descendant `CMakeLists.txt` from `a/CMakeLists.txt` which would avoid `PARENT_SCOPE`, but then you’re leaking the directory hierarchy into the top level file.
`target_sources` seems like a good option, but since I need to copy Python source files, and am not building a library or executable, it doesn’t seem possible to use it in my case.
What is the right way to go about this? **Is there a better way to manually enumerate a tree of source files than what I’m doing?**
Thanks,
Ashish.
---
### Author:  [@jwm](https://discourse.cmake.org/u/jwm)
#### Post date: [June 18, 2020, 8:24pm UTC](https://discourse.cmake.org/t/maintaining-list-of-python-source-files-without-file-glob-recurse/1404/2 "2020-06-18T20:24:24Z")
> [@praetorian](#):
>
> ```auto
> add_custom_command(
> OUTPUT ${CMAKE_BINARY_DIR}/a.stamp
> DEPENDS ${A_PYTHON_SOURCES}
> COMMAND rsync -avz ...
> COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_BINARY_DIR}/a.stamp
> )
>
> ```
This feels like an anti-practice. Shouldn’t your sources be controlled?
I don’t know that I would litter my python module tree with CMakeLists.txt files. If you need python files for unit tests, consider either `configure_file` or modifying the PYTHONPATH environment variable to include this source tree.
---
### Author:  [@praetorian](https://discourse.cmake.org/u/praetorian)
#### Post date: [June 18, 2020, 8:45pm UTC](https://discourse.cmake.org/t/maintaining-list-of-python-source-files-without-file-glob-recurse/1404/3 "2020-06-18T20:45:46Z")
> [@jwm](#):
>
> Shouldn’t your sources be controlled?
Yes, all the files shown in the question are under source control, but why does that matter? I’m trying to figure out when any of the Python files have been modified so I can copy them to the binary directory. (I’m not a CMake expert by any means, so I might be missing something that should be obvious)
> [@jwm](#):
>
> I don’t know that I would litter my python module tree with CMakeLists.txt files
I agree, I don’t like it either, but the other option I can think of is listing them all in `a/CMakeLists.txt`, which I don’t like either.
> [@jwm](#):
>
> If you need python files for unit tests, consider either `configure_file`
Are you saying use `configure_file(... COPYONLY)` for each `.py` file? If so, how do I get the list of files to be copied (which is basically my question)?
> [@jwm](#):
>
> or modifying the PYTHONPATH environment variable to include this source tree
I’m not only running unit tests, but the files under `${CMAKE_BINARY_DIR}/py` are also being built into an sdist. So I need the directory structure created by copying the files under `a` to `${CMAKE_BINARY_DIR}/py`.
In any case, even if I could use `PYTHONPATH` or whatever to run unit tests under the source tree, how do I detect when any of the files have changed so I can re-run unit tests? Again, it seems to me I need a list of all Python source files for doing that.
---
### Author:  [@jwm](https://discourse.cmake.org/u/jwm)
#### Post date: [June 18, 2020, 8:53pm UTC](https://discourse.cmake.org/t/maintaining-list-of-python-source-files-without-file-glob-recurse/1404/4 "2020-06-18T20:53:09Z")
> [@praetorian](#):
>
> Yes, all the files shown in the question are under source control, but why does that matter?
It just struck me as odd. Using **rdist** is stepping outside of CMake, and is extremely unlikely to be portable.
> [@praetorian](#):
>
> Are you saying use `configure_file(... COPYONLY)` for each `.py` file? If so, how do I get the list of files to be copied (which is basically my question)?
Your choices are either use a glob, or explicitly enumerate. While it feels tedious, explicit enumeration is (I think) generally preferred.
> [@praetorian](#):
>
> I’m not only running unit tests, but the files under `${CMAKE_BINARY_DIR}/py` are also being built into an sdist. So I need the directory structure created by copying the files under `a` to `${CMAKE_BINARY_DIR}/py` .
I haven’t made a source distribution, but I would expect you provide these files using something along the lines of `install`.
> [@praetorian](#):
>
> (I’m not a CMake expert by any means, so I might be missing something that should be obvious)
The blind leading the visually-impaired…
---
### Author:  [@praetorian](https://discourse.cmake.org/u/praetorian)
#### Post date: [June 18, 2020, 9:05pm UTC](https://discourse.cmake.org/t/maintaining-list-of-python-source-files-without-file-glob-recurse/1404/5 "2020-06-18T21:05:47Z")
> [@jwm](#):
>
> Using **rdist** is stepping outside of CMake
Ah yes, the `rsync` is an odd choice and unportable, someone else put that in. I should be able to replace it with `${CMAKE_COMMAND} -E copy_directory ...` but since portability is not a concern for this project, I haven’t bothered.
> [@jwm](#):
>
> explicit enumeration is (I think) generally preferred
I’ve read this too, and that’s what my question is essentially - is there a better way to manually enumerate than what I’m doing? I think I’ve drowned that out with too much verbiage, so I’ll clarify the original question.
> [@jwm](#):
>
> I would expect you provide these files using something along the lines of `install`
That’s an interesting thought, thanks! I’m probably not going to be able to use that though because the sdist is being installed into a virtualenv which is created as part of the build, so that yet another suite of tests can be run. Sorry for moving the goal posts, I didn’t think this was relevant information initially.