Suitable versions of Sphinx for building different versions of CMake Docs?

Recently, I tried to configure and build the CMakeHelp project directly to generate CMake Docs locally. The following commands are my simple test of using Sphinx-6.2.1 to build the CMakeHelp of v3.18.6 tag:

sphinx-build --version
git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
cd CMake
git checkout v3.18.6 --quiet
git describe --tag
mkdir build && cd build
cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
cmake --build .

However, I found that the latest Sphinx version (currently, 6.2.1) cannot handle some previous versions of CMake Docs. The following error is generated by the above commands:

Extension error:
Could not import extension cmake (exception: No module named 'sphinx.util.pycompat')
ninja: build stopped: subcommand failed.
Click to expand the full logs
D:\Repo\tmp>sphinx-build --version
sphinx-build 6.2.1

D:\Repo\tmp>git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
Cloning into 'CMake'...
remote: Enumerating objects: 79159, done.
remote: Counting objects: 100% (79159/79159), done.
remote: Compressing objects: 100% (37657/37657), done.
remote: Total 79159 (delta 52343), reused 62038 (delta 38466), pack-reused 0
Receiving objects: 100% (79159/79159), 48.72 MiB | 4.45 MiB/s, done.
Resolving deltas: 100% (52343/52343), done.
Updating files: 100% (22981/22981), done.

D:\Repo\tmp>cd CMake

D:\Repo\tmp\CMake>git checkout v3.18.6 --quiet

D:\Repo\tmp\CMake>git describe --tag
v3.18.6

D:\Repo\tmp\CMake>mkdir build && cd build

D:\Repo\tmp\CMake\build>cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Repo/tmp/CMake/build

D:\Repo\tmp\CMake\build>cmake --build .
[1/1] sphinx-build html: see Utilities/Sphinx/build-html.log
FAILED: doc_format_html D:/Repo/tmp/CMake/build/doc_format_html
cmd.exe /C "cd /D D:\Repo\tmp\CMake\build && C:\Python\Python310\Scripts\sphinx-build.exe -c D:/Repo/tmp/CMake/build -d D:/Repo/tmp/CMake/build/doctrees -b html -A versionswitch=1 D:/Repo/tmp/CMake/Help D:/Repo/tmp/CMake/build/html > build-html.log"

Extension error:
Could not import extension cmake (exception: No module named 'sphinx.util.pycompat')
ninja: build stopped: subcommand failed.

Even though I downgraded the Sphinx gradually, I still met some other different errors sometimes, which bothers me a lot.

Can CMake Team tell me which versions of Sphinx are suitable for building different versions of CMake Docs?

cc: @brad.king @craig.scott

For reference, CMake MR 8324 removed some compatibility code, so CMake 3.27’s docs will require Sphinx 2.x or higher.

If it still doesn’t build with the newest version of Sphinx, further merge requests are welcome to address that.

We do not maintain a mapping of CMake version to Sphinx version, but old CMake versions cannot be expected to support newer Sphinx versions that have removed compatibility with old Sphinx versions.

I noticed that there is an information about the version of Sphinx used to build the CMake Docs in the bottom of page. Maybe I can take it for reference:

image

cc: @brad.king

I found another problem when building the older version of CMake Docs:

  • Sphinx: 2.4.4
  • CMake Docs: v3.11.4

The followings are the commands I used to build the CMake Docs:

sphinx-build --version
git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
cd CMake
git checkout v3.11.4 --quiet
git describe --tag
mkdir build && cd build
cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
cmake --build .

Although there is no error showed up, there is an warning:

WARNING: while setting up extension sphinx.addnodes: node class 'meta' is already registered, its visitors will be overridden
Click to expand the full logs
D:\Repo\tmp>sphinx-build --version
sphinx-build 2.4.4

D:\Repo\tmp>git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
Cloning into 'CMake'...
remote: Enumerating objects: 79159, done.
remote: Counting objects: 100% (79159/79159), done.
remote: Compressing objects: 100% (37674/37674), done.
remote: Total 79159 (delta 52339), reused 62025 (delta 38449), pack-reused 0
Receiving objects: 100% (79159/79159), 48.72 MiB | 4.07 MiB/s, done.
Resolving deltas: 100% (52339/52339), done.
Updating files: 100% (22981/22981), done.

D:\Repo\tmp>cd CMake

D:\Repo\tmp\CMake>git checkout v3.11.4 --quiet

D:\Repo\tmp\CMake>git describe --tag
v3.11.4

D:\Repo\tmp\CMake>mkdir build && cd build

D:\Repo\tmp\CMake\build>cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Repo/tmp/CMake/build

D:\Repo\tmp\CMake\build>cmake --build .
[1/1] sphinx-build html: see Utilities/Sphinx/build-html.log
WARNING: while setting up extension sphinx.addnodes: node class 'meta' is already registered, its visitors will be overridden

After opening the HTML files generated, I found that the gray background of “Content” disappears for some pages:

Comapred with the CMake Docs 3.11.4 hosted in the cmake.org:

I believe this phenomenon is related to the above-mentioned warning. What happened?

I think I found the solution to this warning:

WARNING: while setting up extension sphinx.addnodes: node class 'meta' is already registered, its visitors will be overridden

It seems that this warning is caused by the version of docutils, which has to be <0.18. After I downgrade the docutils from 0.18.1 to 0.17.1, the warning won’t show up again.

  • Sphinx: 2.4.4
  • Docutils: 0.17.1
  • CMake Docs: v3.11.4

The followings are my demo commands:

pip install docutils==0.17.1
sphinx-build --version
git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
cd CMake
git checkout v3.11.4 --quiet
git describe --tag
mkdir build && cd build
cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
cmake --build .
Click to expand the full logs
D:\Repo\tmp>pip install docutils==0.17.1
Collecting docutils==0.17.1
  Using cached docutils-0.17.1-py2.py3-none-any.whl (575 kB)
Installing collected packages: docutils
  Attempting uninstall: docutils
    Found existing installation: docutils 0.18.1
    Uninstalling docutils-0.18.1:
      Successfully uninstalled docutils-0.18.1
Successfully installed docutils-0.17.1

D:\Repo\tmp>sphinx-build --version
sphinx-build 2.4.4

D:\Repo\tmp>git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
Cloning into 'CMake'...
remote: Enumerating objects: 79159, done.
remote: Counting objects: 100% (79159/79159), done.
remote: Compressing objects: 100% (37660/37660), done.
remote: Total 79159 (delta 52339), reused 62041 (delta 38463), pack-reused 0
Receiving objects: 100% (79159/79159), 48.72 MiB | 3.63 MiB/s, done.
Resolving deltas: 100% (52339/52339), done.
Updating files: 100% (22981/22981), done.

D:\Repo\tmp>cd CMake

D:\Repo\tmp\CMake>git checkout v3.11.4 --quiet

D:\Repo\tmp\CMake>git describe --tag
v3.11.4

D:\Repo\tmp\CMake>mkdir build && cd build

D:\Repo\tmp\CMake\build>cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Repo/tmp/CMake/build

D:\Repo\tmp\CMake\build>cmake --build .
[1/1] sphinx-build html: see Utilities/Sphinx/build-html.log

Reference:

As for the Extension Error that I mentioned at the beginning:

Extension error:
Could not import extension cmake (exception: No module named 'sphinx.util.pycompat')
ninja: build stopped: subcommand failed.

It seems that it’s because the pycompact module is deprecated at 4.0.0 and removed at 6.0.0.

Therefore, to remove this error, I just need to install the Sphinx with its version <6.0.0.

Testing with Sphinx-5.3.0 (Another extension error)

  • Sphinx: 5.3.0
  • CMake Docs: v3.18.6

If I install Sphinx with its versions >=4.0.0 and <6.0.0, there will be another extension error showed up:

Extension error:
Could not import extension cmake (exception: cannot import name 'htmlescape' from 'sphinx.util.pycompat' (C:\Python\Python310\lib\site-packages\sphinx\util\pycompat.py))
Click to expand the full logs
D:\Repo\tmp>sphinx-build --version
sphinx-build 5.3.0

D:\Repo\tmp>git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
Cloning into 'CMake'...
remote: Enumerating objects: 79159, done.
remote: Counting objects: 100% (79159/79159), done.
remote: Compressing objects: 100% (37660/37660), done.
remote: Total 79159 (delta 52339), reused 62041 (delta 38463), pack-reused 0
Receiving objects: 100% (79159/79159), 48.72 MiB | 3.41 MiB/s, done.
Resolving deltas: 100% (52339/52339), done.
Updating files: 100% (22981/22981), done.

D:\Repo\tmp>cd CMake

D:\Repo\tmp\CMake>git checkout v3.18.6 --quiet

D:\Repo\tmp\CMake>git describe --tag
v3.18.6

D:\Repo\tmp\CMake>mkdir build && cd build

D:\Repo\tmp\CMake\build>cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Repo/tmp/CMake/build

D:\Repo\tmp\CMake\build>cmake --build .
[1/1] sphinx-build html: see Utilities/Sphinx/build-html.log
FAILED: doc_format_html D:/Repo/tmp/CMake/build/doc_format_html
cmd.exe /C "cd /D D:\Repo\tmp\CMake\build && C:\Python\Python310\Scripts\sphinx-build.exe -c D:/Repo/tmp/CMake/build -d D:/Repo/tmp/CMake/build/doctrees -b html -A versionswitch=1 D:/Repo/tmp/CMake/Help D:/Repo/tmp/CMake/build/html > build-html.log"

Extension error:
Could not import extension cmake (exception: cannot import name 'htmlescape' from 'sphinx.util.pycompat' (C:\Python\Python310\lib\site-packages\sphinx\util\pycompat.py))
ninja: build stopped: subcommand failed.

Testing with Sphinx-3.5.4 (ImportError)

  • Sphinx: 3.5.4
  • CMake Docs: v3.18.6

If I install Sphinx with its versions >=3.0.0 and <4.0.0 , there will be an ImportError showed up:

ImportError: cannot import name 'Union' from 'types' (C:\Python\Python310\lib\types.py)
Click to expand the full logs
D:\Repo\tmp>sphinx-build --version
Traceback (most recent call last):
  File "C:\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Python\Python310\Scripts\sphinx-build.exe\__main__.py", line 4, in <module>
  File "C:\Python\Python310\lib\site-packages\sphinx\cmd\build.py", line 25, in <module>
    from sphinx.application import Sphinx
  File "C:\Python\Python310\lib\site-packages\sphinx\application.py", line 32, in <module>
    from sphinx.config import Config
  File "C:\Python\Python310\lib\site-packages\sphinx\config.py", line 23, in <module>
    from sphinx.util import logging
  File "C:\Python\Python310\lib\site-packages\sphinx\util\__init__.py", line 35, in <module>
    from sphinx.util import smartypants  # noqa
  File "C:\Python\Python310\lib\site-packages\sphinx\util\smartypants.py", line 33, in <module>
    from sphinx.util.docutils import __version_info__ as docutils_version
  File "C:\Python\Python310\lib\site-packages\sphinx\util\docutils.py", line 31, in <module>
    from sphinx.util.typing import RoleFunction
  File "C:\Python\Python310\lib\site-packages\sphinx\util\typing.py", line 34, in <module>
    from types import Union as types_Union
ImportError: cannot import name 'Union' from 'types' (C:\Python\Python310\lib\types.py)

Testing with Sphinx-2.4.5 (No error or warning)

  • Sphinx: 2.4.5
  • CMake Docs: v3.18.6

If I install Sphinx with its versions <3.0.0, there will be no error or warning showed up:

Click to expand the full logs
D:\Repo\tmp>sphinx-build --version
sphinx-build 2.4.5

D:\Repo\tmp>git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
Cloning into 'CMake'...
remote: Enumerating objects: 79159, done.
remote: Counting objects: 100% (79159/79159), done.
remote: Compressing objects: 100% (37660/37660), done.
remote: Total 79159 (delta 52339), reused 62041 (delta 38463), pack-reused 0
Receiving objects: 100% (79159/79159), 48.72 MiB | 4.72 MiB/s, done.
Resolving deltas: 100% (52339/52339), done.
Updating files: 100% (22981/22981), done.

D:\Repo\tmp>cd CMake

D:\Repo\tmp\CMake>git checkout v3.18.6 --quiet

D:\Repo\tmp\CMake>git describe --tag
v3.18.6

D:\Repo\tmp\CMake>mkdir build && cd build

D:\Repo\tmp\CMake\build>cmake ../Utilities/Sphinx -GNinja -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Repo/tmp/CMake/build

D:\Repo\tmp\CMake\build>cmake --build .
[1/1] sphinx-build html: see Utilities/Sphinx/build-html.log

Reference:

cc: @brad.king

Recently, I met a problem when I use sphinx <= 2.4.5 to build the documentation of cmake <= 3.8. That is, there are lots of warning messages showed up after completing the build, for example:

WARNING: 4 column based index found. It might be a bug of extensions you use: [('pair', 'variable ; GRAPHVIZ_GRAPH_TYPE', 'variable:GRAPHVIZ_GRAPH_TYPE', 'main')]
WARNING: 4 column based index found. It might be a bug of extensions you use: [('pair', 'variable ; GRAPHVIZ_GRAPH_NAME', 'variable:GRAPHVIZ_GRAPH_NAME', 'main')]
WARNING: 4 column based index found. It might be a bug of extensions you use: [('pair', 'variable ; GRAPHVIZ_GRAPH_HEADER', 'variable:GRAPHVIZ_GRAPH_HEADER', 'main')]

and the strange background color of the generated HTML files:

The followings are the commands I use:

pip install sphinx==2.4.5 docutils==0.17.1 jinja2==3.0.0
sphinx-build --version
git clone --depth 1 --no-single-branch https://github.com/Kitware/CMake.git
cd CMake
git checkout v3.5.2 --quiet
git describe --tag
mkdir build && cd build
cmake ../Utilities/Sphinx -DSPHINX_HTML=ON -DSPHINX_FLAGS="-A versionswitch=1"
cmake --build .

And the following is the log of the above commands:

log.txt (72.6 KB)

Do you know what happened? I guess it might be related to Sphinx’s dependencies. And the followings are the requirements when I install sphinx==2.4.5:

Requirement already satisfied: sphinx==2.4.5 in c:\python\python310\lib\site-packages (2.4.5)
Requirement already satisfied: docutils==0.17.1 in c:\python\python310\lib\site-packages (0.17.1)
Requirement already satisfied: jinja2==3.0.0 in c:\python\python310\lib\site-packages (3.0.0)
Requirement already satisfied: sphinxcontrib-applehelp in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (1.0.2)
Requirement already satisfied: sphinxcontrib-devhelp in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (1.0.2)
Requirement already satisfied: sphinxcontrib-jsmath in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (1.0.1)
Requirement already satisfied: sphinxcontrib-htmlhelp in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (2.0.0)
Requirement already satisfied: sphinxcontrib-serializinghtml in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (1.1.5)
Requirement already satisfied: sphinxcontrib-qthelp in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (1.0.3)
Requirement already satisfied: Pygments>=2.0 in c:\users\hwhsu1231\appdata\roaming\python\python310\site-packages (from sphinx==2.4.5) (2.14.0)
Requirement already satisfied: snowballstemmer>=1.1 in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (2.2.0)
Requirement already satisfied: babel!=2.0,>=1.3 in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (2.11.0)
Requirement already satisfied: alabaster<0.8,>=0.7 in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (0.7.12)
Requirement already satisfied: imagesize in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (1.4.1)
Requirement already satisfied: requests>=2.5.0 in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (2.28.1)
Requirement already satisfied: setuptools in c:\python\python310\lib\site-packages (from sphinx==2.4.5) (59.8.0)
Requirement already satisfied: packaging in c:\users\hwhsu1231\appdata\roaming\python\python310\site-packages (from sphinx==2.4.5) (23.0)
Requirement already satisfied: colorama>=0.3.5 in c:\users\hwhsu1231\appdata\roaming\python\python310\site-packages (from sphinx==2.4.5) (0.4.6)
Requirement already satisfied: MarkupSafe>=2.0.0rc2 in c:\python\python310\lib\site-packages (from jinja2==3.0.0) (2.1.1)
Requirement already satisfied: pytz>=2015.7 in c:\python\python310\lib\site-packages (from babel!=2.0,>=1.3->sphinx==2.4.5) (2022.6)
Requirement already satisfied: charset-normalizer<3,>=2 in c:\python\python310\lib\site-packages (from requests>=2.5.0->sphinx==2.4.5) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in c:\users\hwhsu1231\appdata\roaming\python\python310\site-packages (from requests>=2.5.0->sphinx==2.4.5) (2.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\python\python310\lib\site-packages (from requests>=2.5.0->sphinx==2.4.5) (1.26.12)
Requirement already satisfied: certifi>=2017.4.17 in c:\python\python310\lib\site-packages (from requests>=2.5.0->sphinx==2.4.5) (2022.9.14)