Since commit 1cdceae8e3 (GoogleTest: Parse discovered test list from
JSON output if supported, 2025-05-02, v4.2.0-rc1~533^2~2) we've placed
the discovery JSON files in the working directory provided by the user.
Place them in the current build tree folder instead.
Fixes: #27840
Introduce a mechanism (via the private and undocumented `ISSUE_WARNING`
option to the `cmake_policy` command) to allow built-in modules to issue
policy warnings. Update modules to use this.
This allows modules to issue policy warnings with the `CMD_POLICY`
diagnostic type, which is consistent with internally issued diagnostics.
This also tends to simplify the logic for issuing such warnings in much
the way the helpers introduced in the previous commit did for the C++
code.
Generate a script that contains a `gtest_discover_tests_impl` call
and use that script for both discovery modes: Either execute it via
`-P` for the POST_BUILD mode, or include it in the test script for
the PRE_TEST mode.
In fc7aa3cd69 (tests: Preserve empty arguments in test command lines,
2024-09-30), the CMP0174 policy setting had to be explicitly handled in
GoogleTestAddTests.cmake because it was part of the same release.
Now that we're past that release, we can remove that and update the
policies based on the CMake version instead. This removes the TODO
added in that original commit.
We also update the policy settings in GoogleTest.cmake so that we are
using consistent policy settings across the GoogleTest implementation.
For single config generators, don't write a file at configure time
as it will be rewritten at generation time.
For multi config generators, use `$<CONFIG>` in the output instead
of looping over `CMAKE_CONFIGURATION_TYPES`.
Covers almost all blocks containing actual code, except:
* Parsed-literal blocks can't be highlighted, including many command
summaries and substitution-heavy docs like find_... commands.
This is a Sphinx limitation.
* Code with errors, like CMP0049, DEPLOYMENT_ADDITIONAL_FILES,
DEPLOYMENT_REMOTE_DIRECTORY, @PACKAGE_INIT@ substitution in the
tutorial, bracket arguments/comments in cmake-language.7 and
cmake-developer.7.
* FindQt4 module, which needs reformatting.
Revise commit fc7aa3cd69 (tests: Preserve empty arguments in test
command lines, 2024-09-30, v3.31.0-rc1~33^2) to delay variable expansion
until code evaluation. Otherwise we may parse the working directory
path as CMake language code.
Test names with square brackets can't be safely parsed in the
list returned through TEST_LIST, so they are omitted from the
list. This was implemented in 8b85b10fb3 (GoogleTest: Add
handling for square brackets in test names, 2022-01-05), but the
documentation wasn't updated as part of that change.
This will now preserve empty values in the TEST_LAUNCHER and
CROSSCOMPILING_EMULATOR target properties for tests added by:
- The add_test() command.
- The ExternalData_Add_Test() command from the ExternalData module.
- The gtest_add_tests() or gtest_discover_tests() commands from the
GoogleTest module.
For the gtest_add_tests() and gtest_discover_tests() commands,
empty elements in the values passed after the EXTRA_ARGS keyword
are also now preserved.
Policy CMP0178 is added to provide backward compatibility with the
old behavior where empty values were silently discarded from the
above cases.
Fixes: #26337
The prose previously gave the impression that discovery was always done
as a post-build operation, but later in the documentation the
DISCOVERY_MODE option is explained where pre-test can also be specified.
Update the earlier prose to mention both methods equally.
- Use an "arg" prefix for cmake_parse_arguments() rather than
an empty string or uppercase ARGS so that usages of parsed
values don't start with a bare underscore or look fully
uppercase. This makes it harder to miss them or mistake
them for keywords.
- Use lowercase for names of positional function arguments
so they don't get mistaken for keywords.
We run test executables to discover tests. Use the same launchers that
are used to run the tests. We already handle `CROSSCOMPILING_EMULATOR`.
Update the logic to account for the `TEST_LAUNCHER` property added by
commit 1ec0372ed4 (add_test: Optionally use a launcher for tests running
in-project targets, 2023-11-11), and for the `CROSSCOMPILING_EMULATOR`
behavior change in commit ca5a300d7f (add_test: Honor
CROSSCOMPILING_EMULATOR only when cross-compiling, 2023-11-02).
Fixes: #25603
Co-authored-by: Brad King <brad.king@kitware.com>
Signed-off-by: Ralf Habacker <ralf.habacker@freenet.de>
The internal helper variable '_GOOGLETEST_DISCOVER_TESTS_SCRIPT' can have gone
out-of-scope when 'gtest_discover_tests()' is called, depending on where the
GoogleTest module is actually included. This leads to a silent failure of
dynamic test discovery, since the custom post-build commands actually does
nothing (it basically invokes 'cmake -P ""'). Ctest will then fail to run the
tests, considering them to be 'not built'.
Fix this by determining the path to the GoogleTest module based on
'${CMAKE_ROOT}' instead, which is always available.
A new test case was added to test suite 'RunCMake/GoogleTest' to ensure that
'gtest_discover_tests()' works correctly when invoked in a different variable
scope.
Fixes: #25477
d4d1f85297 Help: Move a note about arguments and genexes into include file
968224484f Help: Drop unused `PROPERTY_GENEX` replace from `prop_tgt/INTERFACE_*` files
65457e4fae Help: Add _See Also_ sections to related CMake commands
c86ec79d07 Help: Use direct refs to genexes instead of ``$<GENEX-NAME>``
8c6274326d Help: Use CMake highlighting for command synopsis
c356a94512 Help: Use monospaced font for a genex name in a section title
f98c8707f4 Help: Use `code-block:: cmake` instead of `::` for sub-commands descriptions
2ba351ca4f Help: ``CMake`` → CMake
...
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !7873
The `TEST_FILTER` argument can be used to filter tests during the
discovery phase. It combines `--gtest_filter=<expr>` with the
`--gtest_list_tests` argument when invoking the test excutable for
listing defined tests.
Fixes: #17493
When using a file system which only has second resolution timestamps,
there is a reasonably high likelihood of timestamps being the same.
The IS_NEWER_THAN test returns true when timestamps are the same,
so don't redo test discovery when they match exactly.
The generated ..._include.cmake file contained an inclusion of
GoogleTestAddTests without any path. In general, this is a good
approach since it enables to correctly catch possibly
user-customized files.
However, in this case, it didn’t work this way since the
..._include.cmake file is evaluated by a separate CMake call under
a custom command. Because of this, the CMAKE_MODULE_PATH is not set
as expected by the user and the said inclusion catches CMake own
version of the module rather than the possibly user-customized one.
This change fixes this by making the inclusion with an absolute path
determined upon the ..._include.cmake file generation.
Introducing a new DISCOVERY_MODE mode option, which provides greater control
over when gtest_discover_tests perforsm test discovery.
It has two supported modes:
* POST_BUILD
* PRE_TEST
POST_BUILD is the default behavior, which adds a POST_BUILD command
to perform test discovery after the test has been built.
PRE_TEST is a new mode, which delays test discovery until test execution.
DISCOVERY_MODE can be controlled in two ways:
1. Setting the DISCOVERY_MODE when calling gtest_discover_tests
2. Setting the global CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE
prior to calling gtest_discover_tests
By delaying test discovery until ctest runtime,
we can provide better cross compile support,
because we're more likely to be in an environment that can run the test executable.
This was achieved by moving the command for test discovery
into the generated ctest include file.
In PRE_TEST mode, the generated include file now has the form:
if(EXISTS "path/to/test.exe")
if("path/to/test.exe" IS_NEWER_THAN "path/to/ctest_file")
// command to discover tests
// and generate ctest_file
endif()
include("path/to/ctest_file")
endif()
elseif()
// test not built
endif()
Which generates the appropriate CTest files at runtime
and regenerates the CTest files when the executable is updated.
In the old approach, test discovery was always added as POST_BUILD step
and the executable was ran in order to scan for tests.
If the test executable failed to run for any reason,
this resulted in a link failure.
This failure could more commonly occur when cross compiling,
because your build environment might not have
the appropriate runtime environment dlls required to run the test executable.
I also ran into the issue when compiling a Qt application using Qt Creator.
Qt Creator manages its build and runtime environments separately
and only adds the Qt dlls to the environment during runtime -- not during build.
So when gtest_discover_tests ran my test executable,
it promptly crashed because the environment wasn't correct,
which resulted in a build failure.
Setting the DISCOVERY_MODE to PRE_TEST fixed my build failure
by delaying test discovery until runtime,
because this allowed the test exe to run in the proper environment.
A few non-trivial implementation details worth noting:
1. bracket_arguments
In the PRE_TEST side, parameters whose contents might contain special characters,
need to be handled with great care.
To aid in escaping these list arguments, backslashes, and other special characters,
and ensuring that they are preserved in the generated file,
bracket arguments (i.e. [== <...> ==]) are used.
For example:
gtest_discover_tests(
...
EXTRA_ARGS how now "\"brown\" cow"
)
Generates a file with the following call:
gtest_discover_tests_impl(
...
TEST_EXTRA_ARGS [==[how;now;"brown" cow]==]
)
This way the arguments are forwarded correctly.
2. multi-config generators
Multi-Config generators (e.g. MSBuild) now work more correctly in
PRE_TEST than POST_BUILD.
PRE_TEST is more correct because it will generate a unique CTest file for each configuration.
It generates a per config file responsible for test discovery:
foo[1]_include-Debug.cmake
foo[1]_include-MinSizeRel.cmake
foo[1]_include-Release.cmake
foo[1]_include-RelWithDebInfo.cmake
A per config file for containing the google tests:
foo[1]_tests-Debug.cmake
And an outer ctest file:
foo[1]_include-Debug.cmake
That is generically written to include the correct configuration file
by looking at the value of ${CTEST_CONFIGURATION_TYPE} when CTest runs.
POST_BUILD, in contrast, preserves the existing functionality.
Tests are disocvered based on the last configuration that was chosen
and only a single file is produced:
foo[1]_include.cmake
foo[1]_tests.cmake
But it runs with whatever executable requested by ctest --config,
which means it's possible to run the wrong tests
if some tests were enabled in one configuration but not another.
When executing googltests in parallel using 'ctest -j n' and using
'--gtest_output=xml' there is a race condition upon file creation.
See googletest issue https://github.com/google/googletest/issues/2506.
As all testcases (potentially) can be run in parallel each testcase has
to create it's own XML JUnit file. EXTRA_ARGS is not suitable because it
is identical per testsuite.
So instead a new (opitional) parameter has been introduced to specify
the storage location for each test of the testsuite.
If policy settings at the time the GoogleTest module is included are
such that CMP0057 is unset or set to OLD, the use of IN_LIST with
if() will lead to an error. Therefore, explicitly specify the policy
settings for the whole file to ensure the function implementations
have access to the required CMake features.
Fixes: #18198
In gtest_discover_tests(), the TIMEOUT keyword was making it
impossible to set the TIMEOUT test property via the PROPERTIES
keyword. This would be a frequent case, but it doesn't complain
and instead silently does something different to what would
normally be expected. The TIMEOUT keyword has been renamed
to DISCOVERY_TIMEOUT, thereby removing the clash.
This is a breaking change. 3.10.1 and 3.10.2 were the only versions
that supported the TIMEOUT keyword and uses of it were likely
not working as intended.
Fixes: #17801
Add a TIMEOUT option to gtest_discover_tests. This provides a
work-around in case a test goes out to lunch, rather than causing the
build to simply hang. (Although this is still a problem with the user's
project, hanging the build is not cool, especially in the case of
automatically running CI builds. It is much preferred that the build
should actively fail in this case, and it is trivially easy for us to
implement that.)
