dilim/CMake
1
0
Fork 0
mirror of https://github.com/Kitware/CMake.git synced 2026-08-03 14:20:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

CTest: Make ctest_test() label filters multi-value (AND)

Browse Source 
The ctest(1) command line allows repeating -L/-LE to form an AND
filter over test labels, but the ctest_test() and ctest_memcheck()
scripting commands accepted only a single INCLUDE_LABEL/EXCLUDE_LABEL
value, so a dashboard script could not express that filter.

The label-matching engine already stores the include/exclude
expressions as vectors and applies them with AND semantics; only the
command binding was single-value.  Change both keywords to multi-value
(NonEmpty<vector<string>>) and hand the collected list straight to the
engine.  ctest_memcheck() gains the same behavior for free, since it
inherits the option set.

Fixes : #27497
This commit is contained in:
Daksh Mamodiya
2026-07-24 21:22:02 +02:00
parent 3eda12235e
commit 257865e1d4
19 changed files with 160 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
Help/command/ctest_memcheck.rst
View File

@@ -11,8 +11,8 @@ Perform the :ref:`CTest MemCheck Step` as a :ref:`Dashboard Client`.
[STRIDE <stride-number>]
[EXCLUDE <exclude-regex>]
[INCLUDE <include-regex>]
[EXCLUDE_LABEL <label-exclude-regex>]
[INCLUDE_LABEL <label-include-regex>]
[EXCLUDE_LABEL <label-exclude-regex>...]
[INCLUDE_LABEL <label-include-regex>...]
[EXCLUDE_FIXTURE <regex>]
[EXCLUDE_FIXTURE_SETUP <regex>]
[EXCLUDE_FIXTURE_CLEANUP <regex>]

26
Help/command/ctest_test.rst
View File

@@ -11,8 +11,8 @@ Perform the :ref:`CTest Test Step` as a :ref:`Dashboard Client`.
[STRIDE <stride-number>]
[EXCLUDE <exclude-regex>]
[INCLUDE <include-regex>]
[EXCLUDE_LABEL <label-exclude-regex>]
[INCLUDE_LABEL <label-include-regex>]
[EXCLUDE_LABEL <label-exclude-regex>...]
[INCLUDE_LABEL <label-include-regex>...]
[EXCLUDE_FROM_FILE <filename>]
[INCLUDE_FROM_FILE <filename>]
[EXCLUDE_FIXTURE <regex>]
@@ -69,13 +69,31 @@ The options are:
Specify a regular expression matching test names to include.
Tests not matching this expression are excluded.
``EXCLUDE_LABEL <label-exclude-regex>``
``EXCLUDE_LABEL <label-exclude-regex>...``
Specify a regular expression matching test labels to exclude.
``INCLUDE_LABEL <label-include-regex>``
.. versionchanged:: 4.5
More than one ``<label-exclude-regex>`` may be given, in which case a
test is excluded only if each regular expression matches at least one
of the test's labels (i.e. the expressions form an ``AND`` relationship).
See the :ref:`Label Matching` section of the :manual:`ctest(1)` manual.
This mirrors the behavior of repeating the
:option:`ctest --label-exclude` command-line option.
``INCLUDE_LABEL <label-include-regex>...``
Specify a regular expression matching test labels to include.
Tests not matching this expression are excluded.
.. versionchanged:: 4.5
More than one ``<label-include-regex>`` may be given, in which case a
test is included only if each regular expression matches at least one
of the test's labels (i.e. the expressions form an ``AND`` relationship).
See the :ref:`Label Matching` section of the :manual:`ctest(1)` manual.
This mirrors the behavior of repeating the
:option:`ctest --label-regex` command-line option.
``EXCLUDE_FROM_FILE <filename>``
.. versionadded:: 3.29

10
Help/release/dev/ctest-test-label-and.rst Normal file
View File

@@ -0,0 +1,10 @@
ctest-test-label-and
---------------------
* The :command:`ctest_test` and :command:`ctest_memcheck` commands gained the
ability to accept more than one value for their ``INCLUDE_LABEL`` and
``EXCLUDE_LABEL`` options. When multiple regular expressions are given, a
test is selected only if each expression matches at least one of the test's
labels (i.e. the expressions form an ``AND`` relationship), matching the
behavior of repeating the :option:`ctest --label-regex` and
:option:`ctest --label-exclude` command-line options.

7
Source/CTest/cmCTestTestCommand.cxx
View File

@@ -176,13 +176,10 @@ std::unique_ptr<cmCTestGenericHandler> cmCTestTestCommand::InitializeHandler(
handler->TestOptions.IncludeRegularExpression = args.Include;
}
if (!args.ExcludeLabel.empty()) {
handler->TestOptions.ExcludeLabelRegularExpression.clear();
handler->TestOptions.ExcludeLabelRegularExpression.push_back(
args.ExcludeLabel);
handler->TestOptions.ExcludeLabelRegularExpression = args.ExcludeLabel;
}
if (!args.IncludeLabel.empty()) {
handler->TestOptions.LabelRegularExpression.clear();
handler->TestOptions.LabelRegularExpression.push_back(args.IncludeLabel);
handler->TestOptions.LabelRegularExpression = args.IncludeLabel;
}
if (!args.ExcludeTestsFromFile.empty()) {

5
Source/CTest/cmCTestTestCommand.h
View File

@@ -7,6 +7,7 @@
#include <memory>
#include <string>
#include <type_traits>
#include <vector>
#include <cm/optional>
#include <cmext/string_view>
@@ -31,8 +32,8 @@ protected:
std::string Stride;
std::string Exclude;
std::string Include;
std::string ExcludeLabel;
std::string IncludeLabel;
ArgumentParser::NonEmpty<std::vector<std::string>> ExcludeLabel;
ArgumentParser::NonEmpty<std::vector<std::string>> IncludeLabel;
std::string IncludeTestsFromFile;
std::string ExcludeTestsFromFile;
std::string ExcludeFixture;

10
Tests/RunCMake/ctest_memcheck/LabelAndExclude-stdout.txt Normal file
View File

@@ -0,0 +1,10 @@
Memory check project [^
]*/LabelAndExclude-build
Start [0-9]+: RunCMake
[0-9]+/3 MemCheck #[0-9]+: RunCMake \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test1
[0-9]+/3 MemCheck #[0-9]+: test1 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test2
[0-9]+/3 MemCheck #[0-9]+: test2 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 3

6
Tests/RunCMake/ctest_memcheck/LabelAndInclude-stdout.txt Normal file
View File

@@ -0,0 +1,6 @@
Memory check project [^
]*/LabelAndInclude-build
Start [0-9]+: test3
1/1 MemCheck #[0-9]+: test3 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 1

8
Tests/RunCMake/ctest_memcheck/LabelAndIncludeSingle-stdout.txt Normal file
View File

@@ -0,0 +1,8 @@
Memory check project [^
]*/LabelAndIncludeSingle-build
Start [0-9]+: test1
[0-9]+/2 MemCheck #[0-9]+: test1 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test3
[0-9]+/2 MemCheck #[0-9]+: test3 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 2

23
Tests/RunCMake/ctest_memcheck/RunCMakeTest.cmake
View File

@@ -238,3 +238,26 @@ block()
"PRESET my-include-preset PRESETS_FILE /nonexistent/path/presets.json")
run_mc_test(TestPresetBadFile "${PSEUDO_VALGRIND}")
endblock()
#-----------------------------------------------------------------------------
# Verify that INCLUDE_LABEL / EXCLUDE_LABEL accept more than one value that
# form an AND relationship, and that the change propagates to ctest_memcheck().
block()
set(CMAKELISTS_EXTRA_CODE [[
add_test(NAME test1 COMMAND ${CMAKE_COMMAND} -E true)
set_property(TEST test1 PROPERTY LABELS foo)
add_test(NAME test2 COMMAND ${CMAKE_COMMAND} -E true)
set_property(TEST test2 PROPERTY LABELS bar)
add_test(NAME test3 COMMAND ${CMAKE_COMMAND} -E true)
set_property(TEST test3 PROPERTY LABELS foo bar)
]])
set(CTEST_MEMCHECK_ARGS "INCLUDE_LABEL foo bar")
run_mc_test(LabelAndInclude "${PSEUDO_VALGRIND}")
set(CTEST_MEMCHECK_ARGS "EXCLUDE_LABEL foo bar")
run_mc_test(LabelAndExclude "${PSEUDO_VALGRIND}")
set(CTEST_MEMCHECK_ARGS "INCLUDE_LABEL foo")
run_mc_test(LabelAndIncludeSingle "${PSEUDO_VALGRIND}")
unset(CTEST_MEMCHECK_ARGS)
endblock()

1
Tests/RunCMake/ctest_test/LabelAndDuplicate-result.txt Normal file
View File

@@ -0,0 +1 @@
(-1|255)

3
Tests/RunCMake/ctest_test/LabelAndDuplicate-stderr.txt Normal file
View File

@@ -0,0 +1,3 @@
CMake Error at [^
]*/test\.cmake:[0-9]+ \(ctest_test\):
ctest_test called with more than one value for INCLUDE_LABEL

8
Tests/RunCMake/ctest_test/LabelAndExclude-stdout.txt Normal file
View File

@@ -0,0 +1,8 @@
Start [0-9]+: RunCMakeVersion
[0-9]+/3 Test #[0-9]+: RunCMakeVersion \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test1
[0-9]+/3 Test #[0-9]+: test1 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test2
[0-9]+/3 Test #[0-9]+: test2 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 3

10
Tests/RunCMake/ctest_test/LabelAndExcludeEmptyClears-stdout.txt Normal file
View File

@@ -0,0 +1,10 @@
Start [0-9]+: RunCMakeVersion
[0-9]+/4 Test #[0-9]+: RunCMakeVersion \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test1
[0-9]+/4 Test #[0-9]+: test1 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test2
[0-9]+/4 Test #[0-9]+: test2 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test3
[0-9]+/4 Test #[0-9]+: test3 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 4

4
Tests/RunCMake/ctest_test/LabelAndInclude-stdout.txt Normal file
View File

@@ -0,0 +1,4 @@
Start [0-9]+: test3
1/1 Test #[0-9]+: test3 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 1

10
Tests/RunCMake/ctest_test/LabelAndIncludeEmptyClears-stdout.txt Normal file
View File

@@ -0,0 +1,10 @@
Start [0-9]+: RunCMakeVersion
[0-9]+/4 Test #[0-9]+: RunCMakeVersion \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test1
[0-9]+/4 Test #[0-9]+: test1 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test2
[0-9]+/4 Test #[0-9]+: test2 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test3
[0-9]+/4 Test #[0-9]+: test3 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 4

6
Tests/RunCMake/ctest_test/LabelAndIncludeSingle-stdout.txt Normal file
View File

@@ -0,0 +1,6 @@
Start [0-9]+: test1
[0-9]+/2 Test #[0-9]+: test1 \.+ +Passed +[0-9\.]+ sec
Start [0-9]+: test3
[0-9]+/2 Test #[0-9]+: test3 \.+ +Passed +[0-9\.]+ sec
+
100% tests passed out of 2

1
Tests/RunCMake/ctest_test/LabelAndMissingValue-result.txt Normal file
View File

@@ -0,0 +1 @@
(-1|255)

3
Tests/RunCMake/ctest_test/LabelAndMissingValue-stderr.txt Normal file
View File

@@ -0,0 +1,3 @@
CMake Error at [^
]*/test\.cmake:[0-9]+ \(ctest_test\):
Error after keyword "INCLUDE_LABEL":

28
Tests/RunCMake/ctest_test/RunCMakeTest.cmake
View File

@@ -418,6 +418,34 @@ set_property(TEST b PROPERTY LABELS b)
endfunction()
run_changing_labels()
# Verify that INCLUDE_LABEL / EXCLUDE_LABEL accept more than one value that
# form an AND relationship, matching the `ctest -L` / `ctest -LE` behavior.
function(run_label_and)
set(CASE_CMAKELISTS_SUFFIX_CODE [[
add_test(NAME test1 COMMAND ${CMAKE_COMMAND} -E true)
set_property(TEST test1 PROPERTY LABELS foo)
add_test(NAME test2 COMMAND ${CMAKE_COMMAND} -E true)
set_property(TEST test2 PROPERTY LABELS bar)
add_test(NAME test3 COMMAND ${CMAKE_COMMAND} -E true)
set_property(TEST test3 PROPERTY LABELS foo bar)
]])
run_ctest_test(LabelAndInclude INCLUDE_LABEL foo bar)
run_ctest_test(LabelAndExclude EXCLUDE_LABEL foo bar)
run_ctest_test(LabelAndIncludeSingle INCLUDE_LABEL foo)
run_ctest_test(LabelAndMissingValue INCLUDE_LABEL)
run_ctest_test(LabelAndDuplicate INCLUDE_LABEL foo INCLUDE_LABEL bar)
# An explicit empty-string label argument clears a filter pre-seeded on the
# `ctest` command line, and thus runs every test.
set(CASE_CTEST_TEST_RAW_ARGS "INCLUDE_LABEL \"\"")
run_ctest(LabelAndIncludeEmptyClears -L "^foo$")
unset(CASE_CTEST_TEST_RAW_ARGS)
set(CASE_CTEST_TEST_RAW_ARGS "EXCLUDE_LABEL \"\"")
run_ctest(LabelAndExcludeEmptyClears -LE "^foo$")
unset(CASE_CTEST_TEST_RAW_ARGS)
endfunction()
run_label_and()
# Verify that test output can add additional labels
function(run_extra_labels)
set(CASE_CMAKELISTS_SUFFIX_CODE [[