Accept -n as a short option for --dry-run in bootctl,
systemd-oomd, systemd-sysusers, and systemd-tmpfiles.
For systemd-repart, make -n equivalent to --dry-run=yes,
while keeping --dry-run=BOOL available.
Follow-up for: 2479f0bb09
The `Domains=` option in the `[Network]` section did not document its
behaviour when specified repeatedly. In practice the option is additive
(each occurrence accumulates search/routing domains) and assigning an
empty string resets the list, matching the closely related `DNS=`
option. This is implemented by `config_parse_domains()` in
`src/network/networkd-dns.c`, which frees both the search and route
domain sets on an empty `rvalue` and otherwise inserts each
whitespace-separated entry into the corresponding set.
Document this explicitly, using the same wording already used for
`DNS=` in the same man page, so users know repeated assignments are
combined and that an empty value clears them.
Fixes#38740.
- sysupdate: In the auto-enable service, don't enable all features
The auto-enable service should activate suggested components and
features but enabled all features which includes the default components
unsuggested features and any unsuggested features of the suggested
components. This is unexpected behavior and we rather want this service
to be limited to suggested features.
Switch the service flag to suggested and make the wording more explicit
in the man page. Also fix the wrong statement that it operates on
enabled components, it operates on all components, also explicitly
disabled ones.
- sysupdate: Change disabling with
--component-suggested/--feature-suggested
The disabling of features or components with the flag
--component-suggested/--feature-suggested didn't disable the suggested
ones but instead disabled all other ones. This is rather unintuitive due
to how the flags are named and also not really needed because the
intended reconciliation outcome can instead be done by first disabling
everything and then enabling the suggested ones again which is easier to
reason about. For components the tricky part is that they default to
enabled and thus it's better to have the disable/enable commands with
--component-suggested operate only on suggested ones instead of touching
others like "legacy" components that don't explicity say whether they
are enabled and suggested or not.
Make running disablement of components/features with
--component-suggested/--feature-suggested undo a previous enablement
with the same flags. Document how one can align the system to only use
suggested components/features and not anything else by doing it in two
steps, first disabling everything and then enabling suggested ones. This
also makes it clearer now that all components that are not explicitly
enabled nor suggested will be disabled then.
The disabling of features or components with the flag
--component-suggested/--feature-suggested didn't disable the suggested
ones but instead disabled all other ones. This is rather unintuitive due
to how the flags are named and also not really needed because the
intended reconciliation outcome can instead be done by first disabling
everything and then enabling the suggested ones again which is easier to
reason about. For components the tricky part is that they default to
enabled and thus it's better to have the disable/enable commands with
--component-suggested operate only on suggested ones instead of touching
others like "legacy" components that don't explicity say whether they
are enabled and suggested or not.
Make running disablement of components/features with
--component-suggested/--feature-suggested undo a previous enablement
with the same flags. Document how one can align the system to only use
suggested components/features and not anything else by doing it in two
steps, first disabling everything and then enabling suggested ones. This
also makes it clearer now that all components that are not explicitly
enabled nor suggested will be disabled then.
NV indexes created in the storage hierarchy can be undefined and
redefined with TPM owner auth. Because of this, NvPCRs need some way
to prevent them from being redfined in a way that allows spoof
measurements to be replayed.
The current approach requires knowledge of a secret ("anchor secret")
in order to derive the initial NvPCR measurement and to derive a
measurement to an existing PCR (9). The credential is protected by the
TPM with a PCR policy. Without access to the credential, it's not
possible to replay measurements to a newly defined NvPCR without
breaking the binding with the measurement in PCR 9. However, this
approach has a couple of issues:
- The credential is currently only protected by PCR11. As it's not
protected by the rest of the boot chain, it's possible to boot other
operating systems in order to replay the PCR11 measurements and
recover the secret. Note that as the NvPCR anchoring happens in early
boot, the credential is stored in the ESP.
- Someone with privileged access to a system can just create a new
credential containing a known secret and store this in /var/lib and
the ESP. The NvPCRs are anchored with this known secret on subsequent
boots, and therefore the measurements can no longer be trusted.
Imagine the scenario where privileged access is theoretically possible
as a result of some vulnerability. After upgrading the system to fix
this vulnerability, the system should be able to attest that it is
now in a good state. However, if an adversary were able to use their
priviliges to replace the credential, they are able to obtain
persistence and the NvPCR measurements are no longer trustworthy.
This PR changes things to take a different approach. Instead of
requiring knowledge of a secret, the NvPCRs are now created in a way
that requires a policy to be satisfied for writing. The write policy has
2 branches:
- TPM2_PolicyNvWritten(true), which can be satisified without any
further authorization if the NvPCR has already been extended.
- TPM2_PolicyAuthorize(pcrPubKey, SHA256("nvpcr-init")) which can be
satisfied with a signed PCR policy, and must be used to perform the
initial extend to a NvPCR.
The intention here is that the signed PCR policy that can be used to
authorize the initial extend to the NvPCR can only be satisfied during
early boot. During later boot phases, this signed PCR policy must not be
valid. This means that if a NvPCR is undefined and redefined, it won't
be possible to satisfy its write policy in order to able to perform the
initial extend.
In order to anchor the NvPCRs and prevent them from being undefined and
then redefined with a different policy that does allow them to be
extended, the names of the NvPCRs are measured to PCR9. Verifiers must
check that the names of attested NvPCRs match the measurements in PCR9.
This uses the PCR signing key from the currently booted UKI to create
the NvPCRs. If this changes between boots, then tpm2-setup automatically
recreates new NvPCRs with an updated write policy to reflect this. I've
tried to be careful to not undefine arbitrary NV indexes in this case,
so it checks that the existing NV index looks like a NvPCR (ie, it has
the expected attributes) before undefining it.
I did originally try to preserve the old behaviour for existing systems,
but it makes things a lot more complicated. As the new implementation
already creates new NvPCRs when the PCR signing key changes, I ended up
just automatically upgrading the old NvPCRs as well. Again, I check here
that any existing NV index looks like an old style NvPCR (ie, it has the
expected attributes) before undefining it.
I did notice that the initial NvPCR measurement isn't going into the
log. I don't know if that was an intentional choice, but I've preserved
that behaviour in this PR.
This also adds a new option to ukify (--sign-initrd-pcrs) which creates
signed policies (one per PCR bank) that can only be satisfied from the
initrd. These policies are used for initializing the NvPCRs, but can also
be used for protecting TPM2 keyslots enrolled with systemd-cryptenroll
(by using the --tpm2-public-key-policyref=initrd option).
There is one outstanding issue. The NvPCR definitions support different
algorithms, but the use of PolicyAuthorize means that they can only support
SHA-256 for now. This is because the signed policy algorithm must match
the name algorithm, and some additional work is required to support
signed PCR policies for algorithms other than SHA256. I've left a note in
tpm2_nvpcr_initialize that details what's required, and I'll take a look
at that in a subsequent PR.
The auto-enable service should activate suggested components and
features but enabled all features which includes the default components
unsuggested features and any unsuggested features of the suggested
components. This is unexpected behavior and we rather want this service
to be limited to suggested features.
Switch the service flag to suggested and make the wording more explicit
in the man page. Also fix the wrong statement that it operates on
enabled components, it operates on all components, also explicitly
disabled ones.
systemd-escape currently only processes strings passed as
command line arguments. This is awkward for callers that already
have a generated list of strings, because they need to loop around
the tool or use xargs and carefully preserve whitespace and other
special characters.
Add --stdin to read one string per line from standard input and
write one escaped result per output line. Keep command line strings
mutually exclusive with --stdin so the input source remains
unambiguous.
Use an explicit option instead of treating '-' specially, since '-'
is itself a valid string to escape. The existing escape, unescape,
mangle, path, suffix, and template rules are reused unchanged.
Previously, specifying multiple stacked netdevs of the same type required
repeating the corresponding setting, e.g.:
```
[Network]
VLAN=vlan_10
VLAN=vlan_20
VLAN=vlan_30
```
With this change, the same configuration can be written as:
```
[Network]
VLAN=vlan_10 vlan_20 vlan_30
```
Specifying an empty string now also clears all previously assigned stacked
netdevs.
Closes#43103.
status with --static, --pretty, or --transient prints only the
selected hostname in regular output. JSON status output is built from
hostnamed's Describe data and remains a full status object.
Document this distinction so callers do not expect the name type
switches to turn JSON status into a single scalar value.
Follow-up for: ac8a4f6b78
The ExitType= documentation stated that ExitType=main "cannot be used
with Type=oneshot", which is incorrect: ExitType=main is the default
and works fine with Type=oneshot. The restriction actually enforced by
the code is that ExitType=cgroup is refused for Type=oneshot services
(see service_verify() in src/core/service.c).
Move the "cannot be used with Type=oneshot" note from the ExitType=main
bullet to the ExitType=cgroup bullet where it belongs.
Fixes#42327
Signed-off-by: Liu Zheng <liuzheng@uniontech.com>
The normal order of operations when iterating the GLib main loop is:
- prepare() -> sd_event_prepare()
- poll the file descriptors
- check() -> sd_event_wait()
- dispatch() -> sd_event_dispatch()
GLib does not guarantee that check() is always called between two
consecutive prepare() invocations. When another thread attaches or
removes a source with poll fds while the main thread is inside
poll(), g_main_context_check_unlocked() returns immediately without
calling any source's check() callback.
Since the sd_event GSource adapter maps prepare/check/dispatch to
sd_event_prepare/sd_event_wait/sd_event_dispatch, a skipped check()
leaves sd_event in ARMED state. The next prepare() then hits
assertion "e->state == SD_EVENT_INITIAL".
The following program reproduces the faulty scenario by creating a
thread that attaches a new fd during the poll phase:
#include <stdio.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <glib.h>
#include <systemd/sd-event.h>
#include <unistd.h>
/* from glib-event-glue.c */
extern GSource *g_sd_event_create_source(sd_event *event);
static gpointer thread_func(gpointer user_data) {
GSource *source;
GPollFD pollfd;
int fd;
g_usleep(G_USEC_PER_SEC / 10);
fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
if (fd < 0)
abort();
source = g_source_new(&(GSourceFuncs){0}, sizeof(GSource));
pollfd.fd = fd;
pollfd.events = G_IO_IN;
pollfd.revents = 0;
g_source_add_poll(source, &pollfd);
g_source_attach(source, NULL);
g_source_destroy(source);
g_source_unref(source);
close(fd);
return NULL;
}
int main(int argc, char *argv[]) {
sd_event *event = NULL;
GSource *source;
GThread *thread;
int i, r;
r = sd_event_default(&event);
if (r < 0)
return 1;
source = g_sd_event_create_source(event);
if (!source)
return 1;
g_source_attach(source, NULL);
for (i = 0; i < 50; i++) {
thread = g_thread_new("check-skip", thread_func, NULL);
g_main_context_iteration(NULL, TRUE);
g_thread_join(thread);
g_main_context_iteration(NULL, FALSE);
}
g_source_destroy(source);
g_source_unref(source);
sd_event_unref(event);
return 0;
}
When the program uses the current glue code, it crashes with:
Assertion 'e->state == SD_EVENT_INITIAL' failed at src/libsystemd/sd-event/sd-event.c:4560, function sd_event_prepare(). Aborting.
Fix the problem by skipping sd_event_prepare() if the event is
already ARMED: it was already prepared in a previous iteration and
it can be polled immediately.
Nowadays, varlink is used to control systemd-udevd. Let's drop the
legacy socket.
Note, the existence of /run/udev/control socket is widely used in both
our code and external projects. Also, the dependency to
systemd-udevd-control.socket is widely used in many projects. Hence, we
need to create a symlink to the socket file and .socket unit file.
Add a new --no-reconfigure flag to 'networkctl reload' that reloads
.network and .netdev files from disk without reconfiguring any network
interfaces. This may be useful to avoid reconfiguring multiple interfaces
simultaneously when multiple .network files are updated, or when an updated
.network file is applied to multiple interfaces.
On the networkd side, manager_reload() gains a reconfigure_links parameter
that gates the per-link reconfiguration loop. A new io.systemd.Network.Reload
varlink method is added that exposes this as an optional reconfigureLinks
boolean (defaults to true). Both plain 'networkctl reload' and
'--no-reconfigure' now unconditionally call this method first. If an older
networkd returns MethodNotFound, plain reload falls back to
io.systemd.service.Reload for backward compatibility; '--no-reconfigure'
fails with a clear error in that case.
Nowadays, varlink is used to control systemd-udevd. Let's drop the
legacy socket.
Note, the existence of /run/udev/control socket is widely used in both
our code and external projects. Also, the dependency to
systemd-udevd-control.socket is widely used in many projects.
Hence, we need to create a symlink to the socket file and .socket unit
file.
For various cases it is interesting to both sign and encrypted a file
system, for example to prepare it on one host and provide it to another.
Let's explicitly support preparing this in systemd-repart via setting
both Verity= and Encrypt=.
This adds an IPv4 counterpart to `IPv6ProxyNDPAddress=` for adding
manual entries to the kernel's IPv4 neighbour proxy table (check via
`ip -4 neighbour show proxy dev <dev>`). systemd-networkd only exposed
`IPv4ProxyARP=` for per-interface `proxy_arp` sysctl (automatic proxy
ARP) with no way to manage manual entries from a .network file.
To avoid duplicating the IPv6 proxy NDP code path, both families are
now combined into a single new `networkd-neighbor-proxy` module. The
IPv6 behaviour is preserved: `IPv6ProxyNDPAddress=` still implies
`IPv6ProxyNDP=yes` unless `IPv6ProxyNDP=` is explicitly disabled and
entries are still dropped if the kernel has no IPv6 support.
The same rule is applied to `IPv4ProxyARPAddress=`. It implies
`IPv4ProxyARP=yes` when the sysctl is not explicitly set and has no
effect if `IPv4ProxyARP=` has been set to false.
This keeps the user model symmetric and predictable across both
families: a single per-address setting that turns on the matching
per-interface sysctl automatically, while still letting system
administrators opt out by setting the boolean explicitly to false.
Note that the IPv4 manual NTF_PROXY entries installed here would
actually function without `proxy_arp` (unlike IPv6, where `proxy_ndp`
gates the manual entries); the implication is kept for symmetry with
`IPv6ProxyNDPAddress=` and is now called out explicitly in the man
page, together with the fact that enabling `proxy_arp` also activates
interface-wide automatic proxy ARP for routed-toward addresses on
connected subnets.
Parser-time validation rejects addresses the kernel would refuse:
the ANY/null address for both families, IPv4 and IPv6 multicast
and the IPv4 limited broadcast 255.255.255.255.
Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
systemd-repart currently forces newly created image files into NOCOW mode.
That prevents files from being reflinked into the image, making image builds
slower and increasing their disk usage on filesystems that support cloning.
Add a tristate --cow= option. By default, leave the filesystem or parent
directory COW policy unchanged. With --cow=yes, explicitly enable COW; with
--cow=no, retain the previous behavior of forcing NOCOW. Add XO_COW as the
counterpart to XO_NOCOW so xopenat_full() applies either policy while retaining
its normal creation-error cleanup.
Document the new option and extend TEST-58-REPART to verify inherited COW and
NOCOW policies as well as explicit COW and NOCOW overrides. Compare the unset
behavior with the filesystem default so the test also works on nodatacow
mounts, and skip it when the inode attribute is unsupported.
Signed-off-by: Daan De Meyer <daan@amutable.com>
Allow full UAPI.10 version strings, i.e. "+", "_", "~" and "^" too, to
match the recent reworking.
These version strings are generally distro-managed, hence use the more
liberal alphabet.
Fixes: #32785
`systemctl reboot --when=yesterday` reboots the machine immediately, which
surprised users enough to be reported as a bug. It is not one: the timestamp
passed to --when= (and to ScheduleShutdown(), and to shutdown(8)) declares the
earliest point in time the action may be taken, it is not a condition that is
evaluated and that could fail.
Behaving any differently would be racy and surprising: "--when=now" refers to
the past by the time the request is processed, and "--when=+50ms" may well have
elapsed already due to scheduling latencies. In both cases we must still carry
out the action the user asked for.
Document the semantics explicitly in systemctl(1), shutdown(8) and the
org.freedesktop.login1(5) D-Bus interface documentation.
Fixes: #42437
If the read-only FAT file attribute is set on /loader/random-seed,
don't update the seed file — and hence don't use it either, since a
seed we cannot update would be the same on every boot.
This gives users an explicit way to turn off random seed handling by
marking the file read-only, useful for example in pre-built OS images
that are replicated to many systems, where the baked-in seed is shared
and hence must not be credited.
The check is done upfront in process_random_seed(), before any other
work, mirroring the existing check for read-only volumes. This covers
both systemd-boot and systemd-stub, which share this code.
Inspired-by: #42979
Reproducer:
unit=run-wait-rae-$(date +%s)
sudo timeout 3s systemd-run --wait --remain-after-exit \
--unit="$unit" /bin/true
echo $?
systemctl is-active "$unit.service"
Before, the command timed out with exit status 124 while the service
stayed active. --wait waits for deactivation, but RemainAfterExit=yes
keeps the service active after the command exits.
Follow-up for 2a453c2ee3
Reproducer:
sudo systemd-run --wait --verbose --json=short /bin/echo hi
Before, systemd-run printed JSON metadata to stdout while --verbose
also spawned journalctl output on stdout. The resulting stream mixed JSON
with journal lines, so reject the conflicting options.
Follow-up for 744ca8f616
Reproducer:
sudo systemd-run --scope --ignore-failure /bin/false
echo $?
Before, the option was accepted but had no effect because scope mode
executes the command locally after creating the scope. The flag is only
encoded into service ExecStart properties, so accept it only where it can
be applied.
Follow-up for 1072d94731
The current TPM2 PIN mode is flawed as a compromised TPM directly
exposes
the sealed secret which is the LUKS volume key itself
(https://github.com/systemd/systemd/pull/27502 and
https://github.com/systemd/systemd/issues/37386).
Goal: add Argon2id-based PIN hardening to TPM2 enrollment, making
the TPM a second factor rather than a single point of failure:
1. Password + salt → Argon2id → 512-bit key split into Key1 + Key2
2. Key2 (base64-encoded) is used as the PIN to seal a random secret
in the TPM
3. Key1 + unsealed secret → HKDF-SHA256 → final LUKS volume key
This implementation ensures that if the TPM is compromised, an attacker
still needs the password to derive Key1 and combine it with the unsealed
secret.
The --tpm2-with-pin= option now accepts three values:
- false (no PIN used)
- true (PIN hardened with Argon2id - default)
- "direct" (legacy PIN without Argon2id for backward compatibility)
Argon2id parameters are customizable via:
--tpm2-argon2id-memory=
--tpm2-argon2id-iterations=
--tpm2-argon2id-parallelism=
--tpm2-argon2id-iter-time=
These default to a function of available CPUs and physical memory, with
a benchmark that scales iterations to the target time (default: 2s) and
falls back to ARGON2ID_PARAMETERS_DEFAULT (64 MiB, 8 iter, 4 lanes) when
auto detection fails.
Also if the runtime OpenSSL lacks Argon2id support (< 3.2), the feature
silently falls back to direct PIN mode with a warning.
Added includes:
- src/cryptenroll/cryptenroll.c: cpu-set-util.h, limits-util.h,
time-util.h
for Argon2id benchmark auto-tuning (cpus_online, physical_memory_scale,
now/usec_t)
- src/cryptenroll/cryptenroll-tpm2.c: crypto-util.h for
Argon2IdParameters
struct in load_volume_key_tpm2()
- src/shared/tpm2-util.h: crypto-util.h for Argon2IdParameters in
tpm2_make_luks2_json() API
- src/cryptsetup/cryptsetup-tokens/luks2-tpm2.c: crypto-util.h for
kdf_argon2id_derive()/kdf_hkdf_sha256() on the token unlock path
Let's make it possible to "suggest" that certain features or components
are enabled under some conditions.
For this, both features and components gain two things:
1. A Suggested= field which takes a boolean. If true the
feature/component will be suggested for installation, if false it
will not.
2. A set of SuggestedOnXYZ= settings are modelled after ConditionXYZ= in
unit files (and implement a subset of them), will suggest some
component/feature under specific conditions.
The result of the condition is shown in the various output tools.
The --copy-from= argument currently is documented as "copied partitions will have
the same size". This doesn't hold true in the case where a different grain-size is
passed to repart. Because `partition_min/max_size()` currently do rounding, the
size is implicitly rounded to grain size, and therefore partitions are enlarged
to align to grain size whenever possible.
Clarify this behavior and change the manpage, and also add a test for it.
udevadm info rejects positional devices together with
--device-id-of-file=.
Document that behavior instead of saying positional arguments are
ignored.
Reproducer:
udevadm info --device-id-of-file=/etc/passwd /sys
Before:
Devices are not allowed with -d/--device-id-of-file and -c/--cleanup-db.
Follow-up:
31767b92a0
The current TPM2 PIN mode is flawed as a compromised TPM directly exposes
the sealed secret which is the LUKS volume key itself (#27502 and #37386).
Goal: add Argon2id-based PIN hardening to TPM2 enrollment, making
the TPM a second factor rather than a single point of failure:
1. Password + salt -> Argon2id -> 512-bit key split into Key1 + Key2
2. Key2 (base64-encoded) is used as the PIN to seal a random secret
in the TPM
3. Key1 + unsealed secret -> HKDF-SHA256 -> final LUKS volume key
This implementation ensures that if the TPM is compromised, an attacker
still needs the password to derive Key1 and combine it with the unsealed
secret.
The --tpm2-with-pin= option now accepts three values:
- false (no PIN used)
- true (PIN hardened with Argon2id - default)
- "direct" (legacy PIN without Argon2id for backward compatibility)
Argon2id parameters are customizable via:
--tpm2-argon2id-memory=
--tpm2-argon2id-iterations=
--tpm2-argon2id-parallelism=
--tpm2-argon2id-iter-time=
These default to a function of available CPUs and physical memory, with
a benchmark that scales iterations to the target time (default: 2s) and
falls back to ARGON2ID_PARAMETERS_DEFAULT (64 MiB, 8 iter, 4 lanes) when
auto detection fails.
Also if the runtime OpenSSL lacks Argon2id support (< 3.2), the feature
silently falls back to direct PIN mode with a warning.
Added includes:
- src/cryptenroll/cryptenroll.c: cpu-set-util.h, limits-util.h, time-util.h
for Argon2id benchmark auto-tuning (cpus_online, physical_memory_scale,
now/usec_t)
- src/cryptenroll/cryptenroll-tpm2.c: crypto-util.h for Argon2IdParameters
struct in load_volume_key_tpm2()
- src/shared/tpm2-util.h: crypto-util.h for Argon2IdParameters in
tpm2_make_luks2_json() API
- src/shared/tpm2-util.c: limits-util.h, tpm2-util.h for physical_memory() validation
of Argon2id memory cost and function prototypes
- src/cryptsetup/cryptsetup-tokens/luks2-tpm2.c: crypto-util.h for
kdf_argon2id_derive()/kdf_hkdf_sha256() on the token unlock path
This are to the existing sd_varlink_call_and_upgrade() what
sd_varlink_callb() and sd_varlink_callbo() are to sd_varlink_call():
they put together an object on the fly, via the usual JSON builder
logic.
Stores the TPM2 SRK within the credential header, allowing for parameter decryption to be utilized when decrypting the credential.
A new dimension is added to the credential ID matrix to encode this capability.
This also allows for usage of TPM2-bound credentials when a TPM owner password is set since `Esys_CreatePrimary` is no longer used for sealing credentials.
This PR only sets the default to "relaxed" - I can change the default
to "tofu" if desired. But for that we will also need to update the NEWS
file to ensure everyone is aware of this new default.
---
This PR adds a new `systemd.credentials-boot=` kernel
commandline that allows to control if credentials with
a `null` key are accepted.
The possible options are:
* strict: always insist on tpm encryption
* tofu: allow null encryption in firstboot mode and when no tpm is
available
* relaxed: allow null encryption when sb is off, or no tpm is available
* off: allow null encryption always
The default is `relaxed` which is exactly the behavior we had before.
This replaces the initial idea of using plaintext credentials
at firstboot (thanks to Lennart for this nicer and simpler design).
---
With that we can drop `- firstboot: optionally accept credentials at
firstboot without authentication` from TODO.md
Extend real_has_cpu_with_flag to support aarch64 CPU Features using
hwcaps.
With this PR, users can find out if their Arm system supports
architecture-specific features such as BTI as follows:
```
$ systemd-analyze condition 'ConditionCPUFeature=bti'
```
While for sysupdate it's fine to consume a large set of all possible
update payloads in a single directory this is not so handy for managing
and serving the update payloads. Since this large update folder is not
where the build output directly gets written to one has to create
copies and later possibly delete this added set of files.
Support matching for filenames in subdirectories by having a new **/
match pattern prefix which matches any number of nested subdirectories
or no subdirectory at all. For simplicity it's only allowed at the start
of a pattern and not a regular wildcard as the rest because the main
use case is to descend into subdirectories and only do the pattern
matching for the basenames. This way one can create a SHA256SUMS file
in the top folder and have it include all update payloads from the
release-specific (or arch-specific) subdirectories. Something similar
was already supported for directory sources where the match pattern can
start with a subdirectory path. Do also support this for SHA256SUMS for
parity while we are at it. Having the new wildcard makes mirroring also
easier because one does not have to follow the exact subdirectory layout
and one can filter by folder instead of by filename. It also makes it
possible to point the same transfer files with the new wildcard to
either a SHA256SUMS file that uses release-specific (or arch-specific)
subdirectories and includes all versions or to a SHA256SUMS file as
generated from mkosi that does not use subdirectories because it only
has files for a single version.
With the upcoming UAPI.16 JSON format we will also be able to encode
subdirectories and it makes sense to add this to SHA256SUMS for being
able to convert them. It also supports custom URLs for each entry which
is more powerful than the (arbitrary) subdirectory feature used here but
subdirectories have the advantage that they don't break mirroring.
This change also fixes the existing subdirectory handling bugs where
everything greater than two subdirectory levels failed to work because
rel_joined instead of de->d_name got used, symlinks were followed, and
we would continue silently on non-ENOENT errors.
