Creates an image with an ESP via repart, writes an addon into it with
all settings systemd-firstboot can provision, and checks what ended up
in the image: the sections of the (unsigned) addon, and the credentials
in its cpio. Also covers replacing an addon, a second one next to it,
the combinations that are refused, and an image without an ESP.
The images and tools trees install liburing so we could experiment with
io_uring in sd-event. We're going to add io_uring support to sd-event without
liburing, so let's stop installing it.
With the varlink protocol upgrade feature we can use provide
file system access via varlink easily by re-using the existing
sftp standard. SFTP is independant of ssh and provides a well
defined and widely used file transfer protocol.
So here we add two new (optional) varlink interfaces:
- io.systemd.Sftp.Root.Connect
- io.systemd.Sftp.RootReadOnly.Connect
They are behind polkit auth as io.systemd.sftp.root{,-read-only}.
Note that handing this is out is equivalent for giving root
access: even ReadOnly mode will allow to read secrets like
/etc/shadow so its inherently risky. The two actions mostly
exist to prevent accidents not as a (strong) protection
boundary. The unit description warns about this and the polkit
prompt also describes the full implications.
The use cases are:
- file access via the varlink-http-bridge [1]
- file access over AF_VSOCK
The install is optional and turned off by default.
[1] https://github.com/systemd/varlink-http-bridge
The packaging references are part of mkosi's configuration, so tine can't
build a checkout with the same pinned spec. Let's share the pins in one
JSON file keyed by distro and keep mkosi, the packaging updater and
Packit using it.
salsa is very flaky and CI jobs constantly fail when pulling from it.
Switch to github repo in our org so that it is more reliable
(if it's down, the CI jobs are down as well by definition, so no
combinatorial probability of outages
Place the certificate the test images are signed with below
/usr/share/voa/<ID>/kernel-keyring/{dm-verity,fs-verity}/x509/ in the
host image and in the initrd so systemd-keyring-setup enrolls it into
the .dm-verity and .fs-verity keyrings at boot. The initrd mirrors the
copy mkosi installs there for userspace verification.
Add a test CA in keyring-setup-ca.{crt,key} and install its certificate
among the trust anchors for .dm-verity so trust anchors are enrolled as
well. keyring-setup-ca.py mints the certificates without a subject key
identifier that the tests need for the serial number fallback. OpenSSL's
x509 tool adds that extension to every certificate it signs so they
can't be produced with it.
systemd-keyring-setup.service isn't enabled by default but the
integration tests rely on it provisioning the keyrings from the initrd.
So enable it via the mkosi presets.
machine_supports_verity_keyring() looked for the mkosi certificate
anywhere in /proc/keys. Now that .fs-verity holds it as well that
doesn't imply dm-verity signature support anymore. So only look at the
keyrings dm-verity actually consults.
With the service enabled, VMs booted with dm_verity.keyring_unsealed=1
come up with the .dm-verity keyring provisioned and sealed already and
the manual provisioning in TEST-90 fails. So TEST-90 now looks at the
state of the keyring first:
- sealed at boot: the certificate is in place already, check that a
self-signed stranger is refused
- sealed empty: systemd-keyring-setup enrolled nothing, fail
- left open although the kernel left it unsealed and
systemd-keyring-setup is enabled: the service didn't provision it,
fail
- otherwise provision it by hand like before and skip when the kernel
sealed it at boot
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The other distributions get QEMU through the recommends of their systemd
packaging, Debian's doesn't list it, so the tests that boot guests with
systemd-vmspawn (TEST-87-AUX-UTILS-VM, TEST-97-VMSPAWN-MEMORY) have been
skipping on Debian and Ubuntu. Add qemu-system and the matching EDK2
firmware package for the architectures CI runs.
Lennart asked in #44114 to split the preparatory stuff out so it can
be merged in smaller PRs. So this is a stack of four, each based on
the one before it and each showing only its own commits:
1. #44323, the C helpers (merged)
2. #44324, support for programs written in Rust
3. #44325, the systemd_shared wrappers systemd-keyring-setup needs
4. #44114, systemd-keyring-setup itself
This one is 2.
This adds support for writing systemd programs in Rust. It replaces
#43551 and reworks it along the lines the maintainers agreed on at the
Berlin hackfest:
- Rust is only used for new leaf components. Existing C code isn't
ported and nothing that other components depend on is written in
Rust.
- No cargo. Everything is built with meson.
- Only the core, alloc and libc crates. Rust code is no_std and there
are no other crates. The libc crate isn't needed yet, the bindings
generated from the tree's own headers cover libc. And like the
kernel we don't use alloc either. Its collections abort when memory
runs out. systemd_shared has its own Box and Vec instead, which work
like the kernel's: everything that allocates returns an error and
there's no global allocator.
- C linkage only. libsystemd-shared and the dlopen machinery stay as
they are. A program written in Rust links libsystemd-shared
dynamically just like the C programs do and declares its dlopen()
dependencies in .note.dlopen. The Rust crates are linked statically
into the programs using them.
So nothing of the Rust standard library ends up in a program or in
libsystemd-shared. Per HACKING.md the stripped test-cli-rust binary is
32 KB with LTO against 20 KB for its C twin.
Rust support is optional via -Drust= and auto by default.
The bindings for libsystemd-shared are generated at build time by
bindgen from the same headers, flags and include stack the library is
built with. On top of that sits the systemd_shared crate which programs
are written against. It mirrors the C conventions: errno based errors,
the log_*() family, define_main!() and the command line framework of the
C programs. A Rust program prints the same --help, parses options the
same way and answers --introspect-cli exactly like a C program.
test-cli-parity.sh checks that byte for byte.
Secrets go into Vec<u8, Erasing>. Erasing allocates like malloc() but
frees with erase_and_free(). In C that's a convention every caller has
to remember. Here it's part of the type.
You need rustc >= 1.85, bindgen >= 0.71.1 and meson >= 1.4.0 which is
what Debian 13 ships. The tree keeps its meson floor and -Drust=auto
turns itself off on older meson. The mkosi builds of the distributions
that have the toolchain pass -Drust=enabled and the linter runs rustfmt,
the bindings header check and clippy. HACKING.md and CODING_STYLE.md
document how a program is written and what the rules are.
Note that the Arch and openSUSE OBS builds fail because their packaging
builds with --auto-features=enabled and doesn't pull in rustc and
bindgen. They need either -Drust=disabled or build dependencies on both.
It was based on #44323 only because bindings.h lists every header in
src/basic/ and src/shared/ and so has to know voa-util.h. That's merged
now and this PR targets main. The first program written against it is
systemd-keyring-setup in #44114, with the wrappers it needs in
#44325.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Add rustc and bindgen to the build images and tools trees so
-Drust=auto turns Rust support on wherever the packages are new enough.
That's Debian 13 and testing, Fedora, CentOS Stream 10, openSUSE
Tumbleweed and Arch. Ubuntu noble's rustc is too old so Rust stays off
there. Add rustfmt and clippy to the tools trees where the distribution
packages them separately.
The mkosi images of the distributions whose toolchain is known to be
new enough are built with -Drust=enabled so a toolchain that quietly
stops being found fails the build instead of silently dropping Rust.
Debian 13 ships exactly the floors, rustc 1.85.0, bindgen 0.71.1 and
meson 1.7.0, so its image doubles as the check for the oldest supported
toolchain.
rustfmt, the bindings header check and clippy run in the linter
workflow whose tools tree is Fedora rawhide. So that covers the newest
toolchain. Changes to Rust code get the rust label.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The other distributions get QEMU through the recommends of their
systemd packaging, Debian's doesn't list it, so the tests that boot
guests with systemd-vmspawn (TEST-87-AUX-UTILS-VM, TEST-97-VMSPAWN-MEMORY)
have been skipping on Debian and Ubuntu. Add qemu-system and the
matching EDK2 firmware package for the architectures CI runs.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
The minimal-base is not supposed to have systemd installed, and some
tests (e.g. TEST-13-NSPAWN) rely on this to build dummy containers.
But, the systemd package does get pulled in on Ubuntu, so explicitly
remove it in the mkosi config.
Exercises the new switches against every compression algorithm available
in the test image: detection vs. explicitly configured algorithm, "none"
taking the data as is, a mismatching algorithm being strictly used (and
hence failing), and --verify-decoded= checking the decompressed rather
than the downloaded data, also in combination with systemd-pull's
--verify=. A failed digest check must not leave an image in place. The
tar code paths are covered too.
The split out root partitions of the minimal images are used by the
test suite as bare file system images with external Verity data (via
RootImage=, portablectl, systemd-nspawn --image=, …). The validatefs
GPT constraints systemd-repart adds by default can never be fulfilled
for such bare file system images, hence turn them off.
Override mkosi's repart definitions for Format=portable with a
copy that sets AddValidateFS=no (which mkosi supports since the last
update).
* ed3eed38f2 Fix key-file path for extension/portable
\
* 43e8b2d35a Handle repart definitions for extension/portable
* 04816c3b82 Add oci-archive output format
* 7a9c94b63a Install ruff on recent Debian
* 4a7b08985e Change contextlb.contextmanager typing to Generator
\
* 5ca453ab6b Make OUTPUT variable available in postoutput scripts
* 90463ff04d Add tests for BuildStream support
\
* d36b138612 Add BuildStream support
* 2ca0d9c8ce ci: Sync claude-review workflow with systemd
* 1d1763f25a codespell: add "prFiles" to ignore-words-list
* 54b2897425 Revert "Symlink man pages on the website"
\
* 0c087181ed tests: fix pytest deprecation warning
* c9bbec224c docs: bump year in footer
* ae665678e2 Symlink man pages on the website
\
* 5895cd7596 Revert "Symlink man pages on the website"
* a09d5348ed docs: symlink man pages and assign them to the documentation heading
\
* 0bd378b7cb man: use YAML front matter
* 922e4a7f43 build(deps): bump aws-actions/configure-aws-credentials
* 05dfd295d3 initrd: add uas to the default list
* 2d52cc80db Pass through TERMINFO and TERMINFO_DIRS to tput (#4455)
* caa3c3235d Revert "dnf: Only read repositories and variables from the sandbox tree"
* 29b5f4d0d1 Add --umask to sandbox and UMask= setting
* d2e27ab62b Renaming remaining postmarketOS to Nura
On aarch64 there is no SMM to protect Secure Boot variables from the
guest, so Fedora ships aarch64 Secure Boot capable firmware only with
the `host-uefi-vars` feature: it keeps its variables in QEMU's uefi-vars
device instead of pflash. vmspawn previously skipped these firmware
descriptors, so `--secure-boot=yes` failed on aarch64:
> Skipping /usr/share/qemu/firmware/91-edk2-aarch64-qemuvars-sb.json,
> firmware does not have an NVRAM template.
> Failed to find OVMF config: No such file or directory
Select such firmware if its descriptor declares a variable store
template, and give it the `uefi-vars` device. This is backed by a JSON
state file with the same semantics as the pflash NVRAM state. Its
`--efi-nvram-state=auto` path gets a different suffix, so that switching
between the two firmware types never feeds a state file to the wrong
one.
Descriptors are still picked in sort order, so when pflash firmware
satisfies the requested features and is sorted first (as with Fedora's
x86 descriptors), it does not change behaviour.
Signed-off-by: Martin Pitt <martin@amutable.com>
Entry IDs are matched case insensitively by folding the patterns and
relying on the IDs being lower case already (see 23742af522).
Since 4301ad00ef the ID of a`Type #2` entry is rebuilt from the file
name without the boot counter, but it is only
folded when there is no counter, and since 941b91600c the profile ID
appended for secondary UKI profiles is not folded either.
So `EFI/Linux/Foo+3.efi` or a profile with `ID=Profile1` is never
matched by `default`, `preferred`, `@saved`, `bootctl set-default` or
`systemctl reboot --boot-loader-entry=`, while `bootctl list` (which
folds) shows it as the default. Type #1 entries are fine, they fold
after parsing the counter.
This PR folds the ID unconditionally like Type #1 does, and the same for
the profile suffix as well. The second commit extends the existing tests
to cover the capitalized case as well.
Follow-up for 4301ad00ef and
941b91600c.
Rename the UKI to a mixed case file name with a boot counter before
selecting the first profile, and give the second profile a mixed case
ID, so that both are only reachable if systemd-boot folds the entry IDs
of Type #2 entries with boot counter and of secondary profiles.
The entry IDs passed to bootctl set-default are spelled lower case and
without the counter, like loader.conf(5) documents and bootctl list
shows them.
The openSUSE spec installs its own systemd-update-helper shell script over
the systemd-update-helper that meson builds. The integration tests on
openSUSE therefore run the shell script instead of the C implementation.
The introspect-cli subtest of TEST-74-AUX-UTILS fails there, because the
shell script treats --help as an unknown verb:
TEST-74-AUX-UTILS.sh[2883]: ++ systemd-update-helper --help
TEST-74-AUX-UTILS.sh[2883]: Unknown verb '--help'
Remove the install line from the spec, as the build script already does
for other parts of the spec that conflict with the source tree.
- Only download distro systemd packages in mkosi.prepare, set up the
devel sytemd packages in mkosi.postinst so that they're always up to
date.
- Use systemd-analyze for version comparisons
- Add test to make sure units are stopped on downgrade
- Add test to make sure removing a package with a frozen user session
doesn't blow up
Sign a system report through the io.systemd.Report pipeline in the
SNP guest and export the resulting signed report to the host. The host
takes the role of a remote verifier: the tsm signature record must carry
the digest of the signed report bytes, that digest must be embedded in
the REPORT_DATA field of the enclosed SNP attestation report, and the
attestation report must verify against the AMD certificate chain with
snpguest, which is added to the Fedora test image. Certificates must be
pre-provisioned on the host in a sirectory named 'snp-certs' and bound
into the test container via $TEST_BIND_DIRS.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
* 117d382486 nura: use modern url for Alpine binary repos
* 7872fd58f7 nura: drop x86
* 3533864541 gitignore: add mkosi.pkgcache
* 580c4d94d7 Rename postmarketOS --> Nura
* ba96aba91e Bump version to 28~devel
* ecf7f5702b Release 27.1
* ca08e2ccef docs: add notes on nsresourced in execution flow
* 146b5dd4de docs: add a few notes on package manager specific behaviour
* 344e42ee20 Update NEWS
* dde1bf020f debian: fix debian-security/debug url when using a mirror
* 699aa3b565 mkosi: upgrade debug message about fallback path to info
* cb6da3f3ce mkosi: move fallback environment variable check to bypass nsresourced fully
* 88c716a6b9 config: add delegate_ranges
* f4de9a102a mkosi: use the foreign UID range only when requested when running as root
* d091fed134 mkosi: only delegate UID ranges if we're not already root
* 165bc99675 sandbox: rename delegate to delegate_ranges
* bba53c50d0 sandbox: use equality check instead of boolean not for delegate
* a033d03f59 summary: add foreign uid range
* f13e96eb95 user: drop unused variable SUBRANGE
* 847d113863 mkosi-obs: Only configure MakeScriptsExecutable in the main image
* 568b1bce56 dnf: Only read repositories and variables from the sandbox tree
* 938d6a1c21 obs: use policy signing data when available
* 19d045e5e9 Rename sysupdate configuration files to .transfer
* 9539f1771a action: bump fallback apk version to 3.0.8
* 8de4d25596 sandbox: add force_fallback to acquire_privileges and wire it up via env
* af97d6ffef opensuse: tolerate missing RPM keyring
* 0f2b7e0656 initrd: don't fail when encountering directories among ucode files
* 5240191893 Allow using SignInitrdPCRs with SplitArtifacts=pcrs
* ce90009686 Bump ukify required version from 261.999 to 262~ for SignInitrdPCRs
* 58bbd63aca news: fix v27 `DiskType=` setting typo
* 1da3299aec tests: skip test_fork_and_wait if on rc 1
* 5658e7abc9 Bump version to 28~devel
* 4736cd8361 Release 27
* 97f305bf0a action: make shell style consistent
* dd7140cbae action: only use apk version from API if non-empty
* 2cfe0cefb8 news: update NEWS
* 7bca598fa4 action: support container-based self-hosted runners
* 08c86a25e1 action: bump fallback APK version
* cc0fcef38f action: reduce timeouts and number of API calls
* 211993d5d2 action: download latest version of apk
| * f33cc45db3 Add ToolsTreeSnapshot= for pinning the default tools tree
| * ddfc5a2cb5 tests: join options and their values with f-strings
| * a497ff0251 doc: add missing matchers
* | 589226cfd4 apk: disable grub's update-grub trigger during package installation
/
| * 595451bb4b debian: add new split-out systemd packages
| * 7ce1851901 Revert "mkosi-tools: Install systemd-tpm on Debian"
| * 50a1137be5 ci: pin resolved release for integration tests
| * e30b0a0ba4 tests: drop obsolete @ prefix
| * e7f4d57fac news: update NEWS
|/
* bf2addb2ec build(deps): bump aws-actions/configure-aws-credentials
* a58ded6a2a build(deps): bump actions/checkout from 6.0.2 to 7.0.1
* 9b7d87a170 add support for prebuilt UKIs
* 75fdb9be15 mkosi-tools: Install systemd-tpm on Debian
* d5ff0d0d98 Fall back to unprivileged user namespace on polkit denial
* 0e4d5546ac Bump 261~devel version checks to 261
* 180d4a5329 Use systemd nightly OBS version in signing PCR policies auto detection
* f7401bdc8d Add support for signing PCR policies for the initrd
* 33d17b2b92 sandbox: Don't create shared propagation peer group
* a078088f8e sandbox: Declare MS_PRIVATE as well
* e1e9eafc87 kmod: eliminate duplication in calls to modinfo
* df68b27e8e tools: add mount to Debian tools tree
* 205dcedd45 util: don't drop single-character variable names in read_env_file
* 35a9d38ea0 distribution: report the actual architecture in unsupported-arch errors
* d4f44d212c sandbox: handle case where dev node for tty doesn't exist better
* acf08c5eca build(deps): bump actions/upload-artifact from 7.0.0 to 7.0.1
* b3ec53743a tests: Pass --output-directory to vm() and boot()
* 9bc430a7fa config: Isolate build history per output directory
* fd3d9bfe57 tests: Add regression test for build history lookup via configured output dir
| * 55ab00bb13 Fix tar exclude patterns for apivfs paths
|/
* 699da862f8 sandbox: Split a chroot() context manager out of chase()
* 70ce10d132 sandbox: Default FSOperation.execute oldroot/newroot to /
* 610b38538f Forbid AI attribution in commit messages
* b7425bbe42 ci: Sync claude-review workflow with systemd
* b9d4c06239 action: Make apk installation configurable and reliable again
* f908244ec2 systemd: move tmpfiles.d/systemd-network.conf to networkd subpackage
Fixes the following warning on mkosi build:
```
‣ [default-initrd] Copying in extra file trees…
‣ [default-initrd] Generating system users
‣ [default-initrd] Generating volatile files
/buildroot/usr/lib/tmpfiles.d/systemd-network.conf:13: Failed to resolve user 'systemd-network': Unknown user
‣ [default-initrd] Applying presets…
```
* 60fbc2d955 Update systemd to version 261.2 / rev 473 via SR 1368021
* c32d9727e6 Update systemd to version 261.1 / rev 472 via SR 1365923
* 74fd672013 Update systemd to version 260.3 / rev 471 via SR 1362701
* 769e1c4254 Update systemd to version 260.2 / rev 470 via SR 1361320
* b12500baf3 Drop sd-sysupdate for upstream builds, experimental again
* 674dd1e684 d/t/localed-locale: skip in Debian, not supported
* 9bd985d014 d/t/localed-locale: clean up on any error with a trap
* 56153a917f d/t/control: add dependencies missing for incus image
* 9badaae1c4 Drop deprecated meson option
* 8700e5c4e5 Install new files for upstream build
* 7aaf05335d Update d/copyright
* 68b634db29 Update changelog for 261.2-1 release
* 07591e4f9a Open changelog for 261.2-1 release
* 2bd7f0db1f d/libnss-systemd.nss: install after files and compat in nsswitch.conf
* c2e3b3135c Drop OBS workarounds and use pkg.systemd.obs build profile
* aff99fcc49 d/control: clarify systemd-imds Description
* b5006dc64a d/systemd.links: drop procps.service alias (LP: #2153900)
* b630d1569f Make systemd-networkd example consistent with systemd.network(5)
The centos-fedora, opensuse and postmarketos image configurations all
install bpftool but the Debian/Ubuntu ones never did. Only their build
image gets it via the packaging's Build-Depends. So the
TEST-90-RESTRICT-FSACCESS guard and overlay subtests, both gated on
'command -v bpftool', just silently skip there.
Mirror what the build image does. Plain bpftool everywhere except Ubuntu
jammy and noble which still bundle it in linux-tools-generic.
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Prepare scripts run before volatile packages are installed, so parse
the list from the config to ensure they are all included to avoid
failures due to some packages missing from the list.
Follow-up for 28e1f84d6a
Fixes the following failure on Fedora 44:
```
TEST-58-REPART.sh[932]: Executing mkfs command: /usr/bin/mkfs.erofs -U 45745a56-aa2f-4619-8ca7-9cb63667c2ae -zlz4hc,level=3 /dev/loop0 /var/tmp/.#reparteb739e8b8cae7b70
TEST-58-REPART.sh[932]: Successfully forked off '(mkfs)' as PID 933.
TEST-58-REPART.sh[933]: ==933==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
TEST-58-REPART.sh[932]: '(mkfs)' failed with exit status 1.
```
Christian Brauner