With the latest mkosi it's possible for MinimumVersion= to be a git
commit so let's start making use of that. This will make mkosi fail
if it's executed within the systemd repository and the checked out
commit is too old.
Putting the mkosi commit sha in mkosi/mkosi.conf also allows retrieving
it without having the full source tree available.
We also make a bunch of improvements to the fetch-mkosi.py script.
(cherry picked from commit 278d5bfd7e)
Now that mkosi can automatically pick up its main configuration from
a mkosi/ subdirectory if it exists and there is no configuration in the
top level directory, let's make use of it to reduce the amount of clutter
in the top level directory of the repository.
This will also make it easier to install the mkosi configuration files as
part of the testing packages later on.
(cherry picked from commit 342d21ef19)
The tools image is not guaranteed to be the same distribution as the
target distribution and so might have different package environment
variables than the main image yet we currently unconditionally use the
same package environment variables for both of them.
Let's fix this by not passing the package environment variables to the
tools image and subimages anymore, and instead having the main, tools and
build images separately include a config file with the required environment
variables.
(cherry picked from commit ce68d4349e)
Instead of listing dependencies manually for the default tools tree,
let's reuse the prepare scripts from the build image. To make this work,
the sync script has to be configured for the tools tree as well so that
it's invoked both when building the tools tree and for the regular image,
otherwise, when doing the first build in a fresh checkout, the sync script
won't have executed yet as sync scripts for the regular images are executed
after building the default tools tree.
(cherry picked from commit 1b49fb9aaa)
We get the same warning thousands of times:
/work/src/tools/check-version-history.py:28: FutureWarning: This search incorrectly
ignores the root element, and will be fixed in a future version. If you rely on the
current behaviour, change it to
"./refsynopsisdiv/funcsynopsis/funcprototype/funcdef/function[.='udev_device_get_properties_list_entry']"
We also need to update the ignorelist to the new form.
(cherry picked from commit 8d4eda524a)
Maintaining the fmf metadata and script upstream makes it painful
to reuse downstream so let's move the metadata and testing script
downstream and load it upstream instead.
(cherry picked from commit 4d938fa865)
The script runs the binaries which try to find the internal libs via /proc/self/exe due
to glibc's RPATH resolution and fail:
/var/cache/src/systemd/tools/dbus_exporter.py interfaces
/var/cache/src/systemd/build/systemd
/var/cache/src/systemd/build/systemd-homed
/var/cache/src/systemd/build/systemd-hostnamed
/var/cache/src/systemd/build/systemd-importd
/var/cache/src/systemd/build/systemd-localed
/var/cache/src/systemd/build/systemd-logind
/var/cache/src/systemd/build/systemd-machined
/var/cache/src/systemd/build/systemd-networkd
/var/cache/src/systemd/build/systemd-oomd
/var/cache/src/systemd/build/systemd-portabled
/var/cache/src/systemd/build/systemd-resolved
/var/cache/src/systemd/build/systemd-sysupdated
/var/cache/src/systemd/build/systemd-timedated
execve("/var/cache/src/systemd/build/systemd", ["/var/cache/src/systemd/build/sys"..., "--bus-introspect", "list"], 0x7ffc7ab68600 /* 20 vars */) = 0
brk(NULL) = 0x56265bf70000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f56ced7f000
readlinkat(AT_FDCWD, "/proc/self/exe", 0x7ffedeaa7a90, 4096) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=20293, ...}) = 0
mmap(NULL, 20293, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f56ced7a000
close(3) = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v4/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v4/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v3/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v3/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v2/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v2/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/", {st_mode=S_IFDIR|0755, st_size=19312, ...}, 0) = 0
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v4/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v4/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v3/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v3/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v2/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v2/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/", {st_mode=S_IFDIR|0755, st_size=19312, ...}, 0) = 0
openat(AT_FDCWD, "/lib/glibc-hwcaps/x86-64-v4/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/glibc-hwcaps/x86-64-v4/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/glibc-hwcaps/x86-64-v3/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/glibc-hwcaps/x86-64-v3/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/glibc-hwcaps/x86-64-v2/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/glibc-hwcaps/x86-64-v2/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib/", {st_mode=S_IFDIR|0755, st_size=642, ...}, 0) = 0
openat(AT_FDCWD, "/usr/lib/glibc-hwcaps/x86-64-v4/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/glibc-hwcaps/x86-64-v4/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/glibc-hwcaps/x86-64-v3/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/glibc-hwcaps/x86-64-v3/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/glibc-hwcaps/x86-64-v2/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/glibc-hwcaps/x86-64-v2/", 0x7ffedeaa80b0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libsystemd-core-258.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/lib/", {st_mode=S_IFDIR|0755, st_size=642, ...}, 0) = 0
writev(2, [{iov_base="/var/cache/src/systemd/build/sys"..., iov_len=36},
{iov_base=": ", iov_len=2},
{iov_base="error while loading shared libra"..., iov_len=36},
{iov_base=": ", iov_len=2},
{iov_base="libsystemd-core-258.so", iov_len=22},
{iov_base=": ", iov_len=2},
{iov_base="cannot open shared object file", iov_len=30},
{iov_base=": ", iov_len=2},
{iov_base="No such file or directory", iov_len=25},
{iov_base="\n", iov_len=1}],
10/var/cache/src/systemd/build/systemd: error while loading shared libraries: libsystemd-core-258.so: cannot open shared object file: No such file or directory
) = 158
(cherry picked from commit c6a932fd0d)
Then, switch the default value to "0600", due to general security
concerns about terminals being written to by other users.
Closing #35599.
Backport note: the default is changed back to 0620 to keep backward
compatbility intact for the purpose of backporting. This lets
security-conscious users fix the issue if they want to, without
affecting users that do not want changes in the stable branches.
(cherry picked from commit a4d1891475)
[1/9] Generating version.h with a custom command
/home/zbyszek/src/systemd/tools/vcs-tag.sh: line 17: git: command not found
If git is not available, do not try to call it.
We now import the upstream tag in the debian repository, so
this explodes as it tries to walk all upstream commits. Use
--first-parent so that merges only get added via the merge
commit.
In the troff output, this doesn't seem to make any difference. But in the
html output, the whitespace is sometimes preserved, creating an additional
gap before the following content. Drop it everywhere to avoid this.
This is very similar to tools/fetch-distro.py. The idea is that we extend the
commit to update the mkosi hash with a git log --pretty=oneline output, so that
the reader can know what changes were actually included.
The motivation is that I'm always wondering what changed in mkosi when I see a
commit updating the hash, and it's nicer to have this information shown
directly in the commit.
The script does _not_ pull changes from upstream, on the assumption that the
person doing the commit always has a fresh checkout and that they tested with
that checkout.
When building packages of arbitrary commits of systemd-stable,
distributors might want to include a git sha of the exact commit
they're on. Let's extend vcs-tag a little to make this possible.
If we're on a commit matching a tag, don't generate a git sha at all.
If we're not on a commit matching a tag, generate a vcs tag as usually.
However, if we're not in developer mode, don't append a '^' if the tree
is dirty to accomodate package builds applying various patches to the
tree which shouldn't be considered as "dirty" edits.
Let's rename the tool to tools/fetch-distro. It's useful to be able to fetch
the distro directly. But when that functionality is added, the old name is
confusing.
Now --update/-u must be specified to update the commits.
--reference-if-able is used to speed up the clone of debian.
It saves about 75% of the download.
We want the exitrd image to be built with the latest systemd as well.
As the exitrd image is built as part of mkosi.images, and all subimages
are built before the main image, this implies the packages must be built
as a subimage in mkosi.images/ as well. So we introduce the build image and
move all logic related to building distribution packages there.
This also has the nice side effect of slimming down the main image as the
build dependencies are not installed into the main image anymore. It also
makes sure the packages are built in a "clean" chroot without any of the
other packages which we install in the main image available.
This file doesn't document features of systemd, but is more a of a
general description that generalizes/modernizes FHS. As such, the items
listed in it weren't "added" in systemd versions, they simply reflect
general concepts independent of any specific systemd version. hence
let's drop this misleading and confusing version info.
Or in other words, the man page currently claims under "/usr/": "Added
in version 215." – Which of course is rubbish, the directory existed
since time began.
This also rebreaks all paragaphs this touches.
No content changes.
The setting of systemd clock is important and deserves an accurate description,
see for example:
https://discussion.fedoraproject.org/t/f38-to-f39-40-dnf-system-upgrade-can-fail-on-raspberry-pi/92403https://bugzilla.redhat.com/show_bug.cgi?id=2242759
The meat of the description was in systemd-timesyncd.service(8), but
actually it's systemd that sets the clock. In particular, systemd-timesyncd
doesn't know anything about /usr/lib/clock-epoch, and since systemd sets
the clock to the epoch when initializing, systemd-timesyncd would only
get to advance the clock to the epoch under special circumstances.
Also, systemd-timesyncd is an optional component, so we can't even rely
on its man page being installed in all circumstances. The description needs
to be moved to systemd(1).
The description is updated to describe the changes that were made in
previous commits.
Unfortunately, git submodules break in all sorts of ways:
- Various github workflows (dependabot, github pages) try to do a shallow
clone of git submodules which does not work at all when the git repository
is hosted on pagure (https://pagure.io/pagure/issue/5453,
https://github.com/dependabot/dependabot-core/issues/9391).
- If the git forge hosting the git repository uses SHA256, then it breaks our
usage of it as a submodule as SHA256 repositories cannot be used as submodules
in SHA1 repositories (src.opensuse.org moved to SHA256 which broke our usage of
opensuse's systemd spec as a submodule).
- git submodules completely break usage of git worktrees.
- ...
Let's avoid all these issues by just doing our own home grown implementation of
git submodules. We lose the automatic dependabot updates this way but since dependabot
fails to run more often that not with submodules we don't really lose anything.
git rebase does not support a --recurse-submodules switch to automatically
check out the submodules at their registered commits during or after a rebase.
Instead, let's use the post-rewrite git hook to do this ourselves.
Also, this drops Weblate (again) and dependabot from the contributers list.
Moreover, this makes the contributers sorted by git command, rather
than sort command. Then, the authors are sorted by their first name, e.g.
- before
Xiaotian Wu, Yuri Chornoivan, Yu Watanabe, Zbigniew Jędrzejewski-Szmek,
- after
Xiaotian Wu, Yu Watanabe, Yuri Chornoivan, Zbigniew Jędrzejewski-Szmek,
Suggested-by: Matteo Croce <teknoraver@meta.com>
Resolves https://github.com/systemd/systemd/issues/31637.
lld-18 does the section setup differently than older versions. There is a bunch
of ordering chagnes, but it also inserts the following:
Sections:
Idx Name Size VMA LMA File off Algn
...
9 .got 00000000 00000000000283c0 00000000000283c0 000283c0 2**3
CONTENTS, ALLOC, LOAD, DATA
10 .relro_padding 00000c40 00000000000283c0 00000000000283c0 000283c0 2**0
ALLOC
11 .data 00000024 00000000000293c0 00000000000293c0 000283c0 2**4
CONTENTS, ALLOC, LOAD, DATA
...
This causes a problem for us, because we try to map the .got to .rodata,
and the subsequent .data to .data, and round down the VMA to the nearest
page, which causes the PE sections to overlap.
https://github.com/llvm/llvm-project/pull/66042 adds .relro_padding to make
sure that the RELRO segment is properly write protected and allocated. For our
binaries, the .got section is empty, so we can skip it safely, and the
.relro_padding section is not useful once .got has been dropped.
We don't expect .got sections, but they are apparently inserted on i386 and
aarch64 builds. Emit a warning until we figure out why they are there.
RuntimeError is documented as "Unspecified run-time error". It doesn't make
much sense for Python. (It originated in Java, where exceptions that can be
thrown by a function are declared in the function signature. All code calling
such a function must either explicitly catch all possible exception types, or
allow them to propagate by listing them in its own exception type list. This is
nice in theory, but in practice very annoying. Especially during development,
when the list of possible exception types is not finalized, we would end up
adding and removing exceptions to functions signatures all the time. Also for
code which is designed to call functions recursively, we would soon end up with
all functions declaring all possible exception types… To avoid this, people
would quite often do fake handling with a block that either prints and ignores
an exception, or has just a comment like "fix me later", or even nothing. This
often lead to people forgetting to adjust this later on and production code
containing such constructs. An escape hatch was opened with RuntimeException and
its subclasses, which do not need to be pre-declared. Various memory-related
exceptions were added as subclasses of RuntimeException. But later on, people
starting using this to not to have to declare all exception types everywhere.)
In Python, exceptions do no have to be pre-declared, and for code which just
encounters a failure, we should raise a specific exception type. The catch-all
class for unexpected input is ValueError.
For https://github.com/systemd/systemd/issues/31637:
BadSectionError: Section '.data' @0x28000 overlaps previous section @0x28000+0x300=@0x28300
Also, exception strings should not contain trailing periods, because they are
often embedded in sentences.
Luca Boccassi