dilim/systemd
1
0 Fork 0
mirror of https://github.com/systemd/systemd.git synced 2026-11-06 05:17:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

UAPI.16 Manifest tooling and support in systemd-sysupdate (#44292)

Browse Source 
This adds UAPI.16 support to systemd-sysupdate, systemd-importd,
systemd-repart. It also adds a new tool for generating and verifying
manifests.

It also adds support for additional matches in sysupdate transfer files,
against xattrs on files. This is then also replicated in vpick.

Fixes: #44448
Fixes: #38772
Fixes: #32819
Fixes: #35571
Fixes: #38774
Fixes: #37931
This commit is contained in:
Lennart Poettering authored and GitHub committed 2026-11-05 13:41:28 +01:00
2 parents 77dbbd26b4 b8d6a06f5a
commit 27281f9107
17 files changed
+1512 -153
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

TODO.md
-5
View File
@@ -156,8 +156,6 @@ SPDX-License-Identifier: LGPL-2.1-or-later
- make vpick assessment counting work
- sysext: condition confext/sysext on machine tags, too
- sysupdate: add health check logic, revert updates if fail
- confext/sysext: add policy file concept: json files that encode for relevant
@@ -1229,9 +1227,6 @@ SPDX-License-Identifier: LGPL-2.1-or-later
- complete varlink interface
- download images into .v/ dirs
- importd: support image signature verification with PKCS#7 + OpenBSD signify
logic, as alternative to crummy gpg
- In .socket units, add ConnectStream=, ConnectDatagram=,
ConnectSequentialPacket= that create a socket, and then *connect to* rather than
listen on some socket. Then, add a new setting WriteData= that takes some
man/sysupdate.d.xml
+215 -8
View File
@@ -140,7 +140,10 @@
<listitem><para>For HTTP/HTTPS sources a simple scheme is used: a manifest file
<filename>SHA256SUMS</filename>, following the format defined by <citerefentry
project='man-pages'><refentrytitle>sha256sum</refentrytitle><manvolnum>1</manvolnum></citerefentry>,
lists file names and their SHA256 hashes.</para></listitem>
lists file names and their SHA256 hashes. Alternatively, a <filename>Uapi16Manifest</filename> file
following the <ulink url="https://uapi-group.org/specifications/specs/file_manifest/">UAPI.16 File
Manifest</ulink> specification may be used, see <varname>ManifestFormat=</varname>
below.</para></listitem>
</itemizedlist>
<para>Transfers are done in the alphabetical order of the <filename>.transfer</filename> file names they are
@@ -521,6 +524,24 @@
keyring in <filename>/etc/systemd/import-pubring.pgp</filename> (or, if that does not
exist, <filename>/etc/systemd/import-pubring.gpg</filename>).</para>
<para>For UAPI.16 manifests (see <varname>ManifestFormat=</varname>) the signature is embedded in
the manifest itself, as a signature object. Only PKCS#7 signatures are supported, which must have
been made with the private key of one of the X.509 certificates found in the <ulink
url="https://uapi-group.org/specifications/specs/file_hierarchy_for_the_verification_of_os_artifacts/">File
Hierarchy for the Verification of OS Artifacts</ulink> (VOA), i.e. in
<filename><replaceable>OS</replaceable>/manifest/sysupdate/x509/<replaceable>NAME</replaceable>-certificate.pem</filename>
below the VOA load paths <filename>/etc/voa/</filename>, <filename>/run/voa/</filename>,
<filename>/usr/local/share/voa/</filename> and <filename>/usr/share/voa/</filename>, where
<replaceable>OS</replaceable> is derived from
<citerefentry><refentrytitle>os-release</refentrytitle><manvolnum>5</manvolnum></citerefentry> of
the system (or of the root directory or image specified with <option>--root=</option> or
<option>--image=</option>). Only artifact verifiers are supported, i.e. certificates are not checked
against trust anchors, they are trusted as they are. If no certificates are found the transfer
fails. Such signatures may be generated with <command>systemd-manifest make --private-key=</command>,
and checked against the same certificates with <command>systemd-manifest verify
--voa-context=sysupdate</command>, see
<citerefentry><refentrytitle>systemd-manifest</refentrytitle><manvolnum>1</manvolnum></citerefentry>.</para>
<para>This option is essential to provide integrity guarantees for downloaded resources and thus
should be left enabled, outside of test environments.</para>
@@ -634,11 +655,12 @@
<listitem><para>Specifies where to find source versions of this resource.</para>
<para>If the source type is selected as <constant>url-file</constant> or
<constant>url-tar</constant> this must be a HTTP/HTTPS URL. The URL is suffixed with
<filename>/SHA256SUMS</filename> to acquire the manifest file, with
<filename>/SHA256SUMS.gpg</filename> to acquire the detached signature file for it, and with the file
names listed in the manifest file in case an update is executed and a resource shall be
downloaded.</para>
<constant>url-tar</constant> this must be a HTTP/HTTPS URL. Unless <varname>ManifestPath=</varname>
is specified, the URL is suffixed with <filename>/SHA256SUMS</filename> (and
<filename>/SHA256SUMS.gpg</filename> for the detached signature file) or with
<filename>/Uapi16Manifest</filename> (which carries its signature inline), depending on
<varname>ManifestFormat=</varname>, to acquire the manifest file. It is suffixed with the file names
listed in the manifest file in case an update is executed and a resource shall be downloaded.</para>
<para>Alternatively, for these two source types a URL of the form
<literal>provider:[<replaceable>socket</replaceable>]/<replaceable>resource</replaceable></literal>
@@ -698,8 +720,49 @@
<constant>url-file</constant> or <constant>url-tar</constant> source. Takes
<constant>sha256sums</constant>, which selects a <filename>SHA256SUMS</filename> file as generated by
<citerefentry project='man-pages'><refentrytitle>sha256sum</refentrytitle><manvolnum>1</manvolnum></citerefentry>,
accompanied by a detached <filename>SHA256SUMS.gpg</filename> signature, see above. Defaults to
<constant>sha256sums</constant>. This option has no effect for other source resource types.</para>
accompanied by a detached <filename>SHA256SUMS.gpg</filename> signature, see above. Or
<constant>uapi16</constant>, which selects a <filename>Uapi16Manifest</filename> file following the
<ulink url="https://uapi-group.org/specifications/specs/file_manifest/">UAPI.16 File
Manifest</ulink> specification, i.e. a JSON-SEQ stream with one object per file, carrying the file
name, size and SHA256 digest of each file, as generated by
<citerefentry><refentrytitle>systemd-manifest</refentrytitle><manvolnum>1</manvolnum></citerefentry>
or <command>systemd-dissect --uapi16</command>. Only regular files listed in the manifest are
considered, and entries whose GPT partition type does not match the target's
<varname>MatchPartitionType=</varname> are ignored. The signatures embedded in UAPI.16 manifests are
verified against the certificates found in the VOA hierarchy, see
<varname>Verify=</varname> above. Defaults to <constant>sha256sums</constant>. This option has no
effect for other source resource types.</para>
<para>If a file object of a UAPI.16 manifest carries a <literal>contents</literal> array, the file
is acquired from the sources listed there, instead of by its name relative to
<varname>Path=</varname>. Each entry is an alternative location of the same data, tried in the listed
order until one works: entries with a <literal>file</literal> field are downloaded from that
(relative) file name resolved against <varname>Path=</varname>, entries with a
<literal>url</literal> field from that URL, and entries with a <literal>literal</literal> field carry
the data inline. Entries with none of these three fields refer to a file of the object's name,
resolved against <varname>Path=</varname>. Relative file names are resolved against
<varname>Path=</varname> even if <varname>ManifestPath=</varname> is set, as the latter only
redirects the acquisition of the manifest itself. The <literal>encoding</literal> field
may be one of <literal>gzip</literal>, <literal>xz</literal>, <literal>zstd</literal>,
<literal>bzip2</literal> or <literal>lz4</literal>, in which case the data is decompressed on
acquisition. The declared encoding is binding: the data is
strictly decoded with that algorithm (or taken as is, if none is declared), no detection of the
compression format from the data takes place. A source may carry more than the file, in which case
the range of the decoded data starting at <literal>offset</literal> and extending over the file's
<literal>size</literal> is used. Downloaded sources are verified against their
<literal>encodedSha256</literal> digest if they carry one, except for sources that are not encoded
but carry more than the file: of those only the range that makes up the file is downloaded (via an
HTTP range request), and their <literal>encodedSha256</literal> digest is not checked. If the source
is encoded, carries more than the file, or has no digest of its own, the decoded file is checked
against the file's <literal>sha256</literal> digest. Downloaded sources of <literal>url-tar</literal>
resources, as well as inline data that is encoded, must consist of the file and nothing else, other
such sources are ignored.</para>
<para>The extended attributes in the <literal>user</literal> namespace listed in the
<literal>extendedAttributes</literal> field of a file object of a UAPI.16 manifest are applied to the
file or directory created when installing it. This is done on a best-effort basis: if the file system
does not support extended attributes, or they cannot be set for other reasons, the instance is
installed without them.</para>
<xi:include href="version-info.xml" xpointer="v263"/></listitem>
</varlistentry>
@@ -724,6 +787,136 @@
<xi:include href="version-info.xml" xpointer="v263"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>MatchXAttr=</varname></term>
<listitem><para>Takes a boolean argument, defaults to on. If enabled, source instances may be
restricted to specific systems via extended attributes in the <literal>user</literal> namespace. Each
of the following extended attributes is checked against a property of the system that is being
updated (i.e. the system specified with <option>--root=</option>/<option>--image=</option>, if
used):</para>
<table>
<title>Extended attributes checked</title>
<tgroup cols='2'>
<colspec colname='xattr' />
<colspec colname='property' />
<thead>
<row>
<entry>Extended attribute</entry>
<entry>Matched against</entry>
</row>
</thead>
<tbody>
<row>
<entry><literal>user.uapi.match_hostname</literal></entry>
<entry>The hostname, as configured in <filename>/etc/hostname</filename> if operating on a
root directory or image, the current hostname otherwise.</entry>
</row>
<row>
<entry><literal>user.uapi.match_machine_id</literal></entry>
<entry>The machine ID, see
<citerefentry><refentrytitle>machine-id</refentrytitle><manvolnum>5</manvolnum></citerefentry>,
formatted as 32 lowercase hexadecimal characters.</entry>
</row>
<row>
<entry><literal>user.uapi.match_machine_tags</literal></entry>
<entry>The machine tags, see <varname>TAGS=</varname> in
<citerefentry><refentrytitle>machine-info</refentrytitle><manvolnum>5</manvolnum></citerefentry>.</entry>
</row>
<row>
<entry><literal>user.uapi.match_image_id</literal></entry>
<entry><varname>IMAGE_ID=</varname> from
<citerefentry><refentrytitle>os-release</refentrytitle><manvolnum>5</manvolnum></citerefentry>.</entry>
</row>
<row>
<entry><literal>user.uapi.match_image_version</literal></entry>
<entry><varname>IMAGE_VERSION=</varname> from
<citerefentry><refentrytitle>os-release</refentrytitle><manvolnum>5</manvolnum></citerefentry>.</entry>
</row>
<row>
<entry><literal>user.uapi.match_os_id</literal></entry>
<entry><varname>ID=</varname> from
<citerefentry><refentrytitle>os-release</refentrytitle><manvolnum>5</manvolnum></citerefentry>.</entry>
</row>
<row>
<entry><literal>user.uapi.match_os_version</literal></entry>
<entry><varname>VERSION_ID=</varname> from
<citerefentry><refentrytitle>os-release</refentrytitle><manvolnum>5</manvolnum></citerefentry>.</entry>
</row>
</tbody>
</tgroup>
</table>
<para>Each extended attribute contains a list of entries, separated by NUL bytes. A value without
any NUL byte is a list with a single entry. For all extended attributes but the two version ones,
each entry is a shell-style glob pattern (see
<citerefentry project='man-pages'><refentrytitle>fnmatch</refentrytitle><manvolnum>3</manvolnum></citerefentry>),
and the instance matches if <emphasis>any</emphasis> of the entries matches the system property.
For <literal>user.uapi.match_machine_tags</literal> the instance matches if any entry matches any of
the machine's tags. For <literal>user.uapi.match_image_version</literal> and
<literal>user.uapi.match_os_version</literal> each entry is a version comparison expression, i.e.
a version prefixed with one of <literal>&lt;</literal>, <literal>&lt;=</literal>,
<literal>=</literal>, <literal>!=</literal>, <literal>&gt;=</literal>, <literal>&gt;</literal>
(or a glob pattern, optionally prefixed with <literal>$=</literal> or <literal>!$=</literal>), the
same syntax <varname>ConditionVersion=</varname> in
<citerefentry><refentrytitle>systemd.unit</refentrytitle><manvolnum>5</manvolnum></citerefentry>
accepts, and the instance matches if <emphasis>all</emphasis> of the expressions hold.</para>
<para>An instance that does not carry the extended attribute is not restricted by it. If the system
property is not set (for example, if <filename>/etc/os-release</filename> carries no
<varname>IMAGE_ID=</varname>), an instance that carries the extended attribute does not match. An
instance whose extended attribute cannot be parsed does not match either. If multiple of these
extended attributes are set, instances must match all of them. Instances that do not
match are ignored, as if they did not exist. The system properties are only read once an instance
actually carries one of these extended attributes.</para>
<para>The extended attributes are taken from the <literal>extendedAttributes</literal> field of the
file objects in UAPI.16 manifests (see <varname>ManifestFormat=</varname>), as well as from the files
and directories in the file system, for local sources. When an instance is installed from a source
with a UAPI.16 manifest, the <literal>user</literal> extended attributes listed in the manifest are
applied to the file or directory created (see <varname>ManifestFormat=</varname> above), hence the
restrictions are retained. Installed instances are checked only if <varname>MatchXAttr=</varname> is
enabled in the [Target] section too, see below.</para>
<para>This setting is intended to make specific versions unavailable to certain systems
altogether, for example to offer a build only to certain machines, or only to systems of a certain
image type. Instances that do not match are simply not candidates for the system: the newest version
with matching instances is picked instead. It is not intended as a mechanism to make systems pass
through specific intermediary versions while updating, nor to offer different builds of the same
version to different systems.</para>
<para>Note that a version is only offered as update if all transfers have a matching instance of
it. If no instance of a version matches for one transfer, the whole version is considered incomplete
and is not offered at all. Hence, the extended attributes should typically be applied to the
instances of all transfers of a version alike. This also means that this setting cannot be used to
make a transfer optional. To make parts of an update optional, use <varname>Features=</varname>
instead, see above.</para>
<para>Example: the extended attribute of an instance suitable for any system running Fedora 41 or
newer (but older than 50) may be set with:</para>
<programlisting># setfattr -n user.uapi.match_os_version -v '0x3e3d3431003c353000' foo_7.raw</programlisting>
<para>(This sets the NUL separated list <literal>&gt;=41</literal>, <literal>&lt;50</literal>.)
Combined with <literal>user.uapi.match_os_id</literal> set to <literal>fedora</literal>, only Fedora
systems of a suitable version will consider the instance (as long as
<varname>MatchXAttr=</varname> is not turned off in the [Source] section of the transfer
file).</para>
<para>Example: in order to offer version 8 to machines tagged <literal>canary</literal> before
everybody else, set the extended attribute on its instance with:</para>
<programlisting># setfattr -n user.uapi.match_machine_tags -v canary foo_8.raw</programlisting>
<para>Machines that carry the <literal>canary</literal> tag (see <varname>TAGS=</varname> in
<citerefentry><refentrytitle>machine-info</refentrytitle><manvolnum>5</manvolnum></citerefentry>)
will update to version 8, while all others will skip it and stay at (or update to) version 7. Once
version 8 shall be offered to everybody, the extended attribute is simply removed again.</para>
<xi:include href="version-info.xml" xpointer="v263"/></listitem>
</varlistentry>
</variablelist>
</refsect1>
@@ -851,6 +1044,20 @@
<xi:include href="version-info.xml" xpointer="v251"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>MatchXAttr=</varname></term>
<listitem><para>Takes a boolean argument, defaults to off (unlike the setting of the same name in
the [Source] section, which defaults to on). Works like that (see above), but applies to installed
instances, whose extended attributes are read from the installed files and directories. Installed instances that do not match
are ignored, as if they did not exist: they are neither considered when determining the currently
installed version, nor removed to make room for new instances (see
<varname>InstancesMax=</varname>). Not supported for <constant>partition</constant> targets, as
partitions do not carry extended attributes.</para>
<xi:include href="version-info.xml" xpointer="v263"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>PartitionUUID=</varname></term>
<term><varname>PartitionFlags=</varname></term>
src/shared/data-fd-util.c
+39 -16
View File
@@ -18,6 +18,21 @@
/* If memfd didn't work out, then let's use a file in /tmp up to a size of 1M. If it's large than that use /var/tmp/ instead. */
#define DATA_FD_TMP_LIMIT (1U * U64_MB)
static int memfd_seal_and_reopen(int fd, int mode) {
int r;
assert(fd >= 0);
/* Seals the memfd and returns a new fd to it with the specified access mode, positioned at the
* beginning of the file. */
r = memfd_set_sealed(fd);
if (r < 0)
return r;
return fd_reopen(fd, mode);
}
int copy_data_fd_full(int fd, uint64_t max_size) {
_cleanup_close_ int copy_fd = -EBADF, tmp_fd = -EBADF;
const char *td;
@@ -67,20 +82,16 @@ int copy_data_fd_full(int fd, uint64_t max_size) {
if (remaining != UINT64_MAX)
remaining -= n;
if (r == 0)
/* Did it fit into the limit? If so, we are done. (Reopening positions the new fd
* at the beginning of the file, no need to seek.) */
return memfd_seal_and_reopen(copy_fd, O_RDONLY);
off_t f = lseek(copy_fd, 0, SEEK_SET);
if (f < 0)
return -errno;
if (f != 0)
return -EIO;
if (r == 0) {
/* Did it fit into the limit? If so, we are done. */
r = memfd_set_sealed(copy_fd);
if (r < 0)
return r;
return TAKE_FD(copy_fd);
}
}
/* If we have reason to believe this will fit fine in /tmp, then use that as first fallback. */
@@ -223,13 +234,8 @@ int memfd_clone_fd(int fd, const char *name, int mode) {
if (r < 0)
return r;
if (ro) {
r = memfd_set_sealed(mfd);
if (r < 0)
return r;
return fd_reopen(mfd, mode);
}
if (ro)
return memfd_seal_and_reopen(mfd, mode);
off_t f = lseek(mfd, 0, SEEK_SET);
if (f < 0)
@@ -237,3 +243,20 @@ int memfd_clone_fd(int fd, const char *name, int mode) {
return TAKE_FD(mfd);
}
int data_fd_from_iovec(const char *name, const struct iovec *iov) {
_cleanup_close_ int fd = -EBADF;
/* Places the specified data in a sealed memfd and returns a read-only fd to it, positioned at the
* beginning of the file. This is useful for passing literal data to other processes, for example as
* standard input. Unlike copy_data_fd() this never falls back to temporary files, the data is in
* memory already after all. */
assert(iov);
fd = memfd_new_and_seal(name, iov->iov_base, iov->iov_len);
if (fd < 0)
return fd;
return fd_reopen(fd, O_RDONLY);
}
src/shared/data-fd-util.h
+1
View File
@@ -10,3 +10,4 @@ static inline int copy_data_fd(int fd) {
int seekable_data_fd(int fd, uint64_t max_size);
int open_seekable_data_fd(int dir_fd, const char *path, uint64_t max_size);
int memfd_clone_fd(int fd, const char *name, int mode);
int data_fd_from_iovec(const char *name, const struct iovec *iov);
src/shared/forward.h
+2
View File
@@ -95,7 +95,9 @@ typedef struct Table Table;
typedef struct Tpm2Context Tpm2Context;
typedef struct Tpm2Handle Tpm2Handle;
typedef struct Tpm2PCRValue Tpm2PCRValue;
typedef struct Uapi16Contents Uapi16Contents;
typedef struct Uapi16Object Uapi16Object;
typedef struct Uapi16Xattr Uapi16Xattr;
typedef struct UnitInfo UnitInfo;
typedef struct UnitResult UnitResult;
typedef struct UserRecord UserRecord;
src/shared/uapi16-manifest.c
+1 -1
View File
@@ -536,7 +536,7 @@ static int json_dispatch_contents(const char *name, sd_json_variant *variant, sd
if (r < 0)
return r;
if (!!c.file + !!c.url + !!iovec_is_set(&c.literal) > 1)
if (!!c.file + !!c.url + uapi16_contents_has_literal(&c) > 1)
return json_log(variant, flags, SYNTHETIC_ERRNO(EINVAL), "Only one of 'file', 'url', 'literal' may be set for '%s'.", strna(name));
if (c.encoding) {
src/shared/uapi16-manifest.h
+7
View File
@@ -26,6 +26,13 @@ typedef struct Uapi16Contents {
.original_size = UINT64_MAX, \
}
static inline bool uapi16_contents_has_literal(const Uapi16Contents *c) {
/* An empty "literal" field is valid (it encodes empty data), hence check whether the field was set,
* not whether it carries data: when decoding Base64 we always allocate a buffer, even for empty
* data. */
return c->literal.iov_base;
}
/* An extended attribute in the "user" namespace */
typedef struct Uapi16Xattr {
char *name; /* The attribute name, without the "user." prefix */
src/shared/varlink-io.systemd.SysUpdate.Provider.c
+9 -3
View File
@@ -53,7 +53,9 @@ static SD_VARLINK_DEFINE_ENUM_TYPE(
static SD_VARLINK_DEFINE_ENUM_TYPE(
ManifestFormat,
SD_VARLINK_FIELD_COMMENT("A SHA256SUMS file, see ManifestFormat=sha256sums in sysupdate.d(5)."),
SD_VARLINK_DEFINE_ENUM_VALUE(sha256sums));
SD_VARLINK_DEFINE_ENUM_VALUE(sha256sums),
SD_VARLINK_FIELD_COMMENT("A UAPI.16 manifest, see ManifestFormat=uapi16 in sysupdate.d(5)."),
SD_VARLINK_DEFINE_ENUM_VALUE(uapi16));
static SD_VARLINK_DEFINE_STRUCT_TYPE(
TransferSource,
@@ -68,7 +70,9 @@ static SD_VARLINK_DEFINE_STRUCT_TYPE(
SD_VARLINK_FIELD_COMMENT("The format of the manifest file listing the available source files, see ManifestFormat= in the [Source] section of sysupdate.d(5). If unset, defaults to 'sha256sums'."),
SD_VARLINK_DEFINE_FIELD_BY_TYPE(manifestFormat, ManifestFormat, SD_VARLINK_NULLABLE),
SD_VARLINK_FIELD_COMMENT("The URL of the manifest file, see ManifestPath= in the [Source] section of sysupdate.d(5). If unset, the manifest is acquired from the source URL, under the file name appropriate for the manifest format. No specifier expansion is applied."),
SD_VARLINK_DEFINE_FIELD(manifestPath, SD_VARLINK_STRING, SD_VARLINK_NULLABLE));
SD_VARLINK_DEFINE_FIELD(manifestPath, SD_VARLINK_STRING, SD_VARLINK_NULLABLE),
SD_VARLINK_FIELD_COMMENT("Whether to match the user.uapi.match_* extended attributes of instances against the local system, see MatchXAttr= in the [Source] section of sysupdate.d(5). If unset, defaults to true."),
SD_VARLINK_DEFINE_FIELD(matchXAttr, SD_VARLINK_BOOL, SD_VARLINK_NULLABLE));
static SD_VARLINK_DEFINE_STRUCT_TYPE(
TransferTarget,
@@ -105,7 +109,9 @@ static SD_VARLINK_DEFINE_STRUCT_TYPE(
SD_VARLINK_FIELD_COMMENT("Whether to remove temporary files from previous runs, see RemoveTemporary= in the [Target] section of sysupdate.d(5)."),
SD_VARLINK_DEFINE_FIELD(removeTemporary, SD_VARLINK_BOOL, SD_VARLINK_NULLABLE),
SD_VARLINK_FIELD_COMMENT("The path of a symlink to maintain pointing to the newest instance, see CurrentSymlink= in the [Target] section of sysupdate.d(5). No specifier expansion is applied."),
SD_VARLINK_DEFINE_FIELD(currentSymlink, SD_VARLINK_STRING, SD_VARLINK_NULLABLE));
SD_VARLINK_DEFINE_FIELD(currentSymlink, SD_VARLINK_STRING, SD_VARLINK_NULLABLE),
SD_VARLINK_FIELD_COMMENT("Whether to match the user.uapi.match_* extended attributes of instances against the local system, see MatchXAttr= in the [Target] section of sysupdate.d(5). If unset, defaults to false."),
SD_VARLINK_DEFINE_FIELD(matchXAttr, SD_VARLINK_BOOL, SD_VARLINK_NULLABLE));
static SD_VARLINK_DEFINE_STRUCT_TYPE(
Transfer,
src/sysupdate/sysupdate-instance.c
+7
View File
@@ -3,6 +3,7 @@
#include "alloc-util.h"
#include "log.h"
#include "sysupdate-instance.h"
#include "uapi16-manifest.h"
void instance_metadata_destroy(InstanceMetadata *m) {
assert(m);
@@ -58,5 +59,11 @@ Instance *instance_free(Instance *i) {
free(i->path);
partition_info_destroy(&i->partition_info);
FOREACH_ARRAY(s, i->sources, i->n_sources)
uapi16_contents_done(s);
free(i->sources);
uapi16_xattr_array_free(i->user_xattrs, i->n_user_xattrs);
return mfree(i);
}
src/sysupdate/sysupdate-instance.h
+12 -2
View File
@@ -8,8 +8,8 @@
#include "sysupdate-partition.h"
typedef struct InstanceMetadata {
/* Various bits of metadata for each instance, that is either derived from the filename/GPT label or
* from metadata of the file/partition itself */
/* Various bits of metadata for each instance, that is either derived from the filename/GPT label,
* from metadata of the file/partition itself or from the manifest */
char *version;
sd_id128_t partition_uuid;
bool partition_uuid_set;
@@ -49,6 +49,16 @@ struct Instance {
char *path; /* includes the `.sysupdate.partial.` (etc.) prefix, if applicable */
PartitionInfo partition_info;
/* The sources we can acquire the resource from (for web backed sources) */
Uapi16Contents *sources;
size_t n_sources;
/* The "user" extended attributes listed in the manifest (for web backed sources), which we apply
* to the inode we create when installing the instance */
Uapi16Xattr *user_xattrs;
size_t n_user_xattrs;
/* The current state of this instance */
bool is_partial;
bool is_pending;
};
src/sysupdate/sysupdate-resource.c
+395 -17
View File
@@ -8,6 +8,7 @@
#include "blockdev-util.h"
#include "build-path.h"
#include "chase.h"
#include "compress.h"
#include "device-util.h"
#include "dirent-util.h"
#include "env-util.h"
@@ -37,6 +38,9 @@
#include "sysupdate-pattern.h"
#include "sysupdate-resource.h"
#include "time-util.h"
#include "uapi16-manifest.h"
#include "web-util.h"
#include "xattr-match.h"
void resource_destroy(Resource *rr) {
assert(rr);
@@ -62,6 +66,12 @@ bool resource_has_glob_directory_pattern(Resource *rr) {
return false;
}
unsigned resource_xattr_match_mask(const Resource *rr) {
assert(rr);
return rr->match_xattr ? XATTR_MATCH_MASK_ALL : 0;
}
static int resource_add_instance(
Resource *rr,
const char *path,
@@ -98,7 +108,8 @@ static int resource_load_from_directory_recursive(
const char* relpath_for_matching,
mode_t m,
bool ancestor_is_partial,
bool ancestor_is_pending) {
bool ancestor_is_pending,
const XAttrMatchFilter *filter) {
int r;
for (;;) {
@@ -203,7 +214,7 @@ static int resource_load_from_directory_recursive(
return log_error_errno(errno, "Failed to open directory %s/%s: %m", rr->path, rel_joined);
}
r = resource_load_from_directory_recursive(rr, subdir, rel_joined, rel_joined_for_matching, m, is_partial, is_pending);
r = resource_load_from_directory_recursive(rr, subdir, rel_joined, rel_joined_for_matching, m, is_partial, is_pending, filter);
if (r < 0)
return r;
continue;
@@ -212,6 +223,13 @@ static int resource_load_from_directory_recursive(
if (S_ISDIR(st.st_mode) && m != S_IFDIR)
continue;
/* Skip instances restricted to other systems via extended attributes */
r = xattr_match_filter_test_inode(filter, dirfd(d), de->d_name);
if (r < 0)
return log_error_errno(r, "Failed to check extended attributes of '%s': %m", de->d_name);
if (r == 0)
continue;
joined = path_join(rr->path, rel_joined);
if (!joined)
return log_oom();
@@ -239,7 +257,8 @@ static int resource_load_from_directory_recursive(
static int resource_load_from_directory(
Resource *rr,
mode_t m) {
mode_t m,
const XAttrMatchFilter *filter) {
_cleanup_closedir_ DIR *d = NULL;
assert(rr);
@@ -256,7 +275,15 @@ static int resource_load_from_directory(
return log_error_errno(errno, "Failed to open directory '%s': %m", rr->path);
}
return resource_load_from_directory_recursive(rr, d, /* relpath= */ NULL, /* relpath_for_matching= */ NULL, m, /* ancestor_is_partial= */ false, /* ancestor_is_pending= */ false);
return resource_load_from_directory_recursive(
rr,
d,
/* relpath= */ NULL,
/* relpath_for_matching= */ NULL,
m,
/* ancestor_is_partial= */ false,
/* ancestor_is_pending= */ false,
filter);
}
static int resource_load_from_blockdev(Resource *rr) {
@@ -361,11 +388,22 @@ static int resource_load_from_blockdev(Resource *rr) {
return 0;
}
static const char* const resource_manifest_filename_table[_MANIFEST_FORMAT_MAX] = {
[MANIFEST_FORMAT_SHA256SUMS] = "SHA256SUMS",
[MANIFEST_FORMAT_UAPI16] = UAPI16_MANIFEST_FILENAME,
};
DEFINE_PRIVATE_STRING_TABLE_LOOKUP_TO_STRING(resource_manifest_filename, ManifestFormat);
static bool resource_manifest_has_inline_signatures(ManifestFormat style) {
return style == MANIFEST_FORMAT_UAPI16;
}
static int resource_manifest_url(Resource *rr, char **ret) {
int r;
assert(rr);
assert(rr->manifest_format == MANIFEST_FORMAT_SHA256SUMS);
assert(IN_SET(rr->manifest_format, MANIFEST_FORMAT_SHA256SUMS, MANIFEST_FORMAT_UAPI16));
assert(ret);
/* Returns the URL to acquire the manifest from: either the one explicitly configured, or the
@@ -379,9 +417,10 @@ static int resource_manifest_url(Resource *rr, char **ret) {
return 0;
}
r = import_url_append_component(rr->path, "SHA256SUMS", ret);
const char *fn = resource_manifest_filename_to_string(rr->manifest_format);
r = import_url_append_component(rr->path, fn, ret);
if (r < 0)
return log_error_errno(r, "Failed to append SHA256SUMS to URL: %m");
return log_error_errno(r, "Failed to append '%s' to URL '%s': %m", fn, rr->path);
return 0;
}
@@ -394,11 +433,11 @@ static int download_manifest(
int r;
assert(manifest_format == MANIFEST_FORMAT_SHA256SUMS);
assert(IN_SET(manifest_format, MANIFEST_FORMAT_SHA256SUMS, MANIFEST_FORMAT_UAPI16));
assert(manifest_url);
assert(ret);
/* Download a SHA256SUMS file as manifest */
/* Download a SHA256SUMS/Uapi16Manifest file as manifest */
_cleanup_close_pair_ int pfd[2] = EBADF_PAIR;
if (pipe2(pfd, O_CLOEXEC) < 0)
@@ -424,7 +463,7 @@ static int download_manifest(
"raw",
"--direct", /* just download the specified URL, don't download anything else */
"--checksum-file=yes", /* whatever its name, look for <name>.gpg/.asc signature */
"--verify", verify_signature ? "signature" : "no", /* verify the manifest file */
"--verify", verify_signature && !resource_manifest_has_inline_signatures(manifest_format) ? "signature" : "no", /* verify the manifest file */
manifest_url,
"-", /* write to stdout */
NULL
@@ -626,10 +665,318 @@ static int sha256sums_process(
}
}
static int instance_copy_uapi16(Instance *instance, const Uapi16Object *object) {
assert(instance);
assert(object);
/* We verified earlier that these fields are definitely set in the UAPI 16 objects */
assert(object->type == S_IFREG);
assert(iovec_is_set(&object->sha256));
assert(object->size != UINT64_MAX);
if (instance->metadata.sha256sum_set) {
if (memcmp_nn(instance->metadata.sha256sum, SHA256_DIGEST_SIZE, object->sha256.iov_base, object->sha256.iov_len) != 0)
return log_error_errno(SYNTHETIC_ERRNO(EINVAL), "SHA256 sum parsed from filename and manifest don't match, refusing.");
} else {
assert(object->sha256.iov_len == SHA256_DIGEST_SIZE);
memcpy(instance->metadata.sha256sum, object->sha256.iov_base, SHA256_DIGEST_SIZE);
instance->metadata.sha256sum_set = true;
}
if (instance->metadata.size != UINT64_MAX) {
if (instance->metadata.size != object->size)
return log_error_errno(SYNTHETIC_ERRNO(EINVAL), "Size parsed from filename and manifest don't match, refusing.");
} else
instance->metadata.size = object->size;
if (instance->metadata.mode == MODE_INVALID)
instance->metadata.mode = object->mode;
if (instance->metadata.mtime == USEC_INFINITY && object->mtime_nsec < (NSEC_INFINITY / NSEC_PER_USEC) * NSEC_PER_USEC)
instance->metadata.mtime = object->mtime_nsec / NSEC_PER_USEC;
if (!instance->metadata.partition_uuid_set && object->gpt_uuid_set) {
instance->metadata.partition_uuid = object->gpt_uuid;
instance->metadata.partition_uuid_set = true;
}
if (!instance->metadata.partition_flags_set && object->gpt_flags_set) {
instance->metadata.partition_flags = object->gpt_flags;
instance->metadata.partition_flags_set = true;
}
if (instance->metadata.no_auto < 0 && object->gpt_no_auto >= 0)
instance->metadata.no_auto = object->gpt_no_auto;
if (instance->metadata.growfs < 0 && object->gpt_growfs >= 0)
instance->metadata.growfs = object->gpt_growfs;
if (instance->metadata.read_only < 0 && object->read_only >= 0)
instance->metadata.read_only = object->read_only;
return 0;
}
static bool uapi16_encoding_is_supported(const char *encoding) {
/* The compression algorithms systemd-pull and systemd-import can decode, by their canonical names,
* which is what we pass on to them via --compression= */
Compression c = compression_from_string(encoding);
return c >= 0 && c != COMPRESSION_NONE && compression_supported(c);
}
static int uapi16_object_resolve_sources(Resource *rr, Uapi16Object *o) {
size_t n = 0;
int r;
assert(rr);
assert(o);
/* Validates the "contents" array of a file object, i.e. the list of alternative sources the file may
* be acquired from. Resolves relative "file" references against Path= into absolute URLs, and drops
* entries we cannot make use of. Returns > 0 if the object is usable (i.e. at least one source
* remains, or none was listed to begin with, in which case the file is acquired by its name), and 0
* if it shall be skipped. */
for (size_t k = 0; k < o->n_contents; k++) {
Uapi16Contents *c = o->contents + k;
bool usable = true;
/* An entry with neither "file", "url" nor "literal" set refers to a file of the object's
* name, see the UAPI.16 specification. */
if (!c->file && !c->url && !uapi16_contents_has_literal(c)) {
c->file = strdup(o->name);
if (!c->file)
return log_oom();
}
if (c->file) {
/* Refuse "%" for the same reason as in sha256sums_line() */
if (!uapi16_name_is_valid(c->file) || strchr(c->file, '%')) {
log_warning("File '%s' refers to invalid file name '%s' in its contents, ignoring source.", o->name, c->file);
usable = false;
} else {
/* "file" references are relative to Path=, even if the manifest itself is
* acquired from elsewhere via ManifestPath=, the same way as files that list
* no "contents" at all */
_cleanup_free_ char *url = NULL;
r = import_url_append_component(rr->path, c->file, &url);
if (r < 0)
return log_error_errno(r, "Failed to build URL for '%s': %m", c->file);
free_and_replace(c->url, url);
c->file = mfree(c->file);
}
} else if (c->url) {
if (!http_url_is_valid(c->url) && !file_url_is_valid(c->url)) {
log_warning("File '%s' refers to invalid URL '%s' in its contents, ignoring source.", o->name, c->url);
usable = false;
}
}
if (usable && c->encoding && !uapi16_encoding_is_supported(c->encoding)) {
log_warning("File '%s' has a source with unsupported encoding '%s', ignoring source.", o->name, c->encoding);
usable = false;
}
/* Whether the decoded source consists of the file and nothing else */
bool whole = c->offset == 0 && (c->original_size == UINT64_MAX || c->original_size == o->size);
if (usable && !whole) {
if (!c->url && c->encoding) {
/* We cannot cut into encoded inline data, and leave decoding to systemd-import */
log_warning("File '%s' has encoded inline data that carries more than the file, ignoring source.", o->name);
usable = false;
} else if (c->url && rr->type == RESOURCE_URL_TAR) {
/* systemd-pull can only acquire a range of a source for raw images */
log_warning("File '%s' has a source that carries more than the file, which is not supported for tarballs, ignoring source.", o->name);
usable = false;
}
}
if (usable && c->url && !iovec_is_set(&c->encoded_sha256)) {
/* Without the digest of the downloaded bytes we cannot verify the download as a
* whole. If the source is not encoded and consists of the file and nothing else, it
* is identical to the file though, and we can use the file's digest. Otherwise we
* can still verify the file once it has been decoded and cut out of the source, as
* long as its digest is known. */
if (!c->encoding && whole) {
if (!iovec_memdup(&o->sha256, &c->encoded_sha256))
return log_oom();
} else if (!iovec_is_set(&o->sha256)) {
log_warning("File '%s' has a source without SHA256 digest that does not consist of the file alone, and no SHA256 digest of its own, ignoring source.", o->name);
usable = false;
}
}
if (!usable) {
uapi16_contents_done(c);
continue;
}
/* Keep the array compact */
if (n != k) {
o->contents[n] = *c;
*c = UAPI16_CONTENTS_NULL;
}
n++;
}
if (o->n_contents > 0 && n == 0) {
log_warning("File '%s' has no usable source, skipping.", o->name);
o->n_contents = 0;
return 0;
}
o->n_contents = n;
return 1;
}
typedef struct Uapi16ProcessContext {
Resource *resource;
const XAttrMatchFilter *filter;
const GptPartitionType *match_partition_type;
} Uapi16ProcessContext;
static int uapi16_process_object(Uapi16Object *object, size_t index, void *userdata) {
Uapi16ProcessContext *pc = ASSERT_PTR(userdata);
Resource *rr = ASSERT_PTR(pc->resource);
int r;
assert(object);
/* Honour validity properties, but otherwise ignore intro object */
if (index == 0) {
r = uapi16_object_check_validity(object);
if (r < 0) {
if (verify_freshness())
return log_error_errno(r, "Manifest file marked invalid: %m");
log_warning_errno(r, "Manifest file marked invalid, but freshness checks disabled, proceeding.");
}
return 0;
}
/* Ignore non-regular files */
if (!S_ISREG(object->type)) {
log_debug("Skipping file '%s' as it is not a regular file.", object->name);
return 0;
}
/* Ignore files whose partition type does not match the one of the partitions we'd write them to */
if (object->gpt_type_uuid_set && pc->match_partition_type &&
!sd_id128_equal(object->gpt_type_uuid, pc->match_partition_type->uuid)) {
log_debug("Skipping file '%s' as its partition type does not match the target's.", object->name);
return 0;
}
r = uapi16_object_check_validity(object);
if (r < 0) {
if (verify_freshness()) {
log_warning_errno(r, "File '%s' marked invalid, skipping: %m", object->name);
return 0;
}
log_warning_errno(r, "File '%s' marked invalid, but freshness checks disabled, proceeding.", object->name);
}
if (object->size == UINT64_MAX) {
log_warning_errno(SYNTHETIC_ERRNO(EBADMSG), "File '%s' lacks a size, skipping.", object->name);
return 0;
}
if (!iovec_is_set(&object->sha256)) {
log_warning_errno(SYNTHETIC_ERRNO(EBADMSG), "File '%s' lacks a SHA256 digest, skipping.", object->name);
return 0;
}
/* Refuse "%" for the same reason as in sha256sums_line() */
if (strchr(object->name, '%')) {
log_warning_errno(SYNTHETIC_ERRNO(EBADMSG), "File '%s' contains a '%%' character, skipping.", object->name);
return 0;
}
r = uapi16_object_resolve_sources(rr, object);
if (r <= 0)
return r;
_cleanup_(instance_metadata_destroy) InstanceMetadata extracted_fields = INSTANCE_METADATA_NULL;
r = pattern_match_many(rr->patterns, object->name, &extracted_fields);
if (r < 0)
return log_error_errno(r, "Failed to match pattern: %m");
if (!IN_SET(r, PATTERN_MATCH_YES, PATTERN_MATCH_YES_AND_RETRY))
return 0;
/* Skip instances restricted to other systems via extended attributes */
r = xattr_match_filter_test_uapi16_object(pc->filter, object);
if (r < 0)
return log_error_errno(r, "Failed to check extended attributes of '%s' listed in manifest: %m", object->name);
if (r == 0)
return 0;
_cleanup_free_ char *path = NULL;
r = import_url_append_component(rr->path, object->name, &path);
if (r < 0)
return log_error_errno(r, "Failed to build instance URL: %m");
Instance *instance;
r = resource_add_instance(rr, path, &extracted_fields, &instance);
if (r < 0)
return r;
/* Now copy the fields from the manifest into our instances logic */
r = instance_copy_uapi16(instance, object);
if (r < 0)
return r;
/* And take possession of the list of sources, we'll need it when acquiring the file */
instance->sources = TAKE_PTR(object->contents);
instance->n_sources = object->n_contents;
object->n_contents = 0;
/* Similar, for the extended attributes to apply when installing */
instance->user_xattrs = TAKE_PTR(object->user_xattrs);
instance->n_user_xattrs = object->n_user_xattrs;
object->n_user_xattrs = 0;
return 0;
}
static int uapi16_process(
Resource *rr,
const struct iovec *manifest,
X509 *const *certificates,
size_t n_certificates,
const XAttrMatchFilter *filter,
const GptPartitionType *match_partition_type) {
assert(rr);
assert(manifest);
if (!iovec_is_set(manifest))
return log_error_errno(SYNTHETIC_ERRNO(EBADMSG), "Manifest is empty, refusing.");
_cleanup_fclose_ FILE *f = fmemopen_unlocked(manifest->iov_base, manifest->iov_len, "r");
if (!f)
return log_error_errno(errno, "Failed to open manifest buffer as stream: %m");
return uapi16_manifest_read(
f,
certificates,
n_certificates,
uapi16_process_object,
&(Uapi16ProcessContext) {
.resource = rr,
.filter = filter,
.match_partition_type = match_partition_type,
});
}
static int resource_load_from_web(
Resource *rr,
bool verify,
Hashmap **web_cache) {
X509 *const *certificates,
size_t n_certificates,
Hashmap **web_cache,
const XAttrMatchFilter *filter,
const GptPartitionType *match_partition_type) {
_cleanup_(iovec_done) struct iovec buf = {};
struct iovec manifest = {};
@@ -639,7 +986,7 @@ static int resource_load_from_web(
POINTER_MAY_BE_NULL(web_cache);
/* Key the cache by the URL of the manifest itself, not the one of the resource, as the former
* depends on ManifestPath= too. */
* depends on ManifestFormat= and ManifestPath= too. */
_cleanup_free_ char *manifest_url = NULL;
r = resource_manifest_url(rr, &manifest_url);
if (r < 0)
@@ -660,7 +1007,27 @@ static int resource_load_from_web(
manifest = buf;
}
r = sha256sums_process(rr, &manifest);
switch (rr->manifest_format) {
case MANIFEST_FORMAT_SHA256SUMS:
r = sha256sums_process(rr, &manifest);
break;
case MANIFEST_FORMAT_UAPI16:
/* UAPI.16 manifests carry their signatures inline, hence systemd-pull didn't verify
* anything, and we do it here. */
r = uapi16_process(
rr,
&manifest,
verify ? certificates : NULL,
verify ? n_certificates : 0,
filter,
match_partition_type);
break;
default:
assert_not_reached();
}
if (r < 0)
return r;
@@ -696,21 +1063,32 @@ static int instance_cmp(Instance *const*a, Instance *const*b) {
return path_compare((*a)->path, (*b)->path);
}
int resource_load_instances(Resource *rr, bool verify, Hashmap **web_cache) {
int resource_load_instances(
Resource *rr,
bool verify,
X509 *const *certificates,
size_t n_certificates,
Hashmap **web_cache,
const XAttrMatchFilter *filter,
const GptPartitionType *match_partition_type) {
int r;
assert(rr);
/* If match_partition_type is specified, files listed in UAPI.16 manifests that declare a different
* partition type are skipped. This is used to filter the sources by the partition type of the
* target the files would be written to. */
switch (rr->type) {
case RESOURCE_TAR:
case RESOURCE_REGULAR_FILE:
r = resource_load_from_directory(rr, S_IFREG);
r = resource_load_from_directory(rr, S_IFREG, filter);
break;
case RESOURCE_DIRECTORY:
case RESOURCE_SUBVOLUME:
r = resource_load_from_directory(rr, S_IFDIR);
r = resource_load_from_directory(rr, S_IFDIR, filter);
break;
case RESOURCE_PARTITION:
@@ -719,7 +1097,7 @@ int resource_load_instances(Resource *rr, bool verify, Hashmap **web_cache) {
case RESOURCE_URL_FILE:
case RESOURCE_URL_TAR:
r = resource_load_from_web(rr, verify, web_cache);
r = resource_load_from_web(rr, verify, certificates, n_certificates, web_cache, filter, match_partition_type);
break;
default:
src/sysupdate/sysupdate-resource.h
+13 -2
View File
@@ -4,6 +4,7 @@
#include "gpt.h"
#include "manifest-util.h"
#include "sysupdate-forward.h"
#include "xattr-match.h"
typedef enum ResourceType {
RESOURCE_URL_FILE,
@@ -79,8 +80,9 @@ typedef struct Resource {
char **patterns;
GptPartitionType partition_type;
bool partition_type_set;
ManifestFormat manifest_format; /* Only MANIFEST_FORMAT_SHA256SUMS for now */
ManifestFormat manifest_format; /* MANIFEST_FORMAT_SHA256SUMS or MANIFEST_FORMAT_UAPI16 */
char *manifest_path;
bool match_xattr; /* MatchXAttr= */
/* All instances of this resource we found */
Instance **instances;
@@ -94,7 +96,16 @@ void resource_destroy(Resource *rr);
bool resource_has_glob_directory_pattern(Resource *rr);
int resource_load_instances(Resource *rr, bool verify, Hashmap **web_cache);
unsigned resource_xattr_match_mask(const Resource *rr);
int resource_load_instances(
Resource *rr,
bool verify,
X509 *const *certificates,
size_t n_certificates,
Hashmap **web_cache,
const XAttrMatchFilter *filter,
const GptPartitionType *match_partition_type);
Instance* resource_find_instance(Resource *rr, const char *version);
src/sysupdate/sysupdate-transfer.c
+347 -92
View File
@@ -11,6 +11,7 @@
#include "build-path.h"
#include "chase.h"
#include "conf-parser.h"
#include "data-fd-util.h"
#include "dirent-util.h"
#include "errno-util.h"
#include "event-util.h"
@@ -23,6 +24,8 @@
#include "hexdecoct.h"
#include "install-file.h"
#include "json-util.h"
#include "iovec-util.h"
#include "memory-util.h"
#include "mkdir.h"
#include "notify-recv.h"
#include "parse-helpers.h"
@@ -32,6 +35,7 @@
#include "pidref.h"
#include "process-util.h"
#include "rm-rf.h"
#include "sha256.h"
#include "signal-util.h"
#include "specifier.h"
#include "stdio-util.h"
@@ -47,6 +51,7 @@
#include "sysupdate-transfer.h"
#include "time-util.h"
#include "web-util.h"
#include "xattr-util.h"
/* Default value for InstancesMax= for fs object targets */
#define DEFAULT_FILE_INSTANCES_MAX 3
@@ -120,6 +125,9 @@ Transfer* transfer_new(Context *ctx) {
.context = ctx,
};
/* Available instances are matched against the system by default, installed ones are not */
t->source.match_xattr = true;
return t;
}
@@ -611,6 +619,11 @@ static int transfer_finalize(Transfer *t, const char *origin, unsigned line, Has
"Target type '%s' is incompatible with source type '%s', refusing.",
resource_type_to_string(t->target.type), resource_type_to_string(t->source.type));
/* Partitions carry no extended attributes */
if (t->target.type == RESOURCE_PARTITION && resource_xattr_match_mask(&t->target) != 0)
return log_syntax(NULL, LOG_ERR, origin, line, SYNTHETIC_ERRNO(EINVAL),
"MatchXAttr= in the [Target] section is not supported for partition targets, refusing.");
if (!t->source.path && !t->source.path_auto)
return log_syntax(NULL, LOG_ERR, origin, line, SYNTHETIC_ERRNO(EINVAL),
"Source specification lacks Path=.");
@@ -638,7 +651,7 @@ static int transfer_finalize(Transfer *t, const char *origin, unsigned line, Has
"Source path is not a valid HTTP, HTTPS, file, or provider URL: %s", t->source.path);
}
if (t->source.manifest_format != MANIFEST_FORMAT_SHA256SUMS)
if (!IN_SET(t->source.manifest_format, MANIFEST_FORMAT_SHA256SUMS, MANIFEST_FORMAT_UAPI16))
return log_syntax(NULL, LOG_ERR, origin, line, SYNTHETIC_ERRNO(EINVAL),
"Manifest format '%s' not supported for source specifications, refusing.",
manifest_format_to_string(t->source.manifest_format));
@@ -721,11 +734,13 @@ int transfer_read_definition(Transfer *t, const char *path, const char **dirs, H
{ "Source", "MatchPattern", config_parse_resource_pattern, MATCH_PATTERN_SOURCE, &t->source.patterns },
{ "Source", "ManifestFormat", config_parse_resource_manifest_format, 0, &t->source.manifest_format },
{ "Source", "ManifestPath", config_parse_resource_manifest_path, 0, &t->source.manifest_path },
{ "Source", "MatchXAttr", config_parse_bool, 0, &t->source.match_xattr },
{ "Target", "Type", config_parse_resource_type, 0, &t->target.type },
{ "Target", "Path", config_parse_resource_path, 0, &t->target },
{ "Target", "PathRelativeTo", config_parse_resource_path_relto, 0, &t->target.path_relative_to },
{ "Target", "MatchPattern", config_parse_resource_pattern, MATCH_PATTERN_TARGET, &t->target.patterns },
{ "Target", "MatchPartitionType", config_parse_resource_ptype, 0, &t->target },
{ "Target", "MatchXAttr", config_parse_bool, 0, &t->target.match_xattr },
{ "Target", "PartitionUUID", config_parse_partition_uuid, 0, t },
{ "Target", "PartitionFlags", config_parse_partition_flags, 0, t },
{ "Target", "PartitionNoAuto", config_parse_tristate, 0, &t->no_auto },
@@ -930,16 +945,29 @@ static int dispatch_current_symlink(const char *name, sd_json_variant *variant,
return free_and_replace(*current_symlink, p);
}
static int dispatch_source_match(const char *name, sd_json_variant *variant, sd_json_dispatch_flags_t flags, void *userdata) {
bool *b = ASSERT_PTR(userdata);
/* MatchXAttr= defaults to on in the [Source] section, hence map null to true */
if (sd_json_variant_is_null(variant)) {
*b = true;
return 0;
}
return sd_json_dispatch_stdbool(name, variant, flags, userdata);
}
static int dispatch_transfer_source(const char *name, sd_json_variant *variant, sd_json_dispatch_flags_t flags, void *userdata) {
Transfer *t = ASSERT_PTR(userdata);
static const sd_json_dispatch_field dispatch_table[] = {
{ "type", SD_JSON_VARIANT_STRING, dispatch_resource_type, offsetof(Transfer, source.type), SD_JSON_MANDATORY },
{ "path", SD_JSON_VARIANT_STRING, dispatch_resource_path, offsetof(Transfer, source), SD_JSON_MANDATORY },
{ "pathRelativeTo", SD_JSON_VARIANT_STRING, dispatch_path_relative_to, offsetof(Transfer, source.path_relative_to), SD_JSON_NULLABLE },
{ "matchPattern", SD_JSON_VARIANT_ARRAY, dispatch_source_patterns, offsetof(Transfer, source), SD_JSON_MANDATORY },
{ "manifestFormat", SD_JSON_VARIANT_STRING, dispatch_manifest_format, offsetof(Transfer, source.manifest_format), SD_JSON_NULLABLE },
{ "manifestPath", SD_JSON_VARIANT_STRING, sd_json_dispatch_string, offsetof(Transfer, source.manifest_path), SD_JSON_NULLABLE },
{ "type", SD_JSON_VARIANT_STRING, dispatch_resource_type, offsetof(Transfer, source.type), SD_JSON_MANDATORY },
{ "path", SD_JSON_VARIANT_STRING, dispatch_resource_path, offsetof(Transfer, source), SD_JSON_MANDATORY },
{ "pathRelativeTo", SD_JSON_VARIANT_STRING, dispatch_path_relative_to, offsetof(Transfer, source.path_relative_to), SD_JSON_NULLABLE },
{ "matchPattern", SD_JSON_VARIANT_ARRAY, dispatch_source_patterns, offsetof(Transfer, source), SD_JSON_MANDATORY },
{ "manifestFormat", SD_JSON_VARIANT_STRING, dispatch_manifest_format, offsetof(Transfer, source.manifest_format), SD_JSON_NULLABLE },
{ "manifestPath", SD_JSON_VARIANT_STRING, sd_json_dispatch_string, offsetof(Transfer, source.manifest_path), SD_JSON_NULLABLE },
{ "matchXAttr", SD_JSON_VARIANT_BOOLEAN, dispatch_source_match, offsetof(Transfer, source.match_xattr), SD_JSON_NULLABLE },
{},
};
@@ -969,6 +997,7 @@ static int dispatch_transfer_target(const char *name, sd_json_variant *variant,
{ "instancesMax", _SD_JSON_VARIANT_TYPE_INVALID, dispatch_instances_max, offsetof(Transfer, instances_max), 0 },
{ "removeTemporary", SD_JSON_VARIANT_BOOLEAN, sd_json_dispatch_tristate, offsetof(Transfer, remove_temporary), SD_JSON_NULLABLE },
{ "currentSymlink", SD_JSON_VARIANT_STRING, dispatch_current_symlink, offsetof(Transfer, current_symlink), SD_JSON_NULLABLE },
{ "matchXAttr", SD_JSON_VARIANT_BOOLEAN, sd_json_dispatch_stdbool, offsetof(Transfer, target.match_xattr), SD_JSON_NULLABLE },
{},
};
@@ -989,17 +1018,17 @@ int transfer_from_json(Transfer *t, sd_json_variant *v, const char *origin, Hash
* from, for logging purposes. */
static const sd_json_dispatch_field dispatch_table[] = {
{ "id", SD_JSON_VARIANT_STRING, json_dispatch_filename, offsetof(Transfer, id), SD_JSON_MANDATORY|SD_JSON_STRICT },
{ "minVersion", SD_JSON_VARIANT_STRING, json_dispatch_version, offsetof(Transfer, min_version), SD_JSON_NULLABLE },
{ "maxVersion", SD_JSON_VARIANT_STRING, json_dispatch_version, offsetof(Transfer, max_version), SD_JSON_NULLABLE },
{ "protectVersion", SD_JSON_VARIANT_ARRAY, json_dispatch_versions, offsetof(Transfer, protected_versions), SD_JSON_NULLABLE },
{ "verify", SD_JSON_VARIANT_BOOLEAN, sd_json_dispatch_tristate, offsetof(Transfer, verify), SD_JSON_NULLABLE },
{ "changeLog", SD_JSON_VARIANT_ARRAY, json_dispatch_http_urls, offsetof(Transfer, changelog), SD_JSON_NULLABLE },
{ "appStream", SD_JSON_VARIANT_ARRAY, json_dispatch_http_urls, offsetof(Transfer, appstream), SD_JSON_NULLABLE },
{ "features", SD_JSON_VARIANT_ARRAY, sd_json_dispatch_strv, offsetof(Transfer, features), SD_JSON_NULLABLE },
{ "requisiteFeatures", SD_JSON_VARIANT_ARRAY, sd_json_dispatch_strv, offsetof(Transfer, requisite_features), SD_JSON_NULLABLE },
{ "source", SD_JSON_VARIANT_OBJECT, dispatch_transfer_source, 0, SD_JSON_MANDATORY },
{ "target", SD_JSON_VARIANT_OBJECT, dispatch_transfer_target, 0, SD_JSON_MANDATORY },
{ "id", SD_JSON_VARIANT_STRING, json_dispatch_filename, offsetof(Transfer, id), SD_JSON_MANDATORY|SD_JSON_STRICT },
{ "minVersion", SD_JSON_VARIANT_STRING, json_dispatch_version, offsetof(Transfer, min_version), SD_JSON_NULLABLE },
{ "maxVersion", SD_JSON_VARIANT_STRING, json_dispatch_version, offsetof(Transfer, max_version), SD_JSON_NULLABLE },
{ "protectVersion", SD_JSON_VARIANT_ARRAY, json_dispatch_versions, offsetof(Transfer, protected_versions), SD_JSON_NULLABLE },
{ "verify", SD_JSON_VARIANT_BOOLEAN, sd_json_dispatch_tristate, offsetof(Transfer, verify), SD_JSON_NULLABLE },
{ "changeLog", SD_JSON_VARIANT_ARRAY, json_dispatch_http_urls, offsetof(Transfer, changelog), SD_JSON_NULLABLE },
{ "appStream", SD_JSON_VARIANT_ARRAY, json_dispatch_http_urls, offsetof(Transfer, appstream), SD_JSON_NULLABLE },
{ "features", SD_JSON_VARIANT_ARRAY, sd_json_dispatch_strv, offsetof(Transfer, features), SD_JSON_NULLABLE },
{ "requisiteFeatures", SD_JSON_VARIANT_ARRAY, sd_json_dispatch_strv, offsetof(Transfer, requisite_features), SD_JSON_NULLABLE },
{ "source", SD_JSON_VARIANT_OBJECT, dispatch_transfer_source, 0, SD_JSON_MANDATORY },
{ "target", SD_JSON_VARIANT_OBJECT, dispatch_transfer_target, 0, SD_JSON_MANDATORY },
{},
};
@@ -1468,6 +1497,7 @@ static int helper_on_notify(sd_event_source *s, int fd, uint32_t revents, void *
static int run_callout(
const char *name,
char *cmdline[],
int stdin_fd,
const Transfer *transfer,
const Instance *instance,
TransferProgress callback,
@@ -1507,7 +1537,13 @@ static int run_callout(
if (r < 0)
return log_error_errno(r, "Failed to prepare notify socket: %m");
r = pidref_safe_fork(ctx->name, FORK_RESET_SIGNALS|FORK_DEATHSIG_SIGTERM|FORK_LOG, &ctx->pid);
/* Optionally connect the helper's stdin to the specified fd, otherwise it inherits ours */
r = pidref_safe_fork_full(
ctx->name,
stdin_fd >= 0 ? (int[]) { stdin_fd, STDOUT_FILENO, STDERR_FILENO } : NULL,
/* except_fds= */ NULL, /* n_except_fds= */ 0,
FORK_RESET_SIGNALS|FORK_DEATHSIG_SIGTERM|FORK_LOG|(stdin_fd >= 0 ? FORK_REARRANGE_STDIO : 0),
&ctx->pid);
if (r < 0)
return log_error_errno(r, "Failed to fork process %s: %m", ctx->name);
if (r == 0) {
@@ -1633,8 +1669,247 @@ static bool transfer_zero_fill(const Transfer *t) {
return partition_designator_is_verity_sig(t->target.partition_type.designator);
}
int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, TransferProgress cb, void *userdata) {
static int transfer_acquire_source(
Transfer *t,
Instance *i,
const Uapi16Contents *s,
const char *compression,
TransferProgress cb,
void *userdata) {
_cleanup_strv_free_ char **cmdline = NULL;
_cleanup_free_ char *digest = NULL;
_cleanup_close_ int literal_fd = -EBADF;
const char *from, *to;
int r;
assert(t);
assert(i);
assert(s);
assert(compression);
assert(RESOURCE_IS_URL(i->resource->type));
/* Acquires a URL backed resource instance from one of its sources: either downloads it from the URL,
* or takes the literal data the manifest carries. We run everything through systemd-pull or
* systemd-import, so that decompression, sparse file handling and sandboxing are the same regardless
* of where the data comes from. The compression algorithm is passed down explicitly, so that the
* tools strictly decode with what the manifest declares, instead of guessing from the data. */
bool tar = i->resource->type == RESOURCE_URL_TAR;
/* The file may be only a part of the decoded source, i.e. the range [offset, offset+size) of it. If
* the size of the source is not known it is assumed to consist of the file only. */
bool range = s->offset != 0 ||
(s->original_size != UINT64_MAX && i->metadata.size != UINT64_MAX && s->original_size != i->metadata.size);
if (s->url) {
from = s->url;
r = strv_extend_many(
&cmdline,
SYSTEMD_PULL_PATH,
tar ? "tar" : "raw",
"--direct"); /* just download the specified URL, don't download anything else */
if (r < 0)
return log_oom();
/* If the file is cut out of a source that is not encoded, verifying the file's own digest
* (see below) is sufficient, and allows systemd-pull to acquire only the range of the source
* that makes up the file, via an HTTP range request, instead of the whole source. */
if (iovec_is_set(&s->encoded_sha256) && !(range && !s->encoding && i->metadata.sha256sum_set)) {
digest = hexmem(s->encoded_sha256.iov_base, s->encoded_sha256.iov_len);
if (!digest)
return log_oom();
r = strv_extend_many(&cmdline, "--verify", digest); /* validate by explicit SHA256 sum */
} else {
/* Without the digest of the download as a whole, we require the file's digest to be
* known, so that we can validate the decoded file instead, see below. */
if (!i->metadata.sha256sum_set)
return log_error_errno(SYNTHETIC_ERRNO(EINVAL), "SHA256 checksum not known for download '%s', refusing.", s->url);
r = strv_extend_many(&cmdline, "--verify", "no");
}
if (r < 0)
return log_oom();
/* systemd-pull supports ranges for raw images only. Other sources have been dropped when
* the manifest was processed already. */
assert(!range || !tar);
if (s->offset != 0) {
r = strv_extendf(&cmdline, "--source-offset=%" PRIu64, s->offset);
if (r < 0)
return log_oom();
}
if (range && i->metadata.size != UINT64_MAX) {
r = strv_extendf(&cmdline, "--source-size=%" PRIu64, i->metadata.size);
if (r < 0)
return log_oom();
}
} else {
struct iovec data = s->literal;
/* All other sources have been turned into URLs when the manifest was processed. Note that
* the inline data may be empty, for empty files. */
assert(uapi16_contents_has_literal(s));
/* Literal data: check it against its digest right here, cut out the part that makes up the
* file, and pass it to systemd-import via a memory file descriptor as standard input. */
if (iovec_is_set(&s->encoded_sha256) &&
memcmp_nn(SHA256_DIRECT(data.iov_base, data.iov_len), SHA256_DIGEST_SIZE,
s->encoded_sha256.iov_base, s->encoded_sha256.iov_len) != 0)
return log_error_errno(SYNTHETIC_ERRNO(EBADMSG), "Inline data of '%s' does not match its SHA256 digest, refusing.", i->path);
/* Encoded data we cannot cut into, and leave decoding to the importer. Encoded sources that
* carry more than the file have been dropped when the manifest was processed already. */
assert(!range || !s->encoding);
if (!s->encoding) {
if (s->offset > data.iov_len)
return log_error_errno(SYNTHETIC_ERRNO(EBADMSG), "Inline data of '%s' is shorter than the offset of the file in it, refusing.", i->path);
data = IOVEC_MAKE((uint8_t*) data.iov_base + s->offset, data.iov_len - s->offset);
if (i->metadata.size != UINT64_MAX) {
if (data.iov_len < i->metadata.size)
return log_error_errno(SYNTHETIC_ERRNO(EBADMSG), "Inline data of '%s' is shorter than the file, refusing.", i->path);
data.iov_len = i->metadata.size;
}
if (i->metadata.sha256sum_set &&
memcmp_nn(SHA256_DIRECT(data.iov_base, data.iov_len), SHA256_DIGEST_SIZE,
i->metadata.sha256sum, sizeof(i->metadata.sha256sum)) != 0)
return log_error_errno(SYNTHETIC_ERRNO(EBADMSG), "Inline data of '%s' does not match the file's SHA256 digest, refusing.", i->path);
}
/* Place the inline data in a sealed memory file, which we hand to systemd-import as stdin */
literal_fd = data_fd_from_iovec("sysupdate-literal", &data);
if (literal_fd < 0)
return log_error_errno(literal_fd, "Failed to allocate memory file for inline data: %m");
from = "-"; /* i.e. read from stdin */
r = strv_extend_many(
&cmdline,
SYSTEMD_IMPORT_PATH,
tar ? "tar" : "raw",
"--direct"); /* just copy/unpack the specified file, don't do anything else */
}
if (r < 0)
return log_oom();
r = strv_extendf(&cmdline, "--compression=%s", compression);
if (r < 0)
return log_oom();
if (i->metadata.sha256sum_set && (s->encoding || (s->url && (range || !iovec_is_set(&s->encoded_sha256))))) {
/* The digests checked so far cover the encoded data or the whole source only (if any), hence
* let the tool also check the decoded data cut down to the file against the file's digest */
_cleanup_free_ char *decoded_digest = hexmem(i->metadata.sha256sum, sizeof(i->metadata.sha256sum));
if (!decoded_digest)
return log_oom();
r = strv_extendf(&cmdline, "--verify-decoded=%s", decoded_digest);
if (r < 0)
return log_oom();
}
r = strv_extend(&cmdline, t->context->sync ? "--sync=yes" : "--sync=no");
if (r < 0)
return log_oom();
if (tar) {
assert(IN_SET(t->target.type, RESOURCE_DIRECTORY, RESOURCE_SUBVOLUME));
r = strv_extend(&cmdline, t->target.type == RESOURCE_SUBVOLUME ? "--btrfs-subvol=yes" : "--btrfs-subvol=no");
if (r < 0)
return log_oom();
to = t->temporary_partial_path;
} else if (t->target.type == RESOURCE_PARTITION) {
r = strv_extendf(&cmdline, "--offset=%" PRIu64, t->partition_info.start);
if (r < 0)
return log_oom();
r = strv_extendf(&cmdline, "--size-max=%" PRIu64, t->partition_info.size);
if (r < 0)
return log_oom();
r = strv_extend(&cmdline, transfer_zero_fill(t) ? "--zero-fill=yes" : "--zero-fill=no");
if (r < 0)
return log_oom();
to = t->target.path;
} else {
assert(t->target.type == RESOURCE_REGULAR_FILE);
to = t->temporary_partial_path;
}
r = strv_extend_many(&cmdline, from, to);
if (r < 0)
return log_oom();
return run_callout(
s->url ? (tar ? "(sd-pull-tar)" : "(sd-pull-raw)") : (tar ? "(sd-import-tar)" : "(sd-import-raw)"),
cmdline,
literal_fd,
t, i, cb, userdata);
}
static int instance_apply_user_xattrs(Instance *i, const char *path) {
int r;
assert(i);
assert(path);
/* Applies the user extended attributes listed in the manifest to the installed inode, so that they
* are retained on the target. This in particular matters for the ones MatchXAttr= looks at. Failing
* to do so is not fatal, the instance is fully usable without them. Returns > 0 if any attribute
* was applied. */
if (i->n_user_xattrs == 0)
return 0;
_cleanup_close_ int fd = xopenat(AT_FDCWD, path, O_PATH|O_NOFOLLOW);
if (fd < 0) {
log_warning_errno(fd, "Failed to open '%s' to apply extended attributes, ignoring: %m", path);
return 0;
}
bool applied = false;
FOREACH_ARRAY(x, i->user_xattrs, i->n_user_xattrs) {
_cleanup_free_ char *n = strjoin("user.", x->name);
if (!n)
return log_oom();
r = xsetxattr_full(
fd,
/* path= */ NULL,
AT_EMPTY_PATH,
n,
x->value.iov_base,
x->value.iov_len,
/* xattr_flags= */ 0);
if (ERRNO_IS_NEG_NOT_SUPPORTED(r)) {
log_notice_errno(r, "File system backing '%s' does not support extended attributes, not applying them: %m", path);
break;
}
if (r < 0) {
log_warning_errno(r, "Failed to set extended attribute '%s' on '%s', ignoring: %m", n, path);
continue;
}
applied = true;
}
return applied;
}
int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, TransferProgress cb, void *userdata) {
char offset[DECIMAL_STR_MAX(uint64_t)+1], max_size[DECIMAL_STR_MAX(uint64_t)+1];
const char *where = NULL;
Instance *existing;
@@ -1709,18 +1984,6 @@ int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, Tra
log_info("%s Acquiring %s %s %s...", glyph(GLYPH_DOWNLOAD), i->path, glyph(GLYPH_ARROW_RIGHT), where);
if (RESOURCE_IS_URL(i->resource->type)) {
/* For URL sources we require the SHA256 sum to be known so that we can validate the
* download. */
if (!i->metadata.sha256sum_set)
return log_error_errno(SYNTHETIC_ERRNO(EINVAL), "SHA256 checksum not known for download '%s', refusing.", i->path);
digest = hexmem(i->metadata.sha256sum, sizeof(i->metadata.sha256sum));
if (!digest)
return log_oom();
}
switch (i->resource->type) { /* Source */
case RESOURCE_REGULAR_FILE:
@@ -1742,7 +2005,7 @@ int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, Tra
t->context->sync ? "--sync=yes" : "--sync=no",
i->path,
t->temporary_partial_path),
t, i, cb, userdata);
/* stdin_fd= */ -EBADF, t, i, cb, userdata);
break;
case RESOURCE_PARTITION:
@@ -1760,7 +2023,7 @@ int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, Tra
t->context->sync ? "--sync=yes" : "--sync=no",
i->path,
t->target.path),
t, i, cb, userdata);
/* stdin_fd= */ -EBADF, t, i, cb, userdata);
break;
default:
@@ -1784,7 +2047,7 @@ int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, Tra
t->target.type == RESOURCE_SUBVOLUME ? "--btrfs-subvol=yes" : "--btrfs-subvol=no",
i->path,
t->temporary_partial_path),
t, i, cb, userdata);
/* stdin_fd= */ -EBADF, t, i, cb, userdata);
break;
case RESOURCE_TAR:
@@ -1801,68 +2064,54 @@ int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, Tra
t->target.type == RESOURCE_SUBVOLUME ? "--btrfs-subvol=yes" : "--btrfs-subvol=no",
i->path,
t->temporary_partial_path),
t, i, cb, userdata);
/* stdin_fd= */ -EBADF, t, i, cb, userdata);
break;
case RESOURCE_URL_FILE:
switch (t->target.type) {
case RESOURCE_REGULAR_FILE:
/* url file → regular file */
r = run_callout("(sd-pull-raw)",
STRV_MAKE(
SYSTEMD_PULL_PATH,
"raw",
"--direct", /* just download the specified URL, don't download anything else */
"--verify", digest, /* validate by explicit SHA256 sum */
t->context->sync ? "--sync=yes" : "--sync=no",
i->path,
t->temporary_partial_path),
t, i, cb, userdata);
break;
case RESOURCE_PARTITION:
/* url file → partition */
r = run_callout("(sd-pull-raw)",
STRV_MAKE(
SYSTEMD_PULL_PATH,
"raw",
"--direct", /* just download the specified URL, don't download anything else */
"--verify", digest, /* validate by explicit SHA256 sum */
"--offset", offset,
"--size-max", max_size,
transfer_zero_fill(t) ? "--zero-fill=yes" : "--zero-fill=no",
t->context->sync ? "--sync=yes" : "--sync=no",
i->path,
t->target.path),
t, i, cb, userdata);
break;
default:
assert_not_reached();
}
break;
case RESOURCE_URL_TAR:
assert(IN_SET(t->target.type, RESOURCE_DIRECTORY, RESOURCE_SUBVOLUME));
r = run_callout("(sd-pull-tar)",
STRV_MAKE(
SYSTEMD_PULL_PATH,
"tar",
"--direct", /* just download the specified URL, don't download anything else */
"--verify", digest, /* validate by explicit SHA256 sum */
t->target.type == RESOURCE_SUBVOLUME ? "--btrfs-subvol=yes" : "--btrfs-subvol=no",
t->context->sync ? "--sync=yes" : "--sync=no",
i->path,
t->temporary_partial_path),
t, i, cb, userdata);
if (i->n_sources == 0) {
/* The manifest lists no sources for the file, hence acquire it by its name relative
* to the source URL. */
Uapi16Contents fallback = UAPI16_CONTENTS_NULL;
fallback.url = i->path;
fallback.offset = 0;
if (i->metadata.sha256sum_set)
fallback.encoded_sha256 = IOVEC_MAKE(i->metadata.sha256sum, sizeof(i->metadata.sha256sum));
/* For SHA256SUMS manifests we know nothing about the compression, hence let the tool
* figure it out, and decompress implicitly. UAPI.16 manifests declare encodings
* explicitly, via "contents", hence a file without it is to be taken as is: its
* digest is the one of the file as stored. */
r = transfer_acquire_source(
t, i, &fallback,
i->resource->manifest_format == MANIFEST_FORMAT_UAPI16 ? "none" : "auto",
cb, userdata);
} else
/* The manifest lists one or more alternative sources for the file, try them in order
* until one works out. The manifest declares the encoding of each source, hence
* insist on it (or on none) and don't let the tools guess. */
FOREACH_ARRAY(s, i->sources, i->n_sources) {
r = transfer_acquire_source(t, i, s, s->encoding ?: "none", cb, userdata);
if (r >= 0 || r == -ECANCELED)
break;
if (s + 1 >= i->sources + i->n_sources)
break;
log_warning_errno(r, "Failed to acquire '%s', trying alternative source: %m", i->path);
/* In --direct mode the tools leave whatever they wrote in place when they
* fail, hence remove the partial, unverified leftovers before trying the next
* source, so that they are not combined with what that one provides. */
if (RESOURCE_IS_FILESYSTEM(t->target.type)) {
int k = rm_rf(t->temporary_partial_path, REMOVE_ROOT|REMOVE_PHYSICAL|REMOVE_SUBVOLUME|REMOVE_MISSING_OK|REMOVE_CHMOD);
if (k < 0)
return log_error_errno(k, "Failed to remove partially acquired '%s': %m", t->temporary_partial_path);
}
}
break;
default:
@@ -1899,6 +2148,12 @@ int transfer_acquire_instance(Transfer *t, Instance *i, InstanceMetadata *f, Tra
need_sync = true;
}
r = instance_apply_user_xattrs(i, t->temporary_partial_path);
if (r < 0)
return r;
if (r > 0)
need_sync = true;
/* Synchronize */
if (t->context->sync && need_sync) {
if (t->target.type == RESOURCE_REGULAR_FILE)
src/sysupdate/sysupdate.c
+81 -4
View File
@@ -1,5 +1,6 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
#include <fcntl.h>
#include <unistd.h>
#include "sd-bus.h"
@@ -19,6 +20,7 @@
#include "errno-util.h"
#include "fd-util.h"
#include "format-table.h"
#include "fs-util.h"
#include "glyph-util.h"
#include "hashmap.h"
#include "hexdecoct.h"
@@ -50,9 +52,11 @@
#include "sysupdate-transfer.h"
#include "sysupdate-update-set.h"
#include "sysupdate-util.h"
#include "uapi16-manifest.h"
#include "varlink-io.systemd.SysUpdate.h"
#include "varlink-util.h"
#include "verbs.h"
#include "xattr-match.h"
static char *arg_definitions = NULL;
static bool arg_sync = true;
@@ -128,6 +132,12 @@ void context_done(Context *c) {
c->web_cache = hashmap_free(c->web_cache);
host_properties_done(&c->host_properties);
uapi16_certificates_free(c->manifest_certificates, c->n_manifest_certificates);
c->manifest_certificates = NULL;
c->n_manifest_certificates = 0;
c->installdb_fd = safe_close(c->installdb_fd);
c->definitions = mfree(c->definitions);
@@ -572,7 +582,15 @@ static int context_load_installed_instances(Context *c) {
r = resource_load_instances(
&t->target,
c->verify >= 0 ? c->verify : t->verify != 0,
&c->web_cache);
/* certificates= */ NULL,
/* n_certificates= */ 0,
&c->web_cache,
&(const XAttrMatchFilter) {
.mask = resource_xattr_match_mask(&t->target),
.host = &c->host_properties,
.root = c->root,
},
/* match_partition_type= */ NULL);
if (r < 0)
return r;
}
@@ -583,7 +601,15 @@ static int context_load_installed_instances(Context *c) {
r = resource_load_instances(
&t->target,
c->verify >= 0 ? c->verify : t->verify != 0,
&c->web_cache);
/* certificates= */ NULL,
/* n_certificates= */ 0,
&c->web_cache,
&(const XAttrMatchFilter) {
.mask = resource_xattr_match_mask(&t->target),
.host = &c->host_properties,
.root = c->root,
},
/* match_partition_type= */ NULL);
if (r < 0)
return r;
}
@@ -591,6 +617,35 @@ static int context_load_installed_instances(Context *c) {
return 0;
}
static int context_load_manifest_certificates(Context *c) {
int r;
assert(c);
/* Loads the certificates UAPI.16 manifests may be signed with, from the VOA hierarchy */
if (c->manifest_certificates_loaded)
return 0;
_cleanup_close_ int root_fd = -EBADF;
if (c->root) {
root_fd = xopenat(AT_FDCWD, c->root, O_DIRECTORY|O_PATH);
if (root_fd < 0)
return log_error_errno(root_fd, "Failed to open root directory '%s': %m", c->root);
}
r = uapi16_load_voa_certificates(
root_fd >= 0 ? root_fd : XAT_FDROOT,
SYSUPDATE_VOA_CONTEXT,
&c->manifest_certificates,
&c->n_manifest_certificates);
if (r < 0)
return r;
c->manifest_certificates_loaded = true;
return 0;
}
static int context_load_available_instances(Context *c) {
int r;
@@ -600,11 +655,33 @@ static int context_load_available_instances(Context *c) {
FOREACH_ARRAY(tr, c->transfers, c->n_transfers) {
Transfer *t = *tr;
bool verify = c->verify >= 0 ? c->verify : t->verify != 0;
/* UAPI.16 manifests carry their signatures inline, which we verify ourselves, against the
* certificates from the VOA hierarchy */
if (verify && RESOURCE_IS_URL(t->source.type) && t->source.manifest_format == MANIFEST_FORMAT_UAPI16) {
r = context_load_manifest_certificates(c);
if (r < 0)
return r;
if (c->n_manifest_certificates == 0)
return log_error_errno(SYNTHETIC_ERRNO(ENOKEY),
"Transfer '%s' requires verification of the UAPI.16 manifest signature, but no certificates found in the VOA hierarchy (role '%s', context '%s'), refusing.",
t->id, UAPI16_VOA_ROLE, SYSUPDATE_VOA_CONTEXT);
}
r = resource_load_instances(
&t->source,
c->verify >= 0 ? c->verify : t->verify != 0,
&c->web_cache);
verify,
c->manifest_certificates,
c->n_manifest_certificates,
&c->web_cache,
&(const XAttrMatchFilter) {
.mask = resource_xattr_match_mask(&t->source),
.host = &c->host_properties,
.root = c->root,
},
t->target.partition_type_set ? &t->target.partition_type : NULL);
if (r < 0)
return r;
}
src/sysupdate/sysupdate.h
+10
View File
@@ -1,10 +1,14 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
#pragma once
#include "host-properties.h"
#include "sysupdate-component.h"
#include "sysupdate-forward.h"
#include "sysupdate-target.h"
/* The VOA context (see voa-util.h) UAPI.16 manifest certificates are looked up in */
#define SYSUPDATE_VOA_CONTEXT "sysupdate"
typedef enum SelectMode {
SELECT_EXPLICIT, /* Only explicitly specified features/components */
SELECT_ALL, /* All available features/components */
@@ -55,6 +59,12 @@ typedef struct Context {
Hashmap *web_cache; /* Cache for downloaded resources, keyed by URL */
HostProperties host_properties; /* Loaded lazily, for MatchXAttr= */
X509 **manifest_certificates; /* Loaded lazily from the VOA hierarchy, for verifying UAPI.16 manifest signatures */
size_t n_manifest_certificates;
bool manifest_certificates_loaded;
int installdb_fd;
TargetIdentifier target_identifier;
src/test/test-data-fd-util.c
+50
View File
@@ -11,6 +11,7 @@
#include "fileio.h"
#include "fs-util.h"
#include "io-util.h"
#include "iovec-util.h"
#include "memfd-util.h"
#include "pidref.h"
#include "process-util.h"
@@ -59,6 +60,10 @@ TEST(copy_data_fd) {
fd2 = ASSERT_OK(copy_data_fd(fd1));
/* Small data ends up in a sealed memfd, which we return read-only, like the temporary files */
ASSERT_OK_POSITIVE(memfd_get_sealed(fd2));
ASSERT_EQ(ASSERT_OK_ERRNO(fcntl(fd2, F_GETFL)) & O_ACCMODE_STRICT, O_RDONLY);
safe_close(fd1);
fd1 = ASSERT_OK(memfd_new_and_seal_string("data", "hallo"));
@@ -216,4 +221,49 @@ TEST(open_seekable_data_fd) {
ASSERT_ERROR(open_seekable_data_fd(dir_fd, "missing", UINT64_MAX), ENOENT);
}
TEST(data_fd_from_iovec) {
_cleanup_close_ int fd = -EBADF;
_cleanup_free_ uint8_t *large = NULL;
struct stat st;
char buf[16];
fd = ASSERT_OK(data_fd_from_iovec("data", &CONST_IOVEC_MAKE_STRING("hallo")));
assert_fd_contents(fd, "hallo");
/* The fd is sealed, read-only and positioned at the beginning, so that it can be passed as stdin */
ASSERT_OK_POSITIVE(memfd_get_sealed(fd));
ASSERT_EQ(ASSERT_OK_ERRNO(fcntl(fd, F_GETFL)) & O_ACCMODE_STRICT, O_RDONLY);
ASSERT_OK_ZERO_ERRNO(lseek(fd, 0, SEEK_CUR));
ASSERT_ERROR_ERRNO(write(fd, "x", 1), EBADF);
ASSERT_OK_EQ_ERRNO(read(fd, buf, sizeof(buf)), 5);
ASSERT_EQ(memcmp(buf, "hallo", 5), 0);
ASSERT_OK_ZERO_ERRNO(read(fd, buf, sizeof(buf)));
fd = safe_close(fd);
/* Empty data, with or without a buffer, and no name */
fd = ASSERT_OK(data_fd_from_iovec(/* name= */ NULL, &(const struct iovec) {}));
assert_fd_contents(fd, "");
ASSERT_OK_POSITIVE(memfd_get_sealed(fd));
fd = safe_close(fd);
fd = ASSERT_OK(data_fd_from_iovec("data", &IOVEC_MAKE(buf, 0)));
assert_fd_contents(fd, "");
fd = safe_close(fd);
/* Data larger than what copy_data_fd() places in a memfd still ends up in one */
size_t n = 1536 * 1024;
large = ASSERT_NOT_NULL(malloc(n));
for (size_t i = 0; i < n; i++)
large[i] = (uint8_t) (i * 7);
fd = ASSERT_OK(data_fd_from_iovec("large", &IOVEC_MAKE(large, n)));
ASSERT_OK_POSITIVE(memfd_get_sealed(fd));
ASSERT_OK_ERRNO(fstat(fd, &st));
ASSERT_EQ((uint64_t) st.st_size, (uint64_t) n);
_cleanup_free_ uint8_t *copy = ASSERT_NOT_NULL(malloc(n));
ASSERT_OK_EQ_ERRNO(pread(fd, copy, n, 0), (ssize_t) n);
ASSERT_EQ(memcmp(large, copy, n), 0);
}
DEFINE_TEST_MAIN(LOG_DEBUG);
test/units/TEST-72-SYSUPDATE.sh
+323 -3
View File
@@ -1022,9 +1022,9 @@ cmp "$MP/payload/mp-v1.bin" "$MP/target/mp-v1.bin"
# there is none
sed 's|^ManifestPath=.*|&\nManifestPath=|' "$MP/definitions/01-mp.transfer" >"$MP/bad-definitions/01-mp.transfer"
(! "$SYSUPDATE" --definitions="$MP/bad-definitions" --verify=no list)
# Only SHA256SUMS manifests are supported for now
sed 's|^ManifestFormat=.*|ManifestFormat=uapi16|' "$MP/definitions/01-mp.transfer" >"$MP/bad-definitions/01-mp.transfer"
(! "$SYSUPDATE" --definitions="$MP/bad-definitions" --verify=no list) |& grep -F "Manifest format 'uapi16' not supported" >/dev/null
# Only SHA256SUMS and UAPI.16 manifests are supported
sed 's|^ManifestFormat=.*|ManifestFormat=mtree|' "$MP/definitions/01-mp.transfer" >"$MP/bad-definitions/01-mp.transfer"
(! "$SYSUPDATE" --definitions="$MP/bad-definitions" --verify=no list) |& grep -F "Manifest format 'mtree' not supported" >/dev/null
# ManifestPath= must be a URL, and is only supported for URL sources
sed 's|^ManifestPath=.*|ManifestPath=/foo/SHA256SUMS|' "$MP/definitions/01-mp.transfer" >"$MP/bad-definitions/01-mp.transfer"
(! "$SYSUPDATE" --definitions="$MP/bad-definitions" --verify=no list) |& grep -F "Manifest path is not a valid" >/dev/null
@@ -1032,6 +1032,326 @@ sed -e 's|^Type=url-file|Type=regular-file|' -e "s|^Path=file://|Path=|" "$MP/de
(! "$SYSUPDATE" --definitions="$MP/bad-definitions" --verify=no list) |& grep -F "ManifestPath= is only supported for URL sources" >/dev/null
rm -rf "$MP"
# Check that the sources listed in the "contents" arrays of a UAPI.16 manifest are honoured: "file"
# entries are fetched relative to Path= (not relative to ManifestPath=), "url" entries from the specified
# URL, "literal" and "url" entries may carry the file embedded in more data (at "offset"), and alternative
# sources are tried in order. Entries without "file", "url" and "literal" refer to a file of the object's
# name relative to Path=, and an empty "literal" makes an empty file. Files without "contents" are
# acquired by name, as they are, i.e. without decompression.
rm -rf "$WORKDIR/uapi16"
mkdir -p "$WORKDIR/uapi16/payload/blobs" "$WORKDIR/uapi16/manifest/blobs" "$WORKDIR/uapi16/remote" "$WORKDIR/uapi16/target"
head -c 3000 /dev/urandom >"$WORKDIR/uapi16/alt-v1.bin"
cp "$WORKDIR/uapi16/alt-v1.bin" "$WORKDIR/uapi16/payload/blobs/alt-v1.bin"
# A decoy next to the manifest, which must not be used, as "file" entries are not relative to ManifestPath=
head -c 3000 /dev/urandom >"$WORKDIR/uapi16/manifest/blobs/alt-v1.bin"
head -c 2000 /dev/urandom >"$WORKDIR/uapi16/implied-v1.bin"
cp "$WORKDIR/uapi16/implied-v1.bin" "$WORKDIR/uapi16/payload/implied-v1.bin"
: >"$WORKDIR/uapi16/empty-v1.bin"
head -c 5000 /dev/urandom >"$WORKDIR/uapi16/remote-v1.bin"
gzip -k -f "$WORKDIR/uapi16/remote-v1.bin"
mv "$WORKDIR/uapi16/remote-v1.bin.gz" "$WORKDIR/uapi16/remote/"
printf 'inline payload\n' >"$WORKDIR/uapi16/inline-v1.bin"
{ printf 'garbage'; cat "$WORKDIR/uapi16/inline-v1.bin"; printf 'trailing garbage'; } >"$WORKDIR/uapi16/inline-v1.literal"
head -c 4000 /dev/urandom | gzip >"$WORKDIR/uapi16/plain-v1.bin"
cp "$WORKDIR/uapi16/plain-v1.bin" "$WORKDIR/uapi16/payload/plain-v1.bin"
# A disk image like source, which carries the file at an offset, followed by more data
head -c 6000 /dev/urandom >"$WORKDIR/uapi16/slice-v1.bin"
cp "$WORKDIR/uapi16/slice-v1.bin" "$WORKDIR/uapi16/rawslice-v1.bin"
{ head -c 1000 /dev/urandom; cat "$WORKDIR/uapi16/slice-v1.bin"; head -c 2000 /dev/urandom; } >"$WORKDIR/uapi16/remote/disk.raw"
gzip -k -f "$WORKDIR/uapi16/remote/disk.raw"
sha256_of() { sha256sum "$1" | cut -d' ' -f1; }
{
printf '\036{"mediaType":"application/vnd.uapi.16.manifest"}\n'
# The first source does not exist, the second one is found relative to Path=
printf '\036{"name":"alt-v1.bin","size":%s,"sha256":"%s","contents":[{"file":"blobs/missing-v1.bin"},{"file":"blobs/alt-v1.bin"}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/alt-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/alt-v1.bin")"
# The source refers to the file of the object's name relative to Path= implicitly
printf '\036{"name":"implied-v1.bin","size":%s,"sha256":"%s","contents":[{}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/implied-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/implied-v1.bin")"
# An empty file carried inline
printf '\036{"name":"empty-v1.bin","size":0,"sha256":"%s","contents":[{"literal":""}]}\n' \
"$(sha256_of "$WORKDIR/uapi16/empty-v1.bin")"
printf '\036{"name":"remote-v1.bin","size":%s,"sha256":"%s","contents":[{"url":"file://%s/uapi16/remote/remote-v1.bin.gz","encoding":"gzip","encodedSize":%s,"encodedSha256":"%s","originalSize":%s,"originalSha256":"%s"}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/remote-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/remote-v1.bin")" "$WORKDIR" \
"$(stat -c %s "$WORKDIR/uapi16/remote/remote-v1.bin.gz")" "$(sha256_of "$WORKDIR/uapi16/remote/remote-v1.bin.gz")" \
"$(stat -c %s "$WORKDIR/uapi16/remote-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/remote-v1.bin")"
# The file sits at an offset inside the inline data
printf '\036{"name":"inline-v1.bin","size":%s,"sha256":"%s","contents":[{"literal":"%s","offset":7,"originalSize":%s,"originalSha256":"%s"}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/inline-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/inline-v1.bin")" "$(base64 -w0 "$WORKDIR/uapi16/inline-v1.literal")" \
"$(stat -c %s "$WORKDIR/uapi16/inline-v1.literal")" "$(sha256_of "$WORKDIR/uapi16/inline-v1.literal")"
# No sources listed: acquired by name relative to Path=, and must not be decompressed even though it is
# gzip data, as the digest covers the file as it is
printf '\036{"name":"plain-v1.bin","size":%s,"sha256":"%s"}\n' \
"$(stat -c %s "$WORKDIR/uapi16/plain-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/plain-v1.bin")"
# The file is a range of the decoded data of a source that is verified as a whole
printf '\036{"name":"slice-v1.bin","size":%s,"sha256":"%s","contents":[{"url":"file://%s/uapi16/remote/disk.raw.gz","encoding":"gzip","encodedSha256":"%s","originalSize":%s,"offset":1000}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/slice-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/slice-v1.bin")" "$WORKDIR" \
"$(sha256_of "$WORKDIR/uapi16/remote/disk.raw.gz")" "$(stat -c %s "$WORKDIR/uapi16/remote/disk.raw")"
# The same, but without a digest of the source, hence only verified against the file's digest
printf '\036{"name":"rawslice-v1.bin","size":%s,"sha256":"%s","contents":[{"url":"file://%s/uapi16/remote/disk.raw","originalSize":%s,"offset":1000}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/slice-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/slice-v1.bin")" "$WORKDIR" \
"$(stat -c %s "$WORKDIR/uapi16/remote/disk.raw")"
# The source matches its digest, but the range at the offset is not the file
printf '\036{"name":"badslice-v1.bin","size":%s,"sha256":"%s","contents":[{"url":"file://%s/uapi16/remote/disk.raw.gz","encoding":"gzip","encodedSha256":"%s","originalSize":%s,"offset":999}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/slice-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/slice-v1.bin")" "$WORKDIR" \
"$(sha256_of "$WORKDIR/uapi16/remote/disk.raw.gz")" "$(stat -c %s "$WORKDIR/uapi16/remote/disk.raw")"
# The encoded data matches its digest, but does not decode to the file
printf '\036{"name":"bad-v1.bin","size":%s,"sha256":"%s","contents":[{"url":"file://%s/uapi16/remote/remote-v1.bin.gz","encoding":"gzip","encodedSha256":"%s"}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/remote-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/alt-v1.bin")" "$WORKDIR" \
"$(sha256_of "$WORKDIR/uapi16/remote/remote-v1.bin.gz")"
printf '\036{"mediaType":"application/vnd.uapi.16.trailer"}\n'
} >"$WORKDIR/uapi16/manifest/Uapi16Manifest"
for f in alt implied empty remote inline plain slice rawslice; do
cat >"$CONFIGDIR/01-uapi16-$f.transfer" <<EOF
[Source]
Type=url-file
Path=file://$WORKDIR/uapi16/payload
ManifestFormat=uapi16
ManifestPath=file://$WORKDIR/uapi16/manifest/Uapi16Manifest
MatchPattern=$f-@v.bin
[Target]
Type=regular-file
Path=$WORKDIR/uapi16/target
MatchPattern=$f-@v.bin
InstancesMax=2
EOF
done
"$SYSUPDATE" --verify=no update
for f in alt implied empty remote inline plain slice rawslice; do
cmp "$WORKDIR/uapi16/$f-v1.bin" "$WORKDIR/uapi16/target/$f-v1.bin"
done
# The decoded data is verified against the file's digest, not just the encoded data against its own
mkdir -p "$WORKDIR/uapi16/bad-definitions"
sed 's/^MatchPattern=alt-/MatchPattern=bad-/' "$CONFIGDIR/01-uapi16-alt.transfer" >"$WORKDIR/uapi16/bad-definitions/bad.transfer"
(! "$SYSUPDATE" --definitions="$WORKDIR/uapi16/bad-definitions" --verify=no update)
[[ ! -e "$WORKDIR/uapi16/target/bad-v1.bin" ]]
sed -i 's/^MatchPattern=bad-/MatchPattern=badslice-/' "$WORKDIR/uapi16/bad-definitions/bad.transfer"
(! "$SYSUPDATE" --definitions="$WORKDIR/uapi16/bad-definitions" --verify=no update)
[[ ! -e "$WORKDIR/uapi16/target/badslice-v1.bin" ]]
# systemd-pull cannot acquire a tarball that is embedded in a larger source, and we cannot cut into encoded
# inline data, hence such sources are ignored, and the next source listed is used instead
mkdir -p "$WORKDIR/uapi16/ranged/definitions" "$WORKDIR/uapi16/ranged/tree" "$WORKDIR/uapi16/ranged/dirs" "$WORKDIR/uapi16/ranged/target"
printf 'tarball payload\n' >"$WORKDIR/uapi16/ranged/tree/file.txt"
tar -C "$WORKDIR/uapi16/ranged/tree" -cf "$WORKDIR/uapi16/ranged/tree-v1.tar" .
{ head -c 512 /dev/zero; cat "$WORKDIR/uapi16/ranged/tree-v1.tar"; } >"$WORKDIR/uapi16/ranged/embedded.raw"
printf 'encoded inline payload\n' >"$WORKDIR/uapi16/ranged/enc-v1.bin"
{ printf 'garbage'; cat "$WORKDIR/uapi16/ranged/enc-v1.bin"; } | gzip >"$WORKDIR/uapi16/ranged/enc-v1.literal.gz"
{
printf '\036{"mediaType":"application/vnd.uapi.16.manifest"}\n'
printf '\036{"name":"tree-v1.tar","size":%s,"sha256":"%s","contents":[{"url":"file://%s/uapi16/ranged/embedded.raw","offset":512},{"url":"file://%s/uapi16/ranged/tree-v1.tar"}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/ranged/tree-v1.tar")" "$(sha256_of "$WORKDIR/uapi16/ranged/tree-v1.tar")" "$WORKDIR" "$WORKDIR"
printf '\036{"name":"enc-v1.bin","size":%s,"sha256":"%s","contents":[{"literal":"%s","encoding":"gzip","offset":7},{"literal":"%s"}]}\n' \
"$(stat -c %s "$WORKDIR/uapi16/ranged/enc-v1.bin")" "$(sha256_of "$WORKDIR/uapi16/ranged/enc-v1.bin")" \
"$(base64 -w0 "$WORKDIR/uapi16/ranged/enc-v1.literal.gz")" "$(base64 -w0 "$WORKDIR/uapi16/ranged/enc-v1.bin")"
printf '\036{"mediaType":"application/vnd.uapi.16.trailer"}\n'
} >"$WORKDIR/uapi16/ranged/Uapi16Manifest"
cat >"$WORKDIR/uapi16/ranged/definitions/01-tree.transfer" <<EOF
[Source]
Type=url-tar
Path=file://$WORKDIR/uapi16/ranged
ManifestFormat=uapi16
ManifestPath=file://$WORKDIR/uapi16/ranged/Uapi16Manifest
MatchPattern=tree-@v.tar
[Target]
Type=directory
Path=$WORKDIR/uapi16/ranged/dirs
MatchPattern=tree-@v
EOF
cat >"$WORKDIR/uapi16/ranged/definitions/02-enc.transfer" <<EOF
[Source]
Type=url-file
Path=file://$WORKDIR/uapi16/ranged
ManifestFormat=uapi16
ManifestPath=file://$WORKDIR/uapi16/ranged/Uapi16Manifest
MatchPattern=enc-@v.bin
[Target]
Type=regular-file
Path=$WORKDIR/uapi16/ranged/target
MatchPattern=enc-@v.bin
EOF
"$SYSUPDATE" --definitions="$WORKDIR/uapi16/ranged/definitions" --verify=no update &>"$WORKDIR/uapi16/ranged/update.log"
grep -F "File 'tree-v1.tar' has a source that carries more than the file, which is not supported for tarballs, ignoring source." "$WORKDIR/uapi16/ranged/update.log" >/dev/null
grep -F "File 'enc-v1.bin' has encoded inline data that carries more than the file, ignoring source." "$WORKDIR/uapi16/ranged/update.log" >/dev/null
cmp "$WORKDIR/uapi16/ranged/tree/file.txt" "$WORKDIR/uapi16/ranged/dirs/tree-v1/file.txt"
cmp "$WORKDIR/uapi16/ranged/enc-v1.bin" "$WORKDIR/uapi16/ranged/target/enc-v1.bin"
# UAPI.16 manifests carry their signature inline, which is verified against the certificates in the VOA
# hierarchy (role "manifest", context "sysupdate") if verification is enabled
if command -v openssl >/dev/null; then
VOA_OS="$(. /etc/os-release; echo "${ID:-linux}")"
VOA_DIR="/run/voa/$VOA_OS/manifest/sysupdate/x509"
S="$WORKDIR/uapi16/signed"
mkdir -p "$S/payload" "$S/target" "$S/definitions"
head -c 2000 /dev/urandom >"$S/payload/signed-v1.bin"
openssl req -x509 -newkey rsa:2048 -nodes -keyout "$S/good.key" -out "$S/good.crt" -subj /CN=good -days 2 2>/dev/null
openssl req -x509 -newkey rsa:2048 -nodes -keyout "$S/other.key" -out "$S/other.crt" -subj /CN=other -days 2 2>/dev/null
systemd-manifest make --private-key="$S/good.key" --certificate="$S/good.crt" "$S/payload" >"$S/Uapi16Manifest"
systemd-manifest make "$S/payload" >"$S/Unsigned.Uapi16Manifest"
cat >"$S/definitions/signed.transfer" <<EOF
[Source]
Type=url-file
Path=file://$S/payload
ManifestFormat=uapi16
ManifestPath=file://$S/Uapi16Manifest
MatchPattern=signed-@v.bin
[Target]
Type=regular-file
Path=$S/target
MatchPattern=signed-@v.bin
EOF
sed "s|^ManifestPath=.*|ManifestPath=file://$S/Unsigned.Uapi16Manifest|" "$S/definitions/signed.transfer" >"$S/unsigned.transfer"
# No certificates installed: refused
(! "$SYSUPDATE" --definitions="$S/definitions" update)
# Only a different certificate installed: refused
mkdir -p "$VOA_DIR"
cp "$S/other.crt" "$VOA_DIR/test-72-other-certificate.pem"
(! "$SYSUPDATE" --definitions="$S/definitions" update)
[[ ! -e "$S/target/signed-v1.bin" ]]
# The right certificate in a different VOA context: refused
mkdir -p "/run/voa/$VOA_OS/manifest/default/x509"
cp "$S/good.crt" "/run/voa/$VOA_OS/manifest/default/x509/test-72-good-certificate.pem"
(! "$SYSUPDATE" --definitions="$S/definitions" update)
[[ ! -e "$S/target/signed-v1.bin" ]]
rm -r "/run/voa/$VOA_OS/manifest/default"
# The right certificate installed too, but masked: refused
cp "$S/good.crt" "$VOA_DIR/test-72-good-certificate.pem"
mkdir -p "/etc/voa/$VOA_OS/manifest/sysupdate/x509"
ln -s /dev/null "/etc/voa/$VOA_OS/manifest/sysupdate/x509/test-72-good-certificate.pem"
(! "$SYSUPDATE" --definitions="$S/definitions" update)
[[ ! -e "$S/target/signed-v1.bin" ]]
rm -r "/etc/voa/$VOA_OS/manifest"
# Not masked anymore: works
"$SYSUPDATE" --definitions="$S/definitions" update
cmp "$S/payload/signed-v1.bin" "$S/target/signed-v1.bin"
# systemd-manifest can check against the same certificates
systemd-manifest verify --voa-context=sysupdate --field=size,hash "$S/payload" "$S/Uapi16Manifest"
(! systemd-manifest verify --field=size,hash "$S/payload" "$S/Uapi16Manifest")
# An unsigned manifest is refused, unless verification is turned off
rm "$S/target/signed-v1.bin"
mv "$S/unsigned.transfer" "$S/definitions/signed.transfer"
(! "$SYSUPDATE" --definitions="$S/definitions" update)
[[ ! -e "$S/target/signed-v1.bin" ]]
"$SYSUPDATE" --definitions="$S/definitions" --verify=no update
cmp "$S/payload/signed-v1.bin" "$S/target/signed-v1.bin"
rm -r "/run/voa/$VOA_OS/manifest"
fi
# The download tools decode strictly with the declared algorithm if one is passed to them
PULL=/usr/lib/systemd/systemd-pull
IMPORT=/usr/lib/systemd/systemd-import
"$PULL" raw --direct --verify=no --compression=gzip "file://$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-gzip.bin"
cmp "$WORKDIR/uapi16/remote-v1.bin" "$WORKDIR/uapi16/pull-gzip.bin"
"$PULL" raw --direct --verify=no --compression=auto "file://$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-auto.bin"
cmp "$WORKDIR/uapi16/remote-v1.bin" "$WORKDIR/uapi16/pull-auto.bin"
"$PULL" raw --direct --verify=no --compression=none "file://$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-none.bin"
cmp "$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-none.bin"
(! "$PULL" raw --direct --verify=no --compression=xz "file://$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-xz.bin")
(! "$PULL" raw --direct --verify=no --compression=foobar "file://$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-foobar.bin")
"$IMPORT" raw --direct --compression=gzip - "$WORKDIR/uapi16/import-gzip.bin" <"$WORKDIR/uapi16/remote/remote-v1.bin.gz"
cmp "$WORKDIR/uapi16/remote-v1.bin" "$WORKDIR/uapi16/import-gzip.bin"
(! "$IMPORT" raw --direct --compression=xz - "$WORKDIR/uapi16/import-xz.bin" <"$WORKDIR/uapi16/remote/remote-v1.bin.gz")
# … and optionally verify the decoded data
"$PULL" raw --direct --verify=no --compression=gzip --verify-decoded="$(sha256_of "$WORKDIR/uapi16/remote-v1.bin")" \
"file://$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-decoded.bin"
cmp "$WORKDIR/uapi16/remote-v1.bin" "$WORKDIR/uapi16/pull-decoded.bin"
(! "$PULL" raw --direct --verify=no --compression=gzip --verify-decoded="$(sha256_of "$WORKDIR/uapi16/alt-v1.bin")" \
"file://$WORKDIR/uapi16/remote/remote-v1.bin.gz" "$WORKDIR/uapi16/pull-decoded-bad.bin")
"$IMPORT" raw --direct --compression=gzip --verify-decoded="$(sha256_of "$WORKDIR/uapi16/remote-v1.bin")" \
- "$WORKDIR/uapi16/import-decoded.bin" <"$WORKDIR/uapi16/remote/remote-v1.bin.gz"
cmp "$WORKDIR/uapi16/remote-v1.bin" "$WORKDIR/uapi16/import-decoded.bin"
(! "$IMPORT" raw --direct --compression=gzip --verify-decoded="$(sha256_of "$WORKDIR/uapi16/alt-v1.bin")" \
- "$WORKDIR/uapi16/import-decoded-bad.bin" <"$WORKDIR/uapi16/remote/remote-v1.bin.gz")
(! "$IMPORT" raw --direct --verify-decoded=foobar - "$WORKDIR/uapi16/import-decoded-invalid.bin" <"$WORKDIR/uapi16/remote-v1.bin")
rm -f "$CONFIGDIR"/01-uapi16-*.transfer
rm -rf "$WORKDIR/uapi16"
# Check MatchXAttr=: instances restricted to other systems via "user.uapi.match_…" extended attributes are
# ignored, both when listed in UAPI.16 manifests and when found on disk, and the extended attributes listed
# in the manifest are applied to the installed instance.
XM="$WORKDIR/xattr-match"
rm -rf "$XM"
mkdir -p "$XM/definitions" "$XM/definitions-all" "$XM/definitions-target" "$XM/payload/blobs" "$XM/manifest" "$XM/target"
for v in 1 2 3; do
echo "xattr match payload v$v" >"$XM/payload/blobs/xm-v$v.bin"
done
XM_MID="$(cat /etc/machine-id)"
XM_OSID="$(. /etc/os-release; echo "$ID")"
XM_OTHER=0123456789abcdef0123456789abcdef
xm_object() {
local v="${1:?}" xattrs=""
[[ -n "${2:-}" ]] && xattrs=",\"extendedAttributes\":{\"user\":{$2}}"
printf '\036{"name":"xm-v%s.bin","size":%s,"sha256":"%s","contents":[{"file":"blobs/xm-v%s.bin"}]%s}\n' \
"$v" "$(stat -c %s "$XM/payload/blobs/xm-v$v.bin")" "$(sha256_of "$XM/payload/blobs/xm-v$v.bin")" "$v" "$xattrs"
}
{
printf '\036{"mediaType":"application/vnd.uapi.16.manifest"}\n'
# Not restricted in any way
xm_object 1
# Restricted to two machines, one of which is us, and to our OS. Also carries an unrelated attribute.
xm_object 2 "$(printf '"uapi.match_machine_id":{"base64":"%s"},"uapi.match_os_id":{"text":"%s"},"foo":{"text":"bar"}' \
"$(printf '%s\0%s\0' "$XM_OTHER" "$XM_MID" | base64 -w0)" "$XM_OSID")"
# Restricted to some other machine
xm_object 3 "$(printf '"uapi.match_machine_id":{"text":"%s"}' "$XM_OTHER")"
printf '\036{"mediaType":"application/vnd.uapi.16.trailer"}\n'
} >"$XM/manifest/Uapi16Manifest"
for d in definitions definitions-all definitions-target; do
cat >"$XM/$d/xm.transfer" <<EOF
[Source]
Type=url-file
Path=file://$XM/payload
ManifestFormat=uapi16
ManifestPath=file://$XM/manifest/Uapi16Manifest
MatchPattern=xm-@v.bin
[Target]
Type=regular-file
Path=$XM/target
MatchPattern=xm-@v.bin
InstancesMax=1
EOF
done
# MatchXAttr= is enabled by default in the [Source] section, turn it off for "definitions-all"
cat >>"$XM/definitions-all/xm.transfer" <<EOF
[Source]
MatchXAttr=no
EOF
cat >>"$XM/definitions-target/xm.transfer" <<EOF
[Target]
MatchXAttr=yes
EOF
[[ $("$SYSUPDATE" --definitions="$XM/definitions-all" --verify=no --json=short list | jq -cr '.all') == '["v3","v2","v1"]' ]]
[[ $("$SYSUPDATE" --definitions="$XM/definitions" --verify=no --json=short list | jq -cr '.all') == '["v2","v1"]' ]]
"$SYSUPDATE" --definitions="$XM/definitions" --verify=no update
cmp "$XM/payload/blobs/xm-v2.bin" "$XM/target/xm-v2.bin"
[[ ! -e "$XM/target/xm-v3.bin" ]]
# The extended attributes from the manifest have been applied
[[ "$(getfattr --absolute-names --only-values -n user.foo "$XM/target/xm-v2.bin")" == bar ]]
[[ "$(getfattr --absolute-names --only-values -n user.uapi.match_os_id "$XM/target/xm-v2.bin")" == "$XM_OSID" ]]
cmp <(getfattr --absolute-names --only-values -n user.uapi.match_machine_id "$XM/target/xm-v2.bin") <(printf '%s\0%s\0' "$XM_OTHER" "$XM_MID")
# Installed instances are only filtered if MatchXAttr= is enabled in the [Target] section too (it is off by
# default there): an installed instance restricted to some other machine is then neither listed nor vacuumed
echo foreign >"$XM/target/xm-v4.bin"
setfattr -n user.uapi.match_machine_id -v "$XM_OTHER" "$XM/target/xm-v4.bin"
[[ $("$SYSUPDATE" --definitions="$XM/definitions-all" --verify=no --offline --json=short list | jq -cr '.all') == '["v4","v2"]' ]]
[[ $("$SYSUPDATE" --definitions="$XM/definitions" --verify=no --offline --json=short list | jq -cr '.all') == '["v4","v2"]' ]]
[[ $("$SYSUPDATE" --definitions="$XM/definitions-target" --verify=no --offline --json=short list | jq -cr '.all') == '["v2"]' ]]
"$SYSUPDATE" --definitions="$XM/definitions-target" --verify=no vacuum
[[ -e "$XM/target/xm-v2.bin" ]]
[[ -e "$XM/target/xm-v4.bin" ]]
rm -rf "$XM"
# Check that malformed manifest hashes are rejected without aborting.
rm -rf "$WORKDIR/malformed-manifest"
mkdir -p "$WORKDIR/malformed-manifest/definitions" "$WORKDIR/malformed-manifest/source" "$WORKDIR/malformed-manifest/target"