repart: optionally write UAPI.16 manifests of populated partitions and of the partition table

Modelled after SplitName=/--split=: a new ManifestName= setting (defaulting
to "%t", "-" disables) plus --manifest= make systemd-repart write a UAPI.16
manifest for each partition whose file system tree it populates. The
manifest is generated right when the tree is populated (be it in the
staging directory for mkfs --root style population, or in the child
process that copies into the mounted file system), into an O_TMPFILE
file in the image's directory, which is linked into place once the
partition UUIDs and hence the final name are known, next to the image as
<image>.<name>.Uapi16Manifest. The result is what
systemd-manifest would generate for the tree, and can hence be verified
with it.

In addition --manifest-gpt= writes a manifest of the partition table
itself as <image>.Uapi16Manifest: one object per partition, carrying the GPT
type, UUID, label and flags, the partition size and the SHA256 digest of
its contents, and referencing the partition data by its offset in the
image in the "contents" array. Partitions are named like their split-out
files, whether or not --split= is used. For verity signature partitions
the size and digest cover the signature JSON object only, not the NUL
padding after it, matching the (trimmed) split-out file. This is the
format sysupdate consumes for partition resources. It also carries an
object for the image as a whole, named like the image file, with the size
and SHA256 digest of the complete disk. The partition table manifest is
written to a temporary file first, and linked into place once complete.

Incomplete manifests (i.e. where inodes had to be skipped or metadata
omitted) are refused, as systemd-manifest does without --graceful. In
dry-run mode no manifests are written.

The manifest paths show up in the partition table dump, as the split
paths do.

--manifest-fields= selects the per-file fields to include in the
manifests of the partitions' file system trees, like --field= of
systemd-manifest does. If --manifest-private-key= and
--manifest-certificate= (plus optionally --manifest-private-key-source=
and --manifest-certificate-source=) are specified, all generated
manifests are signed with a PKCS#7 signature object. These are
deliberately separate from --private-key= and --certificate=, which
remain in use for verity signature partitions only. All manifest
related switches are grouped in a section of their own in --help and
the man page.

man/repart.d.xml

@@ -994,6 +994,25 @@
         <xi:include href="version-info.xml" xpointer="v252"/></listitem>
       </varlistentry>
 
+      <varlistentry>
+        <term><varname>ManifestName=</varname></term>
+
+        <listitem><para>Configures the suffix to append to the manifest file written for the partition when
+        the <option>--manifest</option> option of
+        <citerefentry><refentrytitle>systemd-repart</refentrytitle><manvolnum>8</manvolnum></citerefentry> is
+        used. The manifest describes the file system tree the partition was populated with (via
+        <varname>CopyFiles=</varname>, <varname>MakeDirectories=</varname>, <varname>MakeSymlinks=</varname>),
+        in the <ulink url="https://uapi-group.org/specifications/specs/file_manifest/">UAPI.16 File
+        Manifest</ulink> format, i.e. what
+        <citerefentry><refentrytitle>systemd-manifest</refentrytitle><manvolnum>1</manvolnum></citerefentry>
+        would generate for the tree. Simple specifier expansion is supported, see below, the same
+        specifiers as for <varname>SplitName=</varname> are understood. Defaults to <literal>%t</literal>.
+        To disable manifest generation for a partition, set <varname>ManifestName=</varname> to
+        <literal>-</literal>.</para>
+
+        <xi:include href="version-info.xml" xpointer="v263"/></listitem>
+      </varlistentry>
+
       <varlistentry>
         <term><varname>Minimize=</varname></term>
 
@@ -1288,7 +1307,8 @@
 
     <para>Specifiers may be used in the <varname>Label=</varname>, <varname>CopyBlocks=</varname>,
     <varname>CopyFiles=</varname>, <varname>MakeDirectories=</varname>, <varname>MakeSymlinks=</varname>,
-    <varname>SplitName=</varname> settings. The following expansions are understood:</para>
+    <varname>SplitName=</varname>, <varname>ManifestName=</varname> settings. The following expansions are
+    understood:</para>
       <table class='specifiers'>
         <title>Specifiers available</title>
         <tgroup cols='3' align='left' colsep='1' rowsep='1'>
@@ -1323,8 +1343,8 @@
         </tgroup>
       </table>
 
-    <para>Additionally, for the <varname>SplitName=</varname> setting, the following specifiers are also
-    understood:</para>
+    <para>Additionally, for the <varname>SplitName=</varname> and <varname>ManifestName=</varname> settings,
+    the following specifiers are also understood:</para>
       <table class='specifiers'>
         <title>Specifiers available</title>
         <tgroup cols='3' align='left' colsep='1' rowsep='1'>

man/systemd-manifest.xml

@@ -274,6 +274,8 @@
 
         <para>The same default set of fields is used by
         <citerefentry><refentrytitle>systemd-dissect</refentrytitle><manvolnum>1</manvolnum></citerefentry>
+        and
+        <citerefentry><refentrytitle>systemd-repart</refentrytitle><manvolnum>8</manvolnum></citerefentry>
         when generating manifests.</para>
 
         <xi:include href="version-info.xml" xpointer="v263"/></listitem>

man/systemd-repart.xml

@@ -371,7 +371,8 @@
 
         <listitem><para>Takes a file system path or an engine or provider specific designation. Configures
         the signing key to use when creating verity signature partitions with the
-        <varname>Verity=signature</varname> setting in partition files.</para>
+        <varname>Verity=signature</varname> setting in partition files. It is not used for signing
+        manifests, see <option>--manifest-private-key=</option> below for that.</para>
 
         <xi:include href="version-info.xml" xpointer="v252"/></listitem>
       </varlistentry>
@@ -393,7 +394,8 @@
 
         <listitem><para>Takes a file system path or a provider specific designation. Configures the PEM
         encoded X.509 certificate to use when creating verity signature partitions with the
-        <varname>Verity=signature</varname> setting in partition files.</para>
+        <varname>Verity=signature</varname> setting in partition files. It is not used for signing
+        manifests, see <option>--manifest-certificate=</option> below for that.</para>
 
         <xi:include href="version-info.xml" xpointer="v252"/></listitem>
       </varlistentry>
@@ -801,6 +803,99 @@
       <xi:include href="standard-options.xml" xpointer="no-legend" />
       <xi:include href="standard-options.xml" xpointer="json" />
     </variablelist>
+
+    <refsect2>
+      <title>Manifest Generation Options</title>
+
+      <para>The following options control generation of <ulink
+      url="https://uapi-group.org/specifications/specs/file_manifest/">UAPI.16 File Manifest</ulink> files
+      describing the generated image:</para>
+
+      <variablelist>
+        <varlistentry>
+          <term><option>--manifest=<replaceable>BOOL</replaceable></option></term>
+
+          <listitem><para>Enables generation of <ulink
+          url="https://uapi-group.org/specifications/specs/file_manifest/">UAPI.16 File Manifest</ulink> files
+          for partitions configured with <varname>ManifestName=</varname>. If enabled, for each partition that
+          is populated with a file system tree (see <varname>CopyFiles=</varname> and related settings in
+          <citerefentry><refentrytitle>repart.d</refentrytitle><manvolnum>5</manvolnum></citerefentry>) a
+          manifest of that tree is written, as
+          <citerefentry><refentrytitle>systemd-manifest</refentrytitle><manvolnum>1</manvolnum></citerefentry>
+          would generate it, which may later be used to verify the contents of the partition. The output file
+          name consists of the image file name suffixed with the name configured with
+          <varname>ManifestName=</varname> and a <literal>.Uapi16Manifest</literal> extension, replacing a
+          <literal>.raw</literal> extension of the image file name if there is one. Partitions that are not
+          populated in this invocation (for example because they already exist) get no manifest.</para>
+
+          <xi:include href="version-info.xml" xpointer="v263"/></listitem>
+        </varlistentry>
+
+        <varlistentry>
+          <term><option>--manifest-gpt=<replaceable>BOOL</replaceable></option></term>
+
+          <listitem><para>Enables generation of a UAPI.16 manifest describing the partition table of the image.
+          If enabled, a manifest with one object per partition is written, carrying the partition's GPT type,
+          UUID, label and flags, its size and the SHA256 digest of its contents, and referencing the
+          partition's data by its offset in the image file in the object's <literal>contents</literal> array.
+          Partitions are named like their split artifacts (see <varname>SplitName=</varname> in
+          <citerefentry><refentrytitle>repart.d</refentrytitle><manvolnum>5</manvolnum></citerefentry>),
+          regardless of whether <option>--split=</option> is used; partitions with <varname>SplitName=</varname>
+          turned off are named after their partition number. For verity signature partitions, the size and
+          digest cover only the signature JSON object, not the NUL padding following it in the partition,
+          matching the split artifact (unless the partition holds no signature at all, in which case the whole
+          partition is covered). Nothing is written in dry-run mode. In addition, an object for the image as a whole is
+          included, named like the image file, carrying the size and SHA256 digest of the complete disk
+          (including the partition table). The output file name is the image file name with a
+          <literal>.Uapi16Manifest</literal> extension, replacing a <literal>.raw</literal> extension if there is
+          one. Such a manifest is suitable for consumption by
+          <citerefentry><refentrytitle>systemd-sysupdate</refentrytitle><manvolnum>8</manvolnum></citerefentry>,
+          see <varname>ManifestFormat=</varname> in
+          <citerefentry><refentrytitle>sysupdate.d</refentrytitle><manvolnum>5</manvolnum></citerefentry>.</para>
+
+          <xi:include href="version-info.xml" xpointer="v263"/></listitem>
+        </varlistentry>
+
+        <varlistentry>
+          <term><option>--manifest-fields=<replaceable>FIELDS</replaceable></option></term>
+
+          <listitem><para>Selects the per-file fields to include in the manifests of populated partitions
+          generated with <option>--manifest=</option>. Takes a comma separated list of field names, the special
+          values <option>all</option>, <option>default</option> and <option>none</option>, and items prefixed
+          with <literal>+</literal> or <literal>-</literal> to add or remove fields from the current set, in
+          the same way as the <option>--field=</option> switch of
+          <citerefentry><refentrytitle>systemd-manifest</refentrytitle><manvolnum>1</manvolnum></citerefentry>,
+          see there for details. Defaults to the same set as <command>systemd-manifest</command>, see there
+          for which fields are included by default and which are not (in particular, owning user and group
+          names and modification times are not). Has no effect
+          on the partition table manifest generated with <option>--manifest-gpt=</option>, whose fields are
+          fixed.</para>
+
+          <xi:include href="version-info.xml" xpointer="v263"/></listitem>
+        </varlistentry>
+
+        <varlistentry>
+          <term><option>--manifest-private-key=</option></term>
+          <term><option>--manifest-private-key-source=</option></term>
+          <term><option>--manifest-certificate=</option></term>
+          <term><option>--manifest-certificate-source=</option></term>
+
+          <listitem><para>Configure the private key and X.509 certificate to sign the manifests generated with
+          <option>--manifest=</option> and <option>--manifest-gpt=</option> with. If
+          <option>--manifest-private-key=</option> is specified the manifests are signed with a PKCS#7 signature
+          object as described in the specification, in the same way as
+          <citerefentry><refentrytitle>systemd-manifest</refentrytitle><manvolnum>1</manvolnum></citerefentry>
+          signs them, otherwise they are not signed. If one of <option>--manifest-private-key=</option> and
+          <option>--manifest-certificate=</option> is specified the other must be specified too. The options
+          take the same arguments as <option>--private-key=</option>,
+          <option>--private-key-source=</option>, <option>--certificate=</option> and
+          <option>--certificate-source=</option>, but are independent of them: the latter are used for verity
+          signature partitions only, the former for manifests only.</para>
+
+          <xi:include href="version-info.xml" xpointer="v263"/></listitem>
+        </varlistentry>
+      </variablelist>
+    </refsect2>
   </refsect1>
 
   <refsect1>

test/units/TEST-58-REPART.sh

@@ -1498,6 +1498,9 @@ EOF
     mkdir -p /run/verity.d
     ln -sf "$defs/verity.crt" /run/verity.d/ok.crt
 
+    # Manifests are signed with a key of their own, separate from the verity one
+    openssl req -x509 -newkey rsa:2048 -nodes -keyout "$defs/manifest.key" -out "$defs/manifest.crt" -subj /CN=manifest -days 2 2>/dev/null
+
     output=$(systemd-repart --offline="$OFFLINE" \
                             --definitions="$defs" \
                             --seed="$seed" \
@@ -1505,6 +1508,11 @@ EOF
                             --empty=create \
                             --size=auto \
                             --split=yes \
+                            --manifest=yes \
+                            --manifest-gpt=yes \
+                            --manifest-fields=size,hash \
+                            --manifest-private-key="$defs/manifest.key" \
+                            --manifest-certificate="$defs/manifest.crt" \
                             --json=pretty \
                             --private-key="$defs/verity.key" \
                             --certificate="$defs/verity.crt" \
@@ -1524,6 +1532,71 @@ EOF
     assert_neq "$sig_split" "null"
     jq . "$sig_split" >/dev/null
 
+    # The manifest of the root partition's file system tree lists the files copied into it, and the manifest
+    # of the partition table describes the partitions, with the split artifacts as sources, hence it must
+    # verify against the directory the split artifacts are in.
+    root_manifest=$(jq -r ".[] | select(.type == \"root-${architecture}\") | .manifest_path" <<<"$output")
+    assert_eq "$root_manifest" "$imgs/verity.root-${architecture}.Uapi16Manifest"
+    # CopyFiles= places the definitions directory at the same path inside the partition
+    jq --seq -s -e ".[] | select(.name == \"${defs#/}/verity-data.conf\")" <"$root_manifest" >/dev/null
+    # Only the requested fields are included
+    jq --seq -s -e ".[] | select(.name == \"${defs#/}/verity-data.conf\") | has(\"size\") and has(\"sha256\") and (has(\"mode\") | not)" <"$root_manifest" >/dev/null
+    # The manifest is signed with the manifest key, not the verity key
+    systemd-manifest verify --certificate="$defs/manifest.crt" / "$root_manifest"
+    (! systemd-manifest verify --certificate="$defs/verity.crt" / "$root_manifest")
+    assert_eq "$(jq -r ".[] | select(.type == \"root-${architecture}-verity-sig\") | .manifest_path" <<<"$output")" "-"
+
+    gpt_manifest="$imgs/verity.Uapi16Manifest"
+    jq --seq -s -e '.[] | select(.mediaType == "application/vnd.uapi.16.signature") | .mechanism == "pkcs7"' <"$gpt_manifest" >/dev/null
+    # The image as a whole is covered too
+    jq --seq -s -e ".[] | select(.name == \"verity\") | .size == $(stat -c %s "$imgs/verity") and .sha256 == \"$(sha256sum "$imgs/verity" | cut -d' ' -f1)\"" <"$gpt_manifest" >/dev/null
+    root_split=$(jq -r ".[] | select(.type == \"root-${architecture}\") | .split_path" <<<"$output")
+    jq --seq -s -e ".[] | select(.name == \"$(basename "$root_split")\") | (.contents | length) == 1 and .contents[0].file == \"$(basename "$imgs/verity")\" and .contents[0].offset > 0 and .gptLabel != null" <"$gpt_manifest" >/dev/null
+    # The verity signature partition is described like its split artifact, i.e. without the NUL padding
+    jq --seq -s -e ".[] | select(.name == \"$(basename "$sig_split")\") | (.contents | length) == 1 and .size == $(stat -c %s "$sig_split")" <"$gpt_manifest" >/dev/null
+    systemd-manifest verify --certificate="$defs/manifest.crt" "$imgs" "$gpt_manifest"
+    (! systemd-manifest verify --certificate="$defs/verity.crt" "$imgs" "$gpt_manifest")
+
+    # A verity signature partition without any signature in it (e.g. an empty B slot) is described with
+    # its full size, rather than failing
+    mkdir "$defs/empty-sig"
+    tee "$defs/empty-sig/10-sig.conf" <<EOF
+[Partition]
+Type=root-${architecture}-verity-sig
+SplitName=-
+EOF
+    systemd-repart --offline="$OFFLINE" \
+                   --definitions="$defs/empty-sig" \
+                   --seed="$seed" \
+                   --dry-run=no \
+                   --empty=create \
+                   --size=auto \
+                   --manifest-gpt=yes \
+                   "$imgs/empty-sig"
+    sig_size=$(sfdisk --json "$imgs/empty-sig" | jq '.partitiontable.partitions[0].size * .partitiontable.sectorsize')
+    jq --seq -s -e ".[] | select(.name == \"empty-sig.0\") | .size == $sig_size" <"$imgs/empty-sig.Uapi16Manifest" >/dev/null
+
+    # Colliding names in the partition table manifest (here: an A/B pair of root partitions with the
+    # default SplitName=) are refused before the partition table is written
+    mkdir "$defs/collide"
+    for i in a b; do
+        tee "$defs/collide/10-root-$i.conf" <<EOF
+[Partition]
+Type=root-${architecture}
+SizeMinBytes=10M
+EOF
+    done
+    truncate -s 64M "$imgs/collide"
+    (! systemd-repart --offline="$OFFLINE" \
+                      --definitions="$defs/collide" \
+                      --seed="$seed" \
+                      --dry-run=no \
+                      --empty=allow \
+                      --manifest-gpt=yes \
+                      "$imgs/collide")
+    (! sfdisk --dump "$imgs/collide")
+    [[ ! -e "$imgs/collide.Uapi16Manifest" ]]
+
     # Check that offline signing works and the resulting image is valid
 
     output=$(systemd-repart --offline="$OFFLINE" \