feat(artifacts): inventory and extract U-Boot FIT images (#7771)
* feat(artifacts): inventory and extract U-Boot FIT images open_binary refused U-Boot Flattened Image Tree files as an unsupported format, so inspect_artifact and extract_artifact could not list or recover a FIT's kernel, devicetree, ramdisk and loadable images. Recognize an FDT v16/v17 with a top-level /images node and parse it with the existing parseDevicetree. Each /images child becomes an inventory entry named images/<unit-name>.bin that extracts its stored bytes from embedded data, data-offset (past the 4-aligned devicetree end) or data-position (from the file start). Image format_metadata records the declared type, arch, os, compression, load and entry; root metadata records the description, timestamp and configurations. crc32, md5, sha1, sha256, sha384 and sha512 hash nodes are verified in one streamed pass per image; other algorithms are reported unverified. Signatures are reported, not verified, and nothing is decompressed, decrypted or executed. Data outside the file, overlapping or duplicate image data, a missing or repeated data source and hash mismatches fail closed. The devicetree is bounded by the fixed 16 MiB allowance; external image data is streamed from disk. Refs #7129 * fix(artifacts): bound FIT configuration references and suggest analysis only for plain images * fix(artifacts): retain buffered FIT stream cancellation * test(artifacts): gate FIT faults on the observed read phase * fix(artifacts): verify FIT links and summary identities * fix(artifacts): bound FIT diagnostics and retain unknown signatures * fix(artifacts): compare FIT declarations and resolved data links * fix(artifacts): admit FIT raw probes through the shared loader contract * fix(artifacts): preserve FIT field availability independently * fix(artifacts): isolate nested entry metadata from reader facts --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: gensokyo-dev <342580529+users.noreply.github.com>
This commit is contained in:
32 files changed
+3511
-24
No files matched your search
+10
@@ -317,6 +317,16 @@ overlap and archive expansion checks. The native streaming decoder keeps the
|
||||
existing 128 MiB Zstandard window cap; frames requiring a larger window are
|
||||
refused. Deprecated method 20 and other unregistered ZIP codecs remain unsupported.
|
||||
|
||||
Inventory and extract U-Boot FIT images:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
See [U-Boot FIT images](fit-images.md) for recognition, data sources, hashes
|
||||
and limits.
|
||||
|
||||
Inspect local model declarations without loading weights:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -323,3 +323,11 @@ addresses are facts from the header, not proven entry points, and compressed
|
||||
payloads stay as stored. Intel HEX/S-record and FIT images are separate
|
||||
readers and are not covered here. See [U-Boot legacy uImage](uimage.md) for
|
||||
component padding, preserved names, resource policies and verification scope.
|
||||
|
||||
## Inspect U-Boot FIT images
|
||||
|
||||
Use `inspect-artifact` / `inspect_artifact` on a U-Boot FIT (`.itb`) for its
|
||||
images, declared load and entry addresses, configurations and verified hashes,
|
||||
then `extract-artifact` / `extract_artifact` for one raw file per image.
|
||||
Decoding is portable and requires no external tool. See
|
||||
[U-Boot FIT images](fit-images.md).
|
||||
@@ -0,0 +1,172 @@
|
||||
# U-Boot FIT images
|
||||
|
||||
REA inventories U-Boot Flattened Image Tree (FIT, `.itb`) files through
|
||||
`inspect-artifact` / `inspect_artifact`, and materializes the stored bytes of
|
||||
each image through `extract-artifact` / `extract_artifact`. Decoding is static
|
||||
and needs no external tool.
|
||||
|
||||
```sh
|
||||
rea inspect-artifact ./boot.itb --json
|
||||
rea inspect-artifact ./boot.itb --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact ./boot.itb --json
|
||||
```
|
||||
|
||||
For MCP, open the file with `open_binary`, then call `inspect_artifact`. Summary
|
||||
delivery retains the complete Evidence; use `parent_evidence_id` with
|
||||
`inspect_analysis_view` to read selected images and fields. `extract_artifact`
|
||||
returns a fresh `output_root` containing one raw file per image. FIT summaries
|
||||
use the current projection identity; there is no pre-coverage FIT summary variant.
|
||||
|
||||
## Recognition
|
||||
|
||||
A FIT is a flattened devicetree (FDT v16 or v17) with a top-level `/images`
|
||||
node. Recognition walks the structure block within the first 4 KiB of the file
|
||||
only; the file name does not matter. A devicetree whose `/images` node begins
|
||||
after 4 KiB, a devicetree without `/images`, and any other FDT version stay on
|
||||
the plain devicetree route: `open_binary` refuses them and
|
||||
[`inspect_devicetree`](devicetree.md) still parses them. A U-Boot control DTB
|
||||
that contains `/images` is treated as a FIT. The container format is `fit`.
|
||||
|
||||
Recognition is a probe. The complete devicetree parse decides, so a file whose
|
||||
prefix declares `/images` but whose structure is malformed fails as a
|
||||
malformed FIT rather than falling back to plain classification.
|
||||
|
||||
## Images and metadata
|
||||
|
||||
Each child of `/images` is one inventory entry named `images/<unit-name>.bin`,
|
||||
listed in devicetree order. Its bytes come from exactly one data source:
|
||||
|
||||
| Source | Stored bytes |
|
||||
| --------------- | --------------------------------------------------------------------------------------- |
|
||||
| `data` | The embedded property value |
|
||||
| `data-offset` | `data-size` bytes at `data-offset` past the devicetree end, aligned to 4 (`mkimage -E`) |
|
||||
| `data-position` | `data-size` bytes at `data-position` from the start of the file (`mkimage -p`) |
|
||||
|
||||
`data-position` is read as U-Boot reads it: an offset from the start of the
|
||||
FIT, not a machine address.
|
||||
|
||||
Image `format_metadata` (`fit-image`) records `index`, `name`, `description`,
|
||||
`type`, `arch`, `os`, `compression`, `load` and `entry`, the `data_source`,
|
||||
the raw `declared_offset` for external data, `image_data` (the shared-data
|
||||
link), `cipher_present`, `cipher_algo`, `hash_algorithms`, `hashes_verified`,
|
||||
`hashes_unverified`, `signature_count`, `signatures_unreadable` and
|
||||
`signature_algorithms`. Algorithm lists are arrays. Nullable declaration fields
|
||||
are absent unless named in `unreadable_fields`, which records parsing failures
|
||||
per field. Hash or signature warnings do not make unrelated declarations unknown.
|
||||
`load` and `entry` fold one or two 32-bit cells into a lowercase
|
||||
`0x…` value, as U-Boot does, independent of `#address-cells`.
|
||||
|
||||
Root `format_metadata` (`fit`) records `fdt_version`, `fdt_totalsize`,
|
||||
`external_data_base` (the aligned devicetree end), the root `description`,
|
||||
`timestamp` (seconds since 1970) and `address_cells`, plus
|
||||
`default_configuration` and `configurations`. A missing description or
|
||||
timestamp is `null`.
|
||||
|
||||
Image bytes are reported as stored. Compressed images are not decompressed and
|
||||
ciphered images are not decrypted: entries report `encrypted: false` and the
|
||||
cipher declaration in `cipher_present` and algorithm in `cipher_algo`. A missing
|
||||
or malformed algorithm stays `null`; it does not establish unciphered bytes.
|
||||
Embedded `image-data` links require byte-identical stored properties; external
|
||||
links require matching physical ranges.
|
||||
|
||||
The extracted file holds the stored bytes. Direct-analysis suggestions require
|
||||
`compression: "none"` and `cipher_present: false`.
|
||||
|
||||
## Hashes and signatures
|
||||
|
||||
Hash subnodes are children whose name starts with `hash`. Inventory verifies
|
||||
`crc32` (4 bytes, big-endian), `md5`, `sha1`, `sha256`, `sha384` and `sha512`
|
||||
over the stored bytes, the same bytes mkimage hashed after any ciphering. All
|
||||
hashes of an image are computed in one streamed pass while the inventory is
|
||||
prepared, so a mismatch fails the whole inventory, naming the image, hash
|
||||
node, algorithm, and the declared and computed values. A supported algorithm
|
||||
whose value has the wrong length also fails.
|
||||
|
||||
Other algorithms such as `crc16-ccitt`, `crc8` or `sm3_256`, a hash without a
|
||||
readable `algo`, and a hash without a `value` are not verified; each is
|
||||
counted in `hashes_unverified` and reported as an image limitation.
|
||||
|
||||
Signature subnodes (names starting with `signature`) on images and
|
||||
configurations retain the total declaration count, readable algorithms and
|
||||
`signatures_unreadable`. Missing or malformed algorithms carry source limitations
|
||||
so unknown declarations remain distinguishable from absence. Signatures are never
|
||||
verified.
|
||||
|
||||
## Configurations
|
||||
|
||||
`/configurations` is optional. Each child records its `name`, `description`,
|
||||
and the image references `kernel`, `firmware`, `fdt` (base plus overlays),
|
||||
`ramdisk` and `loadables`, always as lists, plus its signature algorithms.
|
||||
`compatible`, `fpga`, `script`, `cmdline` and `load-only` are not reported. A
|
||||
reference to a name that is not an `/images` child, and a `default` that names
|
||||
no configuration, are reported as limitations and kept as declared. Each
|
||||
configuration carries `unresolved_reference_count`; dangling references share one
|
||||
configuration-level diagnostic while every original reference remains available.
|
||||
|
||||
## Facts, limitations and failures
|
||||
|
||||
| Situation | Outcome |
|
||||
| ---------------------------------------------------------------------------------------------- | ------------------------------------ |
|
||||
| Malformed devicetree, truncated header, `totalsize` past the end of the file | Fails as malformed |
|
||||
| No data source, more than one data source, external data without `data-size` | Fails as malformed |
|
||||
| `data-offset`, `data-position` or `data-size` not one 32-bit cell, `image-data` not one string | Fails as malformed |
|
||||
| External data outside the file, overlapping the devicetree or another image | Fails as malformed |
|
||||
| Two images with an identical external range linked by `image-data` | Accepted as shared data |
|
||||
| A dangling, chained or mismatched `image-data` link | Fails as malformed |
|
||||
| Hash mismatch, or a supported hash value with the wrong length | Fails integrity |
|
||||
| Unsupported, algorithm-less or value-less hashes | Unverified, reported as a limitation |
|
||||
| Malformed `description`, `type`, `arch`, `os`, `compression`, `load`, `entry` or `timestamp` | `null`, reported as a limitation |
|
||||
| Dangling configuration reference, missing default configuration | Reported as a limitation |
|
||||
|
||||
Failures apply to the whole file and return no partial inventory.
|
||||
|
||||
## Limits
|
||||
|
||||
The devicetree, including any embedded image data, must be at most 16 MiB; a
|
||||
larger `totalsize` fails as a memory limit. Store large images externally
|
||||
(`mkimage -E`): external data of any size is streamed from the file and never
|
||||
loaded into memory. A FIT may declare at most 1,024 images and 1,024
|
||||
configurations, and an image at most 8 hash nodes; the devicetree parser's
|
||||
node, property and depth bounds also apply. The sum of declared image sizes
|
||||
is bounded by the shared archive expansion budget. These are fixed resource
|
||||
bounds with no runtime override. Configurable ceilings are tracked in
|
||||
[#6176](https://github.com/morluto/rea/issues/6176).
|
||||
|
||||
## Analyzing an image
|
||||
|
||||
Inventory does not execute anything, and `load` and `entry` are declarations.
|
||||
After extraction:
|
||||
|
||||
- inspect `flat_dt` images with [`inspect_devicetree`](devicetree.md);
|
||||
- open an uncompressed image with `open_binary` using format `raw-binary`, an
|
||||
architecture matching its declared `arch`, and `entry_point` set to its
|
||||
declared `load` address; raw-binary maps file offset 0 and seeds analysis at
|
||||
`entry_point`. Automatic suggestions require nonempty bytes, an admitted
|
||||
architecture/address/extent, and a declared entry that is absent or equal to
|
||||
load. Distinct or unreadable entry declarations remain in Evidence without
|
||||
a raw-loader suggestion, since this loader cannot represent separate mapping
|
||||
and analysis entry addresses.
|
||||
|
||||
## Comparison
|
||||
|
||||
Artifact comparison reports changes to an image's payload interpretation,
|
||||
including load and entry declarations. Equivalent external offset forms and
|
||||
inventory positions leave those declarations unchanged. Unavailable
|
||||
interpretations remain unknown, while independently known changes survive.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed hashes cover embedded, `data-offset` and
|
||||
`data-position` layouts, shared data, overlap and bounds failures, every
|
||||
verified hash algorithm, configurations, resource limits, inventory,
|
||||
extraction and reader lifecycle. CLI and stdio MCP tests exercise the same
|
||||
artifact provider and advertised schemas. Real U-Boot `mkimage` 2023.01 output
|
||||
has also been verified for embedded, `-E -B 0x200`, and
|
||||
`-E -p 0x2000 -B 0x200` layouts, with byte-exact extraction checked against
|
||||
`dumpimage`. The OpenWrt 24.10.3 ABT ASR3000 recovery FIT was checked against
|
||||
`mkimage`, `fdtget` and `dumpimage`, including its kernel, initrd and devicetree.
|
||||
No image was executed.
|
||||
|
||||
For small containers, complete delivery avoids an additional view request.
|
||||
Choose summary and selected views when the omitted facts justify that extra
|
||||
request; summary delivery does not reduce the underlying inventory work.
|
||||
@@ -78,6 +78,7 @@ import { hasGodotPckHeader } from "../domain/godotPckFormat.js";
|
||||
import { hasUImageMagic } from "../domain/uimageFormat.js";
|
||||
import { hasUnityFsHeader } from "../domain/unityfsFormat.js";
|
||||
import { detectHexRecordEncoding } from "../domain/firmware/hexRecords.js";
|
||||
import { hasFitImagesNode } from "../domain/firmware/fitImage.js";
|
||||
import {
|
||||
ESP8266_REFUSAL,
|
||||
recognizeEspImage,
|
||||
@@ -682,6 +683,7 @@ const detectArtifactFormat = async (
|
||||
constraint: "file_format_rejected",
|
||||
});
|
||||
if (esp === "esp-image") return "esp-image";
|
||||
if (hasFitImagesNode(magic)) return "fit";
|
||||
// Binary plists, including keyed archives, often use other extensions.
|
||||
if (hasBinaryPlistSignature(magic)) return "plist";
|
||||
const named = namedArtifactFormat(lower);
|
||||
@@ -759,6 +761,7 @@ const isArchiveFormat = (
|
||||
"intel-hex",
|
||||
"srec",
|
||||
"esp-image",
|
||||
"fit",
|
||||
].includes(format);
|
||||
|
||||
/** Whether the leading bytes begin any executable header family, valid or not. */
|
||||
|
||||
@@ -90,7 +90,7 @@ export const copyArtifactEntry = (entry: ArtifactEntry): ArtifactEntry => ({
|
||||
entry.declaredIntegrity === null ? null : { ...entry.declaredIntegrity },
|
||||
...(entry.formatMetadata === undefined
|
||||
? {}
|
||||
: { formatMetadata: { ...entry.formatMetadata } }),
|
||||
: { formatMetadata: structuredClone(entry.formatMetadata) }),
|
||||
...(entry.sourceIdentity === undefined
|
||||
? {}
|
||||
: { sourceIdentity: { ...entry.sourceIdentity } }),
|
||||
@@ -136,19 +136,32 @@ const sameDeclaredIntegrity = (
|
||||
left.algorithm === right.algorithm &&
|
||||
left.digest === right.digest);
|
||||
|
||||
/** Format metadata contracts contain only primitive source fields. */
|
||||
const sameFormatMetadata = (
|
||||
left: ArtifactFormatMetadata | undefined,
|
||||
right: ArtifactFormatMetadata | undefined,
|
||||
): boolean =>
|
||||
left === right ||
|
||||
(left !== undefined &&
|
||||
right !== undefined &&
|
||||
/** Compare cloned JSON facts structurally while retaining the primitive fast path. */
|
||||
const sameFormatMetadata = (left: unknown, right: unknown): boolean => {
|
||||
if (left === right) return true;
|
||||
if (Array.isArray(left) || Array.isArray(right))
|
||||
return (
|
||||
Array.isArray(left) &&
|
||||
Array.isArray(right) &&
|
||||
left.length === right.length &&
|
||||
left.every((value, index) => sameFormatMetadata(value, right[index]))
|
||||
);
|
||||
if (
|
||||
left === null ||
|
||||
right === null ||
|
||||
typeof left !== "object" ||
|
||||
typeof right !== "object"
|
||||
)
|
||||
return false;
|
||||
return (
|
||||
Object.keys(left).length === Object.keys(right).length &&
|
||||
Object.entries(left).every(
|
||||
([key, value]) =>
|
||||
Object.hasOwn(right, key) && Reflect.get(right, key) === value,
|
||||
));
|
||||
Object.hasOwn(right, key) &&
|
||||
sameFormatMetadata(value, Reflect.get(right, key)),
|
||||
)
|
||||
);
|
||||
};
|
||||
|
||||
/** One entry or subtree deliberately excluded from a traversal. */
|
||||
export interface ArtifactTraversalOmission {
|
||||
@@ -184,6 +197,7 @@ export interface ArtifactReader {
|
||||
| "intel-hex"
|
||||
| "srec"
|
||||
| "esp-image"
|
||||
| "fit"
|
||||
| "file";
|
||||
entries(
|
||||
signal?: AbortSignal,
|
||||
|
||||
@@ -0,0 +1,536 @@
|
||||
import { getEventListeners, once } from "node:events";
|
||||
import { readFile, truncate, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect, it, vi } from "vitest";
|
||||
|
||||
import { FitArtifactReader } from "./FitArtifactReader.js";
|
||||
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
|
||||
import { ArtifactFileSource } from "./ArtifactFileSource.js";
|
||||
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
||||
import { extractArtifact } from "./extraction/ArtifactExtraction.js";
|
||||
import * as fitStructure from "./fit/FitStructure.js";
|
||||
import * as fileChunks from "../filesystem/readFileHandleChunks.js";
|
||||
import {
|
||||
FIT_DEVICETREE,
|
||||
FIT_EMBEDDED,
|
||||
FIT_EXTERNAL,
|
||||
FIT_KERNEL,
|
||||
fitFixture,
|
||||
} from "./fit/FitImage.fixture.js";
|
||||
import {
|
||||
cells,
|
||||
devicetreeFixture,
|
||||
} from "../firmware/devicetree/Devicetree.fixture.js";
|
||||
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
|
||||
import { awaitAcknowledgement } from "../../tests/fixtures/lifecycleAcknowledgement.js";
|
||||
import { scanCanonicalArtifactInventory } from "../../tests/fixtures/artifactInventory.js";
|
||||
|
||||
const MIB = 1024 * 1024;
|
||||
|
||||
const writeFit = async (name: string, bytes: Buffer): Promise<string> => {
|
||||
const directory = await createTestTempDirectory("rea-fit-");
|
||||
const path = join(directory, name);
|
||||
await writeFile(path, bytes);
|
||||
return path;
|
||||
};
|
||||
|
||||
const withTotalsize = (bytes: Buffer, totalsize: number): Buffer => {
|
||||
const copy = Buffer.from(bytes);
|
||||
copy.writeUInt32BE(totalsize, 4);
|
||||
return copy;
|
||||
};
|
||||
|
||||
const LARGE_FIT = fitFixture({
|
||||
images: [{ name: "kernel", data: Buffer.alloc(MIB, 7), hashes: ["sha256"] }],
|
||||
mode: "offset",
|
||||
});
|
||||
|
||||
const firstEntry = async (reader: FitArtifactReader) => {
|
||||
const { value: entry } = await reader
|
||||
.entries()
|
||||
[Symbol.asyncIterator]()
|
||||
.next();
|
||||
if (entry === undefined) throw new Error("Missing FIT image");
|
||||
return entry;
|
||||
};
|
||||
|
||||
it.each([
|
||||
["embedded", FIT_EMBEDDED, "embedded"],
|
||||
["external", FIT_EXTERNAL, "data-offset"],
|
||||
] as const)(
|
||||
"inventories %s FIT images with root and image facts",
|
||||
async (_mode, bytes, dataSource) => {
|
||||
const snapshot = await scanCanonicalArtifactInventory(
|
||||
await writeFit("boot.itb", bytes),
|
||||
);
|
||||
expect(snapshot.manifest.root_format).toBe("fit");
|
||||
expect(snapshot.coverage).toMatchObject({
|
||||
scope: "container-entries",
|
||||
status: "complete",
|
||||
});
|
||||
expect(
|
||||
snapshot.limitations.some((limitation) =>
|
||||
limitation.startsWith("FIT inventory"),
|
||||
),
|
||||
).toBe(true);
|
||||
const root = snapshot.occurrences.find((row) => row.logical_path === ".");
|
||||
expect(root).toMatchObject({
|
||||
artifact_kind: "container",
|
||||
format_metadata: {
|
||||
kind: "fit",
|
||||
fdt_version: 17,
|
||||
description: "Test FIT",
|
||||
timestamp: 1_700_000_000,
|
||||
address_cells: 1,
|
||||
default_configuration: "conf-1",
|
||||
configurations: [
|
||||
{
|
||||
name: "conf-1",
|
||||
description: "Boot Linux",
|
||||
kernel: ["kernel"],
|
||||
fdt: ["fdt-1"],
|
||||
signature_algorithms: ["sha256,rsa2048"],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
const kernel = snapshot.occurrences.find(
|
||||
(row) => row.logical_path === "images/kernel.bin",
|
||||
);
|
||||
const devicetree = snapshot.occurrences.find(
|
||||
(row) => row.logical_path === "images/fdt-1.bin",
|
||||
);
|
||||
expect(snapshot.occurrences).toHaveLength(3);
|
||||
for (const [row, data] of [
|
||||
[kernel, FIT_KERNEL],
|
||||
[devicetree, FIT_DEVICETREE],
|
||||
] as const) {
|
||||
const offset = row?.source_location?.offset ?? -1;
|
||||
expect(row?.declared_size).toBe(data.length);
|
||||
expect(bytes.subarray(offset, offset + data.length).equals(data)).toBe(
|
||||
true,
|
||||
);
|
||||
}
|
||||
expect(kernel?.format_metadata).toMatchObject({
|
||||
kind: "fit-image",
|
||||
name: "kernel",
|
||||
type: "kernel",
|
||||
arch: "arm64",
|
||||
load: "0x40200000",
|
||||
entry: "0x40200000",
|
||||
data_source: dataSource,
|
||||
hashes_verified: 6,
|
||||
hashes_unverified: 0,
|
||||
});
|
||||
expect(devicetree?.format_metadata).toMatchObject({
|
||||
kind: "fit-image",
|
||||
type: "flat_dt",
|
||||
hashes_verified: 1,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["crc32", "md5", "sha1", "sha256", "sha384", "sha512"])(
|
||||
"fails a %s hash mismatch naming the image, node and algorithm",
|
||||
async (algo) => {
|
||||
const bytes = fitFixture({
|
||||
images: [{ name: "kernel", data: FIT_KERNEL, hashes: [algo] }],
|
||||
mode: "offset",
|
||||
});
|
||||
const kernel = bytes.indexOf(FIT_KERNEL);
|
||||
bytes[kernel + 10] = (bytes[kernel + 10] ?? 0) ^ 1;
|
||||
await expect(
|
||||
scanCanonicalArtifactInventory(await writeFit("bad.itb", bytes)),
|
||||
).rejects.toMatchObject({
|
||||
reason: "integrity",
|
||||
message: expect.stringMatching(
|
||||
new RegExp(
|
||||
`^FIT image kernel hash-1 ${algo} does not match: declared [0-9a-f]+, computed [0-9a-f]+$`,
|
||||
"u",
|
||||
),
|
||||
),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
["embedded", FIT_EMBEDDED],
|
||||
["external", FIT_EXTERNAL],
|
||||
] as const)(
|
||||
"extracts byte-exact %s images that re-inventory as raw files",
|
||||
async (_mode, bytes) => {
|
||||
const path = await writeFit("boot.itb", bytes);
|
||||
const resourceScope = new ArtifactResourceScope();
|
||||
try {
|
||||
const extracted = await extractArtifact({
|
||||
inputPath: path,
|
||||
inputFormat: "fit",
|
||||
outputRoot: join(path, "..", "out"),
|
||||
environment: process.env,
|
||||
integrityPolicy: "fail",
|
||||
resourceScope,
|
||||
});
|
||||
expect(extracted.containment_verified).toBe(true);
|
||||
for (const [name, data] of [
|
||||
["kernel.bin", FIT_KERNEL],
|
||||
["fdt-1.bin", FIT_DEVICETREE],
|
||||
] as const)
|
||||
expect(
|
||||
Buffer.compare(
|
||||
await readFile(join(extracted.output_root, "images", name)),
|
||||
data,
|
||||
),
|
||||
).toBe(0);
|
||||
const reinventoried = await scanCanonicalArtifactInventory(
|
||||
extracted.output_root,
|
||||
);
|
||||
expect(
|
||||
reinventoried.occurrences
|
||||
.filter((row) => row.entry_kind === "file")
|
||||
.map((row) => [row.logical_path, row.artifact_format])
|
||||
.sort(),
|
||||
).toEqual([
|
||||
["images/fdt-1.bin", "file"],
|
||||
["images/kernel.bin", "file"],
|
||||
]);
|
||||
} finally {
|
||||
await resourceScope.close();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("inventories an external-data FIT larger than 16 MiB", async () => {
|
||||
const bytes = fitFixture({
|
||||
images: [
|
||||
{
|
||||
name: "rootfs",
|
||||
data: Buffer.alloc(0),
|
||||
properties: { "data-offset": cells(17 * MIB), "data-size": cells(4) },
|
||||
},
|
||||
],
|
||||
mode: "offset",
|
||||
});
|
||||
const path = await writeFit("large.itb", bytes);
|
||||
const base = Math.ceil(bytes.readUInt32BE(4) / 4) * 4;
|
||||
await truncate(path, base + 17 * MIB + 4);
|
||||
const snapshot = await scanCanonicalArtifactInventory(path);
|
||||
expect(
|
||||
snapshot.occurrences.find(
|
||||
(row) => row.logical_path === "images/rootfs.bin",
|
||||
),
|
||||
).toMatchObject({
|
||||
declared_size: 4,
|
||||
source_location: { offset: base + 17 * MIB, length: 4 },
|
||||
});
|
||||
});
|
||||
|
||||
it("fails an embedded devicetree above 16 MiB with a fixed limit", async () => {
|
||||
const path = await writeFit(
|
||||
"huge.itb",
|
||||
withTotalsize(FIT_EMBEDDED, 16 * MIB + 4),
|
||||
);
|
||||
await truncate(path, 16 * MIB + 8);
|
||||
await expect(scanCanonicalArtifactInventory(path)).rejects.toMatchObject({
|
||||
reason: "limit",
|
||||
boundary: "memory",
|
||||
});
|
||||
});
|
||||
|
||||
it("fails a devicetree totalsize past the end of the file", async () => {
|
||||
await expect(
|
||||
scanCanonicalArtifactInventory(
|
||||
await writeFit(
|
||||
"short.itb",
|
||||
withTotalsize(FIT_EMBEDDED, FIT_EMBEDDED.length + 4),
|
||||
),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
reason: "format",
|
||||
message: expect.stringContaining("FIT devicetree is truncated"),
|
||||
});
|
||||
});
|
||||
|
||||
it("fails /images followed by a malformed structure, and leaves a plain DTB a file", async () => {
|
||||
const bytes = Buffer.from(FIT_EMBEDDED);
|
||||
const end = bytes.readUInt32BE(8) + bytes.readUInt32BE(36) - 4;
|
||||
expect(bytes.readUInt32BE(end)).toBe(9);
|
||||
bytes.writeUInt32BE(10, end);
|
||||
await expect(
|
||||
scanCanonicalArtifactInventory(await writeFit("broken.itb", bytes)),
|
||||
).rejects.toMatchObject({
|
||||
reason: "format",
|
||||
message: "FIT is malformed: Unknown FDT token 10",
|
||||
});
|
||||
const plain = await scanCanonicalArtifactInventory(
|
||||
await writeFit("board.dtb", devicetreeFixture()),
|
||||
);
|
||||
expect(plain.manifest.root_format).toBe("file");
|
||||
});
|
||||
|
||||
it("rejects a changed carrier during extraction preparation", async () => {
|
||||
const reader = new FitArtifactReader(
|
||||
await writeFit("changed.itb", FIT_EMBEDDED),
|
||||
);
|
||||
try {
|
||||
await expect(reader.prepareContainer("0".repeat(64))).rejects.toMatchObject(
|
||||
{ reason: "integrity" },
|
||||
);
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("isolates returned declaration arrays from saved FIT facts", async () => {
|
||||
const reader = new FitArtifactReader(
|
||||
await writeFit("metadata-copy.itb", FIT_EXTERNAL),
|
||||
);
|
||||
try {
|
||||
const entry = await firstEntry(reader);
|
||||
if (entry.formatMetadata?.kind !== "fit-image")
|
||||
throw new Error("Missing FIT image metadata");
|
||||
const original = structuredClone(entry.formatMetadata);
|
||||
entry.formatMetadata.unreadable_fields.push("load");
|
||||
entry.formatMetadata.hash_algorithms[0] = "changed";
|
||||
entry.formatMetadata.signature_algorithms.push("changed");
|
||||
const fresh = await firstEntry(reader);
|
||||
expect(fresh.formatMetadata).toEqual(original);
|
||||
await expect(reader.open(entry)).rejects.toMatchObject({
|
||||
reason: "integrity",
|
||||
});
|
||||
const stream = await reader.open(fresh);
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of stream) chunks.push(chunk as Buffer);
|
||||
expect(Buffer.concat(chunks).equals(FIT_KERNEL)).toBe(true);
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects forged entries and changed carrier identity, then closes idempotently", async () => {
|
||||
const path = await writeFit("identity.itb", FIT_EXTERNAL);
|
||||
const reader = new FitArtifactReader(path);
|
||||
try {
|
||||
const entry = await firstEntry(reader);
|
||||
if (entry.formatMetadata?.kind !== "fit-image")
|
||||
throw new Error("Missing FIT image metadata");
|
||||
await expect(
|
||||
reader.open({
|
||||
...entry,
|
||||
formatMetadata: {
|
||||
...entry.formatMetadata,
|
||||
unreadable_fields: ["load"],
|
||||
},
|
||||
}),
|
||||
).rejects.toMatchObject({ reason: "integrity" });
|
||||
await expect(
|
||||
reader.open({ ...entry, byteOffset: 0 }),
|
||||
).rejects.toMatchObject({ reason: "integrity" });
|
||||
await expect(
|
||||
reader.open({ ...entry, adapterKey: "fit:9" }),
|
||||
).rejects.toMatchObject({ reason: "integrity" });
|
||||
await writeFile(path, FIT_EMBEDDED);
|
||||
await expect(reader.open(entry)).rejects.toMatchObject({
|
||||
reason: "integrity",
|
||||
});
|
||||
} finally {
|
||||
await reader.close();
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("settles an active image stream when closed", async () => {
|
||||
const reader = new FitArtifactReader(await writeFit("close.itb", LARGE_FIT));
|
||||
try {
|
||||
const stream = await reader.open(await firstEntry(reader));
|
||||
await reader.close();
|
||||
expect(stream.destroyed).toBe(true);
|
||||
await expect(
|
||||
reader.entries()[Symbol.asyncIterator]().next(),
|
||||
).rejects.toMatchObject({ reason: "unavailable" });
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps the cancellation reason before admission", async () => {
|
||||
const reader = new FitArtifactReader(
|
||||
await writeFit("cancel.itb", FIT_EMBEDDED),
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const reason = { stage: "before admission" };
|
||||
controller.abort(reason);
|
||||
try {
|
||||
await expect(
|
||||
reader.entries(controller.signal)[Symbol.asyncIterator]().next(),
|
||||
).rejects.toMatchObject({ reason: "cancelled", cause: reason });
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps the cancellation reason during hash verification", async () => {
|
||||
const reader = new FitArtifactReader(
|
||||
await writeFit("hashing.itb", LARGE_FIT),
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const reason = { stage: "hashing" };
|
||||
const create = fitStructure.createFitDigest;
|
||||
const observer = vi
|
||||
.spyOn(fitStructure, "createFitDigest")
|
||||
.mockImplementation((algo) => {
|
||||
const digest = create(algo);
|
||||
return (
|
||||
digest && {
|
||||
update: (chunk) => {
|
||||
controller.abort(reason);
|
||||
digest.update(chunk);
|
||||
},
|
||||
digest: () => digest.digest(),
|
||||
}
|
||||
);
|
||||
});
|
||||
try {
|
||||
await expect(
|
||||
reader.entries(controller.signal)[Symbol.asyncIterator]().next(),
|
||||
).rejects.toMatchObject({ reason: "cancelled", cause: reason });
|
||||
expect(observer).toHaveBeenCalledWith("sha256");
|
||||
} finally {
|
||||
observer.mockRestore();
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps the cancellation reason after an image stream was opened", async () => {
|
||||
const reader = new FitArtifactReader(await writeFit("stream.itb", LARGE_FIT));
|
||||
const controller = new AbortController();
|
||||
const reason = { stage: "image consumption" };
|
||||
try {
|
||||
const stream = await reader.open(
|
||||
await firstEntry(reader),
|
||||
controller.signal,
|
||||
);
|
||||
controller.abort(reason);
|
||||
await expect(
|
||||
(async () => {
|
||||
for await (const _chunk of stream) {
|
||||
/* drain */
|
||||
}
|
||||
})(),
|
||||
).rejects.toMatchObject({ reason: "cancelled", cause: reason });
|
||||
await reader.close();
|
||||
expect(stream.closed).toBe(true);
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it.each([null, { stage: "buffered image" }])(
|
||||
"keeps cancellation attached to buffered image bytes (reason %j)",
|
||||
async (reason) => {
|
||||
const reader = new FitArtifactReader(
|
||||
await writeFit("buffered.itb", FIT_EXTERNAL),
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const readChunks = fileChunks.readFileHandleChunks;
|
||||
let sourceClosed: Promise<unknown> | undefined;
|
||||
let observer: ReturnType<typeof vi.spyOn> | undefined;
|
||||
try {
|
||||
const entry = await firstEntry(reader);
|
||||
observer = vi
|
||||
.spyOn(fileChunks, "readFileHandleChunks")
|
||||
.mockImplementation((...args) => {
|
||||
const source = readChunks(...args);
|
||||
sourceClosed = once(source, "close");
|
||||
return source;
|
||||
});
|
||||
const stream = await reader.open(entry, controller.signal);
|
||||
await once(stream, "finish");
|
||||
expect(sourceClosed).toBeDefined();
|
||||
await sourceClosed;
|
||||
expect(stream.readableLength).toBe(FIT_KERNEL.length);
|
||||
expect(stream.closed).toBe(false);
|
||||
controller.abort(reason);
|
||||
await expect(
|
||||
(async () => {
|
||||
for await (const _chunk of stream) {
|
||||
/* drain */
|
||||
}
|
||||
})(),
|
||||
).rejects.toMatchObject({ reason: "cancelled", cause: reason });
|
||||
await reader.close();
|
||||
expect(stream.closed).toBe(true);
|
||||
expect(getEventListeners(controller.signal, "abort")).toEqual([]);
|
||||
} finally {
|
||||
observer?.mockRestore();
|
||||
await reader.close();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves a final validation failure when cancellation overlaps it", async () => {
|
||||
const reader = new FitArtifactReader(
|
||||
await writeFit("final-validation.itb", FIT_EXTERNAL),
|
||||
);
|
||||
const controller = new AbortController();
|
||||
let acknowledge = (): void => undefined;
|
||||
let resume = (): void => undefined;
|
||||
const entered = new Promise<void>((resolve) => {
|
||||
acknowledge = resolve;
|
||||
});
|
||||
const released = new Promise<void>((resolve) => {
|
||||
resume = resolve;
|
||||
});
|
||||
const failure = new ArtifactReaderFailure("integrity", "carrier changed");
|
||||
const read = ArtifactFileSource.prototype.read;
|
||||
const readChunks = fileChunks.readFileHandleChunks;
|
||||
let owner: ArtifactFileSource | undefined;
|
||||
let imageEnded = false;
|
||||
let observer: ReturnType<typeof vi.spyOn> | undefined;
|
||||
let chunksObserver: ReturnType<typeof vi.spyOn> | undefined;
|
||||
let consumed: Promise<void> | undefined;
|
||||
try {
|
||||
const entry = await firstEntry(reader);
|
||||
observer = vi
|
||||
.spyOn(ArtifactFileSource.prototype, "read")
|
||||
.mockImplementation(async function (this: ArtifactFileSource, signal) {
|
||||
// Only this reader's open/validation receives this caller signal.
|
||||
if (signal === controller.signal) owner ??= this;
|
||||
if (this === owner && imageEnded) {
|
||||
acknowledge();
|
||||
await released;
|
||||
throw failure;
|
||||
}
|
||||
return read.call(this, signal);
|
||||
});
|
||||
chunksObserver = vi
|
||||
.spyOn(fileChunks, "readFileHandleChunks")
|
||||
.mockImplementation((handle, options) => {
|
||||
const source = readChunks(handle, options);
|
||||
if (options?.signal === controller.signal)
|
||||
source.once("end", () => {
|
||||
imageEnded = true;
|
||||
});
|
||||
return source;
|
||||
});
|
||||
const stream = await reader.open(entry, controller.signal);
|
||||
consumed = (async () => {
|
||||
for await (const _chunk of stream) {
|
||||
/* drain */
|
||||
}
|
||||
})();
|
||||
await awaitAcknowledgement(entered, consumed, "FIT final validation");
|
||||
controller.abort({ stage: "final validation" });
|
||||
resume();
|
||||
await expect(consumed).rejects.toBe(failure);
|
||||
await reader.close();
|
||||
expect(stream.closed).toBe(true);
|
||||
expect(getEventListeners(controller.signal, "abort")).toEqual([]);
|
||||
} finally {
|
||||
resume();
|
||||
await consumed?.catch(() => undefined);
|
||||
chunksObserver?.mockRestore();
|
||||
observer?.mockRestore();
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,305 @@
|
||||
import { Readable, Transform, pipeline } from "node:stream";
|
||||
import { setImmediate } from "node:timers/promises";
|
||||
|
||||
import type { FitMetadata } from "../domain/artifactFormatMetadata.js";
|
||||
import { MAX_DEVICETREE_FILE_BYTES } from "../domain/firmware/devicetree.js";
|
||||
import {
|
||||
ArtifactReaderFailure,
|
||||
copyArtifactEntry,
|
||||
sameArtifactEntry,
|
||||
type ArtifactEntry,
|
||||
type ArtifactReader,
|
||||
type ArtifactReaderTraversal,
|
||||
type ArtifactTraversalOptions,
|
||||
} from "./ArtifactReader.js";
|
||||
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
|
||||
import { assertArtifactReadActive } from "./ArtifactFileOperation.js";
|
||||
import { ArtifactFileSource } from "./ArtifactFileSource.js";
|
||||
import {
|
||||
createFitDigest,
|
||||
readFitStructure,
|
||||
type FitImageDeclaration,
|
||||
} from "./fit/FitStructure.js";
|
||||
import { readFileHandleChunks } from "../filesystem/readFileHandleChunks.js";
|
||||
import { readFileHandleRange } from "../filesystem/readFileHandleRange.js";
|
||||
import type { StableRegularFileDescriptor } from "../filesystem/RegularFile.js";
|
||||
|
||||
const FDT_HEADER_BYTES = 40;
|
||||
|
||||
/** Inventory U-Boot FIT /images nodes as stored byte ranges without loading them. */
|
||||
export class FitArtifactReader implements ArtifactReader {
|
||||
readonly format = "fit" as const;
|
||||
readonly #scope = new ArtifactResourceScope();
|
||||
readonly #entries = new Map<string, ArtifactEntry>();
|
||||
readonly #streams = new Set<Readable>();
|
||||
readonly #path: string;
|
||||
readonly #source: ArtifactFileSource;
|
||||
#prepared: Promise<void> | undefined;
|
||||
#metadata: FitMetadata | undefined;
|
||||
#coverage: ArtifactReaderTraversal | undefined;
|
||||
#closing = false;
|
||||
|
||||
constructor(path: string, admitted?: StableRegularFileDescriptor) {
|
||||
this.#path = path;
|
||||
this.#source = new ArtifactFileSource(path, "FIT", admitted);
|
||||
}
|
||||
|
||||
formatMetadata(): FitMetadata | undefined {
|
||||
return this.#metadata === undefined
|
||||
? undefined
|
||||
: structuredClone(this.#metadata);
|
||||
}
|
||||
|
||||
async prepareContainer(
|
||||
expectedSha256: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
await this.#scope.run(async () => {
|
||||
this.#assertOpen(signal);
|
||||
await this.#source.bindSha256(expectedSha256, signal);
|
||||
});
|
||||
}
|
||||
|
||||
async *entries(
|
||||
signal?: AbortSignal,
|
||||
options?: ArtifactTraversalOptions,
|
||||
): AsyncIterable<ArtifactEntry> {
|
||||
await this.#scope.run(async () => {
|
||||
this.#assertOpen(signal);
|
||||
this.#prepared ??= this.#prepare(signal);
|
||||
await this.#prepared;
|
||||
await this.#source.read(signal);
|
||||
});
|
||||
if (this.#coverage !== undefined)
|
||||
options?.onCoverage?.({
|
||||
...this.#coverage,
|
||||
limitations: [...this.#coverage.limitations],
|
||||
});
|
||||
let index = 0;
|
||||
for (const entry of this.#entries.values()) {
|
||||
if (index % 256 === 0) await setImmediate();
|
||||
index += 1;
|
||||
this.#assertOpen(signal);
|
||||
yield copyArtifactEntry(entry);
|
||||
}
|
||||
}
|
||||
|
||||
async open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable> {
|
||||
return this.#scope.run(async () => {
|
||||
this.#assertOpen(signal);
|
||||
const stored = this.#entries.get(entry.adapterKey);
|
||||
if (stored === undefined || !sameArtifactEntry(entry, stored))
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
"FIT entry does not match its owning inventory",
|
||||
);
|
||||
const { handle } = await this.#source.read(signal);
|
||||
this.#assertOpen(signal);
|
||||
if (stored.byteOffset === null || stored.declaredSize === null)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`FIT image has no byte range: ${stored.path}`,
|
||||
);
|
||||
const source =
|
||||
stored.declaredSize === 0
|
||||
? Readable.from([])
|
||||
: readFileHandleChunks(handle, {
|
||||
start: stored.byteOffset,
|
||||
end: stored.byteOffset + stored.declaredSize - 1,
|
||||
signal,
|
||||
cancellationFailure,
|
||||
});
|
||||
const output = this.#imageStream(stored, signal);
|
||||
output.on("error", () => undefined);
|
||||
this.#streams.add(output);
|
||||
let producerComplete = false;
|
||||
const onAbort = (): void => {
|
||||
// Active I/O and final validation own failure precedence. Buffered
|
||||
// readable bytes still belong to the caller after they finish.
|
||||
if (!producerComplete || output.destroyed) return;
|
||||
output.destroy(cancellationFailure(signal?.reason));
|
||||
};
|
||||
const retired = new Promise<void>((resolve) => {
|
||||
output.once("close", () => {
|
||||
this.#streams.delete(output);
|
||||
signal?.removeEventListener("abort", onAbort);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
signal?.addEventListener("abort", onAbort, { once: true });
|
||||
if (signal?.aborted) onAbort();
|
||||
void this.#scope
|
||||
.run(async () => {
|
||||
// Retain both producer work and the unconsumed readable stream.
|
||||
await Promise.all([
|
||||
new Promise<void>((resolve) =>
|
||||
pipeline(source, output, () => {
|
||||
producerComplete = true;
|
||||
if (signal?.aborted) onAbort();
|
||||
resolve();
|
||||
}),
|
||||
),
|
||||
retired,
|
||||
]);
|
||||
})
|
||||
.catch((cause: unknown) =>
|
||||
output.destroy(
|
||||
cause instanceof Error
|
||||
? cause
|
||||
: new Error("FIT stream ownership failed", { cause }),
|
||||
),
|
||||
);
|
||||
return output;
|
||||
});
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
this.#closing = true;
|
||||
for (const stream of this.#streams)
|
||||
stream.destroy(
|
||||
new ArtifactReaderFailure("unavailable", "FIT reader closed"),
|
||||
);
|
||||
await this.#scope.close();
|
||||
await this.#source.close();
|
||||
this.#entries.clear();
|
||||
}
|
||||
|
||||
provenance(): readonly [] {
|
||||
return [];
|
||||
}
|
||||
|
||||
#assertOpen(signal?: AbortSignal): void {
|
||||
assertArtifactReadActive(signal);
|
||||
if (this.#closing)
|
||||
throw new ArtifactReaderFailure("unavailable", "FIT reader is closed");
|
||||
}
|
||||
|
||||
async #prepare(signal?: AbortSignal): Promise<void> {
|
||||
const { handle, initial } = await this.#source.read(signal);
|
||||
const assertActive = (): void => this.#assertOpen(signal);
|
||||
const header = await readFileHandleRange(
|
||||
handle,
|
||||
0,
|
||||
FDT_HEADER_BYTES,
|
||||
assertActive,
|
||||
);
|
||||
if (header.length < FDT_HEADER_BYTES)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"FIT devicetree header is truncated",
|
||||
);
|
||||
const totalsize = header.readUInt32BE(4);
|
||||
if (totalsize > initial.size)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`FIT devicetree is truncated: totalsize ${String(totalsize)} exceeds the ${String(initial.size)}-byte file`,
|
||||
);
|
||||
if (totalsize > MAX_DEVICETREE_FILE_BYTES)
|
||||
throw new ArtifactReaderFailure(
|
||||
"limit",
|
||||
`FIT devicetree is ${String(totalsize)} bytes, above the fixed 16 MiB devicetree allowance; embedded image data this large is not supported (mkimage -E stores data externally). No runtime override.`,
|
||||
{ boundary: "memory" },
|
||||
);
|
||||
// Only the devicetree is read; external image data stays on disk.
|
||||
const structure = readFitStructure(
|
||||
await readFileHandleRange(handle, 0, totalsize, assertActive),
|
||||
this.#path,
|
||||
initial.size,
|
||||
);
|
||||
let index = 0;
|
||||
for (const image of structure.images) {
|
||||
if (index % 256 === 0) await setImmediate();
|
||||
index += 1;
|
||||
this.#assertOpen(signal);
|
||||
await this.#verifyHashes(image, signal);
|
||||
const entry = {
|
||||
path: `images/${image.name}.bin`,
|
||||
sourcePath: `/images/${image.name}`,
|
||||
kind: "file" as const,
|
||||
declaredSize: image.size,
|
||||
compressedSize: null,
|
||||
executable: false,
|
||||
encrypted: false,
|
||||
byteOffset: image.start,
|
||||
declaredIntegrity: null,
|
||||
unpacked: false,
|
||||
limitations: image.limitations,
|
||||
adapterKey: `fit:${String(image.index)}`,
|
||||
formatMetadata: image.metadata,
|
||||
} satisfies ArtifactEntry;
|
||||
this.#entries.set(entry.adapterKey, entry);
|
||||
}
|
||||
this.#metadata = structure.metadata;
|
||||
if (structure.limitations.length > 0)
|
||||
this.#coverage = {
|
||||
status: "complete",
|
||||
limitations: structure.limitations,
|
||||
};
|
||||
await this.#source.read(signal);
|
||||
}
|
||||
|
||||
/** One streamed pass per image feeds every supported digest. */
|
||||
async #verifyHashes(
|
||||
image: FitImageDeclaration,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
const digests = image.hashes.flatMap((hash) => {
|
||||
const digest = createFitDigest(hash.algo);
|
||||
return digest === undefined ? [] : [{ hash, digest }];
|
||||
});
|
||||
if (digests.length === 0) return;
|
||||
const { handle } = await this.#source.read(signal);
|
||||
if (image.size > 0)
|
||||
for await (const chunk of readFileHandleChunks(handle, {
|
||||
start: image.start,
|
||||
end: image.start + image.size - 1,
|
||||
signal,
|
||||
cancellationFailure,
|
||||
}) as AsyncIterable<Buffer>) {
|
||||
this.#assertOpen(signal);
|
||||
for (const { digest } of digests) digest.update(chunk);
|
||||
}
|
||||
for (const { hash, digest } of digests) {
|
||||
const computed = digest.digest();
|
||||
if (!computed.equals(hash.value))
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`FIT image ${image.name} ${hash.node} ${hash.algo} does not match: declared ${hash.value.toString("hex")}, computed ${computed.toString("hex")}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#imageStream(entry: ArtifactEntry, signal?: AbortSignal): Transform {
|
||||
let bytes = 0;
|
||||
const check = async (): Promise<void> => {
|
||||
if (bytes !== entry.declaredSize)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`FIT image ended early: ${entry.path}`,
|
||||
);
|
||||
await this.#source.read(signal);
|
||||
};
|
||||
return new Transform({
|
||||
transform(chunk: Buffer, _encoding, callback) {
|
||||
bytes += chunk.length;
|
||||
callback(null, chunk);
|
||||
},
|
||||
flush(callback) {
|
||||
void check().then(
|
||||
() => callback(),
|
||||
(cause: unknown) =>
|
||||
callback(
|
||||
cause instanceof Error
|
||||
? cause
|
||||
: new Error("FIT read failed", { cause }),
|
||||
),
|
||||
);
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const cancellationFailure = (reason: unknown): ArtifactReaderFailure =>
|
||||
new ArtifactReaderFailure("cancelled", "FIT read cancelled", {
|
||||
cause: reason,
|
||||
});
|
||||
@@ -7,6 +7,7 @@ import type { ArtifactReader } from "./ArtifactReader.js";
|
||||
import { HexRecordArtifactReader } from "./HexRecordArtifactReader.js";
|
||||
import { DirectoryArtifactReader } from "./DirectoryArtifactReader.js";
|
||||
import { EspImageArtifactReader } from "./EspImageArtifactReader.js";
|
||||
import { FitArtifactReader } from "./FitArtifactReader.js";
|
||||
import { GodotPckArtifactReader } from "./GodotPckArtifactReader.js";
|
||||
import { MachOSliceArtifactReader } from "./MachOSliceArtifactReader.js";
|
||||
import { NativeDmgArtifactReader } from "./NativeDmgArtifactReader.js";
|
||||
@@ -33,6 +34,7 @@ export type ArtifactReaderFormat =
|
||||
| "uimage"
|
||||
| "intel-hex"
|
||||
| "srec"
|
||||
| "fit"
|
||||
| "mach-o-universal"
|
||||
| "dmg"
|
||||
| "esp-image"
|
||||
@@ -121,6 +123,8 @@ export function createArtifactReader(
|
||||
case "intel-hex":
|
||||
case "srec":
|
||||
return new HexRecordArtifactReader(path, format, rootSource);
|
||||
case "fit":
|
||||
return new FitArtifactReader(path, rootSource);
|
||||
case "mach-o-universal":
|
||||
return new MachOSliceArtifactReader(
|
||||
path,
|
||||
|
||||
@@ -591,6 +591,7 @@ const requireExtractableFormat = async (
|
||||
format !== "intel-hex" &&
|
||||
format !== "srec" &&
|
||||
format !== "esp-image" &&
|
||||
format !== "fit" &&
|
||||
format !== "mach-o-universal" &&
|
||||
!isZipPackageFormat(format)
|
||||
) {
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { crc32 } from "node:zlib";
|
||||
|
||||
import {
|
||||
cells,
|
||||
devicetreeFixture,
|
||||
dtString,
|
||||
type FixtureNode,
|
||||
} from "../../firmware/devicetree/Devicetree.fixture.js";
|
||||
|
||||
/** One /images node; `hashes` names algorithms whose values the fixture computes. */
|
||||
export interface FitFixtureImage {
|
||||
name: string;
|
||||
data: Buffer;
|
||||
properties?: Record<string, Buffer>;
|
||||
hashes?: string[];
|
||||
children?: FixtureNode[];
|
||||
}
|
||||
|
||||
const align = (value: number, boundary: number): number =>
|
||||
Math.ceil(value / boundary) * boundary;
|
||||
|
||||
const hashValue = (algo: string, data: Buffer): Buffer =>
|
||||
algo === "crc32"
|
||||
? cells(crc32(data))
|
||||
: createHash(algo).update(data).digest();
|
||||
|
||||
/**
|
||||
* Synthetic FIT writer. External modes place each image at a multiple of
|
||||
* `align` in a data store that starts at ALIGN(totalsize, 4), as mkimage -E
|
||||
* does; position mode records absolute file offsets (mkimage -p).
|
||||
*/
|
||||
export const fitFixture = ({
|
||||
images,
|
||||
configurations,
|
||||
mode = "embedded",
|
||||
align: boundary = 4,
|
||||
version = 17,
|
||||
}: {
|
||||
images: FitFixtureImage[];
|
||||
configurations?: FixtureNode;
|
||||
mode?: "embedded" | "offset" | "position";
|
||||
align?: number;
|
||||
version?: number;
|
||||
}): Buffer => {
|
||||
const offsets: number[] = [];
|
||||
let storeBytes = 0;
|
||||
for (const image of images) {
|
||||
storeBytes = align(storeBytes, boundary);
|
||||
offsets.push(storeBytes);
|
||||
storeBytes += image.data.length;
|
||||
}
|
||||
const build = (base: number): Buffer =>
|
||||
devicetreeFixture(
|
||||
{
|
||||
name: "",
|
||||
properties: {
|
||||
description: dtString("Test FIT"),
|
||||
timestamp: cells(1_700_000_000),
|
||||
"#address-cells": cells(1),
|
||||
},
|
||||
children: [
|
||||
{
|
||||
name: "images",
|
||||
children: images.map((image, index) => ({
|
||||
name: image.name,
|
||||
// Explicit properties override the generated data properties.
|
||||
properties: {
|
||||
...(mode === "embedded"
|
||||
? { data: image.data }
|
||||
: {
|
||||
[mode === "offset" ? "data-offset" : "data-position"]:
|
||||
cells(base + (offsets[index] ?? 0)),
|
||||
"data-size": cells(image.data.length),
|
||||
}),
|
||||
...image.properties,
|
||||
},
|
||||
children: [
|
||||
...(image.hashes ?? []).map((algo, hash) => ({
|
||||
name: `hash-${String(hash + 1)}`,
|
||||
properties: {
|
||||
algo: dtString(algo),
|
||||
value: hashValue(algo, image.data),
|
||||
},
|
||||
})),
|
||||
...(image.children ?? []),
|
||||
],
|
||||
})),
|
||||
},
|
||||
...(configurations === undefined ? [] : [configurations]),
|
||||
],
|
||||
},
|
||||
version,
|
||||
);
|
||||
if (mode === "embedded") return build(0);
|
||||
// Cell values do not change the blob size, so a placeholder build fixes it.
|
||||
const store = align(build(0).length, 4);
|
||||
const fdt = build(mode === "position" ? store : 0);
|
||||
const file = Buffer.alloc(store + storeBytes);
|
||||
fdt.copy(file);
|
||||
images.forEach((image, index) =>
|
||||
image.data.copy(file, store + (offsets[index] ?? 0)),
|
||||
);
|
||||
return file;
|
||||
};
|
||||
|
||||
export const FIT_KERNEL = Buffer.from(
|
||||
Array.from({ length: 1000 }, (_, index) => (index * 7 + 3) % 251),
|
||||
);
|
||||
export const FIT_DEVICETREE = devicetreeFixture();
|
||||
|
||||
const FIT_IMAGES: FitFixtureImage[] = [
|
||||
{
|
||||
name: "kernel",
|
||||
data: FIT_KERNEL,
|
||||
properties: {
|
||||
description: dtString("Linux kernel"),
|
||||
type: dtString("kernel"),
|
||||
arch: dtString("arm64"),
|
||||
os: dtString("linux"),
|
||||
compression: dtString("none"),
|
||||
load: cells(0x4020_0000),
|
||||
entry: cells(0, 0x4020_0000),
|
||||
},
|
||||
hashes: ["crc32", "md5", "sha1", "sha256", "sha384", "sha512"],
|
||||
},
|
||||
{
|
||||
name: "fdt-1",
|
||||
data: FIT_DEVICETREE,
|
||||
properties: {
|
||||
type: dtString("flat_dt"),
|
||||
arch: dtString("arm64"),
|
||||
compression: dtString("none"),
|
||||
},
|
||||
hashes: ["sha256"],
|
||||
},
|
||||
];
|
||||
|
||||
const FIT_CONFIGURATIONS: FixtureNode = {
|
||||
name: "configurations",
|
||||
properties: { default: dtString("conf-1") },
|
||||
children: [
|
||||
{
|
||||
name: "conf-1",
|
||||
properties: {
|
||||
description: dtString("Boot Linux"),
|
||||
kernel: dtString("kernel"),
|
||||
fdt: dtString("fdt-1"),
|
||||
},
|
||||
children: [
|
||||
{
|
||||
name: "signature-1",
|
||||
properties: { algo: dtString("sha256,rsa2048") },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
export const FIT_EMBEDDED = fitFixture({
|
||||
images: FIT_IMAGES,
|
||||
configurations: FIT_CONFIGURATIONS,
|
||||
});
|
||||
export const FIT_EXTERNAL = fitFixture({
|
||||
images: FIT_IMAGES,
|
||||
configurations: FIT_CONFIGURATIONS,
|
||||
mode: "offset",
|
||||
});
|
||||
@@ -0,0 +1,615 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
cells,
|
||||
devicetreeFixture,
|
||||
dtString,
|
||||
type FixtureNode,
|
||||
} from "../../firmware/devicetree/Devicetree.fixture.js";
|
||||
import {
|
||||
FIT_EMBEDDED,
|
||||
FIT_KERNEL,
|
||||
fitFixture,
|
||||
type FitFixtureImage,
|
||||
} from "./FitImage.fixture.js";
|
||||
import {
|
||||
FIT_MAX_CONFIGURATION_REFERENCES,
|
||||
FIT_MAX_CONFIGURATIONS,
|
||||
FIT_MAX_HASH_NODES_PER_IMAGE,
|
||||
FIT_MAX_IMAGES,
|
||||
readFitStructure,
|
||||
} from "./FitStructure.js";
|
||||
|
||||
const read = (file: Buffer) =>
|
||||
readFitStructure(
|
||||
file.subarray(0, file.readUInt32BE(4)),
|
||||
"test.itb",
|
||||
file.length,
|
||||
);
|
||||
|
||||
const failureOf = (file: Buffer): unknown => {
|
||||
try {
|
||||
read(file);
|
||||
} catch (cause) {
|
||||
return cause;
|
||||
}
|
||||
throw new Error("Expected readFitStructure to fail");
|
||||
};
|
||||
|
||||
const imageBytes = (file: Buffer, index: number): Buffer => {
|
||||
const image = read(file).images[index];
|
||||
if (image === undefined) throw new Error("Missing image");
|
||||
return file.subarray(image.start, image.start + image.size);
|
||||
};
|
||||
|
||||
const kernel = (properties: Record<string, Buffer> = {}): FitFixtureImage => ({
|
||||
name: "kernel",
|
||||
data: FIT_KERNEL,
|
||||
properties,
|
||||
});
|
||||
|
||||
const shared = (properties: Record<string, Buffer>): FitFixtureImage => ({
|
||||
name: "shared",
|
||||
data: Buffer.alloc(0),
|
||||
properties,
|
||||
});
|
||||
|
||||
/** A FIT whose single image node carries exactly these properties. */
|
||||
const singleImage = (properties: Record<string, Buffer>): Buffer =>
|
||||
devicetreeFixture({
|
||||
name: "",
|
||||
children: [{ name: "images", children: [{ name: "kernel", properties }] }],
|
||||
});
|
||||
|
||||
describe("FIT image facts", () => {
|
||||
it("decodes embedded root and image facts with property ranges", () => {
|
||||
const structure = read(FIT_EMBEDDED);
|
||||
expect(structure.metadata).toMatchObject({
|
||||
kind: "fit",
|
||||
fdt_version: 17,
|
||||
fdt_totalsize: FIT_EMBEDDED.length,
|
||||
description: "Test FIT",
|
||||
timestamp: 1_700_000_000,
|
||||
address_cells: 1,
|
||||
});
|
||||
expect(structure.images[0]?.metadata).toEqual({
|
||||
kind: "fit-image",
|
||||
index: 0,
|
||||
name: "kernel",
|
||||
description: "Linux kernel",
|
||||
type: "kernel",
|
||||
arch: "arm64",
|
||||
os: "linux",
|
||||
compression: "none",
|
||||
load: "0x40200000",
|
||||
entry: "0x40200000",
|
||||
data_source: "embedded",
|
||||
declared_offset: null,
|
||||
image_data: null,
|
||||
cipher_present: false,
|
||||
cipher_algo: null,
|
||||
unreadable_fields: [],
|
||||
hash_algorithms: ["crc32", "md5", "sha1", "sha256", "sha384", "sha512"],
|
||||
hashes_verified: 6,
|
||||
hashes_unverified: 0,
|
||||
signature_count: 0,
|
||||
signatures_unreadable: 0,
|
||||
signature_algorithms: [],
|
||||
});
|
||||
expect(structure.images[0]?.limitations).toEqual([]);
|
||||
expect(Buffer.compare(imageBytes(FIT_EMBEDDED, 0), FIT_KERNEL)).toBe(0);
|
||||
});
|
||||
|
||||
it("bases data-offset at the aligned devicetree end and accepts gaps", () => {
|
||||
const file = fitFixture({
|
||||
images: [kernel(), { name: "second", data: Buffer.from("second") }],
|
||||
mode: "offset",
|
||||
align: 512,
|
||||
});
|
||||
const structure = read(file);
|
||||
const { fdt_totalsize: totalsize, external_data_base: base } =
|
||||
structure.metadata;
|
||||
expect(totalsize % 4).not.toBe(0);
|
||||
expect(base).toBe(Math.ceil(totalsize / 4) * 4);
|
||||
expect(
|
||||
structure.images.map((image) => [
|
||||
image.start,
|
||||
image.metadata.data_source,
|
||||
image.metadata.declared_offset,
|
||||
]),
|
||||
).toEqual([
|
||||
[base, "data-offset", 0],
|
||||
[base + 1024, "data-offset", 1024],
|
||||
]);
|
||||
expect(Buffer.compare(imageBytes(file, 0), FIT_KERNEL)).toBe(0);
|
||||
expect(imageBytes(file, 1).toString()).toBe("second");
|
||||
});
|
||||
|
||||
it("reads data-position as an absolute file offset", () => {
|
||||
const file = fitFixture({ images: [kernel()], mode: "position" });
|
||||
const [image] = read(file).images;
|
||||
expect(image?.metadata.data_source).toBe("data-position");
|
||||
expect(image?.start).toBe(image?.metadata.declared_offset);
|
||||
expect(Buffer.compare(imageBytes(file, 0), FIT_KERNEL)).toBe(0);
|
||||
});
|
||||
|
||||
it("reports malformed descriptive facts as limitations", () => {
|
||||
const [image] = read(
|
||||
fitFixture({
|
||||
images: [
|
||||
kernel({ load: cells(1, 2, 3), arch: Buffer.from([0xff, 0]) }),
|
||||
],
|
||||
}),
|
||||
).images;
|
||||
expect(image?.metadata).toMatchObject({ load: null, arch: null });
|
||||
expect(image?.limitations).toEqual([
|
||||
"/images/kernel: arch is not a NUL-terminated UTF-8 string",
|
||||
"/images/kernel: load is not a 1- or 2-cell address",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("FIT image data ranges", () => {
|
||||
it("accepts an identical range linked by image-data", () => {
|
||||
const [target, linked] = read(
|
||||
fitFixture({
|
||||
images: [
|
||||
kernel(),
|
||||
shared({
|
||||
"image-data": dtString("kernel"),
|
||||
"data-offset": cells(0),
|
||||
"data-size": cells(FIT_KERNEL.length),
|
||||
}),
|
||||
],
|
||||
mode: "offset",
|
||||
}),
|
||||
).images;
|
||||
expect(linked?.start).toBe(target?.start);
|
||||
expect(linked?.metadata.image_data).toBe("kernel");
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"an unlinked identical range",
|
||||
{ "data-offset": cells(0), "data-size": cells(1000) },
|
||||
"image data overlaps: /images/shared and /images/kernel",
|
||||
],
|
||||
[
|
||||
"a dangling link",
|
||||
{
|
||||
"image-data": dtString("missing"),
|
||||
"data-offset": cells(0),
|
||||
"data-size": cells(1000),
|
||||
},
|
||||
"image-data names no image: missing",
|
||||
],
|
||||
[
|
||||
"a mismatched link",
|
||||
{
|
||||
"image-data": dtString("kernel"),
|
||||
"data-offset": cells(0),
|
||||
"data-size": cells(999),
|
||||
},
|
||||
"data does not match its image-data target /images/kernel",
|
||||
],
|
||||
[
|
||||
"a malformed link",
|
||||
{
|
||||
"image-data": dtString("a\0b"),
|
||||
"data-offset": cells(0),
|
||||
"data-size": cells(1000),
|
||||
},
|
||||
"image-data is not one string",
|
||||
],
|
||||
[
|
||||
"a partial overlap",
|
||||
{ "data-offset": cells(500), "data-size": cells(400) },
|
||||
"image data overlaps: /images/shared and /images/kernel",
|
||||
],
|
||||
[
|
||||
"data past the end of the file",
|
||||
{ "data-offset": cells(900), "data-size": cells(200) },
|
||||
"lies outside the",
|
||||
],
|
||||
])("fails %s", (_name, properties, message) => {
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({ images: [kernel(), shared(properties)], mode: "offset" }),
|
||||
),
|
||||
).toMatchObject({
|
||||
reason: "format",
|
||||
message: expect.stringContaining(message),
|
||||
});
|
||||
});
|
||||
|
||||
it("fails a chained image-data link", () => {
|
||||
const link = (target: string) => ({
|
||||
"image-data": dtString(target),
|
||||
"data-offset": cells(0),
|
||||
"data-size": cells(FIT_KERNEL.length),
|
||||
});
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({
|
||||
images: [
|
||||
kernel(),
|
||||
shared(link("kernel")),
|
||||
{ ...shared(link("shared")), name: "chained" },
|
||||
],
|
||||
mode: "offset",
|
||||
}),
|
||||
),
|
||||
).toMatchObject({
|
||||
reason: "format",
|
||||
message:
|
||||
"FIT is malformed: /images/chained: image-data target /images/shared has its own image-data",
|
||||
});
|
||||
});
|
||||
|
||||
it("fails external data inside the devicetree", () => {
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({
|
||||
images: [
|
||||
kernel(),
|
||||
shared({ "data-position": cells(0), "data-size": cells(4) }),
|
||||
],
|
||||
mode: "position",
|
||||
}),
|
||||
),
|
||||
).toMatchObject({
|
||||
reason: "format",
|
||||
message:
|
||||
"FIT is malformed: image data overlaps: /images/shared and the FIT devicetree",
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"missing data",
|
||||
singleImage({ type: dtString("kernel") }),
|
||||
"/images/kernel has no data, data-offset or data-position",
|
||||
],
|
||||
[
|
||||
"data plus data-offset",
|
||||
fitFixture({ images: [kernel({ "data-offset": cells(0) })] }),
|
||||
"/images/kernel declares more than one data source",
|
||||
],
|
||||
[
|
||||
"data-offset without data-size",
|
||||
singleImage({ "data-offset": cells(0) }),
|
||||
"/images/kernel: data-offset requires data-size",
|
||||
],
|
||||
[
|
||||
"an 8-byte data-size",
|
||||
fitFixture({
|
||||
images: [kernel({ "data-size": cells(0, 1000) })],
|
||||
mode: "offset",
|
||||
}),
|
||||
"/images/kernel: data-size is not a single 32-bit cell",
|
||||
],
|
||||
])("fails %s", (_name, file, message) => {
|
||||
expect(failureOf(file)).toMatchObject({
|
||||
reason: "format",
|
||||
message: `FIT is malformed: ${message}`,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("FIT hashes", () => {
|
||||
it("leaves unsupported, algo-less and value-less hashes unverified", () => {
|
||||
const [image] = read(
|
||||
fitFixture({
|
||||
images: [
|
||||
{
|
||||
...kernel(),
|
||||
hashes: ["sha256"],
|
||||
children: [
|
||||
{
|
||||
name: "hash-2",
|
||||
properties: { algo: dtString("crc16-ccitt"), value: cells(0) },
|
||||
},
|
||||
{ name: "hash-3", properties: { value: cells(0) } },
|
||||
{ name: "hash-4", properties: { algo: dtString("sha1") } },
|
||||
{
|
||||
name: "signature-1",
|
||||
properties: { algo: dtString("sha256,rsa2048") },
|
||||
},
|
||||
{ name: "cipher", properties: { algo: dtString("aes256") } },
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
).images;
|
||||
expect(image?.hashes.map((hash) => [hash.node, hash.algo])).toEqual([
|
||||
["hash-1", "sha256"],
|
||||
]);
|
||||
expect(image?.metadata).toMatchObject({
|
||||
hash_algorithms: ["sha256", "crc16-ccitt", "sha1"],
|
||||
hashes_verified: 1,
|
||||
hashes_unverified: 3,
|
||||
signature_algorithms: ["sha256,rsa2048"],
|
||||
cipher_present: true,
|
||||
cipher_algo: "aes256",
|
||||
});
|
||||
expect(image?.limitations).toEqual([
|
||||
'hash-2 algorithm "crc16-ccitt" is not verified',
|
||||
"hash-3 has no NUL-terminated UTF-8 algo; it is not verified",
|
||||
"hash-4 has no value; it is not verified",
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails a supported hash whose value has the wrong length", () => {
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({
|
||||
images: [
|
||||
{
|
||||
...kernel(),
|
||||
children: [
|
||||
{
|
||||
name: "hash-1",
|
||||
properties: {
|
||||
algo: dtString("sha256"),
|
||||
value: Buffer.alloc(31),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
),
|
||||
).toMatchObject({
|
||||
reason: "integrity",
|
||||
message:
|
||||
"FIT image kernel hash-1 sha256 value is 31 bytes; sha256 values are 32 bytes",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("FIT configurations", () => {
|
||||
const images = ["kernel", "fdt-1", "overlay-1"].map((name) => ({
|
||||
name,
|
||||
data: Buffer.from(name),
|
||||
}));
|
||||
const configured = (configurations: FixtureNode) =>
|
||||
read(fitFixture({ images, configurations }));
|
||||
|
||||
it("reports default, references and signature algorithms", () => {
|
||||
const structure = configured({
|
||||
name: "configurations",
|
||||
properties: { default: dtString("conf-a") },
|
||||
children: [
|
||||
{
|
||||
name: "conf-a",
|
||||
properties: {
|
||||
description: dtString("Board A"),
|
||||
kernel: dtString("kernel"),
|
||||
fdt: dtString("fdt-1\0overlay-1"),
|
||||
loadables: dtString("kernel"),
|
||||
firmware: dtString("fdt-1"),
|
||||
},
|
||||
children: [
|
||||
{
|
||||
name: "signature-1",
|
||||
properties: { algo: dtString("sha256,rsa4096") },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(structure.metadata).toMatchObject({
|
||||
default_configuration: "conf-a",
|
||||
configurations: [
|
||||
{
|
||||
name: "conf-a",
|
||||
description: "Board A",
|
||||
kernel: ["kernel"],
|
||||
firmware: ["fdt-1"],
|
||||
fdt: ["fdt-1", "overlay-1"],
|
||||
ramdisk: [],
|
||||
loadables: ["kernel"],
|
||||
signature_algorithms: ["sha256,rsa4096"],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(structure.limitations).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports missing references and a missing default as limitations", () => {
|
||||
const structure = configured({
|
||||
name: "configurations",
|
||||
properties: { default: dtString("conf-x") },
|
||||
children: [{ name: "conf-a", properties: { kernel: dtString("gone") } }],
|
||||
});
|
||||
expect(structure.metadata.configurations[0]).toMatchObject({
|
||||
kernel: ["gone"],
|
||||
unresolved_reference_count: 1,
|
||||
});
|
||||
expect(structure.limitations).toHaveLength(2);
|
||||
expect(
|
||||
structure.limitations.some((value) =>
|
||||
value.includes("/configurations/conf-a"),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(
|
||||
structure.limitations.some((value) => value.includes("conf-x")),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("reports no configurations when the node is absent", () => {
|
||||
expect(read(fitFixture({ images })).metadata).toMatchObject({
|
||||
default_configuration: null,
|
||||
configurations: [],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("FIT fixed limits", () => {
|
||||
it("fails more images than the fixed allowance", () => {
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({
|
||||
images: Array.from({ length: FIT_MAX_IMAGES + 1 }, (_, index) => ({
|
||||
name: `image-${String(index)}`,
|
||||
data: Buffer.from([index & 0xff]),
|
||||
})),
|
||||
}),
|
||||
),
|
||||
).toMatchObject({ reason: "limit", boundary: "capacity" });
|
||||
});
|
||||
|
||||
it("fails more hash nodes than the fixed allowance", () => {
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({
|
||||
images: [
|
||||
{
|
||||
...kernel(),
|
||||
hashes: Array.from(
|
||||
{ length: FIT_MAX_HASH_NODES_PER_IMAGE + 1 },
|
||||
() => "crc32",
|
||||
),
|
||||
},
|
||||
],
|
||||
}),
|
||||
),
|
||||
).toMatchObject({ reason: "limit", boundary: "capacity" });
|
||||
});
|
||||
|
||||
it("fails more configurations than the fixed allowance", () => {
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({
|
||||
images: [kernel()],
|
||||
configurations: {
|
||||
name: "configurations",
|
||||
children: Array.from(
|
||||
{ length: FIT_MAX_CONFIGURATIONS + 1 },
|
||||
(_, index) => ({ name: `conf-${String(index)}` }),
|
||||
),
|
||||
},
|
||||
}),
|
||||
),
|
||||
).toMatchObject({ reason: "limit", boundary: "capacity" });
|
||||
});
|
||||
|
||||
it("bounds configuration references before decoding them", () => {
|
||||
expect(
|
||||
failureOf(
|
||||
fitFixture({
|
||||
images: [kernel()],
|
||||
configurations: {
|
||||
name: "configurations",
|
||||
children: [
|
||||
{
|
||||
name: "conf-1",
|
||||
properties: {
|
||||
loadables: Buffer.alloc(FIT_MAX_CONFIGURATION_REFERENCES + 1),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
}),
|
||||
),
|
||||
).toMatchObject({ reason: "limit", boundary: "capacity" });
|
||||
});
|
||||
|
||||
it("maps a parser limit to a fixed allowance without a caller limit", () => {
|
||||
let nested: FixtureNode = { name: "leaf" };
|
||||
for (let depth = 0; depth < 64; depth += 1)
|
||||
nested = { name: `n${String(depth)}`, children: [nested] };
|
||||
const failure = failureOf(
|
||||
devicetreeFixture({
|
||||
name: "",
|
||||
children: [{ name: "images" }, nested],
|
||||
}),
|
||||
);
|
||||
expect(failure).toMatchObject({ reason: "limit", boundary: "capacity" });
|
||||
// The parser's caller-limit recovery does not apply to fixed FIT reads.
|
||||
expect(String((failure as Error).message)).not.toContain("caller-selected");
|
||||
});
|
||||
|
||||
it("fails a structure without /images", () => {
|
||||
expect(failureOf(devicetreeFixture())).toMatchObject({
|
||||
reason: "format",
|
||||
message: "FIT is malformed: /images node is missing",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("bounds dangling-reference diagnostics while preserving every declared reference", () => {
|
||||
const name = "c".repeat(4096);
|
||||
const references = Array.from(
|
||||
{ length: FIT_MAX_CONFIGURATION_REFERENCES },
|
||||
() => "missing",
|
||||
);
|
||||
const file = fitFixture({
|
||||
images: [{ name: "kernel", data: Buffer.from("K") }],
|
||||
configurations: {
|
||||
name: "configurations",
|
||||
children: [
|
||||
{ name, properties: { loadables: dtString(references.join("\0")) } },
|
||||
],
|
||||
},
|
||||
});
|
||||
const structure = read(file);
|
||||
expect(structure.limitations.length).toBe(1);
|
||||
expect(structure.metadata.configurations[0]).toMatchObject({
|
||||
name,
|
||||
loadables: references,
|
||||
unresolved_reference_count: references.length,
|
||||
});
|
||||
expect(
|
||||
Buffer.byteLength(
|
||||
JSON.stringify({
|
||||
metadata: structure.metadata,
|
||||
limitations: structure.limitations,
|
||||
}),
|
||||
),
|
||||
).toBeLessThan(file.length * 4);
|
||||
});
|
||||
|
||||
it.each(["data-offset", "data-position"] as const)(
|
||||
"accepts a shared external range whose target uses %s",
|
||||
(targetKey) => {
|
||||
const linkedKey =
|
||||
targetKey === "data-offset" ? "data-position" : "data-offset";
|
||||
const data = Buffer.from("payload");
|
||||
const build = (start: number) =>
|
||||
devicetreeFixture({
|
||||
name: "",
|
||||
children: [
|
||||
{
|
||||
name: "images",
|
||||
children: [
|
||||
{
|
||||
name: "target",
|
||||
properties: {
|
||||
[targetKey]: cells(targetKey === "data-offset" ? 0 : start),
|
||||
"data-size": cells(data.length),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "linked",
|
||||
properties: {
|
||||
[linkedKey]: cells(linkedKey === "data-offset" ? 0 : start),
|
||||
"data-size": cells(data.length),
|
||||
"image-data": dtString("target"),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
const start = Math.ceil(build(0).length / 4) * 4;
|
||||
const fdt = build(start);
|
||||
const file = Buffer.concat([fdt, Buffer.alloc(start - fdt.length), data]);
|
||||
const [target, linked] = read(file).images;
|
||||
expect(target?.start).toBe(start);
|
||||
expect(linked?.start).toBe(start);
|
||||
expect(target?.metadata.data_source).toBe(targetKey);
|
||||
expect(linked?.metadata.data_source).toBe(linkedKey);
|
||||
expect(imageBytes(file, 1).equals(data)).toBe(true);
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,614 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { crc32 } from "node:zlib";
|
||||
|
||||
import { artifactExpansionBudgetBytes } from "../../domain/artifactExpansionBudget.js";
|
||||
import type {
|
||||
FitImageMetadata,
|
||||
FitMetadata,
|
||||
} from "../../domain/artifactFormatMetadata.js";
|
||||
import {
|
||||
MAX_DEVICETREE_FILE_BYTES,
|
||||
type Devicetree,
|
||||
} from "../../domain/firmware/devicetree.js";
|
||||
import {
|
||||
DevicetreeLimitFailure,
|
||||
parseDevicetree,
|
||||
} from "../../firmware/devicetree/DevicetreeParser.js";
|
||||
import { BinaryFormatFailure } from "../../native/metadata/BinaryContainer.js";
|
||||
import { ArtifactReaderFailure } from "../ArtifactReader.js";
|
||||
|
||||
// Fixed resource bounds; configurable ceilings are tracked in #6176.
|
||||
export const FIT_MAX_IMAGES = 1024;
|
||||
export const FIT_MAX_HASH_NODES_PER_IMAGE = 8;
|
||||
export const FIT_MAX_CONFIGURATIONS = 1024;
|
||||
/** Bounds configuration references and their limitations across the FIT. */
|
||||
export const FIT_MAX_CONFIGURATION_REFERENCES = 16_384;
|
||||
|
||||
type FitNode = Devicetree["nodes"][number];
|
||||
type FitProperty = FitNode["properties"][number];
|
||||
|
||||
export interface FitImageDeclaration {
|
||||
readonly index: number;
|
||||
readonly name: string;
|
||||
readonly start: number;
|
||||
readonly size: number;
|
||||
readonly metadata: FitImageMetadata;
|
||||
readonly limitations: string[];
|
||||
/** Supported algorithms with a value of the right length, in node order. */
|
||||
readonly hashes: { node: string; algo: string; value: Buffer }[];
|
||||
}
|
||||
|
||||
export interface FitStructure {
|
||||
readonly metadata: FitMetadata;
|
||||
readonly images: FitImageDeclaration[];
|
||||
readonly limitations: string[];
|
||||
}
|
||||
|
||||
interface FitLayout {
|
||||
readonly fdt: Buffer;
|
||||
readonly children: ReadonlyMap<string, readonly FitNode[]>;
|
||||
readonly totalsize: number;
|
||||
readonly fileSize: number;
|
||||
}
|
||||
|
||||
/** Lenient decoding records malformed descriptive facts here instead of failing. */
|
||||
interface FitFacts {
|
||||
readonly fdt: Buffer;
|
||||
readonly limitations: string[];
|
||||
}
|
||||
type FitFact<Value> = (facts: FitFacts, node: FitNode, name: string) => Value;
|
||||
|
||||
// U-Boot's hash value lengths; crc32 is stored as 4 bytes big-endian.
|
||||
const DIGEST_BYTES = new Map([
|
||||
["crc32", 4],
|
||||
["md5", 16],
|
||||
["sha1", 20],
|
||||
["sha256", 32],
|
||||
["sha384", 48],
|
||||
["sha512", 64],
|
||||
]);
|
||||
const DATA_SOURCES = ["data", "data-offset", "data-position"] as const;
|
||||
const CONFIGURATION_REFERENCES = [
|
||||
"kernel",
|
||||
"firmware",
|
||||
"fdt",
|
||||
"ramdisk",
|
||||
"loadables",
|
||||
] as const;
|
||||
|
||||
const align4 = (value: number): number => Math.ceil(value / 4) * 4;
|
||||
const formatFailure = (detail: string): ArtifactReaderFailure =>
|
||||
new ArtifactReaderFailure("format", `FIT is malformed: ${detail}`);
|
||||
const capacityFailure = (message: string): ArtifactReaderFailure =>
|
||||
new ArtifactReaderFailure("limit", message, { boundary: "capacity" });
|
||||
const nodeName = (node: FitNode): string =>
|
||||
node.path.slice(node.path.lastIndexOf("/") + 1);
|
||||
const propertyOf = (node: FitNode, name: string): FitProperty | undefined =>
|
||||
node.properties.find((property) => property.name === name);
|
||||
const valueOf = (fdt: Buffer, property: FitProperty): Buffer =>
|
||||
fdt.subarray(property.offset, property.offset + property.bytes);
|
||||
|
||||
const utf8 = new TextDecoder("utf-8", { fatal: true });
|
||||
const nulCount = (raw: Buffer): number => {
|
||||
let count = 0;
|
||||
for (const byte of raw) if (byte === 0) count += 1;
|
||||
return count;
|
||||
};
|
||||
|
||||
/**
|
||||
* NUL-terminated UTF-8 strings, or undefined when the value is malformed.
|
||||
* More than `maxStrings` strings is malformed, counted before splitting.
|
||||
*/
|
||||
const nulStrings = (
|
||||
fdt: Buffer,
|
||||
property: FitProperty | undefined,
|
||||
maxStrings = 1,
|
||||
): string[] | undefined => {
|
||||
if (property === undefined) return undefined;
|
||||
const raw = valueOf(fdt, property);
|
||||
if (raw.at(-1) !== 0 || nulCount(raw) > maxStrings) return undefined;
|
||||
try {
|
||||
return utf8.decode(raw.subarray(0, -1)).split("\0");
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
|
||||
/** Incremental digest of one verified FIT hash algorithm. */
|
||||
export const createFitDigest = (
|
||||
algo: string,
|
||||
): { update(chunk: Buffer): void; digest(): Buffer } | undefined => {
|
||||
if (!DIGEST_BYTES.has(algo)) return undefined;
|
||||
if (algo !== "crc32") {
|
||||
const hash = createHash(algo);
|
||||
return {
|
||||
update: (chunk) => {
|
||||
hash.update(chunk);
|
||||
},
|
||||
digest: () => hash.digest(),
|
||||
};
|
||||
}
|
||||
let value = 0;
|
||||
return {
|
||||
update: (chunk) => {
|
||||
value = crc32(chunk, value);
|
||||
},
|
||||
digest: () => Buffer.from(value.toString(16).padStart(8, "0"), "hex"),
|
||||
};
|
||||
};
|
||||
|
||||
const fitString: FitFact<string | null> = (facts, node, name) => {
|
||||
const property = propertyOf(node, name);
|
||||
if (property === undefined) return null;
|
||||
const strings = nulStrings(facts.fdt, property);
|
||||
if (strings?.length === 1) return strings[0] ?? null;
|
||||
facts.limitations.push(
|
||||
`${node.path}: ${name} is not a NUL-terminated UTF-8 string`,
|
||||
);
|
||||
return null;
|
||||
};
|
||||
|
||||
const fitStrings: FitFact<string[]> = (facts, node, name) => {
|
||||
const property = propertyOf(node, name);
|
||||
if (property === undefined) return [];
|
||||
// fitConfigurations bounds every reference list before decoding.
|
||||
const strings = nulStrings(
|
||||
facts.fdt,
|
||||
property,
|
||||
FIT_MAX_CONFIGURATION_REFERENCES,
|
||||
);
|
||||
if (strings !== undefined) return strings;
|
||||
facts.limitations.push(
|
||||
`${node.path}: ${name} is not a NUL-terminated UTF-8 string list`,
|
||||
);
|
||||
return [];
|
||||
};
|
||||
|
||||
const fitU32: FitFact<number | null> = (facts, node, name) => {
|
||||
const property = propertyOf(node, name);
|
||||
if (property === undefined) return null;
|
||||
if (property.bytes === 4) return facts.fdt.readUInt32BE(property.offset);
|
||||
facts.limitations.push(`${node.path}: ${name} is not a single 32-bit cell`);
|
||||
return null;
|
||||
};
|
||||
|
||||
/** Fold 32-bit big-endian cells by property length, as U-Boot does. */
|
||||
const fitAddress: FitFact<string | null> = (facts, node, name) => {
|
||||
const property = propertyOf(node, name);
|
||||
if (property === undefined) return null;
|
||||
if (property.bytes !== 4 && property.bytes !== 8) {
|
||||
facts.limitations.push(
|
||||
`${node.path}: ${name} is not a 1- or 2-cell address`,
|
||||
);
|
||||
return null;
|
||||
}
|
||||
let value = 0n;
|
||||
for (let offset = 0; offset < property.bytes; offset += 4)
|
||||
value =
|
||||
(value << 32n) | BigInt(facts.fdt.readUInt32BE(property.offset + offset));
|
||||
return `0x${value.toString(16)}`;
|
||||
};
|
||||
|
||||
/** Byte-locating cells fail closed: a guessed range would misattribute bytes. */
|
||||
const strictCell = (
|
||||
fdt: Buffer,
|
||||
node: FitNode,
|
||||
property: FitProperty,
|
||||
): number => {
|
||||
if (property.bytes !== 4)
|
||||
throw formatFailure(
|
||||
`${node.path}: ${property.name} is not a single 32-bit cell`,
|
||||
);
|
||||
return fdt.readUInt32BE(property.offset);
|
||||
};
|
||||
|
||||
const imageRange = (
|
||||
layout: FitLayout,
|
||||
node: FitNode,
|
||||
): Pick<FitImageDeclaration, "start" | "size"> &
|
||||
Pick<FitImageMetadata, "data_source" | "declared_offset"> => {
|
||||
const declared = DATA_SOURCES.flatMap((name) => {
|
||||
const property = propertyOf(node, name);
|
||||
return property === undefined ? [] : [{ name, property }];
|
||||
});
|
||||
const [source] = declared;
|
||||
if (source === undefined)
|
||||
throw formatFailure(
|
||||
`${node.path} has no data, data-offset or data-position`,
|
||||
);
|
||||
if (declared.length > 1)
|
||||
throw formatFailure(`${node.path} declares more than one data source`);
|
||||
if (source.name === "data")
|
||||
return {
|
||||
start: source.property.offset,
|
||||
size: source.property.bytes,
|
||||
data_source: "embedded",
|
||||
declared_offset: null,
|
||||
};
|
||||
const declaredOffset = strictCell(layout.fdt, node, source.property);
|
||||
const sizeProperty = propertyOf(node, "data-size");
|
||||
if (sizeProperty === undefined)
|
||||
throw formatFailure(`${node.path}: ${source.name} requires data-size`);
|
||||
const size = strictCell(layout.fdt, node, sizeProperty);
|
||||
// data-offset is relative to the 4-byte-aligned end of the devicetree;
|
||||
// data-position is an offset from the start of the FIT (fit + position).
|
||||
const start =
|
||||
source.name === "data-offset"
|
||||
? align4(layout.totalsize) + declaredOffset
|
||||
: declaredOffset;
|
||||
if (start + size > layout.fileSize)
|
||||
throw formatFailure(
|
||||
`${node.path}: data at [${String(start)}, ${String(start + size)}) lies outside the ${String(layout.fileSize)}-byte file`,
|
||||
);
|
||||
return {
|
||||
start,
|
||||
size,
|
||||
data_source: source.name,
|
||||
declared_offset: declaredOffset,
|
||||
};
|
||||
};
|
||||
|
||||
const imageHashes = (
|
||||
facts: FitFacts,
|
||||
image: FitNode,
|
||||
children: readonly FitNode[],
|
||||
) => {
|
||||
const nodes = children.filter((child) => nodeName(child).startsWith("hash"));
|
||||
if (nodes.length > FIT_MAX_HASH_NODES_PER_IMAGE)
|
||||
throw capacityFailure(
|
||||
`FIT image ${nodeName(image)} declares ${String(nodes.length)} hash nodes, above the fixed ${String(FIT_MAX_HASH_NODES_PER_IMAGE)}-node allowance`,
|
||||
);
|
||||
const hashes: FitImageDeclaration["hashes"] = [];
|
||||
const algorithms: string[] = [];
|
||||
let unverified = 0;
|
||||
for (const node of nodes) {
|
||||
const name = nodeName(node);
|
||||
const algos = nulStrings(facts.fdt, propertyOf(node, "algo"));
|
||||
const algo = algos?.length === 1 ? algos[0] : undefined;
|
||||
const value = propertyOf(node, "value");
|
||||
const expected = algo === undefined ? undefined : DIGEST_BYTES.get(algo);
|
||||
if (algo !== undefined) algorithms.push(algo);
|
||||
if (algo === undefined || expected === undefined || value === undefined) {
|
||||
unverified += 1;
|
||||
facts.limitations.push(
|
||||
algo === undefined
|
||||
? `${name} has no NUL-terminated UTF-8 algo; it is not verified`
|
||||
: expected === undefined
|
||||
? `${name} algorithm "${algo}" is not verified`
|
||||
: `${name} has no value; it is not verified`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (value.bytes !== expected)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`FIT image ${nodeName(image)} ${name} ${algo} value is ${String(value.bytes)} bytes; ${algo} values are ${String(expected)} bytes`,
|
||||
);
|
||||
// Copy the value so the devicetree buffer can be released.
|
||||
hashes.push({
|
||||
node: name,
|
||||
algo,
|
||||
value: Buffer.from(valueOf(facts.fdt, value)),
|
||||
});
|
||||
}
|
||||
return { hashes, algorithms, unverified };
|
||||
};
|
||||
|
||||
/** Signature declarations remain visible even when their algorithm is unreadable. */
|
||||
const signatureAlgorithms = (
|
||||
facts: FitFacts,
|
||||
children: readonly FitNode[],
|
||||
): { algorithms: string[]; count: number; unreadable: number } => {
|
||||
const algorithms: string[] = [];
|
||||
let count = 0;
|
||||
let unreadable = 0;
|
||||
for (const child of children) {
|
||||
if (!nodeName(child).startsWith("signature")) continue;
|
||||
count += 1;
|
||||
const algo = fitString(facts, child, "algo");
|
||||
if (algo !== null) algorithms.push(algo);
|
||||
else {
|
||||
unreadable += 1;
|
||||
if (propertyOf(child, "algo") === undefined)
|
||||
facts.limitations.push(`${child.path}: signature algo is missing`);
|
||||
}
|
||||
}
|
||||
return { algorithms, count, unreadable };
|
||||
};
|
||||
|
||||
const imageDeclaration = (
|
||||
layout: FitLayout,
|
||||
node: FitNode,
|
||||
index: number,
|
||||
): FitImageDeclaration => {
|
||||
const facts: FitFacts = { fdt: layout.fdt, limitations: [] };
|
||||
const name = nodeName(node);
|
||||
const children = layout.children.get(node.path) ?? [];
|
||||
const range = imageRange(layout, node);
|
||||
const imageDataProperty = propertyOf(node, "image-data");
|
||||
const imageData = nulStrings(layout.fdt, imageDataProperty);
|
||||
if (imageDataProperty !== undefined && imageData?.length !== 1)
|
||||
throw formatFailure(`${node.path}: image-data is not one string`);
|
||||
const cipher = children.find((child) => nodeName(child) === "cipher");
|
||||
const unreadable = new Set<FitImageMetadata["unreadable_fields"][number]>();
|
||||
const field = <Value>(
|
||||
key: FitImageMetadata["unreadable_fields"][number],
|
||||
reader: FitFact<Value | null>,
|
||||
source = node,
|
||||
property: string = key,
|
||||
): Value | null => {
|
||||
const value = reader(facts, source, property);
|
||||
if (value === null && propertyOf(source, property) !== undefined)
|
||||
unreadable.add(key);
|
||||
return value;
|
||||
};
|
||||
const metadata = {
|
||||
kind: "fit-image" as const,
|
||||
index,
|
||||
name,
|
||||
description: field("description", fitString),
|
||||
type: field("type", fitString),
|
||||
arch: field("arch", fitString),
|
||||
os: field("os", fitString),
|
||||
compression: field("compression", fitString),
|
||||
load: field("load", fitAddress),
|
||||
entry: field("entry", fitAddress),
|
||||
data_source: range.data_source,
|
||||
declared_offset: range.declared_offset,
|
||||
image_data: imageData?.[0] ?? null,
|
||||
cipher_present: cipher !== undefined,
|
||||
cipher_algo:
|
||||
cipher === undefined
|
||||
? null
|
||||
: field("cipher_algo", fitString, cipher, "algo"),
|
||||
};
|
||||
const { hashes, algorithms, unverified } = imageHashes(facts, node, children);
|
||||
const signatures = signatureAlgorithms(facts, children);
|
||||
return {
|
||||
index,
|
||||
name,
|
||||
start: range.start,
|
||||
size: range.size,
|
||||
limitations: facts.limitations,
|
||||
hashes,
|
||||
metadata: {
|
||||
...metadata,
|
||||
unreadable_fields: [...unreadable].sort(),
|
||||
hash_algorithms: algorithms,
|
||||
hashes_verified: hashes.length,
|
||||
hashes_unverified: unverified,
|
||||
signature_count: signatures.count,
|
||||
signatures_unreadable: signatures.unreadable,
|
||||
signature_algorithms: signatures.algorithms,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
/** image-data must name an unchained image whose data properties agree. */
|
||||
const assertImageDataLinks = (
|
||||
images: readonly FitImageDeclaration[],
|
||||
fdt: Buffer,
|
||||
): void => {
|
||||
const byName = new Map(images.map((image) => [image.name, image]));
|
||||
for (const image of images) {
|
||||
const name = image.metadata.image_data;
|
||||
if (name === null) continue;
|
||||
const target = byName.get(name);
|
||||
const path = `/images/${image.name}`;
|
||||
if (target === undefined)
|
||||
throw formatFailure(`${path}: image-data names no image: ${name}`);
|
||||
if (target.metadata.image_data !== null)
|
||||
throw formatFailure(
|
||||
`${path}: image-data target /images/${name} has its own image-data`,
|
||||
);
|
||||
// Embedded copies occupy distinct properties but must contain equal bytes.
|
||||
if (
|
||||
(target.metadata.data_source === "embedded") !==
|
||||
(image.metadata.data_source === "embedded") ||
|
||||
target.size !== image.size ||
|
||||
(image.metadata.data_source === "embedded"
|
||||
? !fdt
|
||||
.subarray(image.start, image.start + image.size)
|
||||
.equals(fdt.subarray(target.start, target.start + target.size))
|
||||
: target.start !== image.start)
|
||||
)
|
||||
throw formatFailure(
|
||||
`${path}: data does not match its image-data target /images/${name}`,
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/** External ranges may not overlap the devicetree or each other, except linked shares. */
|
||||
const assertDisjoint = (
|
||||
images: readonly FitImageDeclaration[],
|
||||
totalsize: number,
|
||||
): void => {
|
||||
const owner = (image: FitImageDeclaration): string =>
|
||||
image.metadata.image_data ?? image.name;
|
||||
const ranges = images
|
||||
.filter(
|
||||
(image) => image.metadata.data_source !== "embedded" && image.size > 0,
|
||||
)
|
||||
.sort((a, b) => a.start - b.start || a.index - b.index);
|
||||
let previous: FitImageDeclaration | undefined;
|
||||
let end = totalsize;
|
||||
for (const image of ranges) {
|
||||
if (image.start < end) {
|
||||
if (
|
||||
previous !== undefined &&
|
||||
previous.start === image.start &&
|
||||
previous.size === image.size &&
|
||||
owner(previous) === owner(image)
|
||||
)
|
||||
continue;
|
||||
throw formatFailure(
|
||||
`image data overlaps: /images/${image.name} and ${previous === undefined ? "the FIT devicetree" : `/images/${previous.name}`}`,
|
||||
);
|
||||
}
|
||||
previous = image;
|
||||
end = image.start + image.size;
|
||||
}
|
||||
};
|
||||
|
||||
const fitConfiguration = (
|
||||
layout: FitLayout,
|
||||
facts: FitFacts,
|
||||
node: FitNode,
|
||||
images: ReadonlySet<string>,
|
||||
): FitMetadata["configurations"][number] => {
|
||||
const references = {
|
||||
kernel: fitStrings(facts, node, "kernel"),
|
||||
firmware: fitStrings(facts, node, "firmware"),
|
||||
fdt: fitStrings(facts, node, "fdt"),
|
||||
ramdisk: fitStrings(facts, node, "ramdisk"),
|
||||
loadables: fitStrings(facts, node, "loadables"),
|
||||
};
|
||||
let unresolved = 0;
|
||||
for (const values of Object.values(references))
|
||||
for (const reference of values) if (!images.has(reference)) unresolved += 1;
|
||||
if (unresolved > 0)
|
||||
facts.limitations.push(
|
||||
`${node.path}: ${String(unresolved)} image references do not name an /images node`,
|
||||
);
|
||||
const signatures = signatureAlgorithms(
|
||||
facts,
|
||||
layout.children.get(node.path) ?? [],
|
||||
);
|
||||
return {
|
||||
name: nodeName(node),
|
||||
description: fitString(facts, node, "description"),
|
||||
...references,
|
||||
unresolved_reference_count: unresolved,
|
||||
signature_count: signatures.count,
|
||||
signatures_unreadable: signatures.unreadable,
|
||||
signature_algorithms: signatures.algorithms,
|
||||
};
|
||||
};
|
||||
|
||||
const fitConfigurations = (
|
||||
layout: FitLayout,
|
||||
facts: FitFacts,
|
||||
node: FitNode | undefined,
|
||||
images: ReadonlySet<string>,
|
||||
): Pick<FitMetadata, "default_configuration" | "configurations"> => {
|
||||
if (node === undefined)
|
||||
return { default_configuration: null, configurations: [] };
|
||||
const children = layout.children.get(node.path) ?? [];
|
||||
if (children.length > FIT_MAX_CONFIGURATIONS)
|
||||
throw capacityFailure(
|
||||
`FIT declares ${String(children.length)} configurations, above the fixed ${String(FIT_MAX_CONFIGURATIONS)}-configuration allowance`,
|
||||
);
|
||||
let references = 0;
|
||||
for (const child of children)
|
||||
for (const name of CONFIGURATION_REFERENCES) {
|
||||
const property = propertyOf(child, name);
|
||||
if (property !== undefined)
|
||||
references += nulCount(valueOf(layout.fdt, property));
|
||||
}
|
||||
if (references > FIT_MAX_CONFIGURATION_REFERENCES)
|
||||
throw capacityFailure(
|
||||
`FIT configurations declare ${String(references)} image references, above the fixed ${String(FIT_MAX_CONFIGURATION_REFERENCES)}-reference allowance`,
|
||||
);
|
||||
const configurations = children.map((child) =>
|
||||
fitConfiguration(layout, facts, child, images),
|
||||
);
|
||||
const defaultConfiguration = fitString(facts, node, "default");
|
||||
if (
|
||||
defaultConfiguration !== null &&
|
||||
!configurations.some((entry) => entry.name === defaultConfiguration)
|
||||
)
|
||||
facts.limitations.push(
|
||||
`${node.path}: default ${defaultConfiguration} names no configuration`,
|
||||
);
|
||||
return { default_configuration: defaultConfiguration, configurations };
|
||||
};
|
||||
|
||||
const parseFit = (fdt: Buffer, path: string): Devicetree => {
|
||||
try {
|
||||
// The artifact field is only echoed back; the reader binds the carrier hash.
|
||||
return parseDevicetree(
|
||||
fdt,
|
||||
{ path, sha256: "", bytes: fdt.length },
|
||||
{
|
||||
path,
|
||||
max_file_bytes: MAX_DEVICETREE_FILE_BYTES,
|
||||
max_nodes: 8192,
|
||||
max_properties: 32768,
|
||||
max_depth: 64,
|
||||
},
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof BinaryFormatFailure)
|
||||
throw formatFailure(cause.message);
|
||||
if (cause instanceof DevicetreeLimitFailure)
|
||||
throw new ArtifactReaderFailure(
|
||||
"limit",
|
||||
`FIT devicetree exceeds the fixed ${cause.limit} allowance`,
|
||||
{
|
||||
boundary: cause.limit === "decoded_bytes" ? "memory" : "capacity",
|
||||
cause,
|
||||
},
|
||||
);
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
/** Decode one complete FIT devicetree whose external data lies in a fileSize-byte file. */
|
||||
export const readFitStructure = (
|
||||
fdt: Buffer,
|
||||
path: string,
|
||||
fileSize: number,
|
||||
): FitStructure => {
|
||||
const tree = parseFit(fdt, path);
|
||||
const byPath = new Map(tree.nodes.map((node) => [node.path, node]));
|
||||
const children = new Map<string, FitNode[]>();
|
||||
for (const node of tree.nodes) {
|
||||
if (node.parent === null) continue;
|
||||
const siblings = children.get(node.parent);
|
||||
if (siblings === undefined) children.set(node.parent, [node]);
|
||||
else siblings.push(node);
|
||||
}
|
||||
const root = byPath.get("/");
|
||||
if (root === undefined || !byPath.has("/images"))
|
||||
throw formatFailure("/images node is missing");
|
||||
const { totalsize } = tree.header;
|
||||
const layout: FitLayout = { fdt, children, totalsize, fileSize };
|
||||
const imageNodes = children.get("/images") ?? [];
|
||||
if (imageNodes.length > FIT_MAX_IMAGES)
|
||||
throw capacityFailure(
|
||||
`FIT declares ${String(imageNodes.length)} images, above the fixed ${String(FIT_MAX_IMAGES)}-image allowance`,
|
||||
);
|
||||
const images = imageNodes.map((node, index) =>
|
||||
imageDeclaration(layout, node, index),
|
||||
);
|
||||
assertImageDataLinks(images, fdt);
|
||||
assertDisjoint(images, totalsize);
|
||||
const declared = images.reduce((sum, image) => sum + image.size, 0);
|
||||
const budget = artifactExpansionBudgetBytes(fileSize);
|
||||
if (declared > budget)
|
||||
throw new ArtifactReaderFailure(
|
||||
"limit",
|
||||
`FIT images declare ${String(declared)} bytes, exceeding this file's ${String(budget)}-byte expansion budget`,
|
||||
{ boundary: "memory" },
|
||||
);
|
||||
const facts: FitFacts = { fdt, limitations: [] };
|
||||
return {
|
||||
images,
|
||||
limitations: facts.limitations,
|
||||
metadata: {
|
||||
kind: "fit",
|
||||
fdt_version: tree.header.version,
|
||||
fdt_totalsize: totalsize,
|
||||
external_data_base: align4(totalsize),
|
||||
description: fitString(facts, root, "description"),
|
||||
timestamp: fitU32(facts, root, "timestamp"),
|
||||
address_cells: fitU32(facts, root, "#address-cells"),
|
||||
...fitConfigurations(
|
||||
layout,
|
||||
facts,
|
||||
byPath.get("/configurations"),
|
||||
new Set(images.map((image) => image.name)),
|
||||
),
|
||||
},
|
||||
};
|
||||
};
|
||||
@@ -24,6 +24,8 @@ import {
|
||||
esp8266Image,
|
||||
espImage,
|
||||
} from "../../../tests/fixtures/espImage.js";
|
||||
import { devicetreeFixture } from "../../firmware/devicetree/Devicetree.fixture.js";
|
||||
import { fitFixture } from "../fit/FitImage.fixture.js";
|
||||
|
||||
it("joins only observed PE signature bytes across stream chunks", () => {
|
||||
const bytes = pe(0x8664, 8190);
|
||||
@@ -491,6 +493,36 @@ it("classifies nested ESP images as containers by bytes", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("classifies a FIT by a /images node within the first 4 KiB", () => {
|
||||
const images = [{ name: "kernel", data: Buffer.from("kernel") }];
|
||||
const late = devicetreeFixture({
|
||||
name: "",
|
||||
properties: { padding: Buffer.alloc(4096) },
|
||||
children: [{ name: "images" }],
|
||||
});
|
||||
const configurationsFirst = devicetreeFixture({
|
||||
name: "",
|
||||
children: [{ name: "configurations" }, { name: "images" }],
|
||||
});
|
||||
for (const [path, bytes] of [
|
||||
["firmware/boot.itb", fitFixture({ images })],
|
||||
["firmware/boot", configurationsFirst],
|
||||
] as const)
|
||||
expect(classifyArtifactBytesAtPath(path, bytes)).toEqual({
|
||||
kind: "container",
|
||||
format: "fit",
|
||||
});
|
||||
for (const bytes of [
|
||||
devicetreeFixture(),
|
||||
fitFixture({ images, version: 15 }),
|
||||
late,
|
||||
])
|
||||
expect(classifyArtifactBytesAtPath("firmware/board.dtb", bytes)).toEqual({
|
||||
kind: "resource",
|
||||
format: "file",
|
||||
});
|
||||
});
|
||||
|
||||
describe("FAT prefix observation coverage", () => {
|
||||
it("does not invent an observation for a header beyond the retained prefix", () => {
|
||||
const bytes = Buffer.alloc(8_224);
|
||||
|
||||
@@ -3,6 +3,7 @@ import { hasUImageMagic } from "../../domain/uimageFormat.js";
|
||||
import { hasUnityFsHeader } from "../../domain/unityfsFormat.js";
|
||||
import { detectHexRecordEncoding } from "../../domain/firmware/hexRecords.js";
|
||||
import { recognizeEspImage } from "../../domain/firmware/espImage.js";
|
||||
import { hasFitImagesNode } from "../../domain/firmware/fitImage.js";
|
||||
import type { ArtifactClassificationEvidence } from "../ArtifactClassificationProbe.js";
|
||||
import { asarHeaderAdmissionFailure } from "../AsarHeader.js";
|
||||
import { hasDestinationPath } from "../ArtifactPaths.js";
|
||||
@@ -291,6 +292,7 @@ export const rootOccurrenceFor = (
|
||||
"intel-hex",
|
||||
"srec",
|
||||
"esp-image",
|
||||
"fit",
|
||||
].includes(metadata.format)
|
||||
? "container"
|
||||
: artifactRoleForFormat(metadata.path, metadata.format).kind,
|
||||
@@ -588,7 +590,8 @@ const artifactRoleForFormat = (
|
||||
format === "uimage" ||
|
||||
format === "intel-hex" ||
|
||||
format === "srec" ||
|
||||
format === "esp-image"
|
||||
format === "esp-image" ||
|
||||
format === "fit"
|
||||
)
|
||||
return { kind: "container", format };
|
||||
return {
|
||||
@@ -612,6 +615,7 @@ export const classifyArtifactBytes = (
|
||||
const hexRecords = detectHexRecordEncoding(prefix);
|
||||
if (hexRecords !== undefined) return hexRecords;
|
||||
if (recognizeEspImage(prefix, fileSize) === "esp-image") return "esp-image";
|
||||
if (hasFitImagesNode(prefix)) return "fit";
|
||||
if (hasBinaryPlistSignature(prefix)) return "plist";
|
||||
if (hasXmlPlistSignature(prefix)) return "plist";
|
||||
const fatFormat = classifyFatArtifact(evidence, fileSize);
|
||||
|
||||
@@ -171,7 +171,8 @@ export const classifyAndHashRootForInventory = async (
|
||||
format === "node-sea" ||
|
||||
format === "intel-hex" ||
|
||||
format === "srec" ||
|
||||
format === "esp-image";
|
||||
format === "esp-image" ||
|
||||
format === "fit";
|
||||
outcome = {
|
||||
kind: "completed",
|
||||
value: {
|
||||
|
||||
@@ -88,6 +88,11 @@ export const inventoryCoverage = (
|
||||
"uImage inventory establishes the 64-byte legacy header facts and stored payload bytes. Compressed payloads are not decompressed, and the image is not executed.",
|
||||
]
|
||||
: []),
|
||||
...(reader.format === "fit"
|
||||
? [
|
||||
"FIT inventory reports each /images node's stored bytes and its declared type, architecture, OS, compression, load and entry. Image data is not decompressed or decrypted, signatures are not verified, nothing is executed, and load and entry addresses are declarations.",
|
||||
]
|
||||
: []),
|
||||
...traversal.limitations,
|
||||
],
|
||||
};
|
||||
@@ -116,7 +121,8 @@ export const inventoryCoverage = (
|
||||
format === "node-sea" ||
|
||||
format === "intel-hex" ||
|
||||
format === "srec" ||
|
||||
format === "esp-image"
|
||||
format === "esp-image" ||
|
||||
format === "fit"
|
||||
)
|
||||
return {
|
||||
coverage: { scope: "container-entries", status: "unknown" },
|
||||
|
||||
@@ -80,6 +80,7 @@ const formatSchema = z.enum([
|
||||
"intel-hex",
|
||||
"srec",
|
||||
"esp-image",
|
||||
"fit",
|
||||
"plist",
|
||||
"javascript",
|
||||
"source-map",
|
||||
|
||||
@@ -6,6 +6,39 @@ import { ARTIFACT_INVENTORY_VIEW_CAVEATS } from "./artifactInventoryViewCaveats.
|
||||
import { digestCanonicalValue } from "../canonicalDigest.js";
|
||||
import { evidenceSemanticContent, type Evidence } from "../evidence.js";
|
||||
|
||||
// The ordinary-root formats actually admitted before ce1f4b1bb/b395190e4.
|
||||
// Godot/PyInstaller summaries already disclosed coverage in those revisions.
|
||||
// Freeze this historical set: newly introduced formats have no older aliases.
|
||||
const HISTORICAL_ORDINARY_ROOT_FORMATS: ReadonlySet<string> = new Set([
|
||||
"dmg",
|
||||
"zip",
|
||||
"pkg",
|
||||
"asar",
|
||||
"javascript-bundle",
|
||||
"source-map",
|
||||
"mach-o-universal",
|
||||
"mach-o",
|
||||
"elf",
|
||||
"pe",
|
||||
"dos-mz",
|
||||
"dos-com",
|
||||
"raw-binary",
|
||||
"hopper",
|
||||
"analysis-database",
|
||||
"ipa",
|
||||
"apk",
|
||||
"msix",
|
||||
"appx",
|
||||
"hap",
|
||||
"hsp",
|
||||
"app-pack",
|
||||
"plist",
|
||||
"entitlements",
|
||||
"directory",
|
||||
"file",
|
||||
"unknown",
|
||||
]);
|
||||
|
||||
/** Exact shipped projections, rather than arbitrary omission or prose tolerance. */
|
||||
export const artifactInventorySummaryProjectionIds = (
|
||||
current: Evidence,
|
||||
@@ -14,13 +47,7 @@ export const artifactInventorySummaryProjectionIds = (
|
||||
current.normalized_result,
|
||||
);
|
||||
const format = result.summary.manifest.root_format;
|
||||
if (
|
||||
format === "godot-pck" ||
|
||||
format === "unityfs" ||
|
||||
format === "pyinstaller" ||
|
||||
format === "node-sea" ||
|
||||
format === "esp-image"
|
||||
)
|
||||
if (!HISTORICAL_ORDINARY_ROOT_FORMATS.has(format))
|
||||
return [current.evidence_id];
|
||||
const { view_digest: _digest, ...projection } = result;
|
||||
// b395190e4 began disclosing unavailable hashes for ordinary root formats.
|
||||
|
||||
@@ -270,7 +270,8 @@ export const projectArtifactInventoryView = (
|
||||
inventory.manifest.root_format === "node-sea" ||
|
||||
inventory.manifest.root_format === "intel-hex" ||
|
||||
inventory.manifest.root_format === "srec" ||
|
||||
inventory.manifest.root_format === "esp-image";
|
||||
inventory.manifest.root_format === "esp-image" ||
|
||||
inventory.manifest.root_format === "fit";
|
||||
const counts = inventoryCounts(source);
|
||||
const limitations = uniqueSorted([
|
||||
...parent.limitations,
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
} from "./artifactInventoryEvidence.js";
|
||||
import {
|
||||
espImageSegmentDeclaration,
|
||||
fitImageDeclaration,
|
||||
nodeSeaEntryDeclaration,
|
||||
pyInstallerEntryDeclaration,
|
||||
} from "./artifactFormatMetadata.js";
|
||||
@@ -464,6 +465,8 @@ interface OccurrenceMetadata {
|
||||
readonly asarPacking: MetadataComparison;
|
||||
readonly nodeSea: MetadataComparison;
|
||||
readonly espImageSegment: MetadataComparison;
|
||||
readonly fitImage: MetadataComparison;
|
||||
readonly fitImageAvailability: MetadataComparison;
|
||||
readonly pyInstaller: MetadataComparison;
|
||||
}
|
||||
|
||||
@@ -471,7 +474,8 @@ interface OccurrenceMetadata {
|
||||
const declarationChanged = (metadata: OccurrenceMetadata): boolean =>
|
||||
metadata.asarPacking === "changed" ||
|
||||
metadata.nodeSea === "changed" ||
|
||||
metadata.espImageSegment === "changed";
|
||||
metadata.espImageSegment === "changed" ||
|
||||
metadata.fitImage === "changed";
|
||||
|
||||
/** Unknown classifications and absent node metadata cannot establish a difference. */
|
||||
const differsWithUnknown = (
|
||||
@@ -550,7 +554,10 @@ const compareOptionalMetadata = <T>(
|
||||
const occurrenceMetadata = (input: OccurrencePair): OccurrenceMetadata => {
|
||||
const left = input.leftOccurrence;
|
||||
const right = input.rightOccurrence;
|
||||
const fit = compareFitImageDeclarations(input);
|
||||
return {
|
||||
fitImage: fit.declaration,
|
||||
fitImageAvailability: fit.availability,
|
||||
asarPacking: compareOptionalMetadata(
|
||||
left?.unpacked,
|
||||
right?.unpacked,
|
||||
@@ -582,6 +589,50 @@ const occurrenceMetadata = (input: OccurrencePair): OccurrenceMetadata => {
|
||||
};
|
||||
};
|
||||
|
||||
/** Keep known FIT changes even when another declaration is unavailable. */
|
||||
const compareFitImageDeclarations = ({
|
||||
leftOccurrence: left,
|
||||
rightOccurrence: right,
|
||||
}: OccurrencePair): {
|
||||
readonly declaration: MetadataComparison;
|
||||
readonly availability: MetadataComparison;
|
||||
} => {
|
||||
const lm =
|
||||
left?.format_metadata?.kind === "fit-image"
|
||||
? left.format_metadata
|
||||
: undefined;
|
||||
const rm =
|
||||
right?.format_metadata?.kind === "fit-image"
|
||||
? right.format_metadata
|
||||
: undefined;
|
||||
if (lm === undefined || rm === undefined) {
|
||||
const result = compareOptionalMetadata(lm, rm, fitImageDeclaration);
|
||||
return {
|
||||
declaration: result,
|
||||
availability: result === "unknown" ? "unknown" : "unchanged",
|
||||
};
|
||||
}
|
||||
const ld = fitImageDeclaration(lm);
|
||||
const rd = fitImageDeclaration(rm);
|
||||
const keys = Object.keys(ld) as (keyof typeof ld)[];
|
||||
let changed = false;
|
||||
let unavailable = false;
|
||||
for (const key of keys) {
|
||||
if (
|
||||
lm.unreadable_fields.some((field) => field === key) ||
|
||||
rm.unreadable_fields.some((field) => field === key)
|
||||
) {
|
||||
unavailable = true;
|
||||
continue;
|
||||
}
|
||||
if (ld[key] !== rd[key]) changed = true;
|
||||
}
|
||||
return {
|
||||
declaration: changed ? "changed" : "unchanged",
|
||||
availability: unavailable ? "unknown" : "unchanged",
|
||||
};
|
||||
};
|
||||
|
||||
const isPyInstallerEntry = (
|
||||
occurrence: ArtifactOccurrence | undefined,
|
||||
): occurrence is ArtifactOccurrence & {
|
||||
|
||||
@@ -197,6 +197,93 @@ export const hexRecordRegionMetadataSchema = z.strictObject({
|
||||
first_line: offset,
|
||||
last_line: offset,
|
||||
});
|
||||
|
||||
/**
|
||||
* U-Boot FIT root facts. Configuration references are declared names; a
|
||||
* reference that names no image is reported as a limitation, not removed.
|
||||
*/
|
||||
export const fitMetadataSchema = z.strictObject({
|
||||
kind: z.literal("fit"),
|
||||
fdt_version: uint32,
|
||||
fdt_totalsize: offset,
|
||||
external_data_base: offset,
|
||||
description: z.string().nullable(),
|
||||
timestamp: uint32.nullable(),
|
||||
address_cells: uint32.nullable(),
|
||||
default_configuration: z.string().nullable(),
|
||||
configurations: z.array(
|
||||
z.strictObject({
|
||||
name: z.string(),
|
||||
description: z.string().nullable(),
|
||||
kernel: z.array(z.string()),
|
||||
firmware: z.array(z.string()),
|
||||
fdt: z.array(z.string()),
|
||||
ramdisk: z.array(z.string()),
|
||||
loadables: z.array(z.string()),
|
||||
signature_count: uint32,
|
||||
signatures_unreadable: uint32,
|
||||
unresolved_reference_count: uint32,
|
||||
signature_algorithms: z.array(z.string()),
|
||||
}),
|
||||
),
|
||||
});
|
||||
export type FitMetadata = z.infer<typeof fitMetadataSchema>;
|
||||
|
||||
/**
|
||||
* One /images node's facts. Null fields are absent unless unreadable_fields
|
||||
* identifies an interpretation failure. Load and entry are declarations;
|
||||
* stored bytes may be compressed or ciphered.
|
||||
*/
|
||||
export const fitImageMetadataSchema = z.strictObject({
|
||||
kind: z.literal("fit-image"),
|
||||
index: uint32,
|
||||
name: z.string(),
|
||||
description: z.string().nullable(),
|
||||
type: z.string().nullable(),
|
||||
arch: z.string().nullable(),
|
||||
os: z.string().nullable(),
|
||||
compression: z.string().nullable(),
|
||||
load: z.string().nullable(),
|
||||
entry: z.string().nullable(),
|
||||
data_source: z.enum(["embedded", "data-offset", "data-position"]),
|
||||
declared_offset: uint32.nullable(),
|
||||
image_data: z.string().nullable(),
|
||||
/** Presence is independent of whether the declared algorithm is readable. */
|
||||
cipher_present: z.boolean(),
|
||||
cipher_algo: z.string().nullable(),
|
||||
unreadable_fields: z.array(
|
||||
z.enum([
|
||||
"description",
|
||||
"type",
|
||||
"arch",
|
||||
"os",
|
||||
"compression",
|
||||
"load",
|
||||
"entry",
|
||||
"cipher_algo",
|
||||
]),
|
||||
),
|
||||
hash_algorithms: z.array(z.string()),
|
||||
hashes_verified: uint32,
|
||||
hashes_unverified: uint32,
|
||||
signature_count: uint32,
|
||||
signatures_unreadable: uint32,
|
||||
signature_algorithms: z.array(z.string()),
|
||||
});
|
||||
export type FitImageMetadata = z.infer<typeof fitImageMetadataSchema>;
|
||||
/** Payload interpretation and sharing declarations, independent of FIT placement. */
|
||||
export const fitImageDeclaration = (metadata: FitImageMetadata) => ({
|
||||
type: metadata.type,
|
||||
arch: metadata.arch,
|
||||
os: metadata.os,
|
||||
compression: metadata.compression,
|
||||
load: metadata.load,
|
||||
entry: metadata.entry,
|
||||
image_data: metadata.image_data,
|
||||
cipher_present: metadata.cipher_present,
|
||||
cipher_algo: metadata.cipher_algo,
|
||||
});
|
||||
|
||||
/** Runtime member identity, independent of blob layout and directory order. */
|
||||
export const nodeSeaEntryDeclaration = (
|
||||
metadata: z.infer<typeof nodeSeaEntryMetadataSchema>,
|
||||
@@ -417,6 +504,8 @@ export const artifactFormatMetadataSchema = z.discriminatedUnion("kind", [
|
||||
espImageMetadataSchema,
|
||||
espImageSegmentMetadataSchema,
|
||||
espImageSignatureBlockMetadataSchema,
|
||||
fitMetadataSchema,
|
||||
fitImageMetadataSchema,
|
||||
]);
|
||||
export type ArtifactFormatMetadata = z.infer<
|
||||
typeof artifactFormatMetadataSchema
|
||||
|
||||
@@ -91,6 +91,7 @@ const artifactFormatSchema = z.enum([
|
||||
"intel-hex",
|
||||
"srec",
|
||||
"esp-image",
|
||||
"fit",
|
||||
"unknown",
|
||||
]);
|
||||
|
||||
@@ -103,6 +104,7 @@ const artifactContentFormatSchema = artifactFormatSchema.extract([
|
||||
"intel-hex",
|
||||
"srec",
|
||||
"esp-image",
|
||||
"fit",
|
||||
"mach-o-universal",
|
||||
"mach-o",
|
||||
"elf",
|
||||
|
||||
@@ -18,6 +18,7 @@ import {
|
||||
} from "./artifactGraph.js";
|
||||
import { digestCanonicalValue } from "./canonicalDigest.js";
|
||||
import { isJavaScriptSourcePath } from "./javascript/javascriptArtifactFiles.js";
|
||||
import { admitRawBinary } from "./rawBinary.js";
|
||||
import { uniqueSorted } from "./canonicalOrdering.js";
|
||||
import { evidenceSchema, parseEvidence, type Evidence } from "./evidence.js";
|
||||
import { jsonValueSchema } from "./jsonValue.js";
|
||||
@@ -330,11 +331,57 @@ const probesFor = (
|
||||
},
|
||||
]
|
||||
: [];
|
||||
if (format === "fit") return fitProbes(inventory);
|
||||
return format === "directory"
|
||||
? directoryProbes(inventory)
|
||||
: probesForFormat(format);
|
||||
};
|
||||
|
||||
const fitProbes = (
|
||||
inventory: ArtifactInventoryResult,
|
||||
): ArtifactInspectionResult["next_probes"] => {
|
||||
// Extraction keeps stored bytes, so only plain images are directly analyzable.
|
||||
const images = inventory.occurrences.flatMap((row) =>
|
||||
row.format_metadata?.kind === "fit-image" &&
|
||||
row.format_metadata.compression === "none" &&
|
||||
row.format_metadata.cipher_present === false
|
||||
? [{ row, image: row.format_metadata }]
|
||||
: [],
|
||||
);
|
||||
let rawProbe: ArtifactInspectionResult["next_probes"][number] | undefined;
|
||||
for (const { row, image } of images) {
|
||||
if (row.declared_size === null) continue;
|
||||
if (image.entry !== null && image.entry !== image.load) continue;
|
||||
// An unavailable entry is not proof that the declaration was absent.
|
||||
if (image.unreadable_fields.includes("entry")) continue;
|
||||
const admitted = admitRawBinary({
|
||||
architecture: image.arch,
|
||||
entryPoint: image.load,
|
||||
fileBytes: row.declared_size,
|
||||
});
|
||||
if (!admitted.ok) continue;
|
||||
rawProbe = {
|
||||
operation: "open_binary",
|
||||
rationale: `Extract ${JSON.stringify(row.logical_path)} with extract_artifact, then open those uncompressed, unciphered bytes with format raw-binary, architecture ${admitted.value.architecture} and entry_point ${admitted.value.entryPoint}. This maps offset 0 and seeds analysis at that address; the declared entry is absent or equal to load.`,
|
||||
authority: "native-analysis-provider",
|
||||
};
|
||||
break;
|
||||
}
|
||||
return [
|
||||
...(images.some(({ image }) => image.type === "flat_dt")
|
||||
? [
|
||||
{
|
||||
operation: "inspect_devicetree",
|
||||
rationale:
|
||||
"Extract images with extract_artifact, then inspect uncompressed, unciphered flat_dt images with inspect_devicetree.",
|
||||
authority: "artifact-bytes" as const,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
...(rawProbe === undefined ? [] : [rawProbe]),
|
||||
];
|
||||
};
|
||||
|
||||
// An application bundle directory is only a container, so its probes follow
|
||||
// the formats it holds: a bundle of Mach-O executables is not a JavaScript tree.
|
||||
// Formats are ordered so a repeated operation keeps its most general rationale.
|
||||
|
||||
@@ -264,7 +264,8 @@ export type BinaryTarget =
|
||||
| "pkg"
|
||||
| "intel-hex"
|
||||
| "srec"
|
||||
| "esp-image";
|
||||
| "esp-image"
|
||||
| "fit";
|
||||
})
|
||||
| (BinaryTargetIdentity &
|
||||
NonExecutableMetadata & {
|
||||
|
||||
@@ -43,6 +43,7 @@ export const evidenceSubjectFormatSchema = z.enum([
|
||||
"intel-hex",
|
||||
"srec",
|
||||
"esp-image",
|
||||
"fit",
|
||||
]);
|
||||
|
||||
/** Format-owned architecture declarations shared by Evidence and persisted executions. */
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
/**
|
||||
* Whether a flattened devicetree prefix declares a top-level /images node, the
|
||||
* U-Boot FIT marker. Only the first 4 KiB is walked, so a FIT whose /images
|
||||
* node starts later stays a plain devicetree; the FIT reader's full parse
|
||||
* remains the authority for everything this probe accepts.
|
||||
*/
|
||||
export const hasFitImagesNode = (prefix: Uint8Array): boolean => {
|
||||
const bytes = prefix.subarray(0, 4096);
|
||||
if (bytes.length < 40) return false;
|
||||
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||
const u32 = (offset: number): number => view.getUint32(offset);
|
||||
const version = u32(20);
|
||||
if (u32(0) !== 0xd00dfeed || (version !== 16 && version !== 17)) return false;
|
||||
const end = Math.min(bytes.length, u32(4));
|
||||
let cursor = u32(8);
|
||||
let depth = 0;
|
||||
while (cursor <= end - 4) {
|
||||
const token = u32(cursor);
|
||||
if (token === 1) {
|
||||
const nul = bytes.indexOf(0, cursor + 4);
|
||||
if (nul < 0 || nul >= end) return false;
|
||||
const name = String.fromCharCode(...bytes.subarray(cursor + 4, nul));
|
||||
if (depth === 0 && name !== "") return false;
|
||||
if (depth === 1 && name === "images") return true;
|
||||
depth += 1;
|
||||
cursor = Math.ceil((nul + 1) / 4) * 4;
|
||||
} else if (token === 2) {
|
||||
depth -= 1;
|
||||
if (depth <= 0) return false;
|
||||
cursor += 4;
|
||||
} else if (token === 3) {
|
||||
if (cursor > end - 8) return false;
|
||||
cursor += Math.ceil((12 + u32(cursor + 4)) / 4) * 4;
|
||||
} else if (token === 4) cursor += 4;
|
||||
else return false;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
@@ -78,6 +78,7 @@ export const liveBinaryTargetSummaryVariants = [
|
||||
"intel-hex",
|
||||
"srec",
|
||||
"esp-image",
|
||||
"fit",
|
||||
]),
|
||||
architecture: z.null(),
|
||||
}),
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, it } from "vitest";
|
||||
import { fitFixture } from "../../src/artifacts/fit/FitImage.fixture.js";
|
||||
import {
|
||||
cells,
|
||||
dtString,
|
||||
} from "../../src/firmware/devicetree/Devicetree.fixture.js";
|
||||
import { artifactComparisonResultSchema } from "../../src/domain/artifactComparison.js";
|
||||
import { artifactInspectionResultSchema } from "../../src/domain/artifactInspection.js";
|
||||
import { artifactInventoryResultSchema } from "../../src/domain/artifactGraph.js";
|
||||
import { parseEvidence, type Evidence } from "../../src/domain/evidence.js";
|
||||
import { connectLocalToolsMcp } from "../fixtures/localToolsMcp.js";
|
||||
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
|
||||
|
||||
it("compares FIT declarations while preserving layout independence and unknowns", async () => {
|
||||
const root = await createTestTempDirectory("rea-fit-comparison-");
|
||||
const { call } = await connectLocalToolsMcp({
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
REA_MCP_TOOL_FAMILIES: "artifact,application,session",
|
||||
});
|
||||
const properties = {
|
||||
type: dtString("kernel"),
|
||||
arch: dtString("arm64"),
|
||||
compression: dtString("none"),
|
||||
load: cells(0x400000),
|
||||
entry: cells(0x400004),
|
||||
};
|
||||
const image = {
|
||||
name: "kernel",
|
||||
data: Buffer.from("payload"),
|
||||
hashes: ["sha256"],
|
||||
};
|
||||
const inspect = async (
|
||||
name: string,
|
||||
overrides: Record<string, Buffer> = {},
|
||||
options: {
|
||||
mode?: "embedded" | "position";
|
||||
prefix?: boolean;
|
||||
omitLoad?: boolean;
|
||||
unsupportedHash?: boolean;
|
||||
} = {},
|
||||
): Promise<Evidence> => {
|
||||
const path = join(root, name);
|
||||
const { load: _load, ...withoutLoad } = properties;
|
||||
await writeFile(
|
||||
path,
|
||||
fitFixture({
|
||||
images: [
|
||||
...(options.prefix
|
||||
? [{ name: "prefix", data: Buffer.from("prefix") }]
|
||||
: []),
|
||||
{
|
||||
...image,
|
||||
properties: {
|
||||
...(options.omitLoad ? withoutLoad : properties),
|
||||
...overrides,
|
||||
},
|
||||
...(options.unsupportedHash
|
||||
? {
|
||||
children: [
|
||||
{
|
||||
name: "hash-unsupported",
|
||||
properties: {
|
||||
algo: dtString("crc16-ccitt"),
|
||||
value: Buffer.alloc(2),
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
],
|
||||
mode: options.mode ?? "embedded",
|
||||
}),
|
||||
);
|
||||
const opened = await call("open_binary", { path });
|
||||
expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
|
||||
const response = await call("inspect_artifact", {});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
return parseEvidence(response.structuredContent);
|
||||
};
|
||||
const compare = async (left: Evidence, right: Evidence) => {
|
||||
const response = await call("compare_artifacts", { left, right });
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
return artifactComparisonResultSchema.parse(
|
||||
parseEvidence(response.structuredContent).normalized_result,
|
||||
);
|
||||
};
|
||||
const occurrence = (evidence: Evidence) => {
|
||||
const inspection = artifactInspectionResultSchema.parse(
|
||||
evidence.normalized_result,
|
||||
);
|
||||
const row = artifactInventoryResultSchema
|
||||
.parse(inspection.substeps[0]?.evidence.normalized_result)
|
||||
.occurrences.find((row) => row.logical_path === "images/kernel.bin");
|
||||
if (row === undefined) throw new Error("Missing kernel occurrence");
|
||||
return row;
|
||||
};
|
||||
const left = await inspect("base.itb");
|
||||
for (const field of ["load", "entry"] as const) {
|
||||
const right = await inspect(field + ".itb", { [field]: cells(0x500000) });
|
||||
expect(occurrence(right).artifact_id).toBe(occurrence(left).artifact_id);
|
||||
const result = await compare(left, right);
|
||||
expect(
|
||||
result.changes.filter((row) => row.logical_path === "images/kernel.bin"),
|
||||
).toMatchObject([{ classification: "changed", dimensions: ["metadata"] }]);
|
||||
}
|
||||
const relocated = await inspect(
|
||||
"relocated.itb",
|
||||
{},
|
||||
{ mode: "position", prefix: true },
|
||||
);
|
||||
expect(occurrence(relocated).artifact_id).toBe(occurrence(left).artifact_id);
|
||||
expect(
|
||||
(await compare(left, relocated)).changes.filter(
|
||||
(row) => row.logical_path === "images/kernel.bin",
|
||||
),
|
||||
).toEqual([]);
|
||||
const unknownLeft = await inspect("unknown-left.itb", {
|
||||
load: cells(1, 2, 3),
|
||||
});
|
||||
const unknownRight = await inspect("unknown-right.itb", {
|
||||
load: cells(4, 5, 6),
|
||||
});
|
||||
expect(
|
||||
(await compare(unknownLeft, unknownRight)).changes.filter(
|
||||
(row) => row.logical_path === "images/kernel.bin",
|
||||
),
|
||||
).toMatchObject([
|
||||
{ classification: "unknown", dimensions: ["availability"] },
|
||||
]);
|
||||
const partialChange = await inspect("partial-entry.itb", {
|
||||
load: cells(4, 5, 6),
|
||||
entry: cells(0x600000),
|
||||
});
|
||||
expect(
|
||||
(await compare(unknownLeft, partialChange)).changes.filter(
|
||||
(row) => row.logical_path === "images/kernel.bin",
|
||||
),
|
||||
).toMatchObject([
|
||||
{ classification: "changed", dimensions: ["metadata", "availability"] },
|
||||
]);
|
||||
const absent = await inspect(
|
||||
"warn-absent.itb",
|
||||
{},
|
||||
{ omitLoad: true, unsupportedHash: true },
|
||||
);
|
||||
const present = await inspect(
|
||||
"warn-present.itb",
|
||||
{},
|
||||
{ unsupportedHash: true },
|
||||
);
|
||||
expect(
|
||||
(await compare(absent, present)).changes.filter(
|
||||
(row) => row.logical_path === "images/kernel.bin",
|
||||
),
|
||||
).toMatchObject([{ classification: "changed", dimensions: ["metadata"] }]);
|
||||
});
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
rebindEvidenceToTargetPath,
|
||||
type Evidence,
|
||||
} from "../../src/domain/evidence.js";
|
||||
import { FIT_EMBEDDED } from "../../src/artifacts/fit/FitImage.fixture.js";
|
||||
import { parseMcpToolError } from "../fixtures/mcpToolError.js";
|
||||
import { connectLocalToolsMcp } from "../fixtures/localToolsMcp.js";
|
||||
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
|
||||
@@ -141,3 +142,50 @@ it("compares a historical alias summary through its retained complete MCP parent
|
||||
summary: { source_coverage: { scope: "root", status: "complete" } },
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a fabricated pre-coverage FIT summary while accepting its current projection", async () => {
|
||||
const root = await createTestTempDirectory("rea-fit-summary-identity-");
|
||||
const path = join(root, "boot.itb");
|
||||
await writeFile(path, FIT_EMBEDDED);
|
||||
const { call } = await connectLocalToolsMcp({
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
REA_MCP_TOOL_FAMILIES: "artifact,application,session",
|
||||
});
|
||||
const opened = await call("open_binary", { path });
|
||||
expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
|
||||
const inspected = await call("inspect_artifact", { detail: "complete" });
|
||||
expect(inspected.isError, JSON.stringify(inspected)).not.toBe(true);
|
||||
const parent = parseEvidence(inspected.structuredContent);
|
||||
const projected = await call("inspect_analysis_view", {
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "summary" },
|
||||
});
|
||||
expect(projected.isError, JSON.stringify(projected)).not.toBe(true);
|
||||
const current = parseEvidence(projected.structuredContent);
|
||||
const result = artifactInventorySummaryViewResultSchema.parse(
|
||||
current.normalized_result,
|
||||
);
|
||||
expect(result.summary.manifest.root_format).toBe("fit");
|
||||
expect(result.summary.source_coverage?.status).toBe("complete");
|
||||
const accepted = await call("compare_artifacts", {
|
||||
left: current,
|
||||
right: parent,
|
||||
});
|
||||
expect(accepted.isError, JSON.stringify(accepted)).not.toBe(true);
|
||||
expect(
|
||||
artifactComparisonResultSchema.parse(
|
||||
parseEvidence(accepted.structuredContent).normalized_result,
|
||||
).status,
|
||||
).toBe("unchanged");
|
||||
const { source_coverage: _coverage, ...summary } = result.summary;
|
||||
const fabricated = resealSummary(current, { ...result, summary });
|
||||
expect(fabricated.evidence_id).not.toBe(current.evidence_id);
|
||||
const rejected = await call("compare_artifacts", {
|
||||
left: fabricated,
|
||||
right: parent,
|
||||
});
|
||||
expect(rejected.isError, JSON.stringify(rejected)).toBe(true);
|
||||
expect(parseMcpToolError(rejected)).toMatchObject({
|
||||
error: { code: "evidence_integrity_mismatch" },
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,415 @@
|
||||
import { readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, onTestFinished } from "vitest";
|
||||
import { cliTest } from "../support/cli/cliFixture.js";
|
||||
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
|
||||
import {
|
||||
FIT_DEVICETREE,
|
||||
FIT_EMBEDDED,
|
||||
FIT_EXTERNAL,
|
||||
FIT_KERNEL,
|
||||
fitFixture,
|
||||
} from "../../src/artifacts/fit/FitImage.fixture.js";
|
||||
import {
|
||||
cells,
|
||||
dtString,
|
||||
} from "../../src/firmware/devicetree/Devicetree.fixture.js";
|
||||
import { parseEvidence } from "../../src/domain/evidence.js";
|
||||
import { artifactInspectionResultSchema } from "../../src/domain/artifactInspection.js";
|
||||
import {
|
||||
artifactExtractionResultSchema,
|
||||
artifactInventoryResultSchema,
|
||||
} from "../../src/domain/artifactGraph.js";
|
||||
|
||||
const cases = [
|
||||
{ mode: "embedded", bytes: FIT_EMBEDDED },
|
||||
{ mode: "external", bytes: FIT_EXTERNAL },
|
||||
] as const;
|
||||
|
||||
cliTest.for(cases)(
|
||||
"inventories and extracts $mode FIT images through the compiled CLI",
|
||||
async ({ bytes }, { cli }) => {
|
||||
const directory = await createTestTempDirectory("rea-fit-cli-");
|
||||
const source = join(directory, "boot.itb");
|
||||
await writeFile(source, bytes);
|
||||
const inspected = await cli.run({
|
||||
arguments: ["inspect-artifact", source, "--json"],
|
||||
});
|
||||
expect(inspected.exitCode, inspected.stdout + inspected.stderr).toBe(0);
|
||||
const inspection = artifactInspectionResultSchema.parse(
|
||||
parseEvidence(inspected.json).normalized_result,
|
||||
);
|
||||
const graph = artifactInventoryResultSchema.parse(
|
||||
inspection.substeps[0]?.evidence.normalized_result,
|
||||
);
|
||||
expect(graph.manifest.root_format).toBe("fit");
|
||||
expect(
|
||||
graph.occurrences.find((row) => row.logical_path === ".")
|
||||
?.format_metadata,
|
||||
).toMatchObject({
|
||||
kind: "fit",
|
||||
description: "Test FIT",
|
||||
default_configuration: "conf-1",
|
||||
});
|
||||
expect(
|
||||
graph.occurrences
|
||||
.filter((row) => row.logical_path !== ".")
|
||||
.map((row) => row.logical_path)
|
||||
.sort(),
|
||||
).toEqual(["images/fdt-1.bin", "images/kernel.bin"]);
|
||||
expect(inspection.next_probes.map((probe) => probe.operation)).toEqual([
|
||||
"inspect_devicetree",
|
||||
"open_binary",
|
||||
]);
|
||||
const extracted = await cli.run({
|
||||
arguments: ["extract-artifact", source, "--json"],
|
||||
environment: { TMPDIR: directory, TMP: directory, TEMP: directory },
|
||||
});
|
||||
expect(extracted.exitCode, extracted.stdout + extracted.stderr).toBe(0);
|
||||
const result = artifactExtractionResultSchema.parse(
|
||||
parseEvidence(extracted.json).normalized_result,
|
||||
);
|
||||
onTestFinished(() =>
|
||||
rm(result.output_root, { recursive: true, force: true }),
|
||||
);
|
||||
expect(result.containment_verified).toBe(true);
|
||||
for (const [path, data] of [
|
||||
["images/kernel.bin", FIT_KERNEL],
|
||||
["images/fdt-1.bin", FIT_DEVICETREE],
|
||||
] as const)
|
||||
expect(
|
||||
Buffer.compare(await readFile(join(result.output_root, path)), data),
|
||||
).toBe(0);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest("rejects a FIT hash mismatch through the CLI", async ({ cli }) => {
|
||||
const directory = await createTestTempDirectory("rea-fit-invalid-cli-");
|
||||
const source = join(directory, "boot.itb");
|
||||
const bytes = Buffer.from(FIT_EXTERNAL);
|
||||
const kernel = bytes.indexOf(FIT_KERNEL);
|
||||
bytes[kernel] = (bytes[kernel] ?? 0) ^ 1;
|
||||
await writeFile(source, bytes);
|
||||
const result = await cli.run({
|
||||
arguments: ["inspect-artifact", source, "--json"],
|
||||
});
|
||||
expect(result.exitCode).not.toBe(0);
|
||||
expect(result.stdout + result.stderr).toContain(
|
||||
"FIT image kernel hash-1 crc32 does not match",
|
||||
);
|
||||
});
|
||||
|
||||
cliTest.for([
|
||||
{ label: "known", properties: { algo: dtString("aes256") } },
|
||||
{ label: "missing", properties: {} },
|
||||
{ label: "malformed", properties: { algo: Buffer.from([0xff, 0]) } },
|
||||
])(
|
||||
"suggests no direct analysis for compressed or ciphered FIT images ($label algorithm)",
|
||||
async ({ label, properties }, { cli }) => {
|
||||
const directory = await createTestTempDirectory("rea-fit-encoded-cli-");
|
||||
const source = join(directory, "boot.itb");
|
||||
await writeFile(
|
||||
source,
|
||||
fitFixture({
|
||||
images: [
|
||||
{
|
||||
name: "kernel",
|
||||
data: FIT_KERNEL,
|
||||
properties: {
|
||||
type: dtString("flat_dt"),
|
||||
compression: dtString("none"),
|
||||
load: cells(0x4020_0000),
|
||||
},
|
||||
children: [{ name: "cipher", properties }],
|
||||
},
|
||||
{
|
||||
name: "fdt-1",
|
||||
data: FIT_DEVICETREE,
|
||||
properties: {
|
||||
type: dtString("flat_dt"),
|
||||
compression: dtString("gzip"),
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
const inspected = await cli.run({
|
||||
arguments: ["inspect-artifact", source, "--json"],
|
||||
});
|
||||
expect(inspected.exitCode, inspected.stdout + inspected.stderr).toBe(0);
|
||||
const inspection = artifactInspectionResultSchema.parse(
|
||||
parseEvidence(inspected.json).normalized_result,
|
||||
);
|
||||
const graph = artifactInventoryResultSchema.parse(
|
||||
inspection.substeps[0]?.evidence.normalized_result,
|
||||
);
|
||||
expect(
|
||||
graph.occurrences.find((row) => row.logical_path === "images/kernel.bin")
|
||||
?.format_metadata,
|
||||
).toMatchObject({
|
||||
cipher_present: true,
|
||||
cipher_algo: label === "known" ? "aes256" : null,
|
||||
});
|
||||
expect(inspection.next_probes.map((probe) => probe.operation)).not.toEqual(
|
||||
expect.arrayContaining(["inspect_devicetree"]),
|
||||
);
|
||||
expect(inspection.next_probes.map((probe) => probe.operation)).not.toEqual(
|
||||
expect.arrayContaining(["open_binary"]),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest.for([true, false])(
|
||||
"checks embedded image-data content (equal=%s)",
|
||||
async (equal, { cli }) => {
|
||||
const directory = await createTestTempDirectory("rea-fit-shared-cli-");
|
||||
const source = join(directory, "shared.itb");
|
||||
await writeFile(
|
||||
source,
|
||||
fitFixture({
|
||||
images: [
|
||||
{ name: "target", data: Buffer.from("AAAA") },
|
||||
{
|
||||
name: "linked",
|
||||
data: Buffer.from(equal ? "AAAA" : "BBBB"),
|
||||
properties: { "image-data": dtString("target") },
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
const inspected = await cli.run({
|
||||
arguments: ["inspect-artifact", source, "--json"],
|
||||
});
|
||||
if (!equal) {
|
||||
expect(inspected.exitCode).not.toBe(0);
|
||||
expect(inspected.stdout + inspected.stderr).toContain(
|
||||
"image-data target /images/target",
|
||||
);
|
||||
return;
|
||||
}
|
||||
expect(inspected.exitCode, inspected.stdout + inspected.stderr).toBe(0);
|
||||
const inspection = artifactInspectionResultSchema.parse(
|
||||
parseEvidence(inspected.json).normalized_result,
|
||||
);
|
||||
const graph = artifactInventoryResultSchema.parse(
|
||||
inspection.substeps[0]?.evidence.normalized_result,
|
||||
);
|
||||
const target = graph.occurrences.find(
|
||||
(row) => row.logical_path === "images/target.bin",
|
||||
);
|
||||
const linked = graph.occurrences.find(
|
||||
(row) => row.logical_path === "images/linked.bin",
|
||||
);
|
||||
expect(linked?.format_metadata).toMatchObject({ image_data: "target" });
|
||||
expect(linked?.artifact_id).toBe(target?.artifact_id);
|
||||
expect(linked?.source_location?.offset).not.toBe(
|
||||
target?.source_location?.offset,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"preserves unreadable image and configuration signature declarations",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-fit-unknown-signatures-");
|
||||
const source = join(root, "signed.itb");
|
||||
const signatures = [
|
||||
{ name: "signature-missing", properties: {} },
|
||||
{
|
||||
name: "signature-malformed",
|
||||
properties: { algo: Buffer.from([0xff, 0]) },
|
||||
},
|
||||
];
|
||||
await writeFile(
|
||||
source,
|
||||
fitFixture({
|
||||
images: [
|
||||
{ name: "plain", data: Buffer.from("plain") },
|
||||
{ name: "signed", data: Buffer.from("signed"), children: signatures },
|
||||
],
|
||||
configurations: {
|
||||
name: "configurations",
|
||||
children: [{ name: "conf", children: signatures }],
|
||||
},
|
||||
}),
|
||||
);
|
||||
const response = await cli.run({
|
||||
arguments: ["inspect-artifact", source, "--json"],
|
||||
});
|
||||
expect(response.exitCode, response.stdout + response.stderr).toBe(0);
|
||||
const inspection = artifactInspectionResultSchema.parse(
|
||||
parseEvidence(response.json).normalized_result,
|
||||
);
|
||||
const graph = artifactInventoryResultSchema.parse(
|
||||
inspection.substeps[0]?.evidence.normalized_result,
|
||||
);
|
||||
expect(
|
||||
graph.occurrences.find((row) => row.logical_path === "images/plain.bin")
|
||||
?.format_metadata,
|
||||
).toMatchObject({ signature_count: 0, signatures_unreadable: 0 });
|
||||
const signed = graph.occurrences.find(
|
||||
(row) => row.logical_path === "images/signed.bin",
|
||||
);
|
||||
expect(signed?.format_metadata).toMatchObject({
|
||||
signature_count: 2,
|
||||
signatures_unreadable: 2,
|
||||
signature_algorithms: [],
|
||||
});
|
||||
expect(
|
||||
signed?.limitations.some((value) => value.includes("signature-missing")),
|
||||
).toBe(true);
|
||||
expect(
|
||||
signed?.limitations.some((value) =>
|
||||
value.includes("signature-malformed"),
|
||||
),
|
||||
).toBe(true);
|
||||
const metadata = graph.occurrences.find(
|
||||
(row) => row.logical_path === ".",
|
||||
)?.format_metadata;
|
||||
expect(metadata).toMatchObject({
|
||||
configurations: [
|
||||
{
|
||||
signature_count: 2,
|
||||
signatures_unreadable: 2,
|
||||
signature_algorithms: [],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(
|
||||
graph.limitations.some((value) =>
|
||||
value.includes("/configurations/conf/signature-missing"),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(
|
||||
graph.limitations.some((value) =>
|
||||
value.includes("/configurations/conf/signature-malformed"),
|
||||
),
|
||||
).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest.for([
|
||||
{
|
||||
label: "matching entry",
|
||||
arch: "arm",
|
||||
load: cells(0x400000),
|
||||
entry: cells(0x400000),
|
||||
bytes: 4,
|
||||
allowed: true,
|
||||
},
|
||||
{
|
||||
label: "unrelated hash warning",
|
||||
arch: "arm",
|
||||
load: cells(0x400000),
|
||||
bytes: 4,
|
||||
allowed: true,
|
||||
unsupportedHash: true,
|
||||
},
|
||||
{
|
||||
label: "absent entry",
|
||||
arch: "arm",
|
||||
load: cells(0x400000),
|
||||
bytes: 4,
|
||||
allowed: true,
|
||||
},
|
||||
{
|
||||
label: "missing architecture",
|
||||
load: cells(0x400000),
|
||||
bytes: 4,
|
||||
allowed: false,
|
||||
},
|
||||
{
|
||||
label: "unsupported architecture",
|
||||
arch: "mips",
|
||||
load: cells(0x400000),
|
||||
bytes: 4,
|
||||
allowed: false,
|
||||
},
|
||||
{
|
||||
label: "empty image",
|
||||
arch: "arm",
|
||||
load: cells(0x400000),
|
||||
bytes: 0,
|
||||
allowed: false,
|
||||
},
|
||||
{
|
||||
label: "misaligned load",
|
||||
arch: "arm",
|
||||
load: cells(0x400001),
|
||||
bytes: 4,
|
||||
allowed: false,
|
||||
},
|
||||
{
|
||||
label: "mapping beyond address width",
|
||||
arch: "arm",
|
||||
load: cells(0xfffffffc),
|
||||
bytes: 8,
|
||||
allowed: false,
|
||||
},
|
||||
{
|
||||
label: "divergent entry",
|
||||
arch: "arm",
|
||||
load: cells(0x400000),
|
||||
entry: cells(0x400004),
|
||||
bytes: 4,
|
||||
allowed: false,
|
||||
},
|
||||
{
|
||||
label: "unreadable entry",
|
||||
arch: "arm",
|
||||
load: cells(0x400000),
|
||||
entry: cells(1, 2, 3),
|
||||
bytes: 4,
|
||||
allowed: false,
|
||||
},
|
||||
])("offers admitted FIT raw analysis ($label)", async (item, { cli }) => {
|
||||
const root = await createTestTempDirectory("rea-fit-raw-probe-");
|
||||
const source = join(root, "raw.itb");
|
||||
await writeFile(
|
||||
source,
|
||||
fitFixture({
|
||||
images: [
|
||||
{
|
||||
name: "kernel",
|
||||
data: Buffer.alloc(item.bytes, 1),
|
||||
...("unsupportedHash" in item
|
||||
? {
|
||||
children: [
|
||||
{
|
||||
name: "hash-unsupported",
|
||||
properties: {
|
||||
algo: dtString("crc16-ccitt"),
|
||||
value: Buffer.alloc(2),
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
: {}),
|
||||
properties: {
|
||||
compression: dtString("none"),
|
||||
load: item.load,
|
||||
...("arch" in item ? { arch: dtString(item.arch) } : {}),
|
||||
...("entry" in item ? { entry: item.entry } : {}),
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
const reply = await cli.run({
|
||||
arguments: ["inspect-artifact", source, "--json"],
|
||||
});
|
||||
expect(reply.exitCode, reply.stdout + reply.stderr).toBe(0);
|
||||
const inspection = artifactInspectionResultSchema.parse(
|
||||
parseEvidence(reply.json).normalized_result,
|
||||
);
|
||||
expect(
|
||||
inspection.next_probes.some((probe) => probe.operation === "open_binary"),
|
||||
).toBe(item.allowed);
|
||||
const graph = artifactInventoryResultSchema.parse(
|
||||
inspection.substeps[0]?.evidence.normalized_result,
|
||||
);
|
||||
expect(
|
||||
graph.occurrences.find((row) => row.logical_path === "images/kernel.bin")
|
||||
?.declared_size,
|
||||
).toBe(item.bytes);
|
||||
});
|
||||
@@ -51,6 +51,11 @@ import {
|
||||
rpathCommand,
|
||||
codeSignatureCommand,
|
||||
} from "../../../src/artifacts/apple/MachoImage.fixture.js";
|
||||
import {
|
||||
FIT_EMBEDDED,
|
||||
FIT_EXTERNAL,
|
||||
} from "../../../src/artifacts/fit/FitImage.fixture.js";
|
||||
import { devicetreeFixture } from "../../../src/firmware/devicetree/Devicetree.fixture.js";
|
||||
|
||||
it.each([
|
||||
[4000, false],
|
||||
@@ -591,6 +596,30 @@ describe("binary target I/O: package classification", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("classifies a FIT by its /images node and leaves a plain DTB rejected", async () => {
|
||||
const directory = await createTestTempDirectory("rea-fit-target-");
|
||||
const itb = join(directory, "boot.itb");
|
||||
const extensionless = join(directory, "boot");
|
||||
const dtb = join(directory, "board.dtb");
|
||||
await Promise.all([
|
||||
writeFile(itb, FIT_EMBEDDED),
|
||||
writeFile(extensionless, FIT_EXTERNAL),
|
||||
writeFile(dtb, devicetreeFixture()),
|
||||
]);
|
||||
for (const path of [itb, extensionless])
|
||||
expect(await parseBinaryTarget(path)).toMatchObject({
|
||||
ok: true,
|
||||
value: { kind: "archive", format: "fit" },
|
||||
});
|
||||
expect(await parseBinaryTarget(dtb)).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "BinaryTargetError",
|
||||
constraint: "file_format_rejected",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
describe("binary target I/O: filesystem and app bundle target resolution", () => {
|
||||
it("resolves relative Hopper databases and rejects unknown or unreadable paths", async () => {
|
||||
const directory = await createTestTempDirectory("rea-target-");
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
import { readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import {
|
||||
FIT_EXTERNAL,
|
||||
FIT_KERNEL,
|
||||
} from "../../../src/artifacts/fit/FitImage.fixture.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { artifactExtractionResultSchema } from "../../../src/domain/artifactGraph.js";
|
||||
|
||||
// The compiled stdio fixture checks advertised schemas against their dialect and
|
||||
// validates actual tool responses, avoiding a second source server module graph.
|
||||
it.each(["full", "compact"] as const)(
|
||||
"exposes FIT facts and image extraction through MCP (%s)",
|
||||
async (profile) => {
|
||||
const root = await createTestTempDirectory("rea-fit-mcp-");
|
||||
const path = join(root, "boot.itb");
|
||||
await writeFile(path, FIT_EXTERNAL);
|
||||
const { call } = await connectLocalToolsMcp({
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
|
||||
});
|
||||
let outputRoot: string | undefined;
|
||||
try {
|
||||
const opened = await call("open_binary", { path });
|
||||
expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
|
||||
const reply = await call("inspect_artifact", { detail: "summary" });
|
||||
expect(reply.isError, JSON.stringify(reply)).not.toBe(true);
|
||||
const summary = z
|
||||
.object({
|
||||
parent_evidence_id: z.string(),
|
||||
summary: z.object({ root_format_metadata: z.unknown() }),
|
||||
})
|
||||
.parse(parseEvidence(reply.structuredContent).normalized_result);
|
||||
expect(summary.summary.root_format_metadata).toMatchObject({
|
||||
kind: "fit",
|
||||
default_configuration: "conf-1",
|
||||
});
|
||||
const retained = await call("inspect_analysis_view", {
|
||||
source: {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: summary.parent_evidence_id,
|
||||
},
|
||||
view: {
|
||||
kind: "inventory",
|
||||
collection: "occurrences",
|
||||
logical_paths: ["images/kernel.bin"],
|
||||
fields: ["format_metadata"],
|
||||
},
|
||||
});
|
||||
expect(retained.isError, JSON.stringify(retained)).not.toBe(true);
|
||||
expect(JSON.stringify(retained.structuredContent)).toContain("fit-image");
|
||||
const extracted = await call("extract_artifact", {});
|
||||
expect(extracted.isError, JSON.stringify(extracted)).not.toBe(true);
|
||||
const result = artifactExtractionResultSchema.parse(
|
||||
parseEvidence(extracted.structuredContent).normalized_result,
|
||||
);
|
||||
outputRoot = result.output_root;
|
||||
expect(
|
||||
Buffer.compare(
|
||||
await readFile(join(result.output_root, "images/kernel.bin")),
|
||||
FIT_KERNEL,
|
||||
),
|
||||
).toBe(0);
|
||||
await call("close_binary", {});
|
||||
const corrupted = Buffer.from(FIT_EXTERNAL);
|
||||
const kernel = corrupted.indexOf(FIT_KERNEL);
|
||||
corrupted[kernel] = (corrupted[kernel] ?? 0) ^ 1;
|
||||
await writeFile(path, corrupted);
|
||||
await call("open_binary", { path });
|
||||
const invalid = await call("inspect_artifact", {});
|
||||
expect(JSON.stringify(parseMcpToolError(invalid))).toContain(
|
||||
"FIT image kernel hash-1 crc32 does not match",
|
||||
);
|
||||
await call("close_binary", {});
|
||||
} finally {
|
||||
if (outputRoot !== undefined)
|
||||
await rm(outputRoot, { recursive: true, force: true });
|
||||
}
|
||||
},
|
||||
);
|
||||
Reference in new issue
Block a user