Policy: add tool metadata conformance (#80056)

* feat(policy): add tool metadata conformance checks

* Add policy trusted tool runtime gate

* Use requireMetadata for tool policy

Make tools.requireMetadata the canonical policy schema for risk, sensitivity, and owner requirements. Update runtime enforcement, doctor findings, evidence parsing, tests, and policy docs to use the new schema.

* fix(policy): persist approval metadata

* fix(policy): refresh approval metadata artifacts

* docs(policy): list all tool finding checks

* fix(policy): parse multiline tool metadata

* test(policy): cover unparseable policy check output

* fix(policy): resolve oc-path api in packaged dist

* fix(policy): clear post-rebase CI failures

* test(policy): clear post-rebase CI failures

* fix(policy): restore watch and align validation

* fix(policy): clear ci gate failures

* Simplify policy tool evidence parsing
This commit is contained in:
Gio Della-Libera
2026-05-20 20:47:32 -07:00
committed by GitHub
parent 6745fe8e70
commit a30ac3f8d7
7 changed files with 1320 additions and 41 deletions

View File

@@ -1,5 +1,5 @@
---
summary: "CLI reference for `openclaw policy` channel conformance checks"
summary: "CLI reference for `openclaw policy` conformance checks"
read_when:
- You want to check OpenClaw settings against an authored policy.jsonc
- You want policy findings in doctor lint
@@ -10,14 +10,23 @@ title: "Policy"
# `openclaw policy`
`openclaw policy` is provided by the bundled Policy plugin. Policy is an
enterprise conformance layer over existing OpenClaw settings: `policy.jsonc`
defines authored requirements, OpenClaw observes the active workspace as
evidence, and policy health checks report drift through `doctor --lint`.
enterprise conformance layer over existing OpenClaw settings. It does not add a
second configuration system. `policy.jsonc` defines authored requirements,
OpenClaw observes the active workspace as evidence, and policy health checks
report drift through `doctor --lint`. The final conformance signal is a clean
`doctor --lint` run; policy contributes findings to that shared lint surface
instead of creating a separate health gate.
This first policy slice manages configured channels. For example, IT can record
that Telegram is not approved, then `doctor --lint` reports any enabled Telegram
channel and `doctor --fix` can turn it off when workspace repairs are explicitly
enabled.
Policy currently manages configured channels and governed tool declarations.
For example, IT or a workspace operator can record that Telegram is not an
approved channel provider, require governed tools to carry risk and sensitivity
metadata, then use `doctor --lint` as the shared conformance gate.
Use policy when a workspace needs a durable statement such as "these channels
must not be enabled" or "governed tools must declare approval metadata" and a
repeatable way to prove that OpenClaw still conforms to that statement. Use
regular config and workspace docs alone when you only need local behavior and
do not need policy findings or attestation output.
## Quick start
@@ -32,7 +41,7 @@ arbitrary plugins. The plugin remains enabled if `policy.jsonc` is missing, so
doctor can report the missing artifact.
Policy is authored, not generated from the user's current settings. A minimal
channel policy looks like this:
policy for channels and tool metadata looks like this:
```jsonc
{
@@ -45,12 +54,16 @@ channel policy looks like this:
},
],
},
"tools": {
"requireMetadata": ["risk", "sensitivity", "owner"],
},
}
```
The rules are the authority. A category block is only a namespace; checks run
when a concrete rule is present. OpenClaw reads current `channels.*` settings
and reports settings that do not conform.
and `TOOLS.md` declarations as evidence, then reports observed state that does
not conform.
Run policy-only checks during authoring:
@@ -122,12 +135,64 @@ Policy config lives under `plugins.entries.policy.config`.
Set `plugins.entries.policy.config.enabled` to `false` to disable policy checks
for a workspace while leaving the plugin installed.
Tool metadata requirements are authored in `policy.jsonc` with
`tools.requireMetadata`, for example `["risk", "sensitivity", "owner"]`.
## Accept policy state
The attestation hash identifies the stable claim: policy hash, evidence hash,
findings hash, and whether the result was clean. It intentionally does not
include `checkedAt`, so the same policy state produces the same attestation
across repeated checks.
Example JSON output:
```json
{
"ok": true,
"attestation": {
"checkedAt": "2026-05-10T20:00:00.000Z",
"policy": {
"path": "policy.jsonc",
"hash": "sha256:..."
},
"workspace": {
"scope": "policy",
"hash": "sha256:..."
},
"findingsHash": "sha256:...",
"attestationHash": "sha256:..."
},
"evidence": {
"channels": [
{
"id": "telegram",
"provider": "telegram",
"source": "oc://openclaw.config/channels/telegram",
"enabled": false
}
],
"tools": [
{
"id": "deploy",
"source": "oc://TOOLS.md/tools/deploy",
"line": 12,
"risk": "critical",
"sensitivity": "restricted",
"capabilities": ["IRREVERSIBLE_EXTERNAL"]
}
]
},
"checksRun": 6,
"checksSkipped": 0,
"findings": []
}
```
The policy hash identifies the authored rule artifact. The evidence block
records the observed OpenClaw state used by the policy checks. The
`workspace.hash` value identifies that evidence payload for the checked scope.
The findings hash identifies the exact finding set returned by the check.
`checkedAt` records when the evaluation ran. The attestation hash identifies
the stable claim: policy hash, evidence hash, findings hash, and whether the
result was clean. It intentionally does not include `checkedAt`, so the same
policy state produces the same attestation across repeated checks. Together,
these form the audit tuple for this policy check.
If a later gateway or supervisor uses policy to block, approve, or annotate a
runtime action, it should record the attestation hash from the last clean policy
@@ -146,20 +211,71 @@ If policy rules change intentionally, update both accepted hashes from a clean
check. If workspace settings change intentionally but policy stays the same,
only `expectedAttestationHash` usually changes.
`openclaw policy watch` runs the same check repeatedly and reports when the
current evidence no longer matches `expectedAttestationHash`:
```bash
openclaw policy watch --json
```
Use `--once` in CI or scripts that only need one drift evaluation. Without
`--once`, the command polls every two seconds by default; use `--interval-ms` to
choose a different interval.
## Findings
Policy currently verifies:
| Check id | Finding |
| ---------------------------------- | ------------------------------------------------------------------- |
| `policy/policy-jsonc-missing` | Policy is enabled but `policy.jsonc` is missing. |
| `policy/policy-jsonc-invalid` | Policy cannot be parsed or has malformed rules. |
| `policy/policy-hash-mismatch` | Policy does not match configured `expectedHash`. |
| `policy/attestation-hash-mismatch` | Current policy evidence no longer matches the accepted attestation. |
| `policy/channels-denied-provider` | An enabled channel matches a channel deny rule. |
| Check id | Finding |
| ---------------------------------------- | ------------------------------------------------------------------- |
| `policy/policy-jsonc-missing` | Policy is enabled but `policy.jsonc` is missing. |
| `policy/policy-jsonc-invalid` | Policy cannot be parsed or has malformed rules. |
| `policy/policy-hash-mismatch` | Policy does not match configured `expectedHash`. |
| `policy/attestation-hash-mismatch` | Current policy evidence no longer matches the accepted attestation. |
| `policy/channels-denied-provider` | An enabled channel matches a channel deny rule. |
| `policy/tools-missing-owner` | A governed tool declaration is missing owner metadata. |
| `policy/tools-missing-risk-level` | A governed tool declaration is missing risk metadata. |
| `policy/tools-missing-sensitivity-token` | A governed tool declaration is missing sensitivity metadata. |
| `policy/tools-unknown-risk-level` | A governed tool declaration uses an unknown risk value. |
| `policy/tools-unknown-sensitivity-token` | A governed tool declaration uses an unknown sensitivity value. |
Policy findings can include `target` and `requirement`: the observed workspace
thing that does not conform, and the authored rule that made it a finding.
Policy findings can include both `target` and `requirement`. `target` is the
observed workspace thing that does not conform. `requirement` is the authored
policy rule that made it a finding. Both values are addresses today, usually
`oc://` paths, but the field names describe their policy role rather than the
address format.
Example JSON finding:
```json
{
"checkId": "policy/channels-denied-provider",
"severity": "error",
"message": "Channel 'telegram' uses denied provider 'telegram'.",
"source": "policy",
"path": "openclaw config",
"ocPath": "oc://openclaw.config/channels/telegram",
"target": "oc://openclaw.config/channels/telegram",
"requirement": "oc://policy.jsonc/channels/denyRules/#0",
"fixHint": "Telegram is not approved for this workspace."
}
```
Example tool finding:
```json
{
"checkId": "policy/tools-missing-risk-level",
"severity": "error",
"message": "TOOLS.md tool 'deploy' has no explicit risk classification.",
"source": "policy",
"path": "TOOLS.md",
"line": 12,
"ocPath": "oc://TOOLS.md/tools/deploy",
"target": "oc://TOOLS.md/tools/deploy",
"requirement": "oc://policy.jsonc/tools/requireMetadata"
}
```
## Repair
@@ -190,5 +306,12 @@ configured channel:
## Exit codes
`policy check` exits `0` when there are no findings at the threshold, `1` when
findings are present, and `2` for argument or runtime failures.
| Command | `0` | `1` | `2` |
| -------------- | ----------------------------------------- | ------------------------------------------------ | ---------------------------- |
| `policy check` | No findings at the threshold. | One or more findings met the threshold. | Argument or runtime failure. |
| `policy watch` | No findings and accepted hash is current. | Findings exist or accepted attestation is stale. | Argument or runtime failure. |
## Related
- [Doctor lint mode](/cli/doctor#lint-mode)
- [Path CLI](/cli/path)

View File

@@ -3,7 +3,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { clearConfigCache } from "openclaw/plugin-sdk/runtime-config-snapshot";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { policyCheckCommand } from "./cli.js";
import { policyCheckCommand, policyWatchCommand } from "./cli.js";
import { resetPolicyDoctorChecksForTest } from "./doctor/register.js";
import {
policyAttestationHash,
@@ -30,6 +30,25 @@ async function runPolicyCheckJson(options: Parameters<typeof policyCheckCommand>
return { exitCode, parsed: JSON.parse(output.at(-1) ?? "{}"), output };
}
async function runPolicyWatchJson(options: Parameters<typeof policyWatchCommand>[0] = {}) {
const output: string[] = [];
const exitCode = await policyWatchCommand(
{ cwd: workspaceDir, json: true, once: true, ...options },
{
writeStdout(value) {
output.push(value);
},
error(value) {
output.push(value);
},
async sleep() {
throw new Error("policy watch should not sleep in --once mode");
},
},
);
return { exitCode, parsed: JSON.parse(output.at(-1) ?? "{}"), output };
}
describe("policy commands", () => {
beforeEach(async () => {
workspaceDir = await fs.mkdtemp(join(tmpdir(), "policy-cli-"));
@@ -102,6 +121,39 @@ describe("policy commands", () => {
});
});
it("reports malformed policy containers in policy check output", async () => {
await fs.writeFile(join(workspaceDir, "policy.jsonc"), JSON.stringify({ tools: [] }), "utf-8");
const { exitCode, parsed } = await runPolicyCheckJson();
expect(exitCode).toBe(1);
expect(parsed).toMatchObject({
ok: false,
findings: [
{
checkId: "policy/policy-jsonc-invalid",
target: "oc://policy.jsonc/tools",
},
],
});
});
it("reports unparseable policy files in policy check output", async () => {
await fs.writeFile(join(workspaceDir, "policy.jsonc"), "{ channels: ", "utf-8");
const { exitCode, parsed } = await runPolicyCheckJson();
expect(exitCode).toBe(1);
expect(parsed).toMatchObject({
ok: false,
findings: [
{
checkId: "policy/policy-jsonc-invalid",
severity: "error",
target: "oc://policy.jsonc",
},
],
});
});
it("links policy findings to evidence and policy requirement refs", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
vi.stubEnv("OPENCLAW_CONFIG_PATH", configPath);
@@ -193,6 +245,92 @@ describe("policy commands", () => {
);
});
it("reports stale accepted attestations in policy watch", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
vi.stubEnv("OPENCLAW_CONFIG_PATH", configPath);
await fs.writeFile(
configPath,
JSON.stringify({
plugins: {
entries: {
policy: {
enabled: true,
config: { enabled: true, expectedAttestationHash: "sha256:not-current" },
},
},
},
}),
"utf-8",
);
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ channels: { denyRules: [] } }),
"utf-8",
);
const { exitCode, parsed } = await runPolicyWatchJson();
expect(exitCode).toBe(1);
expect(parsed).toMatchObject({
status: "stale",
expectedAttestationHash: "sha256:not-current",
findings: [
{
checkId: "policy/attestation-hash-mismatch",
},
],
});
});
it("reports findings instead of stale when policy watch has no attestation to compare", async () => {
await fs.writeFile(join(workspaceDir, "policy.jsonc"), "{ channels: ", "utf-8");
const { exitCode, parsed } = await runPolicyWatchJson();
expect(exitCode).toBe(1);
expect(parsed).toMatchObject({
status: "findings",
findings: [
{
checkId: "policy/policy-jsonc-invalid",
},
],
});
});
it("reports findings before stale when accepted attestation exists", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
vi.stubEnv("OPENCLAW_CONFIG_PATH", configPath);
await fs.writeFile(
configPath,
JSON.stringify({
plugins: {
entries: {
policy: {
enabled: true,
config: { enabled: true, expectedAttestationHash: "sha256:not-current" },
},
},
},
}),
"utf-8",
);
await fs.writeFile(join(workspaceDir, "policy.jsonc"), "{ channels: ", "utf-8");
const { exitCode, parsed } = await runPolicyWatchJson();
expect(exitCode).toBe(1);
expect(parsed).toMatchObject({
status: "findings",
expectedAttestationHash: "sha256:not-current",
findings: [
{
checkId: "policy/policy-jsonc-invalid",
},
],
});
});
it("rejects invalid severity thresholds", async () => {
const errors: string[] = [];

View File

@@ -1,3 +1,4 @@
import { setTimeout as sleep } from "node:timers/promises";
import type { Command } from "commander";
import {
exitCodeFromFindings,
@@ -15,6 +16,7 @@ import { createPolicyAttestation } from "./policy-state.js";
export type PolicyCommandRuntime = {
writeStdout(value: string): void;
error(value: string): void;
sleep?(ms: number): Promise<void>;
};
export interface PolicyCheckOptions {
@@ -23,6 +25,11 @@ export interface PolicyCheckOptions {
readonly cwd?: string;
}
export interface PolicyWatchOptions extends PolicyCheckOptions {
readonly intervalMs?: string | number;
readonly once?: boolean;
}
type PolicyCheckReport = {
readonly ok: boolean;
readonly attestation?: ReturnType<typeof createPolicyAttestation>;
@@ -41,6 +48,9 @@ const defaultRuntime: PolicyCommandRuntime = {
error(value) {
process.stderr.write(`${value}\n`);
},
sleep(ms) {
return sleep(ms);
},
};
export function registerPolicyCli(program: Command): void {
@@ -54,6 +64,17 @@ export function registerPolicyCli(program: Command): void {
.action(async (options: PolicyCheckOptions) => {
process.exitCode = await policyCheckCommand(options);
});
policy
.command("watch")
.description("Watch policy evidence and report accepted-attestation drift")
.option("--json", "Emit JSON output")
.option("--severity-min <severity>", "Minimum severity: info, warning, or error")
.option("--interval-ms <ms>", "Polling interval in milliseconds")
.option("--once", "Run one watch evaluation and exit")
.action(async (options: PolicyWatchOptions) => {
process.exitCode = await policyWatchCommand(options);
});
}
export async function policyCheckCommand(
@@ -70,6 +91,36 @@ export async function policyCheckCommand(
}
}
export async function policyWatchCommand(
options: PolicyWatchOptions,
runtime: PolicyCommandRuntime = defaultRuntime,
): Promise<number> {
try {
const intervalMs = normalizeWatchIntervalMs(options.intervalMs);
let previousKey: string | undefined;
for (;;) {
const report = await buildPolicyCheckReport(options, runtime);
const status = policyWatchStatus(report);
const key = `${status}:${report.attestation?.attestationHash ?? ""}:${report.exitCode}`;
if (previousKey === undefined || previousKey !== key || options.once === true) {
writePolicyWatchReport(report, status, options, runtime);
previousKey = key;
}
if (options.once === true) {
return status === "stale" ? 1 : report.exitCode;
}
if (runtime.sleep !== undefined) {
await runtime.sleep(intervalMs);
} else {
await sleep(intervalMs);
}
}
} catch (err) {
runtime.error(err instanceof Error ? err.message : String(err));
return 2;
}
}
async function buildPolicyCheckReport(
options: PolicyCheckOptions,
runtime: PolicyCommandRuntime,
@@ -204,6 +255,64 @@ function writePolicyCheckReport(
}
}
function writePolicyWatchReport(
report: PolicyCheckReport,
status: "clean" | "findings" | "stale",
options: PolicyWatchOptions,
runtime: PolicyCommandRuntime,
): void {
if (options.json === true || !process.stdout.isTTY) {
runtime.writeStdout(
JSON.stringify({
status,
ok: report.ok,
expectedAttestationHash: report.expectedAttestationHash,
attestation: report.attestation,
findings: report.findings,
}) + "\n",
);
return;
}
if (status === "stale") {
runtime.writeStdout(
`policy watch: accepted attestation is stale (current ${report.attestation?.attestationHash}, expected ${report.expectedAttestationHash}). Review policy check output, then update the supervisor/gateway accepted attestation.\n`,
);
return;
}
if (status === "findings") {
runtime.writeStdout(
`policy watch: ${report.findings.length} finding(s); accepted attestation cannot be updated until policy check is clean.\n`,
);
return;
}
runtime.writeStdout(
`policy watch: clean (attestation ${report.attestation?.attestationHash}, evidence ${report.attestation?.workspace.hash})\n`,
);
}
function policyWatchStatus(report: PolicyCheckReport): "clean" | "findings" | "stale" {
if (
!report.ok &&
report.findings.some((finding) => finding.checkId !== "policy/attestation-hash-mismatch")
) {
return "findings";
}
const expected = report.expectedAttestationHash?.trim();
if (
expected &&
report.attestation !== undefined &&
report.attestation.attestationHash !== expected
) {
return "stale";
}
return report.ok ? "clean" : "findings";
}
function normalizeWatchIntervalMs(value: string | number | undefined): number {
const raw = typeof value === "number" ? value : Number.parseInt(value ?? "", 10);
return Number.isFinite(raw) && raw >= 250 ? raw : 2000;
}
function toJsonFinding(finding: HealthFinding): Record<string, unknown> {
return {
checkId: finding.checkId,

View File

@@ -2,6 +2,7 @@ import { promises as fs } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
runDoctorLintChecks,
type HealthCheck,
type HealthCheckContext,
type HealthFinding,
@@ -9,7 +10,11 @@ import {
type OpenClawConfig,
} from "openclaw/plugin-sdk/health";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { createPolicyAttestation, policyDocumentHash } from "../policy-state.js";
import {
collectPolicyEvidence,
createPolicyAttestation,
policyDocumentHash,
} from "../policy-state.js";
import { registerPolicyDoctorChecks, resetPolicyDoctorChecksForTest } from "./register.js";
let workspaceDir: string;
@@ -83,6 +88,7 @@ async function runDeniedChannelRepair(repairCheckCtx: HealthRepairContext) {
describe("registerPolicyDoctorChecks", () => {
beforeEach(async () => {
resetPolicyDoctorChecksForTest();
workspaceDir = await fs.mkdtemp(join(tmpdir(), "policy-doctor-"));
});
@@ -106,11 +112,16 @@ describe("registerPolicyDoctorChecks", () => {
"policy/policy-hash-mismatch",
"policy/attestation-hash-mismatch",
"policy/channels-denied-provider",
"policy/tools-missing-risk-level",
"policy/tools-unknown-risk-level",
"policy/tools-missing-sensitivity-token",
"policy/tools-missing-owner",
"policy/tools-unknown-sensitivity-token",
]);
expect(duplicateChecks).toEqual([]);
});
it("reports a missing policy file when the policy extension is enabled", async () => {
it("reports a missing policy file when the Policy plugin is enabled", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
@@ -193,6 +204,29 @@ describe("registerPolicyDoctorChecks", () => {
]);
});
it.each([
["top-level array", [], "oc://policy.jsonc"],
["tools array", { tools: [] }, "oc://policy.jsonc/tools"],
["tools settings array", { tools: { settings: [] } }, "oc://policy.jsonc/tools/settings"],
["tools entries object", { tools: { entries: {} } }, "oc://policy.jsonc/tools/entries"],
["channels array", { channels: [] }, "oc://policy.jsonc/channels"],
])("reports malformed policy shape for %s", async (_label, policy, target) => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(join(workspaceDir, "policy.jsonc"), JSON.stringify(policy), "utf-8");
const result = await runPolicyChecks(ctx(configPath, cfgWithPolicy()));
expect(result.findings).toEqual([
expect.objectContaining({
checkId: "policy/policy-jsonc-invalid",
severity: "error",
path: "policy.jsonc",
target,
}),
]);
});
it("reports a policy hash mismatch when expectedHash is configured", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
@@ -328,7 +362,7 @@ describe("registerPolicyDoctorChecks", () => {
checkedAt: "2026-05-10T20:00:00.000Z",
policyPath: "policy.jsonc",
policyHash,
evidence: { channels: [] },
evidence: collectPolicyEvidence({}),
findings: [],
}).attestationHash;
await fs.writeFile(configPath, "{}", "utf-8");
@@ -341,6 +375,29 @@ describe("registerPolicyDoctorChecks", () => {
expect(result.findings).toEqual([]);
});
it("does not include unrelated TOOLS.md evidence in channel-only attestations", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
const policy = { channels: { denyRules: [] } };
const policyHash = policyDocumentHash(policy);
const acceptedAttestationHash = createPolicyAttestation({
ok: true,
checkedAt: "2026-05-10T20:00:00.000Z",
policyPath: "policy.jsonc",
policyHash,
evidence: collectPolicyEvidence({}),
findings: [],
}).attestationHash;
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(join(workspaceDir, "policy.jsonc"), JSON.stringify(policy), "utf-8");
await fs.writeFile(join(workspaceDir, "TOOLS.md"), "## Tools\n\n### deploy\n", "utf-8");
const result = await runPolicyChecks(
ctx(configPath, cfgWithPolicy({ expectedAttestationHash: acceptedAttestationHash })),
);
expect(result.findings).toEqual([]);
});
it("reports configured channels denied by policy", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
const cfg = {
@@ -521,4 +578,267 @@ describe("registerPolicyDoctorChecks", () => {
expect(result.findings).toEqual([]);
});
it("reports invalid requireMetadata policy entries", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["risk", "unsupported"] } }),
"utf-8",
);
await fs.writeFile(join(workspaceDir, "TOOLS.md"), "## Tools\n\n### deploy\n", "utf-8");
const result = await runDoctorLintChecks(ctx(configPath, cfgWithPolicy()), {
checks: registerChecks(),
});
expect(result.findings).toEqual([
expect.objectContaining({
checkId: "policy/policy-jsonc-invalid",
severity: "error",
path: "policy.jsonc",
target: "oc://policy.jsonc/tools/requireMetadata/#1",
}),
]);
});
it("reports blank requireMetadata policy entries", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["risk", " "] } }),
"utf-8",
);
const result = await runDoctorLintChecks(ctx(configPath, cfgWithPolicy()), {
checks: registerChecks(),
});
expect(result.findings).toEqual([
expect.objectContaining({
checkId: "policy/policy-jsonc-invalid",
severity: "error",
path: "policy.jsonc",
target: "oc://policy.jsonc/tools/requireMetadata/#1",
}),
]);
});
it("reports invalid requireMetadata entries against a configured policy path", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "workspace.policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["unsupported"] } }),
"utf-8",
);
const result = await runDoctorLintChecks(
ctx(configPath, cfgWithPolicy({ path: "workspace.policy.jsonc" })),
{
checks: registerChecks(),
},
);
expect(result.findings).toEqual([
expect.objectContaining({
checkId: "policy/policy-jsonc-invalid",
path: "workspace.policy.jsonc",
target: "oc://workspace.policy.jsonc/tools/requireMetadata/#0",
}),
]);
});
it("reports governed tools missing risk and sensitivity metadata", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["risk", "sensitivity", "owner"] } }),
"utf-8",
);
await fs.writeFile(join(workspaceDir, "TOOLS.md"), "## Tools\n\n### deploy\n", "utf-8");
const result = await runDoctorLintChecks(ctx(configPath, cfgWithPolicy()), {
checks: registerChecks(),
});
expect(result.findings).toHaveLength(3);
expect(result.findings).toEqual(
expect.arrayContaining([
expect.objectContaining({
checkId: "policy/tools-missing-risk-level",
severity: "error",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
expect.objectContaining({
checkId: "policy/tools-missing-sensitivity-token",
severity: "error",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
expect.objectContaining({
checkId: "policy/tools-missing-owner",
severity: "error",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
]),
);
});
it("reports governed bullet tools missing required metadata", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["risk", "sensitivity", "owner"] } }),
"utf-8",
);
await fs.writeFile(join(workspaceDir, "TOOLS.md"), "## Tools\n\n- deploy: deploys\n", "utf-8");
const result = await runDoctorLintChecks(ctx(configPath, cfgWithPolicy()), {
checks: registerChecks(),
});
expect(result.findings).toHaveLength(3);
expect(result.findings).toEqual(
expect.arrayContaining([
expect.objectContaining({
checkId: "policy/tools-missing-risk-level",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
expect.objectContaining({
checkId: "policy/tools-missing-sensitivity-token",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
expect.objectContaining({
checkId: "policy/tools-missing-owner",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
]),
);
});
it("accepts governed tool metadata declared on following lines", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["risk", "sensitivity", "owner"] } }),
"utf-8",
);
await fs.writeFile(
join(workspaceDir, "TOOLS.md"),
[
"## Tools",
"",
"### deploy",
"risk: critical",
"sensitivity: restricted",
"owner: ops",
"IRREVERSIBLE_EXTERNAL",
"",
"### inspect",
"risk: low",
"sensitivity: public",
"owner: support",
"",
].join("\n"),
"utf-8",
);
const result = await runDoctorLintChecks(ctx(configPath, cfgWithPolicy()), {
checks: registerChecks(),
});
const evidence = await collectPolicyEvidence(
{},
{
toolsRaw: await fs.readFile(join(workspaceDir, "TOOLS.md"), "utf-8"),
},
);
expect(result.findings).toEqual([]);
expect(evidence.tools).toEqual([
{
id: "deploy",
source: "oc://TOOLS.md/tools/deploy",
line: 3,
risk: "critical",
sensitivity: "restricted",
owner: "ops",
capabilities: ["IRREVERSIBLE_EXTERNAL"],
},
{
id: "inspect",
source: "oc://TOOLS.md/tools/inspect",
line: 9,
risk: "low",
sensitivity: "public",
owner: "support",
},
]);
});
it("reports unknown governed tool risk metadata", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["risk"] } }),
"utf-8",
);
await fs.writeFile(
join(workspaceDir, "TOOLS.md"),
"## Tools\n\n### deploy risk:critcal\n",
"utf-8",
);
const result = await runDoctorLintChecks(ctx(configPath, cfgWithPolicy()), {
checks: registerChecks(),
});
expect(result.findings).toEqual([
expect.objectContaining({
checkId: "policy/tools-unknown-risk-level",
severity: "error",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
]);
});
it("reports unknown governed tool sensitivity metadata", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");
await fs.writeFile(
join(workspaceDir, "policy.jsonc"),
JSON.stringify({ tools: { requireMetadata: ["sensitivity"] } }),
"utf-8",
);
await fs.writeFile(
join(workspaceDir, "TOOLS.md"),
"## Tools\n\n### deploy risk:critical sensitivity:secret\n",
"utf-8",
);
const result = await runDoctorLintChecks(ctx(configPath, cfgWithPolicy()), {
checks: registerChecks(),
});
expect(result.findings).toEqual([
expect.objectContaining({
checkId: "policy/tools-unknown-sensitivity-token",
severity: "error",
path: "TOOLS.md",
ocPath: "oc://TOOLS.md/tools/deploy",
}),
]);
});
});

View File

@@ -19,6 +19,11 @@ const CHECK_IDS = {
policyHashMismatch: "policy/policy-hash-mismatch",
policyInvalidFile: "policy/policy-jsonc-invalid",
policyMissingFile: "policy/policy-jsonc-missing",
policyMissingToolOwner: "policy/tools-missing-owner",
policyMissingToolRisk: "policy/tools-missing-risk-level",
policyMissingToolSensitivity: "policy/tools-missing-sensitivity-token",
policyUnknownToolRisk: "policy/tools-unknown-risk-level",
policyUnknownToolSensitivity: "policy/tools-unknown-sensitivity-token",
} as const;
export const POLICY_CHECK_IDS = [
@@ -27,8 +32,17 @@ export const POLICY_CHECK_IDS = [
CHECK_IDS.policyHashMismatch,
CHECK_IDS.policyAttestationMismatch,
CHECK_IDS.policyDeniedChannelProvider,
CHECK_IDS.policyMissingToolRisk,
CHECK_IDS.policyUnknownToolRisk,
CHECK_IDS.policyMissingToolSensitivity,
CHECK_IDS.policyMissingToolOwner,
CHECK_IDS.policyUnknownToolSensitivity,
] as const;
const KNOWN_RISK_LEVELS = ["low", "medium", "high", "critical"] as const;
const KNOWN_SENSITIVITY_LEVELS = ["public", "internal", "confidential", "restricted"] as const;
const SUPPORTED_TOOL_METADATA = ["risk", "sensitivity", "owner"] as const;
let registered = false;
const policyEvaluationCache = new WeakMap<HealthCheckContext, Promise<PolicyEvaluation>>();
@@ -58,6 +72,11 @@ export function registerPolicyDoctorChecks(host?: PolicyDoctorRegistrationHost):
registerHealthCheck(policyHashMismatchCheck);
registerHealthCheck(policyAttestationMismatchCheck);
registerHealthCheck(policyChannelsDeniedProviderCheck);
registerHealthCheck(policyToolsMissingRiskCheck);
registerHealthCheck(policyToolsUnknownRiskCheck);
registerHealthCheck(policyToolsMissingSensitivityCheck);
registerHealthCheck(policyToolsMissingOwnerCheck);
registerHealthCheck(policyToolsUnknownSensitivityCheck);
registered = true;
}
@@ -78,7 +97,7 @@ export function evaluatePolicy(ctx: HealthCheckContext): Promise<PolicyEvaluatio
const policyMissingFileCheck: HealthCheck = {
id: CHECK_IDS.policyMissingFile,
kind: "plugin",
description: "The enabled policy extension has a policy file to verify.",
description: "The enabled Policy plugin has a policy file to verify.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingFile);
@@ -150,10 +169,60 @@ const policyChannelsDeniedProviderCheck: HealthCheck = {
},
};
const policyToolsMissingRiskCheck: HealthCheck = {
id: CHECK_IDS.policyMissingToolRisk,
kind: "plugin",
description: "TOOLS.md policy entries declare explicit risk levels.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingToolRisk);
},
};
const policyToolsUnknownRiskCheck: HealthCheck = {
id: CHECK_IDS.policyUnknownToolRisk,
kind: "plugin",
description: "TOOLS.md policy entries use known risk levels.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyUnknownToolRisk);
},
};
const policyToolsMissingSensitivityCheck: HealthCheck = {
id: CHECK_IDS.policyMissingToolSensitivity,
kind: "plugin",
description: "TOOLS.md policy entries declare default artifact sensitivity.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingToolSensitivity);
},
};
const policyToolsUnknownSensitivityCheck: HealthCheck = {
id: CHECK_IDS.policyUnknownToolSensitivity,
kind: "plugin",
description: "TOOLS.md policy entries use known sensitivity levels.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyUnknownToolSensitivity);
},
};
const policyToolsMissingOwnerCheck: HealthCheck = {
id: CHECK_IDS.policyMissingToolOwner,
kind: "plugin",
description: "TOOLS.md policy entries declare an accountable owner.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingToolOwner);
},
};
async function evaluatePolicyUncached(ctx: HealthCheckContext): Promise<PolicyEvaluation> {
const settings = policySettings(ctx);
const policyPath = policyDisplayName(ctx);
const evidence = collectPolicyEvidence(ctx.cfg as Record<string, unknown>);
let evidence: PolicyEvidence = collectPolicyEvidence(ctx.cfg as Record<string, unknown>);
const findings: HealthFinding[] = [];
if (!policyChecksEnabled(ctx, settings)) {
@@ -171,7 +240,7 @@ async function evaluatePolicyUncached(ctx: HealthCheckContext): Promise<PolicyEv
findings.push({
checkId: CHECK_IDS.policyMissingFile,
severity: "warning",
message: `${policyPath} is missing for the enabled policy extension.`,
message: `${policyPath} is missing for the enabled Policy plugin.`,
source: "policy",
path: policyPath,
fixHint: `Restore ${policyPath} or add the policy artifact for this workspace.`,
@@ -225,12 +294,34 @@ async function evaluatePolicyUncached(ctx: HealthCheckContext): Promise<PolicyEv
};
}
const policyFindings = channelFindings(
const metadataRequirementFindings = toolMetadataRequirementFindings(
policy,
policyFile.displayName,
policyFile.ocDocName,
evidence,
);
const requiredMetadata =
metadataRequirementFindings.length === 0 ? requiredToolMetadata(policy) : new Set<string>();
if (requiredMetadata.size > 0) {
const toolsFile = await readWorkspaceFile(ctx, "TOOLS.md");
evidence = await collectPolicyEvidence(ctx.cfg as Record<string, unknown>, {
toolsRaw: toolsFile?.raw ?? "",
});
}
const policyFindings: HealthFinding[] = [
...policyContainerShapeFindings(policy, policyFile.displayName, policyFile.ocDocName),
...channelFindings(policy, policyFile.displayName, policyFile.ocDocName, evidence),
...metadataRequirementFindings,
];
if (requiredMetadata.has("risk")) {
policyFindings.push(...toolRiskFindings(policyFile.ocDocName, evidence));
policyFindings.push(...toolUnknownRiskFindings(policyFile.ocDocName, evidence));
}
if (requiredMetadata.has("sensitivity")) {
policyFindings.push(...toolSensitivityFindings(policyFile.ocDocName, evidence));
}
if (requiredMetadata.has("owner")) {
policyFindings.push(...toolOwnerFindings(policyFile.ocDocName, evidence));
}
const attestationFindings = policyAttestationFindings(
policyFile.displayName,
policyHash,
@@ -375,6 +466,127 @@ function toAttestedFinding(finding: HealthFinding): Record<string, unknown> {
};
}
function toolMetadataRequirementFindings(
policy: unknown,
policyPath: string,
policyDocName: string,
): readonly HealthFinding[] {
if (!isRecord(policy) || !isRecord(policy.tools) || policy.tools.requireMetadata === undefined) {
return [];
}
if (!Array.isArray(policy.tools.requireMetadata)) {
return [
{
checkId: CHECK_IDS.policyInvalidFile,
severity: "error",
message: `${policyPath} tools.requireMetadata must be an array of metadata keys.`,
source: "policy",
path: policyPath,
target: `oc://${policyDocName}/tools/requireMetadata`,
fixHint: `Use supported metadata keys: ${SUPPORTED_TOOL_METADATA.join(", ")}.`,
},
];
}
const invalidIndex = policy.tools.requireMetadata.findIndex(
(entry) =>
typeof entry !== "string" ||
!SUPPORTED_TOOL_METADATA.includes(
entry.trim().toLowerCase() as (typeof SUPPORTED_TOOL_METADATA)[number],
),
);
if (invalidIndex < 0) {
return [];
}
return [
{
checkId: CHECK_IDS.policyInvalidFile,
severity: "error",
message: `${policyPath} tools.requireMetadata[${invalidIndex}] must be a supported metadata key.`,
source: "policy",
path: policyPath,
target: `oc://${policyDocName}/tools/requireMetadata/#${invalidIndex}`,
fixHint: `Use supported metadata keys: ${SUPPORTED_TOOL_METADATA.join(", ")}.`,
},
];
}
function policyContainerShapeFindings(
policy: unknown,
policyPath: string,
policyDocName: string,
): readonly HealthFinding[] {
if (!isRecord(policy)) {
return [
policyShapeFinding(
policyPath,
`oc://${policyDocName}`,
`${policyPath} must contain a policy object.`,
`Fix ${policyPath} so the top-level policy is an object.`,
),
];
}
if (policy.tools !== undefined && !isRecord(policy.tools)) {
return [
policyShapeFinding(
policyPath,
`oc://${policyDocName}/tools`,
`${policyPath} tools must be an object.`,
`Fix ${policyPath} so tools is an object.`,
),
];
}
if (isRecord(policy.tools)) {
if (policy.tools.settings !== undefined && !isRecord(policy.tools.settings)) {
return [
policyShapeFinding(
policyPath,
`oc://${policyDocName}/tools/settings`,
`${policyPath} tools.settings must be an object.`,
`Fix ${policyPath} so tools.settings is an object.`,
),
];
}
if (policy.tools.entries !== undefined && !Array.isArray(policy.tools.entries)) {
return [
policyShapeFinding(
policyPath,
`oc://${policyDocName}/tools/entries`,
`${policyPath} tools.entries must be an array.`,
`Fix ${policyPath} so tools.entries is an array.`,
),
];
}
}
if (policy.channels !== undefined && !isRecord(policy.channels)) {
return [
policyShapeFinding(
policyPath,
`oc://${policyDocName}/channels`,
`${policyPath} channels must be an object.`,
`Fix ${policyPath} so channels is an object.`,
),
];
}
return [];
}
function policyShapeFinding(
policyPath: string,
target: string,
message: string,
fixHint: string,
): HealthFinding {
return {
checkId: CHECK_IDS.policyInvalidFile,
severity: "error",
message,
source: "policy",
path: policyPath,
target,
fixHint,
};
}
function invalidChannelDenyRuleFindings(
policy: unknown,
policyPath: string,
@@ -413,6 +625,122 @@ function invalidChannelDenyRuleFindings(
];
}
function toolRiskFindings(
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
return (evidence.tools ?? [])
.filter((tool) => tool.risk === undefined)
.map((tool): HealthFinding => {
return {
checkId: CHECK_IDS.policyMissingToolRisk,
severity: "error",
message: `TOOLS.md tool '${tool.id}' has no explicit risk classification.`,
source: "policy",
path: "TOOLS.md",
line: tool.line,
ocPath: tool.source,
target: tool.source,
requirement: `oc://${policyDocName}/tools/requireMetadata`,
fixHint:
"Declare risk:low, risk:medium, risk:high, risk:critical, or an R0-R5 review alias.",
};
});
}
function toolUnknownRiskFindings(
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
return (evidence.tools ?? [])
.filter(
(tool) =>
tool.risk !== undefined &&
!KNOWN_RISK_LEVELS.includes(tool.risk as (typeof KNOWN_RISK_LEVELS)[number]),
)
.map((tool): HealthFinding => {
return {
checkId: CHECK_IDS.policyUnknownToolRisk,
severity: "error",
message: `TOOLS.md tool '${tool.id}' declares unknown risk '${tool.risk}'.`,
source: "policy",
path: "TOOLS.md",
line: tool.line,
ocPath: tool.source,
target: tool.source,
requirement: `oc://${policyDocName}/tools/requireMetadata`,
fixHint: `Use one of: ${KNOWN_RISK_LEVELS.join(", ")}.`,
};
});
}
function toolSensitivityFindings(
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
return (evidence.tools ?? []).flatMap((tool): HealthFinding[] => {
if (tool.sensitivity === undefined) {
return [
{
checkId: CHECK_IDS.policyMissingToolSensitivity,
severity: "error",
message: `TOOLS.md tool '${tool.id}' has no declared artifact sensitivity.`,
source: "policy",
path: "TOOLS.md",
line: tool.line,
ocPath: tool.source,
target: tool.source,
requirement: `oc://${policyDocName}/tools/requireMetadata`,
fixHint: `Declare sensitivity as one of: ${KNOWN_SENSITIVITY_LEVELS.join(", ")}.`,
},
];
}
if (
KNOWN_SENSITIVITY_LEVELS.includes(
tool.sensitivity as (typeof KNOWN_SENSITIVITY_LEVELS)[number],
)
) {
return [];
}
return [
{
checkId: CHECK_IDS.policyUnknownToolSensitivity,
severity: "error",
message: `TOOLS.md tool '${tool.id}' declares unknown sensitivity '${tool.sensitivity}'.`,
source: "policy",
path: "TOOLS.md",
line: tool.line,
ocPath: tool.source,
target: tool.source,
requirement: `oc://${policyDocName}/tools/requireMetadata`,
fixHint: `Use one of: ${KNOWN_SENSITIVITY_LEVELS.join(", ")}.`,
},
];
});
}
function toolOwnerFindings(
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
return (evidence.tools ?? [])
.filter((tool) => tool.owner === undefined)
.map((tool): HealthFinding => {
return {
checkId: CHECK_IDS.policyMissingToolOwner,
severity: "error",
message: `TOOLS.md tool '${tool.id}' has no declared owner.`,
source: "policy",
path: "TOOLS.md",
line: tool.line,
ocPath: tool.source,
target: tool.source,
requirement: `oc://${policyDocName}/tools/requireMetadata`,
fixHint: "Declare owner:<team-or-person> for this tool.",
};
});
}
async function readPolicyFile(
ctx: HealthCheckContext,
): Promise<{ raw: string; path: string; displayName: string; ocDocName: string } | null> {
@@ -434,6 +762,22 @@ async function readPolicyFile(
}
}
async function readWorkspaceFile(
ctx: HealthCheckContext,
fileName: string,
): Promise<{ raw: string; path: string } | null> {
const path = resolveWorkspacePath(ctx, fileName);
try {
const fs = await import("node:fs/promises");
return { raw: await fs.readFile(path, "utf-8"), path };
} catch (err) {
if (isNotFound(err)) {
return null;
}
throw err;
}
}
function resolveWorkspacePath(ctx: HealthCheckContext, fileName: string): string {
if (isAbsolute(fileName)) {
return fileName;
@@ -603,6 +947,26 @@ function policyChecksEnabled(ctx: HealthCheckContext, settings: PolicySettings):
return settings.enabled !== false;
}
function requiredToolMetadata(policy: unknown): ReadonlySet<string> {
return new Set(readPolicyStringArray(policy, ["tools", "requireMetadata"]) ?? []);
}
function readPolicyStringArray(
policy: unknown,
path: readonly string[],
): readonly string[] | undefined {
let current: unknown = policy;
for (const part of path) {
if (!isRecord(current)) {
return undefined;
}
current = current[part];
}
if (!Array.isArray(current) || !current.every((entry) => typeof entry === "string")) {
return undefined;
}
return current.map((entry) => entry.trim().toLowerCase()).filter(Boolean);
}
function policyPathSetting(ctx: HealthCheckContext): string {
const configured = policySettings(ctx).path;
return typeof configured === "string" && configured.trim() !== ""
@@ -616,5 +980,5 @@ function policyDisplayName(ctx: HealthCheckContext): string {
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null;
return typeof value === "object" && value !== null && !Array.isArray(value);
}

View File

@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { scanPolicyChannels } from "./policy-state.js";
import { scanPolicyChannels, scanPolicyTools } from "./policy-state.js";
describe("scanPolicyChannels", () => {
it("ignores reserved channel config namespaces", () => {
@@ -26,4 +26,60 @@ describe("scanPolicyChannels", () => {
},
]);
});
it("does not treat channel arrays as channel config maps", () => {
expect(
scanPolicyChannels({
channels: [{ enabled: true }],
}),
).toEqual([]);
});
});
describe("scanPolicyTools", () => {
it("scans documented bullet tool declarations", async () => {
await expect(
scanPolicyTools(
[
"## Tools",
"- deploy_tool: risk: critical sensitivity: restricted owner: ops IRREVERSIBLE_EXTERNAL",
"- inspect: risk: low",
" sensitivity: public",
" owner: support",
].join("\n"),
),
).resolves.toEqual([
{
id: "deploy-tool",
source: "oc://TOOLS.md/tools/deploy-tool",
line: 2,
risk: "critical",
sensitivity: "restricted",
owner: "ops",
capabilities: ["IRREVERSIBLE_EXTERNAL"],
},
{
id: "inspect",
source: "oc://TOOLS.md/tools/inspect",
line: 3,
risk: "low",
sensitivity: "public",
owner: "support",
},
]);
});
it("does not treat indented metadata bullets as tool declarations", async () => {
await expect(
scanPolicyTools(["## Tools", "- deploy: risk: critical", " - owner: ops"].join("\n")),
).resolves.toEqual([
{
id: "deploy",
source: "oc://TOOLS.md/tools/deploy",
line: 2,
risk: "critical",
owner: "ops",
},
]);
});
});

View File

@@ -16,6 +16,7 @@ export type PolicyAttestation = {
export type PolicyEvidence = {
readonly channels: readonly PolicyChannelEvidence[];
readonly tools?: readonly PolicyToolEvidence[];
};
export type PolicyChannelEvidence = {
@@ -25,7 +26,20 @@ export type PolicyChannelEvidence = {
readonly enabled?: boolean;
};
export type PolicyToolEvidence = {
readonly id: string;
readonly source: string;
readonly line: number;
readonly risk?: string;
readonly sensitivity?: string;
readonly owner?: string;
readonly capabilities?: readonly string[];
};
const RESERVED_CHANNEL_CONFIG_KEYS = new Set(["defaults", "modelByChannel"]);
const NON_SLUG_CHARS = /[^a-z0-9-]+/g;
const COLLAPSE_HYPHENS = /-+/g;
const TRIM_HYPHENS = /^-+|-+$/g;
export function policyDocumentHash(policy: unknown): string {
return sha256(stableJson(policy));
@@ -82,10 +96,23 @@ export function createPolicyAttestation(input: {
};
}
export function collectPolicyEvidence(cfg: Record<string, unknown>): PolicyEvidence {
return {
channels: scanPolicyChannels(cfg),
};
export function collectPolicyEvidence(
cfg: Record<string, unknown>,
options?: { readonly toolsRaw?: undefined },
): PolicyEvidence;
export function collectPolicyEvidence(
cfg: Record<string, unknown>,
options: { readonly toolsRaw: string },
): Promise<PolicyEvidence>;
export function collectPolicyEvidence(
cfg: Record<string, unknown>,
options: { readonly toolsRaw?: string } = {},
): PolicyEvidence | Promise<PolicyEvidence> {
const channels = scanPolicyChannels(cfg);
if (options.toolsRaw === undefined) {
return { channels };
}
return scanPolicyTools(options.toolsRaw).then((tools) => ({ channels, tools }));
}
export function scanPolicyChannels(cfg: Record<string, unknown>): readonly PolicyChannelEvidence[] {
@@ -110,6 +137,148 @@ export function scanPolicyChannels(cfg: Record<string, unknown>): readonly Polic
});
}
export function scanPolicyTools(raw: string): Promise<readonly PolicyToolEvidence[]> {
return Promise.resolve(scanPolicyToolHeaders(raw));
}
function scanPolicyToolHeaders(raw: string): readonly PolicyToolEvidence[] {
const section = markdownSectionLines(raw, "tools");
if (section.length === 0) {
return [];
}
const tools: PolicyToolEvidence[] = [];
for (let index = 0; index < section.length; index += 1) {
const line = section[index]?.text ?? "";
const heading = /^###\s+([^\s#]+)(.*)$/.exec(line);
const bullet = /^[-*+]\s+([^:\s][^:]*?)\s*:(.*)$/.exec(line);
const match = heading ?? bullet;
if (match === null || slugify(match[1]).length === 0) {
continue;
}
const id = slugify(match[1]);
const entry: {
id: string;
source: string;
line: number;
risk?: string;
sensitivity?: string;
owner?: string;
capabilities?: readonly string[];
} = {
id,
source: `oc://TOOLS.md/tools/${id}`,
line: section[index]?.line ?? index + 1,
};
const metaLines = [match[2] ?? ""];
for (let metaIndex = index + 1; metaIndex < section.length; metaIndex += 1) {
const metaLine = section[metaIndex]?.text ?? "";
if (/^###\s+\S+/.test(metaLine.trim()) || /^[-*+]\s+[^:\s][^:]*?\s*:/.test(metaLine)) {
break;
}
metaLines.push(metaLine);
}
const meta = metaLines.join("\n");
const risk = riskFromMeta(meta);
const sensitivity = /\bsensitivity\s*:\s*([a-z0-9_-]+)\b/i.exec(meta)?.[1]?.toLowerCase();
const owner = /\bowner\s*:\s*([^\s#]+)\b/i.exec(meta)?.[1];
const capabilities = capabilityTokensFromMetaLines(metaLines);
if (risk !== undefined) {
entry.risk = risk;
}
if (sensitivity !== undefined) {
entry.sensitivity = sensitivity;
}
if (owner !== undefined) {
entry.owner = owner;
}
if (capabilities.length > 0) {
entry.capabilities = capabilities;
}
tools.push(entry);
}
return tools;
}
function markdownSectionLines(
raw: string,
sectionSlug: string,
): readonly { readonly line: number; readonly text: string }[] {
const lines = raw.split(/\r?\n/);
let sectionDepth: number | undefined;
const section: { line: number; text: string }[] = [];
for (let index = 0; index < lines.length; index += 1) {
const line = lines[index] ?? "";
const heading = /^(#{1,6})\s+(.+?)\s*#*\s*$/.exec(line);
if (heading !== null) {
const depth = heading[1]?.length ?? 0;
const slug = slugify(heading[2] ?? "");
if (sectionDepth !== undefined && depth <= sectionDepth) {
break;
}
if (sectionDepth !== undefined) {
section.push({ line: index + 1, text: line });
continue;
}
if (sectionDepth === undefined && slug === sectionSlug) {
sectionDepth = depth;
}
continue;
}
if (sectionDepth !== undefined) {
section.push({ line: index + 1, text: line });
}
}
return section;
}
function slugify(text: string): string {
return text
.toLowerCase()
.replace(/_/g, "-")
.replace(NON_SLUG_CHARS, "-")
.replace(COLLAPSE_HYPHENS, "-")
.replace(TRIM_HYPHENS, "");
}
function riskFromMeta(meta: string): string | undefined {
const namedRisk = /\brisk\s*:\s*([a-z0-9_-]+)\b/i.exec(meta)?.[1];
if (namedRisk !== undefined) {
return namedRisk.toLowerCase();
}
const alias = /\bR([0-5])\b/.exec(meta)?.[1];
switch (alias) {
case "0":
case "1":
return "low";
case "2":
case "3":
return "medium";
case "4":
return "high";
case "5":
return "critical";
default:
return undefined;
}
}
function capabilityTokensFromMetaLines(lines: readonly string[]): readonly string[] {
return lines.flatMap((line, index): string[] => {
const trimmed = line.trim();
if (trimmed.length === 0) {
return [];
}
const tokens = trimmed.match(/\b[A-Z][A-Z0-9_]{2,}\b/g) ?? [];
if (index === 0 || /\bcapabilities\s*:/i.test(trimmed)) {
return tokens;
}
const withoutTokens = tokens.reduce((remaining, token) => {
return remaining.replace(token, "");
}, trimmed);
return /^[\s,;:[\](){}#*_-]*$/.test(withoutTokens) ? tokens : [];
});
}
function configuredChannels(cfg: Record<string, unknown>): Record<string, unknown> {
return isRecord(cfg.channels) ? cfg.channels : {};
}
@@ -132,5 +301,5 @@ function stableJson(value: unknown): string {
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null;
return typeof value === "object" && value !== null && !Array.isArray(value);
}