dilim/rea
1
0 Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-11-06 05:17:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(environment): build agent environments from a prompt (#6946)

Browse Source 
* feat(environment): report what an environment serves with an environment-coverage view

A new inspect_analysis_view kind over capture Evidence. It projects the
state an environment derives with the given template rules and reports
each tool as answered from state, replayed with its view reason, a served
write, or a withheld write with its reason, the records behind it and a
hint about what to record next. The hints cover every write reason, so a
new reason without one fails to compile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): build an environment bundle in one command

rea environment-build <capture> [directory] inspects a HAR (or reads
capture Evidence), projects the environment manifest from the in-memory
Evidence, authenticates the pair, prepares the state and writes the bundle
environment-export writes. Every bundle file is byte-identical to the
three-step path for the same capture and options.

The report adds the capture and manifest identities, the coverage summary,
warnings (a loopback capture named after its file, credential-named body
members by pointer) and the next command. --dry-run writes nothing. The
default template rules are the four the published environments use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(environment): cover environment-build and the coverage view

Domain tests check the hint table against the state view's reasons, the
summary, and the view against the environment-state view it summarizes.
Service tests build the fixture capture and compare every file with the
three-step path, a dry run, loopback naming and the refusals. A CLI test
compares a HAR built in one step with the three CLI steps byte for byte,
and an MCP test projects the view from every source kind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(environment): build the real-agentenv lane's bundle with environment-build

The lane now runs the command users run: it builds the Grafana-shaped HAR
with environment-build and the packed checkout, checks the report, then
runs the bundle with agent-env as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): document environment-build and the coverage view

The agent environments guide opens with the one-step build: the report,
the hint for each withheld write reason, and what a bundle ships. The CLI
guide, the capture guide and the skill reference name the command and the
environment-coverage view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): harden environment-build after review

- The credential scan walks with its own stack, so a deeply nested body
  cannot crash the build, and it also reports credential-named header
  names, query names, form body members and string arrays.
- The build checks SIGINT between stages: cancelled before the bundle
  stage it writes nothing, during the write it names the written bundle.
- A dry run opens the package tarball, so it refuses what a build refuses.
- Repeated template rules are refused, a HAR saved as .json gets a hint,
  file-derived names follow the bundle name rules, `localhost.` counts as
  a loopback origin, and the next command is shell-quoted.
- --capture-timeout-ms passes through to the HAR inspection, and a
  missing directory is refused before inspection.
- The no-observation hint covers updates and deletes of entities the state
  never held; the coverage view states that it does not apply the
  serve-time admission checks; the inspect_analysis_view description
  names the view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): state the build's conditions and record its verification

The guides now state when the one-step bundle equals the three-step one,
which headers inspection excludes, what the credential warning scans, how
cancellation behaves, and that the coverage view does not apply the
serve-time admission checks. The spec folder records the census, the end
to end runs, the gates and the review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): dispatch a pending interrupt before each build stage check

The manifest, state and coverage stages run synchronously, so a SIGINT
that arrived during them was only dispatched once the bundle write had
begun. Each check now yields first, so an interrupt before the bundle
stage cancels the build with nothing written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(browser): send declared requests from a scenario page with a request step

A capture_browser_scenario step `request` sends one declared HTTP request
through the page's own fetch, with its cookies and session, the way the
application's frontend would, and reads the whole response so the capture
records the transaction like any other. The path must resolve to the start
URL's origin, and the page must be on that origin when the step runs.
Query values and header values may be literals or declared secrets, and a
header may read its value from a cookie or a local storage entry, which is
then redacted from later records like a declared secret. `expected_status`
fails the step on any other status. Scenarios without a request step are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(browser): verify request steps in a real browser through to a built environment

verify:browser:request records a write session against a fixture item API
that refuses writes without its CSRF cookie header and stored bearer
token. It checks that every request lands in its own step, that the page
values authenticated each write and appear in no record, header or body,
that an unexpected status fails the step and cancels the rest, and that
environment-build serves the recorded create, update and delete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): export environment bundles over MCP

export_environment_bundle writes the bundle environment-build writes, from
capture Evidence retained in the session or one capture Evidence file,
into an absolute output directory that must not exist and whose parent
must exist, as extract_firmware and export_web_scripts require; it never
creates the parent or reuses a directory. It returns the same report,
whose schema now lives in the domain so the CLI and the tool share it, and
dry_run reports without writing. A loopback capture without a file needs a
name. The build's refusal messages no longer name CLI flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan scenario credentials, own the dry-run tarball and serialize Evidence once

A browser scenario's retained header values and the query parameters of its recorded URLs are scanned for credential
names as a HAR's are, through a review projection of its records. A dry run admits and releases the package tarball
through the export's regular-file scope, so a close it cannot confirm keeps its owner, and checks for cancellation
after that last await. Evidence is written into one exactly sized buffer instead of chunks joined by a copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the review round's checks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skill): build agent environments from a prompt with request steps and the bundle tool

Add the environments reference to the skill and install it with setup: record
against a disposable instance, sign in through the UI, write with request
steps and read each write back, check coverage, re-record by withheld reason
and export the bundle. Route it from SKILL.md, name agent environments in the
skill description, and document the request step, the export tool and
recording writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(browser): split the request lane and header resolution into smaller functions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style(environment): format the build spec

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): say when an untimed capture learns no write clock

Coverage of a capture whose records are not all timed, which includes every
browser scenario capture, now states that the state learns no clock, so an
update stamped with a server-set modification time stays unexplained-change
however often it is recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skill): give agents a whole scenario shape with step timeouts

Agents guessed the scenario schema field by field and two hung on steps
without timeout_ms. The reference now starts from a complete scenario with a
timeout on every step, and says a scenario capture cannot serve updates the
application stamps with a modification time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(browser): keep request steps on their origin and their page values redacted

- Follow no redirects: a redirect fails the step and nothing reaches the
  location it names (checked in real Chrome), and check the page's origin
  again in the page as the request is sent.
- Send nothing once the step's time has run out or it was cancelled while
  header values were read; the in-page deadline is what remains of the step's.
- Keep every page value redacted under a repeated or colliding label, keep a
  value's first marker when it is read again, and leave values shorter than
  8 characters unredacted rather than rewriting ids, ports and hosts.
- Refuse a header name given twice, User-Agent (Chrome drops it) and line
  breaks in literal values and prefixes; fail before sending when a secret or
  page value holds them.
- Append query entries after the path's literal query without re-encoding it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): document request step limits and word bundle refusals for both surfaces

Document redirects, timeouts, page value redaction, HttpOnly cookie sources
and header refusals for request steps. Refusals shared by environment-build
and export_environment_bundle no longer name CLI flags, and the retained
loopback refusal no longer prints an empty file name. The skill reference
asks for read-backs after each write, shows the coverage view's source, names
package_tarball for unreleased builds and the browser path a scenario ships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan every form occurrence and MIME-less scenario bodies

The build's credential scan collapsed form bodies with Object.fromEntries, so a
repeated field kept only its last occurrence: `api_key=secret&api_key=` warned
about nothing although the Evidence ships the value, and repeated values were
undercounted. Form fields are now scanned as ordered name/value pairs, as
headers and query strings already are.

The review projection of a capture_browser_scenario dropped a request body
recorded without a media type, so the scan never saw it. Review keeps such a
body's text with an empty media type; the manifest projection still reads only
bodies it can classify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(browser): close request step edge cases from the second review

- Append query entries correctly after a path ending in "?", "&" or "#".
- Refuse 3xx expected statuses, which an unfollowed redirect can never
  report, and say a redirected write may already have been applied.
- Redact a page value also as fetch sends it, without surrounding spaces
  and tabs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): document page value body costs, redirects and MCP dry runs

Document that reading a page value shrinks the bodies a capture keeps under
the default allocation limit, that redirected writes may have landed, that
the MCP dry_run still checks its output directory and which refusals name
the tool, and keep the .har hint in the HAR-as-JSON refusal for both
surfaces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the prompt-to-environment verification

Gates, the real-browser lanes, five environments built from prompts by an
agent on disposable VMs and evaluated with two models, and the two reviews.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(skill): count rollback failures from the managed skill files

The environments reference made the skill eight files, so the recovery tests now derive the counts they expect from
MANAGED_SKILL_FILES instead of fixing them at six and seven.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(adb): restore the portable Java float pattern a merge rewrote

A lint autofix during a merge unescaped the hyphens in javaFloatValuePattern, which is invalid in a v-mode character
class, so the advertised start_adb_activity schema failed to compile in every JS regex mode. The file now matches main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(browser): show the request step inside a whole scenario

Every JSON example in the browser scenario guide is a capture_browser_scenario input the catalog test parses, so the
request step example now sits in a launch scenario instead of standing alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan structured HAR form params and the request side of unattributed scenario transactions

A HAR request recorded with postData.params and no text is scanned by those parameters, each occurrence counting, and
the review projection keeps the request side of a scenario transaction whose response cannot be attributed, so the
credential warning covers what evidence.json ships in both cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): validate build paths and read dry-run tarballs

* fix(environment): report bodies too large for the credential scan

Text bodies over the scan's 4 MiB limit, including a response decoded only as a prefix, are reported as a
credential-scan-incomplete warning by record and side, never by content, so an absent credential-named-members
warning no longer reads as a complete check. The spec and agent-environments guide also state the path validation and
dry-run tarball read that e52ae1cd added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(verification): release cold source transform services

* fix(javascript): retain capture context along covered mutation paths

* fix(environment): guard request dispatch and report export limits

* fix(environment): admit serialized exports and scan HAR form fallbacks

* fix(environment): admit in-memory build inputs against heap headroom and scan SaveHar null-text forms

The inspected HAR's Evidence and the projected manifest are checked against live heap headroom before they are
serialized, with the same check the file route uses, so a capture too large to parse and seal is refused as
resource_constraint before anything is written. A request whose postData text is SaveHar's null is scanned by its
structured params.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): measure the credential scan limit in UTF-8 bytes

A body counts against the 4 MiB limit by its UTF-8 length, not its UTF-16 code units, so multibyte text over the
limit is reported as unscanned instead of parsed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(browser): send request steps only before their deadline and keep page reads on the start origin

The page receives the step's absolute deadline and a one-second start window, so a busy page that runs the request
late, or after the step is cancelled, sends nothing; local storage is read only on the start origin; the response is
drained without keeping its body; an unparseable path is a field issue; retained Evidence is refused for heap headroom
before it is serialized; and the skill says how to reach server-assigned ids.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style: format integrated export admission

* fix(javascript): erase inline type imports before binding values

* fix(browser): run the request lane under the shared verifier command owner

The browser request lane runs each CLI call through createPackageVerifierCommands on a private runtime root, so a
timeout, output overflow or failure stops the CLI and the browser it launched before the root is removed, and a failed
stop is reported with the root retained. The verification record states the lane's current counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): scan scenario content by event and read bodies by their media type

A browser scenario's credential scan reads every retained network-content event, including a response attached
mid-flight and a transaction no record represents, with event pointers, and each transaction URL's query once; the
review record projection it replaces is removed. A body is parsed as a form or as JSON by its recorded media type,
falling back to its shape, so valid forms such as `password=x&` are scanned and sentences are not read as forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
Edgar ArakelyanClaude Opus 5.5Kaoru0822morluto
 authored and GitHub committed 2026-10-24 21:51:41 +08:00
1 parent d361c53c50
commit 02e049744f
75 files changed
+7034 -256
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.agents/skills/reverse-engineer-anything/SKILL.md
+8 -2
View File
@@ -1,8 +1,8 @@
---
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "46"
version: "48"
---
# REA
@@ -133,6 +133,9 @@ explicit path or endpoint and do not need it.
`capture_browser_scenario` with declared actions. See
[runtime observation](references/runtime-observation.md) for connect/launch
and the focused follow-up browser tools.
- Agent environment (AgentEnv bundle) of a web application: read
[environments](references/environments.md) before recording; it records with
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
@@ -471,6 +474,9 @@ as complete while required questions remain open.
[references/runtime-observation.md](references/runtime-observation.md)
- Evidence paging, comparisons, residual unknowns, and verification:
[references/evidence-workflows.md](references/evidence-workflows.md)
- Agent environments (AgentEnv bundles) of a web application, recorded from
a live instance or a HAR:
[references/environments.md](references/environments.md)
## Finish the task
.agents/skills/reverse-engineer-anything/references/environments.md
+215
View File
@@ -0,0 +1,215 @@
# Agent environments from a live application
Use this when the user asks for an agent environment (an AgentEnv bundle) of a
web application: a folder `agent-env` deploys, whose MCP tools answer from what
one recording of the application observed and serve the writes that recording
confirmed. The workflow records the application once with
`capture_browser_scenario`, checks what the environment would serve, records
again if a hint names a gap, then writes the bundle with
`export_environment_bundle`. For a HAR the user already has, skip to
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
`rea environment-build <capture.har> <dir>`.
## Before recording
- Record only against an instance the user says is disposable or seeded for
this purpose: every write step creates, edits or deletes its data. If the
user has not said so, ask before any scenario that writes. REA does not
start, seed or reset applications.
- Credentials come from environment variables the user names. Declare each as a
scenario secret, `{secret_id, environment_variable}`; the variable must be set
in the environment REA's MCP server started with. Never ask for a password in
chat. Declared secrets are redacted as the capture records.
- Map the application first with `inspect_web_page` (and `capture_web_screenshot`
when layout matters): the login form's locators, and the API calls behind the
pages that list and edit the entities the user asked about.
## Record a session an environment can learn writes from
One `capture_browser_scenario` records the whole session. Start from this
shape, with the application's own locators, paths and headers:
```json
{
"browser": { "mode": "launch", "executable_path": "/usr/bin/chromium" },
"start_url": { "url": "http://127.0.0.1:3000/login" },
"secrets": [
{ "secret_id": "admin_password", "environment_variable": "APP_PASSWORD" }
],
"capture": {
"network": {
"request_body": true,
"response_body": true,
"header_values": true
}
},
"actions": [
{
"step_id": "user",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=username]" },
"value": { "source": "literal", "value": "admin" }
},
{
"step_id": "password",
"action": "fill",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=password]" },
"value": { "source": "secret", "secret_id": "admin_password" }
},
{
"step_id": "sign-in",
"action": "press",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "input[name=password]" },
"key": "Enter"
},
{
"step_id": "signed-in",
"action": "wait_for",
"timeout_ms": 15000,
"locator": { "kind": "css", "selector": "nav a[href='/settings']" },
"state": "visible"
},
{
"step_id": "folders-1",
"action": "request",
"timeout_ms": 15000,
"method": "GET",
"path": "/api/folders",
"expected_status": [200]
},
{
"step_id": "create-folder-1",
"action": "request",
"timeout_ms": 15000,
"method": "POST",
"path": "/api/folders",
"json": { "title": "Scratch folder 1" },
"headers": [
{
"name": "X-CSRFToken",
"value": { "source": "cookie", "name": "csrftoken" }
}
],
"expected_status": [200, 201]
}
]
}
```
Give every step a `timeout_ms`. A step without one waits until the scenario
is cancelled, so a locator that never matches hangs the capture, and the
client with it. Match `wait_for` locators to exactly one element.
1. Sign in through the UI: `fill` the user and password fields (the password
as a secret reference), submit, then `wait_for` an element only a signed-in
user sees.
2. Read with `request` steps: each collection the environment needs, twice
before any write, so fields that change on their own show up; one item of
each.
3. Use `request` steps only when the API lets the caller choose each scratch
object's ID or a prior read already established it. Request paths are fixed
strings; no action substitutes a value from an earlier response. For
server-assigned IDs, create through the UI and open the new object's detail
view to record its read, edit and delete through UI actions. If that journey
cannot be scripted, leave the write unserved and report the limitation;
never guess an ID or create objects with no supported cleanup path.
Record two numbered scratch objects per write type,
with varied free text. After each write, and before the next one, read the
object back by id and then its list: create 1, read 1, list, create 2,
read 2, list; never several writes before their reads.
4. Leave more than a second between a create and its first edit
(`wait_for_timeout` with `duration_ms: 1100`).
5. Update both objects and read each back; delete both and read each back
(expect 404) and the list.
About `request` steps:
- `path` is on the start URL's origin; the page must be on that origin.
- Header values are literals, declared secrets, a cookie the browser holds
(`{source: "cookie", name}`), or a local storage entry
(`{source: "local_storage", key, json_pointer}`), with an optional
`prefix` such as `"Bearer "`. Copy the headers the application's own
requests send (a CSRF header, a bearer token). Values read from the page
are redacted from records made after they are read, not from the login
response that returned them, and values shorter than 8 characters are not
redacted at all.
- Reading a page value makes later bodies costlier to keep: after a CSRF
token, bodies over about 47 KiB are kept as `unavailable`. When the API
returns larger responses, raise `capture.limits`
(`max_network_body_allocation_bytes`, `max_total_event_allocation_bytes`).
- Request steps follow no redirects: a redirect fails the step, and
`expected_status` cannot name a 3xx status. Use the path the redirect names,
such as the one with the trailing slash.
- Set `expected_status` on every step, so a refused call fails its step and
the later steps report not having run, instead of recording read-backs of
nothing.
Keep it to about 60 to 160 requests. A scenario with no `request` step is
recorded exactly as before.
The capture returns its whole Evidence, every recorded body included, which is
often more than a client shows inline. Before checking coverage, confirm that
every entry of `normalized_result.steps` has `status: "completed"`; if the
client saved the result to a file, search it for `"status":"failed"` and read
that step's `error`. A failed sign-in or write cancels every step after it.
Use the result's `evidence_id` for coverage and export; do not paste the
Evidence back into another tool.
## Check coverage
`inspect_analysis_view` with `source: {kind: "retained-evidence",
evidence_id}` (the capture's id) and `view: {kind: "environment-coverage",
template_rules: ["after-uid-literal", "opaque-token", "sibling-variance",
"prefix-variance"]}`. It reports read tools
answered from the state or replayed (`reads.recorded_reasons`), the served write
tools, and each withheld write with its `reason` and `hint`. It does not apply
the serve-time catalog and state size checks; the export reports those.
Re-record a withheld write at most twice, and only for a recording gap. Stop at
once when the reason is application behaviour, and report the write as
withheld with its hint:
| Reason | What to do |
| --------------------- | --------------------------------------------------------------------------------------------- |
| `no-state-target` | Re-record: read the collection or item the write changes. |
| `no-observation` | Re-record: read the entity back by id after the write; read it first for an update or delete. |
| `unconfirmed` | Re-record: read the entity and its list after the write. |
| `incomplete-entities` | Re-record: read the entity through every view that lists it. |
| `unexplained-change` | Re-record once with a second, different call of the same write. |
| `no-call` | Re-record once with a write that succeeds. |
| `side-effect` | Stop: the application changes other entities. |
| `contradicted` | Stop: a later read disagrees with the modelled effect. |
| `body-unusable` | Stop: the body is not a JSON object. |
Use new scratch names on each attempt; the first attempt's objects may still
exist.
A browser scenario carries no request times, so the coverage limitations say
the state learns no clock: an update the application stamps with a
modification time stays `unexplained-change` however it is recorded. Report
it rather than re-recording for it.
## Export and hand off
`export_environment_bundle` with `source: {kind: "retained-evidence",
evidence_id}` (or `{kind: "evidence-file", path}`), an absolute
`output_directory` that does not exist yet and whose parent does, and a `name`
(required when the application runs on `localhost` or an IP address). It returns
the coverage again, `tool_count`, the written files and warnings; `dry_run:
true` reports without writing. The bundle installs `rea-agents` of this REA's
version from the registry; when the user runs an unreleased REA build, pass the
`package_tarball` they give you (a packed `rea-agents` tarball) instead.
Tell the user how many tools answer from the state, which writes are served and
which are withheld and why, and the run command (`agent-env run <folder>`). Say
that the bundle ships the recorded responses and the capture's local paths
(for a browser scenario, the browser executable's path), and
list any `credential-named-members` warning pointers for review. Never publish
or share the bundle yourself.
See
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
for what the state, replay and the bundle establish, and
[evidence workflows](evidence-workflows.md) for the manifest and state views.
.agents/skills/reverse-engineer-anything/references/evidence-workflows.md
+7
View File
@@ -106,6 +106,13 @@ each plan states why in its `reason`. It replays the capture instead with
`--replay-only` or when the state is unavailable (not derivable, or a catalog
over the discovery budget), and its stopped document's `state` says which
applied.
Use `view: {kind: "environment-coverage"}` with the same rules to see what an
environment built from a capture would serve: read tools answered from the
state or replayed with their reason, and each write tool served or withheld
with a hint about what to record next; it does not apply the serve-time
catalog and state size checks. `rea environment-build <capture.har> <dir>`
inspects, projects and exports in one step and prints the same coverage, or
`null` when the bundle serves replay.
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
and retains the complete table. Query `inspect_analysis_view` with
docs/agent-environments.md
+232 -9
View File
@@ -1,16 +1,20 @@
# Agent environments from recorded captures
Use `rea environment-serve` to serve an environment manifest as an agent
environment: an MCP server whose tools answer from what one retained web
network capture observed, plus the environment card the agentenv framework
reads to discover it. By default the server derives the
[state the capture observed](#the-state-a-capture-observed) at startup and
answers read tools from it, and the write tools whose effect the capture
confirmed by changing it; `--replay-only` replays the capture's records
Use `rea environment-build` to turn one recorded capture into an agent
environment bundle in one offline step: a folder `agent-env run` builds and
runs, whose MCP server answers from what the capture observed. See
[Build a bundle in one step](#build-a-bundle-in-one-step).
The steps it composes are separate commands too. `rea environment-serve` serves
an environment manifest as an agent environment: an MCP server whose tools
answer from what one retained web network capture observed, plus the
environment card the agentenv framework reads to discover it. By default the
server derives the [state the capture observed](#the-state-a-capture-observed)
at startup and answers read tools from it, and the write tools whose effect the
capture confirmed by changing it; `--replay-only` replays the capture's records
instead. Nothing is fetched from the recorded application; every answer comes
from retained Evidence. `rea environment-export` writes the same pair as a
bundle folder `agent-env run` builds and runs; see
[Export a bundle](#export-a-bundle).
bundle folder; see [Export a bundle](#export-a-bundle).
```sh
rea inspect-web-network-capture /absolute/session.har har --format json > capture.json
@@ -36,6 +40,185 @@ The third command serves that manifest from the state it derives from the
capture; see [Serve from the state](#serve-from-the-state). Both served files
are complete REA Evidence JSON as the commands write it with `--format json`;
the `evidence` member of `view.json` is the whole of `capture.json`.
A `view.json` can also name the capture file as
`{"kind": "evidence-file", "path": "capture.json"}`, which projects it as
retained Evidence instead of charging the whole capture as inline input.
## Build a bundle in one step
```sh
rea environment-build grafana.har ./grafana --json
agent-env run ./grafana
```
`rea environment-build <capture> [directory]` takes a `.har` file, or capture
Evidence JSON from `inspect-web-network-capture` (HAR) or
`capture-browser-scenario`, and writes the bundle
[Export a bundle](#export-a-bundle) describes. It inspects a HAR as
`inspect-web-network-capture` does (Evidence input skips this), projects the
environment manifest from that Evidence as retained Evidence, authenticates
the pair, prepares the state the bundle's server derives at startup, and
writes the bundle with the export. For the same capture path, every file it
writes is byte-identical to what `inspect-web-network-capture`,
`inspect-analysis-view` and `environment-export` write, Evidence ids included,
when `view.json` lists the same rules in the views' order and the export is
given the build's name. It reads one file and writes one folder: it fetches
nothing and launches no browser (HAR inspection runs REA's own decoder
process), so record a browser scenario with `capture-browser-scenario` first
and pass its Evidence. Other capture formats are refused, as the manifest view
refuses them, and a HAR document saved as `.json` is refused with a hint to
name it `.har`.
- `--template-rule <rule>`, repeated, replaces the default opt-in rules
`after-uid-literal`, `opaque-token`, `sibling-variance` and
`prefix-variance`, the rules the published capture-built environments use;
`--format-rules-only` applies none, the views' own default. Rules apply in
the views' fixed order, the report lists them, and a repeated rule is
refused as the views refuse it.
- `--name` names the env folder and env id. Without it the name is the
manifest's, unless the capture's origin is a loopback name or an IP literal:
the capture file then names it (`grafana.har` and `grafana.capture.json`
become `grafana`) and a `name-from-file` warning says so. A file name that
yields no portable label (kebab-case, at most 64 characters, not a Windows
device name) is refused; pass `--name`.
- `--package-tarball` works as for the export, and `--capture-timeout-ms`
as for `inspect-web-network-capture`.
- `--dry-run` runs every stage and prints the report with `status: "planned"`
and the files a build would write, but writes nothing and does not check the
directory, which may then be omitted. It reads and hashes a
`--package-tarball` as a build does before copying it, so it refuses the
tarballs a build refuses, including one whose contents cannot be read.
The directory must be absent or empty, as for the export; the build never
replaces a folder. A refusal before the write leaves nothing behind. A capture
Evidence file, an inspected HAR's Evidence and the projected manifest must each
fit the live heap headroom for parsing and sealing, checked before the file is
read or the Evidence serialized; one that does not is refused as
`resource_constraint`. Each
refusal names its stage: `environment-build` for the build's own checks,
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
projection, `environment-export` for the bundle. SIGINT cancels the build
(`cancelled`); before the bundle stage nothing is written, and a cancellation
during the write names the bundle it wrote. Stage progress is logged to stderr
at info level; the report is the only output on stdout:
```json
{
"status": "built",
"directory": "/work/grafana",
"name": "grafana",
"capture": {
"evidence_id": "ev_…",
"operation": "inspect_web_network_capture",
"records": 105,
"input": "har"
},
"manifest": {
"evidence_id": "ev_…",
"tools": 48,
"excluded_groups": 0,
"template_rules": [
"after-uid-literal",
"opaque-token",
"sibling-variance",
"prefix-variance"
]
},
"state": { "served": true, "evidence_id": "ev_…" },
"coverage": {
"reads": {
"state": 36,
"recorded": 6,
"recorded_reasons": { "query-varies": 3, "…": 1 }
},
"writes": {
"served": [{ "tool": "delete_folders_by_uid", "effect": "delete" }],
"withheld": [
{
"tool": "post_folders",
"effect": "create",
"reason": "unconfirmed",
"hint": "No later read showed the effect. Read the entity and its list after the write.",
"exemplar_ordinal": 94,
"contradicted_by": null
}
]
}
},
"tool_count": 37,
"package_source": { "kind": "registry", "specifier": "rea-agents@…" },
"files": [{ "path": "README.md", "bytes": 4210 }],
"warnings": [{ "code": "name-from-file", "message": "…" }],
"next": ["agent-env run /work/grafana"]
}
```
`coverage` is the state's read views and write plans summarized, and `null`
when the state is unavailable and the bundle serves replay. The
[environment coverage](web-network-captures.md#environment-coverage) view
reports the same summary for any capture without building, so an agent can
check it after each recording. The view summarizes the state as derived; it
does not apply the serve-time admission checks, so for an environment whose
served catalog exceeds the discovery budget, or whose state exceeds what
`data/add` loads, it still reports the state while the build reports `coverage:
null` and the bundle serves replay. Each withheld write carries a hint about
what to record next; hints never change what is served:
| Reason | Hint |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `no-state-target` | The write's path names no collection or item the capture reads. Read the collection or item it changes. |
| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. |
| `body-unusable` | The request body is not a usable JSON object. |
| `no-observation` | Nothing showed the entity or document this write changed. Read a created entity back by id right after the write; for an update or delete the state must already hold the entity, so read it before the write or record the create that made it. |
| `unexplained-change` | The write changed fields it did not send, in a way no rule explains. A second, different call of the same tool can show the pattern. |
| `contradicted` | A later read disagreed with the modelled effect. Check the read at contradicted_by. |
| `side-effect` | The write changed something besides its own entity, which the environment does not model. |
| `unconfirmed` | No later read showed the effect. Read the entity and its list after the write. |
| `incomplete-entities` | The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it. |
The bundle ships `evidence.json`, the whole capture Evidence, response bodies
included. Inspection excludes the values of the standard authentication headers
(`Authorization`, `Cookie` and `Set-Cookie` among them) and URL userinfo; a
custom key header such as `X-Api-Key`, a token in the query string and tokens
an application returns in bodies remain. The environment views refuse captures
inspected with `--sensitive-value`, so scrub a capture before building it. The
report's `credential-named-members` warning counts recorded header names, query
parameter names and JSON or form body members whose names look like
credentials (whole name segments such as `token`, `secret`, `password`,
`api_key`, `authorization` or `session`) and hold a value, and lists the first
20 by record and pointer, never by value. A body is read as a form or as JSON by
its recorded media type, or by its shape when it has none or one naming
neither. A browser scenario is
scanned by event (`event 4 response /token` names
`normalized_result.events.items[4]`): every retained request and response
content event, including a response whose request began before the browser was
attached and a transaction no record represents, and each transaction URL's
query once. Bodies larger than 4 MiB of UTF-8 text are not
scanned; `credential-scan-incomplete` counts them and lists up to 20 record or
event pointers, including base64 bodies whose decoded scan was cut. An empty name
warning does not establish that these bodies were checked. Names do not establish sensitivity,
so the build never refuses on them. Capture
Evidence also records the capture file's absolute path (`subject.local_path`,
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
carries it: build from a neutral folder when that path should not ship.
Over MCP, `export_environment_bundle` builds the same bundle and returns the
same report. Its `source` is capture Evidence the session retained
(`{"kind": "retained-evidence", "evidence_id"}`), such as a
`capture_browser_scenario` or `inspect_web_network_capture` result, or a
capture Evidence file (`{"kind": "evidence-file", "path"}`); inspect a HAR with
`inspect_web_network_capture` first. Retained Evidence is written into the
bundle as `--format json` writes it, so the bundle is the one
`environment-build` writes from that file. `output_directory` must be an
absolute path that does not exist yet, whose parent does; the tool creates it
and refuses an existing directory, empty or not. `name`, `template_rules`,
`package_tarball` and `dry_run` work as `--name`, `--template-rule`,
`--package-tarball` and `--dry-run` do, and retained Evidence has no file to
name the environment after, so a loopback or IP origin needs `name`. Unlike
`--dry-run`, `dry_run` still checks `output_directory`. Destination refusals
name `export_environment_bundle`; the others name their stage as
`environment-build`'s do.
## From a browser scenario
@@ -125,6 +308,46 @@ the replayed body is the bytes the browser exposes for each response, with any
declared secret replaced by its redaction marker (a body that redaction had to
empty replays no body).
To record writes, sign in through the UI and then send the application's API
calls with [request steps](browser-scenario-contract.md#request-steps): each
sends one request to the start origin through the page's own `fetch`, with the
session's cookies, and a header can carry the CSRF cookie or stored token the
application's own requests send. A write is served only when the capture shows
its effect, so read each collection before writing, read every written entity
back by id and in its list right after the write, and leave more than a second
between a create and its first edit. A request step's path is fixed when the
scenario is written and cannot use a value an earlier response returned, so an
id must be known before the run: one the caller chooses, such as a `uid` in the
create body, or one an earlier scenario created. When the server assigns ids,
read the list after each create (the create's response or the list shows the
new entity), and record updates and deletes in a second scenario that reads
the first one's objects by id before editing them, or make them through the
UI. The environment coverage view names each
write it withholds and what to record next. A scenario records no request
times, so the state learns no clock: an update whose response changes a
timestamp the server sets, such as a modification time, is withheld as
`unexplained-change` however it is recorded, and the coverage view says so; a
HAR of the same session times its entries and can serve it. A scenario with write requests
changes the application's data: record against an instance whose data may
change.
```json
{
"step_id": "create-folder-1",
"action": "request",
"method": "POST",
"path": "/api/folders",
"headers": [
{
"name": "X-CSRFToken",
"value": { "source": "cookie", "name": "csrftoken" }
}
],
"json": { "title": "Scratch folder 1" },
"expected_status": [200, 201]
}
```
## What a manifest is
An environment manifest is a projection of one application's recorded request
docs/browser-scenario-contract.md
+113 -3
View File
@@ -10,8 +10,10 @@ liveness deadlines. `INPUT_JSON` may be inline JSON or a JSON file path.
The minimal request contains only browser launch/connect selection, `start_url`,
and at least one `actions` entry. Launch `headless` defaults to `true` and can
be set to `false` where the host supports a visible browser. Navigations follow
the destinations named by the scenario and the page's ordinary browser requests;
there is no separate origin allowlist or request-interception layer. Environment
the destinations named by the scenario and the page's ordinary browser requests,
and [request steps](#request-steps) send only to the start URL's origin;
there is no separate origin allowlist. Request steps use a temporary dispatch
gate for cancellation; ordinary page requests retain their browser behavior. Environment
settings (including service workers blocked by default), empty storage, and a
final sanitized URL capture are supplied by default. Set
`environment.service_workers` to `allow` when the application requires them.
@@ -204,7 +206,8 @@ language environments.
HTTP(S) URLs may include ordinary query values and fragments directly. Use
ordered structured query entries when a value needs to reference a declared
environment-backed secret. Form, storage, and cookie values are either literal
strings or declared secret references. Raw URL userinfo credentials are rejected.
strings or declared secret references; request step headers can also read a
cookie or local storage value from the page. Raw URL userinfo credentials are rejected.
Captured credential-header
values are not retained, and declared secret values are redacted automatically.
A secret may be declared solely to
@@ -315,6 +318,113 @@ external browser without closing it. The request itself defines the operation;
there is no additional REA grant step. The browser and host still enforce their
own access and process rules.
## Request steps
A `request` step sends one HTTP request from the page with the page's own
`fetch`, so the browser attaches the session's cookies and the request is
recorded in the step's network events like any request the application makes.
Use it to record an application's API, including writes, when driving the UI
for each call would be slower or less exact than calling the endpoint the UI
calls. It is the only action that changes application state without a user
gesture, so run scenarios with write requests only against an instance whose
data you may change. This scenario creates one item:
```json
{
"browser": {
"mode": "launch",
"executable_path": "/absolute/path/to/chromium"
},
"start_url": { "url": "http://127.0.0.1:3000" },
"actions": [
{
"step_id": "create-item",
"action": "request",
"method": "POST",
"path": "/api/items",
"query": [
{ "name": "notify", "value": { "source": "literal", "value": "false" } }
],
"headers": [
{
"name": "X-CSRFToken",
"value": { "source": "cookie", "name": "csrftoken" }
},
{
"name": "Authorization",
"value": {
"source": "local_storage",
"key": "auth",
"json_pointer": "/token"
},
"prefix": "Bearer "
}
],
"json": { "title": "Scratch item 1" },
"expected_status": [201],
"timeout_ms": 10000
}
]
}
```
- `method` is `GET`, `HEAD`, `POST`, `PUT`, `PATCH` or `DELETE`. `json` is the
body, sent with `Content-Type: application/json` unless a header sets one; a
`GET` or `HEAD` step cannot carry it.
- `path` starts with one `/` and must resolve to the start URL's origin, which
validation checks before the browser starts. When the step runs, the page must
be on that origin, checked again in the page as the request is sent; add a
`goto` step first if an earlier action left it. A query in `path` is sent as
written (a fragment is not sent), and `query` entries are appended after it
in order and may reference declared secrets.
- Request steps follow no redirects: a redirect answer fails the step, and
nothing is sent to the location it names, though a write the redirect
answered may already have been applied. `expected_status` cannot name a 3xx
status. Give the path the redirect names; with header values selected, the
capture records its `Location`.
- Header values are literals, declared secrets, a cookie the browser holds for
the request URL (`{"source": "cookie", "name"}`, HttpOnly cookies included,
whose value the page's scripts can then see while the request runs), or a
local storage entry on the start origin (`{"source": "local_storage",
"key"}`, with an optional `json_pointer` to a string or number in a stored
JSON value). `prefix` is sent before the value. A value read from the page is
redacted like a declared secret, as `[REDACTED:page:<step_id>.<header>]`,
from records made after the step reads it; a response recorded earlier, such
as the login response that returned a token, keeps the value. A value read
again keeps its first marker, and a value shorter than 8 characters is not
redacted, since redacting it would rewrite ids, ports and hosts throughout
the capture. A value is redacted as stored and as `fetch` sends it, without
surrounding spaces and tabs. A missing cookie, key or pointer fails the step.
Reading a page value raises the estimated cost of redacting every later
body, so the bodies kept within `max_network_body_allocation_bytes` (1 MiB
by default) get smaller: after a 32-character `X-CSRFToken` value, a body over
about 47 KiB is kept as `unavailable`. Raise `capture.limits` when the
application's responses are larger.
- Validation refuses a header name given twice in a step, header names a
page's `fetch` cannot set, such as `Cookie`, `Origin`, `Host` or `Sec-*`, and
`User-Agent`, which Chrome drops; it refuses line breaks and other ASCII
control characters in literal values and prefixes, and the step fails before sending
when a secret or page value holds them.
- The step completes when the whole response has been read. With
`expected_status`, any other status fails the step and the later actions are
recorded as `cancelled`; without it, any response completes the step. A
network error fails the step. `timeout_ms` covers reading the header values
and the request: a step whose time runs out, or that is cancelled, before
its request is sent sends nothing, but a request already sent can still reach
the application after the step fails.
An absolute deadline also covers browser evaluation queued behind page work.
A temporary route checks cancellation again at network dispatch and is removed
when the request evaluation settles. A page controlled by a service worker
is refused because its fetch can bypass that dispatch gate; use the default
blocked service-worker policy for request steps.
The request runs in the page's main world, so an application that wraps
`fetch` sees it, and it goes through the same network observation as other
page requests: select network content (bodies or header values) to record it.
Captures with request steps state in `limitations` that the steps sent their
declared requests through the page's fetch. A scenario without request steps
is unchanged.
## Network evidence
`capture.events: ["network"]` retains request, response, completion, failure,
docs/cli.md
+29
View File
@@ -746,6 +746,31 @@ like a blank `MCP_HOST`. The bound URL is written to stderr as one JSON log line
[agent environments](agent-environments.md) for what the state and replay
return and do not establish.
## Build an environment bundle
`rea environment-build <capture> [dir] [--name <name>] [--template-rule <rule> ...]
[--format-rules-only] [--package-tarball <path>] [--capture-timeout-ms <ms>] [--dry-run]` builds the bundle
`environment-export` writes from one recorded capture: a `.har` file, which it
inspects as `inspect-web-network-capture` does, or capture Evidence JSON from
`inspect-web-network-capture` (HAR) or `capture-browser-scenario`. It projects
the environment manifest from the in-memory Evidence with the opt-in rules
`after-uid-literal`, `opaque-token`, `sibling-variance` and `prefix-variance`
unless `--template-rule` or `--format-rules-only` selects others, prepares the
state, and writes the bundle; every file is byte-identical to the three-step
path's when that path lists the same rules in the views' order and exports
under the build's name. A loopback or IP-literal capture is named after its file unless `--name`
is given. It prints one JSON report: the capture and manifest identities, the
state, a `coverage` summary of which tools answer from the state, which replay
and why each write is withheld with a hint, the files, warnings and the next
command. `--dry-run` writes nothing and may omit the directory.
```sh
rea environment-build grafana.har ./grafana --json
```
See [agent environments](agent-environments.md#build-a-bundle-in-one-step)
for the report, the hints and what the bundle ships.
## Export an environment bundle
`rea environment-export <manifest.json> --evidence <capture.json> <dir>
@@ -921,6 +946,10 @@ server resources. It preserves a primary listener diagnostic, or a
tool count and Evidence identifiers. The successful `stopped` document is
returned only after cleanup is confirmed. The same in-process server owner
retains failed stages for retry; the CLI exposes no separate retry command.
`rea environment-build` is one-shot too: it prints its report and exits `0`,
or prints the refusing stage's structured error and exits `1`, writing nothing
when the refusal comes before the bundle is written. SIGINT cancels it with a
`cancelled` error; a cancellation before the bundle stage writes nothing.
`rea environment-export` is one-shot: it prints the written files and exits
`0`, or prints a structured `invalid_request` error and exits `1` for the same
pair refusals, a non-empty or file-occupied directory, a missing package
docs/specs/agent-environments/environment-build/plan.md
+25
View File
@@ -0,0 +1,25 @@
# Plan
1. Add `environmentCoverageView.ts`: the request schema, the result fields, the
hint per write reason (`satisfies Record<WriteReason, string>`), a
summarizer over state views and write plans, and the projector, which
projects the environment-state view and summarizes it. Register the kind in
the view unions, the grouping dispatch and the view service (state
accumulator, inferred confidence).
2. Split the export's input read into `readEnvironmentExportFile` and
`environmentExportFile` (parse and seal bytes), both naming the calling
operation, and let the export take an already prepared state.
3. Add `environmentBuild.ts`: classify the capture, project the manifest from
the in-memory Evidence, serialize it as `--format json` writes it to a file,
authenticate the pair from those bytes, name the environment, prepare the
state as the export does, summarize coverage, scan credential names, then
plan (dry run) or export. `environmentBuildReport.ts` holds the scan and the
naming helpers.
4. Register `environment-build` in the CLI: inspect a HAR through the capture
service as `inspect-web-network-capture` does, or read Evidence once as the
export does, and log stages at info.
5. Switch `scripts/verify-agentenv-environment.mjs` to the build command.
6. Document the command, the report, the hints, what ships, and the view.
7. Tests: coverage summary and hint completeness, the view against the state
view, build equivalence with the three-step path in process and through the
CLI, dry run, naming, refusals, the credential scan, and the view over MCP.
docs/specs/agent-environments/environment-build/spec.md
+96
View File
@@ -0,0 +1,96 @@
# Build an environment in one step
Issue: https://github.com/morluto/rea/issues/4933
## Outcome
Building an agent environment bundle takes three commands and a hand-written
view request that embeds the whole capture inline, so large captures hit inline
input accounting. The views apply no opt-in template rules by default, while the
published capture-built environments all use the same four, and a local capture
is named after its origin host (`127-0-0-1-3105`). Nothing reports which tools
an environment serves from its state, or why a write tool is withheld.
`rea environment-build <capture> [directory]` builds the bundle in one offline
step and reports what it serves. A new `environment-coverage` analysis view
reports the same summary for any capture, so an agent can check a recording
before building.
## Requirements
- The capture is a `.har` file, inspected as `inspect-web-network-capture`
inspects it, or capture Evidence JSON from `inspect_web_network_capture` or
`capture_browser_scenario`; any other extension is refused before it is
read, and a HAR document saved as `.json` is refused with a hint.
- The manifest is projected from the in-memory capture as retained Evidence,
with the default rules `after-uid-literal`, `opaque-token`,
`sibling-variance` and `prefix-variance` unless `--template-rule` or
`--format-rules-only` selects others; rules apply in the views' order and a
repeated rule is refused, as the views refuse it.
- For the same capture path, every bundle file is byte-identical to what
`inspect-web-network-capture --format json`, `inspect-analysis-view --format
json` (the same rules in the views' order) and `environment-export` (the
build's name) write, Evidence ids included. The Evidence is serialized once
into one exactly sized buffer, whose bytes are both authenticated and shipped.
- The name is `--name`, else the manifest's, unless the origin is a loopback
name or IP literal: then the capture file's name, with a warning; a file name
that yields no portable bundle name is refused, asking for `--name`.
- The report carries the export's tool count, state, package source and files,
plus the capture and manifest identities, `coverage`, warnings and the next
command, shell-quoted. `coverage`
counts read tools answered from the state and replayed (per view reason), and
lists write tools as served or withheld with their reason, plan record, the
contradicting read and a hint; it is null when the state is unavailable.
- Every write reason has a hint; a reason without one fails to compile.
- Credential-named header names, query parameter names and JSON or form body
members that hold a value are reported as a count and the first 20 record
pointers, never values, and never refuse a build; the scan cannot overflow the
stack on deeply nested bodies. A body is read as a form
(`application/x-www-form-urlencoded`, by `URLSearchParams`) or as JSON (the
media types replay parses as JSON) by its recorded media type; a body with
none, or one naming neither, is JSON when it starts with `{` or `[` and a form
when it contains `=` and no whitespace, so plain text and empty bodies yield
no pointers. A browser scenario is scanned by event rather than by record:
every retained request and response network-content event's header values and
body, including a response whose request began before the browser was attached
and a transaction that cannot be attributed to one request and response, and
each transaction URL's query parameters once; its pointers name the event's
index in `events.items` (`event 4 response /token`). Each occurrence of a
repeated form field counts, as each repeated header or query parameter does. A
HAR request recorded with structured form parameters and no text, or the null
text mitmproxy's SaveHar records, is scanned by those parameters. Text bodies
over the 4 MiB scan limit, measured in UTF-8 bytes, including a response body
decoded only as a prefix, are not scanned; they are reported as a
`credential-scan-incomplete` warning with a count and the first 20 record or
event and side pointers (`record 3 response`, `event 4 response`), never
contents, and never refuse a build. Binary bodies hold no JSON or form text
and are neither scanned nor reported.
- The inspected HAR's Evidence and the projected manifest are admitted against
the live heap headroom for parsing and sealing before either is serialized, as
a capture Evidence file is before it is read; one that does not fit is refused
as `resource_constraint` before anything is written.
- An empty or NUL-containing capture, directory or package tarball argument is
refused as invalid input; it is never resolved to the working directory.
- Nothing is written before the bundle stage; `--dry-run` writes nothing,
reports the files a build would write and refuses the package tarballs a build
refuses, admitting, reading and hashing, and releasing the tarball as a build
does before copying it, so a tarball whose contents cannot be read is refused:
a close it cannot confirm is reported as incomplete cleanup that keeps its
owner. The directory must be absent or empty.
- SIGINT is checked between stages and after a dry run's last step: a build
cancelled before the bundle stage writes nothing, and one cancelled during the
write names the written bundle.
- The `environment-coverage` view summarizes the environment-state view with
the same rules, refuses what it refuses and is inferred. It does not apply the
serve-time catalog and state admission checks, and says so in a limitation.
- The `real-agentenv` lane builds its bundle with `environment-build`.
## Non-goals
- No change to the three existing commands' contracts, or to what any view
derives or any environment serves.
- No browser launch or network access: scenarios are recorded with
`capture-browser-scenario` first.
- No `--replace`, no refusal on credential names, and no rewriting of the
capture's absolute path, which Evidence identity includes.
- No MCP export tool; that is a separate change.
docs/specs/agent-environments/environment-build/tasks.md
+13
View File
@@ -0,0 +1,13 @@
# Tasks
- [x] Record the outcome, requirements and non-goals.
- [x] Add the environment-coverage view and register it.
- [x] Split the export's input read and accept a prepared state.
- [x] Add the build module, its report helpers and the CLI command.
- [x] Switch the real-agentenv lane to the build command.
- [x] Document the command, the report, the hints and the view.
- [x] Add domain, service, CLI and MCP tests.
- [x] Run static, formatting, duplication, documentation and test gates.
- [x] Build the 50 published captures and compare them with the three-step path.
- [x] Build environments end to end on Linux VMs and run them with agents.
- [x] Address the independent review.
docs/specs/agent-environments/environment-build/verification.md
+102
View File
@@ -0,0 +1,102 @@
# Verification
Executed on macOS with Node v24.22.1 (the checkout's `.nvmrc`) on main `3d4f5565`, and on Ubuntu 24.04 VMs for the
end-to-end runs.
## Gates
- `npm run -s check:fast`, `npm run -s format:check`, `npx knip`, `npm run -s scan:todos`, `npm run -s docs:check`,
`npm run -s verify:test-discovery` and `npm run -s mcp-catalog:check`: passed.
- `npm run jscpd`: 3.52% duplicated lines over 4348 files, as on main (3.52% over 4341).
- `npx vitest run --project domain --project services`: 9537 passed, 6 skipped and 151 failed. The same 151 fail on main
`3d4f5565` in this environment (9513 passed there); none is new.
- `npm run -s test:prepare`, then `npm run -s test:focused --` the environment build, export, export-inputs and serve CLI
boundaries, the analysis-view CLI and MCP boundaries and the environment serve MCP boundary: 7 files, 67 passed and 1
failed, "projects portable artifact inventory after the original ZIP is removed", which fails the same way on main.
- `npm run verify:agentenv:environment` (the `real-agentenv` lane) passed on Ubuntu 24.04 with `agent-env 0.9.1334`.
## Equivalence and coverage over the published captures
Each of the 50 published capture-built environments' captures (capture Evidence JSON) and four scoped HAR recordings
(EspoCRM, Ghost, NocoDB, Sonarr) was built twice: once with `rea environment-build <capture> <dir> --json`, and once
with the three CLI steps, `inspect-web-network-capture --format json` (HAR only), `inspect-analysis-view` with an
`evidence-file` source and the four default rules, and `environment-export --name <the build's name>`. Every file of
every bundle was compared by SHA-256.
| Inputs | Bundles identical | State served | Read tools from state / replayed | Write tools served / withheld | Build / three steps |
| ------------------------- | ----------------- | ------------ | -------------------------------- | ----------------------------- | ------------------- |
| 50 capture Evidence files | 50 of 50 | 50 | 1,100 / 290 | 50 / 143 | 32.9 s / 59.2 s |
| 4 scoped HARs | 4 of 4 | 4 | 166 / 21 | 5 / 12 | 8.3 s / 12.4 s |
- The coverage of every capture equals an independent census of the environment-state view with the same rules (read
views by shape, write plans by verdict and reason).
- 43 of the 50 captures have a loopback origin and are named after their file (`grafana.capture.json` becomes `grafana`)
with a `name-from-file` warning. 20 carry a `credential-named-members` warning; the published captures were scrubbed by
their pipeline, whose placeholder strings still count.
- The withheld writes' reasons: no-state-target 86, contradicted 16, unconfirmed 14, no-observation 13,
unexplained-change 8, side-effect 4, body-unusable 1, incomplete-entities 1.
## End to end
Five environments last rebuilt on 10-23 were built again with this branch's packed CLI on disposable Ubuntu VMs, four
from their scoped HAR recordings and Metabase from its capture Evidence, with `rea environment-build <capture> <dir>
--name <app> --package-tarball <tarball>`. Each bundle then ran its smoke task, a retrieval eval generated from the
bundle's own manifest and capture, and the write eval from its recorded write spec, with two models through a LiteLLM
proxy (`fireworks_ai/gpt-oss-120b`, `fireworks_ai/glm-5p3-flash`), one run per task.
| App | Input | Served tools of manifest | Served writes (build report) | Smoke | Retrieval gpt-oss / GLM | Writes gpt-oss / GLM |
| -------- | -------- | ------------------------ | -------------------------------------------------- | ------ | ----------------------- | -------------------- |
| Ghost | HAR | 35 of 38 | 0 (3 no-state-target) | passed | 12/12 / 8/12 | 0/2 / 0/2 |
| Metabase | Evidence | 40 of 42 | 1 (create contradicted, update unexplained-change) | passed | 12/12 / 12/12 | 1/2 / 1/2 |
| EspoCRM | HAR | 85 of 88 | 0 (2 contradicted, 1 unexplained-change) | passed | 12/12 / 10/12 | 0/2 / 0/2 |
| NocoDB | HAR | 23 of 28 | 0 (4 no-state-target, 1 unconfirmed) | passed | 6/6 / 5/6 | 0/3 / 0/3 |
| Sonarr | HAR | 49 of 50 | 5 (1 side-effect) | passed | 12/12 / 12/12 | 3/3 / 3/3 |
- Every write task passed with both models when the build report listed its tool as served (Metabase's card delete and
Sonarr's release-profile create, update and delete), and failed when it listed it as withheld: 8 of 24 write episodes
passed, exactly the served ones. On main these apps' other writes are withheld; the published v0.2.0 versions were
built with the state-target and write-rule changes, which serve more of them.
- The retrieval misses are GLM answers on bundles byte-identical to the three-step path's; gpt-oss passed all 54.
- The `real-agentenv` lane, switched to `environment-build`, passed from a fresh checkout on Ubuntu with
`agent-env 0.9.1334` and the host's Docker: five public cases, the smoke task passed in 26.6 s, no new containers.
## Review
An independent review rebuilt the branch and tried to refute it. It confirmed byte-equality for pretty-printed,
CRLF and newline-free Evidence, Unicode HARs across output part boundaries, inline and file view sources, browser
scenario Evidence and relative or symlinked paths, and that the refactored export answers 16 success and refusal cases
exactly as main does. Its findings, all addressed:
- The credential scan recursed and a 4 KB body nested 5,000 deep crashed the build; it now walks with its own stack, and
also scans header names, query names and form bodies.
- SIGINT after the capture was read was ignored; the build now checks the signal between stages and writes nothing when
cancelled before the bundle stage.
- The coverage view does not apply the serve-time catalog and state admission checks; a limitation and the docs now say
so.
- A dry run accepted a tarball the build refuses; it now opens it for reading.
- Repeated `--template-rule` values were deduplicated; they are refused as the views refuse them. A HAR saved as `.json`
gets a hint, file-derived names follow the bundle name rules, `localhost.` is anonymous, and the next command is
shell-quoted.
- The `no-observation` hint now covers updates and deletes of entities the state never held; the docs no longer claim
the build launches no process, state the conditions of byte-equality, and name the headers inspection excludes; the
MCP tool description names the view.
## Review round on main `584120e8`
The branch merged main `584120e8`: the export's file read now returns permission refusals through `Result`, as main
does, under the build's operation name, and the `environment-coverage` sentence sits in main's `inspect_analysis_view`
description. Four automated review findings were fixed, each with a test that fails on the merge:
- A browser scenario's retained header values and the query parameters of its recorded URLs are scanned for credential
names as a HAR's are; the manifest and replay record projections are unchanged.
- `--dry-run --package-tarball` admits and releases the tarball through the export's own regular-file scope, so a close
it cannot confirm is retried and then reported as incomplete cleanup that keeps its owner.
- A dry run checks for cancellation after reading the tarball, before it reports a plan.
- Evidence is serialized into one exactly sized buffer instead of chunks joined by a copy. Those bytes are still parsed
and sealed before authentication, so authentication reads exactly what ships.
On macOS with Node v22.23.1: the static gates passed (`npm run jscpd`: 3.53%), `npx vitest run --project domain
--project services` passed (9753, 6 skipped), the environment boundary projects passed (102), and the focused build,
export, export-inputs, serve and analysis-view boundaries with tool schema validity passed (9 files, 96). Rebuilt over
the same 50 captures and four HARs, all 54 bundles are byte-identical to the three-step path's, with the same coverage,
served writes and warnings as before the round.
docs/specs/agent-environments/environment-prompt/plan.md
+21
View File
@@ -0,0 +1,21 @@
# Plan
1. Add the `request` action to the scenario schema: methods, path, query,
headers with page value sources, JSON body and expected statuses; validate
the origin and the body against the start URL; include query and header
secrets in the declared-secret check.
2. Add `PlaywrightScenarioRequest.ts`: check the page's origin, build the URL,
resolve header values (admitting values read from the page to the scenario's
redaction), fetch in the page with a timeout and read the whole response,
then check the status. Dispatch it from the action runner with the start
origin, and add the limitation when a scenario has a request step.
3. Add a real-browser lane, `verify:browser:request`: a fixture item API that
requires a CSRF cookie header and a stored bearer token for writes, a write
session recorded through the CLI, the build's served writes, and a failing
expected status.
4. Add the `export_environment_bundle` contract, effects, CLI alias, catalog
family and handler: check the destination as other MCP output tools do,
resolve the capture bytes, and call the build with an exclusive destination.
5. Write the skill reference, route it from SKILL.md and install it with setup.
6. Document the request step, the MCP tool and recording writes.
7. Tests: request step unit tests, the MCP tool boundary, setup's skill files.
docs/specs/agent-environments/environment-prompt/spec.md
+83
View File
@@ -0,0 +1,83 @@
# Build an environment from a prompt
## Outcome
`rea environment-build` builds a bundle from one capture, but an agent asked to
"make an environment of this app" still cannot finish the job through REA's MCP
server. A browser scenario can only reach an application's API through the UI or
a `goto` GET, so recording the writes an environment serves means scripting every
form, and there is no MCP tool that writes a bundle. Nothing tells the agent how
to record a session the state can learn writes from, or when to stop recording.
A `request` scenario action sends one same-origin request through the page's own
`fetch`, so a scenario can sign in through the UI and then call the application's
API, writes included, with the session's cookies and the CSRF or bearer values
the application's own requests send. An `export_environment_bundle` MCP tool
builds the bundle from capture Evidence, as `environment-build` does. A skill
reference walks an agent from the prompt to a bundle: map, record, check
coverage, re-record by reason, export and hand off.
## Requirements
- A `request` step names a method (`GET`, `HEAD`, `POST`, `PUT`, `PATCH`,
`DELETE`), a path on the start URL's origin, ordered query entries, headers,
an optional JSON body and optional expected statuses. Validation refuses a
path that resolves to another origin, a body on `GET` or `HEAD`, a 3xx
expected status, header names a page's `fetch` cannot set or Chrome drops
(`User-Agent`), a header name given twice, line breaks or control characters
in literal values and prefixes, and a path the URL parser rejects, as a field
issue at the step's path.
- Header values are literals, declared secrets, a cookie the browser holds for
the request URL, or a local storage value on the start origin, optionally read
at a JSON pointer, with an optional prefix; local storage values are read only
while the page is on the start origin. Values read from the page are redacted
like declared secrets from records made after they are read, as stored and as
sent; a value read again keeps its first marker, a colliding label never
unredacts an earlier value, and values shorter than 8 characters are not
redacted.
- The request runs through the page's `fetch` with same-origin credentials, from
a page on the start origin (checked again in the page as it is sent), follows
no redirects, and completes when the response has been read to its end,
without keeping its body. A redirect, a status outside `expected_status`, a
network error or the step timeout fails the step; later steps are cancelled as
for other actions. The page sends only before the step's deadline, which it
receives as an absolute time, and within one second of being handed the
request; a busy page that starts later sends nothing, and the step hands the
request over again while it is neither cancelled nor out of time. A step
cancelled or out of time before the page starts its request sends nothing,
except a request the page starts within that second of being handed it. A
browser whose clock runs ahead of the host's fails the step. Its network
events are attributed to the step.
- Captures with request steps say so in a limitation. A scenario without one
validates, runs and records exactly as before.
- `export_environment_bundle` takes retained capture Evidence or a capture
Evidence file, an absolute output directory that does not exist and whose
parent does, and the build's name, template rules, package tarball and dry
run. It writes the bundle `environment-build` writes from the same Evidence
bytes (retained Evidence serialized as `--format json` writes it) and returns
the same report, validated by its output schema. It refuses an existing
directory, empty or not, a parentless or relative one, unknown Evidence, an
unnamed loopback capture and repeated rules, and writes nothing when it
refuses. Retained Evidence is refused for heap headroom before it is
serialized, as an Evidence file is before it is read. `environment-build` is
its CLI command.
- The skill reference `references/environments.md` is installed with the skill
and routed from SKILL.md. It covers disposable instances only, credentials
from environment variables, the recording rules the state needs, coverage, the
stop rule, the hand-off, and how to reach objects by id when the server
assigns ids: ids the caller chooses, a second scenario, or the UI.
- Coverage of a capture whose records are not all timed, which includes every
browser scenario capture, says the state learns no write clock.
- Stop rule: re-record a withheld write at most twice for a recording gap
(`no-state-target`, `no-observation`, `unconfirmed`, `incomplete-entities`),
once for `unexplained-change` and `no-call`, and stop at once for application
behaviour (`side-effect`, `contradicted`, `body-unusable`).
## Non-goals
- No cross-origin requests, form or multipart bodies, or arbitrary script
actions.
- No retroactive redaction of values a request step reads: a login response
recorded before the step keeps the token it returned, as captures do today.
- No application start, seed or reset, and no publishing of bundles.
- No change to what any view derives or any environment serves.
docs/specs/agent-environments/environment-prompt/tasks.md
+11
View File
@@ -0,0 +1,11 @@
# Tasks
- [x] Record the outcome, requirements and non-goals.
- [x] Add the request action, its validation and its Playwright runner.
- [x] Add the real-browser request lane.
- [x] Add the export_environment_bundle tool.
- [x] Write and install the skill reference.
- [x] Document the request step, the tool and recording writes.
- [x] Run static, formatting, documentation and test gates, and the existing browser lanes.
- [x] Build environments from prompts with an agent against live applications on Linux VMs.
- [x] Address the two independent reviews.
docs/specs/agent-environments/environment-prompt/verification.md
+82
View File
@@ -0,0 +1,82 @@
# Verification
Executed on macOS with Node v24.22.1 at `feat/environment-prompt` 8dfd5a4d (stacked on the environment-build branch at
eeab0bda), and on Ubuntu 24.04 VMs for the end-to-end runs.
## Gates
- `npm run -s typecheck`, `lint`, `format:check`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery` and
`npx knip`: passed.
- `npx vitest run --project domain --project services --project adapters`: 16,347 passed and 1 failed,
"stops Inspector target authorization after the current stat settles", which fails the same way on the base fce1aba8
in this environment (16,326 passed, 2 failed there); none is new.
- Boundaries: the bundle tool, analysis view, tool schema validity, environment-build CLI, setup skill and client
lifecycle suites (123 passed), and every browser and scenario MCP and CLI boundary suite (68 passed).
- Real Chrome: `verify:browser:request` (new: 22 request steps, the CSRF cookie and stored token authenticate seven
writes and never reach the Evidence, two queued writes are refused, the build serves the three write tools, an unexpected status fails the step and
cancels the rest, and a redirect to another origin is not followed, the other origin receiving nothing) and
`verify:browser:network` passed. `verify:browser` stops at "Real Chrome stable capture identities did not
reconcile", an `inspect_web_page` comparison this change does not touch, and fails the same way on the base.
- `verify:browser:request` runs each CLI call under the shared package verifier command owner, which stops the CLI and
the browser it launched on timeout, output overflow or failure before the lane's private root is removed, and reports
a failed stop with the root retained; `browserRequestVerifierOwnership.test.ts` proves a still-running CLI's
separately grouped browser is stopped before the root goes.
Scenarios without request steps are unchanged: the action union gains a member, the secrets refactor compiles the same
patterns, the request-step limitation appears only with request steps, and the browser suites and the network lane pass.
## End to end: environments from a prompt
Five self-hosted applications were brought up fresh and seeded on four disposable VMs, loopback only. Claude Code 2.1.310
(`claude-sonnet-5-5` through a LiteLLM proxy) got this branch's packed CLI, registered with `rea setup --client
claude_code` (the MCP server and the skill), and one prompt per application: build an agent environment of the app at
this URL, cover these entities, the instance is disposable but leave the seeded data as found, sign in as this user, the
password is in `APP_PASSWORD`, Chromium is here, write the bundle there. The prompt did not mention the skill. The agent
could use the REA tools, the skill and file tools; Bash and web tools were disabled.
Each bundle was then rebuilt with `environment-build` from its own `evidence.json` and this branch's tarball (the
registry's `rea-agents@6.7.0` has no `environment-serve`); `manifest.json` and `evidence.json` were byte-identical to
the agent's in all five. It ran its smoke task, a retrieval eval generated from its manifest and capture, and the
application's recorded write eval with `fireworks_ai/gpt-oss-120b` and `fireworks_ai/glm-5p3-flash`, one run per task.
| App | Agent run | Recordings | Records | Tools (state / replayed reads) | Served writes | Smoke | Retrieval gpt-oss / GLM | Writes gpt-oss / GLM |
| -------- | --------- | ---------- | ------- | ------------------------------ | ------------- | ------ | ----------------------- | -------------------- |
| Linkding | 182 s | 9 | 66 | 8 (7 / 1) | 0 | passed | 9/9 / 8/9 | 0/3 / 0/3 |
| Grocy | 253 s | 7 | 154 | 22 (16 / 0) | 6 | passed | 10/11 / 10/11 | 3/3 / 3/3 |
| Homebox | 391 s | 9 | 268 | 13 (11 / 2) | 0 | passed | 9/9 / 9/9 | 0/2 / 0/2 |
| Directus | 202 s | 7 | 100 | 25 (16 / 4) | 5 | passed | 12/12 / 11/12 | 2/2 / 2/2 \* |
| Grafana | 143 s | 4 | 100 | 15 (12 / 2) | 1 | passed | 5/5 / 5/5 | 2/2 / 2/2 |
\* The Directus write eval checks that the state holds exactly the provisions the scripted recording read (2); the
agent's recording read all 5, so its count checks fail although every field check passes (0/2 with both models). With
the counts relative to the state's own provisions it passes 2/2 with both.
- Write tasks passed exactly when the build served their tool, with both models.
- The skill loaded unprompted in every run. All five agents signed in through the form with the declared secret,
recorded the API with request steps, checked coverage, stopped by the withheld reasons and exported with
`export_environment_bundle`; each reported the credential-named warning pointers, and none published anything.
- The scripted recordings the published environments were built from, built with this branch, read more (11, 39, 21,
31 and 35 state-backed reads) and serve 7, 3, 2, 2 and 0 writes; the prompted recordings serve 0, 6, 0, 5 and 1.
- Linkding's and Homebox's updates are withheld because a browser scenario carries no request times: the state learns
no clock, so an update the application stamps with a modification time is `unexplained-change`. Linkding's scripted
HAR serves the same PATCH with a clock rule. The coverage view now says so; recording request times on scenario
network events would lift it. Linkding derives each bookmark's archive URL from its creation time, so its create is
`contradicted` in both recordings.
Earlier rounds: in the first, two agents hung on steps without `timeout_ms` and most guessed the scenario schema, so the
reference now starts from a whole scenario with a timeout on every step; the second (same five apps, before the review
fixes) served 1, 6, 0, 5 and 0 writes with the same eval pattern.
## Review
Two independent reviews tried to refute the change.
- The first found a 302 that took a request step to another origin, page values left unredacted under a repeated or
colliding header label, short page values that rewrote ids and hosts across the capture, `User-Agent` accepted but
dropped by Chrome, a request that could still be sent after its step timed out, an origin check separated from the
send, a literal query re-encoded, line breaks accepted in header values, and CLI wording in MCP refusals. All are
fixed and tested; redirects are now checked in real Chrome.
- The second confirmed those fixes and found query entries mishandled after a path ending in `?` or `#`, 3xx expected
statuses that can never match, and padded stored values escaping redaction (fixed), plus the body cost of reading a
page value, short values kept in plain text and the MCP dry run checking its directory (documented). In the
prompted captures, only static assets lost bodies to the size limits.
docs/specs/agent-environments/scenario-capture/spec.md
+2 -1
View File
@@ -61,7 +61,8 @@ change.
- No new browser action: scenarios still cover what the UI requests plus GET
reads through `goto`. A fetch or script action would be a new effect and
needs its own scope decision.
needs its own scope decision. (The `request` action was later scoped and
added by [environment-prompt](../environment-prompt/spec.md).)
- No change to what the scenario redacts. Values the application returns in
response bodies are retained, as they are in HAR captures.
- The api-surface view keeps HAR parents only.
docs/testing.md
+9 -7
View File
@@ -1935,13 +1935,15 @@ would remain after the pause, the verifier waits 90 seconds and runs once more,
recording `attempts: 2` and `retried_for` in the receipt. Any other failure
ends the lane at once.
The verifier writes the Grafana-shaped HAR the view tests own, inspects it and
projects the `environment-manifest` view with all three template rules through
the built CLI, packs the checkout with `npm pack`, exports the bundle with
`--package-tarball` so the image installs this checkout rather than the
registry package, and runs `agent-env run` on the bundle. It checks that the
export reports the planned file set with the inputs copied byte-exact and a
bundle served from the derived state, which serves fewer tools than the
The verifier writes the Grafana-shaped HAR the view tests own, packs the
checkout with `npm pack`, builds the bundle from the HAR with
`rea environment-build --package-tarball` through the built CLI, the command
users run, so the image installs this checkout rather than the registry
package, and runs `agent-env run` on the bundle. It checks that the build
reports the 28-record capture, the default template rules, the planned file set
with the shipped manifest naming the reported capture, a coverage summary whose
read and served write tools add up to the served tool count, and a bundle
served from the derived state, which serves fewer tools than the
manifest because write tools the capture did not confirm are not served, that the runner exits `0` with
`tasks/smoke.json` reported as passed and the verifier's description naming
that served tool count, and that no container exists
docs/web-network-captures.md
+22
View File
@@ -802,6 +802,28 @@ derives it from the capture at startup, serves read tools and the served write
plans from it and loads replacement states through the AgentEnv data plane, as
[serve from the state](agent-environments.md#serve-from-the-state) describes.
## Environment coverage
`view: {"kind":"environment-coverage"}` with the same `template_rules` reports
what the state an environment derives from the parent would serve, without the
state's entities: `reads.state` counts read tools answered from the state,
`reads.recorded` those that replay their recordings with
`reads.recorded_reasons` per view reason, `writes.served` lists the write tools
whose effect later reads confirmed, and `writes.withheld` lists every other
write tool with its reason, the record its plan was built from
(`exemplar_ordinal`), the read that disagreed (`contradicted_by`) and a `hint`
about what to record next. Hints are advice; they never change what is served.
The view projects the environment-state view and summarizes it, so it refuses
every parent that view refuses, with the same errors, and it is always
inferred. Call it after each recording to see which writes an environment
built from it would serve. It does not apply the serve-time admission checks:
an environment whose served catalog exceeds the discovery budget, or whose
state exceeds what `data/add` loads, serves replay, and `rea environment-build`
then reports `coverage: null`; otherwise the build prints this summary as its
`coverage`. See
[Build a bundle in one step](agent-environments.md#build-a-bundle-in-one-step)
for the hint per reason.
## Resource and lifecycle boundaries
Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting
package.json
+1
View File
@@ -220,6 +220,7 @@
"verify:frida:bindings": "node scripts/run-typescript.mjs --noEmit -p tsconfig.frida-verification.json",
"verify:browser:dom": "node scripts/verify-browser-dom-destinations.mjs",
"verify:browser:network": "npm run build:cached && node scripts/verify-browser-network.mjs",
"verify:browser:request": "npm run build:cached && node scripts/verify-browser-request.mjs",
"verify:electron": "npm run build:cached && node scripts/verify-real-electron.mjs",
"verify:inspector": "npm run build:cached && node scripts/verify/inspector/runtime-observation.mjs",
"verify:javascript-runtime": "npm run build:cached && node scripts/verify/javascript/runtime-observation.mjs",
scripts/fixtures/browser-request-site.mjs
+130
View File
@@ -0,0 +1,130 @@
import { once } from "node:events";
import { createServer } from "node:http";
/** The CSRF cookie and the stored bearer token the page holds; request steps must send both and the capture must not keep them. */
export const requestFixtureCsrf = "csrf-fixture-7c1d2e";
export const requestFixtureToken = "bearer-fixture-9f3a6b";
const html = `<!doctype html><meta charset="utf-8"><title>Items</title><body><main id="ready">Items</main>
<script>localStorage.setItem('auth', JSON.stringify({ token: ${JSON.stringify(requestFixtureToken.split("-"))}.join('-') }));</script>`;
const json = (response, status, value) => {
response.writeHead(status, { "Content-Type": "application/json" });
response.end(value === undefined ? undefined : JSON.stringify(value));
};
/** The item API's answer to one authorized request, as a status and body. */
const answerItems = (store, method, id, body) => {
const { items } = store;
if (id === undefined) {
if (method === "GET")
return [200, { count: items.size, results: [...items.values()] }];
if (method === "POST") {
const created = { id: store.next++, title: body.title, done: false };
items.set(created.id, created);
return [201, created];
}
return [405, { error: "method not allowed" }];
}
const item = items.get(id);
if (item === undefined) return [404, { error: "not found" }];
if (method === "GET") return [200, item];
if (method === "PUT") {
const updated = { ...item, title: body.title, done: body.done };
items.set(id, updated);
return [200, updated];
}
if (method === "DELETE") {
items.delete(id);
return [204, undefined];
}
return [405, { error: "method not allowed" }];
};
/** Start a small item API that, like a cookie-session app with a CSRF header and a stored bearer token, refuses writes without both. */
export async function startBrowserRequestSite() {
const store = {
items: new Map([
[1, { id: 1, title: "Seeded item", done: false }],
[2, { id: 2, title: "Second seeded item", done: true }],
]),
next: 3,
};
const writes = [];
const hosts = [];
const server = createServer(async (request, response) => {
const url = new URL(request.url, "http://fixture.test");
hosts.push(`${request.headers.host} ${url.pathname}`);
const chunks = [];
for await (const chunk of request) chunks.push(chunk);
const body =
chunks.length === 0 ? undefined : JSON.parse(Buffer.concat(chunks));
if (url.pathname === "/") {
response.writeHead(200, {
"Content-Type": "text/html; charset=utf-8",
"Set-Cookie": `csrftoken=${requestFixtureCsrf}; Path=/; SameSite=Lax`,
});
response.end(html);
return;
}
if (url.pathname === "/favicon.ico") {
response.writeHead(204);
response.end();
return;
}
if (url.pathname === "/api/elsewhere") {
const port = new URL(`http://${request.headers.host}`).port;
response.writeHead(302, {
Location: `http://localhost:${port}/api/items`,
});
response.end();
return;
}
if (url.pathname === "/api/session") {
json(response, 200, { token: requestFixtureToken });
return;
}
const match = /^\/api\/items(?:\/(\d+))?$/u.exec(url.pathname);
if (match === null) {
json(response, 404, { error: "not found" });
return;
}
const id = match[1] === undefined ? undefined : Number(match[1]);
if (request.method !== "GET") {
writes.push({
method: request.method,
path: url.pathname,
csrf: request.headers["x-csrftoken"] === requestFixtureCsrf,
bearer:
request.headers.authorization === `Bearer ${requestFixtureToken}`,
});
if (
request.headers["x-csrftoken"] !== requestFixtureCsrf ||
request.headers.authorization !== `Bearer ${requestFixtureToken}`
) {
json(response, 403, { error: "forbidden" });
return;
}
}
const [status, value] = answerItems(store, request.method, id, body);
json(response, status, value);
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
const address = server.address();
if (typeof address !== "object" || address === null)
throw new Error("Request fixture has no TCP address");
return {
origin: `http://127.0.0.1:${address.port}`,
writes,
/** Host header and path of every request the server received. */
hosts,
items: store.items,
async close() {
server.closeAllConnections();
await new Promise((resolve, reject) =>
server.close((error) => (error ? reject(error) : resolve())),
);
},
};
}
scripts/lib/browser-request-e2e.mjs
+376
View File
@@ -0,0 +1,376 @@
import assert from "node:assert/strict";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { chromium } from "playwright-core";
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
import { browserScenarioCaptureSchema } from "../../dist/domain/browserScenarioCapture.js";
import { parseEvidence } from "../../dist/domain/evidence.js";
import { performScenarioRequest } from "../../dist/browser/PlaywrightScenarioRequest.js";
import { BrowserScenarioSecrets } from "../../dist/browser/BrowserScenarioSecrets.js";
import { PrivateRuntimeRoot } from "../../dist/process/PrivateRuntimeRoot.js";
import {
requestFixtureCsrf,
requestFixtureToken,
startBrowserRequestSite,
} from "../fixtures/browser-request-site.mjs";
import { withVerifierCleanup } from "./verifier-cleanup.mjs";
import { createPackageVerifierCommands } from "./verify-package-commands.mjs";
const authenticated = [
{ name: "X-CSRFToken", value: { source: "cookie", name: "csrftoken" } },
{
name: "Authorization",
value: { source: "local_storage", key: "auth", json_pointer: "/token" },
prefix: "Bearer ",
},
];
const read = (step_id, path) => ({
step_id,
action: "request",
method: "GET",
path,
expected_status: [200],
});
const write = (step_id, method, path, { json, status }) => ({
step_id,
action: "request",
method,
path,
headers: authenticated,
...(json === undefined ? {} : { json }),
expected_status: [status],
});
const pause = (step_id) => ({
step_id,
action: "wait_for_timeout",
duration_ms: 1100,
});
/** A recording that writes two scratch items and reads each write back at once, as an environment needs. */
const writeSession = (executable, origin) =>
browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: executable },
start_url: { url: `${origin}/` },
actions: [
{
step_id: "ready",
action: "wait_for",
locator: { kind: "css", selector: "#ready" },
state: "visible",
timeout_ms: 10_000,
},
read("list-1", "/api/items"),
read("list-2", "/api/items"),
read("item-1", "/api/items/1"),
write("create-1", "POST", "/api/items", {
json: { title: "Scratch item 1" },
status: 201,
}),
read("created-1", "/api/items/3"),
read("list-created-1", "/api/items"),
write("create-2", "POST", "/api/items", {
json: { title: "Scratch item 2" },
status: 201,
}),
read("created-2", "/api/items/4"),
read("list-created-2", "/api/items"),
pause("settle-creates"),
write("update-1", "PUT", "/api/items/3", {
json: { title: "Scratch item 1 renamed", done: true },
status: 200,
}),
read("updated-1", "/api/items/3"),
read("list-updated-1", "/api/items"),
write("update-2", "PUT", "/api/items/4", {
json: { title: "Scratch item 2 renamed", done: false },
status: 200,
}),
read("updated-2", "/api/items/4"),
read("list-updated-2", "/api/items"),
write("delete-1", "DELETE", "/api/items/3", { status: 204 }),
{ ...read("deleted-1", "/api/items/3"), expected_status: [404] },
read("list-deleted-1", "/api/items"),
write("delete-2", "DELETE", "/api/items/4", { status: 204 }),
{ ...read("deleted-2", "/api/items/4"), expected_status: [404] },
read("list-deleted-2", "/api/items"),
read("session", "/api/session"),
],
capture: {
at_end: ["url"],
network: { request_body: true, response_body: true, header_values: true },
},
});
/** Run the CLI under the owner, which stops it and every browser it launched on timeout or failure. */
const cliOf = (commands, entrypoint) => async (args, maxBuffer) =>
(
await commands.exec(process.execPath, [entrypoint, ...args, "--json"], {
env: {
...process.env,
REA_LOG_LEVEL: "silent",
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-provider/hopper",
},
timeout: 120_000,
maxBuffer,
})
).stdout;
const captureOf = async (cli, scenario, maxBuffer) => {
const stdout = await cli(
["capture-browser-scenario", JSON.stringify(scenario)],
maxBuffer,
);
const evidence = parseEvidence(JSON.parse(stdout));
return {
stdout,
evidence,
capture: browserScenarioCaptureSchema.parse(evidence.normalized_result),
};
};
/** Every request step completed and sent exactly its own request. */
const assertRequestsAttributed = (scenario, capture) => {
const steps = capture.steps.filter(
(step) => step.step_id !== "scenario_start",
);
assert.equal(steps.length, scenario.actions.length);
for (const step of steps)
assert.equal(step.status, "completed", JSON.stringify(step));
const requests = capture.events.items.filter(
(event) => event.kind === "request",
);
scenario.actions.forEach((action, index) => {
if (action.action !== "request") return;
const stepIndex = capture.steps.findIndex(
(step) => step.step_id === action.step_id,
);
assert.equal(capture.steps[stepIndex].action, "request");
const sent = requests.filter(
(event) =>
event.step_index === stepIndex &&
event.method === action.method &&
new URL(event.url.url).pathname === action.path,
);
assert.equal(sent.length, 1, `step ${String(index)} ${action.step_id}`);
});
};
/** The CSRF cookie and stored token reached the server but no record keeps them. */
const assertPageValuesRedacted = (evidence, capture) => {
// Bodies are base64 in the Evidence, so decode them before looking for the values.
const contents = capture.events.items.filter(
(event) => event.kind === "network-content",
);
const bodies = contents
.filter((event) => event.body.state === "captured")
.map((event) => Buffer.from(event.body.content, "base64").toString());
const headerValues = contents.flatMap((event) =>
event.headers.state === "captured"
? event.headers.items.map((item) => item.value ?? "")
: [],
);
const retained = [JSON.stringify(evidence), ...bodies, ...headerValues];
for (const value of [requestFixtureCsrf, requestFixtureToken])
assert.ok(
retained.every((text) => !text.includes(value)),
`${value} was retained`,
);
assert.ok(
headerValues.some((value) =>
value.includes("[REDACTED:page:create-1.X-CSRFToken]"),
),
"the CSRF header was not recorded redacted",
);
assert.ok(
bodies.some(
(body) =>
body.includes('"token"') && body.includes("[REDACTED:page:create-1."),
),
"the session response was not recorded redacted",
);
assert.ok(
capture.limitations.some((line) => line.startsWith("Request steps sent")),
);
};
/** A status outside expected_status fails the step and cancels the rest; a redirect to another origin is not followed. */
const assertUnexpectedStatusFails = async (cli, scenario, site) => {
const failing = browserScenarioSchema.parse({
...scenario,
actions: [
write("delete-missing", "DELETE", "/api/items/999", { status: 204 }),
read("after", "/api/items"),
],
});
const { capture } = await captureOf(cli, failing, 16 * 1024 * 1024);
const missing = capture.steps.find(
({ step_id }) => step_id === "delete-missing",
);
assert.equal(missing.status, "failed");
assert.match(missing.error ?? "", /answered 404; expected 204/u);
assert.equal(
capture.steps.find(({ step_id }) => step_id === "after").status,
"cancelled",
);
const redirected = browserScenarioSchema.parse({
...scenario,
actions: [read("elsewhere", "/api/elsewhere")],
});
const followed = await captureOf(cli, redirected, 16 * 1024 * 1024);
const step = followed.capture.steps.find(
({ step_id }) => step_id === "elsewhere",
);
assert.equal(step.status, "failed");
assert.match(step.error ?? "", /request steps do not follow/u);
// Chrome reports the redirected request and aborts it; the other origin must never receive it.
assert.ok(
site.hosts.every((line) => !line.startsWith("localhost")),
"a request step followed a redirect to another origin",
);
};
/** A genuinely busy page must not dispatch a write after its deadline or host cancellation. */
const assertQueuedWritesStopped = async (executable, site) => {
const browser = await chromium.launch({
executablePath: executable,
headless: true,
});
try {
const page = await browser.newPage({ serviceWorkers: "block" });
await page.goto(site.origin);
for (const cancelled of [false, true]) {
const scenario = browserScenarioSchema.parse({
...writeSession(executable, site.origin),
actions: [
{
...write("queued", "POST", "/api/items", {
json: { title: "must not be created" },
status: 201,
}),
headers: [
{
name: "X-CSRFToken",
value: { source: "literal", value: requestFixtureCsrf },
},
{
name: "Authorization",
value: {
source: "literal",
value: `Bearer ${requestFixtureToken}`,
},
},
],
timeout_ms: cancelled ? 5000 : 50,
},
],
});
const secrets = BrowserScenarioSecrets.resolve(scenario, {});
if (!secrets.ok) throw secrets.error;
// A preceding evaluation monopolizes the page thread while Node can
// still cancel. Its protocol request is queued before the request step.
const busy = page.evaluate(() => {
const until = Date.now() + 300;
while (Date.now() < until) {}
});
const controller = new AbortController();
const before = site.writes.length;
const request = performScenarioRequest({
page,
action: scenario.actions[0],
secrets: secrets.value,
startOrigin: site.origin,
signal: controller.signal,
});
const outcome = request.then(
() => ({ ok: true }),
(error) => ({ ok: false, error }),
);
const timer = cancelled
? setTimeout(() => controller.abort("queued request cancelled"), 30)
: undefined;
try {
const result = await outcome;
assert.equal(result.ok, false, "queued write unexpectedly completed");
assert.equal(
site.writes.length,
before,
"the server received a late write",
);
await busy;
} finally {
if (timer !== undefined) clearTimeout(timer);
}
}
} finally {
await browser.close();
}
};
/**
* Record a write session through request steps in a real browser, then build
* an environment from it: every request lands in its own step, the page's
* CSRF cookie and stored token authenticate the writes and never reach the
* Evidence, and the build serves the recorded writes.
*/
export async function verifyBrowserRequestSteps(
executable,
entrypoint = fileURLToPath(new URL("../rea.mjs", import.meta.url)),
) {
const site = await startBrowserRequestSite();
let root;
let commands;
const proof = async () => {
root = await PrivateRuntimeRoot.create({ prefix: "rea-request-e2e-" });
commands = await createPackageVerifierCommands({ workspace: root });
const cli = cliOf(commands, entrypoint);
const scenario = writeSession(executable, site.origin);
const { stdout, evidence, capture } = await captureOf(
cli,
scenario,
64 * 1024 * 1024,
);
assertRequestsAttributed(scenario, capture);
assert.equal(site.writes.length, 6);
assert.ok(site.writes.every(({ csrf, bearer }) => csrf && bearer));
assertPageValuesRedacted(evidence, capture);
const capturePath = join(root.path, "items.capture.json");
await writeFile(capturePath, stdout);
const built = JSON.parse(
await cli(
["environment-build", capturePath, join(root.path, "env")],
16 * 1024 * 1024,
),
);
assert.equal(built.status, "built");
assert.equal(built.name, "items");
const served = built.coverage.writes.served.map(({ tool }) => tool).sort();
assert.deepEqual(
served,
["delete_items_by_id", "post_items", "put_items_by_id"],
JSON.stringify(built.coverage),
);
await assertUnexpectedStatusFails(cli, scenario, site);
await assertQueuedWritesStopped(executable, site);
return {
request_steps: scenario.actions.filter(
({ action }) => action === "request",
).length,
authenticated_writes: site.writes.length,
served_writes: served,
tool_count: built.tool_count,
queued_writes_refused: 2,
};
};
// The owner stops its commands' processes before removing the root; a failed stop keeps the root and is reported.
return withVerifierCleanup(proof, [
() => (commands === undefined ? root?.close() : commands.close()),
() => site.close(),
]);
}
scripts/lib/catalog-builders.mjs
+1
View File
@@ -127,6 +127,7 @@ export const toolFamilyCatalog = (sources) => {
...sources.webScriptContracts.WEB_SCRIPT_TOOL_CONTRACTS,
...sources.javascriptRecoveryContracts
.JAVASCRIPT_RECOVERY_TOOL_CONTRACTS,
...sources.environmentContracts.ENVIRONMENT_TOOL_CONTRACTS,
],
},
{
scripts/lib/catalog-core.mjs
+1
View File
@@ -116,6 +116,7 @@ const SOURCE_PATHS = {
applicationContracts: "dist/contracts/applicationToolContracts.js",
analysisViewContracts: "dist/contracts/analysisViewToolContracts.js",
webScriptContracts: "dist/contracts/webScriptToolContracts.js",
environmentContracts: "dist/contracts/environmentToolContracts.js",
javascriptRecoveryContracts:
"dist/contracts/javascript/javascriptRecoveryToolContracts.js",
javascriptRecoveryProvider: "dist/javascript/recovery/WakaruRelease.js",
scripts/verify-agentenv-environment.mjs
+36 -56
View File
@@ -22,11 +22,6 @@ const QUOTA_RETRY_PAUSE_MS = 90_000;
const RUN_BUDGET_MS = 22 * 60_000;
const FIRST_ATTEMPT_MS = 15 * 60_000;
const MIN_RETRY_MS = 5 * 60_000;
const TEMPLATE_RULES = [
"after-uid-literal",
"opaque-token",
"sibling-variance",
];
// Both tools are caller-supplied absolute paths, as every real-* lane names its upstream.
for (const key of ["REA_AGENT_ENV_COMMAND", "REA_DOCKER_COMMAND"])
@@ -80,43 +75,6 @@ try {
);
const harPath = join(runtime.path, "capture.har");
await writeFile(harPath, JSON.stringify(environmentManifestHar()));
const capture = await cli(
"inspect-web-network-capture",
harPath,
"har",
"--format",
"json",
);
assert.equal(capture.operation, "inspect_web_network_capture");
assert.equal(capture.normalized_result.total_records, 28);
cases++;
const viewPath = join(runtime.path, "view.json");
await writeFile(
viewPath,
JSON.stringify({
source: { kind: "inline", evidence: capture },
view: { kind: "environment-manifest", template_rules: TEMPLATE_RULES },
}),
);
const manifest = await cli(
"inspect-analysis-view",
viewPath,
"--format",
"json",
);
assert.equal(manifest.normalized_result.kind, "environment-manifest");
assert.equal(
manifest.normalized_result.parent_evidence_id,
capture.evidence_id,
);
assert.equal(manifest.normalized_result.name, "grafana-example");
const toolCount = manifest.normalized_result.tools.length;
assert.ok(toolCount > 0);
cases++;
const capturePath = join(runtime.path, "capture.json");
const manifestPath = join(runtime.path, "manifest.json");
await writeFile(capturePath, JSON.stringify(capture));
await writeFile(manifestPath, JSON.stringify(manifest));
// The lane proves this checkout, not the registry package: pack it and install the tarball in the image.
const { stdout: packed } = await commands.execNpm(
["pack", "--ignore-scripts", "--json", "--pack-destination", runtime.path],
@@ -124,25 +82,43 @@ try {
);
const tarball = join(runtime.path, packedFilename(JSON.parse(packed)));
await access(tarball);
// The one-step build users run: it inspects the HAR, projects the manifest and writes the bundle.
const bundle = join(runtime.path, "grafana-example");
const exported = await cli(
"environment-export",
manifestPath,
"--evidence",
capturePath,
const built = await cli(
"environment-build",
harPath,
bundle,
"--package-tarball",
tarball,
);
assert.equal(exported.status, "exported");
assert.equal(exported.name, "grafana-example");
assert.equal(built.status, "built");
assert.equal(built.capture.operation, "inspect_web_network_capture");
assert.equal(built.capture.records, 28);
cases++;
assert.equal(built.name, "grafana-example");
const toolCount = built.manifest.tools;
assert.ok(toolCount > 0);
assert.deepEqual(built.manifest.template_rules, [
"after-uid-literal",
"opaque-token",
"sibling-variance",
"prefix-variance",
]);
cases++;
// The bundle serves the state its server derives from the capture, which leaves the
// manifest's write tools unserved; the export reports the count the bundle serves.
assert.equal(exported.state.served, true);
const servedCount = exported.tool_count;
// manifest's write tools unserved; the build reports the count the bundle serves.
assert.equal(built.state.served, true);
const servedCount = built.tool_count;
assert.ok(servedCount > 0 && servedCount < toolCount);
assert.equal(exported.package_source.kind, "tarball");
assert.deepEqual(exported.files.map((file) => file.path).sort(), [
// Every read tool and every served write tool is in the served catalog.
assert.equal(
built.coverage.reads.state +
built.coverage.reads.recorded +
built.coverage.writes.served.length,
servedCount,
);
assert.equal(built.package_source.kind, "tarball");
assert.deepEqual(built.files.map((file) => file.path).sort(), [
"README.md",
"artifacts/check_grafana-example/verify.py",
"envs/grafana-example/Dockerfile",
@@ -152,9 +128,13 @@ try {
"envs/grafana-example/rea-agents.tgz",
"tasks/smoke.json",
]);
assert.equal(
const shipped = JSON.parse(
await readFile(join(bundle, "envs/grafana-example/manifest.json"), "utf8"),
await readFile(manifestPath, "utf8"),
);
assert.equal(shipped.evidence_id, built.manifest.evidence_id);
assert.equal(
shipped.normalized_result.parent_evidence_id,
built.capture.evidence_id,
);
const dockerfile = await readFile(
join(bundle, "envs/grafana-example/Dockerfile"),
scripts/verify-browser-request.mjs
+18
View File
@@ -0,0 +1,18 @@
#!/usr/bin/env node
import { access } from "node:fs/promises";
import { verifyBrowserRequestSteps } from "./lib/browser-request-e2e.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
import { requireOrdinaryWindowsVerifierCaller } from "./lib/windows-ordinary-user-verifier.mjs";
const executable = process.env.REA_BROWSER_EXECUTABLE;
if (executable === undefined || executable === "")
throw new Error(
"verify:browser:request requires REA_BROWSER_EXECUTABLE pointing to an installed Chrome-family browser",
);
await requireOrdinaryWindowsVerifierCaller();
await access(executable);
const run = createVerifierRun();
const proof = await verifyBrowserRequestSteps(executable, process.argv[2]);
process.stdout.write(
`${JSON.stringify({ ...proof, verifier_run: await completeVerifierRun(run), verified: true })}\n`,
);
src/application/SetupSkill.ts
+1
View File
@@ -30,6 +30,7 @@ export const MANAGED_SKILL_FILES = [
"references/android-applications.md",
"references/runtime-observation.md",
"references/evidence-workflows.md",
"references/environments.md",
] as const;
export const isManagedSkillManifest = (content: string): boolean =>
src/application/analysisView/AnalysisViewService.ts
+4 -2
View File
@@ -193,7 +193,8 @@ export const inspectAnalysisViewValidated = (
)
: undefined;
const stateAccumulator =
input.view.kind === "environment-state"
input.view.kind === "environment-state" ||
input.view.kind === "environment-coverage"
? new EnvironmentStateAccumulator(undefined, input.view.template_rules)
: undefined;
const accumulator =
@@ -387,8 +388,9 @@ const createAnalysisViewEvidence = (
rawResult: null,
confidence:
// Grouping is whole-capture, so any templated group or parameter makes the result inferred;
// entity types and identity in an extracted state always are.
// entity types and identity in an extracted state, and the coverage summarizing it, always are.
result.kind === "environment-state" ||
result.kind === "environment-coverage" ||
((result.kind === "api-surface" ||
result.kind === "environment-manifest") &&
result.templated_groups > 0)
src/application/environment/environmentBuild.test.ts
+881
View File
@@ -0,0 +1,881 @@
import { evidenceFileBytes } from "./environmentEvidenceBytes.js";
import {
chmod,
mkdir,
open,
readdir,
readFile,
writeFile,
type FileHandle,
} from "node:fs/promises";
import { join, sep } from "node:path";
import { Writable } from "node:stream";
import { describe, expect, it, vi } from "vitest";
import {
analysisViewCaptureEvidence,
environmentManifestCaptureFixture,
} from "../../../tests/fixtures/analysisView.js";
import {
scenarioCaptureEvidence,
scenarioCaptureFixture,
} from "../../../tests/fixtures/browserScenarioCapture.js";
import { environmentManifestHar } from "../../../tests/fixtures/historicalHar.js";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { ArtifactFileCleanupFailure } from "../../artifacts/ArtifactFileOperation.js";
import { writeJsonOutput } from "../../cli/jsonOutput.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { analysisViewRequestSchema } from "../../domain/analysisView/analysisView.js";
import { summarizeEnvironmentCoverage } from "../../domain/analysisView/environmentCoverageView.js";
import type { Evidence } from "../../domain/evidence.js";
import { inspectAnalysisViewValidated } from "../analysisView/AnalysisViewService.js";
import {
DEFAULT_BUILD_TEMPLATE_RULES,
buildEnvironment,
canonicalTemplateRules,
classifyBuildCapture,
type EnvironmentBuildInput,
} from "./environmentBuild.js";
import {
exportEnvironmentBundle,
readEnvironmentExportFile,
readEnvironmentExportInputs,
} from "./environmentExport.js";
import { loadEnvironmentSource } from "./environmentSource.js";
import { prepareServedState } from "./environmentState.js";
const listFiles = async (root: string): Promise<string[]> =>
(await readdir(root, { recursive: true, withFileTypes: true }))
.filter((entry) => entry.isFile())
.map((entry) =>
join(entry.parentPath, entry.name)
.slice(root.length + 1)
.split(sep)
.join("/"),
)
.sort();
/** The Grafana-shaped capture with every recorded URL moved to a loopback origin. */
const loopbackCapture = (): Evidence =>
analysisViewCaptureEvidence(
JSON.parse(
JSON.stringify(environmentManifestCaptureFixture()).replaceAll(
/https?:\/\/grafana\.example/gu,
"http://127.0.0.1:3000",
),
) as ReturnType<typeof environmentManifestCaptureFixture>,
);
/** Write capture Evidence as `--format json` writes it, then read it as the build command does. */
const captureFile = async (
root: string,
evidence: Evidence,
name = "grafana.capture.json",
) => {
const path = join(root, name);
await writeFile(path, evidenceFileBytes(evidence));
const read = await readEnvironmentExportFile(
"evidence",
path,
"environment-build",
);
if (!read.ok) throw read.error;
return read.value;
};
const buildInput = async (
root: string,
evidence: Evidence = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
),
overrides: Partial<EnvironmentBuildInput> = {},
): Promise<EnvironmentBuildInput> => {
const capture = await captureFile(root, evidence);
return {
capture,
captureInput: "evidence",
capturePath: capture.path,
templateRules: DEFAULT_BUILD_TEMPLATE_RULES,
directory: join(root, "built"),
dryRun: false,
...overrides,
};
};
/** The three-step path: project the manifest, write both files, export them. */
const threeStep = async (
root: string,
evidence: Evidence,
name: string | undefined,
) => {
const manifest = inspectAnalysisViewValidated({
source: { kind: "inline", evidence },
view: analysisViewRequestSchema.parse({
kind: "environment-manifest",
template_rules: [...DEFAULT_BUILD_TEMPLATE_RULES],
}),
});
if (!manifest.ok) throw manifest.error;
const manifestPath = join(root, "manifest.json");
const capturePath = join(root, "capture.json");
await writeFile(manifestPath, evidenceFileBytes(manifest.value));
await writeFile(capturePath, evidenceFileBytes(evidence));
const inputs = await readEnvironmentExportInputs(manifestPath, capturePath);
if (!inputs.ok) throw inputs.error;
const source = loadEnvironmentSource(
inputs.value.manifest.value,
inputs.value.evidence.value,
);
if (!source.ok) throw source.error;
const directory = join(root, "three-step");
const exported = await exportEnvironmentBundle({
...inputs.value,
source: source.value,
directory,
name,
});
if (!exported.ok) throw exported.error;
return { directory, exported: exported.value };
};
/** The prototype every host file handle shares, so a test can fail one handle's host call. */
const fileHandlePrototype = async (path: string): Promise<FileHandle> => {
const handle = await open(path, "r");
await handle.close();
return Object.getPrototypeOf(handle) as FileHandle;
};
const sameTree = async (left: string, right: string): Promise<number> => {
const files = await listFiles(left);
expect(await listFiles(right)).toEqual(files);
for (const file of files)
expect(await readFile(join(left, file)), file).toEqual(
await readFile(join(right, file)),
);
return files.length;
};
describe("buildEnvironment", () => {
it("writes the bundle the three-step path writes, byte for byte", async () => {
const root = await createTestTempDirectory("rea-environment-build-same-");
const evidence = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
);
const input = await buildInput(root, evidence);
const built = await buildEnvironment(input);
if (!built.ok) throw built.error;
const reference = await threeStep(
await createTestTempDirectory("rea-environment-build-ref-"),
evidence,
undefined,
);
expect(await sameTree(input.directory!, reference.directory)).toBe(7);
expect(built.value).toMatchObject({
status: "built",
name: "grafana-example",
capture: {
evidence_id: evidence.evidence_id,
operation: "inspect_web_network_capture",
input: "evidence",
},
manifest: {
evidence_id: reference.exported.manifest_evidence_id,
template_rules: [...DEFAULT_BUILD_TEMPLATE_RULES],
},
state: reference.exported.state,
tool_count: reference.exported.tool_count,
package_source: reference.exported.package_source,
files: reference.exported.files,
warnings: [],
next: [`agent-env run ${input.directory!}`],
});
});
it("reports the coverage of the state the bundle serves", async () => {
const root = await createTestTempDirectory("rea-environment-build-cov-");
const evidence = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
);
const built = await buildEnvironment(
await buildInput(root, evidence, { dryRun: true, directory: undefined }),
);
if (!built.ok) throw built.error;
const manifest = inspectAnalysisViewValidated({
source: { kind: "inline", evidence },
view: analysisViewRequestSchema.parse({
kind: "environment-manifest",
template_rules: [...DEFAULT_BUILD_TEMPLATE_RULES],
}),
});
if (!manifest.ok) throw manifest.error;
const source = loadEnvironmentSource(manifest.value, evidence);
if (!source.ok) throw source.error;
const { state } = prepareServedState(source.value, evidence);
if (state === undefined) throw new Error("Expected a served state");
expect(built.value.coverage).toEqual(
summarizeEnvironmentCoverage({
views: state.views.values(),
writes: state.writes.values(),
}),
);
expect(
built.value.coverage!.reads.state + built.value.coverage!.reads.recorded,
).toBe(state.views.size);
});
it("plans the same files without writing anything on a dry run", async () => {
const root = await createTestTempDirectory("rea-environment-build-dry-");
const planned = await buildEnvironment(
await buildInput(root, undefined, { dryRun: true }),
);
if (!planned.ok) throw planned.error;
await expect(readdir(join(root, "built"))).rejects.toMatchObject({
code: "ENOENT",
});
const built = await buildEnvironment(await buildInput(root));
if (!built.ok) throw built.error;
const { status, next, ...plannedRest } = planned.value;
const { status: builtStatus, next: builtNext, ...builtRest } = built.value;
expect([status, builtStatus]).toEqual(["planned", "built"]);
expect(next).toEqual([]);
expect(builtNext).toHaveLength(1);
expect(plannedRest).toEqual(builtRest);
});
});
describe("buildEnvironment naming and refusals", () => {
it("names a loopback capture after its file and says so", async () => {
const root = await createTestTempDirectory("rea-environment-build-name-");
const evidence = loopbackCapture();
const input = await buildInput(root, evidence);
const built = await buildEnvironment(input);
if (!built.ok) throw built.error;
expect(built.value.name).toBe("grafana");
expect(built.value.warnings).toEqual([
expect.objectContaining({
code: "name-from-file",
message: expect.stringContaining("http://127.0.0.1:3000"),
}),
]);
const reference = await threeStep(
await createTestTempDirectory("rea-environment-build-name-ref-"),
evidence,
"grafana",
);
await sameTree(input.directory!, reference.directory);
const named = await buildEnvironment({
...input,
name: "my-grafana",
directory: join(root, "named"),
});
if (!named.ok) throw named.error;
expect(named.value.name).toBe("my-grafana");
expect(named.value.warnings).toEqual([]);
});
it("refuses a loopback capture whose file name yields no name", async () => {
const root = await createTestTempDirectory("rea-environment-build-noname-");
const capture = await captureFile(root, loopbackCapture(), "--.json");
const built = await buildEnvironment({
capture,
captureInput: "evidence",
capturePath: capture.path,
templateRules: DEFAULT_BUILD_TEMPLATE_RULES,
directory: join(root, "built"),
dryRun: false,
});
if (built.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(built.error)).toMatchObject({
code: "invalid_request",
details: {
operation: "environment-build",
issues: [{ path: ["name"] }],
},
});
await expect(readdir(join(root, "built"))).rejects.toMatchObject({
code: "ENOENT",
});
});
it("refuses evidence that is not a capture, and a build without a directory", async () => {
const root = await createTestTempDirectory("rea-environment-build-refuse-");
const evidence = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
);
const manifest = inspectAnalysisViewValidated({
source: { kind: "inline", evidence },
view: analysisViewRequestSchema.parse({ kind: "environment-manifest" }),
});
if (!manifest.ok) throw manifest.error;
const notCapture = await buildEnvironment(
await buildInput(root, manifest.value),
);
if (notCapture.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(notCapture.error)).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["capture", "operation"] }] },
});
const noDirectory = await buildEnvironment(
await buildInput(root, evidence, { directory: undefined }),
);
if (noDirectory.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(noDirectory.error)).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["directory"] }] },
});
await expect(readdir(join(root, "built"))).rejects.toMatchObject({
code: "ENOENT",
});
});
it("refuses an occupied directory and leaves it as it was", async () => {
const root = await createTestTempDirectory("rea-environment-build-busy-");
const input = await buildInput(root);
await mkdir(input.directory!);
await writeFile(join(input.directory!, "notes.md"), "keep");
const built = await buildEnvironment(input);
if (built.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(built.error).code).toBe("invalid_request");
expect(await listFiles(input.directory!)).toEqual(["notes.md"]);
});
});
describe("buildEnvironment cancellation and input checks", () => {
it("writes nothing when cancelled before the bundle stage", async () => {
const root = await createTestTempDirectory("rea-environment-build-stop-");
const controller = new AbortController();
const input = await buildInput(root, undefined, {
signal: controller.signal,
progress: (stage) => {
if (stage === "state") controller.abort();
},
});
const built = await buildEnvironment(input);
if (built.ok) throw new Error("Expected a cancellation");
expect(projectAnalysisError(built.error)).toMatchObject({
code: "cancelled",
details: { operation: "environment-build" },
});
await expect(readdir(input.directory!)).rejects.toMatchObject({
code: "ENOENT",
});
});
it("names the written bundle when cancelled during the write", async () => {
const root = await createTestTempDirectory("rea-environment-build-late-");
const controller = new AbortController();
const input = await buildInput(root, undefined, {
signal: controller.signal,
progress: (stage) => {
if (stage === "bundle") controller.abort();
},
});
const built = await buildEnvironment(input);
if (built.ok) throw new Error("Expected a cancellation");
const projected = projectAnalysisError(built.error);
expect(projected.code).toBe("cancelled");
expect(JSON.stringify(projected)).toContain(
`"output_directory":${JSON.stringify(input.directory)}`,
);
expect(await listFiles(input.directory!)).toHaveLength(7);
});
it("refuses repeated rules and a HAR document saved as JSON", async () => {
const root = await createTestTempDirectory("rea-environment-build-input-");
const repeated = await buildEnvironment(
await buildInput(root, undefined, {
templateRules: ["opaque-token", "opaque-token"],
}),
);
if (repeated.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(repeated.error)).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["template_rules"] }] },
});
const harPath = join(root, "grafana.json");
await writeFile(harPath, JSON.stringify({ log: { entries: [] } }));
const har = await readEnvironmentExportFile("evidence", harPath);
if (!har.ok) throw har.error;
const asJson = await buildEnvironment({
capture: har.value,
captureInput: "evidence",
capturePath: harPath,
templateRules: DEFAULT_BUILD_TEMPLATE_RULES,
dryRun: true,
});
if (asJson.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(asJson.error)).toMatchObject({
code: "invalid_request",
message: expect.stringContaining(".har"),
details: { issues: [{ path: ["capture"] }] },
});
});
it("refuses an unreadable package tarball on a dry run as a build does", async () => {
if (process.getuid?.() === 0) return;
const root = await createTestTempDirectory("rea-environment-build-tgz-");
const tarball = join(root, "rea-agents.tgz");
await writeFile(tarball, "packed");
await chmod(tarball, 0o000);
try {
const planned = await buildEnvironment(
await buildInput(root, undefined, {
dryRun: true,
packageTarball: tarball,
}),
);
const built = await buildEnvironment(
await buildInput(root, undefined, { packageTarball: tarball }),
);
if (planned.ok || built.ok) throw new Error("Expected both refusals");
expect(projectAnalysisError(planned.error).code).toBe("access_denied");
expect(projectAnalysisError(built.error).code).toBe("access_denied");
} finally {
await chmod(tarball, 0o600);
}
});
it("refuses a loopback capture whose file name is too long, and quotes the next command", async () => {
const root = await createTestTempDirectory("rea-environment-build-long-");
const long = await captureFile(
root,
loopbackCapture(),
`${"a".repeat(70)}.capture.json`,
);
const refused = await buildEnvironment({
capture: long,
captureInput: "evidence",
capturePath: long.path,
templateRules: DEFAULT_BUILD_TEMPLATE_RULES,
dryRun: true,
});
if (refused.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(refused.error)).toMatchObject({
details: { operation: "environment-build", issues: [{ path: ["name"] }] },
});
const built = await buildEnvironment(
await buildInput(root, undefined, { directory: join(root, "my bundle") }),
);
if (!built.ok) throw built.error;
expect(built.value.next).toEqual([
`agent-env run '${join(root, "my bundle")}'`,
]);
});
});
describe("classifyBuildCapture", () => {
it("inspects .har files, reads .json Evidence and refuses anything else", () => {
expect(classifyBuildCapture("/x/grafana.HAR")).toEqual({
ok: true,
value: "har",
});
expect(classifyBuildCapture("/x/grafana.capture.json")).toEqual({
ok: true,
value: "evidence",
});
const refused = classifyBuildCapture("/x/grafana.saz");
if (refused.ok) throw new Error("Expected a refusal");
expect(projectAnalysisError(refused.error)).toMatchObject({
code: "invalid_request",
details: {
operation: "environment-build",
issues: [{ path: ["capture"] }],
},
});
});
});
describe("canonicalTemplateRules", () => {
it("orders rules as the views apply them", () => {
expect(
canonicalTemplateRules(["prefix-variance", "after-uid-literal"]),
).toEqual(["after-uid-literal", "prefix-variance"]);
});
});
describe("buildEnvironment credential names", () => {
it("flags the credential-named headers and query parameters a scenario capture retains, as a HAR's", async () => {
const root = await createTestTempDirectory("rea-environment-build-scan-");
const evidence = scenarioCaptureEvidence(
scenarioCaptureFixture(
[
{
method: "GET",
url: "/api/users/2?api_key=k-1&page=1",
status: 200,
mediaType: "application/json",
body: '{"id":2,"login":"probe"}',
requestHeaders: [
{ name: "X-Api-Key", value: "k-123" },
{ name: "Accept", value: "application/json" },
{ name: "Authorization", value: null },
],
responseHeaders: [{ name: "X-Auth-Token", value: "t-9" }],
},
],
{ request_body: true, response_body: true, header_values: true },
),
);
const planned = await buildEnvironment(
await buildInput(root, evidence, {
dryRun: true,
directory: undefined,
name: "rota",
}),
);
if (!planned.ok) throw planned.error;
expect(planned.value.capture.records).toBe(1);
expect(planned.value.warnings).toEqual([
expect.objectContaining({
code: "credential-named-members",
pointers: [
"event 0 request query api_key",
"event 1 request header X-Api-Key",
"event 3 response header X-Auth-Token",
],
}),
]);
const report = JSON.stringify(planned.value);
for (const value of ["k-123", "t-9"]) expect(report).not.toContain(value);
});
});
/** A scenario capture whose event list `edit` rewrote, with its retained count kept consistent. */
const editedScenarioEvidence = (
capture: ReturnType<typeof scenarioCaptureFixture>,
edit: (
items: ReturnType<typeof scenarioCaptureFixture>["events"]["items"],
) => ReturnType<typeof scenarioCaptureFixture>["events"]["items"],
): Evidence => {
const items = edit(capture.events.items);
return scenarioCaptureEvidence({
...capture,
events: { ...capture.events, retained: items.length, items },
});
};
/** Drop a transaction's request event, as a capture attached after the request began records it. */
const attachedMidFlight =
(transaction: string) =>
(items: ReturnType<typeof scenarioCaptureFixture>["events"]["items"]) =>
items.filter(
(event) =>
event.kind !== "request" || event.transaction_id !== transaction,
);
const plannedScan = async (prefix: string, evidence: Evidence) => {
const root = await createTestTempDirectory(prefix);
const planned = await buildEnvironment(
await buildInput(root, evidence, {
dryRun: true,
directory: undefined,
name: "rota",
}),
);
if (!planned.ok) throw planned.error;
return planned.value;
};
const USERS_EXCHANGE = {
method: "GET",
url: "/api/users/2",
status: 200,
mediaType: "application/json",
body: '{"id":2,"login":"probe"}',
} as const;
describe("buildEnvironment credential scan of scenario events", () => {
it("scans retained content that no record represents: a response attached mid-flight and an ambiguous transaction", async () => {
const capture = scenarioCaptureFixture(
[
USERS_EXCHANGE,
{
method: "GET",
url: "/api/session?api_key=k-mid",
status: 200,
mediaType: "application/json",
body: '{"token":"t-mid"}',
responseHeaders: [{ name: "X-Auth-Token", value: "t-header" }],
},
{
method: "GET",
url: "/api/me",
status: 200,
mediaType: "application/json",
body: '{"sessionId":"s-ambiguous"}',
},
],
{ request_body: true, response_body: true, header_values: true },
);
const duplicate = capture.events.items.find(
(event) =>
event.kind === "response" && event.transaction_id === "request-3",
);
if (duplicate === undefined) throw new Error("Fixture lacks a response");
// The session's request began before attachment; the third transaction recorded two responses.
const planned = await plannedScan(
"rea-environment-build-events-",
editedScenarioEvidence(capture, (items) => [
...attachedMidFlight("request-2")(items),
{ ...duplicate, sequence: 99 },
]),
);
expect(planned.capture.records).toBe(2);
expect(planned.warnings).toEqual([
expect.objectContaining({
code: "credential-named-members",
pointers: [
"event 3 response query api_key",
"event 4 response header X-Auth-Token",
"event 4 response /token",
"event 7 response /sessionId",
],
}),
]);
const report = JSON.stringify(planned);
for (const value of ["k-mid", "t-mid", "t-header", "s-ambiguous"])
expect(report).not.toContain(value);
});
it("reports a response attached mid-flight that is too large to scan", async () => {
const capture = scenarioCaptureFixture([
USERS_EXCHANGE,
{
method: "GET",
url: "/api/export",
status: 200,
mediaType: "application/json",
body: JSON.stringify({
token: "t-large",
pad: "a".repeat(4 * 1024 * 1024),
}),
},
]);
const planned = await plannedScan(
"rea-environment-build-events-large-",
editedScenarioEvidence(capture, attachedMidFlight("request-2")),
);
expect(planned.warnings).toEqual([
expect.objectContaining({
code: "credential-scan-incomplete",
pointers: ["event 4 response"],
}),
]);
});
});
describe("buildEnvironment credential scan coverage", () => {
it("warns that a body too large to scan was not scanned", async () => {
const root = await createTestTempDirectory("rea-environment-build-large-");
const [entry] = environmentManifestHar().log.entries;
const text = JSON.stringify({
token: "t-large",
pad: "a".repeat(4 * 1024 * 1024),
});
const evidence = analysisViewCaptureEvidence(
environmentManifestCaptureFixture([
{
...entry!,
response: {
...entry!.response,
content: { ...entry!.response.content, text },
},
},
]),
);
const planned = await buildEnvironment(
await buildInput(root, evidence, {
dryRun: true,
directory: undefined,
name: "large",
}),
);
if (!planned.ok) throw planned.error;
expect(planned.value.warnings).toEqual([
expect.objectContaining({
code: "credential-scan-incomplete",
pointers: ["record 0 response"],
}),
]);
expect(JSON.stringify(planned.value.warnings)).not.toContain("t-large");
});
});
describe("buildEnvironment dry-run package tarball", () => {
it("refuses a tarball whose descriptor stats successfully but fails to read", async () => {
const root = await createTestTempDirectory("rea-environment-build-read-");
const tarball = join(root, "rea-agents.tgz");
await writeFile(tarball, "packed");
const input = await buildInput(root, undefined, {
dryRun: true,
packageTarball: tarball,
});
const prototype = await fileHandlePrototype(tarball);
const failure = Object.assign(
new Error("controlled tarball read failure"),
{ code: "EIO" },
);
const read = vi.spyOn(prototype, "read").mockRejectedValue(failure);
try {
const planned = await buildEnvironment(input);
expect(read).toHaveBeenCalled();
if (planned.ok) throw new Error("Expected a tarball read refusal");
expect(projectAnalysisError(planned.error)).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["package_tarball"] }] },
});
expect(await readdir(root)).not.toContain("bundle");
} finally {
read.mockRestore();
}
});
it.each([false, true])(
"retries a failed tarball close and retains its owner when the failure persists: %s",
async (persistent) => {
const root = await createTestTempDirectory(
"rea-environment-build-close-",
);
const tarball = join(root, "rea-agents.tgz");
await writeFile(tarball, "packed");
const input = await buildInput(root, undefined, {
dryRun: true,
packageTarball: tarball,
});
const prototype = await fileHandlePrototype(tarball);
const stat = prototype.stat;
let owner: FileHandle | undefined;
let close: (() => Promise<void>) | undefined;
let attempts = 0;
let released = false;
const failClose = (handle: FileHandle): void => {
const original = handle.close.bind(handle);
owner = handle;
close = original;
handle.close = () => {
attempts += 1;
return !released && (persistent || attempts === 1)
? Promise.reject(
Object.assign(new Error("controlled tarball close failure"), {
code: "EIO",
}),
)
: original();
};
};
// The tarball is the only file a dry run opens; fail its handle's close.
const spy = vi.spyOn(prototype, "stat").mockImplementation(function (
this: FileHandle,
...args: unknown[]
) {
if (owner === undefined) failClose(this);
return Reflect.apply(stat, this, args) as ReturnType<
FileHandle["stat"]
>;
} as FileHandle["stat"]);
try {
const planned = await buildEnvironment(input);
if (!persistent) {
if (!planned.ok) throw planned.error;
expect(attempts).toBe(2);
expect(planned.value.files.at(-1)).toEqual({
path: expect.stringMatching(/\.tgz$/u),
bytes: 6,
});
return;
}
if (planned.ok) throw new Error("Expected a cleanup refusal");
expect(planned.error).toMatchObject({
_tag: "AnalysisOutputError",
operation: "environment-export",
cleanupIncomplete: true,
});
const failure = planned.error.cause;
if (!(failure instanceof ArtifactFileCleanupFailure))
throw new Error("Missing exact cleanup owner");
released = true;
await failure.cleanupLease.release();
expect(planned.error.cleanupIncomplete).toBe(false);
expect(owner?.fd).toBe(-1);
} finally {
released = true;
spy.mockRestore();
if (owner !== undefined && owner.fd >= 0) await close?.();
}
},
);
it("is cancelled by an interrupt that arrives while it reads the tarball", async () => {
const root = await createTestTempDirectory(
"rea-environment-build-tgz-stop-",
);
const tarball = join(root, "rea-agents.tgz");
await writeFile(tarball, "packed");
const controller = new AbortController();
const input = await buildInput(root, undefined, {
dryRun: true,
packageTarball: tarball,
signal: controller.signal,
});
const prototype = await fileHandlePrototype(tarball);
const original = prototype.stat;
const stat = vi.spyOn(prototype, "stat").mockImplementation(function (
this: FileHandle,
...args: unknown[]
) {
controller.abort();
return Reflect.apply(original, this, args) as ReturnType<
FileHandle["stat"]
>;
} as FileHandle["stat"]);
try {
const planned = await buildEnvironment(input);
if (planned.ok) throw new Error("Expected a cancellation");
expect(projectAnalysisError(planned.error)).toMatchObject({
code: "cancelled",
details: { operation: "environment-build" },
});
} finally {
stat.mockRestore();
}
});
});
describe("evidenceFileBytes", () => {
it("writes the bytes --format json writes, into one buffer", async () => {
// A long multibyte body with a surrogate pair across the serializer's first part boundary.
const [entry] = environmentManifestHar().log.entries;
const text = `${"a".repeat(8191)}😀${"é😀".repeat(40_000)}`;
const evidence = analysisViewCaptureEvidence(
environmentManifestCaptureFixture([
{
...entry!,
response: {
...entry!.response,
content: { ...entry!.response.content, text },
},
},
]),
);
const written: Buffer[] = [];
await writeJsonOutput(
evidence,
new Writable({
write(chunk: Buffer, _encoding, done) {
written.push(chunk);
done();
},
}),
);
const expected = Buffer.concat(written);
const concat = vi.spyOn(Buffer, "concat");
try {
const bytes = evidenceFileBytes(evidence);
expect(concat).not.toHaveBeenCalled();
expect(bytes.length).toBeGreaterThan(200_000);
expect(bytes.equals(expected)).toBe(true);
} finally {
concat.mockRestore();
}
});
});
src/application/environment/environmentBuild.ts
+409
View File
@@ -0,0 +1,409 @@
import { setImmediate } from "node:timers/promises";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
import { summarizeEnvironmentCoverage } from "../../domain/analysisView/environmentCoverageView.js";
import {
TEMPLATE_RULES,
type TemplateRule,
} from "../../domain/analysisView/environmentManifestRules.js";
import type { Evidence } from "../../domain/evidence.js";
import type { EnvironmentBuildReport } from "../../domain/environmentBundleExport.js";
import { err, ok, type Result } from "../../domain/result.js";
import { inspectAnalysisView } from "../analysisView/AnalysisViewService.js";
import {
ENVIRONMENT_EXPORT_OPERATION,
isPortableBundleName,
planEnvironmentBundle,
registryPackageSource,
type EnvironmentPackageSource,
} from "./environmentBundle.js";
import {
anonymousOrigin,
credentialScanWarnings,
nameFromCaptureFile,
type EnvironmentBuildWarning,
} from "./environmentBuildReport.js";
import {
environmentExportEvidenceFile,
exportEnvironmentBundle,
packageTarballBytes,
type EnvironmentExportFile,
type WrittenBundleFile,
} from "./environmentExport.js";
import {
loadEnvironmentSource,
parseCaptureEvidence,
parseEvidenceFile,
} from "./environmentSource.js";
import { prepareServedState } from "./environmentState.js";
/** Operation name the build command's own refusals report; each stage keeps its own. */
export const ENVIRONMENT_BUILD_OPERATION = "environment-build";
/** The opt-in rules the rebuild pipeline selects for every published capture-built environment. */
export const DEFAULT_BUILD_TEMPLATE_RULES: readonly TemplateRule[] = [
"after-uid-literal",
"opaque-token",
"sibling-variance",
"prefix-variance",
];
/** What the capture argument is, decided from its file name before anything is read. */
export type EnvironmentBuildCaptureInput = "har" | "evidence";
/** A HAR file is inspected; a JSON file must already be capture Evidence. */
export const classifyBuildCapture = (
path: string,
): Result<EnvironmentBuildCaptureInput, AnalysisInputError> => {
if (/\.har$/iu.test(path)) return ok("har");
if (/\.json$/iu.test(path)) return ok("evidence");
return err(
new AnalysisInputError(
ENVIRONMENT_BUILD_OPERATION,
{
userMessage: `${path} is neither a .har capture nor a .json capture Evidence file.`,
},
[
{
path: ["capture"],
reason: "invalid_value",
expected: ".har or .json",
message:
"Pass a HAR file, or capture Evidence JSON from inspect-web-network-capture (HAR) or capture-browser-scenario.",
},
],
),
);
};
/** Rules in the views' fixed order, so the report and manifest list them canonically. */
export const canonicalTemplateRules = (
rules: readonly TemplateRule[],
): TemplateRule[] => TEMPLATE_RULES.filter((rule) => rules.includes(rule));
export interface EnvironmentBuildInput {
/** The capture Evidence the bundle ships, read or serialized once. */
readonly capture: EnvironmentExportFile;
readonly captureInput: EnvironmentBuildCaptureInput;
/** The capture argument, absolute; names a loopback capture's environment. */
readonly capturePath: string;
readonly templateRules: readonly TemplateRule[];
/** Absolute bundle root, absent or empty; may be absent only for a dry run. */
readonly directory?: string | undefined;
readonly name?: string | undefined;
readonly packageTarball?: string | undefined;
readonly dryRun: boolean;
/** Called as each stage starts. */
readonly progress?: ((stage: string) => void) | undefined;
/** Checked between stages; a build cancelled before the write writes nothing. */
readonly signal?: AbortSignal | undefined;
/** Refuse an existing directory and never create its parent, as MCP output directories require. */
readonly exclusiveDestination?: boolean | undefined;
}
/** The report environment-build prints and export_environment_bundle returns. */
export type { EnvironmentBuildReport };
const captureOperations = new Set([
"inspect_web_network_capture",
"capture_browser_scenario",
]);
const harDocument = (value: unknown): boolean => {
if (typeof value !== "object" || value === null) return false;
const log: unknown = Reflect.get(value, "log");
return (
typeof log === "object" &&
log !== null &&
Array.isArray(Reflect.get(log, "entries"))
);
};
const harAsJson = (path: string): AnalysisInputError =>
new AnalysisInputError(
ENVIRONMENT_BUILD_OPERATION,
{
userMessage: `${path} is a HAR document, not capture Evidence; save it as a .har file to build from it, or inspect it as a HAR first (inspect_web_network_capture) and build from that Evidence.`,
},
[
{
path: ["capture"],
reason: "invalid_value",
expected: "capture Evidence JSON, or a .har file",
message:
"A .json capture must be capture Evidence; a HAR document is inspected only from a .har file.",
},
],
);
const repeatedRules = (): AnalysisInputError =>
new AnalysisInputError(ENVIRONMENT_BUILD_OPERATION, undefined, [
{
path: ["template_rules"],
reason: "invalid_value",
message: "template_rules must not repeat a rule",
},
]);
/** A cancellation, naming the bundle when it arrived after the bundle was written. */
const cancelled = async (
signal: AbortSignal | undefined,
written?: string,
): Promise<AnalysisCancelledError | undefined> => {
// The stages are synchronous; yield so a pending interrupt is dispatched before checking.
if (signal !== undefined) await setImmediate();
return signal?.aborted === true
? new AnalysisCancelledError(ENVIRONMENT_BUILD_OPERATION, {
cause: signal.reason,
...(written === undefined
? {}
: {
userMessage: `The build was cancelled after it wrote the bundle at ${written}.`,
diagnostics: { output_directory: written, bundle_written: true },
}),
})
: undefined;
};
const notCaptureEvidence = (operation: string): AnalysisInputError =>
new AnalysisInputError(
ENVIRONMENT_BUILD_OPERATION,
{
userMessage: `The capture file is ${operation} Evidence, not capture Evidence.`,
},
[
{
path: ["capture", "operation"],
reason: "invalid_value",
expected: "inspect_web_network_capture or capture_browser_scenario",
message:
"Pass a HAR file, or capture Evidence JSON from inspect-web-network-capture (HAR) or capture-browser-scenario.",
},
],
);
const unnamed = (path: string): AnalysisInputError =>
new AnalysisInputError(
ENVIRONMENT_BUILD_OPERATION,
{
userMessage: `The capture's origin is a loopback or IP address, and ${path === "" ? "retained Evidence has no file name to take one from" : `${path} yields no portable environment name (a kebab-case label of at most 64 characters that is not a Windows device name)`}; name the environment.`,
},
[
{
path: ["name"],
reason: "invalid_value",
message:
"Name the environment: a kebab-case label of lowercase letters, digits and single hyphens.",
},
],
);
const missingDirectory = (): AnalysisInputError =>
new AnalysisInputError(
ENVIRONMENT_BUILD_OPERATION,
{ userMessage: "Name the bundle directory, or plan a dry run." },
[
{
path: ["directory"],
reason: "invalid_value",
message: "The bundle directory may be omitted only with --dry-run.",
},
],
);
/** The given name; else the manifest's, unless its origin is anonymous, when the capture file names it. */
const environmentName = (
input: EnvironmentBuildInput,
manifest: { readonly name: string; readonly origin: string },
warnings: EnvironmentBuildWarning[],
): Result<string | undefined, AnalysisInputError> => {
if (input.name !== undefined) return ok(input.name);
if (!anonymousOrigin(manifest.origin)) return ok(undefined);
const name = nameFromCaptureFile(input.capturePath);
if (!isPortableBundleName(name)) return err(unnamed(input.capturePath));
warnings.push({
code: "name-from-file",
pointers: [],
message: `The capture's origin ${manifest.origin} names no application, so the environment is named ${name} after the capture file; name it explicitly to choose another.`,
});
return ok(name);
};
/** A path as one POSIX shell word. */
const shellWord = (path: string): string =>
/^[\w./@%+=:,-]+$/u.test(path) ? path : `'${path.replaceAll("'", "'\\''")}'`;
const packageSourceValue = (
source: EnvironmentPackageSource,
): EnvironmentBuildReport["package_source"] =>
source.kind === "registry"
? { kind: "registry", specifier: source.specifier }
: { kind: "tarball", path: source.path };
/**
* Build an agent environment bundle from capture Evidence: project the
* manifest from the in-memory capture, authenticate the pair, prepare the
* state the bundle's server derives and summarize what it serves, then write
* the bundle with the export unless this is a dry run. Every file equals what
* inspect-analysis-view and environment-export produce from the same capture.
*/
export const buildEnvironment = async (
input: EnvironmentBuildInput,
): Promise<Result<EnvironmentBuildReport, AnalysisError>> => {
if (input.directory === undefined && !input.dryRun)
return err(missingDirectory());
if (new Set(input.templateRules).size !== input.templateRules.length)
return err(repeatedRules());
if (input.captureInput === "evidence" && harDocument(input.capture.value))
return err(harAsJson(input.capturePath));
const evidence = parseEvidenceFile(
ENVIRONMENT_BUILD_OPERATION,
"evidence",
input.capture.value,
);
if (!evidence.ok) return evidence;
if (!captureOperations.has(evidence.value.operation))
return err(notCaptureEvidence(evidence.value.operation));
const capture = evidence.value;
const templateRules = canonicalTemplateRules(input.templateRules);
input.progress?.("manifest");
const projected = await inspectAnalysisView(
{
source: { kind: "retained-evidence", evidence_id: capture.evidence_id },
view: { kind: "environment-manifest", template_rules: templateRules },
},
(id) => (id === capture.evidence_id ? capture : undefined),
);
if (!projected.ok) return projected;
const stopped = await cancelled(input.signal);
if (stopped !== undefined) return err(stopped);
const manifest = await environmentExportEvidenceFile(
"manifest",
"manifest.json",
projected.value,
ENVIRONMENT_BUILD_OPERATION,
);
if (!manifest.ok) return manifest;
const source = loadEnvironmentSource(
manifest.value.value,
input.capture.value,
ENVIRONMENT_BUILD_OPERATION,
);
if (!source.ok) return source;
const warnings: EnvironmentBuildWarning[] = [];
const named = environmentName(input, source.value.manifest, warnings);
if (!named.ok) return named;
const name = named.value ?? source.value.manifest.name;
const beforeState = await cancelled(input.signal);
if (beforeState !== undefined) return err(beforeState);
input.progress?.("state");
// Prepared as the export prepares it, so its report and the bundle's README match the three-step path.
const served = prepareServedState(source.value, input.capture.value, {
name,
operation: ENVIRONMENT_EXPORT_OPERATION,
});
const recorded = parseCaptureEvidence(
ENVIRONMENT_BUILD_OPERATION,
input.capture.value,
);
if (!recorded.ok) return recorded;
warnings.push(...credentialScanWarnings(recorded.value));
const packageSource: EnvironmentPackageSource =
input.packageTarball === undefined
? registryPackageSource()
: { kind: "tarball", path: input.packageTarball };
const base = {
name,
capture: {
evidence_id: capture.evidence_id,
operation: capture.operation,
records: recorded.value.records.length,
input: input.captureInput,
},
manifest: {
evidence_id: source.value.manifestEvidenceId,
tools: source.value.manifest.tools.length,
excluded_groups: source.value.manifest.excluded.length,
template_rules: templateRules,
},
state: served.report,
coverage:
served.state === undefined
? null
: summarizeEnvironmentCoverage({
views: served.state.views.values(),
writes: served.state.writes.values(),
}),
warnings,
};
if (input.dryRun) {
const beforePlan = await cancelled(input.signal);
if (beforePlan !== undefined) return err(beforePlan);
const plan = planEnvironmentBundle(source.value, {
name,
packageSource,
served: () => served,
});
if (!plan.ok) return plan;
const files: WrittenBundleFile[] = [
...plan.value.files.map((file) => ({
path: file.path,
bytes: Buffer.byteLength(file.content),
})),
{ path: plan.value.copies.manifest, bytes: manifest.value.bytes.length },
{ path: plan.value.copies.evidence, bytes: input.capture.bytes.length },
];
if (
plan.value.copies.tarball !== null &&
input.packageTarball !== undefined
) {
const bytes = await packageTarballBytes(input.packageTarball);
if (!bytes.ok) return bytes;
files.push({ path: plan.value.copies.tarball, bytes: bytes.value });
}
const afterPlan = await cancelled(input.signal);
if (afterPlan !== undefined) return err(afterPlan);
return ok({
status: "planned",
directory: input.directory ?? null,
...base,
tool_count: plan.value.toolCount,
package_source: packageSourceValue(packageSource),
files,
next: [],
});
}
const directory = input.directory;
if (directory === undefined) return err(missingDirectory());
const beforeWrite = await cancelled(input.signal);
if (beforeWrite !== undefined) return err(beforeWrite);
input.progress?.("bundle");
const exported = await exportEnvironmentBundle({
manifest: manifest.value,
evidence: input.capture,
source: source.value,
served,
directory,
name,
packageTarball: input.packageTarball,
exclusiveDestination: input.exclusiveDestination,
});
if (!exported.ok) return exported;
const report: EnvironmentBuildReport = {
status: "built",
directory,
...base,
tool_count: exported.value.tool_count,
package_source: packageSourceValue(packageSource),
files: [...exported.value.files],
next: [`agent-env run ${shellWord(directory)}`],
};
// A cancellation that arrives during the write still names the bundle it wrote.
const afterWrite = await cancelled(input.signal, directory);
return afterWrite === undefined ? ok(report) : err(afterWrite);
};
src/application/environment/environmentBuildReport.test.ts
+373
View File
@@ -0,0 +1,373 @@
import { describe, expect, it } from "vitest";
import { scenarioCaptureFixture } from "../../../tests/fixtures/browserScenarioCapture.js";
import type { WebNetworkCaptureRecord } from "../../domain/webNetworkCapture.js";
import {
anonymousOrigin,
credentialNamed,
nameFromCaptureFile,
scanCredentialNames,
scanScenarioCredentialNames,
} from "./environmentBuildReport.js";
const record = (
ordinal: number,
request: unknown,
response: unknown,
): WebNetworkCaptureRecord =>
({
ordinal,
reported: {
request: {
method: "POST",
url: "http://127.0.0.1/api",
...(request === undefined
? {}
: {
postData: {
mimeType: "application/json",
text: JSON.stringify(request),
},
}),
},
response: {
status: 200,
content: {
mimeType: "application/json",
text: JSON.stringify(response),
},
},
},
redactions: [],
binary_fields: [],
}) as unknown as WebNetworkCaptureRecord;
/** Every body of these records was small enough to scan. */
const NONE_SKIPPED = { count: 0, pointers: [] };
describe("credentialNamed", () => {
it("matches whole name segments, not substrings", () => {
for (const name of [
"password",
"authToken",
"api_key",
"apiKey",
"sessionId",
"client_secret",
"X-Session",
"refresh.token",
"Authorization",
"APIToken",
"accesstoken",
"passphrase",
"api_keys",
"ssh_key",
])
expect(credentialNamed(name), name).toBe(true);
for (const name of ["author", "authority_level", "keyboard", "tokenizer"])
expect(credentialNamed(name), name).toBe(false);
});
});
describe("scanCredentialNames", () => {
it("counts string values under credential names in request and response bodies, by pointer and never by value", () => {
const scanned = scanCredentialNames([
record(
3,
{ name: "Folder", secret: "s3cr3t-value" },
{ data: [{ id: 1, sessionId: "abc123def" }], hasPassword: true },
),
record(4, undefined, { token: "", "a/b": { apiKey: "k-1" } }),
]);
expect(scanned).toEqual({
count: 3,
pointers: [
"record 3 request /secret",
"record 3 response /data/0/sessionId",
"record 4 response /a~1b/apiKey",
],
skipped: NONE_SKIPPED,
});
expect(JSON.stringify(scanned)).not.toContain("s3cr3t");
});
it("scans header and query names, form bodies and string arrays", () => {
const withHeaders = {
ordinal: 7,
reported: {
request: {
method: "POST",
url: "http://127.0.0.1/api?api_key=k&page=1",
headers: [
{ name: "X-Api-Key", value: "k-123" },
{ name: "Accept", value: "application/json" },
{ name: "Authorization", value: null },
],
queryString: [
{ name: "api_key", value: "k" },
{ name: "page", value: "1" },
],
postData: {
mimeType: "application/x-www-form-urlencoded",
text: "user=a&password=hunter22",
},
},
response: {
status: 200,
headers: [{ name: "X-Auth-Token", value: "t-9" }],
content: {
mimeType: "application/json",
text: JSON.stringify({ tokens: ["a", "b"], token_count: 2 }),
},
},
},
redactions: [],
binary_fields: [],
} as unknown as WebNetworkCaptureRecord;
expect(scanCredentialNames([withHeaders])).toEqual({
count: 5,
pointers: [
"record 7 request header X-Api-Key",
"record 7 response header X-Auth-Token",
"record 7 request query api_key",
"record 7 request /password",
"record 7 response /tokens",
],
skipped: NONE_SKIPPED,
});
});
it("counts every occurrence of a repeated form field that holds a value", () => {
const form = (text: string): WebNetworkCaptureRecord => {
const scanned = record(3, undefined, undefined);
(
scanned.reported as { request: { postData?: unknown } }
).request.postData = {
mimeType: "application/x-www-form-urlencoded",
text,
};
return scanned;
};
// A later empty occurrence does not hide the earlier value the Evidence ships.
expect(scanCredentialNames([form("api_key=secret&api_key=")])).toEqual({
count: 1,
pointers: ["record 3 request /api_key"],
skipped: NONE_SKIPPED,
});
expect(scanCredentialNames([form("token=a&page=1&token=b")])).toEqual({
count: 2,
pointers: ["record 3 request /token", "record 3 request /token"],
skipped: NONE_SKIPPED,
});
});
it("walks bodies nested deeper than the call stack allows", () => {
const depth = 20_000;
const deep = record(1, undefined, undefined);
(
deep.reported as { response: { content: { text: string } } }
).response.content.text =
`${"[".repeat(depth)}{"token":"x"}${"]".repeat(depth)}`;
expect(scanCredentialNames([deep]).count).toBe(1);
});
});
describe("scanCredentialNames body formats", () => {
const posted = (mimeType: string, text: string): WebNetworkCaptureRecord => {
const scanned = record(3, undefined, undefined);
(scanned.reported as { request: { postData?: unknown } }).request.postData =
{ mimeType, text };
return scanned;
};
const FORM = "application/x-www-form-urlencoded; charset=UTF-8";
it("reads a form body as the form parser does, by its media type or, without one, by its shape", () => {
for (const mimeType of [FORM, ""]) {
expect(
scanCredentialNames([posted(mimeType, "password=example&")]),
mimeType,
).toEqual({
count: 1,
pointers: ["record 3 request /password"],
skipped: NONE_SKIPPED,
});
expect(
scanCredentialNames([posted(mimeType, "user=a&&flag&token=example")]),
mimeType,
).toEqual({
count: 1,
pointers: ["record 3 request /token"],
skipped: NONE_SKIPPED,
});
}
});
it("finds nothing in plain text that is not a form", () => {
for (const mimeType of ["text/plain", ""])
for (const text of ["token=abc to sign in", "Use token=abc", ""])
expect(
scanCredentialNames([posted(mimeType, text)]),
`${mimeType} ${text}`,
).toEqual({ count: 0, pointers: [], skipped: NONE_SKIPPED });
});
});
describe("scanCredentialNames over structured form parameters", () => {
it.each([undefined, null])(
"scans each structured form parameter with absent text: %s",
(text) => {
const structured = record(4, undefined, undefined);
(
structured.reported as { request: { postData?: unknown } }
).request.postData = {
mimeType: "multipart/form-data",
text,
params: [
{ name: "api_key", value: "secret" },
{ name: "api_key", value: "" },
{ name: "user", value: "a" },
{ name: "token", value: "t" },
],
};
expect(scanCredentialNames([structured])).toEqual({
count: 2,
pointers: ["record 4 request /api_key", "record 4 request /token"],
skipped: NONE_SKIPPED,
});
},
);
});
describe("scanScenarioCredentialNames", () => {
it("scans each content event, and each transaction URL once, by event index", () => {
const capture = scenarioCaptureFixture(
[
{
method: "POST",
url: "/api/session?api_key=k-1",
status: 200,
mediaType: "application/json",
body: "{}",
// Recorded without a media type, so its shape decides how it is read.
requestBody: '{"password":"x"}',
requestHeaders: [{ name: "X-Api-Key", value: "k-2" }],
},
],
{ request_body: true, response_body: true, header_values: true },
);
const response = capture.events.items.find(
(event) => event.kind === "response",
);
if (response === undefined) throw new Error("Fixture lacks a response");
// A second response leaves the transaction unattributable; its events still ship.
const scanned = scanScenarioCredentialNames([
...capture.events.items,
{ ...response, sequence: 99 },
]);
expect(scanned).toEqual({
count: 3,
pointers: [
"event 0 request query api_key",
"event 1 request header X-Api-Key",
"event 1 request /password",
],
skipped: NONE_SKIPPED,
});
expect(JSON.stringify(scanned)).not.toContain("k-2");
});
});
describe("scanCredentialNames coverage", () => {
it("measures the scan limit in UTF-8 bytes for multibyte text", () => {
const multibyte = record(
9,
{ token: "界".repeat(2 * 1024 * 1024) },
undefined,
);
expect(scanCredentialNames([multibyte])).toMatchObject({
count: 0,
skipped: { count: 1, pointers: ["record 9 request"] },
});
});
it("reports text bodies too large to scan by record and side, never by content", () => {
const large = JSON.stringify({
token: "t-large",
pad: "a".repeat(4 * 1024 * 1024),
});
const oversized = record(5, undefined, undefined);
const reported = oversized.reported as {
request: { postData?: unknown };
response: { content: { text: string } };
};
reported.request.postData = { mimeType: "application/json", text: large };
reported.response.content.text = large;
const base64 = (ordinal: number, bytes: Buffer): WebNetworkCaptureRecord =>
({
...record(ordinal, undefined, undefined),
reported: {
request: { method: "GET", url: "http://127.0.0.1/api" },
response: {
status: 200,
content: {
mimeType: "application/json",
text: "",
encoding: "base64",
},
},
},
binary_fields: [
{
pointer: "/response/content/text",
state: "retained",
content_base64: bytes.toString("base64"),
},
],
}) as unknown as WebNetworkCaptureRecord;
const scanned = scanCredentialNames([
oversized,
// Decoded only as a prefix, so its end is never read.
base64(6, Buffer.from(large)),
// Binary bytes hold no JSON or form text, so nothing was left unscanned.
base64(7, Buffer.alloc(5 * 1024 * 1024, 0xff)),
record(8, { token: "t-small" }, {}),
]);
expect(scanned).toEqual({
count: 1,
pointers: ["record 8 request /token"],
skipped: {
count: 3,
pointers: [
"record 5 request",
"record 5 response",
"record 6 response",
],
},
});
expect(JSON.stringify(scanned)).not.toContain("t-large");
});
});
describe("anonymousOrigin", () => {
it("is true for loopback names and IP literals", () => {
for (const origin of [
"http://127.0.0.1:3101",
"http://localhost:3000",
"http://app.localhost",
"http://[::1]:8080",
"http://10.0.0.5",
])
expect(anonymousOrigin(origin), origin).toBe(true);
expect(anonymousOrigin("http://localhost.:3000")).toBe(true);
for (const origin of ["https://grafana.example", "not a url"])
expect(anonymousOrigin(origin), origin).toBe(false);
});
});
describe("nameFromCaptureFile", () => {
it("drops capture suffixes and keeps a kebab-case label", () => {
expect(nameFromCaptureFile("/w/grafana.har")).toBe("grafana");
expect(nameFromCaptureFile("/w/grafana.capture.json")).toBe("grafana");
expect(nameFromCaptureFile("/w/My App.scoped.HAR")).toBe("my-app-scoped");
expect(nameFromCaptureFile("/w/--.json")).toBe("");
});
});
src/application/environment/environmentBuildReport.ts
+358
View File
@@ -0,0 +1,358 @@
import { basename } from "node:path";
import { isIP } from "node:net";
import { isJsonLike } from "../../domain/analysisView/EnvironmentManifestAccumulator.js";
import {
decodeRetainedBase64,
readRecordedResponseBody,
type RecordedResponseBody,
} from "../../domain/analysisView/recordedResponseBody.js";
import { scenarioContentMediaType } from "../../domain/analysisView/scenarioCaptureRecords.js";
import { mediaTypeEssence } from "../../domain/analysisView/webNetworkRouteTemplates.js";
import type { BrowserScenarioEvent } from "../../domain/browserScenarioCaptureValues.js";
import type { EnvironmentBuildReport } from "../../domain/environmentBundleExport.js";
import {
iterateUrlQueryParameters,
parseRecordedTransportUrl,
} from "../../domain/recordedTransportUrl.js";
import type { WebNetworkCaptureRecord } from "../../domain/webNetworkCapture.js";
/** A note the build report carries; warnings never change what is built. */
export type EnvironmentBuildWarning =
EnvironmentBuildReport["warnings"][number];
/** Whole name segments that name credentials; `author` is not `auth`. */
const CREDENTIAL_SEGMENTS = new Set([
"accesstoken",
"apikey",
"apikeys",
"apitoken",
"auth",
"authorization",
"bearer",
"cookie",
"credential",
"credentials",
"jwt",
"passphrase",
"passwd",
"password",
"privatekey",
"pwd",
"secret",
"secrets",
"session",
"sessionid",
"token",
"tokens",
]);
const CREDENTIAL_PAIRS = new Set([
"access key",
"api key",
"api keys",
"client secret",
"private key",
"session id",
"ssh key",
]);
/** Split a member name into lowercase segments at case changes and separators. */
const nameSegments = (name: string): string[] =>
name
.replaceAll(/([a-z0-9])([A-Z])/gu, "$1 $2")
.replaceAll(/([A-Z]+)([A-Z][a-z])/gu, "$1 $2")
.toLowerCase()
.split(/[^a-z0-9]+/u)
.filter((segment) => segment.length > 0);
/** Whether a member, header or query name looks like it holds a credential. */
export const credentialNamed = (name: string): boolean => {
const segments = nameSegments(name);
return segments.some(
(segment, index) =>
CREDENTIAL_SEGMENTS.has(segment) ||
(index > 0 && CREDENTIAL_PAIRS.has(`${segments[index - 1]} ${segment}`)),
);
};
/** Bodies larger than this are not scanned. */
const SCAN_BODY_LIMIT_BYTES = 4 * 1024 * 1024;
/** Pointers the report lists; the count covers every match. */
const REPORTED_POINTERS = 20;
const pointerToken = (name: string): string =>
name.replaceAll("~", "~0").replaceAll("/", "~1");
/** A held value: a non-empty string, or an array holding one. */
const holdsValue = (value: unknown): boolean =>
(typeof value === "string" && value !== "") ||
(Array.isArray(value) &&
value.some((item) => typeof item === "string" && item !== ""));
const member = (value: unknown, name: string): unknown =>
typeof value === "object" && value !== null
? Reflect.get(value, name)
: undefined;
/** A recorded body: a JSON value to walk, form fields in order with every occurrence kept, as headers and query strings are, or text too large to scan. */
type ScannedBody =
| { readonly kind: "json"; readonly value: unknown }
| { readonly kind: "form"; readonly fields: readonly [string, string][] }
| { readonly kind: "skipped" };
const FORM_MEDIA_TYPE = "application/x-www-form-urlencoded";
/** The parser a recorded media type names; otherwise JSON by its first character, or a form when it has `=` and no whitespace, which form encoding never leaves raw. */
const bodyFormat = (
text: string,
mediaType: unknown,
): "json" | "form" | undefined => {
const essence =
typeof mediaType === "string" ? mediaTypeEssence(mediaType) : null;
if (essence === FORM_MEDIA_TYPE) return "form";
if (isJsonLike(essence)) return "json";
const start = text.trimStart()[0];
if (start === "{" || start === "[") return "json";
return text.includes("=") && !/\s/u.test(text) ? "form" : undefined;
};
/** A recorded body as JSON or form fields, skipped when it is cut or over the limit, or undefined when it is neither. */
const parsedBody = (
text: unknown,
mediaType: unknown,
cut = false,
): ScannedBody | undefined => {
if (typeof text !== "string") return undefined;
if (cut || Buffer.byteLength(text, "utf8") > SCAN_BODY_LIMIT_BYTES)
return { kind: "skipped" };
const format = bodyFormat(text, mediaType);
if (format === "form")
return { kind: "form", fields: [...new URLSearchParams(text)] };
if (format === undefined) return undefined;
try {
return { kind: "json", value: JSON.parse(text) };
} catch {
return undefined;
}
};
/** Decoded body text as the scan reads it; binary bytes hold no JSON or form text. */
const decodedBody = (
body: RecordedResponseBody,
mediaType: unknown,
): ScannedBody | undefined =>
body.bodyEncoding === "base64"
? undefined
: parsedBody(body.bodyText, mediaType, body.bodyCut === true);
/** Name/value pairs such as HAR headers and query strings. */
const namedValues = (value: unknown): (readonly [string, unknown])[] =>
Array.isArray(value)
? value.flatMap((entry: unknown) => {
const name = member(entry, "name");
return typeof name === "string"
? [[name, member(entry, "value")] as const]
: [];
})
: [];
/** A count of matches and the first of their pointers. */
interface ScanTally {
count: number;
readonly pointers: string[];
}
type CredentialScan = Readonly<ScanTally> & {
readonly skipped: Readonly<ScanTally>;
};
const tallied = (tally: ScanTally, pointer: string): void => {
tally.count++;
if (tally.pointers.length < REPORTED_POINTERS) tally.pointers.push(pointer);
};
/**
* The tallies both capture readers fill: a named value counts when its name
* looks like a credential and it holds a value. The body walk keeps its own
* stack, so nesting depth cannot overflow it.
*/
const credentialTally = () => {
const matches: ScanTally = { count: 0, pointers: [] };
const skipped: ScanTally = { count: 0, pointers: [] };
const field = (pointer: string, name: string, value: unknown): void => {
if (credentialNamed(name) && holdsValue(value)) tallied(matches, pointer);
};
const walk = (root: unknown, label: string): void => {
const stack: [unknown, string][] = [[root, ""]];
while (stack.length > 0) {
const [value, pointer] = stack.pop()!;
if (typeof value !== "object" || value === null) continue;
if (Array.isArray(value)) {
for (let index = value.length - 1; index >= 0; index--)
stack.push([value[index], `${pointer}/${String(index)}`]);
continue;
}
const entries = Object.entries(value);
for (let index = entries.length - 1; index >= 0; index--) {
const [name, child] = entries[index]!;
const path = `${pointer}/${pointerToken(name)}`;
field(`${label} ${path}`, name, child);
stack.push([child, path]);
}
}
};
const body = (scanned: ScannedBody | undefined, label: string): void => {
if (scanned?.kind === "skipped") tallied(skipped, label);
else if (scanned?.kind === "json") walk(scanned.value, label);
else if (scanned?.kind === "form")
for (const [name, value] of scanned.fields)
field(`${label} /${pointerToken(name)}`, name, value);
};
return {
field,
body,
result: (): CredentialScan => ({ ...matches, skipped }),
};
};
/**
* Credential-named members that hold a value in recorded JSON and form
* bodies, headers and query strings: a count and the first pointers, never
* values, and the bodies too large to scan by record and side.
*/
export const scanCredentialNames = (
records: readonly WebNetworkCaptureRecord[],
): CredentialScan => {
const scan = credentialTally();
for (const record of records) {
const ordinal = String(record.ordinal);
const request = member(record.reported, "request");
const response = member(record.reported, "response");
for (const [side, message] of [
["request", request],
["response", response],
] as const)
for (const [name, value] of namedValues(member(message, "headers")))
scan.field(`record ${ordinal} ${side} header ${name}`, name, value);
for (const [name, value] of namedValues(member(request, "queryString")))
scan.field(`record ${ordinal} request query ${name}`, name, value);
const postData = member(request, "postData");
const text = member(postData, "text");
// A HAR may record form fields only as structured params; each occurrence counts, as in a form body.
scan.body(
typeof text !== "string"
? {
kind: "form",
fields: namedValues(member(postData, "params")).flatMap(
([name, value]) =>
typeof value === "string"
? [[name, value] as [string, string]]
: [],
),
}
: parsedBody(text, member(postData, "mimeType")),
`record ${ordinal} request`,
);
scan.body(
decodedBody(
readRecordedResponseBody(record, SCAN_BODY_LIMIT_BYTES),
member(member(response, "content"), "mimeType"),
),
`record ${ordinal} response`,
);
}
return scan.result();
};
/**
* The same scan over a browser scenario's retained events, by index in
* `events.items`: every network-content event's headers and body, whether or
* not a record represents its transaction, and each transaction URL's query
* once.
*/
export const scanScenarioCredentialNames = (
events: readonly BrowserScenarioEvent[],
): CredentialScan => {
const scan = credentialTally();
const urls = new Set<string>();
for (const [index, event] of events.entries()) {
const label = `event ${String(index)}`;
if (event.kind === "network-content") {
const side = `${label} ${event.phase}`;
if (event.headers.state === "captured")
for (const { name, value } of event.headers.items)
scan.field(`${side} header ${name}`, name, value);
if (event.body.state === "captured")
scan.body(
decodedBody(
decodeRetainedBase64(event.body.content, SCAN_BODY_LIMIT_BYTES),
scenarioContentMediaType(event),
),
side,
);
} else if ("transaction_id" in event) {
// Every event of a transaction repeats its URL; a response attached mid-flight may be the only one.
const key = `${event.transaction_id} ${event.url.url}`;
if (urls.has(key)) continue;
urls.add(key);
const { query } = parseRecordedTransportUrl(event.url.url, {
includeQuery: true,
});
for (const [name, value] of iterateUrlQueryParameters(query ?? ""))
scan.field(`${label} ${event.kind} query ${name}`, name, value);
}
}
return scan.result();
};
/** The credential-name scan's warnings: what it found, and the bodies it could not scan. */
export const credentialScanWarnings = (capture: {
readonly records: readonly WebNetworkCaptureRecord[];
readonly scenarioEvents: readonly BrowserScenarioEvent[] | null;
}): EnvironmentBuildWarning[] => {
const scanned =
capture.scenarioEvents === null
? scanCredentialNames(capture.records)
: scanScenarioCredentialNames(capture.scenarioEvents);
const warnings: EnvironmentBuildWarning[] = [];
if (scanned.count > 0)
warnings.push({
code: "credential-named-members",
message: `${String(scanned.count)} recorded body members have credential-like names and a value; evidence.json ships them, so review it before sharing.`,
pointers: scanned.pointers,
});
if (scanned.skipped.count > 0)
warnings.push({
code: "credential-scan-incomplete",
message: `${String(scanned.skipped.count)} recorded bodies exceed the credential-name scan's 4 MiB limit and were not scanned; evidence.json ships them, so review them before sharing.`,
pointers: scanned.skipped.pointers,
});
return warnings;
};
/** Whether an origin's host is a loopback name or an IP literal, which says nothing about the application. */
export const anonymousOrigin = (origin: string): boolean => {
let host: string;
try {
host = new URL(origin).hostname.toLowerCase();
} catch {
return false;
}
const bracketless = host.startsWith("[") ? host.slice(1, -1) : host;
const bare = bracketless.endsWith(".")
? bracketless.slice(0, -1)
: bracketless;
return (
bare === "localhost" || bare.endsWith(".localhost") || isIP(bare) !== 0
);
};
/** A kebab-case name from a capture file's name: `grafana.har` and `grafana.capture.json` become `grafana`. */
export const nameFromCaptureFile = (path: string): string =>
basename(path)
.replace(/\.(?:har|json)$/iu, "")
.replace(/\.capture$/iu, "")
.toLowerCase()
.replaceAll(/[^a-z0-9]+/gu, "-")
.replace(/^-+/u, "")
.replace(/-+$/u, "");
src/application/environment/environmentBundle.ts
+7 -1
View File
@@ -30,6 +30,12 @@ const BUNDLE_NAME_MAX_LENGTH = 64;
/** Win32 reserves these device names for files and folders, so the env folder can never take one. */
const WINDOWS_RESERVED_NAME = /^(?:con|prn|aux|nul|com[1-9]|lpt[1-9])$/u;
/** Whether a name can be the env folder and env id: the checks the export applies. */
export const isPortableBundleName = (name: string): boolean =>
BUNDLE_NAME.test(name) &&
name.length <= BUNDLE_NAME_MAX_LENGTH &&
!WINDOWS_RESERVED_NAME.test(name);
/** Where the Dockerfile installs the CLI from. */
export type EnvironmentPackageSource =
| { readonly kind: "registry"; readonly specifier: string }
@@ -103,7 +109,7 @@ const reservedNameError = (name: string): AnalysisInputError =>
new AnalysisInputError(
ENVIRONMENT_EXPORT_OPERATION,
{
userMessage: `The bundle name ${JSON.stringify(name)} is a device name Windows reserves, so it cannot name the env folder; pass --name with another label.`,
userMessage: `The bundle name ${JSON.stringify(name)} is a device name Windows reserves, so it cannot name the env folder; choose another name.`,
},
[
{
src/application/environment/environmentBundleTool.ts
+82
View File
@@ -0,0 +1,82 @@
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { SafeOutputTree } from "../../artifacts/SafeOutputTree.js";
import { outputDirectoryAdmissionRefusal } from "../../artifacts/SafeOutputTreeAdmission.js";
import type {
EnvironmentBuildReport,
ExportEnvironmentBundleInput,
} from "../../domain/environmentBundleExport.js";
import { err, type Result } from "../../domain/result.js";
import {
resolveEvidenceInput,
type EvidenceLookup,
} from "../EvidenceInputResolver.js";
import {
DEFAULT_BUILD_TEMPLATE_RULES,
buildEnvironment,
} from "./environmentBuild.js";
import {
environmentExportEvidenceFile,
readEnvironmentExportFile,
type EnvironmentExportFile,
} from "./environmentExport.js";
/** MCP tool and operation name of the bundle export. */
export const EXPORT_ENVIRONMENT_BUNDLE_OPERATION = "export_environment_bundle";
/** The capture's bytes as the bundle ships them: a file's own bytes, or retained Evidence as `--format json` writes it. */
const captureFile = async (
source: ExportEnvironmentBundleInput["source"],
lookup: EvidenceLookup | undefined,
): Promise<Result<EnvironmentExportFile, AnalysisError>> => {
if (source.kind === "evidence-file")
return readEnvironmentExportFile(
"evidence",
source.path,
EXPORT_ENVIRONMENT_BUNDLE_OPERATION,
);
const retained = resolveEvidenceInput(source, lookup);
if (!retained.ok) return retained;
return environmentExportEvidenceFile(
"evidence",
source.evidence_id,
retained.value,
EXPORT_ENVIRONMENT_BUNDLE_OPERATION,
);
};
/**
* Build an environment bundle from capture Evidence into an absent output
* directory whose parent exists, as environment-build does, and return the
* same report. A dry run checks the directory and writes nothing.
*/
export const exportEnvironmentBundleTool = async (
input: ExportEnvironmentBundleInput,
lookup: EvidenceLookup | undefined,
options: { readonly signal?: AbortSignal | undefined } = {},
): Promise<Result<EnvironmentBuildReport, AnalysisError>> => {
try {
await SafeOutputTree.checkDestination(input.output_directory);
} catch (cause: unknown) {
const refusal = outputDirectoryAdmissionRefusal(
EXPORT_ENVIRONMENT_BUNDLE_OPERATION,
"output_directory",
cause,
);
if (refusal instanceof AnalysisError) return err(refusal);
throw refusal;
}
const capture = await captureFile(input.source, lookup);
if (!capture.ok) return capture;
return buildEnvironment({
capture: capture.value,
captureInput: "evidence",
capturePath: input.source.kind === "evidence-file" ? input.source.path : "",
templateRules: input.template_rules ?? DEFAULT_BUILD_TEMPLATE_RULES,
directory: input.output_directory,
name: input.name,
packageTarball: input.package_tarball,
dryRun: input.dry_run,
signal: options.signal,
exclusiveDestination: true,
});
};
src/application/environment/environmentBundleWrite.ts
+57 -37
View File
@@ -94,19 +94,17 @@ const writeBytes = async (
return { path: written.relativePath, bytes: written.bytesWritten };
};
interface BundleTarball {
/** The package tarball instance admitted once and retained until its owner closes it. */
export interface BundleTarball {
readonly path: string;
readonly handle: FileHandle;
readonly size: number;
}
/** Hash and copy only the instance retained before destination admission. */
const writeTarball = async (
tree: SafeOutputTree,
directory: string,
relative: string,
/** Read and hash the admitted instance, checking its complete length without retaining its contents. */
export const bundleTarballDigest = async (
source: BundleTarball,
): Promise<WrittenBundleFile> => {
): Promise<string> => {
const hash = createHash("sha256");
let observed = 0;
for await (const chunk of tarballChunks(source)) {
@@ -116,7 +114,17 @@ const writeTarball = async (
hash.update(bytes);
}
if (observed !== source.size) throw new RegularFileChangedError(source.path);
const sha256 = hash.digest("hex");
return hash.digest("hex");
};
/** Hash and copy only the instance retained before destination admission. */
const writeTarball = async (
tree: SafeOutputTree,
directory: string,
relative: string,
source: BundleTarball,
): Promise<WrittenBundleFile> => {
const sha256 = await bundleTarballDigest(source);
const written = await writingBundleFile(directory, relative, () =>
tree.write(relative, Readable.from(tarballChunks(source)), {
bytes: source.size,
@@ -142,10 +150,11 @@ const writeBundleFiles = async (
}
| { readonly kind: "failed"; readonly cause: unknown };
try {
await mkdir(dirname(input.directory), { recursive: true, mode: 0o700 });
if (input.exclusiveDestination !== true)
await mkdir(dirname(input.directory), { recursive: true, mode: 0o700 });
try {
tree = await SafeOutputTree.create(input.directory, process.platform, {
acceptExistingEmpty: true,
acceptExistingEmpty: input.exclusiveDestination !== true,
});
} catch (cause: unknown) {
if (cause instanceof SafeOutputTreeCreationFailure) tree = cause.tree;
@@ -208,14 +217,31 @@ const writeBundleFiles = async (
return outcome.files;
};
/** Retain one tarball instance across planning output writes and final stability verification. */
export const writeEnvironmentBundle = async (
input: EnvironmentExportInput,
plan: EnvironmentBundlePlan,
): Promise<readonly WrittenBundleFile[]> => {
if (plan.packageSource.kind !== "tarball")
return writeBundleFiles(input, plan);
const path = plan.packageSource.path;
/** A package tarball path that names no readable regular file. */
export const packageTarballRefusal = (
path: string,
cause: unknown,
): AnalysisInputError =>
new AnalysisInputError(
ENVIRONMENT_EXPORT_OPERATION,
{
cause,
userMessage: `${path} must name a readable regular package tarball.`,
},
[
{
path: ["package_tarball"],
reason: "invalid_value",
message: "The package tarball must be an existing regular file.",
},
],
);
/** Admit the package tarball once and retain it through `use`; a close it cannot confirm stays bound to its owner. */
export const withBundleTarball = async <Value>(
path: string,
use: (tarball: BundleTarball) => Promise<Value>,
): Promise<Value> => {
const resources = new ArtifactResourceScope();
let admitted = false;
try {
@@ -223,11 +249,7 @@ export const writeEnvironmentBundle = async (
path,
async (handle, state) => {
admitted = true;
return writeBundleFiles(input, plan, {
path,
handle,
size: state.size,
});
return use({ path, handle, size: state.size });
},
{ resources },
);
@@ -250,21 +272,19 @@ export const writeEnvironmentBundle = async (
String(cause.code),
)
)
throw new AnalysisInputError(
ENVIRONMENT_EXPORT_OPERATION,
{
cause,
userMessage: `${path} must name a readable regular package tarball.`,
},
[
{
path: ["package_tarball"],
reason: "invalid_value",
message: "The package tarball must be an existing regular file.",
},
],
);
throw packageTarballRefusal(path, cause);
}
throw cause;
}
};
/** Retain one tarball instance across planning output writes and final stability verification. */
export const writeEnvironmentBundle = (
input: EnvironmentExportInput,
plan: EnvironmentBundlePlan,
): Promise<readonly WrittenBundleFile[]> =>
plan.packageSource.kind === "tarball"
? withBundleTarball(plan.packageSource.path, (tarball) =>
writeBundleFiles(input, plan, tarball),
)
: writeBundleFiles(input, plan);
src/application/environment/environmentEvidenceBytes.ts
+19
View File
@@ -0,0 +1,19 @@
import type { Evidence } from "../../domain/evidence.js";
import { jsonByteLength, jsonParts } from "../../domain/jsonSerialization.js";
/** An Evidence file's bytes as `--format json` writes the Evidence to a file: compact JSON and a newline. */
export const evidenceFileBytes = (
evidence: Evidence,
admit?: (bytes: number) => void,
): Buffer => {
// Measured first, so the document is written into one exactly sized buffer.
const size = jsonByteLength(evidence) + 1;
admit?.(size);
const bytes = Buffer.allocUnsafe(size);
let offset = 0;
for (const part of jsonParts(evidence)) offset += bytes.write(part, offset);
if (offset !== bytes.length - 1)
throw new Error("Evidence serialized to a different length than measured");
bytes[offset] = 0x0a;
return bytes;
};
src/application/environment/environmentExport.ts
+146 -73
View File
@@ -12,6 +12,8 @@ import {
type AnalysisInputIssue,
} from "../../domain/analysisErrorCore.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import type { Evidence } from "../../domain/evidence.js";
import { evidenceFileBytes } from "./environmentEvidenceBytes.js";
import { err, ok, type Result } from "../../domain/result.js";
import { RegularFileChangedError } from "../../filesystem/RegularFile.js";
import {
@@ -24,6 +26,9 @@ import { RegularFileCleanupFailure } from "../../filesystem/OwnedRegularFile.js"
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import {
BundleFileHostFailure,
bundleTarballDigest,
packageTarballRefusal,
withBundleTarball,
writeEnvironmentBundle,
} from "./environmentBundleWrite.js";
import { sealOwnedJsonInput } from "../OwnedJsonInput.js";
@@ -58,6 +63,10 @@ export interface EnvironmentExportInputs {
export interface EnvironmentExportInput extends EnvironmentExportInputs {
/** The authenticated pair the bundle serves, loaded from the parsed values of the two files. */
readonly source: EnvironmentSource;
/** The state already prepared under `name` from the evidence value; prepared here when absent. */
readonly served?: PreparedState | undefined;
/** Refuse an existing directory and never create its parent; otherwise an empty directory is accepted. */
readonly exclusiveDestination?: boolean | undefined;
/** Absolute bundle root; absent or empty. */
readonly directory: string;
readonly name?: string | undefined;
@@ -86,12 +95,11 @@ export interface EnvironmentExportResult {
}
const inputError = (
operation: string,
issue: AnalysisInputIssue,
userMessage: string,
): AnalysisInputError =>
new AnalysisInputError(ENVIRONMENT_EXPORT_OPERATION, { userMessage }, [
issue,
]);
new AnalysisInputError(operation, { userMessage }, [issue]);
const systemCode = (cause: unknown): string | undefined =>
cause instanceof Error && "code" in cause && typeof cause.code === "string"
@@ -129,6 +137,7 @@ const accessDeniedError = (
path: string,
diagnostics?: Readonly<Record<string, JsonValue>>,
access: "read" | "write" = "read",
operation: string = ENVIRONMENT_EXPORT_OPERATION,
): AnalysisAccessDeniedError | AnalysisOutputError | undefined => {
const seen = new Set<Error>();
let current = cause;
@@ -148,14 +157,9 @@ const accessDeniedError = (
};
const error =
access === "read"
? new AnalysisAccessDeniedError(
ENVIRONMENT_EXPORT_OPERATION,
path,
code,
options,
)
? new AnalysisAccessDeniedError(operation, path, code, options)
: new AnalysisOutputError(
ENVIRONMENT_EXPORT_OPERATION,
operation,
`Host filesystem permissions denied write access: ${path}`,
{
...options,
@@ -204,6 +208,7 @@ const NOT_A_FILE_CODES = new Set([
/** Refuse a whole read, native parse and owned snapshot traversal that cannot fit live heap headroom. */
const headroomRefusal = (
operation: string,
path: string,
inputBytes: number,
): AnalysisResourceConstraintError | undefined => {
@@ -212,7 +217,7 @@ const headroomRefusal = (
return requiredBytes <= heap.total_available_size
? undefined
: new AnalysisResourceConstraintError(
ENVIRONMENT_EXPORT_OPERATION,
operation,
"memory",
"Insufficient heap headroom to read, parse and seal the complete JSON file",
{
@@ -226,10 +231,64 @@ const headroomRefusal = (
);
};
/** Read one input once: the same bytes are parsed for authentication and copied into the bundle. */
const readInput = async (
/** Measure and admit every serialized copy before allocating its buffer. */
export const environmentExportEvidenceFile = async (
role: "manifest" | "evidence",
path: string,
evidence: Evidence,
operation: string = ENVIRONMENT_EXPORT_OPERATION,
): Promise<Result<EnvironmentExportFile, AnalysisError>> => {
try {
const bytes = evidenceFileBytes(evidence, (size) => {
const refused = headroomRefusal(operation, path, size);
if (refused !== undefined) throw refused;
});
return environmentExportFile(role, path, bytes, operation);
} catch (cause: unknown) {
if (cause instanceof AnalysisResourceConstraintError) return err(cause);
throw cause;
}
};
/** Parse and seal bytes the bundle will copy, so authentication reads exactly what ships. */
export const environmentExportFile = async (
role: "manifest" | "evidence",
path: string,
bytes: Buffer,
operation: string = ENVIRONMENT_EXPORT_OPERATION,
): Promise<Result<EnvironmentExportFile, AnalysisError>> => {
const refusal = headroomRefusal(operation, path, bytes.length);
if (refusal !== undefined) return err(refusal);
let parsed: ReturnType<typeof parseUtf8Json>;
try {
parsed = parseUtf8Json(bytes, operation, path);
} catch (cause: unknown) {
if (cause instanceof AnalysisResourceConstraintError) return err(cause);
throw cause;
}
if (!parsed.ok)
return err(
inputError(
operation,
{
path: [role],
reason: "invalid_format",
expected: "JSON",
message: parsed.error,
},
`${path} is not valid JSON: ${parsed.error}`,
),
);
const snapshot = await sealOwnedJsonInput(parsed.value, operation);
if (!snapshot.ok) return snapshot;
return ok({ path, bytes, value: snapshot.value });
};
/** Read one input once: the same bytes are parsed for authentication and copied into the bundle. */
export const readEnvironmentExportFile = async (
role: "manifest" | "evidence",
path: string,
operation: string = ENVIRONMENT_EXPORT_OPERATION,
): Promise<Result<EnvironmentExportFile, AnalysisError>> => {
const resources = new ArtifactResourceScope();
let bytes: Buffer;
@@ -239,7 +298,7 @@ const readInput = async (
bytes = await readRegularFile(path, {
resources,
admit: (state) => {
const refused = headroomRefusal(path, state.size);
const refused = headroomRefusal(operation, path, state.size);
if (refused !== undefined) throw refused;
},
});
@@ -250,7 +309,7 @@ const readInput = async (
AnalysisCleanupLease.forCloseOwner(cause.owner, path),
);
const cleanup = new AnalysisOutputError(
ENVIRONMENT_EXPORT_OPERATION,
operation,
`Could not confirm closing the ${role} input file.`,
{
cause: failure,
@@ -266,7 +325,7 @@ const readInput = async (
? analysisErrorWithCleanupFailure(
primary,
cleanup,
ENVIRONMENT_EXPORT_OPERATION,
operation,
).bindCleanupDependencies([primary, cleanup])
: cleanup,
);
@@ -275,6 +334,7 @@ const readInput = async (
if (cause instanceof RegularFileChangedError)
return err(
inputError(
operation,
{
path: [role],
reason: "invalid_value",
@@ -287,6 +347,7 @@ const readInput = async (
if (NOT_A_FILE_CODES.has(systemCode(cause) ?? ""))
return err(
inputError(
operation,
{
path: [role],
reason: "invalid_value",
@@ -295,35 +356,11 @@ const readInput = async (
`${path} could not be read as the ${role} file.`,
),
);
const denied = accessDeniedError(cause, path);
const denied = accessDeniedError(cause, path, undefined, "read", operation);
if (denied !== undefined) return err(denied);
throw cause;
}
let parsed: ReturnType<typeof parseUtf8Json>;
try {
parsed = parseUtf8Json(bytes, ENVIRONMENT_EXPORT_OPERATION, path);
} catch (cause: unknown) {
if (cause instanceof AnalysisResourceConstraintError) return err(cause);
throw cause;
}
if (!parsed.ok)
return err(
inputError(
{
path: [role],
reason: "invalid_format",
expected: "JSON",
message: parsed.error,
},
`${path} is not valid JSON: ${parsed.error}`,
),
);
const snapshot = await sealOwnedJsonInput(
parsed.value,
ENVIRONMENT_EXPORT_OPERATION,
);
if (!snapshot.ok) return snapshot;
return ok({ path, bytes, value: snapshot.value });
return environmentExportFile(role, path, bytes, operation);
};
/** Read both files once; the caller authenticates the parsed values and the export copies the same bytes. */
@@ -331,9 +368,9 @@ export const readEnvironmentExportInputs = async (
manifestPath: string,
evidencePath: string,
): Promise<Result<EnvironmentExportInputs, AnalysisError>> => {
const manifest = await readInput("manifest", manifestPath);
const manifest = await readEnvironmentExportFile("manifest", manifestPath);
if (!manifest.ok) return manifest;
const evidence = await readInput("evidence", evidencePath);
const evidence = await readEnvironmentExportFile("evidence", evidencePath);
if (!evidence.ok) return evidence;
return ok({ manifest: manifest.value, evidence: evidence.value });
};
@@ -343,6 +380,67 @@ const packageSourceValue = (source: EnvironmentPackageSource): JsonValue =>
? { kind: "registry", specifier: source.specifier }
: { kind: "tarball", path: source.path };
const changedArtifact = (
cause: RegularFileChangedError,
): AnalysisArtifactChangedError =>
new AnalysisArtifactChangedError(
ENVIRONMENT_EXPORT_OPERATION,
cause.path,
cause.message,
{ cause },
);
/** A reader failure as an output refusal that keeps an unconfirmed cleanup bound to its owner. */
const readerFailureError = (
cause: ArtifactReaderFailure,
directory: string | undefined,
): AnalysisOutputError => {
const host = hostSystemError(cause);
const error = new AnalysisOutputError(
ENVIRONMENT_EXPORT_OPERATION,
host === undefined ? cause.message : `${cause.message}: ${host.message}`,
{
cause,
// Only a host failure gets host recovery; an integrity or other
// refusal keeps its own message without disk-space advice.
...(cause.cleanup !== undefined
? { cleanup: cause.cleanup }
: directory === undefined
? {}
: host === undefined
? { diagnostics: { output_directory: directory } }
: hostWriteFailure(directory)),
},
);
bindArtifactFileCleanup(cause, error);
return error;
};
/** The package tarball's size after the same complete read and integrity check export uses. */
export const packageTarballBytes = async (
path: string,
): Promise<Result<number, AnalysisError>> => {
try {
return ok(
await withBundleTarball(path, async (tarball) => {
await bundleTarballDigest(tarball);
return tarball.size;
}),
);
} catch (cause: unknown) {
if (cause instanceof AnalysisError) return err(cause);
if (cause instanceof RegularFileChangedError)
return err(changedArtifact(cause));
if (cause instanceof ArtifactReaderFailure)
return err(readerFailureError(cause, undefined));
if (cause instanceof BundleFileHostFailure)
return err(packageTarballRefusal(path, cause));
if (systemCode(cause) !== undefined)
return err(packageTarballRefusal(path, cause));
throw cause;
}
};
/** Export the authenticated pair as an agent environment bundle folder. */
export const exportEnvironmentBundle = async (
input: EnvironmentExportInput,
@@ -356,6 +454,7 @@ export const exportEnvironmentBundle = async (
name: input.name,
packageSource,
served: (name) =>
input.served ??
prepareServedState(input.source, input.evidence.value, {
name,
operation: ENVIRONMENT_EXPORT_OPERATION,
@@ -379,35 +478,9 @@ export const exportEnvironmentBundle = async (
);
if (denied !== undefined) return err(denied);
if (cause instanceof RegularFileChangedError)
return err(
new AnalysisArtifactChangedError(
ENVIRONMENT_EXPORT_OPERATION,
cause.path,
cause.message,
{ cause },
),
);
if (cause instanceof ArtifactReaderFailure) {
const host = hostSystemError(cause);
const error = new AnalysisOutputError(
ENVIRONMENT_EXPORT_OPERATION,
host === undefined
? cause.message
: `${cause.message}: ${host.message}`,
{
cause,
// Only a host failure gets host recovery; an integrity or other
// refusal keeps its own message without disk-space advice.
...(cause.cleanup !== undefined
? { cleanup: cause.cleanup }
: host === undefined
? { diagnostics: { output_directory: input.directory } }
: hostWriteFailure(input.directory)),
},
);
bindArtifactFileCleanup(cause, error);
return err(error);
}
return err(changedArtifact(cause));
if (cause instanceof ArtifactReaderFailure)
return err(readerFailureError(cause, input.directory));
if (
cause instanceof BundleFileHostFailure ||
(cause instanceof Error && systemCode(cause) !== undefined)
src/application/environment/environmentScenario.test.ts
+1 -1
View File
@@ -82,7 +82,7 @@ describe("environment manifests from capture_browser_scenario parents", () => {
expect(projectAnalysisError(refused.error)).toMatchObject({
code: "unsupported_target",
message: expect.stringContaining(
"supports only the environment-manifest and environment-state views",
"supports only the environment-manifest, environment-state and environment-coverage views",
),
});
});
src/application/environment/environmentSource.ts
+5
View File
@@ -16,6 +16,7 @@ import {
type RecordedRequestLine,
} from "../../domain/analysisView/webNetworkRouteTemplates.js";
import { retainedBrowserScenarioCaptureSchema } from "../../domain/browserScenarioCapture.js";
import type { BrowserScenarioEvent } from "../../domain/browserScenarioCaptureValues.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisInputError,
@@ -263,6 +264,8 @@ interface CaptureFacts {
readonly responseBodies: boolean;
/** Whether the capture recorded request bodies, likewise. */
readonly requestBodies: boolean;
/** A browser scenario's retained events, including those no record represents; null for a HAR. */
readonly scenarioEvents: readonly BrowserScenarioEvent[] | null;
}
/** Issue paths of a capture payload refusal start here. */
@@ -297,6 +300,7 @@ const parseScenarioCapture = (
captureSha256: definition.success ? definition.data.scenario_sha256 : null,
responseBodies: scenario.data.scenario.network_content.response_body,
requestBodies: scenario.data.scenario.network_content.request_body,
scenarioEvents: scenario.data.events.items,
});
};
@@ -347,6 +351,7 @@ const parseHarCapture = (
captureSha256: capture.data.artifact.sha256,
responseBodies: true,
requestBodies: true,
scenarioEvents: null,
});
};
src/browser/BrowserScenarioPartialObservation.ts
+5
View File
@@ -43,6 +43,11 @@ const browserScenarioLimitations = (
: []),
"Playwright scenario events do not expose request initiator stacks; receipt order does not prove causality.",
"Storage values are hashed only after declared-secret redaction.",
...(scenario.actions.some(({ action }) => action === "request")
? [
"Request steps sent their declared requests through the page's own fetch with its cookies and session, following no redirects; they are REA-initiated, not user-interface activity, and header values of at least 8 characters they read from page cookies or local storage are redacted like declared secrets from records made after they were read.",
]
: []),
...(scenario.storage.local_storage.length === 0 &&
scenario.storage.session_storage.length === 0
? []
src/browser/BrowserScenarioSecrets.ts
+35 -7
View File
@@ -15,6 +15,9 @@ import type { BrowserStorageValueFingerprint } from "../domain/browserScenarioCa
const REDACTION_PREFIX = "[REDACTED:";
/** Page values shorter than this are not redacted: redacting them would rewrite ids, ports and hosts throughout the capture. */
export const MIN_REDACTED_PAGE_VALUE_LENGTH = 8;
const encodedSecretValues = (secret: string): ReadonlySet<string> =>
new Set([
encodeURIComponent(secret),
@@ -227,14 +230,19 @@ const secretNeedles = (
/** Resolved secret values kept only for one in-memory scenario session. */
export class BrowserScenarioSecrets {
private readonly urlPatterns: readonly SecretPattern[];
private readonly textPatterns: readonly SecretPattern[];
private readonly urlReplacements: readonly SecretReplacement[];
private readonly textReplacements: readonly SecretReplacement[];
private readonly textNeedles: readonly SecretNeedle[];
private readonly redactionCostPerCharacter: number;
private urlPatterns: readonly SecretPattern[] = [];
private textPatterns: readonly SecretPattern[] = [];
private urlReplacements: readonly SecretReplacement[] = [];
private textReplacements: readonly SecretReplacement[] = [];
private textNeedles: readonly SecretNeedle[] = [];
private redactionCostPerCharacter = 8;
private constructor(private readonly values: ReadonlyMap<string, string>) {
private constructor(private readonly values: Map<string, string>) {
this.compile();
}
private compile(): void {
const values = this.values;
this.urlPatterns = declaredPatterns(values, "url");
this.textPatterns = declaredPatterns(values, "text");
this.urlReplacements = secretReplacements(values, "url");
@@ -255,6 +263,26 @@ export class BrowserScenarioSecrets {
);
}
/**
* Redact a value a request step read from the page, such as a CSRF cookie
* or a stored bearer token, from everything recorded after it, as a
* declared secret is. The label names it in redaction markers; a value
* already redacted keeps its first marker, and a value shorter than
* {@link MIN_REDACTED_PAGE_VALUE_LENGTH} is not redacted.
*/
admitPageValue(label: string, value: string): void {
// fetch sends a header value without surrounding spaces and tabs, so that is the spelling records hold.
const sent = value.replace(/^[\t ]+|[\t ]+$/gu, "");
if (sent !== value) this.admitPageValue(label, sent);
if (value.length < MIN_REDACTED_PAGE_VALUE_LENGTH) return;
for (const held of this.values.values()) if (held === value) return;
let id = `page:${label}`;
for (let suffix = 2; this.values.has(id); suffix += 1)
id = `page:${label}~${String(suffix)}`;
this.values.set(id, value);
this.compile();
}
static resolve(
scenario: BrowserScenario,
environment: Readonly<Record<string, string | undefined>>,
src/browser/PlaywrightScenarioActions.ts
+21 -2
View File
@@ -5,6 +5,7 @@ import type {
BrowserScenarioValue,
} from "../domain/browserScenarioValues.js";
import type { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
import { performScenarioRequest } from "./PlaywrightScenarioRequest.js";
type ScenarioLocator = Extract<
BrowserScenarioAction,
@@ -34,13 +35,17 @@ interface PerformActionInput {
readonly page: Page;
readonly action: BrowserScenarioAction;
readonly secrets: BrowserScenarioSecrets;
/** Origin of the scenario's start URL, the only origin a request step may address. */
readonly startOrigin: string;
/** Cancellation of the step; a request step that sees it before sending sends nothing. */
readonly signal?: AbortSignal | undefined;
}
/** Execute one admitted declarative action with no caller-supplied code. */
/** Execute one admitted declarative action; a request step sends one declared request through the page's own fetch, and no action runs caller-supplied code. */
export const performPlaywrightScenarioAction = async (
input: PerformActionInput,
): Promise<void> => {
const { page, action, secrets } = input;
const { page, action, secrets, startOrigin, signal } = input;
const timeout = "timeout_ms" in action ? action.timeout_ms : undefined;
const options = timeout === undefined ? {} : { timeout };
switch (action.action) {
@@ -86,5 +91,19 @@ export const performPlaywrightScenarioAction = async (
return;
case "wait_for_timeout":
await page.waitForTimeout(action.duration_ms);
return;
case "request":
await performScenarioRequest({
page,
action,
secrets,
startOrigin,
signal,
});
return;
default: {
const unhandled: never = action;
throw new Error(`Unhandled browser scenario action ${String(unhandled)}`);
}
}
};
src/browser/PlaywrightScenarioRequest.test.ts
+591
View File
@@ -0,0 +1,591 @@
import type { Page, Route } from "playwright-core";
import { afterEach, describe, expect, it, vi } from "vitest";
import { browserScenarioSchema } from "../domain/browserScenario.js";
import type { BrowserScenarioAction } from "../domain/browserScenarioValues.js";
import { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
import { performPlaywrightScenarioAction } from "./PlaywrightScenarioActions.js";
const ORIGIN = "https://app.example.test";
const scenario = (actions: readonly unknown[], secrets: unknown[] = []) =>
browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/opt/chromium/chrome" },
start_url: { url: `${ORIGIN}/` },
actions,
storage: { cookies: [], local_storage: [], session_storage: [] },
secrets,
capture: {},
});
const requestStep = (fields: Record<string, unknown>) => ({
step_id: "create",
action: "request",
method: "POST",
path: "/api/folders",
...fields,
});
const issues = (actions: readonly unknown[], secrets: unknown[] = []) => {
const parsed = browserScenarioSchema.safeParse({
browser: { mode: "launch", executable_path: "/opt/chromium/chrome" },
start_url: { url: `${ORIGIN}/` },
actions,
storage: { cookies: [], local_storage: [], session_storage: [] },
secrets,
capture: {},
});
return parsed.success
? []
: parsed.error.issues.map((issue) => issue.message);
};
describe("request step schema", () => {
it("refuses repeated header names, User-Agent and line breaks in literal values or prefixes", () => {
const literal = (value: string) => ({ source: "literal", value });
expect(
issues([
requestStep({
headers: [
{ name: "X-A", value: literal("one") },
{ name: "x-a", value: literal("two") },
],
}),
]),
).toContainEqual(expect.stringContaining("more than once"));
expect(
issues([
requestStep({
headers: [{ name: "User-Agent", value: literal("agent") }],
}),
]),
).toContainEqual(expect.stringContaining("cannot set"));
expect(
issues([requestStep({ expected_status: [200, 303] })]),
).toContainEqual(expect.stringContaining("3xx"));
for (const header of [
{ name: "X-A", value: literal("a\r\nX-B: b") },
{ name: "X-A", value: literal("a"), prefix: "Bearer\n" },
])
expect(issues([requestStep({ headers: [header] })])).toContainEqual(
expect.stringContaining("line breaks"),
);
});
it("accepts a same-origin request with query, headers and a JSON body", () => {
const parsed = scenario(
[
requestStep({
query: [{ name: "org", value: { source: "literal", value: "1" } }],
headers: [
{
name: "X-CSRFToken",
value: { source: "cookie", name: "csrftoken" },
},
{
name: "Authorization",
value: { source: "secret", secret_id: "token" },
prefix: "Bearer ",
},
],
json: { title: "Scratch folder 1" },
expected_status: [200, 201],
}),
],
[{ secret_id: "token", environment_variable: "APP_TOKEN" }],
);
expect(parsed.actions[0]).toMatchObject({
action: "request",
method: "POST",
path: "/api/folders",
});
});
it("refuses paths that leave the start origin", () => {
for (const path of [
"//evil.test/x",
"/\\evil.test/x",
"https://evil.test/",
"/\n/[bad]",
])
expect(issues([requestStep({ path })]).length, path).toBeGreaterThan(0);
});
it("refuses a body on GET, a header fetch cannot set and an undeclared secret", () => {
expect(
issues([requestStep({ method: "GET", json: { a: 1 } })]),
).toContainEqual(expect.stringContaining("cannot send a body"));
expect(
issues([
requestStep({
headers: [
{ name: "Cookie", value: { source: "literal", value: "a=b" } },
],
}),
]),
).toContainEqual(expect.stringContaining("cannot set this header"));
expect(
issues([
requestStep({
headers: [
{ name: "X-Key", value: { source: "secret", secret_id: "key" } },
],
}),
]),
).toContainEqual(expect.stringContaining("not declared"));
});
});
interface FakeFetch {
readonly url: string;
readonly init: RequestInit;
}
const fakePage = (options: {
readonly url?: string;
/** The origin the page reports when the request is sent, if it moved after the check. */
readonly sendingOrigin?: string;
readonly cookies?: readonly { name: string; value: string }[];
readonly cookieDelayMs?: number;
readonly evaluateDelayMs?: number;
readonly storage?: Record<string, string>;
}) => {
const url = options.url ?? `${ORIGIN}/dashboard`;
vi.stubGlobal("localStorage", {
getItem: (key: string) => options.storage?.[key] ?? null,
});
vi.stubGlobal("location", {
origin: options.sendingOrigin ?? new URL(url).origin,
});
return {
route: async () => {},
unroute: async () => {},
url: () => url,
context: () => ({
cookies: async () => {
if (options.cookieDelayMs !== undefined)
await new Promise((resolve) =>
setTimeout(resolve, options.cookieDelayMs),
);
return options.cookies ?? [];
},
}),
evaluate: async <Arg, Result>(
fn: (arg: Arg) => Result | Promise<Result>,
arg: Arg,
): Promise<Result> => {
if (options.evaluateDelayMs !== undefined)
await new Promise((resolve) =>
setTimeout(resolve, options.evaluateDelayMs),
);
return fn(arg);
},
} as unknown as Page;
};
const stubFetch = (status: number) => {
const calls: FakeFetch[] = [];
vi.stubGlobal("fetch", async (url: string, init: RequestInit) => {
calls.push({ url, init });
return new Response("{}", { status });
});
return calls;
};
const perform = async (
page: Page,
action: BrowserScenarioAction,
secrets: BrowserScenarioSecrets,
) =>
performPlaywrightScenarioAction({
page,
action,
secrets,
startOrigin: ORIGIN,
});
const resolved = (parsed: ReturnType<typeof scenario>, environment = {}) => {
const secrets = BrowserScenarioSecrets.resolve(parsed, environment);
if (!secrets.ok) throw secrets.error;
return secrets.value;
};
describe("request step execution", () => {
afterEach(() => {
vi.unstubAllGlobals();
});
it("sends the declared request through the page's fetch with its cookies", async () => {
const calls = stubFetch(201);
const parsed = scenario([
requestStep({
query: [{ name: "org", value: { source: "literal", value: "1" } }],
json: { title: "Scratch folder 1" },
}),
]);
await perform(fakePage({}), parsed.actions[0]!, resolved(parsed));
expect(calls).toEqual([
{
url: `${ORIGIN}/api/folders?org=1`,
init: expect.objectContaining({
method: "POST",
credentials: "same-origin",
headers: [["content-type", "application/json"]],
body: '{"title":"Scratch folder 1"}',
}),
},
]);
});
it("reads header values from a cookie and from stored JSON, and redacts them afterwards", async () => {
const calls = stubFetch(200);
const parsed = scenario([
requestStep({
headers: [
{
name: "X-CSRFToken",
value: { source: "cookie", name: "csrftoken" },
},
{
name: "Authorization",
value: {
source: "local_storage",
key: "auth",
json_pointer: "/token",
},
prefix: "Bearer ",
},
],
}),
]);
const secrets = resolved(parsed);
await perform(
fakePage({
cookies: [{ name: "csrftoken", value: "csrf-value-123" }],
storage: { auth: JSON.stringify({ token: "jwt-value-456" }) },
}),
parsed.actions[0]!,
secrets,
);
expect(calls[0]!.init.headers).toEqual([
["X-CSRFToken", "csrf-value-123"],
["Authorization", "Bearer jwt-value-456"],
]);
const redacted = secrets.redact("csrf-value-123 and jwt-value-456");
expect(redacted).not.toContain("csrf-value-123");
expect(redacted).not.toContain("jwt-value-456");
expect(redacted).toContain("[REDACTED:page:create.X-CSRFToken]");
});
it("refuses to run off the start origin, and fails on an unexpected status or a missing cookie", async () => {
stubFetch(403);
const parsed = scenario([requestStep({ expected_status: [200, 201] })]);
const action = parsed.actions[0]!;
await expect(
perform(
fakePage({ url: "https://other.test/" }),
action,
resolved(parsed),
),
).rejects.toThrow("start origin");
await expect(
perform(fakePage({}), action, resolved(parsed)),
).rejects.toThrow("answered 403; expected 200 or 201");
const withCookie = scenario([
requestStep({
headers: [{ name: "X-T", value: { source: "cookie", name: "absent" } }],
}),
]);
await expect(
perform(fakePage({}), withCookie.actions[0]!, resolved(withCookie)),
).rejects.toThrow("cookie absent is not set");
});
it("completes on any status when none is expected and reports fetch errors", async () => {
stubFetch(500);
const parsed = scenario([requestStep({})]);
await expect(
perform(fakePage({}), parsed.actions[0]!, resolved(parsed)),
).resolves.toBeUndefined();
vi.stubGlobal("fetch", async () => {
throw new TypeError("Failed to fetch");
});
await expect(
perform(fakePage({}), parsed.actions[0]!, resolved(parsed)),
).rejects.toThrow("Failed to fetch");
});
});
describe("request dispatch boundary", () => {
afterEach(() => {
vi.unstubAllGlobals();
});
it("sends nothing when browser evaluation starts after the absolute deadline", async () => {
const calls = stubFetch(200);
const parsed = scenario([requestStep({ timeout_ms: 5 })]);
await expect(
perform(
fakePage({ evaluateDelayMs: 30 }),
parsed.actions[0]!,
resolved(parsed),
),
).rejects.toThrow("ran out of time");
expect(calls).toEqual([]);
});
it("guards dispatch after cancellation while evaluation was queued, and removes its route", async () => {
const parsed = scenario([requestStep({})]);
const controller = new AbortController();
const page = fakePage({});
let gate: ((route: Route) => Promise<void>) | undefined;
vi.spyOn(page, "route").mockImplementation(async (_match, handler) => {
gate = handler as typeof gate;
});
const unroute = vi.spyOn(page, "unroute");
const sent = vi.fn();
vi.spyOn(page, "evaluate").mockImplementation(async (fn, arg) => {
controller.abort("abandoned while queued");
return (fn as (arg: unknown) => unknown)(arg);
});
vi.stubGlobal("fetch", async (_url: string, init: RequestInit) => {
if (gate === undefined) throw new Error("Missing dispatch owner");
await gate({
request: () => ({
method: () => init.method,
postData: () => init.body ?? null,
}),
fallback: sent,
abort: async () => {
throw new Error("aborted at dispatch");
},
} as unknown as Route);
return new Response("{}", { status: 200 });
});
await expect(
performPlaywrightScenarioAction({
page,
action: parsed.actions[0]!,
secrets: resolved(parsed),
startOrigin: ORIGIN,
signal: controller.signal,
}),
).rejects.toThrow("aborted at dispatch");
expect(sent).not.toHaveBeenCalled();
expect(unroute).toHaveBeenCalledOnce();
});
it("refuses foreign-origin storage before reading its value", async () => {
const calls = stubFetch(200);
const read = vi.fn();
const parsed = scenario([
requestStep({
headers: [
{
name: "Authorization",
value: { source: "local_storage", key: "token" },
},
],
}),
]);
const page = fakePage({ sendingOrigin: "https://other.test" });
vi.stubGlobal("localStorage", { getItem: read });
await expect(
perform(page, parsed.actions[0]!, resolved(parsed)),
).rejects.toThrow("local storage");
expect(read).not.toHaveBeenCalled();
expect(calls).toEqual([]);
});
it("drains response chunks and releases the reader without materializing the body", async () => {
const read = vi
.fn()
.mockResolvedValueOnce({ done: false, value: new Uint8Array(8) })
.mockResolvedValueOnce({ done: true });
const releaseLock = vi.fn();
const arrayBuffer = vi.fn();
vi.stubGlobal("fetch", async () => ({
status: 200,
type: "basic",
body: { getReader: () => ({ read, releaseLock }) },
arrayBuffer,
}));
const parsed = scenario([requestStep({})]);
await perform(fakePage({}), parsed.actions[0]!, resolved(parsed));
expect(read).toHaveBeenCalledTimes(2);
expect(releaseLock).toHaveBeenCalledOnce();
expect(arrayBuffer).not.toHaveBeenCalled();
});
});
describe("request step confinement", () => {
afterEach(() => {
vi.unstubAllGlobals();
});
it("follows no redirect and sends nothing once the page has left the start origin", async () => {
const calls: FakeFetch[] = [];
vi.stubGlobal("fetch", async (url: string, init: RequestInit) => {
calls.push({ url, init });
return {
type: "opaqueredirect",
status: 0,
arrayBuffer: async () => new ArrayBuffer(0),
};
});
const parsed = scenario([requestStep({})]);
await expect(
perform(fakePage({}), parsed.actions[0]!, resolved(parsed)),
).rejects.toThrow("request steps do not follow");
expect(calls[0]!.init.redirect).toBe("manual");
await expect(
perform(
fakePage({ sendingOrigin: "https://other.test" }),
parsed.actions[0]!,
resolved(parsed),
),
).rejects.toThrow("the page moved");
expect(calls).toHaveLength(1);
});
it("sends nothing when its time runs out while header values are read, or the step is cancelled", async () => {
const calls = stubFetch(200);
const parsed = scenario([
requestStep({
timeout_ms: 5,
headers: [{ name: "X-T", value: { source: "cookie", name: "t" } }],
}),
]);
const cookies = [{ name: "t", value: "token-value-1" }];
await expect(
perform(
fakePage({ cookies, cookieDelayMs: 30 }),
parsed.actions[0]!,
resolved(parsed),
),
).rejects.toThrow("ran out of time");
const cancelled = new AbortController();
cancelled.abort();
await expect(
performPlaywrightScenarioAction({
page: fakePage({ cookies }),
action: scenario([requestStep({})]).actions[0]!,
secrets: resolved(parsed),
startOrigin: ORIGIN,
signal: cancelled.signal,
}),
).rejects.toThrow("ran out of time");
expect(calls).toEqual([]);
});
it("appends query entries after the path's literal query without re-encoding it", async () => {
const calls = stubFetch(200);
const parsed = scenario([
requestStep({
path: "/api/items?flag&a=b%20c&s=x;y",
query: [{ name: "q", value: { source: "literal", value: "a b" } }],
}),
]);
await perform(fakePage({}), parsed.actions[0]!, resolved(parsed));
expect(calls[0]!.url).toBe(
`${ORIGIN}/api/items?flag&a=b%20c&s=x;y&q=a%20b`,
);
for (const [path, sent] of [
["/api/items?", "/api/items?q=1"],
["/api/items#", "/api/items?q=1"],
["/api/items?a=1&", "/api/items?a=1&q=1"],
["/api/items?a=1#top", "/api/items?a=1&q=1"],
] as const) {
const edge = scenario([
requestStep({
path,
query: [{ name: "q", value: { source: "literal", value: "1" } }],
}),
]);
await perform(fakePage({}), edge.actions[0]!, resolved(edge));
expect(calls.at(-1)!.url, path).toBe(`${ORIGIN}${sent}`);
}
});
it("refuses page values a header cannot carry, and stored values it cannot read", async () => {
const calls = stubFetch(200);
const header = (value: Record<string, unknown>) =>
scenario([requestStep({ headers: [{ name: "X-T", value }] })]);
const cookie = header({ source: "cookie", name: "t" });
await expect(
perform(
fakePage({ cookies: [{ name: "t", value: "a\r\nX-B: b" }] }),
cookie.actions[0]!,
resolved(cookie),
),
).rejects.toThrow("cannot carry");
for (const [storage, pointer, message] of [
[{}, undefined, "local storage has no auth"],
[{ auth: "not json" }, "/token", "is not JSON"],
[{ auth: '{"other":1}' }, "/token", "no string or number at /token"],
] as const) {
const stored = header({
source: "local_storage",
key: "auth",
...(pointer === undefined ? {} : { json_pointer: pointer }),
});
await expect(
perform(fakePage({ storage }), stored.actions[0]!, resolved(stored)),
).rejects.toThrow(message);
}
expect(calls).toEqual([]);
const numeric = header({
source: "local_storage",
key: "auth",
json_pointer: "/org",
});
await perform(
fakePage({ storage: { auth: '{"org":42}' } }),
numeric.actions[0]!,
resolved(numeric),
);
expect(calls[0]!.init.headers).toEqual([["X-T", "42"]]);
});
});
describe("page value redaction", () => {
const secrets = () => resolved(scenario([requestStep({})]));
it("keeps every admitted value redacted, even under a repeated or colliding label", () => {
const held = secrets();
held.admitPageValue("s.x-a", "first-secret-AAAA");
held.admitPageValue("s.x-a", "second-secret-BBBB");
held.admitPageValue("a.b.c", "third-secret-CCCC");
held.admitPageValue("a.b.c", "fourth-secret-DDDD");
const text = held.redact(
"first-secret-AAAA second-secret-BBBB third-secret-CCCC fourth-secret-DDDD",
);
for (const value of ["AAAA", "BBBB", "CCCC", "DDDD"])
expect(text).not.toContain(value);
});
it("keeps a value's first marker when later steps read it again", () => {
const held = secrets();
held.admitPageValue("create-1.X-CSRFToken", "csrf-token-value");
held.admitPageValue("create-2.X-CSRFToken", "csrf-token-value");
expect(held.redact("csrf-token-value")).toBe(
"[REDACTED:page:create-1.X-CSRFToken]",
);
});
it("redacts the value fetch sends when the stored one has surrounding spaces", () => {
const held = secrets();
held.admitPageValue("s.X-Tok", " padded-token-4f9e2a\t");
expect(held.redact("X-Tok: padded-token-4f9e2a")).not.toContain(
"padded-token-4f9e2a",
);
});
it("does not redact values too short to tell from ids and ports", () => {
const held = secrets();
held.admitPageValue("s.X-Org-Id", "1");
held.admitPageValue("s.X-Port", "3000");
const text = "http://127.0.0.1:3000/api/items/12?page=1";
expect(held.redact(text)).toBe(text);
});
});
src/browser/PlaywrightScenarioRequest.ts
+323
View File
@@ -0,0 +1,323 @@
import type { Page, Route } from "playwright-core";
import {
HEADER_FIELD_VALUE,
type BrowserScenarioAction,
type BrowserScenarioRequestHeader,
} from "../domain/browserScenarioValues.js";
import type { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
type RequestAction = Extract<BrowserScenarioAction, { action: "request" }>;
/** What the page's fetch returned; a redirect is not followed, so its status is unknown. */
type PageResponse =
| { readonly status: number }
| { readonly redirected: true }
| { readonly left: string }
| { readonly error: string };
/** A request step that could not be sent or did not answer as expected. */
export class ScenarioRequestError extends Error {
override readonly name = "ScenarioRequestError";
}
const originOf = (url: string): string | undefined => {
try {
return new URL(url).origin;
} catch {
return undefined;
}
};
/** The string or number at a JSON pointer, as text. */
const pointerText = (value: unknown, pointer: string): string | undefined => {
let current = value;
if (pointer !== "")
for (const token of pointer.slice(1).split("/")) {
const key = token.replaceAll("~1", "/").replaceAll("~0", "~");
if (Array.isArray(current) && /^(?:0|[1-9][0-9]*)$/u.test(key))
current = current[Number(key)];
else if (
typeof current === "object" &&
current !== null &&
Object.hasOwn(current, key)
)
current = (current as Record<string, unknown>)[key];
else return undefined;
}
return typeof current === "string" || typeof current === "number"
? String(current)
: undefined;
};
/** Resolve one header value; values read from the page are redacted from later records. */
const headerValue = async (
step: {
readonly page: Page;
readonly url: URL;
readonly stepId: string;
readonly secrets: BrowserScenarioSecrets;
},
header: BrowserScenarioRequestHeader,
): Promise<string> => {
const { page, url, stepId, secrets } = step;
const { value } = header;
if (value.source === "literal" || value.source === "secret")
return secrets.value(value);
let read: string | undefined;
if (value.source === "cookie") {
const cookies = await page.context().cookies(url.href);
read = cookies.find((cookie) => cookie.name === value.name)?.value;
if (read === undefined)
throw new ScenarioRequestError(
`Request step ${stepId}: cookie ${value.name} is not set for ${url.origin}.`,
);
} else {
const stored = await page.evaluate(
({ key, origin }) => {
if (
(globalThis as { readonly location?: { origin: string } }).location
?.origin !== origin
)
throw new Error(
"The page moved off the start origin before its local storage was read.",
);
return (
(
globalThis as {
readonly localStorage?: { getItem(key: string): string | null };
}
).localStorage?.getItem(key) ?? null
);
},
{ key: value.key, origin: url.origin },
);
if (stored === null)
throw new ScenarioRequestError(
`Request step ${stepId}: local storage has no ${value.key} on ${url.origin}.`,
);
if (value.json_pointer === undefined) read = stored;
else {
let parsed: unknown;
try {
parsed = JSON.parse(stored);
} catch {
throw new ScenarioRequestError(
`Request step ${stepId}: local storage ${value.key} is not JSON.`,
);
}
read = pointerText(parsed, value.json_pointer);
if (read === undefined)
throw new ScenarioRequestError(
`Request step ${stepId}: local storage ${value.key} has no string or number at ${value.json_pointer}.`,
);
}
}
secrets.admitPageValue(`${stepId}.${header.name}`, read);
return read;
};
/** Append declared query entries after any literal query the path carries, leaving that query as written. */
const withQuery = (
url: URL,
entries: readonly (readonly [string, string])[],
): string => {
if (entries.length === 0) return url.href;
const added = entries
.map(
([name, value]) =>
`${encodeURIComponent(name)}=${encodeURIComponent(value)}`,
)
.join("&");
// fetch drops the fragment; cut it at its "#" even when nothing follows it.
const hash = url.href.indexOf("#");
const base = hash < 0 ? url.href : url.href.slice(0, hash);
const separator = !base.includes("?")
? "?"
: base.endsWith("?") || base.endsWith("&")
? ""
: "&";
return `${base}${separator}${added}`;
};
/** What a request step sends: the URL, the origin it must be sent from, and the call. */
interface PageRequest {
readonly url: string;
readonly origin: string;
readonly method: string;
readonly headers: [string, string][];
readonly body: string | undefined;
readonly deadline: number | undefined;
}
/** Send the request with the page's own fetch, from the page's main world, and read the whole answer. */
const sendFromPage = (
page: Page,
request: PageRequest,
): Promise<PageResponse> =>
page.evaluate(async (request): Promise<PageResponse> => {
// The page may have navigated since the origin check; never send from another origin.
const here = (globalThis as { readonly location?: { origin: string } })
.location?.origin;
if (here !== request.origin) return { left: String(here) };
if (
(
globalThis as {
readonly navigator?: { serviceWorker?: { controller?: unknown } };
}
).navigator?.serviceWorker?.controller
)
return {
error:
"Request steps require a page without a controlling service worker so cancellation can guard dispatch.",
};
const remaining =
request.deadline === undefined
? undefined
: request.deadline - Date.now();
if (remaining !== undefined && remaining <= 0)
return { error: "The request ran out of time before it was sent." };
const controller = new AbortController();
const timer =
remaining === undefined
? undefined
: setTimeout(() => controller.abort(), remaining);
try {
const answer = await fetch(request.url, {
method: request.method,
headers: request.headers,
credentials: "same-origin",
redirect: "manual",
signal: controller.signal,
...(request.body === undefined ? {} : { body: request.body }),
});
const reader = answer.body?.getReader();
if (reader !== undefined)
try {
while (!(await reader.read()).done) {
// Wait for completion without retaining a second response body.
}
} finally {
reader.releaseLock();
}
return answer.type === "opaqueredirect"
? { redirected: true }
: { status: answer.status };
} catch (cause: unknown) {
return {
error: cause instanceof Error ? cause.message : String(cause),
};
} finally {
if (timer !== undefined) clearTimeout(timer);
}
}, request);
/**
* Send one request from the page with its own fetch, cookies and session, as
* the application's frontend would, and read the whole response so the
* capture can record it. The page must be on the scenario's start origin.
*/
export const performScenarioRequest = async (input: {
readonly page: Page;
readonly action: RequestAction;
readonly secrets: BrowserScenarioSecrets;
readonly startOrigin: string;
readonly signal?: AbortSignal | undefined;
}): Promise<void> => {
const { page, action, secrets, startOrigin, signal } = input;
const deadline =
action.timeout_ms === undefined
? undefined
: Date.now() + action.timeout_ms;
const pageOrigin = originOf(page.url());
if (pageOrigin !== startOrigin)
throw new ScenarioRequestError(
`Request step ${action.step_id} runs from a page on the start origin ${startOrigin}; navigate back to it with a goto step first.`,
);
const url = new URL(action.path, startOrigin);
if (url.origin !== startOrigin)
throw new ScenarioRequestError(
`Request step ${action.step_id} must stay on the start origin ${startOrigin}.`,
);
const target = withQuery(
url,
action.query.map(({ name, value }) => [name, secrets.value(value)]),
);
const step = { page, url, stepId: action.step_id, secrets };
const headers: [string, string][] = [];
for (const header of action.headers) {
const value = `${header.prefix ?? ""}${await headerValue(step, header)}`;
if (!HEADER_FIELD_VALUE.test(value))
throw new ScenarioRequestError(
`Request step ${action.step_id}: the ${header.name} value holds characters a header cannot carry, such as a line break.`,
);
headers.push([header.name, value]);
}
const hasContentType = headers.some(
([name]) => name.toLowerCase() === "content-type",
);
if (action.json !== undefined && !hasContentType)
headers.push(["content-type", "application/json"]);
// A step the boundary abandoned, or whose time ran out while values were read, sends nothing.
const remaining = deadline === undefined ? undefined : deadline - Date.now();
if (signal?.aborted === true || (remaining !== undefined && remaining <= 0))
throw new ScenarioRequestError(
`Request step ${action.step_id} ran out of time before its request was sent.`,
);
const request = {
url: target,
origin: startOrigin,
method: action.method,
headers,
body: action.json === undefined ? undefined : JSON.stringify(action.json),
deadline,
};
// The browser can queue evaluate behind a busy page. Keep a host-side gate
// until that evaluation settles, including after the step boundary stops
// waiting. A queued fetch then observes live cancellation at dispatch.
const dispatchUrl = new URL(target);
dispatchUrl.hash = "";
const matches = (url: URL): boolean => url.href === dispatchUrl.href;
const gate = async (route: Route): Promise<void> => {
const sent = route.request();
if (
sent.method() !== request.method ||
sent.postData() !== (request.body ?? null)
) {
await route.fallback();
return;
}
if (
signal?.aborted === true ||
(deadline !== undefined && Date.now() >= deadline)
)
await route.abort("aborted");
else await route.fallback();
};
await page.route(matches, gate);
let response: PageResponse;
try {
response = await sendFromPage(page, request);
} finally {
await page.unroute(matches, gate);
}
if ("error" in response)
throw new ScenarioRequestError(
`Request step ${action.step_id} failed in the page: ${response.error}`,
);
if ("left" in response)
throw new ScenarioRequestError(
`Request step ${action.step_id} runs from a page on the start origin ${startOrigin}; the page moved before the request was sent.`,
);
if ("redirected" in response)
throw new ScenarioRequestError(
`Request step ${action.step_id} was answered with a redirect, which request steps do not follow; a write may already have been applied. The capture records the redirect and, with header values selected, its Location.`,
);
if (
action.expected_status !== undefined &&
!action.expected_status.includes(response.status)
)
throw new ScenarioRequestError(
`Request step ${action.step_id} answered ${String(response.status)}; expected ${action.expected_status.join(" or ")}.`,
);
};
src/browser/PlaywrightScenarioSession.ts
+6
View File
@@ -82,6 +82,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
readonly version: string;
readonly initialUrl: string;
private readonly secrets: BrowserScenarioSecrets;
private readonly startOrigin: string;
private readonly eventCapture: PlaywrightScenarioEvents;
private readonly storage: PlaywrightScenarioStorage | undefined;
private readonly serviceWorkers: PlaywrightScenarioServiceWorkers | undefined;
@@ -101,6 +102,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
options: {
readonly mode: BrowserScenario["browser"]["mode"];
readonly secrets: BrowserScenarioSecrets;
readonly startOrigin: string;
readonly eventCapture: PlaywrightScenarioEvents;
readonly storage: PlaywrightScenarioStorage | undefined;
readonly serviceWorkers: PlaywrightScenarioServiceWorkers | undefined;
@@ -111,6 +113,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
this.processOwnership =
options.mode === "launch" ? "provider-owned" : "external";
this.secrets = options.secrets;
this.startOrigin = options.startOrigin;
this.eventCapture = options.eventCapture;
this.storage = options.storage;
this.serviceWorkers = options.serviceWorkers;
@@ -327,6 +330,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
return new PlaywrightScenarioSession(opened, {
mode: scenario.browser.mode,
secrets,
startOrigin: new URL(scenario.start_url.url).origin,
eventCapture: events,
storage,
serviceWorkers,
@@ -439,6 +443,8 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
page: this.opened.page,
action,
secrets: this.secrets,
startOrigin: this.startOrigin,
signal,
}),
"timeout_ms" in action ? action.timeout_ms : undefined,
signal,
src/cli/environmentCommands.ts
+247 -4
View File
@@ -1,7 +1,11 @@
import { basename } from "node:path";
import { localPathStringSchema } from "../domain/localPath.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import type { EnvironmentExportResult } from "../application/environment/environmentExport.js";
import type { EnvironmentBuildCaptureInput } from "../application/environment/environmentBuild.js";
import type {
EnvironmentExportFile,
EnvironmentExportResult,
} from "../application/environment/environmentExport.js";
import type { EnvironmentSource } from "../application/environment/environmentSource.js";
import type { ServedState } from "../application/environment/environmentState.js";
import { z } from "incur";
@@ -21,13 +25,17 @@ import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.
import type { StreamableHttpListenError } from "../server/http/streamableHttpListenError.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import { createLogger } from "../logger.js";
import { TEMPLATE_RULES } from "../domain/analysisView/environmentManifestRules.js";
import { inspectWebNetworkCaptureInputSchema } from "../domain/webNetworkCapture.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
import type { CliInstance } from "./types.js";
/** Same envelope label as the other JSON-input commands and `parseCliJsonInput`. */
@@ -36,8 +44,8 @@ const cliError = (error: AnalysisError): JsonValue => ({
...projectAnalysisError(error),
});
/** The listener announces its URL at info like `rea mcp --http`; an explicit REA_LOG_LEVEL still wins. */
const listenerLogger = (
/** The listener announces its URL, and a build its stages, at info like `rea mcp --http`; an explicit REA_LOG_LEVEL still wins. */
const infoLogger = (
environment: Readonly<NodeJS.ProcessEnv>,
logger: Logger,
): Logger =>
@@ -178,7 +186,7 @@ const serveEnvironmentCommand = async (
source,
served: { state, report },
} = inputs.value;
const commandLogger = listenerLogger(environment, logger);
const commandLogger = infoLogger(environment, logger);
if (!report.served && report.reason === "unavailable")
commandLogger.warn(
{ code: report.code, error: report.message },
@@ -305,6 +313,240 @@ const exportEnvironmentCommand = async (
: cliError(exported.error);
};
interface BuildArguments {
readonly capture: string;
readonly directory?: string | undefined;
readonly name?: string | undefined;
readonly templateRule?:
| readonly (typeof TEMPLATE_RULES)[number][]
| undefined;
readonly formatRulesOnly: boolean;
readonly packageTarball?: string | undefined;
readonly dryRun: boolean;
readonly captureTimeoutMs?: number | undefined;
}
const conflictingRules = (): JsonValue =>
cliError(
new AnalysisInputError(CLI_COMMANDS.environmentBuild, undefined, [
{
path: ["format_rules_only"],
reason: "invalid_value",
message:
"--format-rules-only applies no opt-in template rules; it cannot be combined with --template-rule.",
},
]),
);
const isEvidenceValue = (value: unknown): boolean =>
typeof value === "object" &&
value !== null &&
"evidence_id" in value &&
"normalized_result" in value;
/** Inspect a HAR as inspect-web-network-capture does and serialize the Evidence as it prints it; a refusal is the command's output. */
const inspectBuildHar = async (
capturePath: string,
captureTimeoutMs: number | undefined,
environment: Readonly<NodeJS.ProcessEnv>,
signal: AbortSignal,
): Promise<Result<EnvironmentExportFile, JsonValue>> => {
const { createWebNetworkCaptureService } =
await import("../composition/webNetworkCaptures.js");
const { parseEvidence } = await import("../domain/evidence.js");
const { environmentExportEvidenceFile } =
await import("../application/environment/environmentExport.js");
const { ENVIRONMENT_BUILD_OPERATION } =
await import("../application/environment/environmentBuild.js");
const service = createWebNetworkCaptureService(environment);
const inspected = await finishAnalysisCommand(
() =>
service.inspect(
{
capture_path: capturePath,
format: "har",
...(captureTimeoutMs === undefined
? {}
: { capture_timeout_ms: captureTimeoutMs }),
},
{ signal },
),
service,
"web-network-capture",
"inspect_web_network_capture",
signal,
);
if (!isEvidenceValue(inspected))
return err({
error: "Application workflow failed",
...(inspected as Record<string, JsonValue>),
});
const file = await environmentExportEvidenceFile(
"evidence",
capturePath,
parseEvidence(inspected),
ENVIRONMENT_BUILD_OPERATION,
);
return file.ok ? ok(file.value) : err(cliError(file.error));
};
/** Read the capture once, then build or plan the bundle from it. */
const buildEnvironmentCommand = async (
input: BuildArguments,
environment: Readonly<NodeJS.ProcessEnv>,
logger: Logger,
signal: AbortSignal,
): Promise<JsonValue> => {
if (input.formatRulesOnly && input.templateRule !== undefined)
return conflictingRules();
const {
buildEnvironment,
classifyBuildCapture,
DEFAULT_BUILD_TEMPLATE_RULES,
ENVIRONMENT_BUILD_OPERATION,
} = await import("../application/environment/environmentBuild.js");
const capturePath = absoluteCallerPath(input.capture);
const kind = classifyBuildCapture(capturePath);
if (!kind.ok) return cliError(kind.error);
if (input.directory === undefined && !input.dryRun)
return cliError(
new AnalysisInputError(
CLI_COMMANDS.environmentBuild,
{ userMessage: "Name the bundle directory, or pass --dry-run." },
[
{
path: ["directory"],
reason: "invalid_value",
message: "The bundle directory may be omitted only with --dry-run.",
},
],
),
);
const commandLogger = infoLogger(environment, logger);
const progress = (stage: string): void => {
commandLogger.info({ stage }, "Building environment");
};
let capture: EnvironmentExportFile;
if (kind.value === "har") {
progress("inspect");
const inspected = await inspectBuildHar(
capturePath,
input.captureTimeoutMs,
environment,
signal,
);
if (!inspected.ok) return inspected.error;
capture = inspected.value;
} else {
const { readEnvironmentExportFile } =
await import("../application/environment/environmentExport.js");
const read = await readEnvironmentExportFile(
"evidence",
capturePath,
ENVIRONMENT_BUILD_OPERATION,
);
if (!read.ok) return cliError(read.error);
capture = read.value;
}
if (signal.aborted)
return cliError(new AnalysisCancelledError(CLI_COMMANDS.environmentBuild));
const captureInput: EnvironmentBuildCaptureInput = kind.value;
const built = await buildEnvironment({
capture,
captureInput,
capturePath,
templateRules: input.formatRulesOnly
? []
: (input.templateRule ?? DEFAULT_BUILD_TEMPLATE_RULES),
directory:
input.directory === undefined
? undefined
: absoluteCallerPath(input.directory),
name: input.name,
packageTarball:
input.packageTarball === undefined
? undefined
: absoluteCallerPath(input.packageTarball),
dryRun: input.dryRun,
progress,
signal,
});
return built.ok
? (built.value as unknown as JsonValue)
: cliError(built.error);
};
/** Register the one-step build from a recorded capture to a bundle folder. */
const registerBuildCommand = (
cli: CliInstance,
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
): void => {
cli.command(CLI_COMMANDS.environmentBuild, {
description:
"Build an agent environment bundle from a recorded capture in one offline step: inspect a HAR (or read capture Evidence), project the environment manifest, derive the state, report which tools the environment answers from state, which replay and why each write is withheld, then write the bundle environment-export writes",
args: z.object({
capture: localPathStringSchema.describe(
"A .har file, or capture Evidence JSON from inspect-web-network-capture (HAR) or capture-browser-scenario",
),
directory: localPathStringSchema
.optional()
.describe(
"Bundle directory to create; must be absent or empty, nothing is overwritten; omit only with --dry-run",
),
}),
options: z.object({
name: z
.string()
.optional()
.describe(
"Env folder and env id (kebab-case); default the manifest's name, or the capture file's name when the origin is a loopback or IP address",
),
templateRule: z
.array(z.enum(TEMPLATE_RULES))
.optional()
.describe(
"Opt-in template rule; repeat to select several. Replaces the default after-uid-literal, opaque-token, sibling-variance and prefix-variance",
),
formatRulesOnly: z
.boolean()
.default(false)
.describe("Apply only the four format rules, the views' own default"),
packageTarball: localPathStringSchema
.optional()
.describe(
"Packed rea-agents tarball (npm pack) copied into the env folder and installed by the Dockerfile instead of the registry package pinned to this version",
),
dryRun: z
.boolean()
.default(false)
.describe("Run every stage and report, but write nothing"),
captureTimeoutMs:
inspectWebNetworkCaptureInputSchema.shape.capture_timeout_ms,
}),
run: ({ args, options }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.environmentBuild, () =>
buildEnvironmentCommand(
{
capture: args.capture,
directory: args.directory,
name: options.name,
templateRule: options.templateRule,
formatRulesOnly: options.formatRulesOnly,
packageTarball: options.packageTarball,
dryRun: options.dryRun,
captureTimeoutMs: options.captureTimeoutMs,
},
environment,
logger,
signal,
),
),
),
});
};
const manifestArgument = z
.string()
.describe(
@@ -431,6 +673,7 @@ export const registerEnvironmentCommands = (
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
): void => {
registerBuildCommand(cli, logger, environment);
registerExportCommand(cli, logger);
registerCompareCommand(cli, logger);
cli.command(CLI_COMMANDS.environmentServe, {
src/cliCommandNames.ts
+2
View File
@@ -140,6 +140,7 @@ export const CLI_COMMANDS = Object.freeze({
environmentServe: "environment-serve",
environmentExport: "environment-export",
environmentCompare: "environment-compare",
environmentBuild: "environment-build",
compare: "compare",
captureProcess: "capture-process",
compareProcessCaptures: "compare-process-captures",
@@ -179,6 +180,7 @@ export const CLI_COMMAND_NAMES = Object.freeze(Object.values(CLI_COMMANDS));
/** Canonical MCP operations reached by CLI commands with different names. */
export const CLI_COMMAND_TOOL_ALIASES = Object.freeze({
"environment-build": ["export_environment_bundle"],
analyze: ["binary_overview"],
inspect: ["binary_overview"],
decompile: ["procedure_pseudo_code"],
src/contracts/analysisViewToolContracts.ts
+1 -1
View File
@@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
...toolContractMetadata("inspect_analysis_view"),
kind: "application",
description:
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views rLine truncated
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views rLine truncated
inputSchema: inspectAnalysisViewInputSchema,
outputSchema: evidenceResultOf(analysisViewResultSchema),
examples: [
src/contracts/browserScenarioToolContracts.ts
+1 -1
View File
@@ -26,7 +26,7 @@ export const BROWSER_SCENARIO_TOOL_CONTRACTS = [
name: "capture_browser_scenario",
...toolContractMetadata("capture_browser_scenario"),
description:
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. Environment, empty storage, a URL-only final capture, and measured event-retention budgets are defaults. capture.network selects request_body, response_body, and header_values independently; capture.limits lets callers raise or lower the event count, retained-allocation estimates, network transaction count, and network content budgets. Selected content is returned inline with transaction and event references, byte digests, and explicit unavailable states. Budget omissions identify affected event kinds and network transactions. Credentials and declared secrets are redacted. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Results are returned inline.",
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. A request action sends one HTTP request to the start origin through the page's own fetch, with header values from secrets, page cookies or local storage; write methods change application state. Environment, empty storage, a URL-only final capture, and measured event-retention budgets are defaults. capture.network selects request_body, response_body, and header_values independently; capture.limits lets callers raise or lower the event count, retained-allocation estimates, network transaction count, and network content budgets. Selected content is returned inline with transaction and event references, byte digests, and explicit unavailable states. Budget omissions identify affected event kinds and network transactions. Credentials and declared secrets are redacted. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Results are returned inline.",
kind: "browser-provider",
inputSchema: browserScenarioInputSchema,
outputSchema: evidenceResultOf(browserScenarioCaptureSchema),
src/contracts/environmentToolContracts.ts
+41
View File
@@ -0,0 +1,41 @@
import {
environmentBuildReportSchema,
exportEnvironmentBundleInputSchema,
} from "../domain/environmentBundleExport.js";
import type { ToolContract } from "./toolContractTypes.js";
import { toolContractMetadata } from "./toolEffects.js";
/** Agent environment bundle contracts shared with the environment-build CLI command. */
export const ENVIRONMENT_TOOL_CONTRACTS = [
{
name: "export_environment_bundle",
...toolContractMetadata("export_environment_bundle"),
description:
"Write an agent environment bundle folder from capture Evidence (capture_browser_scenario, or HAR inspect_web_network_capture Evidence) into an absent absolute output_directory whose parent exists, as rea environment-build does. It projects the environment manifest with the default template rules, derives the state the bundle serves, and reports read tools answered from the state or replayed with their reason, and write tools served or withheld with a hint about what to record next; check the same coverage first with inspect_analysis_view view environment-coverage. dry_run reports without writing. The bundle copies the capture Evidence byte for byte, recorded response bodies included, and its README names how the agentenv framework uses it. Reads that Evidence, and package_tarball when given, and writes only that folder; no network access. dry_run still checks output_directory. Destination refusals name this tool; the others name their stage as environment-build's do.",
kind: "application",
inputSchema: exportEnvironmentBundleInputSchema,
outputSchema: environmentBuildReportSchema,
examples: [
{
title: "Write the bundle for a retained browser scenario capture",
input: {
source: {
kind: "retained-evidence",
evidence_id:
"ev_0000000000000000000000000000000000000000000000000000000000000001",
},
output_directory: "/tmp/grafana-env",
name: "grafana",
},
},
{
title: "Check what a capture file's bundle would serve",
input: {
source: { kind: "evidence-file", path: "/tmp/grafana.capture.json" },
output_directory: "/tmp/grafana-env",
dry_run: true,
},
},
],
},
] as const satisfies readonly ToolContract[];
src/contracts/toolContracts.ts
+2
View File
@@ -41,6 +41,7 @@ import { REVERSE_ENGINEERING_TOOL_CONTRACTS } from "./reverseEngineeringToolCont
import { GDB_TOOL_CONTRACTS } from "./gdbToolContracts.js";
import { RIZIN_DEBUG_TOOL_CONTRACTS } from "./rizinDebugToolContracts.js";
import { FRIDA_TOOL_CONTRACTS } from "./fridaToolContracts.js";
import { ENVIRONMENT_TOOL_CONTRACTS } from "./environmentToolContracts.js";
/** Complete ordered public inventory used by registration and verification. */
export const TOOL_CONTRACTS = [
@@ -80,6 +81,7 @@ export const TOOL_CONTRACTS = [
...WEB_SCRIPT_TOOL_CONTRACTS,
...WEB_RUNTIME_TOOL_CONTRACTS,
...WEB_NETWORK_CAPTURE_TOOL_CONTRACTS,
...ENVIRONMENT_TOOL_CONTRACTS,
...JAVASCRIPT_RECOVERY_TOOL_CONTRACTS,
...SESSION_TOOL_CONTRACTS,
...CUTTER_TOOL_CONTRACTS,
src/contracts/toolEffects.ts
+4
View File
@@ -681,6 +681,10 @@ export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
writesFilesystem: true,
idempotent: false,
}),
export_environment_bundle: effects({
writesFilesystem: true,
idempotent: false,
}),
reconcile_javascript_runtime: evidence,
recover_javascript_sources: effects({
mutatesSession: true,
src/domain/analysisView/analysisView.ts
+29 -3
View File
@@ -235,6 +235,11 @@ import {
environmentStateViewRequestSchema,
projectEnvironmentStateView,
} from "./environmentStateView.js";
import {
environmentCoverageFields,
environmentCoverageViewRequestSchema,
projectEnvironmentCoverageView,
} from "./environmentCoverageView.js";
import {
pcapViewRequestSchema,
pcapViewPageSchema,
@@ -374,6 +379,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
apiSurfaceViewSchema,
environmentManifestViewRequestSchema,
environmentStateViewRequestSchema,
environmentCoverageViewRequestSchema,
captureViewRequestSchema,
protobufViewRequestSchema,
qlogViewRequestSchema,
@@ -966,6 +972,16 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
]),
view: environmentStateViewRequestSchema,
}),
z.strictObject({
...viewResultBase,
...environmentCoverageFields,
kind: z.literal("environment-coverage"),
parent_operation: z.enum([
"inspect_web_network_capture",
"capture_browser_scenario",
]),
view: environmentCoverageViewRequestSchema,
}),
z.discriminatedUnion("parent_operation", [
erlangBeamSummaryViewResultSchema,
pythonPycSummaryViewResultSchema,
@@ -1241,7 +1257,7 @@ const unsupportedParent = (
INSPECT_ANALYSIS_VIEW_OPERATION,
parentArtifactPath(parent),
parent.operation === "capture_browser_scenario"
? `Parent Evidence operation "capture_browser_scenario" supports only the environment-manifest and environment-state views.`
? `Parent Evidence operation "capture_browser_scenario" supports only the environment-manifest, environment-state and environment-coverage views.`
: `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed Evidence from a supported parent operation with a compatible view. For a summary receipt, use its normalized_result.parent_evidence_id rather than the receipt's own evidence_id.`,
);
@@ -1305,9 +1321,16 @@ export interface AnalysisViewAccumulators {
readonly state?: EnvironmentStateAccumulator | undefined;
}
/** Views that group a capture's records into tools. */
export type GroupingViewKind =
| "api-surface"
| "environment-manifest"
| "environment-state"
| "environment-coverage";
type GroupingViewRequest = Extract<
AnalysisViewRequest,
{ kind: "api-surface" | "environment-manifest" | "environment-state" }
{ kind: GroupingViewKind }
>;
const isGroupingView = (
@@ -1315,7 +1338,8 @@ const isGroupingView = (
): view is GroupingViewRequest =>
view.kind === "api-surface" ||
view.kind === "environment-manifest" ||
view.kind === "environment-state";
view.kind === "environment-state" ||
view.kind === "environment-coverage";
/** Views that group a capture's records, each under the budget its caller created. */
const projectGroupingView = (
@@ -1331,6 +1355,8 @@ const projectGroupingView = (
);
if (view.kind === "environment-manifest")
return projectEnvironmentManifestView(parent, view, accumulators.manifest);
if (view.kind === "environment-coverage")
return projectEnvironmentCoverageView(parent, view, accumulators.state);
return projectEnvironmentStateView(parent, view, accumulators.state);
};
src/domain/analysisView/binaryLayoutView.ts
+2 -4
View File
@@ -9,6 +9,7 @@ import type {
AnalysisViewParent,
AnalysisViewRequest,
UnsignedAnalysisView,
GroupingViewKind,
} from "./analysisView.js";
import {
boundedViewMessages,
@@ -157,10 +158,7 @@ const layoutSummary = (layout: BinaryLayout) => ({
export const projectBinaryLayoutView = (
parent: AnalysisViewParent,
layout: BinaryLayout,
view: Exclude<
AnalysisViewRequest,
{ kind: "api-surface" | "environment-manifest" | "environment-state" }
>,
view: Exclude<AnalysisViewRequest, { kind: GroupingViewKind }>,
): Result<UnsignedAnalysisView, AnalysisError> => {
const shared = parentFields(parent, layout);
if (view.kind === "native")
src/domain/analysisView/environmentCoverageView.test.ts
+221
View File
@@ -0,0 +1,221 @@
import { expect, it } from "vitest";
import {
analysisViewCaptureEvidence,
environmentStateCaptureFixture,
} from "../../../tests/fixtures/analysisView.js";
import {
SCENARIO_EXCHANGES,
scenarioCaptureEvidence,
scenarioCaptureFixture,
} from "../../../tests/fixtures/browserScenarioCapture.js";
import type { Evidence } from "../evidence.js";
import {
analysisViewRequestSchema,
projectAnalysisView,
type AnalysisViewParent,
type AnalysisViewResult,
} from "./analysisView.js";
import {
WITHHELD_WRITE_HINTS,
summarizeEnvironmentCoverage,
} from "./environmentCoverageView.js";
import type { StateView } from "./environmentStateRules.js";
import { environmentStateFields } from "./environmentStateView.js";
import type { StateWritePlan } from "./environmentStateWriteRules.js";
const RULES = [
"after-uid-literal",
"opaque-token",
"sibling-variance",
] as const;
const parentOf = (
evidence: Evidence = analysisViewCaptureEvidence(
environmentStateCaptureFixture(),
),
): AnalysisViewParent => ({
evidenceId: evidence.evidence_id,
operation: evidence.operation,
normalizedResult: evidence.normalized_result,
subject: evidence.subject,
limitations: [],
parameters: evidence.parameters,
});
const project = (
kind: "environment-state" | "environment-coverage",
parent: AnalysisViewParent = parentOf(),
): AnalysisViewResult => {
const projected = projectAnalysisView(
parent,
analysisViewRequestSchema.parse({ kind, template_rules: [...RULES] }),
);
if (!projected.ok) throw projected.error;
return projected.value;
};
const plan = (
tool: string,
reason: StateWritePlan["reason"],
extra: Partial<StateWritePlan> = {},
): StateWritePlan =>
({
tool,
effect: "create",
target: "/api/folders",
served: reason === null,
reason,
exemplar_ordinal: 4,
observation: null,
response: null,
body: null,
rules: [],
collection: null,
insert: null,
merge: null,
checked_record_ordinals: [],
contradicted_by: null,
...extra,
}) as StateWritePlan;
it("has a hint for every reason the state view reports for a withheld write", () => {
const reasons = environmentStateFields.writes.element.shape.reason
.unwrap()
.options.slice()
.sort();
expect(Object.keys(WITHHELD_WRITE_HINTS).sort()).toEqual(reasons);
for (const hint of Object.values(WITHHELD_WRITE_HINTS))
expect(hint.length).toBeGreaterThan(0);
});
it("counts read tools by how they answer and lists writes as served or withheld with hints", () => {
const views: StateView[] = [
{ tool: "get_folders", shape: "document", checked_record_ordinals: [] },
{ tool: "get_users", shape: "recorded", reason: "query-varies" },
{ tool: "get_teams", shape: "recorded", reason: "envelope-varies" },
{ tool: "get_orgs", shape: "recorded", reason: "query-varies" },
];
const writes = [
plan("post_folders", null),
plan("put_folders_by_uid", "contradicted", {
effect: "update",
contradicted_by: 9,
}),
plan("delete_folders_by_uid", "no-state-target", {
effect: "delete",
exemplar_ordinal: null,
}),
];
expect(summarizeEnvironmentCoverage({ views, writes })).toEqual({
reads: {
state: 1,
recorded: 3,
recorded_reasons: { "envelope-varies": 1, "query-varies": 2 },
},
writes: {
served: [{ tool: "post_folders", effect: "create" }],
withheld: [
{
tool: "put_folders_by_uid",
effect: "update",
reason: "contradicted",
hint: WITHHELD_WRITE_HINTS.contradicted,
exemplar_ordinal: 4,
contradicted_by: 9,
},
{
tool: "delete_folders_by_uid",
effect: "delete",
reason: "no-state-target",
hint: WITHHELD_WRITE_HINTS["no-state-target"],
exemplar_ordinal: null,
contradicted_by: null,
},
],
},
});
});
it("summarizes the state the environment-state view derives with the same rules", () => {
const state = project("environment-state");
const coverage = project("environment-coverage");
if (
state.kind !== "environment-state" ||
coverage.kind !== "environment-coverage"
)
throw new Error("Expected state and coverage views");
const summary = summarizeEnvironmentCoverage(state);
expect(coverage.reads).toEqual(summary.reads);
expect(coverage.writes).toEqual(summary.writes);
expect(coverage.reads.state + coverage.reads.recorded).toBe(
state.views.length,
);
expect(coverage.writes.served.length + coverage.writes.withheld.length).toBe(
state.writes.length,
);
expect(coverage).toMatchObject({
view: { kind: "environment-coverage", template_rules: [...RULES] },
parent_evidence_id: state.parent_evidence_id,
origin: state.origin,
name: state.name,
unknowns: state.unknowns,
coverage: state.coverage,
});
expect(coverage.limitations).toEqual(
expect.arrayContaining([
...state.limitations,
expect.stringContaining("does not run the environment"),
]),
);
});
it("says an untimed capture learns no clock, and only for an untimed capture", () => {
const untimed = /learns no clock/u;
const scenarioParent = parentOf(
scenarioCaptureEvidence(
scenarioCaptureFixture(SCENARIO_EXCHANGES, {
request_body: true,
response_body: true,
header_values: true,
}),
),
);
for (const [parent, timeline] of [
[parentOf(), "recorded-start-time"],
[scenarioParent, "record-order"],
] as const) {
const state = project("environment-state", parent);
const coverage = project("environment-coverage", parent);
if (
state.kind !== "environment-state" ||
coverage.kind !== "environment-coverage"
)
throw new Error("Expected state and coverage views");
expect(state.timeline).toBe(timeline);
expect(coverage.limitations.some((line) => untimed.test(line))).toBe(
timeline === "record-order",
);
}
});
it("refuses a browser scenario without response bodies as the state view does", () => {
const evidence = scenarioCaptureEvidence(
scenarioCaptureFixture(SCENARIO_EXCHANGES, {
request_body: true,
response_body: false,
header_values: true,
}),
);
const view = (kind: "environment-state" | "environment-coverage") =>
projectAnalysisView(
parentOf(evidence),
analysisViewRequestSchema.parse({ kind, template_rules: [] }),
);
const state = view("environment-state");
const coverage = view("environment-coverage");
if (state.ok || coverage.ok) throw new Error("Expected both refusals");
expect(coverage.error).toMatchObject({
_tag: "AnalysisUnsupportedTargetError",
message: state.error.message,
});
});
src/domain/analysisView/environmentCoverageView.ts
+186
View File
@@ -0,0 +1,186 @@
import { z } from "zod";
import type { AnalysisError } from "../analysisErrorBase.js";
import { ok, type Result } from "../result.js";
import type {
AnalysisViewParent,
AnalysisViewRequest,
UnsignedAnalysisView,
} from "./analysisView.js";
import { EnvironmentStateAccumulator } from "./EnvironmentStateAccumulator.js";
import { templateRulesSchema } from "./environmentManifestView.js";
import type { StateView } from "./environmentStateRules.js";
import {
environmentStateFields,
projectEnvironmentStateView,
} from "./environmentStateView.js";
import type {
StateWritePlan,
WriteEffect,
WriteReason,
} from "./environmentStateWriteRules.js";
export const environmentCoverageViewRequestSchema = z.strictObject({
kind: z.literal("environment-coverage"),
template_rules: templateRulesSchema.describe(
"Opt-in template rules as for environment-manifest, each listed at most once. Use the rules of the manifest the environment serves so tool names match.",
),
});
/** What to record next for each reason a write tool is withheld; advice only, it never changes what is served. */
export const WITHHELD_WRITE_HINTS = {
"no-state-target":
"The write's path names no collection or item the capture reads. Read the collection or item it changes.",
"no-call":
"No recorded call of this tool was modelled, for example because every call failed. Record a successful call.",
"body-unusable": "The request body is not a usable JSON object.",
"no-observation":
"Nothing showed the entity or document this write changed. Read a created entity back by id right after the write; for an update or delete the state must already hold the entity, so read it before the write or record the create that made it.",
"unexplained-change":
"The write changed fields it did not send, in a way no rule explains. A second, different call of the same tool can show the pattern.",
contradicted:
"A later read disagreed with the modelled effect. Check the read at contradicted_by.",
"side-effect":
"The write changed something besides its own entity, which the environment does not model.",
unconfirmed:
"No later read showed the effect. Read the entity and its list after the write.",
"incomplete-entities":
"The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it.",
} as const satisfies Record<WriteReason, string>;
const writeReasons = Object.keys(WITHHELD_WRITE_HINTS) as [
WriteReason,
...WriteReason[],
];
const effectSchema = z.enum(["create", "update", "delete", "document", "none"]);
const ordinalSchema = z.number().int().nonnegative();
/** Which tools an environment answers from its state, which replay their recordings, and why each write is withheld. */
export const environmentCoverageSchema = z.strictObject({
reads: z.strictObject({
state: z
.number()
.int()
.nonnegative()
.describe("Read tools answered from the state."),
recorded: z
.number()
.int()
.nonnegative()
.describe("Read tools that replay their recordings."),
recorded_reasons: z
.record(z.string(), z.number().int().positive())
.describe(
"Replayed read tools per reason, as the environment-state view names the reasons.",
),
}),
writes: z.strictObject({
served: z.array(z.strictObject({ tool: z.string(), effect: effectSchema })),
withheld: z.array(
z.strictObject({
tool: z.string(),
effect: effectSchema,
reason: z.enum(writeReasons),
hint: z
.string()
.describe(
"What to record next; advice that never changes what is served.",
),
exemplar_ordinal: ordinalSchema
.nullable()
.describe(
"The first successful recorded call the plan was built from.",
),
contradicted_by: ordinalSchema
.nullable()
.describe("The read that disagreed with the modelled effect."),
}),
),
}),
});
export type EnvironmentCoverage = z.infer<typeof environmentCoverageSchema>;
export const environmentCoverageFields = {
origin: environmentStateFields.origin,
name: environmentStateFields.name,
...environmentCoverageSchema.shape,
unknowns: environmentStateFields.unknowns,
};
/** Summarize a state's read views and write plans as served, replayed and withheld tools. */
export const summarizeEnvironmentCoverage = (state: {
readonly views: Iterable<StateView>;
readonly writes: Iterable<StateWritePlan>;
}): EnvironmentCoverage => {
let answered = 0;
let recorded = 0;
const reasons: Record<string, number> = {};
for (const view of state.views)
if (view.shape === "recorded") {
recorded++;
reasons[view.reason] = (reasons[view.reason] ?? 0) + 1;
} else answered++;
const served: { tool: string; effect: WriteEffect }[] = [];
const withheld: EnvironmentCoverage["writes"]["withheld"] = [];
for (const plan of state.writes)
if (plan.served || plan.reason === null)
served.push({ tool: plan.tool, effect: plan.effect });
else
withheld.push({
tool: plan.tool,
effect: plan.effect,
reason: plan.reason,
hint: WITHHELD_WRITE_HINTS[plan.reason],
exemplar_ordinal: plan.exemplar_ordinal,
contradicted_by: plan.contradicted_by,
});
return {
reads: {
state: answered,
recorded,
recorded_reasons: Object.fromEntries(
Object.entries(reasons).sort(([left], [right]) =>
left < right ? -1 : left > right ? 1 : 0,
),
),
},
writes: { served, withheld },
};
};
/** Project the state an environment would serve and summarize which tools it answers from that state. */
export const projectEnvironmentCoverageView = (
parent: AnalysisViewParent,
view: Extract<AnalysisViewRequest, { kind: "environment-coverage" }>,
accumulator = new EnvironmentStateAccumulator(undefined, view.template_rules),
): Result<UnsignedAnalysisView, AnalysisError> => {
const projected = projectEnvironmentStateView(
parent,
{ kind: "environment-state", template_rules: view.template_rules },
accumulator,
);
if (!projected.ok) return projected;
const state = projected.value;
return ok({
kind: "environment-coverage",
view,
parent_evidence_id: state.parent_evidence_id,
parent_operation: state.parent_operation,
parent_digest: state.parent_digest,
artifact: state.artifact,
coverage: state.coverage,
limitations: [
...state.limitations,
"Coverage summarizes the environment-state view an environment derives at startup; it does not run the environment. An environment whose served catalog exceeds the discovery budget, or whose state exceeds what data/add loads, serves replay instead, which this view does not check. Hints are advice about what to record next.",
...(state.timeline === "record-order"
? [
"Not every record carries a start time (browser scenario captures carry none), so the state learns no clock: a write whose response changes a timestamp the server sets, such as a modification time, is withheld as unexplained-change however often it is recorded.",
]
: []),
],
unknowns: state.unknowns,
origin: state.origin,
name: state.name,
...summarizeEnvironmentCoverage(state),
});
};
src/domain/analysisView/environmentStateView.ts
+4 -1
View File
@@ -466,7 +466,10 @@ export const projectEnvironmentStateView = (
parent: AnalysisViewParent,
view: Extract<AnalysisViewRequest, { kind: "environment-state" }>,
accumulator = new EnvironmentStateAccumulator(undefined, view.template_rules),
): Result<UnsignedAnalysisView, AnalysisError> => {
): Result<
Extract<UnsignedAnalysisView, { kind: "environment-state" }>,
AnalysisError
> => {
const grouped = groupCaptureRecords(parent, view.kind, view.template_rules, {
accumulator,
noun: "state",
src/domain/analysisView/javascriptApplicationView.ts
+2 -4
View File
@@ -23,6 +23,7 @@ import type {
AnalysisViewParent,
AnalysisViewRequest,
UnsignedAnalysisView,
GroupingViewKind,
} from "./analysisView.js";
const OPERATION = "inspect_analysis_view";
@@ -340,10 +341,7 @@ const selectModule = (
export const projectJavaScriptApplicationView = (
parent: AnalysisViewParent,
analysis: JavaScriptApplicationAnalysisResult,
view: Exclude<
AnalysisViewRequest,
{ kind: "api-surface" | "environment-manifest" | "environment-state" }
>,
view: Exclude<AnalysisViewRequest, { kind: GroupingViewKind }>,
): Result<UnsignedAnalysisView, AnalysisError> => {
if (
view.kind !== "summary" &&
src/domain/analysisView/recordedResponseBody.ts
+22 -16
View File
@@ -28,6 +28,27 @@ const strictUtf8 = (): TextDecoder =>
export const base64PrefixChars = (limitBytes: number): number =>
Math.ceil((limitBytes + 4) / 3) * 4;
/** Decode retained base64 as strict UTF-8, only as far as `limitBytes`; bytes that are not UTF-8 stay base64. */
export const decodeRetainedBase64 = (
encoded: string,
limitBytes: number,
): RecordedResponseBody => {
const prefixChars = base64PrefixChars(limitBytes);
const cut = encoded.length > prefixChars;
const prefix = cut ? encoded.slice(0, prefixChars) : encoded;
const flags = cut ? { bodyCut: true as const } : {};
try {
// A cut prefix may end inside a character; streaming leaves that tail undecoded.
const text = strictUtf8().decode(Buffer.from(prefix, "base64"), {
stream: cut,
});
return { bodyText: text, ...flags };
} catch {
// Binary or non-UTF-8 bytes: keep them exactly, as base64, instead of inventing replacement characters.
return { bodyText: prefix, bodyEncoding: "base64", ...flags };
}
};
/** Read producer-decoded text or supported HAR base64; unknown encoding declarations stay opaque. Base64 beyond `limitBytes` is decoded only that far. */
export const readRecordedResponseBody = (
record: WebNetworkCaptureRecord,
@@ -53,22 +74,7 @@ export const readRecordedResponseBody = (
);
if (binary === undefined || binary.content_base64 === null)
return { bodyText: null };
const prefixChars = base64PrefixChars(limitBytes);
const cut = binary.content_base64.length > prefixChars;
const encoded = cut
? binary.content_base64.slice(0, prefixChars)
: binary.content_base64;
const flags = cut ? { bodyCut: true as const } : {};
try {
// A cut prefix may end inside a character; streaming leaves that tail undecoded.
const text = strictUtf8().decode(Buffer.from(encoded, "base64"), {
stream: cut,
});
return { bodyText: text, ...flags };
} catch {
// Binary or non-UTF-8 bytes: keep them exactly, as base64, instead of inventing replacement characters.
return { bodyText: encoded, bodyEncoding: "base64", ...flags };
}
return decodeRetainedBase64(binary.content_base64, limitBytes);
};
/** Prove decoded emptiness with a bounded prefix, preserving encoding and cut uncertainty. */
src/domain/analysisView/scenarioCaptureRecords.ts
+6 -3
View File
@@ -81,7 +81,10 @@ const utf8Text = (body: CapturedBody): string | undefined => {
}
};
const mediaType = (content: ContentEvent | undefined): string | undefined =>
/** A content event's media type: its captured body's, else its retained Content-Type value. */
export const scenarioContentMediaType = (
content: ContentEvent | undefined,
): string | undefined =>
captured(content)?.media_type ??
headerValue(content?.headers, "content-type");
@@ -93,7 +96,7 @@ const reportedRequest = (
): { readonly [key: string]: JsonValue } => {
const body = purpose === "manifest" ? captured(content) : undefined;
const text = body === undefined ? undefined : utf8Text(body);
const mimeType = mediaType(content);
const mimeType = scenarioContentMediaType(content);
return {
method: request.method,
bodySize: body !== undefined && !body.redacted ? body.bytes : -1,
@@ -142,7 +145,7 @@ const record = (
const requestContent = onlyContent(index, request.sequence);
const responseContent = onlyContent(index, response?.sequence);
const body = retainedBody(responseContent);
const mimeType = mediaType(responseContent) ?? "";
const mimeType = scenarioContentMediaType(responseContent) ?? "";
return {
record: {
ordinal,
src/domain/analysisView/webNetworkCaptureView.ts
+1 -1
View File
@@ -458,7 +458,7 @@ export const projectWebNetworkCaptureView = (
path: ["view"],
reason: "invalid_value",
message:
"Historical captures support summary, capture record pages, qlog fact pages and admitted api-surface, environment-manifest or environment-state views.",
"Historical captures support summary, capture record pages, qlog fact pages and admitted api-surface, environment-manifest, environment-state or environment-coverage views.",
},
]),
);
src/domain/browserScenario.ts
+62
View File
@@ -10,6 +10,7 @@ import {
browserScenarioSecretSchema,
browserScenarioStorageSchema,
browserScenarioUrlSchema,
HEADER_FIELD_VALUE,
type BrowserScenarioAction,
type BrowserScenarioUrl,
type BrowserScenarioValue,
@@ -72,9 +73,69 @@ const actionSecretReferences = (
return secretReferencesInValue(action.value);
if (action.action === "goto")
return destinationSecretReferences(action.destination);
if (action.action === "request")
return [
...action.query.flatMap(({ value }) => secretReferencesInValue(value)),
...action.headers.flatMap(({ value }) =>
value.source === "secret" ? [value.secret_id] : [],
),
];
return [];
};
/** Request steps stay on the start origin and send a body only with methods that carry one. */
const validateRequestSteps = (
scenario: ScenarioShape,
context: z.RefinementCtx,
): void => {
const origin = new URL(scenario.start_url.url).origin;
scenario.actions.forEach((action, index) => {
if (action.action !== "request") return;
const resolved = URL.parse(action.path, origin);
if (resolved === null)
addIssue(
context,
["actions", index, "path"],
"The request step path is not a URL path the browser can resolve",
);
else if (resolved.origin !== origin)
addIssue(
context,
["actions", index, "path"],
`Request steps must stay on the start URL's origin ${origin}`,
);
if (
action.json !== undefined &&
(action.method === "GET" || action.method === "HEAD")
)
addIssue(
context,
["actions", index, "json"],
`A ${action.method} request step cannot send a body`,
);
const names = new Set<string>();
action.headers.forEach((header, position) => {
const name = header.name.toLowerCase();
if (names.has(name))
addIssue(
context,
["actions", index, "headers", position, "name"],
`Request step headers name ${header.name} more than once`,
);
names.add(name);
if (
header.value.source === "literal" &&
!HEADER_FIELD_VALUE.test(header.value.value)
)
addIssue(
context,
["actions", index, "headers", position, "value", "value"],
"A header value cannot carry line breaks or control characters",
);
});
});
};
const validateStorage = (scenario: ScenarioShape): readonly string[] => {
const references: string[] = [];
scenario.storage.cookies.forEach((cookie) => {
@@ -136,6 +197,7 @@ const validateBrowserScenario = (
context,
);
validateRequestSteps(scenario, context);
const references = scenario.actions.flatMap(actionSecretReferences);
references.push(...destinationSecretReferences(scenario.start_url));
references.push(...validateStorage(scenario));
src/domain/browserScenarioValues.ts
+126
View File
@@ -1,6 +1,7 @@
import { z } from "zod";
import { IDENTIFIER_PATTERN } from "./stringPatterns.js";
import { jsonValueSchema } from "./jsonValue.js";
import { absoluteLocalPathStringSchema } from "./localPath.js";
import { browserNetworkContentSelectionSchema } from "./browserNetworkEvidence.js";
@@ -67,6 +68,97 @@ const queryEntrySchema = z.strictObject({
value: browserScenarioValueSchema,
});
/** HTTP methods a request step may send. */
export const BROWSER_SCENARIO_REQUEST_METHODS = [
"GET",
"HEAD",
"POST",
"PUT",
"PATCH",
"DELETE",
] as const;
/** Header names a page's fetch cannot set (and User-Agent, which Chrome drops), so a request step refuses them rather than having them dropped. */
const FORBIDDEN_REQUEST_HEADERS = new Set([
"accept-charset",
"accept-encoding",
"access-control-request-headers",
"access-control-request-method",
"connection",
"content-length",
"cookie",
"cookie2",
"date",
"dnt",
"expect",
"host",
"keep-alive",
"origin",
"referer",
"set-cookie",
"te",
"trailer",
"transfer-encoding",
"upgrade",
"user-agent",
"via",
]);
/** Characters an HTTP field value may carry: visible ASCII, space, tab and obs-text. */
export const HEADER_FIELD_VALUE = /^[\t\x20-\x7e\x80-\xff]*$/u;
const settableRequestHeader = (name: string): boolean => {
const lower = name.toLowerCase();
return (
!FORBIDDEN_REQUEST_HEADERS.has(lower) &&
!lower.startsWith("proxy-") &&
!lower.startsWith("sec-")
);
};
/** A header value a request step reads from the page as it runs; it is redacted like a declared secret. */
const browserScenarioPageValueSchemas = [
z.strictObject({
source: z.literal("cookie"),
name: z
.string()
.min(1)
.describe("Cookie the browser holds for the request URL."),
}),
z.strictObject({
source: z.literal("local_storage"),
key: z.string().min(1).describe("Local storage key on the start origin."),
json_pointer: z
.string()
.regex(/^(?:\/.*)?$/u)
.optional()
.describe(
"Parse the stored value as JSON and use the string or number at this pointer; empty selects the whole value.",
),
}),
] as const;
const requestHeaderSchema = z.strictObject({
name: z
.string()
.regex(/^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/u)
.refine(settableRequestHeader, {
message: "A page's fetch cannot set this header",
}),
value: z.discriminatedUnion("source", [
...browserScenarioValueSchema.options,
...browserScenarioPageValueSchemas,
]),
prefix: z
.string()
.regex(HEADER_FIELD_VALUE, {
message: "A header value cannot carry line breaks or control characters",
})
.optional()
.describe("Text sent before the value, such as `Bearer `."),
});
export type BrowserScenarioRequestHeader = z.infer<typeof requestHeaderSchema>;
/** URL with ordinary inline details and optional structured secret query references. */
export const browserScenarioUrlSchema = z.strictObject({
url: browserScenarioBaseUrlSchema,
@@ -221,6 +313,40 @@ export const browserScenarioActionSchema = z.discriminatedUnion("action", [
locator: locatorSchema,
state: z.enum(["attached", "detached", "visible", "hidden"]),
}),
z.strictObject({
...stepBase,
action: z.literal("request"),
method: z.enum(BROWSER_SCENARIO_REQUEST_METHODS),
path: z
.string()
.regex(/^\/(?![/\\])/u)
.describe(
"Path, and optional literal query, on the start URL's origin, such as /api/folders; it must resolve to that origin.",
),
query: z.array(queryEntrySchema).default([]),
headers: z.array(requestHeaderSchema).default([]),
json: jsonValueSchema
.optional()
.describe(
"JSON request body, sent with Content-Type application/json unless a header sets one; not allowed for GET or HEAD.",
),
expected_status: z
.array(
z
.int()
.min(100)
.max(599)
.refine((status) => status < 300 || status > 399, {
message:
"Request steps follow no redirects, so a 3xx status cannot complete one",
}),
)
.min(1)
.optional()
.describe(
"Statuses that complete the step; any other status fails it. Omitted means any response completes it.",
),
}),
z.strictObject({
step_id: scenarioIdentifierSchema,
action: z.literal("wait_for_timeout"),
src/domain/environmentBundleExport.ts
+121
View File
@@ -0,0 +1,121 @@
import { z } from "zod";
import { environmentCoverageSchema } from "./analysisView/environmentCoverageView.js";
import { TEMPLATE_RULES } from "./analysisView/environmentManifestRules.js";
import { prefixedDigestSchema } from "./digests.js";
import { retainedEvidenceReferenceSchema } from "./evidenceReference.js";
import { absoluteLocalPathStringSchema } from "./localPath.js";
/** Capture Evidence retained in this session, or one capture Evidence file on this host. */
const environmentCaptureSourceSchema = z.discriminatedUnion("kind", [
retainedEvidenceReferenceSchema.describe(
"Capture Evidence this session retained, such as a capture_browser_scenario or inspect_web_network_capture result.",
),
z.strictObject({
kind: z.literal("evidence-file"),
path: absoluteLocalPathStringSchema.describe(
"Absolute path of one capture Evidence JSON file, copied byte for byte into the bundle.",
),
}),
]);
export const exportEnvironmentBundleInputSchema = z.strictObject({
source: environmentCaptureSourceSchema.describe(
"The capture the environment is built from: capture_browser_scenario Evidence, or HAR inspect_web_network_capture Evidence.",
),
output_directory: absoluteLocalPathStringSchema.describe(
"Absolute, absent output directory for the bundle folder; parent must exist.",
),
name: z
.string()
.min(1)
.optional()
.describe(
"Env folder and env id (kebab-case). Default: the manifest's name; required when the capture's origin is a loopback or IP address.",
),
template_rules: z
.array(z.enum(TEMPLATE_RULES))
.optional()
.describe(
"Opt-in template rules, each at most once; [] applies only the format rules. Default: after-uid-literal, opaque-token, sibling-variance and prefix-variance.",
),
package_tarball: absoluteLocalPathStringSchema
.optional()
.describe(
"Absolute path of a packed rea-agents tarball the bundle installs instead of the registry package of this version.",
),
dry_run: z
.boolean()
.default(false)
.describe("Report what the bundle would hold without writing anything."),
});
export type ExportEnvironmentBundleInput = z.infer<
typeof exportEnvironmentBundleInputSchema
>;
/** What a built or planned bundle serves and holds; environment-build prints the same report. */
export const environmentBuildReportSchema = z.strictObject({
status: z.enum(["built", "planned"]),
directory: z.string().nullable(),
name: z.string(),
capture: z.strictObject({
evidence_id: prefixedDigestSchema("ev"),
operation: z.string(),
records: z.int().nonnegative(),
input: z.enum(["har", "evidence"]),
}),
manifest: z.strictObject({
evidence_id: prefixedDigestSchema("ev"),
tools: z.int().nonnegative(),
excluded_groups: z.int().nonnegative(),
template_rules: z.array(z.enum(TEMPLATE_RULES)),
}),
state: z
.union([
z.strictObject({
served: z.literal(true),
evidence_id: z.string(),
}),
z.strictObject({
served: z.literal(false),
reason: z.literal("unavailable"),
code: z.string(),
message: z.string(),
}),
])
.describe(
"Whether the bundle serves from the derived state, or replays and why.",
),
coverage: environmentCoverageSchema
.nullable()
.describe(
"Read tools answered from the state or replayed, and write tools served or withheld; null when the bundle serves replay.",
),
tool_count: z.int().nonnegative(),
package_source: z.union([
z.strictObject({ kind: z.literal("registry"), specifier: z.string() }),
z.strictObject({ kind: z.literal("tarball"), path: z.string() }),
]),
files: z.array(
z.strictObject({ path: z.string(), bytes: z.int().nonnegative() }),
),
warnings: z.array(
z.strictObject({
code: z.enum([
"name-from-file",
"credential-named-members",
"credential-scan-incomplete",
]),
message: z.string(),
pointers: z
.array(z.string())
.describe(
"Where the warning applies, such as record pointers; never values.",
),
}),
),
next: z.array(z.string()),
});
export type EnvironmentBuildReport = z.infer<
typeof environmentBuildReportSchema
>;
src/generatedPackageMetadata.ts
+1 -1
View File
@@ -6,5 +6,5 @@ export const PACKAGE_METADATA = {
serverSdkVersion: "2.3.1",
clientSdkVersion: "2.3.1",
coreSdkVersion: "2.3.1",
skillVersion: "46",
skillVersion: "48",
} as const;
src/server/registerApplicationTools.ts
+2
View File
@@ -6,6 +6,7 @@ import { registerCompareApplicationVersionsTool } from "./registerApplicationToo
import { registerCompareSourceToBundleTool } from "./registerApplicationTools/compareSourceToBundle.js";
import { registerCompareJavaScriptExportShapesTool } from "./registerApplicationTools/compareExportShapes.js";
import { registerCoverageTools } from "./registerApplicationTools/coverage.js";
import { registerEnvironmentBundleTool } from "./registerApplicationTools/environmentBundle.js";
import { registerReconstructionObligationLedgerTool } from "./registerApplicationTools/obligationLedger.js";
import { registerProjectMobileApplicationGraphTools } from "./registerApplicationTools/projectMobileApplicationGraph.js";
import type { ApplicationToolRegistration } from "./registerApplicationTools/types.js";
@@ -22,5 +23,6 @@ export const registerApplicationTools = (
registerCompareJavaScriptExportShapesTool(server, options);
registerReconstructionObligationLedgerTool(server, options);
registerCoverageTools(server, options);
registerEnvironmentBundleTool(server, options);
registerProjectMobileApplicationGraphTools(server, options);
};
src/server/registerApplicationTools/environmentBundle.ts
+54
View File
@@ -0,0 +1,54 @@
import type { EvidenceMcpServer } from "../EvidenceMcpServer.js";
import { runAdmittedToolOperation } from "../admittedToolOperation.js";
import { exportEnvironmentBundleTool } from "../../application/environment/environmentBundleTool.js";
import { toolContract } from "../../contracts/toolContracts.js";
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
import { err } from "../../domain/result.js";
import { logToolExecution } from "../toolLogging.js";
import { applicationToolRegistrationOptions } from "../toolRegistrationOptions.js";
import type { ApplicationToolRegistration } from "./types.js";
const contract = toolContract("export_environment_bundle");
/** Register the agent environment bundle export; it validates its own input so refusals name the field. */
export const registerEnvironmentBundleTool = (
server: EvidenceMcpServer,
options: ApplicationToolRegistration,
): void => {
server.registerTool(
contract.name,
applicationToolRegistrationOptions(contract),
async (rawInput, context) =>
runAdmittedToolOperation(
server,
options.withAdmittedAnalysis,
contract.name,
context.mcpReq.signal,
async () => {
const result = await logToolExecution(
options.logger,
contract.name,
async () => {
const parsed = contract.inputSchema.safeParse(rawInput);
if (!parsed.success)
return err(
analysisInputErrorFromIssues(
contract.name,
parsed.error.issues,
rawInput,
{ cause: parsed.error },
),
);
return exportEnvironmentBundleTool(
parsed.data,
options.evidenceById,
{ signal: context.mcpReq.signal },
);
},
);
return server.delivery.toCallToolResult(result, contract);
},
),
);
};
tests/boundary/cli/environmentBuildCli.test.ts
+191
View File
@@ -0,0 +1,191 @@
import { mkdir, readdir, readFile, writeFile } from "node:fs/promises";
import { join, sep } from "node:path";
import { expect } from "vitest";
import { environmentManifestHar } from "../../fixtures/historicalHar.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
const RULES = [
"after-uid-literal",
"opaque-token",
"sibling-variance",
"prefix-variance",
];
const listFiles = async (root: string): Promise<string[]> =>
(await readdir(root, { recursive: true, withFileTypes: true }))
.filter((entry) => entry.isFile())
.map((entry) =>
join(entry.parentPath, entry.name)
.slice(root.length + 1)
.split(sep)
.join("/"),
)
.sort();
cliTest(
"builds the bundle the three CLI steps build, byte for byte",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-environment-build-cli-");
const har = join(root, "grafana.har");
await writeFile(har, JSON.stringify(environmentManifestHar()));
const built = await cli.run({
arguments: ["environment-build", har, join(root, "built"), "--json"],
timeoutMs: 60_000,
});
expect(built.exitCode, built.stderr).toBe(0);
expect(built.json).toMatchObject({
status: "built",
name: "grafana-example",
capture: { input: "har", operation: "inspect_web_network_capture" },
manifest: { template_rules: RULES },
state: { served: true },
warnings: [],
});
const inspected = await cli.run({
arguments: [
"inspect-web-network-capture",
har,
"har",
"--format",
"json",
],
});
expect(inspected.exitCode, inspected.stderr).toBe(0);
const capture = join(root, "capture.json");
await writeFile(capture, inspected.stdout);
const request = join(root, "view.json");
await writeFile(
request,
JSON.stringify({
source: { kind: "evidence-file", path: capture },
view: { kind: "environment-manifest", template_rules: RULES },
}),
);
const projected = await cli.run({
arguments: ["inspect-analysis-view", request, "--format", "json"],
});
expect(projected.exitCode, projected.stderr).toBe(0);
const manifest = join(root, "manifest.json");
await writeFile(manifest, projected.stdout);
const exported = await cli.run({
arguments: [
"environment-export",
manifest,
"--evidence",
capture,
join(root, "three-step"),
"--json",
],
timeoutMs: 60_000,
});
expect(exported.exitCode, exported.stderr).toBe(0);
const files = await listFiles(join(root, "built"));
expect(files).toEqual(await listFiles(join(root, "three-step")));
expect(files).toHaveLength(7);
for (const file of files)
expect(await readFile(join(root, "built", file)), file).toEqual(
await readFile(join(root, "three-step", file)),
);
expect(built.json).toMatchObject({
tool_count: (exported.json as { tool_count: number }).tool_count,
files: (exported.json as { files: unknown[] }).files,
});
const coverageRequest = join(root, "coverage.json");
await writeFile(
coverageRequest,
JSON.stringify({
source: { kind: "evidence-file", path: capture },
view: { kind: "environment-coverage", template_rules: RULES },
}),
);
const coverage = await cli.run({
arguments: ["inspect-analysis-view", coverageRequest, "--format", "json"],
});
expect(coverage.exitCode, coverage.stderr).toBe(0);
const view = (
coverage.json as {
normalized_result: { kind: string; reads: unknown; writes: unknown };
}
).normalized_result;
const report = (
built.json as { coverage: { reads: unknown; writes: unknown } }
).coverage;
expect(view.kind).toBe("environment-coverage");
expect(view.reads).toEqual(report.reads);
expect(view.writes).toEqual(report.writes);
},
);
cliTest(
"plans without writing on a dry run and refuses unusable arguments",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-environment-build-plan-");
const har = join(root, "grafana.har");
await writeFile(har, JSON.stringify(environmentManifestHar()));
const planned = await cli.run({
arguments: ["environment-build", har, "--dry-run", "--json"],
timeoutMs: 60_000,
});
expect(planned.exitCode, planned.stderr).toBe(0);
expect(planned.json).toMatchObject({
status: "planned",
directory: null,
next: [],
});
expect(await readdir(root)).toEqual(["grafana.har"]);
const other = join(root, "grafana.saz");
await writeFile(other, "not a capture");
const unsupported = await cli.run({
arguments: ["environment-build", other, join(root, "out"), "--json"],
});
expect(unsupported.exitCode).toBe(1);
expect(unsupported.json).toMatchObject({
code: "invalid_request",
details: { operation: "environment-build" },
});
const conflicting = await cli.run({
arguments: [
"environment-build",
har,
join(root, "out"),
"--format-rules-only",
"--template-rule",
"opaque-token",
"--json",
],
});
expect(conflicting.exitCode).toBe(1);
expect(conflicting.json).toMatchObject({ code: "invalid_request" });
const noDirectory = await cli.run({
arguments: ["environment-build", har, "--json"],
timeoutMs: 60_000,
});
expect(noDirectory.exitCode).toBe(1);
expect(noDirectory.json).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["directory"] }] },
});
const emptyWorkingDirectory = join(root, "empty-cwd");
await mkdir(emptyWorkingDirectory);
const emptyDirectory = await cli.run({
arguments: ["environment-build", har, "", "--json"],
cwd: emptyWorkingDirectory,
});
expect(emptyDirectory.exitCode).toBe(1);
expect(await readdir(emptyWorkingDirectory)).toEqual([]);
expect((await readdir(root)).sort()).toEqual([
"empty-cwd",
"grafana.har",
"grafana.saz",
]);
},
);
tests/boundary/filesystem/setupSkill.test.ts
+1
View File
@@ -638,6 +638,7 @@ describe("canonical skill transaction", () => {
"SKILL.md",
"references/android-applications.md",
"references/connection-and-recovery.md",
"references/environments.md",
"references/evidence-workflows.md",
"references/javascript-applications.md",
"references/native-and-artifacts.md",
tests/boundary/filesystem/setupSkillPublication.test.ts
+7 -2
View File
@@ -18,6 +18,7 @@ import { SkillFilePublication } from "../../../src/application/SkillFilePublicat
import {
canonicalSkillNeedsInstall,
installCanonicalSkill,
MANAGED_SKILL_FILES,
skillDestinations,
type SkillInstallFileOperations,
} from "../../../src/application/SetupSkill.js";
@@ -164,7 +165,9 @@ describe("skill publication recovery", () => {
reason.includes("First-install skill recovery"),
),
).toBe(true);
expect(result.error.rollbackFailures).toHaveLength(6);
expect(result.error.rollbackFailures).toHaveLength(
MANAGED_SKILL_FILES.length - 1,
);
},
);
@@ -185,7 +188,9 @@ describe("skill publication recovery", () => {
},
});
if (result.ok) throw new Error("Expected readback failure");
expect(result.error.rollbackFailures).toHaveLength(7);
expect(result.error.rollbackFailures).toHaveLength(
MANAGED_SKILL_FILES.length,
);
expect(
result.error.rollbackFailures.every(({ reason }) =>
reason.includes("manual removal"),
tests/boundary/mcp/analysisViewMcp.test.ts
+45
View File
@@ -1428,6 +1428,51 @@ it("extracts an environment state identically from retained, portable and file M
await client.ping();
});
it("reports environment coverage identically from retained, portable and file MCP Evidence", async () => {
const { client, session } = await connect();
const { ajv, inputSchema, outputSchema } = await advertisedMcpToolSchemas(
client,
"inspect_analysis_view",
);
const parent = analysisViewCaptureEvidence(environmentStateCaptureFixture());
expect(session.recordEvidence(parent).ok).toBe(true);
const evidencePath = join(
await createTestTempDirectory("rea-coverage-file-mcp-"),
"capture.json",
);
await writeFile(evidencePath, JSON.stringify(parent));
const rules = ["after-uid-literal", "opaque-token", "sibling-variance"];
const outputs = [];
for (const source of [
{ kind: "inline", evidence: parent },
{ kind: "retained-evidence", evidence_id: parent.evidence_id },
{ kind: "evidence-file", path: evidencePath },
]) {
const view = { kind: "environment-coverage", template_rules: rules };
expect(ajv.validate(inputSchema, { source, view })).toBe(true);
const response = await client.callTool({
name: "inspect_analysis_view",
arguments: { source, view },
});
expect(response.isError).not.toBe(true);
expect(
ajv.validate(outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
).toBe(true);
const evidence = parseEvidence(response.structuredContent);
expect(evidence.confidence).toBe("inferred");
expect(evidence.normalized_result).toMatchObject({
kind: "environment-coverage",
reads: { state: expect.any(Number) },
writes: { served: expect.any(Array), withheld: expect.any(Array) },
});
outputs.push(evidence.normalized_result);
}
expect(outputs[1]).toEqual(outputs[0]);
expect(outputs[2]).toEqual(outputs[0]);
await client.ping();
});
it("projects an environment manifest identically from retained, portable and file MCP Evidence", async () => {
const { client, session } = await connect();
const { ajv, inputSchema, outputSchema } = await advertisedMcpToolSchemas(
tests/boundary/mcp/environmentBundleMcp.test.ts
+271
View File
@@ -0,0 +1,271 @@
import { access, mkdir, readdir, readFile, writeFile } from "node:fs/promises";
import { join, sep } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { expect, it, onTestFinished } from "vitest";
import {
DEFAULT_BUILD_TEMPLATE_RULES,
buildEnvironment,
} from "../../../src/application/environment/environmentBuild.js";
import { evidenceFileBytes } from "../../../src/application/environment/environmentEvidenceBytes.js";
import { environmentExportFile } from "../../../src/application/environment/environmentExport.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import type { Evidence } from "../../../src/domain/evidence.js";
import { createServer } from "../../../src/server/createServer.js";
import { advertisedMcpToolSchemas } from "../../fixtures/advertisedMcpTool.js";
import {
analysisViewCaptureEvidence,
environmentManifestCaptureFixture,
} from "../../fixtures/analysisView.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const TOOL = "export_environment_bundle";
const connect = async () => {
const session = createTestBinarySession(() => {
throw new Error("the bundle export must not start a deep provider");
});
const server = createServer(
{ kind: "session", session },
{
environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: "full" },
},
);
const client = new Client({ name: "environment-bundle-mcp", version: "1" });
onTestFinished(async () => {
await client.close();
await server.close();
await session.close();
});
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
await server.connect(serverTransport);
await client.connect(clientTransport);
return { client, session };
};
const listFiles = async (root: string): Promise<string[]> =>
(await readdir(root, { recursive: true, withFileTypes: true }))
.filter((entry) => entry.isFile())
.map((entry) =>
join(entry.parentPath, entry.name)
.slice(root.length + 1)
.split(sep)
.join("/"),
)
.sort();
const loopbackCapture = (): Evidence =>
analysisViewCaptureEvidence(
JSON.parse(
JSON.stringify(environmentManifestCaptureFixture()).replaceAll(
/https?:\/\/grafana\.example/gu,
"http://127.0.0.1:3000",
),
) as ReturnType<typeof environmentManifestCaptureFixture>,
);
it("writes the bundle environment-build writes from retained capture Evidence", async () => {
const { client, session } = await connect();
const { ajv, inputSchema, outputSchema } = await advertisedMcpToolSchemas(
client,
TOOL,
);
const capture = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
);
expect(session.recordEvidence(capture).ok).toBe(true);
const root = await createTestTempDirectory("rea-bundle-mcp-");
const input = {
source: { kind: "retained-evidence", evidence_id: capture.evidence_id },
output_directory: join(root, "grafana"),
};
expect(ajv.validate(inputSchema, input)).toBe(true);
const response = await client.callTool({ name: TOOL, arguments: input });
expect(response.isError, JSON.stringify(response)).not.toBe(true);
expect(
ajv.validate(outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
).toBe(true);
const report = toolContract(TOOL).outputSchema.parse(
response.structuredContent,
);
expect(report).toMatchObject({
status: "built",
name: "grafana-example",
directory: input.output_directory,
capture: { evidence_id: capture.evidence_id, input: "evidence" },
manifest: { template_rules: [...DEFAULT_BUILD_TEMPLATE_RULES] },
});
const file = await environmentExportFile(
"evidence",
"capture.json",
evidenceFileBytes(capture),
);
if (!file.ok) throw file.error;
const reference = await buildEnvironment({
capture: file.value,
captureInput: "evidence",
capturePath: "capture.json",
templateRules: DEFAULT_BUILD_TEMPLATE_RULES,
directory: join(root, "reference"),
dryRun: false,
});
if (!reference.ok) throw reference.error;
const files = await listFiles(input.output_directory);
expect(files).toEqual(await listFiles(join(root, "reference")));
for (const path of files)
expect(await readFile(join(input.output_directory, path)), path).toEqual(
await readFile(join(root, "reference", path)),
);
expect(report.coverage).toEqual(reference.value.coverage);
});
it("copies a capture Evidence file byte for byte and plans without writing on a dry run", async () => {
const { client } = await connect();
const root = await createTestTempDirectory("rea-bundle-mcp-file-");
const path = join(root, "grafana.capture.json");
await writeFile(
path,
`${JSON.stringify(analysisViewCaptureEvidence(environmentManifestCaptureFixture()), null, 2)}\n`,
);
const planned = await client.callTool({
name: TOOL,
arguments: {
source: { kind: "evidence-file", path },
output_directory: join(root, "planned"),
dry_run: true,
},
});
expect(planned.isError).not.toBe(true);
expect(planned.structuredContent).toMatchObject({ status: "planned" });
await expect(access(join(root, "planned"))).rejects.toMatchObject({
code: "ENOENT",
});
const built = await client.callTool({
name: TOOL,
arguments: {
source: { kind: "evidence-file", path },
output_directory: join(root, "built"),
},
});
expect(built.isError).not.toBe(true);
expect(
await readFile(join(root, "built/envs/grafana-example/evidence.json")),
).toEqual(await readFile(path));
});
it("refuses an existing or parentless output directory and leaves it as it was", async () => {
const { client, session } = await connect();
const capture = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
);
expect(session.recordEvidence(capture).ok).toBe(true);
const root = await createTestTempDirectory("rea-bundle-mcp-dirs-");
const source = {
kind: "retained-evidence",
evidence_id: capture.evidence_id,
};
const empty = join(root, "empty");
const busy = join(root, "busy");
await mkdir(empty);
await mkdir(busy);
await writeFile(join(busy, "notes.md"), "keep");
for (const output_directory of [empty, busy]) {
const refused = await client.callTool({
name: TOOL,
arguments: { source, output_directory },
});
expect(refused.isError).toBe(true);
expect(parseMcpToolError(refused).error).toMatchObject({
code: "invalid_request",
details: {
operation: TOOL,
issues: [{ path: ["output_directory"], reason: "invalid_value" }],
},
});
}
expect(await readdir(empty)).toEqual([]);
expect(await readdir(busy)).toEqual(["notes.md"]);
const parentless = await client.callTool({
name: TOOL,
arguments: { source, output_directory: join(root, "missing", "env") },
});
expect(parentless.isError).toBe(true);
expect(parseMcpToolError(parentless).error).toMatchObject({
details: { issues: [{ path: ["output_directory"] }] },
});
await expect(access(join(root, "missing"))).rejects.toMatchObject({
code: "ENOENT",
});
const relative = await client.callTool({
name: TOOL,
arguments: { source, output_directory: "relative/env" },
});
expect(parseMcpToolError(relative).error).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["output_directory"] }] },
});
});
it("refuses unknown, unnamed loopback and non-capture Evidence, and repeated rules", async () => {
const { client, session } = await connect();
const root = await createTestTempDirectory("rea-bundle-mcp-refuse-");
const output_directory = join(root, "env");
const unknown = await client.callTool({
name: TOOL,
arguments: {
source: {
kind: "retained-evidence",
evidence_id: `ev_${"0".repeat(63)}9`,
},
output_directory,
},
});
expect(unknown.isError).toBe(true);
const loopback = loopbackCapture();
expect(session.recordEvidence(loopback).ok).toBe(true);
const unnamed = await client.callTool({
name: TOOL,
arguments: {
source: { kind: "retained-evidence", evidence_id: loopback.evidence_id },
output_directory,
},
});
expect(parseMcpToolError(unnamed).error).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["name"] }] },
});
const named = await client.callTool({
name: TOOL,
arguments: {
source: { kind: "retained-evidence", evidence_id: loopback.evidence_id },
output_directory,
name: "grafana",
dry_run: true,
},
});
expect(named.structuredContent).toMatchObject({
status: "planned",
name: "grafana",
warnings: [],
});
const repeated = await client.callTool({
name: TOOL,
arguments: {
source: { kind: "retained-evidence", evidence_id: loopback.evidence_id },
output_directory,
name: "grafana",
template_rules: ["opaque-token", "opaque-token"],
},
});
expect(parseMcpToolError(repeated).error).toMatchObject({
code: "invalid_request",
});
await expect(access(output_directory)).rejects.toMatchObject({
code: "ENOENT",
});
});
tests/boundary/process/browserRequestVerifierOwnership.test.ts
+158
View File
@@ -0,0 +1,158 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { pathToFileURL } from "node:url";
import { expect } from "vitest";
import { z } from "zod";
import { liveProcesses } from "../../../src/process/ProcessOwnershipProcessTree.js";
import { systemProcessOwnershipHost } from "../../../src/process/ProcessOwnershipObservation.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { processTest } from "../../support/process/processFixture.js";
// Records once if the lane's root disappears while this stand-in browser still runs.
const browserSource = `
import { appendFileSync, existsSync, readdirSync } from "node:fs";
const rootExists = () => readdirSync(process.env.OWNER_TMP).some((name) => name.startsWith("rea-request-e2e-"));
let seen = false;
setInterval(() => {
if (existsSync(process.env.OWNER_RELEASE)) process.exit(0);
if (rootExists()) seen = true;
else if (seen) {
appendFileSync(process.env.OWNER_EVENTS, JSON.stringify({ kind: "browser-outlived-root" }) + "\\n");
seen = false;
}
}, 10);
setTimeout(() => process.exit(0), 20000);
process.send({ pid: process.pid });
`;
// Launches its browser in a new process group, as Playwright does, then overruns the
// lane's output budget while still running.
const cliSource = `
import { spawn } from "node:child_process";
import { writeFileSync } from "node:fs";
const browser = spawn(process.execPath, [process.env.OWNER_BROWSER_ENTRY], {
detached: true,
stdio: ["ignore", "ignore", "ignore", "ipc"],
});
const ready = await new Promise((resolve, reject) => {
browser.once("message", resolve);
browser.once("error", reject);
});
browser.disconnect();
writeFileSync(process.env.OWNER_READY, JSON.stringify({ pids: [ready.pid] }));
const chunk = Buffer.alloc(1024 * 1024, 120);
for (;;) await new Promise((resolve) => process.stdout.write(chunk, resolve));
`;
const reportSchema = z.object({
pids: z.array(z.number().int().positive()).length(1),
live: z.array(z.number().int().positive()),
roots: z.array(z.string()),
failure: z.object({ name: z.string(), message: z.string() }),
});
const releaseBrowser = async (release: string, ready: string) => {
await writeFile(release, "release the stand-in browser");
const pids: readonly number[] = await readFile(ready, "utf8").then(
(text) => reportSchema.shape.pids.parse(JSON.parse(text).pids),
(cause: unknown) => {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
return [];
throw cause;
},
);
const deadline = Date.now() + 7_500;
while (
liveProcesses(await systemProcessOwnershipHost.listProcesses()).some(
({ pid }) => pids.includes(pid),
)
) {
if (Date.now() >= deadline)
throw new Error("The stand-in browser did not consume its release");
await delay(25);
}
};
// The stand-in CLI and browser are authored; the lane, its owner, the private root
// and process cleanup are real. No browser capture is simulated or claimed.
processTest.skipIf(process.platform === "win32")(
"stops a failed capture's CLI and the browser it launched before removing the request lane's root",
{ timeout: 40_000 },
async ({ processes }) => {
const root = await createTestTempDirectory("rea-browser-request-owner-");
const tmp = join(root, "tmp");
await mkdir(tmp);
const cli = join(root, "cli.mjs");
const browser = join(root, "browser.mjs");
const ready = join(root, "ready.json");
const release = join(root, "release-browser");
const events = join(root, "events.jsonl");
const driver = join(root, "driver.mjs");
await writeFile(cli, cliSource);
await writeFile(browser, browserSource);
const lane = pathToFileURL(
resolve("scripts/lib/browser-request-e2e.mjs"),
).href;
const observations = pathToFileURL(
resolve("dist/process/ProcessOwnershipObservation.js"),
).href;
const tree = pathToFileURL(
resolve("dist/process/ProcessOwnershipProcessTree.js"),
).href;
await writeFile(
driver,
`
import { readFile, readdir } from "node:fs/promises";
import { verifyBrowserRequestSteps } from ${JSON.stringify(lane)};
import { systemProcessOwnershipHost } from ${JSON.stringify(observations)};
import { liveProcesses } from ${JSON.stringify(tree)};
let failure;
try {
await verifyBrowserRequestSteps("/rea-unlaunched-browser", process.env.OWNER_CLI);
} catch (cause) {
failure = { name: cause?.name ?? typeof cause, message: cause?.message ?? String(cause) };
}
const { pids } = JSON.parse(await readFile(process.env.OWNER_READY, "utf8"));
const live = liveProcesses(await systemProcessOwnershipHost.listProcesses()).filter(({ pid }) => pids.includes(pid)).map(({ pid }) => pid);
const roots = (await readdir(process.env.OWNER_TMP)).filter((name) => name.startsWith("rea-request-e2e-"));
process.stdout.write(JSON.stringify({ pids, live, roots, failure }));
`,
);
try {
const result = await processes.run(process.execPath, [driver], {
timeoutMs: 30_000,
env: {
...process.env,
TMPDIR: tmp,
OWNER_TMP: tmp,
OWNER_CLI: cli,
OWNER_BROWSER_ENTRY: browser,
OWNER_READY: ready,
OWNER_RELEASE: release,
OWNER_EVENTS: events,
},
});
expect(result.exitCode, result.stderr).toBe(0);
const report = reportSchema.parse(JSON.parse(result.stdout));
expect(report.live).toEqual([]);
const receipts = await readFile(events, "utf8").catch(
(cause: unknown) => {
if (
cause instanceof Error &&
"code" in cause &&
cause.code === "ENOENT"
)
return "";
throw cause;
},
);
expect(receipts).toBe("");
expect(report.roots).toEqual([]);
expect(report.failure.message).toMatch(/output exceeded/u);
} finally {
await releaseBrowser(release, ready);
}
},
);
tests/fixtures/browserScenarioCapture.ts
Vendored
+43 -14
View File
@@ -27,8 +27,28 @@ export interface ScenarioExchange {
readonly body?: string | Buffer;
readonly requestMediaType?: string;
readonly requestBody?: string;
/** Header values as `header_values` retains them; a null value is one the capture excluded. */
readonly requestHeaders?: readonly ScenarioHeader[];
readonly responseHeaders?: readonly ScenarioHeader[];
}
export interface ScenarioHeader {
readonly name: string;
readonly value: string | null;
}
const retainedHeaders = (headers: readonly ScenarioHeader[] | undefined) =>
headers === undefined
? { state: "not_requested" as const }
: {
state: "captured" as const,
items: headers.map(({ name, value }) => ({
name,
value,
redacted: value === null,
})),
};
export const SCENARIO_ORIGIN = "http://127.0.0.1:3101";
/** A small app session: a page load, JSON reads, a search, a JSON write, a redacted URL, binary bytes and a failure. */
@@ -131,7 +151,7 @@ const exchangeEvents = (
sequence: number,
source: number,
phase: "request" | "response",
body: ReturnType<typeof capturedBody>,
body: ReturnType<typeof capturedBody> | { readonly state: "not_requested" },
) => ({
sequence,
step_index: 1,
@@ -139,7 +159,9 @@ const exchangeEvents = (
transaction_id: transaction,
source_event_sequence: source,
phase,
headers: { state: "not_requested" as const },
headers: retainedHeaders(
phase === "request" ? exchange.requestHeaders : exchange.responseHeaders,
),
body,
});
const events: unknown[] = [
@@ -152,17 +174,22 @@ const exchangeEvents = (
},
];
let next = first + 1;
if (exchange.requestBody !== undefined) {
if (
exchange.requestBody !== undefined ||
exchange.requestHeaders !== undefined
) {
events.push(
content(
next,
first,
"request",
capturedBody(
exchange.requestBody,
"browser-exposed-request-bytes",
exchange.requestMediaType ?? null,
),
exchange.requestBody === undefined
? { state: "not_requested" }
: capturedBody(
exchange.requestBody,
"browser-exposed-request-bytes",
exchange.requestMediaType ?? null,
),
),
);
next += 1;
@@ -186,17 +213,19 @@ const exchangeEvents = (
failure: null,
});
next += 1;
if (exchange.body !== undefined) {
if (exchange.body !== undefined || exchange.responseHeaders !== undefined) {
events.push(
content(
next,
response,
"response",
capturedBody(
exchange.body,
"playwright-response-body-bytes",
exchange.mediaType ?? null,
),
exchange.body === undefined
? { state: "not_requested" }
: capturedBody(
exchange.body,
"playwright-response-body-bytes",
exchange.mediaType ?? null,
),
),
);
next += 1;
tests/process-global/environmentBuildHeadroom.test.ts
+119
View File
@@ -0,0 +1,119 @@
import { readdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import * as v8 from "node:v8";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
DEFAULT_BUILD_TEMPLATE_RULES,
buildEnvironment,
} from "../../src/application/environment/environmentBuild.js";
import { evidenceFileBytes } from "../../src/application/environment/environmentEvidenceBytes.js";
import { readEnvironmentExportFile } from "../../src/application/environment/environmentExport.js";
import { createCli } from "../../src/cli.js";
import {
analysisViewCaptureEvidence,
environmentManifestCaptureFixture,
} from "../fixtures/analysisView.js";
import { environmentManifestHar } from "../fixtures/historicalHar.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
vi.mock("node:v8", { spy: true });
// The HAR decoder runs as a compiled child process; inspection returns the capture Evidence it decodes.
vi.mock("../../src/composition/webNetworkCaptures.js", () => ({
createWebNetworkCaptureService: () => ({
inspect: async () => ({
ok: true,
value: analysisViewCaptureEvidence(environmentManifestCaptureFixture()),
}),
close: async () => {},
}),
}));
/** Every later heap reading reports no headroom. */
const starveHeap = (): void => {
const heap = v8.getHeapStatistics();
vi.mocked(v8.getHeapStatistics).mockImplementation(() => ({
...heap,
total_available_size: 0,
}));
};
afterEach(() => {
vi.mocked(v8.getHeapStatistics).mockReset();
});
describe("environment build heap headroom", () => {
it("refuses to serialize the manifest the live heap cannot hold, before writing", async () => {
const root = await createTestTempDirectory("rea-environment-build-heap-");
const path = join(root, "grafana.capture.json");
await writeFile(
path,
evidenceFileBytes(
analysisViewCaptureEvidence(environmentManifestCaptureFixture()),
),
);
const capture = await readEnvironmentExportFile(
"evidence",
path,
"environment-build",
);
if (!capture.ok) throw capture.error;
starveHeap();
const built = await buildEnvironment({
capture: capture.value,
captureInput: "evidence",
capturePath: path,
templateRules: DEFAULT_BUILD_TEMPLATE_RULES,
directory: join(root, "built"),
dryRun: false,
});
expect(built).toMatchObject({
ok: false,
error: {
_tag: "AnalysisResourceConstraintError",
operation: "environment-build",
resource: "memory",
reportedLimits: {
input_path: "manifest.json",
input_file_bytes: expect.any(Number),
available_heap_bytes: 0,
},
},
});
expect(await readdir(root)).toEqual(["grafana.capture.json"]);
});
it("refuses to serialize an inspected HAR's Evidence the live heap cannot hold, before writing", async () => {
const root = await createTestTempDirectory("rea-environment-build-heap-");
const har = join(root, "grafana.har");
await writeFile(har, JSON.stringify(environmentManifestHar()));
const exitCode = process.exitCode;
starveHeap();
let stdout = "";
try {
await createCli({ REA_LOG_LEVEL: "silent" }).serve(
["environment-build", har, join(root, "built"), "--json"],
{
env: {},
exit: () => {},
stdout: (text) => {
stdout += text;
},
},
);
} finally {
process.exitCode = exitCode;
}
expect(JSON.parse(stdout)).toMatchObject({
code: "resource_constraint",
details: {
operation: "environment-build",
resource: "memory",
reported_limits: { input_path: har, available_heap_bytes: 0 },
},
});
expect(await readdir(root)).toEqual(["grafana.har"]);
});
});
tests/process-global/environmentBundleMemory.test.ts
+48
View File
@@ -0,0 +1,48 @@
import { readdir } from "node:fs/promises";
import { join } from "node:path";
import { getHeapStatistics } from "node:v8";
import { expect, it, vi } from "vitest";
import { exportEnvironmentBundleTool } from "../../src/application/environment/environmentBundleTool.js";
import { projectAnalysisError } from "../../src/domain/analysisErrorProjection.js";
import {
analysisViewCaptureEvidence,
environmentManifestCaptureFixture,
} from "../fixtures/analysisView.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
vi.mock("node:v8", { spy: true });
it("refuses retained export before allocation when its expanded copies cannot fit", async () => {
const capture = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
);
const root = await createTestTempDirectory("rea-retained-export-memory-");
const statistics = getHeapStatistics();
vi.mocked(getHeapStatistics).mockReturnValue({
...statistics,
total_available_size: 1,
});
const allocation = vi.spyOn(Buffer, "allocUnsafe");
try {
const result = await exportEnvironmentBundleTool(
{
source: { kind: "retained-evidence", evidence_id: capture.evidence_id },
output_directory: join(root, "bundle"),
dry_run: false,
},
() => capture,
);
if (result.ok) throw new Error("Expected a memory refusal");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "resource_constraint",
details: { operation: "export_environment_bundle", resource: "memory" },
});
expect(allocation).not.toHaveBeenCalled();
expect(await readdir(root)).toEqual([]);
} finally {
allocation.mockRestore();
vi.mocked(getHeapStatistics).mockReset();
}
});
tests/process-global/environmentExportMemory.test.ts
+57
View File
@@ -0,0 +1,57 @@
import { getHeapStatistics } from "node:v8";
import { expect, it, vi } from "vitest";
import { evidenceFileBytes } from "../../src/application/environment/environmentEvidenceBytes.js";
import {
environmentExportEvidenceFile,
environmentExportFile,
} from "../../src/application/environment/environmentExport.js";
import { projectAnalysisError } from "../../src/domain/analysisErrorProjection.js";
import {
analysisViewCaptureEvidence,
environmentManifestCaptureFixture,
} from "../fixtures/analysisView.js";
vi.mock("node:v8", { spy: true });
it.each(["serialize", "parse"] as const)(
"refuses in-memory export before %s when its copies cannot fit",
async (stage) => {
const capture = analysisViewCaptureEvidence(
environmentManifestCaptureFixture(),
);
const bytes = evidenceFileBytes(capture);
const statistics = getHeapStatistics();
const allocate = vi.spyOn(Buffer, "allocUnsafe");
vi.mocked(getHeapStatistics).mockReturnValue({
...statistics,
total_available_size: 1,
});
try {
const result =
stage === "serialize"
? await environmentExportEvidenceFile(
"evidence",
"capture.json",
capture,
"environment-build",
)
: await environmentExportFile(
"evidence",
"capture.json",
bytes,
"environment-build",
);
if (result.ok) throw new Error("Expected a memory refusal");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "resource_constraint",
details: { operation: "environment-build", resource: "memory" },
});
expect(allocate).not.toHaveBeenCalled();
} finally {
vi.mocked(getHeapStatistics).mockReset();
allocate.mockRestore();
}
},
);