* feat(artifacts): inventory and extract UF2 and DfuSe regions
open_binary refused UF2 and DfuSe firmware containers as an unsupported
format, so inspect_artifact and extract_artifact could not show which
bytes a drag-and-drop or DFU update writes at which addresses.
UF2 is recognized by a complete first block with all three magic
numbers. Valid main-flash payloads are joined into address-contiguous
regions per familyID, named family-0x<id>/0x<start>-0x<end>.bin, and
known IDs are named from a small table copied from Microsoft's
uf2families.json at a pinned commit. Numbering is counted per familyID
and block count: absent, repeated, out-of-order and out-of-range block
numbers are reported, and absent blocks leave coverage unknown.
Not-main-flash blocks are excluded and reported; file-container blocks
are reported and leave coverage unknown; invalid blocks and a partial
final block are excluded with their position. A rewrite of one address
with different bytes fails closed, naming both block spans.
DfuSe is recognized only by its prefix together with a DFU 1.1a suffix.
Each element becomes one region at its file range with its target's
alternate setting and name. Structural faults fail closed with a byte
offset; overlapping elements stay separate and are reported. A suffix
CRC mismatch follows integrity_policy: fail refuses with a dfu-crc32
integrity error, and record-and-continue records a root contradiction
that extraction also carries.
HEX, UF2 and DfuSe share one retained-region reader and one run
merger, so the HEX decoder and reader lose their private copies.
* docs(artifacts): document UF2 and DfuSe images and their esptool lane
Describe recognition, region names, metadata, numbering, failures, the
DfuSe suffix CRC under integrity_policy, and limits, and link the page
from the CLI and firmware guides and the agent skill reference.
npm run verify:uf2 runs esptool merge-bin --format uf2 on generated
inputs at known addresses, with and without MD5, with 256-byte
payloads, and on a concatenation of two chips' outputs. Inspection and
extraction through the built CLI must return every input at its address
under the familyID esptool itself reports.
* fix(artifacts): accept both DfuSe image size conventions
UM0391 calls DFUImageSize the file length, and dfu-util's dfuse-pack.py
writes that value, while ST's DfuFileMgr, MicroPython's dfu.py and pydfu
leave the 16-byte suffix out. The decoder accepted only the second form,
so it refused dfuse-pack.py output as malformed.
Accept either value and keep the declared one in image_size; any other
value still fails with the suffix offset and the file length. The tests
pin both producers' output for one image, CRC included, from their
build() functions transcribed and run under Python 3.
* fix(artifacts): keep reused UF2 block numbers from hiding missing blocks
esptool numbers every merged input from 0, so two inputs with the same
block count share one numbering group. A block lost from one input left
its number present in the other, and the file reported complete
coverage. Each block number now records the target addresses it was
seen at: equal address counts are complete copies, a resend to the same
address is one block, and unequal counts make coverage unknown with a
limitation and ambiguous_numbering_group_count. sequence_count becomes
numbering_group_count, and region block positions become
first_file_block and last_file_block.
Review follow-ups in the same change:
- compare_artifacts compares a DfuSe element's alternate setting and
target name, so bytes moved to another device memory are changed;
- the shared reader keeps no region bytes outside its map after close;
- one domain type replaces four identical decoder failure types;
- the UF2 decoder returns a typed limit instead of throwing on bytes
beyond its admitted size, and caps retained regions, not runs;
- DfuSe caps elements with bytes only, names empty elements and
element-less targets, drops the constant bVersion and bcdDFU fields,
and says whether the CRC matches when DFUImageSize fits neither
convention;
- classification evidence carries a suffix only when the probe saw the
end, and root classification reads the tail only for a DfuSe prefix;
- the UF2 domain region owns family_name, and the root records the
family registry revision; the table's MIT notice is preserved;
- docs pin the producer revisions (dfu-util ee6b393d and f9a537c9,
MicroPython cb99ca98), give the MD5 structure's real meaning, and
describe container contradictions in the MCP contract guide;
- tests and the esptool lane add the equal-count inputs and a block
lost from one of them, accepted boundaries beside each refusal, a
DfuSe root larger than the classification prefix, and CLI refusals.
* feat(artifacts): name the UF2 families that LibreTiny builds for
LibreTiny writes UF2 for Realtek, Beken and Bouffalo Lab modules. The
Microsoft registry revision REA already cites (cf3bd7f9) lists eight of
those families: RTL8710A, RTL8710B, RTL8720C, RTL8720D, BK7231U, BK7231N,
BK7251 and BL602. Their regions now carry those names. LN882H, which
that revision omits, keeps its familyID with no name.
* fix(artifacts): count DfuSe declarations without bytes instead of listing them
Empty DfuSe elements have no entry, so DFUSE_MAX_ELEMENTS does not cap
them, and the target walk kept a description string for each one. At 8
bytes per element, a file under the 128 MiB ceiling can declare about
16.7 million of them. The walk now counts every declaration without bytes
and builds only the first NAMED_ELEMENTS descriptions, which is all that
the limitation shows. The limitation text and empty_element_count do not
change. The guide now says that both DfuSe limitations name the first four.
On this host, decoding 4,000,000 empty elements (a 32 MB file) ran out of
memory under a 128 MB V8 heap before this change, and it peaked at 1.59 GB
RSS with the default heap. It now decodes in 114 ms with a 220 MB peak.
---------
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* feat(pe): inspect import and delay-load tables
Add inspect_pe_imports / inspect-pe-imports to read PE32/PE32+ import
and delay-load directories without executing the image. The report keeps
DLL names, named and ordinal functions, ILT/IAT RVAs, and a pefile
import hash of the standard import table. A missing directory is
observed absence, not proof the image never imported APIs.
* fix(pe): default PE import fixture options
Allow peImportImage() without an argument so CLI and filesystem tests
can use the KERNEL32 sample without repeating an empty options object.
* test(pe): keep import path cases on the dedicated MCP contract
inspect_pe_imports already rejects relative MCP paths in its own
boundary test. Leave the shared absolute-path matrix to tools whose
handler issue projection matches that suite's expected shape.
* fix(native): index virtual-to-file mapping lookups
mappedFileOffset scans every mapping twice per span, so a parser that
translates one span per table entry costs sections x entries. Add
fileMappingLookup, which builds an O(n log n) interval index once and
answers each span in O(log n) with the same accept, reject and
translation results. A differential test checks it against
mappedFileOffset over randomized overlapping mappings and zero fills.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ra7LM2NyQ7DumdssxbUq5g
* fix(pe): report partial import evidence and exact pefile hashes
- Withhold imphash with an explicit reason whenever the standard import
table is partial (unreadable names or thunks, missing tables), or when
pefile 2024.8.26 would hash a different row sequence: invalid or
truncated names, ordinal 0, descriptors without functions, tables its
bounds or repeat/spread heuristics discard, and its 8192-thunk import
symbol cap. Delay-load issues do not withhold it.
- Hash ordinal imports with pefile's complete ordlookup tables and its
ord<decimal> fallback instead of declining unmapped ordinals.
- Report a nonterminal descriptor whose FirstThunk is zero, and a
delay-load descriptor whose IAT or INT RVA is zero, as malformed_entry
partial coverage instead of a complete empty inventory.
- Directory evidence lists the file spans actually consumed for the
descriptor array and terminator; discontiguous mappings produce
separate spans and separate Evidence ranges.
- Carry exhausted limits as PeImportLimitFailure.limit and derive
recovery from it; the fixed 16 MiB retained-text ceiling reports a
fixed allowance instead of max_file_bytes advice.
- Translate RVAs through the indexed mapping lookup and yield with a
cancellation check every 1024 table entries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ra7LM2NyQ7DumdssxbUq5g
* fix(pe): read only own ordinal tables for DLL names
DLL names such as constructor and __proto__ pass pefile's name rule but
resolved inherited object members, throwing before the ord<n> fallback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ra7LM2NyQ7DumdssxbUq5g
* fix(native): preserve PE import facts and prove hash compatibility
* test(native): verify PE import workflows and measured costs
* refactor(native): narrow PE facts across the container union
* docs(native): record integrated PE import validation
* fix(native): bound every PE import RVA span
* docs(native): measure RVA guard completed workflows
* fix(pe): reconcile import coverage with retained rows
* fix(pe): retain partial imports at descriptor mapping gaps
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
* feat(kuna): serve procedures, pseudocode and assembly from Kuna exports
Register Kuna as a bring-your-own deep-analysis provider. It answers
list_documents, list_procedures, search_procedures, procedure_address,
address_name, procedure_pseudo_code, procedure_assembly and
analyze_function (and batch_decompile through composition) from the
verified project export. While a cold export builds in the background,
pseudocode comes from a single-function `kuna decompile` run on a private
copy and says so; assembly and data names wait for the export.
Each answer records its source and file span in raw_result, limitations
are scoped per operation, and the session-owned export is stopped on
close unless another session still waits for it. The literal/regex
procedure search predicate moves from the IDA runner into a shared
domain helper. `npm run verify:kuna` adds a real-provider lane with an
optional Ghidra comparison.
Refs #1957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(kuna): refuse a Ghidra comparison without caller-built targets
Refs #1957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(kuna): retry owned cleanup and keep answers bound to the opened Kuna
- Re-verify the Kuna executable's digest before every launch (memoized by
file identity), so no answer names one build while another produced it.
- Keep the process supervisor of an unproven stop, and retry it on every
later close in both the session and the cache; a failed close keeps
failing until the stop is proven instead of reporting success.
- Report an abandoned export's retained cleanup from session close.
- Forget and rebuild an adopted export whose files another REA process
evicted, instead of failing every later read.
- Close a private copy whose setup failed, and retry failed removals.
- Let the session own the shared quick inventory; each caller only stops
waiting when it cancels.
- Report a failed export as `failed` with its reason.
- Lane: release the MCP client when open_binary rejects a target, and accept
either valid source for the first answer.
Refs #1957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(kuna): drain session requests before releasing workspaces
Combine caller cancellation with session shutdown and wait for active reads before releasing their process supervisors or private inputs. Retain unproven cleanup for later close attempts. Measure UTF-8 reply bytes without implying model token usage.
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* perf(kuna): keep document metadata reads free of target analysis
Listing the selected path no longer initializes the project cache or starts an export that the request never consumes.
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* fix(kuna): reject verification matrices with missing functions
Validate requested matrix coverage before building the fixture or opening providers. Empty matrices, unmatched names and partially matched function sets must fail instead of reporting an unperformed comparison as successful.
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* fix(kuna): retain component provenance in function dossiers
Record each source facet as one file span and preserve both pseudocode and assembly digests/ranges in combined reports. Keep direct text source fields compatible and verify every combined span through the real MCP lane.
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* fix(kuna): keep retired file cleanup bound to its export
Retain the export key with each failed file removal so a cancelled session reports and retries its own stopped resources. Preserve that association across sweeping and process-cleanup recovery without attributing unrelated cache failures to the session.
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* docs(kuna): document the Kuna provider and list it in the catalog (#3855)
* docs(kuna): document the Kuna provider and list it in the catalog
Add the Kuna provider guide (configuration, answered tools, cold and warm
paths, provenance, cache ownership, verification lane), link it from the
README and its translations, the CLI, installation and index pages and the
site navigation, describe `npm run verify:kuna` in the testing guide, credit
Kuna in THIRD_PARTY_NOTICES, teach the agent skill when to select Kuna, and
register its identity and operations in the generated product catalog.
Refs #1957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(kuna): state the Linux cache fallback, raw_result scope and comparison inputs
Refs #1957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(kuna): note that cache pins hold within one REA process
Refs #1957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(kuna): describe retired staging and retried cleanup
Refs #1957
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(kuna): supervise regex and bind reads to one export
Share a cancellable ECMAScript matching worker with Hopper while preserving its error contract. Bind C and assembly indexes to immutable project readers, retry whole dossiers after eviction, and enforce the existing buffered-text capacity before allocating cached spans. Document the verified Linux scope and keep complete exports accessible.
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Reuse the domain ZIP format list for session schemas, verify full Harmony
CLI/MCP lifecycles, and correct native view guidance and audited docs drift.
Refresh locked dependency notices and gate inventory/provenance consistency,
including notice inputs in the cached documentation gate.
Fixes#2454
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Ship the responsible-use statement, third-party attribution records and
preserved license texts in npm packages. Clarify historical-version and
channel coverage without changing existing licenses or installation behavior.
Assisted-by: Eiki (gensokyo-skills:eiki-rule-review)
Assisted-by: Aya (gensokyo-skills:aya-source-investigation)
morluto