Files
rea/docs/specs/desktop-app-environments/document-views/verification.md
Edgar ArakelyanandClaude Opus 5.5 ac23dc8077 feat(desktop): read XML with a plain DOCTYPE, and check an element's whole text (#9050)
* feat(desktop): read XML with a plain DOCTYPE, and check an element's whole text

- Both readers accept a DOCTYPE with only a name and an optional PUBLIC
  or SYSTEM identifier, record it as the view's doctype, and never fetch
  the DTD or expand entities beyond XML's five; an internal subset, an
  external entity and a parameter entity stay refused alike.
- text_equals and text_contains compare the concatenated text of the
  element (or text node or attribute) a pointer reaches, as XPath
  string() does, with optional normalize_whitespace; both evaluators
  agree, and a text check on a truncated element is undecided.
- Whitespace-only text between elements is dropped only when the element
  holds no other text, so text split into runs keeps its spaces.
- The draft suggests text_equals for an element whose text changed.

So an AbiWord .abw reads as XML, and its paragraph's text is checked
whole however AbiWord splits it into <c> runs, as Writer's spans are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): draft whole-text checks on the paragraph, read only text as text, and skip asctime stamps

- A changed value is read at both ends, so text that became a run still
  marks its paragraph, and the drafted text_equals moves up out of mixed
  content to the nearest ancestor named by an attribute, such as
  AbiWord's p by its xid.
- Text checks accept only a text node or an attribute value; element
  names and doctype fields fail, in both evaluators.
- isTimestamp also recognises asctime dates (Www Mmm dd hh:mm:ss yyyy).
- XML files are drafted with the keyed diff the archive entries use, so
  no suggestion reaches an element by position; the shared logic moves to
  appTaskDraftNamed.ts.
- Text-check tests assert facts, not whole sentences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep checks on a changed attribute inside a paragraph whose text check replaces its runs'

A drafted whole-text check on a paragraph hid every change inside it, so a
link whose label and href both changed drafted only the paragraph's text and
the new destination went unchecked. The text check now covers text and runs
added or removed whole; a changed name or attribute of an element both
versions hold keeps its own check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): read an attribute called name or attributes as an attribute when the draft keeps its check

The owner of `.../attributes/name` was taken as `.../attributes`, so a
changed `name` attribute inside a paragraph with a text check was dropped
again. The owner is now read by segment, attributes first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): draft text checks from text in runs added whole, and name a text element by its own id before its container

Text typed into a run the diff lists whole was never read, so the draft kept
only a presence check on the run, which wrong text also passed. Such text now
marks the nearest element both ends hold, or, when none can take the check,
the added element's own text. The climb also names an element by an
identifier of its own (id, xml:id, xid) before a larger container, so a
headline under a long layer keeps a whole-text check that survives re-split
runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(desktop): expect the whole-text check where #8988's pairing test meets text checks

On this branch a changed element's text is drafted as `text_equals` on the
element, so the identity pairing test now expects that check, under the
same `[@role=body]` key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep checks for elements without text beside a text edit, and normalize DOCTYPE system id line endings

A review found two gaps.

- A paragraph's whole-text check covered every element added or removed
  inside it, including ones holding no text: an image frame, a line break.
  A solver who typed the text but never inserted the image passed. Such an
  element now keeps its own presence or absence check.
- The Python reader kept a DOCTYPE system literal's CRLF where TypeScript
  normalizes it to LF. Both now report LF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(desktop): import textSyntaxFormat once in the state format tests

main's mixed-case JSON suffix fix and this branch both added the import,
and the merge kept both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 03:20:12 +00:00

14 KiB

Verification

Run on 2026-11-05 on top of the vendor .deb files branch, on macOS with Node v22.23.1 and Python 3.11.14.

Gates

  • npm run typecheck, lint, lint:typed, format:check, knip (after build:cached), jscpd, docs:check, docs:build, verify:test-discovery and verify:architecture-guards: passed, with no lint or typed-lint diagnostics in the touched files. The one clone jscpd lists in a touched file, DesktopApplicationState.ts against GpgvKeyCheck.ts, was there before. After main was merged in, lint and lint:typed fail only on main's own tests/acceptance/webSourceLocationCli.test.ts.
  • npx vitest run src/domain/desktop src/application/desktop src/desktop with tests/boundary/providers/desktopExportPython.test.ts, desktopTaskPython.test.ts and tests/boundary/cli/cliStartupImports.test.ts: 377 passed and 18 skipped (Linux-only cases). The new tests/boundary/providers/desktopZipPython.test.ts: 13 passed, under Python 3.9.6, 3.10, 3.11, 3.12, 3.13 and 3.14 alike; desktopStateHeap.test.ts: 2 passed, and desktopZipHeap.test.ts: 1. The desktop CLI boundary tests desktopApplicationCli, cliInputSchemaRequired and cliToolCoverage: 28 passed.
  • Namespaced XML, on the same day and toolchain: the same gates passed, with no lint, typed-lint or jscpd findings in the touched files, and the desktop tests with the four provider parity files, desktopXmlPython.test.ts among them: 370 passed and 18 skipped.

Tests

Each new test fails on the code before this change, run from a copy of that commit with only the new tests and fixtures added; the tests already in the touched files still pass there.

  • Parity of the two zip readers, REA's and the exported module's, on the same archives: entries in order with JSON, key file, XML, text, UTF-8-named and binary entries, a directory left out, and an archive, a SQLite database and an inexact JSON number inside, each not read for its reason; an encrypted entry, bzip2, a declared 17 MiB entry, a 943-times expansion, a bad CRC-32 and a short declared size; a damaged archive; 1,002 entries; and the 64 MiB archive budget through both readers' refusal functions.
  • FreeCAD: the real Document.xml of FreeCAD 0.21 and 1.0, before and after the hole, zipped into .FCStd archives. Both readers give the same views, and both evaluators give the same results and descriptions: a Part::Cut exists, the Hole radius is 5 ± 0.01 and the cut's tool links Hole all fail before the edit and pass after it, in both versions, by the same pointers.
  • OpenDocument: an .odt built in the test reads as mimetype text and content.xml and META-INF/manifest.xml as XML, with the paragraph's text in the view; the readers give the same view, namespaced names included, and the export keeps a check of the paragraph by [name=text:p].
  • Namespaced XML, in desktopXmlPython.test.ts: the real saves of an Inkscape 1.2.2 poster and a LibreOffice 24.2 Writer flyer, before and after one edit, read alike by both readers, the SVG as a file and content.xml in an .odt built in the test. Both evaluators give the same results and descriptions: the headline [name=text][@id=headline] under the layer [@inkscape:label=Layer 1] and the [@text:style-name=Subtitle] date fail before the edit and pass after it, and inkscape:document-units passes in both. Built documents cover xmlns declarations in document order, an undeclared default namespace, xml: attributes, CDATA, comments and processing instructions, what both parsers refuse (an undeclared prefix, one attribute twice under two prefixes, a prefix bound to "", two colons in a name, a colon in a processing instruction's target, a DOCTYPE with an internal subset), and the node and depth limits on prefixed names.
  • Selectors and tolerance in the shared check cases both evaluators run: one member matched, several, none before or at the last segment, a malformed selector, an escaped / in an entry name, an entry by index, a selector over strings in an array, per-entry truncation, an entry the listing left out; numbers and numeral strings within a tolerance, outside it, a string and a boolean that are not numbers, and tolerance: 0.
  • locatePointer and the member keys: the most general selector unique among siblings, passed over when it names several members of the other version, and values that hold ][ or look like a time left out.
  • The keyed zip diff: an object inserted before another shifts nothing, and a comparison is incomplete when an entry is unread, truncated or not listed.
  • The draft: an archive's added and removed elements and changed values by selector, inside namespaced XML too, where a paragraph is named by [name=text:p][@text:style-name=Subtitle]; and through the application, a capture with a FreeCAD document and an .odt whose draft, with a check of the list's paragraph, the export then accepts.

Review fixes

Each fix has a regression test that fails on the reviewed commit and passes after it:

  • A selector over an array a view may hold only part of is undecided in both evaluators: a truncated XML view, a cut listing searched by entry name, and a truncated entry's XML, beside positions into the same views that stay decided.
  • The zip diff pairs members one to one: an element whose attributes split across two new siblings leaves one paired and the other added.
  • The draft checks an entry or a JSON value added whole through its members, and suggests nothing that reaches an element by position, the last step included.
  • The export accepts a check on the whole listing whose value holds namespaced XML, since both readers name it alike.
  • The exported reader keeps a JSON entry nested 1,100 deep on that entry, with its siblings read, and refuses JSON nested more than 500 levels as REA does, 501 levels read and 502 refused, in a file or an entry.
  • The exported reader reports an entry damaged when its declared size is short and its checksum is the prefix's, deflated or stored, as REA does.
  • An array that becomes an object with the same keys, or the reverse, is a changed value, with nothing in it added or removed, so the draft suggests no absence the end state contradicts.
  • The draft checks an empty array or object added to a JSON entry as it is.
  • An entry REA could not read, whose view is therefore null, gets no drafted check, and both evaluators fail value_equals null into its view, while a JSON entry holding null still passes it.
  • The exported reader's depth check holds one iterator per level: on a 4,000,001-byte array of 800,000 nulls, _text_view peaks at 6.7 MB of traced allocations, as before the check, against 58.2 MB when it queued every member.
  • A binary entry fails value_equals null and value_absent into its view in both evaluators, its size still checked, and the draft suggests no absence in an entry that ends binary.
  • REA's own depth check (deeperThan, from the state view) also holds one iterator per level: on 800,000 nulls it grows the heap by 7.3 MB of short-lived iterator results, against 72.7 MB when it queued a pair per member, and runs under an 18 MiB old-space limit where the queue needed more than 64 MiB. A test runs it under a 32 MiB limit.
  • The exported reader reads an entry in 64 KiB chunks up to a byte past its declared size: a 32,731-byte archive holding 32 MiB of zeros declared as one byte peaks at 0.1 MiB of traced allocations, against 77.5 MiB when it read the stream whole before clipping it, both reporting the entry damaged as REA does. An 8.7 MiB text entry peaks at 48.5 MiB against 47.6 MiB. REA's zip.js reader already stops at the first chunk past the declared size: reading that archive grows the peak resident size by 0.8 MiB, as a one-byte archive does.
  • A check of a namespaced entry's size, of its absence, or comparing the entry whole exports, since both readers name namespaced XML alike.
  • The diff finds a counterpart by the evaluator's own step: a bracketed key against an array is a selector, so {"[name=x]": 1} becoming [{"name": "x"}] reports nothing removed, in an archive or a JSON file, and the draft suggests no absence of a value the selector still names.
  • Both readers bound the central directory: an archive of 600,000 empty entries (52.4 MiB) reads in a 24 MiB V8 old space against more than 48 MiB when zip.js parsed every record, and grows peak RSS by 37 MiB against 184 MiB; the exported reader peaks at 1.3 MiB of traced allocations against 293.6 MiB, and at 1.3 MiB against 25.1 MiB for 50,000 entries. Directories are told apart as zip.js tells them in both.
  • Both readers follow a Zip64 locator whether or not the classic end record holds sentinels, as zipfile does: an archive whose classic record keeps its own counts, size and offset, which zipfile and zip.js both read, was unreadable to both readers and now lists its entries in both. A last name ending in bytes that only look like a locator keeps the classic values in both, and the exported module gives zipfile real Zip64 end records for the directory it shows it, so zipfile no longer refuses such an archive; the parity tests pass under Python 3.9 to 3.14.
  • A diff comparing an object of selector-shaped keys with an array indexes the array once per set of selector fields: 2,000 keys read its members 4,000 times against 4,002,000, and 8,000 and 20,000 keys diff in 19 and 31 ms against 980 ms and 5.4 s.
  • The keys are resolved together, their conditions sorted and de-duplicated, holding only the values they ask for rather than an index of the array per set of fields. 127 field combinations over 5,000 members grow the heap by 5.5 MB against 68.0 MB, in 138 ms against 174 ms; 160 spellings of one selector over 800 members by 4.9 MB against 75.9 MB, in 4 ms against 388 ms, reading the members 1,600 times against 128,800. Both diff in a 16 MiB V8 old space where they needed more than 48 MiB; a test runs them under a 32 MiB limit.
  • Member pairing keeps an element's tag apart from its attribute conditions and spells a selector only once it is chosen: an element with a 38,588-character tag and 2,000 attributes pairs with itself in 4 ms, growing the heap by 1.1 MiB, against 7.2 s and 148 MiB when the tag was copied into a selector per attribute. A test pairs two such elements under a 32 MiB V8 old space.
  • An element pairs by its identifiers first, and by a later selector both versions share before one that names nothing in the other version keys it as added: <item color="red" id="1"/> becoming <item color="green" id="1"/> beside another item is a changed color under [name=item][@id=1], in the diff and the draft, rather than one element removed and another added. A FreeCAD Hole deleted and a Hole001 added, both Part::Cylinder, stay one removal and one addition rather than a rename found by their shared type. An identified element's other attributes still count against its siblings' selectors: [@color=blue] is not reported removed when it moves to the identified item, and [@role=body], [@role=second] and an Inkscape group's [@inkscape:label=note] key their elements rather than a color, class or fill an identified sibling shares, which would match both. The real FreeCAD 0.21 and 1.0 documents, and the Inkscape, LibreOffice and AbiWord ones of the branches built on this one, zipped as entries, give the same diffs, drafts and member keys as before.
  • Both readers refuse a JSON or key file over 4 MiB before building its value, telling JSON from text by a token scan. REA's reader detects a 4,200,004-byte array of empty objects in 58 ms, growing the heap by 4.2 MiB (the decoded text), against 129 ms and 85.5 MiB when detection parsed it, and a test reads it under a 32 MiB V8 old space. The exported reader's entry peaks at 8.4 MB of traced allocations against 110.0 MB, a 16 MiB one at 35.2 MB against 442.5 MB, and a 4.9 MB key file at 10.4 MB against 86.7 MB; its scan takes 0.9 s and 3.5 s for those entries against 0.7 s and 2.7 s for the parse. The exported reader now applies the limit to files as REA does, where it read them whole before.
  • The exported reader takes text nested past Python's recursion limit as JSON only when the token scan finds all of it JSON, as REA does: 1,100 opening brackets and 1,099 closing ones are text in both readers, where the export called them JSON nested too deeply, and 1,100 of each are still refused for their depth in both.

FreeCAD captures

The real .FCStd files the FreeCAD investigation captured, from FreeCAD 0.21 (Ubuntu 24.04 with the FreeCAD PPA) and 1.0 (Ubuntu 25.10), read by the built CLI:

  • inspect-desktop-application-state: plate.FCStd is zip in every version, 2 entries before and 15 or 16 after, with a complete values change listing /entries/[name=Document.xml]/view/root/children/[name=Objects]/children/[name=Object][@name=Hole] among the added values; the .FCBak backups are zip too.
  • REA's evaluator and the exported module's, each on its own view of the retained bytes: identical views, and the hole task's two checks fail before and pass after in both versions.
  • export-desktop-application --task on the 0.21 capture keeps the task with both checks failing on the seed and writes the selectors and the tolerance into verify.py. The 1.0 capture is not exported, as before this change: its package installs only on Ubuntu 25.10.
  • describe-desktop-application-task on the 0.21 capture drafts ten checks on plate.FCStd, all by selector (the six objects and dependencies FreeCAD added, then counts and statuses), and the export keeps that draft with all ten failing on the seed.