Kanban workers no longer run `chat -q`: the spawned process is
hermes_cli.kanban_worker_client, which submits kanban.run to the profile's
owner gateway, and the owner runs the attempt in _run_task_turns. Those
three run_conversation calls (first turn, in-place recovery, goal turns)
passed no turn_author, so memory providers still filed the dispatcher's
prompt and the worker's notes under the operator (#137406). They now pass
a fixed bot author. Its id is kanban:dispatcher rather than bot:kanban,
which a Bot Mode profile named kanban would also produce.
The dispatcher spawns `hermes chat -q "work kanban task <id>"` with no turn
author, so memory providers filed the dispatcher prompt and the worker's reply
under the human user's peer. Honcho, for example, derived facts such as
"the user mentioned kanban task t_..." and promoted worker progress notes into
the user's profile.
The spawn now sets HERMES_TURN_AUTHOR to a bot author (`bot:kanban`), and the
non-quiet `-q` one-shot path consumes it the same way `-Q` already does and
forwards it to every turn of the run, including recovery and goal follow-ups.
Providers that understand bot authors route these turns away from the human
peer; a human `-q` run keeps today's unattributed call shape.
Since the default profile's env allowlist was restored under multiplex,
authorization does reach DISCORD_ALLOWED_USERS for the default bot; only a
secondary profile's own bot depends on its adapter config. Only the
default-profile cases use the real config loader. The own-bot case also
wrote a profile .env and a home config.yaml that nothing read.
The thread-click and guild-channel cases pass on main since the default
profile's env allowlist was restored under multiplex, so they are guards,
not regressions. The thread-route case proves the live verdict resolves the
routed profile; it does not prove a user paired only there can answer the
card (the click gate consults the live verdict only for users already in the
connect-time allowlist). The own-bot case never loaded the profile
config.yaml it wrote, and the route case took an unused monkeypatch.
Since 70872e7fa6 the default profile's config-only allow_from reaches
the env under multiplex again, so the guild-channel cases above pass
without the allow_from_covers_groups fallback. A profile running its
own Discord bot still depends on it: only the default profile's list
reaches the env, so its guild channels can be admitted only through its
adapter's allow_from, as its DMs already are.
Under multiplex, authorization reads each profile's own secret scope, so the
DISCORD_ALLOWED_USERS the YAML bridge copies from discord.allow_from into os.environ is never
seen. The fallback to the adapter's extra then read group_allow_from for 'group' sources:
Telegram's group-only list, which Discord does not have. An allowlisted user's guild-channel
messages were dropped and their button clicks refused, while threads and DMs worked.
Discord declares allow_from_covers_groups, so its groups read allow_from. Adapters with a
separate group list (Telegram) are unchanged; a test pins that a Telegram DM-only allow_from
still does not admit the user in a group.
#133213 confirms a snapshot-admitted clicker with the gateway's live check, but labelled every
guild click 'group' and passed no thread id. Message intake labels the same channel 'thread',
so a user the gateway admits in a thread could be refused on its approval card ("This bot is
private..."), and a thread_id profile route never matched the click. Label the click like
intake does: a discord.Thread is 'thread' with its id as thread_id.
With compression.prepare_ahead, compress() re-planned the tail cut and threw the
completed background summary away whenever the fresh cut landed before the
summary's boundary (the no-LLM prune made tail rows cheaper, tool-group
alignment moved the cut), then wrote a second summary inline while the user
waited. That breaks the one-summary rule: once the background summary exists it
is used, and everything after its boundary is the tail.
The splice check no longer consults the fresh cut at all. A summary is still
discarded only when the messages it covers changed (rewind, shape change, other
session, moved head). The pass planned its own window with a full tail, so the
tail is never empty. The now-unreachable inside_tail reason leaves the
(unreleased) prepared-compaction metric contract.
_admission_outcome read the admission row, then its result blob, in two
statements. The drain settles from a worker thread (commit_and_publish), so
the commit could land between them: the result was present while the stale
started row had no outcome, and a completed turn projected as a terminal
`failed` receipt carrying the reply as its error. Terminal receipts leave
_bot_receipt_index, so the settle wake re-read nothing and no reply watcher
existed; the sender kept the false failure for good.
Read the result first; when one exists but the row read was not terminal,
re-read the row (the result and terminal outcome commit together, and
terminal is final).
This is the CI flake test_paused_receipt_rearms_on_resume_without_polling_rpc
("the paused delivery never settled"). Repro: an in-process loop of that
scenario with a 0-20 ms stall between the two reads strands 251/2400
receipts on main (all `failed`/error=reply, row terminal/completed), 0/2400
with this fix. The new test commits the settlement between the two reads
through refresh_receipt and fails on main with ('failed', 'pong').
Two paths left a moved local conversation's mount wrong (Greptile on 757486d):
- Startup recovery of a lane's queued input (`recover_native_sessions`) and an
automation wake built the mount without `handed_off`. Once the chat's idle
local routing entry is pruned (session_store_max_age_days), recovery is what
mounts it after a restart, and when another chat was then handed off to the
lane the moved chat stayed writable. Every mount of a messaging route now
goes through `session_route_recovery.mount`, which reads `handed_off` from
the row (`moved_local`).
- `_publish_route` moved a mount's source and route only for a local ->
platform move. A chat a `/resume` brought to another chat kept its old
route, so a viewer's turn or Stop through it acted on the conversation that
replaced it there. A mount published on a different route now moves there.
Not routed through `mount`: the local receipt restore (a LOCAL source is never
a moved chat), the hosted-room bind and the two exhaustion-rotation mounts (rows
minted by that platform path), the source-less history mount.
Independent review of ff6409bf1ba: Bot Chat X, a local reset to X2, then /handoff of X.
follow_hand_off flagged the Desktop's mount of X as handed off, but the X2 head it then
published was a plain LiveSession (handed_off and history_mount both false). When another
chat was handed off to the same lane, or the platform's /new and its next inbound published a
new conversation, the takeover supersede loop skipped X2: it still authorized session:submit
and session:control, and the turn resolver ran its turns in the new conversation.
The same hole existed without a reset whenever the head was mounted by anything other than
the Desktop that followed it: the lane's inbound after a restart (a fresh LiveSession), or a
viewer's cold open (a history mount the next inbound clears).
Handoff provenance now comes from the row, not from the path that mounts it:
session_route_recovery.moved_local (a row created by a local surface, published on a
messaging route) sets LiveSession.handed_off on every newly constructed mount, in
SessionAuthority._publish_route and in restore_routed_session. Reads stay open; submit and
control answer session_superseded once the lane takes up another conversation.
Review finding on #144708 (independent review)
Refs #143286
Round-3 re-review of #144708:
- A lane's /new frees a handed-off Bot Chat's title from the store's worker
thread, outside the titled-create lock. A Bot delivery it landed in the
middle of (after the title read, or under the lock before its own displace
write) re-read an empty title and crashed on row['id'], so the delivery
failed and nothing was minted. _create_bot_chat now mints when its displace
write lost to that rename and the title is free; _target mints when its
re-read finds the title gone.
- The platform's /resume of a local conversation resolved its title to the
first segment, so after a local /new it moved that stale segment onto the
lane and released the receipt of the live segment, which stayed local and
was refused storage_unavailable after a restart. It now resolves the
conversation's current head through the same conversation_head step
/handoff uses, and a Desktop still viewing it follows it onto the lane.
Greptile on df9746d5f09:
- A chat handed off to a route again after another one took it (H, then Y,
then H) skipped the supersede loop because it was already on that route, so
Y stayed writable while the route named H. _publish_route now treats a
superseded session coming back as a takeover too.
- The Bot Chat a delivery minted under its automatic creation id, reset
locally and handed off, leaves its first segment untitled on the lane once
the name follows the head. _bot_chat_creation_id read that untitled row as an
interrupted creation and reused its id after the platform's /new, so the next
delivery hit the released receipt (storage_unavailable). A creation handed
off to a messaging lane is spent.
- A moved receipt whose lineage holds a non-string entry raised TypeError in
the boot heal and stopped the profile's start; hand_off_conversations now
leaves such a receipt as it is (its own reads refuse it, logged).
- The title follow-the-head rule ranks provenance with SessionDB._title_rank
instead of a copy (a pre-provenance title ranks as the user's).
Three defects an independent re-review found on #144708 (#143286):
- Data safety. After a local /new (receipt now names the new segment), a
/handoff of the chat's id moved the stale first segment to the lane and the
owner-row branch of moved_receipts released the receipt guarding the
post-/new segment, which was still local: that transcript was reachable from
nowhere and the platform resumed the stale history. The handoff now repoints
the receipt's current head (latest /new segment, then its compression tip,
hermes_state_local.handoff_head); a receipt is released only when the row its
CURRENT target names moved to a messaging lane, and the rest of that
conversation moves with it in the same write (hand_off_conversations): its
earlier segments open as the lane's read-only history and its name follows
the head over a generated title. A Desktop still holding the chat follows it
(follow_hand_off).
- Bot Chat after the platform's /new. The roster's canonical_session (raw title
registry) still named the superseded handed-off chat while the title lookup
hid it, so Desktop's bot row failed closed ("Could not confirm ... not
starting a new chat") on every click. The title now moves aside at the moment
the lane moves off the chat (SessionStore._promote_session_reset, every route
transition, and at boot), so the registry, the roster and the lookup agree
at every step; the lookup-side filter and the create-side move-aside go.
- Second handoff to the same home chat. A live local chat handed off to a route
that already carried a handed-off chat skipped the supersede loop, leaving
the first chat writable while the route named the second. _publish_route now
supersedes every other handed-off/history mount on a route whenever a
session takes it over.
The PR's own history='reset' case asserted the old (stale-segment) behaviour;
it now asserts the head carries on and the earlier segment is read-only history.
Desktop's titled create renamed and unhid the handed-off Bot Chat before staging the new one,
so a refused create (an invalid cwd) left the saved chat renamed with nothing minted in its
place (Greptile, #144708). The rename now runs after the replacement commits, right before it
takes the title.
Matching a receipt by its owner row joined receipts whose value is not JSON, and the route
comparison then raised `malformed JSON` out of every repoint and boot heal, where it used to skip
them (tests/gateway/test_local_recovery_pre_profile_origin.py). Only well-formed receipts are
compared; a malformed one stays as it is, refused and logged when touched.
Review of #144708 (Bot Mode keeps one canonical forever-chat per profile + title):
- A Bot/cron delivery after the handoff treated the handed-off Bot Chat as displaceable
(a messaging row with no receipt), renamed it aside and minted a new canonical chat, so
the bot row silently switched chats. `displaceable_bot_chat` keeps a handed-off chat whose
lane still holds it, and `_target` delivers to it on that lane.
- The Desktop view that followed the chat onto the lane stayed writable after the platform's
own /new, with a route naming the new conversation (Stop, agent lookup and turns would act
on it). A handed-off live session is now superseded like a history mount when its lane
publishes another conversation.
- After that /new and a restart the bot row adopted the read-only chat forever. The title
lookup no longer resolves a handed-off Bot Chat whose lane moved on, and Desktop's titled
create (and the delivery door) move it aside, `Bot Chat (<platform> <id>)`, before minting
a fresh one.
- The in-handoff receipt release applies the startup heal's messaging-lane filter.
Also: a chat with a /new reset segment handed off by its conversation id (a platform
`/resume <title>` or a resumed CLI) kept a receipt naming the reset segment while its owner
row moved to the lane, so every cold open failed the lineage check. A receipt whose owner
row a repoint moved is released too (in the write and by the startup heal).
The reporter of #143286 runs a named bot profile under a multiplexed gateway. Drive that
topology with a real GatewayRunner serving default + `bot`: the Bot Chat lives in the bot
profile's own state.db, the handoff watcher's `_process_handoff(row, 'bot')` runs in that
profile's scope and keys the lane `agent:bot:...`, and after a cold boot the browser Webapp
finds it by title and may control it; the default profile's store never sees the row.
Red on origin/main (`stored identity mismatch` on the still-open Desktop view; the title
lookup answers `not_found` after a restart), green with the fix.
A compressed local chat's receipt names its newest compression child.
The handoff repointed the route at the logical id, so the rewrite walked
only that row and its ancestors, missed the receipt, and left the tip on
its local identity (the next turn's tip walk then broke it the old way).
Resolve the compression tip before repointing: the whole chain moves onto
the lane with its receipt, the handoff carries the live transcript, and the
still-open viewer is published by its logical owner.
The repoint moves into GatewayStartupMixin._handoff_repoint; the
regression test drives it on the real store, plain and compressed.
Greptile review on #144708.
`/handoff <platform>` repoints the platform route at the local session
(SessionStore.switch_session -> record_gateway_session_peer), which rewrites
the row's routing identity to the platform's. Its local receipt
(gateway.local_policy.v1 / gateway.local_legacy.v1) stayed behind, so the row
matched neither lane:
- authorize took restore_local_session (receipt present) and _checked_entry
refused it with `stored identity mismatch`: resume, pin/unpin, rename and
archive answered storage_unavailable, and every boot logged
`receipt refused`.
- the title registry (session.list title=, session.resume title=, the Bots
roster's Bot Chat lookup) sent the row to legacy adoption, which answers
not_found for a messaging row and for any non-native actor (browser
Webapp), so a handed-off canonical Bot Chat could never be opened again.
Now:
- the explicit repoint retires a local receipt whose target row it moved to
another route, in the same write (hermes_state_gateway ancestors mode);
- gateway boot heals stores an older build left that way: receipts whose
target row a messaging lane now holds are released (logged once);
- the title resolver returns a local conversation a messaging lane took over
(created by a local surface, mountable on its route) through the routed
mount, so `/resume <title>` and the Bot Chat registry open it there;
- the handoff publishes the lane on the authority, and _publish_route moves a
still-open LOCAL live entry onto it, so the Desktop that ran /handoff keeps
its view without a restart;
- resume refreshes the creation policy only for local routes (a platform
lane has none; it logged `keeps its stored policy` on every open).
Fixes#143286
test_takeover_revokes_waiting_workers_and_waits_for_an_admitted_write[True]
writes OLD then NEW to a pseudo-terminal's master and then read the slave
once. A write to the master returns before the slave can read it: Linux
hands the bytes to the slave's line discipline later, from a kernel worker,
so under load the first read returns b"OLD\n" with NEW still on its way.
An instrumented copy recorded both master writes returning in order, the
read seeing only OLD, and NEW arriving ~0.03-0.5 ms later (CI: #144804,
run 40274889783).
The test now writes an END marker to the master once every sink write has
finished and reads until it arrives. PTY delivery is ordered, so the marker
comes after everything the session wrote, and the assertion compares the
complete stream. That is also stricter: a revoked worker writing late (the
production race the [False] case guards) now shows up as NEW, OLD, END,
which the single read could miss.
Repro: a relay between the real slave and the test that delivers each line
0.5 s after the one before fails base on every [True] run (20/20) and passes
the fix (0/20).
Pinned to hermes-official-plugins 77b179d9a4a (PR #181 merge): the sound effects tool, the
bundled sound-design skill (ffmpeg post-processing incl. mono and game variation packs) and the
Desktop sound card + library tab.
Follow-up to #144850. flush_before_hard_exit drained only the per-profile
runtime stores, so a deferred write to the root (default) profile's store,
where the host-level active-profiles accumulator lives and which no runtime
owns, was still lost at every graceful os._exit (gateway, compute host,
kanban worker, slash worker).
Each SharedMetricsStore now registers itself in a process-level list the
first time it defers a write (next to its atexit drain), and the bounded
exit flush drains that whole list within the same budget. Each deferred
write runs against the store that deferred it, so no profile's rows land
in another's database; collection off creates no store, so nothing is
registered.
Every launchctl call (finished-job cleanup, bootstrap, the job's poll and its pgrep) now gets
min(cap, time left before the caller's deadline), so a slow or wedged launchctl can no longer
hold a client past the startup timeout it asked for. The update-time watcher has no deadline
and keeps the fixed caps.
macOS Local Network Privacy attributes a process to its responsible app. The gateway that
`hermes gateway ensure` (Desktop's launch path, CLI clients) and the update-time restart
watcher start is a plain detached child, so it is attributed to whichever app started it.
The gateway outlives that app by design; once the app quits, macOS holds every surviving
descendant responsible for itself, and a bare venv Python is denied: every connect to a
model server on another machine fails with `[Errno 65] No route to host`, even though Hermes
is allowed in Local Network settings. Reported in Discord: Test and chat against a LAN Ollama
broke on every Desktop quit/reopen and after in-app updates, and worked only with a gateway
run from Terminal or the installed launchd service (which already runs through osascript,
#71206).
On macOS both spawn sites now start the gateway as a one-off launchd job through the same
osascript JXA wrapper the installed service uses, so osascript (an exempt Apple platform
binary) stays the responsible code for the gateway's whole life. Nothing is installed: the
plist is loaded once and deleted, it never lives in ~/Library/LaunchAgents, has no KeepAlive
and is labelled outside ai.hermes.* so no fleet scan or restart route mistakes it for a
service; the gateway sees XPC_SERVICE_NAME=0 and still reads as unsupervised. When launchd
refuses the job (no GUI session, e.g. over SSH) the old plain-child spawn runs instead.
Review finding 4 on #144850. A store another process held past the 250 ms
busy timeout deferred the exit rows, then the export's drain_deferred dropped
them on the same contention, and os._exit skipped the store's atexit drain.
drain_deferred now re-queues on a busy/locked store (the _write_or_defer
promise), and flush_before_hard_exit drains every live store within the
remaining budget, so contention that clears inside the 3 s bound no longer
loses the open sessions' rows.
Review finding 3 on #144850. The flush (up to 3 s with a managed call wedged
in flight) ran before _mark_exited, so a supervisor kill during it recorded a
graceful stop as an unclean death. Order is now release locks, stamp the clean
exit, flush metrics, drain logs; the early-exit paths keep releasing the
PID/lock first. The wedge test now drives the real exit funnel with a real
Relay managed call in flight and asserts both the bound and the ordering.
Review findings 1 (critical) and 2 on #144850, plus the send-pass sibling.
A multiplex gateway or compute host exits from the launch profile's scope, and
the os._exit flush (like main's atexit hook, which has the same flaw) shut down
every profile's runtime under it: profile B (send: false) got the launch
profile's send=True reconciled into its store, an open-ended consent window and
a send pass over its outbox; a profile that turned collection off since its last
turn still had its open sessions written at exit.
- _Runtime.shutdown binds its own profile home (host.profile_key) for every
caller (atexit, shutdown_runtimes, tests) and checks enabled() inside that
scope first: a revoked runtime is deactivated, dropping its data unexported.
- _flush_and_export exports under the owner's scope, whoever triggered it.
- The send pass thread inherits that scope (spawn_context_thread): on main it
ran on a bare Thread, so still_consented() re-checked the LAUNCH profile's
consent (B's pass read A's send:false; a B revoke would not stop its pass).
- Tests: wait_until replaces the sleep-poll; one three-profile invariant test
through the real gateway exit funnel.
The same skipped-atexit loss on the other graceful hard exits: the compute host's
shutdown frame and orphan exit (both via close()), a SIGTERMed kanban worker (whose
normal one-shot exit already runs shutdown_runtimes) and an orphaned slash worker
each now run the bounded flush_before_hard_exit before os._exit. One test file covers
every path with a real NeMo Relay runtime and an open session; red on main.
Every graceful gateway stop (SIGTERM, update restart, idle exit, --replace takeover)
funnels through _exit_after_graceful_shutdown, which ends in os._exit and so skips the
shared-metrics runtimes' atexit hook. Sessions still open at the stop (a messaging or
Desktop chat) never closed, so their hermes.session.count / context-peak / tool-usage
rows never reached the local store.
shared_metrics_process.flush_before_hard_exit runs that hook's work (shutdown_runtimes)
on a daemon thread joined for at most 3 s, never raises, and never imports the runtime
module, so with collection off nothing is created. The gateway runs it first, before
the PID/lock release.
Review follow-up. An id entered _rows_pending before a save that can raise, and the fresh-route
path only cleared it inside _finish_route_transition, so a failed save left the id pending for
good and _is_session_ended_in_db would never call that row ended again. Every publisher now
clears its own id in a finally (get_or_create's create path, reset_session, switch_session),
_replace_route_locked marks the id only after its save returns, and the create kwargs are
computed before the entry is published. The test uses tests.waiting.async_wait_until.
SessionStore publishes a route transition in two halves: the new entry under _lock, then the
state.db row (insert for /new and a fresh route, reopen for /resume) after releasing it. Anything
that resolved the route in between saw the named conversation's row missing or still ended and
took the #54878 stale-route self-heal: drop the entry and recover the newest open row, which is
the conversation the user just left. A Desktop/Webapp turn finishing on a chat's previous
conversation does exactly that (its post-turn bookkeeping calls get_or_create_session), so after
/new the chat's route named A again and its next message could land there.
The store now remembers the ids it published whose row is not live yet, and
_is_session_ended_in_db does not call those ended. Every reader of that check (the routing
self-heal, the reaped-run eviction, the cached-agent liveness check) gets the fix.
Greptile on 815681917c3:
- a boundary landing after the pass published (and cleared its pending token) but before the
waiting compaction read the summary still read as a failed pass; take() now notices a discard
that happened during its wait and leaves the boundary's row as the only one;
- compaction_failed was lost when a pre-commit cancel restored the compressor's attempt telemetry
(and its prepared_summary flag) before the record was emitted; the spliced attempt id now lives
on the prepared-compaction state, outside the restored snapshot;
- the deadline-extension test starts its 2 s deadline only once the pass is running.
Re-review findings:
- a summary take() handed to compress() that the compaction then aborted or no-opped before
adopt() spliced it was counted nowhere; it is now counted once at the next touch as
discarded/compaction_aborted;
- a session boundary landing after the pass published but before the waiting compaction woke
read as a failed pass (a second discarded row); the wake now reads the entry and the cancel flag
under one lock, so the boundary's row is the only one.
Live runs (flag on and off) show the next compaction retries the configured summary model before
falling back again, so the comment no longer claims the fallback is sticky for the session.
Independent review of the telemetry found:
- a session boundary that woke a waiting compaction was counted twice (the boundary's discard
plus the woken wait's 'discarded_while_waiting'); the boundary's row is now the only one and the
dead reason code is dropped from the vocabulary;
- 'used' was recorded in take(), before a stale-attempt abort could still drop the summary; it is
now recorded by adopt(), the splice itself, with the wait carried on the entry;
- the cancel reason is set under the lock, so a pass finishing in that gap reports the real reason;
- skipped_busy counts once per refused window instead of every tool batch.
The live matrix on real processes found two ways a prepared-compaction pass was paid for and never
usable:
- `hermes chat -q` / `-z` runs end with their turn (the gateway evicts the agent right after the
reply), so a turn-end pass there was always cancelled ~0.5 s later. One-shot sessions now join
cron runs and delegated children in skipping turn-end passes.
- Releasing an agent without a session boundary (gateway cache eviction, a Desktop rewind,
managed-worker retire, close()) left a running pass billed to completion. release_clients() and
close() now discard it (shared-metrics reason `agent_released`).
Pass log lines now carry the session id, so concurrent sessions in one gateway are attributable.
Wire hermes.prepared_compaction.count into the prepared-compaction state
machine so a profile that opts in to shared metrics reports whether its
passes pay off, and if not, why:
- pass side (pass thread): pass_started / skipped_busy (process slot
cap) / pass_completed (summary published) / pass_failed (classified
as an inline attempt would be, terminal summary failure first) /
pass_cancelled (reason set by whoever cancelled: superseded,
session_boundary, wait_timeout, host_cancelled) / internal_error.
- compaction side: used (wait bucket, aux_fallback_main when the pass
fell back to the main model), discarded (closed miss code + wait),
auto_disabled (first memory-provider compaction), and compaction_failed
from the attempt telemetry when a spliced compaction did not commit.
Splice-miss reasons become a str subclass carrying a closed code, so the
log text is unchanged. The pass thread already copies contextvars, so its
rows land in the owning profile.
Opt-in shared metrics could not say whether anyone turns on
compression.prepare_ahead, or whether the passes it starts pay off.
- hermes.install.snapshot gains compression_prepare_ahead (on/off), read
from the profile config with the agent's own truthiness. Pre-upgrade
snapshot rows keep a legacy field set and the v4 schema field is
optional, so they still validate and package.
- New closed-enum counter hermes.prepared_compaction.count (mark name =
metric name, projected by the decision-mark table): event, origin,
reason, failure_class (compression vocabulary), wait_bucket.
record_prepared_compaction() takes raw values, normalises inside, is a
no-op unless collection is on and never raises.
- Vocabularies live in shared_metrics_prepared_compaction.py (the
contract sits at the 2,000-line cap); the contract imports it.
- v4 schema extended in place (additive), docs row + snapshot bullet,
relay smoke asserts the snapshot field.
Review follow-up (Greptile on 41ac847b3ee): caller-supplied conversation_history
that is never persisted carries no message_uid, so the structure-only check
accepted a dropped user/assistant pair once the transcript regrew to the same
shape, and the old summary replaced messages it never summarized.
Each message identity now also carries a text digest. A message whose uid is
present on both sides is still matched by uid alone, so plugin text rewrites
of persisted messages keep the summary; a message without a uid on one side
is matched by text.
Maintainer ruling on the prepared-compaction design, plus the review findings
(ours and Greptile's) on #144684:
- One pass per compaction. The extension passes (a new summary call every ~8k
tokens of growth) and the "stale after one tail budget" check are gone: the
summary splices at its own boundary and every message after it, however
many, stays verbatim in the tail. No second summary is ever paid for.
- The splice check compares message structure, not text: role, tool call
ids/names, tool result ids, plus the durable message_uid where both sides
carry one. Plugins that rewrite text in place (redaction, pre-compaction
notes) no longer throw the summary away; a dropped/inserted message or a
rewind into the window (new message_uids) still misses. This also settles
the fingerprint gap on input_text/output_text parts: text is not compared.
- A pass that recovered on the main model after the summary model failed now
hands that over on a splice, as an inline call would: summary_model goes
sticky to the main model, and the "configured compression model failed"
warning reaches the user. The attempt telemetry carries the pass's aux call
(model, tokens, duration, fallback) and a new prepared_summary flag.
- The first compaction that needs memory-provider context stops further
passes for the agent; each would have been discarded and paid twice.
- Turn-end passes are skipped for cron runs and delegated children, whose
sessions end with the turn.
Round-two review of the cancel and fast-path commits:
- A cancelled pass's stream raises AuxiliaryExplicitCancellation, a
BaseException that _generate_summary and _run_pass did not catch, so
every superseded pass printed "Exception in thread compaction-prepare"
to stderr (seen in the live probe). _run_pass now catches it and logs
the cancel.
- take() snapshotted the candidate at its start; a pass that published
while compaction was checking that snapshot survived the compaction.
It never spliced (its fingerprint no longer matched), but the
prompt-size skip could then keep it and start no pass for a cycle.
take() now clears every candidate on exit.
- The prompt-size skip measured growth since the plan, so once the
prompt grew past the threshold while the window gap stayed small,
every tool batch re-planned again. Keeping a candidate after a full
check now re-arms the skip from that check.
Tests: one stub models the aux client ending a cancelled stream, the
local wait helper gives way to tests.waiting.wait_until, and the
skipped small-window-delta path gets its own test. Docs and the config
comment name the discard case.
While a completed candidate was still fresh, maybe_prepare re-planned the
whole compaction window and re-fingerprinted the transcript before every
tool batch and at every turn end, only to keep the candidate. Measured on
the turn thread: ~23 ms per call at 180k tokens, ~31 ms at 540k.
The candidate now remembers the provider-reported prompt size its pass
was planned at; while the prompt has grown by less than the extension
threshold since then, the hook returns at once. A shrunk prompt or a
larger growth falls through to the full plan, and compaction still runs
the authoritative fingerprint and gap check.
Passes are also skipped while the summary model is benched after two
fallback summaries in a row: inline compaction would not call it then
either (#63008), and each pass would be a paid call.
A pass that compaction or a session boundary superseded kept streaming
its summary to the provider's own ceiling: billed, then thrown away. The
same happened when a waiting compaction gave up at the host deadline or
on a host cancel, so the summary was paid for twice (once in the
background, once inline). Live probe: after the deadline cut a wait
short, the pass still completed 4 s later and was billed (2 aux calls).
Each pass now carries a cancel event, wired into the existing aux
hard-cancel path (aux_interrupt_protection(cancel_event=...)) so the
stream is closed. take() sets it whenever it consumes or supersedes the
pass; discard() sets it on reset, end and rebind. A compaction waiting
on a discarded pass wakes at once instead of waiting out the pass.
Also:
- the waiting compaction installs its progress hooks before the rest of
the setup; resolving the aux timeout loads config on first use
(~0.8 s), and progress streamed in that window was dropped;
- a miss now says why: deadline, host cancel, discarded, or the pass
failed (every miss used to read "did not finish in time");
- the config comment mentions the wait.
Probe after the fix: deadline miss -> the pass exits ~0.1 s later, 1
aux call billed; reset during the wait -> the wait returns at once.
Inline, a llama.cpp server reading a long summary prompt pushes the host's
compression deadline back (compression_deadline_scope installs a
prompt_reading_hook that extends the fence). A compaction waiting on a
prepared pass lost that: the pass thread had no reading hook, and the wait
read the deadline once. A slow-but-healthy local prefill could hit the
original ceiling and fall back to an inline summary that pays the same
prefill again.
The pass now installs a reading hook that forwards to the waiting
compaction's hook, and the wait re-reads the live host deadline on every
poll.
Rebasing #120603 onto main lost a line in the conflict resolution: main's
summary success path (86bf8886c2 / 4f134d4c37) resets the sustained-overload
budget, but the extracted _record_summary_success() did not. Two overload
aborts, then a healthy summary whose compaction is refused (would-grow), left
the strikes armed, so one isolated overload later escalated straight to the
destructive fallback. The helper now clears terminal failures with the shared
helper and restarts the overload budget, exactly as main does.
The overload tests move to tests/agent/test_context_compressor_overload.py
(unchanged) so the new case fits the test-file size cap.
When automatic compaction fires with no completed candidate but a pass is
still writing a summary whose window fits, compaction waited for nothing and
wrote the same summary again inline, so the user paid the full summary
latency and the background call was thrown away.
Compaction now waits for that pass. Each streamed chunk of the pass touches
the compacting thread's aux progress hook, so the host's idle watchdog treats
the wait like an inline summary stream. The wait ends at the host's stream
deadline (the compression aux timeout when none is installed) or when the
host cancels, and compaction then summarizes inline as before. A running pass
whose window no longer fits is not waited on. Annotations in the module and
its tests move to builtin generics.
kshitijk4poor