0 Fork 0

mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-11-06 05:17:15 +00:00

Files

T

Siddharth Balyan 334f5c13b4 feat(auth): hand the free-tier guest to the account after sign-in (#143234 )

* feat(auth): hand the free tier's apps to the account that replaces it

When a Nous account is saved over a free-tier identity, Hermes now moves
that identity's connected apps into the account with one call to the Nous
Portal claim endpoint, authorized by the account's own token. Nothing asks
first; the result (account email, what moved, what needs reconnecting) is
shown afterwards in `hermes auth status nous` and exposed for the desktop.

Sign-in code does not know about it. A hook at the two lowest write seams
(`_store_provider_state`, `_write_shared_nous_state`) writes a pending
record under the existing locks and does no network. The claim runs
outside every lock: after a shared-store write, when a `deferred()` block
exits (the desktop poller and `run_sign_in` save under their own lock),
after a free-tier reconcile, and on each Nous keepalive tick, which
resumes retries from an earlier process.

Records sit beside the shared Nous store, keyed by a hash of the free-tier
credential and bound to the destination account, so profiles sharing one
free-tier identity share one record and one claim. A claim lock spans
read, claim and write. A Claimed record is never reopened; a lost success
followed by an unknown-credential answer reads as "probably moved". The
account token is refreshed after the move only when the cohort carried.
The same path runs `settle_after_upgrade` once per replacement, so every
replacing writer settles the config.

The legacy promotion sign-in already moves the apps on the server, so it
records the handoff as done and sends no claim.

* fix(auth): the free-tier handoff only ever claims into its bound account

Review round 1 on the guest handoff.

P1, Bearer not proven to be the destination: the runner checked the
profile's account against the record's destination, then resolved a
Bearer separately. A sign-in landing between the two could move the apps
into that other account and mark the record Claimed. The Bearer is now
resolved first and its JWT subject is compared to the record's
destination inside the reservation, so only a token proven to belong to
the destination is ever sent.

P1, stale reservation: the destination check and the attempt increment
were separate record reads, so a sibling could rebind the destination in
between. `_reserve` now does one locked read-modify-write: still active,
due (`next_attempt_at`, so a waiting runner honours a Retry-After another
runner saved), under `MAX_ATTEMPTS` (a crash after the last send now
closes the record as Failed instead of sending again), and bound to the
Bearer's subject. A record without a destination subject is never written.

P2, config completion in the runner: `drain` no longer runs
`settle_after_upgrade`. Settling stays with the sign-in that saved the
account (run_sign_in, the desktop poller), as before this PR. This
deletes the thread-local queue and the batch's settle fallback.

P2, keepalive: a handoff failure is logged and token renewal carries on.
An unparseable records file is moved aside to .corrupt on the next write
instead of blocking every Nous account save.

P2, budget: the remaining sync budget is checked before each record,
token resolve and request, and is passed as their timeout, so a refusal
at the edge starts no refresh or second claim.

P2, status: an active move bound to an account the profile no longer
holds reads as paused, with copy that says so.

* fix(auth): free-tier handoff retries reach every profile and every long-lived process

Review round 2 on the guest handoff.

P1, queued moves had no worker: retries rode the token keepalive, which
does not start in a process that boots with a guest (a Desktop backend
before sign-in). A move left by a CLI sign-in during an outage then
waited for a restart. Retries now run on a dedicated worker that
`start_nous_auth_keepalive` starts in every long-lived Nous process,
guest or not; the keepalive tick no longer drains.

P1, named profiles never retried: the keepalive drained only the launch
profile's records. Each worker tick now runs every due record under its
owning profile's home and secret scope (the way the cron ticker binds a
profile), so a named profile's move retries from a process launched as
another profile. The bound-account check and the Bearer proof in the
reservation still decide whether anything is sent.

P2, cached guest token: the resolver's 5 s memo could still hold the
guest's token from a connector call just before the save; the
reservation then refused its subject and the move waited for a later
run. The handoff now drops that memo before reading the account token.

P2, budget through locks: the token read waited the resolver's own
15 s floors for the auth and shared locks. The handoff now takes both
locks itself within the remaining budget (same order), which makes the
resolver's acquisitions reentrant, and re-checks the budget before
reserving or sending.

* fix(auth): free-tier handoff never gives up on a transient failure before the reap

Compatibility audit, items B1, B4 and B5.

B1: a handoff that reached Failed was never retried, so the free tier's
apps were lost when the idle free-tier identity was reaped after 14
days. Transient failures (5xx, the 503 breaker with no Retry-After,
timeouts, busy, unknown codes) became final after 8 attempts, about 81
minutes. Now only codes no later attempt can change are final
(client_not_allowed, account_locked, invalid_request). Every other
outcome retries, the slowest rung (1 h) repeating, until 13 days after
the record was made. The attempt reservation then closes it as Failed
`retry_window_ended`. One predicate (`_retryable`) picks the work for
every run, so a record an older build failed on a transient code re-arms
on the next sign-in, `hermes auth status nous`, `hermes auth upgrade`, or
retry-worker tick, with no new state. The attempt cap is deleted.

B4: failure copy printed raw server codes. Final codes map to a plain
sentence, and anything else shows only the generic sentence.

B5: a claim answer without a `connectors` field (an older service) read
as "There were no free-tier apps to move", and so did Gone on a first
attempt, which follows a released ticket that already moved the apps.
The first now reads as unknown with neutral copy, and the second as
"Nothing left to move from the free tier."

* fix(auth): a free tier with no apps reads as nothing moved after sign-in

A free tier that only read its connectors gets a vendor id row on the
service, so the claim answers kind `moved` (or `swapped`) with an empty
`moved_toolkits`. The status copy keyed on the kind and printed "Your
apps moved to <email>." for an account that received nothing.

One rule: an empty moved list reads as nothing moved, for moved and
swapped alike. The service now refuses a move it could not read, so an
empty list is the truth. The promotion sign-in's record, which reports
no list at all, now carries kind `unknown` and keeps its neutral copy
instead of falling under the rule.

2026-10-23 06:08:24 +05:30

17 KiB

Raw Blame History

sidebar_position, title, description

sidebar_position	title	description
15	CLI Internals	How hermes_cli is shaped: slash dispatch, config loaders, the skin engine, the transactional update pipeline, and process-identity rules

CLI Internals

Companion to hermes_cli/AGENTS.md (the rules) — this page holds the longer explanations.

Update pipeline

User-facing behaviour (receipts, --plan, snapshot modes) is in Updating; hermes_cli/AGENTS.md carries the short form of the rules.

Fleet-update campaign #91277 (Aug 2026). A PR that weakens a stage must answer for the failure class it guards. plan → snapshot → apply → restart-per-kind → verify → report

Plan (update_inventory.py, hermes update --plan): read-only inventory — install kind, all profiles, every live gateway with supervisor + running code version. Deployment kinds are first-class: git updates in place; docker/nix/apt are NOT in-place-updatable and the updater reports the correct external command instead of fighting the deployment model.
Snapshot (backup.py): pre-update quick snapshot for EVERY profile (the code swap + fleet restart touch all of them), each into its own state-snapshots/, identical file set, 1 GiB per-file cap, keep=1. Never add a partial/tiered snapshot set — mixed coverage creates torn-restore states across schema generations. Quick snapshots are FILE-LOSS RECOVERY (the per-profile cron-jobs safety net restores from them), NOT code-rollback insurance; --backup full mode owns rollback.
Apply: git pull, or the Windows ZIP fallback — which fires ONLY when git itself failed (_should_zip_fallback_on_update_error, argv-classified; a dependency-install failure must never trigger a tree-clobbering re-download), REFUSES a dirty working tree (-uall + a pre-swap TOCTOU re-check — but classifies a !! line by whether the swap would destroy it: an ignored path under a root entry the ZIP does not ship (.bytecode-fingerprint, .hermes-bootstrap-complete, hermes_agent.egg-info/; tracked root entries stand in for the ZIP set before the download, the re-check gets the real one), a nested __pycache__/node_modules, or a _ZIP_PRESERVED_NESTED output is admitted; other ignored files under shipped dirs still block), and grafts the live nested build outputs (_ZIP_PRESERVED_NESTED: apps/desktop/{release,dist,node_modules,build}, hermes_cli/web_dist, ui-tui/{dist,node_modules,packages/hermes-ink/dist}, web/node_modules, scripts/whatsapp-bridge/node_modules) into the staged swap by hardlink (the GitHub source ZIP has none of them; without the graft the swap deletes them). Post-swap, the Desktop rebuild decision also trusts the build stamp under HERMES_HOME, so an install that already lost its artifacts in an earlier update is rebuilt instead of "forgotten" (#90495).
Restart-per-kind: systemd and launchd restarts are FLEET-WIDE within the updating install (every hermes-gateway* unit / ai.hermes.gateway* LaunchAgent whose home is the updating root or one of its profiles/<name>), drain-first (SIGUSR1), with per-unit/per-label failure isolation. Restarting only the invoking profile's service leaves siblings on stale sys.modules until they crash — the largest dupe-PR cluster in the repo's history came from that bug. The fleet is bounded by HOME, not by namespace: hermes_cli/update_fleet_scope.py judges every unit/label/process by the home it actually runs on (live environ, unit Environment=, plist HERMES_HOME), and a runtime of another HERMES_HOME on the same account — a sibling install, the real hermes-gateway.service seen from a scratch home — is named and left alone, never restarted (#93349).
Verify: gateways stamp code_sha/code_version into gateway_state.json on every runtime-status write (gateway/status.py); the updater compares each live gateway against the fresh checkout and prints a fleet version matrix. After the commit point (the tree moved) nothing fails hermes update (exit 0 unless the code was rolled back): a provably-stale or stopped gateway, a failed build/launcher/config migration/bytecode sweep/channel adoption/Windows resume each print ⚠, land in the receipt as outcome: "success" with followups: [{step, reason}], and keep their own obligation armed (source-completion-pending for the tail, the fleet-restart obligation for gateways) so the next launch or hermes update — including "Already up to date" — retries them. The owed-fleet-restart CLI startup warning is what keeps a mixed-version fleet from ever looking healthy. A completion process that could not be started (temporary directory, request file, spawn) or died without a correlated result (OOM, SIGKILL) is the same kind of debt: an owed completion follow-up with the source-update tail re-armed, exit 0. Exit 2 (refused/concurrent) and exit 1 (nothing committed / rolled back) keep their meaning; the one committed run that exits 1 is an autostash whose restore conflicted and stays parked: the receipt is partial with a user_action, since nothing retries what only the user can re-apply (#122557).
Report: every run writes a machine-readable receipt to the ROOT home's logs/update_receipts/ (never a sticky profile's; update.log likewise) — latest.json pointer; steps, skips WITH reasons, restart outcome, plan, fleet snapshot, followups. The receipt is on disk as outcome: "running" from the start and refreshed at each stage boundary; the next update marks a running record whose processes are gone interrupted and says so. Readers (hermes logs update, debug bundle, dashboard status, pm receipts) use the root home too. A dependency-sync failure in the completion bootstrap is a dependencies follow-up (A6), and Ctrl-C after the commit point finalizes interrupted (exit 130), never failed. Before a source swap, the parent captures plan/snapshots/receipt and its Windows pause token. update_completion.py runs new-code PM preparation with site initialization disabled, then selected-Python builds, maintenance, scans/restarts and verification. Git/current/ZIP share this owner; never reload or purge modules to continue in the old interpreter. The parent keeps the lock, waits, and accepts only a correlated terminal result. cmd_update still finalizes early failures and missing/killed-child outcomes; PM refusal data survives the handoff. See website/docs/developer-guide/source-update-completion.md. A begun-but-unwritten receipt is a bug.
Nothing runs pulled code in the pre-pull interpreter. This tree finishes updates through update_completion.run_completion / _update_takeover.py. hermes_cli/update_handoff.py and hermes_cli/update_serve_obligations.py are the FROZEN COMPAT SURFACE for releases that lazily import those module names from the NEW tree after the checkout swap. Keep their public names importable and behavior-preserving. They call into _old_updater.stop_for_relaunch → _run_child. Removing a name bricks every release mid-update. see tests/compat/old_updater_surface.json.

Process-scan coordination between updater, serve/dashboard, and gateway is being replaced by a gateway-owned control socket (#92091); scans are the fallback layer for old/crashed processes — read #92091 before adding any heuristic. Process identity rules (never argv substrings; canonical matchers; parser-derived flag sets; never blanket-exclude gateway ancestors, #87594): root AGENTS.md and website/docs/developer-guide/cli-internals.md.

The systemd blunt-restart fallback waits for the unit's TimeoutStopUSec plus TimeoutStartUSec, with 15 seconds of client-side slack. It reads the target unit in the same manager scope as the restart; both the initial attempt and retry use this budget, including the catch-up restart after an interrupted update. A start after a graceful drain uses only the start budget plus slack. A missing, unparseable, or infinite phase limit falls back to 90 seconds for that phase, keeping unattended updates bounded. Timing out the systemctl client does not cancel the manager's transaction. Custom multi-command stop chains or EXTEND_TIMEOUT_USEC can still outlast this estimate; a real timeout remains an incomplete restart, and successful commands still require the existing service-health and fleet-version verification. Raw numeric *USec values are microseconds, while formatted values use systemd's fixed units, including days, weeks, months and years. The combined timeout is capped below the native signed 32-bit millisecond poll limit (with rounding headroom), so exceptionally long unit limits cannot overflow subprocess polling. Zero/unknown/infinite phase limits use the bounded fallback. This does not change active-turn drain settings.

Sign-in completion is one function, settle_after_upgrade, called by every caller that persists an account over a free-tier identity (CLI upgrade_guest, the desktop poller): it moves a config on the welcome route to the account's host and the tier's recommended default (models.recommended_nous_default_model, shared with GET /api/model/recommended-default).

The shared flow, states, and copy live in anon_sign_in.py; CLI rendering lives in anon_sign_in_cli.py. anon_auth.py keeps identity, promotion polling, and settlement, and re-exports the existing sign-in API. The flow resolves identity and persistence collaborators through anon_auth at call time to preserve module-attribute monkeypatch seams.

The sign-in itself is one composition: anon_auth.run_sign_in() yields SignInStates (Code, Waiting, Completed, Declined, Superseded, TimedOut, Retired, Failed, AlreadySignedIn, Unavailable). It reads the current state itself, holds one absolute deadline across both waits, persists only after a completed promotion and a token grant, runs settle_after_upgrade exactly once per completion, and never lets a persist or settle failure escape as an exception — it becomes Failed. Every state carries its own .copy (the chat form, which never contains a raw exception, a URL or a hermes verb) and .copy_terminal, so no caller maps a reason to a string. cancelled() stops an attempt; cancel_wins_after_promotion decides what happens when the server had already completed the transfer — the desktop keeps True (a DELETE means "not on this machine"), the gateway passes False (a supersede must not discard a transfer the user actually approved). scope is entered only around the precondition and persist blocks, never across a yield or a network wait, because run_in_executor does not carry contextvars. upgrade_guest (hermes auth upgrade), the CLI /login handler and the desktop promotion poller are renderers over it; a surface that needs the cancel check and the save to be atomic passes persist_guard. The desktop's plain "connect another Nous account" device-code login is a separate path (_nous_plain_poller) and must stay one.

When any writer saves a Nous account over a free-tier identity, hermes_cli/guest_handoff.py hands that identity's connected apps to the account: POST /api/anonymous/claim {"token": "<anon_...>"} with the account's own Bearer token. Nothing asks the user first; the result is shown afterwards (hermes auth status nous, guest_handoff.handoff_status() for the desktop). Sign-in code does not call it:

note_replacement runs at the two lowest write seams (auth._store_provider_state and auth_nous._write_shared_nous_state) under the caller's auth-store locks and only writes a pending record. Records live in guest_handoff.json beside the shared Nous store, keyed by a hash of the anon_ credential and bound to the destination account, so profiles sharing one free-tier identity share one record and one claim.
drain runs the claim outside every lock, bounded to about 10 s (each record, lock wait, token read and request counts against the budget): after a shared-store write, when a guest_handoff.deferred() block exits (a surface that saves under its own lock, such as the desktop poller, wraps the save in it), and after a free-tier reconcile. It only moves apps; config completion (settle_after_upgrade) stays with the sign-in that saved the account.
Retries left by any process are finished by a retry worker that every long-lived Nous process starts (start_nous_auth_keepalive starts it, guest or not). Each tick runs every due record under its owning profile's home and secrets, so a named profile's move retries from a process launched as another profile.
Every attempt is reserved in one locked read-modify-write that checks the record is still due, inside its retry window, and bound to the account whose Bearer is about to be sent (the token's sub must equal the record's destination). Apps never go to any other account.

Answers: claimed records what moved (and refreshes the account token only when cohort_carried; an answer without connectors reads as unknown, never "nothing moved"); unknown_token is Gone, read as "probably moved" after an earlier attempt (a lost success); already_retired is Gone too; a 401 refreshes the account token once. Only codes no later attempt can change are final (client_not_allowed, account_locked, invalid_request). Every other refusal, unknown codes included, and every transport failure retries with backoff (Retry-After honoured), the slowest rung repeating, until 13 days after the record was made: the service reaps an idle free-tier identity, apps included, after 14. A record an older build failed on a transient code re-arms on the next run (a sign-in, hermes auth status, hermes auth upgrade, the retry worker). A Claimed record is never reopened, and user copy never shows a server code.

Process identity: never infer it from argv substrings

The bug class behind ~10 fleet-update issues (#90778, #87594, #78089, #76129, #91964, ...): classifying a process by "serve" in cmdline or similar. kanban --preserve-cache contains "serve"; a flag VALUE can equal a subcommand (-m dashboard serve); truncated cmdlines hide the real subcommand. Rules:

Use the canonical matchers: gateway.status.looks_like_gateway_command_line (gateway run), hermes_cli.update_cmd._hermes_holder_subcommand (top-level subcommand of any Hermes argv). Never hand-roll token scans.
Flag sets must be DERIVED from the parser (_holder_value_flags() introspects build_top_level_parser()), never hand-written lists — they drift.
Never blanket-exclude ancestors from process scans: when /update runs as the gateway's child, a gateway ancestor must stay visible to the pause machinery (#87594). Exclude interactive ancestry, carve out gateway-shaped ancestors.
Match on FULL cmdlines; truncate only at display time (#78089).
Before adding any new scan heuristic, read #92091 — the gateway control socket replaces scans as the primary coordination mechanism; scans are the fallback layer for old/crashed processes.

Skin engine — what skins customize

Element	Skin key	Used by
Banner panel border / title / section headers / dim / body	`colors.banner_border`, `banner_title`, `banner_accent`, `banner_dim`, `banner_text`	`banner.py`
Response box border	`colors.response_border`	`cli.py`
Spinner faces (waiting / thinking)	`spinner.waiting_faces`, `spinner.thinking_faces`	`display.py`
Spinner verbs / wings (optional)	`spinner.thinking_verbs`, `spinner.wings`	`display.py`
Tool output prefix / per-tool emojis	`tool_prefix`, `tool_emojis`	`display.py` → `get_tool_emoji()`
Agent name / welcome / response label / prompt symbol	`branding.agent_name`, `welcome`, `response_label`, `prompt_symbol`	`banner.py`, `cli.py`

Built-in skins (_BUILTIN_SKINS in hermes_cli/skin_engine.py): default (classic gold/kawaii), ares (crimson/bronze with custom spinner wings), mono (grayscale), slate (cool blue). Add a built-in as a dict entry {"name", "description", "colors", "spinner", "branding", "tool_prefix"}. User skins are ~/.hermes/skins/<name>.yaml with the same keys, activated with /skin <name> or display.skin: <name>; the full YAML template is in the Skins & Themes user guide.

Profiles: multi-instance support

Hermes supports profiles — fully isolated instances, each with its own HERMES_HOME (config, API keys, memory, sessions, skills, gateway). For single-profile commands (hermes -p x <cmd>), _apply_profile_override() in hermes_cli/main.py sets HERMES_HOME before any module imports, so every get_hermes_home() reference scopes to the active profile. The multiplex gateway and the Desktop/dashboard serve backend serve several profiles from one process instead: the active profile is a contextvar override bound per activity, os.environ["HERMES_HOME"] stays the launch profile's, and a module-level constant derived from the home freezes to that launch profile (see Gateway Internals § Multiplexed profiles). Profile operations are HOME-anchored (_get_profiles_root() returns Path.home() / ".hermes" / "profiles", not get_hermes_home() / "profiles") so hermes -p coder profile list sees all profiles regardless of which one is active — intentional. Profile-safe coding rules are in the root AGENTS.md; multiplex secret-scope rules in gateway/AGENTS.md.

17 KiB Raw Blame History