Files
hermes-agent/gateway/session_cron_commands.py
kshitijk4poor db481bfade fix(gateway): wait for /cron run on the runner's turn pool, not the pool its job needs
Symptom: with a one-thread default executor on the owner loop, a
/cron run sent on the classic CLI's command socket never answered;
the job never started and the cron layer paused it as unverified. The
default pool has min(32, cpu+4) threads, so on a small host a handful
of manual runs (Ctrl+C'd waits included, since the gateway side keeps
waiting) froze every other asyncio.to_thread caller on the owner:
session.create commits, admissions, reads.

Root cause: run_cron built every /cron reply through owner_lines,
which ran it under asyncio.to_thread. For `run`, that thread blocks in
run_canonical_job on run_coroutine_threadsafe(...).result() until the
job ends, while the job's admission (tracked_write) and execution
(session_cron.execute) take threads from the same default pool.

Change: owner_lines takes an optional executor hop; run_cron passes
the runner's _run_in_executor_with_context (the unbounded turn pool
that already carries long blocking work and keeps the session
contextvars). Runners without it keep asyncio.to_thread.

Test: two concurrent /cron runs on a one-thread default pool both
answer "Ran now: succeeded." (timed out before this change).

Refs #134889 (T1)
Refs #139158
2026-10-20 13:08:24 +05:30

30 lines
1.6 KiB
Python

"""``/cron`` on the shared gateway: the classic CLI's handler (hermes_cli/cli_cron_lines.py), run by the owner.
The profile's cron store belongs to the gateway whose ticker fires its jobs, so no client writes it. Bare
``/cron`` and ``list`` read, for every local session (main's TUI ran the same text in its slash worker). Every
other form writes the store: it runs here when the classic CLI sends it on its own command socket, and a
viewer's connection (Ink, Desktop, ACP) keeps main's refusal (gateway/session_commands.py).
"""
from __future__ import annotations
def cron_read(tokens):
"""``'overview'`` / ``'list'`` / ``'list-all'`` for the read forms of the shlex-split ``/cron`` argument;
None for a write."""
if not tokens:
return 'overview'
if tokens[0].lower() == 'list' and set(tokens[1:]) <= {'--all', '-a'}:
return 'list-all' if tokens[1:] else 'list'
return None
async def run_cron(authority, arg) -> dict:
"""``/cron <arg>`` off the owner loop in the owning profile: the store is written here, never by a client."""
from gateway.session_info_reads import owner_lines
from hermes_cli.cli_cron_lines import cron_command_lines
# A /cron run waits for its job, and the job's admission and execution take threads from the loop's
# default pool (tracked_write, session_cron.execute). The wait therefore holds a thread of the runner's
# unbounded turn pool, never one of the pool the job needs, however many runs (or Ctrl+C'd waits) pile up.
return await owner_lines(authority, lambda: cron_command_lines(arg),
getattr(authority.runner, '_run_in_executor_with_context', None))