docs(delegation): prefix_stagger_seconds is a cap on the warm-up wait, not a stagger

This commit is contained in:
Teknium committed 2026-11-04 21:50:30 -08:00
1 parent 17a8d1e201
commit c58e6de692
3 files changed
+4 -4

No files matched your search

+1 -1
View File
@@ -1357,7 +1357,7 @@ DEFAULT_CONFIG = {
"reasoning_effort": "",
# Max parallel children per batch AND concurrent background units (async beyond it runs sync). Floor 1.
"max_concurrent_children": 10,
"prefix_stagger_seconds": 0, # batch prompt-cache warm-up cap (s), 0 = off; see delegate_tool_prefix_gate
"prefix_stagger_seconds": 0, # max s siblings wait for the 1st child's 1st chunk (not a stagger); 0 = off
# Background fan-outs return as ONE message when the whole call finishes. true = each task
# (or `group`) returns on its own as it finishes — more new turns for the orchestrator.
"independent_completions": False,
+1 -1
View File
@@ -97,7 +97,7 @@ def _get_prefix_stagger_seconds() -> float:
its request reads the shared prompt cache (0 = off, the default). See tools/delegate_tool_prefix_gate.py."""
return _knob(
"prefix_stagger_seconds", None, lambda v: min(60.0, max(0.0, float(v))), 0.0,
"delegation.prefix_stagger_seconds=%r is not a number; the launch stagger is off",
"delegation.prefix_stagger_seconds=%r is not a number; the prompt-cache warm-up wait is off",
)
+2 -2
View File
@@ -3150,7 +3150,7 @@ delegation:
# provider:
# sort: throughput
max_concurrent_children: 3 # Parallel children per batch (floor 1, no ceiling). Also via DELEGATION_MAX_CONCURRENT_CHILDREN env var.
prefix_stagger_seconds: 0 # Prompt-cache warm-up for batches: siblings wait (at most N s, max 60) until the first child's response starts so they read its cached prefix. 0 = off.
prefix_stagger_seconds: 0 # Prompt-cache warm-up for batches: siblings start when the first child's response starts streaming; N (max 60) only caps that wait, it is not a fixed stagger. 0 = off.
worktree_isolation: false # Give each child its own git worktree branched from HEAD (local backend + git repos only; inspired by Muse Code). See Subagent Delegation → Worktree Isolation.
max_spawn_depth: 1 # Delegation tree depth cap (1-3, clamped). 1 = flat (default): parent spawns leaves that cannot delegate. 2 = orchestrator children can spawn leaf grandchildren. 3 = three levels.
orchestrator_enabled: true # Global kill switch. When false, role="orchestrator" is ignored and every child is forced to leaf regardless of max_spawn_depth.
@@ -3190,7 +3190,7 @@ The delegation provider uses the same credential resolution as CLI/gateway start
**Child process notifications:** background processes started by subagents route their completion/watch notifications to the parent conversation, but those are **suppressed** there by default — the child's consolidated result is the deliverable. Set `delegation.surface_child_process_notifications: true` to deliver them (with subagent attribution). Delegation results themselves are never suppressed. See [Subagent Delegation → Child background-process notifications](features/delegation.md#child-background-process-notifications).
**Prompt-cache warm-up for batches:** siblings in one `tasks` batch send the same tools and system-prompt scaffold; only the goal differs (and the end of the system prompt, when tasks carry different `context`). Started together, each child's first request misses the provider's prompt cache and pays the full prefix. With `prefix_stagger_seconds: N` the first child starts at once and the siblings on its provider/model wait until its first response starts streaming (by then its request has written the cache), until it finishes, or N seconds, whichever comes first. The cost is roughly one time-to-first-token added to the batch; routes Hermes does not prompt-cache (most non-Anthropic models) and siblings routed to a different model start without waiting. The leader is always the batch's first task: when task 0 is on an uncached or different route, the batch is not gated even if later tasks share a cached route, and a one-task unit (e.g. under `independent_completions`) never waits. Off (`0`) by default.
**Prompt-cache warm-up for batches:** siblings in one `tasks` batch send the same tools and system-prompt scaffold; only the goal differs (and the end of the system prompt, when tasks carry different `context`). Started together, each child's first request misses the provider's prompt cache and pays the full prefix. With `prefix_stagger_seconds: N` the first child starts at once and the siblings on its provider/model wait until its first response starts streaming (by then its request has written the cache), until it finishes, or N seconds, whichever comes first. N is a cap on that one wait, not a spacing between children: siblings normally start about one time-to-first-token after the leader, whatever N is. The cost is roughly one time-to-first-token added to the batch; routes Hermes does not prompt-cache (most non-Anthropic models) and siblings routed to a different model start without waiting. The leader is always the batch's first task: when task 0 is on an uncached or different route, the batch is not gated even if later tasks share a cached route, and a one-task unit (e.g. under `independent_completions`) never waits. Off (`0`) by default.
## Clarify