Diagnose Stalled Hermes Background Subagents

A long-running subagent is not necessarily stalled. Hermes deliberately has no wall-clock timeout for delegated children by default, so a child can spend more than ten minutes on a deep review and remain healthy as long as it is making progress. Diagnosis should therefore start with activity signals, not elapsed time.

On surfaces that support later delivery, top-level delegate_task calls detach automatically. Do not confuse them with /background <prompt>, which creates a separate session; this guide covers delegated children.

Inspect the live delegation tree

Run this in the session that owns the work:

/agents

/tasks is an alias. On the classic CLI and messaging platforms, the command prints background delegations with per-child activity, including API-call count, current tool or between-turn state, and how recently activity occurred. The TUI adds a richer overlay with a recursive tree, per-branch token and cost rollups, files touched, turn-by-turn history, and controls to pause or kill a specific branch.

Interpret the fields together:

  • A rising API-call count means the child is advancing between model turns.
  • A current tool plus recent activity usually means the child is working, even if the tool is slow.
  • "Between turns" with a recent timestamp is normal.
  • A frozen API-call count and an old last-activity time are stronger stall evidence than total runtime.
  • stalling · no progress 450s — interrupting means the built-in monitor has already intervened.

Follow the live transcript

Every delegation task gets an append-only human-readable log as soon as it is dispatched. The delegate_task response includes these paths in live_transcripts. The documented layout is:

~/.hermes/cache/delegation/live/<delegation-id>/task-<n>.log

Watch a specific child from a shell:

tail -f ~/.hermes/cache/delegation/live/<delegation-id>/task-0.log

Each timestamped line can include assistant text, thinking snippets, tool calls, tool results, and a final status marker. The neighboring manifest.json describes the batch goals, task count, and per-task status. The logs persist after completion and directories older than seven days are pruned when new delegations are dispatched.

The last transcript event usually identifies the phase. A child that never reached an API call points toward provider, credential, or tool-schema setup. A child parked in a terminal command or web fetch may simply be inside a slow tool. A child with completed tool output but no next model activity may be wedged between turns.

Understand the built-in stall monitor

Hermes watches background delegations with a progress-based monitor enabled by default and requiring no configuration. Progress includes every streamed token, tool transition, and API-call boundary.

  • A progressing child is never interrupted merely for running a long time.
  • A completely quiet idle child is considered stale after 450 seconds.
  • A child currently inside a tool gets a 1200-second threshold so legitimate terminal and web operations have more room.
  • Once stale, the child is interrupted and gets a 120-second grace period to unwind and deliver partial results.
  • If it still does not return, Hermes force-finalizes a terminal stalled completion event and frees the asynchronous slot.

A stalled result includes structured fields: stalled_after_quiet_seconds, stall_threshold_seconds, stall_phase (idle or in_tool), and stall_grace_seconds. Use those fields instead of inferring the cause from an error sentence.

Separate stalls from configured timeouts

delegation.child_timeout_seconds is a different mechanism. Its default is 0, meaning no wall-clock timeout. A positive value, with a 30-second floor, imposes a hard per-child cap:

delegation:
  child_timeout_seconds: 1800

Use a cap for deliberate unattended cost control, not as the first response to slow legitimate work. A timeout result reports timeout_seconds, timed_out_after_seconds, and timeout_phase; a stall reports the quiet-time fields above.

If a configured hard cap fires before the child makes any API call, Hermes writes a diagnostic file named like ~/.hermes/logs/subagent-timeout-<session>-<timestamp>.log. It contains the child configuration snapshot, credential-resolution trace, early errors, and stack traces for all live threads. Inspect that file, then run hermes doctor and hermes auth list to check the installation and configured credentials. The special dump is tied to a zero-call timeout; it is not promised for every stall.

Choose the safest intervention

  • Keep waiting when /agents or the transcript shows advancing signals.
  • Cancel one branch in the TUI when a single child is clearly wrong and its siblings should continue.
  • Use /stop when you intend to cancel all running background delegations owned by the current session.
  • Do not close or reset the owning session if you still want its children; explicit close or reset interrupts them.
  • Inspect external state before retrying after a Hermes process restart. Running children are not resumed, and their attempt becomes unknown because Hermes cannot prove which side effects happened.

A completion event that was already stored before a restart can be delivered afterward, but that is durable completion delivery, not durable execution. For work that must survive session closure or process restart, the official docs recommend a cron job or a tracked background terminal process with completion notification rather than delegation.

Pitfalls

  • Confusing /background sessions with delegated subagents sends you to the wrong observability surface.
  • Treating elapsed time as a stall ignores progress-based monitoring and can kill healthy deep work.
  • Adding a short hard timeout can convert valid slow work into repeated failures and higher cost.
  • /stop is broad for the owning session. Use branch-specific TUI controls when only one child should be cancelled.
  • A process restart does not resume delegated execution. Retrying a task with external side effects before checking state can duplicate work.
  • Live transcripts may contain operational details and tool output. Treat them as local diagnostic records, not automatically shareable reports.

Verification checklist

  • Verify /agents shows the expected delegation ID, child count, API-call counts, current tool, and activity age.
  • Verify the live transcript path came from the dispatch response and its final event matches the reported child status.
  • Verify a quiet child has crossed the correct threshold for its phase before calling it stalled.
  • Verify the result metadata identifies a stall or a configured timeout rather than relying on prose alone.
  • Verify zero-call timeout diagnostics were inspected for provider, credential, schema, or thread evidence when such a dump exists.
  • Verify any cancellation targets the intended branch or owning session.
  • Verify external side effects before retrying an attempt marked unknown after restart.
  • Verify durable work uses cron or a tracked background process instead of assuming delegation will resume.

Official references