hermes-agent/plugin-catalog/alans-way-computer.yaml

name: alans-way-computer
repo: https://github.com/capthvnsen/alans-way-agents
sha: 23cce292c867a42ccf9853b99d798add028788f0
subdir: alans-way-computer
version: "0.21.10"
title: "Alan's Way Plugin: Computer Use"
description: "Computer-use provider for Hermes: the stock computer_use tool drives desktop apps on the user's computer through the Alan's Workspace app, and on the VM's own desktop when the computer is offline. Needs the Alan's Way Plugin's workspace setup. Select it with computer_use.backend: alans-way-computer (the Alan's Way Plugin's setup.sh sets this when it installs the provider; in the catalog flow it prints the command). Disclosure — starts the workspace router from the profile's mcp_servers.workspace_browser block (command and args from config.yaml, with a filtered environment plus HERMES_WORKSPACE_ALLOW_DESKTOP_ACTIONS=1, which lets this private router child serve desktop input) as a long-running node child per session. The filtered environment passes PATH, HOME, locale and OS basics, plus DISPLAY, XAUTHORITY and DBUS_SESSION_BUS_ADDRESS so the router's VM-desktop fallback finds the X display and session bus, plus HERMES_HOME set to the bound profile's home, under which the router appends a content-free serving log to logs/workspace-router.log; no keys or tokens. It talks to the router over stdio (app lists, element trees, screenshots and input actions). The router reaches the user's computer over SSH on the user's tailnet and falls back to the VM's desktop, where it uses xdotool, a screenshot tool and the AT-SPI accessibility bus if they are installed (the provider installs none of them). On the user's Mac the provider can bring the target app to the front when the model asks for it (focus_app with raise_window, or an input action with delivery_mode foreground), and the Alan's Workspace host does so only after the user has been idle for at least 2 seconds; focus_app can also launch a named app, or open a document or folder by absolute path in its app (the host refuses app bundles, executables, scripts and off-limits apps). Every input action still goes through Hermes's computer_use approval gate, with raising and foreground input under their own approval scopes (the cua:<action>:background entries in command_allowlist that the Alan's Way Plugin's --allow-desktop-actions adds skip the prompt for background input, focus_app and launches, not for raising or foreground input). The provider itself makes no network calls. On Hermes 0.21.5, which has no provider API, it installs and does nothing. Since 0.21.0, captures keep element numbers stable across action replies on Mac and Windows, fold in changed controls and report removed numbers; a truncated list never treats unseen controls as removed. Linux keeps the whole-tree generation guard, so capture again before acting by number after a change. Action replies describe nearby changes and can mark the effect confirmed. Default som captures skip the screenshot call entirely when finite snapshot window geometry proves named controls cover at least 35%; older snapshots without geometry still fetch a screenshot to measure coverage and may omit it from the model; mode vision explicitly includes it and mode ax requests no screenshot. Replies identify the coordinate space and explain locked screens, missing permissions, unreadable or cut trees, hung apps and uncertain outcomes before a retry. Captures request generation-based snapshot deltas, validate them against the held tree and retry once for a full list on invalid or mismatched deltas; older apps may return full or unchanged lists. Once a reply identifies a window, later captures, screenshots and input stay bound to that window; capture(window_id=...) selects another. Mismatched window replies clear the target and require a fresh capture. Desktop actions carry expect_host from the observed host so a routing switch refuses input before dispatch; app IDs and element numbers must be reacquired after a switch. Element trees, generation, bound window ID, screenshots' geometry and host OS are session memory, with no new persistent provider state, tools, hooks, middleware, required environment variables or direct network destinations. Hermes's unmodified CI-pinned scanner reports SAFE: 0 HIGH, 2 MEDIUM (python_subprocess: the long-running
  router Popen in __init__.py and its bounded 40-second --probe subprocess) and 0 LOW. Browser commit approvals are separate from this provider's unchanged computer_use approval gate."
maintainer: capthvnsen
tier: community
category: tools
requires_hermes: ">=0.21.6"
docs_url: https://github.com/capthvnsen/alans-way-agents/tree/23cce292c867a42ccf9853b99d798add028788f0/alans-way-computer
readme: true
platforms: [linux, macos]
known_issues:
  - "Needs a Hermes build with the pluggable computer-use API (Hermes 0.21.6 or newer). On older Hermes the plugin installs and does nothing. Install alans-way first and run its workspace setup, which writes the workspace_browser block this provider reuses."
  - "Hermes asks for approval on every computer_use action type until it is granted. For an unattended Telegram bot, grant them once with Always in Telegram, or add cua:<action>:background entries to command_allowlist (see the README). Takes effect after a gateway restart."
  - "The pinned README requires in-process loading and says plugins.isolation: host is unsupported because the provider hands Hermes a live router session. The current admission validator reports host readiness; remote isolation compatibility has not been verified."
  - "Typing without a target field types into the focused field (still approval-gated). On the user's Mac this needs Alan's Workspace 0.21.9: the target app is brought forward only after the user has been idle for at least 2 seconds, password fields are refused, and the focused field must belong to the target app."
  - "The VM desktop path needs xdotool, a screenshot tool (scrot, maim or ImageMagick import), the AT-SPI bus and python3-gi on the VM. Without them those calls fail and the agent reports it; setup.sh warns about what is missing."
  - "A default som capture skips the screenshot request when snapshot geometry establishes 35% named-control coverage; older apps without geometry still require a screenshot to check it and may omit it from the model. Use mode vision to request it explicitly, or mode ax for an element-only capture. When a screenshot is omitted, coordinate= and element bounds are screen pixels."
  - "Mac and Windows action replies update the held element list without shifting surviving numbers. On Linux, capture again before acting by number after a tree change; cut replies do not prove an unseen control is gone. A timeout or hung app can leave the action's outcome uncertain: capture before retrying."
  - "This release changes provider code: focus_app launches a named app, or opens a document or folder by absolute path in its app on the user's computer (needs Alan's Workspace 0.21.9 with computer-open-v1; app bundles, executables, scripts and off-limits apps are refused). It binds the window showing the opened item. When an action lands on an unfocused window it returns that error and tells the model to ask for a raise explicitly (focus_app with raise_window=true, or delivery_mode='foreground'), which Hermes approves under its foreground scopes; the provider never raises on its own. It follows the focused window after an app shortcut, and passes the app's step notes to the model. Older apps refuse path opens and keep the earlier behavior."
  - "Admission validation passes with a private-import warning: __init__.py uses tools.mcp_tool_config._resolve_stdio_command to resolve the router launcher; this internal helper may change across Hermes releases."
capabilities:
  provides_tools: []
  provides_hooks: []
  provides_middleware: []
  requires_env: []