mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-11-06 05:17:15 +00:00
f7dde7f56d
Scraping vendors answer 200 and report the target page's own status inside the
payload. Firecrawl (direct key, Nous Tool Gateway, keyless REST) put it in
metadata.status_code / statusCode and Hermes never read it, so
web_extract("https://httpbin.org/status/403") returned the stub error page
with error: null: the agent could not tell blocked from blank, the keyless
rescue never ran, and the error page was cached as content.
One shared helper in plugins/web/_common.py (origin_http_error /
with_origin_status / vendor_error_entry) now turns an origin status >= 400
into "HTTP <code> from origin[: reason]", keeping any error-page body as
content. Applied to:
- Firecrawl SDK/gateway scrape and the keyless REST scrape (both key spellings)
- Parallel keyed + keyless (error was the bare "http_error"; the payload's
http_status_code was dropped)
- Exa keyed (URLs Exa could not fetch were silently missing from results;
their reason lives in response.statuses)
Tavily, Perplexity and Keenable already surface per-URL failures (Tavily's
failed_results carry no status code).
Salvages the Firecrawl statusCode check from #99549 (and #99556), which read
only the camelCase key and only flagged empty content, so it missed the
Gateway/SDK case where the 403 body is a non-empty stub page.
Found on Hermes Cloud staging.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Co-authored-by: Ahmett101 <ahmet.tunc@gmail.com>