mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-11-06 05:17:15 +00:00
docs(stt): clarify language inheritance and auto-detection
Explain why a blank provider language can still resolve to English. Align the guides and environment reference with first-nonempty resolution while preserving configuration values and documenting command-backend limits. Constraint: Documentation must describe current runtime behavior, not pending default-language or CLI changes. Confidence: high Scope-risk: narrow Reversibility: clean Tested: Four configuration cases through the real loader and resolver; parsed YAML unchanged. Tested: Clean English Docusaurus build and rendered table/cross-link checks; git diff --check. Not-tested: Other documentation locales; no runtime code changed.
This commit is contained in:
1 parent
f97e97234e
commit
961e275288
5 files changed
+47
-14
No files matched your search
@@ -1770,10 +1770,10 @@ stt:
|
||||
# no_speech_prob_threshold: 0.6 # drop a segment only if no_speech_prob > this...
|
||||
# logprob_threshold: -1.0 # ...AND avg_logprob < this (both must hit — quiet real speech survives)
|
||||
# unload_after_idle_seconds: 0 # 0=never unload (default); e.g. 300 releases the model after 5min idle (frees VRAM on GPU; next voice message reloads it)
|
||||
language: "en" # GLOBAL language hint for every STT provider (per-provider language wins). Set "" for auto-detect.
|
||||
language: "en" # Global hint; a nonempty provider language wins. Blank falls back to HERMES_LOCAL_STT_LANGUAGE, then auto-detect.
|
||||
# groq:
|
||||
# model: "whisper-large-v3-turbo"
|
||||
# language: "" # blank = stt.language > HERMES_LOCAL_STT_LANGUAGE > auto-detect
|
||||
# language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
openai:
|
||||
model: "whisper-1" # whisper-1 | gpt-4o-mini-transcribe | gpt-4o-transcribe | gpt-transcribe
|
||||
language: "" # blank = inherit stt.language; set "en", "es", "fr", etc. to force
|
||||
|
||||
@@ -117,7 +117,7 @@ Hermes reads environment variables from the process environment and, for user-ma
|
||||
| `HERMES_MODEL` | Override model name at process level (used by cron scheduler; prefer `config.yaml` for normal use) |
|
||||
| `VOICE_TOOLS_OPENAI_KEY` | Preferred OpenAI key for OpenAI speech-to-text and text-to-speech providers |
|
||||
| `HERMES_LOCAL_STT_COMMAND` | Optional local speech-to-text command template. Supports `{input_path}`, `{output_dir}`, `{language}`, and `{model}` placeholders |
|
||||
| `HERMES_LOCAL_STT_LANGUAGE` | Default language hint for STT. Used by the `local` (faster-whisper) provider, `HERMES_LOCAL_STT_COMMAND`, the local `whisper` CLI fallback (default: `en`), Groq, and xAI when no per-provider `language` is set in `config.yaml` |
|
||||
| `HERMES_LOCAL_STT_LANGUAGE` | Fallback STT language hint, used only when both the provider's own `language` and the global `stt.language` are empty. Command backends (`HERMES_LOCAL_STT_COMMAND`, the local `whisper` CLI) may apply their own default (`en`) when no hint resolves. |
|
||||
| `HERMES_HOME` | Select the configuration and user-data home. A literal `~` or `$VAR` in the value is expanded (fish does not expand `~` inside `VAR=~/…`), so it never resolves relative to the current directory. Defaults to `~/.hermes` on POSIX and `%LOCALAPPDATA%\hermes` on Windows; the official Docker image uses `/opt/data`. Profile/runtime context can select a more specific home. |
|
||||
| `HERMES_CONFIG_BACKEND` | Select where the profile's `config.yaml` user layer is read and written. `file` (the default) is the local `config.yaml`; `remote` is [Remote Config](../user-guide/remote-config.md). Any other value, or a remote backend that cannot serve config, stops Hermes at startup with an error instead of falling back to defaults. An explicitly empty value in a `.env` resets a stale shell value to `file`. Read from the launch environment, the launch home's `.env` and the managed `.env`, never from `config.yaml`. See [Config backend](../developer-guide/config-backend.md). |
|
||||
| `HERMES_CONFIG_REMOTE_URL` | Base URL of the [Remote Config](../user-guide/remote-config.md) plane (default `https://config-config.nousresearch.com`; `https://` only, plain `http://` only to loopback, no user info). An env var because it is needed before any config can be read: it says where config comes from. |
|
||||
|
||||
@@ -2558,7 +2558,7 @@ stt:
|
||||
enabled: true # Auto-transcribe inbound voice messages (default: true)
|
||||
echo_transcripts: true # Post raw transcripts back to the chat as 🎙️ "..." (default: true)
|
||||
provider: "local" # "local" | "groq" | "openai" | "mistral" | "xai" | "elevenlabs" | "deepinfra" | ...
|
||||
language: "en" # GLOBAL language hint for every provider (per-provider language wins); set "" for auto-detect
|
||||
language: "en" # Global hint; a nonempty provider language wins. Blank falls back to HERMES_LOCAL_STT_LANGUAGE, then auto-detect
|
||||
streaming: false # live partial text while you speak (openai, xai, elevenlabs); see Voice Mode > Live transcription
|
||||
cloud_trim_silence: true # trim long pauses with ffmpeg before uploading to a cloud provider (default: true)
|
||||
cloud_trim_threshold_db: -40 # audio quieter than this counts as silence
|
||||
@@ -2566,7 +2566,7 @@ stt:
|
||||
# prompt: "Hermes, Teknium, Nous Research, kanban" # Static vocabulary hint (see below)
|
||||
local:
|
||||
model: "base" # tiny, base, small, medium, large-v3, turbo
|
||||
language: "" # per-provider override of stt.language
|
||||
language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
initial_prompt: "" # optional whisper prompt to bias vocabulary/script (e.g. Simplified Chinese)
|
||||
vad: true # Silero VAD filter (default on) — silence never reaches whisper; false = raw behavior (music/ambient)
|
||||
vad_min_silence_ms: 500 # min silence (ms) that splits speech chunks when vad is on
|
||||
@@ -2574,19 +2574,48 @@ stt:
|
||||
logprob_threshold: -1.0 # ...AND avg_logprob < this (both must hit — quiet real speech survives)
|
||||
unload_after_idle_seconds: 0 # 0=never unload (default); e.g. 300 = release the model after 5min idle
|
||||
groq:
|
||||
language: "" # per-provider override of stt.language
|
||||
language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
openai:
|
||||
model: "whisper-1" # whisper-1 | gpt-4o-mini-transcribe | gpt-4o-transcribe | gpt-transcribe
|
||||
language: "" # per-provider override of stt.language
|
||||
language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
timeout: 60 # seconds per transcription request; raise for self-hosted model cold starts
|
||||
max_retries: 1 # SDK transport retries (connection errors, 408/409/429/5xx); 0 = single attempt
|
||||
xai:
|
||||
model: "grok-voice-transcribe-2.0" # or grok-voice-transcribe-1.0; sent on every request
|
||||
language: "" # per-provider override of stt.language
|
||||
language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
# model: "whisper-1" # Legacy fallback key still respected
|
||||
```
|
||||
|
||||
Language resolution is the same for **every** STT provider (local, groq, openai, mistral, xai, elevenlabs, deepinfra, command providers, and plugins): `stt.<provider>.language` → `stt.language` → `HERMES_LOCAL_STT_LANGUAGE` env var → provider auto-detect. **The default is `stt.language: "en"`** — Whisper auto-detection frequently misidentifies short or accented clips, which shows up as voice notes transcribed in the wrong language. Non-English speakers should set `stt.language` to their language code once (e.g. `"es"`, `"zh"`, `"uk"`); set it to `""` to restore auto-detection for multilingual use.
|
||||
Language resolution uses the **first nonempty string** in this order:
|
||||
|
||||
```text
|
||||
stt.<provider>.language -> stt.language -> HERMES_LOCAL_STT_LANGUAGE -> no language hint
|
||||
```
|
||||
|
||||
A blank value means **continue to the next source**, not "force auto-detection". ElevenLabs also accepts `stt.elevenlabs.language_code` as a provider-specific alias. **The global default is `stt.language: "en"`**, so leaving only the provider language blank still resolves to English. To pin another language, set a provider or global language code such as `"zh"`, `"es"`, or `"uk"`.
|
||||
|
||||
Without a request-level plugin hook override:
|
||||
|
||||
| Provider language | Global language | `HERMES_LOCAL_STT_LANGUAGE` | Resolved hint |
|
||||
| --- | --- | --- | --- |
|
||||
| `""` | `"en"` | unset | `en` |
|
||||
| `"zh"` | `""` | unset | `zh` |
|
||||
| `""` | `""` | `en` | `en` |
|
||||
| `""` | `""` | unset | none |
|
||||
|
||||
For automatic language detection with **local faster-whisper**, use:
|
||||
|
||||
```yaml
|
||||
stt:
|
||||
provider: local
|
||||
language: ""
|
||||
local:
|
||||
language: ""
|
||||
```
|
||||
|
||||
Also remove any `HERMES_LOCAL_STT_LANGUAGE` override from the process environment or `.env`. A nonempty provider setting or a `pre_transcription` hook can still select a language.
|
||||
|
||||
When no hint is resolved, faster-whisper and the native cloud handlers omit the language argument; plugin providers receive `None`. Command backends have additional behavior: the current `local_command` path and named command providers fall back to `en` for the `{language}` placeholder. Custom templates may omit that placeholder or use their own engine's detection option. If `local` falls back to a Whisper CLI because faster-whisper is unavailable, the same blank settings therefore do not currently guarantee auto-detection. See [command providers](features/tts.md#stt-custom-command-providers) for their configuration.
|
||||
|
||||
`stt.openai.timeout` and `stt.openai.max_retries` shape the OpenAI-SDK transcription client that the `openai`, `groq` and `deepinfra` providers share (there are no per-provider siblings yet, and the SDK reads no environment variables for these). The defaults are `60` / `1` rather than the previous fixed 30 s / no retries because a self-hosted OpenAI-compatible endpoint can take longer than 30 s to load its model on the first request, which used to lose that voice message outright. The trade-off: an unreachable backend now holds a voice message for up to two attempts × the timeout before Hermes gives up; set `timeout: 30` and `max_retries: 0` for the old shape.
|
||||
|
||||
|
||||
@@ -511,21 +511,23 @@ Local transcription works out of the box when `faster-whisper` is installed. If
|
||||
# In ~/.hermes/config.yaml
|
||||
stt:
|
||||
provider: "local" # "local" | "groq" | "openai" | "mistral" | "xai" | "elevenlabs" | "deepinfra"
|
||||
language: "en" # Global language hint applied to every provider unless a per-provider language overrides it; set "" to restore auto-detect
|
||||
language: "en" # Global hint; blank falls back to HERMES_LOCAL_STT_LANGUAGE, then auto-detect
|
||||
local:
|
||||
model: "base" # tiny, base, small, medium, large-v3, turbo
|
||||
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else auto-detect
|
||||
language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
groq:
|
||||
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else auto-detect
|
||||
language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
openai:
|
||||
model: "whisper-1" # whisper-1, gpt-4o-mini-transcribe, gpt-4o-transcribe, gpt-transcribe
|
||||
mistral:
|
||||
model: "voxtral-mini-latest" # voxtral-mini-latest, voxtral-mini-2602
|
||||
xai:
|
||||
model: "grok-voice-transcribe-2.0" # or grok-voice-transcribe-1.0
|
||||
language: "" # optional ISO-639-1 hint; blank = use HERMES_LOCAL_STT_LANGUAGE if set, else "en"
|
||||
language: "" # blank inherits stt.language, then HERMES_LOCAL_STT_LANGUAGE
|
||||
```
|
||||
|
||||
Blank provider language settings in this example inherit the global `"en"`; they do not select auto-detection. See [STT language resolution](../configuration.md#speech-to-text-stt) for precedence, an auto-detection example, and command-backend limits.
|
||||
|
||||
### Provider Details
|
||||
|
||||
**Local (faster-whisper)** — Runs Whisper locally via [faster-whisper](https://github.com/SYSTRAN/faster-whisper). Uses CPU by default, GPU if available. Model sizes:
|
||||
@@ -547,7 +549,7 @@ HF_HUB_DISABLE_XET=1
|
||||
|
||||
`HF_HUB_DISABLE_XET=1` keeps downloads on the mirror's regular HTTP path instead of contacting Xet CAS hosts that do not honor `HF_ENDPOINT`.
|
||||
|
||||
**Groq API** — Requires `GROQ_API_KEY`. Good cloud fallback when you want a free hosted STT option. Set `stt.groq.language` (or the global `HERMES_LOCAL_STT_LANGUAGE` env var) to skip Whisper's auto-detect and reduce latency on known-language audio.
|
||||
**Groq API** — Requires `GROQ_API_KEY`. Good cloud fallback when you want a free hosted STT option. Set `stt.groq.language` or the global `stt.language` (with `HERMES_LOCAL_STT_LANGUAGE` as the last fallback) to skip Whisper's auto-detect and reduce latency on known-language audio.
|
||||
|
||||
**OpenAI API** — Accepts `VOICE_TOOLS_OPENAI_KEY` first and falls back to `OPENAI_API_KEY`. Supports `whisper-1`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe`, and `gpt-transcribe`.
|
||||
|
||||
|
||||
@@ -542,6 +542,8 @@ tts:
|
||||
device: cpu
|
||||
```
|
||||
|
||||
Blank provider language settings inherit the global `stt.language` value (default `"en"`) before consulting the environment. See [STT language resolution](../configuration.md#speech-to-text-stt) for the auto-detection conditions and command-backend limits.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
|
||||
Reference in new issue
Block a user