mirror of
https://github.com/Haidra-Org/AI-Horde.git
synced 2026-11-06 05:17:31 +00:00
docs: add checked chat wire examples and link the reference pages to them
- docs/reference/chat_wire_examples.md holds the JSON bodies of native requesters, OpenAI front clients and text workers, with a sequence diagram of one chat job - test_chat_wire_examples.py and test_chat_front_wire_examples.py capture each response body live and compare it with the page: keys, the values of rc, code, type, state, finish_reason, kind, reason and booleans, and the JSON type of every other scalar; documented requests are posted, and worker request keys are checked against the route's parser and models - tests/integration/_wire_examples.py holds the page parsing and comparison both test modules share - serve_chat_jobs.md opens with one chat job in full; it, chat_requests.md and chat_front.md include wire examples through pymdownx.snippets sections, so a shown body cannot drift from the checked one - chat_requests.md and chat_front.md link each major section to its examples; the 202 description states when message and warnings appear, and the front's error descriptions carry the 404 type and metadata.rc
This commit is contained in:
14 files changed
+2472
-129
No files matched your search
@@ -261,6 +261,36 @@ client can dispatch or answer such a reply. A refused submit faults the job on t
|
||||
the request, so the check refuses only what OpenAI would never return. A trimmed prompt or a hidden tool-selection pass
|
||||
does not show in the reply and is not verified.
|
||||
|
||||
### Prompt fit on the worker
|
||||
|
||||
The guarantee `prompt_overflow_refused` promises that a worker faults a conversation that does not fit instead of
|
||||
trimming it. Only the worker can keep this promise. The horde does not run a tokenizer and sizes context at 3
|
||||
characters per token (`CHARS_PER_TOKEN_ESTIMATE` in `horde/classes/kobold/request_fit.py`), and a trimmed prompt
|
||||
does not show in the reply. The worker's count has to be at least the count used by generation. A lower count lets a
|
||||
prompt pass the fit check and still overflow, and the backend then trims it.
|
||||
|
||||
An exact counter renders the conversation with the backend's template and tokenizes it. Its count matches the count
|
||||
used by generation when both render with the same template and the same BOS token, and falls short when they differ.
|
||||
The minimal worker in `tools/chat_worker_example/` therefore compares its counter with the backend's
|
||||
`usage.prompt_tokens` for one probe conversation at startup. It declares the guarantee only when the counter is not
|
||||
short.
|
||||
|
||||
A backend without a counter route can still back the guarantee with an upper bound. A byte-level BPE tokenizer emits
|
||||
at least one UTF-8 byte per token, so the byte length of the messages' JSON bounds the token count of the message
|
||||
text. The JSON keys around each message stand in for its role markup. The template's fixed part is the BOS token, any
|
||||
system prompt injected by the template, and the generation prompt. The worker measures it once from the probe's
|
||||
`usage.prompt_tokens` and adds it to the bound. The probe has no tools, so it does not measure a tool preamble, and
|
||||
the bound backs the guarantee for tool requests only when the operator adds the preamble's size with
|
||||
`--template-overhead`.
|
||||
|
||||
The bound costs capacity. It counts one token per byte, which for ASCII text is about three times the horde's
|
||||
estimate of 3 characters per token, and more for text outside ASCII. The horde sends a worker conversations that fit
|
||||
by its own estimate, so a worker on the byte bound faults long conversations with `prompt_overflow`. A worker that
|
||||
aborts more than 20 jobs in an hour is put into maintenance. The alternative is to trim the prompt, which keeps the
|
||||
job and breaks the guarantee, so a worker that trims does not declare it and serves only `best_effort` requests. The
|
||||
counters and the check script are in
|
||||
[Write a minimal chat worker](../how-to/write_a_minimal_chat_worker.md#fit-check-and-the-overflow-guarantee).
|
||||
|
||||
### Strict handling of unsupported parameters
|
||||
|
||||
A key or value the service does not implement is refused with `400 UnsupportedChatParameter`, and the message states
|
||||
|
||||
@@ -24,6 +24,6 @@ the ordered steps, the checks between them, and the way back if a step fails.
|
||||
| [Run the OpenAI-compatible chat front](run_openai_chat_front.md) | Configure and start the horde.oai process that serves /v1/chat/completions and /v1/models in front of a horde, then check it with a real request. |
|
||||
| [Send chat completions through the OpenAI-compatible front](send_chat_completions.md) | Point an existing OpenAI client at the Horde: base URL, key and attribution, model choice, accepted and refused keys, the horde object, images, tools, structured output, streams, errors and limits. |
|
||||
| [Add chat completions to a web frontend](add_chat_to_a_frontend.md) | Call the Horde's chat front from a browser: bearer key over CORS, OpenAI-only request keys, a stream parser that reads content, tool calls and error events, the tool loop, and model card limits. |
|
||||
| [Write a minimal chat worker](write_a_minimal_chat_worker.md) | Run a tested one-file Python worker that serves chat jobs from any OpenAI-compatible backend, check its declaration, send it a chat request and read the reply. |
|
||||
| [Write a minimal chat worker](write_a_minimal_chat_worker.md) | Run a tested one-file Python worker that serves chat jobs from any OpenAI-compatible backend, check its token counter and confirm its chat declaration on the horde. |
|
||||
| [Serve chat jobs from a text worker](serve_chat_jobs.md) | Declare chat on text pop, recognise a chat job, render and fit it, serve tools, structured output and images, and submit the structured reply, with koboldcpp's embedded worker as the worked example. |
|
||||
<!-- END GENERATED: documents -->
|
||||
@@ -37,6 +37,37 @@ For why chat travels this way, see [Chat requests on the Horde](../explanation/c
|
||||
requests as a client, see [Send chat completions through the OpenAI-compatible front](send_chat_completions.md).
|
||||
To run this end to end with koboldcpp for the first time, follow [Serve your first chat job with koboldcpp](../tutorial/first_chat_worker.md).
|
||||
|
||||
## One chat job in full
|
||||
|
||||
A worker adds the five fields of its chat declaration to every pop, beside the usual text fields. The body in
|
||||
[step 1](#1-declare-chat-on-pop) is the smallest declaration: two optional features, the limits, the model and no
|
||||
image transport.
|
||||
|
||||
A chat job arrives with `chat: true` and the conversation in `payload.messages`:
|
||||
|
||||
```json
|
||||
--8<-- "docs/reference/chat_wire_examples.md:pop-chat-job"
|
||||
```
|
||||
|
||||
The worker renders `payload.messages` with its chat template, generates, and submits an assistant message:
|
||||
|
||||
```json
|
||||
--8<-- "docs/reference/chat_wire_examples.md:submit-reply"
|
||||
```
|
||||
|
||||
The submit is answered with the kudos the worker earned:
|
||||
|
||||
```json
|
||||
--8<-- "docs/reference/chat_wire_examples.md:submit-reply-response"
|
||||
```
|
||||
|
||||
Every other body a worker meets (tools, images, offloaded messages, faults, progress posts, refusals and worker
|
||||
details) is in [Chat wire examples](../reference/chat_wire_examples.md#worker), checked against the running code.
|
||||
[Write a minimal chat worker](write_a_minimal_chat_worker.md) is a complete worker for an OpenAI-compatible backend
|
||||
in one file.
|
||||
|
||||
## The pop loop
|
||||
|
||||
The steps below build one pop loop. A pass through it, with the step that covers each part:
|
||||
|
||||
```mermaid
|
||||
@@ -127,7 +158,8 @@ Add five fields to every pop body, beside the usual `name`, `models`, `max_lengt
|
||||
then not sent to the worker, under either conformance mode, since a clamped value changes the request. Leave it out
|
||||
when the backend applies every temperature as sent.
|
||||
- `model_id` is the model the worker serves chat jobs with, and it has to be one of `models`. The native route treats
|
||||
it as an opaque string. The OpenAI-compatible front reserves `:` and answers 404 for any model id that contains one, so a model name with a colon can be served natively and cannot be reached through the front
|
||||
it as an opaque string. The OpenAI-compatible front reserves `:` and answers 404 for any model id that contains
|
||||
one, so a model name with a colon can be served natively and cannot be reached through the front
|
||||
([ADR 0020](../decisions/0020-select-chat-service-by-request-body-and-model-variants.md)). Choose a name without a
|
||||
colon.
|
||||
- `image_transports` lists how the worker accepts images: `url`, `base64` or both. Leave it out, or send it empty,
|
||||
@@ -214,14 +246,16 @@ Only the worker knows how many tokens its template and tokenizer produce, which
|
||||
worker ([Chat job pop response](../reference/chat_requests.md#chat-job-pop-response)).
|
||||
|
||||
The fit check must not run side effects. A backend's request transform may do more than render. koboldcpp's chat
|
||||
transform runs a short in-process generation to choose a tool when `tool_choice` is `auto` or `required`. Running that
|
||||
transform to estimate the prompt would generate once more for every chat job with tools, outside the job's budget.
|
||||
transform, when it formats without a Jinja template, runs a short in-process generation to choose a tool when
|
||||
`tool_choice` is `auto` or `required`. Running that transform to estimate the prompt could generate once more for every
|
||||
chat job with tools, outside the job's budget.
|
||||
koboldcpp's embedded worker therefore does not render for the check. It counts the tokens of the JSON of `messages`,
|
||||
`tools` and `response_format`, adds 32 tokens, 16 for each message and 512 for each image, and faults the job with
|
||||
`prompt_overflow` when that count plus `max_length` exceeds its context.
|
||||
|
||||
Verification: send a conversation longer than `chat_limits.max_context_length` allows. The worker submits
|
||||
`{"id": "<job-id>", "state": "faulted", "model_id": "<model-name>"}` and does not generate.
|
||||
`{"id": "<job-id>", "state": "faulted", "model_id": "<model-name>", "fault_reason": "prompt_overflow"}` and does not
|
||||
generate.
|
||||
|
||||
## 4. Serve tools and structured output
|
||||
|
||||
@@ -333,8 +367,10 @@ Send `message`, `finish_reason` and `model_id` in place of `generation`
|
||||
beside `finish_reason`.
|
||||
- `prompt_tokens` and `completion_tokens` are sent only when the backend reported them, and `reasoning_tokens`, how
|
||||
many of the completion tokens were reasoning, only when the backend counted it. Leave a count out and do not send
|
||||
zero; the horde stores a missing count as null and shows it to the client, and the counts never change the
|
||||
charge or the reward ([Pricing](../reference/chat_requests.md#pricing)).
|
||||
zero. The horde stores a missing count as null, and the native status shows it as null. The OpenAI-compatible
|
||||
front estimates a missing `prompt_tokens`, `completion_tokens` or `reasoning_tokens` from the text and sets
|
||||
`x_horde.estimated_usage` to true ([Response shape](../reference/chat_front.md#response-shape)). The counts do
|
||||
not change the charge or the reward ([Pricing](../reference/chat_requests.md#pricing)).
|
||||
- `model_id` is the model the job was assigned, the `model_id` the worker declared when it popped the job.
|
||||
- `state` is `ok`, `censored`, `csam` or `faulted`, as for text jobs.
|
||||
|
||||
@@ -526,46 +562,17 @@ endpoint, and the worker posts its progress about once a second; a `cancelled: t
|
||||
generation, and the job is not submitted. A job with tools is generated whole, since the tool calls are parsed from the complete reply.
|
||||
|
||||
The reply is submitted with the backend's `finish_reason` and with each token count the backend reported. The worker
|
||||
faults the job with `prompt_overflow` when the fit check fails, and with `reply_refused` when the horde answers the
|
||||
faults the job with `prompt_overflow` when the fit check fails, and with `reply_refused` when the horde refuses the
|
||||
reply with `ChatSubmitInvalid`, `ChatSubmitNonconformant` or `ChatModelMismatch`. A failed fetch, a failed generation
|
||||
or a finish reason outside the four the horde accepts faults the job without a reason. A chat job is not generated a
|
||||
second time.
|
||||
|
||||
### Start koboldcpp as a chat worker
|
||||
|
||||
```bash
|
||||
koboldcpp --model <model.gguf> --contextsize 4096 --jinja \
|
||||
--hordekey <api-key> --hordeworkername <worker-name> \
|
||||
--hordemodelname <model-name> --hordemaxctx 4096 --hordegenlen 512
|
||||
```
|
||||
|
||||
`--jinja` formats the conversation with the model's Jinja chat template, including tool calls. Add `--mmproj
|
||||
<projector.gguf>` for a vision model.
|
||||
|
||||
Verification: the log prints `Worker <n> - Embedded Horde Worker '<worker-name>' is started.`, and the worker's
|
||||
details show the declaration:
|
||||
|
||||
```bash
|
||||
curl -s "https://aihorde.net/api/v2/workers/name/<worker-name>" | jq '{accepts_chat_jobs, accepts_prompt_jobs, chat}'
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"accepts_chat_jobs": true,
|
||||
"accepts_prompt_jobs": true,
|
||||
"chat": {
|
||||
"model_id": "koboldcpp/<model-name>",
|
||||
"features": ["json_object", "json_schema", "presence_penalty", "prompt_overflow_refused", "reasoning_effort", "seed", "tools"],
|
||||
"limits": {"max_context_length": 4096, "max_length": 512, "tools_max_temperature": 1.0},
|
||||
"image_transports": ["messages_url"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
With a projector loaded, `features` adds `vision` and `image_transports` adds `base64`.
|
||||
|
||||
The embedded worker does not pop while koboldcpp's local API served a request in the last 20 seconds, so a local
|
||||
client in use delays Horde jobs.
|
||||
[Step 1 of Serve your first chat job with koboldcpp](../tutorial/first_chat_worker.md#1-start-koboldcpp-as-a-chat-worker)
|
||||
has the command and its log line, and step 2 checks the declaration. For a vision model, add `--mmproj
|
||||
<projector.gguf>` to that command. The worker's details show `accepts_prompt_jobs: true`, because the embedded worker
|
||||
also serves text jobs.
|
||||
|
||||
Reversal: stop koboldcpp. A worker counts as online for 300 seconds after its last pop, so the declaration leaves the
|
||||
chat model list within that time.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: "Write a minimal chat worker"
|
||||
summary: "Run a tested one-file Python worker that serves chat jobs from any OpenAI-compatible backend, check its declaration, send it a chat request and read the reply."
|
||||
summary: "Run a tested one-file Python worker that serves chat jobs from any OpenAI-compatible backend, check its token counter and confirm its chat declaration on the horde."
|
||||
topics: [chat, workers]
|
||||
order: 61
|
||||
---
|
||||
@@ -24,11 +24,11 @@ llama.cpp's `llama-server`, vLLM and koboldcpp among them. It uses `requests` an
|
||||
in-process horde. [Serve chat jobs from a text worker](serve_chat_jobs.md) is the full procedure with every option;
|
||||
this page is the shortest path to a working worker.
|
||||
|
||||
Preconditions: Python 3.12 or later with `requests` and `horde_sdk` installed; a registered API key for the horde at
|
||||
`<HORDE_URL>`; a model that the backend serves with a chat template; and `curl` and `jq`.
|
||||
Preconditions: Python 3.12 or later with `requests` and `horde_sdk` 0.33.3 or later installed; a registered API key
|
||||
for the horde at `<HORDE_URL>`; a model that the backend serves with a chat template; and `curl` and `jq`.
|
||||
|
||||
End state: the worker's details show `accepts_chat_jobs: true`, `accepts_prompt_jobs: false` and its declaration, and
|
||||
a chat request for `<model-name>` on the native route completes with the backend's reply.
|
||||
End state: the worker's details show `accepts_chat_jobs: true`, `accepts_prompt_jobs: false` and its declaration, so
|
||||
the horde sends it chat requests for `<model-name>`.
|
||||
|
||||
## The worker
|
||||
|
||||
@@ -39,7 +39,7 @@ a chat request for `<model-name>` on the native route completes with the backend
|
||||
What it does on each pass of `serve_one_pop`:
|
||||
|
||||
- Pops `POST /api/v2/generate/text/pop` with `chat_jobs: true` and `prompt_jobs: false`, so it receives chat jobs only
|
||||
and needs no text path. An empty pop returns `False`, and `main` sleeps 2 seconds before the next one.
|
||||
and does not need a text path. An empty pop returns `False`, and `main` sleeps 2 seconds before the next one.
|
||||
- Counts the prompt (see [Fit check and the overflow guarantee](#fit-check-and-the-overflow-guarantee)) and faults
|
||||
the job with `prompt_overflow` when the count plus `max_length` exceeds the job's `max_context_length`.
|
||||
- Sends the job to the backend without streaming, with the sampling values as the horde sent them. It does not post
|
||||
@@ -50,7 +50,7 @@ What it does on each pass of `serve_one_pop`:
|
||||
`reply_refused`. Any other refused submit, such as `AbortedGen` for a job past its 150-second lease, is logged and
|
||||
not faulted.
|
||||
|
||||
It declares no optional feature unless `--features` lists one. `--features` accepts `seed`, `presence_penalty`,
|
||||
It does not declare an optional feature unless `--features` lists one. `--features` accepts `seed`, `presence_penalty`,
|
||||
`frequency_penalty`, `tools`, `json_object`, `json_schema` and `reasoning_effort`; list only what the backend honours,
|
||||
since the horde does not test a declaration.
|
||||
|
||||
@@ -59,7 +59,7 @@ since the horde does not test a declaration.
|
||||
Every `strict` request needs the guarantee `prompt_overflow_refused`, and a request is strict unless the client asks
|
||||
for `best_effort` ([step 1 of Serve chat jobs](serve_chat_jobs.md#1-declare-chat-on-pop)). The worker declares it only
|
||||
when its fit check can back it. A job fits when the prompt's token count plus `max_length` is at most the job's
|
||||
`max_context_length`; `--token-counter` sets how the prompt is counted:
|
||||
`max_context_length`. `--token-counter` sets how the prompt is counted:
|
||||
|
||||
| `--token-counter` | Backend routes per job | Count |
|
||||
| ----------------- | --------------------------------------------------------- | ---------------------------------------------- |
|
||||
@@ -68,21 +68,20 @@ when its fit check can back it. A job fits when the prompt's token count plus `m
|
||||
| `vllm` | `POST /v1/chat/completions/render` | Exact, tools included |
|
||||
| `bytes` (default) | none | An upper bound from the UTF-8 length of the JSON |
|
||||
|
||||
koboldcpp's count runs its chat transform, which generates once to choose a tool when tools are present, so the
|
||||
worker leaves tools out of that count and adds their bytes instead.
|
||||
koboldcpp's count runs its chat transform, which generates once to choose a tool when tools are present and no Jinja
|
||||
template is in use, so the worker leaves tools out of that count and adds their bytes instead.
|
||||
|
||||
At startup the worker sends the backend one empty user message with `max_tokens: 1` and reads `usage.prompt_tokens`,
|
||||
the count generation uses:
|
||||
the count used by generation:
|
||||
|
||||
- With an exact counter, the counter has to reach that count for the same body, or the worker logs the mismatch and
|
||||
does not declare the guarantee. This catches a counter that renders with another template or without the BOS token.
|
||||
A counter route that does not answer stops the worker.
|
||||
- With `bytes`, the count is the template's fixed part (a BOS token, a system prompt the template injects, the
|
||||
generation prompt), added to the bound. A byte-level BPE tokenizer emits at least one UTF-8 byte per token, so the
|
||||
JSON's length bounds the message text, and the JSON keys cover each message's role markup. For English the bound
|
||||
is about four times the real count. The horde sizes context at about 3 characters per token, so a worker on the
|
||||
byte bound faults long conversations the horde sends it with `prompt_overflow`, and more than 20 faults in an hour
|
||||
put a worker into maintenance. Use an exact counter on a backend that has one.
|
||||
- With an exact counter, a count below `usage.prompt_tokens` for the same body is logged as a mismatch, and the
|
||||
worker does not declare the guarantee. A counter route that does not respond stops the worker.
|
||||
- With `bytes`, `usage.prompt_tokens` is added to the bound as the template's fixed part.
|
||||
|
||||
The byte bound counts far more tokens than generation uses, so a worker on it faults long conversations with
|
||||
`prompt_overflow`. Use an exact counter on a backend that has one.
|
||||
[Prompt fit on the worker](../explanation/chat_requests.md#prompt-fit-on-the-worker) explains why each counter backs
|
||||
the guarantee and what the byte bound costs.
|
||||
|
||||
The worker does not declare the guarantee, and serves only `best_effort` requests, when the backend reports no
|
||||
`usage`, when the counter undercounts, or when the byte bound runs with `tools` in `--features` and without
|
||||
@@ -106,8 +105,8 @@ tools counter 175 generation 175 ok
|
||||
oversized prompt status 400 refused
|
||||
```
|
||||
|
||||
That output is from llama.cpp build b12020 serving Llama 3.2 3B Instruct; the counts differ by model. It exits
|
||||
non-zero when a count falls short or the backend trims.
|
||||
The counts differ by model and chat template. The script exits non-zero when a count falls short or the backend
|
||||
trims.
|
||||
|
||||
## 1. Start an OpenAI-compatible backend
|
||||
|
||||
@@ -120,7 +119,7 @@ llama-server --model <model.gguf> --ctx-size 4096 --parallel 1 --jinja --port 80
|
||||
```
|
||||
|
||||
`--jinja` renders each conversation with the model's chat template, and `--parallel 1` gives one conversation the
|
||||
whole context. With vLLM, start it with `--served-model-name <model-name>`, since vLLM answers only requests for a
|
||||
whole context. With vLLM, start it with `--served-model-name <model-name>`, since vLLM accepts only requests for a
|
||||
model it serves and the worker sends `<model-name>` as `model`.
|
||||
|
||||
Verification: the probe the worker sends at startup returns a prompt count:
|
||||
@@ -193,52 +192,12 @@ Verification: the details show the declaration of the last pop
|
||||
|
||||
Worker details are cached for 30 seconds, so a changed declaration can take that long to show.
|
||||
|
||||
## 4. Send a chat request
|
||||
|
||||
```bash
|
||||
REQUEST_ID=$(curl -s "$HORDE_URL/api/v2/generate/chat/async" \
|
||||
-H "apikey: $HORDE_WORKER_API_KEY" \
|
||||
-H "Client-Agent: minimal-chat-worker-howto:1:unknown" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "<model-name>",
|
||||
"messages": [
|
||||
{"role": "system", "content": "Answer in one short sentence."},
|
||||
{"role": "user", "content": "What colour is a clear daytime sky?"}
|
||||
],
|
||||
"max_tokens": 64
|
||||
}' | tee /dev/stderr | jq -r .id)
|
||||
```
|
||||
|
||||
Verification: the horde answers `202` with `{"id": "<request-id>", "kudos": <kudos>}`, and within a few seconds the
|
||||
worker logs `Job <job-id>: stop`.
|
||||
|
||||
## 5. Read the reply
|
||||
|
||||
```bash
|
||||
curl -s "$HORDE_URL/api/v2/generate/chat/status/$REQUEST_ID" -H "apikey: $HORDE_WORKER_API_KEY" \
|
||||
| jq '{done, generations: [.generations[] | {worker_name, message, finish_reason, prompt_tokens, completion_tokens}]}'
|
||||
```
|
||||
|
||||
Verification: once `done` is true, the generation carries the backend's reply:
|
||||
|
||||
```json
|
||||
{
|
||||
"done": true,
|
||||
"generations": [
|
||||
{
|
||||
"worker_name": "<worker-name>",
|
||||
"message": {"role": "assistant", "content": "A clear daytime sky is blue."},
|
||||
"finish_reason": "stop",
|
||||
"prompt_tokens": 38,
|
||||
"completion_tokens": 9
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
A faulted job shows in the status as `restarted` and goes to the next eligible worker. The worker logs the job with
|
||||
`prompt_overflow`, `generation_error` or `reply_refused` in place of the finish reason.
|
||||
To send the worker a chat request and read the reply, follow steps 3 and 4 of
|
||||
[Serve your first chat job with koboldcpp](../tutorial/first_chat_worker.md#3-send-a-chat-request) with
|
||||
`HORDE_API_KEY=$HORDE_WORKER_API_KEY` and `MODEL_NAME=<model-name>`, without the `koboldcpp/` prefix. The worker logs
|
||||
`Job <job-id>: stop` for a completed job. A faulted job shows in the status as `restarted` and goes to the next
|
||||
eligible worker. The worker logs it with `prompt_overflow`, `generation_error` or `reply_refused` in place of the
|
||||
finish reason.
|
||||
|
||||
## Extensions
|
||||
|
||||
@@ -259,6 +218,6 @@ Each extension adds to `serve_one_pop` or the declaration:
|
||||
## Related
|
||||
|
||||
- [Serve chat jobs from a text worker](serve_chat_jobs.md): every declaration field, fault reason and submit rule.
|
||||
- [Serve your first chat job with koboldcpp](../tutorial/first_chat_worker.md): koboldcpp's embedded worker, which needs
|
||||
no separate worker process.
|
||||
- [Serve your first chat job with koboldcpp](../tutorial/first_chat_worker.md): koboldcpp's embedded worker, which does
|
||||
not need a separate worker process.
|
||||
- [Chat requests reference](../reference/chat_requests.md): every pop, payload and submit field.
|
||||
@@ -25,6 +25,7 @@ The concepts they assume (what a worker is, what a job's lifecycle means) are in
|
||||
| [Image model reference loader reference](model_reference.md) | Where the image model reference comes from, how beta (pending) models are merged over it, the environment that configures both, and what the loader exposes. |
|
||||
| [Style contract reference](style_contract.md) | What an image or text style may declare about the params a request may set and the placeholders its prompt accepts, and how a text request is sized against the prompt it ends up sending. |
|
||||
| [Chat requests reference](chat_requests.md) | Chat routes and their text mapping, known models, operation keys, chat declaration on pop, chat job payload, structured submit, pricing, chat model list, admission flag and completion signal. |
|
||||
| [Chat wire examples](chat_wire_examples.md) | Annotated JSON for the chat bodies of native requesters, OpenAI-compatible front clients and text workers: requests, statuses, completions, stream chunks, refusals, model lists and jobs. |
|
||||
| [Chat front reference](chat_front.md) | The horde.oai package by concern, its request keys, message roles, response, stream and model card shapes, its metrics and labels, and its per-request log line. |
|
||||
| [Outbound requests reference](outbound_requests.md) | Which client-supplied URLs the horde requests, the address check every hop passes, and the timeouts, redirect limits and byte caps of image fetches and webhooks. |
|
||||
<!-- END GENERATED: documents -->
|
||||
@@ -51,6 +51,8 @@ model list routes over the horde's native `/v2/generate/chat` routes (see
|
||||
|
||||
## Request keys
|
||||
|
||||
Example: [Front request](chat_wire_examples.md#front-request).
|
||||
|
||||
The front validates a request against `ChatCompletionRequest` after three screening steps in `horde/oai/schema.py`.
|
||||
A body that is not JSON, or that holds `NaN`, `Infinity` or `-Infinity` (which Python's parser otherwise accepts), is
|
||||
400 `BadRequest` before screening.
|
||||
@@ -122,6 +124,10 @@ An unserved role (such as `function`) or content part type (such as `input_audio
|
||||
|
||||
## Response shape
|
||||
|
||||
Examples: [Completion](chat_wire_examples.md#completion), [Tool-call completion](chat_wire_examples.md#tool-call-completion),
|
||||
and the streamed [Content chunk](chat_wire_examples.md#content-chunk),
|
||||
[Finish chunk](chat_wire_examples.md#finish-chunk) and [Usage chunk](chat_wire_examples.md#usage-chunk).
|
||||
|
||||
`choices[i].message` is `{"role": "assistant", "content": ..., "tool_calls": [...]}`. `content` is always written and
|
||||
is null only when the worker returned tool calls without text; `tool_calls` is written only when the worker returned
|
||||
calls, each `{"id", "type": "function", "function": {"name", "arguments"}}` as the worker sent it. `finish_reason` is
|
||||
@@ -265,7 +271,12 @@ the status carries an `impossible_reason` of kind `declaration_gap` (see
|
||||
[Impossible reason](chat_requests.md#impossible-reason)), it cancels at once and returns 404
|
||||
`no_worker_supports_features`, OpenRouter's status for no endpoint supporting the requested features, without
|
||||
a `Retry-After`. The message lists the missing declarations or the exceeded limit, and adds that guarantees are not
|
||||
required under `best_effort` when a guarantee is missing.
|
||||
required under `best_effort` when a guarantee is missing. The `type` is `not_found_error`, `param` is null, and
|
||||
`metadata.request_id` holds the cancelled native request:
|
||||
|
||||
```json
|
||||
--8<-- "docs/reference/chat_wire_examples.md:front-no-worker-supports-features"
|
||||
```
|
||||
|
||||
A gap of only `max_length` or `max_context_length` in `exceeded_limits` is a request over every online worker's
|
||||
size limits for the request's models. At submit the Horde lowers a `max_tokens` above the largest
|
||||
@@ -319,12 +330,19 @@ An `impossible_reason` of a kind the front does not know is ignored, so the grac
|
||||
A request carrying `horde.max_kudos` is forwarded with it, and the native route refuses it before creating it when
|
||||
the kudos estimate exceeds the cap (`ChatKudosCapExceeded`, see [Request body](chat_requests.md#request-body)).
|
||||
`_kudos_cap_error` in `horde/oai/app.py` returns 400 `invalid_request_error` with code `max_kudos_exceeded`, `param`
|
||||
`horde.max_kudos`, the native message, and `metadata.kudos_estimate` and `metadata.max_kudos`. The native error body
|
||||
holds only the message and the `rc`, so both values are read from the message with `KUDOS_CAP_EXCESS_PATTERN` in
|
||||
`horde/chat_contract.py`, the module that writes it; a message that does not match leaves them out of `metadata`.
|
||||
`horde.max_kudos`, the native message, and `metadata.rc`, `metadata.kudos_estimate` and `metadata.max_kudos`. The
|
||||
native error body holds only the message and the `rc`, so both values are read from the message with
|
||||
`KUDOS_CAP_EXCESS_PATTERN` in `horde/chat_contract.py`, the module that writes it; a message that does not match
|
||||
leaves them out of `metadata`.
|
||||
|
||||
```json
|
||||
--8<-- "docs/reference/chat_wire_examples.md:front-kudos-cap"
|
||||
```
|
||||
|
||||
## Model card fields
|
||||
|
||||
Example: [Model list](chat_wire_examples.md#model-list).
|
||||
|
||||
`GET /v1/models` lists each text model with at least one online chat worker. `context_length` and
|
||||
`top_provider.max_completion_tokens` are the largest the workers declare; `architecture.input_modalities` adds `image`
|
||||
for `vision`. `supported_parameters` is `max_tokens`, `temperature`, `top_p` and `stop`, then `seed`,
|
||||
|
||||
@@ -87,7 +87,7 @@ those workers keep receiving text jobs.
|
||||
|
||||
| Method and path | Behavior |
|
||||
| ------------------------------------------ | -------------------------------------------------------------------------- |
|
||||
| `POST /v2/generate/chat/async` | Queue a chat request. `202` with `{id, kudos, message, warnings}`, as text; `200` with `{id, kudos}` for a repeated operation key |
|
||||
| `POST /v2/generate/chat/async` | Queue a chat request. `202` with `{id, kudos}`, plus `message` and `warnings` when the horde warns, as text; `200` with `{id, kudos}` for a repeated operation key. See [Request accepted](chat_wire_examples.md#request-accepted) |
|
||||
| `GET /v2/generate/chat/status/<id>` | Full status, with each finished generation and its structured reply |
|
||||
| `DELETE /v2/generate/chat/status/<id>` | Cancel, as text |
|
||||
|
||||
@@ -199,6 +199,9 @@ returns `429`.
|
||||
|
||||
## Request body
|
||||
|
||||
Examples: [Request](chat_wire_examples.md#request), [Request accepted](chat_wire_examples.md#request-accepted),
|
||||
[Dry run](chat_wire_examples.md#dry-run), [Status when done](chat_wire_examples.md#status-when-done).
|
||||
|
||||
| Key | Mapping |
|
||||
| ---------------------------------------- | -------------------------------------------------------------------------------- |
|
||||
| `model` | The model the request accepts. A request sends exactly one of `model` and `models` |
|
||||
@@ -445,6 +448,8 @@ the client's; only the worker's copy is narrowed (see [Chat job pop response](#c
|
||||
|
||||
### Refused keys and values
|
||||
|
||||
Example: [UnsupportedChatParameter](chat_wire_examples.md#unsupportedchatparameter).
|
||||
|
||||
Any key outside the lists above returns `400 UnsupportedChatParameter`, and the message holds the key. This covers
|
||||
`logprobs`, `top_logprobs`, `logit_bias`, `modalities`, `audio` and `prediction`, as well as:
|
||||
|
||||
@@ -469,6 +474,8 @@ UUID, neither `model` nor `models`, a `models` that is empty, holds an empty ent
|
||||
|
||||
### Context adjustments
|
||||
|
||||
Example: [Status with context adjustments](chat_wire_examples.md#status-with-context-adjustments).
|
||||
|
||||
When sizing under `grow` or `truncate_oldest` raises the context length to fit the messages, the request stores the change and the status
|
||||
route returns it as `context_adjustments`: `[{"from": 2048, "to": 4096, "reason": "grow"}]`, where `from` is the
|
||||
requested `max_context_length` or the default. It is `null` when the horde did not change the context or
|
||||
@@ -539,6 +546,8 @@ arguments, the tool definitions and the response format are JSON documents and a
|
||||
|
||||
## Chat declaration on pop
|
||||
|
||||
Examples: [Pop request](chat_wire_examples.md#pop-request), [ChatDeclarationInvalid](chat_wire_examples.md#chatdeclarationinvalid).
|
||||
|
||||
`POST /v2/generate/text/pop` takes six further fields. The worker row stores them at every pop, including a pop
|
||||
inside the 30 second window in which the other check-in fields are not rewritten, and the pop is matched against
|
||||
them. A pop without `chat_jobs` therefore withdraws the whole declaration and does not receive a chat job.
|
||||
@@ -655,6 +664,10 @@ not redefine work already handed out.
|
||||
|
||||
### Chat job pop response
|
||||
|
||||
Examples: [Chat job](chat_wire_examples.md#chat-job), [with tools](chat_wire_examples.md#chat-job-with-tools),
|
||||
[with an image](chat_wire_examples.md#chat-job-with-an-image),
|
||||
[with offloaded messages](chat_wire_examples.md#chat-job-with-offloaded-messages), [No job](chat_wire_examples.md#no-job).
|
||||
|
||||
A text job's pop response does not carry a `chat` key. A chat job's response has the same top-level fields plus
|
||||
`chat: true`, and its payload is:
|
||||
|
||||
@@ -755,6 +768,10 @@ pop.
|
||||
|
||||
## Structured submit
|
||||
|
||||
Examples: [Reply](chat_wire_examples.md#reply), [Tool-call reply](chat_wire_examples.md#tool-call-reply),
|
||||
[Reply with reasoning](chat_wire_examples.md#reply-with-reasoning), [Fault](chat_wire_examples.md#fault), and the
|
||||
refusals from [ChatSubmitInvalid](chat_wire_examples.md#chatsubmitinvalid) on.
|
||||
|
||||
`POST /v2/generate/text/submit` keeps `generation` for text jobs. A chat job sends these instead:
|
||||
|
||||
| Field | Type | Required | Meaning |
|
||||
@@ -847,10 +864,8 @@ a completion without visible text is logged with its completion limit (see [Metr
|
||||
Before the compare-and-set, `verify_chat_reply` checks the reply against the request's stored `chat_extras` (the
|
||||
client's values, not the narrowed copy the worker received) under its `chat_conformance`. It is the third layer of
|
||||
ADR 0036, after eligibility and dispatch narrowing, and does not trust any worker declaration. A refused submit leaves the
|
||||
generation pending, the koboldcpp worker faults the job with `reply_refused` after `ChatSubmitInvalid`,
|
||||
`ChatSubmitNonconformant` or `ChatModelMismatch`, and the request faults after
|
||||
`FAULTED_GENERATIONS_LIMIT` faulted jobs, so the checks cover only what the payload proves and what OpenAI never
|
||||
returns.
|
||||
generation pending for the worker to fault ([Fault a job](../how-to/serve_chat_jobs.md#7-fault-a-job)), and the
|
||||
request faults after `FAULTED_GENERATIONS_LIMIT` faulted jobs. The checks therefore cover only what the payload proves and what OpenAI never returns.
|
||||
|
||||
| Rule | `strict` | `best_effort` |
|
||||
| ----------------------------------------------------------------------------------- | -------------------------------- | ------------------ |
|
||||
@@ -904,6 +919,8 @@ whose reply was refused as nonconformant. Worker details (`GET /v2/workers`, `/v
|
||||
|
||||
### Worker details
|
||||
|
||||
Example: [Worker details](chat_wire_examples.md#worker-details).
|
||||
|
||||
Text worker details carry the declaration of the worker's last pop. `accepts_chat_jobs` and `accepts_prompt_jobs` are
|
||||
the pop's `chat_jobs` and `prompt_jobs`. `chat` mirrors the pop's chat fields, and is absent when the worker does not
|
||||
accept chat jobs:
|
||||
@@ -1008,11 +1025,7 @@ the model" for exceeded limits. With a gap, the body also carries `impossible_re
|
||||
reports (see [Impossible reason](#impossible-reason)):
|
||||
|
||||
```json
|
||||
{
|
||||
"message": "No online worker can serve this request: no online worker serving the model declares vision.",
|
||||
"rc": "NoEligibleWorker",
|
||||
"impossible_reason": {"kind": "declaration_gap", "missing_declarations": ["vision"], "exceeded_limits": []}
|
||||
}
|
||||
--8<-- "docs/reference/chat_wire_examples.md:refusal-no-eligible-worker"
|
||||
```
|
||||
|
||||
A request that covers its hold, or fits the floor, is created and queued whether or not a worker can serve it.
|
||||
@@ -1066,6 +1079,8 @@ the stored `max_length` at assignment. The downgrade log line is kept beside the
|
||||
|
||||
## Chat model list
|
||||
|
||||
Example: [Chat model list](chat_wire_examples.md#chat-model-list).
|
||||
|
||||
`GET /v2/status/models?type=text&chat=true` lists only the text models that at least one online chat worker serves,
|
||||
each with the usual fields plus a `chat` object. Without `chat=true` the entries do not carry `chat` and the
|
||||
response is the plain list. `chat=true` with another `type` returns `400`.
|
||||
@@ -1088,6 +1103,10 @@ changes when the earliest of those workers goes offline; the Horde does not keep
|
||||
|
||||
## Streaming progress
|
||||
|
||||
Examples: [Progress post](chat_wire_examples.md#progress-post), [Progress accepted](chat_wire_examples.md#progress-accepted),
|
||||
[Progress after a cancel](chat_wire_examples.md#progress-after-a-cancel),
|
||||
[Progress out of sequence](chat_wire_examples.md#progress-out-of-sequence).
|
||||
|
||||
A request sent with `stream: true` is stored with the stream hint, `stream_progress: true` in `chat_extras`, and its
|
||||
job payload carries the same key. The hint does not change the reply and is not in the operation fingerprint. A worker posts
|
||||
the text it generated so far to `POST /v2/generate/text/progress` only for a job whose payload carries the hint, and
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -39,6 +39,7 @@ procedure for one subject sit together.
|
||||
- How-to: [Add chat completions to a web frontend](how-to/add_chat_to_a_frontend.md)
|
||||
- How-to: [Write a minimal chat worker](how-to/write_a_minimal_chat_worker.md)
|
||||
- How-to: [Serve chat jobs from a text worker](how-to/serve_chat_jobs.md)
|
||||
- Reference: [Chat wire examples](reference/chat_wire_examples.md)
|
||||
- Tutorial: [Your first chat completion through the Horde](tutorial/first_chat_completion.md)
|
||||
- Tutorial: [Serve your first chat job with koboldcpp](tutorial/first_chat_worker.md)
|
||||
|
||||
@@ -91,6 +92,7 @@ procedure for one subject sit together.
|
||||
- Reference: [Moderation operations reference](reference/moderation_operations.md)
|
||||
- Reference: [Style contract reference](reference/style_contract.md)
|
||||
- Reference: [Chat requests reference](reference/chat_requests.md)
|
||||
- Reference: [Chat wire examples](reference/chat_wire_examples.md)
|
||||
- Reference: [Chat front reference](reference/chat_front.md)
|
||||
- Reference: [Outbound requests reference](reference/outbound_requests.md)
|
||||
- Tutorial: [Your first chat completion through the Horde](tutorial/first_chat_completion.md)
|
||||
@@ -105,5 +107,6 @@ procedure for one subject sit together.
|
||||
- Reference: [Image baseline policy reference](reference/baseline_policy.md)
|
||||
- Reference: [Image model reference loader reference](reference/model_reference.md)
|
||||
- Reference: [Chat requests reference](reference/chat_requests.md)
|
||||
- Reference: [Chat wire examples](reference/chat_wire_examples.md)
|
||||
- Tutorial: [Serve your first chat job with koboldcpp](tutorial/first_chat_worker.md)
|
||||
<!-- END GENERATED: topics -->
|
||||
@@ -65,7 +65,7 @@ Worker <n> - Embedded Horde Worker '<worker-name>' is started.
|
||||
```
|
||||
|
||||
Leave koboldcpp running, and do not use its local web interface during the tutorial: the embedded worker pauses
|
||||
popping for 20 seconds after each local request.
|
||||
popping for 20 seconds after each local generation request.
|
||||
|
||||
## 2. Watch the worker register as a chat worker
|
||||
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
# SPDX-FileCopyrightText: 2026 Tazlin <tazlin@haidra.net>
|
||||
#
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""The examples of ``docs/reference/chat_wire_examples.md`` as the wire example tests read and compare them.
|
||||
|
||||
[`DOCUMENTED`][tests.integration._wire_examples.DOCUMENTED] holds the page's examples by marker id.
|
||||
[`structure_differences`][tests.integration._wire_examples.structure_differences] compares a documented body with a
|
||||
live one, and [`filled`][tests.integration._wire_examples.filled] replaces a documented request's placeholders with
|
||||
live values before it is posted.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
__all__ = [
|
||||
"DOCUMENTED",
|
||||
"FRONT_EXAMPLE_PREFIX",
|
||||
"MODEL_NAME_PLACEHOLDER",
|
||||
"PAGE_PATH",
|
||||
"DocumentedExample",
|
||||
"filled",
|
||||
"structure_differences",
|
||||
]
|
||||
|
||||
PAGE_PATH = Path(__file__).resolve().parents[2] / "docs" / "reference" / "chat_wire_examples.md"
|
||||
"""The page whose examples the wire example tests check."""
|
||||
|
||||
EXAMPLE_PATTERN = re.compile(r"<!-- wire-example: (?P<id>[a-z0-9-]+) -->\r?\n```json\r?\n(?P<body>.*?)```", re.DOTALL)
|
||||
"""One example: its marker line, then the fenced ``json`` block that follows it."""
|
||||
|
||||
SNIPPET_SECTION_LINE = re.compile(r"^.*--8<-- \[(?:start|end):[a-z][-_0-9a-z]*\].*\r?\n", re.MULTILINE)
|
||||
"""A ``pymdownx.snippets`` section marker line inside an example. Other pages include the example through its section,
|
||||
and the build removes the marker lines."""
|
||||
|
||||
PLACEHOLDER_PATTERN = re.compile(r"<[a-z0-9-]+>")
|
||||
"""A whole string the page writes in place of an id, a key, a name or a kudos value, such as ``<job-id>``."""
|
||||
|
||||
FRONT_EXAMPLE_PREFIX = "front-"
|
||||
"""The id prefix of the OpenAI-compatible front's examples, whose scenarios live in
|
||||
``test_chat_front_wire_examples.py`` because they need the front harness."""
|
||||
|
||||
MODEL_NAME_PLACEHOLDER = "<model-name>"
|
||||
"""The placeholder the page writes for the chat model."""
|
||||
|
||||
VALUE_KEYS = frozenset(
|
||||
{
|
||||
"rc",
|
||||
"code",
|
||||
"type",
|
||||
"object",
|
||||
"state",
|
||||
"finish_reason",
|
||||
"kind",
|
||||
"reason",
|
||||
},
|
||||
)
|
||||
"""The keys whose string values are part of the contract, so a documented value is compared with the live one; every
|
||||
other string can differ by scenario and is compared by type."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DocumentedExample:
|
||||
"""Represents one example as the page holds it."""
|
||||
|
||||
body: Any
|
||||
"""The parsed JSON block."""
|
||||
|
||||
prose: str
|
||||
"""The page text between the previous example and this example's marker."""
|
||||
|
||||
|
||||
def _documented_examples() -> dict[str, DocumentedExample]:
|
||||
page = PAGE_PATH.read_text(encoding="utf-8")
|
||||
examples: dict[str, DocumentedExample] = {}
|
||||
previous_end = 0
|
||||
for match in EXAMPLE_PATTERN.finditer(page):
|
||||
assert match["id"] not in examples, f"The marker {match['id']} appears twice."
|
||||
body = json.loads(SNIPPET_SECTION_LINE.sub("", match["body"]))
|
||||
examples[match["id"]] = DocumentedExample(body=body, prose=page[previous_end : match.start()])
|
||||
previous_end = match.end()
|
||||
return examples
|
||||
|
||||
|
||||
DOCUMENTED = _documented_examples()
|
||||
"""The page's examples by marker id, read once at import."""
|
||||
|
||||
|
||||
def filled(documented: Any, placeholders: dict[str, str]) -> Any:
|
||||
"""Return a documented body with each placeholder string replaced by its live value.
|
||||
|
||||
Args:
|
||||
documented: The documented body.
|
||||
placeholders: The live value of each placeholder.
|
||||
|
||||
Returns:
|
||||
The body to post.
|
||||
"""
|
||||
if isinstance(documented, dict):
|
||||
return {key: filled(value, placeholders) for key, value in documented.items()}
|
||||
if isinstance(documented, list):
|
||||
return [filled(element, placeholders) for element in documented]
|
||||
if isinstance(documented, str):
|
||||
return placeholders.get(documented, documented)
|
||||
return documented
|
||||
|
||||
|
||||
def _json_type(value: Any) -> str:
|
||||
if value is None:
|
||||
return "null"
|
||||
if isinstance(value, bool):
|
||||
return "boolean"
|
||||
if isinstance(value, int | float):
|
||||
return "number"
|
||||
return "string"
|
||||
|
||||
|
||||
def _scalar_difference(documented: Any, live: Any, path: str, key: str | None) -> list[str]:
|
||||
is_placeholder = isinstance(documented, str) and PLACEHOLDER_PATTERN.fullmatch(documented) is not None
|
||||
if is_placeholder:
|
||||
return []
|
||||
is_compared_by_value = isinstance(documented, bool) or (isinstance(documented, str) and key in VALUE_KEYS)
|
||||
if is_compared_by_value:
|
||||
has_documented_value = _json_type(documented) == _json_type(live) and documented == live
|
||||
return [] if has_documented_value else [f"{path}: documented {documented!r}, live {live!r}"]
|
||||
if _json_type(documented) != _json_type(live):
|
||||
return [f"{path}: documented a {_json_type(documented)}, live {live!r}"]
|
||||
return []
|
||||
|
||||
|
||||
def structure_differences(documented: Any, live: Any, path: str = "$", key: str | None = None) -> list[str]:
|
||||
"""Return where a documented body departs from the live body.
|
||||
|
||||
Objects are compared by their key sets and arrays by their first elements. A string under one of ``VALUE_KEYS``
|
||||
and every boolean are compared by value, and every other scalar by its JSON type; a placeholder such as
|
||||
``<kudos>`` matches any live value.
|
||||
|
||||
Args:
|
||||
documented: The documented body.
|
||||
live: The body the code sent or accepted.
|
||||
path: The JSON path of ``documented`` within the example, for the messages.
|
||||
key: The object key ``documented`` sits under; an array element inherits its array's key.
|
||||
|
||||
Returns:
|
||||
One message per difference, empty when the bodies agree.
|
||||
"""
|
||||
if isinstance(documented, dict) or isinstance(live, dict):
|
||||
if not (isinstance(documented, dict) and isinstance(live, dict)):
|
||||
return [f"{path}: one side is an object and the other is not"]
|
||||
differences = (
|
||||
[f"{path}: keys {sorted(documented.keys() ^ live.keys())} are on one side only"] if documented.keys() != live.keys() else []
|
||||
)
|
||||
for shared_key in sorted(documented.keys() & live.keys()):
|
||||
differences += structure_differences(documented[shared_key], live[shared_key], f"{path}.{shared_key}", shared_key)
|
||||
return differences
|
||||
if isinstance(documented, list) or isinstance(live, list):
|
||||
if not (isinstance(documented, list) and isinstance(live, list)):
|
||||
return [f"{path}: one side is an array and the other is not"]
|
||||
if bool(documented) != bool(live):
|
||||
return [f"{path}: one side is an empty array"]
|
||||
return structure_differences(documented[0], live[0], f"{path}[0]", key) if documented else []
|
||||
return _scalar_difference(documented, live, path, key)
|
||||
@@ -0,0 +1,290 @@
|
||||
# SPDX-FileCopyrightText: 2026 Tazlin <tazlin@haidra.net>
|
||||
#
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""The OpenAI-compatible front's wire examples in ``docs/reference/chat_wire_examples.md`` match the front.
|
||||
|
||||
The examples whose id starts with ``FRONT_EXAMPLE_PREFIX`` are checked here against the front served by the harness
|
||||
of ``test_chat_front_sdk.py``: uvicorn in front of the in-process Horde, with that module's fake worker serving the
|
||||
chat model. Each response example is compared with the live body as
|
||||
[`structure_differences`][tests.integration._wire_examples.structure_differences] describes; the bodies are read as
|
||||
raw HTTP, so each is the JSON the front writes and not an SDK object. The request example is posted and has to get the
|
||||
status of the response example that answers it; the front refuses unknown keys, so a misspelt key fails the post.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Callable, Iterator
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from tests.integration._fake_chat_worker import take_worker_offline
|
||||
from tests.integration._wire_examples import (
|
||||
DOCUMENTED,
|
||||
FRONT_EXAMPLE_PREFIX,
|
||||
MODEL_NAME_PLACEHOLDER,
|
||||
filled,
|
||||
structure_differences,
|
||||
)
|
||||
from tests.integration.test_chat_front_sdk import (
|
||||
CHAT_MESSAGES,
|
||||
CHAT_MODEL,
|
||||
SDK_TIMEOUT_SECONDS,
|
||||
TOOL_MESSAGES,
|
||||
WEATHER_TOOL,
|
||||
WORKER_NAME,
|
||||
FixedReplyWorker,
|
||||
_no_rate_limit, # noqa: F401 (registers the autouse fixture in this module)
|
||||
_worker_kept_up, # noqa: F401 (registers the autouse fixture in this module)
|
||||
fixed_reply_worker, # noqa: F401 (registers the fixture in this module)
|
||||
front_base_url, # noqa: F401 (registers the fixture in this module)
|
||||
horde_api_root, # noqa: F401 (registers the fixture in this module)
|
||||
owner_api_key, # noqa: F401 (registers the fixture in this module)
|
||||
signal_redis_url, # noqa: F401 (registers the fixture in this module)
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from flask import Flask
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
COMPLETIONS_PATH = "chat/completions"
|
||||
"""The chat completions route, relative to the front's ``/v1`` base URL."""
|
||||
|
||||
SSE_DATA_PREFIX = "data: "
|
||||
"""The prefix of each server-sent event line that carries a chunk."""
|
||||
|
||||
SSE_DONE = "[DONE]"
|
||||
"""The data of the event that ends a stream."""
|
||||
|
||||
FRONT_CHAT_BODY: dict[str, Any] = {
|
||||
"model": CHAT_MODEL,
|
||||
"messages": CHAT_MESSAGES,
|
||||
"max_tokens": 64,
|
||||
"temperature": 0.5,
|
||||
}
|
||||
"""The body the response and stream examples send, which the documented front request holds as well."""
|
||||
|
||||
FEATURE_GAP_REASONING_EFFORT = "low"
|
||||
"""A ``reasoning_effort``, which needs a feature the fake worker does not declare, so the request has a declaration
|
||||
gap."""
|
||||
|
||||
KUDOS_CAP = 1
|
||||
"""A ``horde.max_kudos`` below the quote of any chat request, so the native route refuses the request."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FrontCapture:
|
||||
"""One body the front sent, with the HTTP status of the response that carried it."""
|
||||
|
||||
status: int
|
||||
"""The HTTP status; for a streamed chunk, the status of the stream."""
|
||||
|
||||
body: Any
|
||||
"""The parsed JSON body or chunk."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FrontRun:
|
||||
"""Holds one test's HTTP client for the front."""
|
||||
|
||||
http: httpx.Client
|
||||
"""A client addressed at the front's ``/v1`` base URL, carrying the owner's key as its bearer token."""
|
||||
|
||||
def complete(self, chat_body: dict[str, Any]) -> FrontCapture:
|
||||
"""Send a non-streamed completion request and return its answer.
|
||||
|
||||
Args:
|
||||
chat_body: The request body.
|
||||
|
||||
Returns:
|
||||
The response's status and JSON body.
|
||||
"""
|
||||
response = self.http.post(COMPLETIONS_PATH, json=chat_body)
|
||||
return FrontCapture(status=response.status_code, body=response.json())
|
||||
|
||||
def stream_chunks(self) -> tuple[int, list[dict[str, Any]]]:
|
||||
"""Send the documented request streamed with usage and return the stream's status and its chunks in order.
|
||||
|
||||
Returns:
|
||||
The HTTP status and every ``data`` event before ``[DONE]``, parsed.
|
||||
"""
|
||||
streamed_body = {**FRONT_CHAT_BODY, "stream": True, "stream_options": {"include_usage": True}}
|
||||
with self.http.stream("POST", COMPLETIONS_PATH, json=streamed_body) as response:
|
||||
event_data = [line.removeprefix(SSE_DATA_PREFIX) for line in response.iter_lines() if line.startswith(SSE_DATA_PREFIX)]
|
||||
assert event_data[-1] == SSE_DONE, event_data
|
||||
return response.status_code, [json.loads(data) for data in event_data[:-1]]
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def _front_worker_offline(app: Flask, request: pytest.FixtureRequest) -> Iterator[None]:
|
||||
"""Start the fake worker, then stop it and take it offline when the module ends.
|
||||
|
||||
The harness leaves the worker online, which would make later tests' requests for its model possible. The thread
|
||||
stops first, since its next pop would bring the worker back online.
|
||||
"""
|
||||
worker: FixedReplyWorker = request.getfixturevalue("fixed_reply_worker")
|
||||
yield
|
||||
worker.stop()
|
||||
take_worker_offline(app, WORKER_NAME)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def front(request: pytest.FixtureRequest, _front_worker_offline: None) -> Iterator[FrontRun]:
|
||||
"""A scenario context with an HTTP client for the front.
|
||||
|
||||
The harness fixtures are read by name, since parameters named after the imported fixtures would shadow them.
|
||||
"""
|
||||
with httpx.Client(
|
||||
base_url=request.getfixturevalue("front_base_url"),
|
||||
headers={"Authorization": f"Bearer {request.getfixturevalue('owner_api_key')}"},
|
||||
timeout=SDK_TIMEOUT_SECONDS,
|
||||
) as http:
|
||||
yield FrontRun(http=http)
|
||||
|
||||
|
||||
def _post_front_request(front: FrontRun, documented: dict[str, Any]) -> httpx.Response:
|
||||
return front.http.post(COMPLETIONS_PATH, json=filled(documented, {MODEL_NAME_PLACEHOLDER: CHAT_MODEL}))
|
||||
|
||||
|
||||
def _live_front_response(front: FrontRun) -> FrontCapture:
|
||||
return front.complete(FRONT_CHAT_BODY)
|
||||
|
||||
|
||||
def _live_stream_chunk(front: FrontRun, select: Callable[[dict[str, Any]], bool]) -> FrontCapture:
|
||||
"""Return the one chunk of the documented stream that ``select`` picks."""
|
||||
status, chunks = front.stream_chunks()
|
||||
[chunk] = [chunk for chunk in chunks if select(chunk)]
|
||||
return FrontCapture(status=status, body=chunk)
|
||||
|
||||
|
||||
def _is_content_chunk(chunk: dict[str, Any]) -> bool:
|
||||
return bool(chunk["choices"]) and "content" in chunk["choices"][0]["delta"]
|
||||
|
||||
|
||||
def _is_finish_chunk(chunk: dict[str, Any]) -> bool:
|
||||
return bool(chunk["choices"]) and chunk["choices"][0]["finish_reason"] is not None
|
||||
|
||||
|
||||
def _is_usage_chunk(chunk: dict[str, Any]) -> bool:
|
||||
return not chunk["choices"]
|
||||
|
||||
|
||||
def _live_front_tool_call_response(front: FrontRun) -> FrontCapture:
|
||||
return front.complete({"model": CHAT_MODEL, "messages": TOOL_MESSAGES, "tools": [WEATHER_TOOL]})
|
||||
|
||||
|
||||
def _live_front_no_worker_supports_features(front: FrontRun) -> FrontCapture:
|
||||
return front.complete({**FRONT_CHAT_BODY, "reasoning_effort": FEATURE_GAP_REASONING_EFFORT})
|
||||
|
||||
|
||||
def _live_front_kudos_cap(front: FrontRun) -> FrontCapture:
|
||||
return front.complete({**FRONT_CHAT_BODY, "horde": {"max_kudos": KUDOS_CAP}})
|
||||
|
||||
|
||||
def _live_front_model_list(front: FrontRun) -> FrontCapture:
|
||||
response = front.http.get("models")
|
||||
return FrontCapture(status=response.status_code, body=response.json())
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FrontScenario:
|
||||
"""Produces one front response example's live body, or posts the front request example."""
|
||||
|
||||
live: Callable[[FrontRun], FrontCapture] | None = None
|
||||
"""Returns the live ``FrontCapture``; None for a request example."""
|
||||
|
||||
status: int | None = None
|
||||
"""The HTTP status of a response example, which its prose states; None for a request example."""
|
||||
|
||||
post: Callable[[FrontRun, dict[str, Any]], httpx.Response] | None = None
|
||||
"""Posts the documented request body with its placeholders filled in; None for a response example."""
|
||||
|
||||
answered_by: str | None = None
|
||||
"""The response example whose status the posted documented body gets."""
|
||||
|
||||
|
||||
FRONT_SCENARIOS: dict[str, FrontScenario] = {
|
||||
"front-request": FrontScenario(
|
||||
post=_post_front_request,
|
||||
answered_by="front-response",
|
||||
),
|
||||
"front-response": FrontScenario(
|
||||
live=_live_front_response,
|
||||
status=200,
|
||||
),
|
||||
"front-stream-content-chunk": FrontScenario(
|
||||
live=lambda front: _live_stream_chunk(front, _is_content_chunk),
|
||||
status=200,
|
||||
),
|
||||
"front-stream-finish-chunk": FrontScenario(
|
||||
live=lambda front: _live_stream_chunk(front, _is_finish_chunk),
|
||||
status=200,
|
||||
),
|
||||
"front-stream-usage-chunk": FrontScenario(
|
||||
live=lambda front: _live_stream_chunk(front, _is_usage_chunk),
|
||||
status=200,
|
||||
),
|
||||
"front-tool-call-response": FrontScenario(
|
||||
live=_live_front_tool_call_response,
|
||||
status=200,
|
||||
),
|
||||
"front-no-worker-supports-features": FrontScenario(
|
||||
live=_live_front_no_worker_supports_features,
|
||||
status=404,
|
||||
),
|
||||
"front-kudos-cap": FrontScenario(
|
||||
live=_live_front_kudos_cap,
|
||||
status=400,
|
||||
),
|
||||
"front-model-list": FrontScenario(
|
||||
live=_live_front_model_list,
|
||||
status=200,
|
||||
),
|
||||
}
|
||||
"""Every front example on the page by its marker id."""
|
||||
|
||||
|
||||
def test_every_front_marker_has_a_scenario_and_every_scenario_a_marker() -> None:
|
||||
"""The other examples are matched to their scenarios in ``test_chat_wire_examples.py``."""
|
||||
assert sorted(example_id for example_id in DOCUMENTED if example_id.startswith(FRONT_EXAMPLE_PREFIX)) == sorted(FRONT_SCENARIOS)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("example_id", sorted(example_id for example_id, scenario in FRONT_SCENARIOS.items() if scenario.live is not None))
|
||||
def test_each_front_response_example_matches_the_live_response(front: FrontRun, example_id: str) -> None:
|
||||
"""A front response example has the live body's keys, contract values and value types."""
|
||||
scenario = FRONT_SCENARIOS[example_id]
|
||||
assert scenario.live is not None
|
||||
|
||||
live = scenario.live(front)
|
||||
|
||||
assert live.status == scenario.status, json.dumps(live.body, indent=2)
|
||||
assert structure_differences(DOCUMENTED[example_id].body, live.body) == [], json.dumps(live.body, indent=2)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"example_id",
|
||||
sorted(example_id for example_id, scenario in FRONT_SCENARIOS.items() if scenario.post is not None),
|
||||
)
|
||||
def test_each_documented_front_request_body_gets_the_documented_status(front: FrontRun, example_id: str) -> None:
|
||||
"""The documented front request gets the status of the response example that answers it."""
|
||||
scenario = FRONT_SCENARIOS[example_id]
|
||||
assert scenario.post is not None
|
||||
assert scenario.answered_by is not None
|
||||
|
||||
response = scenario.post(front, DOCUMENTED[example_id].body)
|
||||
|
||||
assert response.status_code == FRONT_SCENARIOS[scenario.answered_by].status, response.text
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"example_id",
|
||||
sorted(example_id for example_id, scenario in FRONT_SCENARIOS.items() if scenario.status is not None),
|
||||
)
|
||||
def test_each_front_response_example_states_its_status(example_id: str) -> None:
|
||||
"""The prose before a front response example states the status the live response has."""
|
||||
assert f"`{FRONT_SCENARIOS[example_id].status}`" in DOCUMENTED[example_id].prose
|
||||
@@ -0,0 +1,757 @@
|
||||
# SPDX-FileCopyrightText: 2026 Tazlin <tazlin@haidra.net>
|
||||
#
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""The worker and native requester chat wire examples in ``docs/reference/chat_wire_examples.md`` match the code.
|
||||
|
||||
Each response example has a scenario here that produces the live body through the in-process app, compared as
|
||||
[`structure_differences`][tests.integration._wire_examples.structure_differences] describes. Each request example is
|
||||
posted with its placeholders filled in and has to get the status of the response example that answers it. The worker
|
||||
routes ignore keys they do not read, so each worker request example's keys are also checked against the route's
|
||||
parser and models. The examples whose id starts with ``FRONT_EXAMPLE_PREFIX`` are checked against the
|
||||
OpenAI-compatible front in ``test_chat_front_wire_examples.py``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import uuid
|
||||
from collections.abc import Callable, Iterator
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import pytest
|
||||
from flask_restx import fields as restx_fields
|
||||
|
||||
from tests.integration._fake_chat_worker import (
|
||||
CHAT_MESSAGES,
|
||||
CHAT_MODEL,
|
||||
FakeChatWorker,
|
||||
build_chat_submit_body,
|
||||
build_faulted_chat_submit_body,
|
||||
cancel_request,
|
||||
chat_status,
|
||||
submit_chat_request,
|
||||
take_worker_offline,
|
||||
)
|
||||
from tests.integration._wire_examples import (
|
||||
DOCUMENTED,
|
||||
FRONT_EXAMPLE_PREFIX,
|
||||
MODEL_NAME_PLACEHOLDER,
|
||||
filled,
|
||||
structure_differences,
|
||||
)
|
||||
from tests.integration.test_chat import (
|
||||
ANONYMOUS_HEADERS,
|
||||
LONG_HISTORY,
|
||||
LOOKUP_TOOL,
|
||||
TOOL_HISTORY,
|
||||
TOOLS_FEATURES,
|
||||
TOOLS_REQUEST_EXTRAS,
|
||||
_no_rate_limit, # noqa: F401 (registers the autouse fixture in this module)
|
||||
_serve_reply,
|
||||
)
|
||||
from tests.integration.test_chat_images import (
|
||||
MESSAGES_URL_TRANSPORTS,
|
||||
VISION_FEATURES,
|
||||
_image_body,
|
||||
_png_data_url,
|
||||
_submit,
|
||||
messages_offload, # noqa: F401 (registers the fixture in this module)
|
||||
)
|
||||
from tests.integration.test_chat_progress import (
|
||||
_post_progress,
|
||||
lua_redis, # noqa: F401 (registers the fixture in this module)
|
||||
progress_redis, # noqa: F401 (registers the fixture in this module)
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from flask import Flask
|
||||
from flask.testing import FlaskClient
|
||||
from flask_restx import Model
|
||||
from flask_restx.reqparse import RequestParser
|
||||
from werkzeug.test import TestResponse
|
||||
|
||||
CHAT_ASYNC_ROUTE = "/api/v2/generate/chat/async"
|
||||
"""The native route a requester queues a chat request on."""
|
||||
|
||||
NO_ELIGIBLE_WORKER_MAX_TOKENS = 1024
|
||||
"""A completion length above the free output floor of 512, which the anonymous key cannot hold kudos for, and within
|
||||
the scribe's chat limit, so the only gap the refusal states is the missing ``vision`` declaration."""
|
||||
|
||||
JOB_ID_PLACEHOLDER = "<job-id>"
|
||||
"""The placeholder the page writes for a generation id."""
|
||||
|
||||
WORKER_NAME_PLACEHOLDER = "<worker-name>"
|
||||
"""The placeholder the page writes for the worker name."""
|
||||
|
||||
BRIDGE_AGENT_PLACEHOLDER = "<bridge-agent>"
|
||||
"""The placeholder the page writes for the worker's bridge agent string."""
|
||||
|
||||
IMAGE_TRANSPORTS = ["url", "base64"]
|
||||
"""The image transports the documented worker details declare."""
|
||||
|
||||
SCRIBE_NAME = "CICD Wire Example Scribe"
|
||||
"""The worker most scenarios pop with; the user's worker count is capped, so scenarios share one."""
|
||||
|
||||
DECLARED_FEATURES = [
|
||||
"seed",
|
||||
"presence_penalty",
|
||||
"vision",
|
||||
"tool_choice_enforced",
|
||||
"prompt_overflow_refused",
|
||||
]
|
||||
"""The features and guarantees the documented worker details declare."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class WireRun:
|
||||
"""Holds one test's client and the workers and requests it creates, released when the test ends."""
|
||||
|
||||
app: Flask
|
||||
"""The in-process app."""
|
||||
|
||||
client: FlaskClient
|
||||
"""The Flask test client."""
|
||||
|
||||
headers: dict[str, str]
|
||||
"""The requesting user's headers, who also owns the workers."""
|
||||
|
||||
request: pytest.FixtureRequest
|
||||
"""The test's fixture request, for fixtures only some scenarios need."""
|
||||
|
||||
worker_names: list[str] = field(default_factory=list)
|
||||
"""The workers to take offline when the test ends."""
|
||||
|
||||
open_requests: list[str] = field(default_factory=list)
|
||||
"""The chat requests to cancel when the test ends."""
|
||||
|
||||
def worker(self, *, fresh_name: bool = False, **pop_options: Any) -> FakeChatWorker:
|
||||
"""Return a worker and bring it online with one pop, which finds no job.
|
||||
|
||||
Each pop replaces the stored declaration, so scenarios share ``SCRIBE_NAME`` with their own pop options.
|
||||
|
||||
Args:
|
||||
fresh_name: Whether to pop under a new name, for a scenario that reads worker details: single worker
|
||||
details are cached for 30 seconds per worker.
|
||||
**pop_options: Pop body options the worker sends on every pop.
|
||||
|
||||
Returns:
|
||||
The online worker.
|
||||
"""
|
||||
worker = FakeChatWorker(
|
||||
client=self.client,
|
||||
headers=self.headers,
|
||||
name=self.worker_name(fresh_name=fresh_name),
|
||||
models=[CHAT_MODEL],
|
||||
pop_options=pop_options,
|
||||
)
|
||||
assert worker.pop()["id"] is None
|
||||
return worker
|
||||
|
||||
def worker_name(self, *, fresh_name: bool = False) -> str:
|
||||
"""Return the name a scenario's worker pops under, tracked to be taken offline when the test ends.
|
||||
|
||||
Args:
|
||||
fresh_name: Whether to return a new name in place of ``SCRIBE_NAME``.
|
||||
|
||||
Returns:
|
||||
The worker name.
|
||||
"""
|
||||
worker_name = f"{SCRIBE_NAME} {uuid.uuid4().hex[:8]}" if fresh_name else SCRIBE_NAME
|
||||
if worker_name not in self.worker_names:
|
||||
self.worker_names.append(worker_name)
|
||||
return worker_name
|
||||
|
||||
def queue(self, **overrides: Any) -> str:
|
||||
"""Queue a chat request through the native route and track it for cancellation.
|
||||
|
||||
Args:
|
||||
**overrides: Request body overrides.
|
||||
|
||||
Returns:
|
||||
The request id.
|
||||
"""
|
||||
return self.track(submit_chat_request(self.client, self.headers, **overrides))
|
||||
|
||||
def track(self, request_id: str) -> str:
|
||||
"""Track a request queued elsewhere for cancellation.
|
||||
|
||||
Args:
|
||||
request_id: The request id.
|
||||
|
||||
Returns:
|
||||
The same request id.
|
||||
"""
|
||||
self.open_requests.append(request_id)
|
||||
return request_id
|
||||
|
||||
def cancel(self, request_id: str) -> None:
|
||||
"""Cancel a tracked request now, while the fixtures it depends on are still in place.
|
||||
|
||||
Args:
|
||||
request_id: The request id.
|
||||
"""
|
||||
cancel_request(self.client, self.headers, "chat", request_id)
|
||||
self.open_requests.remove(request_id)
|
||||
|
||||
def popped_job(self, worker: FakeChatWorker) -> str:
|
||||
"""Pop the queued job and return its generation id.
|
||||
|
||||
Args:
|
||||
worker: The worker that pops.
|
||||
|
||||
Returns:
|
||||
The generation id.
|
||||
"""
|
||||
pop_result = worker.pop()
|
||||
generation_id: str | None = pop_result["id"]
|
||||
assert generation_id is not None, pop_result
|
||||
return generation_id
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def wire(app: Flask, client: FlaskClient, request_headers: dict[str, str], request: pytest.FixtureRequest) -> Iterator[WireRun]:
|
||||
"""A scenario context that cancels its requests and takes its workers offline afterwards.
|
||||
|
||||
A worker left online would make later tests' requests possible, which tests of an unserved model rely on not being.
|
||||
"""
|
||||
run = WireRun(app=app, client=client, headers=request_headers, request=request)
|
||||
yield run
|
||||
for request_id in run.open_requests:
|
||||
client.delete(f"/api/v2/generate/chat/status/{request_id}", headers=request_headers)
|
||||
for worker_name in run.worker_names:
|
||||
take_worker_offline(app, worker_name)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RouteSchema:
|
||||
"""Represents what a worker route reads from a request body: its parser's arguments and its body model."""
|
||||
|
||||
parser: RequestParser
|
||||
"""The parser the route parses its arguments with."""
|
||||
|
||||
model: Model
|
||||
"""The model the route validates the body against, whose nested models hold the keys of nested objects."""
|
||||
|
||||
def json_arguments(self) -> set[str]:
|
||||
"""Return the body keys the parser reads; empty for a parser that reads only headers."""
|
||||
return {argument.name for argument in self.parser.args if "json" in argument.location}
|
||||
|
||||
|
||||
def _pop_schema() -> RouteSchema:
|
||||
from horde.apis.v2.kobold import models, parsers
|
||||
|
||||
return RouteSchema(parser=parsers.job_pop_parser, model=models.input_model_job_pop)
|
||||
|
||||
|
||||
def _chat_submit_schema() -> RouteSchema:
|
||||
from horde.apis.v2.kobold import models, parsers
|
||||
|
||||
return RouteSchema(parser=parsers.chat_job_submit_parser, model=models.input_model_job_submit)
|
||||
|
||||
|
||||
def _progress_schema() -> RouteSchema:
|
||||
from horde.apis.v2.kobold import models, parsers
|
||||
|
||||
return RouteSchema(parser=parsers.chat_progress_parser, model=models.input_model_chat_progress)
|
||||
|
||||
|
||||
def _unread_nested_keys(documented: Any, model_field: Any, path: str) -> list[str]:
|
||||
"""Return the keys of a documented nested object that its model field does not declare."""
|
||||
if isinstance(model_field, restx_fields.Nested) and isinstance(documented, dict):
|
||||
nested_model = model_field.nested
|
||||
unread = [f"{path}.{key}" for key in documented if key not in nested_model]
|
||||
for key, value in documented.items():
|
||||
unread += _unread_nested_keys(value, nested_model.get(key), f"{path}.{key}")
|
||||
return unread
|
||||
if isinstance(model_field, restx_fields.List) and isinstance(documented, list):
|
||||
return [
|
||||
unread_key
|
||||
for index, element in enumerate(documented)
|
||||
for unread_key in _unread_nested_keys(element, model_field.container, f"{path}[{index}]")
|
||||
]
|
||||
return []
|
||||
|
||||
|
||||
def _unread_keys(documented: dict[str, Any], schema: RouteSchema) -> list[str]:
|
||||
"""Return the documented keys the route does not read.
|
||||
|
||||
A top-level key has to be an argument of the route's parser, or a field of its model when the parser reads only
|
||||
headers, as the progress route's does. A nested object's keys have to be fields of the nested model the route's
|
||||
model declares for it.
|
||||
"""
|
||||
body_model = schema.model.resolved
|
||||
top_level_keys = schema.json_arguments() or set(body_model)
|
||||
unread = [f"$.{key}" for key in documented if key not in top_level_keys]
|
||||
for key, value in documented.items():
|
||||
unread += _unread_nested_keys(value, body_model.get(key), f"$.{key}")
|
||||
return unread
|
||||
|
||||
|
||||
def _post_pop_request(wire: WireRun, documented: dict[str, Any]) -> TestResponse:
|
||||
# A fresh name keeps the details read after this pop clear of the 30-second worker details cache.
|
||||
placeholders = {
|
||||
WORKER_NAME_PLACEHOLDER: wire.worker_name(fresh_name=True),
|
||||
MODEL_NAME_PLACEHOLDER: CHAT_MODEL,
|
||||
BRIDGE_AGENT_PLACEHOLDER: wire.headers["Client-Agent"],
|
||||
}
|
||||
return wire.client.post("/api/v2/generate/text/pop", json=filled(documented, placeholders), headers=wire.headers)
|
||||
|
||||
|
||||
def _live_pop_chat_job(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker()
|
||||
wire.queue()
|
||||
return worker.pop_response()
|
||||
|
||||
|
||||
def _live_pop_tools_job(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker(chat_features=TOOLS_FEATURES)
|
||||
wire.queue(messages=TOOL_HISTORY, **TOOLS_REQUEST_EXTRAS)
|
||||
return worker.pop_response()
|
||||
|
||||
|
||||
def _live_pop_image_job(wire: WireRun) -> TestResponse:
|
||||
wire.request.getfixturevalue("object_store_ready")
|
||||
worker = wire.worker(chat_features=VISION_FEATURES, image_transports=["base64"])
|
||||
wire.track(_submit(wire.client, wire.headers, _image_body(_png_data_url())))
|
||||
return worker.pop_response()
|
||||
|
||||
|
||||
def _live_pop_messages_url_job(wire: WireRun) -> TestResponse:
|
||||
wire.request.getfixturevalue("object_store_ready")
|
||||
wire.request.getfixturevalue("messages_offload")
|
||||
worker = wire.worker(chat_features=VISION_FEATURES, image_transports=MESSAGES_URL_TRANSPORTS)
|
||||
wire.track(_submit(wire.client, wire.headers, _image_body(_png_data_url())))
|
||||
return worker.pop_response()
|
||||
|
||||
|
||||
def _live_pop_no_job(wire: WireRun) -> TestResponse:
|
||||
return wire.worker().pop_response()
|
||||
|
||||
|
||||
def _post_submit(wire: WireRun, documented: dict[str, Any], *, finished: bool, **request_overrides: Any) -> TestResponse:
|
||||
"""Pop a job for a request with ``request_overrides`` and post the documented submit for it."""
|
||||
worker = wire.worker(chat_features=TOOLS_FEATURES)
|
||||
request_id = wire.queue(**request_overrides)
|
||||
response = worker.submit(filled(documented, {JOB_ID_PLACEHOLDER: wire.popped_job(worker), MODEL_NAME_PLACEHOLDER: CHAT_MODEL}))
|
||||
if finished and response.status_code == 200:
|
||||
wire.open_requests.remove(request_id)
|
||||
return response
|
||||
|
||||
|
||||
def _live_submit_reply_response(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker()
|
||||
wire.queue()
|
||||
return worker.submit(build_chat_submit_body(generation_id=wire.popped_job(worker), content="Blue.", model_id=CHAT_MODEL))
|
||||
|
||||
|
||||
def _live_submit_fault_response(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker()
|
||||
wire.queue()
|
||||
return worker.submit(
|
||||
build_faulted_chat_submit_body(generation_id=wire.popped_job(worker), model_id=CHAT_MODEL, fault_reason="prompt_overflow"),
|
||||
)
|
||||
|
||||
|
||||
def _streamed_job(wire: WireRun) -> tuple[str, str]:
|
||||
"""Pop a streamed chat job with its lease on an in-memory Redis, and return the request and generation ids."""
|
||||
wire.request.getfixturevalue("lua_redis")
|
||||
worker = wire.worker()
|
||||
request_id = wire.queue(stream=True)
|
||||
return request_id, wire.popped_job(worker)
|
||||
|
||||
|
||||
def _post_progress_post(wire: WireRun, documented: dict[str, Any]) -> TestResponse:
|
||||
request_id, generation_id = _streamed_job(wire)
|
||||
response = wire.client.post(
|
||||
"/api/v2/generate/text/progress",
|
||||
json=filled(documented, {JOB_ID_PLACEHOLDER: generation_id}),
|
||||
headers={"apikey": wire.headers["apikey"]},
|
||||
)
|
||||
wire.cancel(request_id)
|
||||
return response
|
||||
|
||||
|
||||
def _live_progress_accepted(wire: WireRun) -> TestResponse:
|
||||
request_id, generation_id = _streamed_job(wire)
|
||||
response = _post_progress(wire.client, wire.headers["apikey"], generation_id, 1, "Blue")
|
||||
wire.cancel(request_id)
|
||||
return response
|
||||
|
||||
|
||||
def _live_progress_cancelled(wire: WireRun) -> TestResponse:
|
||||
request_id, generation_id = _streamed_job(wire)
|
||||
_post_progress(wire.client, wire.headers["apikey"], generation_id, 1, "Bl")
|
||||
wire.cancel(request_id)
|
||||
return _post_progress(wire.client, wire.headers["apikey"], generation_id, 2, "ue")
|
||||
|
||||
|
||||
def _live_progress_out_of_sequence(wire: WireRun) -> TestResponse:
|
||||
request_id, generation_id = _streamed_job(wire)
|
||||
_post_progress(wire.client, wire.headers["apikey"], generation_id, 1, "Bl")
|
||||
response = _post_progress(wire.client, wire.headers["apikey"], generation_id, 3, "ue")
|
||||
wire.cancel(request_id)
|
||||
return response
|
||||
|
||||
|
||||
def _live_chat_declaration_invalid(wire: WireRun) -> TestResponse:
|
||||
return wire.worker().pop_response(model_id="koboldcpp/model_Q4")
|
||||
|
||||
|
||||
def _live_chat_submit_invalid(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker()
|
||||
wire.queue()
|
||||
submit_body = build_chat_submit_body(generation_id=wire.popped_job(worker), content="Blue.", model_id=CHAT_MODEL)
|
||||
del submit_body["finish_reason"]
|
||||
return worker.submit(submit_body)
|
||||
|
||||
|
||||
def _live_chat_submit_nonconformant(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker()
|
||||
request_id, submit_response = _serve_reply(
|
||||
wire.client,
|
||||
wire.headers,
|
||||
worker,
|
||||
{"tools": [LOOKUP_TOOL], "tool_choice": "required"},
|
||||
content="Sunny.",
|
||||
)
|
||||
wire.track(request_id)
|
||||
refusal: TestResponse = submit_response
|
||||
return refusal
|
||||
|
||||
|
||||
def _live_chat_model_mismatch(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker()
|
||||
wire.queue()
|
||||
return worker.submit(build_chat_submit_body(generation_id=wire.popped_job(worker), content="Blue.", model_id="another/model"))
|
||||
|
||||
|
||||
def _live_worker_details(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker(
|
||||
fresh_name=True,
|
||||
chat_features=DECLARED_FEATURES,
|
||||
chat_guarantees=[],
|
||||
tools_max_temperature=0.7,
|
||||
image_transports=IMAGE_TRANSPORTS,
|
||||
)
|
||||
return wire.client.get(f"/api/v2/workers/name/{worker.name}", headers={"Client-Agent": wire.headers["Client-Agent"]})
|
||||
|
||||
|
||||
def _post_chat_request(wire: WireRun, documented: dict[str, Any]) -> TestResponse:
|
||||
wire.worker()
|
||||
response = wire.client.post(CHAT_ASYNC_ROUTE, json=filled(documented, {MODEL_NAME_PLACEHOLDER: CHAT_MODEL}), headers=wire.headers)
|
||||
if response.status_code == 202:
|
||||
wire.track(response.get_json()["id"])
|
||||
return response
|
||||
|
||||
|
||||
def _live_chat_request_accepted(wire: WireRun) -> TestResponse:
|
||||
wire.worker()
|
||||
response = wire.client.post(CHAT_ASYNC_ROUTE, json={"model": CHAT_MODEL, "messages": CHAT_MESSAGES}, headers=wire.headers)
|
||||
if response.status_code == 202:
|
||||
wire.track(response.get_json()["id"])
|
||||
return response
|
||||
|
||||
|
||||
def _live_chat_dry_run(wire: WireRun) -> TestResponse:
|
||||
wire.worker()
|
||||
return wire.client.post(
|
||||
CHAT_ASYNC_ROUTE,
|
||||
json={"model": CHAT_MODEL, "messages": CHAT_MESSAGES, "max_tokens": 64, "horde": {"dry_run": True}},
|
||||
headers=wire.headers,
|
||||
)
|
||||
|
||||
|
||||
def _status_response(wire: WireRun, request_id: str) -> TestResponse:
|
||||
return wire.client.get(f"/api/v2/generate/chat/status/{request_id}", headers=wire.headers)
|
||||
|
||||
|
||||
def _live_chat_status_waiting(wire: WireRun) -> TestResponse:
|
||||
wire.worker()
|
||||
return _status_response(wire, wire.queue(max_tokens=64))
|
||||
|
||||
|
||||
def _live_chat_status_done(wire: WireRun) -> TestResponse:
|
||||
worker = wire.worker()
|
||||
request_id = wire.queue(max_tokens=64)
|
||||
submit_response = worker.submit(
|
||||
build_chat_submit_body(
|
||||
generation_id=wire.popped_job(worker),
|
||||
content="Blue.",
|
||||
model_id=CHAT_MODEL,
|
||||
prompt_tokens=21,
|
||||
completion_tokens=3,
|
||||
native_finish_reason="eos_token",
|
||||
),
|
||||
)
|
||||
assert submit_response.status_code == 200, submit_response.get_data(as_text=True)
|
||||
wire.open_requests.remove(request_id)
|
||||
assert chat_status(wire.client, wire.headers, request_id)["done"] is True
|
||||
return _status_response(wire, request_id)
|
||||
|
||||
|
||||
def _live_chat_status_context_adjustments(wire: WireRun) -> TestResponse:
|
||||
wire.worker()
|
||||
return _status_response(wire, wire.queue(messages=LONG_HISTORY, max_tokens=64, horde={"max_context_length": 2048}))
|
||||
|
||||
|
||||
def _live_no_eligible_worker(wire: WireRun) -> TestResponse:
|
||||
wire.request.getfixturevalue("object_store_ready")
|
||||
wire.worker()
|
||||
return wire.client.post(
|
||||
CHAT_ASYNC_ROUTE,
|
||||
json=_image_body(_png_data_url(), max_tokens=NO_ELIGIBLE_WORKER_MAX_TOKENS),
|
||||
headers=ANONYMOUS_HEADERS,
|
||||
)
|
||||
|
||||
|
||||
def _live_unsupported_chat_parameter(wire: WireRun) -> TestResponse:
|
||||
return wire.client.post(CHAT_ASYNC_ROUTE, json={"model": CHAT_MODEL, "messages": CHAT_MESSAGES, "logprobs": True}, headers=wire.headers)
|
||||
|
||||
|
||||
def _live_chat_model_list(wire: WireRun) -> TestResponse:
|
||||
from horde.database.threads import store_available_models
|
||||
|
||||
wire.worker()
|
||||
# The test app runs no maintenance threads, so the model list cache is refreshed here, as the production thread would.
|
||||
store_available_models() # type: ignore[no-untyped-call]
|
||||
return wire.client.get("/api/v2/status/models?type=text&chat=true", headers={"Client-Agent": wire.headers["Client-Agent"]})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Scenario:
|
||||
"""Produces one response example's live body, or posts one request example."""
|
||||
|
||||
live: Callable[[WireRun], TestResponse] | None = None
|
||||
"""Returns the live response; None for a request example."""
|
||||
|
||||
status: int | None = None
|
||||
"""The HTTP status of a response example, which its prose states; None for a request example."""
|
||||
|
||||
post: Callable[[WireRun, dict[str, Any]], TestResponse] | None = None
|
||||
"""Posts the documented request body with its placeholders filled in; None for a response example."""
|
||||
|
||||
answered_by: str | None = None
|
||||
"""The response example whose status the posted documented body gets."""
|
||||
|
||||
schema: Callable[[], RouteSchema] | None = None
|
||||
"""Returns what the route reads from the body, for a worker request example; the route ignores other keys."""
|
||||
|
||||
object_storage: bool = False
|
||||
"""Whether the scenario stores images, which needs the object store."""
|
||||
|
||||
|
||||
SCENARIOS: dict[str, Scenario] = {
|
||||
"pop-request": Scenario(
|
||||
post=_post_pop_request,
|
||||
answered_by="pop-no-job",
|
||||
schema=_pop_schema,
|
||||
),
|
||||
"pop-chat-job": Scenario(
|
||||
live=_live_pop_chat_job,
|
||||
status=200,
|
||||
),
|
||||
"pop-chat-job-tools": Scenario(
|
||||
live=_live_pop_tools_job,
|
||||
status=200,
|
||||
),
|
||||
"pop-chat-job-image": Scenario(
|
||||
live=_live_pop_image_job,
|
||||
status=200,
|
||||
object_storage=True,
|
||||
),
|
||||
"pop-chat-job-messages-url": Scenario(
|
||||
live=_live_pop_messages_url_job,
|
||||
status=200,
|
||||
object_storage=True,
|
||||
),
|
||||
"pop-no-job": Scenario(
|
||||
live=_live_pop_no_job,
|
||||
status=200,
|
||||
),
|
||||
"submit-reply": Scenario(
|
||||
post=lambda wire, documented: _post_submit(wire, documented, finished=True),
|
||||
answered_by="submit-reply-response",
|
||||
schema=_chat_submit_schema,
|
||||
),
|
||||
"submit-tool-calls": Scenario(
|
||||
post=lambda wire, documented: _post_submit(
|
||||
wire,
|
||||
documented,
|
||||
finished=True,
|
||||
messages=TOOL_HISTORY,
|
||||
**TOOLS_REQUEST_EXTRAS,
|
||||
),
|
||||
answered_by="submit-reply-response",
|
||||
schema=_chat_submit_schema,
|
||||
),
|
||||
"submit-reasoning": Scenario(
|
||||
post=lambda wire, documented: _post_submit(wire, documented, finished=True),
|
||||
answered_by="submit-reply-response",
|
||||
schema=_chat_submit_schema,
|
||||
),
|
||||
"submit-fault": Scenario(
|
||||
post=lambda wire, documented: _post_submit(wire, documented, finished=False),
|
||||
answered_by="submit-fault-response",
|
||||
schema=_chat_submit_schema,
|
||||
),
|
||||
"submit-reply-response": Scenario(
|
||||
live=_live_submit_reply_response,
|
||||
status=200,
|
||||
),
|
||||
"submit-fault-response": Scenario(
|
||||
live=_live_submit_fault_response,
|
||||
status=200,
|
||||
),
|
||||
"progress-post": Scenario(
|
||||
post=_post_progress_post,
|
||||
answered_by="progress-accepted",
|
||||
schema=_progress_schema,
|
||||
),
|
||||
"progress-accepted": Scenario(
|
||||
live=_live_progress_accepted,
|
||||
status=200,
|
||||
),
|
||||
"progress-cancelled": Scenario(
|
||||
live=_live_progress_cancelled,
|
||||
status=200,
|
||||
),
|
||||
"progress-out-of-sequence": Scenario(
|
||||
live=_live_progress_out_of_sequence,
|
||||
status=409,
|
||||
),
|
||||
"refusal-chat-declaration-invalid": Scenario(
|
||||
live=_live_chat_declaration_invalid,
|
||||
status=400,
|
||||
),
|
||||
"refusal-chat-submit-invalid": Scenario(
|
||||
live=_live_chat_submit_invalid,
|
||||
status=400,
|
||||
),
|
||||
"refusal-chat-submit-nonconformant": Scenario(
|
||||
live=_live_chat_submit_nonconformant,
|
||||
status=400,
|
||||
),
|
||||
"refusal-chat-model-mismatch": Scenario(
|
||||
live=_live_chat_model_mismatch,
|
||||
status=400,
|
||||
),
|
||||
"worker-details": Scenario(
|
||||
live=_live_worker_details,
|
||||
status=200,
|
||||
),
|
||||
"chat-request": Scenario(
|
||||
post=_post_chat_request,
|
||||
answered_by="chat-request-accepted",
|
||||
),
|
||||
"chat-request-accepted": Scenario(
|
||||
live=_live_chat_request_accepted,
|
||||
status=202,
|
||||
),
|
||||
"chat-dry-run": Scenario(
|
||||
live=_live_chat_dry_run,
|
||||
status=200,
|
||||
),
|
||||
"chat-status-waiting": Scenario(
|
||||
live=_live_chat_status_waiting,
|
||||
status=200,
|
||||
),
|
||||
"chat-status-done": Scenario(
|
||||
live=_live_chat_status_done,
|
||||
status=200,
|
||||
),
|
||||
"chat-status-context-adjustments": Scenario(
|
||||
live=_live_chat_status_context_adjustments,
|
||||
status=200,
|
||||
),
|
||||
"refusal-no-eligible-worker": Scenario(
|
||||
live=_live_no_eligible_worker,
|
||||
status=403,
|
||||
object_storage=True,
|
||||
),
|
||||
"refusal-unsupported-chat-parameter": Scenario(
|
||||
live=_live_unsupported_chat_parameter,
|
||||
status=400,
|
||||
),
|
||||
"chat-model-list": Scenario(
|
||||
live=_live_chat_model_list,
|
||||
status=200,
|
||||
),
|
||||
}
|
||||
"""Every worker and native requester example on the page by its marker id."""
|
||||
|
||||
|
||||
def _scenario_params(example_ids: list[str]) -> list[Any]:
|
||||
return [
|
||||
pytest.param(example_id, id=example_id, marks=[pytest.mark.object_storage] if SCENARIOS[example_id].object_storage else [])
|
||||
for example_id in example_ids
|
||||
]
|
||||
|
||||
|
||||
def test_every_marker_has_a_scenario_and_every_scenario_a_marker() -> None:
|
||||
"""The front's examples are matched to their scenarios in ``test_chat_front_wire_examples.py``."""
|
||||
assert sorted(example_id for example_id in DOCUMENTED if not example_id.startswith(FRONT_EXAMPLE_PREFIX)) == sorted(SCENARIOS)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"example_id",
|
||||
_scenario_params(sorted(example_id for example_id, scenario in SCENARIOS.items() if scenario.live is not None)),
|
||||
)
|
||||
def test_each_response_example_matches_the_live_response(wire: WireRun, example_id: str) -> None:
|
||||
"""A response example has the live response's keys, contract values and value types."""
|
||||
scenario = SCENARIOS[example_id]
|
||||
assert scenario.live is not None
|
||||
|
||||
live = scenario.live(wire)
|
||||
|
||||
assert live.status_code == scenario.status, live.get_data(as_text=True)
|
||||
assert structure_differences(DOCUMENTED[example_id].body, live.get_json()) == [], json.dumps(live.get_json(), indent=2)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("example_id", sorted(example_id for example_id, scenario in SCENARIOS.items() if scenario.schema is not None))
|
||||
def test_each_worker_request_example_holds_only_keys_the_route_reads(example_id: str) -> None:
|
||||
"""A misspelt key in a worker request example would be ignored by the route, so the post alone does not catch it."""
|
||||
scenario = SCENARIOS[example_id]
|
||||
assert scenario.schema is not None
|
||||
|
||||
assert _unread_keys(DOCUMENTED[example_id].body, scenario.schema()) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"example_id",
|
||||
_scenario_params(sorted(example_id for example_id, scenario in SCENARIOS.items() if scenario.post is not None)),
|
||||
)
|
||||
def test_each_documented_request_body_gets_the_documented_status(wire: WireRun, example_id: str) -> None:
|
||||
"""The documented request, posted with live ids, gets the status of the response example that answers it."""
|
||||
scenario = SCENARIOS[example_id]
|
||||
assert scenario.post is not None
|
||||
assert scenario.answered_by is not None
|
||||
|
||||
response = scenario.post(wire, DOCUMENTED[example_id].body)
|
||||
|
||||
assert response.status_code == SCENARIOS[scenario.answered_by].status, response.get_data(as_text=True)
|
||||
|
||||
|
||||
def test_the_documented_pop_request_reaches_the_worker_details(wire: WireRun) -> None:
|
||||
"""The worker details read back the chat declaration the documented pop request sends."""
|
||||
documented = DOCUMENTED["pop-request"].body
|
||||
response = _post_pop_request(wire, documented)
|
||||
assert response.status_code == 200, response.get_data(as_text=True)
|
||||
|
||||
[worker_name] = wire.worker_names
|
||||
details = wire.client.get(f"/api/v2/workers/name/{worker_name}", headers=wire.headers).get_json()
|
||||
|
||||
assert details["accepts_chat_jobs"] is documented["chat_jobs"]
|
||||
assert details["chat"] == {
|
||||
"model_id": CHAT_MODEL,
|
||||
"features": sorted(documented["chat_features"]),
|
||||
"limits": {**documented["chat_limits"], "tools_max_temperature": None},
|
||||
"image_transports": sorted(documented["image_transports"]),
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("example_id", sorted(example_id for example_id, scenario in SCENARIOS.items() if scenario.status is not None))
|
||||
def test_each_response_example_states_its_status(example_id: str) -> None:
|
||||
"""The prose before a response example states the status the live response has."""
|
||||
assert f"`{SCENARIOS[example_id].status}`" in DOCUMENTED[example_id].prose
|
||||
+4
-2
@@ -44,6 +44,7 @@ nav = [
|
||||
{ "Outbound requests" = "reference/outbound_requests.md" },
|
||||
{ "Chat requests" = "reference/chat_requests.md" },
|
||||
{ "Chat front" = "reference/chat_front.md" },
|
||||
{ "Chat wire examples" = "reference/chat_wire_examples.md" },
|
||||
] },
|
||||
{ "How-to" = [
|
||||
{ "Overview" = "how-to/README.md" },
|
||||
@@ -127,8 +128,9 @@ pygments_lang_class = true
|
||||
custom_fences = [
|
||||
{ name = "mermaid", class = "mermaid", format = "pymdownx.superfences.fence_code_format" },
|
||||
]
|
||||
# Example programs live as tested files under tools/ and are included into their pages, so a page shows the code the
|
||||
# tests run. check_paths fails the build when an included file is missing.
|
||||
# Example programs under tools/, and sections of docs/reference/chat_wire_examples.md, are included into the pages
|
||||
# that show them, so a page shows the code and bodies the tests check. check_paths fails the build when an included
|
||||
# file is missing.
|
||||
[project.markdown_extensions.pymdownx.snippets]
|
||||
base_path = ["."]
|
||||
check_paths = true
|
||||
Reference in new issue
Block a user