mirror of
https://github.com/Haidra-Org/AI-Horde.git
synced 2026-11-06 05:17:31 +00:00
tazlin
7d13ce4508
OpenAI's default `reasoning_effort` is `medium`. koboldcpp maps the value to a thinking budget as a fraction of `max_length` and treats an absent value as unlimited, so a request with a small `max_tokens` spent the whole budget on reasoning and returned an empty reply. Reasoning cannot be resumed through the OpenAI API, so a reply cut off while thinking is lost, and `max_tokens` has to cover the reasoning and the answer in one request. - `DEFAULT_CHAT_REASONING_EFFORT` in `horde/chat_contract.py` is `medium` - `apply_worker_chat_defaults` adds it to one job's payload copy in `TextWaitingPrompt.get_pop_payload`, only for a worker that declares the `reasoning_effort` feature and only when the client sent no value. An omitted value does not require the feature, so routing, the stored request, the operation fingerprint and the kudos estimate do not change. An explicit value, `none` included, is passed unchanged - Unit tests cover the default, the untouched explicit values and non-declaring workers. Integration tests pop a declaring and a non-declaring worker - ADR 0033 and 0036, the pop payload and request field descriptions, and the chat request, serve-chat-jobs and send-chat-completions docs state the behaviour - The send-chat-completions how-to, the chat request reference and explanation, ADR 0033 and the native and front request field descriptions publish the budget policy: reasoning is generated within `max_tokens`, an omitted effort gives a declaring worker a budget of about half on koboldcpp, the budget applies per thinking block and does not guarantee an answer, and a reply cut off while thinking cannot be continued. The `include_reasoning` swagger field states its true default