fix(chat): require an image transport of a capable vision worker

- `capable_chat_worker_clauses` in `horde/classes/kobold/worker.py` gains the SQL form of `serves_chat_images`: a request that requires `vision` has a capable worker only when that worker declares the `url` or `base64` image transport. A worker that declares `vision` with no way to receive images, or with `messages_url` alone, no longer keeps an image request from a slower worker that can take it. The clause is built from `TextArrayContainsAll`, so it compiles on both dialects
- the pop clause comment in `horde/database/text_functions.py`, ADR 0041 and the chat request reference state the transport condition
- a parametrized unit test in `tests/unit/test_text_job_ttl.py` covers a faster vision worker with no transport, an empty list, `messages_url` only, `url` and `base64`

docs/decisions/0041-enforce-the-speed-gate-for-chat-requests-and-shadow-it-for-legacy-text.md

@@ -72,8 +72,9 @@ measured?
 Chosen option: "Prefer capable workers for chat requests and record every over-capacity assignment".
 
 A capable worker for a chat request is an online chat worker for one of the request's models, not in maintenance or
-paused, whose declared chat limits admit the request, which declares every chat feature the request requires, and
-whose capacity is at least its `max_length` (`capable_chat_worker_clauses`). `TextWorker.can_generate_with_eligibility_inputs` refuses a chat request over the
+paused, whose declared chat limits admit the request, which declares every chat feature the request requires and the
+`url` or `base64` image transport when the request requires `vision`, and whose capacity is at least its `max_length`
+(`capable_chat_worker_clauses`). `TextWorker.can_generate_with_eligibility_inputs` refuses a chat request over the
 worker's capacity with `lease_speed` while a capable worker is online, and admits it when none is. In the pop query,
 `text_wp_speed_gate_clause` applies the same rule through a correlated `NOT EXISTS` over the capable workers, and
 passes every legacy text request. The feature test is `TextArrayContainsAll`, which compiles to `@>` on PostgreSQL
@@ -141,8 +142,8 @@ record at assignment, since the worker's measured speed moves between pop and su
 
 `tests/unit/test_text_job_ttl.py` pins the pop clause and the eligibility check agreeing for chat and legacy
 requests with and without a capable worker online, an offline, unavailable or under-declared faster worker not
-holding a chat request back, a faster worker counting as capable only when it declares every required feature, an
-unmeasured worker counting as capable at the assumed speed, a slow worker refused a
+holding a chat request back, a faster worker counting as capable only when it declares every required feature, a
+faster `vision` worker counting as capable only with the `url` or `base64` image transport, an unmeasured worker counting as capable at the assumed speed, a slow worker refused a
 chat request a capable worker can take and assigned a legacy one, the capacity-limited count reading only the chat
 declaration, and the shadow records: the marker and assignment count for legacy and chat jobs, the completed outcome
 with its service time, the faulted outcome counted once, the aborted outcome, and no record for an assignment within

docs/reference/chat_requests.md

@@ -714,6 +714,8 @@ the job's `max_length` is offered it only when no capable chat worker is online.
 - is not in maintenance and not paused,
 - declares a chat `max_length` and `max_context_length` at or above the request's,
 - declares every feature in the request's `chat_required_features`,
+- declares the `url` or `base64` image transport when `chat_required_features` holds `vision` (`messages_url` alone
+  delivers no image),
 - has a capacity of at least the request's `max_length`.
 
 The pop query applies this rule as a correlated `NOT EXISTS` over the workers table (`text_wp_speed_gate_clause`,

horde/classes/kobold/worker.py

@@ -13,7 +13,7 @@ from typing import TYPE_CHECKING, Any
 
 from horde_sdk.generation_parameters.text.chat_policy import KNOWN_CHAT_DECLARATIONS, ChatFeature, ChatLimitField
 from horde_sdk.worker.chat_declaration import IMAGE_PART_TRANSPORTS, ChatImageTransport
-from sqlalchemy import JSON, ScalarSelect, case
+from sqlalchemy import JSON, ScalarSelect, case, literal, or_
 from sqlalchemy.dialects.postgresql import ARRAY, UUID
 from sqlalchemy.orm import Mapped, relationship
 from sqlalchemy.sql import ColumnElement, expression
@@ -167,8 +167,9 @@ def capable_chat_worker_clauses(
     """Return the clauses met by a chat worker row that can take a chat request of these lengths inside the base lease.
 
     The worker is online, accepts chat jobs, is not in maintenance or paused, declares chat limits at or above the
-    request's lengths, declares every chat feature the request requires, and has a lease capacity of at least
-    ``max_length``. The caller adds the model condition. A chat request over a worker's capacity is offered to that
+    request's lengths, declares every chat feature the request requires, declares the ``url`` or ``base64`` image
+    transport when the request requires ``vision``, and has a lease capacity of at least ``max_length``. The caller
+    adds the model condition. A chat request over a worker's capacity is offered to that
     worker only when no worker meets these clauses for one of the request's models. A faster worker that lacks a
     required feature cannot serve the request, so it does not keep the request from a slower worker that has it.
 
@@ -190,10 +191,24 @@ def capable_chat_worker_clauses(
         worker_entity.chat_max_length >= max_length,
         worker_entity.chat_max_context_length >= max_context_length,
         TextArrayContainsAll(worker_entity.chat_features, required_features),
+        # The SQL form of serves_chat_images: a worker receives a request's images only through url or base64, so a
+        # vision declaration without one of them does not make the worker capable of a request with images.
+        or_(
+            ~TextArrayContainsAll(required_features, _text_array_literal([ChatFeature.VISION.value])),
+            *(
+                TextArrayContainsAll(worker_entity.image_transports, _text_array_literal([transport.value]))
+                for transport in sorted(IMAGE_PART_TRANSPORTS)
+            ),
+        ),
         text_worker_lease_capacity_covers_clause(worker_entity, max_length),
     ]
 
 
+def _text_array_literal(values: list[str]) -> ColumnElement[Any]:
+    """Return ``values`` bound as a ``text_array_column_type`` value, a ``text[]`` on PostgreSQL and JSON on SQLite."""
+    return literal(values, type_=text_array_column_type)
+
+
 class TextWorkerSoftprompts(db.Model):
     __tablename__ = "text_worker_softprompts"
     id = db.Column(db.Integer, primary_key=True)

horde/database/text_functions.py

@@ -230,7 +230,7 @@ def text_wp_speed_gate_clause(worker, models_list):
     # - is_chat IS false: a legacy text request is offered at any max_length.
     # - max_length <= capacity: a chat request the worker finishes inside the base lease is offered.
     # - NOT EXISTS capable worker: a chat request over the capacity is offered when no online chat worker for its
-    #   models finishes it in time and declares the chat limits and features that admit it.
+    #   models finishes it in time and declares the chat limits, features and image transport that admit it.
     # For a popping worker measured at 2 tokens/s (capacity 2 * 120 = 240), with one other online chat worker for
     # model m at 10 tokens/s (capacity 1200) and none for model n:
     #

tests/unit/test_text_job_ttl.py

@@ -23,6 +23,7 @@ import fakeredis
 import pytest
 from flask.testing import FlaskClient
 from horde_sdk.generation_parameters.text.chat_policy import ChatFeature
+from horde_sdk.worker.chat_declaration import ChatImageTransport
 from sqlalchemy import JSON, Text, create_engine, literal, select
 from sqlalchemy.dialects.postgresql import ARRAY
 from sqlalchemy.orm import Session
@@ -609,6 +610,49 @@ class TestSpeedGate:
         assert [wp.id for wp in popped] == ([waiting_prompt.id] if slow_worker_offered else [])
         assert worker.can_generate(waiting_prompt) == ([True, None] if slow_worker_offered else [False, "lease_speed"])
 
+    @pytest.mark.parametrize(
+        ("faster_worker_transports", "slow_worker_offered"),
+        [
+            (None, True),
+            ([], True),
+            ([ChatImageTransport.MESSAGES_URL.value], True),
+            ([ChatImageTransport.URL.value], False),
+            ([ChatImageTransport.BASE64.value], False),
+        ],
+        ids=[
+            "faster_worker_declares_no_transport",
+            "faster_worker_declares_an_empty_transport_list",
+            "faster_worker_declares_only_messages_url",
+            "faster_worker_declares_url",
+            "faster_worker_declares_base64",
+        ],
+    )
+    def test_a_faster_vision_worker_counts_as_capable_only_with_an_image_transport(
+        self,
+        db_session: Session,
+        fake_redis: object,
+        make_user: MakeUser,
+        faster_worker_transports: list[str] | None,
+        slow_worker_offered: bool,
+    ) -> None:
+        """A 512-token chat request that requires ``vision`` reaches a worker at 2 tokens/s (capacity 240) that
+        receives images when the only faster worker declares ``vision`` without the ``url`` or ``base64`` transport,
+        and is left to the faster worker when it declares one of them. ``messages_url`` alone delivers no image."""
+        user = make_user()
+        worker = _make_worker(user, speed=2)
+        waiting_prompt = _make_request(user, 512, is_chat=True)
+        waiting_prompt.chat_required_features = [ChatFeature.VISION.value]
+        db.session.commit()
+        _declare_chat(worker, waiting_prompt)
+        worker.image_transports = [ChatImageTransport.URL.value]
+        db.session.commit()
+        _make_capable_chat_worker(user, waiting_prompt, image_transports=faster_worker_transports)
+
+        popped = text_functions.get_sorted_text_wp_filtered_to_worker(worker, [MODEL_NAME])
+
+        assert [wp.id for wp in popped] == ([waiting_prompt.id] if slow_worker_offered else [])
+        assert worker.can_generate(waiting_prompt) == ([True, None] if slow_worker_offered else [False, "lease_speed"])
+
     def test_an_unmeasured_worker_counts_as_capable_at_the_assumed_speed(
         self,
         db_session: Session,