Commit Graph
5 Commits
Author SHA1 Message Date
tazlin f374fed8bd fix: refresh the fleet reference when due and tolerate bad text entries
- The quorum publisher job now polls every 60 s and publishes only when the served snapshot is degraded or older than FLEET_PUBLISH_INTERVAL_SECONDS (publish_fleet_snapshot_if_due), so a newly elected owner or a fleet on a degraded document refreshes within a minute instead of after a full hourly interval
- refresh_text_reference drops individual entries without a numeric parameter count with a warning instead of rejecting the whole payload; only an unusable or empty payload retains the previous text reference
- Document both behaviors in docs/reference/model_reference.md
2026-08-30 10:57:24 -04:00
tazlin 279545de6a feat: distribute the image model reference as one fleet-wide Redis snapshot
- Add horde/model_reference_snapshot.py: one complete, hash-verified ImageReferenceSnapshot per publication, replaced atomically under a fenced lease so a slow publisher cannot overwrite a newer revision, with sequence-guarded fan-out to every REDIS_SERVERS master
- ModelReference no longer fetches per process: the quorum owner publishes hourly via publish_model_reference_snapshot, every process polls Redis every 5 s, and cold start contends for the bootstrap lease instead of fetching independently
- Publication refuses an empty reference, an empty catalog, or HMR's GitHub fallback so the last-known-good document is never replaced by a degraded read; a fleet with no snapshot at all may bootstrap one degraded document from fallback sources after the strict window, logged as errors and flagged in the document
- Unknown top-level snapshot fields are ignored with a warning so older consumers keep loading during a rolling deploy
- Report snapshot age and degraded state as gauges on every poll and warn once the served snapshot is three hours old
- Require PrefetchStrategy.NONE on the HMR manager; reject a pre-existing manager built with another strategy
- Validate the text reference before replacing it and expose loader state through immutable per-type state objects
- store_known_image_models guards on a non-empty reference now that reference is never None
- Document the fleet distribution in docs/reference/model_reference.md and add the bootstrap and lease knobs to .env_template
2026-08-30 10:48:41 -04:00
tazlin 52613a2d75 fix: publish image reference snapshots atomically
- Refresh and stage the served baseline catalog before loading models, then expose both through one copy-on-write snapshot.
- Keep the previous complete snapshot serving when either fetch or validation fails, rather than leaking a new model against stale baseline policy.
- Add loader coverage for coherent publication and failed-refresh rollback; document the snapshot contract.
2026-08-30 10:48:40 -04:00
tazlin b4cf664882 fix: harden uncatalogued baseline feature gating
- Fixes baseline policy to treat uncatalogued baselines conservatively: plain generation is still allowed, but baseline-dependent workflows (such as QR code/remix and other gated features) are rejected until explicit catalog/bridge support exists.
- Renames the default capability constant to `UNCATALOGUED_CAPABILITIES`, updates parity and policy tests to reflect the new behavior.
- Adds Anima baseline coverage, added by bumping `horde_model_reference` to 8.5.0, which also supports the new "baseline" record semantics on models.aihorde.net.
2026-08-30 10:48:40 -04:00
tazlin 20c4f08065 feat: image reference from models.aihorde.net with baseline-driven policy
- model_reference.py loads image_generation through horde_model_reference's ModelReferenceManager (REPLICA, GitHub fallback) with the PRIMARY's pending queue overlaid via PendingModelProvider, so beta models are known models; HORDE_BETA_MODEL_CATEGORIES and HORDE_BETA_MODELS_API_KEY control the overlay, the raw GitHub URL overrides and the expired zmodel branch gate are gone, and stable_diffusion_names is every record rather than a hard-coded baseline allowlist.
- Baselines are the horde_model_reference enum spelling end to end; get_model_baseline returns a record's baseline verbatim (unknown strings pass through) with the customizer name-suffix fallbacks, get_all_model_baselines shares them.
- baseline_policy.py: baseline_violation composes the request rules from two authorities, the served baseline record (weights and mechanisms that exist: remix, transparent, qr_code, unavailable control types, flow matching) and bridge_supports (which bridge releases render hires_fix, control_type, flow_shift, transparent); a feature no known worker kind renders is refused with the existing return codes. policy() reads kudos, batching, TTL and resolution floor from the record with par defaults, replacing the duplicated kudos ladders, BASELINE_BATCHING_MULTIPLIERS, FLOW_SHIFT_BASELINES and the per-baseline branches in validation.py, stable.py, worker.py, waiting_prompt.py and processing_generation.py.
- ImageAsyncGenerate.validate validates against the effective model list, which a style may have substituted; get_highest_model_batching_multiplier resolves baselines instead of looking model names up in a baseline-keyed dict; the dry-run quote applies the same qr_code and hires factors the charge does; resolution floors cover every large architecture.
- The KnownImageModel mirror takes ImageGenerationModelRecord objects; the manager's baseline store is refreshed on the hourly reference refresh.
- Tests seed the reference with real records (model_reference_seed.py); docs for the loader, the policy table and adding a baseline.
2026-08-30 10:48:39 -04:00