mirror of
https://github.com/Haidra-Org/AI-Horde.git
synced 2026-11-06 05:17:31 +00:00
tazlin
8928d4d49f
- Rewrite README.md to reflect the current service: quick-start links, OpenAI-compatible chat endpoint, updated tech stack, funding, docs, contributing and license sections. - Replace the long FAQ.md with a pointer to aihorde.net/faq. Shrink README_StableHorde.md and README_KoboldAIHorde.md to short pointers to the website and main readme. Add a private-horde intro to README_docker.md. - Remove the now-unused FAQ and GUI images (and their .license files), and update the join-the-horde and FAQ links in index_stable.md.
105 lines
3.4 KiB
Markdown
105 lines
3.4 KiB
Markdown
<!--
|
|
SPDX-FileCopyrightText: 2023 Tazlin
|
|
|
|
SPDX-License-Identifier: AGPL-3.0-or-later
|
|
-->
|
|
|
|
# Running ai-horde with Docker
|
|
|
|
You can run your own private horde with this software, for example to pool the GPUs of a team. It is free software
|
|
under the AGPL-3.0-or-later license, so you can use, modify and share it under that license's terms. Hordes do not
|
|
share workers with each other, so a private horde needs its own workers.
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
- [Install Docker](https://docs.docker.com/get-docker/)
|
|
|
|
## Build
|
|
|
|
Run the following command in your project root folder (the folder where your Dockerfile is located):
|
|
|
|
(The image name can be changed to any name you want.)
|
|
|
|
```bash
|
|
docker build -t aihorde:latest .
|
|
```
|
|
|
|
The default image intentionally omits Pyroscope's native profiling wheels. To
|
|
build an image that can run with `PYROSCOPE_ENABLED=true`, include the
|
|
`telemetry-profiling` dependency group:
|
|
|
|
```bash
|
|
docker build \
|
|
--build-arg AI_HORDE_DEPENDENCY_GROUPS=telemetry-profiling \
|
|
-t aihorde:telemetry .
|
|
```
|
|
|
|
The published GHCR images follow the same split: `ghcr.io/haidra-org/ai-horde:main`
|
|
is the default runtime image, while `ghcr.io/haidra-org/ai-horde:main-telemetry`
|
|
includes the profiling group.
|
|
|
|
## with Docker Compose
|
|
|
|
[docker-compose.yaml](docker-compose.yaml) is provided to run the AI-Horde with Redis and Postgres.
|
|
|
|
Copy the `.env_template` file in the root folder to create the `.env_docker` file.
|
|
|
|
```bash
|
|
cp .env_template .env_docker
|
|
```
|
|
|
|
To use the supplied `.env_template` with the supplied `docker-compose.yaml`, you will need to set:
|
|
|
|
```bash
|
|
# .env_docker
|
|
REDIS_IP="redis"
|
|
REDIS_SERVERS='["redis"]'
|
|
USE_SQLITE=0
|
|
POSTGRES_URL="postgres"
|
|
```
|
|
|
|
Then run the following command in your project root folder:
|
|
|
|
```bash
|
|
# run in background
|
|
docker compose up --build -d
|
|
```
|
|
|
|
The Compose volume is mounted at PostgreSQL's version-aware parent directory.
|
|
This is required by PostgreSQL 18 and later, and keeps data for different major
|
|
versions in separate subdirectories when the development image is updated.
|
|
The API waits for the database health check before it starts.
|
|
|
|
The project name gives the test stack its own named volume. Stop any stack that
|
|
already owns ports 7001, 5432, or 6379 before starting the isolated one.
|
|
|
|
## Optional tuning environment variables
|
|
|
|
These are all optional - the image boots with safe defaults. See
|
|
[`tests/DEPLOYMENT_CONTRACT.md`](tests/DEPLOYMENT_CONTRACT.md) for the full,
|
|
authoritative list (required vs optional) that downstream deployers rely on.
|
|
|
|
### SQLAlchemy connection pool
|
|
|
|
Each replica opens up to `SQLALCHEMY_POOL_SIZE + SQLALCHEMY_MAX_OVERFLOW`
|
|
Postgres connections. When running multiple replicas, keep the aggregate under
|
|
the database's `max_connections`.
|
|
|
|
```bash
|
|
SQLALCHEMY_POOL_SIZE=15 # base pooled connections per replica
|
|
SQLALCHEMY_MAX_OVERFLOW=5 # extra burst connections per replica
|
|
SQLALCHEMY_POOL_TIMEOUT=30 # seconds to wait for a connection before erroring
|
|
SQLALCHEMY_POOL_PRE_PING=0 # 1 to validate connections before use
|
|
```
|
|
|
|
### Telemetry (opt-in)
|
|
|
|
Telemetry activates when `AI_HORDE_TELEMETRY_ENABLED=1` or any
|
|
`OTEL_EXPORTER_OTLP_*` endpoint is set (and `OTEL_SDK_DISABLED` is not `true`).
|
|
Common knobs: `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_SERVICE_NAME`,
|
|
`DEPLOYMENT_ENVIRONMENT`, `OTEL_INSTRUMENT_SQLALCHEMY`, `OTEL_INSTRUMENT_REDIS`,
|
|
`OTEL_TRACES_SAMPLER_ARG`, and
|
|
`PYROSCOPE_ENABLED` (requires the `-telemetry` image variant).
|