dilim/AI-Horde
1
0 Fork 0
mirror of https://github.com/Haidra-Org/AI-Horde.git synced 2026-11-06 05:17:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: rewrite readmes and point FAQ to website

Browse Source 
- Rewrite README.md to reflect the current service: quick-start links, OpenAI-compatible chat endpoint, updated tech stack, funding, docs, contributing and license sections.
- Replace the long FAQ.md with a pointer to aihorde.net/faq. Shrink README_StableHorde.md and README_KoboldAIHorde.md to short pointers to the website and main readme. Add a private-horde intro to README_docker.md.
- Remove the now-unused FAQ and GUI images (and their .license files), and update the join-the-horde and FAQ links in index_stable.md.
This commit is contained in:
tazlin committed 2026-11-04 12:57:31 -05:00
1 parent 99f09720ad
commit 8928d4d49f
15 files changed
+167 -426
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

FAQ.md
+10 -214
View File
@@ -1,214 +1,10 @@
<!--
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: AGPL-3.0-or-later
-->
# Frequently Asked Questions
## How does this all work?
This is a very big subject which will be later be expanded in a devlog. But putting it simply:
* As a requestor, you send a prompt to the horde which adds it into a smart "first in/first out" queue, prioritizing it according to the amount of Kudos you have. If the request has multiple generations required, it is split into multiple jobs by the horde.
* Any joined workers periodically check-in to the horde and request a job to do. The horde sends them the first available request they can fulfil, by order of kudos-priority, respecting the worker's wishes for specific workers who have priority.
* The worker generates the job's prompt and submits it back to the horde
* Once all the component jobs of a request have been submitted, the horde marks the request as "Done"
* The requestor can at any point check the status of their waiting requests and retrieve the results, even if they're not "done"
## Workers?
![](img_faq/worker.png)
### What is a worker?
A worker is computer with usually a mid-range or higher class GPU, who has installed specific software to generate images via Stable Diffusion or KoboldAI locally and it connected to the horde through what we call the "bridge". It is constantly polling the horde for new generations and receiving Kudos in return for creating them.
### What is in it for the worker?
Generating images for others 24/7 technically costs electricity, we know.
People want to contribute to the horde for many reasons
* They want to provide constant processing power for tools they've built on top of Stable Diffusion or KoboldAI, such as video games, chat bots or software plugins
* They want to accumulate Kudos so that when it's time to generate their own requests they can get them done faster.
* They want to warm their room productively (this is indeed a literal reason people have given)
* They are just nice people and want to support this endeavor.
### Can workers spy on my prompts or generations?
*Technically*, yes. While the worker software and the bridge code is not set to allow it, ultimately the software resides at someone else's computer and is open source. As such, anyone with the know-how can modify their own code to not just see all prompts passing through, but even save all results they generate.
However workers do not have any identifying information about individual requestors as they cannot see their ID or IP.
Nevertheless always request generations as if you're posting in a public forum like using a discord bot. While the horde is technically more private than that, it's a good practice anyway.
### Can I turn off my worker whenever I want?
Yes! We do not require workers to stay always online. We only request that you put your worker in maintenance before you do so to avoid messing with a currently running generation your worker might have picked up.
### Do workers support multiple models
Yes, but you select your model manually. You can also not select a model and get the first model the next worker that picks up your work provides
### Can I prioritize myself and my friends on my own worker?
Yes! By default, your own requests are always served first by your own worker, regardless of Kudos. But you can also specify specific usernames which will also be prioritized the same way.
## Kudos?
![](img_faq/kudos.png)
### What, what is Kudos?
Another big subject. This one actually [has a devlog about it](https://dbzer0.com/blog/the-kudos-based-economy-for-the-koboldai-horde/)
### How do I get Kudos?
Connect a worker to the horde, that is all! You will generate kudos for each request you fulfil, relevant to its difficulty, and you will also generate kudos every 10 minutes your worker stays online.
### How is image Kudos consumption calculated?
The Kudos cost reflects the amount of processing required to generate the image.
As each payload on the horde can have too many variables which affect its speed, we have trained a neural network which takes as input a request payload, and calculates how much kudos it would require, based on how much faster or slower it would generate compared to a baseline of 10 kudos for a 50 step 512x512 image. The baseline costs 10 kudos. So if a payload is expected to take double that time, it will be valued as 20 kudos.
The AI Horde API provides a `dry_run` payload key. When set to true, it will return the kudos cost for an image, without actually requesting a generation.
On top of that there is what's known as the "horde tax" which represents the extra costs to the infrastructure for each request. These kudos are not received by the worker but are rather "burnt" forever.
* There is a 1 kudos tax per request. This is applied even if the request it cancelled, faulted or aborted.
* There is a 1 kudos tax per job in a request.
* When requesting only fast workers, there's an added +20% kudos burn.
* When requesting a worker blacklist, there's an added +10% kudos burn.
### I don't have a powerful GPU. How can I get Kudos?
We use Kudos to support good behaviour in the community. As such we have ways to receive Kudos outside of generating images for others (although that's the best way)
* If you have at least 2GB VRAM, you can run an alchemist, which is used to interrogate or post-process images.
* Rate some images. Almost all clients should have a way to rate images while waiting which will provide kudos per rating, and you can also rate the images you just generated for a kudos refund! [Artbot has a very easy rating page](https://tinybots.net/artbot/rate)
* Fulfill a bounty from our discord bounties forum
* Subscribe to [the patreon supporting the development of the AI Horde](https://www.patreon.com/db0).
* Generate and share some cool art in our discord.
* Politely request some people to transfer some to you in our discord server. People tend to give plenty to new users and helpful or funny comments.
### Can I transfer my Kudos?
**Yes!** Check the `api/v2/transfer` endpoint.
Remember however that the Kudos is merely a prioritization mechanism, **not a currency**. The AI Horde are under no obligation to maintain Kudos totals or current rate of return, and we may tweak them to ensure more optimal operation of the system.
### Can I sell my kudos?
**NO!** Kudos is inherently valueless and we do not allow anyone exchanging kudos for money. Bypassing this requirement is an existential threat to the AI Horde. Please do not attempt to do this under the table. **If you exchange money for Kudos and we discover it, we might zero out your account and whoever you bought it from!**
### Is Kudos a cryptocurrency?
**No!** Kudos is completely centralized and involved no blockchain tech whatsoever. The AI Horde is explicitly hostile to blockchain technologies and we will never integrate with any of them. Likewise, there is no way to convert kudos to anything other than favours benefiting the improvement of the AI Horde.
### Can I exchange kudos for cryptocurrencies?
**No!** Same rules and reasoning applies as for selling kudos, see above.
## Not Safe for Work?
![](img_faq/nsfw.png)
### Can I request NSFW images/text?
Yes, but you might have a smaller a pool of workers to fulfil your request, which will lead to longer generation times.
### Do you censor generations?
Horde-wide, we censor only one type of generation: [CSAM](https://en.wikipedia.org/wiki/Child_sexual_abuse_material) Images. We have two mechanisms to achieve this, one is a regex replacement filter during initial API request. The other is the anti-CSAM AI running on each worker. See more detailed answers below.
Other than this one instance, the horde does not censor text generations at all, or images which do not appear to be CSAM.
However each individual worker might have its own censorship guidelines. And each requestor can voluntarily opt-in to accidental NSFW censorship.
## Why are some of my images just black with white text?
Those generations have been NSFW-censored by the worker generating them. If you've specified your request as SFW, individual SFW workers who fulfil it might have the NSFW censorship model active, which will return just this black image. To avoid such images, turn on NSFW, or ensure your prompt is not too close to the edge of SFW/NSFW. If the image mentions censor due to the anti-CSAM filter, this cannot be turned off.
## Why are some of my images are still censored even though I'm requesting NSFW?
Each individual worker can optionally define a censorlist. If any word inside that list is found, the worker will automatically post-process using a NSFW censorship model. These words are things that should never be combined with NSFW content or would run into legal troubles for the worker if they did.
This means your censored images triggered one such worker's censorlist. You can rerun the prompt and hope you get a generation with a seed that doesn't trigger the NSFW model, or hope to get a new worker, or tweak your prompt.
If you feel a worker is using the censorlist maliciously, or improperly, please contact us with the content of your prompt and the worker name, and we'll address it.
## Can you explain how the Anti-CSAM regex filter works?
When an image request first comes into the AI Horde, it is passed through a private regex filter looking for combination of two contexts: "Underage" context and "Lewd" Context. An example of an underage context might be "child" and an example of a lewd context might be "without clothes".
If none, or one of these contexts is detected in a prompt, then the prompt is allowed to pass through unaffected. For example "child in the playground" is ok. "Without clothes in the bathroom" is also OK.
If however both of these terms are present in the same prompt, this will trigger the regex protection. This has two modes of operation:
If the `replacement_filter` is `true` in your API payload and the prompt is less than 1000 chars, each triggering term will be automatically replaced by an "adult" version of those terms. For example "school" will be replaced by "university". This attempts to point the inference to try and generate safe content.
If however `replacement_filter` is `false` or your prompt is above 1000 chars, then the request is automatically blocked instead, and you get an IP timeout for a couple of minutes. This IP timeout increases further, every time you get caught by the regex filter. This is to prevent people deliberately trying to out the filter to reverse engineer loopholes.
## Can you explain how the Anti-CSAM AI works?
We have written [a detailed devlog about this](https://dbzer0.com/blog/ai-powered-anti-csam-filter-for-stable-diffusion/)
### Where can I read more of NSFW controls?
* [The NSFW Question](https://www.patreon.com/posts/nsfw-question-72771484)
* [Blacklists](https://www.patreon.com/posts/72890784)
## Horde?
![](img_faq/horde.png)
### Why "AI Horde"?
This project started as a way to consolidate resources for the KoboldAI Client. As we needed a name for it, I came up with something thematic for the concept of "Kobolds". "A Horde of Kobolds". When I started doing image generation as well, I kept the "Horde" part.
### Can you explain the terminology around the AI Horde?
We have [a dedicated wiki page](https://github.com/Haidra-Org/AI-Horde/wiki/Terminology) where you can look a lot of the cross-referenced terms commonly mentioned in the context of the AI Horde
### Does the horde spy on my prompts and generations?
No, the horde itself is not storing such details. The prompts and the generations are only stored in-memory transiently and deleted shortly after the generation is delivered or cancelled.
### Why should I use the Horde and not my local PC?
Not everyone has a power GPU in their PC. The horde allows anyone to use fast Stable Diffusion and KoboldAI, not only the ones privileged enough to be able to afford an expensive graphics card.
Furthermore, local clients, even at the best of times, are difficult to setup up and often error prone due to python dependencies. They also need plenty of internet bandwidth to download 4GB of models. The stable horde provides no-install clients, as well as browser clients you can use even on your phone!
Finally if you wanted to provide a service built on image or text generation, you can now use your own PC to power your image generations, and therefore avoid all the complexity and capital costs required with setting up a server infrastructure.
### Why should I use the Horde and not a service like Stability.ai?
Because the Horde is free! You will never need to pay to use the horde. Sure if the demand is high, your delivery speed might not be great, but that is true with other services like midjourney
Second, the Horde gives you all the benefits of a local installation, such as freedom in prompts, while still allowing a browser interface. and flexibility.
Finally unlike many of these services, Horde also provides a fully fledged REST API you can use to integrate your applications, without worrying about costs.
### Why should I use the Horde and not a free service?
Because when the service is free, you're the product!
Other services which run on centralized servers have costs. Someone has to pay for the electricity, and the server infrastructure. The horde is explicit of how these costs are crowdsourced and there's no need for us to ever add anything in the future to change our existing model. Other free services tend to be deliberately obscure in how they use your prompts, results, and data, or explicitly say that your data is going to be the product. Such services eventually pivot their usercount to make money through advertisements and data brokering.
If you're fine with that, go ahead and use them.
Finally a lot of these services do not provide free REST APIs, so if you need to integrate with them, you have to use a browser interface, so that you can see the adverts.
### Can I run my own private horde?
Of course! This software is FOSS and you are welcome to use, modify and share, so long as you respect the AGPL3 license.
If you set up your own horde of course, you will need to also maintain your own workers, as hordes do not share workers with each other.
### Can I built paid services integration into the AI Horde?
Yes, with some restrictions. Due to the voluntary nature of the AI Horde, you **must** give back to the AI horde at least as much as you take out to make a profit. Please see the detailed explanation [in this devlog](https://dbzer0.com/blog/what-about-paid-services-on-top-of-the-ai-horde/)
<!--
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: AGPL-3.0-or-later
-->
# Frequently Asked Questions
The FAQ is on the website at [aihorde.net/faq](https://aihorde.net/faq). For running your own private horde, see the
[Docker readme](README_docker.md).
README.md
+126 -90
View File
@@ -7,150 +7,186 @@ SPDX-License-Identifier: AGPL-3.0-or-later
# AI-Horde: Community-Powered AI Generation
The [AI Horde](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#ai-horde) is a free community service that lets anyone create [AI-generated images](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#image-generation) and [text](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#text2text). In the spirit of projects like Folding@home (sharing compute for medical research) or SETI@home (sharing compute for the search for alien signals), AI Horde lets volunteers share their computer power through [workers](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#worker) to help others create AI art and writing.
[Generate images](https://aihorde.net/generate/image) · [Apps and integrations](https://aihorde.net/guis) · [Image guide](https://aihorde.net/docs/image) · [REST API](https://aihorde.net/api/) · [Service documentation](https://haidra-org.github.io/AI-Horde/) · [Discord](https://discord.gg/3DxrhksKzn)
When you make a [request](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#request) - like asking for "a painting of a sunset over mountains" - the AI Horde system finds available volunteer computers that can handle your [job](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#job). It's similar to how ride-sharing apps connect passengers with nearby drivers, but instead of rides, you're getting AI-generated content.
> **What is it?** You can make images and text with open models for free, without an expensive computer, and no account or setup is needed. The Horde has no ads and does not sell your data. It is run by a non-profit and paid for by donations.
>
> **How do I start?**
>
> - **Images**: open the [image generator](https://aihorde.net/generate/image) in your browser, type a prompt, optionally select a style, and click generate. There are guided tutorials as well to get started.
> - **Text and chat**: use [KoboldAI Lite](https://lite.koboldai.net) in your browser, or another app from the [list of integrations](https://aihorde.net/guis).
> - **Faster results**: [register](https://aihorde.net/register) for a free API key and enter it in any app. Registered users get better queue priority than anonymous ones. [Contribute your GPU (Become a worker)](#contribute-your-gpu-become-a-worker) to earn kudos. The kudos you earn is equal to the amount of work your GPU contributes. You can always redeem them for as many requests as you generated, allowing you to "bank" compute.
> - **Your own software**: use the [REST API](https://aihorde.net/api/) or the [OpenAI-compatible endpoint](#openai-compatible-chat-completions).
The system uses "[kudos](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/kudos.md)" points to keep things fair. [Workers](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#worker) earn kudos when their computers help process requests, which they can use to get priority for their own requests or leave unspent to help others. Importantly, kudos can never be bought or sold - this is strictly against the [Terms of Service](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#terms-of-service). If you would like to learn more about kudos, see [our detailed explanation](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/kudos.md).
The [AI Horde](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#ai-horde) is a free, community-run service for [AI image generation](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#image-generation) and [text generation](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#text2text). Volunteers run [workers](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#worker) on their own GPUs, and the Horde sends each [request](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#request) to a worker that can serve it. The idea is the same as Folding@home or SETI@home: people share spare compute so that others can use it.
What makes AI Horde special is that it's completely free and community-run, with a strong commitment to staying that way. The kudos system is specifically designed to ensure that access to these resources remains equitable. While [users](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#user) with more kudos get faster service, anyone can use it for free, even [anonymously](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#anonymous), and kudos never expire.
Anyone can use the AI Horde for free, even [anonymously](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#anonymous). The Horde uses only open-source models and software, and this repository is the server that runs it. Our [mission page](https://aihorde.net/mission) explains why the project exists and what it commits to, and the [motivations document](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/why.md) goes into more detail.
The AI-Horde hopes to ensure that everyone gets a chance to use these exciting AI technologies, regardless of their financial means or technical resources. You can read more about why we do this in the [motivations document](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/why.md).
Priority in the queue is decided by [kudos](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/kudos.md). Workers earn kudos for the jobs they complete and can spend them to move their own requests forward, or leave them unspent. Kudos cannot be bought or sold. Trading them is against the [Terms of Service](https://aihorde.net/terms). Users with more kudos get faster service, but everyone gets served, and kudos do not expire.
## Sponsors
## Funding and Sponsors
The AI Horde is run by [Haidra](https://haidra.net), a volunteer-run, registered non-profit in Luxembourg. It is paid for entirely by donations. Volunteers donate the GPU time that does the generation, and financial donations pay for the servers, storage and bandwidth that connect everything. If the Horde is useful to you, please consider [supporting Haidra](https://haidra.net/donate/) or [running a worker](#contribute-your-gpu-become-a-worker).
[![NLnet logo](assets/logo_nlnet.svg)](https://nlnet.nl/project/AI-Horde/)
The AI Horde has received funding from [NLnet](https://nlnet.nl/project/AI-Horde/) through the NGI0 Core fund. See all [sponsors and credits](https://aihorde.net/sponsors).
## Table of Contents
- [AI-Horde: Community-Powered AI Generation](#ai-horde-community-powered-ai-generation)
- [Table of Contents](#table-of-contents)
- [Sponsors](#sponsors)
- [Technical Introduction](#technical-introduction)
- [Key Features](#key-features)
- [Public Version](#public-version)
- [Private Deployment](#private-deployment)
- [Technologies Used](#technologies-used)
- [How It Works](#how-it-works)
- [Getting Started with AI Horde](#getting-started-with-ai-horde)
- [OAuth2 Registered Account (Recommended)](#oauth2-registered-account-recommended)
- [Pseudonymous Account](#pseudonymous-account)
- [Anonymous Usage](#anonymous-usage)
- [Why Register?](#why-register)
- [Contribute your GPU (Become a worker)](#contribute-your-gpu-become-a-worker)
- [Image](#image)
- [Text](#text)
- [Integrate with the AI-Horde](#integrate-with-the-ai-horde)
- [Community](#community)
- [Funding and Sponsors](#funding-and-sponsors)
- [Table of Contents](#table-of-contents)
- [Technical Introduction](#technical-introduction)
- [Features](#features)
- [Public Service](#public-service)
- [OpenAI-Compatible Chat Completions](#openai-compatible-chat-completions)
- [Private Deployment](#private-deployment)
- [Technologies Used](#technologies-used)
- [Documentation](#documentation)
- [Getting Started with AI Horde](#getting-started-with-ai-horde)
- [OAuth2 Registered Account (Recommended)](#oauth2-registered-account-recommended)
- [Pseudonymous Account](#pseudonymous-account)
- [Anonymous Usage](#anonymous-usage)
- [Why Register?](#why-register)
- [Contribute your GPU (Become a worker)](#contribute-your-gpu-become-a-worker)
- [Image](#image)
- [Text](#text)
- [Integrate with the AI-Horde](#integrate-with-the-ai-horde)
- [Contributing to This Repository](#contributing-to-this-repository)
- [Community](#community)
- [License](#license)
## Technical Introduction
The AI Horde is an enterprise-level ML-Ops crowd-sourced distributed inference cluster for AI Models. This middleware supports both image and text generation, making it highly versatile. It is designed to be highly scalable, allowing for seamless drop-in/drop-out of compute resources.
The AI Horde is a distributed inference service. The server queues requests, matches them to volunteer workers, collects the results and keeps the kudos accounts. Workers can join and leave at any time, and the Horde routes work to whichever workers are online and able to serve it.
### Key Features
### Features
- **Scalability**: The system can scale effortlessly to accommodate varying workloads, ensuring efficient use of available resources.
- **Flexibility**: Supports both image and text generation, catering to a wide range of AI applications.
- **Community-Powered**: Relies on spare/idle resources provided by the community, making advanced AI accessible to everyone.
- **Image generation**: text-to-image, img2img, inpainting, ControlNet, LoRAs, textual inversions and post-processing, across Stable Diffusion, SDXL, Flux and other open model families.
- **Alchemy**: image interrogation, captioning, upscaling and other post-processing on existing images.
- **Text generation**: prompt completion through the native API and chat completions through the [OpenAI-compatible endpoint](#openai-compatible-chat-completions), with streaming, tool calls, structured output, reasoning and image inputs where the worker supports them.
- **Styles**: shareable generation presets with a published parameter policy. Frontends can apply them without reimplementing the rules.
- **Kudos accounting**: an event-sourced ledger with reservations, so every charge and payment is recorded and can be audited.
- **Moderation tools**: content filters, evidence records and review queues for the volunteer moderators.
### Public Version
### Public Service
The [public version](https://aihorde.net) allows individuals without powerful GPUs to use advanced AI models like Stable Diffusion or Large Language Models (LLMs) such as Pygmalion/Llama. This is achieved by leveraging the spare computing power provided by the community, running software we call '[workers](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/definitions.md#worker)'. Additionally, it supports clients, such as third party websites, games and apps, to utilize AI-generated content through a [REST API](https://aihorde.net/api/). You can see the public instance's performance and application statistics on [our grafana instance](https://grafana.aihorde.net/d/jSb16YLVk/performance?orgId=1).
The [public service](https://aihorde.net) lets people without powerful GPUs use open models through the compute shared by the community. You can generate images directly on the website at [aihorde.net/generate/image](https://aihorde.net/generate/image), with no account required. The [image generation guide](https://aihorde.net/docs/image) walks through a first image and explains each setting. The [model reference](https://reference.aihorde.net) lists the models known to the Horde by category, with search.
Many apps, bots, games and plugins are built on the Horde. The [list of integrations](https://aihorde.net/guis) covers frontends for images and text, Discord and fediverse bots, editor plugins and developer tools such as the [Python SDK](https://github.com/Haidra-Org/horde-sdk).
Third-party websites, games and apps can use the [REST API](https://aihorde.net/api/) directly. Live status is on the [status page](https://aihorde.net/status), and performance statistics are on [our Grafana instance](https://grafana.aihorde.net/d/jSb16YLVk/performance?orgId=1).
### OpenAI-Compatible Chat Completions
The Horde has an OpenAI-compatible endpoint at `https://oai.aihorde.net/v1`. It supports `POST /v1/chat/completions` (streamed and non-streamed) and `GET /v1/models`, so most OpenAI client libraries and chat frontends work with it after changing the base URL. Send a Horde API key as the bearer token, or use the anonymous key `0000000000`.
```bash
curl https://oai.aihorde.net/v1/chat/completions \
-H "Authorization: Bearer 0000000000" \
-H "Content-Type: application/json" \
-d '{"model": "<a model id from /v1/models>", "messages": [{"role": "user", "content": "Hello"}]}'
```
The [interactive API documentation](https://oai.aihorde.net/docs) lists the supported request keys and the differences from OpenAI, and the [chat front reference](https://haidra-org.github.io/AI-Horde/reference/chat_front/) covers them in full. The endpoint runs as its own process (`python -m horde.oai`) in front of the native chat API. To get started, see [your first chat completion](https://haidra-org.github.io/AI-Horde/tutorial/first_chat_completion/) and [send chat completions](https://haidra-org.github.io/AI-Horde/how-to/send_chat_completions/). Worker operators can read [serve your first chat job](https://haidra-org.github.io/AI-Horde/tutorial/first_chat_worker/).
### Private Deployment
The AI Horde middleware can also be deployed privately within any enterprise environment. It can be installed within hours and can scale your ML solution within days of deployment, providing a robust and flexible solution for enterprise-level AI needs.
The Horde server can also be deployed privately, for a team or an organization that wants to pool its own GPUs. Container images are published to the GitHub Container Registry, and the [Docker readme](README_docker.md) describes a full deployment with Docker Compose. A private horde does not share workers with the public one, so it needs its own.
### Technologies Used
- **Python**: 3.9 or later
- **PostgreSQL**
- **Redis**: For caching
- **Docker** support for full containerization
- **Python** 3.12 or 3.13, with dependencies managed by [uv](https://docs.astral.sh/uv/)
- **Flask** and **Flask-RESTX** for the native REST API, served by **Waitress**
- **FastAPI** and **Uvicorn** for the OpenAI-compatible chat endpoint
- **PostgreSQL** for the main database and the kudos ledger
- **Redis** for caching, rate limits, locks and chat streams
- **S3-compatible object storage** (Cloudflare R2 on the public service) for images and large payloads
- **Docker** images and a Docker Compose file for the full stack
- **OpenTelemetry** instrumentation for tracing and metrics
### How It Works
### Documentation
For a high-level overview of how the AI Horde operates, including diagrams of the request/job lifecycle, see the [request/job lifecycle explanation](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/workers.md).
- [Service documentation](https://haidra-org.github.io/AI-Horde/) (published from [docs/](docs/README.md)): tutorials, how-to guides, reference and explanation pages for developers, integrators and operators of this server, and the [architecture decision records](https://haidra-org.github.io/AI-Horde/decisions/).
- [Horde concepts](https://github.com/Haidra-Org/haidra-assets/tree/main/docs): what the Horde is, workers, the [request and job lifecycle](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/workers.md) and [kudos](https://github.com/Haidra-Org/haidra-assets/blob/main/docs/kudos.md).
- [Image generation guide](https://aihorde.net/docs/image) on the website, for people making images.
- [FAQ](https://aihorde.net/faq) on the website.
- [Return codes](README_return_codes.md) for API errors.
Maintainers and contributors can find repository-specific architecture documentation in [the local documentation
index](docs/README.md), including the kudos accounting explanation, reference, and operational procedure.
## Getting Started with AI Horde
There are also individual readmes for each specific mode supported by the AI Horde.
You can use any service built on the AI Horde with a registered account or anonymously. To try it right away, [generate an image on the website](https://aihorde.net/generate/image) or pick an app from the [list of integrations](https://aihorde.net/guis).
- [Image generation Readme](README_StableHorde.md)
- [Text generation Readme](README_KoboldAIHorde.md)
- [Docker Readme](README_docker.md)
### OAuth2 Registered Account (Recommended)
For common questions, check the [FAQ](FAQ.md). If you'd like more details on features of the horde or the reasoning behind it, see the [documentation which goes greater into depth](https://github.com/Haidra-Org/haidra-assets/tree/main/docs).
> Note: The only information we store from your OAuth2 account is its unique ID, and we do not use it for any other purpose. See our [privacy policy](https://aihorde.net/privacy) for details.
# Getting Started with AI Horde
You can use any service powered by the AI Horde either with a registered account or anonymously. You can find a partial list of services powered by the AI-Horde on our [main website](https://aihorde.net/).
## OAuth2 Registered Account (Recommended)
> Note: The only information we store from your account is your unique ID. We do not use your id for any other purpose. See our [privacy policy](https://aihorde.net/privacy) for more details.
- Visit [AI Horde Registration](https://aihorde.net/register)
- Log in using one of the supported OAuth2 services
- Choose your username and receive your API key
- Visit [AI Horde Registration](https://aihorde.net/register).
- Log in with one of the supported OAuth2 services.
- Choose your username and receive your API key.
- Benefits:
- Start with higher priority in generation queues
- Can change username or reset API key if needed
- Maintain and track your kudos balance
- Can recover account access through OAuth2 service
- Higher starting priority in the generation queues
- You can change your username or reset your API key
- Your kudos balance is kept and tracked
- You can recover access through the OAuth2 service
- Minimum kudos balance of 25
## Pseudonymous Account
### Pseudonymous Account
- Visit [AI Horde Registration](https://aihorde.net/register)
- This method is the default if you do not log in with an OAuth2 service (google, github, discord, etc)
- **Important**: If you lose your API key, the account cannot be recovered
- Cannot change username or reset API key
- Still earns and maintains kudos
- Still better priority than anonymous users
- Visit [AI Horde Registration](https://aihorde.net/register).
- This is the default when you do not log in with an OAuth2 service (Google, GitHub, Discord, etc).
- **Important**: if you lose your API key, the account cannot be recovered.
- You cannot change your username or reset your API key.
- You still earn and keep kudos.
- You still get better priority than anonymous users.
## Anonymous Usage
### Anonymous Usage
- Use API key '0000000000'
- Lowest priority in generation queues
- No kudos tracking
- No need to register
- Service may be restricted for anonymous users during high load
- Use the API key `0000000000`.
- Lowest priority in the generation queues.
- No kudos tracking.
- No registration needed.
- Anonymous use may be restricted when the Horde is under high load.
## Why Register?
### Why Register?
The main benefit of registration is participating in the kudos system. Kudos determine your priority in the queue - the more kudos you have, the faster your requests are processed. You can earn kudos by:
Registering lets you take part in the kudos system. Kudos decide your place in the queue: the more you have, the sooner your requests are processed. You can earn kudos by:
- Running a worker to help process other users' requests
- Receiving kudos as a thank-you for donations (though kudos cannot be directly bought or sold)
- Being online and available as a worker
- Participating in the kudos economy on the [official discord](https://discord.gg/3DxrhksKzn) by logging in with the bot. (Use the command `/login` in any channel, click the blue button and enter your API key at the popup).
- Running a worker that processes other users' requests, and keeping it online
- Receiving monthly kudos as a thank-you for a [subscription to Haidra](https://haidra.net/donate/) (kudos still cannot be bought or sold)
- Taking part in the kudos economy on the [official Discord](https://discord.gg/3DxrhksKzn) by logging in with the bot. Use the command `/login` in any channel, click the blue button and enter your API key in the popup.
Remember: AI Horde is committed to remaining free and accessible. While the kudos system provides priority benefits, it's designed to encourage community contribution rather than commercialization. All users, even anonymous ones, can access the service's core features. Read about this on the [official developer's blog.](https://dbzer0.com/blog/the-kudos-based-economy-for-the-koboldai-horde/)
The kudos system rewards contribution. It does not gate access, and every user, anonymous ones included, can use the service's core features. The [developer's blog post on the kudos economy](https://dbzer0.com/blog/the-kudos-based-economy-for-the-koboldai-horde/) explains the reasoning.
## Contribute your GPU (Become a worker)
Be sure to [register](#getting-started-with-ai-horde) first.
[Register](#getting-started-with-ai-horde) first, then run one of the workers below with your API key. The website has a [worker guide](https://aihorde.net/contribute/workers).
### Image
The officially supported way to add your GPU for image generation is by running the worker software [horde-worker-reGen](https://github.com/Haidra-Org/horde-worker-reGen).
The officially supported image worker is [horde-worker-reGen](https://github.com/Haidra-Org/horde-worker-reGen).
### Text
There are multiple ways to contribute your GPU for text generation:
- [KoboldCpp](https://github.com/LostRuins/koboldcpp) has a built-in Horde worker, including chat jobs.
- [AI-Horde-Worker](https://github.com/Haidra-Org/AI-Horde-Worker) connects an inference backend to the Horde.
- Any OpenAI-compatible backend can serve chat jobs. See [write a minimal chat worker](https://haidra-org.github.io/AI-Horde/how-to/write_a_minimal_chat_worker/) and [serve chat jobs](https://haidra-org.github.io/AI-Horde/how-to/serve_chat_jobs/).
- [AI-Horde-Worker](https://github.com/Haidra-Org/AI-Horde-Worker) can connect to inference backends and 'bridge' you to the AI-Horde
- [KoboldAI](https://github.com/henk717/KoboldAI)
- [KoboldCPP](https://github.com/lostruins/koboldcpp)
- [Aphrodite Engine](https://github.com/PygmalionAI/aphrodite-engine)
- ... and many others. Join the discussion in the #horde channel of the [KoboldAI Discord Server](https://discord.gg/XuQWadgU9k) to learn more.
Join the #horde channel on the [KoboldAI Discord server](https://discord.gg/XuQWadgU9k) to learn more about text workers.
## Integrate with the AI-Horde
If you want to build an integration to the AI Horde (Bot, application, scripts etc), please consult our [Integration Readme](README_integration.md)
To build a bot, app or script on the AI Horde, read the [integration readme](README_integration.md). It covers the request workflow, the endpoints for each generation type, shared keys and error handling. For Python, the [horde-sdk](https://github.com/Haidra-Org/horde-sdk) handles the request lifecycle for you. Chat clients can use the [OpenAI-compatible endpoint](#openai-compatible-chat-completions), and frontend authors can follow [add chat to a frontend](https://haidra-org.github.io/AI-Horde/how-to/add_chat_to_a_frontend/).
## Contributing to This Repository
See [CONTRIBUTING.md](CONTRIBUTING.md) for the development setup, tests, code style and documentation conventions. Changes are listed in the [changelog](CHANGELOG.md).
## Community
If you have any questions or feedback, we have a vibrant community on [discord](https://discord.gg/3DxrhksKzn)
Questions and feedback are welcome on [Discord](https://discord.gg/3DxrhksKzn), [Mastodon](https://sigmoid.social/@stablehorde) and [Lemmy](https://lemmy.dbzer0.com/c/aihorde). Project news is posted at [aihorde.net/news](https://aihorde.net/news).
## License
The AI Horde server is free software under the [GNU Affero General Public License v3.0 or later](LICENSE). Licensing follows the [REUSE](https://reuse.software/) specification, and the license texts are in [LICENSES/](LICENSES/).
README_KoboldAIHorde.md
+13 -78
View File
@@ -1,78 +1,13 @@
<!--
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: AGPL-3.0-or-later
-->
# AI Horde LLM Generation
Information specific to the Text generation
# Generating Prompts
## GUI
The easiest way to start using the [KoboldAI Lite](https://lite.koboldai.net) requiring no installation and no technical expertise
## Command Line
I have provided [a small python script](https://github.com/Haidra-Org/AI-Horde-CLI) with which you can use to call the horde. Please check the [README](https://github.com/Haidra-Org/AI-Horde-CLI/blob/main/README.md) for usage instructions
## Using KoboldAI client
KoboldAI Client supports using the AI Horde directly from its interface. To use it go to the AI menu on the top left, then select Online Services > KoboldAI Horde. In the next window that opens, you have to fill in the url of the Horde (You can use `https://aihorde.net`) and then type a username. If the models window hasn't appeared yet, click away from the textbox and it should. You can select one of more models that you want to use for your generations, or All, if you don't care. Then finally click `Load`.
![](gui_select.png)
You can also start KAI directly in horde mode by using the command line in the `play.(sh|bat)` file. Pass the arguments to start a KAI instance in cluster mode like so (Change "0000000000" to your own API KEY, if you have one.)
LINUX
```bash
APIKEY=0000000000
./play.sh --path https://aihorde.net --model CLUSTER --apikey ${APIKEY}
```
WINDOWS
```bash
play.bat --path https://aihorde.net --model CLUSTER --apikey 0000000000
```
This will use any available model on the cluster. If you want to use only specific models, pass the wanted modules via one or more `req_model` args. Example `--req_model "KoboldAI/fairseq-dense-13B-Nerys-v2" --req_model "KoboldAI/fairseq-dense-2.7B-Nerys"`
Once the KAI starts in cluster mode, any request will be sent to the AI Horde
# Joining the AI Horde
Anyone can convert their own PC into a worker which generates or interrogates images for other people in the horde and gains kudos for doing so. To do so, they need to run a software we call the AI Horde Worker, which bridges your Stable Diffusion inference to the AI Horde via REST API.
We have prepared a [very simple installation procedure for running the bridge on each OS](https://github.com/Haidra-Org/AI-Horde-Worker#readme).
## KoboldAI Client UI2
**You KoboldAI client must be using the UNITED branch!**
The United branch of KoboldAI now supports becoming a worker for the AI Horde directly from its interface. To use it you need to switch to UI2, load a model, and then in your settings sidebar, fill out your worker name and API key.
# Other Info
The cluster does not save any prompts nor generations locally. It's all stored in memory. Furthermore, requested prompts and their generations are wiped after 20 minutes of inactivity
The bridge also does not save any prompts, but of course this is not under my control as the bridges run locally on the various cluster nodes. As such, you should not prompt with anything you do not want others to see!
This system stores how many tokens you requested to generate, and how many your own servers have generated for others. This is not used yet, but eventually this will be how we balance resources among the users.
# Advanced Usage: Local + Horde KAI
If you want to both play with KAI AND share resources with the community, you can achieve this by running two instances of KAI side by side. One normal one, and one in cluster mode. That way when you're using the clustered KAI, you will ensure there's always at least one instance to serve you (your own), while also taking advantage of any other instances that are onboarded at the time.
1. start KAI as you would normally do with `play.(sh|bat)` and load the model you want.
2. open a terminal window at your KAI installation, and now run `play.(sh|bat)` using a different port in CLUSTER mode. This will start open another KAI window, while leaving your model-loaded KAI intact. Make sure the `req_model` you pass, includes the model loaded in your own KAI instance.
```bash
play.bat --port 5002 --path https://aihorde.net --model CLUSTER --apikey 0000000000 --req_model "KoboldAI/fairseq-dense-13B-Nerys-v2" --req_model "KoboldAI/fairseq-dense-2.7B-Nerys"
```
Now use the CLUSTER KAI to play. You will see the requests being fulfilled by the model-loaded KAI after you press the button.
<!--
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: AGPL-3.0-or-later
-->
# AI Horde Text Generation
- To generate text, use an app from the [list of integrations](https://aihorde.net/guis), such as
[KoboldAI Lite](https://lite.koboldai.net) or [SillyTavern](https://sillytavern.app/).
- To use the Horde from software written for the OpenAI API, see
[OpenAI-compatible chat completions](README.md#openai-compatible-chat-completions) in the main readme.
- To share your GPU, see [Contribute your GPU](README.md#contribute-your-gpu-become-a-worker) in the main readme.
README_StableHorde.md
+12 -30
View File
@@ -1,30 +1,12 @@
<!--
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: AGPL-3.0-or-later
-->
# AI Horde Image Generation
Information specific to the Stable Diffusion image generation.
# Generating Prompts
## GUI
We provide [a client interface](https://dbzer0.itch.io/lucid-creations) requiring no installation and no technical expertise.
<img src="https://raw.githubusercontent.com/Haidra-Org/Lucid-Creations/main/screenshot.png" width="500" />
You can also see our wiki which [describes multiple available frontends made by the Haidra community](https://github.com/Haidra-Org/AI-Horde/wiki/Available-client-interfaces).
## Command Line
I have provided [a small python script](https://github.com/Haidra-Org/AI-Horde-CLI) with which you can use to call the horde. Please check the [README](https://github.com/Haidra-Org/AI-Horde-CLI/blob/main/README.md) for usage instructions.
# Joining the horde
Anyone can convert their own PC into a worker which generates or interrogates images for other people in the horde and gains kudos for doing so. To do so, they need to run a software we call the AI Horde Worker, which bridges your Stable Diffusion inference to the AI Horde via REST API.
We have prepared a [very simple installation procedure for running the bridge on each OS](https://github.com/Haidra-Org/horde-worker-reGen#readme).
<!--
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: AGPL-3.0-or-later
-->
# AI Horde Image Generation
- To generate images, use the [on-site generator](https://aihorde.net/generate/image) or an app from the
[list of integrations](https://aihorde.net/guis). The [image generation guide](https://aihorde.net/docs/image) walks
through a first image.
- To share your GPU, see [Contribute your GPU](README.md#contribute-your-gpu-become-a-worker) in the main readme.
README_docker.md
+4
View File
@@ -6,6 +6,10 @@ SPDX-License-Identifier: AGPL-3.0-or-later
# Running ai-horde with Docker
You can run your own private horde with this software, for example to pool the GPUs of a team. It is free software
under the AGPL-3.0-or-later license, so you can use, modify and share it under that license's terms. Hordes do not
share workers with each other, so a private horde needs its own workers.
---
## Prerequisites
gui_select.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

img_faq/horde.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 399 KiB

img_faq/horde.png.license
-3
View File
@@ -1,3 +0,0 @@
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: CC0-1.0
img_faq/kudos.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 302 KiB

img_faq/kudos.png.license
-3
View File
@@ -1,3 +0,0 @@
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: CC0-1.0
img_faq/nsfw.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 307 KiB

img_faq/nsfw.png.license
-3
View File
@@ -1,3 +0,0 @@
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: CC0-1.0
img_faq/worker.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 475 KiB

img_faq/worker.png.license
-3
View File
@@ -1,3 +0,0 @@
SPDX-FileCopyrightText: 2022 Konstantinos Thoukydidis <mail@dbzer0.com>
SPDX-License-Identifier: CC0-1.0
index_stable.md
+2 -2
View File
@@ -6,9 +6,9 @@ SPDX-License-Identifier: AGPL-3.0-or-later
# {page_title}
<img style="float:right" src="{horde_img_url}" width="300" /> This is a [crowdsourced distributed cluster](https://github.com/Haidra-Org/AI-Horde) of [Image generation workers](https://github.com/Haidra-Org/horde-worker-reGen) and [text generation workers](https://github.com/Haidra-Org/AI-Horde-Worker). If you like this service, consider [joining the horde yourself](https://github.com/Haidra-Org/AI-Horde/blob/main/README_StableHorde.md#joining-the-horde)!
<img style="float:right" src="{horde_img_url}" width="300" /> This is a [crowdsourced distributed cluster](https://github.com/Haidra-Org/AI-Horde) of [Image generation workers](https://github.com/Haidra-Org/horde-worker-reGen) and [text generation workers](https://github.com/Haidra-Org/AI-Horde-Worker). If you like this service, consider [joining the horde yourself](https://github.com/Haidra-Org/AI-Horde#contribute-your-gpu-become-a-worker)!
For more information, check [the FAQ](https://github.com/Haidra-Org/AI-Horde/blob/main/FAQ.md). Finally you can also follow the [main developer's blog](https://dbzer0.com)
For more information, check [the FAQ](https://aihorde.net/faq). Finally you can also follow the [main developer's blog](https://dbzer0.com)
## Latest [News](/api/v2/status/news)