dilim/skills
1
0 Fork 0
mirror of https://github.com/mattpocock/skills.git synced 2026-11-06 05:17:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

software-factory: glossary skill for designing a factory (#1316)

Browse Source 
* software-factory: add glossary skill for designing a factory

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: add example runs grouped by problem

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: separate automated review from human review, add ratchet example

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: drop review definitions, keep the stricter wording

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: optional observability platform, slot surveys

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: slot surveys orient, the agent researches current options

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: add Task; example runs become example tasks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: start slow, accelerators and brakes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: start slow with throttled automated triggers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: tighten start slow

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: define the three brakes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* software-factory: tasks work on tickets, PRs or the codebase

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Matt PocockandClaude Opus 5.5 authored and GitHub committed 2026-10-31 14:23:30 +00:00
1 parent 5ae481693d
commit 4ae2317ddb
8 files changed
+522
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.changeset/software-factory-skill.md
+5
View File
@@ -0,0 +1,5 @@
---
"mattpocock-skills": patch
---
Add `software-factory`, an in-progress user-invoked reference skill: a glossary for designing a software factory (tasks, runs, runners, triggers, and the services they plug into), example tasks grouped by the problem they solve, surveys of the products that fill each slot (ticket trackers, git hosts, runners, observability platforms), and the questions that finish a design. Call it alongside a grilling session.
skills/in-progress/README.md
+1
View File
@@ -14,3 +14,4 @@ npx skills@latest add mattpocock/skills --skill=<name>
- **[writing-shape](./writing-shape/SKILL.md)**: Take a markdown file of raw material and shape it into an article paragraph by paragraph, arguing format choices at each step.
- **[claude-handoff](./claude-handoff/SKILL.md)**: Hand the current conversation off to a fresh background agent that picks up the work immediately, seeded with a handoff summary via `claude --bg`. User-invoked.
- **[setup-ts-deep-modules](./setup-ts-deep-modules/SKILL.md)**: Wire dependency-cruiser into a TypeScript repo so each package is a deep module: implementation hidden in subfolders, reachable only through its entry-point files, tests exercising it through those. User-invoked.
- **[software-factory](./software-factory/SKILL.md)**: Shared language for designing a software factory: the tasks it runs and the services it plugs into. Call it alongside a grilling session. User-invoked.
skills/in-progress/software-factory/EXAMPLE-TASKS.md
+80
View File
@@ -0,0 +1,80 @@
# Example tasks
Tasks grouped by the problem they solve. Each names the trigger that starts its runs, what a run does, and what it triggers next. Most factories start as one of these and grow by chaining the next.
## Untriaged tickets pile up
**Explore task**
- Trigger: a ticket is created, or a human labels it.
- Does: checks the ticket against the codebase and sorts it into a category (easy call, needs a human, not a bug).
- Triggers: an implementer run for easy calls. Anything else ends with the ticket labelled as waiting for a human.
## Ready tickets wait for someone to build them
**Implementer task**
- Trigger: a ticket is labelled ready, by a human or an explore run.
- Does: builds the ticket until the automated checks pass, and opens a draft PR.
- Triggers: an automated review run.
## PRs wait for review
**Automated review task**
- Trigger: a PR opens with no automated review yet, or a human labels it.
- Does: reviews the diff against the codebase's standards and the ticket it came from.
- Triggers: a rework run on a fail. On a pass it ends, and the PR waits for human review before a human merges it.
## Failed automated reviews sit unfixed
**Rework task**
- Trigger: an automated review run fails a PR.
- Does: fixes only what the automated review named, and pushes.
- Triggers: an automated review run. After a set number of fails, it ends with the PR labelled as waiting for a human.
## Specs are too big for one run
**Decompose task**
- Trigger: a spec ticket is labelled.
- Does: splits the spec into tickets, each recording which tickets it depends on.
- Triggers: an implementer run.
The implementer run here picks the next ticket whose dependencies are all closed, and triggers itself again until no ticket is left.
## Migrating to a stricter automated check
A new lint rule (or a stricter type setting) is switched on, but only some files pass it. The rule enforces on the passing files, and the failing ones sit on an exception list.
**Ratchet task**
- Trigger: a daily clock.
- Does: takes one file off the exception list, fixes it until the stricter check passes, and opens a PR.
- Triggers: an automated review run. Once the exception list is empty, the ratchet run has nothing to take and the migration is done.
One small PR a day keeps human review cheap, and the check only ever gets stricter.
## Users report bugs nobody reads
A feedback button in the app is a ticket source: it files each report as a ticket with the explore label already on it. The explore run does the rest.
## Production problems are found late
A monitor (slow queries, error rates) is a ticket source. It files a ticket per problem, and comments on the open ticket instead when it sees the same problem again. After a fix merges, it reopens the ticket if the problem persists.
## The factory's own runs fail
**Fix task**
- Trigger: the orchestrator reports a failed run.
- Does: files the failing log as a ticket, and attempts a fix as a PR when the failure is fixable.
- Triggers: an automated review run. A failure in a fix run's own work triggers nothing, so fixes never chain. Because the fix changes the factory itself, it always gets human review.
## The codebase drifts
**Architecture task**
- Trigger: a nightly clock.
- Does: scans recent changes for refactoring candidates and files them as one ticket.
- Triggers: nothing. A human picks a candidate and labels it, which starts the usual chain.
## The same mistakes keep coming back
**Retro task**
- Trigger: a PR merges, throttled to one merged PR in ten.
- Does: reads the PR, its ticket, its automated reviews and its run logs, and proposes changes to the factory's own instructions as a PR.
- Triggers: an automated review run.
skills/in-progress/software-factory/GIT-HOSTS.md
+84
View File
@@ -0,0 +1,84 @@
# Git hosts
> Surveyed 2026-10, and not guaranteed to be up to date. Use it to get oriented, then research the current options yourself: products, prices, limits and betas change fast.
## The types
### All-in-one platforms
The git host bundled with a ticket tracker, a CI system that serves as orchestrator, and hosted runners, all sharing one event stream and one permission model. One product fills git host, ticket tracker, orchestrator and runner, and its CI run pages often cover the observability platform too. Every PR, review, label and check event is both a webhook and a native trigger, so this type has the richest trigger catalogue.
**Pick it when** the factory starts small, the code already lives there, or the user wants one bill and one login. This is the "All on GitHub" design in [SKILL.md](SKILL.md). The vendors also ship their own agents, which either compete with the user's runs or slot in as one of them.
- [GitHub](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api): the default; Issues, Actions, Apps and the largest integration ecosystem.
- [GitLab](https://about.gitlab.com/pricing/): issues, merge requests, CI/CD and runners in one product, SaaS or self-managed.
### Suite git hosts
A git host sold as one module of a wider suite, where the ticket tracker is a sibling product (Jira, Azure Boards) joined by links and webhooks. Tracker events arrive on a second stream, so the orchestrator joins two event sources.
**Pick it when** the organisation already runs the suite and its humans live in Jira or Azure Boards. The factory inherits that tracker as its human interface.
- [Bitbucket Cloud](https://support.atlassian.com/bitbucket-cloud/docs/api-request-limits/): Atlassian's git host, paired with Jira and Bitbucket Pipelines.
- [Azure DevOps (Azure Repos)](https://learn.microsoft.com/en-us/azure/devops/integrate/concepts/rate-limits): Repos, Boards, Pipelines and Entra identities in one organisation, so it fills all four slots.
- [AWS CodeCommit](https://aws.amazon.com/blogs/devops/aws-codecommit-returns-to-general-availability/): a bare git host with IAM identities; pair it with an external tracker and CI.
### Self-hosted git hosts
A git host the user runs on their own infrastructure, often beside the orchestrator and runners on the same network. The operator sets the rate limits, so the ceiling is the hardware. The user owns upgrades, backups and uptime for every slot it fills.
**Pick it when** code must stay on-premises or air-gapped, the user wants zero per-seat cost, or a SaaS API budget throttles a busy factory. Forgejo and Gitea suit small teams and bundle issues plus GitHub-Actions-compatible CI; GitLab self-managed and GitHub Enterprise Server suit organisations and keep their Apps and service accounts.
- [Forgejo](https://forgejo.org/2026-10-release-v17-0/): community-governed, lightweight, with Forgejo Actions.
- [Gitea](https://about.gitea.com/): lightweight with Gitea Actions; also sold managed.
- [GitLab self-managed](https://about.gitlab.com/install/) and [GitHub Enterprise Server](https://docs.github.com/en/enterprise-server@latest): the full platforms on the user's own infrastructure.
### Review-first and decentralised git hosts
Git hosts whose review model departs from the branch-and-PR shape: per-commit patchsets with voted labels, or repositories federated across user-run nodes. Most factory tooling assumes PRs, so expect to build more of the glue. In Gerrit, an automated check run votes `Verified` from a bot account and a human votes `Code-Review`.
**Pick it when** the user's team already reviews this way, or values federation over ecosystem.
- [Gerrit](https://www.gerritcodereview.com/): patchset review with label votes; the Android and Chromium model.
- [Tangled](https://blog.tangled.org/ci/): git hosting on the AT Protocol with stacked PRs and its own CI runner.
### Agent-native git hosts
New git hosts built around agents as the main authors: high commit throughput, server-side reads in place of clones, and per-agent attribution. The whole type is early (as of 2026-10), so gates and rate limits may be unpublished: ask the vendor.
**Pick it when** the factory runs many concurrent runs against one large repository and clone time or API limits dominate. Pilot one on a mirror of a single repository, keeping GitHub as the fallback.
- [Cursor Origin](https://cursor.com/origin): from the Graphite team, with GitHub sync and signed PR, review and push webhooks.
- [GitLab next-gen SCM](https://about.gitlab.com/blog/gitlab-next-gen-scm/): rebuilt GitLab backend with clone-free agent APIs.
### Git storage for runs
Headless, API-first git repositories created by the thousand: branches, commits and diffs, with no PRs, reviews or gates. They fill half the slot, with short-lived per-repository tokens minted per run.
**Pick it when** runs need cheap per-run scratch repositories or forks (a fork per ticket, an agent's working state) and the PR still lands on a full git host from the types above.
- [Cloudflare Artifacts](https://www.cloudflare.com/product/artifacts/): git-compatible storage on Workers; fork a baseline per task.
- [Pierre code.storage](https://code.storage/docs): white-label git API with ephemeral branches.
## Researching further
Check each candidate against these questions:
1. **Triggers**: which events (PR opened, push, review submitted, label added, check finished) reach the orchestrator as webhooks, and are deliveries signed?
2. **Identity**: can a run act as a dedicated bot or app identity? That keeps runs attributable, lets an automated review run approve a PR another identity authored, and decides whether a run's own pushes trigger the next run.
3. **Gates**: are required checks, required approvals and branch protection available on the user's plan and repository visibility, and can a bot sit on a bypass list?
4. **Rate limits**: what API budget does every run on every runner share, per what unit (installation, token, user, workspace), and how does it scale?
5. **Slots filled**: which of ticket tracker, orchestrator, runner and observability platform does the same product cover?
Find the answers in primary sources: the REST API and webhook reference, the rate-limit page, the pricing and plan comparison page, the access-token or app docs, and the changelog or release notes.
The fastest to go stale: beta and preview status (the whole agent-native and git-storage types), rate-limit numbers, which plan unlocks gates, and availability after acquisitions, shutdowns and reopenings. As of 2026-10, for example, AWS CodeCommit had reopened to new customers and Bitbucket Data Center was sold only as a hybrid licence with Cloud.
## Durable gotchas
Verify each still holds before the design relies on it.
- **GitHub `GITHUB_TOKEN` ends chains.** Events caused by the built-in Actions token start no new workflow runs (except `workflow_dispatch` and `repository_dispatch`), so a run pushing with it silently stops the next run. Give runs a GitHub App identity from day one.
- **Gates often sit behind a paid plan.** As of 2026-10, GitHub rulesets on private repositories need Pro, Team or Enterprise, GitLab approval rules need Premium, and enforced Bitbucket merge checks need Premium.
- **Rate limits are per identity, and the bot's type changes them.** As of 2026-10, a GitHub App installation gets 5,000 requests/hour against 1,000 for `GITHUB_TOKEN`; Bitbucket user tokens stay at the base rate while access tokens scale; Azure DevOps raises a bot's budget with its access level.
- **Suite trackers mean two event streams.** The orchestrator joins git host and ticket tracker events itself.
skills/in-progress/software-factory/OBSERVABILITY-PLATFORMS.md
+75
View File
@@ -0,0 +1,75 @@
# Observability platforms
> Surveyed 2026-10, and not guaranteed to be up to date. Use it to get oriented, then research the current options yourself: products, prices, limits and betas change fast.
Orientation for filling the observability platform slot, or deciding on a deliberate none. The types split by **what the human needs to see inside a run**: its log, its agent trace (each model call and tool call as a span), or that trace joined to the rest of the user's system. Self-hosted vs SaaS cuts across the last three types. When a run should act on what the platform sees, make the platform a ticket source: a monitor files the ticket, and a run reads it.
## How runners hook in
Every type receives data through one of five hooks. Name the hook alongside the product when filling the slot.
- **Harness OTel**: the agent harness exports OpenTelemetry itself. Claude Code and the Claude Agent SDK take `CLAUDE_CODE_ENABLE_TELEMETRY=1` plus `OTEL_*` variables, with prompts and tool output opt-in; Codex CLI takes an `[otel]` block in `config.toml`. ([Claude Agent SDK](https://code.claude.com/docs/en/agent-sdk/observability), [Claude Code](https://code.claude.com/docs/en/monitoring-usage), [Codex config](https://developers.openai.com/codex/config-advanced))
- **Platform push**: the model platform sends traces for you, as Claude Managed Agents do to an OTLP destination set in the Claude Console. ([Claude docs](https://platform.claude.com/docs/en/manage-claude/opentelemetry/api))
- **SDK wrapper**: one call in the runner's code, such as LangSmith's `configure_claude_agent_sdk()`.
- **Hook plugin**: a Claude Code hook reads the session transcript at each Stop and ships it, as in Langfuse's Claude Code plugin.
- **Gateway**: the runner's model base URL points at a logging proxy. Zero runner code, and it sees model calls only: tool calls inside the harness stay invisible.
OpenTelemetry is the common wire: harness OTel and platform push land in any OTLP backend. The GenAI semantic conventions (`invoke_agent`, `chat`, `execute_tool` spans) are still settling, so expect span names to drift. ([OTel GenAI agent spans](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-agent-spans))
## Type 1: None (the runner or orchestrator product shows enough)
**What it is**: the human reads each run where the runner or orchestrator already shows it: the CI run page, the sandbox dashboard, the agent's transcript saved as a run artifact. No hook to build, as self-hosted as the runner, and bundled with its cost.
**When a factory picks it**: few runs a day, one human watching, and the question is "did it fail, and what did it print?" Most factories start here, and grow out of it when they need search across runs, a token or cost rollup per run, or retention longer than the runner keeps.
**Starting points**: [GitHub Actions run logs](https://docs.github.com/en/organizations/managing-organization-settings/configuring-the-retention-period-for-github-actions-artifacts-and-logs-in-your-organization), [E2B sandbox logs and metrics](https://e2b.dev/docs), [Daytona sandbox telemetry](https://www.daytona.io/docs/en/observability/otel-collection/), [OpenAI Agents SDK traces](https://openai.github.io/openai-agents-python/tracing/). Sandbox products often forward the same data over OTLP, which is the path out to Type 2.
**Verify**: retention days per runner product; some sandboxes keep very little (Daytona: 3 days, as of 2026-10).
## Type 2: LLM and agent tracing platform
**What it is**: a backend built around the agent trace: each run as a tree of model calls and tool calls, with token cost, prompts and sessions, usually with evals and datasets on top.
**When a factory picks it**: the human asks "why did the agent do that?" across many runs; a retro run or a human compares runs; token spend needs a per-run rollup. Self-hosting an open-source one keeps transcripts in-house.
**Traits of the type**: takes harness OTel, an SDK wrapper or a hook plugin, often the platform push too. Billed by volume, and agent runs are span-heavy (one span per tool call), so budget per run rather than per request.
**Starting points**: [Langfuse](https://langfuse.com/pricing) and [Arize Phoenix](https://phoenix.arize.com) (open source, self-hostable), [LangSmith](https://www.langchain.com/pricing-langsmith), [Braintrust](https://www.braintrust.dev/pricing).
**Verify**: free-tier limits, the billing unit (trace, span, unit or GB; Braintrust moved to GB in 2026-03), which tier unlocks self-hosting, and current ownership.
## Type 3: General APM and logging backend
**What it is**: the user's existing observability stack (logs, metrics, distributed traces for every service), with an agent or LLM view added.
**When a factory picks it**: the team already pays for one; runs need joining to the app's own traces (the Agent SDK propagates `TRACEPARENT`, so a run nests inside its caller's trace); or a monitor ticket source already reads from it.
**Traits of the type**: harness OTel goes straight to the vendor's OTLP endpoint. Priced per GB, event or host, so a factory's volume is a rounding error on an existing bill and a surprise on a new one.
**Starting points**: [Datadog Agent Observability](https://www.datadoghq.com/product/llm-observability/), [Grafana Cloud](https://grafana.com/pricing/), [Honeycomb](https://www.honeycomb.io/pricing), [SigNoz](https://signoz.io/pricing/) (open source).
**Verify**: whether the agent view is GA or preview, which harnesses have a setup guide, and free-tier volumes.
## Type 4: LLM gateway with request logging
**What it is**: a proxy between the runner and the model provider that logs every request and response, with routing, keys, budgets and caching alongside.
**When a factory picks it**: the user wants per-run spend caps or one key vault for many runners, and logging comes with it; or the harness has no export of its own. It sees model calls only, so pair it with Type 2 or 3 for tool calls.
**Starting points**: [LiteLLM Proxy](https://docs.litellm.ai/docs/proxy/logging) (open source, forwards logs to Langfuse or OTel), [Portkey](https://portkey.ai/pricing).
**Verify**: per-request pricing on SaaS, and whether the product is still actively developed (Helicone went into maintenance mode after an acquisition, as of 2026-10).
## Researching further
Check each candidate against the questions that decide this slot:
1. **Does the runner or orchestrator already show enough?** Compare what the human needs (cross-run search, per-run cost, retention) with what the run page or sandbox dashboard gives. A yes here means a deliberate none.
2. **How does the agent harness export traces?** Find which of the five hooks the harness supports, then which the product accepts. Confirm tool calls arrive as spans, and whether prompts and tool output need an opt-in.
3. **Is it self-hostable?** Licence, which tier allows it, and what it runs on. Decisive when transcripts must stay in-house.
4. **What is the cost shape?** The billing unit, the free tier, and an estimate per run given one span per tool call, multiplied by runs per day.
5. **How long is retention?** Days kept on each tier, against how far back the human looks when a run goes wrong.
**Where the answers live**: the vendor's pricing page (units, tiers, retention); the vendor's integration doc for the specific harness; the harness's own monitoring docs (what it exports, which flags are beta); the OpenTelemetry GenAI semantic conventions; the product's repo for licence and release activity; the vendor's changelog or blog for ownership changes.
**What goes stale fastest**: free-tier limits and prices; beta and preview flags (Claude Code trace export, Console platform push, vendor agent views); ownership and maintenance status; billing units; retention days; GenAI span names.
skills/in-progress/software-factory/RUNNERS.md
+94
View File
@@ -0,0 +1,94 @@
# Runners
> Surveyed 2026-10, and not guaranteed to be up to date. Use it to get oriented, then research the current options yourself: products, prices, limits and betas change fast.
A runner is **disposable**: every run starts on a fresh machine and leaves nothing behind. Five types fill the slot. They differ mostly in who owns the agent loop and who owns the machine:
| Type | Agent loop | Machine | Also fills |
| --- | --- | --- | --- |
| CI runner | yours, inside the CI job | CI vendor's, or yours | orchestrator, observability platform |
| MicroVM API | yours, in the orchestrator or on the runner | provider's | nothing |
| Bundled agent platform | vendor's | vendor's | part of orchestrator, observability platform |
| Bring-your-own worker | vendor's | yours | part of orchestrator, observability platform |
| Own VPS or containers | yours | yours | nothing |
Runner choice is per run: a cheap CI runner for review runs and a microVM for implementer runs is a normal design.
## CI runners
A fresh VM or pod per CI job, started by the git host's own workflow engine; the agent CLI is one step in the job. The orchestrator starts a run with a workflow trigger (issue, PR, schedule, manual dispatch), and a concurrency group keeps two runs off the same ticket. Hosted runners are disposable by construction; self-hosted runners get there through an ephemeral mode, one pod per job, deleted after. Cost is per minute.
**Pick it when** the git host is GitHub or GitLab and every trigger is a git host event or a clock: the workflow file is the orchestrator and the run page is the observability platform, so one product fills three slots.
Starting points:
- [GitHub Actions](https://docs.github.com/en/actions/reference/limits): the default for an all-GitHub factory; [pricing](https://docs.github.com/en/billing/reference/actions-runner-pricing).
- [Claude Code GitHub Action](https://code.claude.com/docs/en/github-actions): Claude Code as a workflow step.
- [GitLab hosted runners](https://docs.gitlab.com/ci/runners/hosted_runners/linux/): an ephemeral VM per job.
- [Actions Runner Controller](https://docs.github.com/en/actions/concepts/runners/actions-runner-controller): self-hosted, one ephemeral pod per job on the user's Kubernetes.
## MicroVM APIs
An SDK call creates an isolated microVM (vendors say "sandbox") in about a second; the orchestrator execs commands into it, or starts an agent CLI inside it. Cost is per second of vCPU and RAM. The orchestrator creates one per run, kills it at the end, and sets a timeout as the backstop. Providers also sell persistence (pause/resume, snapshots, drives); a factory uses it as a cache, and keeps memory between runs in the ticket tracker, git host and orchestrator.
**Pick it when** the factory has its own orchestrator (a server, a queue) and wants many short-lived, parallel runs with no machines to manage. Compare on max run length, concurrency cap, and whether time spent waiting on the model is billed: as of 2026-10, Vercel bills only active CPU, so model-wait time is nearly free there.
Starting points: [E2B](https://e2b.dev/pricing), [Daytona](https://www.daytona.io/pricing), [Modal Sandboxes](https://modal.com/docs/guide/sandboxes), [Vercel Sandbox](https://vercel.com/docs/vercel-sandbox/pricing).
## Bundled agent platforms
A hosted coding agent with its runner included: the vendor provisions a fresh VM or container per run, runs its own agent loop there, and pushes a branch or PR. Only the branch survives. Runs start from an @-mention or assignment in the git host or ticket tracker, a chat message, a schedule, or an HTTP call, so the platform fills much of the orchestrator; its transcript view fills the observability platform. Cost is folded into a seat or subscription, or charged per run.
**Pick it when** the user wants the fewest moving parts and is happy with that vendor's agent. Confirm its triggers cover the user's ticket tracker.
Starting points:
- [Claude Code cloud](https://code.claude.com/docs/en/claude-code-on-the-web), with [scheduled tasks](https://code.claude.com/docs/en/routines) for cron, API and GitHub event triggers.
- [Claude Managed Agents](https://platform.claude.com/docs/en/managed-agents/overview): API-first; create a run against an environment and stream events.
- [Codex cloud](https://developers.openai.com/codex/cloud): a container per task.
- [Copilot coding agent](https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent): assign an issue to Copilot; its runner is a GitHub Actions runner.
## Bring-your-own worker
The bundled platform keeps its agent loop and its queue of runs, but tool calls execute on a worker the user hosts. The worker polls the vendor outbound, claims a run, executes it, and exits: the same shape as a self-hosted CI runner. Disposability is the user's job: run the worker under Kubernetes or a microVM API so each run gets a fresh disk. Cost is the vendor's seat or usage plus the user's machines.
**Pick it when** the user wants a bundled agent but runs must reach a private network, keep data on the user's machines, or use custom hardware.
Starting points: [Claude Code self-hosted environments](https://code.claude.com/docs/en/self-hosted-environments), [Cursor self-hosted pool](https://cursor.com/docs/cloud-agent/self-hosted-pool), [Copilot coding agent on ARC](https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment).
## Own VPS or containers
A machine the user rents or runs, with a fresh container (or machine) per run: the machine may live for months, the container lives for one run. Max run length is whatever deadline the orchestrator sets. Disposability comes from `docker run --rm`, a Kubernetes Job with `ttlSecondsAfterFinished: 0`, or a machine destroyed at the end; every cache sits on a path the next run treats as optional. Cost is flat monthly, or per second while running.
**Pick it when** runs are long, steady or heavy, cost per hour dominates, or a server already hosts the orchestrator.
Starting points: [Fly Machines](https://docs.fly.io/machines/overview) ([pricing](https://docs.fly.io/about/pricing)), [Hetzner Cloud](https://www.hetzner.com/cloud/) plus Docker, [Kubernetes Jobs](https://kubernetes.io/docs/concepts/workloads/controllers/job/).
## Researching further
### The questions that decide the slot
Check each candidate against these, with the factory's runs in hand:
1. **How does the orchestrator start one?** A workflow trigger, an SDK call, an @-mention, an HTTP call, or a vendor queue that a worker claims. Does that fit the orchestrator the design already names?
2. **What is the max run length, and the concurrency cap?** Find both the default timeout and the ceiling, per plan tier, and what happens at the cap. Hold them against the design's longest run and busiest burst.
3. **What is the cost shape?** Per minute, per second of vCPU and RAM, active CPU only, per run, per seat, or flat monthly. Is time waiting on the model billed? Are model tokens inside the price or on top?
4. **How is disposability achieved?** Fresh by default, an ephemeral flag, or the orchestrator's own teardown. Which persistence features exist, and can the design keep them to caches?
5. **What can a run reach?** Private network access, outbound firewall or allowlist, how secrets reach the runner, and who else can read them.
6. **Which other slots does it fill?** Count the triggers it reads as orchestrator coverage, and its run page or transcript as observability platform coverage.
### Where the answers live
- The vendor's **limits or quotas page** and **pricing page**, over the landing page.
- The **SDK or API reference** for the create call: its timeout, lifecycle and network parameters.
- The **security or networking docs** for firewalls, allowlists and secret handling.
- The **changelog or release notes** for beta, preview and GA status, and for plan gating (Team, Enterprise).
- For CI runners, the git host's **billing docs**.
### What goes stale fastest
- **Prices and free allowances.** As of 2026-10, GitHub had announced a per-minute fee for self-hosted runners, then postponed it.
- **Beta, preview and early-access status**, and which plans unlock a feature. As of 2026-10, Claude Managed Agents and Claude Code self-hosted environments were both in beta.
- **Default timeouts and max run length** per tier.
- **Trigger integrations** on bundled platforms: which ticket trackers and chat tools start a run.
- **Startup times**, which are vendor claims; measure them if they matter.
The type traits above (who owns the loop, who owns the machine, which slots it fills) change slowly; trust them over any number here.
skills/in-progress/software-factory/SKILL.md
+98
View File
@@ -0,0 +1,98 @@
---
name: software-factory
description: Shared language for designing a software factory: the tasks it runs and the services it plugs into.
disable-model-invocation: true
---
A **software factory** is a system of agents that turns tickets, PRs and schedules into merged code, deployed on remote infrastructure the user chooses. This skill is the language for designing one. Use these terms, and only these, when discussing or whiteboarding a factory; when the user reaches for a word under _Avoid_, name the canonical term.
## Tasks and runs
**Task**:
A kind of work the factory performs, defined once and run many times: review, explore, implement, ratchet. A task may work on a ticket, a PR, or the codebase as a whole.
_Avoid_: job, workflow, action
**Trigger**:
The event that starts a run: a change in the ticket tracker or git host, a clock in the orchestrator, or another run.
**Run**:
One execution of a task on one runner, from its trigger to its finish, treated as a black box. A run triggers further runs either directly, through logic in the orchestrator, or by changing the ticket tracker or git host. A run never waits for a human: it ends, and the human's later change to the ticket tracker or git host triggers a new run.
_Avoid_: session
**Runner**:
The disposable machine a run executes on. Nothing on it survives the run, even when the machine itself is long-lived: a run's only memory of earlier runs is the ticket tracker, the git host and the orchestrator.
_Avoid_: server, host, box, compute, sandbox
## Services
**Ticket tracker**:
Where tickets live, and the factory's human interface: humans schedule work by filing tickets, talk to agents in their comments, and their changes to tickets trigger runs.
_Avoid_: issue tracker, ticket manager, project board
**Ticket source**:
Anything outside the factory that creates tickets in the ticket tracker: a feedback button, a monitor, an error tracker, an inbox.
_Avoid_: data source, trigger source, intake
**Git host**:
Where branches, PRs and reviews live.
_Avoid_: code host, forge, repo
**Orchestrator**:
The component that receives triggers, starts runs on runners, and records which runs are queued, in flight or finished.
_Avoid_: dispatcher, scheduler, control plane
**Observability platform**:
Where runners send logs, traces and status, so a human can see what happened inside each run. Optional: the product filling the runner or orchestrator slot often shows enough on its own. A run never decides on it until something writes it into the ticket tracker or git host.
_Avoid_: observability, logs, monitoring, dashboard
## Designing a factory
A design has two halves: the **services**, each a slot filled with a product, and the **tasks** whose runs move work through them.
Start from the user's problem, not the factory: untriaged tickets, PRs waiting for automated review, a stricter automated check to migrate to, bugs nobody reads. When the user names a problem or doesn't know what to build, read [EXAMPLE-TASKS.md](EXAMPLE-TASKS.md) for the tasks that solve common ones.
### Start slow
Every trigger is an accelerator. Accelerators without brakes make a slop cannon. There are three brakes:
- **Automated checks**: deterministic gates (types, lint, tests) that every run must pass.
- **Automated review**: a separate run that reads the diff cold against the codebase's standards and the ticket, when there is one.
- **Human review**: a human reads the PR before it merges; what it catches becomes new automated checks and sharper automated review.
Add one automated trigger at a time, throttled at first (one bug report in ten, a clock once a day), with human review on every PR. Open the throttle as the brakes earn trust, and cap every loop of runs at a few tries before a human.
### Fill every slot
Each service is a slot. One product can fill several slots, and one server can host several services. Three designs, side by side:
| Service | All on GitHub | Linear tickets | Own tracker |
| ---------------------- | ------------------------------- | ------------------------------ | ------------------------ |
| Ticket tracker | GitHub Issues | Linear | own app |
| Ticket source | feedback button | feedback button, monitor | feedback button, monitor |
| Git host | GitHub | GitHub | GitHub |
| Orchestrator | GitHub Actions | own server + database | own app |
| Runner | Actions runner | Actions, E2B, Daytona or a VPS | own app, or sandboxes |
| Observability platform | none: Actions run pages suffice | an LLM tracing platform | own app |
When the user is choosing a product for a slot, read that slot's survey to get oriented, then research the current options yourself; each survey groups the options by type and lists the questions that decide the slot:
- [TICKET-TRACKERS.md](TICKET-TRACKERS.md)
- [GIT-HOSTS.md](GIT-HOSTS.md)
- [RUNNERS.md](RUNNERS.md)
- [OBSERVABILITY-PLATFORMS.md](OBSERVABILITY-PLATFORMS.md)
Buy the services and build the tasks: the tasks carry the user's design, the services are generic. Favour an off-the-shelf ticket tracker: it is robust, people already know it, and outside users are easy to let in.
### Question every task
For each task, answer:
1. What triggers it?
2. What does it do, and which services does it read and write?
3. What does it trigger next?
4. Where does a human re-enter?
5. What stops two runs doing the same work?
For each ticket source, answer: what stops it filing the same ticket twice?
A design is finished when every slot names a product (or, for the observability platform, a deliberate none) and every task and ticket source has an answer to every question.
skills/in-progress/software-factory/TICKET-TRACKERS.md
+85
View File
@@ -0,0 +1,85 @@
# Ticket trackers
> Surveyed 2026-10, and not guaranteed to be up to date. Use it to get oriented, then research the current options yourself: products, prices, limits and betas change fast.
The ticket tracker is the factory's human interface. Each type below is a starting point: read what it is and when a factory picks it, then check the examples against the questions in [Researching further](#researching-further).
## The types
### 1. Git-host tracker
The tracker built into the git host, so tickets, branches and PRs share one product, one permission model and one identity. A factory picks it when it is small, the team already lives on the git host, or the repo is public. This is the "All on GitHub" design: the git host's CI fills the orchestrator slot with no glue. Labels are the dependable router, because statuses live on a separate board. Outside developers file where the code is; non-developers need an account, so a feedback button files through the API. Self-hosted git hosts carry their tracker with them.
- [GitHub Issues](https://docs.github.com/en/copilot/concepts/agents/about-third-party-agents): the default; assign-to-agent for Copilot, Claude and Codex.
- [GitLab Issues](https://about.gitlab.com/blog/how-to-use-a-work-item-created-trigger/): Duo Agent Platform flows triggered from work items.
- [Forgejo](https://forgejo.org/2026-10-release-v17-0/): self-hosted, Actions-compatible.
To verify: which assign-to-agent features are still in preview and which plans include them; whether a native agent reads later comments or only the ticket at assignment (as of 2026-10, Copilot reads it at assignment only).
### 2. Agent-native dev tracker
A standalone SaaS tracker for software teams that treats agents as first-class assignees, with an agent API and its own coding agent. A factory picks it when product, support or design staff schedule work, the team already lives there, or it wants polished triage and planning views. The "Linear tickets" design. Statuses and workflows are first-class, so "moved to Ready" is a clean trigger. Agents appear as app users a human delegates to, and runs can stream progress into the ticket. Native agents here are the deepest of any type, so one can partly fill the orchestrator and runner slots too.
- [Linear](https://linear.app/developers/agents): the most agent-shaped tracker.
- [Jira](https://support.atlassian.com/jira-software-cloud/docs/collaborate-on-work-items-with-ai-agents/): status-triggered agents; fits an org already on Atlassian.
- [Shortcut](https://www.shortcut.com/help/integrations/using-ai-agents): assign a story or mention an agent in a comment.
- [YouTrack](https://www.jetbrains.com/youtrack/): workflows-as-code; self-hosts.
To verify: agent API status (Linear's was a developer preview as of 2026-10); how native agent runs are metered and priced; agent seat cost; acknowledgement deadlines on agent webhooks; which plan unlocks customer-facing forms; self-hosted edition end-of-life dates.
### 3. Open-source dev tracker
A Linear-style tracker you can run on your own infrastructure, with a cloud edition beside it. A factory picks it when data stays in-house (air-gapped, regulated) or the user wants to read and patch the tracker's source while still buying it. Routing by states, labels and custom properties works much as in type 2; native agents are younger.
- [Plane](https://developers.plane.so/dev-tools/agents/overview): Linear-like; bot users per app; agents reply in comments.
- [Huly](https://github.com/hcengineering/huly-selfhost): all-in-one workspace with two-way GitHub sync; a heavy stack to operate.
- [OpenProject](https://www.openproject.org/): mature, classic project management.
To verify: agent feature status (Plane's was beta as of 2026-10); which edition carries guest roles and intake forms; licence; the self-hosted stack's operating cost.
### 4. General work tool
A work-management or docs tool whose database or task list doubles as the tracker. A factory picks it when it serves a non-engineering team already living there, or tickets are mostly content, ops or research rather than code. Forms are the strength: anyone with a link files a ticket. Routing is flexible database properties, so every factory invents its own schema. Developer context (diffs, PRs) sits a click away on the git host. SaaS only.
- [Notion](https://www.notion.com/help/custom-agents): database-as-tracker; agents fire on database changes.
- [Asana](https://asana.com/product/ai/ai-teammates): AI Teammates take assigned tasks.
- [ClickUp](https://clickup.com/brain/cloud-agents): Cloud Agents return a PR in the task thread.
- [Trello](https://developer.atlassian.com/cloud/trello/guides/rest-api/webhooks/): boards and lists; webhooks plus Butler automation.
To verify: webhook latency and granularity (as of 2026-10, Notion batched content updates, arriving a minute or two late); which plan includes agents.
### 5. Repo-native tracker
Tickets stored as files in the repo (JSONL, Markdown), edited by agents through a CLI or MCP server and merged like code. A factory picks it when one developer drives agents locally and wants the plan in git, or as a run's working memory beside a real tracker. A push to the git host is the only trigger, so the orchestrator diffs files to find what changed. Identity is the git host's. Outside filing is a commit or PR, the weakest of any type, so a factory graduates to a git-host tracker once others file tickets.
- [Beads](https://github.com/steveyegge/beads): JSONL with dependency tracking, so a run picks the next unblocked ticket.
- [Backlog.md](https://github.com/MrLesk/Backlog.md): Markdown tasks with a Kanban view and MCP server.
### 6. Build-your-own
The factory's own app holds tickets in its own database, usually beside the orchestrator. The "Own tracker" design. A factory picks it when the tracker *is* the product (an internal platform, a customer-facing support factory) or no off-the-shelf model fits. Triggers become function calls; every trait in the questions below becomes code to write and maintain, which is why the skill favours buying. An [Airtable](https://airtable.com/developers/web/api/webhooks-overview) base with webhooks is a half-built middle.
## Researching further
Check each candidate against these questions. They decide the slot:
- **Triggers**: which ticket changes (created, labelled, status moved, assigned, commented, mentioned) emit a webhook or event the orchestrator can turn into a run? How fast and how granular?
- **Routing**: which labels, statuses or fields can a run read to know its kind (explore, implement, rework)?
- **Thread**: can a human answer an agent in comments, and does that reply trigger the next run?
- **Identity**: is there an API plus a bot identity, so runs act as themselves and humans tell agent from colleague? Does the bot cost a seat?
- **Native agents**: can a ticket be assigned to an agent directly? Which agents, on which plan, metered how? What does it read: the ticket at assignment, or the whole thread?
- **Outside filing**: how does a ticket source or customer create tickets: forms, email, chat, API? Which plan unlocks it?
- **Hosting**: SaaS only, or self-hosted, and under which licence?
Answers live in primary sources: the API and webhook reference (event lists, payloads, delivery and retry rules), the agent or app developer docs, the pricing page and plan comparison table, and the changelog or release notes. Prefer these over blog posts and comparison sites.
Stalest first: agent features and their preview or beta status, agent metering and credit prices, plan gating of agents and forms, rate limits, and acquisitions, shutdowns and end-of-life dates for self-hosted editions. Webhook event lists and routing models change slowly.
## Durable gotchas
Verify each still holds before relying on it:
- **GitHub's `GITHUB_TOKEN` trap**: changes made with the default `GITHUB_TOKEN` trigger no new workflow runs. A run that labels or comments on a ticket to trigger the next run acts through a GitHub App token.
- **Assignment-time context**: some native agents read the ticket only when assigned, so later context goes on the PR.
- **Agent webhooks expect a fast acknowledgement**: the orchestrator acknowledges at once and starts the run afterwards.
- **Batched webhooks in general work tools** arrive late; a factory needing prompt runs triggers on property changes or picks another type.