Skip to content
Ludmal De SilvaAI Engineer

open-factory: a software factory for AI coding agents

7 min read

open-factory is an Agent Skill that takes a ticket to a merged pull request. Agents analyse the work, write an OpenSpec spec, build it in parallel, test it against hidden scenarios, review it and merge it. This post explains how it works and how to control it.

I have been working on a simple idea. Give a coding agent one line of instructions, and get back a merged pull request that has been specified, built, tested and reviewed.

/factory "Add CSV export to the invoices page"

The result is open-factory. It is a single Agent Skill made of an orchestrator, six agent roles and one configuration file.

It is a proof of concept, built on Claude Code. Codex, Gemini CLI, GitHub Copilot and Cursor read the same skill format, so they are supported, but I have not tested them yet.

What is a software factory?

A software factory is a pipeline of AI agents that takes a spec to merged code. Other agents and automated tests check the work at every step.

StrongDM runs one where no human writes or reviews the code. This is called a dark factory. Dan Shapiro's five levels of AI coding place it at level 5, the highest level.

open-factory also runs as a dark factory by default, with one important difference: it stops at the merge. It never deploys to production.

How it works

open-factory flow: factory run, Analyse, Spec, Develop with several developers, then Verify and Review in parallel, then Merge to main. Failures go back to Develop. The factory stops before production.

Your main agent session acts as the orchestrator. It does not write code. It starts a new agent for each step, reads the result and moves the work forward.

All the state is kept in GitHub:

  • Labels: each issue has one factory:<stage> label that shows where it is.
  • Comments: each step leaves a short comment with its result.
  • Spec: the OpenSpec change lives on the change branch.

If a run stops halfway, the next /factory run continues from where it stopped.

The steps

Step Agent What it does
Analyse Analyst Reads the issue and the code. Decides if the work is ready, needs more information, should be split, or should be rejected. Flags sensitive areas such as authentication, payments and migrations.
Spec Spec writer Writes an OpenSpec change: a proposal, requirements with WHEN/THEN scenarios, a design, and tasks grouped into work packages. Also writes 3 to 8 hidden scenarios.
Develop Developers One developer per work package, each in its own git worktree. Tests first, then code. One pull request per issue.
Verify Verifier Runs every check, starts the app, and tests every public and hidden scenario.
Review Reviewer Reads the change against the spec. Checks coverage, bugs, security, data safety and any weakened checks.
Merge Release Merges one pull request at a time. Updates the branch, archives the spec, waits for CI, merges and closes the issue.

Verify and review run in parallel on the same commit. If either one fails, a developer gets the findings and fixes them. By default this can happen up to two times. After that, the issue is labelled factory:blocked and waits for a person.

Why you can trust the result

Agents checking other agents only works if the checks are independent. open-factory uses three rules for this.

1. Every checker starts fresh. The verifier and the reviewer did not write the code, and they never see the developers' notes. They judge the code, the spec and the running app.

2. Hidden scenarios. The spec writer creates 3 to 8 extra test scenarios and stores them outside the code the developers work on. Only the verifier runs them. When one fails, the developer is told which requirement failed and what happened, but not the scenario itself. This makes it hard to write code that only passes the known tests. The scenarios are hidden by convention, not by a sandbox, which is enough for a proof of concept.

3. No agent can weaken its own checks. Agents may not delete or skip tests, relax lint or type settings, or change CI or factory.yaml to make a build pass. The reviewer looks for this and blocks it. Changes to the factory's own settings are always merged by a person.

This is what one fix round looks like:

sequenceDiagram
    participant O as Orchestrator
    participant D as Developer
    participant V as Verifier
    participant R as Reviewer
    par Same commit
        O->>V: PR and hidden scenarios
    and
        O->>R: PR, no developer notes
    end
    V-->>O: Hidden scenario H2 fails
    R-->>O: One blocking finding
    O->>D: The finding, and the requirement H2 broke
    D-->>O: Fix pushed
    O->>V: Verify again
    O->>R: Review again
    V-->>O: Pass
    R-->>O: Pass

How it compares

quadrantChart
    title Automation and human review
    x-axis Humans do more --> Agents do more
    y-axis No human review --> Human review
    quadrant-1 Agents build, humans approve
    quadrant-2 Human-led
    quadrant-3 Unreviewed
    quadrant-4 Dark factory
    Anthropic playbook: [0.38, 0.88]
    Stripe Minions: [0.62, 0.74]
    open-factory: [0.76, 0.12]
    StrongDM: [0.92, 0.08]
open-factory (default) Anthropic's playbook Stripe Minions StrongDM
Spec Written by an agent Agent drafts, product owner approves Engineer gives the task Written by people
Hidden tests Yes Not described Not described Yes
Code review Independent agent Code owner approves Engineer reviews every PR None
Merge Agent Person Person Agent
Production Never deploys Agent cannot pass the production gate Not described Not described

Anthropic's AI-native SDLC playbook keeps people in charge of the spec and the merge. The agent can work up to the production gate, but it cannot pass it. Stripe's Minions merge more than a thousand pull requests a week, and an engineer reviews every one. StrongDM removed human review completely. open-factory is close to StrongDM by default, but it stops at the merge.

open-factory takes four ideas from these approaches:

  • Keep the agents that build the code separate from the agents that check it.
  • Limit the number of retries, then hand over to a person.
  • Never let an agent weaken its own checks.
  • Keep merging separate from releasing.

Turning off the dark factory

By default, no person reads the spec or the code before it is merged. If you want more control, change two settings in factory.yaml:

policy:
  merge: human           # stop at a ready pull request; you merge it
  spec_approval: always  # stop after the spec until you approve it

These safety controls also work in the default mode:

  • Hold categories: hold_for_human stops selected changes before the merge, for example [destructive-migration, infrastructure, payments, auth]. A revert does not bring back deleted data.
  • Circuit breaker: if more than two factory pull requests are reverted in seven days, merging switches to manual.
  • Stop label: add factory:stop to an issue to stop it before the next step.
  • Target branch: if every push to main deploys to production, set git.target_branch to a branch that does not deploy.

How to try it

You need:

  • an AI coding tool that supports Agent Skills, ideally with subagents
  • git
  • the GitHub CLI, signed in with write access to the repo
  • the OpenSpec CLI: npm i -g @fission-ai/openspec

In Claude Code:

/plugin marketplace add ludmal/open-factory
/plugin install open-factory@open-factory
/factory init

init creates factory.yaml with your lint, test and build commands, sets up OpenSpec and the labels, and opens a setup pull request. You merge that one yourself. After that, add the factory:ready label to an issue and run /factory run, or give it an instruction directly.

For other tools, copy skills/factory/ to the folder your tool reads skills from. The README lists the paths.

Two things to keep in mind:

  • Cost: each step is a new agent, and several developer agents can run at the same time, so check your usage on the first runs.
  • Permissions: git worktrees are not sandboxes. The agents run with your shell's permissions and credentials.

You can watch the agents while the factory runs with agents-flow.

Summary

A software factory is only as good as its checks. open-factory keeps the agents that build the code separate from the agents that check them, hides some tests from the builders, and never lets an agent weaken its own checks. The code is on GitHub, and feedback is welcome.

ShareXLinkedIn