How to Build an AI Software Factory: Architectural Blueprint for Autonomous Pull Request Orchestration

On January 6, 2026, .NET runtime core maintainer Stephen Toub opened nine pull requests from his smartphone while sitting at 35,000 feet on a commercial flight. Seven of those pull requests merged cleanly into production. The milestone captured a fundamental structural inversion in modern software development: the economics of code production have permanently decoupled from human typing speed. When a single engineer equipped with a background coding agent can saturate an engineering organization's code review capacity from an airplane seat, the critical operational question stops being how to make models write code, and becomes how to build systems capable of absorbing the output.

An AI software factory is the architectural pattern enterprise engineering teams have converged on to solve this absorption bottleneck. Distinct from running an interactive coding assistant in an editor window—where a developer manually prompts, inspects, and reviews changes—a software factory operates unattended infrastructure. Work enters from a queue, executes inside isolated disposable workspaces, undergoes rigorous multi-stage automated verification, and surfaces to engineers only at an explicit merge gate where human accountability is required.

Cross-referencing architectures deployed across technology organizations reveals a shared five-stage skeleton:

1. 1.Intake and Deduplication: Deciding What Deserves Compute

Unfiltered software factories quickly descend into compute waste by spawning agents on poorly scoped bugs. Leading implementations enforce strict intake filters. Sentry's Seer evaluates incoming issue stacks for actionability before triggering an agent. Shopify routes requests through public Slack channels rather than private direct messages, ensuring all agent interventions remain searchable across the organization. Furthermore, Microsoft telemetry on the .NET runtime demonstrates that agent PRs modifying 1 to 50 lines achieve a 76% to 80% success rate, whereas complex architectural refactors land at only 54.5%. Successful intake pipelines filter by scope, verify reproducibility, and query external dependency indices to confirm an issue has not already been resolved upstream.

1. 1.Isolation Models: Preventing Workspace Collisions

Allowing concurrent agents to modify a shared repository environment introduces immediate file locking and dependency corruption. Factories tier isolation into three distinct operational models:

• Git Worktrees: Lightest-weight isolation, allocating separate working directories on dedicated local branches while sharing one repository cache. Tools like Claude Code manage worktrees via project configuration files to ensure environment files and secrets are mapped cleanly.
• Containerized Sandboxes: Essential when agents require conflicting system dependencies or execute untrusted shell scripts.
• Pre-Warmed Cloud Devboxes: Deployed at enterprise scale, platforms like Stripe's Minions provision dedicated virtual machines in under ten seconds, and Ramp utilizes ephemeral filesystem snapshots to support unlimited parallel execution.

1. 1.Tools and Company Context: Giving Agents Hands

A model restricted to repository text operates as advanced autocompletion; a model connected to an organization's test runners, telemetry pipelines, feature flags, and deploy tooling functions as a colleague. Stripe's Toolshed exposes approximately 500 internal utilities behind a unified Model Context Protocol (MCP) server with role-based access controls. Similarly, teams are consolidating cross-functional documentation into company-wide monorepos, allowing autonomous agents to reference marketing, compliance, and product specifications alongside source code.

1. 1.Nested Verification Loops: The Confidence-Score Trap

Verification is the stage where software factories succeed or fail. Spotify structures this as nested loops: an inner loop executing deterministic local linting and compilation in under five seconds; an intermediate LLM judge that audits whether the diff adhered strictly to task scope (vetoing roughly 25% of unaligned sessions); and an outer continuous integration run. Crucially, engineering audits from Faire disproved the common assumption that filtering agent suggestions by self-reported model confidence scores improves code review quality. Self-reported confidence proved uninformative; what actually elevated comment acceptance to 73% was deploying an independent secondary model tasked solely with asking whether a code comment was worth an engineer's time.

1. 1.The Merge Gate: Governance by Blast Radius

An agent merge policy determines whether an automated factory remains an asset or becomes technical debt. Mature organizations categorize changes into explicit risk tiers: mechanical dependency updates auto-merge upon passing comprehensive automated suites; scoped bug fixes require one human review; non-trivial feature changes mandate two human approvals; and security-sensitive domains (authentication, payments, data retention) require explicit sign-off from named domain owners.

While software factories unlock unprecedented delivery velocity, real-world data highlights clear trade-offs. Microsoft's ten-month longitudinal analysis of Copilot coding agents on the .NET runtime revealed that agent-authored pull requests merged 67.9% of the time, compared to 87.1% for human engineers. When engineers collaborated directly on an agent's branch, merge rates rebounded to 86.2%, whereas fully autonomous unattended runs landed at 55.1%. Notably, revert rates remained identical (0.6% for agents versus 0.8% for humans), demonstrating that changes that clear rigorous verification are structurally sound.

However, high generation volume carries a measurable maintenance burden. Longitudinal studies by GitClear analyzing 211 million lines of code found duplicated blocks increased eightfold, while code refactoring rates dropped by half. DORA research similarly identifies a transient J-curve dip in initial delivery stability, attributed to the verification tax of human teams adjusting to review overload.

The strategic lesson is decisive: the competitive moat in software engineering has shifted from the volume of code written to the operational rigor of the factory that produces and verifies it.

• How to Build an AI Software Factory: Agents That Open, Review, and Merge PRs — Firecrawl Engineering
• Ten Months with Copilot Coding Agent in dotnet/runtime — Microsoft Developer Blog / Stephen Toub
• Under the River: How Shopify Ships Multi-Agent Coding Fleets — Shopify Engineering
• Running a Software Factory Efficiently at Scale — Uber Engineering Blog