The software factory pyramid explained
Seven layers of an AI software factory, from versioned rules and working context to models, testing, orchestration and the ways people request work.
A software factory is a repeatable system for turning an agreed request into software that has been built, tested and reviewed. AI agents are programs that use models and tools to carry out parts of that work. The factory supplies their instructions, working environment, checks and handoffs.
The pyramid groups that system into seven responsibilities. Read from the foundation upwards, from the files that define the setup to the ways people request work. It is a practical architecture map rather than an industry standard or a maturity score. One product may cover several layers, and work does not travel through them in numerical order.
00. Factories-as-code
Factories-as-code means keeping the repeatable setup in versioned files. Those files can define instructions, available tools, workflow steps, test commands and the environments where work runs. A version history lets a team inspect a change, compare it with the previous setup and restore an earlier version when needed.
For example, a saved workflow can build the application and run its tests whenever someone proposes a code change. The checks belong to the project rather than relying on a person remembering a checklist. Versioning the setup does not make AI produce the same result every time or guarantee that the checks are sufficient.
01. Data and context
Context is the information an AI model works from while completing a task. It can include the request, relevant code, project instructions, documents, earlier decisions and test results. Stored information only helps when the system retrieves the useful parts and supplies them at the right time.
For a broken sign-in page, useful context could include the reported error, the sign-in code and the expected behaviour. A large folder of unrelated documents can add noise. Access controls should restrict the information each task can read, and saved notes should retain enough detail for a later worker to understand the decision.
02. Compute
The compute layer provides the working environment where tools edit files, run commands, start the application and execute tests. A sandbox is an isolated environment with limits on what those actions can reach. Its permissions, network access and resource limits still need configuring.
In the sign-in example, the workspace holds a copy of the code and runs the proposed fix against the tests. Separating that work from the live application reduces direct exposure to experimental changes. Isolation is one control, not a guarantee that every permitted action is safe.
- Vercel documents isolated environments for running generated code
- Read our explanation of AI sandboxes
03. Inference
Inference means running a trained AI model on an input to produce an output. Inside a software factory, that output might be a plan, a code change, a review comment or a proposed tool action. Choosing a model involves trade-offs in capability, response time and cost.
The model might read the failing sign-in test and propose an edit. The compute layer then runs the tool that applies or tests that edit. Model inference itself also needs computing resources, often hosted elsewhere. The pyramid separates the responsibilities of model execution and the workspace that runs tools.
04. Improvement
Improvement uses evidence from completed work to test whether a change to the factory helps. Useful evidence includes task outcomes, failed tests, recorded tool actions, time and cost. An evaluation is a repeatable test for an AI system, with defined inputs and a way to judge the result.
Suppose the same instruction repeatedly causes a test to be skipped. A team could revise it, replay representative tasks and compare the results with the previous setup. It should also check for regressions, where existing behaviour becomes worse. Better scores on a small task set do not prove reliability in every real situation.
This layer feeds back across the system. It can change instructions, information selection, tools or model choice. It does not imply that the model automatically retrains itself, or that proposed changes should be adopted without review.
- Anthropic explains evaluations, recorded runs and outcome checks
- Read our explanation of software factory improvement loops
05. Orchestration
Orchestration coordinates the work. It decides which step runs next, which tools are available, how results pass between steps and when a person must decide. A factory can use a fixed workflow, one agent or several agents. More agents bring extra coordination work as well as possible benefits.
A sign-in repair might move through an agreed requirement, implementation, tests and review. A failed test returns the change for another attempt. Limits on retries, time and spending stop an unsuccessful attempt continuing indefinitely. Passing tests alone does not grant permission to release the change.
06. Access
Access is the layer through which people and other systems request work, provide direction and inspect results. Entry points can include chat, a task ticket, an application or an API, a way for one program to talk to another.
A person might report the broken sign-in through a ticket and later review the proposed fix in a different interface. The interface should make the task, status, evidence and required decisions clear. A convenient entry point does not remove the controls applied further down the system.
How the layers work together
- A person submits the sign-in problem through an approved entry point.
- The workflow selects the task instructions, relevant code and permitted tools.
- A model proposes a change, and tools apply it in the working environment.
- Tests check the change. Failures send useful evidence back for another attempt.
- The required review and release checks decide whether the change can go live.
- Recorded results help the team test later improvements to the factory.
Permissions, budgets, recorded actions and human approval apply throughout. The seven layer numbers are labels for responsibilities. They are not a requirement to buy seven tools or an instruction to build every capability before doing useful work.
We would start with a small, repeatable task and strengthen whichever part prevents a dependable result. Missing instructions need clearer context. Unsafe tool access needs better restrictions. Repeated failures need evidence and a tested change to the process. A more capable model can help some problems, but it cannot supply a requirement that nobody recorded.
The same approach informs our consulting work. Understand the intended outcome, the current workflow and where it falls short before choosing the technology.