How I run AI coding agents so a business can trust what they build
How it works
The loop every build ran on: agents draft and build, a person decides at every consequential step.
Rulebook
Each session starts by reading the project's rules, the decisions reserved for a person, and every past incident.
Frozen spec
Work is opened from a written spec that's approved before any code.
A person decides
Agents build
The agents draft and build in small phases. They're told to refuse rather than guess.
Rules become checks
Where possible a rule turns into a test, a gate or a database rule.
Check reality
Look at what production serves and query the database, whatever the agent reports.
Sign-off
Every release, deploy, spend and user-facing word is approved per action.
A person decides
The outcome
Five systems built with AI coding agents, each shipped with automated tests, a release gate and my sign-off. Two security defects were caught before any real data existed, a pre-launch review found six defects that were fixed the same day, and an ask-before-any-paid-call rule went in after an agent made 89 unapproved calls.
The problem
AI agents are fast and capable, and they fail in predictable ways. They take a loose instruction as permission, report success they haven't checked, spend money nobody approved, and copy a pattern because it looks right.
What I decided: the method
- A written rulebook per project, read at the start of every session: what the product is for, the rules it must never break, which decisions belong to a person, and each past incident with the rule it produced.
- Human approval for anything consequential: database and security changes, user-facing wording, moving to the next phase, deploys, ad changes and any spending. Approval is per action. "Yes, launch it" doesn't cover whatever comes next.
- Specs frozen before code. Work moves in small phases, each opened from a written spec and closed at a tagged version after testing.
- "Refuse rather than guess." Agents are told a correction is worth more than compliance. On one build, those stops caught errors in the spec and two real security defects.
- Rules become checks. A rule that lives only in a document gets forgotten, so where possible it becomes a test, a gate or a database rule, such as a release gate that fails if private content leaks into the public code.
- Check reality. Look at what production serves, query the database and render the page, whatever the agent reports.
- A separate review before launch, by a different session, before real money or real users touch the build.
The incidents behind the rules
- 89 unapproved paid calls. I told an agent "I'll leave it to you how you do this", meaning the creative direction. It made 89 paid API calls to rewrite a programme, mostly resending the same policy document, and logged no cost. New rules: ask before any paid call outside the product's standing path, say what it will cost, and log the usage of every paid script.
- The deploy that didn't happen. An agent read the last two lines of a deploy command, saw what looked like success and reported it done. Nothing had deployed. New rule: read the whole output and confirm against what production serves.
- The silent payment failure. A payment event failed silently and went unrecorded. New rule: any failed or stuck payment event alerts me within a minute.
- Green tests, broken product. Every database test passed while two functions couldn't be reached from the app. New rule: test through the real interface, as a real user.
Results
Five systems built with this method:
- an AI sales-research pipeline: about USD 0.11 a run, 290 of 290 stored facts sourced, 20 of 20 planted errors caught, 273 tests;
- a sales, delivery and lifecycle-automation platform: about 1,900 tests and 66 database test suites in one release gate;
- a 57-table execution-monitoring SaaS: about 10 weeks against a 14 to 21 week plan, about 2,000 tests;
- a programme platform for sensitive health-related data: access control on all 71 tables, two security defects caught before launch;
- the company website: Lighthouse 100 on desktop, 94 to 96 on mobile.
What I'd do next, and bring to a team
Start with one painful, measurable workflow. Set the rules and spending limits first, build in small steps behind approval gates, build a labelled test set before trusting output, and turn every incident into a check, so agents can do real work without anyone losing control.
Screens from the build
