Is AI-generated code safe to ship?

The honest answer is: not by default, and not without someone qualified reading it.

That's an unhelpful answer on its own, so here is the useful version — the specific ways AI-generated code fails, why those failures are harder to catch than ordinary bugs, and what actually works as a control.

Why this is a different problem from ordinary bad code

A junior developer writing beyond their level produces code that looks uncertain. Variable names are vague, structure is muddled, and the reviewer's instinct fires immediately.

AI-generated code has the opposite property. It is fluent. It is idiomatic. It has clean names, sensible structure, and often a helpful comment explaining what it does. It reads like code written by someone competent and confident — including when it is wrong.

The danger isn't that the code looks bad. It's that it looks finished.

This inverts the usual review economics. Reviewers are trained to slow down when something looks messy. Nothing here looks messy, so the natural review speed is far too fast for the actual risk.

The six failure modes we actually see

1. Plausible-but-wrong business logic

The most common and the most dangerous. Ask for a discount calculation and you'll get one that is correct for the obvious case and subtly wrong at a boundary — the day a subscription renews, the moment a currency rounds, the second item in a bulk order.

It compiles. It passes the tests, because the tests were generated from the same misunderstanding. It ships. It's found in month four by a customer doing their accounts.

2. Authorisation that checks the wrong thing

Generated endpoints reliably check that a user is authenticated. They much less reliably check that the user is allowed to touch this particular record. The result is the single most common serious vulnerability in modern web applications, and AI produces it enthusiastically.

// Generated: checks WHO you are, never WHETHER it's yours
export async function GET(req, { params }) {
  const user = await requireSession(req);   // authenticated ✓
  return json(await db.invoice.find(params.id));  // authorised ✗
}

Any authenticated user can read any invoice by changing a number in the URL. The code is clean, typed, and completely wrong.

3. Tests that assert the implementation

If you ask a model to write tests for code it just wrote, it will write tests that pass. They describe what the code does, not what the requirement was. Coverage goes up, confidence goes up, and nothing has actually been verified.

This is worse than having no tests, because it produces a false signal that survives review.

4. Dependency invention and typosquat exposure

Models occasionally import packages that don't exist, or that exist but aren't the one you meant. The first case fails loudly and is harmless. The second is the problem — and attackers have caught on, publishing packages under names models are known to hallucinate.

5. Silent behaviour changes during refactors

Ask for a refactor and you may get one that is cleaner, better named, and behaviourally different in one edge case nobody mentioned. Diffs on refactors look boring, so they get skimmed. This is where regressions hide.

6. Performance that only fails at scale

Queries inside loops, missing indexes, loading a full collection to count it. All correct. All fine on a development machine with forty rows. All catastrophic at forty thousand.

What actually works as a control

Not 'we review everything carefully'. Everyone says that, and attention degrades over a long day of fluent-looking diffs. The controls have to be structural.

Specify acceptance criteria before generating anything

If the requirement is written down as checkable statements first, tests can be written against the requirement rather than the implementation. This single practice removes failure modes 1 and 3, which are the two that reach production most often.

Run an adversarial machine review before human review

A second pass whose only job is to look for injection paths, missing authorisation, unsafe dependencies, N+1 queries and behaviour changes — tuned to be pessimistic, and run before a human sees the diff. It catches the mechanical failures cheaply and leaves humans reviewing the things that need judgment.

Make a named senior engineer accountable for the merge

Not a rota, not a rubber stamp. A specific person who has shipped production systems for years and whose name is on the merge. Accountability changes review behaviour in a way that process documents do not.

Review refactors harder than features

Counterintuitive but correct. Feature diffs get attention because they're interesting. Refactor diffs get skimmed because they're meant to be no-ops — which is precisely why a behaviour change hidden in one is likely to survive.

Pin and scan dependencies in CI

Lockfiles, an allowlist for new packages, and automated vulnerability scanning on every build. Cheap, boring, and closes the supply-chain path entirely.

What to ask a vendor

• Who specifically reviews generated code before it reaches main, and how long have they been shipping production systems?
• Are acceptance criteria written before code is generated, or are tests written afterwards?
• What automated checks run on every diff, and do they run before or after human review?
• How do you handle dependency additions?
• Is our code submitted to any model for training?

A team using AI seriously will answer these immediately, because they've had to solve them. A team that added AI to the marketing page will not.