AI-written code: the checks reviewers skip

Ask an engineering leader what share of their production code was written or substantially modified by an assistant in the last quarter and the honest answers now range from a third to most of it. Ask their security lead what changed in review as a result and the honest answer is usually nothing. Generated code is not worse than human code. It fails differently, and review processes designed for human failure modes do not see it.

How generated code fails

Human-written insecure code tends to look wrong: a hurried patch, a copied snippet, an inconsistent style that draws a reviewer's eye. Generated code looks right. It is idiomatic, consistently named, well commented, and complete in every respect except the one the prompt did not mention. The prompt said "add an endpoint to fetch an order by ID". It did not say "only if the caller owns the order", because the author assumed that was obvious. The model did exactly what it was asked.

Five checks that go missing

• →Ownership. The record is fetched by ID; the caller's relationship to it is never verified. This is the single most common generated-code finding we see, and it is a full account takeover in a multi-tenant product.
• →Bounds. Pagination without a maximum page size, uploads without a size limit, loops over caller-supplied lists. Each is a spend or availability problem at scale.
• →Rate. A new endpoint inherits no rate limit because the limit was configured per route by hand, elsewhere, by someone else.
• →Secrets. Credentials that were supposed to be injected end up in a default parameter, a test fixture, or a log line, because the model needed a working example.
• →Trust. Content from a user, a webhook or a document is treated as data by the human who wrote the prompt and as instruction by the agent that consumes it.

The agent did what it was told. The problem was who got to tell it.From the MATT record, on the Black Hat 2026 coding-agent findings

Why review does not catch it

Reviewers check what is present. A missing check has no diff line to comment on. Static analysis has the same blind spot: it flags dangerous patterns, and the absence of an authorisation call is not a pattern. The only reliable way to find a missing check is to test the behaviour of the deployed product as an outsider: fetch someone else's order, page through ten million rows, send the webhook the agent will act on.

What to do about it

• →Write the assumptions down as rules. "A user must never act on another user's account" is testable; "be careful with auth" is not.
• →Run the rules against the live product on every release, regardless of who or what wrote the change. Authorship is irrelevant to behaviour.
• →Give agents their own identities and minimum permissions, and treat everything they read from outside as untrusted data.
• →Keep using assistants. The productivity is real. Pair it with tests that check the thing the prompt forgot to say.