AI‑written code: the checks reviewers skip

AI-written code fails differently from human code. It is clean, idiomatic and complete in every respect except the check nobody asked for, most often the one that confirms the caller owns what they are asking for. Review reads what is present, so a missing check has no line to comment on. Testing the deployed product as an outsider is what finds it.

The share of code written this way is now large. In October 2024 Sundar Pichai said more than a quarter of all new code at Google ↗ was generated by AI and then reviewed by engineers; at Google Cloud Next in April 2026 he put the figure at 75% ↗. GitHub reported in Octoverse 2025 ↗ that nearly 80% of new developers on the platform use Copilot within their first week. Ask a security lead what changed in review as a result, and the honest answer is often very little.

How generated code fails

Human-written insecure code often looks wrong: a hurried patch, a copied snippet, an inconsistent style that draws a reviewer's eye. Generated code looks right. It is idiomatic, consistently named, well commented and complete in every respect except the one the prompt did not mention.

The prompt said "add an endpoint to fetch an order by ID". It did not say "only if the caller owns the order", because the author assumed that was obvious. The model did exactly what it was asked, and the endpoint returns any order to anyone who can guess an ID.

What the research shows

Independent studies have measured this for four years, with different methods, and they point the same way.

• →About 40% of programs vulnerable. Researchers asked GitHub Copilot to complete 89 security-relevant scenarios; of the 1,689 programs it produced, roughly 40% were vulnerable ↗ (Pearce et al., IEEE Symposium on Security and Privacy 2022).
• →Less secure, and more confident. In a Stanford user study, participants with an AI assistant wrote significantly less secure code ↗ and were more likely to believe it was secure (Perry et al., ACM CCS 2023).
• →45% of tasks. Veracode tested more than 100 models on 80 coding tasks and found the generated code introduced a vulnerability in 45% of cases ↗, with Java above 70% (July 2025).
• →Dependencies that do not exist. Across 576,000 code samples from 16 models, 19.7% of the packages the models recommended were hallucinated ↗: 205,474 unique invented names (Spracklen et al., USENIX Security 2025).

Developers sense it. In Stack Overflow's 2025 survey more developers distrusted the accuracy of AI tools than trusted it ↗, 46% against 33%, and 66% named AI answers that are almost right, but not quite, as a frustration.

Six checks that go missing

• →Ownership. The record is fetched by ID and the caller's relationship to it is never verified. OWASP ranks broken object level authorization first among API risks ↗, it is the most common generated-code finding we see, and in a multi-tenant product it is a full account takeover. The ownership pattern shows how it is tested.
• →Bounds. Pagination without a maximum page size, uploads without a size limit, loops over caller-supplied lists. Each is a spend or availability problem at scale, written up as unbounded input.
• →Rate. A new endpoint inherits no rate limit because limits were configured per route, by hand, somewhere else. On a metered AI feature the result arrives as a bill: metered resource abuse.
• →Secrets. Credentials that were meant to be injected end up in a default parameter, a test fixture or a log line, because the model needed a working example.
• →Dependencies. A suggested package that does not exist is an unclaimed name anyone can publish under. A model that recommends it once will often recommend it again.
• →Trust. Content from a user, a webhook or a document is data to the person who wrote the prompt and instructions to the agent that reads it. OWASP lists this as LLM01, prompt injection ↗; the indirect form arrives through websites and files. Prompt injection into tooling shows the test.

The agent did what it was told. The problem was who got to tell it.From the MATT record, on the Black Hat 2026 coding-agent findings

Why review does not catch it

Reviewers check what is present. A missing check has no diff line to comment on, and a clean, well-named function gives the eye nothing to snag on. Static analysis has a similar blind spot: it flags dangerous patterns, and the absence of an authorisation call is not a pattern. Broken access control has topped the OWASP Top 10 since 2021, and in the 2025 edition ↗ every application tested was found to have some form of it.

The dependable way to find a missing check is to test the behaviour of the deployed product as an outsider: fetch someone else's order from a second account, page through ten million rows, send the webhook the agent will act on. Behaviour does not depend on who wrote the code, so the same test covers a human change and a generated one.

What to do on Monday

• →Write the assumptions down as rules. "A user must never act on another user's account" is testable; "be careful with auth" is a wish.
• →Test with two accounts in two tenants. Most of the checks above only fail across accounts.
• →Re-prove the rules on every release, whoever or whatever wrote the change. Teams that want wider coverage run MATT on developer machines as well, so each rule is also checked as the code is written.
• →Confirm that every new dependency exists and is the one you meant before it is installed, and pin it.
• →Give agents their own identities and the minimum permissions their task needs, and treat everything they read from outside as untrusted data.
• →Keep using assistants. The productivity is real. Pair it with tests for the thing the prompt forgot to say.

Generated code is here to stay, and most of it is fine. The part that needs attention is narrow and predictable: the checks an author assumed and never wrote down. Writing them down as rules and re-proving them on every release is how a team keeps both the speed and the checks. AI-written and agentic code describes how MATT tests it.