Why annual pentests fail against AI attackers

An annual penetration test is a careful, expert look at one version of a product on one date. What falls short against AI-assisted attackers is the annual cadence when it is the only test: products change every week, and the time from disclosure to exploitation is now measured in days. Keep the pentest, and pair it with rules re-proven on every release.

The numbers moved quickly. Mandiant measured the average time from a vulnerability's disclosure to its exploitation at 63 days in 2018–19 and five days in 2023 ↗. Its M-Trends 2026 ↗ report estimates the mean is now minus seven days: on average, exploitation began before a patch was available. Verizon's 2026 Data Breach Investigations Report found exploitation of vulnerabilities was the leading way into breaches, at 31% ↗.

What the annual pentest was built for

For two decades the annual penetration test has been the backbone of application security: a skilled tester, one or two weeks, a report, a remediation deadline and a retest. It matched the economics of attack. Finding a serious flaw took a skilled person days or weeks, so a yearly test by a skilled person was a fair stand-in for what an attacker could do.

Standards wrote the cadence down. PCI DSS requires internal and external penetration tests at least once every 12 months and after any significant infrastructure or application upgrade or change ↗. Many organisations do less: the UK government's Cyber Security Breaches Survey 2025/26 ↗ found 13% of UK businesses had run a penetration test in the previous 12 months.

What changed: finding a flaw got cheap

AI systems have been finding real vulnerabilities in public for two years. Google's Big Sleep agent found an exploitable flaw in SQLite ↗ in late 2024. At the final of DARPA's AI Cyber Challenge in August 2025, the competing systems found 54 of 63 planted vulnerabilities and 18 real ones ↗, at about $152 a task. In June 2025 XBOW said its autonomous tester had reached the top of HackerOne's US leaderboard ↗.

In 2026 the cost fell again. Anthropic reported that Claude Mythos Preview found a 27-year-old flaw in OpenBSD in a run costing under $50, and Mozilla fixed 271 Firefox bugs in two weeks with the same model. Google's Threat Intelligence Group counted 18 exploited vulnerabilities a month ↗ in the first eight months of 2026, up from 10.5 in 2025, and reported a flaw discovered with AI being exploited within four days of its disclosure.

Four gaps a yearly snapshot leaves

• →Coverage. A tester samples: some endpoints, some IDs, some roles, within the days the engagement allows. A model can enumerate every one, and the flaw is often in the endpoint nobody sampled.
• →Time. The product ships every week. A test in March describes the March build; the handler merged in May waits for the next engagement.
• →Output. A report becomes a queue of tickets, and few teams can say six months later which controls hold today. Verizon found only 26% of known exploited vulnerabilities were fully fixed in 2025 ↗, with a median of 43 days to fix.
• →Authorship. Assistants now write a large share of production code, and generated code fails differently: clean, reviewed and missing the check the author assumed. A test scoped a year ago was written before that code existed.

Nothing that was deployed seven or 10 years ago is prepared to handle AI at machine speed.Nikesh Arora, CEO, Palo Alto Networks, in Yahoo Finance, September 2026

What to pair it with

The unit that keeps pace with a weekly release is the rule: one plain-English sentence, such as "a user must never be able to act on another user's account", with a check that runs against the live product. An audit writes the rules and fixes what fails them. Continuous assurance re-proves every rule on every release against a staging or production deployment, and reports each one as holding or broken. Teams that want wider coverage run MATT on developer machines as well.

Three things change. Coverage widens to the 1,700+ attack patterns the rules are drawn from. A regression shows up on the release that introduced it. And the board's question, "are we secure?", gets an answer dated this week.

How to get more from the next pentest

The pentest stays. PCI DSS requires one, SOC 2 auditors and enterprise customers ask for one, and a skilled tester finds things no rule covers yet. It does the most good as an input to the rule set.

• →Write the scope first: the surfaces, a test account for every role, the limits and the rules the testers must check. The pentest scope builder produces that document.
• →Ask for every finding to be mapped to a rule, new or existing, so it is re-tested on every release after the engagement ends.
• →Run a test after a significant change, as PCI DSS already asks, and point it at what changed.
• →Rotate the focus from year to year: identity and access one year, AI features and agents the next.

What to do on Monday

• →List what has shipped since your last penetration test. That list is what the last report did not see.
• →Write the three rules that matter most as plain-English sentences, starting with ownership.
• →Test them with two accounts against the current release, then again on the next one.

The annual pentest was a good answer to the old cost of attack. Kept, and paired with rules re-proven on every release, it is still part of a good answer to the new one. The AI security audit is where MATT starts that pairing.