SOC 2 in the age of agentic attacks
SOC 2 shows controls operated over an audit period. How to make the evidence for technical controls a test of the live product, mapped to the criteria.
A SOC 2 report shows that the controls a company described were in place and, in a Type 2 report, that they operated over a period. For technical controls the strongest evidence is a test of the live product, run on every release and mapped to the Trust Services Criteria. SOC 2 stays essential; what changes is the evidence behind it.
This matters more in 2026 because testing a control from the outside has become cheap. In April Anthropic reported a model run costing under $50 that found a 27-year-old flaw in OpenBSD. In September NIST and Anthropic measured a downloadable model building almost as many working exploits as Claude Mythos Preview. A model with an ordinary account can try a control every day; a report describes the period the auditor examined.
What a SOC 2 report shows
SOC 2 is an attestation against the AICPA's Trust Services Criteria ↗. A Type 1 report covers how the controls were designed as of a date. A Type 2 report adds the auditor's opinion on whether they operated effectively throughout a period ↗, with the tests the auditor performed and their results.
The evidence behind most technical controls is familiar: a policy, a screenshot of a configuration, a ticket, a quarterly access review, and the auditor's sample of each. It shows that a control existed and that someone performed it. It says less about how the product behaves when someone tries to get around the control.
The gap between evidence and behaviour
Take a common control: access to customer data is restricted to authorised users. The SOC 2 evidence is an access review and a policy document. The behavioural test is different: create an ordinary account, request another customer's record by ID, and see what comes back. Both can be true at once. The access review can be complete and the endpoint can still answer.
Vercel had a compliance programme. In April 2026, according to its own bulletin ↗, it was entered through the OAuth tokens of a third-party AI tool an employee had connected to Google Workspace. The controls an auditor samples and the paths an attacker tries are two different sets.
From screenshot to test result
Each technical criterion can be expressed as one or more plain-English rules, each with a check that runs against the live product. The evidence for the criterion becomes the check's result, timestamped, on every release. The criteria below are paraphrased from the AICPA's 2017 Trust Services Criteria.
| Criterion | What it asks for | Example rule | Evidence |
|---|---|---|---|
| CC6.1 | Logical access controls over protected information assets | A user must never act on another user's account. | Result per release, from two test accounts |
| CC6.6 | Protection against threats from outside the system's boundaries | No unauthenticated endpoint returns customer data. | Result per release for every public endpoint |
| CC7.1 | Detecting changes that introduce vulnerabilities, and newly discovered ones | A release must not reopen a fixed finding. | Rule results compared release to release |
| CC7.2 | Monitoring for anomalies that indicate malicious acts or errors | A broken rule reaches a named owner the same day. | The alert log |
| CC8.1 | Testing and approving changes before they are implemented | Every release is re-proven against every rule. | The results of each run |
The auditor receives a control map backed by results. A customer's security questionnaire gets the same answer, and it is current.
What a good test record contains
An auditor samples. A record that answers any sample on any date in the period has five parts.
- →The rule, in its plain-English sentence, and the criterion it supports.
- →The environment and the release it ran against, so each result ties to a change.
- →The test accounts and roles it used, with no customer data in the record.
- →The time it ran and the result: holding or broken.
- →For a broken rule, who was told and when, and the release in which it held again.
Kept this way, the record answers the auditor's request for any date and the question a customer actually asks: does this control hold today?
Every company on the record had a security team, scanners and a pentest on file. The missing piece was a loop that knew what had become exploitable since.From the MATT home page
What the standards already say
The criteria already point this way. CC4.1, taken from COSO's internal control principles, asks a company to perform "ongoing and/or separate evaluations" of whether its controls are present and functioning. NIST describes the same idea for federal systems: SP 800-137 ↗ defines continuous monitoring as "maintaining ongoing awareness of information security, vulnerabilities, and threats to support organizational risk management decisions", and the NIST Cybersecurity Framework 2.0 ↗ gives it a category of its own, DE.CM.
Auditors are also watching how evidence is gathered. In February 2026 the AICPA's Journal of Accountancy ↗ described compliance tools that pull evidence from client systems automatically, where clients once sent screenshots and spreadsheets, and the chair of the AICPA's SOC 2 working group cautioned that a report can end up leaning too heavily on inquiry. A behavioural test sits at the other end from inquiry: it records what happened when someone tried to get around the control.
The frontier labs accepted a similar standard for themselves in September, in the White House Accord on Super Intelligence: controls shown to be operating as intended, assessed by an independent outsider.
What to do on Monday
- →Take the current SOC 2 control list and mark which controls are technical and can be tested from outside the product. It is usually a long list.
- →Write each one as a plain-English rule with a check. The first pass usually surfaces a control that was described and never verified.
- →Test with two accounts in two tenants, so ownership and isolation are checked across accounts.
- →Re-prove the rules on every release and keep every result with the time it ran.
- →Agree the evidence format with your auditor before the period starts, then hand customers the same control map for questionnaires.
- →Keep the annual penetration test, and give the testers the rules, so the report says which ones held.
Compliance and security were always meant to describe the same thing. Evidence from the live product, re-proven on every release, is how they do again. Continuous assurance is how MATT runs it.
When an auditor or a customer asks for a penetration test, the pentest scope builder turns the same rules into a scope a testing firm can quote and work from.
Questions
What is the difference between SOC 2 Type 1 and Type 2?
A SOC 2 Type 1 report describes a company's system and whether its controls were suitably designed as of a specific date. A Type 2 report adds the auditor's opinion on whether those controls operated effectively throughout a period, with the tests performed and their results. Both are attestations against the AICPA Trust Services Criteria, and MATT's rule results can support the evidence for technical controls in either.
Can automated security tests be used as SOC 2 evidence?
The auditor decides what evidence is sufficient, so agree the format before the period starts. For technical controls, a timestamped result from a test of the live product, such as a second account failing to read another customer's record, shows the control operating under a real attempt. MATT keeps every rule result with the time it ran, mapped to the criteria it supports.
What is continuous control monitoring?
Continuous control monitoring means checking that controls keep working as systems change, often enough to support risk decisions. NIST SP 800-137 defines information security continuous monitoring as maintaining ongoing awareness of information security, vulnerabilities and threats, and NIST CSF 2.0 lists it as category DE.CM. MATT applies the idea to product behaviour by re-proving each rule on every release.