How Mandiant combines AI code review with expert validation

AI code review gains credibility from expert constraints

Google Cloud reports that its Agentic Vulnerability Discovery Harness found more than 100 true-positive critical vulnerabilities in two days during an incident involving stolen corporate repositories. Alex Tselevich and Michael Maturi, authors of the Google Cloud account, describe a system that places expert controls around the models.

Mandiant uses the system in security assessments and incident response work. Its AI agents, model-driven components assigned specific investigation tasks, search code, collect context, trace execution paths and generate vulnerability hypotheses. Human experts provide specialist knowledge, approve the threat model and reproduce suspected exploits before findings survive. The system expands how much code experts can investigate while leaving final security judgment with people.

The claims come from Google Cloud and Mandiant, which have a commercial stake in demonstrating the value of this approach. Google Cloud argues that stolen proprietary source code creates urgency because attackers can rapidly use automated tools to search it for exploitable weaknesses.

The gap between manual review and conventional scanning

Source-code review has a scaling problem. Tselevich and Maturi say Mandiant has applied the harness to environments containing tens of millions of lines of code. Manual review at that scale demands substantial expert time.

Conventional static analysis automates parts of the work. Tselevich and Maturi argue that these tools can struggle with business logic, execution reachability and the distinction between privileged and public-facing functions. Those distinctions can determine whether suspicious code exposes an exploitable path.

The Agentic Vulnerability Discovery Harness combines automated investigation with expert decisions at defined stages. Tselevich and Maturi describe it as a point-in-time architecture.

The architecture puts expertise around the models

The harness uses a sequential pipeline that divides the investigation into specialized stages. Tselevich and Maturi say this structure makes analysis more predictable and reduces erratic model behavior.

Threat modelling comes first. An explorer agent identifies the software domain, reads documentation and marks directories such as unit tests for exclusion. Specialist subagents investigate areas including authentication, authorisation and routing, then a synthesis agent combines their work into a threat model.

A human consultant must verify that model before broader analysis proceeds. This checkpoint determines what the automated system considers important and in scope.

The next phase searches every file in scope for entry points, meaning locations where requests, messages or other external inputs enter the software. These include HTTP routes and inter-process communication listeners. Additional agents collect the surrounding code, bringing sanitisers, permission checks and nested function calls into the analysis so a suspicious function can be evaluated in its execution context.

Deeper inspection separates control-flow from data-flow analysis. Control-flow analysis examines the paths program execution can take. Its agents look for access-control problems including missing authorisation, privilege escalation and cross-site request forgery.

Data-flow analysis tracks how information moves through the software. Its agents follow user-controlled input to determine whether it can reach dangerous functions associated with SQL injection, cross-site scripting, command injection and path traversal.

Mandiant consultants also encode specialist knowledge into structured rules covering software domains, programming languages, frameworks and individual vulnerability classes. Tselevich and Maturi say these rules capture where frameworks define entry points, how they behave and how particular flaws should be found and validated.

Several validation agents challenge the AI-generated hypotheses. A synthesis stage classifies a suspected flaw as confirmed, disproven or rejected. Confirmed findings are risk-rated and deduplicated.

Human review is the final gate. Reviewers attempt to reproduce exploits and test proof-of-concept code. Findings that fail dynamic testing are discarded. Tselevich and Maturi say this stage removes false positives and catches incorrect model assumptions.

Reported deployments establish operating scale

Tselevich and Maturi say Mandiant has used the system during the past 10 months in proactive code reviews, penetration tests, red team operations and investigations following source code theft.

At large-codebase scale, they report that the process has generated thousands of analysis pipelines and tens of thousands of findings. Reviews have uncovered dozens of flaws in web extensions and open-source projects. They say 12 Common Vulnerabilities and Exposures identifiers have been assigned so far, with about a dozen additional issues in active disclosure.

They also describe a client engagement in which the harness processed a customer’s web application source code and rapidly identified a remote code execution vulnerability that provided an initial route into the target environment.

These outcomes are reported by Google Cloud employees about technology used by Mandiant, part of Google Cloud. Both organizations benefit commercially when customers see value in their security capabilities. The cases establish reported usage and operating scale. Broader comparative performance across organizations, codebases and review methods is a separate question.

Evaluation separates discovery from memorization

The role of specialists defines the system’s autonomy. Experts verify the threat model, contribute structured security knowledge, reproduce exploits and discard findings that fail testing. This design supports a narrower proposition: AI can scale an investigation directed and adjudicated by security experts.

Tselevich and Maturi also describe an evaluation designed to address contamination from model training data. Public vulnerability datasets may contain flaws encountered during model training, allowing a model to appear to discover a vulnerability when it is recalling information it has already seen.

The evaluation uses internal synthetic codebases with injected vulnerabilities. Human experts verify those flaws. A grading process compares system findings with known ground truth, meaning the verified answers used as the reference for evaluation, before manual review checks the results. This design aims to distinguish vulnerability discovery from recall of public vulnerabilities.

That test addresses memorization. Comparative performance against other review approaches is a separate question. The reported deployments and synthetic evaluation support different claims and should be assessed separately.

Expert adjudication sets the operating boundary

For a security leader, the decision is whether this expert-designed workflow can expand the amount of code a team can investigate when time and scale constrain manual review. Model capability is one part of that decision. The surrounding controls determine where specialist judgment enters the process and what evidence is required before an AI-generated suspicion becomes a security finding.

Tselevich and Maturi describe the strategy as “embedding frontier models within an expert-defined harness” to automate discovery of routine vulnerabilities. In the workflow they describe, the consequential boundary comes at validation: reviewers attempt to reproduce the exploit and discard findings that fail dynamic testing. That human gate determines which machine-generated hypotheses become findings a security team can act on.

Key highlights

• Expert controls make AI code review more credible: Mandiant reports finding more than 100 critical vulnerabilities in two days by combining AI agents with expert threat modelling, validation and exploit reproduction. Leaders should evaluate the controls around AI discovery.
• AI can extend security review beyond manual scale: Mandiant has applied its harness to codebases containing tens of millions of lines, while conventional scanning can miss business logic and execution context. Security teams should consider where agentic analysis can complement manual review and static analysis.
• Specialized agents need defined human checkpoints: The harness separates threat modelling, entry-point discovery, control-flow analysis, data-flow analysis and validation. Human approval of the threat model and final findings keeps consequential security judgments with specialists.
• Reported deployments show scale: Google Cloud says the system has generated thousands of analysis pipelines, tens of thousands of findings and multiple CVEs. Buyers should treat these vendor-reported outcomes as evidence of operating scale while seeking independent or comparative benchmarks.
• Evaluation should distinguish discovery from memorization: Mandiant tests the system on internal synthetic codebases with injected, expert-verified vulnerabilities to reduce the risk that models merely recall public flaws from training data. Leaders evaluating AI security tools should ask how benchmarks control for training-data contamination.
• Human adjudication remains the operating boundary: AI agents can expand how much code security teams investigate, but Mandiant requires reviewers to reproduce exploits and reject findings that fail dynamic testing. Organizations adopting similar systems should preserve explicit human gates before AI-generated hypotheses become actionable findings.