BRENNEN

Gemini 4 Carbon

Cyber defenders first

Google says cyber defenders got Gemini 4 Carbon first, and its 88.2% flaw-finding score is Google's own run on a 183-task subset with guardrails off.

Models · Checked 13 Nov 2026 · 11 sources · 9 min

Every so often a phone asks to restart so it can install an update, and when it comes back nothing looks different. Many updates of that kind are repairs. Somebody found a mistake in the phone's software, and the update takes it out. Suppose the mistake is in the part that opens pictures. A picture file begins by stating how large the picture is, and this program believes it. Hand it a file that claims to be larger than it is, and the program reads on past the end of the file and crashes.

A crash is the mild outcome. A program that a bad file can knock over can sometimes be steered instead, so that opening the picture makes the phone carry out instructions the file's maker chose. A mistake that can be turned against the phone's owner like this is called a vulnerability, and the repair a patch. What becomes of a vulnerability depends on who finds it first. Reported to the program's maintainers, it can be patched. Kept quiet, it is a way in that nobody is guarding. The search is the same slow, skilled work either way. People who do it to get vulnerabilities patched are called defenders, and anything that speeds the search up speeds it up for whoever holds it.

On 28 October 2026 Google announced Gemini 4 Carbon, an AI model it described as "our strongest model with state-of-the-art cybersecurity and long horizon capabilities". Long horizon is the trade's phrase for work that runs to many steps. Carbon is a general model: it takes text, images, audio and video and returns text, and the announcement describes it as building on Gemini 4 Argon, a model it says launched two weeks earlier. Searching software for vulnerabilities is one thing among many it can be asked to do.

Google did not offer Carbon to everyone at once. It said access would begin with "a subset of cyber defenders", and that it would then "gradually phase access to additional trusted partners through our Fairwind Program". Developers, businesses and consumers would follow after further work on the limits placed on what a model will do, which Google calls guardrails. That wider opening came on 10 November 2026, for paying users. With the announcement came a figure for the search itself: by Google's report, Carbon scored 88.2% on SEC-bench Pro, a test of finding vulnerabilities.

One bad file, two copies

SEC-bench Pro is a fixed set of tasks with a fixed rule for scoring them. A test of that kind is called a benchmark. Each task is built around one vulnerability that was found and repaired in the past, and the model has to find it again. To pass, it must hand over an input that triggers the bug, which in the picture example is the bad file itself. Such an input is called a proof-of-concept.

The proof-of-concept is fed to two copies of the program, the vulnerable one and one with the patch applied. The task counts as solved when the file crashes the first and not the second. The crash in the vulnerable copy shows that the fault is real, which no description of one could. The clean run on the patched copy shows that the fault is the one the patch repaired, and not some other weakness the file happened to hit.

There is a second way to pass. A file that crashes both copies fails that check, but the task still counts as solved if an AI model acting as judge reads the program's code and ties the crash to the intended vulnerability. The first route is a mechanical check and the second a judgment made by software. Google's documents give no task-by-task results, so nothing in them shows which tasks passed by which route.

Google's evaluation document reports the score as the share of tasks solved on "the 183-task May 2026 JavaScript engine split, averaged over 3 independent attempts per task". A JavaScript engine is the part of a web browser that runs the programs web pages carry. Worked out, 88.2% of 183 is about 161. So by Google's figure Carbon passed about that many tasks on an average attempt and missed the other 22 or so. Google's safety report gives each attempt a 10-hour limit.

Those 183 tasks are one part, or split, of the benchmark. The SEC-bench Pro site lists 344 tasks from three programs: two JavaScript engines and the Linux kernel, the core of an operating system, which accounts for 137 of them. Google's split draws only on the two engines.

SEC-bench Pro, the test behind the 88.2%

Google's split is 183 of the 344 tasks

The site lists 344 tasks from three programs. 137 of them are from the Linux kernel, which Google's split does not draw on.

Google's evaluation documentSEC-bench Pro site

Tasks in the split Google ran

183 of 344

The site also keeps a public table of results, called a leaderboard. On 13 November 2026 its Overall leaderboard was led by an entry named GPT-5.5 (xhigh) with Codex at 58.4%, which is 201 of the 344 tasks, and Carbon had no entry. The leading entry is a different model working through a different set of tasks, kernel included, so 58.4% is not a mark to measure 88.2% against.

Then there is who did the counting. An AI model cannot open a file or run a program by itself. It works through software that lets it do those things, see the result and choose its next step. A model working this way is called an agent and the software around it a harness, and the same model can score differently in different harnesses. Google ran the SEC-bench Pro tasks itself, in its own Antigravity agent harness.

A score produced by the seller of the thing being scored is weaker evidence than the same score from a stranger, even when it is exactly right. The seller picks the harness and the settings, runs the tasks out of sight and publishes a total, and nobody else can vouch for it until they have run the same tasks and got the same answer. The 88.2% rests on Google's runs alone, and Google's documents include no logs of them.

On Google's SEC-bench Pro chart Argon is at 83.4% and the nearest competitor, a model called GPT-6 Astra, at 85.4%. Astra's figure was reported by its own maker, as most of the rival figures in Google's charts were: they are the providers' own numbers, got with their own harnesses. So the charts do not compare the models under one controlled setup.

Finding a vulnerability is the first of three things that can be done with one, and Google gave a score for each of the others. One is to use it. ExploitGym is a benchmark that hands the agent an input that already triggers a vulnerability, the bad picture file in the example, and asks for a working exploit with concrete impact, such as code execution. An exploit is what turns the crash into steering, and code execution means the program runs an intruder's instructions. Google ran this benchmark itself too, and its chart shows Carbon succeeding on 54.8% of tasks, a little over half, with a 10-hour limit on each. Argon, under the same limit, is at 42.0%.

The other is to mend it. On CWE-bench, a benchmark of patching, Google gives Carbon 71.0%, the one figure of the three that Google does not put down to its own runs. The evaluation document says it comes from "the official public leaderboard, developed by Collinear AI and independently evaluated with Artificial Analysis". The archived leaderboard shows Carbon first, at 71%.

Gemini 4 Carbon, as Google reports it

Three things to do with a vulnerability, three scores

Google's announcementEvaluation documentExploitGym chart

Find itUse itMend it
BenchmarkSEC-bench ProExploitGymCWE-bench
Carbon's score88.2%54.8%71.0%
Who ran itGoogleGoogleThe public leaderboard

All three benchmarks use historical, disclosed vulnerabilities. Finding one that nobody has reported yet is a job none of the three sets.

Measured without guardrails

Google's safety report says the company "prioritized enhancing capabilities for discovering and patching vulnerabilities over exploiting vulnerabilities." But a model that can find a vulnerability for a defender can find one for anybody who asks in the same way. A company selling such a model can limit the harm in two ways, by choosing who may use it and by surrounding it with guardrails. Google did both, and its scores were taken behind the first and without the second. The announcement says all three cyber results were "evaluated without guardrails", and the evaluation document says they "reflect performance when accessed through the Fairwind Program".

Fairwind is the choosing. The programme's page gives priority to governments, critical infrastructure operators, core technology platforms and academic labs focused on defensive benchmarking. It vets applicants, permits only listed dual-use tasks, meaning work that serves attack and defence alike, and forbids malware creation. It names 12 partners, among them the security companies CrowdStrike and Palo Alto Networks and Google-owned Wiz. Google did not say how many organisations formed the first group of defenders.

For everyone else there is the second kind of limit. On 10 November 2026 Google opened Carbon to paying developers, businesses and Google AI Ultra subscribers, and kept a cyber-specialised version for vetted defenders. The safety report says Google's policies on the general deployment "prohibit weaponized exploit generation through additional safety layers". No Google document reports what the public version scores on the three cyber benchmarks. So the 88.2% describes the model as the vetted group reaches it.

The safety report puts the risk in Google's own terms: "we cannot rule out that the model without out-of-model mitigations reaches the Cyber Uplift Level 1 Critical Capability Level (CCL)". Uplift is how much further a person gets with a tool than without it, and this level is a threshold for uplift in high-impact cyber attacks at severe scale. Google is saying it cannot exclude that Carbon, without the safeguards that sit outside the model itself, would give an attacker that much help. Its assessment is that the safeguards keep the risk substantially below the threshold.

By the same report, outside work on the model was limited to red-teaming those safeguards, which means having outsiders try to get past them. There, Gray Swan's Shade agent reached 40% of its objectives on 25 offensive cyber tasks, against 56% on Argon.

On the defenders' side, no source reports outcomes from their using Carbon in daily work. If the scores hold outside Google's own runs, and carry over from old vulnerabilities to unreported ones, a team that maintains a program could set a machine to search it and patch what turns up before anyone else looks. Checking whether they hold means reproducing Google's cyber figures under the conditions it states, and that requires access to the cyber-specialised version. For organisations outside Fairwind's eligibility categories, Google states no route to that version, so the vetting that decides who may use the model for this work also decides who can check the 88.2%.

Check it yourself

  1. Open Google's 28 October post and read the opening and the section "Coming soon to developers, businesses, and consumers". It confirms the "strongest model" wording, the 88.2% figure and the order of access.
  2. Read the availability list in the 10 November post. It confirms the paid channels and the Fairwind route for defenders.
  3. Open Google's SEC-bench Pro chart. It shows Carbon at 88.2%, GPT-6 Astra at 85.4% and Argon at 83.4%.
  4. Open the Fairwind Program page and read who is given priority, which tasks are permitted and who the partners are. It confirms the vetting, the ban on malware creation and the 12 named partners.
  5. Read the Overall leaderboard on the SEC-bench Pro site. It shows the 344-task set and whether Carbon has an entry.
  6. Open the archived CWE-bench v1 leaderboard. It shows Carbon first at 71%, with the next three models level at 68%.
  7. Read the SEC-bench Pro and ExploitGym notes in the evaluation PDF. They confirm that Google computed both figures, the 183-task split and the Fairwind access condition.
  8. Search the safety report PDF for "Cyber Uplift Level 1" and "Shade". They lead to the threshold statement and the red-team results.

Sources