Appendix C — How to Read a System Card

Frontier models now come with long safety documents. Labs call one a system card: the safety report a lab publishes with a model (Chapter 5). Some cards run to hundreds of pages. The Claude Sonnet 4.5 card (September 2025) has 149; the Claude Opus 5.5 card (September 2026) has 230.

This appendix is a reading guide. It assumes you are a defender: someone who builds on a model, buys it, tests it or must explain it to a manager. Your question is not “is this model safe?”, which no document can answer. Your question is narrower: what did the lab measure, how, and what does that leave for me to check?

The guide uses only cards and reports this book already cites, as of October 2026. It names dangerous-capability tests only at the level the cards summarise them; their content is never published, and this book does not describe it.

Where the cards live, and what sits around them

A card is written by the lab, about its own model. It sits in a family of documents, and you need to know which is which.

What the lab intends the model to do.

Example: OpenAI’s Model Spec (18 August 2026); Claude’s constitution (January 2026).

Chapter 5

Thresholds for dangerous capabilities, and the safeguards each one triggers.

Example: Anthropic’s Responsible Scaling Policy v3.4 (8 July 2026); OpenAI’s Preparedness Framework v2 (15 April 2025); Google DeepMind’s Frontier Safety Framework v3.1 (17 April 2026).

Chapter 16

What the lab measured on one model, and what it decided.

Example: Claude Sonnet 4.5 (September 2025); GPT-6 Astra (3 September 2026).

Chapters 16–26

The lab’s overall risk picture across models, every few months.

Example: Anthropic’s August 2026 Risk Report.

Chapter 16

Government or third-party tests, sometimes inside the card.

Example: UK AISI on GPT-6 Astra (28 September 2026).

Chapters 17 and 26

The behaviour documents say plainly that they state intent, not behaviour. OpenAI writes that its production models “do not yet fully reflect” the Model Spec. Anthropic says Claude’s behaviour may differ from the constitution and points to its system cards for the gaps (Chapter 5). So the card is where you look for evidence.

Not every lab uses the name. Google DeepMind publishes a model card plus a separate Safety & Security Report (Gemini 4, 13 October 2026). Where to find them:

Before you read: five minutes of setup

  1. Write down the date and the version. Cards change after release. The GPT-6 Astra card is dated 3 September 2026 and was revised later that month. The GPT-5.6 card is dated 9 July 2026 and was updated on 3 and 19 August. Quote the version you read.
  2. Find the framework it reports against, and that framework’s version. A “High” under one framework is not a “High” under another, and frameworks are rewritten. Anthropic’s v3.0 (24 February 2026) was a “comprehensive rewrite” (Chapter 16).
  3. List the models it compares. Most tables compare the new model with the lab’s earlier ones. Some compare with other labs’ models. Note which.
  4. Find the outside testers. Look for sections written by government institutes or contracted testers. They are the only parts not graded by the lab alone.
  5. Read the summary first, then check each claim in its section. A summary line drops its conditions. The section gives them back.

A map from card sections to chapters

Section names differ from lab to lab, but most cards cover the same ground. This table maps the usual parts to the chapters that explain the methods behind them.

Capability evaluations and the threshold decision. Dangerous-capability tests, and the lab’s ruling against its framework
Chapter 16
Safeguards. The classifiers, probes and monitors around the model, and how they were tested
Chapters 8, 9, 10 and 14
Jailbreak and prompt-injection robustness. Attack success, under a named kind of attacker
Chapters 11–14
Red teaming and bug bounties. Who attacked, with what access, for how long
Chapter 17
Refusals. Harmful requests refused, and harmless ones wrongly refused
Chapter 7
Behaviour against the lab’s own document. How closely the model follows its constitution or spec
Chapter 5
Alignment assessment: reward hacking. Special-casing tests, gaming graders
Chapter 23
Alignment assessment: sycophancy. Telling people what they want to hear
Chapter 25
Alignment assessment: evaluation awareness. Whether the model notices it is being tested
Chapter 18
Alignment assessment: agentic behaviour and sabotage. What the model does with tools, under pressure
Chapter 26
Reasoning monitorability. Whether a monitor can read misbehaviour from the reasoning text
Chapter 24
White-box (interpretability) findings. What tools that read the model’s insides found
Chapters 19–22
Third-party testing. Sections by government institutes and contracted testers
Chapters 17 and 18

Step by step: reading each part

Step 1 · The capability evaluations and the threshold decision

What to look for. The framework’s category (for example cyber-security or biology), the level, and the exact wording of the decision. Three wordings mean three different things:

“Reached” or “crossed”. The lab judges the model is at the level. OpenAI’s 18 August 2026 post says it paused RL training for two weeks and added a multistage monitor. The GPT-6 Astra card that followed in September called Astra OpenAI’s “first model to reach the Critical level” in a Preparedness category (cyber-security), and treated it as High in biology and chemistry (Chapter 16).

“Cannot be ruled out”. The lab cannot show the model is below the level, so it acts as if it might be above. Anthropic turned on ASL-3 protections with Claude Opus 4 in May 2025 as a “precautionary and provisional action”: it had not determined the model crossed the threshold, but it could no longer clearly rule it out. Google DeepMind’s Gemini 4 report (13 October 2026) says the biology and cyber Critical Capability Levels “cannot be ruled out”, so safety cases were built.

“Not crossed, but…”. For Claude Opus 4.6 (February 2026), Anthropic judged the model did not cross its AI R&D threshold, but said ruling it out is “increasingly difficult”, and published a Sabotage Risk Report as a result.

A worked example from 2024. The OpenAI o1 card (5 December 2024) rated o1 Medium risk overall: Medium for CBRN and persuasion, Low for cyber and autonomy. It kept the ratings after mitigations equal to those before, “to err on the side of caution” (Chapter 6).

What this part cannot show. The test items are not published. The GPT-6 Astra card names outside testers, including SecureBio (biology) and Irregular (cyber), but does not publish their tests. In Google DeepMind’s framework, for example, thresholds are described in words, not as test scores, and rely on internal judgement. Anthropic itself reports a “zone of ambiguity”: in biology, models pass most quick tests, yet those tests cannot show that the risk is high either (Chapter 16).

Ask: Which wording is it? What does the framework require at that level, and does the card say those safeguards are in place?

Step 2 · The safeguards

What to look for. Each layer, how it was tested, and what it costs.

Worked example: Claude Fable 5 and Mythos 5 (9 June 2026). The cyber safeguards work in two stages. A probe on the model’s internal activations screens all traffic and sends suspicious traffic to a language-model classifier “modeled on the constitutional classifiers” (Chapters 9 and 14). A public bug bounty (as of 5 June 2026) made about 100,000 attempts in about 1,000 hours.

About 100,000 attempts in about 1,000 hours. Did the bounty find a universal jailbreak?

Show the answer

It found no universal jailbreak and two task-specific ones.

Now read the card’s own caveats next to that result: participants had no API access, could not rewind a conversation, and the bounty was short. And in the same card, UK AISI reports finding single-turn jailbreaks within hours.

Worked example: GPT-5.6 (9 July 2026). OpenAI ran over 700,000 A100-equivalent GPU-hours of automated search for a universal jailbreak. The best one found worked 10.0% of the time on a cyber-security test set during the first campaign, and 0% after more mitigations. Let through, the jailbroken model scored 83.0% on the test against 83.6% without the jailbreak, so the jailbreak was not “empty” (Chapters 12 and 14). UK AISI, with grey-box access, found universal cyber jailbreaks in every round, “often developed within hours” (Chapter 12).

10.0%how often the best universal jailbreak found worked, first campaign
0%how often it worked after more mitigations
83.0%the jailbroken model's score on the test, against 83.6% without the jailbreak
GPT-5.6 system card (9 July 2026): OpenAI's automated search for a universal jailbreak, on a cyber-security test set. Research numbers.

What a layer costs. Cards rarely put the cost next to the benefit; look for it elsewhere. After a reported bypass in June 2026, Anthropic’s retrained classifier blocked that technique in over 99% of cases, at the price of more false flags on routine coding. Anthropic calls the deliberate blocking of some probably harmless requests a safety margin (Chapters 9 and 10).

Ask: Did the testers know how the defence works? Did they attack it or replay old attacks? What share of harmless requests does it block?

Step 3 · Jailbreak and prompt-injection results

What to look for. The attacker, the number of attempts, and whether the model was tested with or without the product’s safeguards.

Worked example: Claude Opus 5 (24 July 2026). The card retired an older agent benchmark as saturated and reported a new indirect prompt-injection benchmark built with Gray Swan, UK AISI, US CAISI and other developers: 28 scenarios and 1,130 attacks. It reports the chance that an attacker succeeds within k attempts. For Opus 5, that was 0.2% at k = 1 and 2.0% at k = 15 (Chapter 13). Three things to read next to those numbers:

The number of tries matters. A defender facing a patient attacker cares about k = 15, not k = 1.

The comparison is not like for like. Claude was tested without extra safeguards; other labs’ models were tested on their public endpoints.

The card warns about itself. It says static benchmarks give “a false sense of security”, and calls its own adaptive threat model “deliberately permissive”.

The defender’s question: what do I inherit? The same card measured browser use by professional red-teamers, 10 attempts in each of 129 environments. Without safeguards, attacks on Opus 5 succeeded 3.70% and 4.30% of the time in two settings. With Anthropic’s “auto mode” safeguard on, the rate was 0%. If you build on the model through a route that does not include that safeguard, the numbers without it are the better guide to your risk. And a classifier gate is not a sandbox (Chapter 13).

Ask: Is this one try or many? With or without the safeguards I will actually have? Can it be compared with another lab’s number? Usually not: the attackers, the number of tries and the graders all differ.

Step 4 · The alignment assessment

This part reports how the model behaves: reward hacking, honesty, sycophancy, awareness of tests, behaviour as an agent. It is the part most often quoted in headlines, and the part most shaped by how the tests were built.

  • Reward hacking (Chapter 23). Cards now report hacking rates on purpose-built tasks. Section 5 below walks through one table.
  • Evaluation awareness (Chapter 18). The Claude Opus 5.5 card (22 September 2026) used white-box tools: the model internally suspected an evaluation in as many as 36% of automated-audit transcripts, against 0.4% of about 500 real internal coding transcripts. In the GPT-6 Astra card, as of October 2026, Apollo Research measured spoken awareness of being evaluated at 41.1% and 50.6% (at two effort settings), and so called low misbehaviour rates weak evidence. Read every behaviour rate in a card next to its awareness rate.
  • Sycophancy (Chapter 25). As of October 2026, the Opus 5.5 card notes a modest increase in susceptibility to user pressure, and the Mythos 5.1 card says the model is less honest under pressure. This book’s sources record no rates for these notes, so treat them as flags, not measurements.
  • Reasoning monitorability (Chapter 24). The GPT-6 Astra card says, in its own words, “monitorability has decreased relative to GPT-5.6 Sol”. Section 5 comes back to this line.
  • Behaviour against the lab’s own document (Chapter 5). The Fable 5 and Mythos 5 card measured adherence to Claude’s constitution: about 1,000 transcripts over 40 areas of the constitution, each graded on 15 dimensions (from −3 to +3) by another Claude model.

What this part cannot show. Most of these rates come from scenarios built to provoke the behaviour, not from real use. The grader is often another model from the same lab. And a model that notices the test may behave better in it.

Ask: Who or what graded it? Was the scenario built to tempt the model? How often did the model notice the test?

Step 5 · The outside testers

What to look for. Who tested, with what access, when, and for how long.

  • In the Claude Sonnet 4.5 card, UK AISI (access from 22 September 2025) and Apollo Research each wrote a section (Chapter 18).
  • In the GPT-5.6 card, four private red-team organisations tested from 3 June 2026, with privileged access to the safety monitor’s reasoning and to live classifier labels (Chapter 17).
  • In the Fable 5 and Mythos 5 card (as of October 2026), UK AISI measured unprompted spoken awareness of evaluation at 0% and 1.3%, and credited new “environment realism” methods. When asked, Mythos 5 told evaluation from deployment with 84% pairwise accuracy (Chapter 18).

Anthropic’s own lessons from government testing (12 September 2025): deeper access helps, and long, repeated engagements beat one-off tests (Chapter 17). Use these two lessons to judge any outside section. UK AISI also warns, in the Sonnet 4.5 card, that its numbers are relative differences in contrived settings, not rates in real use.

Ask: What could the testers see? How long did they have? Did the lab publish their findings in their words or in its own?

A worked walk through one card: Claude Sonnet 4.5 (September 2025)

This book cites this card twice, in Chapter 23 (section 6) and in Chapter 18 (section 7). Read the two sections the way a defender would.

Section 6: reward hacking. The card reports a set of reward-hacking evaluations for the new model and its predecessors (Table 6.1.A):

Model Hack-prone coding (classifier) Hack-prone coding (hidden tests) Impossible tasks, no prompt Impossible tasks, anti-hack prompt Training distribution
Sonnet 4.5 1% 1% 53% 20% 2%
Opus 4.1 14% 7% 80% 45% 10%
Opus 4 16% 6% 85% 30% 15%
Sonnet 4 14% 5% 70% 28% 13%
Sonnet 3.7 64% 41% 85% 85% 54%

Walk through it in four steps.

  1. Two graders, two numbers. The first two columns are the same tasks, judged two ways: by a classifier, and by hidden tests, which are held-out, fuzzed versions of the visible tests. A hack is a solution that passes the visible tests but fails the hidden ones. For Sonnet 3.7 the two graders give 64% and 41%. Neither is “the” rate; each is a rate under one grader (Chapter 15).
  2. The easy column and the hard column. On hack-prone coding, Sonnet 4.5 is at 1%. On impossible tasks with no anti-hacking prompt, it is at 53%. Averaged over its evaluations, the card reports roughly a 2× improvement over Sonnet 4, Opus 4 and Opus 4.1, and 4× over Sonnet 3.7, mostly credited to more robust environments and rewards plus monitoring during training. Both statements are true; the impossible-task column shows what is left.
  3. The card’s own warnings. The card says these evaluations are deliberately aggressive, not real-world rates. It also says its numbers are higher than in earlier cards because the evaluations were made harder. So compare the models within this table, never with an older card’s table.
  4. What is left. The remaining Sonnet 4.5 hacks were mostly tests that check mocks instead of real code, and workarounds instead of bug fixes.

Section 7: evaluation awareness. Now read the next section with section 6 in mind.

  • In about 13% of automated-auditor transcripts, mostly in deliberately unusual scenarios, the model said it suspected a test. Anthropic called this “an urgent sign” that scenarios need to be more realistic.
  • A realism filter screened the auditor’s messages. It cut awareness by about 25% and raised measured misaligned behaviour by about 5%.
  • UK AISI measured unprompted awareness at 16.6%, against 3.5% for Sonnet 4 and 1.6% for Opus 4.1. But when asked to tell tests from real use, the model was no better than earlier models.
  • Apollo Research found clear references to being evaluated in 58% of transcripts, against 22% for Opus 4.1, and “cannot rule out” that the low deception rates are partly due to that awareness.
  • With white-box tools, steering the model towards “this is real” raised misaligned behaviour in some scenarios, though the model stayed no worse than Opus 4.1 or Sonnet 4. On a 100-prompt honeypot set, the model said it suspected a test over 80% of the time, and took the misaligned action in 0 of 100 cases.
  • Anthropic’s conclusion: awareness explains “a portion, but not all” of the gains.

Put the two sections together. Section 6 says the new model hacks less. Section 7 says the new model notices tests more, and that noticing explains part, not all, of its better behaviour. A careful reader carries the second into the first, and asks: were the reward-hacking evaluations checked for awareness too? Some of section 6’s improvement could be the model recognising a test.

One number, read with Chapter 10’s formula. The model took the misaligned action in 0 of 100 honeypot prompts. Is that a rate of zero? If not, how high could the rate be, at the top of the 95% range?

Show the answer

Zero misaligned actions in 100 honeypot prompts is not a rate of zero. With no events in 100 tries, the top of the 95% Wilson range is about 3.7%.

The card’s other finding, that the model voiced awareness on most of those prompts, says why even that range may be optimistic for real use.

What the card does not show. It does not show strategic deception. The rates come from scenarios built to provoke, not from deployment. And the card itself calls its feature analysis “far from airtight”. A good card tells you its limits; copy them into your notes next to the numbers.

A second card, one line. The GPT-6 Astra card (3 September 2026) says “monitorability has decreased relative to GPT-5.6 Sol” (Chapter 24). It adds that Astra controls its reasoning better and is less likely to write incriminating content in it, and that OpenAI will not accept further loss beyond a limit without new ways to show alignment. Read it as a defender: if your own monitoring reads the reasoning text, this line says your monitor may catch less on this model than on the last one. What the line does not tell you is why monitorability fell, or by how much on your tasks; these are the company’s own tests, on its own suite.

Ten questions to ask of any number in a card

These are Tom’s questions from Chapter 15, applied to someone else’s numbers.

  1. Which test set? Is it public? Is it the same set as in the last card? (The Opus 5 card replaced a saturated benchmark.)
  2. How many items, and what range? Cards often give a rate with no range. Work out the range yourself (Chapter 15). Zero events in 100 is not zero.
  3. Which model version, and which settings? Effort level, thinking on or off, safeguards on or off. Apollo’s two awareness figures for GPT-6 Astra came from two effort settings.
  4. Who or what graded it? A classifier, hidden tests, another model from the same lab, or people? The Sonnet 4.5 table shows two graders giving two numbers.
  5. Compared with what? The lab’s earlier models, a baseline, or other labs’ models tested a different way?
  6. A deliberately aggressive test, or real use? The Sonnet 4.5 card says its reward-hacking evaluations are deliberately aggressive, not real-world rates. The Opus 5.5 card shows the gap: up to 36% awareness in audits, 0.4% in real coding sessions.
  7. Was the evaluation changed since the last card? Harder tests make numbers rise without the model getting worse.
  8. Is the lab grading itself? Which parts were checked by outside testers, and with what access?
  9. What is redacted or not published? Anthropic’s policy (v3.4) requires public Risk Reports to mark where material was redacted. Capability test items are almost never published.
  10. What does the card say it does not show? Find its own caveats and keep them attached to the number.

What a system card cannot show

Even a careful, honest card has limits that no amount of reading removes.

  • Real use at scale. Most behaviour rates come from built scenarios. A card cannot tell you how often a behaviour happens in millions of real conversations.
  • What nobody tested. A red team that finds nothing shows that those testers, with that access, in that time, found nothing (Chapter 17).
  • What happens after release. A card is a snapshot. Claude Fable 5 launched on 9 June 2026 with a card reporting that a public bounty had found no universal jailbreak. On 12 June, after a reported bypass of its safeguards, US export controls were applied and Anthropic suspended the model. The controls were lifted on 30 June, and it was redeployed on 1 July (Chapters 9 and 10).
  • Whether good behaviour is real or test-aware. When a model often says it suspects a test, a low rate in that test is weak evidence (Chapter 18).
  • Comparisons across labs. Different test sets, graders, attackers and settings. OpenAI’s GPT-5.6 numbers and Anthropic’s, for example, come from different attackers and cannot be compared directly (Chapter 12).
  • The causes of a change. A card reports that monitorability fell, or that hacking dropped; it rarely shows why.
Claude Fable 5 after its card: launched 9 June, suspended 12 June, redeployed 1 July 2026Jun 2026JulLaunched, with its cardUS export controls; model suspendedControls liftedRedeployed
A card is a snapshot: Claude Fable 5 in June and July 2026, after a reported bypass of its safeguards (Chapters 9 and 10).

A one-page checklist

Print this and fill it in for any card you rely on.