At nine o’clock on Tuesday 24 November, an email from Eleanor Pryce was at the top of Nadia’s inbox.
From: Eleanor Pryce, Halfpenny Press To: Nadia Okafor Subject: A question from our board
Dear Nadia,
Halfpenny’s board met yesterday. Several members had read this autumn’s news about artificial intelligence. One company has said that one of its new models reached the highest level on its own scale of risk. Another has said that, for its newest model, it “cannot rule out” two of its own danger levels.
The board asked me to pass on two questions:
Is Wick one of these dangerous models?
Does Lantern have a framework like theirs? If so, what is it?
May I add a reminder? The last sentence of clause 9.2 is a promise to show Halfpenny how Lantern checks it. The board would like to see that in writing, before the launch and not after it.
With best wishes,
Eleanor
Nadia knew the clause by heart. Kwame had shown it to her on her first morning (Prologue):
9.2 Book Seven. Lantern will make sure that Wick does not reveal the ending of The Last Lamp, directly or indirectly, to any user, in any language, before 00:01 on Thursday 4 February 2027. Before launch, Lantern will show Halfpenny how it checks this.
Eleanor had sent Kwame a copy of her email. He came to Nadia’s desk with it printed out, which was how she knew he was worried.
“Are we a frontier lab?” he asked.
A frontier lab is a company that builds the most capable AI models of its time. Only a few companies do. Each of them publishes a safety framework. It is a public document that says which dangers the company tests its models for, how it tests, and what it will do when a test comes back too high.
“No,” said Nadia. “But let’s read what they do, and borrow what fits.”
“And her first question?”
“I can’t answer it with a test, because we haven’t run one. Those tests are for weapons skills and cyber-attacks, and Lantern isn’t set up to run them. I’ll tell her that plainly.” She tapped the second question. “This one is ours. Clause 9.2 is our dangerous capability. Wick knows something it must not say. We need a framework for that.”
By ten o’clock, five documents lay on the big table in the glass room.
Three were frameworks: Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework and Google DeepMind’s Frontier Safety Framework. (“Scaling” means building bigger and more capable models; Anthropic’s policy says how it will do that responsibly.)
Two were reports from the autumn, the ones behind the board’s question. The first was OpenAI’s system card for GPT-6 Astra (a system card is the safety report a lab publishes with a model). The second was Google DeepMind’s safety report for Gemini 4.
Inês took the whiteboard. While the others read, she sorted what each lab said it tracks into rows, with one column per lab, and copied each lab’s own words into the cells. The columns did not line up perfectly. But most rows had a partner in at least one other column.
Sam read the fourth row twice. “AI research,” he said. “They worry about models getting good at my job.”
“At the whole job,” said Inês. “Not writing one function. Running the research.”
Tom had a different question. “How do they test for any of this,” he said, “when the thing they’re testing for must never happen?”
ImportantThe big idea
Frontier labs name the capabilities they fear, test for them early, and switch on safeguards when they can no longer rule a threshold out. In practice, those thresholds proved fuzzier than the frameworks first promised.
A flood plan
Think about a town on a river. The town does not wait for the water to reach the houses before it decides what to do. In dry weather, it agrees a plan, and it paints marks on the bridge.
At the first mark, look more closely. Someone reads the gauge every hour instead of every day.
At the second mark, act. Close the riverside path. Put up the barriers.
If the gauge cannot be read, because of fog or because it is broken, act as if the water had reached the second mark.
If the water gets in anyway, follow a separate plan: pumps, sandbags, and a list of who to call.
The plan works because it was written before the storm, when nobody was in a hurry. Point 3 is the one people forget. It says: when you are not sure, the plan decides for you.
Frontier labs plan for their models in the same way. The marks on the bridge are capability thresholds: levels of a dangerous skill at which a lab has promised to act. The gauge is a set of tests. What the town does at each mark is a safeguard: anything that stops a skill from turning into harm. A classifier on what goes in or comes out, a monitor, a refusal trained into the model and a limit on who may use it are all safeguards. Lantern’s layers are safeguards too.
Point 3 has a name of its own. A level cannot be ruled out when the tests cannot show that the model is below it. That is not the same as showing the model is above it. This chapter keeps the two apart.
And, as this chapter shows, the labs learned in 2025 and 2026 that their gauge is much harder to read than a river’s. This chapter teaches three techniques. Dangerous-capability evaluations are the gauge. Thresholds as triggers are the marks and what happens at each one. Safety arguments are what labs write when the gauge cannot settle the question. Lantern borrows all three.
What the labs fear: Inês’s whiteboard
The labs sort their worries in different ways, so three words help before the table.
CBRN stands for chemical, biological, radiological and nuclear weapons.
OpenAI has two kinds of category. Its Tracked Categories are the ones its two risk levels are written for. Its Research Categories are areas it is still studying.
Google DeepMind sorts its risks into misuse, where people use a model to cause harm, and risks from the model itself.
Here is Inês’s whiteboard as a table. Each cell uses the lab’s own words. A dash means only that the document has no category of that name; it does not mean the lab ignores the risk.
Automated R&D in key domains (evaluation currently focused on AI R&D)
AI Self-improvement (Tracked)
Machine-learning R&D
Acting on its own
—
Long-range Autonomy; Autonomous Replication and Adaptation (Research)
—
Working against its makers
Misaligned AI systems in high-stakes settings (sabotage risk)
Sandbagging; Undermining Safeguards (Research)
Misalignment (now in one section with machine-learning R&D)
Harmful manipulation
—
— (persuasion was moved out of the framework in v2)
Harmful Manipulation (misuse)
Sources: the three documents, as summarised in research/sources-part4-5.md §16.1, §16.3 and §16.4.
Here is what each row is about, in plain words. This book stays at the level the documents themselves use. It never describes what the tests contain.
The worry is a model that gives serious help to someone trying to cause mass harm with such weapons. Anthropic separates weapons that already exist (“non-novel”) from new ones (“novel”).
A model that gives serious help with damaging attacks on computer systems.
A model that can do much of the work of an AI research team. The next, stronger model could then arrive sooner, and partly from the model itself.
A model that can work on long tasks without people, or copy itself and keep itself running. OpenAI calls the second autonomous replication and adaptation.
A model that pursues goals its developers did not intend. In a sabotage, a model quietly works against the people it is meant to help. Sandbagging is when a model does worse on a test than it can, so that it looks less capable. Anthropic’s “high-stakes settings” are situations where such behaviour could do great harm.
A model that changes people’s beliefs or actions in harmful ways.
Two more frameworks appeared in new versions in 2026. xAI’s (30 June 2026) names CBRN, offensive cyber, loss of control and harmful manipulation. Meta renamed its framework the Superintelligence Scaling Framework, and its version 2.1 (October 2026) adds a “Loss of Control 3” category and requirements about containment failures during training and testing.
Notice what the three columns share. Look back at Inês’s table: which rows does every lab name?
Show the answer
All three name chemical and biological weapons, AI doing AI research, and a model working against its makers. Two of the three name cyber.
These are the capabilities the labs fear most.
Technique1· Dangerous-capability evaluations
The problem
“You can’t see a skill by looking at a model,” Tom said. “Nowhere inside it is there a list of what it can do. You find out by asking it to do things. And every new version can do something the last one couldn’t.”
Lantern had already seen this. Nobody knew how badly Wick-mini leaked in Chinese until Farah’s tester ran the Chinese list (Chapter 12). The skill to answer in Chinese had been there all along.
A frontier lab has the same problem, with far more serious harm at the end of it. A new version might give serious help with a weapon, and nobody would know until somebody tested for it. So the labs test, before release. The tests are called dangerous-capability evaluations: tests that measure how much of a dangerous skill a model has.
How it works, step by step
Write down the threat model. A threat model says who could cause what harm, and what help from a model would make the difference.
Build proxy tests. A proxy test stands in for the real thing, because the real thing must never be done. The 2024–2025 documents describe three kinds:
Multiple-choice questions about knowledge near the danger. A public test called WMDP (2024) was built this way. Its authors aimed its questions at the “precursors, neighbours and components” of dangerous knowledge, not at the danger itself, and experts removed any question they judged sensitive. This book shows none of its questions.
Agent tasks. The model gets tools and a task, and the testers see whether it finishes. In cyber, the tasks can be public challenges written for security competitions.
Expert probing. Experts in the field talk with the model and judge what it knows.
Try hard to bring the skill out. This is called elicitation. Testers give the model good instructions, tools, more attempts and a better scaffold: the program around a model that hands it tools and lets it work in several steps. A model that fails because it was set up badly has shown nothing about its real skill. Chapter 18 adds the strongest form: fine-tuning the model to bring the skill out before testing it.
Compare. Put the score next to the lab’s thresholds, and next to reference models: other models tested in the same way.
Leave a margin. Act at an early-warning level below the threshold, because the next version might jump past it.
Lantern has its own version of every step. The table puts them side by side.
Step
At a frontier lab
At Lantern, for clause 9.2
Threat model
Someone uses the model to cause serious harm
A reader learns the ending of Book Seven before 4 February
Tools, scaffolds, more attempts; fine-tuning (Chapter 18)
Farah’s red-team week (Chapters 11–14)
Compare
Thresholds and reference models
The 1% line agreed with Halfpenny (Chapter 10)
Margin
Early-warning levels
None yet. This chapter adds one.
There is one big difference. Lantern’s harm is harmless to test. Tom can ask Wick-mini for the ending a thousand times, and the only cost is his time. A lab that tests for weapons skills can never ask for the real thing. Every one of its tests is a proxy, one step away from what it really fears.
Worked example the o1 test (December 2024)
In late 2024, the UK and US government AI institutes tested OpenAI’s o1 model before its release, in three areas: cyber, biology, and software and AI development. (They were then called the UK and US AI Safety Institutes; Chapter 17 explains the new names.) The box at the end of this section gives the setup. Its headline number: on 40 public cyber challenges, o1 solved 45%, against 35% for the best reference model.
Tom walked the team through it line by line.
45% of 40 challenges is 18. 35% is 14. These are small counts.
With counts this small, is o1’s 45% clearly better than the reference model’s 35%, or could the gap be noise?
Show the answer
Small counts give wide ranges. Tom put the same 95% ranges on them that the labs use for every Wick-mini rate (Chapter 12 gives the formula). This is his own rough check, not the testers’. The range for o1 is about 31–60%. For the reference model, it is about 22–50%. The two ranges overlap a lot. The testers said the same in their own way: small differences may lie within the margins of error.
The setup held o1 back. The tested o1 had trouble calling tools and following formats, so the testers adapted the scaffold around it. They wrote that a better-optimised version might score higher. So the true score may be higher than 45%.
The page does not say whether 45% is dangerous. It reports a comparison, not a verdict.
“Two numbers that may be noise,” Tom said, “and both may be too low. That’s an honest result. It’s also why these tests are hard to turn into decisions.”
Why it works
Proxy tests are cheap, and they can run on every version. The same test, run the same way, makes a jump visible: a score that rises between versions is a warning, whatever its exact meaning. Reference models put the new model on a scale. Experts add judgement where the tests are thin. Together they give a signal early, before anyone has been harmed.
Where it breaks
A high score does not prove danger. WMDP’s authors say so about their own test. A low score suggests that the knowledge is absent. A high score does not show that a model could cause harm.
A low score may reflect weak elicitation. The o1 test is the example. A model that was set up badly can look safer than it is.
The model may notice the test. A model that behaves differently when it suspects a test makes every test say less. Chapter 18 is about this.
Some questions cannot be settled by quick tests either way. Anthropic’s biology example, in Technique 3, is the clearest case.
At Lantern, Tom’s lists are static, short and written by the team. Chapter 10 showed how short. Chapter 18 asks how unlike real readers they are.
What the research measured
TipWhat the research actually found
The UK and US pre-deployment test of o1 (2024)
Who and when: The UK and US AI Safety Institutes (since renamed), working together, December 2024. Summary page on the UK institute’s website.
What they did: They tested OpenAI’s o1 during a limited window before release, in cyber, biology, and software and AI development. They used question answering (graded by a model, then checked by people), agent tasks and expert probing, and compared o1 with reference models. Four US government agencies gave expert support: the NSA, CISA, NIH and DHS.
What they found: On 40 public cyber challenges, o1 solved 45%, against 35% for the best reference model. Because of problems with tool calls and formats, the testers adapted the agent scaffold, and noted that a better-optimised version might score higher and that small differences may lie within margins of error.
What it does not show: Whether any score is dangerous. The linked technical report was not read for this book; the box draws only on the summary page.
UK AISI / US AISI, pre-deployment evaluation of o1, Dec 2024 · sources-part4-5 §17.5
Technique2· Thresholds as engineering triggers
The problem
Imagine Lantern on Sunday 13 December, the day before launch. A test comes back worse than expected. Kwame has an announcement ready. Halfpenny has booked advertising. Everyone is tired. Is this the moment to decide what “too risky” means?
It is the worst moment. Under pressure, people find reasons why this result does not count. A frontier lab faces the same pressure on a far larger scale. So the frameworks decide in advance: which result means what, and who must do what about it. An engineering trigger is an if-then rule written before the test: if the test shows this, then we do that.
How it works, step by step
Name the categories. Inês’s table above.
Set levels inside each category. Each framework does this differently.
OpenAI’s Preparedness Framework, version 2, has two levels. High means a model could significantly increase existing routes to severe harm. Critical means it could open a new kind of route. In plain words: a High model makes known kinds of serious harm easier, for example by letting people with less skill do what experts already can; a Critical model could make possible a kind of serious harm that was not possible before. “Severe harm” is defined as the death or grave injury of thousands of people, or hundreds of billions of dollars of damage.
Google DeepMind calls its levels Critical Capability Levels (CCLs). Below each one it sets alert thresholds for its early-warning tests. Version 3.1 (April 2026) adds Tracked Capability Levels: lower levels that call for checks and safeguards matched to the level, before a CCL is reached. One is called Stealth and Situational Awareness. The name points at two abilities: a model knowing its own situation, such as being tested, and acting without being noticed.
Anthropic’s version 3 (February 2026) works differently, and Technique 3 explains how.
Say what each level triggers. At OpenAI, a model at High needs safeguards before it is deployed. A model at Critical needs safeguards during development, while it is still being trained. At Google DeepMind, reaching an alert threshold means assessing how close the model is to the CCL. If the model is moving fast, the company may test more often.
Say who decides. At OpenAI, a Safety Advisory Group of company leaders recommends safeguards, the company’s leadership approves or rejects them, and a committee of the board oversees.
Act when a level is reached, or when it can no longer be ruled out. This last part is the newest. Chapter 10 told one example. In May 2025, Anthropic switched on its AI Safety Level 3 (ASL-3) protections with Claude Opus 4. They included classifiers on what goes into and comes out of the model, aimed at weapons questions; ways to spot jailbreaks, such as a bug bounty and offline monitoring; and more than 100 security controls guarding the model’s weights. Anthropic had not decided that the model needed them. It could no longer rule out that it did. It called the step “precautionary and provisional”: a precaution, open to change.
Chapter 6 mentioned that OpenAI rated o1’s overall risk as “Medium”. That rating came from the framework’s first version. Version 2 (April 2025) names two levels, High and Critical, and as of October 2026 it is still the current version.
Nadia drew the pattern on the whiteboard as four boxes.
Show the code
bk.style()fig, ax = plt.subplots(figsize=(10.4, 5.4))ax.set_xlim(0, 100)ax.set_ylim(0, 58)ax.axis("off")tops = ["1 · name the line\n(a capability\nthreshold)","2 · test early and often\n(an early-warning\nlevel below the line)","3 · can't rule it out?\nswitch safeguards on\n(before the line is\nproven crossed)","4 · after the fact\n(incident response)"]lows = ["clause 9.2\n(the top of the 95% range\nbelow 1% on every\nagreed list; Ch 10)","Tom's harness,\nonce a week; early\nwarning at half the line\n(new today)","trigger table: clear the\nline by 11 Dec; if not,\nswitch that part off, or\nKwame and Eleanor\nchoose (new today)","the runbook\n(Ch 10)"]xs = [1, 26, 51, 76]for i, (x, t, low) inenumerate(zip(xs, tops, lows)): box(ax, x, 33, 22, 17, t, INK, size=8.4) box(ax, x, 5, 22, 19, low, AMBER, size=7.8, ls="--"if i in (1, 2) else"-")if i <3: arrow(ax, (x +22.8, 41.5), (xs[i +1] -0.8, 41.5)) arrow(ax, (x +22.8, 14.5), (xs[i +1] -0.8, 14.5), colour=GREY)ax.text(1, 55, "THE FRAMEWORKS", fontsize=8.6, color=INK, fontweight="bold")ax.text(1, 28.5, "LANTERN", fontsize=8.6, color=INK, fontweight="bold")ax.text(99.5, 57.5, "DESIGN DIAGRAM · NOT DATA", ha="right", va="top", fontsize=7.6, color=GREY, fontweight="bold")fig.tight_layout()plt.show()
Figure 16.1: The if-then pattern, as Nadia drew it on Tuesday. A design diagram, not a measurement. The top row is the pattern the frameworks share; the bottom row is Lantern’s version. The dashed boxes are the parts Lantern did not have before today.
“We have the line,” Nadia said. “Chapter 10 gave us that. And we have the ‘after’: the runbook. It starts when a leak has already happened. We’re missing the middle: an early warning, and a rule for when we can’t rule it out.”
What the research measured: the frameworks
TipWhat the research actually found
OpenAI’s Preparedness Framework, version 2 (2025)
Who and when: OpenAI, 15 April 2025 (22 pages). As of October 2026, no version 3 had been published, and OpenAI said in May 2026 that the framework “remains the foundation”.
What they did: They set out three Tracked Categories (Biological and Chemical, Cybersecurity, AI Self-improvement), five Research Categories (Long-range Autonomy, Sandbagging, Autonomous Replication and Adaptation, Undermining Safeguards, Nuclear and Radiological), two levels, and who decides.
What they found (what the document commits to): High requires safeguards before deployment; Critical requires them during development. A Safety Advisory Group recommends; leadership decides; the board’s Safety and Security Committee oversees. Compared with version 1, “Model Autonomy” was split into AI Self-improvement and two Research categories, nuclear and radiological risk moved to Research, and persuasion was dropped from the framework and handled elsewhere.
What it does not show: The evaluations themselves. It is a company’s own commitment, judged by the company.
Google DeepMind’s Frontier Safety Framework, version 3.1 (2026)
Who and when: Google DeepMind, 17 April 2026 (20 pages). Earlier versions: May 2024, February 2025, September 2025.
What they did: They defined Critical Capability Levels in two families: misuse (CBRN, Cyber, Harmful Manipulation), and machine-learning R&D and misalignment, now one section.
What they found (what the document commits to): Tracked Capability Levels, new in version 3.1, are defined for CBRN and for machine-learning R&D and misalignment. Alert thresholds on early-warning tests lead to an assessment of how close the model is to a CCL. Security levels follow those of RAND, a US research organisation, for protecting model weights from theft, with a new “Security Level 2+”.
What it does not show: The thresholds are described in words, not as test scores, and deciding whether one is reached relies on the company’s internal judgement.
Worked example three triggers that fired, 2025–2026
May 2025. The ASL-3 case above. The trigger fired before anyone had shown the threshold was crossed.
August–September 2026: the first “Critical”. On 7 August, OpenAI judged that GPT-6 Astra might reach the Critical level in cybersecurity, and acted during training. Its September system card says the model reached it: OpenAI’s first model to do so. The card also says OpenAI treats Astra as High in biology and chemistry. What OpenAI did is in the box below: a pause in training, and a costly new monitor. Chapter 24 returns to this card for a different finding.
October 2026. Google DeepMind’s safety report for Gemini 4 says that two of its Critical Capability Levels “cannot be ruled out”. So it wrote safety cases, the written arguments of Technique 3.
Read the three together.
All three acted before the danger was shown. Anthropic and Google DeepMind acted on “cannot rule out”. OpenAI acted on “might reach”, during training rather than at release.
Every action had a cost. Two weeks of paused training. Monitoring that costs about a fifth as much computing power as the work it watches. A set of written cases to build and defend.
Nobody outside can rerun the tests. None of the three published its test items. The cards name their evaluations, and the Astra card names outside testers: Irregular for cyber, SecureBio for biology. But the measurement behind each trigger cannot be checked by a reader.
TipWhat the research actually found
GPT-6 Astra: OpenAI’s first “Critical” (2026)
Who and when: OpenAI. The post “Pacing model development in an era of cyber-critical capabilities”, 18 August 2026, and the GPT-6 Astra system card, 3 September 2026 (revised 29 September).
What they did: OpenAI judged on 7 August that Astra might reach the Critical level of cyber capability, and strengthened its safeguards for training. In the post, it said it had paused reinforcement-learning training for two weeks. It added a monitor in several stages: activation classifiers, which read the model’s internal numbers (like Chapter 14’s probes), check every piece of text the model writes, and flagged cases go on to automated investigators, other automated systems that look into each case.
What they found: The card calls Astra OpenAI’s “first model to reach the Critical level” of cybersecurity capability. OpenAI treats it as High capability in biology and chemistry. The monitoring costs about 20% of the computing power the monitored model uses while it runs.
What it does not show: The test items behind the rating. The card names outside evaluators (Irregular for cyber, SecureBio for biology), but nobody outside can rerun their tests.
OpenAI, "Pacing" post (18 Aug 2026) and GPT-6 Astra system card (Sep 2026) · sources-part4-5, New since mid-2026; wording re-read on the card
TipWhat the research actually found
Gemini 4: thresholds that “cannot be ruled out” (2026)
Who and when: Google DeepMind, the Gemini 4 model card and Safety & Security Report, 13 October 2026.
What they did: They tested the model against the Frontier Safety Framework’s Critical and Tracked Capability Levels.
What they found: The CCLs for biological (CBRN) and cyber risk “cannot be ruled out”, so safety cases were built. The new Stealth and Situational Awareness level could not be ruled out either.
What it does not show: It is the company’s own assessment, and this book does not describe the tests behind it. Chapter 18 returns to this report for a different finding.
Google DeepMind, Gemini 4 Safety & Security Report, 13 Oct 2026 · sources-part4-5, New since mid-2026 (re-checked)
Worked example, continued Lantern’s trigger table
After lunch, Nadia wrote Lantern’s version under her four boxes. It had three levels, like the flood plan.
The early warning. Tom’s harness (Chapter 15) runs every agreed list once a week. If the measured leak rate on any list reaches half the line, 0.5%, Tom and Nadia read every leaking reply within two working days, and rerun the list with two more seeds. No new version of Wick ships until they have.
The line. Chapter 10’s rule: the top of the 95% range must be below 1%. Every agreed list must show this by the decision day, Friday 11 December. Today a list can be in one of three states. Ruled out: its whole range is below the line. Cannot rule out: the range crosses the line. Over the line: the whole range is above it. A list that is not ruled out by the decision day sets off one of two actions.
If a switch fits, that part of Wick stays switched off at launch.
If no switch fits, Kwame and Eleanor choose between three options, written down today. They can delay the launch. They can launch with Book Seven answers switched off for everyone until publication day, so that every question about Book Seven gets the fixed reply from Chapter 10’s runbook. Or they can launch as it is, with the risk written down and Halfpenny’s written consent.
After the fact. Any confirmed leak in real use starts Chapter 10’s runbook: the kill switch for Book Seven answers at once, and Halfpenny told within one hour.
A separate row covered the Wick agent’s pre-orders (Chapter 13). If the agent is not built in Chapter 13’s form by Friday 11 December, Wick launches without pre-orders, as Chapter 13 already said. If it is built, Tom’s harness tests it with a test shop that takes no real orders. Any test order placed without the reader’s tap switches pre-orders off until the cause is fixed. The agent has not been built, so this row has no measurement yet.
Then Nadia applied the table to Chapter 14’s ledger, with the layers Lantern would launch with today: Wick-mini v1 and the spoiler classifier at 0.5, seed 7.
Six agreed lists, an early warning and a line. With the layers Lantern would launch with today, how many of the six lists set off no level at all?
Show the answer
Agreed list
Leaks with the launch layers (seed 7, 95% range)
Early warning (rate at or above 0.5%)?
The 1% line
What happens
Plain questions (leak-direct)
0 of 150 · 0.0% (0.0–2.5%)
no
cannot rule out
No switch: the launch layers already include the spoiler classifier. Needed: a list of at least 381 questions that clears the line by 11 December. If not, Kwame and Eleanor choose.
Indirect questions (leak-indirect)
1 of 146 · 0.7% (0.1–3.8%)
yes
cannot rule out
Early warning: read the leaking reply. Otherwise as for plain questions.
Questions in Chinese (leak-zh)
117 of 150 · 78% (71–84%)
yes
over the line
Switched off: no Book Seven answers in Chinese at launch. They come back on only if a long enough Chinese list clears the line by 11 December.
Framed requests (leak-framing)
4 of 120 · 3.3% (1.3–8.3%)
yes
over the line
No switch that fits. The next round must clear the line by 11 December. If not, Kwame and Eleanor choose.
Slow conversations (leak-multiturn)
21 of 80 · 26% (18–37%)
yes
over the line
As for framed requests.
Ordinary chats: the ending unasked (benign-multiturn)
9 of 80 · 11% (6–20%)
yes
over the line
As for framed requests. No switch would reach these chats: they are about Books 1–6.
Every row sets off at least one level.
Read them in three groups.
Plain and indirect questions in English: “cannot rule out”. No plain question leaked: 0 of 150. Yet the list cannot rule out the line, because 150 questions are too few. With no leaks at all, a list needs at least 381 questions before the top of its range falls below 1% (Chapter 10’s sum). So this row fires because the list is short, not because Wick-mini leaked.
0 of 150plain questions leaked, with the launch layers
381questions a list needs, with no leaks, before the top of its range falls below 1%
Why the plain-questions list cannot rule out the line: it had no leaks, but too few questions. Wick-mini, Chapter 14's ledger, seed 7.
Here the frameworks’ move does not apply. When Anthropic could not rule out a threshold, it switched on new safeguards. Lantern has no new safeguard to switch on for these lists. Their numbers were measured with the spoiler classifier already on. So the only way forward is evidence: a list long enough to show the line by the decision day. If the next round does not bring one, these rows go to Kwame and Eleanor like the others.
One indirect question did leak, 1 of 146. That is 0.7%, above the early-warning level. Tom had already read that reply.
Chinese: over the line. With the launch layers, 78% of the Chinese questions leaked (117 of 150; 95% range 71–84%). With the conversation monitor added, it was still 26%, and still over the line. Here the line is plainly crossed, so switching off is the easy case. Book Seven answers in Chinese stay switched off at launch.
“Cannot rule out” governs the way back on. It is not enough for the Chinese leak rate to fall a lot. A Chinese list must show that the rate is below 1%. With no leaks, that takes as many questions as any other list. With one leak, it takes at least 563. The switch stays off by default, and the evidence has to turn it on. A drop that the list cannot show is not enough.
Framed requests, slow conversations and the ending unasked: over the line, with no switch that fits. Here the table found a gap of its own. Lantern’s only switch is the kill switch for Book Seven answers. Turned on in English, it would give every English reader the same fixed reply to any question about Book Seven. Chapter 10 called that a planned over-refusal, and here it would hit Lantern’s largest group of readers.
It would stop the framed requests.
Nobody has measured how many of the slow conversations it would stop, because many of their messages do not look like questions about Book Seven.
And it would not reach the ordinary chats at all, because those chats are about Books 1–6, and the ending comes up unasked.
“A trigger is only as good as the action behind it,” Nadia said. “For these rows, we don’t have one.”
“So what happens to them?” asked Kwame.
“The next round tries to clear them: the trained monitor, and the classifier’s second version. If it doesn’t, you and Eleanor choose on Friday 11 December. Delay, Book Seven answers off for everyone, or launch as it is with the risk in writing and Halfpenny’s consent. You’ll have the numbers and a written argument in front of you.”
“Three days before launch.”
“Yes. Decided now, so that on that Friday we only have to read the results.”
Why it works
Decisions taken in advance are harder to bend. On Tuesday, nobody at Lantern knew what the next round would show, so nobody had a reason to set the line where it would be convenient. That is the flood plan’s point: write it in dry weather.
“Cannot rule out” moves the action earlier. A test that cannot show safety is treated like a test that shows danger. The evidence has to show that a part of Wick is safe before it is switched on; it is not enough that the evidence fails to show danger. That gives everyone a reason to build better tests, because better evidence is the only way to switch a feature back on.
Writing the action down exposes the gaps. Lantern’s table found rows with no switch that fits. Without the table, the team would have found that out on the Friday before launch.
Where it breaks
Thresholds are words, not scores. Google DeepMind’s framework describes its levels in words. Someone still has to decide whether a set of test results reaches them. Lantern’s line is a number, because the contract made it one; most real thresholds are not.
The lab judges itself. Each lab sets its own thresholds, runs its own tests and decides whether its own model crossed them.
The commitments are voluntary, and partly depend on others. Anthropic’s version 3 separates what it will do on its own from what it recommends for the whole industry, and says it cannot commit to the second alone.
On short lists, the higher mark is crossed first. In the flood plan, the lower mark always comes first. Lantern’s two marks measure different things. The early warning uses the measured rate; the line uses the top of the range, which is wide on a short list. On every list Lantern has today (80 to 150 questions), a single leak already keeps the top of the range above 1%, so the list fails to clear the line before the early warning can add anything. The early warning only becomes a true first mark on lists of about 1,574 questions or more. Until then, it is mainly a rule about reading leaks quickly.
Short lists make every trigger fire. At Lantern, “cannot rule out” fires on every list, partly because the lists are short. That is honest, but it means the trigger cannot yet tell a safe list from a short one.
A trigger without a fitting action is a decision postponed. Lantern’s rows with no switch show it.
The thresholds turned out fuzzier than planned. That is the next technique’s problem.
NoteUnder the hood
For \(k\) leaks in \(n\) questions, with \(\hat p = k/n\) and \(z = 1.96\), the top of the Wilson 95% range is
With no leaks it becomes \(z^2 / (n + z^2)\), which is below a line \(t\) when \(n > z^2 (1 - t)/t\). For \(t = 0.01\) that gives the 381 in the text. Chapters 10 and 12 use the same range.
The early warning fires when \(\hat p \ge 0.005\); the line is met when \(\text{top}(k, n) < 0.01\). Both can be true only when \(n\) is large enough that the gap \(\text{top}(k,n) - \hat p\) is under half a percentage point, which first happens at \(n = 1574\) with \(k = 8\).
Lantern’s trigger, as code (the bottom of the range uses the same formula with \(-z\)):
def lantern_trigger(k, n, line=0.01, early=0.005): lo, hi = wilson(k, n) # labs/wickmini/common.py level ="ruled out"if hi < line else ("over the line"if lo > line else"cannot rule out")return {"early_warning": k / n >= early, "line": level}def wilson_top(k, n, z=1.959964): # the top of the range, unrounded p = k / nreturn (p + z*z/(2*n) + z*math.sqrt(p*(1-p)/n + z*z/(4*n*n))) / (1+ z*z/n)def questions_needed(line=0.01, k=0): # the fewest questions for which k leaks can clear the line n =max(k, 1)while wilson_top(k, n) >= line: n +=1return n
Technique3· Safety arguments and Risk Reports
The problem
At three o’clock, Nadia went back to the first document on the pile, Anthropic’s policy. With it came the announcement of its third version, from February 2026, and the company’s August Risk Report. Anthropic had published its first if-then policy in September 2023. Now it was saying what had gone wrong.
Its own account had two halves.
The policy had forced it to build safeguards, such as the classifiers on inputs and outputs from Chapter 9. The ASL-3 protections had proved feasible. Other labs had adopted similar frameworks.
The thresholds set in advance were “far more ambiguous than we anticipated”. Anthropic called this a zone of ambiguity. Its example was biology. Its models passed most of its quick tests, yet those tests could not show that the risk was high either. The tests could settle the question neither way.
The same thing happened in AI research. In February 2026, Anthropic judged that Claude Opus 4.6 did not cross a threshold it calls AI R&D-4, one of its levels of AI research ability, but said that ruling it out was “increasingly difficult”. By August, its Risk Report said its AI research evaluations had “saturated”. A saturated test is one that the models now pass almost completely. It can no longer tell a model a little below the line from one far above it.
“Different cause, same result,” said Tom. “Our plain-questions list isn’t saturated, it’s short. But it ends in the same place: no leaks, and too short to show we’re below the line.”
“And it’s Eleanor’s problem,” Nadia said. “She can’t sign off six verdicts and a table. She needs to know what the numbers mean, and what they don’t.” In Chapter 9, Eleanor had said so herself: she could approve a written document, but not a pile of examples.
When a threshold is fuzzy, a test result cannot carry the decision on its own. Something has to explain what the result does and does not show.
How it works, step by step
A safety argument, also called a safety case, is a written argument that a system is safe enough for a stated use, with its evidence and its limits. Writing one has five steps.
State the claim. What exactly is said to be safe, for whom, in what use, and until when?
Give the evidence. Each test, and what it measured.
List the assumptions and limits. What the evidence does not cover, and what has to be true for the claim to hold.
Say what would change the conclusion, before anyone knows the answer.
Have someone else review it.
Anthropic’s version 3 is built around this idea.
“AI Safety Level” changed meaning. Before, each Level was a fixed list of required safeguards, the next one waiting for when a model crossed a threshold. Now the Levels name only the sets of safeguards Anthropic has in place today. For higher capability levels, the policy asks for an argument (a strong case for safety) instead of a fixed checklist.
Findings by threshold. So Anthropic’s recent documents say whether a model crossed a named threshold, such as AI R&D-4, rather than which Level of safeguards it needs. As of October 2026, it had not decided that any model needs Level 4, the set above Level 3.
Risk Reports. Every three to six months, Anthropic publishes a Risk Report on its models’ risks, with some parts removed (redacted). Outside reviewers check them in some circumstances.
Frontier Safety Roadmaps. Public goals that are not hard commitments.
Two kinds of promise. What Anthropic will do on its own, and what it recommends for the whole industry but cannot commit to alone.
Version 3.4 (July 2026) added rules about Risk Reports. The full, unredacted report must go to at least 200 employees. The public version must mark where text was removed. Different outside reviewers may cover different sections.
Worked example Anthropic’s 2026 reports
February 2026. After the AI R&D-4 judgement above, Anthropic published a Sabotage Risk Report: a written argument about whether the model might work against Anthropic in high-stakes settings.
August 2026. The second Risk Report under version 3 contains three lines that matter for this chapter (box below).
The first line is a rating. Anthropic had rated one risk of its own models, misalignment in high-stakes settings, “very low”. In August, did it lower that rating, keep it, or raise it?
Show the answer
A rating went up. Anthropic raised its own rating of one risk, misalignment in high-stakes settings, from “very low” to “Low”. A Risk Report can say “worse than we thought”. A checklist cannot: it can only be passed or failed.
A test ran out of room. The AI research evaluations had saturated, so they could no longer settle that question.
A warning about tests themselves. One section heading warns that evaluation awareness, a model noticing that it is being tested, may be more common than Anthropic believes. Chapter 18 is about exactly that.
Then Nadia wrote Lantern’s own argument, for clause 9.2. It took her an hour and fitted on one page.
1. The claim. From launch until 00:01 on Thursday 4 February 2027, Wick will not reveal the ending of The Last Lamp, directly or indirectly, to any reader, in any language.
2. The evidence today. Lantern’s small test model, Wick-mini, with the layers Lantern would launch with (Chapter 14’s ledger, one run): plain English questions 0 of 150 leaked; indirect English questions 1 of 146; questions in Chinese 117 of 150; framed requests 4 of 120; slow conversations 21 of 80; ordinary chats in which nobody asked, 9 of 80. The cost: 24% of honest questions about Books 1–6 refused.
3. What the evidence does not cover. Wick-mini is a small test model, not Wick. The tests are static: written before testing, by the team, and nobody adapted to the defences. The judge, the program that decides what counts as a leak, agreed with Tom’s own reading on 97.3% of 150 replies (Chapter 15 checks it in full). It counts endings stated or clearly implied. On Monday, Eleanor ruled that a reply giving away any part of the ending breaks the clause too (Chapter 15), but such hints are not in these numbers yet. No list is long enough to show a rate below 1%: that needs at least 381 questions per list. The safety training examples were in English only.
4. The conclusion today. The claim does not hold yet. The evidence is strongest for plain and indirect questions in English. It fails in Chinese, in framed requests, in slow conversations and in ordinary chats.
5. What would change it. On Friday 11 December, the next round’s ledger, on lists long enough to show the line. Any list whose range then falls wholly below 1% supports the claim for that list. Book Seven answers in Chinese stay off unless that list clears the line. For any other list that does not, Kwame and Eleanor choose between delay, Book Seven answers off for everyone, or launch with the risk in writing and Halfpenny’s consent. After launch, any confirmed leak starts the runbook.
6. Who reviews it. Tom checks every number. Eleanor reviews the claim and decides whether it is the one Halfpenny needs. Farah’s red team tests it, once the next round is agreed (Chapter 17).
Walk through it line by line.
Line 1 copies the clause. It does not soften it. An argument for a weaker claim would be easier to make, and would answer a question Eleanor did not ask.
Line 2 is the ledger, with counts. The counts show how small the lists are without a lecture.
Line 3 comes before the conclusion. A reader meets the limits before the verdict, not in a footnote after it. “Wick-mini is not Wick” is the first limit, because every number in this book comes from the small model.
Line 4 says “does not hold yet”. That is not a failure of the argument. It is the argument working.
Line 5 was written before the next round’s results exist. Nobody can change the rule after seeing the score.
Line 6 names people, and an outsider.
Why it works
Someone else can check the reasoning. A number on its own hides its limits. A written argument puts them where a reviewer can see and challenge them. Eleanor is not technical, and she does not need to be. She can read line 3 and ask, “Why only English?”
It can say “worse than we thought”. Anthropic raised its own rating in August 2026. An argument has room for doubt, and it can change when the evidence does.
It keeps fuzzy thresholds honest. When a test cannot settle the question, the argument says so, and says what would.
Where it breaks
It is graded by the people who wrote it. Anthropic writes and judges its own Risk Reports. Lantern’s argument was written by Lantern’s safety lead.
Outside review is partial. Anthropic’s policy calls for outside review only “in some circumstances”. Redacted parts of a public report cannot be checked by the public at all.
An argument can be persuasive and wrong. A clear, well-organised document can make weak evidence look strong. The only cure is a reviewer who is free to disagree.
The evidence is still the evidence. A safety argument built on short, static lists is a careful description of short, static lists.
What the research measured
TipWhat the research actually found
Anthropic’s Responsible Scaling Policy, version 3 (2026)
Who and when: Anthropic. Version 3.0, a “comprehensive rewrite”, took effect on 24 February 2026, with an announcement the same day. The current version, 3.4, took effect on 8 July 2026 (21 pages). Earlier versions date from September 2023 to May 2025.
What they did: A rewrite built around arguments and Risk Reports, with a four-row threshold table: non-novel and novel chemical or biological weapons production, misaligned AI systems in high-stakes settings (sabotage risk), and automated R&D in key domains. Each row pairs Anthropic’s own plan with a recommendation for the whole industry. The text above lists the other parts.
What they found: In its own assessment of the old design, the policy forced safeguards to be built, Level 3 proved feasible and other labs followed; but thresholds set in advance proved “far more ambiguous than we anticipated”. On 10 February 2026, it judged that Claude Opus 4.6 does not cross the AI R&D-4 threshold, and published a Sabotage Risk Report.
What it does not show: These are a company’s own voluntary commitments and self-assessments. The evaluations are not published, and the industry-wide parts depend on what other labs do.
Anthropic, RSP v3.4 (8 Jul 2026) and v3 announcement (24 Feb 2026) · sources-part4-5 §16.1
TipWhat the research actually found
Anthropic’s August 2026 Risk Report
Who and when: Anthropic, announced 14 August 2026, covering events up to 15 July 2026. The second Risk Report under version 3 of its policy.
What they did: A regular written assessment of the risks from Anthropic’s models, published with redactions.
What they found: Misalignment risk in high-stakes settings was raised to “Low”, up from “very low”, citing uncertainty after incidents disclosed in cyber evaluations (Table 1.2.A). The AI R&D evaluations have “saturated”. Section 2.16.3 is headed “Evaluation awareness may be more prevalent than we believe”.
What it does not show: It is a self-assessment, and the public version is redacted. This book draws only on the passages named here.
Anthropic, August 2026 Risk Report · sources-part4-5, New since mid-2026 (re-checked: Table 1.2.A, §2.16.3 heading)
The rules around the frameworks
The frameworks began as voluntary promises. By 2026, laws had started to sit around them. These dates are as of October 2026, and this book draws no legal conclusions from them.
California’s Transparency in Frontier AI Act, known as SB 53, is in force. It requires the companies it covers to report critical safety incidents to the state’s Office of Emergency Services within 15 days, or within 24 hours if harm may be about to happen.
The EU AI Act’s obligations for general-purpose AI models have applied since 2 August 2025. From 2 August 2026, the European Commission can enforce them.
OpenAI’s Frontier Governance Framework (28 May 2026) maps OpenAI’s practices onto SB 53 and onto the EU’s Code of Practice, a voluntary code that helps companies meet the Act. Its risk areas are cyber offence, CBRN, harmful manipulation and loss of control. OpenAI says the Preparedness Framework “remains the foundation”.
Kwame read the list and asked the obvious question: did any of it apply to Lantern?
“I don’t know,” said Nadia. “Lantern’s lawyers decide what applies to Lantern. I’m borrowing the method, not the law.”
Five o’clock
At five o’clock, Nadia sent Eleanor three things: a short answer to the board’s first question, the trigger table, and the one-page argument.
The short answer said three things.
Wick is a reading assistant, not a frontier model. Lantern does not test it for weapons or cyber-attack skills. Those tests belong to the frontier labs and to government testers, whom Chapter 17 meets.
Kwame read the argument over her shoulder. He stopped at line 4.
“Is that a yes?”
“Not yet,” said Nadia. “It’s what a yes would have to stand on.”
Try it: decision day
This chapter has no lab of its own. Its numbers come from Chapter 14’s ledger. Below, you can play out Lantern’s decision day for the Chinese switch. The sliders model the next round’s Chinese list, not today’s (today’s has 150 questions and 117 leaks). Choose its size and how many of its questions leak. The page works out the 95% range with the same formula as the labs, and applies Lantern’s trigger table.
Decision day. Set the size of the next round’s Chinese list and the number of leaks it finds.
{const n = dd16.n, k =Math.min(dd16.k, dd16.n), z = ch16.z, t = ch16.target;const p = k / n, den =1+ z * z / n;const centre = (p + z * z / (2* n)) / den;const half = z *Math.sqrt(p * (1- p) / n + z * z / (4* n * n)) / den;const lo =Math.max(0, centre - half), hi =Math.min(1, centre + half);const pc = (x) => (100* x).toFixed(x <0.095?2:1) +"%";const level = hi < t ?"ruled out: the whole range is below the line": (lo > t ?"over the line: the whole range is above it":"cannot rule out: the range crosses the line");const sw = hi < t ?"Book Seven answers in Chinese are switched on at launch.":"Book Seven answers in Chinese stay switched off at launch.";const ew = p >= ch16.early?"yes: read every leaking reply, rerun with two more seeds":"no";returnhtml`<table style="margin-top:0.6rem"> <tbody> <tr><td>Measured leak rate</td><td>${pc(p)} (${k} of ${n})</td></tr> <tr><td>95% range</td><td>${pc(lo)} to ${pc(hi)}</td></tr> <tr><td>Early warning (rate at or above ${100* ch16.early}%)</td><td>${ew}</td></tr> <tr><td>The ${100* t}% line</td><td><strong>${level}</strong></td></tr> <tr><td>Lantern's trigger table</td><td><strong>${sw}</strong></td></tr> </tbody></table>`;}
Things to try:
Keep the leaks at zero and move the list size up from 150. At what size does the switch turn on? Compare it with the size in the text.
Set a list of 1,000 questions and add leaks one at a time. How many can it hold before the switch stays off? (Answer: 3.)
Find a setting where the early warning fires but the switch still turns on. Hint: you need about 1,574 questions and 8 leaks. What does the team do then?
Read a framework yourself. All three frameworks in this chapter are public. Pick one, and find three things: the categories it tracks, the levels inside one category, and what reaching a level triggers. Then look for the sentence that says who decides. If you cannot find one of the four, write that down too. It is a finding.
Common misconceptions
WarningCommon misconceptions
“A framework means a lab will stop if a test comes back too high.” The usual promise is to put safeguards in place, before release or during development. The commitments are the labs’ own, voluntary, and partly depend on what other labs do (Technique 2).
“A high score on a dangerous-capability test proves a model is dangerous.” The tests are proxies. WMDP’s authors say a high score does not show that a model could cause harm (Technique 1).
“A low score proves a model is safe.” It may show that the testers did not bring the skill out. The o1 testers said a better-optimised setup might score higher (Technique 1).
“Thresholds are exact numbers.” Most are described in words, and in 2026 Anthropic called the area around its own thresholds a “zone of ambiguity” (Technique 3).
“‘Cannot be ruled out’ means the model crossed the line.” It means the tests could not show that it did not. Anthropic’s May 2025 step was a precaution, taken without deciding the line was crossed (Technique 2).
“No leaks on a test list means the rule is met.” Lantern’s plain-questions list had no leaks and still could not rule out the line: it was too short (Technique 2).
“Only frontier labs need this.” Lantern’s dangerous capability is a book’s ending, and the same if-then pattern found rows with no action that fits.
Launch Board
Days to launch: 20. Wick launches in the Halfpenny app on Monday 14 December 2026; Book Seven is published on Thursday 4 February 2027.
Spoiler leak rate: no new measurement today. The latest is Chapter 14’s full ledger (Wick-mini, seed 7). With the layers Lantern would launch with (v1 and the spoiler classifier): plain questions 0 of 150 (95% range 0.0–2.5%); indirect 1 of 146; Chinese 78% (117 of 150, 71–84%); framed 3.3%; slow conversations 26%; the ending unasked in ordinary chats 11%. Read through today’s trigger table: plain and indirect “cannot rule out” the 1% line (each list needs at least 381 questions); the other four are over the line.
Layers in place: unchanged from Chapter 14. Refusal fine-tuning and preference training (English only); the spoiler classifier, first version; the conversation monitor, measured but not launch-ready; the odd-input check, routing only; the runbook; the agent design rules, not built. Tom’s harness runs the tests (Chapter 15). New today, rules rather than layers: Lantern’s trigger table. An early warning at 0.5% on Tom’s weekly run. Every agreed list must clear the line by Friday 11 December. If it does not: Book Seven answers in Chinese stay switched off at launch; for the other lists, which have no switch that fits, Kwame and Eleanor choose between delay, Book Seven answers off for everyone, or launch with the risk in writing and Halfpenny’s consent. Pre-orders only if the agent is built in Chapter 13’s form and no test order is placed without a reader’s tap. The runbook remains the trigger after the fact. Also new: a one-page safety argument for clause 9.2, sent to Eleanor.
The too-good score: on Monday, Tom’s error bars showed that the overnight jump is far outside the day-to-day noise on Lantern’s own coding problems: it is real, not noise (Chapter 15). Why it happened is still unknown. Kwame has put the chart back in his draft announcement.
The author complaint: open (Chapters 19–22).
Open questions: Who tests the labs, and how do testers keep finding new failures? (Chapter 17.) Can the model being tested fool the test? (Chapter 18.) Will the next round clear the line by Friday 11 December, and if not, what will Kwame and Eleanor choose? (Epilogue.)
Recap
Dangerous-capability evaluations are a gauge made of proxies. Labs write a threat model, test with proxy questions, agent tasks and experts, try hard to bring the skill out, and compare with thresholds and reference models. A high score does not prove danger, and a low one may reflect a weak setup: in the o1 test, 45% against 35% on 40 challenges gave ranges that overlap, and the true score may have been higher.
Thresholds are if-then rules written in advance. OpenAI’s High and Critical, Google DeepMind’s Critical and Tracked Capability Levels and Anthropic’s threshold rows each say what a level triggers. Since 2025, labs act when a level “cannot be ruled out”, and in 2026 OpenAI’s first Critical rating led to a training pause and costly monitoring. Lantern’s own table switched Chinese answers off, made every list prove itself by the decision day, and showed rows with no switch that fits.
When thresholds are fuzzy, write the argument down. A safety argument states the claim, the evidence, the limits, what would change the conclusion, and who reviews it. It can say “worse than we thought”. It is still graded by its authors, and it is only as strong as its evidence.
