Seuil analysisA security control becomes a product constraint when it changes history, fallbacks, errors or billing. Those behaviors belong in acceptance tests.
What Anthropic announced on October 12
Anthropic described three defenses against unauthorized model distillation: controlling where reasoning can go, detecting extraction attempts with a two-stage classifier, and limiting or banning accounts that persist. Distillation here means using large volumes of one model's outputs to train another model to imitate its capabilities.
The company estimates that its methods reduced reasoning data harvested by attackers by around 90 to 99 percent. On its own platform, it says extracted reasoning tokens fell by a factor of 10 to 100 after the classifier was introduced.
These are Anthropic's measurements, not an independent audit. The company says its classifier catches all techniques from its September report on its most capable generally available models, with a false-positive rate below 0.1 percent.
Simple transcript controls did not stop extraction
Anthropic first summarized reasoning before showing it to users. In its experiment, a model trained on full Claude Opus 4.8 transcripts scored 80.4 percent on SWE-bench Verified, while one trained on summarized transcripts scored 71.8 percent.
Attackers then tested 12,000 prompt variations and moved protected reasoning between conversations, models and accounts. Anthropic says proxy networks controlled hundreds of thousands of fraudulent accounts. That scale explains why the new design binds reasoning to more of its original context.
The practical lesson is narrow: do not design a feature around extracting hidden reasoning. Ask for an answer, sources, confidence signals or a structured explanation that the product can display and verify.
Preserved thinking makes conversation history append-only
Anthropic's preserved thinking checks which model produced a reasoning block and whether the system prompt, tools and preceding messages changed. Its documentation tells developers to send assistant turns back exactly as received and append new messages at the end.
The prefix check is on by default for API accounts created on or after August 31, 2026. Depending on configuration, a changed prefix can return a 400 error or cause invalid thinking blocks to be dropped.
Imagine an agent that inserts a fresh system instruction halfway through a conversation or removes a tool before retrying. The visible chat may look intact while preserved reasoning becomes invalid. Test the exact history builder, compaction path and tool loop used in production.
| Confirmed change | Observable effect | Product test |
|---|---|---|
| Prefix binding | Edited prior context can error or drop thinking blocks | Replay a multi-turn tool flow after compaction and steering |
| Model and account binding | A fallback may not read earlier reasoning | Switch model and organization, then verify continuity |
| Extraction classifier | A benign request can be blocked | Surface the refusal and preserve a safe recovery path |
| Charging for blocked requests | A failed user journey may still incur usage | Separate blocked cost from successful completion cost |
Model fallbacks can lose reasoning continuity
Claude models do not all read the same prior thinking blocks. The API may ignore or drop blocks that a fallback model cannot use, and some newer blocks are tied to the account or organization that produced them.
A support agent might switch models after a capacity error, then continue with the same visible messages. The final answer can still change because the fallback no longer has the same reasoning context. Record the selected model and whether blocks were transformed or dropped.
Define that behavior in the AI product requirements document. A fallback is not complete because the API returned 200. It is complete when the user's task continues with known context and permissions.
Blocked requests now affect reliability and cost
Anthropic says it resumed charging on September 24 for requests blocked before Claude responds in areas, including distillation, where the classifier's false-positive rate is below 0.1 percent. A legitimate request can therefore fail and still consume budget.
Do not merge these events into a generic provider error. Log the stop category, model, account, retry path, user-facing message and charged usage. Alert on a rate change rather than hiding repeated blocks behind automatic retries.
The recent Anthropic model actions analysis covers controls for what an agent does. This announcement adds a different concern: whether the provider will accept the reasoning flow that led to that action.
The reported gains are early, not permanent
Anthropic says it does not know how long the gains will last and expects a continuing contest with attackers. It also says these defenses have not yet faced a highly sophisticated and persistent attempt to circumvent them.
Legitimate users carry tradeoffs too. Preserved thinking required some integration changes, the classifier can block benign requests, and blocked requests can be charged. A product team should verify those costs without pretending it can independently validate Anthropic's security result.
What Claude API teams should test now
Use the AI product audit guide to turn one production flow into reproducible acceptance cases. If the integration needs a broader rebuild, compare responsibilities with the AI product development partner guide before changing providers or architecture.
- Send prior assistant turns back unchanged and append new messages only.
- Exercise compaction, context editing and tool changes in a long conversation.
- Switch the fallback model and account, then verify visible task continuity.
- Capture dropped-block transformations, stop categories and 400 responses.
- Show a useful recovery state when a legitimate request is blocked.
- Track blocked usage separately from successful task completion.
- Retest after model, SDK, prompt, tool or history-builder changes.