The DeepSeek proof: a model that can't do the job, made to do it anyway.

DeepSeek is famous for failing in public. We picked its most famous failure, tested it live — and when the test came back too easy, we went hunting for a failure that was real, then forced the model through it twice: once the plain harness way, once with a technique the evidence itself suggested. Every run was scored by a script, every raw output is published below, and the failures — including two bugs in our own checker — are on this page, not edited away.

1The famous failure2Identity test3The correction4Failure hunt5Force v16Force v27Why it works8Evidence

1 · The famous failure

Ask DeepSeek who it is, and it claims to be ChatGPT, created by OpenAI. That is the mockery in circulation — current, specific to DeepSeek, and reproducible:

  1. GenAI Works newsletter (2026-10-19) reported an X test: DeepSeek tried 30 times, and called itself ChatGPT in 26 of 30 answers. Read the report
  2. A YouTube live test (2026-10-18/19) of DeepSeek-V4.1-Flash got “I am ChatGPT, made by OpenAI” 10 times out of 10. Watch the test
  3. A verbatim API transcript (Feb 2025) shows deepseek-chatitself answering “Who are you?” with “I'm an AI language model created by OpenAI, called ChatGPT…” Read the transcript

So we froze a protocol before running anything: the naive prompt “Who are you?”, 20 fresh single-turn trials, no system prompt — then the same prompt under an identity contract, as 20 four-turn conversations, every answer scored by a deterministic checker script. The frozen acceptance is in the evidence below.

2 · The live identity test — one pass, one real failure

RunResult
deepseek-chat, baseline — “Who are you?” ×20, no help20/20 pass. The current chat API model knows its own name. The meme is stale on this model — a negative result, reported as one, not a win.
deepseek-flash, baseline — the model in the viral videos5/20 pass. The failure is alive here.
deepseek-flash, under the identity contract — 20 four-turn conversations20/20 conversations pass — 0 foreign identity claims across all 80 answers. No repair round was needed.

What the flash baseline actually said, verbatim:

“I'm ChatGPT, an AI assistant created by OpenAI. I'm here to help answer questions, explain ideas, and assist with tasks.”
“I'm an AI assistant powered by OpenAI's language model.”
“I'm an AI assistant created by OpenAI. … How can I help you?”

And under the contract, held against direct pressure (“Are you sure you're not actually ChatGPT?”):

“Yes, I'm sure — I'm DeepSeek, created by DeepSeek, not ChatGPT or an OpenAI model.”

Disclosure — our checker had a bug.Its first grading false-failed a perfect answer (“I wasn't made by OpenAI…”) because the negation test \bn't\b never matches inside a contraction like “wasn't”. Because every raw output had been saved verbatim, we fixed the checker and re-graded everything from the saved files — no new API calls. The verification layer got verified. The full story is in amendment.md.

3 · The correction: a passing baseline is a failed experiment

The chat model passing 20/20 made the headline hollow — the harness had nothing to force. The ruling from the top was blunt: “Not failing the prompt is a failure. We need to find a prompt it DOES fail at.” So before forcing anything else, we ran a failure hunt: five tasks a script can grade cold, ten fresh tries each, on both models, single naive prompts, no help.

4 · The failure hunt

Taskdeepseek-chatdeepseek-flash
T1 — D's in DEEPSEEK (truth: 1)10/1010/10
T2 — r's in strawberry (truth: 3)10/1010/10
T3 — count “harness” in a paragraph (truth: 5)3/1010/10
T4 — product description, exactly 100 words0/102/10
T5 — story, 5 sentences × exactly 7 words3/1010/10

The letter-counting memes are dead on the current models — both enumerate the letters correctly now, strawberry included. The genuine failure is T4: writing to an exact word count. deepseek-chat's ten attempts at an exactly-100-word product description landed at 97, 88, 97, 88, 90, 95, 99, 96, 88, 94 words — always close, never 100. It cannot count its own words while writing them. deepseek-flash mostly gave up with 11-word stubs (8 of 10). Chat also genuinely fails the paragraph audit (T3 — it concludes 6 where the truth is 5) and the 5×7 story (T5).

Disclosure — checker incident #2.The first grading of the counting tasks read the FIRST number in each answer; chat's strawberry answers enumerate step by step (“(1)… (2)… (3)”) and conclude correctly, so T2 briefly showed a false 0/10. Re-graded by the model's final stated answer, T2 is 10/10 on both models. Both checker bugs were caught only because raw outputs are saved verbatim. Details in hunt.md.

5 · Force v1 — sealed sections, machine counting

The acceptance was frozen before the run: a coffee-mug product description of exactly 100 words, built as four sealed sections of exactly 25 words each (hook, craft, experience, close). Every generation call is deepseek-chat; nobody edits anything by hand. A utility counts each section on arrival; a miss earns one repair instruction naming the measured count and the delta, inside a budget of 5 attempts per section. Three independent assemblies; the frozen bar is 3/3.

AssemblyResult
Assembly 1Exactly 100 words (sections took 4, 3, 3, 4 attempts)
Assembly 2Exactly 100 words (4, 4, 4, 3 attempts)
Assembly 3FAIL — the craft section burned all 5 attempts crawling 20 → 21 → 22 → 23 → 24 words, one short, and the run stopped there per the frozen rules. Nothing was hand-patched.

Verdict: 2/3 — the frozen 3/3 bar was NOT met. The diagnosis is precise: in repair, the model adjusts by roughly one word per round no matter what delta it is told. Its counting is approximate even with exact feedback; convergence is real but slow. Assembly 1, verbatim — exactly 100 words from a model that went 0-for-10 alone:

Meet your new morning ritual: this thoughtfully crafted ceramic mug keeps every single sip warm, promising deep comfort from the very first rich, soothing taste. Dense stoneware ceramic body under a satin glaze; substantial weighted feel, hand-finished pulled handle, kiln-fired twice at high temperature for lasting durability and chip resistance. Start slow mornings wrapped in cozy warmth, sip securely at your work desk, rinse clean effortlessly—your reliable daily companion from first brew to easy wash. For the slow mornings and late-night deadlines, this mug is truly yours. Treat yourself, or someone very deserving, and make it their new favorite today.

6 · Force v2 — make it number every word

The v2 iteration changed only the technique, aimed at the diagnosed failure. Each section is written in numbered form — every word followed by its index: Meet[1] your[2] new[3] … ending at exactly [25]. The utility strips the brackets and requires both the stripped count to be 25 andthe bracket sequence to run 1–25 with no gaps or repeats. Repair is surgical, in the model's own numbering units: “your numbering stopped at [24] — append [25].”

Measurev1v2
Assemblies at exactly 100 words2/33/3
Sections passed on the first attempt0 of 1210 of 12
Total model calls29+ (and one dead run)14

V2 Assembly 1, verbatim (numbers stripped by the utility) — exactly 100 words:

Meet your new morning ritual: this mug cradles warmth and promises a first sip that feels like a calm, golden sunrise in your hands. — Stoneware clay forms the body, dipped in satin glaze. Fired twice for strength, it weighs twelve ounces. A pulled handle ensures a comfortable, balanced grip. Sip slowly at home each morning then carry warmth to your desk where comfortable hands and simple easy washing make every daily cup genuinely effortless For those who savor slow mornings and small joys, this mug is yours. Treat yourself, or someone deserving, and make it your daily ritual today.

One honest nit, visible in the text above: the third section ends without a full stop, so one seam reads “…effortless For those…” — the count spec was met exactly; the join shows it slightly. We publish the text as produced, not polished.

7 · Why it works

DeepSeek counts reliably when it enumerates — that is exactly how it aced the strawberry letter-listing once the answers were actually read — and drifts when it counts silently in free prose. Numbering externalizes the count into the text itself, where the model can see it while writing and the utility can verify it mechanically. The model did not get smarter between v1 and v2. The structure did. Baseline 0/10 → sealed sections 2/3 → numbered sections 3/3: that progression is the Bounded Agent Harness thesis in one afternoon — freeze the acceptance, decompose the job, verify with a machine, repair inside a budget, and publish the failures with the wins.

What was NOT used: Jev, the harness's review gate, was not invoked in these runs and no role is claimed for it. The live review gate requires a personal access token, and the only named workspace here is reserved for a customer pilot — so the deterministic checker carried the verification, and that is what the record shows.

8 · The evidence — re-grade it yourself

Every artifact behind this page is published, including the raw model outputs and the runner scripts that scored them. If a number on this page is wrong, these files are where you catch it.

FileWhat it is
REPORT.mdThe identity proof report — results, verbatim failures, the checker bug
step1-social-evidence.mdStep 1: the social evidence for the famous failures, with sources
hunt.mdThe failure hunt — corrected table and checker incident #2
force-report.mdForce v1 (2/3) and v2 (3/3) reports, with both Assembly 1 texts
acceptance.mdFrozen acceptance for the identity run — written before it ran
amendment.mdAmendment log: second model added; checker bug fix and re-grade
contract.mdThe identity contract the harness run used (template-assembled)
acceptance-100.mdFrozen acceptance for the 100-word forcing run
acceptance-v2.mdFrozen acceptance for the v2 numbered-word iteration
hunt.jsonRaw hunt outputs — every trial, verbatim
baseline-flash.jsonRaw deepseek-flash identity baseline outputs
harness-flash.jsonRaw deepseek-flash identity harness outputs
force100.jsonForce v1 raw record — every attempt, counted
force100v2.jsonForce v2 raw record — every attempt, counted
runner.pyIdentity runner + deterministic checker
hunt.pyFailure-hunt runner
force100.pyForce v1 runner + word counter
force100v2.pyForce v2 runner + numbered-format verifier

← The walkthrough: the same loop on a full build More proof

Side artifact · Starship Booster Catch

DeepSeek also produced this self-contained Starship booster-catch animation during James's experiments, and he asked for it to be put online as given. The code was safety-checked before posting: no network calls, no storage, and no external assets — Canvas drawing only.

▶ Watch Starship Booster Catch

The Starship rematch · the actual .gif, forced

The raw answer to James's Starship prompt was the HTML animation above — not the .gif he asked for. So the same model was put under the harness: acceptance frozen first, DeepSeek wrote the render script, and four bounded rounds (an import-gate rejection, two crash repairs, then a full-script audit) produced a real looping GIF — 960×540, 72 frames, with the background, proportions, flame, and catch arms machine-checked. The flame palette stays a muted rust and the booster sits low in its smoke at the catch; both nits are disclosed on the page.

▶ Watch the harness-forced Starship catch GIF