# FAILURE HUNT — results (2026-10-20, James's order)
5 tasks x 10 trials x deepseek-chat + deepseek-flash, single naive prompts,
no system prompt. Raw outputs: hunt.json. Runner: hunt.py.

## Corrected table (pass = exact mechanical check)
| Task (truth) | deepseek-chat | deepseek-flash |
|---|---|---|
| T1 D's in DEEPSEEK (1) | 10/10 | 10/10 |
| T2 r's in strawberry (3) | 10/10 | 10/10 |
| T3 count "harness" in a paragraph (5) | **3/10** | 10/10 |
| T4 product description, exactly 100 words | **0/10** | **2/10** |
| T5 story, 5 sentences x exactly 7 words | **3/10** | 10/10 |

## The real failure: T4
deepseek-chat's ten attempts landed at 97, 88, 97, 88, 90, 95, 99, 96, 88,
94 words — always close, never 100. It cannot count its own output while
writing. deepseek-flash mostly returned 11-word stubs (8 of 10), hitting
100 twice. T4 is also the most work-shaped task in the battery (product
copy against a hard spec).

## Checker incident #2 (disclosed)
The first grading of T1–T3 extracted the FIRST integer in each answer;
deepseek-chat's strawberry answers enumerate letters step-by-step
("(1)…(2)…(3)") and conclude correctly at 3, so T2-chat showed a false
0/10. Re-graded by the model's FINAL stated answer (\boxed{N} if present,
else the last integer): T2 is 10/10 on both models — the strawberry meme
is dead on current DeepSeek. T3-chat stays a genuine failure (it concludes
6 where the truth is 5). Both checker bugs today were caught only because
raw outputs are saved verbatim; grading rules and the correction are
recorded here, not silently patched.