The Splitter at scale: we kept making the job bigger. The one-shot kept passing.

The setup

Demo A asked whether splitting a build into sealed parts lets a weak model do work it couldn't do in one shot — and the honest answer at four modules was "no proof either way": the one-shot passed everything too. A test that can't separate the two methods proves nothing, so under this slate's escalation rule we scrapped the small version and rebuilt the test around something we actually wanted: a complete, playable tower-defense browser game — Gatekeeper — and then made it harder twice more when the baseline refused to break.

Three levels of the same game, each frozen in writing before its runs: Level 1, 14 files, one map, 447 automated checks. Level 2 adds a second map, branching tower upgrades, armored enemies, and enemies that spawn more enemies when they die — 421 checks. Level 3 adds saving and reloading a complete in-progress game, mid-wave, projectiles in flight — 15 files, 458 checks.

The test — protocol frozen before any run

Same model in both arms (deepseek-flash, the cheap one). The baseline gets the complete spec in one prompt and writes all files in one response — five attempts per level. The split arm gets the same model writing each file alone, from a sealed brief carrying the shared contracts and nothing else — no part ever sees another part's code — three full builds per level, recombined mechanically.

The pre-registered bar — split passes at least 2 of 3 while the baseline passes at most 1 of 5 — was not met at any level. NOT PROVEN.

A follow-on arm, registered before its runs, asked the parity question on the final level: the same job, one-shot, by the strong model (deepseek-v4-pro, three attempts). It scored 0 of 3 — below the weak split's 2 of 3 and the weak one-shot's 4 of 5 on the identical checks. Its builds were compact and passed 21–22 of 23 check groups each, but all three skipped the same sentence of the spec (an unguarded UI bind that must tolerate a headless document). Small numbers, one task, one model family — reported as measured: on this build, the tier of the model did not predict the score; diligence on the spec's defensive clauses did.

What the split arm's failures actually were

Not weak parts. With one exception in nine builds, every part of every failed split build passed its own unit checks. The builds died at the seams, on single sentences the spec hadn't pinned:

• Level 1: one part called the gold-earning function as a method

on the state object; another part — equally blind — provided it as a module function and even hedged the call correctly elsewhere. Five integration groups failed on that one call.

• Level 2, twice: the map data part emitted waypoints as

[x, y] pairs; the geometry part read them as {x, y} objects. Both readings fit the spec's words. Positions came back as NaN and everything downstream collapsed. The third run's parts happened to agree — and passed everything.

• Level 3: the spec pinned the representation up front (the

disclosed fix), and the seam failures disappeared: two full passes, and a third build that failed only because one file's punctuation arrived double-encoded — garbled text on screen, verified at the byte level.

The one-shot arm isn't immune to self-contradiction — one Level 2 attempt contradicted its own convention across a single response — but writing everything in one context makes consistency the default. Sealed parts have to win consistency at every boundary, every run.

What this changes

The headline "splitting lets a weak model do strong-model work" is not something this slate can say. At 14–15 interlocking files, a current cheap model with a complete spec doesn't need the split — and the split cost roughly 1.8× the tokens for fewer passes.

What the experiment did measure, precisely, is where the split's reliability is bought: in the explicitness of its contracts. A split is exactly as strong as its contracts are explicit — pin the call conventions and the data representations, and sealed parts integrate; leave one surface ambiguous, and the failure is total and silent. That lesson is now part of how the packs here are written. The harness claim this slate did prove is the other half of the system — Demo B's rescue: a fresh weak model finishing a stranded project from its continuation pack, 3 of 3, against 0 of 3 without it.

Disclosures

• Every checker in this demo was self-verified against an

operator-written reference build (passes) and an empty attempt (fails) before scoring, and every level's checker still caught defects in itself during grading — three re-grades in total, each documented, each re-scored from saved files with no model re-called, and each correction improved the baseline's score (L1 4/5 → 5/5; L2 0/5 → 3/5; L3 3/5 → 4/5). The original verdicts are preserved beside the corrected ones.

• The Level 2 split briefs carried one disclosed clarification over

Level 1 (the earning function's call form, stated outright after Level 1's failure); Level 3's spec pinned the waypoint representation for both arms. No interface changed; the ambiguity the record convicted was removed, in the open.

• The Level 3 checker records a limit it cannot cross: with the

save format deliberately left to each builder, rejection of a same-format save with swapped values is not black-box testable. That gap is stated in the record rather than tested around.

• Two runner processes died mid-run during the demo (a Level 1

split run, a Level 2 split run); both partial runs were discarded in full and re-run, and their token usage is excluded from the totals and disclosed in the record.

• The baseline was never hobbled: full spec, one competent prompt,

five attempts, its passes published at every level — including the levels where publishing them ends the claim.

• The strong-model follow-on arm surfaced two further checker

fidelity defects (the sandbox lacked browser timers; one assertion counted dead enemies an implementation may sweep). Fixed, all eleven Level 3 attempts re-graded from saved files; no verdict changed in any arm.

• Spend: 2,809,846 tokens across 147 recorded generation calls

(including the strong-model arm) — on the order of a few dollars at the provider's published rates.

Evidence — re-grade it yourself

← All proofs How the harness is tested