We asked a local LLM to build Pac-Man. Cold, it failed. One prompt change fixed it.

Part of an Altronis series on running real AI on hardware you own.

We run both kinds of model: frontier models in the cloud, and open models on our own machines. The interesting question for anyone weighing private, on-prem AI isn't "which benchmarks does the local model win" - it's how far can a small local model actually build real, working software? So we ran a deceptively hard test: a playable Pac-Man, written by Qwen 3.6 - a 35-billion-parameter model with only ~3B parameters active per token - running entirely on a mini-PC in Singapore, no cloud, no API.

Cold prompt: it looks like Pac-Man - now try to play it

We asked for a Pac-Man clone in plain language. The local model produced a full file that renders a convincing board - maze, dots, four ghosts, score, lives. Then you try to play it. Pac-Man won't turn cleanly at junctions, and it slides straight through the walls. It is not a playable game. And here's the part that matters: across five rounds of "it's still broken," the model could not fix it.

The bug it couldn't see

Reading the code, the cause is subtle and systemic. The model chose to move Pac-Man by a fractional speed - about 0.06 of a tile per frame. But its own logic only checks for a turn or a wall when Pac-Man is exactly centred on a tile. With a step of 0.06, the position never lands exactly on a tile centre - it steps 0.94, 1.00-ish, 1.06… skipping over the exact centre almost every time. So the turn-and-collision check almost never fires. Nothing turns; nothing collides. Every symptom traces back to one wrong architectural choice - and the small model kept editing around it instead of seeing it. It took a frontier model reading the code to name the bug.

Same model, 150 more words: it works

We did not switch to a bigger model. We gave the same local Qwen the same task - plus one block of architecture in the prompt: represent everything on an integer grid; only change direction or check walls when the actor is exactly centred on a tile; and choose a speed that divides the tile size evenly so it always lands on a centre. It one-shot this - a genuinely playable Pac-Man, with real cornering, collision, four chasing ghosts, and mobile swipe:

The block that made the difference was about 150 words:

MOVEMENT MODEL (critical - do not deviate):
- Represent Pac-Man and every ghost by INTEGER grid coordinates
  {col, row}, a direction {dx, dy}, and an integer progress 0..TILE.
- Movement is TILE-LOCKED. An actor may ONLY change direction or
  test walls when it is EXACTLY centred on a tile (progress === 0).
- Choose SPEED so TILE is an exact integer multiple of SPEED
  (TILE=24, SPEED=3). NEVER use fractional speeds like 0.06 - they
  drift past tile centres so turns and wall-checks never fire.
- Each tick: if progress===0, decide direction / run AI, then if the
  target tile is a WALL, cancel movement. Add speed to progress; when
  progress>=TILE, commit col+=dx,row+=dy and reset progress=0.

The rest of the prompt was generic - the stuff anyone would write. The difference-maker was those ~150 words: domain knowledge a frontier model has internalised, and a 3B-active local model simply doesn't carry.

The lesson: the prompt is where the expertise lives

The gap was never the local model's ability to write code - it writes clean code, fast, for free, on your own box. The gap is the meta-knowledge of which architecture avoids the bug in the first place. Frontier models carry that knowledge unprompted; local models need it supplied. Which points to a durable division of labour, not a winner-takes-all:

• Frontier model = architect and judge. It writes the tight spec (the "movement model" block, scaled up to a whole system) and reviews the output for the coherence bugs a small model is blind to.
• Local model = executor. It builds most of the actual code - cheaply, privately, and quickly - given a spec that pre-empts the bug classes.

We tested exactly this at larger scale too - the same local model built most of a full 3D world from a detailed spec, and needed a frontier model only to catch a handful of last-mile correctness bugs. Same pattern, same lesson.

Part 2 — the same idea, scaled to a 3D world

Pac-Man is a 2D puzzle. So we pushed the pattern much harder: we wrote a genuinely complex spec for a high-fidelity 3D explorable island — procedural terrain, custom GLSL water with waves and foam, an articulated animated character, day-night lighting, instanced trees, a minimap — and had the same local Qwen 3.6 build the whole thing as one file. It produced a complete 68KB app in five minutes. It ran — after a frontier model fixed a handful of bugs it couldn't see: a three.js class that doesn't exist in the loaded version, and GLSL helper functions defined after they were called (the shader language has no hoisting). Same lesson as Part 1: local writes the bulk, frontier catches the coherence.

Then the real question: can it fix its OWN bugs?

We fed the local model nothing but its own runtime errors — no diagnosis, no code from us — and asked it to repair itself. The first run was brutal: it rewrote the entire 68KB file three times and never fixed a single one-line bug. It kept re-emitting a call to a three.js class that doesn't exist in this version, even after being told three times that the call errored. Verdict, apparently: the frontier judge isn't optional.

But that verdict was unfair — and why it was unfair is the most useful lesson here. That first run had the model's reasoning mode switched off, and its thinking budget capped at ~500 tokens. We were asking it to debug with one hand tied behind its back.

So we ran it properly: reasoning on, thinking budget raised to 4,096 tokens, and — crucially — an agentic loop so it could see each new error and fix that one next. This time it converged in two rounds, to zero errors, entirely on its own: round one fixed the first bug using ~2,800 reasoning tokens (six times what the old cap allowed — so it genuinely had been starved), and round two caught a bug it had just introduced and fixed that too. The result is a clean, running 3D world — built and self-repaired by a local model on a mini-PC, no cloud:

The sharper lesson

Local models can self-debug real, stateful bugs — but only under the right conditions: enough reasoning budget to actually think it through, and an agentic loop to react to their own errors. Starve either one and they thrash. So the frontier model's role isn't "do the fix the small model can't" — it's "design the spec, set up the loop, pick the right settings, and catch whatever's left." That's a cheaper and more capable division of labour than either model working alone. The exact local setup we used - the llama.cpp flags, the reasoning-budget config, and the systemd units - is open-source: github.com/sypherin/strix-halo-setup.

Why this matters if you run AI on your own hardware

For regulated or cost-sensitive teams, local models are an enormous lever: near-zero marginal cost per token, and data that never leaves the building. This experiment is the practical playbook for actually getting correct output from them - not "hope the model is clever," but "encode the expertise in the spec, and keep a frontier model in the loop as architect and judge." That is precisely how we build private and on-prem AI systems. If you want the hardware, the benchmarks, and the deployment patterns, start with our Private & On-Prem LLM Deployment guide.