← mino.mobi

crucible

overgenerate, then judge — the studio's house algorithm

in the fire: mappa · biome/gacha · hoop/econ · fable  ·  the field: generate-and-test → propose-and-verify

Build a big, awkward, noisy set. Put a judge in charge. Keep the good ones.

It turns out that's not a trick — it's a method, and it has been one for sixty years under half a dozen names. We kept reaching for the same hammer across unrelated projects: a world generator that spits out thousands of planets, a gacha that rolls whole ecosystems, a town engine that breeds societies, a puzzle factory that mints logic grids. None of them try to author the good artifact directly. Each one overproduces — floods the space with candidates — and then hands the pile to a second machine whose only job is to say which of these is any good. The good ones aren't designed. They're survived.

The shape of the hammer

generatorseeded, cheap, noisy — floods the space with candidates
→
judgesolver / oracle / fitness / verifier — scores or certifies each one
→
keepthe survivors: ranked, tiered, or filtered to a certified set

The split is the whole idea. The generator can be dumb and prolific because it doesn't have to be right — it only has to be diverse. The judge can be expensive and exacting because it runs after the fact, on finished candidates, where the question "is this good?" is usually far easier than the question "how do I make a good one?". That asymmetry — that verifying is cheaper than generating — is the lever the whole family pulls on. It's why a small model with a verifier can match a model thirty times its size, why a chess program can beat you by guessing badly and checking well, and why our gacha can find a self-sustaining ecosystem it could never have planned.

Three things make our version of it work:

1 · The judge is real, not a vibe

Every judge here is a piece of running code with teeth: a constraint solver that counts solutions, a breadth-first search that finds the optimal par, an eigenvalue test that asks whether a food web is actually stable, a tribunal of playouts that measures whether a card game rewards skill. The verdict is a certificate, not an opinion.

2 · Interestingness is a designed aesthetic — and we say so

Where the judge scores taste rather than correctness — is this world interesting? is this town alive? — we don't pretend the number is objective. mappa's own notes put it best:

"The score is for ranking — surfacing the gems out of a seed line — not for claiming a world is objectively good… the score is deliberately a designed aesthetic, not a discovered one." mappa — lib/world-signals.js

The honest posture is half the discipline: a battery of weighted signals plus a list of degeneracy flags — the boring failure modes (waterworld, iceball, monobiome, monoculture) we explicitly screen out.

3 · Seeded generation makes the verdict a permalink

Because every generator is deterministic (xmur3 → mulberry32 and friends), candidate n is the same artifact on every machine, forever. So the judge's certificate can be attached to a URL: /gacha/?n=…, /econ/?seed=…, /mappa/?seed=…, fable's page-n puzzles. The good one you found is the good one anyone else finds at that address — solved, graded, and provably so.

Our crucibles

The same loop, four shapes of fire. The judge changes form completely from project to project — that's the point — but the discipline never does.

surfacethe noisy set (generator)the judgewhat ships (certificate)
worlds — the judge scores viability & interestingness
mappaworld plates, tectonics, climate, rivers & biomes simulated from a seed — ~9–16k Voronoi cells per planet an interestingness battery: 6 weighted signals (sea, structure, relief, climate, hydrology, habitability) + 9 degeneracy flags score 0–100 + auto-descriptor ("a frozen archipelago, 71% ocean, with inland sea and great rivers")
biome/gachaworld roll K=14–34 organisms from a 60-creature deck under guild quotas; wire who-eats-whom by ecological rules; prune the starvers a viability oracle: does the loop close? does it persist? is it stable (community-matrix eigenvalue α<0)? fed, aired, robust? Legendary≥88EpicRare … rarity = viability minus degeneracy
hoop/econworld a social genome (role-mix, building footprints, affiliations, archetype) breeds a 6,000-cell town where people wear many hats a vitality scorer: supply closure, interaction thickness, Granovetter weave, bridging, third-places, employment — then a hub-removal shock test vitality 0–100 → Thriving … Failing; every (genome, seed) is atproto-stable
games & puzzles — the judge certifies, never opines · fable's doctrine: seeded · certified · graded
puzzgame roll a logic-puzzle genus, generate a full solution, carve fair clues from it CSP solver — countSolutions() (bounded backtrack) + logicSolve() (pure propagation, no guessing) uniquefair + technique-fingerprint difficulty
knackgame composable grid mechanics (push, ice, keys, gates, pits) across 6 genres BFS over the state graph: solvable? optimal par? did the headline mechanic actually fire? solvableoptimal par + the mechanic earns its place
fluxgame continuous-space physics puzzles — gravity, magnets, goo, bumpers (deterministic f64) action-space sweep: 96×18 launches → win-map → basins → a fine-robust answer that survives a 0.4° nudge (Rust cross-check) solvablebasin-robust + difficulty = how small the winning basin is
gyregame the same physics, but on a torus — geodesics, Christoffel terms, forces projected to the tangent plane the sweep adapted to the surface, plus winding numbers only the topology can express solvablefine-robust + topologically interesting
morphgame a generator of generators: sample substrate (grid · cylinder · torus · Möbius · Klein · hex) × law × goal × aesthetic one substrate-agnostic BFS oracle certifies any rolled genome solvable; reject the duds solvable on any manifold + a stable genome label
driftgame puzzles in meaning-space — 7,000 word embeddings; cross a semantic gulf, or fold 12 words into 3 families BFS on the kNN graph (ladder par) + a margin oracle (fold separability ≥ 0) optimal crossingmargin-separable over a frozen substrate
forgegame laws no one wrote: sample a movement genome from a closed rule-DSL three gates — ALIVE (≥30 reachable states), NOVEL (fingerprint ≥ 0.22 from all prior — novelty search), PLAYABLE (BFS certifies) a deterministic codex — the n-th admitted law is a permalink
dealgame a card-game grammar: trick-taking or shedding × suits × ranks × scoring → a self-writing rulebook the tribunal: seeded random-vs-greedy playouts, measured on three gates — terminates, rewards skill, plays fair terminatesskillfulfair — games no one wrote, proved playable
fable says it best. "Six puzzle-spaces, one discipline. Each takes a vast combinatorial space, generates it deterministically from a seed, and sets an oracle over it — a real solver that certifies every artifact: solvable, fair, graded. Nothing here was authored; everything here was proved."

Notice forge in particular: its NOVEL gate keeps a candidate law only if its behavioral fingerprint is far enough from every law admitted before it. That isn't "keep the best" — it's "keep the different." Which is the exact move that names a whole branch of the literature below.

The field — names for one hammer

This is the part the projects kept gesturing at: surely someone studies this. They do — many someones, mostly without talking to each other. The same architecture (a cheap proposer + a separate evaluator) has been reinvented in classic AI, evolutionary computation, natural-language processing, deep learning, game design, and now large language models. Here is the through-line and the canon.

What the communities call it

generate-and-testclassic AI / cognitive science. The original and most general term — a generator proposes, a tester rejects the unfit (Newell & Simon). Degenerate baseline: the "British Museum algorithm."
search-based generationprocedural content generation / game AI. The judge is a fitness function driving an iterative search; contrasted with pure generate-and-test PCG.
quality-diversity / illuminationevolutionary computation. Keep the best of each kind, not one winner — novelty search, MAP-Elites. (This is forge's gate.)
overgenerate-and-rankNLP / parsing / MT / speech. Emit an n-best list, then re-score it with a discriminative judge.
propose-and-verify · generator–verifiermodern LLM / RL. A cheap verifier filters an expensive generator's samples (best-of-N) — the current restatement, riding the "generation–verification gap."
generator–discriminatordeep learning. The special case where the judge is learned jointly and adversarially (GANs).

Classic AI generate-and-test

  1. Newell, Shaw & Simon (1958). Elements of a Theory of Human Problem Solving. Psychological Review 65(3), 151–166. psycnet Coins the "British Museum algorithm" — exhaustively generate every possibility — as the naive baseline a selective judge improves on. The canonical statement that generation needs a guide.
  2. Newell & Simon (1972). Human Problem Solving. Prentice-Hall. archive The book where "generate-and-test" is articulated as a basic method: a generator proposes candidates, a tester rejects the unfit. The most-cited home of the named pattern.

Search-based procedural content search-based generation

  1. Togelius, Yannakakis, Stanley & Browne (2011). Search-Based Procedural Content Generation: A Taxonomy and Survey. IEEE TCIAIG 3(3), 172–186. doi:10.1109/TCIAIG.2011.2148116 Explicitly contrasts generate-and-test PCG (generate, test against constraints, discard, regenerate) with search-based PCG (a fitness function drives the search). The cleanest field-internal articulation of our loop — and Browne is the bridge to automated game design below.

Quality-diversity & aesthetic evolution illumination

  1. Lehman & Stanley (2011). Abandoning Objectives: Evolution Through the Search for Novelty Alone. Evolutionary Computation 19(2), 189–223. pdf Novelty search: the judge rewards being different, not better. Exactly forge's NOVEL gate — keep a candidate for how far it is from everything seen.
  2. Mouret & Clune (2015). Illuminating Search Spaces by Mapping Elites. arXiv:1504.04909. arxiv MAP-Elites: file each candidate into a behavior-grid cell, keep the elite per cell. The output is a diverse map of high performers — "keep the best of each kind."
  3. Pugh, Soros & Stanley (2016). Quality Diversity: A New Frontier for Evolutionary Computation. Frontiers in Robotics and AI 3:40. doi:10.3389/frobt.2016.00040 Unifies novelty search + MAP-Elites under "quality-diversity," the sub-paradigm where "keep the good ones" becomes "keep the best of each type."
  4. Sims (1991). Artificial Evolution for Computer Graphics. SIGGRAPH; Computer Graphics 25(4), 319–328. karlsims.com Mutating Lisp expressions generate images; the human eye is the judge (interactive/aesthetic selection). Noisy procedural generator, a person picking the good ones.
  5. Dawkins (1986). The Blind Watchmaker — the Biomorphs program. overview Random mutations of a 9-gene tree, the user breeds the offspring they like. The earliest popular demo of "random generator + selective judge → complexity."

Overgenerate-and-rank in language overgenerate-and-rank

  1. Collins & Koo (2005). Discriminative Reranking for Natural Language Parsing. Computational Linguistics 31(1), 25–70. mit press A generative parser overgenerates n candidate parses; a discriminative model reranks them. The textbook "noisy generator + learned judge → pick the best" in NLP.
  2. Charniak & Johnson (2005). Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking. ACL 2005, 173–180. aclanthology Produce 50-best parses, then a MaxEnt reranker selects. The standard reference for n-best reranking, mirrored in the MT lineage (Och et al.).

Adversarial: the judge learns too generator–discriminator

  1. Goodfellow et al. (2014). Generative Adversarial Nets. NeurIPS; arXiv:1406.2661. arxiv The judge (discriminator) is trained jointly with the generator and its gradient teaches the generator. The pattern made differentiable and co-adaptive — a minimax game.

The LLM era: verifiers & the generation–verification gap propose-and-verify

  1. Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems (GSM8K). arXiv:2110.14168. arxiv Sample ~100 solutions, train a separate verifier to score them, take best-of-N. A 6B model + verifier matches a 175B model — the empirical seed of "verifying is cheaper than generating."
  2. Wang et al. (2022). Self-Consistency Improves Chain-of-Thought Reasoning. arXiv:2203.11171. arxiv Sample many reasoning chains, keep the majority answer. The judge is a free consensus vote — generate-many-then-select with no extra model.
  3. Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685. arxiv A strong LLM can be the judge, agreeing with humans >80%. The enabling result for LLM-as-a-judge ranking/filtering pipelines.
  4. Silver et al. (2016). Mastering the Game of Go with Deep Neural Networks and Tree Search (AlphaGo). Nature 529, 484–489. doi:10.1038/nature16961 A policy network proposes moves, a value network judges positions, MCTS searches the proposals under the judge. The RL incarnation; AlphaZero generalizes it with self-play.

Automated game design it has been done

  1. Browne (2008 thesis / 2011 Springer). Evolutionary Game Design — the Ludi system; Yavalath. springer Ludi recombines game "ludemes," plays each candidate against itself, and judges by aesthetic metrics (drama, depth, uncertainty). Yavalath — the survivor — is the first commercially published computer-designed board game. This is fable/deal's tribunal, fifteen years earlier.
The through-line. Across six decades and nine communities the architecture is one thing: split a hard problem into a cheap, often stochastic proposer and a separate evaluator; lean on the proposer for coverage, and let the evaluator do the discriminating the proposer can't. The communities differ mostly in who the judge is — a hard constraint, a hand-designed aesthetic, a human eye, a learned discriminator, a trained verifier, or a cheap consensus — and in what they keep (one winner, or a diverse illuminated set). The reason it keeps winning is the generation–verification gap: scoring a good solution is usually easier than producing one, so many candidates + a good judge beats one-shot genius.

What makes a good crucible

Do

Don't

The oracle changes shape every time — CSP, BFS, action-sweep, eigenvalues, embeddings, a tribunal of playouts — but the discipline never does: overgenerate, then judge.