Ensembles, Voting, and Debate
Run the same task several ways and combine the answers: majority voting, best-of-n with a judge, and structured debate — plus what they cost.
Ask a model the same hard question five times and you won't always get the same answer. That variability is usually treated as a nuisance. Ensembles turn it into a tool. If you run a task several ways and combine the results, independent mistakes tend to cancel out, and the amount of disagreement tells you how much to trust the answer.
This lesson covers the three ensemble patterns that matter in practice: majority voting for questions with a definite answer, best-of-n with a judge for open-ended work, and structured debate, where agents critique each other before a decision. You'll build the first two in TypeScript, learn where diversity actually comes from, and see what each pattern costs, because every one of them multiplies your token bill.
Why combining answers works, and when it doesn't
The idea is old and simple. Suppose each of five voters is right 80% of the time, and their mistakes are independent, meaning one voter's error tells you nothing about another's. A wrong majority needs at least three of the five to fail at once, and the arithmetic says that happens only about 6% of the time, compared with 20% for a single voter. This is why a panel of reviewers beats one reviewer, and why polling several people about a fact works better than asking one.
The catch is in the word "independent." Five copies of the same model, given the same prompt, tend to make the same mistakes. If the model misreads an ambiguous ticket one way, it'll often misread it that way every time. Correlated errors don't cancel; they get outvoted into confident wrong answers.
So the craft of ensembles is mostly the craft of diversity: making the members different enough that their errors don't line up. You have four levers:
- Different framings of the prompt. Ask the same question from different angles: by root cause, by what the customer needs, as a routing decision.
- Different models. A Claude Haiku 5.5 voter and a Claude Sonnet 5.5 voter have different strengths.
- Different approaches or evidence. One member reasons from the text alone, another uses a search tool, a third runs code.
- Different input order. When the input is a list of items, shuffling it changes what the model attends to first.