Andrej Karpathy · Released, not on our provider
LLM council prompt: three templates for the answer, review and chairman stages (2026)
An LLM council prompt is a three-step way to ask several models the same question, have them review and rank each other’s anonymised answers, and then have a chairman model write one final answer, a flow Andrej Karpathy published in his llm-council repository on GitHub, and this page gives you three templates to run it by hand.
Use it now
Chat
This chat starts with Claude Haiku 5.5, not LLM council prompt.
≈ 1 credit per reply · See all models
New accounts get 5 free credits once, valid for 7 days. They work on the lower-priced models here; the priciest ones need credits you bought.
Start chatting — 5 free credits- Maker
- Andrej Karpathy
- Type
- Text
- Released
- No date given
- OpenRouter id
- None
Checked
LLM council prompt: facts, with sources
- Andrej Karpathy’s GitHub repository llm-council was created on November 22, 2025, according to GitHub. Its README calls it a simple, local web app that looks like ChatGPT but sends your query to several LLMs through OpenRouter, asks them to review and rank each other’s work, and then has a Chairman LLM produce the final response.Source: github.com · checked 2026-11-19
- The README describes three stages. Stage 1, first opinions: the question goes to every model on its own and the answers are shown in tabs. Stage 2, review: each model is given the others’ answers with the identities anonymised "so that the LLM can’t play favorites", and ranks them for accuracy and insight. Stage 3, final response: the Chairman compiles everything into one answer.Source: github.com · checked 2026-11-19
- In the code (backend/council.py), stage 1 sends the question alone, as one user message, to every council model in parallel and keeps the replies that succeed. Stage 2 labels those answers "Response A", "Response B" and so on and sends the same labelled set to every council model, including the one that wrote each answer, in the same order for every reviewer. Each model must judge every response and end with a fixed "FINAL RANKING:" list, which the code parses and averages per model, lower being better.Source: github.com · checked 2026-11-19
- Stage 3 sends the chairman the question, every stage-1 answer with its model name, and every reviewer’s full text, and asks for one clear, well-reasoned final answer reflecting where the council agreed and disagreed. So the chairman is not blind to who wrote what, and the rankings are input to it, not a vote the code obeys. For N council models the code makes N calls for stage 1, N for stage 2 and one for the chairman, plus a separate call to a small model for the chat title.Source: github.com · checked 2026-11-19
- The default config lists four council models, GPT-5.1, Gemini 3 Pro Preview, Claude Sonnet 4.5 and Grok 4, all called through OpenRouter, with Gemini 3 Pro Preview as chairman. The notes say the chairman can be the same as a council member or a different model, and that if some models fail the run goes on with the ones that answered.Source: github.com · checked 2026-11-19
- The author says the project was a fun Saturday hack, about 99 percent vibe coded (his words), written to compare several models side by side while reading books with LLMs. He says he will not support or improve it and offers it as is for other people’s inspiration. The repository shows no license.Source: github.com · checked 2026-11-19
- The repository’s technical notes call anonymised peer review the key idea and say it stops models playing favorites. They list ideas not built, such as custom ranking criteria and analytics on model performance over time. Neither the README nor the notes report any measurement of answer quality; we found none on the pages we read.Source: github.com · checked 2026-11-19
- This site does not run the council automatically and is not part of the repository. Its chat lets you pick any usable text model, so you can play each stage by hand with models such as Claude Haiku 5.5, GPT-6 Luna and DeepSeek V4.1 Flash (checked November 19, 2026).Source: openrouter.ai · checked 2026-11-19 · second-hand
Our table
LLM council prompt: our three templates by stage
- The repository’s ranking prompt asks each model to evaluate every response one by one, then end with a numbered final ranking in a strict format so the code can parse it. Our three templates follow the same order of stages but are written by us for a chat window where you paste the answers yourself; they do not copy the repository’s prompts.Source: github.com · checked 2026-11-19
Prompts by stage
LLM council prompt templates for stage 1, stage 2 and stage 3, with what to paste in
To run a council by hand, open the chat, start a new chat for each council model, pick the model and send the stage 1 prompt. Copy each answer, remove the model names, then send the stage 2 prompt to each model in a fresh chat, with the answers in a different order each time. Last, send the stage 3 prompt to the chairman. Three details of this chat shape the templates: a message holds at most 4,000 characters; line breaks become spaces and angle brackets, curly braces, backticks and double quotes are removed from what you send; and the chat’s fixed instructions ask models for plain text under 700 words. So each template is one paragraph with plain labels and asks for short answers. The character budget is our own sizing, not a test run; count your message before you send it.
| Stage | Purpose | Our prompt text | What to paste in |
|---|---|---|---|
| Stage 1: answer | Get an independent answer from each council model, short enough to paste three of them into one message | Answer the question below on your own, in plain text, in at most 100 words. Start with your answer, then give the one or two reasons it rests on. If you are unsure of a point, say so in one short sentence. Question: [QUESTION] | Your question in place of [QUESTION]. Send it as the first message of a new chat for each model, so no model sees another model’s answer. |
| Stage 2: anonymous review | Have each model judge all three answers without knowing who wrote them, and rank them | Below are a question and three answers written by different assistants, labelled A, B and C. You do not know who wrote which. For each answer write one line of at most 15 words that names its biggest strength or its biggest mistake. Then write RANKING: followed by the three letters from best to worst, judging correctness first and usefulness second. Question: [QUESTION] Answer A: [TEXT] Answer B: [TEXT] Answer C: [TEXT] | The question and the three stage 1 answers with names removed. Use a new chat per reviewer and change the order of A, B and C each time: the repository shows every reviewer the same order, and we have not measured whether order matters. |
| Stage 3: chairman | Have one model write the final answer from the three answers and the reviewers’ short verdicts | You are the chair of a panel. Below are a question, three answers labelled A, B and C, and the one-line verdicts and rankings from three reviewers. Write one final answer in at most 150 words. Keep the points the answers agree on. Where they disagree, decide by the reasons the reviewers give, not by the count of votes alone. End with one line saying what is still uncertain. Question: [QUESTION] Answers: A [TEXT] B [TEXT] C [TEXT] Reviews: [REVIEW 1] [REVIEW 2] [REVIEW 3] | The question, the three answers, and only the one-line verdicts and RANKING lines from each reviewer, not their full text. In a new chat with the chairman model. Keep the whole message under 4,000 characters. |
| Tally: you, no model | Turn the three RANKING lines into one order before the chairman sees them | No prompt. Add up each answer’s position across the three rankings; the lowest total is first. The repository’s code does the same with an average position per model. | The three RANKING lines. Nothing is sent to a model at this step. |
Our table
LLM council prompt: what one question costs through a three-model council
- In the repository’s flow the number of model calls is 2N + 1 for N council models: N first opinions, N reviews and one chairman call. Each stage-2 call carries all N answers, and the chairman call carries the answers and all the reviews, so later calls have longer inputs than the first.Source: github.com · checked 2026-11-19
Cost of one run
LLM council prompt: credits for one question through a three-model council on this site, light and heavy
A credit is about $0.01 in the $5 pack, and each reply costs its real token cost, at least 1 credit. On this site a typical reply to GPT-6 Luna, Claude Haiku 5.5 or DeepSeek V4.1 Flash costs about 1 credit, Claude Sonnet 5.5 about 3, and Claude Fable 5.2 about 11. The light council below uses the three cheap models and Claude Sonnet 5.5 as chairman; the heavy one uses Claude Fable 5.2, GPT-6.2 Sol and Gemini 4 Carbon, with Fable 5.2 as chairman.
| Stage | Light council (3 cheap models) | Heavy council (Fable 5.2, GPT-6.2 Sol, Gemini 4 Carbon) |
|---|---|---|
| Stage 1: 3 calls | About 1 + 1 + 1 = 3 credits | About 11 + 3 + 3 = 17 credits |
| Stage 2: 3 calls | About 1 + 1 + 1 = 3 credits | About 11 + 3 + 3 = 17 credits |
| Stage 3: 1 call | Claude Sonnet 5.5: about 3 credits | Claude Fable 5.2: about 11 credits |
| Total: 7 calls | About 9 credits, roughly $0.09 | About 45 credits, roughly $0.45 |
Nearest available
Alternatives to LLM council prompt you can use now
Claude Haiku 5.5
Anthropic$0.10 / $0.50 (per million tokens)
- Anthropic’s model page says Claude Haiku 5.5 was released on October 7, 2026, is the latest Haiku, and will not be retired sooner than October 7, 2027. Anthropic’s launch post is dated the same day.Source: platform.claude.com · checked 2026-11-17
- In its launch post Anthropic calls Haiku 5.5 "the cheapest, fastest, and most capable small model we’ve ever released", for high-volume, cost-sensitive tasks such as summaries, compactions, database queries and classification, and as a coding subagent alongside Opus 5.5 and Sonnet 5.5. It calls Haiku 5.5 its first Haiku-class model with an adjustable effort setting.Source: anthropic.com · checked 2026-11-17
- Anthropic’s model page says it is built for high-volume, latency-sensitive work such as classification, routing, extraction and subagent tasks, and rates its latency "Fastest" in the current lineup (Fable 5.2 and Opus 5.5 are "Moderate", Sonnet 5.5 is "Fast").Source: platform.claude.com · checked 2026-11-17
GPT-6 Luna
OpenAI$0.10 / $0.50 (per million tokens)
- OpenAI: "Our most efficient model for focused, high-volume tasks".Source: developers.openai.com · checked 2026-11-11
- Context 1,050,000 tokens; maximum output 128,000 tokens; text and image input, text output; knowledge cutoff May 18, 2026.Source: developers.openai.com · checked 2026-11-11
- Reasoning effort can be set to none, low, medium (the default), high, xhigh or max.Source: developers.openai.com · checked 2026-11-11
DeepSeek V4.1 Flash
DeepSeek$0.30 / $1.2 (per million tokens)
- DeepSeek released V4.1-Flash on September 10, 2026.Source: api-docs.deepseek.com · checked 2026-11-11
- Only the Flash version of V4.1 exists: DeepSeek has published no V4.1 Pro, and the Pro model on its price page is still deepseek-v4-pro (V4-Pro-0813).Source: api-docs.deepseek.com · checked 2026-11-11
- DeepSeek’s pricing page says the API model deepseek-flash is now served by the V4.1-Flash-1017 snapshot.Source: api-docs.deepseek.com · checked 2026-11-11
FAQ
LLM council prompt FAQ
What is an LLM council prompt?
It is a way of asking one question to several AI models and then having them check each other. Each model answers alone, then reviews the anonymised answers of the group and ranks them, and a chairman model writes the final answer from all of it. The name comes from Andrej Karpathy’s llm-council repository on GitHub.
What are the three stages of the LLM council?
Per the repository’s README: stage 1 collects first opinions from each model, stage 2 has each model review and rank the anonymised answers for accuracy and insight, and stage 3 has a chairman model compile one final answer. Our table gives a prompt template for each stage.
Can I run an LLM council on this site?
By hand, yes. The chat lets you pick any usable text model, so you can send the stage 1 prompt to three models, paste their answers into the stage 2 prompt, and send the stage 3 prompt to a chairman. Use a new chat for each model, or the later models will see the earlier answers. There is no automatic council button.
How much does an LLM council cost?
It multiplies. Three council models plus a chairman means 7 calls per question: 3 answers, 3 reviews and 1 final answer. On this site a typical reply to GPT-6 Luna costs about 1 credit (a credit is about $0.01 in the $5 pack), billed by its real token cost, at least 1 credit; Claude Haiku 5.5 and DeepSeek V4.1 Flash are about 1 credit each and Claude Sonnet 5.5 about 3. The cheap council above is about 9 credits per question.
How many models should be in the council?
The repository’s default is four council models plus a chairman. We use three in the templates because a message in this chat holds at most 4,000 characters, and the stage 2 and stage 3 messages must carry every answer. Fewer than three gives little to compare; more models mean more calls and longer messages.
Does an LLM council give better answers?
We do not know. The repository’s README and notes describe the design and report no measurement of answer quality, and we have not run any council ourselves. Rankings written by models are opinions, not ground truth, and the chairman in the code sees which model wrote each answer. Check the final answer against a source when it matters.
Is this the code from Karpathy’s repository?
No. We read the README and the Python code to describe the flow accurately, but the prompt templates here are our own wording and are not copied from it. The repository is a weekend project its author says he will not support. We have not run our templates against any model.
Is this the official LLM council website?
No. This page is unofficial and not affiliated with or endorsed by Andrej Karpathy. Facts link to the pages we read.