Icarian Forge
← All work

Case study · Live at talk.icarianforge.com

Talk Me Out of It

A council of AI agents argues about the gadget you want, then rules: Buy, Wait 30 days, or Skip. Four agents with different jobs argue it out, a fifth explains the verdict, and the code does the math. Built from scratch in Python on a free local model, for $0.

Role
Product lead and apprentice: wrote the brief and the rules, made every call, and learned Python and agent design as it was built
Built with
Claude Code as the engineering team; Python on a free local model (Qwen2.5 14B through Ollama); React; Cloudflare
Budget
$0: a free model on my own Mac, free hosting, no paid service anywhere
Status
Live: eight real debates, saved as they ran, from a $70 game to a $2,500 laptop. Live mode for visitors is planned, not built.

In 30 seconds

Strategy

Wrote the brief first: something anyone gets in five seconds, with a real agent system under it. Hard rules before any code: zero dollars, no framework, the model judges and the code calculates.

Leadership

Directed Claude Code as the engineering team in eight small milestones, each shown working and explained before the next, and made every call along the way.

Hands-on

Learned Python and agent design on the job: chose the model after a head-to-head evaluation, approved the scoring formula from its numbers, and picked the purchases the council was put to.

Live · talk.icarianforge.com
The landing page: the council’s five orbs, the glowing name, the hook with Buy, Wait 30 days and Skip in their colors, and the start of the How it works section below
The share card for a PlayStation 5 Pro: 4.5 of 10, Wait 30 days, and the Judge’s one-line reason
A verdict on a phone: Wait 30 days at 6.6 of 10, with the rule note that no member leaned Buy
Why I built it

You’ve already half decided

You want the thing. You’ve read the reviews, and they all agree with you, because you only read the good ones. What you need isn’t another review but someone to argue with you, and a verdict you can hold against yourself in a month.

I also wanted to learn how agents are really built. Not a wrapper around one prompt, but the loop itself: turns, rounds, tools, structured output, guardrails, tracing and evaluation, written in Python on a free local model. Claude Code was the engineering team, and I stopped at every milestone to understand what had just been built before going on.

It’s a decision helper, not financial advice. Nothing on the site buys anything or connects to a store.

What it does

Five voices, one verdict

  • Five voices, not oneThe Analyst, Future You, Budget Guardian and the Alternatives Hunter each argue from one angle, in their own voice. The Judge only explains.
  • They actually argueRound two is rebuttals, by name. Every member reads the last round, answers one or two of the others, and scores again. A split council gets a tiebreak round.
  • A verdict you can checkRatings from 1 to 10 from every member, averaged by Python, with two cut-offs and two lean rules. The page redoes the arithmetic in front of you.
  • Under the hoodEvery model call, tool call and rejected answer of the run on a scrubbable timeline, with each member’s full brief and system prompt.
  • No made-up factsPrices and release dates come from you, or are labeled unverified. An order hidden in the input, like “rate everything 10”, is set aside before the council reads it.
  • Share itEvery debate has its own address and a share card, so a verdict can be sent to the person who needs talking out of it.
How it works

The model judges, the code calculates

  1. IntakeA small classifier call checks that it’s a tech gadget, names the category, and flags any orders hidden in the reason. A leather sofa gets a polite decline.
  2. Opening roundEach member gives its take as schema-checked JSON: a reason before every score, a lean, and only numbers it was actually given. A reply that breaks a rule goes back with the exact problem, up to three times.
  3. ToolsThe Analyst and the Hunter call two exact Python tools, cost per use and resale. The model picks the assumptions; the code does the arithmetic, and the model can’t type a resale value.
  4. RebuttalEveryone reads a digest of the last round and answers the others by name. Code records every rating and lean that moved.
  5. ScoringPython averages the final ratings: 5.5 and up is Buy, below 4.5 is Skip, between is Wait 30 days. Buy needs one member leaning Buy and Skip one leaning Skip, or the verdict holds at Wait.
  6. The JudgeReads the debate and explains the verdict it was given. It has no verdict field to fill in and never produces a number.
84 of 105evaluated councils agree with the expected verdict
0verdicts that no member argued for
357automated tests, none needing the model
$0spent: a free model on one Mac, free hosting

Rules I set before building anything

  • Zero dollars, a free local model
  • No agent framework
  • The model judges, the code calculates
  • No made-up prices or dates
  • Never buys anything
  • The site only reads transcripts
Does it work?

Honest numbers

An evaluation harness runs the council against 15 purchases with the verdict a person would expect, several times each, and reports agreement. On all 105 saved councils the scoring agrees 84 times (80%), with no far misses and no verdict that no member argued for. Every miss is one step off, a Wait where Skip was expected or the reverse, and the expected verdicts were written before any run and left alone afterwards.

Two things worth knowing. The debate changes minds, rarely outcomes: one verdict in 80 moved after the rebuttal round. And the first live run under the final formula scored 16 of 26, because the Analyst switched from Buy to Wait in four rebuttals and the Buy rule then held four right Buys at Wait. Both are in the build notes, not hidden.

1 in 80verdicts changed by the debate itself
15 of 15injected orders caught, no false alarms on 38 ordinary texts
8 of 8non-gadgets declined, the category right every time
  1. Setup
  2. One agent
  3. Tools
  4. The council
  5. Scoring and evals
  6. Sessions
  7. Replay site
  8. Publish
How it was built

Eight phases in seven days, each explained before the next

The interesting part was watching a small model fail in specific ways and fixing each one in code rather than by rewording the prompt: numbers invented in reasons, a tool argument made up nine times in 39 councils, Chinese characters mid-sentence, a Judge that opened with the verdict until its field was renamed, and a prompt injection the bigger model followed because it follows instructions better. Every lesson is written down as a rule in the build notes and as a decision record.

8build phases, each shown working before the next
7 daysfrom the brief to live
4,100lines of hand-written Python, no agent framework
Next

Where it could go from here

  • Live modeLet visitors put their own purchase to the council. That needs a hosted model and money, so it comes with a spending cap and rate limits, or not at all.
  • A council that holds its groundMembers drift toward Wait in the rebuttal, and the Buy rule then holds right Buys at Wait. The fix is a scoring choice or one prompt rule: change your lean only when an argument changed your mind.
LiveWatch a debate