IN PRACTICE
Can You Actually Estimate? Calibration Is Measurable
Anyone can call a winner. Calling the probability is a craft — and it comes down to a single number. Ten statements will show you whether you trust yourself too much, too little, or just about right.
On a prediction market, the winner isn't the person who calls the outcome. It's the person who calls the probability. That's an unfamiliar idea, because the world around us rewards certainty: a pundit who said "this will happen", and it happened, looks smart. On a market the reward hides somewhere else — in whether your seventies come true seventy percent of the time.
The good news is that this is measurable. Not by feel, not in hindsight, but with a number you can find out about yourself in ten minutes. The bad news is that the number tends to be unpleasant for almost everyone at first.
Calibration: when "eighty percent" really means eighty
CalibrationThe match between the confidence you state and how often you're actually right. A calibrated person is wrong about exactly one in five of the claims they give 80% — no more, no less. isn't about being right often. It's about your numbers not lying. Someone who says ninety percent to everything and gets sixty right is dangerous to themselves: they overpay on every bet, because they're buying a certainty they don't have.
The opposite mistake is rarer, but real. Someone who's afraid to say more than sixty percent, even when they know their stuff, leaves money on the table — and never finds the bet worth going in on properly.
You measure it with the Brier scoreThe mean squared difference between your estimate and reality. Zero is perfection, 0.25 is what you get by saying 50% to everything, and anything above 0.25 means you'd have done better with a coin.. It doesn't care whether you "were right" — it punishes you for how certain you were at the moment you got it wrong.
The best way to get it is to try it. Ten statements; for each one, move the slider to how strongly you believe it holds. Fifty percent means a pure guess, zero means "this is definitely false":
- 1
Polymarket runs on the Polygon network.
True. It rests on Ethereum in spirit, but the trading happens on Polygon, where a transaction costs a fraction.
- 2
At most 21 million bitcoin can ever exist.
True. The cap is written into the network's rules and can't be raised without the agreement of the overwhelming majority.
- 3
Ethereum switched to proof of stake in 2022.
True. The switch happened in September 2022 and cut energy use by orders of magnitude.
- 4
USDC and USDT are issued by the same company.
False. They are two different issuers, each with its own reserves and its own history of trouble.
- 5
A share on a prediction market pays exactly one dollar when it wins.
True. That is precisely why a price in cents reads directly as a probability in percent.
- 6
A new Bitcoin block appears on average once every ten minutes.
True. The network retunes its own difficulty so the average stays at ten minutes.
- 7
A seed phrase always has exactly twelve words.
False. Twelve is the most common, but twenty-four is routine and the standard allows other lengths too.
- 8
Fees on Polygon are paid in ETH.
False. They're paid in POL. Without a few cents of POL no transaction leaves your wallet, even with a thousand dollars of USDC sitting in it.
- 9
An optimistic oracle puts every proposed outcome to a vote.
False. Only what somebody disputes goes to a vote. The rest is approved by silence — hence "optimistic".
- 10
The Kelly criterion maximises long-run growth of an account.
True. It does not maximise the profit on one bet, but where you end up after hundreds of them.
Ten questions is too few for a proper measurement — plenty to see yourself, though. Anyone taking it seriously writes down estimates on real bets and reviews them every fifty.
How to read that score
0.25 is the line of indifference — exactly what you get by leaving all ten sliders at fifty percent. Anything below means you know something. Anything above means your confidence is hurting you: either you're betting against your own knowledge or — far more often — you trust yourself more than reality warrants.
Look above all at the pair average confidence and actual hit rate. When the first is clearly higher than the second, you have textbook overconfidence. It's not a disgrace, it's the best-documented feature of human judgement there is — and the only known cure is writing estimates down in advance and then looking at them.
An estimate starts with the base rate, not the story
When news breaks, people tend to rewrite the whole estimate around it. That's the mistake markets punish daily. The right order is the opposite: start from the base rateHow often the thing happens in general, regardless of today's news. For questions like "will the minister resign by the end of the month?" the base rate is remarkably low, however loud the headlines are. and let the news nudge it. By how much depends on one thing only: how often that news would arrive if nothing were happening.
it happens and the news arrives it doesn't happen, and the news arrives anyway
News that also arrives when nothing is happening moves the estimate very little. The rarer the false alarm, the more the news weighs.
Try pushing the false alarm up to eighty percent. The estimate barely budges — and that's exactly the situation where markets move the most: news lands that distinguishes practically nothing, and the price jumps anyway. That's where the edge appears for whoever does the arithmetic.
Three habits that move the needle
Write the estimate down first. Without a record, everyone remembers having "sort of expected" whatever happened. A column in a spreadsheet is enough: date, question, your percentage. After fifty rows you'll see yourself better than after five years of impressions.
Ask what would convince you otherwise. If you have no answer, you don't have an estimate, you have a stance. An estimate can be moved by evidence; a stance can't.
Separate "I don't know" from "I don't care". Fifty percent is an honest answer to a question you've thought about and that came out balanced. For a question you never read, the answer isn't fifty percent — it's don't bet.
What to take away
- The market doesn't pay for calling the outcome, it pays for calling the probability.
-
A Brier score below
0.25means you know something. Above it, you'd be better off with a coin. - The gap between average confidence and hit rate is your overconfidence expressed as a number.
- An estimate starts at the base rate. News only shifts it — and the less, the more often that news also arrives for nothing.
- Calibration gets you into the game. What earns is the gap against the market.
With this one you have the full set: you know what a prediction market is, how to get into one, who decides the outcome, how much to put into a bet, what can be read from big wallets — and now also how to tell whether your own numbers mean anything. The rest is repetition and honest record-keeping.