Mind Forge All tools
Numeracy · calibration

Calibrated Estimation Game

See a few seconds of dots, an image of buildings, or a weight. Estimate the quantity. We don't just check if you're right — we check whether your confidence matches your accuracy.

What makes this one different

A right answer with high confidence is the gold standard. A wrong answer with low confidence is honest. We compute a calibration score: when you're 80% confident, are you actually right 80% of the time? Most people are over-confident.

Type —
Round 1 / 12
Press Start
Ad slot.

Calibration curve

Ideal: the line follows the diagonal. Above the line = under-confident. Below = over-confident.

History

No sessions yet.

What problem does it solve?

Most estimation exercises focus on accuracy: were you close? But being right by luck isn't the same as being right by judgment. Calibration — the relationship between confidence and accuracy — is a better signal of "good judgment." It's measured in forecasting tournaments (Good Judgment Project), intelligence analysis, and weather prediction.

Who is this for

How to interpret your results

Calibration score near 0 is excellent. Below 0.05 is great. Above 0.15 means your confidence doesn't match your accuracy.

Over-confident people say "90%" but are right only 70%. Under-confident people say "60%" but are right 80%. Both are bad calibration.

Limitations

  • 12 rounds is enough for a rough calibration curve, not a precise one. For research-grade, you'd want 50+ rounds.
  • Some question types (e.g., counting dots) are inherently easier to calibrate than others (e.g., distance to a far city). We mix types but a per-type breakdown is in the history.

FAQ

How is "close enough" defined?

Within a factor of 2 (i.e., your estimate is between half and double the truth). That's a standard "order-of-magnitude" looseness for sanity-check estimates.

Can I get better at calibration?

Yes — there's a strong literature showing calibration improves with feedback. The trick is to treat 70% confidence as "3 out of 10 times I'm wrong."

How do I use Calibrated Estimation Game step by step without mistakes?Follow the order given in the usage section: Estimate quantities (count, distance, weight, area). We score calibration — whether your confidence matches your accuracy — instead of just right-or-wrong. The step-by-step part matters less than the unit check that sits inside it. Confirm every unit matches the thing you are actually trying to measure before you press calculate - that single check catches more wrong answers than any other habit, and it costs about ten seconds.
What are the common mistakes when using Calibrated Estimation Game?Most common mistakes come down to four habits: entering 20 when the field expects 0.2 because the field is a percentage; leaving a field blank and accepting the default without checking what it stands for; changing one input and reading a result that was calculated before the change; and quoting the output as an exact figure when the page describes it as a reference estimate. Press calculate again after any edit, and treat the number as a reference point rather than a final answer.
How do I read the result from Calibrated Estimation Game?Read the final figure first, then come back to this section for the conditions behind it: Calibration score near 0 is excellent. Below 0.05 is great. Above 0.15 means your confidence doesn't match your accuracy. Over-confident people say "90%" but are right only 70%. Under-confident people say "60%" but are right 80%. Both are bad calibration. Keep two things apart: the number itself, which comes from the values you typed, and the assumptions it depends on, which decide whether that number is meaningful. The page sets out what a higher or lower result means for your case, so check that direction as well as the magnitude.
Can I act on the Calibrated Estimation Game result directly?Treat it as one reference sheet before you decide, rather than as the answer itself: 12 rounds is enough for a rough calibration curve, not a precise one. For research-grade, you'd want 50+ rounds. Some question types (e.g., counting dots) are inherently easier to calibrate than others (e.g., distance to a far city). We mix types but a per-typ The output describes a magnitude under the conditions you entered. It does not include what it actually costs to act, what might change while you act, or how the other side behaves. Before you commit, compare the result against reality and note which assumptions do not hold where you are.