Every forecast is scored in public when it resolves.

Cutover: engine fix deployed Aug 5, 2026. 185 forecasts on the record in total (131 pre-fix + 54 post-fix).

Scoreable on the record: 15 of 54 (current engine) and 17 of 131 (pre-fix), as of 2026-10-21 09:37 UTC. “Scoreable” means bound to a named public series and an explicit threshold that can settle it — the only way we will ever put a Brier score against it. A forecast that is not scoreable is not pending; it will never produce a number, and we would rather print that than let an empty column imply patience. Generation itself fails sometimes and we publish that too: 236 runs attempted, 51 failed (21.6%).

How scoring works

1. Forecast

   p(YES) timestamped at issue

2. Bound series + threshold

   public series named · rule written

3. Resolution date

   fixed in advance or on publication

4. Observed value

   read from the series · YES / NO

5. Brier

   (p − outcome)² · published

A forecast whose series stops publishing cannot resolve and is shown as unscoreable, not dropped.

• Timestamped at generation. A forecast is stamped when it is written. The database does not yet lock a forecast row against later edits; the prediction-market ledger, sealed by hash before settlement, is the record that cannot move.
• Distributions, not guesses. Forum Forecast decomposes a question into driver hypotheses, retrieves cited evidence for each, and runs a Monte-Carlo simulation in which each driver's estimated effect is scaled by its estimated weight. It draws 10,000 samples when the request asks for a confidence target of 0.9 or higher and 1,000 otherwise, and publishes a probability distribution with a 95% sampling interval, not a single number. 183 of 185 on the record used 10,000 samples; the rest used as few as 1,000, which is what the engine does when a caller asks for a lower confidence target.
• Brier-scored on resolution. When the outcome is known, we score it. 0 = perfect; 0.25 = an uninformed 50/50 on a binary; lower is better. Multi-outcome baselines differ — we publish the method, not just the number.
• In public. The track record is attached to every forecast so you can judge the model before you rely on it.
• Commit-reveal, where it applies. Forecasts on the public prediction-market ledger are sealed as a salted SHA-256 commitment before the market settles and revealed afterwards, so a changed number would produce a different hash. Forum Forecast outputs in the product are timestamped rows, not hashed commitments. The ledger.

By forecast type

Both cohorts combined — see the pre-fix note above.

See the full methodology, the commit–reveal ledger and our citation discipline.

Forecasts and the ledger are readable on the free tier. Start free