Published · Updated
The Brier score measures how well calibrated probability predictions are to actual outcomes. Lower is better. Coin flip baseline is 0.25: a model has to score below it to add information. Brier score is the gold-standard metric for evaluating a sports betting model's probability calibration.
The Brier score is the mean squared difference between predicted probability and actual outcome (0 or 1). For each prediction: (prob - outcome)^2. Average across all predictions. Lower score = better calibration.
Example: model predicts 60% probability for an outcome that happens (outcome = 1). Squared error: (0.60 - 1)^2 = 0.16.
Same model predicts 60% probability for an outcome that doesn't happen (outcome = 0). Squared error: (0.60 - 0)^2 = 0.36.
If half the 60% predictions hit, average squared error is (0.16 + 0.36) / 2 = 0.26. The model is poorly calibrated (60% predictions should hit 60% of the time, not 50%).
Hit rate alone is misleading. A model that always predicts 50% on coin flips will have 50% hit rate forever, but the predictions provide no information. Brier score penalizes overconfidence AND underconfidence. A model that predicts 80% on something that hits 65% of the time gets penalized for overconfidence even though its predictions are technically correct.
Brier score is the cleanest single number for "is this model calibrated to reality?"
Hit rate and Brier score measure different things:
A model can have decent hit rate but bad Brier score if its high-confidence predictions are systematically overconfident. Sharp bettors care about Brier score because they bet by probability, not by binary picks.
Aggregate Brier score hides important information. Better: compute Brier per probability bucket.
Per-bucket calibration tells you WHERE the model is miscalibrated. A model with strong overall Brier might still be overconfident in 55-60% predictions (check every bucket, not only the aggregate). Sharp bettors identify those buckets and skip them.
Sportsbooks track their own Brier scores on de-vigged closing lines. Closing lines are the market's best estimate, which is why beating them consistently is hard. Bettors who consistently beat the closing line are, on average, pricing in information the book had not yet absorbed, and some sportsbooks limit those accounts.
Bookie Bullies reports its overall Brier score on the track record page. The score changes as picks are graded, so check that page for the current number and compare it with the 0.25 coin-flip baseline. See the methodology page for details on the Platt-scaling recalibration applied to displayed probabilities.
The Brier score measures how well calibrated probability predictions are to actual outcomes. It's the mean squared difference between predicted probability and outcome (0 or 1). Lower is better. Coin flip baseline is 0.25: a model has to score below it to add information.
On this site's scale, 0.18-0.22 is decent, 0.22-0.25 is weak (barely better than always predicting 50%), and anything above 0.25 is worse than always predicting 50%.
Brier score is better than hit rate because hit rate alone is misleading. A model that always predicts 50% on coin flips has 50% hit rate forever but provides no information. Brier penalizes overconfidence (predicting 80% on things that hit 65%) AND underconfidence. It's the cleanest single number for 'is this model calibrated to reality?'
Four main ways to improve Brier score: (1) Platt scaling, logistic regression on historical (predicted_prob, outcome) pairs to recalibrate future predictions; (2) isotonic regression when the calibration curve is not sigmoid-shaped; (3) per-bucket recalibration for different adjustments per probability band; (4) ensemble methods combining models with different miscalibration patterns.
Bookie Bullies' overall Brier score is shown on the track record page and changes as picks are graded. Compare it with the 0.25 coin-flip baseline. The track record breakdown shows per-bucket Brier identifying which probability bands have the strongest calibration. Moneyline probabilities are recalibrated with Platt scaling and a per-bucket adjustment. First-inning reads are not Platt-scaled. See methodology for full detail.