Abstract

Simulation evidence obtained within a Bayesian model of price-setting in a betting market, where anonymous gamblers queue to bet against a risk-neutral bookmaker, suggests that a gambler who wants to maximize future profits should trade on the advice of the analyst cum probability forecaster who records the best probability score, rather than the highest trading profits, during the preceding observation period. In general, probability scoring rules, specifically the log score and better known “Brier” (quadratic) score, are found to have higher probability of ranking rival analysts in predetermined “correct” order than either (i) the more usual method of counting categorical forecast errors (misclassifications), or (ii) an economic measure of forecasting success, described here as the “Kelly score” and defined as the trading profits accumulated by making log optimal bets (i.e. Kelly betting) against the market maker based on the probability forecasts of the analyst being assessed. This runs counter to the conventional wisdom that financial forecasts are more aptly evaluated in terms of their financial consequences than by an abstract non-monetary measure of statistical accuracy such as the number of misclassifications or a probability score