Calibration, coherence, and scoring rules

Philosophy of Science 52 (2):274-294 (1985)
  Copy   BIBTEX

Abstract

Can there be good reasons for judging one set of probabilistic assertions more reliable than a second? There are many candidates for measuring "goodness" of probabilistic forecasts. Here, I focus on one such aspirant: calibration. Calibration requires an alignment of announced probabilities and observed relative frequency, e.g., 50 percent of forecasts made with the announced probability of.5 occur, 70 percent of forecasts made with probability.7 occur, etc. To summarize the conclusions: (i) Surveys designed to display calibration curves, from which a recalibration is to be calculated, are useless without due consideration for the interconnections between questions (forecasts) in the survey. (ii) Subject to feedback, calibration in the long run is otiose. It gives no ground for validating one coherent opinion over another as each coherent forecaster is (almost) sure of his own long-run calibration. (iii) Calibration in the short run is an inducement to hedge forecasts. A calibration score, in the short run, is improper. It gives the forecaster reason to feign violation of total evidence by enticing him to use the more predictable frequencies in a larger finite reference class than that directly relevant

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 90,616

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Analytics

Added to PP
2009-01-28

Downloads
117 (#141,542)

6 months
33 (#90,518)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Teddy Seidenfeld
Carnegie Mellon University

Citations of this work

A nonpragmatic vindication of probabilism.James M. Joyce - 1998 - Philosophy of Science 65 (4):575-603.
What is justified credence?Richard Pettigrew - 2021 - Episteme 18 (1):16-30.
Accuracy and Coherence: Prospects for an Alethic Epistemology of Partial Belief.James M. Joyce - 2009 - In Franz Huber & Christoph Schmidt-Petri (eds.), Degrees of Belief. Synthese. pp. 263-297.
Being Rational and Being Wrong.Kevin Dorst - 2023 - Philosophers' Imprint 23 (1).

View all 25 citations / Add more citations

References found in this work

The Foundations of Statistics.Leonard J. Savage - 1954 - Synthese 11 (1):86-89.
Coherence and the axioms of confirmation.Abner Shimony - 1955 - Journal of Symbolic Logic 20 (1):1-28.
Elicitation of Personal Probabilities and Expectations.Leonard Savage - 1971 - Journal of the American Statistical Association 66 (336):783-801.
Subjective probability: Criticisms, reflections, and problems.H. Kyburg - 1978 - Journal of Philosophical Logic 7 (1):157 - 180.

View all 11 references / Add more references