With this treatise, an insightful exploration of the probabilistic connection between philosophy and the history of science, the famous economist breathed new life into studies of both disciplines. Originally published in 1921, this important mathematical work represented a significant contribution to the theory regarding the logical probability of propositions. Keynes effectively dismantled the classical theory of probability, launching what has since been termed the “logical-relationist” theory. In so doing, he explored the logical relationships between classifying a proposition as “highly probable” (...) and as a “justifiable induction.” Unabridged republication of the classic 1921 edition. (shrink)

The mathematical theory of communication.

Information theory and statistical mechanics.

Information theory and statistical mechanics II.

Of course, the rationale of PME is so different from what has been taught in “orthodox” statistics courses for fifty years, that it causes conceptual hangups for many with conventional training. But beginning students have no difficulty with it, for it is just a mathematical model of the natural, common sense way in which anybody does conduct his inferences in problems of everyday life.The difficulties that seem so prominent in the literature today are, therefore, only transient phenomena that will disappear (...) automatically in time. Indeed, this revolution in our attitude toward inference is already an accomplished fact among those concerned with a few specialized applications; with a little familarity in its use its advantages are apparent and it no longer seems strange. It is the idea that inference was once thought to be tied to frequencies in random experiments, that will seem strange to future generations. (shrink)

The principle of maximum entropy is a method for assigning values to probability distributions on the basis of partial information. In usual formulations of this and related methods of inference one assumes that this partial information takes the form of a constraint on allowed probability distributions. In practical applications, however, the information consists of empirical data. A constraint rule is then employed to construct constraints on probability distributions out of these data. Usually one adopts the rule that equates the expectation (...) values of certain functions with their empirical averages. There are, however, various other ways in which one can construct constraints from empirical data, which makes the maximum entropy principle lead to very different probability assignments. This paper shows that an argument by Jaynes to justify the usual constraint rule is unsatisfactory and investigates several alternative choices. The choice of a constraint rule is also shown to be of crucial importance to the debate on the question whether there is a conflict between the methods of inference based on maximum entropy and Bayesian conditionalization. (shrink)

This essay is, primarily, a discussion of four results about the principle of maximizing entropy (MAXENT) and its connections with Bayesian theory. Result 1 provides a restricted equivalence between the two: where the Bayesian model for MAXENT inference uses an "a priori" probability that is uniform, and where all MAXENT constraints are limited to 0-1 expectations for simple indicator-variables. The other three results report on an inability to extend the equivalence beyond these specialized constraints. Result 2 established a sensitivity of (...) MAXENT inference to the choice of the algebra of possibilities even though all empirical constraints imposed on the MAXENT solution are satisfied in each measure space considered. The resulting MAXENT distribution is not invariant over the choice of measure space. Thus, old and familiar problems with the Laplacian principle of Insufficient Reason also plague MAXENT theory. Result 3 builds upon the findings of Friedman and Shimony (1971; 1973) and demonstrates the absence of an exchangeable, Bayesian model for predictive MAXENT distributions when the MAXENT constraints are interpreted according to Jaynes's (1978) prescription for his (1963) Brandeis Dice problem. Lastly, Result 4 generalizes the Friedman and Shimony objection to cross-entropy (Kullback-information) shifts subject to a constraint of a new odds-ratio for two disjoint events. (shrink)

Many statistical problems, including some of the most important for physical applications, have long been regarded as underdetermined from the standpoint of a strict frequency definition of probability; yet they may appear wellposed or even overdetermined by the principles of maximum entropy and transformation groups. Furthermore, the distributions found by these methods turn out to have a definite frequency correspondence; the distribution obtained by invariance under a transformation group is by far the most likely to be observed experimentally, in the (...) sense that it requires by far the least “skill.” These properties are illustrated by analyzing the famous Bertrand paradox. On the viewpoint advocated here, Bertrand's problem turns out to be well posed after all, and the unique solution has been verified experimentally. We conclude that probability theory has a wider range of useful applications than would be supposed from the standpoint of the usual frequency definitions. (shrink)

An important contribution to the foundations of probability theory, statistics and statistical physics has been made by E. T. Jaynes. The recent publication of his collected works provides an appropriate opportunity to attempt an assessment of this contribution.

The concepts of uncertainty in prediction and inference are introduced and illustrated using the diffraction of light as an example. The close relationship between the concepts of uncertainty in inference and resolving power is noted. A general quantitative measure of uncertainty in inference can be obtained by means of the so-called statistical distance between probability distributions. When applied to quantum mechanics, this distance leads to a measure of the distinguishability of quantum states, which essentially is the absolute value of the (...) matrix element between the states. The importance of this result to the quantum mechanical uncertainty principle is noted. The second part of the paper provides a derivation of the statistical distance on the basis of the so-called method of support. (shrink)