Letter to the
editor, The American Statistician, Vol. 25, No. 5 (Dec., 1971), pp. 62–63.
Dear Sir:
46656 Varieties of Bayesians
Some attacks and defenses of the Bayesian position assume that it is unique so it should be helpful to point out that there are at least 46656 different interpretations. This is shown by the following classification based on eleven facets. The count would be larger if I had not artificially made some of the facets discrete and my heading would have been “On the Infinite Variety of Bayesians”.
All Bayesians, as I understand the term, believe that it is usually meaningful to talk about the probability of a hypothesis and they make some attempt to be consistent in their judgments. Thus von Mises (10) would not count as a Bayesian, on this definition. For he considered that Bayes' theorem is applicable only when the prior is itself a physical probability distribution based on a large sample from a superpopulation. If he is counted as a Bayesian, then there are at least 46657 varieties, which happens to rhyme with the number of Heinz varieties. But no doubt both numbers will increase on a recount.
Here are the eleven facets:
1. Type II rationality. (a) Consciously recognized; (b) not. Here Type II rationality is defined as the recommendation to maximize expected utility allowing for the cost of theorizing (4). It involves the recognition that judgments can be revised, leading at best to consistency of mature judgments.
2. Kinds of judgments. (a) Restricted to a specific class or classes, such as preferences between actions; (b) all kinds permitted, such as of probabilities and utilities, and any functions of them such as expected utilities, weights of evidence, likelihoods, surprise indices (2), (3). This facet could of course be broken up into a large number.
3. Precision of judgments. (a) Sharp; (b) based on inequalities, i.e. partially ordered; (c) partially ordered, but sharp judgments often assumed for the sake of simplicity (in accordance with l(a)).
4. Extremeness. (a) Formal Bayesian procedure recommended for all applications; (b) non-Bayesian methods used provided that some set of axioms of intuitive probability are not seen to be contradicted (the Bayes/non-Bayes compromise: Hegel and Marx would call it a synthesis); (c) non-Bayesian methods used only after they have been given a rough Bayesian justification.
5. Utilities. (a) Brought in from the start; (b) avoided, as by Jeffreys (9); (c) utilities introduced separately from intuitive probabilities.
6. Quasi-utilities, (a) Only one kind of utility recognized; (b) explicit recognition that "quasi-utilities" (7) (8) are worth using, such as amounts of information or "weights of evidence" (Peirce (11), Good (1)); (c) using quasi-utilities without noticing that they are substitutes for utilities. The use of quasi-utilities is as old as the words "information" and "evidence", but I think
the name "quasi-utility" serves a useful purpose in focussing the Issue.
7. Physical probabilities. (a) Assumed to exist; (b) denied; (c) used as if they exist but without philosophical commitment (6).
8. Intuitive probability. (a) Subjective probabilities regarded as primary; (b) credibilities (logical probabilities) primary; (c) regarding it as mentally healthy to think of subjective probabilities as estimates of credibilities, without being sure that credibilities really exist; (d) credibilities in principle definable by an international body (with Harold Jeffreys as the chairman).
9. Device of imaginary results. (a) Explicit use; (b) not. The device involves imaginary experimental results used for judging final or posterior probabilities from which are inferred discernments about the initial probabilities. For examples see (1), (5).
10. Axioms. (a) As simple as possible; (b) incorporating Kolmogorov's axiom (complete additivity); (c) use Kolmogorov's axiom when mathematically convenient but regarding it as barely relevant to the philosophy of applied statistics.
11. Probability "types". (a) Consider that priors can have parameters with “Type III” distributions, as a convenient technique for making judgments; (b) not. Here (a) leads, by a compromise with non-Bayesian statistics, to such techniques as Type II maximum likelihood and Type II likelihood-ratio tests (5).
Thus there are at least 24.36.4 = 46656 categories. This is more than the number of professional statisticians so some of the categories must be empty. Thomas Bayes hardly wrote enough to be properly categorized; a partial attempt is b- -aaa?-b--. My own category is abcbcbccaca. What's yours?
Sincerely,
I. J. Good
University Professor
Virginia Polytechnic Inst.
and State University
REFERENCES
1. Good, I. J. Probability and the Weighing of Evidence, Charles Griffin, London; Hafners, New York; 1950, pp.119.
2. Good, I. J. Contribution to the discussion on R.D. Clark's paper "The Concept of Probability", J. Inst. Actuaries, 80 (1954), 17-18.
3. Good,
4. Good, I. J. "How
Rational should a Manager be?" Management
Science, 8, (1962) 383-393. Reprinted, with numerous minor improvements, in
Executive Readings in Management Science,
5. Good,
6. Good, I. J. Discussion of de Finetti's paper at the 1968 Colloquium in Philosophy of Science, Synthese 20 (1969), 17-24; reprinted in Induction, Physics and Ethics (ed. P. Weingartner and G. Zecha), D. Reidel, Dordrecht, 1970; pp.18-25.
7. Good, I J. Information, Rewards, and Quasi-utilities, in Science, Decision, and Value (ed. J. J. Leach), D. Reidel, Dordrecht; in the press. (Based on an invited lecture at he Second World Congress of the Econometric Society, 1970.)
8. Good,
9. Jeffreys,
H. Theory of Probability, Clarendon Press,
10. Mises, R. von, On the Correct use of Bayes's Formula, Annals of Mathematical Statistics, 13 (1942),156-165.
11. Peirce, C. S. The Probability of Induction, Popular Science Monthly.
12, (1878). 705-718. Reprinted in C. Hartshorne
and P. Weiss (eds.), Collected Papers of Charles Saunders Peirce, Volume
2, 4th edn. 415–423, The Belknap Press of