The Relevance of Proofs of the Rationality of Probability Theory to Automated Reasoning and Cognitive Models
This is an incremental critique for researchers in AI and cognitive science, questioning the practical applicability of foundational probability theory results.
The paper critiques the relevance of proofs like Cox's and de Finetti's theorems to automated reasoning and cognitive models, arguing they do not ensure useful or rational probabilistic models for specific tasks, as illustrated by examples where reasonable models differ significantly.
A number of well-known theorems, such as Cox's theorem and de Finetti's theorem. prove that any model of reasoning with uncertain information that satisfies specified conditions of "rationality" must satisfy the axioms of probability theory. I argue here that these theorems do not in themselves demonstrate that probabilistic models are in fact suitable for any specific task in automated reasoning or plausible for cognitive models. First, the theorems only establish that there exists some probabilistic model; they do not establish that there exists a useful probabilistic model, i.e. one with a tractably small number of numerical parameters and a large number of independence assumptions. Second, there are in general many different probabilistic models for a given situation, many of which may be far more irrational, in the usual sense of the term, than a model that violates the axioms of probability theory. I illustrate this second point with an extended examples of two tasks of induction, of a similar structure, where the reasonable probabilistic models are very different.