The no-free-lunch theorems promote a skeptical conclusion that all possible machine learning algorithms equally lack justification. But how could this leave room for a learning theory, that shows that some algorithms are better than others? Drawing parallels to the philosophy of induction, we point out that the no-free-lunch results presuppose a conception of learning algorithms as purely data-driven. On this conception, every algorithm must have an inherent inductive bias, that wants justification. We argue that many standard learning algorithms should rather (...) be understood as model-dependent: in each application they also require for input a model, representing a bias. Generic algorithms themselves, they can be given a model-relative justification. (shrink)

I critically discuss a recent innovation in the debate surrounding the plausibility of rational choice theory : the appeal to evolutionary theory. Specifically, I assess Gigerenzer and colleagues’ claim that considerations based on natural selection show that, instead of making decisions in a RCT-like way, we rely on ‘simple heuristics’. As I try to make clearer here, though, Gigerenzer and colleagues’ arguments are unconvincing: we lack the needed information about our past to determine whether the premises on which they are (...) built are true—and, hence, we cannot tell whether they, in fact, speak against RCT. (shrink)

Is there some general reason to expect organisms that have beliefs to have false beliefs? And after you observe that an organism occasionally occupies a given neural state that you think encodes a perceptual belief, how do you evaluate hypotheses about the semantic content that that state has, where some of those hypotheses attribute beliefs that are sometimes false while others attribute beliefs that are always true? To address the first of these questions, we discuss evolution by natural selection and (...) show how organisms that are risk-prone in the beliefs they form can be fitter than organisms that are risk-free. To address the second question, we discuss a problem that is widely recognized in statistics – the problem of over-fitting – and one influential device for addressing that problem, the Akaike Information Criterion (AIC). We then use AIC to solve epistemological versions of the disjunction and distality problems, which are two key problems concerning what it is for a belief state to have one semantic content rather than another. (shrink)

Given free information and unlimited processing power, should decision algorithms use as much information as possible? A formal model of the decision-making environment is developed to address this question and provide conditions under which informationally frugal algorithms, without any information or processing costs whatsoever, are optimal. One cause of compression that allows optimal algorithms to rationally ignore information is inverse movement of payoffs and probabilities (e.g., high payoffs occur with low probably and low payoffs occur with high probability). If inversely (...) related payoffs and probabilities cancel out, then predictors that correlate with payoffs and consequently condition the probabilities associated with different payoffs will drop out of the expected-payoff objective function, severing the link between information and optimal action rules. Stochastic payoff processes in which rational ignoring occurs are referred to as compressed environments, because optimal action depends on a reduced-dimension subset of the environmental parameters. This paper considers benefits and limitations of economic models versus other methods for studying links between environmental structure and the real-world success of simple decision procedures. Different methods converge on the normative proposition of ecological rationality, as opposed to axiomatic rationality based on informational efficiency and internal consistency axioms, as a superior framework for comparing the effectiveness of decision strategies and prescribing decision algorithms in application. (shrink)

Intellectual progress involves forming a more accurate picture of the world. But it also figuring out which concepts to use for theorizing about the world. Bayesian epistemology has had much to say about the former aspect of our cognitive lives, but little if at all about the latter. I outline a framework for formulating questions about conceptual change in a broadly Bayesian framework. By enriching the resources of Epistemic Utility Theory with a more expansive conception of epistemic value, I offer (...) a picture of our cognitive economy on which adopting new conceptual tools can sometimes be epistemically rational. (shrink)