Millikan's famous oil drop experiment is scrutinized from the viewpoint of the methodological dicta of the credibility revolution.

While the history and methodology of laboratory experiments in economics have been extensively studied by philosophers, those of field experiments have not attracted much attention until recently. What is the historical context in which field experiments have been advocated? And what are the methodological rationales for conducting experiments in the field as opposed to in the lab? This article addresses these questions by combining historical and methodological perspectives. In terms of history, we show that the movement toward field experiments in (...) economics has two distinct roots. One is the general orientation of medical and social sciences to evidence-based policy evaluation, which gave rise to randomized field experiments in economics. The other is an awareness of several methodological limitations of lab experiments in economics, which required practitioners to get out of the lab and into the field. In these senses, the movement is a consequence of influences from both outside and inside economics: the general evidence-based trend in policy science and an internal methodological development of experimental economics. In terms of methodology, we show that these two roots resulted in two somewhat different notions of “external validity” as methodological rationales of field experiment. Finally, we suggest that analysis of experiments as exhibits highlights a methodological strategy in which both strands complement each other. (shrink)

Randomized experiments, as developed by Esther Duflo and Abhijit Banerjee at the Abdul Latif Jameel Poverty Action Lab, offer a novel, evidence-based approach to fighting poverty. This approach is original, in that it imports the methodology of clinical trials for application in development economics. This paper examines the analogy between J-PAL’s field experiments in development economics and randomized controlled trials in medicine. RCTs and randomized field experiments are commonly treated as identical, but such treatment neglects some of the major distinguishing (...) features that make each experiment specifically apt for use in its respective field. The central claim of this paper is that the analogy between medicine and development economics is incomplete because the central dimensions of RCTs are not simply different but altogether lacking in J-PAL’s approach. This weakens both the political and the theoretical power of such experiments in development economics. (shrink)

In this paper, we critically and constructively examine the methodology of evidence-based development economics, which deploys randomized field experiments as its main tool. We describe the...

Glenn Harrison [Journal of Economic Methodology, 2013, 20, 103–117] discusses four related forms of methodological intolerance with respect to field experiments: field experiments should rely on some form of randomization, should be disconnected from theory, the concept of causality should only be defined in terms of observables, and the role of laboratory experiments is dismissed. As is often the case, the cause of intolerance is ignorance, as it is here. To acquire knowledge about potential influences, which we need for both (...) the evaluation of internal and external validity of experimental results, we cannot do without theory. A purely empiricist methodology will be unable to give us sufficient understanding of the validity of these results. An account of causality only based on directly observed things, is an account based on factual influences only. This account will be too restricted, because it will not deal with the unobserved potential influences, which we need – again – for the evaluat... (shrink)

In ‘Field Experiments and Methodological Intolerance,’ Glenn Harrison develops a criticism of randomized field experiments and denounces ‘intolerance’ for lab experiments and a ‘disconnect from theory.’ We argue that lab experiments and RCTs are based on different methodological approaches and therefore fulfill different scientific objectives; key features of the RCT methodology make it highly relevant for policy-making, which accounts for a large part of its uptake; RCTs foster a convergence of interests between policy-makers and researchers around the study of economic (...) mechanisms, which makes it possible to challenge and improve economic theory based on real-world facts; and hybrid forms of experimentation are being increasingly used and open promising avenues. In sum, economic research based on RCTs is more and more connected to theory and other forms of experiment, while also helping policy-makers tackle daunting problems of poverty and exclusion. (shrink)