Evidence based-policy (EBP) is a popular research paradigm in the applied social sciences and within government agencies. Informally, EBP represents an explicit commitment to applying scientific methods to public affairs, in contrast to ideologically-driven or merely intuitive “common-sense” approaches to public policy. More specifically, the EBP paradigm places great weight on the results of experimental research designs, especially randomised controlled trials (RCTs), and systematic literature reviews that place evidential weight on experimental results. One hope is that (...) such research designs and approaches to analysing the scientific literature are sufficiently robust that they can settle what really ‘works’ in public policy. Can EBP succeed in displacing reliance on domain-specific expertise? On our account, this is seldom, if ever, the case. The key reason for this is that underlying this approach is generally an appeal to argument by induction, which always requires further assumptions to underwrite its validity, and if not induction, some other argument form that also requires assumptions that are very often not validated for the case at hand. (shrink)

Acceding to the demand that public policy should be based on “the best available evidence” can come at significant moral cost. Important policy questions cannot be addressed using “the best available evidence” as defined by the evidence-based policy paradigm; the paradigm can change the meaning of questions so that they can be addressed using the preferred kind of evidence; and important evidence that does not meet the standard defined by the (...) paradigm can get ignored. We illustrate these problems in three contexts of evidence-based policy. (shrink)

This paper critically analyses the concept of evidence in evidence-based-policy arguing that there is key problem: that there is no existing practicable theory of evidence, one which is philosophically grounded and yet applicable for evidencebased policy. The paper critically considers both philosophical accounts of evidence and practical treatments of evidence in evidence-based-policy. It argues that both fail in different ways to provide a theory of evidence that is adequate (...) for evidence-basedpolicy. The paper is a valuable contribution to the part of the research project that explores how evidence can and should be used to reduce contingency in science, and policy based on science. (shrink)

Public policymakers and institutional decision-makers routinely face questions about whether interventions “work”: does universal basic income improve people’s welfare and stimulate entrepreneurial activity? Do gated alleyways reduce burglaries or merely shift the crime burden to neighbouring communities? What is the most cost-effective way to improve students’ reading abilities? These are empirical questions that seem best answered by looking at the world, rather than trusting speculations about what will be effective. Evidence-based policy (EBP) is a movement that concretizes (...) this intuition. It maintains that policy should be based on evidence of “what works.” Not any evidence will do, however. Following on the heels of its intellectual progenitor, evidence-based medicine (EBM), EBP insists on the use of high-quality evidence produced in accordance with rigorous methodological standards. Though intuitively compelling, EBP has attracted significant criticism from methodologists, political scientists, and philosophers. This chapter provides a critical overview of EBP through a philosophical lens, reviewing and discussing some of the most pressing challenges that EBP faces and outlining some proposals for improving it. I first provide a brief overview of EBP and distinguish a broader and narrower understanding of it. I then review existing lines of criticism of EBP, two of which are considered in detail. The first elaborates how EBP struggles with extrapolation, that is, the use of evidence from study populations to make inferences about the effects of policies on novel target populations. The second maintains that EBP’s methodological tenets are deeply entwined with moral and political values, which can threaten EBP’s promise to promote objectivity in policymaking. Finally, I make some proposals for how EBP could be improved going forward. (shrink)

This article critically analyses the concept of evidence in evidence‐based policy, arguing that there is a key problem: there is no existing practicable theory of evidence, one which is philosophically‐grounded and yet applicable for evidence‐based policy. The article critically considers both philosophical accounts of evidence and practical treatments of evidence in evidence‐based policy. It argues that both fail in different ways to provide a theory of evidence (...) that is adequate for evidence‐based policy. The article contributes to the debate about how evidence can and should be used to reduce contingency in science and in policy based on science. (shrink)

This paper critically analyses the concept of evidence in evidence-based-policy arguing that there is key problem: that there is no existing practicable theory of evidence, one which is philosophically grounded and yet applicable for evidencebased policy. The paper critically considers both philosophical accounts of evidence and practical treatments of evidence in evidence-based-policy. It argues that both fail in different ways to provide a theory of evidence that is adequate (...) for evidence-basedpolicy. The paper is a valuable contribution to the part of the research project that explores how evidence can and should be used to reduce contingency in science, and policy based on science. (shrink)

How can philosophy of science be of more practical use? One thing we can do is provide practicable advice about how to determine when one empirical claim is relevant to the truth of another; i.e., about evidential relevance. This matters especially for evidence-based policy, where advice is thin—and misleading—about how to tell what counts as evidence for policy effectiveness. This paper argues that good efficacy results (as in randomized controlled trials), which are all the rage (...) now, are only a very small part of the story. To tell what facts are relevant for judging policy effectiveness, we need to construct causal scenarios about will happen when the policy is implemented. (shrink)

How can philosophy of science be of more practical use? One thing we can do is provide practicable advice about how to determine when one empirical claim is relevant to the truth of another; i.e., about evidential relevance. This matters especially for evidence-based policy, where advice is thin—and misleading—about how to tell what counts as evidence for policy effectiveness. This paper argues that good efficacy results (as in randomized controlled trials), which are all the rage (...) now, are only a very small part of the story. To tell what facts are relevant for judging policy effectiveness, we need to construct causal scenarios about will happen when the policy is implemented. (shrink)

Toward evidence‐based policy decisions: a case study of nursing health human resources in Ontario, CanadaThis paper reflects how health services research ‘evidence’ was used to influence decisions in the province of Ontario, Canada. The process involved interaction among a variety of stakeholders and decision‐makers with researchers to reduce uncertainty and to substantiate emerging service provision issues in the province. The issues presented here focus specifically on an analysis of the nursing situation completed in 1998 for the (...) Minister of Health’s Nursing Task Force, which examined key issues in service delivery. The issues were: restructured work environments; nurse supply and declining enrollments; labour trends and utilization of the nursing workforce; patient acuity and complexity of work environments and the influence on workload; and the paucity of reliable and valid data bases for analysis of nursing’s contribution to the health system. Ontarians can be confident that the Task Force recommendations were born from solid research‐based evidence and now the challenge becomes to monitor the implementation of these resolutions over time. (shrink)

Evidence-based policy is gaining support in many areas of government and in public affairs more generally. In this paper we outline what evidence—based policy is then discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible account of evidence. This account needs to be at least in the spirit of the hierarchy of evidence subscribed to by evidence-based medicine (from which (...) class='Hi'>evidence—based policy derives its name and inspiration). Yet evidence-based policy’s hierarchy needs to be tailored to the kinds of evidence relevant and available to the policy arena. The evidence required for policy decisions does not easily lend itself to randomised controlled trials (the "gold standard" in evidence-based medicine), nor, for that matter, being listed in a single all—purpose hierarchy. (shrink)

Evidence-based policy has support in many areas of government and in public affairs more generally. In this paper we outline what evidence-based policy is, then we discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible, over-arching account of evidence. We contrast evidence-based policy with evidence-based medicine, especially the role of evidence in assessing the effectiveness of medicines. The (...) evidence required for policy decisions does not easily lend itself to randomized controlled trials, nor, for that matter, being listed in a single all-purpose hierarchy. (shrink)

Consider any choice that affects some social policy. A decision that considers evidence will, at its heart, contain some kind of explicit or implicit “because” statement: “We are doing X because the evidence says Y.” But can evidence ever truly speak for itself, in the sense of being reducible to objective utterances that are either correct or in need of correction? Before answering, consider what you'd prefer. Would you rather receive evidence that was free of (...) any value judgments imposed by human actors, that was laden with value judgments that you agree with, or laden with value judgments that you disagree with? The central assertion of this essay is that, throughout policy analysis but especially in assessments of the costs and benefits of regulating versus encouraging new technologies (cost‐benefit analysis, or CBA), the first possibility above is a mirage, and the second and third are self‐contradictory. Instead, we are overwhelmingly confronted with a fourth possibility: we receive evidence that appears to be (or is deliberately touted as) value neutral but is suffused with hidden value judgments.In the second part of this essay, entitled “A Guided Tour through Inevitable Value Judgments,” I identify in a systematic way approximately sixty‐five value judgments that are routinely (in some cases, invariably) made in CBA, but that are kept hidden. For each judgment, I discuss its genesis as it is most commonly invoked in CBA, explain how it is hidden in plain sight, and offer one or more value judgments that could be made instead of or in addition to the conventional one. The alternative judgments highlight the width of the spectrum of reasonable conclusions an analyst could reach merely by substituting other judgments for the ones currently embedded.Bringing hidden value judgments to light is doubly valuable. First, it allows discussion to ensue on a level playing field; instead of conclusory statements about what the evidence says, transparency permits statements taking the form of “when channeled through these value judgments, the evidence says this.” Perhaps more importantly, transparency about value judgments permits participants in the discussions to offer interpretations of evidence contingent on there being different value judgments chosen at one or more points in the analysis, interpretations that may suggest that alternative course(s) of action are preferable to the one being championed. (shrink)

This paper will critically examine the use of evidence in creating policy in the area of reproductive technologies. The use of evidence in health care and policy is not a new phenomenon. However, codified strategies for evidence appraisal in health care technology assessments and attempts to create evidence based policy initiatives suggest that the way evidence is used in practice and policy has changed. This paper will examine this trend by (...) considering what is counted as ‘good’ evidence, difficulties in translating evidence into policy and practice and how evidence interacts with principles. To illustrate these points the removal of gamete donor anonymity in the UK in 2005 and the debates that preceded this change in the law will be examined. It will be argued that evidence will only ever take us so far and attention should also be paid to the underlying principles that guide policy. The paper will conclude with suggestions for how underlying principles can be more rigorously used in policy formation. (shrink)

Evidence-based approaches to policy-making are growing in popularity. A generally embraced view is that with the appropriate evidence at hand, decision and policy making will be optimal, legitimate and publicly accountable. In practice, however, evidence-based policy making is constrained by a variety of problems of evidence. Some of these problems will be explored in this article, in the context of the debates on evidence from which they originate. It is argued (...) that the source of much disagreement might be a failure to addressing crucial philosophical assumptions that inform, often silently, these debates. Three controversial questions will be raised which appear central to some of the challenges faced by evidence-based policy making: firstly, how do certain types of facts candidate themselves as evidence; secondly, how do we decide what evidence we have, and how much of it; and thirdly, can we combine evidence. In addressing these questions it will be shown how a philosophically informed debate might prove instrumental in clarifying and settling practical difficulties. (shrink)

It has recently been argued that successful evidence-based policy should rely on two kinds of evidence: statistical and mechanistic. The former is held to be evidence that a policy brings about the desired outcome, and the latter concerns how it does so. Although agreeing with the spirit of this proposal, we argue that the underlying conception of mechanistic evidence as evidence that is different in kind from correlational, difference-making or statistical evidence, (...) does not correctly capture the role that information about mechanisms should play in evidence-based policy. We offer an alternative account of mechanistic evidence as information concerning the causal pathway connecting the policy intervention to its outcome. Not only can this be analyzed as evidence of difference-making, it is also to be found at any level and is obtainable by a broad range of methods, both experimental and observational. Using behavioral policy as an illustration, we draw the implications of this revised understanding of mechanistic evidence for debates concerning policy extrapolation, evidence hierarchies, and evidence integration. (shrink)

Proponents of evidence-based policy (EBP) call for public policy to be informed by high-quality evidence from randomized controlled trials. This methodological preference aims to promote several epistemic values, e.g. rigor, unbiasedness, precision, and the ability to obtain causal conclusions. I argue that there is a trade-off between these epistemic values and several non-epistemic, moral and political values. This is because the evidence afforded by preferred EBP methods is differentially useful for pursuing different moral and (...) political values. I expand on how this challenges ideals of value-freedom and -neutrality in EBP, and offer suggestions for how EBP methodology might be revised. (shrink)

Safety assessment of technologies and interventions is often underdetermined by evidence. For example, scientists have collected evidence concerning genetically modified plants for decades. This evidence was used to ground opposing safety protocols for “stacked genetically modified” plants, in which two or more genetically modified plants are combined. Evidence based policy would thus be rendered more effective by an approach that accounts for underdetermination. Douglas (2012) proposes an explanatory approach, based on the criteria of (...) transparency, empirical competence, internal consistency of explanations, and predictive potency. However, sometimes multiple explanations can satisfy these criteria. We propose an additional criterion based on converse abduction, where explanations are selected on the basis of ontological background assumptions as well as by evidence. We then apply our proposed scheme to the case of the regulation of stacked genetically modified plants. We discuss the implications and suggest follow-up work concerning the generalizability of the approach. (shrink)

Indikatoren sind wesentliche Teile des Monitorings und dienen in diesem Zusammenhang der politischen Steuerung auf verschiedenen Ebenen – vom lokalen Kleinprojekt bis zur globalen Governance. Sie sind ein Instrument zur Produktion von Daten als Grundlage für rationale Entscheidungen und ermöglichen so eine „evidence-based policy“. Die grundlegende Idee eines solchen Konzepts rationaler Politik besteht darin, dass politische Entscheidungsträger Indikatoren als Erhebungsinstrumente im Rahmen eines Monitoring- und Evaluationssystems einsetzen und so auf diese Weise ihre Instrumente in Richtung besserer positiver (...) Wirksamkeit und Verringerung schädlicher Nebenwirkungen lenken können. (shrink)

An evidence-based approach to crime and justice policy can go a long way toward ensuring that the best available research is considered in decisions that bear on the public good. However, the term "evidence-based" is characterized by a great deal of rhetoric. Indeed, there remains a marked disjuncture between calls for "evidence-based" policy and an understanding of what it means for policy to be "evidence-based." The calls for evidence- (...) class='Hi'>based policy nonetheless provide a powerful foundation for propelling a movement toward bringing about rational, cost-effective, and humane policies for the betterment of society. This handbook showcases the state of research on evidence-based crime and justice policy and the challenges that impede its creation and use. The volume has three core objectives: to promote new and productive ways to think about evidence-based policy; to demonstrate how research can contribute to and guide evidence-based policy in juvenile justice, criminal justice, and alternatives to system responses; and to identify strategies that can increase reliance on evidence-based policy. To meet these objectives, each chapter is guided by several central questions: What do we know about evidence-based policy and practice in crime and justice? How can we improve knowledge of evidence-based policy and practice? How can we promote more use of evidence-based policy and practice? Taken as a whole, the volume emphasizes the critical need for policies that are grounded in high-quality research, that address critical research gaps, and that fully acknowledge the limitations of what extant research can do to inform policy decisions. (shrink)

Smoking remains the leading cause of preventable disease and death in the United States, killing approximately 480,000 people each year. This crushing health burden falls disproportionately, and recent CDC data shows that large disparities in adult cigarette smoking remain. One factor in these disparities is the use of flavors. Menthol cigarettes and other flavored tobacco products are used at higher rates by vulnerable populations including youth and young adults, African Americans, women, Hispanics and Asian Americans. This is no accident; the (...) tobacco industry has long targeted these same groups. Given FDA's failure to act to ban flavored tobacco products, states and municipalities are considering taking matters into their own hands to protect their communities from these dangerous products. The authors will explore state and local authority to restrict the sales of these products and review the evidence base indicating that removing flavored tobacco products – and menthol combustible products in particular - from the market would significantly reduce the toll of illness and death caused by these products. (shrink)

WE AIM HERE to outline a theory of evidence for use. More specifically we lay foundations for a guide for the use of evidence in predicting policy effectiveness in situ, a more comprehensive guide than current standard offerings, such as the Maryland rules in criminology, the weight of evidence scheme of the International Agency for Research on Cancer (IARC), or the US ‘What Works Clearinghouse’. The guide itself is meant to be well-grounded but at the same (...) time to give practicable advice, that is, advice that can be used by policy-makers not expert in the natural and social sciences, assuming they are well-intentioned and have a reasonable but limited amount of time and resources available for searching out evidence and deliberating. (shrink)

With the UK government considering a ban on the prescription of homeopathic remedies on the NHS, John Worrall examines the rationale for such a proposal and suggests that the decision is not as simple as it might initially seem.

What’s so bad about the placebo effect? John Worrall discusses the recent Nurofen labelling “scandal”.

This article tries to present a broad view on the values and ethicalissues that are at stake in efforts to rationalize health policy on thebasis of economic evaluations (like cost-effectiveness analysis) andrandomly controlled clinical trials. Though such a rationalization isgenerally seen as an objective and `value free' process, moral valuesoften play a hidden role, not only in the production of `evidence', butalso in the way this evidence is used in policy making. For example, thedefinition of effectiveness (...) of medical treatment or health care serviceis heavily dependent on dominant individual or social views about thegoals of the particular treatment or service. There is also a concernthat a reliance on EBM in health policy will occur at the expense ofwidely shared social values like equity and solidarity. Moreover, thereis a concern that when economic considerations and rational proceduresbecome more influential, various `outside' groups third parties likeinsurance companies and policy makers will get a stronger influence onmedical practice which may lead to a change in the patient-providerrelationship. The authors conclude that social values and patientpreference should be explicitly addressed when health policy making isbased on economic and other scientific evidence. (shrink)

In this paper we argue for a naturalistic solution to some of the methodological controversies in regulatory science, on the basis of two case studies: toxicology and health claim regulation. We analyze the debates related to the scientific evidence that is considered necessary for regulatory decision making in each of those two fields, with a particular attention to the interactions between scientific and regulatory aspects. This analysis allows us to identify two general stances in the debate: a) one that (...) argues for more permissive standards of evidence and for methodological pluralism, and b) an opposing one that not only defends strict evidence requirements but also stipulates the use of one particular scientific methodologies for data generation. We argue that the real-world outcomes produced by alternative regulatory options are a vital piece of information that allows for the empirical assessment of these two stances. In particular, this information on outcomes makes it possible to analyze which standards of evidence and scientific methods generate the most useful knowledge as input for regulatory decision making. Our conclusion is that instead of an a priori selection of methodologies and standards, such decisions ought to be based on empirical evidence related to real-world outcomes. (shrink)

In the first part of this paper, I will sketch the main features of traditional models of evidence, indicating idealizations in such models that I regard as doing more harm than good. I will then proceed to elaborate on an alternative model of evidence that is functionalist, complex, dynamic, and contextual, which I will call DYNAMIC EVIDENTIAL FUNCTIONALISM. I will demonstrate its application to an illuminating example of scientific inquiry, and defend it from some likely objections. In the (...) second part, I will use that alternative to solve a variety of classic and contemporary problems in the literature on scientific evidence having to do with the empirical basis of science and the use of evidence in public policy. (shrink)

This article draws upon issues within the debate on evidence-based policy raised in the academic literature and in recent government documentation. The article assesses the extent to and ways in which policy development and implementation on adult basic skills (literacy and numeracy) within the National Probation Service (NPS) are evidencebased. It is argued that the albeit limited amount of empirical evidence on adult basic skills, methodological insights gained through empirical research, and expert opinion have shaped (...) the NPS basic skills strategy. To enhance policy implementation, findings from the NPS pathfinder evaluation have been disseminated to practitioners. It is aimed to build research evidence on basic skills in the NPS in a cumulative way and to use findings to shape policy development on an ongoing basis. The lack of a systematic review on adult basic skills is used to explore reasons for gaps in evidence-based policy on basic skills in the NPS. The article concludes that a range of types of evidence has shaped policy development on basic skills in the NPS. (shrink)

This article examines the conventional Rehnist interpretation of active labor market policy in Sweden during the period from the war to the early 1970s and asks how it relates to the basic principles of evidence-based social science research.It is shown that the interpretation is misleading, reproducing and communicating material, stories, and images that were supplied bysome of the leading Swedish actors rather than producing an analysis based on the empirical examination of actual developments.The conclusion of the (...) article offers suggestions for an alternative evidencebased interpretation of the Swedish case. (shrink)

This article briefly reviews the various papers contained in this volume. They were originally presented at a research workshop held at Keele University in the UK in February 2003. It is suggested that the different papers raise a series of related legal, social and ethical issues and can be collectively seen to demonstrate the fact that policy formation in relation to reproductive matters is highly contested. It is concluded that ethical policy formation in this area needs to be (...) based on actual evidence of harm rather than assumed harm and that this therefore entails more empirical research into reproductive matters. (shrink)

Several countries maintain universal health coverage, which implies responsibility to organize delivery formats of healthcare services and products for citizens. In Brazil, the health system has a principle of universal access for more than 30 years, but many deficiencies remain and the country observes a day practice for those seeking judicial decisions to determine provision of healthcare.

To help eliminate perinatal HIV transmission, the US Department of Health and Human Services recommends against breastfeeding for women living with HIV, regardless of viral load or combined antiretroviral therapy status. However, cART radically improves HIV prognosis and virtually eliminates perinatal transmission, and breastfeeding's health benefits are well-established. In this setting, pregnancy is increasing among American women with HIV, and a harm reduction approach to those who breastfeed despite extensive counseling is suggested. We assess the evidence and ethical justification (...) for current policy, with attention to pertinent racial and health disparities. We first review perinatal transmission and breastfeeding data relevant to US infants. We compare hypothetical risk of HIV transmission from breastmilk to increased mortality from sudden infant death syndrome, necrotizing enterocolitis and sepsis from avoiding breastfeeding, finding that benefits may outweigh risks if mothers maintain undetectable viral load on cART. We then review maternal health considerations. We conclude that avoidance of breastfeeding by women living with HIV may not maximize health outcomes and discuss our recommendation for revising national guidelines in light of autonomy, harm reduction and health inequities. (shrink)

The COVID-19 pandemic poses extraordinary public health challenges. In order to respond to such challenges, most democracies have relied on so-called ‘evidence-based’ policies, which supposedly devolve to science the burden of their justification. However, the biomedical sciences can only provide a theory-laden evidential basis, while reliable statistical data for policy support is often scarce. Therefore, scientific evidence alone cannot legitimise COVID-19 public health policies, which are ultimately based on political decisions. Given this inevitable input on (...) policy-making, the risk of arbitrariness is ubiquitous and democratic scrutiny becomes essential to counter it. During the COVID-19 pandemic, the standards of scientific and democratic scrutiny have been, as a matter of fact, substantially lowered. This erosion potentially damages democracy. (shrink)

In my response to Andrew Webster's examples I point to certain limitations, while fully supporting the thrust of his argument for a re-engagement of science and technology studies with policy making. When analyzing the policy implications of knowledge, the larger context must be considered. New criteria, like transparency, have arisen and the tendency for evidence-based policy making has become widespread. The managerial side of policy making emphasizes that "only what can be measured, can be (...) managed." The crucial question is how well prepared-and how willing-the STS community is to tackle policy making in this new guise. In the end, what is needed are institutions that are capable of developing their own reflexivity-a task to which STS can certainly contribute, but the road to be taken may be less direct than Andrew Webster suggests. (shrink)

Evidence-based policy-making (EBP) is widely seen as a key instrument for good policy-making. Yet in the wake of the Covid-19 pandemic, the debate on the relation of science and policy-making gained new momentum. The premises of EBP, its narrow understanding of what kind of knowledge counts and how to make decisions, appeared inapt to provide a sound foundation for policy-making under conditions of high complexity and uncertainty. This paper addresses the major shortcomings of EBP (...) and argues for revising its evidentialist framework. First, the idea of evidence itself has to be extended and pluralized by including alternative forms of knowledge. Second, uncertainty has to be taken seriously as a fundamental epistemic condition. To arrive at a more apt model of policy-making, this paper suggests to adopt a pragmatist perspective on the relation of politics, science and the public. This leads to an experimentalist account in the meliorist spirit. Such an account highlights the role of experience, participatory processes and collective learning. Furthermore, and in contrast to scientistic models, it underlines political responsibility and demands to cultivate sensitivity for possible failures and tragic constellations. Finally, a non-evidentialist model of policy advice is outlined. (shrink)