The problem of collecting, analyzing and evaluating evidence on adverse drug reactions (ADRs) is an example of the more general class of epistemological problems related to scientific inference and prediction, as well as a central problem of the health-care practice. Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special (...) attention to the privileged role assigned to randomized controlled trials in Evidence Based Medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. This point has been emphasized by epidemiologists in the last decade. Their main focus is methodological, and regards the fact that bias and confounding do not affect studies on intended and unintended effects in the same way. However, deeper concerns ground the intuition for such a distinction; these are related to the constraints which we impose on evidence and their epistemological justification. My thesis is that such constraints ought to be understood to be different in the case of evidence for risk vs. benefit assessment. I present the recent debate on the causal association between acetaminophen and asthma in order to illustrate the point at issue. (shrink)

The claims of randomized controlled trials to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This article describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive (...) method. It argues that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The article reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

The claims of RCTs to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This paper describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. (...) It argues that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The paper reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

The claims of RCTs to be the gold standard rest on the fact that the ideal RCT is a deductive method: if the assumptions of the test are met, a positive result implies the appropriate causal conclusion. This is a feature that RCTs share with a variety of other methods, which thus have equal claim to being a gold standard. This paper describes some of these other deductive methods and also some useful non-deductive methods, including the hypothetico-deductive method. (...) It argues that with all deductive methods, the benefit that the conclusions follow deductively in the ideal case comes with a great cost: narrowness of scope. This is an instance of the familiar trade-off between internal and external validity. RCTs have high internal validity but the formal methodology puts severe constraints on the assumptions a target population must meet to justify exporting a conclusion from the test population to the target. The paper reviews one such set of assumptions to show the kind of knowledge required. The overall conclusion is that to draw causal inferences about a target population, which method is best depends case-by-case on what background knowledge we have or can come to obtain. There is no gold standard. (shrink)

We are less optimistic than Madole & Harden that family-based genome-wide association studies (GWASs) will lead to significant second-generation causal knowledge. Despite bearing some similarities, family-based GWASs and randomised controlled trials (RCTs) are not identical. Most RCTs assess a relatively homogenous causal stimulus as a treatment, whereas GWASs assess highly heterogeneous causal stimuli. Thus, GWAS results will not translate so easily into second-generation causal knowledge.

Error statistics is an important methodological view in philosophy of statistics and philosophy of science that can be applied to scientific experiments such as clinical trials. In this paper, I raise two potential issues for ES when it comes to guiding, and explaining early stopping of randomized controlled trials : ES provides limited guidance in cases of early unfavorable trends due to the possibility of trend reversal; ES is silent on how to prospectively control error rates in experiments requiring multiple (...) interim analyses. The method of conditional power, together with a rationing principle for RCTs, can assist ES in addressing such issues. (shrink)

This presentation addresses these questions: • “Upon what ethical grounds may the physician offer RCT enrollment to a patient?” • Which is the preferred moral basis of the RCT?

This article brings a historical perspective to explain the recent dissemination of randomized controlled trials (RCTs) as the new “gold standard” method to assess international development projects. Although the buzz around RCT evaluations dates from the 2000s, we show that what we are witnessing now is a second wave of RCTs, while a first wave began in the 1960s and ended by the early 1980s. Drawing on content analysis of 123 RCTs, participant observation, and secondary sources, we (...) compare the two waves in terms of the participants in the network of expertise required to carry out field experiments and the characteristics of the projects evaluated. The comparison demonstrates that researchers in the second wave were better positioned to navigate the political difficulties caused by randomization. We explain the differences in the expertise network and in the type of projects as the result of concurrent transformations in the fields of development aid and the economics profession. We draw on Andrew Abbott’s concept of “hinges,” as well as on Bourdieu’s concept of “homology” between fields, to argue that the similar positions and parallel struggles conducted by two groups of actors in the two fields served as the basis for a cross-field alliance, in which RCTs could function as a “hinge” linking together the two fields. (shrink)

For evidence-based practice and policy, randomised controlled trials (RCTs) are the current gold standard. But exactly why? We know that RCTs do not, without a series of strong assumptions, warrant predictions about what happens in practice. But just what are these assumptions? I maintain that, from a philosophical stance, answers to both questions are obscured because we don't attend to what causal claims say. Causal claims entering evidence-based medicine at different points say different things and, I would suggest, (...) failure to attend to these differences makes much current guidance about evidence for medical and social policy misleading. (shrink)

This article brings a historical perspective to explain the recent dissemination of randomized controlled trials (RCTs) as the new “gold standard” method to assess international development projects. Although the buzz around RCT evaluations dates from the 2000s, we show that what we are witnessing now is a second wave of RCTs, while a first wave began in the 1960s and ended by the early 1980s. Drawing on content analysis of 123 RCTs, participant observation, and secondary sources, we (...) compare the two waves in terms of the participants in the network of expertise required to carry out field experiments and the characteristics of the projects evaluated. The comparison demonstrates that researchers in the second wave were better positioned to navigate the political difficulties caused by randomization. We explain the differences in the expertise network and in the type of projects as the result of concurrent transformations in the fields of development aid and the economics profession. We draw on Andrew Abbott’s concept of “hinges,” as well as on Bourdieu’s concept of “homology” between fields, to argue that the similar positions and parallel struggles conducted by two groups of actors in the two fields served as the basis for a cross-field alliance, in which RCTs could function as a “hinge” linking together the two fields. (shrink)

In this article, I ask whether a principle analogous to the principle of clinical equipoise should govern the design and conduct of RCTs evaluating the effectiveness of policy interventions. I answer this question affirmatively, and introduce and defend the principle of policy equipoise. According to this principle, all arms of a policy RCT must be, at minimum, in a state of equipoise with the best proven policy that is also morally and practically attainable and sustainable. For all arms of (...) a policy RCT, policy experts must either reasonably disagree about whether the trial arms are more effective than this policy, or know that they are. (shrink)

BackgroundRandomized controlled trials (RCTs) are often complex and expensive to perform. Less than one third achieve planned recruitment targets, follow-up can be labor-intensive, and many have limited real-world generalizability. Designs for RCTs conducted using cohorts and routinely collected health data, including registries, electronic health records, and administrative databases, have been proposed to address these challenges and are being rapidly adopted. These designs, however, are relatively recent innovations, and published RCT reports often do not describe important aspects of their (...) methodology in a standardized way. Our objective is to extend the Consolidated Standards of Reporting Trials (CONSORT) statement with a consensus-driven reporting guideline for RCTs using cohorts and routinely collected health data.MethodsThe development of this CONSORT extension will consist of five phases. Phase 1 (completed) consisted of the project launch, including fundraising, the establishment of a research team, and development of a conceptual framework. In phase 2, a systematic review will be performed to identify publications (1) that describe methods or reporting considerations for RCTs conducted using cohorts and routinely collected health data or (2) that are protocols or report results from such RCTs. An initial “long list” of possible modifications to CONSORT checklist items and possible new items for the reporting guideline will be generated based on the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) and The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statements. Additional possible modifications and new items will be identified based on the results of the systematic review. Phase 3 will consist of a three-round Delphi exercise with methods and content experts to evaluate the “long list” and generate a “short list” of key items. In phase 4, these items will serve as the basis for an in-person consensus meeting to finalize a core set of items to be included in the reporting guideline and checklist. Phase 5 will involve drafting the checklist and elaboration-explanation documents, and dissemination and implementation of the guideline.DiscussionDevelopment of this CONSORT extension will contribute to more transparent reporting of RCTs conducted using cohorts and routinely collected health data. (shrink)

Statistical power is often a concern for clustered randomized control trials (RCTs) due to variance inflation from design effects and the high cost of adding study clusters (such as hospitals, schools, or communities). While covariate pre-specification can improve power for estimating regression-adjusted average treatment effects (ATEs), further precision gains can be achieved through covariate selection once primary outcomes have been collected. This article uses design-based methods underlying clustered RCTs to develop Lasso methods for the post-hoc selection of covariates (...) for ATE estimation that avoids a lack of transparency and model overfitting. Our focus is on two-stage estimators: in the first stage, Lasso estimation is conducted using data on cluster-level averages or sums, and in the second stage, standard ATE estimators are adjusted for covariates using the first-stage Lasso results. We discuss l 1 {l}_{1} consistency of the estimated Lasso coefficients, asymptotic normality of the ATE estimators, and design-based variance estimation. The nonparametric approach applies to continuous, binary, and discrete outcomes. We present simulation results and demonstrate the method using data from a federally funded clustered RCT testing the effects of school-based programs promoting behavioral health. (shrink)

Randomized controlled trials sometimes test interventions that aim to improve existing services targeted to a subset of individuals identified after randomization. Accordingly, the treatment could affect the composition of service recipients and the offered services. With such bias, intention-to-treat estimates using data on service recipients and nonrecipients may be difficult to interpret. This article develops causal estimands and inverse probability weighting estimators for complier populations in these settings, using a generalized estimating equation approach that adjusts the standard errors for estimation (...) error in the IPW weights. While our focus is on more general clustered RCTs, the methods also apply to nonclustered RCTs. Simulations show that the estimators achieve nominal confidence interval coverage under the assumed identification conditions. An empirical application demonstrates the methods using data from a large-scale RCT testing the effects of early childhood services on children’s cognitive development scores. An R program for estimation is available for download. (shrink)

Despite efforts from regulatory agencies (e.g. NIH, FDA), recent systematic reviews of randomised controlled trials (RCTs) show that top medical journals continue to publish trials without requiring authors to report details for readers to evaluate early stopping decisions carefully. This article presents a systematic way of modelling and simulating interim monitoring decisions of RCTs. By taking an approach that is both general and rigorous, the proposed framework models and evaluates early stopping decisions of RCTs based on a (...) clear and consistent set of criteria. The framework allows decision analysts to generate and quickly answer ‘what-if’ questions by simulating alternate trial scenarios. I illustrate the framework with a case study of an RCT that was stopped early due to harm. This was a trial of vitamin A supplement in relation to HIV transmission from mother-to-child through breastfeeding. (shrink)

The article covers the simulation approach as a part of communicative-oriented teaching, where its main task is to remove psychological barriers in education leading to the fast and effective language learning. The main aim of this method is to provide a life experience through the modern educational environment. Modeling creates situations which helps us to try other strategies while sinking into screenplay. An example in the article we consider a linguo-didactic resource "Время говорить по-русски". The research relevant is vivid: the (...) traditional forms of education are being replaced by the modern technologies. (shrink)

Medical scientists employ ‘quality assessment tools’ (QATs) to measure the quality of evidence from clinical studies, especially randomized controlled trials (RCTs). These tools are designed to take into account various methodological details of clinical studies, including randomization, blinding, and other features of studies deemed relevant to minimizing bias and error. There are now dozens available. The various QATs on offer differ widely from each other, and second-order empirical studies show that QATs have low inter-rater reliability and low inter-tool reliability. (...) This is an instance of a more general problem I call the underdetermination of evidential significance. Disagreements about the strength of a particular piece of evidence can be due to different—but in principle equally good—weightings of the fine-grained methodological features which constitute QATs. (shrink)

Randomized controlled trials (RCTs) are widely taken as the gold standard for establishing causal conclusions. Ideally conducted they ensure that the treatment ‘causes’ the outcome—in the experiment. But where else? This is the venerable question of external validity. I point out that the question comes in two importantly different forms: Is the specific causal conclusion warranted by the experiment true in a target situation? What will be the result of implementing the treatment there? This paper explains how the probabilistic (...) theory of causality implies that RCTs can establish causal conclusions and thereby provides an account of what exactly that causal conclusion is. Clarifying the exact form of the conclusion shows just what is necessary for it to hold in a new setting and also how much more is needed to see what the actual outcome would be there were the treatment implemented. (shrink)

An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. (...) Different meta-analyses of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

The problem of collecting, analyzing and evaluating evidence on adverse drug reactions (ADRs) is an example of the more general class of epistemological problems related to scientific inference and prediction, as well as a central problem of the health-care practice. Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special (...) attention to the privileged role assigned to randomized controlled trials in Evidence Based Medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. This point has been emphasized by epidemiologists in the last decade. Their main focus is methodological, and regards the fact that bias and confounding do not affect studies on intended and unintended effects in the same way. However, deeper concerns ground the intuition for such a distinction; these are related to the constraints which we impose on evidence and their epistemological justification. My thesis is that such constraints ought to be understood to be different in the case of evidence for risk vs. benefit assessment. I present the recent debate on the causal association between acetaminophen and asthma in order to illustrate the point at issue. (shrink)

Governments are increasingly using randomized controlled trials (RCTs) to evaluate policy interventions. RCTs are often understood to provide the highest quality evidence regarding the causal efficacy of an intervention. While randomization plays an essential epistemic role in the context of policy RCTs however, it also plays an important distributive role. By randomly assigning participants to either the intervention or control arm of an RCT, people are subject to different policies and so, often, to different types and levels (...) of benefits. In this paper, I identify one necessary condition as well as a set of sufficient conditions for the permissible use of random assignment by government agencies. I argue first that random assignment is permissible only if it is consistent with governments’ duty to realize morally important outcomes. I argue second that random assignment is permissible in cases where investigators are in a state of genuine equipoise regarding all arms of the experiment and the policy to which people have a claim of justice. Finally, I defend a set of conditions under which random assignment is permissible in cases where one or more arms of a policy RCT are reasonably expected to be either superior or inferior to this policy. (shrink)

We raise a problem of applicability of RCTs to validate nuclear diagnostic imaging tests. In spite of the wide application of PET and other similar techniques that use radiopharmaceuticals for diagnostic purposes, RCT-based evidence on their validity is sparse. We claim that this is due to a general conceptual problem that we call Prevalence of Treatment, which arises in connection with designing RCTs for testing any diagnostic procedure in the present context of medical research, and is particularly apparent (...) in this case. We also identify three practical reasons why RCTs do not qualify as the best option for PET validation, which have to do with specific characteristics of nuclear diagnostic imaging, and of radiopharmaceuticals. The paper is meant to contribute both to the philosophical discussion on the EBM hierarchy of evidence, and on the specific debate on radiopharmaceuticals in nuclear medicine. (shrink)

The transferability problem—whether the results of an experiment will transfer to a treatment population—affects not only Randomized Controlled Trials but any type of study. The problem for any given type of study can also, potentially, be addressed to some degree through many different types of study. The transferability problem for a given RCT can be investigated further through another RCT, but the variables to use in the further experiment must be discovered. This suggests we could do better on the epistemological (...) problem of transferability by promoting, in the repeated process of formulating public health guidelines, feedback loops of information from the implementation setting back to researchers who are defining new studies. (shrink)

Randomized control trials (RCTs) are the gold standard for estimating causal effects, but often use samples that are non-representative of the actual population of interest. We propose a reweighting method for estimating population average treatment effects in settings with noncompliance. Simulations show the proposed compliance-adjusted population estimator outperforms its unadjusted counterpart when compliance is relatively low and can be predicted by observed covariates. We apply the method to evaluate the effect of Medicaid coverage on health care use for a (...) target population of adults who may benefit from expansions to the Medicaid program. We draw RCT data from the Oregon Health Insurance Experiment, where less than one-third of those randomly selected to receive Medicaid benefits actually enrolled. (shrink)

Randomised controlled trials (RCTs) sometimes recruit participants who are desperate to receive the experimental treatment. This paper defends the practice against three arguements that suggest it is unethical first, desperate volunteers are not in equipoise. Second clinicians, entering patients onto trials are disavowing their therapeutic obligation to deliver the best treatment; they are following trial protocols rather than delivering individualised care. Research is not treatment; its ethical justification is different. Consent is crucial. Third, desperate volunteers do not give proper (...) consent: effectively, they are coerced. This paper responds by advocating a notion of equipoise based on expert knowledge and widely shared values. Where such collective, expert equipoise exists there is a prima facie case for an RCT. Next the paper argues that trial entry does not involve clinicians disavowing their therapeutic obligation; individualised care based on insufficient evidence is not in patients best interest. Finally, it argues that where equipoise exists it is acceptable to limit access to experimental agents; desperate volunteers are not coerced because their desperation does not translate into a right to receive what they desire. (shrink)

The randomised controlled trial (RCT) constitutes a quantitative, comparative, controlled study of a particular treatment, and provides invaluable evidence regarding its pharmacotherapeutic efficacy. These studies are generally predicated upon the ethical principle of clinical equipoise. However, this may be insufficient to justify withholding treatment from a control group while assessing drug therapy in a potentially fatal disease. Thus, the criteria for randomisation, informed consent methodology and timing, and consideration of treatment options in such a scenario remain the province of medical (...) ethics. This paper addresses the need for an RCT of ribavirin in the treatment of Crimean Congo haemorrhagic fever, and highlights underlying ethical concerns in light of the current medical, virological and ethical literature. (shrink)

Randomised controlled trials (RCTs) are often seen as the 'gold standard' of evaluative research. However, whilst randomisation will ensure comparable groups, trials are still vulnerable to a range of biases that can undermine their internal validity. In this paper we describe a number of common threats to the internal validity of RCTs and methods of countering them. We highlight a number of examples from randomised trials in education and health care where problems of execution and analysis of the (...) RCT has undermined their internal validity. However, awareness of these potential biases can lead to careful planning to avoid or reduce their occurrence. If good quality randomised trials are to inform policy and practice in education then rigorous trials need to be designed that are the least susceptible to threats to their validity. (shrink)

In his book “Medical Philosophy: Conceptual issues in Medicine”, Mario Bunge provides a unique account of medical philosophy that is deeply rooted in a realist ontology he calls “systemism”. According to systemism, the world consists of systems and their parts, and systems possess emergent properties that their parts lack. Events within systems may form causes and effects that are constantly conjoined via particular mechanisms. Bunge supports the views of the evidence-based medicine movement that randomized controlled trials (RCTs) provide the (...) best evidence to establish the truth of causal hypothesis; in fact, he argues that _only_ RCTs have this ability. Here, we argue that Bunge neglects the important feature of patients being open systems which are in steady interaction with their environment. We show that accepting this feature leads to counter-intuitive consequences for his account of medical hypothesis testing. In particular, we point out that (i) the confirmation of hypotheses is inherently stochastic and affords a probabilistic account of both confirmation and evidence which we provide here; (ii) RCTs are neither necessary nor sufficient to establish the truth of a causal claim; (iii) testing of causal hypotheses requires taking into account background knowledge and the context within which an intervention is applied. We conclude that there is no “best” research methodology in medicine, but that different methodologies should coexist in a complementary fashion. (shrink)

Evidence based-policy (EBP) is a popular research paradigm in the applied social sciences and within government agencies. Informally, EBP represents an explicit commitment to applying scientific methods to public affairs, in contrast to ideologically-driven or merely intuitive “common-sense” approaches to public policy. More specifically, the EBP paradigm places great weight on the results of experimental research designs, especially randomised controlled trials (RCTs), and systematic literature reviews that place evidential weight on experimental results. One hope is that such research designs (...) and approaches to analysing the scientific literature are sufficiently robust that they can settle what really ‘works’ in public policy. Can EBP succeed in displacing reliance on domain-specific expertise? On our account, this is seldom, if ever, the case. The key reason for this is that underlying this approach is generally an appeal to argument by induction, which always requires further assumptions to underwrite its validity, and if not induction, some other argument form that also requires assumptions that are very often not validated for the case at hand. (shrink)

Various publications claim that medical AI systems perform as well, or better, than clinical experts. However, there have been very few controlled trials and the quality of existing studies has been called into question. There is growing concern that existing studies overestimate the clinical benefits of AI systems. This has led to calls for more, and higher-quality, randomized controlled trials of medical AI systems. While this a welcome development, AI RCTs raise novel methodological challenges that have seen little discussion. (...) We discuss some of the challenges arising in the context of AI RCTs and make some suggestions for how to meet them. (shrink)

In recent years, there has been a surge of high-profile publications on applications of artificial intelligence (AI) systems for medical diagnosis and prognosis. While AI provides various opportunities for medical practice, there is an emerging consensus that the existing studies show considerable deficits and are unable to establish the clinical benefit of AI systems. Hence, the view that the clinical benefit of AI systems needs to be studied in clinical trials—particularly randomised controlled trials (RCTs)—is gaining ground. However, an issue (...) that has been overlooked so far in the debate is that, compared with drug RCTs, AI RCTs require methodological adjustments, which entail ethical challenges. This paper sets out to develop a systematic account of the ethics of AI RCTs by focusing on the moral principles of clinical equipoise, informed consent and fairness. This way, the objective is to animate further debate on the (research) ethics of medical AI. (shrink)

Millions of pounds are spent on educational research each year in the UK alone. By far the greatest proportion of this expenditure is on research which is thought to have practical relevance to educational problems and the vast majority of this is spent on empirical educational research, that is educational research which examines and seeks explanations for actual or proposed educational practices or the kinds of activities, institutions or policies that prepare young people for life (Pring, 2015, p. 27). Invariably, (...) the aim of such research is not merely to gain understanding of these practices, but to improve or replace them with better ones. This practical aspect of educational research exercises most commentators, although it should not be forgotten that educational research can be carried out in a relatively disinterested manner, out of intellectual curiosity (Hammersley, 2006). There are two related issues to be considered here: 1] is it possible to gain knowledge of how educational practices work?; 2] is it possible to use such knowledge to improve existing practices, replace existing ones with better ones or introduce practices which will improve the overall practice of education? In this book, both these questions will be addressed. If the answer to 1] is negative, then it is futile to address 2]. By implication then the answer to 1] will be positive. The answer to 2] will not be negative, but there are degrees of positivity attached to a positive answer and what those degrees are will be crucial to the ultimate claims of the book. It is no exaggeration to say that in recent decades educational research has undergone a crisis of confidence, particularly but not exclusively within the world of educational policymaking. Dissatisfaction has been expressed both with the quality and the usefulness of such research. Conceptual research on education tends to be regarded with some suspicion as something that is either of little practical value or which undermines existing practices. But empirical research fares little better, with scepticism expressed both about its quality (Tooley and Darby, 1998) and about its applicability (Hillage et al., 1998). More recently there have been moves to tie funding to preferred approaches to empirical research such as intervention studies and randomised control trials (RCTs) and to disseminate in accessible form what the findings of such studies are and how they are relevant to the improvement of educational practice. In the UK alone we see organisations such as the Educational Policy Institute (EPI), the Educational Endowment Foundation (EEF), and the Sutton Trust, not to mention meta-researchers Robert Slavin and John Hattie gaining a significant amount of influence in the propagation of studies of 'what works' in educational practice. One should also mention the rising interest in the deliverances of educational research from within the teaching profession, such as ResearchEd in the UK, which betoken a wish by teachers to become more active. (shrink)

Introduction: Clinical practice guidelines (CPGs) are an important source of justification for clinical decisions in modern evidence-based practice. Yet, we have given little attention to how they argue their evidence. In particular, how do CPGs argue for treatment with long-term medications that are increasingly prescribed to older patients? Approach and rationale: I selected six disease-specific guidelines recommending treatment with five of the medication classes most commonly prescribed for seniors in Ontario, Canada. I considered the stated aims of these CPGs and (...) the techniques employed towards those aims. Finally, I reconstructed and logically analysed the arguments supporting recommendations for pharmacotherapy. Analysis: The primary function of CPGs is rhetorical, or persuasive, and their means of persuasion include both a display of their credibility and their argumentation. Arguments supporting pharmacotherapy recommendations for the target population follow a common inductive pattern: statistical generalization from randomized controlled trial (RCT) and meta-analysis evidence. Two of the CPGs also argue their treatment recommendations for older patients in this style, while three fail to justify pharmacotherapy specifically for the older population. Discussion: The arguments analysed lack the auxiliary assumptions that would warrant making a generalization about the clinical effectiveness of medications for the older population. Guidelines reason using simple induction, while ignoring important inferential gaps. Future guidelines should aspire to be well-reasoned rather than simply evidence-based; argue from a plurality of evidence; be wary of hasty inductions; appropriately limit the scope of their recommendations; and avoid making law-like, prescriptive generalizations. (shrink)

Randomized experiments, as developed by Esther Duflo and Abhijit Banerjee at the Abdul Latif Jameel Poverty Action Lab, offer a novel, evidence-based approach to fighting poverty. This approach is original, in that it imports the methodology of clinical trials for application in development economics. This paper examines the analogy between J-PAL’s field experiments in development economics and randomized controlled trials in medicine. RCTs and randomized field experiments are commonly treated as identical, but such treatment neglects some of the major (...) distinguishing features that make each experiment specifically apt for use in its respective field. The central claim of this paper is that the analogy between medicine and development economics is incomplete because the central dimensions of RCTs are not simply different but altogether lacking in J-PAL’s approach. This weakens both the political and the theoretical power of such experiments in development economics. (shrink)

When a new intervention or drug is developed, this has to pass through various phases of clinical testing before it achieves market approval, which can take many years. This raises an issue for drugs which could benefit terminally ill patients. These patients might set their hopes on the experimental drug but are unable to wait since they are likely to pass away before the drug is available. As a means of nevertheless getting access to experimental drug, many seriously ill and (...) terminally ill patients are therefore very willing to participate in randomised controlled trials. However, only very few terminally ill patients are able to actually participate, and those that do participate are at risk of participating solely as a way of getting experimental drugs. Currently, there are, however, ways of getting access to drugs that have not gained market approval. One such mean is via expanded access or compassionate use programs where terminally ill patients receive experimental new drugs that are not yet market approved. In this paper, I examine some of the common justifications for such programs. The most frequently voiced justifications for compassionate use or expanded access programs could be put in one of three categories. First, there are justifications of justice, where compassionate use programs could be seen as a just or fair way to distribute experimental new drugs to patients who are denied access to RCT’s through no fault of their own. Second, such programs could be justified by reference to the ethical principle of beneficence where it could be claimed that terminally ill patients stand to benefit greatly at very little risk. Third, there are considerations of autonomy where, it is claimed, patients should be able to exercise their autonomy and have access to such drugs if that is there free choice and they are fully aware of the risks associated with that choice. In this paper, I argue currently all justifications are potentially problematic. If they truly form the basis for justification, compassionate use programs should be designed to maximize justice, beneficence and autonomy. (shrink)

Philosophical discussions on causal inference in medicine are stuck in dyadic camps, each defending one kind of evidence or method rather than another as best support for causal hypotheses. Whereas Evidence Based Medicine advocates the use of Randomised Controlled Trials and systematic reviews of RCTs as gold standard, philosophers of science emphasise the importance of mechanisms and their distinctive informational contribution to causal inference and assessment. Some have suggested the adoption of a pluralistic approach to causal inference, and an (...) inductive rather than hypothetico-deductive inferential paradigm. However, these proposals deliver no clear guidelines about how such plurality of evidence sources should jointly justify hypotheses of causal associations. We here develop such guidelines by first giving a philosophical analysis of the underpinnings of Hill’s viewpoints on causality. We then put forward an evidence-amalgamation framework adopting a Bayesian net approach to model causal inference in pharmacology for the assessment of harms. Our framework accommodates a number of intuitions already expressed in the literature concerning the EBM vs. pluralist debate on causal inference, evidence hierarchies, causal holism, relevance, and reliability. (shrink)