We simulate trial data to test speculative claims about research methods, such as the impact of publication bias.

Medical scientists employ ‘quality assessment tools’ (QATs) to measure the quality of evidence from clinical studies, especially randomized controlled trials (RCTs). These tools are designed to take into account various methodological details of clinical studies, including randomization, blinding, and other features of studies deemed relevant to minimizing bias and error. There are now dozens available. The various QATs on offer differ widely from each other, and second-order empirical studies show that QATs have low inter-rater reliability and low inter-tool reliability. This (...) is an instance of a more general problem I call the underdetermination of evidential significance. Disagreements about the strength of a particular piece of evidence can be due to different—but in principle equally good—weightings of the fine-grained methodological features which constitute QATs. (shrink)

Meditation is an umbrella term for a number of mental training practices designed to improve the monitoring and regulation of attention and emotion. Some forms of meditation are now being used for clinical intervention. To accompany the increased clinical interest in meditation, research investigating the neural basis of these practices is needed. A central hypothesis of contemplative neuroscience is that meditative states, which are unique on a phenomenological level, differ on a neurophysiological level. To identify the electrophysiological correlates of meditation (...) practice, the electrical brain activity of highly skilled meditators engaging in one of six meditation styles was recorded. A mind-wandering task served as a control. Lempel–Ziv complexity showed differences in nonlinear brain dynamics during meditation compared with mind wandering, suggesting that meditation, regardless of practice, affects neural complexity. In contrast, there were no differences in power spectra at six different frequency bands, likely due to the fact that participants engaged in different meditation practices. Finally, exploratory analyses suggest neurological differences among meditation practices. These findings highlight the importance of studying the electroencephalography correlates of different meditative practices. (shrink)

In the biomedical context, policy makers face a large amount of potentially discordant evidence from different sources. This prompts the question of how this evidence should be aggregated in the interests of best-informed policy recommendations. The starting point of our discussion is Hunter and Williams’ recent work on an automated aggregation method for medical evidence. Our negative claim is that it is far from clear what the relevant criteria for evaluating an evidence aggregator of this sort are. What is the (...) appropriate balance between explicitly coded algorithms and implicit reasoning involved, for instance, in the packaging of input evidence? In short: What is the optimal degree of ‘automation’? On the positive side: We propose the ability to perform an adequate robustness analysis as the focal criterion, primarily because it directs efforts to what is most important, namely, the structure of the algorithm and the appropriate extent of automation. Moreover, where there are resource constraints on the aggregation process, one must also consider what balance between volume of evidence and accuracy in the treatment of individual evidence best facilitates inference. There is no prerogative to aggregate the total evidence available if this would in fact reduce overall accuracy. (shrink)

In this paper we consider the problem of how to measure the strength of statistical evidence from the perspective of evidence amalgamation operations. We begin with a fundamental measurement amalgamation principle : for any measurement, the inputs and outputs of an amalgamation procedure must be on the same scale, and this scale must have a meaningful interpretation vis a vis the object of measurement. Using the p value as a candidate evidence measure, we examine various commonly used approaches to amalgamation (...) of evidence across similar studies, including standard forms of meta-analysis. We show that none of these methods satisfies MAP. Thus an underlying measurement problem remains. We argue that a successful approach to evidence amalgamation necessitates a solution to the problem of evidence measurement, and we suggest some lines of reasoning that might guide further work towards this end. (shrink)

In the last decade, many problematic cases of scientific conduct have been diagnosed; some of which involve outright fraud others are more subtle. These and similar problems can be interpreted as caused by lack of scientific objectivity. The current philosophical theories of objectivity do not provide scientists with conceptualizations that can be effectively put into practice in remedying these issues. We propose a novel way of thinking about objectivity for individual scientists; a negative and dynamic approach.We provide a philosophical conceptualization (...) of objectivity that is informed by empirical research. In particular, it is our intention to take the first steps in providing an empirically and methodologically informed inventory of factors that impair the scientific practice. The inventory will be compiled into a negative conceptualization, which could in principle be used by individual scientists to assess objectivity of scientific practice. We propose a preliminary outline of a usable and testable instrument for indicating the objectivity of scientific practice. (shrink)

According to public health data, the US Hispanic population is far healthier than would be expected for a population with low socioeconomic status. Ever since Kyriakos Markides and Jeannine Coreil highlighted this in a seminal 1986 article, public health researchers have sought to explain the so-called “Hispanic paradox.” Several candidate explanations have been offered over the years, but the debate goes on. This article offers a philosophical analysis that clarifies how two sets of obstacles make it particularly difficult to explain (...) the Hispanic paradox. First, different research projects define the Hispanic paradox phenomenon in substantially different ways. Moreover, using Bas van Fraassen’s pragmatic theory of explanation and Sean Valles’s extension of it with the concept of “phenomenon choice,” it also becomes clear that there are also multiple ways of explaining each individual definition of the phenomenon. A second set of philosophical and methodological challenges arises during any attempt to study “Hispanic” phenomena, with one key challenge being that the “Hispanic” panethnic concept was intentionally made vague as it was developed and popularized during the 1960s–1970s. After comparing this case with similar cases in the philosophical literature, the article concludes with observations on what makes this problem unique, particularly its ethical features. (shrink)

Philosophy of epidemiology has recently emerged as a distinct branch of philosophy. The field will surely benefit from pluralism, reflected in the broad range of topics and perspectives in this special issue. Here, I argue that a healthy pluralistic field of philosophy of epidemiology has social responsibilities that require the field as a whole to engage actively with research on social determinants of health and health disparities. Practicing epidemiologists and the broader community of public health scientists have gradually acknowledged that (...) much of their attention ought to be paid to these, i.e. inequitable between-population health variations that largely reflect the world’s inequitable distribution of resources and social conditions. The paper illustrates how and why health disparities and social determinants of health, through no ill will, become de facto secondary concerns in Alex Broadbent’s field-defining Philosophy of Epidemiology, showing the ease with which these topics can be incidentally sidelined. As means for philosophy of epidemiology to meet its social responsibility obligations, I suggest philosophy of epidemiology expand its attention to two particular lines research. First, the paper discusses Geoffrey Rose’s concept of “causes of incidence”—causes of disparities between populations—and argues for the importance of engaging with these causes. Second, the paper argues for the value of engaging with Bruce Link and Jo Phelan’s “fundamental causes” model of how flexible social resources serve as buffers from harms, generating a plethora of shifting and locally-contingent health effects. (shrink)

Establishing the effectiveness of medical treatments is one of the most important aspects of medical practice. Bradford Hill's viewpoints play an important role in inferring causality in medicine,...

This article examines the relationship between the precautionary principle and the well-known Hill criteria of causation. Some have charged that the Hill criteria are anti-precautionary because the...

Consensus conferences are social techniques which involve bringing together a group of scientific experts, and sometimes also non-experts, in order to increase the public role in science and related policy, to amalgamate diverse and often contradictory evidence for a hypothesis of interest, and to achieve scientific consensus or at least the appearance of consensus among scientists. For consensus conferences that set out to amalgamate evidence, I propose three desiderata: Inclusivity, Constraint, and Evidential Complexity. Two examples suggest that consensus conferences can (...) readily satisfy Inclusivity and Evidential Complexity, but consensus conferences do not as easily satisfy Constraint. I end by discussing the relation between social inclusivity and the three desiderata. (shrink)

Harms of medical interventions are systematically underestimated in clinical research. Numerous factors—conceptual, methodological, and social—contribute to this underestimation. I articulate the depth of such underestimation by describing these factors at the various stages of clinical research. Before any evidence is gathered, the ways harms are operationalized in clinical research contributes to their underestimation. Medical interventions are first tested in phase 1 ‘first in human’ trials, but evidence from these trials is rarely published, despite the fact that such trials provide the (...) foundation for assessing the harm profile of medical interventions. If a medical intervention is deemed safe in a phase 1 trial, it is tested in larger phase 2 and 3 clinical trials. One way to think about the problem of underestimating harms is in terms of the statistical ‘power’ of a clinical trial—the ability of a trial to detect a difference of a certain effect size between the experimental group and the control group. Power is normally thought to be pertinent to detecting benefits of medical interventions. It is important, though, to distinguish between the ability of a trial to detect benefits and the ability of a trial to detect harms. I refer to the former as power-B and the latter as power-H. I identify several factors that maximize power-B by sacrificing powerH in phase 3 clinical trials. If a medical intervention is approved for general use, it is evaluated by phase 4 post-market surveillance. Phase 4 surveillance of harms further contributes to underestimating the harm profile of medical interventions. At every stage of clinical research the hunt for harms is shrouded in secrecy, which further contributes to the underestimation of the harm profiles of medical interventions. (shrink)

There are two competing views regarding the role of mechanistic knowledge in inferences about the effectiveness of interventions. One view holds that inferences about the effectiveness of interventions should be based only on data from population-level studies (often statistical evidence from randomised trials). The other view holds that such inferences must be based in part on mechanistic evidence. The competing views are local principles of inference, the plausibility of which can be assessed by a more general normative principle of inference. (...) Bayesianism tells us to base inferences on both the ‘likelihood’ and the ‘prior’. The likelihood represents statistical evidence. One influence on the prior probability of a hypothesis like ‘d causes x’ is mechanistic knowledge of how d causes x. Thus, reasoning about such inferences by appealing to both statistical and mechanistic evidence is vindicated by our best general theory of inference. The primary contribution of this paper is to assess the merits and weaknesses of the arguments on both sides of the debate, using the Bayesian framework. This analysis lends support to those who argue that we should base our causal inferences about interventions in part on mechanistic evidence. (shrink)

Evidence hierarchies are widely used to assess evidence in systematic reviews of medical studies. I give several arguments against the use of evidence hierarchies. The problems with evidence hierarchies are numerous, and include methodological shortcomings, philosophical problems, and formal constraints. I argue that medical science should not employ evidence hierarchies, including even the latest and most-sophisticated of such hierarchies.

The concept of a “validator” as a unit of evidence for the validity of a psychiatric category has been important for more than fifty years. Validator evidence is aggregated by expert committees (for the Diagnostic and Statistical Manual of Mental Disorders (DSM), these are referred to as “workgroups”), which use the results to make nosological decisions. Through an examination of the recent history of psychiatric research, this paper argues that it is time to reassess this traditional practice. It concludes with (...) specific suggestions for going forward. (shrink)

Advocates of the self-corrective thesis argue that scientific method will refute false theories and find closer approximations to the truth in the long run. I discuss a contemporary interpretation of this thesis in terms of frequentist statistics in the context of the behavioral sciences. First, I identify experimental replications and systematic aggregation of evidence (meta-analysis) as the self-corrective mechanism. Then, I present a computer simulation study of scientific communities that implement this mechanism to argue that frequentist statistics may converge upon (...) a correct estimate or not depending on the social structure of the community that uses it. Based on this study, I argue that methodological explanations of the “replicability crisis” in psychology are limited and propose an alternative explanation in terms of biases. Finally, I conclude suggesting that scientific self-correction should be understood as an interaction effect between inference methods and social structures. (shrink)

Naturalistic philosophers rely on literature search and review in a number of ways and for different purposes. Yet this article shows how processes of literature search and review are likely to be affected by widespread and systematic biases. A solution to this problem is offered here. Whilst the tradition of systematic reviews of literature from scientific disciplines has been neglected in philosophy, systematic reviews are important tools that minimize bias in literature search and review and allow for greater reproducibility and (...) transparency. If naturalistic philosophers wish to reduce bias in their research, they should then supplement their traditional tools for literature search and review by including systematic methodologies. (shrink)

Amalgamating evidence from heterogeneous sources and across levels of inquiry is becoming increasingly important in many pure and applied sciences. This special issue provides a forum for researchers from diverse scientific and philosophical perspectives to discuss evidence amalgamation, its methodologies, its history, its pitfalls, and its potential. We situate the contributions therein within six themes from the broad literature on this subject: the variety-of-evidence thesis, the philosophy of meta-analysis, the role of robustness/sensitivity analysis for evidence amalgamation, its bearing on questions (...) of extrapolation and external validity of experiments, its connection with theory development, and its interface with causal inference, especially regarding causal theories of cancer. (shrink)

November 2009’s announcement of the USPSTF’s recommendations for screening for breast cancer raised a firestorm of objections. Chief among them were that the panel had insufficiently valued patients’ lives or allowed cost considerations to influence recommendations. The publicity about the recommendations, however, often either simplified the actual content of the recommendations or bypassed significant methodological issues, which a philosophical examination of both the science behind screening recommendations and their import reveals. In this article, I discuss two of the leading ethical (...) considerations at issue in screening recommendations: respect for patient autonomy and beneficence and then turn to the most significant methodological issues raised by cancer screening: the potential biases that may infect a trial of screening effectiveness, the problem of base rates in communicating risk, and the trade-offs involved in a judgment of screening effectiveness. These issues reach more broadly, into the use of “evidence-based” medicine generally, and have important implications for informed consent. (shrink)

Evidence-based medicine (EBM) has long deemphasized mechanistic reasoning and pathophysiological rationale in assessing the effectiveness of interventions. The EBM+ movement has challenged this stance, arguing that evidence of mechanisms and comparative studies should both be seen as necessary and complementary. Advocates of EBM+ provide a combination of theoretical arguments and examples of mechanistic reasoning in medical research. However, EBM+ proponents have not provided recent examples of how downplaying mechanistic reasoning resulted in worse medical results than would have occurred otherwise. Such (...) examples are necessary to make the case that EBM+ responds to a problem in clinical practice that urgently demands a solution. In light of this, we examine the failed rollout of efavirenz as a first-line HIV treatment in Zimbabwe as evidence of the importance of mechanistic reasoning in improving clinical practice and public health policy decisions. We suggest that this case is analogous to examples commonly given to support EBM. (shrink)

The problem of collecting, analyzing and evaluating evidence on adverse drug reactions (ADRs) is an example of the more general class of epistemological problems related to scientific inference and prediction, as well as a central problem of the health-care practice. Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special (...) attention to the privileged role assigned to randomized controlled trials in Evidence Based Medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. This point has been emphasized by epidemiologists in the last decade. Their main focus is methodological, and regards the fact that bias and confounding do not affect studies on intended and unintended effects in the same way. However, deeper concerns ground the intuition for such a distinction; these are related to the constraints which we impose on evidence and their epistemological justification. My thesis is that such constraints ought to be understood to be different in the case of evidence for risk vs. benefit assessment. I present the recent debate on the causal association between acetaminophen and asthma in order to illustrate the point at issue. (shrink)

The problem of collecting, analyzing and evaluating evidence on adverse drug reactions (ADRs) is an example of the more general class of epistemological problems related to scientific inference and prediction, as well as a central problem of the health-care practice. Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special (...) attention to the privileged role assigned to randomized controlled trials in Evidence Based Medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended effects, in that the unknowns at stake are heterogonous in the two contexts. This point has been emphasized by epidemiologists in the last decade. Their main focus is methodological, and regards the fact that bias and confounding do not affect studies on intended and unintended effects in the same way. However, deeper concerns ground the intuition for such a distinction; these are related to the constraints which we impose on evidence and their epistemological justification. My thesis is that such constraints ought to be understood to be different in the case of evidence for risk vs. benefit assessment. I present the recent debate on the causal association between acetaminophen and asthma in order to illustrate the point at issue. (shrink)

Philosophical discussions have critically analysed the methodological pitfalls and epistemological implications of evidence assessment in medicine, however they have mainly focused on evidence of treatment efficacy. Most of this work is devoted to statistical methods of causal inference with a special attention to the privileged role assigned to randomized controlled trials (RCTs) in evidence based medicine. Regardless of whether the RCT’s privilege holds for efficacy assessment, it is nevertheless important to make a distinction between causal inference of intended and unintended (...) effects, in that the unknowns at stake are heterogonous in the two contexts. However, although “lower level” evidence is increasingly acknowledged to be a valid source of information contributory to assessing the risk profile of medications on theoretical or empirical grounds, current practices have difficulty in assigning a precise epistemic status to this kind of evidence because they are more or less implicitly parasitic on the (statistical) methods developed to test drug efficacy. My thesis is that (1) “lower level” evidence is justified on distinct grounds and at different conditions depending on the different epistemologies which one wishes to endorse, in that each impose different constraints on the methods we adopt to collect and evaluate evidence; (2) such constraints ought to be understood to be different in the case of evidence for risk versus benefit assessment for a series of reasons which I will illustrate on the basis of the recent debate on the causal association between acetaminophen (a.k.a. paracetamol) and asthma. (shrink)

Reluctance to endorse mask wearing to slow transmission of SARS-Cov-2 has been rationalized by the failure of randomized control trials (RCTs) to provide supportive evidence. In contrast, a mechanism-based approach suggests that mask wearing should be expected to reduce transmission: so that contrary evidence from RCTs likely reflects the need to focus policy attention on addressing interacting or mediating factors that offset the basic positive effect. The differing conclusions that result from these two approaches reflect the limitations of RCT-based approaches (...) that are compounded in scenarios, such as pandemics, where urgent decisions are required with limited evidence. (shrink)

Scientific consensus is widely deferred to in public debates as a social indicator of the existence of knowledge. However, it is far from clear that such deference to consensus is always justified. The existence of agreement in a community of researchers is a contingent fact, and researchers may reach a consensus for all kinds of reasons, such as fighting a common foe or sharing a common bias. Scientific consensus, by itself, does not necessarily indicate the existence of shared knowledge among (...) the members of the consensus community. I address the question of under what conditions it is likely that a consensus is in fact knowledge based. I argue that a consensus is likely to be knowledge based when knowledge is the best explanation of the consensus, and I identify three conditions—social calibration, apparent consilience of evidence, and social diversity, for knowledge being the best explanation of a consensus. (shrink)

A consensus in a scientific community is often used as a resource for making informed public-policy decisions and deciding between rival expert testimonies in legal trials. This paper contains a social-epistemic analysis of the high-profile Bendectin drug controversy, which was decided in the courtroom inter alia by deference to a scientific consensus about the safety of Bendectin. Drawing on my previously developed account of knowledge-based consensus, I argue that the consensus in this case was not knowledge based, hence courts’ deference (...) to it was not epistemically justified. I draw sceptical lessons from this analysis regarding the value of scientific consensus as a desirable and reliable means of resolving scientific controversies in public life. (shrink)

It is commonly argued that values “fill the logical gap” of underdetermination of theory by evidence, namely, values affect our choice between two or more theories that fit the same evidence. The underdetermination model, however, does not exhaust the roles values play in evidential reasoning. I introduce WAVE – a novel account of the logical relations between values and evidence. WAVE states that values influence evidential reasoning by adjusting evidential weights. I argue that the weight-adjusting role of values is distinct (...) from their underdetermination gap-filling role. Values adjust weights in three ways. First, values affect our trust in the testimony of others. Second, values influence the evidential thresholds required for justified epistemic judgments. Third, values influence the relative weight of a certain type of evidence within a body of multimodal discordant evidence. WAVE explains, from an epistemic perspective, rather than psychological, how smokers, for example, can find the same evidence about the dangers of smoking less persuasive than non-smokers. WAVE allows for a wider effect of values on our accepted scientific theories and beliefs than the effect for which the underdetermination model allows alone; therefore, science studies scholars must consider WAVE in their research and analysis of evidential case studies. (shrink)

Increasing philosophical attention is being directed to the rapidly growing discipline of evidence-based medicine. Philosophical discussions of EBM, however, remain narrowly focused on randomization, mechanisms, and the sociology of EBM. Other aspects of EBM have been all but ignored, including the nature of clinical reasoning and the question of whether it can be standardized; the application of EBM principles to the logic, value, and ethics of diagnosis and prognosis; evidence synthesis ; and the nature and ethics of placebo controls. Philosophical (...) analysis in each of these areas has the potential to enhance the discussion of EBM methodology and practice. (shrink)

The Concussion in Sport Group guidelines have successfully brought the attention of brain injuries to the global medical and sport research communities, and has significantly impacted brain injury-related practices and rules of international sport. Despite being the global repository of state-of-the-art science, diagnostic tools and guides to clinical practice, the ensuing consensus statements remain the object of ethical and sociocultural criticism. The purpose of this paper is to bring to bear a broad range of multidisciplinary challenges to the processes and (...) products of sport-related concussion movement. We identify lacunae in scientific research and clinical guidance in relation to age, disability, gender and race. We also identify, through multidisciplinary and interdisciplinary analysis, a range of ethical problems resulting from conflicts of interest, processes of attributing expertise in sport-related concussion, unjustifiably narrow methodological control and insufficient athlete engagement in research and policy development. We argue that the sport and exercise medicine community need to augment the existing research and practice foci to understand these problems more holistically and, in turn, provide guidance and recommendations that help sport clinicians better care for brain-injured athletes. (shrink)

The current strategy of searching for an effective treatment for COVID-19 relies mainly on repurposing existing therapies developed to target other diseases. Conflicting results have emerged in regard to the efficacy of several tested compounds but later results were negative. The number of conducted and ongoing trials and the urgent need for a treatment pose the risk that false-positive results will be incorrectly interpreted as evidence for treatments’ efficacy and a ground for drug approval. Our purpose is twofold. First, we (...) show that the number of drug-repurposing trials can explain the false-positive results. Second, we assess the evidence for treatments’ efficacy from the perspective of evidential pluralism and argue that considering mechanistic evidence is particularly needed in cases when the evidence from clinical trials is conflicting or of low quality. Our analysis is an application of the program of Evidence Based Medicine Plus to the drug repurposing trials for COVID. Our study shows that if decision-makers applied EBM+, authorizing the use of ineffective treatments would be less likely. We analyze the example of trials assessing the efficacy of hydroxychloroquine as a treatment for COVID-19 and mechanistic evidence in favor of and against its therapeutic power to draw a lesson for decision-makers and drug agencies on how excessive hypothesis testing can lead to spurious findings and how studying negative mechanistic evidence can be helpful in discriminating genuine from spurious results. (shrink)

This article examines an often neglected topic in the history of science, namely clinical observation, specifically the objectivity and knowledge production associated with therapeutic trials. It will describe an eighteenth and nineteenth century pharmacological concept of objectivity and exemplify that concept using late nineteenth century European cocaine research. As conceived within clinical drug research, this concept of objectivity does not correspond with those described by Daston and Galison in their seminal book Objectivity (2007). I will explore the implications of this (...) “new” concept of objectivity—as compared to the “old” ones postulated by Daston and Galison—for their core propositions. (shrink)

It is argued in this paper that amalgamating confirmation from various sources is relevantly different from social-choice contexts, and that proving an impossibility theorem for aggregating confirmation measures directs attention to irrelevant issues.

Philosophical discussions on causal inference in medicine are stuck in dyadic camps, each defending one kind of evidence or method rather than another as best support for causal hypotheses. Whereas Evidence Based Medicine advocates the use of Randomised Controlled Trials and systematic reviews of RCTs as gold standard, philosophers of science emphasise the importance of mechanisms and their distinctive informational contribution to causal inference and assessment. Some have suggested the adoption of a pluralistic approach to causal inference, and an inductive (...) rather than hypothetico-deductive inferential paradigm. However, these proposals deliver no clear guidelines about how such plurality of evidence sources should jointly justify hypotheses of causal associations. We here develop such guidelines by first giving a philosophical analysis of the underpinnings of Hill’s viewpoints on causality. We then put forward an evidence-amalgamation framework adopting a Bayesian net approach to model causal inference in pharmacology for the assessment of harms. Our framework accommodates a number of intuitions already expressed in the literature concerning the EBM vs. pluralist debate on causal inference, evidence hierarchies, causal holism, relevance, and reliability. (shrink)

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way (...) in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed. (shrink)

The trustworthiness of nutrition science has been questioned recently. According to the critics, the food industry has corrupted scientists in the field. I argue that the worries that commercialization threatens the epistemic trustworthiness of nutrition science are indeed well-founded. However, it is problematic that the discussion has revolved around how funding can threaten the integrity of researchers and the methodological quality of the studies. By extending Wilholt’s :233–253, 2013) account of epistemic trustworthiness, I argue that when assessing the epistemic trustworthiness (...) of research that forms the basis for different health policy measures, it is necessary to evaluate research at the macro-level and whether agenda setting advances the goals that are assigned to the field. The prevalence of commercial funding becomes problematic if it leads to a situation where the body of available evidence that is used for making health policy decisions does not reflect the shared sense of what epistemic and non-epistemic goals of the inquiry are important. (shrink)

This paper argues for a novel account of deceitful scientific communication, as “wishful speaking”. This concept is of relevance both to philosophy of science and to discussions of the ethics of lying and misleading. Section 1 outlines a case-study of “ghost-managed” research. Section 2 introduces the concept of “wishful speaking” and shows how it relates to other forms of misleading communication. Sections 3–5 consider some complications raised by the example of pharmaceutical research; concerning the ethics of silence; how research strategies—as (...) well as the communication of results—may be misleading; and questions of multiple authorship. The conclusion suggests some more general conclusions. (shrink)

EBM, and the hierarchy of evidence it prescribes, is a controversial model when it comes to research into the effectiveness of psychotherapeutic treatments. This is due in part to the so-called ‘Dodo Bird verdict’, which claims that all psychotherapies are equally effective, and that their effectiveness is largely due to the placebo effect. In response to this controversy, I argue that EBM can nevertheless be made to fit research into the effectiveness of psychotherapy, once a piecemeal approach to conducting RCTs (...) is taken. Such an approach involves studying the contributions made by individual treatment components. (shrink)

Arguments that medical decision making should rely on a variety of evidence often begin from the claim that meta-analysis has been shown to be problematic. In this paper, I first examine Stegenga’s argument that meta-analysis requires multiple decisions and thus fails to provide an objective ground for medical decision making. Next, I examine three arguments from social epistemologists that contend that meta-analyses are systematically biased in ways not appreciated by standard epistemology. In most cases I show that critiques of meta-analysis (...) fail to account for the full range of meta-analytic procedures. In the remainder of cases, I argue that the critiques identify problems that do not uniquely cut against meta-analysis. I close by suggesting one reason why it may be pragmatically rational to violate the principle of total evidence and by outlining the criteria for a successful argument against meta-analysis. A set of criteria I contend remain unmet. (shrink)

This paper explores the relation between scientific knowledge and common sense intuitions as a complement to Hoyningen-Huene’s account of systematicity. On one hand, Hoyningen-Huene embraces continuity between these in his characterization of scientific knowledge as an extension of everyday knowledge, distinguished by an increase in systematicity. On the other, he argues that scientific knowledge often comes to deviate from common sense as science develops. Specifically, he argues that a departure from common sense is a price we may have to pay (...) for increased systematicity. I argue that to clarify the relation between common sense and scientific reasoning, more attention to the cognitive aspects of learning and doing science is needed. As a step in this direction, I explore the potential for cross-fertilization between the discussions about conceptual change in science education and philosophy of science. Particularly, I examine debates on whether common sense intuitions facilitate or impede scientific reasoning. While contending that these debates can balance some of the assumptions made by Hoyningen-Huene, I suggest that a more contextualized version of systematicity theory could supplement cognitive analysis by clarifying important organizational aspects of science. (shrink)

As the quality movement in health care now enters its fourth decade, the language of quality is ubiquitous. Practitioners, organizations, and government agencies alike vociferously testify their commitments to quality and accept numerous forms of governance aimed at improving quality of care. Remarkably, the powerful phrase ‘‘quality of care’’ is rarely defined in the health care literature. Instead it operates as an accepted and assumed goal worth pursuing. The status of evidence-based medicine, for instance, hinges on its ability to improve (...) quality of care, and efforts are made by both proponents and detractors to unpack the contents and outcomes of evidence-based practice while the contents of ‘‘quality of care’’ are presumed to be understood. Because the goals of medicine are far from obvious, this paper investigates the neglected term, ‘‘quality of care,’’ in an effort to understand what it is that health care practices are so uncritically assumed to be striving for. Finding lack of consensus on the terminology in the quality literature, I propose that the term operates rhetorically by way of persuasive appeal (and lack of descriptive meaning). Unsatisfied that ‘‘quality of care’’ operates as a mere buzzword in morally contentious debates over resource allocation and duties of care, I implore health care communities to go beyond mere commitments to quality and, instead, to focus attention on the difficult task of specifying what counts as quality care within an economically constrained health care system. (shrink)

The replicability crisis refers to the apparent failures to replicate both important and typical positive experimental claims in psychological science and biomedicine, failures which have gained increasing attention in the past decade. In order to provide evidence that there is a replicability crisis in the first place, scientists have developed various measures of replication that help quantify or “count” whether one study replicates another. In this nontechnical essay, I critically examine five types of replication measures used in the landmark article (...) “Estimating the reproducibility of psychological science” (Open Science Collaboration, Science, 349, ac4716, 2015) based on the following techniques: subjective assessment, null hypothesis significance testing, comparing effect sizes, comparing the original effect size with the replication confidence interval, and meta-analysis. The first four, I argue, remain unsatisfactory for a variety of conceptual or formal reasons, even taking into account various improvements. By contrast, at least one version of the meta-analytic measure does not suffer from these problems. It differs from the others in rejecting dichotomous conclusions, the assumption that one study replicates another or not simpliciter. I defend it from other recent criticisms, concluding however that it is not a panacea for all the multifarious problems that the crisis has highlighted. (shrink)

A coherent and helpful public policy based on science is difficult to achieve for at least three reasons. First, there are purely practical problems—for example, that scientific experts often disagree on policy-relevant questions and their debates often continue well beyond policy appropriate timelines. Second, there are epistemic problems—for example, that science is hardly the neutral supplier of factual information that traditionally has been supposed. And third, there are social problems: given the commercialization of today’s science and its enduring limitations, much (...) of scientific research today fails to meet the moral and political standards one would expect it to meet in order to inform public policy. In this paper, we examine such problems in the context of breast cancer screening policy and suggest the role philosophy of science should play in dealing with the situation. (shrink)

The setting of evidential standards is a core practice of scientific assessment for policy. Persuaded by considerations of inductive risk, philosophers generally agree that the justification of evidential standards must appeal to non-epistemic values but debate whether the balance of non-epistemic reasons favours varying evidential standards versus maintaining fixed high evidential standards in assessment, as both sets of standards promote different and important political virtues of advisory institutions. In this paper, I adjudicate the evidential standards debate by developing a novel (...) argument from justice, drawing on the IPCC’s assessment of climate impacts as a case study. I argue that in assessments marked by background evidential inequality, maintaining fixed high evidential standards results in an unequal distribution of ‘epistemic power’ among stakeholders, producing a ‘powerful assessment’ for the data-rich (a high rate of findings) and a ‘weak assessment’ for the data-poor (a low rate of findings). Where such inequalities of epistemic power disadvantage those in data-poor regions with respect to fundamental interests, such as basic human rights, we have decisive reasons of justice to vary evidential standards. (shrink)

The philosophy of medicine is an area that has experienced significant growth and change in the last decade or so. While usually thought to be distinct from bioethics, the philosophy of medicine addresses a wide range of questions having to do with epistemology, metaphysics, and values. In this survey, I use the concept of the reference class to draw connections between a number of questions of longstanding and of more recent concern in the philosophy of medicine. By focusing on the (...) way in which the reference class plays a role in questions and approaches that sometimes seem to be at odds with one another, I hope to illustrate the ways in which these oppositions are less complete than they might at first appear, and to show how the connections between different problems are being redrawn. (shrink)

Many approaches to evidence amalgamation focus on relatively static information or evidence: the data to be amalgamated involve different variables, contexts, or experiments, but not measurements over extended periods of time. However, much of scientific inquiry focuses on dynamical systems; the system’s behavior over time is critical. Moreover, novel problems of evidence amalgamation arise in these contexts. First, data can be collected at different measurement timescales, where potentially none of them correspond to the underlying system’s causal timescale. Second, missing variables (...) have a significantly different impact on time series measurements than they do in the traditional static setting; in particular, they make causal and structural inference much more difficult. In this paper, we argue that amalgamation should proceed by integrating causal knowledge, rather than at the level of “raw” evidence. We defend this claim by first outlining both of these problems, and then showing that they can be solved only if we operate on causal structures. We therefore must use causal discovery methods that are reliable given these problems. Such methods do exist, but their successful application requires careful consideration of the problems that we highlight. (shrink)

Evidence-based medicine (EBM) makes use of explicit procedures for grading evidence for causal claims. Normally, these procedures categorise evidence of correlation produced by statistical trials as better evidence for a causal claim than evidence of mechanisms produced by other methods. We argue, in contrast, that evidence of mechanisms needs to be viewed as complementary to, rather than inferior to, evidence of correlation. In this paper we first set out the case for treating evidence of mechanisms alongside evidence of correlation in (...) explicit protocols for evaluating evidence. Next we provide case studies which exemplify the ways in which evidence of mechanisms complements evidence of correlation in practice. Finally, we put forward some general considerations as to how the two sorts of evidence can be more closely integrated by EBM. (shrink)