An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis (...) falls far short of that standard. Different meta-analyses of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

The “erasing race” effect is the reduction of the salience of “race” as an alliance cue when recalling coalition membership, once more accurate information about coalition structure is presented. We conducted a random-effects model meta-analysis of this effect using five United States studies (containing nine independent effect sizes). The effect was found (ρ = 0.137, K = 9, 95% CI = 0.085 to 0.188). However, no decline effect or moderation effects were found (a “decline effect” in this context (...) would be a decrease in the effect size over time). Furthermore, we found little evidence of publication bias. Synthetically correcting the effect size for bias stemming from the use of an older method for calculating error base rates reduced the magnitude of the effect, but the it remained significant. Taken together, these findings indicate that the “erasing race” effect generalizes quite well across experimental contexts and would, therefore, appear to be quite robust. We reinterpret the theoretical basis for these effects in line with Brunswikian evolutionary-developmental theory and present a series of predictions to guide future research in this area. (shrink)

Scholars have proposed a number of courses and programs intended to improve the ethical behavior of scientists in an attempt to maintain the integrity of the scientific enterprise. In the present study, we conducted a quantitative meta-analysis based on 26 previous ethics program evaluation efforts, and the results showed that the overall effectiveness of ethics instruction was modest. The effects of ethics instruction, however, were related to a number of instructional program factors, such as course content and delivery (...) methods, in addition to factors of the evaluation study itself, such as the field of investigator and criterion measure utilized. An examination of the characteristics contributing to the relative effectiveness of instructional programs revealed that more successful programs were conducted as seminars separate from the standard curricula rather than being embedded in existing courses. Furthermore, more successful programs were case based and interactive, and they allowed participants to learn and practice the application of real-world ethical decision-making skills. The implications of these findings for future course development and evaluation are discussed. (shrink)

Meta‐analysis: the glass eye of evidence‐based practice?Meta‐analysis was developed as a technique for combining the results of many different quantitative studies: it is often used to produce quantitative estimates of causal relations and/or association between variables. Meta‐analysis is sometimes regarded as a central component of evidence‐based practice. We draw attention to an incompatibility in the epistemology and methods of reasoning in quantitative meta‐analysis and the epistemology and reasoning implicit in expert practice. We (...) argue that this may be because the common perception of meta‐analysis appeals to truth as correspondence; we suggest that rejecting the naïve realism that underpins truth as correspondence allows meta‐analysis to be understood in terms of truth as coherence. We can then develop an account of meta‐analysis that does not depend upon reduction to a mathematical procedure but is an attempt to maximise coherence in beliefs about what works that is consistent with clinical reasoning in expert practice. (shrink)

Recent research highlights the relationship between perceived brand ethicality (PBE), consumer purchase intention, and the consumer–brand relationship. Existing empirical studies offer mixed findings on whether these three relate positively, negatively, or not at all. Moreover, their relationships have not been the primary focus of existing meta-analytic reviews. Therefore, we conducted a meta-analysis to provide an empirical consensus to this debate by studying the magnitude of the association between PBE and consumer responses (purchase intention, brand trust, and brand (...) loyalty). Moreover, we examined the moderating effects of self-accountability and brand experience to expand our understanding of this relationship. After a thorough literature review from major databases and cross-referencing of the relevant articles, we selected 31 peer-reviewed articles for this meta-analysis. The results reveal that consumer response to PBE positively influences attitude formation towards the brand and purchase intentions. Additionally, moderation analyses reveal the crucial roles of self-accountability and brand experience in influencing the effects of PBE on consumer–brand relationships and purchase intention. Crucial theoretical and practical implications are discussed regarding these relationships, followed by limitations and future research directions. (shrink)

Emotional clarity and attention to emotions represent the extent to which people understand and attend to their own emotions, respectively, and are broad facets of emotional awareness, alexithymia, and emotional intelligence. To examine the extent to which these two constructs are associated, we conducted a meta-analysis of studies including well-validated self-report measures of trait clarity and attention to emotion. Clarity and attention were moderately, positively associated. Assessment instrument, but not sample gender or age, moderated the association between clarity (...) and attention. Analyses of between-study heterogeneity and publication bias suggested that results were valid and generalizable. We discuss potential causes of the association between clarity and attention, and elaborate on the implications of our results for emotion regulation, well-being, and psychopathology. (shrink)

E-learning has gained popularity since the outbreak of COVID-19. This study aims to identify gender differences in e-learners' self-efficacy, satisfaction, motivation, attitude, and performance across the world. Through a meta-analysis and systematic review, this study concludes that there are generally no significant gender differences in e-learning outcomes except in a few countries. Females significantly outperformed males in Spain and the UK. In Austria, India, and mixed countries, females hold significantly more positive attitudes toward e-learning than males. In the (...) USA, females present significantly higher self-efficacy than males. Future research into the gender issue in e-learning across the world may adopt cross-disciplinary research methods except for a meta-analysis. (shrink)

This paper presents a series of meta-analyses of the validity of general mental ability (GMA) for predicting five occupational criteria, including supervisory ratings of job performance, production records, work sample tests, instructor ratings, and grades. The meta-analyses were conducted with a large database of 467 technical reports of the validity of the General Aptitude Test Battery (GATB) which included 630 independent samples. GMA showed to be a consistent predictor of the five criteria, but the magnitude of the operational (...) validity was not the same across the five criteria. Results also showed that job complexity is a moderator of the GMA validity for the performance criteria. We also found that the GMA validity estimates are slightly smaller than the previous ones obtained by Hunter (1983a; Hunter & Hunter, 1984). Finally, we discuss the implications of these findings for the research and practice of personnel selection. (shrink)

In evolutionary psychology predictions, women’s mate preferences shift between fertile and nonfertile times of the month to reflect ancestral fitness benefits. Our meta-analytic test involving 58 independent reports was largely nonsupportive. Specifically, fertile women did not especially desire sex in short-term relationships with men purported to be of high genetic quality. The few significant preference shifts appeared to be research artifacts. The effects declined over time in published work, were limited to studies that used broader, less precise definitions of (...) the fertile phase, and were found only in published research. (shrink)

This meta-analysis of 31 studies over 20 years advances our understanding of the gender gap in entrepreneurial bank finance. Findings from previous research on the relationship between entrepreneurs’ gender and bank financing are mixed, which suggests the need to pay particular attention to entrepreneurs’ social context. In this study, we develop a model of how social gender norms explain variation in women entrepreneurs’ (vis-à-vis men entrepreneurs’) access to bank finance. Specifically, we theorize how women’s formal (their nations’ political (...) ideologies) and informal (women’s empowerment) social standing within their societies influence gender discrimination in entrepreneurial bank financing. Consistent with most previous studies, our baseline results show that women entrepreneurs’ business loan applications are rejected to a greater extent than men entrepreneurs’ loan applications. Women entrepreneurs also pay higher interest rates on loans than men entrepreneurs. Further, in societies dominated by a conservative (rather than a liberal) political ideology, the positive relationship between women entrepreneurs and loan interest rates is more positive. Interestingly, gender discrimination in loan rejection and interest rates is magnified in societies with greater women’s empowerment. Taking a social gender-norm perspective, our analysis establishes a gender gap in entrepreneurial bank finance, and we outline an agenda for further research. (shrink)

Mental contrasting with implementation intentions is a self-regulation strategy that enhances goal attainment. This meta-analysis evaluated the efficacy of MCII for goal attainment and explored potential moderators. A total of 21 empirical studies with 24 independent effect sizes were included in the analysis. Results showed that MCII to be effective for goal attainment with a small to medium effect size. The effect was mainly moderated by intervention style. Specifically, studies with interventions based on interactions between participants and (...) experimenters had stronger effects than studies with interventions based on interactions between participants and documents. The results revealed that MCII is a brief and effective strategy for goal attainment with a small to moderate effect; however, because of some publication bias, the actual effect sizes may be smaller. Due to small number of studies in this meta-analysis, additional studies are needed to determine the role of moderator variables. (shrink)

OBJECTIVES: This reliability generalization study aimed to estimate the mean and variance of the interrater reliability coefficients (ryy) of supervisory ratings of overall, task, contextual, and positive job performance. The moderating effect of the appraisal purpose and the scale type was examined. It was hypothesized that the ratings collected for research purposes and multi-item scales have higher ryy. It was also examined whether ryy was similar for the four performance dimensions. METHOD: A database consisting of 224 independent samples was created (...) and hierarchical sub-grouping meta-analyses were conducted. RESULTS: The appraisal purpose was a moderator of ryy for the four performance dimensions. Scale type was a moderator of ryy for overall and task performance collected for research purposes. The findings also suggest that supervisors seem to have less difficulty evaluating overall job performance than task, contextual, and positive performance. The best estimates of the observed ryy for overall job performance are.61 for research-collected ratings and.45 for administrative-collected ratings. MAIN CONCLUSIONS: (1) Appraisal purpose moderates ryy and researchers and practitioners should be aware of its effects before collecting ratings or using empirically-derived interrater reliability distributions, (2) Scale type seems to moderate ryy in the case of the ratings collected for research purposes, only, (3) overall job performance is more reliably rated than task, contextual, and positive performance. Implications for research and practice are discussed. (shrink)

The biophilia hypothesis posits an innate biological and genetic connection between human and nature, including an emotional dimension to this connection. Biophilic design builds on this hypothesis in an attempt to design human-nature connections into the built environment. This article builds on this theoretical framework through a meta-analysis of experimental studies on the emotional impacts of human exposure to natural and urban environments. A total of 49 studies were identified, with a combined sample size of 3,201 participants. The (...) primary findings indicated that exposure to natural environments had a medium to large effect on both increasing positive affect and decreasing negative affect. This finding supported the anticipated emotional dimension of the biophilia hypothesis and lends credibility to biophilic design theory. Evidence was revealed in support of the affective/arousal response model. Immersion in environments indicated a larger effect size than laboratory simulation of environments. Methodological recommendations for future experimental research were few, however the Positive and Negative Affect Schedule outcome measure was recommended as a measure of both positive and negative affect for further studies. A combination measurement of stress related outcome variables was proposed to further explore the affective/arousal response model and its potential relationship to the biophilia hypothesis. The meta-analysis provides evidence for fundamental theories regarding human-nature connection, while revealing gaps in current knowledge. (shrink)

Emotional intelligence is an emerging field since the 1990s due to its important outcomes for employees. This study is a psychometric meta-analysis examining the links between emotional intelligence and organizational commitment, organizational citizenship behavior, job satisfaction, job performance, and job stress of employees. In this meta-analysis, carefully selected studies on emotional intelligence since the origin of the concept in 1990 were included along with studies examining its outcomes. For this analysis, three streams of emotional intelligence, (...) consistent with previous meta-analyses, were considered: ability, self-report, and mixed emotional intelligence. This meta-analysis is an attempt to add to the literature by analyzing the relationships between emotional intelligence and selected employee outcomes over a period of time beginning in 1990. The three streams of emotional intelligence were separately analyzed to examine their relationship with employee outcomes. These outcomes were included in the study based on select research studies. Our study results showed that emotional intelligence and its three streams were positively related to organizational commitment, organizational citizenship behavior, job satisfaction, and job performance and negatively related to job stress. (shrink)

Are there relationships between consciousness and the material world? Empirical evidence for such a connection was reported in several meta-analyses of mind-matter experiments designed to address this question. In this paper we consider such meta-analyses from a statistical modeling perspective, emphasizing strategies to validate the models and the associated statistical procedures. In particular, we explicitly model increased data variability and selection mechanisms, which permits us to estimate 'selection profiles ' and to reassess the experimental effect in view of (...) potential other effects. An application to the data pool considered in the influential meta-analysis of Radin and Nelson (1989) yields indications for the presence of random and selection effects Adjustment for possible selection is found to render the,without such an adjustment significant, experimental effect non-significant. Somewhat different conclusions apply to a subset of the data deserving separate consideration. The actual origin of the data features that are described as experimental, random, or selection effects within the proposed model cannot be clarified y our approach and remains open. (shrink)

Computational models of child language development can help us understand the cognitive underpinnings of the language learning process, which occurs along several linguistic levels at once (e.g., prosodic and phonological). However, in light of the replication crisis, modelers face the challenge of selecting representative and consolidated infant data. Thus, it is desirable to have evaluation methodologies that could account for robust empirical reference data, across multiple infant capabilities. Moreover, there is a need for practices that can compare developmental trajectories of (...) infants to those of models as a function of language experience and development. The present study aims to take concrete steps to address these needs by introducing the concept of comparing models with large-scale cumulative empirical data from infants, as quantified by meta-analyses conducted across a large number of individual behavioral studies. We formalize the connection between measurable model and human behavior, and then present a conceptual framework for meta-analytic evaluation of computational models. We exemplify the meta-analytic model evaluation approach with two modeling experiments on infant-directed speech preference and native/non-native vowel discrimination. (shrink)

Emotion Review, Volume 14, Issue 1, Page 66-83, January 2022. To better define the boundaries of conceptually overlapping constructs of intrapersonal emotion knowledge, we examined meta-analytic correlations among five intrapersonal EK-related constructs and attention to emotion. Affect labelling, alexithymia, and emotional clarity were strongly associated, and they were moderately associated with attention to emotion. Alexithymia and emotional awareness were weakly associated, and emotion differentiation was unrelated with emotional clarity. Sample characteristics and measures moderated some of the associations. Publication bias (...) was not found, except for the alexithymia-emotional awareness association. This study helped to clarify the extent to which similarly defined constructs overlap or are distinct, which can inform our decision to adequately label important constructs and employ corresponding measures. (shrink)

Emotion Review, Volume 14, Issue 1, Page 66-83, January 2022. To better define the boundaries of conceptually overlapping constructs of intrapersonal emotion knowledge, we examined meta-analytic correlations among five intrapersonal EK-related constructs and attention to emotion. Affect labelling, alexithymia, and emotional clarity were strongly associated, and they were moderately associated with attention to emotion. Alexithymia and emotional awareness were weakly associated, and emotion differentiation was unrelated with emotional clarity. Sample characteristics and measures moderated some of the associations. Publication bias (...) was not found, except for the alexithymia-emotional awareness association. This study helped to clarify the extent to which similarly defined constructs overlap or are distinct, which can inform our decision to adequately label important constructs and employ corresponding measures. (shrink)

Philosophical work on meta-analysis focuses on biomedical research and revolves around the question: Is meta-analytic evidence the best kind of evidence? By focusing on conservation science rather than biomedical science, I identify further questions and puzzles for meta-analysis and show their importance for the epistemology of meta-analysis.

Chow's defense of the null-hypothesis significance- test procedure is thoughtful and compelling in many respects. Nevertheless, techniques such as meta-analysis, power analysis, effect size estimation, and confidence intervals can be useful supplements to NHSTP in furthering the cumulative nature of behavioral research, as illustrated by the history of research on the spontaneous recovery of verbal learning.

Lloyd (2011) presents highly suggestive results regarding the specificity of the link between particular brain areas and cognitive tasks. Some of his evidence is derived from the analysis of data from the BrainMap database (available: www.brainmap.org), which has become a fundamental resource for the conduct of functional neuroimaging meta-analysis. In the present note, some observations regarding the possibilities and pitfalls of meta-analysis of functional neuroimaging data are given as a complement to Lloyd's excellent exposition of (...) the topic. Additionally, some comments are made on the particular meta-analytic results presented by Lloyd.Functional neuroimaging studies usually present their findings in the form .. (shrink)

Scholars have suggested that leaders’ ethical failures at the beginning of the twenty-first century have raised awareness about the importance of ethical leadership (EL). Yet, there has been no systematic effort to evaluate whether this awareness indeed led to changes in EL or how followers react to this leadership style over time. To address this gap, we examine the evolution in EL means, variability, and its associations with follower outcomes between 2004 and 2019. Our cross-temporal meta-analysis included 359 (...) independent samples from 314 studies published between 2005 and 2020 and focused on followers’ ratings of their leaders using the Ethical Leadership Scale (ELS; Brown et al., in Organizational Behavior and Human Decision Processes, 97(2), 117–134, 2005). Using cubic spline meta-regressions, our results indicated no global changes in EL mean levels and variability across the 15-year period. Nevertheless, country-level comparisons revealed different EL-trends in China (i.e. decrease) versus the United States (i.e. stable), and further moderation analyses highlighted the role of cultural value dimensions and national corruption rates. Finally, we also found that the relationship between EL and desirable follower outcomes (e.g. organizational citizenship behavior) became gradually stronger over time on a global level. These results provide a solid empirical basis to evaluate cross-temporal trends in EL and its (changing) impact on follower outcomes across the globe. (shrink)

The authors meta-analytically examined the relationship between intrinsic religiosity and internal locus of control. Thirty-seven independent samples, comprising 9,924 participants, yielded an average effect size of r =.06, which was small, but significant, indicating a positive correlation between intrinsic religiosity and internal locus of control. Moderation analyses showed a significant trend of effects becoming weaker over time. The judged religiousness of samples significantly predicted the strength of the correlation, such that more religious samples showed stronger effect sizes.

The ability to recognise others’ emotions from nonverbal cues is measured with performance-based tests and has many positive correlates. Although researchers have...

One of the new topics that has attracted the attention of researchers in recent years is neuro-marketing. The purpose of the present study is to achieve an insight into the progress of studies on neuro-marketing through review of scientific articles in this field with methodology text-mining. A total of 394 articles were selected between 2005 and 2017 using the search for “neuro-marketing” in valid databases. By reviewing the title, abstract, and keywords at various stages of screening, the researchers selected 311 (...) articles related to the neuro-marketing topic in order to carry out the text mining process. Then, pre-processing of data was performed and using cluster analysis, 6 optimal clusters were obtained according to Davis Bouldin index. Using the multi-attribute decision-making technique, the most important roots were ranked in each cluster. The results showed that interest in research on neuro-marketing has increased in recent years and clusters created around topics such as marketing strategies, advertising and video ads, ethical issues, advertising message components, neuroscience in marketing, economics and consumer behavior, and decision making and brand selection. The study shows that the researchers’ attention is moving from the concepts and principles of the use of neuroscience in marketing, economics and consumer behavior to design and develop marketing strategies using neuroscience. (shrink)

This study aims to reveal the impact of proactive personality on career success and the sequential mediation effect of organizational citizenship behavior and task performance on the relationship. Utilizing meta-analytic structural equation modeling technology sampling 101,131 employees from multiple organizations and industries, which deeply decreased sampling error, the results indicated slightly different findings of proactive personality and three types of career success. Specifically, in relation to salary, OCB and task performance independently transmit the effects of proactive personality to subjective (...) career success, but they sequentially mediate this link as well. In regard to subjective career success and promotion, OCB mediates the relationship between proactive personality and promotion. OCB and task performance sequentially mediate these links. We discussed findings cautiously and purpose future research directions. (shrink)

Arguments that medical decision making should rely on a variety of evidence often begin from the claim that meta-analysis has been shown to be problematic. In this paper, I first examine Stegenga’s argument that meta-analysis requires multiple decisions and thus fails to provide an objective ground for medical decision making. Next, I examine three arguments from social epistemologists that contend that meta-analyses are systematically biased in ways not appreciated by standard epistemology. In most cases I (...) show that critiques of meta-analysis fail to account for the full range of meta-analytic procedures. In the remainder of cases, I argue that the critiques identify problems that do not uniquely cut against meta-analysis. I close by suggesting one reason why it may be pragmatically rational to violate the principle of total evidence and by outlining the criteria for a successful argument against meta-analysis. A set of criteria I contend remain unmet. (shrink)

Based on research on the increasingly popular unconscious thought effect (UTE), it has been suggested that physicians might make better diagnostic decisions after a period of distraction than after an equivalent amount of time of conscious deliberation. However, published attempts to demonstrate the UTE in medical decision making have yielded inconsistent results. In the present study, we report the results of a meta-analysis of all the available evidence on the UTE in medical decisions made by expert and novice (...) clinicians. The meta-analysis failed to find a significant contribution of unconscious thought to the accuracy of medical decisions. This result cannot be easily attributed to any of the potential moderators of the UTE that have been discussed in the literature. Furthermore, a Bayes Factor analysis shows that most experimental conditions provide positive support for the null hypothesis, suggesting that these null results do not reflect a simple lack of statistical power. We suggest ways in which new studies could usefully provide further evidence on the UTE. Unless future research shows otherwise, the recommendation of using unconscious thought to improve medical decisions lacks empirical support. (shrink)

Commentators on empirical research in business ethics have recommended that use should be made of meta-analysis – the quantitative analysis of a group of research studies. This paper elaborates upon those recommendations by conducting, as a "case study" for further reflection, a meta-analysis of studies of accountants' organizational-professional conflict (OPC) previously published in accounting and psychology journals. Of five variables capable of analysis, only the population correlation coefficient between OPC and organizational tenure is identified. (...) It was not possible to find the population correlation coefficient of the other four variables, for which several possible reasons are suggested. The paper then shows how meta-analysis can make a valuable contribution, even when the substantive findings are limited, by providing guidance for future research and a safeguard against the drawing of unjustified conclusions, to which the more usual narrative reviews of literature are prone. (shrink)