My aim in this paper is to show how the problem of inflated effect sizes corrupts the severity measure of evidence. This has never been done. In fact, the Winner’s Curse is barely mentioned in the philosophical literature. Since the severity score is the predominant measure of evidence for frequentist tests in the philosophical literature, it is important to underscore its flaws. It is also crucial to bring the philosophical literature up to speed with the limits of classical testing. (...) The Winner’s Curse is one of them. The problem is that when a significant result is obtained by using an underpowered test, the severity score becomes particularly high for large discrepancies from the null-hypothesis. This means that such discrepancies are very well supported by the evidence according to that measure. However, it is now well documented that significant tests with low power display inflated effect sizes. They systematically show departures from the null hypothesis H0 that are much greater than they really are. From an epistemological point of view this means that a significant result produced by an underpowered test does not provide evidence for large discrepancies from H0. Therefore, the severity score is an inadequate measure of evidence. Given that we are now aware of the phenomenon of inflated effect sizes, it would be irresponsible to rely on the severity score to measure the strength of the evidence against the null. Instead, one must take appropriate measures to try and avoid using underpowered tests by setting a threshold for the sample size or by replicating the results of the experiment. (shrink)

This is a brief response to David Wright’s commentary on my paper, “Critical Thinking Instruction: A Realistic Evaluation: The Dream vs. Reality.” Wright claims that if one looks more closely at the literature on critical thinking assessment that the reported effect sizes for CT instruction are quite respectable and my standards are too high. My comments will focus is on whether effect size is both problematic and an adequate measure for assessment.

DOI: 10.1080/00031305.2018.1564697 When the editors of Basic and Applied Social Psychology effectively banned the use of null hypothesis significance testing (NHST) from articles published in their journal, it set off a fire-storm of discussions both supporting the decision and defending the utility of NHST in scientific research. At the heart of NHST is the p-value which is the probability of obtaining an effect equal to or more extreme than the one observed in the sample data, given the null hypothesis (...) and other model assumptions. Although this is conceptually different from the probability of the null hypothesis being true, given the sample, p-values nonetheless can provide evidential information, toward making an inference about a parameter. Applying a 10,000-case simulation described in this article, the authors found that p-values’ inferential signals to either reject or not reject a null hypothesis about the mean (α = 0.05) were consistent for almost 70% of the cases with the parameter’s true location for the sampled-from population. Success increases if a hybrid decision criterion, minimum effect size plus p-value (MESP), is used. Here, rejecting the null also requires the difference of the observed statistic from the exact null to be meaningfully large or practically significant, in the researcher’s judgment and experience. The simulation compares performances of several methods: from p-value and/or effect size-based, to confidence-interval based, under various conditions of true location of the mean, test power, and comparative sizes of the meaningful distance and population variability. For any inference procedure that outputs a binary indicator, like flagging whether a p-value is significant, the output of one single experiment is not sufficient evidence for a definitive conclusion. Yet, if a tool like MESP generates a relatively reliable signal and is used knowledgeably as part of a research process, it can provide useful information. (shrink)

Sex dimorphism occurs when group means differ by four or more standard deviations. However, the average size of the corpus callosum is greater in males by about one standard deviation in rats, 0.2 standard deviation in humans, and virtually zero in mice. Furthermore, variations in corpus callosum size are related to brain size and are not sex specific.

Absolute and relative outcome measures measure a treatment’s effect size, purporting to inform treatment choices. I argue that absolute measures are at least as good as, if not better than, relative ones for informing rational decisions across choice scenarios. Specifically, this dominance of absolute measures holds for choices between a treatment and a control group treatment from a trial and for ones between treatments tested in different trials. This distinction has hitherto been neglected, just like the role of (...) absolute and baseline risks in rational decision-making that my analysis reveals. Recognizing both aspects advances the discussion on reporting outcome measures. (shrink)

Alternative displays of effect size statistics can enhance the understandability and impact of validity evidence in a variety of applied settings. Arguably, the proliferation of alternative effect size statistics has been limited due to the lack of user-friendly tools to create them. Common statistical packages do not readily produce these alternative effect sizes and existing tools are outdated and inaccessible. In this paper, I introduce a free-to-use web-based calculator (https://dczhang.shinyapps.io/expectancyApp/) for generating alternative effect (...) class='Hi'>size displays from empirical data. This calculator requires no mathematical or programming expertise, and therefore, is ideal for academics and practitioners. I also present results from an empirical study that demonstrates the benefits of alternative effect size displays for enhancing lay people’s perceived understandability of validity information and attitudes toward the use of standardized testing for college admissions. (shrink)

The usefulness of effect-size differs in utilitarian experiments from its use in theory corroborations. Chow introduces the question of the relationship of effect-size to practical validity and the role of the assessment of “importance” in this. This review develops this question and suggests the actuarial table as a replacement for effect-size in practical decision-making.

The variance effective population size (NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document}) is frequently used to quantify the expected rate at which a population’s allele frequencies change over time. The purpose of this paper is to find expressions for the global NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document} of a spatially structured population that are of interest for conservation of species. Since NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document} depends on (...) allele frequency change, we start by dividing the cause of allele frequency change into genetic drift within subpopulations (I) and a second component mainly due to migration between subpopulations (II). We investigate in detail how these two components depend on the way in which subpopulations are weighted as well as their dependence on parameters of the model such a migration rates, and local effective and census sizes. It is shown that under certain conditions the impact of II is eliminated, and NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document} of the metapopulation is maximized, when subpopulations are weighted proportionally to their long term reproductive contributions. This maximal NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document} is the sought for global effective size, since it approximates the gene diversity effective size NeGD\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eGD}$$\end{document}, a quantifier of the rate of loss of genetic diversity that is relevant for conservation of species and populations. We also propose two novel versions of NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document}, one of which (the backward version of NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document}) is most stable, exists for most populations, and is closer to NeGD\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eGD}$$\end{document} than the classical notion of NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document}. Expressions for the optimal length of the time interval for measuring genetic change are developed, that make it possible to estimate any version of NeV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_{eV}$$\end{document} with maximal accuracy. (shrink)

Recent downward trends in corporate giving have renewed interest in the factors that shape corporate philanthropy. This paper examines the relationships between charitable contributions, firm size and industry. Improvements over previous studies include an IRS data base that covers a much broader range of firm sizes and industries as compared to previous studies and estimation using an instrumental variable technique that explicitly addresses potential simultaneity between charitable contributions and profitability. Important findings provide evidence of a cubic relationship between charitable (...) giving and firm size and evidence of strong industry effects. The plus-minus-plus regression coefficient sign pattern for the cubic firm size model suggests that small and large firms give more relative to total receipts with lower giving ratios among medium size firms. One interpretation for this finding is that small firms are close to the communities they serve while high visibility creates a need for large firm philanthropy. Strong industry effects provide evidence of inter-industry differences in giving culture and/or different public relations requirements across industries. (shrink)

Difficulties understanding speech form one of the most prevalent complaints among older adults. Successful speech perception depends on top-down linguistic and cognitive processes that interact with the bottom-up sensory processing of the incoming acoustic information. The relative roles of these processes in age-related difficulties in speech perception, especially when listening conditions are not ideal, are still unclear. In the current study, we asked whether older adults with a larger working memory capacity process speech more efficiently than peers with lower capacity (...) when speech is presented in noise, with another task performed in tandem. Using the Eye-tracking of Word Identification in Noise Under Memory Increased Load an adapted version of the “visual world” paradigm, 36 older listeners were asked to follow spoken instructions presented in background noise, while retaining digits for later recall under low or high memory load. In critical trials, instructions directed listeners’ gaze to pictures of objects whose names shared onset or offset sounds with the name of a competitor that was displayed on the screen at the same time. We compared listeners with different memory capacities on the time course for spoken word recognition under the two memory loads by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results indicated two trends. For older adults with lower working memory capacity, increased memory load did not affect online speech processing, however, it impaired offline word recognition accuracy. The reverse pattern was observed for older adults with higher working memory capacity: increased task difficulty significantly decreases online speech processing efficiency but had no effect on offline word recognition accuracy. Results suggest that in older adults, adaptation to adverse listening conditions is at least partially supported by cognitive reserve. Therefore, additional cognitive capacity may lead to greater resilience of older listeners to adverse listening conditions. The differential effects documented by eye movements and accuracy highlight the importance of using both online and offline measures of speech processing to explore age-related changes in speech perception. (shrink)

Business ethics journals have appeared on a few ranked lists that are specific to this niche discipline. As with more traditional academic disciplines, these rankings are used for academic rewards such as faculty tenure and promotion, along with department and school ratings. Journal ranking has been subject to considerable criticism even as its administrative use persists. Among the criticisms are that journal quality is a poor proxy for article quality, citation rate is an imperfect reflection of article influence, and bias (...) may be introduced into rankings by visibility characteristics such as journal age, size, circulation, and experience of the rater with a journal. This research note studies the effect of journal age and size on the rankings of business ethics journals compiled by Beets, Lewis, and Brower, by Albrecht, Thompson, Hoopes, and Rodrigo, and by Serenko and Bontis. Significant correlation was found for journal age with the administratively derived Beets et al. ranking. No significant correlation was found for size in any ranking study. Results were not significant for the Albrecht et al. and the Serenko and Bontis rankings representing the perspectives from surveys of active researchers or citation analysis. Perhaps sometimes a journal’s reputation precedes it, as perception of journal quality may be biased by journal visibility, either because it has been published and available for a number of years, or because it is well known and likely to be cited. (shrink)

Propagation of novel linguistic terms is an important aspect of language use and language change. Here, we test how social network size influences people's likelihood of adopting novel labels by examining hashtag use on Twitter. Specifically, we test whether following fewer Twitter users leads to more varied and malleable hashtag use on Twitter, because each followed user is ascribed greater weight and thus exerts greater influence on the following user. Focusing on Dutch users tweeting about the terrorist attack in (...) Brussels in 2016, we show that people who follow fewer other users use a larger number of unique hashtags to refer to the event, reflecting greater malleability and variability in use. These results have implications for theories of language learning, language use, and language change. (shrink)

Despite current moves in the UK to limit class sizes for young children in school, there is still a disturbing lack of research evidence on the effect of class size differences on pupils' educational progress and experience. Past research has concentrated on the effects on outcomes such as pupils' school attainments in basic areas. Much less is known about classroom processes that might mediate any such effects, though such knowledge is more useful for practice and policy. Drawing on (...) a current programme of research at the Institute of Education, University of London, this paper identifies five processes likely to be connected to class size: within class grouping practices, the nature and quality of teaching, pupil attention in class, pupil adjustment to school and effects on teachers' morale, stress and enthusiasm. Relevent research is reviewed, and in line with this research the paper concentrates on the primary school years. (shrink)

An emergent body of literature examined why some firms apply some environmental initiatives while other firms do not take responsibility for their natural environment? Thus, firm environmental orientation (responsiveness and performance) are linked in the literature to several variables. Unfortunately, the relationship between firm environmental orientation and either available resources or firm size showed mixed results and inconclusive evidence. Therefore, the aim of this paper is to show empirically how available resources and firm size can explain differences in (...) firm environmental responsiveness and environmental performance. Econometric results of environmental responsiveness using the logistic regression model demonstrated that firm size does appear to add something unique in explaining differences in environmental responsiveness while available resource can be safely dropped from the model. However, econometric analysis of environmental performance using the maximum-likelihood random effects model showed strong evidence that available resources and firm size are significant predictors of firm environmental performance. (shrink)

This study investigates the distinct and joint effects of corporate social performance, firm size, and visibility on a company’s decision to disclose sustainability-related information through sustainability reports. It seeks to provide more nuanced explanations for why certain companies tend to extensively report on their sustainability performance. First, while prior studies have predominantly focused on environmental reporting, the current analysis considers comprehensive sustainability reports that include both environmental and social issues. Second, the article argues that the effects of two important (...) antecedents of legitimacy pressure—firm size and organizational visibility—should be analyzed separately rather than restricting the analysis on the effects of legitimacy pressure per se. Third, it argues that the hypothesized effects are nonlinear because the marginal costs and benefits of sustainability reporting vary with a company’s CSP level, its size, and its visibility in the public. Finally, although there is a strong link between CSP and sustainability reporting, the strength of this link depends on its size and visibility. The study of 280 companies in environmentally and socially sensitive industries provides considerable support for these hypotheses, including evidence that size and visibility independently affect sustainability reporting and that the shape of the CSP/sustainability reporting link is contingent upon firm size and visibility. (shrink)