This paper summarizes recent and ongoing experimental work regarding the reality, nature, effects, and causes of the underrepresentation of women in academic philosophy. We first present empirical data on several aspects of underrepresentation, and then consider various reasons why this gender imbalance is problematic. We then turn to the published and preliminary results of empirical work aimed at identifying factors that might explain it.

We examine the criticisms and subsequent changes that arise in the course of peer review. Fifty-two scholars who had recently published in Administrative Science Quarterly were surveyed regarding their peer review experience and how their article changed from initial journal submission to eventual publication. Papers that challenged theoretical perspectives faced distinctively high levels of criticism and change, particularly with attention to methodology, while those that offered a new perspective or that extended or combined established perspectives were less criticized and changed. (...) The number of challenge-oriented publications was small as well, suggesting that either few such submissions survive the review process or few are submitted in the first place. Overall, peer review appears open to expansion of the variety of theoretical argument but does little to aid in the winnowing out of established perspectives. (shrink)

This paper presents a framework that editors, peer reviewers, and authors can use to identify and resolve efficiently disputes that arise during peer review in scientific journals. The framework is called a scientific dialectical brief. In this framework, differences among authors and reviewers are formatted into specific assertions and the support each party provides for its position. A literature review suggests that scientists use five main types of support; empirical data, reasoning, speculation, feelings, and status. It is suggested that the (...) scientific dialectical brief format can streamline the review process by facilitating rapid differentiation between stronger and weaker support, so that valuable time can be focused on the better-substantiated claims. The paper concludes with some suggestions for implementation. (shrink)

Peer review is a topic of considerable concern to many researchers, and there is a correspondingly large body of research on the topic. This issue of Science and Engineering Ethics presents recent work on peer review that is both grounded in empirical science and is applicable to policy decisions. This research raises two basic questions; (a) how does current peer review operate, and (b) how can it be improved? Topics addressed include descriptions of how peer review is used in Federal (...) agencies. whether peer review leads to better manuscripts, demographic characteristics of authors or reviewers (status or institutional affiliation), blinding of reviewers, authors, or results, reliability and consistency of reviews, accepting a paper before the study is done, simultaneous submission, and use of dispute resolution procedures such as scientific dialectical and pleading protocols. (shrink)

Peer review is widely recognized as a mechanism for quality control of academic content. This research article aims at comparing the review reports and decisions of reviewers who are members of the editorial board of the European Scientific Journal with those reviewers suggested by the authors and who are not affiliated with the journal. 457 review reports on 378 papers submitted to the ESJ in the period of October–December 2017 were analysed. Statistical methods including OLS and Wilcoxon rank-sum test were (...) applied based on the score approach toward the reviewers’ assessments of the papers and their characteristics related to the country, gender, and time of revisions. Results show the difference between the decisions these two groups of reviewers made. Even though editor-suggested and author-suggested reviewers need equal time to review a paper, the former are less favourable towards the authors of the papers. It is also concluded that factors such as time and country of the reviewers influence their decisions. In this regard, the editors should avoid relying their decisions solely on review reports received from reviewers suggested by the authors. However, further research with larger sample sizes should be conducted. (shrink)

One of the most dramatic techniques for constructing accounts of "undiscovery" or incompetence in science involves the manipulative deception—in some accounts, the "hoaxing"—of the putatively incompetent researcher, ostensibly as an experiment to evaluate his or her methodology and the soundness of his or her knowledge claims. In this article, the author examines five cases in which such deceptions have been employed, noting the patterns of argument that typically follow these deceptions and the factors that seem to determine the power of (...) these arguments. The author suggests that the "hoaxer" in such cases is most likely to be successful when his or her views about the targeted researcher's methodology and knowledge claims are orthodox with respect to his or her intended audience. (shrink)

Saul_Jennifer, Scepticism and Implicit Bias.

A growing body of literature has identified potential problems that can compromise the quality, fairness, and integrity of journal peer review, including inadequate review, inconsistent reviewer reports, reviewer biases, and ethical transgressions by reviewers. We examine the evidence concerning these problems and discuss proposed reforms, including double-blind and open review. Regardless of the outcome of additional research or attempts at reforming the system, it is clear that editors are the linchpin of peer review, since they make decisions that have a (...) significant impact on the process and its outcome. We consider some of the steps editors should take to promote quality, fairness and integrity in different stages of the peer review process and make some recommendations for editorial conduct and decision-making. (shrink)

Born in the 17th century, journal peer review is an extremely diverse technology, constantly torn between two often incompatible goals: the validation of manuscripts conceived as a collective industrial-like reproducible process performed to assert scientific statements, and the dissemination of articles considered as a means to spur scientific discussion, raising controversies, and civically challenging a state of knowledge. Such a situation is particularly conducive to clarifying the processes of valuation and evaluation in journal peer review. In this article, such processes (...) are considered as specific tests in order to emphasize the uncertain properties of pre-tests manuscripts. On the one hand, evaluation tests are examined at the core of the validation of manuscripts, such as defining the coordination of judging instances or controlling the modalities of inter-knowledge between reviewers and authors. They are also studied regarding the dissemination of articles, notably through the contemporary conception of a continuing evaluation test termed “post publication peer review”. On the other hand, valuation tests are both part of the validation of manuscripts, such as the weighting of different judgments of the same manuscript and the tensions that these hierarchies cause, and of the dissemination of articles, such as attention metrics recording the uses of articles. The conclusion sketches out how the articulation of these different tests has recently empowered readers as a new key judging instance for dissemination and for validation, potentially transforming the definition of peers, and thus the whole process of journal peer review. (shrink)

The authors review three theoretical approaches to strategic corporate social responsibility (CSR), which can be defined as voluntary CSR actions that enhance a firm’s competitiveness and reputation. The end result of such activities should be an improvement in financial and economic performance. Based on an overview of recent empirical evidence, the authors conclude that economic theories of strategic CSR have the greatest potential for advancing this field of inquiry, although theories of strategic leadership should also be incorporated into this perspective. (...) In the remainder of the article, they provide focused summaries of the articles presented in this special issue and outline an agenda for future research on strategic CSR and environmental sustainability. (shrink)

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s knowledge claims, but on the beliefs and (...) norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. (shrink)

Peer review of journal submissions has become one of the most important pillars of quality management in academic publishing. Because of growing concerns with the quality and effectiveness of the system, a host of enthusiastic innovators has proposed and experimented with new procedures and technologies. However, little is known about whether these innovations manage to convince other journal editors. This paper will address open questions regarding the implementation of new review procedures, the occurrence rate of various peer review procedures and (...) their distribution over scientific disciplines or academic publishers, as well as the motivations for editors or publishers to engage in novel review procedures. It shows that in spite of enthusiastic innovation, the adoption of new peer review procedures is in fact very slow, with the exception of text similarity scanners. For now, peer review innovations appear to be restricted to specific niches in academic publishing. Analysing these niches, the article concludes with a reflection on the circumstances in which innovations might be more widely implemented. (shrink)

Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. (...) We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)

Policy disputes arise at all scales of governance: in clubs, non-profits, firms, nations, and alliances of nations. Both the means and ends of policy are disputed. While many, perhaps most, policy disputes arise from conflicting ends, important disputes also arise from differing beliefs on how to achieve shared ends. In fact, according to many experts in economics and development, governments often choose policies that are “inefficient” in the sense that most everyone could expect to gain from other feasible policies. Many (...) other kinds of experts also see existing policies as often clearly inferior to known alternatives. (shrink)

This study seeks to examine the impact of alumni connections between the evaluators and evaluatees on the results of peer review ratings for the Korean national R&D project and selection success rate. Specifically, this study analyzed the evaluation results of 8,402 research proposal entries submitted between 2007 and 2011 for the “general researchers support project,” all in the Natural Science and Engineering areas and sponsored by the National Research Foundation of Korea. Each proposal entry was evaluated by three evaluators, and (...) approximately 39 percent of the proposals had at least one evaluator from the same university. The results of this study showed that evaluators have a tendency to give relatively high scores to research proposals submitted by the alumni of the same universities as their alma mater. Also, when an evaluator from the same university as an evaluatee was included in the evaluator group, the results of this study showed that the percentage of entry submissions was higher compared to the contrary. Such results show that in the process of peer review–based research proposal evaluations for national R&D projects, alumni connections have significant influence on evaluation results in South Korea. (shrink)

BackgroundTriggered by a series of controversies and diversifying expectations of editorial practices, several innovative peer review procedures and supporting technologies have been proposed. However, adoption of these new initiatives seems slow. This raises questions about the wider conditions for peer review change and about the considerations that inform decisions to innovate. We set out to study the structure of commercial publishers’ editorial process, to reveal how the benefits of peer review innovations are understood, and to describe the considerations that inform (...) the implementation of innovations.MethodsWe carried out field visits to the editorial office of two large academic publishers housing the editorial staff of several hundreds of journals, to study their editorial process, and interviewed editors not affiliated with large publishers. Field notes were transcribed and analysed using coding software.ResultsAt the publishers we analysed, the decision-making structure seems to show both clear patterns of hierarchy and layering of the different editorial practices. While information about new initiatives circulates widely, their implementation depends on assessment of stakeholder’s wishes, impact on reputation, efficiency and implementation costs, with final decisions left to managers at the top of the internal hierarchy. Main tensions arise between commercial and substantial arguments. The editorial process is closely connected to commercial practices of creating business value, and the very specific terms in which business value is understood, such as reputation considerations and the urge to increase efficiency. Journals independent of large commercial publishers tend to have less hierarchically structured processes, report more flexibility to implement innovations, and to a greater extent decouple commercial and editorial perspectives.ConclusionOur study demonstrates that peer review innovations are partly to be understood in light of commercial considerations related to reputation, efficiency and implementations costs. These arguments extend beyond previously studied topics in publishing economics, including publishers’ choice for business or publication models and reach into the very heart of the editorial and peer review process. (shrink)

Business ethics journals have appeared on a few ranked lists that are specific to this niche discipline. As with more traditional academic disciplines, these rankings are used for academic rewards such as faculty tenure and promotion, along with department and school ratings. Journal ranking has been subject to considerable criticism even as its administrative use persists. Among the criticisms are that journal quality is a poor proxy for article quality, citation rate is an imperfect reflection of article influence, and bias (...) may be introduced into rankings by visibility characteristics such as journal age, size, circulation, and experience of the rater with a journal. This research note studies the effect of journal age and size on the rankings of business ethics journals compiled by Beets, Lewis, and Brower, by Albrecht, Thompson, Hoopes, and Rodrigo, and by Serenko and Bontis. Significant correlation was found for journal age with the administratively derived Beets et al. ranking. No significant correlation was found for size in any ranking study. Results were not significant for the Albrecht et al. and the Serenko and Bontis rankings representing the perspectives from surveys of active researchers or citation analysis. Perhaps sometimes a journal’s reputation precedes it, as perception of journal quality may be biased by journal visibility, either because it has been published and available for a number of years, or because it is well known and likely to be cited. (shrink)

This article reports the results of a study of confirmational response bias among social work journals. A contrived research paper with positive findings and its negative mirror image were submitted to two different groups of social work journals and to two comparison groups of journals outside social work. The quantitative results, suggesting bias, are tentative; but the qualitative findings based upon an analysis of the referee comments are clear and consistent. Few referees from prestigious or nonprestcgrous social work journals prepared (...) reviews that were knowledgeable, scientifically astute, or objective. The best reviews came from journals outside of soccal work or from journals that are accepted as social work journals but originate with other disciplines. (shrink)

The notion of quality is analysed for its functional roots as a social heuristic for reusing others’ quality judgements and hence aiding choice. This is applied to the context of academic publishing, where the costs of publishing have greatly decreased, but the problem of finding the papers one wants has become harder. This paper suggests that instead of relying on generic quality judgements, such as those delivered by journal reviewers, that the maximum amount of judgemental information be preserved and then (...) made available to potential readers to help them find papers that meet their particular needs. The suggestion is that: multidimensional quality data be captured on review of papers, this information is stored on a database, and then used to filter papers according to the criteria set by the searcher—personalising the quality filter. In other words the quality judgements and subsequent use are maintained in a disaggregated form, maintaining the maximum informational context of the judgements for future use. The advantages, disadvantages, challenges and possible variations of this proposal are discussed. (shrink)