Abstract

The aim of this paper is to understand whether the probability of receiving positive peer reviews is influenced by having been published in an independently assessed, high-ranking journal: we interpret a positive relationship among peer evaluation and journal ranking as evidence that journal ratings are good predictors of article quality. The analysis is based on a large dataset of more than 11,500 research articles published in Italy between 2004 and 2010 in the areas of architecture, arts and humanities, history and philosophy, law, sociology and political sciences. These articles received a score from a significant number of externally appointed referees during the Italian research assessment exercise. Similarly, journal scores were assigned in a panel-based independent assessment, which involved academic journals in which Italian scholars were published, carried out using a different procedure. An article’s score was compared with that of the journal in which it was published. We first estimate an ordered probit model, assessing the paper’s probability receiving a higher score due to the higher journal’s score. In a second step, we concentrate on the top papers, evaluating the probability of a paper receiving an excellent score after it was published in a top-rated journal. In doing so, we control some characteristics of the paper and its author, including the publication’s language, the scientific field and its size, the author’s age and academic status. We add to journal classification literature by providing a large-scale test of the robustness of expert-based classification for the first time.