Study reveals journal impact factor manipulation
A study into worldwide trends in impact factor manipulation in academic journals suggests potential large-scale and increasing manipulation since the beginning of this century.
This backs up the academic community's growing criticism of the impact factor as a measure of a journal's quality. Caspar Chorus from the TU Delft's Faculty of TPM and Ludo Waltman from Leiden University present the startling results of their research in the journal PLoS ONE. The paper brings quite a stir in the academic world. The head of Research of ThomsonReuters wants to discuss with the researchers how to make their metrics less susceptible for manipulation.
In their study, the researchers present a measure of potential manipulation of impact factors. It is based on suspected patterns in citation data. Chorus: “The measure provides an indication of the extent to which a journal disproportionately cites itself, and thereby positively influences the impact factor. This may, for example, be caused by editorial staff ‘requesting’ that authors include citations of work recently published in the journal in their articles.” The researchers validated this measure and then used it to investigate trends in impact factor manipulation across the decades, for all journals worldwide. “This kind of large-scale analysis is totally unprecedented”, says Chorus.
The study reveals a strong suspicion that impact factor manipulation played virtually no role at all in the years 1987-2004, but significantly gathered pace after that. This was precisely the period in which academics and policymakers began to become obsessed with the impact factor. The study backs up the view that the impact factor provides an incomplete picture of a journal's quality and of the articles published in it.