European Association of Science Editors Statements

Statements

EASE statement

Inappropriate use of impact factors

Below is the final EASE statement on inappropriate use of impact factors. The draft text was open for comments until 1 September 2007. The EASE Council then decided on the final text. We especially sought quotes (with references) that official bodies (such as research councils, universities, government organisations, etc.) use journal impact factors when judging researchers, research programmes or grant applications. The final text was published in the November 2007 issue of European Science Editing as an official EASE statement.

Download a copy:
EASE statement on inappropriate use of impact factors

EASE statement on inappropriate use of impact factors

The journal impact factor was developed as a means to measure the impact of scientific journals [1,2]. Over time, its use has been extended to measuring the quality of scientific journals, the quality of individual articles and the productivity of individual researchers [3,4]. Impact factors are nowadays even used in academic appointments, to evaluate grant applications and to allocate other financial support for research programmes [5,6].

The impact factor, however, is not always a reliable instrument for measuring the quality of journals [7,8]. Its use for purposes for which it was not intended causes even greater unfairness [9-12].

Therefore the European Association of Science Editors recommends that journal impact factors are used only - and cautiously - for measuring and comparing the influence of entire journals, but not for the assessment of single papers, and certainly not for the assessment of researchers or research programmes either directly or as a surrogate.

1     "The "impact factor" is similar to the quantitative measure obtained by Gross in evaluating the relative importance of scientific journals [...]".

Garfield E. Citation indexes for science. A new dimension in documentation through association of ideas. Science. 1955 Jul 15;122(3159):108-111.

2     "Measures of citation frequency and impact factor should be helpful in determining the optimum makeup of both special and general [library journal] collections."

Garfield E. Citation analysis as a tool in journal evaluation. Journals can be ranked by frequency and impact of citations for science policy studies. Science. 1972 Nov 3;178(60):471-479.

3     "While the IFS [impact factor score] was designed to assess journals, there are frequent mentions in the literature of the IFS being used as an indicator of the eventual impact of a scholar's work."

Holden G, Rosenberg G, Barker K, Onghena P. Should decisions about your hiring, reappointment, tenure, or promotion use the impact factor score as a proxy indicator of the impact of your scholarship? Medscape General Medicine. 2006 Jul 28;8(3):21.

4     "[...] the Higher Education Funding Council in Britain came to understand that it was assessing science in a fundamentally unscientific way by using the impact factor of journals as a surrogate for the impact of articles published in them."

Smith R. Commentary: the power of the unrelenting impact factors is it a force for good or harm? International Journal of Epidemiology. 2006 Sep 19;35:1129-30.

5    "Evaluationsgrundlage sind die Impactfaktoren [bzw. die Journal-Reihungen] aus der unveranderten Impactfactor-Liste des ISI, jeweils letzte verfagbare Ausgabe zum Zeitpunkt des Einreichsdatums zur Habilitation. Die Publikationen der/s Habilitand/in/en werden getrennt nach Erst- und Koautorschaften" [The basis for evaluation are the impact factors [respectively the journal rankings] from the unchanged impact factor list of ISI, always the most recent available issue at the time of submitting the application. The publications of the applicant are distinguished in first authorship and co-authorship]

Habilitationsrichtlinien der Medizinische Universitat Wien [Guidelines for qualifation as a university teacher at the Medical University of Vienna]. Wien: Medizinische Universitat Wien; 2004 May.

6 "Universities in Germany, for instance, regularly plug the impact factor of journals in which scientists publish into formulae to help them determine departmental funding. The Italian Association for Cancer Research requires grant applicants to complete worksheets calculating the average impact factor of the journals in which their publications appear. [...] [In Finland] government funding for university hospitals is partly based on publications points, with a sliding scale corresponding to the impact factor of the journals in which researchers publish their work."

Adam D. The counting house. Nature. 2002 Feb 14;415(6873)726-729.

7    "All citation studies should be adjusted to account for variables such as specialty, citation density, and half-life."

Garfield E. The history and meaning of the journal impact factor. JAMA. 2006 Jan 4;295(1):90-3.

8     "Apart from being non-representative, the journal impact factor is encumbered with several shortcomings of a technical and more fundamental nature. [...] Pure technicalities can therefore account for several-fold differences in journal impact."

Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997 Feb 15;314(7079):498-502.

9    "The IFS [impact factor score] was the best predictor of both short- and long-term impact [of journal articles], yet even when the IFS was combined with other predictors, the overall amount of variance in both short- and long-term impact was less than 13%."

Holden G, Rosenberg G, Barker K, Onghena P. Should decisions about your hiring, reappointment, tenure, or promotion use the impact factor score as a proxy indicator of the impact of your scholarship? Medscape General Medicine. 2006 Jul 28;8(3):21.

10     "Indeed, of 38 million items cited from 1900-2005, only 0.5% were cited more than 200 times. Half [of the published articles] were not cited at all [...]. The skewness of citations is well known and repeated as a mantra by critics of the impact factor. [...] The use of JIFs [journal impact factors] instead of actual article citation counts to evaluate individuals is a highly controversial issue. Granting and other policy agencies often wish to bypass the work involved in obtaining citation counts for individual articles and authors. [...] Thus, the JIF is used to estimate the expected count of individual papers, which is rather dubious considering the known skewness observed for most journals."

Garfield E. The history and meaning of the journal impact factor. JAMA. 2006 Jan 4;295(1):90-3.

11    "[In Finland] a single paper published in a journal with an impact factor of 3, rather than 2, could have boosted a hospital's funding by about US$7,000 in 2000."

Adam D. The counting house. Nature. 2002 Feb 14;415(6873)726-729.

12     "Even the uncited articles are then given full credit for the impact of the few highly cited articles that predominantly determine the value of the journal impact factor. [...] However, the correlation between journal impact and actual citation rate of articles from individual scientists or research groups is often poor."

Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997 Feb 15;314(7079):498-502.

Statements
EASE statement on impact factors - Apr 20, 2008 - 5:30 PM
Download a copy of EASE's statement on inappropriate use of impact factors

Download attachment(s): [ EASE statement on impact factors ]

[ Read More... ]

<< prev page next page >>