Impact factor

Assessment | Biopsychology | Comparative | Cognitive | Developmental | Language | Individual differences | Personality | Philosophy | Social |
Methods | Statistics | Clinical | Educational | Industrial | Professional items | World psychology |

Professional Psychology: Debating Chamber · Psychology Journals · Psychologists

---

An impact factor is a measure of the importance of scientific journals. It is calculated each year by the Institute for Scientific Information for those journals which it indexes, and the factors are published in the Journal Citation Reports. Impact factors have a huge, but controversial, influence on the way published scientific research is perceived and evaluated.

Calculation

The impact factor is calculated based on a three-year period. For example, the 2003 impact factor for a journal would be calculated as follows:

A = the number of times articles published in 2001-2 were cited in indexed journals during 2003

B = the number of articles, reviews, proceedings or notes published in 2001-2

2003 impact factor = A/B

There are some nuances to this: ISI excludes certain article types (such as news items, correspondence, and errata) from the denominator. New journals, that are indexed from their first published issue, will receive an Impact Factor after the completion of two years' indexing; in this case, the citations to the year prior to Volume 1, and the number of articles published in the year prior to Volume 1 are known zero values. Journals that are indexed starting with a volume other than the first volume will not have an Impact Factor published until three complete data-years are known.

Debate

It is sometimes useful to be able to compare different journals and research groups. For example, a sponsor of scientific research might wish to compare the results to assess the productivity of its investments. An objective measure of the importance of different publications is then required and the impact factor is one of such. Consequently, there is a demand for measures such as this. In comparitive use, however, it is important to remember that different scholarly disciplines can have very different publication and citation practises, which affect not only the number of citations, but how quickly, after publication, most articles in the subject reach their highest level of citation. In these cases, it would be more relevant to consider the rank of the journal in a category of its peers, rather than the raw Impact Factor value.

Favorable properties of the impact factor include:

• Wide international coverage with more than 8400 journals from 60 countries
• Results are publicly and freely available to use and understand

Impact factors are not infallible, however. For example, it is questionable whether the number of citations measures the quality or sheer number of publications. Furthermore, in a journal which has long publication times, it might be impossible to cite articles new enough to fall inside the three-year window. Indeed, for some journals, the time between submission and publication can be over two years, which leaves less than a year for citation. On the other hand, a longer temporal window would be slow to adjust to trend changes.

The most commonly mentioned faults of the impact factor include:

• Number of citations does not really measure quality of publication, but also quantity of publication. The Impact Factor denominator adjusts for this quantity.
• The temporal window for citation is too short. Classic articles are cited frequently even after several decades.
• The nature of results in different research areas yield different amounts of publications and at different pace, which has an effect on impact factors. Generally, for example, medical journals have higher impact factors than mathematical journals.
• By merely counting the frequency of citations per article and disregarding the prestige of the citing journals, the ISI IF is a metric of popularity, not of prestige.

Alternatives

In 2006, Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel [1] proposed using the PageRank algorithm used by Google to distinguish the "quality" of citations and hence improve Impact Factor calculation.

   Isi Impact Factor          PageRank            Combined
 1 52.28 ANNU REV IMMUNOL     16.78 NATURE        51.97 NATURE
 2 37.65 ANNU REV BIOCHEM     16.39 J BIOL CHEM   48.78 SCIENCE
 3 36.83 PHYSIOL REV          16.38 SCIENCE       19.84 NEW ENGL J MED
 4 35.04 NAT REV MOL CELL BIO 14.49 PNAS          15.34 CELL
 5 34.83 NEW ENGL J MED        8.41 PHYS REV LETT 14.88 PNAS
 6 30.98 NATURE                5.76 CELL          10.62 J BIOL CHEM
 7 30.55 NAT MED               5.70 NEW ENGL J MED 8.49 JAMA
 8 29.78 SCIENCE               4.67 J AM CHEM SOC  7.78 LANCET
 9 28.18 NAT IMMUNOL           4.46 J IMMUNOL      7.56 NAT GENET
10 28.17 REV MOD PHYS          4.28 APPL PHYS LETT 6.53 NAT MED

The table shows the top 10 journals by ISI Impact Factor, PageRank, and a modified system that combines the two. Nature and Science are generally regarded as the most prestigous journals, and in the combined system they come out on top.

Inflation of impact factors

A journal can adopt editorial policies that increase its impact factor[2]. These editorial policies may not solely involve improving the quality of published scientific work. Journals sometimes may publish a larger percentage of review articles. While many research articles remain uncited after 3 years, nearly all review articles receive at least one citation within three years of publication, therefore review articles can raise the impact factor of the journal. Editorials in a journal do not count as publications. However when they cite published articles, often articles from the same journal, those citations increase the cite count for the article. An editor of a journal may encourage authors to cite articles from that journal in the papers they submit.

Skewness

An editorial in Nature (Vol 435, pp 1003-1004 , 23 June 2005) stated

For example, we have analysed the citations of individual papers in Nature and found that 89% of last year’s figure was generated by just 25% of our papers. The most cited Nature paper from 2002−03 was the mouse genome, published in December 2002. That paper represents the culmination of a great enterprise, but is inevitably an important point of reference rather than an expression of unusually deep mechanistic insight. So far it has received more than 1,000 citations. Within the measurement year of 2004 alone, it received 522 citations. Our next most cited paper from 2002−03 (concerning the functional organization of the yeast proteome) received 351 citations that year. Only 50 out of the roughly 1,800 citable items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations.

This emphasizes the fact that the impact factor refers to the average number of citations per paper, and this is more a skewed than a gaussian distribution. A paper published in a high impact factor journal is likely to, itself, have a much lower number of cites than the impact factor suggests. Therefore the Impact Factor of the source journal should not be used as a substitute measure of the impact of individual articles in the journal.

Use in scientific employment

Though the impact factor was originally intended as an objective measure of the reputability of a journal (Garfield), it is now being increasingly applied to measure the productivity of scientists.

See also

• Smallest publishable unit
• Eugene Garfield
• Citation index
• Hirsch number, a proposed alternate kind of impact factor

References

• Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique response to a crisis. American Psychologist, 58, 15–23.
• Anseel,F., Duyck,W.,De Baene,W., and Brysbaert, M. (2004) Journal Impact Factors and Self-Citations: Implications for Psychology Journals. American Psychologist, ,49-51 Full text
• Boor, M. (1982). The citation impact factor:Another dubious index of journal quality.American Psychologist, 37, 975–977.
• Gottfredson, S. D. (1978). Evaluating psychological research reports: Dimensions, reliability, and correlates of quality judgments. American Psychologist, 33, 920–934.
• McGarty, C. (2000). The citation impact factor in social psychology: A bad statistic that encourages bad science? Current Research in Social Psychology, 5(1), 1–16.

External links

• The Impact Factor, from the Thomson ISI website
• Impact factors from recent years.
• The Number That's Devouring Science, an article from The Chronicle of Higher Education debating the use of impact factors in academic jobs.
• Citation data: the wrong impact?
• Impact factor: a valid measure of journal quality?
• Per O Seglen, Why the impact factor of journals should not be used for evaluating research, BMJ 1997;314:497
• Popularity and/or prestige