Purpose of this page: to alert faculty and administrators to selected resources on citation-based metrics.

Cited Reference Searching in Web of Science is the oldest and best-known measure of productivity and impact of an individual or a group of researchers^1,2,3. It is not without problems or controversy.

Other measures that have been developed include the h-index, new search engines, and open access publishers such as BioMed Central flag articles as Highly Accessed.¹⁰ These competing measures "...and the increased emphasis on quality control...have led to the emergence of an evaluation industry."⁸

A new editorial (December 2008) by Phillip Bourne provides a glimpse of what may be possible in the future to identify the scholarly output of an individual online, completely and "...in perpetuity and returned in a variety of easy-to-digest ways.¹²

Suggestions & Caveats for Faculty and Administrators about citation metrics :

• See New report on citation statistics from the International Mathematical Union 6/12/08
  
  See also commentary about the report, for example:
  Chronicle of Higher Education article 6/08
  Steven Harnad's Blog on Open Access
  "Cite Check" Article from Inside Higher Education July 8, 2008
  
• Not all journals are covered in the creation of the source records. Consult the journal lists for the Arts & Humanities Citation Index, the Science Citation Index (expanded), or the Social Sciences Citation Index at http://scientific.thomson.com/mjl/.
• A list of publications, with full authorship, should be used to search for cited references. First authors (last name plus initials) should be used for maximum retrieval of cited references in the ISI Web of Knowledge. Secondary authors are indexed only if the cited reference has been used as a source document in the citation database.
• Consider the following when searching for an author:
• Names can be misspelled.
• Names with hyphens and apostrophes may not be handled consistently.
• Inconsistent use of initials by authors/editors/journals
• Common names and initials may retrieve papers written by someone with the same name but in a different discipline.
• If a person is not the first author on a paper that is not covered by the citation index, then one must search for that paper by the first author.
• Errors may occur in the year, volume and/or page numbers of a citation.
• Self-citation may account for some of the cited references.
• Not all cited references are positive. The initial paper on "cold fusion" has a large number of citations from papers covering the fact that the results could not be reproduced.
• Book reviews, letters, meeting abstracts, etc. are also source records.
• Review articles may be cited more than those that offer new concepts or ideas.
• Cited papers may be cited for reasons having more to do with the sociology of science rather than the importance of the paper - e.g. a former graduate student may cite his/her advisor; the more widely known are cited more than the less widely known for political reasons or due to grantsmanship.⁹
• See references 8 and 13 below for additional caveats.

For more information, consult:

1. Courtois, Martin. "Tips for searching the ISI citation indexes for personnel decisions." Database, 16(3): 60-67. 1993.
2. Garfield, Eugene. "How to use citation analysis for faculty evaluations, and when is it relevant? Part 1." Essays of an Information Scientist. http://www.garfield.library.upenn.edu/essays/v6p354y1983.pdf
3. Garfield, Eugene. "How to use citation analysis for faculty evaluations, and when is it relevant? Part 2." Essays of an Information Scientist. http://www.garfield.library.upenn.edu/essays/v6p363y1983.pdf
4. Lawrence, Steve. "Free online availability substantially increases a paper's impact," Nature 411, 521 (2001).
5. Meho, Lokman I. "The Rise and Rise of Citation Analysis." Physics World 20, no. 1 (January 2007): 32-36.
6. Saha S, Saint S, and Christakis DA. "Impact factor: a valid measure of journal quality?" Journal of the Medical Library Association 91, no.1 (January 2003): 42-46.
7. Hirsch Jorge E. "An index to quantify an individual's scientific research output," Proceedings of the National Academy of Sciences of the United States of AmericaVolume: 102 Issue: 46 Pages: 16569-16572 Published: NOV 15 2005
8. Leydesdorff "Caveats for the use of citation indicators in research and journal evaluations" Journal of the American Society for Information Science and Technology [1532-2882] yr:2008 vol:59 iss:2 pg:278 -287
9. Wendl MC "H-index: however ranked, citations need context," Nature Vol 449 27 Sept. 2007 page 403.
10. Most viewed articles on BioMed Central Web page accessed 2/14/08 http://www.biomedcentral.com/info/about/mostviewed.
11. Vucovich, LA; Baker, JB; Smith, JT. "Analyzing the impact of an author's publications," Journal of the Medical Library Association 96 (1): 63-66. Jan. 2008.
12. Bourne, PE; Fink, JL. "I am not a scientist, I am a number," PLos Computational Biology 4 (12):e1000247. doi: 10.1371/journal.pcbi.1000247.
13. Moed, HF. Citation Analysis in Research Evaluation, Dordrecht: Springer, 2005. Available in the UT Hodges Library PN171.F56 M64 2005 in paper and electronically in the UT Libraries' online catalog or SpringerLink database.
    This book has been cited at least 82 times, according to a search of Web of Science that was run in 2/09.
14. The journal Scientometrics publishes articles in this area. From the About section of the journal: "Scientometrics is concerned with the quantitative features and characteristics of science. Emphasis is placed on investigations in which the development and mechanism of science are studied by statistical mathematical methods."

Gayle Baker
University of Tennessee, Knoxville Libraries
December 2000

Updated Feb 2009 by Ann Viera, Veterinary Medicine Reference Librarian and Josh Hogan, Graduate Assistant
Pendergrass AG-VET MED Library