BioMed Central Page 1 of 8 (page number not for citation purposes) Biomedical Digital Libraries Open Access Research Three options for citation tracking: Google Scholar, Scopus and Web of Science Nisa Bakkalbasi���1, Kathleen Bauer*���1, Janis Glover���2 and Lei Wang���2 Address: 1Yale University Library, 130 Wall St., P.O. Box 208240, New Haven, CT 06520-8240, USA and 2Cushing/Whitney Medical Library, Yale School of Medicine, 333 Cedar St. P.O. Box 20804, New Haven, CT 06520-8014, USA Email: Nisa Bakkalbasi - nisa.bakkalbasi@yale.edu Kathleen Bauer* - kathleen.bauer@yale.edu Janis Glover - janis.glover@yale.edu Lei Wang - lei.wang@yale.edu * Corresponding author ���Equal contributors Abstract Background: Researchers turn to citation tracking to find the most influential articles for a particular topic and to see how often their own published papers are cited. For years researchers looking for this type of information had only one resource to consult: the Web of Science from Thomson Scientific. In 2004 two competitors emerged ��� Scopus from Elsevier and Google Scholar from Google. The research reported here uses citation analysis in an observational study examining these three databases comparing citation counts for articles from two disciplines (oncology and condensed matter physics) and two years (1993 and 2003) to test the hypothesis that the different scholarly publication coverage provided by the three search tools will lead to different citation counts from each. Methods: Eleven journal titles with varying impact factors were selected from each discipline (oncology and condensed matter physics) using the Journal Citation Reports (JCR). All articles published in the selected titles were retrieved for the years 1993 and 2003, and a stratified random sample of articles was chosen, resulting in four sets of articles. During the week of November 7���12, 2005, the citation counts for each research article were extracted from the three sources. The actual citing references for a subset of the articles published in 2003 were also gathered from each of the three sources. Results: For oncology 1993 Web of Science returned the highest average number of citations, 45.3. Scopus returned the highest average number of citations (8.9) for oncology 2003. Web of Science returned the highest number of citations for condensed matter physics 1993 and 2003 (22.5 and 3.9 respectively). The data showed a significant difference in the mean citation rates between all pairs of resources except between Google Scholar and Scopus for condensed matter physics 2003. For articles published in 2003 Google Scholar returned the largest amount of unique citing material for oncology and Web of Science returned the most for condensed matter physics. Conclusion: This study did not identify any one of these three resources as the answer to all citation tracking needs. Scopus showed strength in providing citing literature for current (2003) oncology articles, while Web of Science produced more citing material for 2003 and 1993 condensed matter physics, and 1993 oncology articles. All three tools returned some unique material. Our data indicate that the question of which tool provides the most complete set of citing literature may depend on the subject and publication year of a given article. Published: 29 June 2006 Biomedical Digital Libraries 2006, 3:7 doi:10.1186/1742-5581-3-7 Received: 18 April 2006 Accepted: 29 June 2006 This article is available from: http://www.bio-diglib.com/content/3/1/7 �� 2006 Bakkalbasi et al licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Biomedical Digital Libraries 2006, 3:7 http://www.bio-diglib.com/content/3/1/7 Page 2 of 8 (page number not for citation purposes) Background Many researchers have an interest in finding citation information about a given article ��� both how many times the article is cited and who is citing that article. This may be for the completeness of a literature search, or perhaps to find how often his or her own publications are cited. Eugene Garfield made possible the widespread use of cita- tion analysis in academe through his creation of three citation indices: Science, Humanities and Social Science Citation Indices, which were combined and transformed into an electronic version called the Web of Science. These indices were based on the concept that a carefully selected subset of journals would produce the majority of impor- tant citing literature for any given article. Citation analysis has real world implications: for good or bad, citedness is considered in grants, hiring and tenure decisions. For many reasons professors and researchers may want to demonstrate the impact of their work and citation analy- sis is one way (albeit a controversial one [1-3]) to accom- plish this. For many years Web of Science had a virtual monopoly on the provision of citedness tracking. Late in 2004 two competitors to Web of Science emerged ��� Google Scholar and Scopus. The Internet search giant Google sponsored the creation of Google Scholar, a tool that attempts to give users a sim- ple way to broadly search the scholarly literature. Google Scholar uses a matching algorithm to look for keyword search terms in the title, abstract or full text of an article from multiple publishers and web sites (Google Scholar does not share the specifics of how this algorithm works). The number of times a journal article, book chapter, or web site is cited also plays an important part in Google Scholar's ranking algorithm. Search results are displayed so that the more cited and highly relevant articles rise to the top of the set. This varies from the more traditional default "reverse chronological" order employed by most scholarly databases. Google Scholar neither lists the jour- nal titles it includes, nor the dates of coverage although they have indicated that they have agreements with most major publishers (except Elsevier). Another area of differ- ence for Google Scholar is that unlike most scholarly research databases, it looks beyond journal literature to cover other modes of scholarly communication. Other sources covered in Google Scholar include preprint servers such as arXiv (physics) and government and academic Web sites. Google Scholar does not state how a Web site qualifies for inclusion in its searches. At approximately the same time that Google Scholar was made public, Elsevier introduced Scopus, an indexing and abstracting service that contains its own citation-tracking tool. Scopus indexes a larger number of journals than Web of Science, and includes more international and open access journals. Citation coverage however only dates to 1996 (abstracts, but not citation coverage, are available back to 1966 for some journals.) Scopus includes its own Web search engine, Scirus. Scirus results are presented separately from other Scopus journal results. Also, material from Scirus does not figure into cita- tion counts for Scopus journal records. Table 1 provides a comparison summary of features in Web of Science, Sco- pus, and Google Scholar. Citation analysis has been the focus of research and dis- cussion for decades. Much has been written about citation analysis techniques [18-29], application to different disci- plines [1,28,30], and controversies surrounding the use of citation analysis and journal impact factors to gauge the value and impact of a given journal title or the corpus of a given author [1-3]. With the introduction of Scopus and Google Scholar, there have been many recent articles that include careful analysis of the features of each individual tool as well as comparisons among two or more of these tools, and others (for example, PubMed and Scirus) [9- 17]. While these articles discuss the general characteristics and report the results of sample searches the authors have completed, they do not systematically review the citation analysis functions. In a 2005 study analyzing Google Scholar, Noruzi [14] briefly compared citation counts for two products ��� Google Scholar and Web of Science ��� in the field of webometrics. First, the author selected the first article to establish the word "webometrics" [18], and pro- vided the "times cited" for both the Web of Science and Google Scholar. The author then compared the number of unique and overlapping citations to this one article in Table 1: Comparison of features in Web of Science, Scopus, and Google Scholar Web of Science Scopus Google Scholar Indexing and abstracting Yes Yes No Years covered-journals 1900 to present (Science) 1956-present (Social Science) 1975-present (Arts and Humanities) 1966 to present for some journals, but many date back to 1996 to present Not revealed Years covered-citations 1900 to present 1996 to present Not revealed Fee-based Yes Yes No Contents 9300 journals (Science, Social Science and Arts and Humanities) 15,000 journals (Science and Social Science) Not revealed