New ways of getting your work noticed via the web has been a very frequent topic of our posts here. We’ve written about raising your online visibility, making your work more discoverable, and many other aspects of getting your work noticed online. That’s why it makes me very happy to announce that a workshop is being convened to discuss these very topics. At the 3rd International Conference on Web Science (14-15 June), a workshop on tracking scholarly impact on the social web has been organized. Read the post for more details. (more…)
Posts Tagged ‘Impact factor’
Editor’s note: This is a guest crosspost from Dario Taraborelli who created the ReaderMeter application on top of the Mendeley API. Dario also blogs at Academic Productivity. He asked if he could talk a bit about citation metrics. Over to Dario….
Readers of this blog are not new to my ramblings on soft peer review, social metrics and post-publication impact measures:
- how can we measure the impact of scientific research based on usage data from collaborative annotation systems, social bookmarking services and social media?
- should we expect major discrepancies between citation-based and readership-based impact measures?
- are online reference management systems more robust a data source to study scholarly readership than traditional usage factors (e.g. downloads, clickthrough rates etc.)?
Having seen a lot of ‘top 10 lists of 2009’ around, we thought we’d throw in our two cents and give you the top 10 most read articles on Mendeley, published in 2009!
The top paper for 2009 was written by Uri Alon, entitled: ‘How to choose a good scientific problem’, published in the journal “Molecular Cell.” Our stats tell us that there are currently 74 Mendeley users who have read this paper, even though it was only published in late 2009.
The full list of the top ten articles published in 2009 on Mendeley (as of 28th January 2010) is:
1. Uri Alon, ‘How to choose a good scientific problem’, Molecular Cell (2009), Volume: 35, Issue: 6

7. Fatih Ozsolak et al, ‘Direct RNA sequencing’, Nature, Volume: 461, Issue: 7265

We’d like to point out that this isn’t an authoritative list of all the ‘most read articles for 2009’. Instead, these are the ones that appear in Mendeley user libraries and show some early indications of the popularity of a journal article. We will also track the evolution of these stats over the course of 2010.
Readership complementing the impact factor
With Mendeley’s growing user base, the readership count can complement other measures, such as citation metrics, adding an extra dimension to assessing a journal article’s impact.
For example, the article “How to choose a good scientific problem” is a general interest article, rather than being specific to biology which suggests it is not likely to have a high citation count in future primary research literature.
Nonetheless, it is already the most read paper on Mendeley published in 2009, a factor that would otherwise be missed. This indicates that the readership count can allude to other ways in which articles are used within a community, and therefore increase awareness of what should be read. The next step will be to anonymously track reading time and quality rating metrics to gather the most accurate data possible for our upcoming personalized recommendation engine.
Predicting research trends?
Understanding and predicting research trends is an important part of research. The citation count, used for decades as the gold standard in article-level metrics, can verify broad trends occurring within academic disciplines such as biology. While quite accurate, official citation metrics take two years to calculate. In contrast, readership statistics may be able to predict similar trends in real-time.
For example, look at The Scientist’s list of the hottest biology papers in 2009 (all published in 2007). The readership count for these papers on Mendeley correlates with ISI’s citation count at r=.76 (two-tailed, p=.13 due to the low sample size) – a near perfect correlation, even if only based on five papers and our userbase of just over 100,000 users:
Comparison of Mendeley’s most read papers with the ISI Citations
We look forward to comparing the top 10 list shown above to the official ISI citation metrics for 2009 publications when they are calculated and released later in 2010 or 2011.
In summary, using Mendeley’s readership figures alongside the citation metrics should make it possible in the future to evaluate the scope of a journal article within the community more effectively. Finally, further refinements and understanding of readership metrics might make it possible to identify the next big trend in the academic world.
Methodology
The top 10 list was made by noting how many times a paper appears in the libraries of individual Mendeley users (readership count) and how many distinct user tags were attributed to that paper (tag count), then we filtered the results to include only papers from 2009 – done!
Exciting news: Jason Hoyt, the founder of Ologeez (a semantic frontend for PubMed), is joining Mendeley! Jason holds a Ph.D. in Genetics from Stanford University. At the moment, he is still based in Palo Alto, but once the visa issues are sorted out, Jason will be joining us here in London as our new Research Director. TechCrunch broke the story today with a headline that made our geek hearts beat faster, comparing us to a Klingon battle cruiser de-cloaking in London.
To get started, Jason wrote up his reasons for joining us, and how Mendeley can help change the Impact Factor. Over to him:
———————————
Changing the Journal Impact Factor
Right, so the first thing I had to ask myself was “Why on earth would I move from San Francisco, lea
ving behind a cushy life for London, and work for a reference management start-up?” Surely any rational person would find this a bit odd.
Well, I’m not going to answer by talking about how great the team is or how enthusiastic the founders are about improving research, which is certainly all true. Rather, let’s take a real-world example of how the “tech” behind Mendeley is already making a difference with how we view the impact factors of research.










