ITSPOTLIGHT

by DONOVAN GRIFFIN

ImpactStory: Tabulating Tomorrows Research

Priem describes ImpactStory as a way to aggregate all the impact data of all the products that scholars make. Unlike the traditional lters for scholarly impact that marked a successful scholar of yesteryear, ImpactStory rates more than just a scholars articles. at the University of Washington supplemented his research with alt metrics in his tenure packet, moving from assistant to associate professor. But not everyone is thrilled with altmetrics. Priem says theyve encountered difculty in appealing to scholars accustomed to the nearly 40-year-old system of journal-based impact, with reactions ranging from uninterested to outright hostile. One researcher basically said we were opening the gates to the barbarians, says Priem with a laugh. Some people in the scholarly community were afraid that the rise of altmetrics would herald a system where the number of tweets would decide a professors tenure, he says. For Priem, one important aspect of ImpactStory is the long-due recognition of web-native scholarship that is currently being performed outside the traditional reward structure. But for the long term, tools such as ORCID IDs for researchers and semantic data collection will differentiate feedback from a well-respected professor in the relevant eld from that of the researchers own family, which would prevent unscientic citation from deciding the course of scholarly research.

eres a pioneering nonprot organization that harnes ses alternate metrics to measure the impact proles of researchers in the growing elds beyond traditional scholarly citation.

A Little Background
Before the internet, there were few options available to scholars who wanted to assess the impact or relative importance of their published material on an individual basis. Top on the list of options is peer review, which provides the most intimate feedback, though the process can be lengthy. Then there are the more esoteric practices, such as citationcounting measures including the h-index, which are decidedly narrow, or the journal impact factor, which is too all-encompassing. But now that a signicant portion of the scholarly population has put down roots and settled on the internet, the pure volume of data to categorize and rate has increased exponentiallyand so have the options for sorting it. ImpactStorys origin takes place at an event called Beyond Impact 2011 in the U.K., where participants engaged in a workshop/hackathon explored different forms of research impact. Heather Piwowar and Jason Priem, along with a small group, worked through the hackathon and into the night to construct an inchoate ImpactStory, a program that was an attempt to support web-native scholarship. Energized by the project, they continued on it in their spare time. After a few lackluster showings in other contests, the team met Joshua M. Greenberg, program director for the digital information technology program at the Alfred P. Sloan Foundation, at a demonstration for Microsoft academics. Based on the strength of their project, Piwowar and Priem secured a $125,000 grant for ImpactStory from the Sloan Foundation. The team was granted another round of funding (this time for $500,000) from Sloan at the beginning of summer 2013.

We think in the age of the web, increasingly, these sorts of other things, the datasets, the codes, the blog posts, these are rst-class citizens, says Priem. They are different kinds of products and theyre not necessarily appropriate in the same way a paper might be, but sometimes papers are inappropriate, and in the same way data might be preferable.

numbers; research has its own impact flavor, as well. Both members of the team are committed to open data, open metrics, and radical transparency. We think in an age where we CAN make data open, its borderline immoral not to, says Priem. He foresees ImpactStorys value for the future not as a tool on which administrators base hiring or promotion decisions but as a foundation for scholarly tools yet to come that will draw their data from ImpactStorys open source code. We think that by building this open infrastructure, not just of researchers webpages, but of their conversations, of their usage, and of a fabric of their scholarly life, if we can build that graph, that graph of connections between researchers, and then we make that graph open, says Priem. People will be able to build those sort of next generation recommendation tools, next generation assessment tools, next generation search tools, all of that can be built on this conversation graph were building.

Five-Year Plan
Piwowar and Priem plan on hiring some help on a contract basis with the $500,000 grant from Sloan, a step they hope will lead them toward self-sufciency. Priem says that he expects their battle to convince skeptical researchers of the value of altmetrics to web-native scholarship will become easier. Piwowar weighs in on the National Information Standards Organizations (NISO) recent move to study and propose community-based standards in the eld of altmetrics: Standard denitions are a great idea, so we know when we are comparing apples to apples. Furthermore, standards about transparency will be great. But Piwowar is cautious about advancing standardized practices too early. I think we want to be careful so that we dont adopt standards that could calcify innovation: these are early days for altmetrics, she says. Tools that can deliver a customized recommendation feed to a user will become a lot bigger part of what we do in ve years, Priem says.

The Technology
ImpactStory hosts a prole for a researcher at a custom-persistent URL, similar to an online curriculum vitae (CV), which lists the persons published articles, datasets, slides, software, and webpages. But this prole has a decidedly egalitarian approach to the value of a scholars outputs. That researcher inputs his existing products, and ImpactStory searches through a host of APIs across the web, with data from popular sites including Twitter and Wikipedia, as well as specialized sites that appeal to scholars, such as CrossRef, Mendeley, and Delicious. The information collected is fed into the prole, which gives scholars real-time access to the exact impact their research has generated on the web. One particular article may be rated as highly cited because of its number of Scopus citations when compared to items indexed in the Web of Science in 2012 or highly saved due to the number of Mendeley users who added the article to their libraries. ImpactStory is already being used by its target audience: One scholar

Meet the Team

For Piwowar and Priem, the idea for ImpactStory didnt pop up out of thin air. Priem is a Ph.D. student studying information science at the University of North CarolinaChapel Hill. In 2010, he co-wrote an article with three other authors, dening the term altmetrics and laying down a manifesto for its future. Piwowar, co-founder of ImpactStory, studies research data availability and data reuse as a postdoc research associate with Duke University and the University of British Columbia. She says that research should be measured in more than just impact

Donovan Grifn is the assistant editor at Information Today. Send your comments about this article to itletters@infotoday.com.

8 | INFORMATION TODAY | SEPTEMBER 2013

I N FO TO DAY. C O M