Professional Documents
Culture Documents
Purnell Citation-Indices
Purnell Citation-Indices
Purnell Citation-Indices
Philip Purnell
March 2010
PART I
RESEARCH EVALUATION
HOW DO WE EVALUATE RESEARCH?
• Research grants
– Number and value
• Prestigious awards
– Nobel Prizes
• Patents
– Demonstrating innovative research
• Faculty
– Number of post-graduate researchers
• Citation analysis
– Publication and citation counts
– Normalised by benchmarks
• Peer Evaluation
– Expensive, time consuming and subjective
DO WE NEED MORE METRICS?
A SIMPLE COUNT IS GOOD ENOUGH (1962)
CHAMPIONS LEAGUE 2009
METRICS ARE HERE TO STAY !
Barcelona Team Statistics Manchester United
2 Goals 0
1 1st Half Goals 0
7 Shots on Target 2
4 Shots off Target 7
1 Blocked Shots 2
4 Corners 7
7 Fouls 10
1 Offsides 5
1 Yellow Cards 3
0 Red Cards 0
87 Passing Success 82.9
18 Tackles 21
72.2 Tackles Success 85.7
52.7 Possession 47.3
55.3 Territorial Advantage 44.7
5
A BRIEF HISTORY OF THE CITATION INDEX
• Concept first developed by Dr Eugene Garfield
– Science, 1955
• The Science Citation Index (1963)
– SCI print (1960’s)
– On-line with SciSearch in the 1970’s
– CD-ROM in the 1980’s
– Web interface (1997) Web of Science
• Content enhanced:
– Social Sciences Citation Index (SSCI)
– Arts & Humanities Citation Index (AHCI)
• The Citation Index
– Primarily developed for purposes of information retrieval
– Development of electronic media and powerful searching tools have increased
its use and popularity for purposes of Research Evaluation
WEB OF SCIENCE
JOURNAL SELECTION POLICY
20 4% of the journals:
0 • 30% of the publications
0 1000 2000 3000 4000 5000 6000
# of journals • 51% of cited papers
Articles Citations
COMPARE LIKE WITH LIKE
WHAT IS THE VALUE OF A CITATION?
• Why do people cite?
– Pay homage / give credit to pioneer
– Identifying a methodology
– Provide background reading
– Quotations
– Authenticating data, reproducing work etc
– Corrections
– Criticizing/Disclaiming someone's work/opinions
Journal ‘quality’
Web of Science
• Editorial content
– Addition to knowledge in specific subject field
• Diversity
– International, regional influence of authors, editors, advisors
• Citation analysis
– Editors and authors’ prior work
Regional Journal Content Expansion 2007-2009
Contribution of Each Region to Subject Areas
Czech 30 56 26 + 87%
Journals
Indexed
13
GROWTH IN COVERAGE OF CZECH
JOURNALS IN WEB OF SCIENCE
Web of 2005 2010 Growth Variation
Science
Science 23 36 13 + 57%
Citation
Index –
Expanded
Social 4 11* 7 + 175%
Sciences
Citation
Index
16
EVALUATING COUNTRIES
CHANGING NATURE OF CZECH
RESEARCH
Czech researchers
averaged just over
4,000 papers a year in
the 70s – 90s.
Now over 10,000
papers per year
18
CHANGING NATURE OF CZECH RESEARCH
INTERNATIONAL COLLABORATION
19
CHANGING NATURE OF CZECH RESEARCH
INTERNATIONAL COLLABORATION
20
SCIENTIFIC RESEARCH PRODUCTIVITY
IN CENTRAL EUROPE
21
Thomson Reuters InCites
SCIENTIFIC RESEARCH IMPACT
IN CENTRAL EUROPE
22
Thomson Reuters InCites
CZECH RESEARCH
BY OUTPUT AND IMPACT
23
Ranking by percentage share of Thomson Reuters (ISI) indexed papers 2004 - 2008
EVALUATING INSTITUTIONS (1)
EXTERNAL COMPARISONS
EVALUATING INSTITUTIONS
Number of citations
to North American
scientific papers
26
COMPARE YOUR INSTITUTION’S
PERFORMANCE AGAINST GLOBAL PEERS
27
Thomson Reuters InCites
INSTITUTIONAL COMPARISONS
COMPARING IMPACT IN BIOLOGY?
Cambridge
Oxford
Stirling
29
Thomson Reuters InCites
INSTITUTIONAL COMPARISONS
IN SPECIFIC FIELDS
30
Thomson Reuters InCites
INSTITUTIONAL COMPARISONS
BIOLOGY PERFORMANCE
31
Thomson Reuters InCites
EVALUATING INSTITUTIONS (2)
INTERNAL ANALYSIS
WHAT IS OUR RESEARCH OUTPUT?
33
BENCHMARKING YOUR PAPERS
AGAINST GLOBAL AVERAGES
34
WHAT IS OUR RESEARCH OUTPUT?
35
WHICH ARE OUR
CENTERS OF EXCELLENCE?
37
WHICH OF THOSE COLLABORATIONS ARE
THE MOST VALUABLE?
38
INCITES
CITING ARTICLES LISTING
40
Thomson Reuters InCites
EVALUATING INDIVIDUALS
FRANTISEK VYSKOCIL
PROF. AND D.Sc CHARLES UNIV, PRAGUE
• Described: Potassium movement in the brain during migraine
• Member: Physiological Society of Cambridge and London
• Founding Member: Learned Society of the Czech Republic
• Awards:
– Prize of the Czech Academy of Sciences
– Lifelong Contribution Award of the Czech Academy of Sciences
– Purkynje Medal of the Czech Academy of Sciences
42
FRANTISEK VYSKOCIL
WEB OF SCIENCE PUBLICATIONS
247 Publications
43
FRANTISEK VYSKOCIL
GLOBAL INFLUENCE
Citations by country
Citations: 2.375
Cites per paper: 9,73
H-Index = 30
Citations by journal
WHO ARE OUR
MOST PRODUCTIVE AUTHORS?
45
Thomson Reuters InCites
WHO ARE OUR
MOST INFLUENTIAL RESEARCHERS?
46
Thomson Reuters InCites
WHICH AUTHORS HAVE
MOST IMPACT?
47
Thomson Reuters InCites
WHICH AUTHORS’ PAPERS HAVE
PERFORMED BEST IN THEIR FIELD?
48
Thomson Reuters InCites
HOW CAN WE COMPARE RESEARCHERS?
49
Thomson Reuters InCites
EVALUATING JOURNALS
EFFICIENCY
JOURNAL IMPACT FACTOR
2008 Impact
Factor
Citations
Citations in 2008
To items published in 2007 = 78
To items published in 2006 = 100
Sum = 178
178
= 0,784
53
CITATION BEHAVIOUR VARIES
BETWEEN SUBJECT CATEGORIES
JOURNAL 5-YEAR IMPACT FACTOR
CZECH JOURNALS
55
HOW DO CZECH INSTITUTIONS
EVALUATE JOURNALS FOR LIBRARIES?
• Faculty head request
• Publisher packages
• Budget constraints
• Library recommendation
56
IN WHICH JOURNALS DO OUR
BIOLOGISTS PUBLISH?
57
Thomson Reuters InCites
AND THE GLOBAL COMMUNITY
INFLUENCED BY YOUR RESEARCH?
58
Thomson Reuters InCites
SUMMARY:
THE DATA SOURCE
• Thomson Reuters solutions utilize the data from the Web of Science to
provide reliable and consistent evaluation tools
• It is essential to have a consistent, authoritative and clearly
defined body of data to create meaningful and dependable
statistics. This is why we use Web of Science:
– The gold standard citation resource, used by over 4,000 institutions in
more than 90 countries
– A comprehensive resource covering over 11,000 journals and thousands of
conference proceedings in the fields of Science, Social Science, Arts and
Humanities
– Unmatched retrospective depth of citation data (up to 110 years!)
– Selected content using unbiased, time tested and stringent journal
selection process
– Web of Science is also the source of the Impact Factor the most widely
accepted indicator of journal performance
PART II
THE DATA
THE DATA
• Web of Science
– Conference Proceedings Citation Index
– Century of Social Sciences
• Research Analytics
61
THE DATA
• Web of Science
– Conference Proceedings Citation Index
– Century of Social Sciences
• Research Analytics
62
CPCI – ON WEB OF SCIENCE
Search by
- Conference title
- Location
- Date
- Sponsor
63
WEB OF SCIENCE
CONFERENCE PROCEEDINGS CITATION INDEX
• Two Editions:
- Science & Technology
- Social Sciences & Humanities
• Approx 5.5 million records
• Updated Weekly
• Over 385,000 records per year
• More than 120,000 conferences covered in total
• Approximately 12,000 conferences covered annually
• Coverage from 1990, cited references from 1999
• Comprehensive, multidisciplinary and international coverage from
conferences proceedings published in books, journals, reports,
series, and preprints
• Details of each conference are indexed and searchable, data
includes: conference title, sponsor, location, description and date
A combined set of results
from both Conference
Proceedings and Journal
literature.
Details of both
conference and
source
publication
Classified by
Document Type
• Web of Science
– Conference Proceedings Citation Index
– Century of Social Sciences
• Research Analytics
69
WEB OF SCIENCE
DEPTH OF CITATION INDICES
Thornthwaite
produced 28 highly
influential works in the
Social Sciences which
have been
consistently cited
throughout their
history
THE DATA
• Web of Science
– Conference Proceedings Citation Index
– Century of Social Sciences
• Research Analytics
76
ISI WEB OF KNOWLEDGE 5
webofknowledge.com
77
ISI WEB OF KNOWLEDGE 5
CITATION UNIVERSE
78
ELIMINATION OF STOPWORDS
79
LEMMATIZATION
80
FULL RESULT COUNTS
NO MAXIMUM ON ‘ANALYZE RESULTS’
81
LEFT-HAND TRUNCATION
82
LEFT-HAND TRUNCATION
83
ABSTRACT PREVIEW
84
ABSTRACT PREVIEW
85
CITATION COUNTS USING
WEB OF KNOWLEDGE VALUES
86
WEB OF KNOWLEDGE
CITATION BREAKDOWN TABLE
87
MARKED LIST ENHANCEMENTS
- ANALYZE TOOL & CITATION REPORTS
Analyze tool and citation
report avaiable for specific
products. Future release will
include these features for the
All Databases results
88
CITED REFERENCE SEARCHING
NOW FOR ALL CITATION INDICES
89
CITATION COUNTS IN FULL
90
THE DATA
• Web of Science
– Conference Proceedings Citation Index
– Century of Social Sciences
• Research Analytics
91
BIOSIS CITATION INDEX
BIOSIS CONTENT WITH CITATION DATA
• Biosis Previews
– Widest collection of life sciences fields in the world
– agriculture, biodiversity, biotechnology, botany, drug
discovery, gene therapy, marine biology, wildlife
conservation, zoology
• Biosis Citation Index
– Provides cited references for BIOSIS Previews
• Cited references for all Previews content (beginning 2006)
• Cited references for 60% of high impact Previews journals
(1926 - 2005)
93
BIOSIS CITATION INDEX
SEARCHING
94
BIOSIS CITATION INDEX
REFINE RESULTS
Refine results or exclude
Identify the most abundant results from your dataset
sources of relevant results via a series of parameters
or concentrate on harder-
to-find document types
95
BIOSIS CITATION INDEX
SORTING AND EXPORTING RESULTS
Sort your results to find
breaking new research papers
or to identify the most
influential or relevant papers
Select the records of your in your field
choice and print, email or
export them via your favourite
bibliographic management
system
96
BIOSIS CITATION INDEX
MARKED LIST
97
BIOSIS CITATION INDEX
CITATION BREAKDOWN TABLE
98
BIOSIS CITATION INDEX
CITATION REPORT
99
BIOSIS CITATION INDEX
ANALYZE RESULTS
100
BIOSIS CITATION INDEX
CITATION MAP
Citation maps can find
relevant papers not returned
in your search results
2nd generation
citations reveal
citation pathways
101
BIOSIS CITATION INDEX
CITED REFERENCE SEARCHING
Cited reference
searching helps users
to find ‘missing
citations’
102
BIOSIS CITATION INDEX
SPECIALIZED INDEXING
103
BIOSIS CITATION INDEX
SPECIALIZED INDEXING
• All BIOSIS records are indexed by biology specialists using
the following unique fields
104
BIOSIS CITATION INDEX
SPECIALIZED INDEXING
All the specialized indexing
fields are searchable
105
BIOSIS CITATION INDEX
SUMMARY
• Combines
– World-leading Biosis life sciences content
– with valuable cited references back to 1926
• Enhances
– Web of Science searches via the All Database Search
using common indexing backbone
• Employs
– ISI Web of Knowledge cited reference searching
• Exclusive
– ISI Web of Knowledge
106
THE DATA
• Web of Science
– Conference Proceedings Citation Index
– Century of Social Sciences
• Research Analytics
107
EMERGENCE OF CHINESE RESEARCH
108
CHINESE SCIENCE CITATION DATABASE
AVAILABLE NOW!
• Over 2 million records
• Over 13 million cited references
• Subject Areas
– Mathematics, Physics, Chemistry, Geosciences, Biology,
Agricultural Sciences, Engineering (Computer Science,
Mechanics, etc.), Medicine, and Management
• Dual Language
– Since 2002, 80% of records have full English bibliographic
information and most have English abstracts
Bibliographic
information in
English and
Chinese
111
THE DATA
• Web of Science
– Conference Proceedings Citation Index
– Century of Social Sciences
• Research Analytics
112
AUTHOR NAMES
113
AUTHOR NAMES
114
RESEARCHERID
SCHOLARLY RESEARCH COMMUNITY
• Accurate Identification
• Organize and Manage
• Increase Visibility and Recognition
• Measure Performance
• Collaboration
• Security
See personalized
metrics using Web
of Science citation
data
ResearcherID Profile:
http://www.researcherid.com/rid/A-
9180-2008
RESEARCHER ID
GLOBAL PARTICIPATION
Top Ten Institutions
TopCollege
University Five Institutions
London Top Five Countries
The University of Queensland
Monash University
Harvard University
University of Michigan
University of Pennsylvania
ETH Zurich
University of Cambridge
Stanford University
McGill University
RESEARCHERID
UPLOAD SERVICE