Scientometrics

https://doi.org/10.1007/s11192-018-2844-1
(0123456789().,-volV)(0123456789().,-volV)

A comparative study of citations to chemical encyclopedias

in scholarly articles: Kirk-Othmer Encyclopedia of Chemical
Technology and Ullmann’s Encyclopedia of Industrial
Chemistry

Robert Tomaszewski1

Received: 8 January 2018

 Akadémiai Kiadó, Budapest, Hungary 2018

Abstract
Subject encyclopedias have long been regarded as authoritative reference works that
contain credible and reliable information on a subject area or discipline for use by scholars
and researchers. In the digital age, when subject-specific information is readily accessible
online for free or through a subscription-based resource, researchers may no longer rely on
standard reference sources such as subject encyclopedias. To determine whether subject
encyclopedias are used in research, a citation analysis study was conducted using two of
the best-known subject encyclopedias relevant to chemists and chemical engineers: the
Kirk-Othmer Encyclopedia of Chemical Technology and Ullmann’s Encyclopedia of
Industrial Chemistry. Citations to each encyclopedia were obtained from the Science
Citation Index Expanded in Web of Science (WoS) from 2000 to 2015 (inclusive) and
compared by document type. Article citations to these encyclopedias were compared by
year, journal, country of publication, organizational affiliations, and WoS subject category.
Over the sixteen-year period, citations to both encyclopedias have doubled from 943 to
1876 peer-reviewed articles. The study supports the view that such long-standing reference
sources are still needed for research and valued by the scientific community despite the
availability of online resources. The study used VOSviewer to analyze and compare the
key terms in the title and abstract of all articles citing each encyclopedia. This analysis
revealed sub-disciplinary differences in the use of the two encyclopedias.

Keywords Encyclopedias  Authoritative  Chemistry  Citation analysis  Scholarly

articles  VOSviewer

& Robert Tomaszewski

rtomaszewski@fullerton.edu
1
California State University, Fullerton, 800 North State College Blvd, Fullerton, CA 92834-4150,
USA

123
 Scientometrics

Introduction

In the era of the Internet, access to information is quick and easy, often through a click of
the mouse or touchpad, a swipe or touch of a screen, or a spoken word or phrase to a smart
device. Consequently, the question arises whether traditional, long-standing reference
sources like the general or subject encyclopedia are still used by academics. Encyclopedias
are authoritative sources written by scholars and subject specialists and having undergone
extensive evaluation by editors and reviewers. General encyclopedias cover a broad range
of topics for use by the public; subject or specialized encyclopedias focus on a subject-
specific area and are designed to provide information for specialists. Katz (1978, p. 174)
described the general encyclopedia as the ‘‘Ford or Chevrolet’’ and the subject encyclo-
pedia as the ‘‘Rolls Royce’’ of a library collection. East (2010) reviewed the development
of subject encyclopedias in light of the digital age and suggested that libraries need to
promote encyclopedias more effectively by making them available online, discoverable,
and cross-searchable. Moreover, as academic libraries deal with declining budgets and
space issues, they need to continually analyze, evaluate, and provide supportive evidence
that high-cost reference materials, such as standard, authoritative handbooks, and subject
encyclopedias are indeed needed and used by the research community despite the free
availability of online resources.
The advantages of using encyclopedias are their in-depth yet concisely and clearly
written information about topics including concepts and theories; a wide range of termi-
nology for use in searching databases, and the inclusion of a bibliography for further
information. Also, encyclopedias can often provide a quick to use and easy to understand
answers that could otherwise take hours to locate in journals on a specific subject or topic.
Encyclopedias contain context for the research topic and highlight important issues,
illustrations, dates and events, definitions as well as explaining information in a simpler
language than scholarly articles. Moreover, encyclopedias explain topics and concepts in a
more understandable and elementary context to the novice researcher which is occasionally
taken for granted, missed or omitted in articles. According to Flaxbart (2004), ‘‘Broad
subject encyclopedias in the sciences and engineering, such as the venerable Kirk-Oth-
merEncyclopedia of Chemical Technology, continue to prove their worth and authority in
both print and online formats, because of their wide scope, ease of use, and consistent
quality. They are scholarly, yet intellectually accessible to students and the layperson.
Reference librarians know them and refer users to them regularly. Niche encyclopedias,
which seem to be proliferating at an alarming pace, are a different matter. Their potential
audience is much smaller, and their cost–benefit ratios may not stand up to serious
scrutiny.’’
Two of the best-known and pioneering encyclopedias of interest to chemists and
chemical engineers are the Kirk-OthmerEncyclopedia of Chemical Technology and Ull-
mann’s Encyclopedia of Industrial Chemistry. The Kirk-OthmerEncyclopedia of Chemical
Technology was first published in 1947 by Prof. Raymond Eller Kirk and Prof. Donald
Frederick Othmer. It later grew to a 27-volume set. Ullmann’s Encyclopedia of Industrial
Chemistry was first published in 1914 by Prof. Fritz Ullmann and originally written in
German. In 1985 it began publication in English and is now a 40-volume set. The print and
online versions are both available and published by Wiley (Kirk-OthmerEncyclopedia of
Chemical Technology 2017; Ullmann’s Encyclopedia of Industrial Chemistry 2017). The
two works are complementary (Bottle and Rowland 1993; Craig 2006; Maizell 1998;
Wolman 1988).

123
Scientometrics

When referring to the Kirk-OthmerEncyclopedia of Chemical Technology, Grayson

et al. (1979) stated that it is ‘‘designed for the library user who does not know what
questions to ask, or for the specialist from another branch of that science taking a first look
at a neighboring field to uncover general principles, to obtain an overview to see if a
detailed search is needed, or simply to browse for new ideas’’ (p. 117) and went on to say
that ‘‘The Encyclopedia provides the reader the equivalent of consulting an expert in a
given field, someone who can summarize the state of the art, interpret the trends, and tell
where the best sources are for the details that initial inquiry may lead one to require’’ (p.
117). According to Matley (1986), ‘‘…unless there were funds to spare, the library might
not need to include Ullmann and Kirk-Othmer, because they are so similar, although each
offers features, attractions and conveniences not shared by the other’’ (p. 98). Chadwick
(1988) stated that ‘‘Ullmann is superior to Kirk-Othmer in its coverage of international
chemical technology and industrial chemistry’’ (p. 34). Other reviews of the two ency-
clopedias mention that ‘‘Inevitably Ullmann will be compared to the Kirk-Othmer Ency-
clopedia, and since they are both expensive, libraries may only be able to purchase one. For
the process chemist and engineer, Ullmann contains more detail on processes, but less on
marketing, use, economics, and safety than Kirk-Othmer’’ (Laird 1997, p. 391); another
writes, ‘‘The coverage is similar to that of the Kirk-Othmer Encyclopedia of Chemical
Technology. The contributors are primarily European with a limited number of contributors
from Asia and the U.S. A strong industrial influence is maintained through the selection of
contributors with broad industrial experience’’ (Eldridge 1998, p. 10972); a third writes,
‘‘Kirk-Othmer is to the disparate fields of chemical technology what the Encyclopaedia
Britannica is to general knowledge. There is no more authoritative resource available’’
(Flaxbart 1999, p. 2339); a fourth goes on to say, ‘‘…the user-friendly Ullmann’s Ency-
clopedia of Industrial Chemistry remains the single most authoritative, comprehensive,
well-organized, and convenient source book of accurate, reliable information on industrial
chemistry and chemical engineering, reflecting current practice around the world. The
unparalleled international authority, it is an indispensable gold mine of data for students,
individual researchers, patent attorneys, and library collections in industrial chemistry,
chemical engineering, materials chemistry, plastics engineering, biotechnology, food
technology, and environmental science’’ (Kauffman 2000, p. 52); and a fifth writes, ‘‘Kirk-
Othmer Encyclopedia of Chemical Technology (ECT) has retained its unique role as the
‘‘Bible’’ and ‘‘Britannica’’ of chemical technology for almost six decades by providing the
necessary perspective and insight into pertinent aspects, rather than merely presenting
information that can be found elsewhere…’’ (Kaufmann 2007, p. 375) and went on to say
that ‘‘The only possible competitor to the complete ECT 5 is the larger and more expensive
Ullmann’s Encyclopedia of Industrial Chemistry, which began publishing in English with
the fifth edition (1985–1996). Kirk-Othmer is more popular in English-speaking countries,
because it has always been published in English. Both encyclopedias are widely viewed as
having similar coverage and are complementary, with Kirk-Othmer having a more North
American viewpoint and Ullmann having a more European/Japanese viewpoint’’ (Kauf-
mann 2007, p. 377).
Recent developments in bibliometric software tools allow researchers to construct,
analyze, and visualize citation data (Chen and Song 2017; Cobo et al. 2011; Pan et al.
2018; Van Eck and Waltman 2014). Citations to journals, articles, web pages, topic areas,
institutions, and specific research groups have long been used to assess and evaluate their
impact on the literature. Citation analysis can also be a general indicator of the use and
usefulness of a resource. Citation to a source suggests that it likely has been used or
browsed. Further, a researcher’s citation to a source provides evidence in support of its

123
 Scientometrics

credibility. Citations to resources such as Wikipedia, an open-access online encyclopedia to

which anyone can contribute are increasing (Tomaszewski and MacDonald 2016), but
standard authoritative reference sources, such as the CRC Handbook of Chemistry and
Physics and The Merck Index are cited more frequently (Tomaszewski 2017), indicating
their continuing value to scientists.
The Kirk-Othmer Encyclopedia of Chemical Technology (Kirk-Othmer) and Ullmann’s
Encyclopedia of Industrial Chemistry (Ullmann’s) are major subject encyclopedias and
well-regarded reference sources. It would be useful to discover if and how their use has
changed with the increasing availability of potential online competitors. To the author’s
knowledge, no study has examined citations to scientific encyclopedias. A Web of Science
(WoS) topic search using the Boolean operation, (encyclo* AND citation) yielded 49
records, none of which contained scientific encyclopedias. The following study uses
citation analysis to evaluate these two well-known yet similar encyclopedias in the
chemical sciences and answer the following research questions:
1. Are major subject encyclopedias in the chemical sciences used by scientists?
2. Can citation data be used to compare major subject encyclopedias in the chemical
sciences?

Methodology

Clarivate Analytics maintains the Web of Science (WoS) database. The WoS Core Col-
lection is a part of WoS that incorporates three major searchable indexes—the Science
Citation Index Expanded (SCIE), the Social Sciences Citation Index (SSCI), and the Arts
& Humanities Citation Index (AHCI). Journals in these indexes have an acceptance rate of
10-12 percent and are regarded as top tier that go through an evaluation and selection
process before being indexed by WoS (Testa 2016). The WoS Cited Reference Search
mode with the SCIE was used in combination with Boolean operators to extract citations to
each encyclopedia from 2000 to 2015 (inclusive). The Boolean strings used with the ‘‘cited
work’’ field were as follows:
Kirk-Othmer Encyclopedia of Chemical Technology:
ðk  oth  encyÞ OR ðk  othmÞ OR ðkir  encyÞ OR ðoth  encyÞ
OR ðency  chem  techÞ

Ullmann’s Encyclopedia of Industrial Chemistry:

ðull  encyÞ OR ðul  ency  ind  chemÞ OR ðency  ind  chemÞ
NOT ðency  ind  chem  aÞ

The results from the Cited Reference Search were saved in several parts and combined
from the search history. The scholarly journal articles were filtered from other document
types and analyzed using the ‘‘Analyze Results’’ function in WoS. The incident of citation
to each of the encyclopedias was compared by document type and number of article
citations by year, journal, country, organization, and WoS subject category. The articles
were then saved 500 at a time using the tab-delimited format in WoS by choosing the ‘Save
to Other Formats’ option. The individual files were combined and extracted to create co-
word maps using the VOSviewer (Visualization of Similarities) software, version 1.6.8
(Van Eck and Waltman 2010, 2018). A thesaurus file (corpus file) for each encyclopedia

123
Scientometrics

was constructed and loaded into the VOSviewer. This file removed irrelevant terms and
combined spelling variations and synonyms on terms and concepts including abbrevia-
tions, singular and plural terms.

Discussion and findings

Document types and time curves of the citing articles

Both Kirk-Othmer and Ullmann’s Encyclopedias are cited in various document types, such
as articles, review papers, proceedings, and book chapters. However, the majority of
citations are in articles (Table 1). In this sixteen-year period, Kirk-Othmer has been cited
in 11,131 articles, and Ullmann’s in 9,168 articles. The time curves for the number of
article citations are shown in Fig. 1. Citations to Kirk-Othmer throughout the sixteen-year
period shows a constant line graph between 600 and 800 article citations yearly. Citations
to Ullmann’s shows a steep growth rate, from 324 article citations in 2000 to 1,166 article
citations in 2015. Kirk-Othmer has been cited more than Ullmann’s in the period
2000–2012, however, in the last 3 years of the study (2013–2015), article citations to
Ullmann’s have surpassed Kirk-Othmer, presumably in part due to the easier online access
from Wiley publishers. The combined number of citations to these encyclopedias have
doubled between 2000 and 2015 from 943 to 1,876 article citations suggesting that these
resources are continuously used and valued by researchers. Both encyclopedias have been
cited more frequently than Wikipedia in the same periods (Tomaszewski and MacDonald
2016).

Journals of the citing articles

A search in WoS revealed 14,956 journals indexed by SCIE between the years 2000–2015
(inclusive). A total of 1,670 journals have cited Kirk-Othmer (11% of all journals in WoS),
and 1,409 journals have cited Ullmann’s (9% of all journals in WoS). The range of
citations was 1 to 336 journal article citations to Kirk-Othmer, and 1–332 journal article
citations to Ullmann’s. Table 2 lists the top-10 journals with the most article citations
along with their corresponding 2016 impact factors. The most citations to both

Table 1 The total number of

citations to chemical encyclope-
dias by document type (number
of documents | percent),
2000–2015. Data collected, April
2018
Kirk-Othmer
Article (11,131 | 91.0)
Review (1026 | 8.4)
Proceeding Paper (743 | 6.1)
Book Chapter (43 | \ 1)
Editorial Material (36 | \ 1)
Letter (26 | \ 1)
Reprint (7 | \ 1)
Meeting Abstract (4 | \ 1)
Correction (1 | \ 1)
Software Review (1 | \ 1)
Ullmann’s
Article (9,168 | 90.6)
Review (899 | 8.9)
Proceeding Paper (581 | 5.7)
Book Chapter (47 | \ 1)
Editorial Material (37 | \ 1)
Letter (14 | \ 1)
Meeting Abstract (3 | \ 1)
Retracted Publication (3 | \ 1)
Hardware Review (1 | \ 1)
News Item (1 | \ 1)
Reprint (1 | \ 1)

123
 Scientometrics

Fig. 1 The total number of 2000

Combined totals for both encyclopedias
chemical encyclopedia citations 1800 Kirk-Othmer Encyclopedia of Chemical Technology
in scholarly articles by year, Ullmann's Encyclopedia of Industrial Chemistry
2000–2015. Data collected, April 1600

Number of Arcle Citaons

2018 1400

1200

1000

800

600

400

200

0
2000 2002 2004 2006 2008 2010 2012 2014 2016 2018
Publicaon Year

Table 2 The number of article citations to chemical encyclopedias by most frequent journals (number of
articles | 2016 journal impact factor), 2000–2015. Data collected, April 2018
Kirk-Othmer Ullmann’s

Industrial & Engineering Chemistry Research Industrial & Engineering Chemistry Research
(336 | 2.843) (332 | 2.843)
Journal of Applied Polymer Science (261 | 1.860) Applied Catalysis A: General (203 | 4.339)
Applied Catalysis A: General (206 | 4.339) Journal of Molecular Catalysis A: Chemical
(148 | 4.211)
Green Chemistry (143 | 9.125) Green Chemistry (135 | 9.125)
Journal of Molecular Catalysis A: Chemical Catalysis Today (120 | 4.636)
(131 | 4.211)
Journal of Hazardous Materials (106 | 6.065) Chemie Ingenieur Technik (118 | 0.877)
Journal of Catalysis (102 | 6.844) Journal of Catalysis (116 | 6.844)
Catalysis Today (99 | 4.636) RSC Advances (114 | 3.108)
Journal of Chemical and Engineering Data Angewandte Chemie International Edition
(88 | 2.323) (112 | 11.994)
Journal of Membrane Science (88 | 6.035) Chemical Communications (109 | 6.319)

encyclopedias appear in Industrial & Engineering Chemistry Research (IF of 2.843) from
over 300 articles citations. Seven journals have 100 or more citations to Kirk-Othmer,
among them, the Journal of Applied Polymer Science (261 articles), Applied Catalysis A:
General (206 articles), Green Chemistry (143 articles), and Journal of Molecular Catalysis
A: Chemical (131 articles). Eleven journals have 100 or more citations to Ullmann’s,
among them, Applied Catalysis A: General (203 articles), Journal of Molecular Catalysis
A: Chemical (148 articles), Green Chemistry (135 articles), Catalysis Today (120 articles),
and Chemie Ingenieur Technik (118 articles). Both encyclopedias are cited by the same
journals or by journals with similar subject focus. Many journals are high impact journals
and predominately focus on catalysis, polymer science, and engineering.

123
Scientometrics

Countries of publication of the citing articles

The highest number of scholarly article citations to Kirk-Othmer come from institutions in
the United States; those to Ullmann’s are from Germany (Table 3). This may not be too
surprising, given that Kirk-Othmer has a largely North American outlook and background;
Ullmann’s has a more European (and Japanese) outlook and background (Maizell 1998;
Matley 1986; Wolman 1983). Scholars from over 100 different countries worldwide,
including developing countries are citing these chemical encyclopedias.

Organizational affiliations of the citing articles

The highest incidence of citations is from academic institutions worldwide. Comparatively

high numbers of citations come from third-world institutions and minor European schools
which may in part be due to a limited budget for subscription-based resources. The rela-
tively high number of citations from developing countries can also be attributed to higher
reliance on traditional, printed resources (like these encyclopedias) than in the developed
countries where online tools are more readily available. High numbers of citations to Kirk-
Othmer were found from the Indian Institute of Technology, the Chinese Academy of
Sciences, the Islamic Azad University, and the University of Tehran, whereas those to
Ullmann’s came from the Chinese Academy of Sciences, the Swiss Federal Institute of
Technology in Zurich, the Eindhoven University of Technology, and the Technical
University of Munich (Table 4). The cited articles can be further classified by address,
state, and department using a combined search. For example, combining the article set with
state (PS = CA) in the advanced search revealed that both encyclopedias are cited by, and
presumably available at, large research institutions in California: Kirk-Othmer is cited
from the University of California (UC) at Berkeley (39 article citations), Davis (28 article
citations), Los Angeles (26 article citations), and Santa Barbara (19 article citations).
Ullmann’s is highly cited by UC at Berkeley (30 article citations), Davis (20 article
citations), San Diego (15 article citations), Los Angeles (11 article citations), and Santa
Barbara (11 article citations). Refining citations to resources by institution can provide

Table 3 The number of article citations to chemical encyclopedias by most frequent countries (number of
articles | percent), 2000–2015. Data collected, April 2018
Kirk-Othmer Ullmann’s

USA (2,322 | 20.9) Germany (1,699 | 18.5)

India (1,342 | 12.1) USA (1,136 | 12.4)
People’s Republic of China (948 | 8.5) India (776 | 8.5)
Iran (699 | 6.3) People’s Republic of China (760 | 8.3)
Spain (517 | 4.6) Spain (443 | 4.8)
Japan (496 | 4.5) Italy (420 | 4.6)
Germany (454 |4.1) France (395 | 4.3)
Canada (416 | 3.7) Netherlands (358 | 3.9)
England (413 | 3.7) Switzerland (357 | 3.9)
France (400 | 3.6) Japan (347 | 3.8)

123
 Scientometrics

Table 4 The number of article citations to chemical encyclopedias by most frequent institutions (number of
articles), 2000–2015. Data collected, April 2018
Kirk-Othmer Ullmann’s

Indian Institute of Technology (184) Chinese Academy of Sciences (113)

Chinese Academy of Sciences (154)
Islamic Azad University (132)
University of Tehran (99)
National Chemical Laboratory, India (88)
Indian Institute of Chemical Technology (87)
Russian Academy of Sciences (75)
Consejo Superior de Investigaciones Cientificas
(CSIC) (71)
Eindhoven University of Technology (67)
Consiglio Nazionale delle Ricerche (CNR) (62)
University of São Paulo (62)
Swiss Federal Institute of Technology in Zurich (86)
Eindhoven University of Technology (83)
Technical University of Munich (80)
Indian Institute of Technology (78)
Rheinisch-Westfälische Technische Hochschule
Aachen (77)
University of Erlangen-Nuremberg (74)
Delft University of Technology (72)
National Chemical Laboratory, India (66)
University of Porto (61)

supportive feedback on usage to justify subscription renewal, weeding or relocating ref-

erence materials to stacks.

WoS subject categories of the citing articles

Each journal in WoS is assigned one or more subject categories attributed to the source
publication. WoS contains 252 subject categories in science, social sciences, arts and
humanities. Examples of WoS subject categories include Chemistry, Analytical; Chem-
istry, Applied; Chemistry, Organic; Engineering, Chemical; Engineering, Industrial;
Energy & Fuels; etc. (Clarivate Analytics 2017). There are over 100 different WoS subject
categories of articles citing each encyclopedia. Both encyclopedias have the highest
incidence of citations from journals in chemical engineering and physical chemistry.
Table 5 shows the top-10 WoS subject categories of articles citing each encyclopedia.
Kirk-Othmer has more citations from journals dealing with polymer science, and Ull-
mann’s contains slightly more citations from journals dealing with organic chemistry. Both
encyclopedias have citations to a wide range of subject categories in the sciences.

Content analysis of the citing articles

Researchers have used co-word maps (also referred to as term maps) to study scientific
topics. However, few studies have used co-word maps to analyze reference material
citation such as encyclopedias. The proposed hypothesis is that a relationship exists
between the content of research articles that cite a resource and the content of a resource.
Density visualization maps (Fig. 2) and network visualization maps (Fig. 3) from the
VOSviewer illustrate the different co-words from articles citing each of the two ency-
clopedias. The co-word maps contain key terms from the articles’ titles and abstracts
arranged by clusters (represented by color). From each cluster, the size of each circle

123
Scientometrics

Table 5 The number of article citations to chemical encyclopedias by most frequent Web of Science
categories (number of articles | percent), 2000–2015. Data collected, April 2018
Kirk-Othmer Ullmann’s

Engineering, Chemical (2,311 | 20.8) Engineering, Chemical (2,228 | 24.3)

Chemistry, Physical (2,171 | 19.5)
Chemistry, Multidisciplinary (1,765 | 15.9)
Polymer Science (1,232 | 11.1)
Materials Science, Multidisciplinary (921 | 8.3)
Environmental Sciences (917 | 8.2)
Chemistry, Applied (697 | 6.3)
Chemistry, Analytical (687 | 6.2)
Chemistry, Organic (674 | 6.1)
Engineering, Environmental (571 | 5.1)
Chemistry, Physical (1,948 | 21.2)
Chemistry, Multidisciplinary (1,679 | 18.3)
Chemistry, Organic (772 | 8.4)
Materials Science, Multidisciplinary (664 | 7.2)
Environmental Sciences (662 | 7.2)
Polymer Science (622 | 6.8)
Chemistry, Applied (559 | 6.1)
Engineering, Environmental (465 | 5.1)
Energy & Fuels (429 | 4.7)

represents the number of key term occurrences (the higher the number of terms, the bigger
the circle diameter) and the distance between the terms on the map represents the fre-
quency of occurrence (the more occurrences [relatedness] of terms, the closer the circles).
Terms that appear together are thus likely related to the same or similar subject matter and
hence grouped in a cluster.
Kirk-Othmer’s co-word map resulted in six clusters, identified from key terms in 11,131
articles that cite it. These clusters are evident from the density visualization map (Fig. 2A).
A network visualization map (Fig. 3A) identified three major clusters as technology (green
cluster, 368 key terms), polymer (blue cluster, 320 key terms), and catalyst (red cluster,
293 key terms) as well as three smaller clusters identified as phase (yellow cluster, 166 key
terms), adsorption (cyan cluster, 43 key terms), and analytical chemistry (purple cluster, 40
key terms). The larger three clusters of co-words are discrete and unique. The smaller three
clusters blend into the larger clusters. The adsorption cluster blends into the technology
cluster, whereas the phase and analytical chemistry clusters are sandwiched between the
technology and catalyst cluster suggesting a close relationship. The technology cluster
(green) contains co-words associated with biochemical and industrial technology, such as
fatty acid, protein, bacteria, strain, fungi, enzyme, fish, cytotoxicity, fermentation, envi-
ronmental pollution, pilot scale, food industry, agriculture, and pharmaceutical industry.
Experimental characterization techniques are also present in this cluster and evident from
terms that include mass spectrometry, gas chromatography, and high-performance liquid
chromatography. The polymer cluster (blue) outlines areas of polymer science in identi-
fication, characterization, and property information. Co-words from the polymer cluster
include polyurethane, radical polymerization, graft copolymerization, emulsion polymer-
ization, composite, mechanical strength, viscoelastic property, dielectric property, thermal
conductivity, impact strength, tensile test, stress, crystallinity, glass transition temperature,
surface morphology, differential thermogravimetry, and differential scanning calorimetry.
The catalyst cluster (red) contains co-words dealing with chemicals, reaction conditions,
characterization, selectivity, kinetics, and reactions. The phase cluster (yellow) contains
experimental key terms and concepts, such as binary and ternary mixture, Wilson model,
non-random two-liquid model, phase diagram, solubility parameter, distribution coeffi-
cient, and isentropic compressibility. The adsorption cluster (cyan) includes co-words like

123
 Scientometrics

Fig. 2 Density visualization maps of key terms to article titles and abstracts citing Kirk-Othmer
Encyclopedia of Chemical Technology (A) and Ullmann’s Encyclopedia of Industrial Chemistry (B).
Software used was VOSviewer. Data collected, April 2018

123
Scientometrics

Fig. 3 Network visualization maps to key terms to article titles and abstracts citing Kirk-Othmer
Encyclopedia of Chemical Technology (A) and Ullmann’s Encyclopedia of Industrial Chemistry (B).
Software used was VOSviewer. Data collected, April 2018

123
 Scientometrics

Freundlich equation, Langmuir adsorption equation, adsorption isotherm, and sorption

capacity. The analytical chemistry cluster (cyan) contains co-words, such as titration,
potentiometric titration, chemiluminescence, and photoluminescence.
Ullmann’s co-word map resulted in five clusters, identified from key terms in 9,168
articles that cite it. These clusters are visible from the density visualization map (Fig. 2B).
A network visualization map (Fig. 3B) shows five clusters identified as technology (green
cluster, 348 key terms), polymer (blue cluster, 174 key terms), catalyst (red cluster, 165
key terms), catalytic analysis (pink cluster, 165 key terms), and phase (yellow cluster, 80
key terms). The polymer (blue) and technology (green) clusters overlap suggesting a close
relationship. The catalytic analysis cluster (pink) extends into the catalyst cluster (red),
whereas the phase cluster (yellow) penetrates into the technology and catalyst clusters. The
technology cluster (green) has the highest number of co-words with a focus on industrial
chemistry, such as textile industry, food industry, pharmaceutical industry, plant, pilot
plant, membrane reactor, bioreactor, heat exchanger, feedstock, optimization, petro-
chemical, and environmental impact. This cluster also contains adsorption terms like
adsorption capacity, adsorption isotherm, and adsorption process. The polymer cluster
(blue) includes a broader description of polymer terminology and physical property co-
words like copolymer, homopolymer, copolymerization, radical polymerization, initiator,
epoxy resin, silicones, hydrophilicity, grafting, nanocomposite, glass transition tempera-
ture, hardness, molecular weight distribution. This cluster also contains material charac-
terization techniques, such as gel permeation chromatography, differential scanning
calorimetry, electron microscopy, Fourier-transform infrared spectroscopy, and nuclear
magnetic resonance. The catalyst cluster (red) contains co-words dealing with industrial
catalyst terms, such as catalyst precursor, cocatalyst, heterogeneous and homogeneous
catalyst, catalytic cycle, turnover number, pot synthesis as well as organic chemicals and
name reactions. The catalytic analysis cluster (pink) contains information about the cata-
lyst itself like catalytic activity, catalyst support, catalyst surface, catalytic conversion,
catalytic performance, active species, selectivity, and zeolites. This cluster also holds terms
relevant to characterization and spectroscopy, such as X-ray diffraction, electron param-
agnetic resonance, atomic absorption spectroscopy, and ultraviolet–visible spectroscopy.
The phase cluster (yellow) contains terms related to various phases in a chemical system
and includes terms like vapor–liquid equilibrium, ternary mixture, and phase diagram.
The topics of both encyclopedias fall within the clusters’ co-words as represented in
Table 6. There is a relationship between the articles’ co-words to that of the encyclopedias’
content topic titles (Kirk-Othmer Encyclopedia of Chemical Technology, 2017; Ullmann’s
Encyclopedia of Industrial Chemistry, 2017). Further, co-word maps reveal more simi-
larities than differences between the two encyclopedias. Kirk-Othmer, however, is cited
more frequently by articles dealing with polymer property and characterization on a bench
or pilot scale. Kirk-Othmer contains more emphasis on pure chemistry and theoretical
chemical engineering and less on plant design, production processes, and applications in
chemical and allied industries. Ullmann’s citation coverage derives from articles about
applied and industrial chemistry, industrial engineering, and large-scale chemical engi-
neering. Ullmann’s is cited more by articles on plant design and applications and less on
kinetics and theoretical information.
Institutions that deal with a smaller library budget are often required to carefully
determine the best resources needed by researchers in a discipline. Analyzing the content
of articles that cite a resource allows one to deduce which potential purchases are most
used in a specific research area. This may be particularly helpful when user statistics for a
resource are not available or difficult to obtain.

123
Scientometrics

Table 6 Encyclopedias content topics represented by co-word clusters. Data collected, April 2018
Encyclopedia topics Cluster name Example of extracted co-words

Agrochemicals Technology Agriculture, fertilizer, pesticide, soil

Analytical techniques Analytical Atomic absorption spectroscopy, capillary electrophoresis,
Chemistry cyclic voltammetry, elemental analysis, gas
Catalyst chromatography, infrared spectroscopy, ultraviolet-
Catalytic visible spectroscopy, X-ray diffraction
Analysis
Polymer
Technology
Biochemistry and Technology Bacteria, biodiesel, biofuel, enzyme, fermentation, gene,
Biotechnology microorganism, protein
Chemical reactions Catalyst Active species, aldol condensation, alkylation, catalyst,
Catalytic catalytic activity, epoxidation, hydrogenation,
Analysis oligomerization, pot synthesis, selectivity
Dyes and pigments Adsorption Azo dye, crystal violet, methylene blue, pigment
Catalyst
Technology
Energy Technology Biofuel, fossil fuel, fuel cell, solar energy, renewable raw
material
Environmental protection Polymer Environmental assessment, pollution, safety data sheet,
and industrial safety Technology sustainability
Fat, oil, food and feed, Technology Corn, cosmetic, feedstock, food industry, triglyceride
cosmetics
Inorganic chemicals Catalyst Alumina, iron oxide, transition metal, zeolites
Catalytic
Analysis
Materials Polymer Carbon black, cellulose, cement, composite, filler, glass,
polyester, rubber
Metals and alloys Analytical Cadmium, gold, heavy metal, steel
Chemistry
Catalyst
Catalytic
Analysis
Technology
Organic chemicals Catalyst Adipic acid, alcohols, alkane, alkene, amide, amine,
Phase ethylene, nitrobenzene, propylene, styrene
Polymer
Pharmaceuticals Polymer Drug release, medicine, pharmaceutical industry
Technology
Polymers and plastics Polymer Copolymerization, crosslinking, epoxy resin, glass
transition temperature, polyester, polyethylene,
polyurethane, radical polymerization, tensile strength
Processes and process Adsorption Adsorption process, batch reactor, continuous process,
engineering Catalyst distillation column, kinetics, microreactor, phase
Phase diagram, pilot scale, power plant, tank reactor
Polymer
Technology
Renewable resources Polymer Biofuel, biomass, cotton, renewable raw material
Technology

123
 Scientometrics

Limitations of the study and future work

The study was limited to journals in the SCIE of WoS between the selected years 2000 and
2015 (inclusive). Citation errors outside the selected year range in the Cited Reference
Search are not picked up. The study data reflects the use of only these two chemical
encyclopedias through citation in published peer-reviewed research articles. It does not
show use in other instances like unpublished information, class assignments, or as a quick
reference look-up by students, faculty, and librarians. Future work could use the outlined
methodology to analyze citation counts to encyclopedias in other subject-specific disci-
plines as well as online resources like subscription-based and freely available databases.

Conclusion

Within the sixteen-year time-frame of the study, the Kirk-Othmer Encyclopedia of

Chemical Technology has been cited in 11,131 articles and Ullmann’s Encyclopedia of
Industrial Chemistry in 9168. The combined number of citations to these encyclopedias
have doubled between 2000 and 2015, from 943 to 1876. This research reveals that there
are more similarities than differences between the two subject encyclopedias. Citations to
the two encyclopedias are published in articles from scholars and institutions worldwide, in
high impact journals, and particularly in the disciplines of chemistry, engineering, material
science, and polymer science. Authoritative subject encyclopedias, such as the Kirk-
Othmer Encyclopedia of Chemical Technology and Ullmann’s Encyclopedia of Industrial
Chemistry are continued to be cited by scientists in peer-reviewed articles and steadily
increasing in citation counts. This study provides supportive evidence that such long-
standing reference sources are still needed for research and valued by the scientific
community despite the availability of freely accessible online resources. The prominence
of citations from countries outside North America and Europe reflect the reliance on
subject encyclopedias. It would be interesting to see, for example, if these citation practices
continue once developing countries increase their access to online resources. The VOS-
viewer was successfully used to critically assess and compare the nature and content use of
each encyclopedia from the key terms found in the titles and abstracts of articles that cite it.
Such encyclopedias should be promoted through library instruction and online guides as
valuable authoritative tools for finding information.

Acknowledgements I thank the reviewers for their insightful and intelligent suggestions with the
manuscript.

References
Bottle, R. T., & Rowland, J. F. B. (1993). Information sources in chemistry (4th ed.). London: Bowker-Saur.
Chadwick, S. S. (1988). Ullmann’s encyclopedia of industrial chemistry. Reference Services Review, 16(4),
31–34. https://doi.org/10.1108/eb049034.
Chen, C., & Song, M. (2017). Science mapping tools and applications. In Representing scientific knowledge:
The role of uncertainty (pp. 57–137). Cham, Switzerland: Springer.
Clarivate Analytics. (2017). Web of Science Core Collection help. Retrieved from https://images.
webofknowledge.com/images/help/WOS/hp_subject_category_terms_tasca.html. Accessed 26 June
2018.

123
Scientometrics

Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). Science mapping software
tools: Review, analysis, and cooperative study among tools. Journal of the Association for Information
Science and Technology, 62(7), 1382–1402. https://doi.org/10.1002/asi.21525.
Craig, C. (2006). Kirk-Othmer encyclopedia of chemical technology and Ullmann’s encyclopedia of
industrial chemistry. Issues in Science & Technology Librarianship. https://doi.org/10.5062/
F4RF5S0T.
East, J. W. (2010). The rolls royce of the library reference collection: The subject encyclopedia in the age of
Wikipedia. Reference & User Services Quarterly, 50(2), 162–169.
Eldridge, R. B. (1998). Book Reviews [Review of the book Ullmann’s Encyclopedia of Industrial Chem-
istry, 5th Completely Revised Edition, by B. Elvers, S. S. Hawkins & W. Raussey]. Journal of the
American Chemical Society, 120(41), 10792.
Flaxbart, D. (1999). Book Reviews [Review of the book Kirk-Othmer Encyclopedia of Chemical Tech-
nology, Fourth Edition, 27-Volume Set, by Wiley Interscience]. Journal of the American Chemical
Society, 121(10), 2339.
Flaxbart, D. (2004). Death of an encyclopedia salesman? The fate of science reference resources in the
digital age. Issues in Science & Technology Librarianship. https://doi.org/10.5062/F4VQ30M9.
Grayson, M., Eckroth, D., & Kromer, R. (1979). The making of an encyclopedia: The 3rd edition of the
Kirk-Othmer Encyclopedia of Chemical Technology. Journal of Chemical Information and Computer
Sciences, 19(3), 117–122.
Katz, W. A. (1978). Encyclopedias: General and subject. In Introduction to reference work, vol. 1, Basic
information sources (3rd ed.) (pp. 173–174). New York: McGraw-Hill.
Kauffman, G. B. (2000). Media Reviews [Review of the book Ullmann’s Encyclopedia of Industrial
Chemistry, Sixth Edition, by T. Kellersohn, B. Elvers, S. Hawkins, & U. Winter]. The Chemical
Educator, 5(1), 49–52. http://chemeducator.org/bibs/0005001/510049gk.htm.
Kauffman, G. B. (2007). Media Reviews [Review of the book Kirk-Othmer Encyclopedia of Chemical
Technology, Fifth Edition, by A. Seidel]. The Chemical Educator, 12(5), 375–377. http://
chemeducator.org/bibs/0012005/12070374mr.htm.
Kirk-Othmer Encyclopedia of Chemical Technology. (2017). [Online]. Retrieved from http://onlinelibrary.
wiley.com/book/10.1002/0471238961. Accessed 26 June 2018.
Laird, T. (1997). Book Review [Review of the book Ullmann’s Encyclopedia of Industrial Chemistry, 5th
Edition]. Organic Process Research & Development, 1(5), 391–392. https://doi.org/10.1021/
op970020u.
Maizell, R. E. (1998). How to find chemical information: A guide for practicing chemists, educators, and
students. New York: Wiley.
Matley, J. (1986). [Review of the book Ullmann’s Encyclopedia of Industrial Chemistry, 5th Edition, by W.
Gerhartz & Y. S. Yamamoto]. Chemical Engineering, 93(8), 95–98.
Pan, X., Yan, E., Cui, M., & Hua, W. (2018). Examining the usage, citation, and diffusion patterns of
bibliometric mapping software: A comparative study of three tools. Journal of Informetrics, 12(2),
481–493. https://doi.org/10.1016/j.joi.2018.03.005.
Testa, J. (2016). The Web of Science journal selection process. Retrieved from http://wokinfo.com/essays/
journal-selection-process/. Accessed 26 June 2018.
Tomaszewski, R. (2017). Citations to chemical resources in scholarly articles: CRC Handbook of Chemistry
and Physics and The Merck Index. Scientometrics, 112(3), 1865–1879. https://doi.org/10.1007/s11192-
017-2437-4.
Tomaszewski, R., & MacDonald, K. I. (2016). A study of citations to Wikipedia in scholarly publications.
Science & Technology Libraries, 35(3), 246–261. https://doi.org/10.1080/0194262X.2016.1206052.
Ullmann’s Encyclopedia of Industrial Chemistry. (2017). [Online]. Retrieved from http://onlinelibrary.
wiley.com/book/10.1002/14356007. Accessed 26 June 2018.
Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric
mapping. Scientometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-0146-3.
Van Eck, N. J., & Waltman, L. (2014). Visualizing bibliometric networks. In Y. Ding, R. Rousseau, & D.
Wolfram (Eds.), Measuring scholarly impact: Methods and practice (pp. 285–320). Berlin: Springer.
Van Eck, N. J., & Waltman, L. (2018). VOSviewer manual. Version 1.6.8 (27 April). Centre for Science and
Technology Studies. Leiden, NL: Leiden University. Retrieved from http://www.vosviewer.com/
getting-started#VOSviewer%20manual. Accessed 26 June 2018.
Wolman, Y. (1983). Chemical information: A practical guide to utilization. New York: Wiley.
Wolman, Y. (1988). Chemical information: A practical guide to utilization (2nd ed.). New York: Wiley.

123