Article
whole variety of knowledge sources and targets. A
An analysis of knowledge
acquisition roles and
participants
William P. Wagner
Department of Management, Villanova
University, Villanova, PA 19085, USA
Clyde W. Holsapple
Decision Science and Information Systems Area,
College of Business and Economics, University of
Kentucky, Lexington,
KY 40506–0034, USA
Abstract: Much recent research effort in the field of
knowledge acquisition (KA) has focussed on extending
knowledge acquisition techniques and processes to
include a wider array of participants and knowledge
sources in a variety of knowledge acquisition scenarios.
As the domain of expert systems applications and
research has expanded, techniques have been developed
to acquire and incorporate knowledge from groups of
experts and from various sources such as text, video, and
audio tapes. However, the dominant participant-role
model remains that of the knowledge engineer eliciting
knowledge from one or more human experts. This
conceptual gap has contributed to the major divisions in
the KA field between researchers interested in manual
KA and those developing tools for automated KA. This
article considers the wide variety of possible KA
scenarios and presents a meta-view of KA participants
and the roles they may assume. We suggest that it is
more appropriate to think of knowledge acquisition
participants as playing one or more roles. These include
knowledge sources, agents and targets for KA processes.
We also present a participant model drawn from
research in decision support systems that more
accurately characterizes the diversity of the entities
participating in the KA process. This view is more
inclusive as it allows us to consider both human-human
and human-computer KA interactions as well as the
Expert Systems, February 1997, Vol. 14, No. 1
careful consideration of the meta-view and its associated
role-participant mappings also yields the new ideas of
the elemental and composite role and the multi-role
entity. These new constructs are then used to identify
areas where research is currently needed and to generate
specific research issues. Taken altogether, this view
allows a more flexible consideration of the many possible
combinations that can and frequently do occur in actual
KA situations.
Keywords: knowledge acquisition, expert systems,
knowledge acquisition roles, knowledge acquisition
participants
1. Introduction
Research in the field of knowledge acquisition (KA) has
expanded in recent years to include techniques and situ-
ations involving multiple experts (Boose, Bradshaw, Kosz-
arek, and Shema, 1993; Grabowski, Massey, and Wallace,
1992; Liou, 1990; Liou, Weber, and Nunamaker, 1990) and
also a wide array of knowledge sources such as text, video
and auditory media (Kaplan and Berry-Rogghe, 1991;
Slator, 1989; Boose, 1989; McGraw and Harbison-Briggs,
1989). Within the KA field, many researchers have been
working at integrating the many different manual and auto-
mated techniques required when a diverse set of knowledge
sources is used in the development of an expert system
(Reinartz and Schmalhofer, 1994; Gil and Paris, 1994;
Sommer, Morik, Andre and Uszynski, 1994; Barbuceanu,
1993; Morik, 1991; Rappaport and Gaines, 1990; Boose,
1989). But as the set of possible knowledge sources and
knowledge representations has expanded our ability to sys-
tematically describe and organize the existing research
within the field has not kept pace. Recent knowledge acqui-
sition research frameworks still focus primarily on the
classic scenario of a single knowledge engineer eliciting
the knowledge of a single human expert though knowledge
acquisition activities include a much broader range of
possibilities (Holsapple and Wagner, 1996a; Dhaliwal and
Benbasat, 1990; Liou, 1990; McGraw and Harbison-Briggs,
1989; Kim and Courtney, 1988). This conceptual gap in the
research serves to exaggerate some of the current divisions
between researchers involved in human-centered or
‘manual’ KA and those focussing solely on automated KA
techniques (Wagner, 1992; Dhaliwal and Benbasat; 1990;
Fellers, 1987).
In order to bridge these apparent divisions, this article
3
proposes a new way of viewing participants in KA pro- Table 1: Types of knowledge extraction methods
cesses that allows for the wide variety of possible sources (Hoffman, 1987)
and scenarios. We suggest that it is more appropriate to
view participants as playing one or more roles in the con- METHOD CATEGORY DESCRIPTION
text of one or more KA processes (Holsapple and Wagner,
1996a). To aid research in this area it is suggested that the METHOD OF Analysis of the tasks that
possible roles include the knowledge source, agent or tar- ‘FAMILIAR’ TASKS the expert usually
get. We also allow that participants can fill one or more performs.
roles and that a role can be filled by one or more parti- STRUCTURED AND The expert is queried with
cipants. The combination of this conception of KA parti- UNSTRUCTURED regard to knowledge of
cipants and their roles leads us to propose role-participant INTERVIEWS facts and procedures.
and participant-role mappings that capture the complex
LIMITED A familiar task is
many-to-many KA scenario where a participant plays sev-
INFORMATION TASKS performed, but the expert
eral roles and roles are filled by multiple participants. This
is not given certain
simple yet powerful view allows us to consider the many
information that is
possible combinations of participants and roles that actually
typically available.
do occur within the context of KA processes.
In the following section, we suggest criteria for evaluat- CONSTRAINED A familiar task is
ing the value of a meta-view and how this should guide PROCESSING TASKS performed but the expert
the construction of this one in particular. Then we present must do so under time or
the meta-view itself; breaking it down into separate sections other constraints.
on knowledge source, target, and agent types of roles along METHOD OF ‘TOUGH Analysis of a familiar task
with a consideration of the resulting role-participant map- CASES’ that is conducted for a set
pings. Finally we show how this taxonomy can be applied of data that presents a
to organizing current research in the field and generating ‘tough case’ for the expert.
many new and interesting research issues.

structed from the perspective of a single, human knowledge

1. 1. The value of classification schemata for knowledge
acquisition research
Knowledge acquisition research has produced a variety of
taxonomies, frameworks, and classification schemata
(Holsapple and Wagner, 1996b; Dhaliwal and Benbasat,
1990; Boose, 1989; Hoffman, 1987; Clancy, 1986). There
are classifications of knowledge types, problem domains,
task types, and KA tools and methods. Classification
schemata and taxonomies play fundamental roles in the
development of a discipline, as a means for organizing
phenomena into classes that are amenable to systematic
investigation and theory development. They offer perspec-
tives on how to distinguish among elements of a set. Such
distinctions become the basis for stating hypotheses and
designing empirical studies. They can guide more special-
ized and focused research endeavors. They also can serve
as valuable pedagogical devices.
For example, Hoffman (1987) proposed a taxonomy of
knowledge extraction techniques. This is summarized in
Table 1.
Hoffman goes on to suggest a number of criteria that
might be used to evaluate and compare each of these
knowledge extraction methods. These criteria include the
time the task takes, its simplicity, its flexibility, its arti-
ficiality, the format and validity of the resulting data, and
the efficiency of the method itself.
What is interesting to note is that this taxonomy is con-
source. The operant characteristic of the taxonomy is the
expert’s familiarity with the task that is to serve as the
vehicle for knowledge elicitation. So, it may only describe
a portion of the knowledge extraction phenomenon, which
is in turn, only a portion of the knowledge acquisition prob-
lem itself. However, with respect to usefulness, Hoffman’s
taxonomy is an important contribution. He was one of the
first researchers to recognize the need for conducting
empirical work that would directly compare knowledge
acquisition techniques. Approaching the problem from the
perspective of experimental psychology, he attempted to
develop a general taxonomy of knowledge extraction
methods and the associated criteria needed to evaluate
them. As such, his work represents a valuable contribution
to knowledge acquisition research.
Another very different framework illustrates how
researchers might approach the wide array of KA scenarios.
Kim and Courtney (1988) attempted to support the KA pro-
cess by making conceptual mappings between different
types of managerial problems and the different available
KA techniques (Fig. 1). Their approach is based on classi-
fying problems by their size, complexity, and degree of
structure. They proceed to advocate the use of certain KA
methods for these classes. In order to make a conceptual
mapping between KA methods and the different types of
managerial decisions, they argue that each type of decision
involves some combination of the three knowledge types.

4 Expert Systems, February 1997, Vol. 14, No. 1

Figure 1: A conceptual mapping of KA methodologies with decision types (Kim & Courtney, 1988).
For instance, because operating decisions are characterized
as involving heuristic and conceptual knowledge, they
deduce that a knowledge engineer-driven method such as
protocol analysis would be the most appropriate.
Their approach, which has been described as a ‘selection-
oriented’ framework (Holsapple and Wagner, 1996b) is
quite different from other development-oriented frame-
works, with its greater consideration of the knowledge
acquisition context. Here, the problem domain is seen as a
crucial part of the KA context, though it has been delimited
to include only managerial decision making. Unlike
Hoffman’s view, the allowable participants in any occur-
rence of knowledge acquisition are implied by the KA tech-
niques included in the mapping to be only single human
sources, a knowledge engineer, and a computer as a recep-
tacle for the knowledge base. However, Kim and Courtney
do not justify their framework’s specific mappings of
knowledge types and problem attributes to knowledge
acquisition methods, either deductively or with empirical
evidence. Moreover, problem attributes such as size and
complexity are not operationalized in their work. As such,
the framework focuses on giving advice about what kind of
KA technique should be adopted, given a problem domain.
1. 2. Criteria for evaluating KA classification schemata
Regardless of the procedure used for inventing a classi-
fication scheme, there are common criteria for evaluating
the resulting schemata. Yet within the KA literature itself,
Expert Systems, February 1997, Vol. 14, No. 1
there has been very little discussion of how alternative
knowledge acquisition taxonomies may be evaluated. The
best-known experimental research on comparative KA
methods is that of Burton, Shadbolt, Hedgecock, and Rugg
(1987). However, their research was limited to the ‘tra-
ditional’ KA techniques of interviewing, card sorting and
goal decomposition. The few pioneering studies are typified
by confusing terminology, conflicting operationalizations,
and the proliferation of ad hoc taxonomies. Each piece of
research allows for different knowledge types, techniques
and even participants. For this reason numerous KA frame-
works and taxonomies have been presented and used but it
is very difficult to evaluate the contributions of each. To
that end this section proposes five different criteria to use
in evaluating knowledge acquisition taxonomies. These five
criteria are summarized in Table 2.
The first criterion requires that the phenomenon being
Table 2: Criteria for Evaluating KA Taxonomies
1. Is the phenomenon that is to be classified
accurately described by the taxonomy?
2. Are the properties or characteristics on which the
taxonomy is based accurately specified?
3. Are the categories mutually exclusive?
4. Are the categories collectively exhaustive?
5. Is the taxonomy useful?
5
classified be carefully defined, so that the taxonomy accu-
rately describes it. It may be that a particular knowledge
acquisition taxonomy reflects only a part of the phenom-
enon and could be identified in a more appropriate manner.
In the case of Hoffman’s taxonomy above, it only refers to
interactions between single knowledge engineers and lone
experts. The second criterion asks whether properties that
have been chosen to do the classifying have been
adequately specified. In the case of the same taxonomy
above, the characteristics may be inferred by examining the
description of each category. Hoffman’s category descrip-
tions are not inconsistent with each other, but some proper-
ties, such as information content and the time allowed to
perform each method, are only referred to in two categories
of methods. Otherwise, we would expect that what is called
the ‘method of familiar tasks’ would include categories
where task time is limited or unlimited, and where the given
information is limited and unlimited. This category would
also suggest the existence of a set of methods using tasks
with varying degrees of familiarity with respect to the
expert, as Hoffman does describe.
The third criterion suggests that all the categories at the
same level should be mutually exclusive. That is, if one
item fits a particular category, it will not fit another. In the
case of Hoffman’s classification of knowledge extraction
methods, some of the categories are not entirely mutually
exclusive. This is because the degree of knowledge engin-
eer involvement in a particular knowledge extraction
method is not included as a characteristic of all categories.
So, it may be that the knowledge engineer merely analyzes
a protocol of the expert’s performance under information
and/or time limits. Or, it may be the case that the knowl-
edge engineer is allowed to interact with the expert to some
degree in each of these situations. Either way, it is not com-
pletely clear from the descriptions given.
The fourth criterion holds that taxonomies should strive
to be collectively exhaustive. That is, every example of the
phenomenon that is to be classified can be placed within
one of the categories. If we equate Hoffman’s knowledge
extraction with the more common notion of knowledge
elicitation, it may be more appropriate to think of his tax-
onomy as describing a range of tasks that a knowledge
elicitation method can use to focus an expert’s attention.
This taxonomy does not include other types of knowledge
sources such as text, data, computer systems, all of which
can require different knowledge extraction or elicitation
methods.
The last criterion simply asks ‘Is the taxonomy useful?’
Does it adequately serve its intended purpose? How does
it compare with other alternative taxonomies? All classi-
fications are designed to help solve some kind of problem.
It is therefore very important to keep in mind the purpose
of the particular taxonomy being evaluated. As Harvey
(1969) states:
6 Expert Systems, February 1997, Vol. 14, No. 1
‘Classification may be regarded as a means for searching
reality for hypotheses or for structuring reality to test
hypotheses. It may also be regarded as a beginning point
or the culmination of scientific investigation. We pos-
sess, therefore, no means of assessing the adequacy or
efficiency of a given classification independently of the
job it is designed to do.’
In this respect, Hoffman’s taxonomy certainly was
important and useful because it provided a basis for per-
forming better empirical work in the KA field.
2. Knowledge acquisition participants
Perhaps the most interesting and complex variable in KA is
the set of participants in the knowledge acquisition project.
These vary in terms of type, number, abilities, and roles
played. As mentioned in section 1, knowledge sources can
include data, texts, and also individual or groups of human
experts. In the literature, techniques as diverse as machine
learning, text understanding systems, and the more tra-
ditional set of human-human knowledge engineering tech-
niques all rightfully claim to be involved in knowledge
acquisition processes. This diversity of knowledge sources
and KA techniques is summarized in Figure 2.
Despite this wide variation in what is described as KA
research, a review of the KA literature reveals that past
knowledge acquisition models and frameworks have
touched on only a limited view of the KA process and of
the allowable project participants (Holsapple, Raj, and
Wagner, 1993; Dhaliwal and Benbasat, 1990, Adelman,
1989; Boose, 1989). Keeping the proposed taxonomy cri-
teria in mind, we introduce a more complete characteriz-
ation, which is crucial to an understanding of the KA
phenomenon.
In order to get beyond the major division within the KA
field between those doing ‘manual’ or ‘traditional’ KA and
those involved in ‘automated’ KA we need to go beyond
the idea that KA participants must be considered to be only
humans. Past work has postulated that a participant set in
an expert system development project can be understood
by viewing them as entities playing roles as determined
by a schema (Holsapple and Wagner, 1995). Certainly,
there can be other entities and roles in a development pro-
ject beyond those needed for KA. Here, our focus is on the
various possible participants in the KA portion of the
project.
2. 1. KA participants as entities
We consider the knowledge acquisition process in terms of
potential participants and the roles they play, rather than
limiting our framework to simply explaining the ‘tra-
ditional’ knowledge acquisition problem of mapping the
knowledge from a human source via a human knowledge
Figure 2: Summary of KA sources and techniques (adapted from Selbig, 1986).

engineer to a computer where the expertise is to reside. The

expertise needed to complete an expert system project may
not exist in a single human expert. As mentioned earlier,
it may reside in multiple humans, text, visual media, audio
form, or even another computer. Similar diversity may exist
at the target side of KA. Moreover, we allow that single or
multiple entities can mediate a KA process. In our view,
an entity is a general description of a KA participant. In
this sense then an entity can be a human, computer, or a
proxy for either, and can be either an individual or aggre-
gate as will be discussed later.
The entities involved in a particular KA process are
determined by the project. Drawing from research in
decision support systems (DSS) we suggest that the generic
Figure 3: Generic decision support framework (Bonczek,
DSS framework could be one way of characterizing entities
Holsapple, and Whinston, 1981).
involved in KA. This framework posits that a DSS may
consist of a language system (LS), presentation system
(PS), problem-processing system (PPS), and a knowledge both computerized decision support and human-centered
system (KS) as shown in Figure 3 (Dos Santos and Holsap- decision support as well (Holsapple and Whinston, 1996;
ple, 1989; Bonczek, Holsapple, and Whinston, 1981). Holsapple and Wagner, 1995; Dos Santos and Holsapple,
The LS represents the set of all messages that the entity 1989).
can accept, while the complementary PS represents the set Although each entity that participates in a knowledge
of all the messages that the entity can issue. The entity’s acquisition process may be characterized with this model,
knowledge is stored in the KS. The PPS of the entity con- obviously, not all entities are alike. For example, a human
sists of its cognitive (i.e. knowledge/message processing) expert can be thought of as using his/her PPS to retrieve
abilities. It draws on the KS to process incoming messages, or even derive new knowledge from his/her KS after receiv-
produce outgoing messages, and solve problems. This ing a message from another entity via the LS and then
model has been applied in the DSS literature to describe transmitting the appropriate knowledge via its PS. Simi-

Expert Systems, February 1997, Vol. 14, No. 1 7

larly, we can use the same model to describe the possibility
of an existing expert system acting as a knowledge source
for another expert system. This model, in combination with
our conception of the possible participant roles, is used to
describe participants in knowledge acquisition processes in
the following section.
2. 2. Roles
In the literature we also find a general class of KA where
an expert interacts with a computer program that helps to
elicit his/her expertise (Boose, 1989). Here, the expert plays
one role and the computer plays another. In other cases, an
expert simply learns to use an expert system shell and
directly inputs his/her knowledge. Here, the same person
plays two roles. There are still other possibilities, but in
general, an entity participates in a knowledge acquisition
process by playing one or more roles. For our purposes, a
role is defined as:
A function (i.e. set of behaviors) that can be assumed
by an entity in a knowledge acquisition process.
The set of roles involved in KA could be variously speci-
fied by different researchers, depending on the level of
detail desired. The most obvious roles are the traditional
roles of expert(s) and knowledge engineers. For the sake
of completeness and clarity, we postulate the existence of
three distinct roles to which entities involved in KA may be
assigned: knowledge source (S), agent (A), and knowledge
target (T). Simply put, a knowledge source serves as a
repository of expertise and an agent’s role is to transfer
this expertise in a form acceptable to some entity playing
a target role.
As discussed earlier, analysis of our working definition
identifies these three roles. They are also consistent with
the KA literature, although the concept of the agent role is
a more inclusive notion than knowledge engineer and the
notion of a knowledge target role has not been explicitly
identified in prior frameworks. We now define and describe
each of these roles in greater detail, using set theoretic
notation where appropriate.
2. 2. 1. Knowledge sources Aside from individual human
experts, many other kinds of knowledge sources (e.g. com-
puters, data sets, textual passages, multi-participant human
sources, and combinations of these) can form part of the
context for a KA process. To encompass all of these, we
define a knowledge source as follows:
A knowledge source provides a repository of expertise
(i.e. reasoning knowledge) that is operated on in a KA
process to satisfy a project’s expert system develop-
ment objectives.
This general definition of a knowledge source is
important in several ways. First, it does not preclude the
8 Expert Systems, February 1997, Vol. 14, No. 1
possibility of a project assigning any of the various kinds
of entities (e.g. humans, computers, data sets, textual
passages) to the source role. Second, it permits multiple
entities (both individual and aggregate) to play the source
role. Third, it encompasses the notions of both active and
passive sources. How a source role is enacted or manifests
for a particular KA occasion depends on the entity (or
entities) that is (are) assigned to that role. The general
model of an entity can be used to describe and analyze such
possibilities. For instance, when an individual human plays
the knowledge source role, we can study a LS for incoming
messages (e.g. from an agent), a PS for presenting outgoing
messages (e.g. to an agent), a KS serving as a repository
of expertise, and a PPS to extract and derive knowledge
for presentation.
Consider the case where one or more of these parameters
is lacking. For example, a human entity playing the role of
knowledge source may not possess a LS (relative to the
knowledge acquisition agent) or may possess it only to a
degree that does not enable the process to function
properly:
S ← (0, PS, PPS, KS).
This would be the case if the knowledge engineer were to
listen to an expert lecture on the topic of interest, but did
not have the opportunity to interact with the expert. Rela-
tive to the knowledge engineer, this source did not have
the opportunity to process any incoming messages which
might help it clarify its own outgoing messages.
A similar situation would be the use of audio tapes as
sources of domain expertise. However, there would also be
the absence of a KS to draw on and a PPS to process the
KS. That is, enactment of the source role would consist
only of a PS, with null values for other entity parameters:
S ← (0, PS, 0, 0)
Here, the PS functions as the repository of expertise. Data
and text as entities playing the source role also fit this pro-
file. They can be viewed as subsets of some human entity’s
PS. Relative to the agent, all other entity parameters have
null values.
This kind of analysis can be extended to describe human
source limitations with respect to any one of the particular
parameters. For example, a human source’s KS can be
limited with regard to both the type of knowledge it con-
tains and the way in which it is organized. Research along
these lines has been performed in cognitive psychology to
show that there may be a correlation between a person’s
personality and the way in which he/she organizes his/her
knowledge. Changes in the KS of a human may also be
linked to capabilities of both the LS to handle incoming
messages and the PS to present outgoing messages. The
problem of eliciting ‘tacit’ knowledge from a human expert
certainly is linked to such limitations on the KS, LS, and
PS. Expertise may be retrieved or derived from its KS and
expressed as a set of messages via the source’s PS, or it
may be prepackaged in the PS of the entity playing the
source role.
2. 2. 2. Agent The notion of the knowledge acquisition
agent role allows us to readily consider both human and
computer-based entities with the same model. The idea of
modelling both human and computer KA agents has also
been noted in other recent research (Schreiber, Wielinga,
de Hoog, Akkermans, and Van de Velde, 1994). As with
the source role, we view the agent as a role being played by
one or more entities involved in the knowledge acquisition
process. This allows us to characterize an agent as having
a language system (LS), a presentation system (PS), a prob-
lem processing system (PPS), and a knowledge system
(KS).
The role of a KA agent is to transfer/transform knowl-
edge from source entities to target entities to meet the
project’s objectives.
The basic idea of the agent role is that one or more
entities act to facilitate the mapping of knowledge. This
characterization of the agent role suggests that (unlike a
source role) it cannot be passive. That is, it must possess
a PPS. This role may go so far as entailing the enforcement
of regulations that pertain to the particular knowledge
acquisition process being implemented. For example, one
type of protocol analysis may require that the agent not
interfere with the expert’s problem-solving process or struc-
tured interviewing may require that the agent use a prior
protocol in order to derive a set of in-depth questions for
the source expert.
As stated earlier, a traditional major dividing line in KA
research has separated the knowledge engineering mode of
KA from modes that automate the KA process via com-
puter-based methods. Here, we have a unified way of view-
ing these traditionally compartmentalized KA modes. Also,
unlike past research, we have the notion that it may be
appropriate within the scope of a project to assign multiple
entities to the agent role. These may be a mix of human
and computer-based entities. Each such entity can be an
individual (a person or computer) or an aggregate (e.g. a
team of specialists, a person supported by a computer, etc.).
2. 2. 3. Target Our characterization of participant roles
would not be complete without the idea of a knowledge
target role.
A knowledge target holds a repository of expertise
resulting from a KA process undertaken to satisfy a pro-
ject’s development objectives.
As with the source role, this allows for both active and
passive targets. When the objectives and resources of the
Expert Systems, February 1997, Vol. 14, No. 1
knowledge acquisition project are defined prior to
implementation of the process, a target is defined in which
the acquired knowledge is expected to reside. Thinking of
the target as describing a role filled by an entity, we can
again use the generic entity model to describe the manifes-
tations of a knowledge target. This means a knowledge tar-
get, like the knowledge source and knowledge acquisition
agent, can be described as having a LS, PS, PPS, and a KS.
In the context of typical knowledge acquisition research
this target is usually the KS of an expert system or other
computer-based system. With the notion of a knowledge
target however, we can expand the model to describe other
targets that have not been recognized in the KA literature.
For example, the notion of a knowledge target role allows
us to consider the possibility that a text may be the desired
target of a particular knowledge acquisition process. This
would be the case when a project aims to generate just the
expert system specifications. In this case the target entity
would consist only of a language system with respect to
the KA process itself:
T ← (LS, 0, 0, 0).
The range of representations accepted by the knowledge
target in this case would depend on the nature of the text
desired.
It is important to note that the knowledge target is not to
be confused with the knowledge representation itself. The
knowledge representation is the message output from the
agent, based on its presentation strategy. There is consider-
able research about various knowledge representations and
their relative advantages (McGraw & Harbison-Briggs,
1989), but the KA literature says little about the entity for
which they are targeted and possible ways its capabilities
could affect the KA process and outcome. Our meta-view
also accommodates the possibility of multiple represen-
tations with respect to one or more entities playing the tar-
get role in a KA process.
The idea of a knowledge target is important in order to
completely characterize the knowledge acquisition process.
This is because an active knowledge target will have its
own abilities and attributes, which affect the type of knowl-
edge representations it will accept from the agent. Indeed,
from the view presented here, it is conceivable that the tar-
get may even drive the KA process.
It is also possible that acquired knowledge could be
mapped to a human target. Allowing a human knowledge
target lets the framework deal with situations such as com-
puter-aided instruction, or even technology transfer (in the
micro sense), which may also be characterized as knowl-
edge acquisition phenomena. That is, the view presented
here suggests that it would be possible to have a computer
as a source for one or more human knowledge targets. To
the extent that this has been previously discussed, it has
been characterized as an aspect of organizational learning.
9
2. 3. Mappings between entities and roles
Prior knowledge acquisition frameworks do not acknowl-
edge the possibility of participants in a KA process playing
multiple roles (Holsapple and Wagner, 1996b; Holsapple,
Raj, and Wagner, 1993). In some automated knowledge
acquisition approaches (e.g. some that make use of reper-
tory grids) a computer plays both the role of the agent and
the role of the target (Shaw and Woodward, 1988). In this
scenario, a computer is essentially interviewing the expert
and deriving rules from the expert’s input that would be
suitable for the resulting expert system. Likewise it is not
unusual for a person to function as both source and agent.
In this scenario, the expert self-extracts knowledge with or
without the help of the computer (Bradshaw and Boose,
1989).
Prior frameworks generally do not account for the possi-
bility that multiple entities, each with differing abilities,
may be assigned to a single role. This could occur if several
experts were assigned to the source role or multiple knowl-
edge engineers played the agent role. In practice however,
KA tools such as AQUINAS have been building in features
that allow them to acquire knowledge from multiple experts
(Gaines, 1987) Thus, the framework’s description of poss-
ible KA participant scenarios needs a way of describing
how the entities are related to the various roles and also
how the roles may be related to the entities playing them.
To support these cases we introduce the notion of a
schema:
A project schema describes how an expert system pro-
ject maps entities to the roles they play and how each
role is mapped to the entities that fill it.
This definition of a schema allows an individual entity
or aggregate entity to play more than one type of role, such
as in the case where an entity serving as an agent also
serves as the target. It also allows for the scenario where
each type of role is replicated, being filled by multiple enti-
ties. An example is the fairly common case where multiple
experts are used as knowledge sources. To accommodate
these distinct possibilities within our characterization of
KA participants, we describe two functions; the entity-role
(er) and the role-entity (re) function. The idea of the er
function is that it can be used to describe how entities (Ei)
are mapped to specific roles in the KA process. The con-
verse is true of the re function which maps each role to
the entities that perform that type of role in the KA process.
Taken together, these two functions describe the project
schema. Both are necessary in order to fully account for
the possible entity-role and role-entity variations. In this
way, all possible project scenarios involving single and
aggregate entities playing single and multiple roles are
accounted for in our meta-view.
For example, if the er function is applied to an entity it
will yield at least one role from the set of allowable roles.
10
If the re function is applied to a role, it will yield at least
one of the entities. In general:
er(Ei) ; R, where Ei ∈ E
re(Rj) ; E, where Rj ∈ R
A closer examination of these two functions helps
describe what we term as elemental and composite roles.
If the function, re, is applied to a certain role and yields
only one entity, we have an elemental role. For example,
re (S) = {E2}
indicates that the project has specified that an elemental
knowledge source be used in this KA process. If E2 is an
individual entity, this example describes the traditional case
of a single human expert playing the role of a knowledge
source. If E 2 is an aggregate entity, the example charac-
terizes the situation where a panel of experts function as a
unified knowledge source. The nature of their interactions
is internal to the entity, rather than being imposed by the
KA process. In neither case is there an issue of the KA
process needing to coordinate the use of multiple sources,
because there is only a single source. The same holds for
elemental agent and target roles.
The project schema also allows multiple entities to be
mapped to the same type of role. We refer to this as a
composite role. For instance, if
er(E1) = {S}
er(E2) = {S}
er(E3) = {S}
then the source role is composite. The project constraints
allow that three entities (E1, E2 and E3) play the source type
of role. Each of these entities will be either individuals or
aggregate. The coordination of these three entities must be
addressed (e.g. the sequence in which they are used, the
reconciliation of differences among them). So, if the er
function, when applied to all available E i, results in more
than one entity being mapped to the same role (Rj), then
that role is said to be a composite role. This kind of schema
accommodates a scenario where multiple entities are used
as distinct knowledge sources. Each such entity could be a
human, a computer, a database, an aggregate (e.g. an expert
panel), or even a textbook. In actual practice, it is quite
common for knowledge engineers to refer to texts, proto-
cols, and distinct experts or groups of experts during the
course of a development project as in the classic ExperTax
system developed by Coopers and Lybrand (Shpilberg,
Graham, and Schatz, 1986).
It is also interesting to consider a case where the er func-
tion yields two roles for the same entity. For example,
Expert Systems, February 1997, Vol. 14, No. 1
 er(E1) = {A, T}
reveals that one entity is playing the A role and the T role.
A review of the knowledge acquisition literature indicates
that this characterizes some of the automated KA scenarios
mentioned earlier (Boose, 1989; McGraw and Harbison-
Briggs, 1989). Within the view presented here, we refer to
E1 as a multi-role entity because we have an entity playing
more than one role. Generally, if when applied to an entity,
Ei, the er function yields more than one role, then that
entity is said to be a multi-role entity.
The remainder of this article demonstrates that taxo-
nomies can be generated from the meta-view framework of
knowledge acquisition roles and participants. These taxo-
nomies are created via the logical partitioning approach
with the criteria of Table 2 in mind. These taxonomies are
meant to suggest alternative ways of characterizing con-
structs identified by the framework, as a basis for organiz-
ing and stimulating future KA research. Our intent is to
illustrate rather than be exhaustive or prescriptive.
3. Deriving knowledge acquisition participant
taxonomies
One way of showing the proposed framework’s utility is
to use it to generate KA taxonomies that can guide empiri-
cal and developmental research. This section introduces
several new taxonomies based on the concepts introduced
to describe KA participants in the previous section.
Knowledge acquisition roles have been characterized
here as being composite and elemental, while entities were
classified as being either individual or aggregate. When
combined, these two ideas define a multi-level taxonomy
of entities and roles. Figure 4 summarizes four entity categ-
ories that exist for any of the defined roles (Rj): (1) an
individual entity that alone fills the role, (2) an aggregate
Figure 4: Sample KA role and entity taxonomy.
Expert Systems, February 1997, Vol. 14, No. 1
entity that alone fills the role, (3) an individual that is one
of multiple entities filling a type of role, and (4) an aggre-
gate entity that is one of multiple entities assigned to a
role type.
This way of viewing project participants is interesting
when we begin to consider how it may impact the knowl-
edge acquisition process at any level. For example, if we
look at the potential variations in a source-agent interaction,
using Figure 4 as a starting point, we obtain the scenario
matrix shown in Table 3.
As a descriptor of the potential scenarios involved in a
source-agent interaction, the matrix captures all of the the-
oretic possibilities suggested by the framework. For
instance, cell 1:2 denotes* an interaction where an individ-
ual elemental source interacts with an aggregate elemental
agent (e.g. an individual human expert with a knowledge
engineer group). Cell 2:4 denotes the case where an aggre-
gate elemental source (e.g. an expert panel) interacts with
multiple aggregate agents (e.g. several knowledge engineer
groups). It is interesting that prior research has focussed
almost exclusively on the scenarios described by cells 1:1
and 3:1. We contend that the other cells deserve to be inves-
tigated with respect to such issues as appropriate KA
methods, tool support possibilities, and their advantages/
disadvantages relative to each other.
A similar KA scenario matrix is shown in Table 4. Here,
the classification is with respect to the agent-target interac-
tions rather than source-agent relationships. Only cell 1:1
has been examined extensively in the KA research literature
(Holsapple and Wagner, 1995; Wagner, 1992).
A special case of this taxonomy is shown in Table 5
where the source and agent are the same (S = A). Included
are scenarios where the S learns to interact with the T
directly and is able to self-elicit his/her knowledge
Table 3: Source-agent interaction matrix
*By convention, the first number of a cell identifier refers to the
rows of the matrix and the second number refers to the columns
of the matrix.
11
 Table 4: Agent-target interaction matrix
Table 5: Source = agent-target interaction matrix
(Shephard, 1993; Boose, 1989). Also included are cases
where expert system shells have sufficiently accessible
interfaces to enable the source to represent his/her knowl-
edge in a way that is understandable to the target computer.
These types of interactions are described by cell 1:1. The
matrix suggests that many other types of interactions can
occur and may be of interest to investigate.
Table 6 shows the taxonomy that results where the agent
and target are the same (A = T). A knowledge acquisition
project characterized by S:A = T interactions encompasses
attempts to automate the knowledge acquisition process in
various ways. One method is to have a source expert per-
form a simulation of the problem on a computer, which
automatically transforms the expert’s performance into
appropriate knowledge representations. Another interaction
of this type occurs when a text understanding system or
other machine learning system induces a rule set or other
suitable knowledge representation from a text or data set.
Both of these examples fall into the category of interaction
described by cell 1:1. Other categories have not been
reported on by KA researchers.
We can use the five criteria in Table 2 to critique this set
12
Table 6: S-Agent = target interaction matrix
of taxonomies. Each contains categories that are mutually
exclusive and collectively exhaustive with respect to the
interaction with which they are concerned. The properties
of entities and roles also accurately describe the possible
combinations of interactions in a way not covered by prior
frameworks. By showing combinations of interactions that
have yet to be studied or suggested, the criterion of useful-
ness is also met.
3. 1. Research issues generated by the analysis
The purpose of this section is to demonstrate the usefulness
of the meta-view framework as a tool for stimulating our
thinking as KA researchers, developers of knowledge
acquisition techniques, and builders of expert systems.
These issues are not explored further in the context of the
present work but are posed as questions that emerge from a
consideration of the framework. This list is not necessarily
exhaustive, but is included to demonstrate the utility of the
framework as an aid to KA research.
As previously mentioned, project participants have been
considered in limited ways by KA researchers. Interest in
these has primarily centered on the attributes of human
knowledge sources and how they may affect the knowledge
acquisition process. More issues emerge from the para-
meterization of the participants as introduced in this
research.
I1. How are aggregate S’s, A’s, and T’s integrated
into the process?
I2. How are composite and elemental S’s, A’s, and
T’s integrated into the process?
I3. How is KA affected if the aggregate S’s, A’s, or
T’s are distributed either spatially or temporally?
I4. What happens to KA if only certain elements of
a participant or group of participants (e.g. LS, PS,
PPS, or KS) are distributed?
Expert Systems, February 1997, Vol. 14, No. 1
I5. What if the sources are not just humans but a mix
of text, data, computers, and humans?
I6. Can a computer serve as a S? What kind of special
KA support might this require?
I7. How can human and computer A’s work together?
I8. How would multiple human A’s work together?
What support exists for this?
I9. Can a target be other than a computer? Can it be
a human, text, or even a mixed group?
I10. How can the target interact with the A or the S to
improve the efficiency of KA? How do T abilities
affect KA?
I11. What happens when individual entities play mul-
tiple roles? Can S = A and A = T? Can the S ever
equal the T?
I12. Can aggregate entities play multiple roles? Does
this situation ever occur?
I13. How can additional entity attributes be expressed?
Can they be adequately discussed in terms of LS,
PS, PPS, and KS abilities?
4. Conclusion
The set of role and participant taxonomies presented here
represents an innovative way of viewing the many vari-
ations in KA participants and can help KA researchers
structure further research. Here, we have introduced taxo-
nomies derived directly from the meta-view presented.
They have not previously appeared in the KA literature.
The taxonomies introduced here are not meant to be
exhaustive with respect to knowledge acquisition pheno-
mena or to the meta-view itself, but simply illustrate how
it can be useful in deriving taxonomies and research
issues via logical partitioning. Moreover, the taxonomies
generated appear to be useful in their own rights, for
helping organize comparative knowledge acquisition
experiments and suggesting KA research topics that are
as yet uninvestigated.
References
Adelman, L. (1989) ‘Measurement Issues in Knowledge Engin-
eering.’ IEEE Transactions on Systems, Man and Cybernetics.
(19), 483–488.
Barbuceanu, M. (1993). ‘Models: Toward Integrated Knowl-
edge Modeling Environments.’ Knowledge Acquisition. Vol. 5,
no. 3.
Bonczek, R., C. Holsapple, and A. Whinston (1981) Foun-
dations of Decision Support Systems. Orlando, FL: Academic
Press Inc.
Boose, J. (1989). ‘A Survey of Knowledge Acquisition Tech-
niques and Tools.’ Knowledge Acquisition. Vol. 1 no. 1.
Boose, J., J. Bradshaw, J. Koszarek, and D. Shema (1993).
‘Knowledge Acquisition for Group Decision Support.’ Knowl-
edge Acquisition. Vol. 5, no. 4.
Bradshaw, J. and J. Boose (1989). ‘Decision Analytic Tech-
niques for Knowledge Acquistion: Combining Situation and
Expert Systems, February 1997, Vol. 14, No. 1
Preference Models Using Aquinas.’ International Journal of
Man-Machine Studies. no. 31.
Burton, A.M., N.R. Shadbolt, A. Hedgecock and G. Rugg
(1987). ‘A Formal Evaluation of Knowledge Elicitation Tech-
niques for Expert Systems.’ Proceedings of the 1st European
Workshop on Knowledge Acquisition for Knowledge-Based
Systems, Reading University, September, 1987.
Clancy, W.J. (1986). ‘Heuristic Classification.’ in J. Kowalik,
Ed. Knowledge-Based Problem-Solving. New York: Prentice-
Hall.
Dhaliwal, J.S. and I. Benbasat (1990). ‘A Framework for the
Comparative Evaluation of Knowledge Acquisition Tools and
Techniques.’ Knowledge Acquisition. Vol. 2, no. 2.
Dos Santos, B. and C. Holsapple (1989). ‘A Framework for
Designing Adaptive DSS Interfaces.’ Decision Support Sys-
tems. Vol. 5, no. 1.
Fellers, J.W. (1987). ‘Key Factors in Knowledge Acquisition.’
Computer Personnel. Vol. 11, no. 1.
Gaines, B. (1987). ‘An Overview of Knowledge Acquisition and
Transfer.’ International Journal of Man-Machine Studies. no.
26.
Gil, Y. and C. Paris (1994). ‘Towards Method-Independent
Knowledge Acquisition.’ Knowledge Acquisition. Vol. 6, no. 2.
Grabowski, M., A. Massey and W. Wallace (1992). ‘Focus
Groups as a Group Knowledge Acquisition Technique.’ Knowl-
edge Acquisition. Vol. 4, no. 4.
Harvey, D. (1969). Explanation in Geography. New York: St
Martins Press.
Hoffman, R. (1987). ‘The Problem of Extracting the Knowledge
of Experts From the Perspective of Experimental Psychology.’
AI Magazine. Vol. 8, no. 2.
Holsapple, C., V. Raj and W. Wagner (1993). ‘Knowledge
Acquisition: Recent Theoretic and Empirical Developments.’ in
Recent Developments in Decision Support Systems (C. Holsap-
ple and A. Whinston). Berlin: Springer Verlag.
Holsapple, C. and W. Wagner (1995). ‘Contextual Factors in
Knowledge Acquisition.’ International Journal of Expert Sys-
tems: Research and Applications. Vol 8, no. 4.
Holsapple, C. and W. Wagner (1996a). ‘Process Factors in
Knowledge Acquisition.’ Expert Systems. Vol. 13, no. 1.
Holsapple, C. and W. Wagner (1996b). ‘Knowledge Acqui-
sition Frameworks: A Survey, Assessment, and Implications.’
forthcoming in Journal of Computer Information Systems.
fall/winter.
Holsapple, C. and A. Whinston (1996). Decision Support Sys-
tems: A Knowledge-Based Approach. St. Paul, MN: West Pub-
lishing Company.
Kaplan, R. and G. Berry-Rogghe (1991). ‘Knowledge-Based
Acquisition of Causal Relationships in Text.’ Knowledge
Acquisition. Vol. 3, no. 3.
Kim, J. and J. Courtney (1988). ‘A Survey of Knowledge Acqui-
sition Techniques and Their Relevance to Managerial Problem
Domains.’ Decision Support Systems. Vol. 4, no. 3.
Liou, Y. (1990). ‘Knowledge Acquisition: Issues, Techniques,
and Methodology.’ Proceedings of the 1990 ACM SIGBDP
Conference on Trends and Directions in Expert Systems.
Orlando, FL.
Liou, Y., E. Weber and J. Nunamaker (1990). ‘A Methodology
for Knowledge Acquisition in a Group Decision Support Sys-
tem Environment.’ Knowledge Acquisition. Vol. 2, no. 2.
McGraw, K. L. and K. Harbison-Briggs (1989). Knowledge
Acquisition: Principles and Guidelines. Englewood Cliffs, NJ:
Prentice-Hall.
Morik, K. (1991). ‘Underlying Assumptions of Knowledge
Acquisition and Machine Learning.’ Knowledge Acquisition.
Vol. 3, no. 2.
13
Rappaport, A. and B. Gaines (1990). ‘Integrated Knowledge
Base Building Environments.’ Knowledge Acquisition. Vol. 2,
no. 1.
Reinartz, T. and F. Schmalhofer (1994). ‘An Integration of
Knowledge Acquisition Techniques and EBL for Real-World
Production Planning.’ Knowledge Acquisition. Vol. 6, no. 2.
Schreiber, G., B. Wielinga, R. de Hoog, H. Akkermans and
W. Van de Velde (1994). ‘CommonKADS: A Comprehensive
Methodology for KBS Development.’ IEEE Expert. Vol. 9,
no. 6.
Selbig, J. (1986). ‘Knowledge Acquisition by Inductive Learning
from Examples.’ in Lecture Notes in Computer Science.
Springer Verlag. no. 265.
Shaw, M. and J. Woodward (1988). ‘Validation in a Knowledge
Support System: Construing Consistency With Multiple
Experts.’ International Journal of Man-Machine Studies. no.
29.
Shephard, G. (1993). ‘Providing Descriptive Power to Guided
Self-Elicitation.’ Knowledge Acquisition. Vol. 5, no. 3.
Slator, B. (1989). ‘Extracting Lexical Knowledge from Diction-
ary Text.’ Knowledge Acquisition. Vol. 1, no. 1.
Sommer, E., K. Morik, J. Andre and M. Uszynski (1994).
‘What Online Machine Learning Can Do for Knowledge Acqui-
sition — A Case Study.’ Knowledge Acquisition. Vol. 6, no. 4.
Shpilberg, D., L.E. Graham and H. Schatz (1986).
‘ExperTAX: An Expert System for Corporate Tax Planning.’
Expert Systems. Vol. 3, no. 3.
Wagner, W. (1992) dissertation entitled ‘A Phenomenon-
Oriented Knowledge Acquisition Framework.’
14
The authors
William P. Wagner
Professor Wagner is an Assistant Professor of Management
in MIS at Villanova University. His teaching and research
interests are in the areas of decision support systems, expert
systems and database management. Much of his recent
work has focused on the knowledge acquisition problem
and has been presented at several professional conferences
such as the Decision Science Institute, ACM SIGBDP,
TIMS/ORSA and the NATO Advanced Studies Institute.
Clyde W. Holsapple
Professor Holsapple holds the Rosenthal Endowed Chair in
MIS at the University of Kentucky. He does teaching and
research in decision support systems, knowledge manage-
ment, expert systems and organisational computing. His
books include Foundations of Decision Support Systems,
Micro Database Management, Business Expert Systems,
The Information Jungle and the just published Decision
Support Systems: A Knowledge-Based Approach. His
research articles span such diverse journals as Decision
Sciences, Operations Research, Decision Support Systems,
Datamation, IEE-SMC, IEE Expert, Information Systems,
Expert Systems, Knowledge Acquisition, The Computer
Journal, Information and Management, Financial Manage-
ment, OMEGA, Journal of Organizational Computing and
Organization Science. Dr. Holsapple is Area Editor for
Decision Support Systems and Associate Editor for Man-
agement Science. In 1993 he was named ‘Computer Edu-
cator of the Year’, by the International Association For
Computer Information Systems.
Expert Systems, February 1997, Vol. 14, No. 1