integrating Nafurwi Language

Processha and hmRormation

Reh.ievaiIm a 7koubieshooting
Help Desk
Peter 6. Anick, Digital Equipment Corporation

ation provides telephone and electronic

support to its customers through a world-
AS THE SIZE AND IMPORTANCE OF DEC'S ON-LINE
wide network of support centers and field DATABASE HAVE GROWN, ITS CUSTOMER SUPPORT
offices. In addition to troubleshooting cus-
SOFTWARE HAS EVOLVED TO HANDLE INCREASED DATA
tomer problems, Digital's service and sup- I

port specialists share their knowledge and
experiences with other specialists by writ-
ing up their problems and solutions and
making them available in a worldwide on-
line information base. By sharing this
knowledge among a large body of physi-
cally distributed support engineers. cus-
tomer problems that have been previously
encountered can be resolved much more
quickly, thereby increasing customer sat-
isfaction and decreasing Digital's cost of
providing service.
Over the past decade, the range of prod-
ucts supported by Digital's specialists has
grown steadily and now includes non-Dig-
ita1 products as well. This growth has been
matched by a massive increase in the size
of the on-line inforniation base, which has
surged from less than 10,000 article5 into
the hundreds of thousands.
As the size and importance of the data-
base has grown, the customer support cen-
ters have improved their software's data
management and information retrieval ca-
pabilities. In the early 1980s. articles were
MANAGEMENT AND INFORMATION RETRIEVAL NEEDS.
NATURAL LANGUAGE PROCESSING PLAYS AN lMl'ORlXhT ROLE,
indexed with a controllcd vocabulary and
retrieved using Boolean expressions over
this vocabulary. By the mid- 1980s. how-
ever, this system was replaced by a full-
text system. Stars, which proved much
more effective. In a full-text system, all the
words in a text are available as keywords
for querying. Users do not have to learn a
controlled vocabulary, and they can ex-
press their queries in natural language rather
than the syntax of Boolean expressions.
Stars did not try to "understand" the
natural language queries, nor did it simply
match the query's tokens with articles con-
taining those strings. Instead. i t truncated
words to remove suffixes. eliminated noise
words like "a" and "of." added synonyms.
and converted the results into Boolean ex-
pressions for matching against articles in
the database. It could also remove terms
from a query to broaden it if the original
search resulted in no hits.
On the whole. this approach to informa-
tion retrieval has worked well for special-
ists hunting down problem-solving infor-
mation. However, the rapid growth in the
size of databases has contributed to an
increase in the number of false hits re-
trieved during searches, as well as an in-
crease in the need to reformulate queries to
guide the search. Particularly frustrating
were cases in which a specialist knew an
article existed but couldn't easily craft a
query that would retrieve it. Problems like
this led Digital's Stars development group
to contact us at the company's AI Technol-
ogy Center to explore the applicability of
AI technologies to this critical component
of customer service and support. Intrigued
by the possibilities for exploiting expert
systems, intelligent user/system dialogues,
blackboard-based integration of knowledge
sources, and natural language processing,
we eagerly accepted the challenge and chris-
tened our collaboration the AI-Stars project.
Interviewing the users
Information retrieval research in the
1980shad already pursued anumber of AI-
influenced approaches. The Rubric sys-
tem, for example, cast queries as rules for
building up evidence about a document’s
relevance.’ 13R used a blackboard archi-
tecture to orchestrate a dialogue with the
user, elaborate the user’s information needs,
and find appropriate documents.* Natural
language processing was used to automate
the role of an expert intermediary in inter-
active document r e t r i e ~ a l Statistical
.~
ods had also emerged as attractive alterna-
tives to Boolean and controlled vocabulary
approaches; for example, Wide Area In-
formation Servers4drew on the vector space
model, in which queries and documents are
represented as weighted term vectors, al-
lowing for the computation of statistical
similarity metrics between them.5 Such
systems work best with long queries, and it
is possible to use the text of one article as
a query to find others similar to it.
One of the conclusions researchers had
come to after exploring this wide range of
approaches was that no one retrieval meth-
od is likely to meet the needs of all users.6
Not only do the methods tend to retrieve
different sets of relevant documents, but
users themselves come with different task
orientations, interaction preferences, skill
levels, and so on. We therefore began our
AI-Stars investigation by interviewing
members of our user population and watch-
ing them as they performed their daily
work of computer troubleshooting.
We had discovered many different styles
of on-line searching. Some specialists liked
to enter a broad (one- or two-word) query
and then, based on the size of the resulting
set, further restrict the search expression.
Others started with longer queries of per-
haps half a dozen terms and removed or
added related terms if the initial query
appeared to miss its mark. Users tended to
understand that their natural language ex-
pression was transformed into a Boolean
expression and, on some occasions, would
resort to using optional explicit Boolean
IO
operators, including negation. The object
of their task, to respond as quickly as pos-
sible to a specific customer inquiry, meant
that they were primarily interested in “pre-
cision” searching: finding the one article
that addressed a particular problem, as
opposed to all the articles that might be
even peripherally related.
For the most part, users reported liking
the Stars system and appeared to be effec-
tive at using it. However, their confidence
was shaken when they had trouble locating
an article that they knew to exist; and they
W E FELT OBLIGED TO
EXPLORE SOLUTIONS THAT
meth- WOULD MINIMIZE THE
USER’S NEED TO DO
ANYTHING MORE THAN TYPE
IN A SIMPLE NATURAL
LANGUAGE QUERY.
had little way of knowing, when a search
failed, whether that meant there was no
article that met their need or whether fur-
ther searching was likely to be fruitful.
Users also reported some trouble coming
to grips with the internal heuristics used by
Stars to convert natural language queries
into search expressions. Indeed, each of
Stars’ internal query-processing strategies
could in some cases lead to poor results and
confuse users:
Stemming sometimes reduced unrelated
words to the same stem.
Some apparent noise words were in fact
content words in certain contexts.
Automatic use of synonyms sometimes
broadened the query in inappropriate
ways.
Users intended certain combinations of
words in their queries to be contiguous
phrases rather than independent search
terms.
The strategy for automatically broaden-
ing a query with no hits did not always
do the “right” thing.
Perhaps reacting to these kinds of expe-
riences, the users we interviewed were
wary of many of the “enhancements” we
initially proposed. Intelligent automated
query reformulation and statistical nearest
neighbor matching methods were perceived
as potentially making the system even more
difficult to understand. They argued that
users should be able to readily comprehend
and easily override any “intelligence” built
into the system. Furthermore, their practi-
cal need to perform searches quickly ar-
gued against a user interface that engaged
in interactive dialogues to elicit long and
“accurate” statements of users’ information
needs. They liked the convenience of
launching searches with short natural lan-
guage queries.
Users did volunteer suggestions for im-
provements, however. They wanted accu-
rate stemming, which behaved according
to their linguistic intuitions and did not, for
example, accidentally conflate technical
terms with similar natural language words.
They wanted the system to match phrases,
so that “C library” would be interpreted
as a single expression rather than the
combination of two potentially noncon-
tiguous terms. They wanted more control
over the system’s default behavior with
respect to noise words, synonyms, and
query broadening. They wanted the system
to recognize certain expressions such as
version numbers, so as to retrieve articles
containing variant forms or specializa-
tions (for example, a query containing
“V5.4” should match an article contain-
ing “version 5.4-la”). Help in reformulat-
ing queries would be valuable as well, so
long as the user remained firmly in control
of reformulation.
These interviews dampened our enthusi-
asm for a number of possible AI approaches;
we concluded that our initial efforts should
focus on improving the linguistic perfor-
mance of the existing Stars system. While
some of the users’ requested functionality
could be achieved through extensions to
the user query language, such as proximity
operators and wildcarding, we felt obliged
to explore solutions that would minimize
the user’s need to do anything more than
type in a simple natural language query.
We therefore decided to evaluate the feasi-
bility of incorporating some of the basic
building blocks of a standard natural lan-
guage processing system, namely a
computational lexicon, morphological an-
alyzer, and parser.
IEEE EXPERT
 I

Developing a prototype
To implement our first prototype, we
adapted a set of tools we had developed for
a machine-aided translation system. This
included a 17,000-word English dictionary
containing information about each word’s
grammatical category and stem forms. Its
morphological analyzer contained a set of
rules encoding English orthography and
inflectional suffixation. Given a string, it
proposed a set of potential stems that could
then be verified against the lexicon. The
parser was a simple bottom-up chartpars-
er,’ which, given a set of syntactic rules
and an English string, generated a table, or
chart, by applying the grammar to the to-
kens in the input string and storing all the
well-formed substrings. The chart can be
Uversion 5

thought of as a two-dimensional array, in

which the input tokens span single col-
umns along the topmost row. Larger con-
stituents recognized by the parser span
multiple columns directly below the to-
kens or phrases from which they are com-
posed. For natural language processing,
this data structure conveniently represents
competing ambiguous interpretations as
edges in different rows that span the same
columns.
Our original intent was to use the chart
parser to recognize potential noun phrases
in user queries. However, many user que-
ries consist simply of a string of nouns,
without punctuation or function words to
indicate noun phrase boundaries. We de-
cided that a purely syntactic approach to
recognizing noun phrases in user queries
was impractical, since any sequence of two
or more nouns in a query would be inter-
preted as a potential noun phrase. Instead,
we adopted the strategy of storing useful
noun phrases in the lexicon and constrain-
ing the parser to recognize only those
phrases. The parser was also used to recog-
nize special expressions, such as variants
of operating systems’ version numbers.
One common technique in implement-
ing an efficient text retrieval system is to
construct inverted indexes for descriptors,
facilitating rapid access to the objects as-
sociated with each descriptor.8 For our
prototype, we chose to index articles by the
citation forms of the words and phrases
contained in them (that is, the string you
would typically use to look up a word in a
traditional dictionary). This required run-
ning a morphological and phrasal analysis
Figure 1. The query input window and query reformulation workspace.
over the entire text base at load time, but
had the effect of reducing all inflectional
variants of a word or phrase to a single
index entry. By indexing on the citation
form, we were also collapsing across the
word’s part of speech. That is, our index
did not distinguish between “program” as a
noun and as a verb, for example. However,
since noun and verb senses of English
words are often semantically related, we
felt that any attempt to disambiguate words
into grammatical categories at index time
would have little tangible benefit for sub-
sequent information retrieval.
At query time, the AI-Stars prototype
parsed the user’s query, as described above,
to construct a chart containing the citation
forms of words, phrases, and special ex-
pressions. It then applied heuristics to de-
termine which of the identified terms should
be included in the actual search expression
to be matched against the article index.
Function words like “the” and “of” were
marked inactive. Likewise, any term whose
chart column span was exceeded by the
span of another term was labeled inactive,
thus eliminating any individual terms that
were part of a larger phrase. The prototype
then applied an algorithm to the chart to
construct a Boolean query from the re-
maining active terms. Roughly speaking,
the algorithm ORed together terms that
appeared in the same column on the chart,
and then ANDed together these disjunct^.^
AI-Stars ran the resulting query expression
against the database index, and displayed
the size of the matching article set to the
user, who could then examine the list or
modify the query and rerun it.
Reformulatingqueries
Adding linguistic components helped
correct a number of shortcomings in the
existing Stars engine, but we still had to
address our users’ second directive: to make
the system’s search behavior easier to in-
terrogate and manipulate. Our recent ex-
perience with generating Boolean expres-
sions from the underlying 2D chart data
structure led us to craft a visual representa-
tion of the chart to serve as a user window
into the internals of the Stars query inter-
preter. Figure 1 shows the visual display
we developed; specifically, the query input
window with the query reformulation work-
space. Each term in the query is depicted as
a tile in a 2D layout, with active terms in
reverse video to visually distinguish them

DECEMBER 1993 11
 I

I
H-r

I n
UdadTenar
Flk,
___
T m "I I .

igure 2. The thesaurus window, query input window, and modified query reformulation workspate.

from inactive terms. Each tile displays the Clicking on a tile changes it from active Selecting a synonym from the Thesaurus
term's citation form and the number of to inactive, and vice versa. window broadens the query by adding
article postings for that term. Through the Dragging a tile to a new location chang- the item as an active tile to the same
display, the results of all the default behav- es its Boolean relationship with the oth- column(s) as the previously selected
iors of the query interpreter are available at er tiles. workspace term, without changing the
aglance: the morphological analysis, phrase Stretching a tile across multiple col- term's activation.
matching, special expression recognition, umns effectively ORs the term with the Selecting a conceptually related term
and selection of active terms. The article AND of terms in the columns it spans. narrows the query by appending the item
postings provide feedback on the discrim- Selecting a tile option can pull up a as an active tile after the last occupied
inative role played by each term in the thesaurus window for that term (see Fig- column of the chart (visually establish-
query. For most configurations of the tiles, ure 2). ing a new column).
even the Boolean interpretation is visually Selecting a compound term has the same
The thesaurus provides a way to access
manifest; users can apply the rule of thumb effect as adding a synonym. Compound
related terms that can help broaden or nar-
that tiles in the same column are ORed and terms are superstrings of the term that
row a search. To minimize user effort in
those in different columns are ANDed. The
tiles in Figure 1 can thus be interpreted as
1 adding thesaurus terms to the query, we
contain at least one nonalphabetic char-
acter. They are common in computer
the niienr
1 divided the window into four catgories of
science domains where many terms, such
Y""J
1 relationships:
as error messages and facility names, are
composed of mnemonic strings embed-
. . . ded in longer strings, using nonalphabetic
But the visual display is not only a win- phrase for the selected term in the work- characters as delineators.
dow into the system's internals. It also space: It is placed as an active tile in the
serves as a direct manipulation workspace same column(s) as the previously selected Selected thesaurus terms are automatically
for performing iterative query refinement: workspace term, which is made inactive. placed on the chart in aposition (relative to

IEEE EXPERT
the initial term) that makes sense for their
relationship to that term, so the user does
not have to decide where to place the terms.
Since each default thesaurus operation
has an immediate visual manifestation, the
user can adjust the query if the default is
not what is desired. Figure 2 shows how the
query reformulation workspace from Fig-
ure 1 would look after clicking on the tiles
for “copy,” “BACKUP,” and “saveset,”
and adding the phrase “scratch tape” from
the thesaurus. This new tile configuration
would translate into the query
((“BACKUP” AND “saveset”) OR “BACKUP
saveset”) AND ”scratch tape” AND (“v5.0”
OR “version 5.0”))
In lieu of interacting directly with the
thesaurus, a user can opt to have prespeci-
fied categories of related terms (such as
synonyms) added to the query automati-
cally. In addition, other terms that might be
useful in query reformulation can be in-
cluded in the display as inactive tiles. In
Figure 2, for example, an inactive tile has
been added by default to generalize the
version number (“version 5.0” + “version
5”). In this way, the ability to broaden the
query according to this useful domain-
specific generalization is only a mouse
click away.
By making it easy to iteratively permute
and rerun a query, we were hoping to over-
come the traditional brittleness of all-or-
none responses to single Boolean queries,
without sacrificing the almost “surgical”
precision of the Boolean model. Likewise,
we believed that giving the user a conve-
nient way to grasp how the various compo-
nent terms contribute to the result would
improve the user’s sense of whether to
continue trying to refine a search or to
assume that no relevant article exists.
From prototype to product
Evaluating an information retrieval sys-
tem has always been problematic.’” Accu-
rately calculating precision and recall for a
given document collection and a given set
of queries requires a full understanding of
the collection’s contents and some way to
determine the subjective relevance of each
document to each query. Furthermore, dif-
ferent task orientations, user backgrounds,
and the nature of the information stored in
the collection might need to be considered.
DECEMBER 1993
Given the additional problem that our pro-
spective users were always too busy to ex-
tensively test a prototype, we had to make
do with a series of short demonstrations
followed by informal user evaluations. In
general, users were pleased with the lin-
guistic enhancements underlying the query
interpreter, especially the handling of phrases
and special expressions. Initial reactions to
our slide presentation about the query refor-
mulation workspace ranged widely; some
users were intrigued and enthusiastic, oth-
ers were baffled and skeptical. While we do
THERE
WAS NO WAY TO
UPDATE THE ARTlCLE INDEXES
WITHOUT REAPPLmG THE
ANALYZER TO ALL THE T m T S ,
A TlME-CONSUMING PROCESS.
not know how much of the latter response
was due to a general unfamiliarity with
direct-manipulation graphical user inter-
faces, we were pleased to find that most
users interviewed, including some of the
initially most dubious critics, felt at ease
with the workspace after about five minutes
of supervised interaction.
Encouraged by this feedback, the Stars
development team decided to include the
workspace in the next major release of the
Stars product. Also slated for inclusion
were a number of major architectural chang-
es and functional enhancements, many de-
signed specifically to support the customer
support centers. The centers wanted to
evolve their corporate network of loosely
coupled Stars databases into a true world-
wide distributed database, from which us-
ers could retrieve any piece of information
in an identical manner from any site. New
features included full clientkerver com-
puting, user-defined semistructured infor-
mation objects, data replication, hyperin-
formation linking, and logical database
partitioning. As we prepared to reimple-
ment the linguistic features we had proto-
typed, we had to consider how they would
integrate into this new architecture. In
addition, our experience implementing the
prototype had made us aware of several
weaknesses in our original design.
A dynamic lexicon. One of the first
problems we encountered was the need to
support a dynamic rather than static lexi-
con. The constant influx of new textual
information from a variety of sources, in-
cluding corporate publications, internal
bulletin boards, and external support
documentation, guranteed that lexical
k n o w l e d g d h e words and phrases that
comprise the application’s working vo-
cabulary-would also remain highly dy-
namic. Indeed, each new computing prod-
uct or technology is likely to introduce a
wealth of new phrases, endow existing
words with new meanings or new gram-
matical categories (for instance, a noun
might be adopted as a verb), and even coin
altogether novel words (such as “widget”
and “laptop”).
Compared to a static lexicon, an evolv-
ing lexicon has significant implications for
any information retrieval system. Since we
cannot assume that all required vocabulary
is known before the first article is loaded
into the system, the system must include
some mechanism for ensuring that the lex-
icon and the article database always re-
main synchronized. Not only must the lex-
icon be updated if new vocabulary is used
in an article, but, if article index keys are
based on linguistic analysis, then reindex-
ing might be necessary whenever the lin-
guistic information changes. Secondly, a
dynamic lexicon is likely to require many
of the same administrative functions that
apply to a dynamic textual database, such
as data distribution, security, activity log-
ging, data subsetting, transactional consis-
tency, and so on. We were therefore inter-
ested in exploring whether an information
retrieval system could itself be an effective
repository of lexical information, allowing
the lexicon to share all the administrative
facilities already provided.
As we began to add new words and
phrases to our prototype’s lexicon, we were
forced to confront the disadvantage of in-
dexing articles by their citation forms: There
was no way to update the article indexes
without reapplying the parser/morpholog-
ical analyzer to all the texts, a time-
consuming process. What was worse, a
new article might be added while the previ-
ous index transaction was still in progress
~
13
 and, conversely, new lexicon entries might
be added while an article index update was
already in progress to accommodate previ-
ous lexicon modifications. We did not want
to block either lexicon or article updates,
so we initially developed a scheme that
time-stamped the updates. This facilitated
the coordination of lexicon and index up-
dates, and gave users the illusion of a
consistent system by making new lexicon
entries “invisible” until all article reindex-
ing had completed. From a user’s perspec-
tive, however, it made no sense that, after
updating the lexicon, it took some inde-
terminate amount of time before the update
was reflected in the system’s search behav-
ior. Therefore, we began to look into other
possibilities.
We recognized that if we indexed on
surface forms (the strings just as they ap-
pear in the text) in addition to citation
forms, we could take advantage of a much
more efficient scheme for updating article
indexes. That is, whenever a new word was
to be added to the lexicon, we could use our
morphological generator to generate a list
of all the inflectional variants of the word.
We could then do a query on the OR of
those terms, and the resulting article set
would be the precise set that should be
indexed by the newly added citation form.
Likewise, for phrases, we could do a
query on the AND of the phrase compo-
nents to identify the articles that poten-
tially contained the phrase. However, with-
out information specifying the locations of
phrase components within the articles, it
was impossible to compute the precise set
without reparsing each article in the set to
check for a confirmed occurrence of the
phrase. We thus chose to build a concor-
dance index, which contains not only the
set of articles that contain each word but
also the location (token number) of each
occurrence of the word. Word adjacency
can be tested by comparing token numbers
in the index, enabling AI-Stars to find
phrases in articles without reparsing.
We also decided to try indexing only on
surface strings, and using query-time mor-
phological analysis and generation as well
as phrase detection (via concordance adja-
cency checking) to simulate the matching
of articles by citation forms of words and
phrases. This approach has the architectur-
al advantage of functionally isolating lin-
guistic processing from article indexing. If
indexes no longer depend on linguistic
14
processing, there is no longer a need to
coordinate updates of the lexicon with up-
dates to the article indexes. As soon as a
new word or phrase is added to the lexicon,
it can be used in a user query to access
articles.
The disadvantage of this approach is that
it moves more of the computational burden
to query time. However, we have found
that the major performance cost (the disk
110s required to fetch the index entries for
each inflectional variant) can be effectively
mitigated by using a B-tree index that sorts
I F INDEXES NO LONGER
DEPEND ON LINGUISTIC
PROCESSING, THERE IS NO
LONGER A NEED TO
COORDINATE UPDATES OF THE
LEXlCON WTH UPDATES TO
THE ARTICLE INDEXES.
the surface string keys according to a case
insensitive alphabetical sort. Since most
inflectional variants of a word (in English)
share the same prefix string, they will tend
to cluster on the same disk block, making it
likely that successive disk fetches will not
be required to access the index entries for
a set of inflectional variants.
Implementing a lexicon on top of an
information retrieval engine. As men-
tioned earlier, treating the lexicon as a
dynamic rather than a static data structure
requires support for the full range of activ-
ities implied by a dynamic database. These
include not only Add, Modify, and Delete
functions, but also a host of other features
that might be required in the real-world
environment in which the lexicon must
operate. Digital’s customer support lexi-
con is managed by a dedicated group of
database administrators, whose responsi-
bility also includes administrative over-
sight of the contents of the on-line textual
databases. Activity logging and reporting,
security, and data organization, subsetting,
and distribution are some of the functions
that not only must be provided for the text
database but are desirable for lexical data
as well.
We initially prototyped the AI-Stars lex-
icon on top of a standard relational data-
base management system. However, as we
added features and extended the system
into a wide-area distributed clientkerver
implementation, we began to explore the
possibility of layering the natural language
processing functions directly on top of the
AI-Stars Collection Services, the system’s
storage and indexing facilities. Several as-
pects of the system made this particularly
attractive:
In our distributed environment, Collec-
tion Services provide for the replication
of document collections at multiple net-
work sites, for both performance and
reliability (availability) reasons. Updates
to document collections are automati-
cally propagated to all replication sites.
By using the same facilities for lexical
data, lexicon updates can be handled in
an identical fashion.
Collection Services support the con-
struction of virtual databases (so-called
derived collections), defined by apply-
ing a query filter to the union of one or
more other collections. For textual data-
bases, this provides a mechanism to log-
ically categorize the data in multiple
ways, as well as to provide restricted
views of data for security reasons. For
lexical data, this capability can be used
to maintain sublanguage vocabularies or
vocabularies involving security restric-
tions. Collections also provide a natural
vehicle for providing separate lexicons
for different natural languages.
Lexical entries are semistructured ob-
jects. They can include free text data,
such as definitions and usage examples.
Layering the lexicon on the information
retrieval system lets users query lexical
data in exactly the same manner as any
other semistructured information object
in the AI-Stars database.
In spite of these important advantages, it
was not clear to us at first whether per-
formance (again, measured in disk I/Os)
would be competitive with other ap-
proaches. Retrieving individual words is
intrinsically less efficient in a concordance-
based inverted indexing system, since each
stem lookup retrieves not the set of words
but a set of object identification tags that
must in turn be used to access the word
IEEE EXPERT
objects. Fortunately, however, this indi-
rection (the use of intermediate sets of
object IDS) can be exploited to recognize
phrases. Rather than use an on-disk dis-
crimination net, for example, we can per-
form phrase recognition using set intersec-
tion over the concordance index, testing
whether contiguous words in a query are
members of the same phrase (or phrases).
Since linguistic analysis at query time must
include both word and phrase recognition,
the query time cost of using our concor-
dance-indexing scheme turns out, on the
whole, to be equivalent to that of alterna-
tive approaches. We therefore chose to
reimplement the lexicon on top of the AI-
Stars Collection Services.
Extending the query reformulation
workspace. Another piece of code that
underwent significant redesign was the
query reformulation workspace. We rec-
ognized that the workspace interface, while
originally designed for manipulating natu-
ral language queries, generalized nicely to
the case of structured queries, which range
over the values of structured data fields,
such as author and date. For manipulating
structured data, we needed to extend the
chart workspace functionality to allow for
the nesting of subcharts. That is, we allow
a tile to correspond to either a value term,
such as a word in a natural language query,
or a full <field relational-operator value>
restriction expression.
A user can expand any tile of the second
type into its own chart window to manipu-
late its value content. But in addition, the
Boolean relationships among multiple-re-
striction expressions are now available for
direct manipulation in the top-level query
reformulation workspace. This structured-
query interface is illustrated in Figure 3, in
which the tile representing the restriction
for the All Text field has been expanded in
its own subquery window. The user can
manipulate the tiles in either window (or
both) and then rerun the query.
When other engineering groups expressed
interest in experimenting with this interface
for other applications, we decided to repack-
age the operations on the chart data struc-
ture, which originally had been tied directly
to the natural language parser. As our chart
now had at least two clients (the parser and
the direct manipulation interface), we decid-
ed to implement it as an abstract data type
with its own callable interface. The abstract
DECEMBER 1993
igure 3. The structured query interface for a query containing multiple restrictions. The
subquery window shows the workspate for the value of the All Text field.
interface supports the creation, movement,
and deletion of tiles. The operational seman-
tics of tiles is implemented via callbacks
associated with annotations on the tiles. A
chart is itself defined as a subtype of tile,
thereby allowing charts to be nested (as
needed for the structured query interface
described earlier.)
Using the morphological analyzer for
technical terms. Among the most useful
terms in some computer troubleshooting
queries are technical symbols such as
io$m-abort and sys$unwind. These terms,
identifying such things as function library
entry points and error messages, tend to
have a fixed number of systematic syntac-
tic variants, such as io$v-abort, $unwind-s,
$unwind_g, and so on. Just as with inflec-
tional variants of English words, users typ-
ically want to match on all variants; they
also tend to be inconsistent about which
variant they use in their queries. We have
therefore found it convenient to treat such
terms as we do inflectional variants, add-
ing rules to our morphological analyzer/
generator to cover technical symbols that
undergo regular patterns of variation.
Populating the lexicon. Given the re-
quirement for a dynamic lexicon, we de-
signed an interactive interface to make it
easy for database administrators to add
new vocabulary, phrases, and thesaurus
entries. However, manually identifying the
thousands of new words and phrases that
appear over time in on-line technical data-
bases would be economically infeasible.
We have therefore constructed a number of
tools, based on techniques developed in
'
the field of corpus linguistics,' to assist in
the (semi)automatic population of our lex-
ical knowledge base.
With each word in the lexicon, we store
its uninflected form, its part of speech,
inflectional paradigm, and orthographic
properties, such as whether its final conso-
nant gets doubled. Based on our assump-
tion that in a large enough text corpus,
words that inflect are likely to appear in
several of their inflected forms, we devel-
oped the following algorithm for extract-
ing information about unknown words in a
corpus:
(1) For each surface string in the corpus,
test whether it is already in the lexicon,
using morphological analysis. Omit
known surface strings from further
consideration.
(2) Since we generally do not need to store
inflectional information for proper
nouns in the lexicon, eliminate them
from the data using the simple heuristic
IS
, . -~~ .
- -~
I . I ‘.,.
i (VMS system” ((S “VMS”) (L “system“ mum)))
(“LTA device” ((S ‘LTA“) (L “device” nouns)))
(“load host” (6“load”) IL ”host” nouns)))
(“patch file” ((S “patch”) (L “file” nouns)))
(“server node” ((S “server”) (L “node” nouns)))
that proper nouns are usually capital-
ized in English. Use the morphological
guesser to generate a set of candidate
lexical entries for each of the remain-
ing (unknown and noncapitalized)
strings.
Apply abductive reasoning to choose a
“best guess” explanation for each un-
known surface form in the corpus. That
is, choose the lexical entry that ac-
counts for the greatest number of dif-
ferent surface forms that are actually
found in the text. For example, if we
found both “instantiate” and “instanti-
ates” in a corpus, the algorithm would
propose that “instantiate” is either the
singular form of a noun whose plural is
“instantiates” or it is the infinitive of a
verb whose third person singular form
is “instantiates.” However. if it finds
“instantiated” in the text as well, then
it prefers the verb interpretation, as
this interpretation would account for
three unknown strings in the corpus.
whereas the noun explanation covers
only two.
Many English verbs are also nouns, so it
is useful to supplement morphological
data with syntactic collocational data to
test whether a word is a noun in addition
to a verb. A simple heuristic for this
would be to test for the appearance of a
determiner (for example, “the” or “a”)
appearing directly before the potential
noun somewhere in the corpus.
Use heuristics to break ties. For words
that have more than one candidate ex-
planation with the same degree of cor-
pus covering, or for words that do not
appear to have any inflection variants
in the corpus. we apply a set of suffix
heuristics. The suffix - / y typically in-
dicates an adverb, whereas - i o r i indi-
cates a noun. For words for which no
suffix heuristic applies, we classify the
word as a noun by default.
. _- .~ . .-__ .-
, Star> without ;in cxce\.riw burdm of hand-
In English. a language with relatively
little morphological inflection, the vast
majority of new unknown words are nouns,
and this classification is often made on the
basis of the default case in step 4. How-
ever. we have found this algorithm partic-
ularly useful for languages such as French
in which a great deal of morphological
inflection exists. We are currently experi-
menting with it to populate a French lexi-
con from scratch.
Extracting phrases. Even with the best
parsers. syntactic analysis of large text
corpora can be computationally expensive,
slow, and error prone. Given that our goal
was to extract possible noun phrases in an
expedient manner, we eschewed full pars-
ing in favor of a more streamlined, if less
accurate, approach. We apply a three-pass
algorithm that
( I ) uses an ordered set of local syntactic
context rules (along with the
morphological analyzer) to heuristi-
cally tag each word in the corpus with
a single unambiguous part of speech,
(2) uses afinite-state network that encodes
a set of noun phrase recognition rules
tc) identify certain sequences of parts
of speech as potential noun phrases,
*..A
aiiu
(3) applies a filter to the result of step 2 to
remove a known set of false alarms.
These include candidate phrases that a
person in the loop has already explicit-
ly rejected.
This algorithm has been quite effective,
generating more than 15,000 noun phrases
from a 100-Mbyte corpus, with an accept-
ably low percentage of false alarms (Figure
4 gives examples of acquiredphrases). Most
false alarms are due to errors made in part-
of-speech tagging in step I . These knowl-
edge acquisition procedures make it possi-
ble to bring up new databases for use in
t;iiloring the Icxicm. S i n x the typical hclp
d e A en\ tronmcnt 4tiinlv doc\ not ha\.e the
manual administration of linguistic data- ,
base$, the ability to automate much of the 1
tion is still largely required is in specifying
thesaurus relationships among words. While
we are pursuing research in automatically
extracting thesaurus relationships from cor-
pora, this is still in the very early stages.’?
Fortunately, syntactically related words
generally are also semantically related. Thus,
we could use the database of noun phrases
as a source for generating related terms.
That is, words that appear together in
phrasal relationships can be made available
via the thesaurus as related terms.
r H E NEW VERSION O F STARS IS
in the early stages of field testing at several
of Digital’s customer support centers. Us-
ers have indicated that, subjectively at least,
the new search engine allows them to re-
trieve articles with greater confidence and
precision. Because they are adjusting to a
number of new facilities in the tool (in-
cluding the switch from a character cell
interface to a window-based one). many
have not yet begun to take full advantage of
the query reformulation facilities. Thus it
is still premature to speculate how much
more effective users will be once they are
comfortable with the range of new options.
We are continuing to alter the user inter-
face as users request. We are also looking
into the possibility of integrating other
retrieval models, where appropriate. For
example, if the user finds an article that is
closely related to but that does not com-
1 pletely fulfill some information need, it
might be useful to allow a vector space or
~
probabilistic search to test for other similar
~
articles in the database.
We view our present work as part of an
evolutionary approach to incorporating even
more sophisticated linguistic processing.
For example, a system that tried word sense
disambiguation, concept identification, or
the generation of syntactic variants of larger
phrasal units could also use a graphical
device like the query reformulation work-
space to make its inferences public and

malleable. At the Same time, our exper]-
ence integrating even modest amounts of
natural language processing in an informa-
system has made us
aware of the subtle relationships that can
exist between components. More sophisti-
cated use of natural language processing in
information retrieval will have to be ac-
companied by continued practical research
in system architecture and performance.
Acknowledgments
The evolution of the ideas embodied in AI-
Stars and its implementation have been a grow 1
effort. Jeffrey Robbins was the principal design-
er and implementor of the original Stars system.
Past and current members of the AI-Stars team
who have contributed to the work cited here
include Bryan Alvey, Suzanne Artemieff, Jeff
Brennen, Rex Flynn, David Hanssen, Jong Kim,
Jim Moore, Mayank Prakash, and Clark Wright.
I also wish to acknowledge Jim Miller andRalph
Swick of Digital’s Cambridge Research Lab for
their collaboration on the chart abstract data
type, as well as the many database administra-
tors and specialists who have offered their in-
sights, critiques, and support.
References
1. R.M. Tong andD.G. Shapiro, “Experimen-
tal Investigations of Uncertainty in a Rule-
Based System for Information Retrieval,”
Int’l J. Man-Machine Studies, V o l . 22, No.
3, 1985, pp. 265-282.
2. W.B. Croft and R.T. Thompson, “13R: A
New Approach to the Design of Document
Retrieval Systems,” J . Am. Soc. for Infor-
mation Science, Vol. 38,1987, pp. 389-404.
3. G . Guida and C. Tasso, “IR-NLI: An Ex-
pert Natural Language Interface to On-
Line Databases,” Proc. ACL Conf Applied
Natural Language Processing, Assoc. for
Computational Linguistics, Morristown,
N.J., 1983.
4. B. Kahle, “An Information System for Cor-
porate Users: Wide-Area Information Serv-
ers,’’ Tech. Report TMC-199, Thinking
Machines, Cambridge, Mass.
5. G. Salton and M.J. McGill, Inwoduction to
Modern Information Retrieval, McGraw-
Hill, New York, 1983.
6. N.J. Belkin and W.B. Croft, “Retrieval
Techniques,” in Ann. Rev. Information Sci-
ence and Technology, Vol. 22, Martha
Williams, ed., Elsevier Science Publishers,
New York, 1987, pp. 109-145.
7. M. Kay, “Algorithm Schemata and Data
Structures in Syntactic Processing,” Tech.
Report CSL-80-12, Xerox Palo Alto Re-
search Center, Palo Alto, Calif., 1980.
8. G. Salton, Automatic Text Processing: The
DECEMBER 1993
1
Transformarion. Analysis, and Retrieval of
Informarion by Computer, Addison-Wes-
1
~
ley, Reading, Mass., 1989.
9. P.G. Anick et al., “A Direct Manipulation
~
Interface for Boolean Information Retriev-
1 Tech. Report TR91- I 2063 expert system interfaces,
1 Dept. of Computer Science, Cornell Univ.,
Ithaca. N,Y,, 1991, full-text informationretrieval, and machine-aided
translation. He holds a MS from the University
1 1 . Compururional Linguisrics, special issue on of Wisconsin-Madison and is pursuing a PhD at
using large corpora, Vol. 19, No. I , 1993. Brandeis University. He is a member of the
I
I 12. J. Pustejovsky, S . Bergler, and P. Anick, Association for Computational Linguistics and
’ “Lexical Semantic Techniques for Corpus the ACM SIGIR. He can be reached at DEC,
Analysis,” Cnmputarinnal Linguistics, Vol. 1 1 I Locke Dr., LM02-1/D12, Marlboro,
19, No. 2, 1993, pp. 331-358. MA 01 752; e-mail, anick@aiag.enet.dec.com
RAL: Rule-Based Systems
RAL represents a new approach to rule-based
systems. It was developed by one of the oldest
and most experienced companies in the industry
to meet the needs of real-world developers. RAL
offers:
Intewration With C. Based o n a new implementation technology, RAL is
the first rule-based system to permit the seamless
integration of rules into C programs. When you use
RAL, you need no clumsy interface code between
the rules and the procedures. And you need no
“bridges” to access other software packages.
High performance. PST products are based o n the proprietary Rete I1
algorithm. Independently-developed benchmarks
have shown PST tools to run significantly faster
than the fastest competing tool tested.
True ease Of use. T h e rich rule language of RAL enables you to
solve problems with fewer rules. At PST, we
believe that this kind of expressive power defines
true ease of use. With RAL, you can concentrate
o n solving your problems, not o n working
around limitations of the tool.
Proven reliability. Systems developed with PST’s tools are used in
critical applications throughout North America,
Europe, and Japan. A number of systems devel-
oped using PST tools are now being sold as com-
mercial products. You may already be using RAL
systems without knowing it.
RAL is available for PCs, workstations, superminis, and mainframes
ProductionSystems Technologies Inc If you are inrerested in solving real-world problems with rule-based
5001 Baum Boulevard systems, write for our technical brochure today Or call us at
Pinsburgh Pennsylvania 15213 4 12-683-4000. (FAX: 412-683-6347)
Reader Service Number 2