USING NLP OR NLP RESOURCES FOR INFORMATION RETRIEVAL TASKS

ALAN F. SMEATON Dublin City University Glasnevin, Dublin 9, IRELAND

1. Abstract
The imact of NLP on information retrieval tasks has largely been one of promise rather than substance. While there are exceptions to this as some of the chapters in the present volume demonstrate, for the most part NLP and information retrieval have only recently started to dovetail together. In this chapter we will present a precis of our experiments in information retrieval using NLP which have had mixed successover the last few years. We introduce the respective roles of NLP and IR and then we summarise our early experiments on using syntactic analysis to derive term dependencies and structured representations of term-term relationships. We then re-thought the role that NLP could have for IR tasks and decided to concentrate our e orts onto using NLP resources rather than NLP tools in information retrieval and our more recent experiments in this area in which we use WordNet are summarised. Finally we present our conclusions and the status of our work.1

2. Introduction
The development of tools and resources for automatic natural language processing (NLP) has been ongoing for many decades albeit with many ups and downs. The principal driving forces behind this have been applications such as machine translation (MT), information management and natural language interfaces.
1 in: \Natural Language Information Retrieval", T. Strzalkowski (Ed.), Kluwer Academic Publishers, 1997.

ALAN F. SMEATON

The relationship between NLP and MT has always been close, perhaps too close, and much of NLP research and the evolution of natural language processing techniques has been tailored to the MT application. This may have been to the detriment of other application areas as we shall discuss. The relationship between NLP and information management based on content has not been quite as symbiotic compared to NLP and MT. The content-based manipulation operations we refer to include indexing and retrieval, categorisation, classi cation, ltering, and so on. If we broadly de ne information retrieval to be the retrieval of textual information based on its content then we see that NLP tools and techniques do not have very much impact on our current generation of information retrieval systems. Operational IR systems are predominantly based on statistical measures of overlap between documents and queries, counting the numbers of words or index terms in common between the two as part of some similarity measure. The kind of NLP that been developed for applications like MT until recently has had little in uence on information retrieval. The present book goes some way towards highlighting the fact that NLP can and does have a greater role in information retrieval than many believe and many of the chapters report successful uses of NLP tools and techniques for IR applications. In this chapter we will present a precis of our experiments in information retrieval using NLP and these have had mixed successes over the years. In the following section we summarise our early experiments on using syntactic analysis to derive term dependencies and structured representations of term-term relationships. Following the negative results obtained with these approaches we then re-thought the role that NLP has for IR tasks and decided to concentrate our e orts onto using NLP resources rather than NLP tools in information retrieval as we discuss in section 4, and our experiments in this area are summarised in section 5. A concluding section outlines our plans for future work and presents our thoughts on whether using the NLP tools that are currently at our disposal, which are a legacy of the development for applications like machine translation, holds any long-term promise for the successful use of NLP in information retrieval tasks.

3. Early Experiments
Over the last 10 years, this author and his research group have tried a number of di erent ways of applying NLP tools and techniques to information retrieval tasks, with varying degrees of success and failure. During the mid-1980s we developed and experimented with techniques for parsing users' natural language queries and from the resulting parse trees we identi ed word pair and word triple dependencies between query terms,

NLP OR NLP RESOURCES FOR INFORMATION RETRIEVAL

which were then used as part of a term weighting retrieval. The improvements obtained in terms of retrieval e ectiveness were all but negligible which was disappointing given the fact that using heuristic rules based on simple word adjacency yielded comparable e ectiveness performances as our computationally heavyweight approach (Smeaton, 1988). This point was further supported by Joel Fagan from Cornell who parsed documents and user queries for the same purpose (Fagan, 1897) and obtained similar results. The syntactic level analysis of documents and queries which we performed in order to derive pair-wise and triple dependencies between index terms was not leading to progress in retrieval e ectiveness because syntactic and lexical ambiguities in queries and in documents generated too many noisy dependencies. It was clear that the type of syntactic analysis we were performing would always yield ambiguous output as this is an inherent feature of natural language and of the kind of analysis that parsing is. Realising that we could not eliminate syntactic ambiguities without encoding extra domain-dependent knowledge, to address this we developed a way in which to embed such ambiguities into a structured representation of text which we called tree structured analytics (TSAs). These TSAs were directly derivable from a morpho-syntactic analysis of input text based on Karlsson's ENGCG analysis which uses a Constraint Grammar framework (Karlsson et al., 1995). TSAs were formulated to encode within their structures the most commonly-occurring syntactic ambiguities due to PP attachment, conjunction, the scope of modi ers and some others. The structured representations of text can be used as a representation for complex phrases or for entire sentences and in a sense they postpone the interpretation of a document text until retrieval. If a text fragment has multiple (syntactic) interpretations, all of these are encoded into the TSAs and all are available at retrieval time with the TSA matching algorithm weighting various interpretations of the text fragment. Associated with the TSA format we developed a complex algorithm for matching TSAs and numerically quanitifying their degree of overlap. In-built into this algorithm, which essentially matched phrase or sentence pairs, was the ability to measure the degree of overlap between input phrases which may or may not have been about the same topic, but which used the same words or perhaps morphological variants of the same words though sometimes in di erent contexts. For example, given the input phrase:
communication among groups that use di erent type hierarchies

4 and the matching phrases:

ALAN F. SMEATON

a group of hierarchies for di erent types of communication using type hierarchies for communication

from our interpretation of these we can deduce that the second phrase is semantically closer to the input phrase than the rst, even though the rst phrase has more base words in common with the input than the second. This can be inferred from the structural roles di erent words play in the phrases, acting as heads, as modi ers or as attachments. In order to test whether our approach of syntactic structure matching had any utility, we developed a phrase set of seed phrases and for each, a set of 9 syntactic variants and we asked human volunteers to rank the derived phrases by their semantic similarity with the seed. Each seed/variant set was ranked by 10 di erent users and a Friedman test showed strong correlation among the volunteers in their rankings of the same phrase sets, indicating that people tended to agree with each other when doing this. We then used our TSA matching algorithm to rank the phrase set for each seed phrase and compared the TSA ranking against the averaged human ranking, using a Spearman correlation, and we found a strong overlap. We then repeated the experiments on a second dataset in order to re-enforce our results and obtained similar ndings. These experiments were reported (Sheridan and Smeaton, 1992). To move the TSA matching beyond a phrase-phrase matching function, we took part in category B of the TREC-3 benchmarking exercise performing ad hoc retrieval on c.550 Mbytes of Wall Street Journal data. We ran the ENGCG morpho-syntactic analyser on the documents and indexed them by the base forms of the words occurring within them. We then indexed the topic statements and generated a set of TSAs for each. Because of the computational complexity of the TSA matching algorithm we were unable to perform a TSA-TSA match between the TSAs in a query and the set of TSAs in each document so we performed a pre-fetch to retrieve the top 1000 documents, according to a tf I DF term weighting function, and ran the TSA-TSA match between each query TSA and each document TSA, for each of the top 1000 documents, for each query. We then aggregated TSA-TSA match scores into query-document scores and ranked the documents for evaluation. In terms of retrieval e ectiveness, the performance of our TSA-based retrieval runs was disappointing, much worse, in fact, than our own baseline tf I DF ranking which we used as a pre-fetch. Suspecting that it may have had something to do with the type of documents retrieved in the pre-

NLP OR NLP RESOURCES FOR INFORMATION RETRIEVAL

fetch, we applied our TSA match to a pre-fetch for each of the top 1000 document sets returned by research groups at Cornell using SMART, at Bellcore using an LSI-based retrieval and at UMass using INQUERY.2 In each case we managed to dis-improve their baseline performances as well. There are a number of reasons for our failure to make any progress in terms of information retrieval e ectiveness with our TSA-based approach to matching. One could be that the version of the ENGCG analyser we used (a prototype version which has since been improved) was of poor accuracy; another reason could be that the TREC topic statements consisted mostly of interrogative constructs whereas our TSA match had been developed for matching descriptive statements; yet another plausible reason could have been that our TSA-match would score highly and would retrieve texts that other approaches would not, especially if those texts had few words in common with a query but where those words played the same or similar structural roles in query and in document. This suggests that we should have run the TSA match on the entire document collection and not on the top-1000 retrieved by an alternative strategy. Computational resources and the demands of the TSA-match procedure prevented us from doing this at that time.

4. Using Natural Language Processes or NLP Resources

One of the observations we noted repeatedly in our work on trying to gainfully use language syntax to improve retrieval e ectiveness was that even if we had managed to overcome the problems caused by lexical and syntactic ambiguity there were many relevant documents which we would not have been able to retrieve with any degree of con dence simply because they did not contain many, or any, of the query terms in the rst place. Our early attempts at using NLP in IR were really attempts at furthering the conventional approach to information retrieval which is based on matching the \bag" of terms in a user's query with the \bags" of terms representing documents. In e ect this means we cannot handle cases of di erent words being used to represent the same meaning or concepts within documents or within queries, and we cannot handle polysemous words where single words have multiple meanings. In information retrieval, queries and their relevant documents are rarely identical and documents are compositional meaning that they contain information on more than one topic. It may be just one or it may be a combination of topics treated in a document that cause it to be relevant to a query. Furthermore, the query-document ranking process is only a part
2 Thanks are due to Chris Buckley, Sue Dumais and Jamie Callan respectively for providing these datasets.

ALAN F. SMEATON

of a user's information seeking task and during that overall task a user's information need may be stable or may vary, and it may shift slightly or evolve into something completely di erent during a single session (Ingwersen, 1992). All of these highlighted inadequacies with the role that information retrieval plays in information seeking show that the IR operation which we concentrate on is just one part of a larger process and if we are to make improvements in that process then we need to see information retrieval as something more than exact matching of bags of index terms. This realisation caused us to re-think how we could use natural language processing in IR. Clearly this does not require an incremental evolution in research progress but a complete revolution and re-examination of the IR task. Our research group was not and is not equipped to do this so we content ourselves with trying to make one small step and tackle one problem in this direction. A speci c problem in information retrieval that interested us was the case of di erent words being used to describe the same thing, either in queries or in documents. This requires some kind of knowledge base or repository which can equate or relate the terms used in language, i.e. a thesaurus. A thesaurus is a data structure which groups synonymous terms and relates them as either broader or narrower. Machine-readable thesauri, however, tend to be small in size or speci c to a certain domain, or both. Our concern was not to be constrained to any given domain but to remain domain-independent in order that our methods may be tried on large collections of documents. Information retrieval research has moved to the stage where new techniques really have to be evaluated and proved on test collections of the size of TREC (Harman, 1996) before they can be considered practical. The use of an NLP-related knowledge structure in information retrieval tasks does not require the use of natural language processing techniques per se. It is possible to utilise NLP resources only, within IR tasks without including what we could call the traditional NL processes like parsing or part-of-speech tagging. While this may seem like \cherry-picking" of ideas and resources from one discipline to use them in another if it can be used to improve the latter then it will be worthwhile. When we looked to see what kind of NLP resources were possible to use in IR we identi ed machine-readable dictionaries (MRDs) and knowledge bases (KBs) as candidates. The incorporation of MRDs into information retrieval was ongoing at the University of Massachusetts through work by Bob Krovetz and others (Krovetz and Croft, 1992)and at Glasgow University through work by Mark Sanderson (Sanderson, 1994). These sites were having mixed results in their experiments.

NLP OR NLP RESOURCES FOR INFORMATION RETRIEVAL

An even richer resource than an MRD is a knowledge base and for largescale, domain-independent knowledge bases there are only two, CYC and WordNet. Cyc is a long-term project initially started at MCC in Texas and now carried out at Cycorp. Its aim is to develop a huge knowledge base of common-sense knowledge (Lenat, 1995). Up to the present time its development is un nished and even preliminary versions are unavailable for commercial reasons. CYC holds a lot of potential for information retrieval and for many other areas as well but for the moment its contribution to IR remains untested. An alternative to CYC is WordNet which is large-scale, domainindependent, and its use is free for research purposes (Miller, 1995). WordNet has been used for a number of applications including information retrieval (Fellbaum, 1996). In our work we used WordNet as a starting point in developing a measurement of word-word semantic distance and in the next section of this chapter we describe this work.

5. Using WordNet for Information Retrieval

WordNet

is the product of a research project at Princeton University which has attempted to model the lexical knowledge of a native speaker of English. Information in WordNet is organised around logical groupings of related terms called synsets, each of which consists of a list of synonymous word forms and semantic pointers that describe relationships between the current synset and other synsets. A word form in a synset can be a single word or a group of up to 6 words which form a phrase or collocation. In our work we used the nouns from WordNet as a knowledge base and we ignored the verbs, adjectives and adverbs. We did this on the basis that nouns are the most content-bearing of all word classes and occur in every sentence. Our approach to using WordNet was to use it as a basis for measuring the semantic similarity between pairs of nouns. If we can develop a numerical estimate of how close two nouns are, for any pairs of nouns, then we can try to address the problems of di erent word use in queries and documents one of the features which makes the information retrieval task so di cult. WordNet organises synsets into an enormous network but it is an undirected network, with no root node or starting point, and no terminal nodes. In order to make handling the WordNet network more manageable we partitioned it into 11 smaller, overlapping graphs based on the IS-A hierarchical links between synsets, each of which had a root node and was itself a hierarchy. We call these WordNet partitions Hierarchical Concept Graphs (HCGs). The roots of these HCGs were chosen so as to give minimal overlap but 100% coverage of the WordNet network. One could imagine

ALAN F. SMEATON

this partitioning operation as choosing 11 nodes in the network, lifting them clear and having them pull up other nodes using the IS-A links only, until all nodes were associated with one HCG. We realised that our HCGs derived from WordNet in their raw state were inadequate for measuring semantic distances between synsets as HCG leaf nodes are very speci c in their content while those closer to HCG roots represent very broad and general concepts. Thus the signi cance of \distance" in the HCGs depends entirely on how far up or down the hierarchy one is. We therefore decided to assign an information content value to each synset and the approach we took was based on that taken by Resnik (1995). Essentially, the information content of each synset is approximated by estimating the probability of occurrence of all nouns in a synset, plus the probability of occurrence of all nouns in all subordinate synsets, within a corpus of text. To estimate these probabilities we applied the ENGCG analyser used in our earlier work on TSAs to a corpus of 278 Mbytes of text in order to identify all noun occurrences. These 11 million noun occurrences, including the occurrence of WordNet phrases or collocations, were used as a basis for estimating information content values of synsets. Having computed these values, we can say that the semantic similarity between two nouns (or more accurately the two synsets from which the nouns are drawn) is the information content of the rst synset which subsumes the two synsets from which the two input words are drawn (Richardson, 1995). The theory behind the development of our semantic similarity estimator is reasonably well-founded but its usefulness in practice remained to be evaluated. Rather than try it in an information retrieval application where its e ect would be diluted by so many other factors involved in IR processes, we decided to perform some tests on the e ectiveness of the semantic distance estimator independent of any application. Although there is no standard way of evaluating computational measures of semantic similarity, George Miller, developer of WordNet has stated that: \Semantic similarity is easily estimated by asking people to rate pairs of words with respect to their likelihood of meaning." (Miller et al., 199). A collection of 30 pairs of nouns was used in a similar exercise by Resnik (1995) and by Miller and Charles (Miller and Charles, 1991). In Miller and Charles' experiments they had humans rank the similarity between these noun pairs, some of which are semantically very close (journey and voyage, coast and shore, for example) and some of which have little overlap in meaning (glass and magician, noon and string, for example). We tested

NLP OR NLP RESOURCES FOR INFORMATION RETRIEVAL

the performance of our semantic similarity estimator against the averaged performance of a set of human judges and obtained a product moment correlation of r = 0:7939 where a value of +1:0 would have indicated an exact duplication of the ranking. These preliminary results were very encouraging and led us to try using word-word semantic distance measurement in an information retrieval application. To do this we used a portion of the TREC-3 collection, the category B data consisting of 550 Mbytes of articles from the Wall Street Journal. In theory, our similarity measure for query-document similarity computed all pair-wise noun-noun similarities between words identi ed as nouns in the input document and in the query, and normalised the sum of these similarity estimates to account for variable document and query lengths. To reduce the e ects of noisy similarity measures we applied a threshold to the similarity values used such that only query term-document term similarity values above this threshold would be used in computing a document's score. One issue in using word-word distances as part of information retrieval which we have not mentioned is due to polysemous words and the fact that each sense of a polysemous word has a separate entry in WordNet. Thus when we want to measure the similarity between pub and bar we must determine which sense of pub and which sense of bar we intend. To address this we developed a simple word sense disambiguation process for documents which choose the single most likely sense of a noun occurrence. In subsequent evaluations of this word sense disambiguation process it correctly disambiguated between 60% and 70% of word occurrences in the manually tagged Brown corpus (Richarson, 1995). The threshold values we used for word-word similarity were based on experimental runs and estimates of the time required to rank all documents in this collection based entirely on word-word similarities meant that such runs would be impossible. We therefore implemented our experiments by pre-fetching the top 1000 documents from the collection using a conventional IR technique of term weighting, and we performed our exhaustive word distance based measure on these top 1000 documents. This approach is reminiscent of that we took for evaluating the use of TSAs in document retrieval described earlier. The retrieval e ectiveness results we obtained with using word-word distances, in terms of precision and recall, are poor compared to the baseline tf I DF term weighting strategy but the possibility for errors in the di erent stages of these experiments are enormous (Richardson, 1995). We could have, and probably have had, errors in the syntactic tagging of documents to determine what are the noun occurrences, in the word sense disambiguation of documents, and at the level of semantic matching be-

10

ALAN F. SMEATON

tween words. When we performed a failure analysis of retrieved documents we found that the inaccuracy of our word sense disambiguation process and the large number of proper nouns in TREC queries had a severely debilitating e ect on retrieval e ectiveness. This suggested to us that if we were to pursue the notion of incorporating word-word semantic distance in information retrieval applications we needed to isolate its contribution to query-document matching to see if it was contributing something useful, so we developed another set of experiments in another application domain which did this. The nal set of experiments we mention in this chapter isolate the contribution of word-word semantic distances by eliminating the problems of word sense disambiguation in documents, and the issues raised by proper names. The \documents" in these experiments are manually-de ned captions used to describe a collection of over 4,000 images. For this image collection we manually captioned each image based on a short description of its content yielding captions such as
Fisherman in a canoe on a lake with a mountain peak behind A female clown painting a boy with war paint The smiling captain of a shing boat with a bridge behind

For each polysemous word or phrase in all captions we then manually disambiguated these into one or more than one WordNet word sense as appropriate. We then built a collection of user queries and relevance assessments using pooled output from several retrieval strategies, in a manner similar to the way in which the TREC document collection was formed. This created an environment in which we could try a variety of query-document similarity measures using word-word semantic distances while removing the noises caused by word sense disambiguation and proper name occurrences. One of the handicaps we su ered when incorporating word-word similarity into large-scale document retrieval experiments was the execution speed due to the costs of keeping HCGs on secondary storage and having to access them randomly during the computation of a query-document similarity. In the IR experiments on the TREC data, each query took a number of hours to run, and though clearly we were not concerned with e ciency aspects (yet) this could easily have been improved upon. To alleviate a possible operational bottleneck for our image caption retrieval, we pre-computed the similarity between 150,000,000 noun pairs from WordNet and stored each similarity as a 2-byte integer in a single large lookup table. These similarity values were placed within this table at some location computed using a hash function on the two nouns whose similarity is

NLP OR NLP RESOURCES FOR INFORMATION RETRIEVAL

11

being estimated. Thus two hash function computations and a single disk IO operation are now used to retrieve a noun-noun similarity value. The set of experiments we conducted to develop a query-caption similarity formula are reported in (Smeaton and Quigley, 1996) and show a signi cant improvement in retrieval e ectiveness over our baseline tf I DF term weighting. The formula uses query and caption length normalisation and also applies a noise threshold for noun-noun similarity values, below which the similarity is not used in the overall calculation. We are very encouraged by these results though we do concede that they are in a much more contrived environment by virtue of the captions being much shorter, more focused and being manually sense disambiguated when compared to documents. We also acknowledge that the image caption retrieval experiments are small-scale, notwithstanding our comments earlier in this chapter on experiments on large document collections being the true indicator in modern experimental information retrieval. We are at least encouraged to continue our work on incorporating word-word semantic distances into IR.

6. Status and Plans

The experiments we have reported over the last 10 years represent a slow development of a document indexing and retrieval strategy. Having been initially de ated by the prospects of using word-word distances in largescale IR experiments we are now encouraged by the prospects. We now recognise the limitation of blindly indexing a document by all contentbearing words which occur within it, and we are now of the opinion that a document can be pre-processed to extract only the most important of its themes which can be used for indexing. This uses the fact that documents are compositional and we are presently investigating ways in which this can be implemented. Lexical chaining is a technique in which the frequencies of occurrence and co-occurrence of words in text or in dialogue, as well as their adjacency in a given text is used to determine chains of related words which can be used to determine text content. Lexical chaining is being developed as a document indexing technique at the University of Waterloo (Kazman, 1996), where the documents are transcripts of meetings, and at UMIST where the application is conventional document retrieval (Stairmand, 1997). We are at present in the process of incorporating our word-word semantic distance estimator into a document indexing process. The appealing aspects of this are that the negative impacts of proper noun occurrences and word sense disambiguation are diluted. This is because the word-word distance estimator is only part of the process of determining lexical chains and can be used when document words occur in WordNet and discarded without

12

ALAN F. SMEATON

e ect when they do not, as happens with proper nouns. There are clear drawbacks to the approach we are taking. For a start, the knowledge base we used as a basis for our distance estimator, WordNet, is manually constructed but it is US-centric, the word senses used are static and xed, the vocabulary is limited, and there is no domain-speci city. This latter point is a criticism because it means that proper nouns, which are the characteristics that make a domain-speci c application, are always going to be a problem for our approach. For the foreseeable future we plan to continue our exploration of the use of semantic distance estimation in IR applications and we do this because we believe results are obtainable within the short term, our techniques are general purpose and not domain-dependent, and they will be computable within reasonable response time. In the longer term, we believe that current approaches to using natural language processing in information retrieval are a rough coupling rather than an integration. This is so because NLP is simply not suited to information retrieval tasks having been developed for applications which are exact and precise, such as machine translation. Information retrieval is not an exact application and approximation is inherent to its operation because of the many degrees of uncertainty within the processes involved. A completely new approach to NLP for IR which really is natural language processing speci cally for information retrieval applications, is required before signi cant leaps in e ectiveness can result.
ACKNOWLEDGEMENTS: Much of the experimental work reported here was performed in conjunction with former students Fergus Kelledy, Ruairi O'Donnell, Ian Quigley, Ray Richardson and Paraic Sheridan,. Their contributions are acknowledged though the opinions expressed herein are entirely those of the author. Various parts of the work reported were funded by the CEC's ESPRIT, VALUE and LIBRARIES programmes, the National Software Directorate, FORBAIRT and IBM IISL.
Fagan, J.L. (1987). Experiments in Automatic Phrase Indexing for Document Retrieval: A Comparison of Syntactic and Non-Syntactic Methods. PhD thesis, Department of Computer Science, Cornell University, TR-87-868. Fellbaum, C. (1996). WordNet: An Electronic Lexical Database and Some of its Applications. MIT Press. Harman, D.K. (Ed.) (1996). The Fourth Text REtrieval Conference (TREC-4). NIST Special Publication 500-236. Ingwersen, P. (1992). Information Retrieval Interaction. Taylor Graham. Karlsson, F., Voutilainen, A., Heikkila, J. and Anttila, a. (1995). Constraint Grammar: A Language-Independent System for Parsing Unrestricted Text. Mouton de Gruyter:

References

NLP OR NLP RESOURCES FOR INFORMATION RETRIEVAL

13

Berlin and New York. Kazman, R., Al-Halimi, R., Hunt, W. and Mantei, M. (1996). Four Paradigms for Indexing Video Conferences. IEEE Multimedia, 3(1), pp. 63{73. Krovetz, R. and Croft, W.B. (1992). Lexical Ambiguity and Information Retrieval. ACM Transactions on Information Systems, 10(2), pp. 115{141. Lenat, D.B. (1995). CYC: A Large-Scale Investment in Knowledge Infrastructure. Communications of the ACM, 38(11), pp. 33{38. Miller, G.A. and Charles, G. (1991). Contextual Correlates of Semantic Similarity. Language and Cognitive processes, 6(1), pp. 1{28. Miller, G.A., Beckwith, R., Felbaum, C., Gross, D. and Miller, K. (1990). Introduction to WordNet: An On-Line Lexical Database. International Journal of Lexicography,, 3(4), pp. 235-244. Miller, G.A. (1995). WordNet: A Lexical Database for English. Communications of the ACM, 38(11), pp. 39{41. Resnik, P. (1995). Using Information Content to Evaluate Semantic Similarity in a Taxonomy. in: Proceedings of the 14th International Joint Conference on Arti cial Intelligence. Richardson, R. and Smeaton, A.F. (1995). Automatic Word Sense Disambiguation in a KBIR Application. Presented at the 17th BCS-IRSG Colloquium, Manchester and reprinted in TheNew Review of Document and text Management,, 1, pp. 299{319. Richardson, R. (1994). A Semantic-Based Approach to Information Processing. PhD Thesis, Dublin City University, Glasnevin, Dublin, Ireland. Sanderson, M. (1994). Word Sense Disambiguation and Information Retrieval. in Proceedings of the 17th ACM-SIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland, pp. 49-57. Sheridan, P. and Smeaton, A.F. (1992). The Application of Morpho-Syntactic Language processing to E ective Phrase Matching. Information Processing and Management,

Smeaton, A.F. and van Rijsbergen, C.J. (1988). Experiments on Incorporating Syntactic Processing of User Queries into a Document Retrieval Strategy in Proceedings of the 11th International ACM-SIGIR Conference on Research and Development in Information Retrieval, Grenoble, France, pp. 31-54. Smeaton, A.F. and Quigley, I. (1996). Experiments on Using Semantic Distances between Words in Image Caption Retrieval. in Proceedings of the 19th ACM-SIGIR Conference on Research and Development in Information Retrieval, Zurich, Switzerland, pp. 174180. Stairmand, M.A. (1997). Textual Context Analysis for Information Retrieval. in Proceedings of the 20th ACM-SIGIR Conference on Research and Development in Information Retrieval, Philadelphia, USA.

28(3).