Apex Institute of Technology

Department of Computer Science & Engineering

NATURAL LANGUAGE PROCESSING

DISCOVER . LEARN . EMPOWER

1
NATURAL LANGUAGE PROCESSING : Course Objectives

COURSE OBJECTIVES

The Course aims to:

•This course is an introduction to the fundamental concepts and techniques of natural
language processing (NLP).

2
COURSE OUTCOMES

On completion of this course, the students shall be able to:-

CO2 Recognize the feasibility of applying parsing techniques and its application through problem solving BT 1

CO3 Apply commonly used algorithms for processing linguistic information BT 3

3
Words have structured meanings
• Lexeme – a pairing of a form with a sense
• Orthographic form – the way the lexeme looks on the page
• Phonological form – the way the lexeme sounds
• Lexicon – finite list of lexemes

eats

eaten eat Lexeme eat

ate
 Lexical Relations

• Homonymy
• Polysemy
• Synonymy
• Hyponymy
 Homonymy

A relation that holds between two lexemes that have the same form with unrelated
meanings
• Homophones
• Homographs
• “found”

Lexeme sloping
mound

bank

Lexeme Financial
institution
Homonymy causes problems

• Spelling correction
• Confusables – your vs. you’re
• Speech recognition
• Homophones and pure homonyms
• Text-to-speech
• Homographs – conduct
• Information retrieval
• Homographs and pure homonyms
 Polysemy
The phenomenon where a single lexeme has multiple related meanings

Biological
repository

bank Lexeme

Financial
institution
Polysemy

• How many senses does a word have?

• Zeugma: Does Midwest Express serve breakfast and Philadelphia
• Kim has an uncle and so does Sandy
• Kim has a bat and so does Sandy
• How are they related?
• How can you tell which sense should be attributed to a given word?
 Synonymy
A relation that holds between two lexemes with the same sense

older

big lexeme

Positive
size
large lexeme
hyponymy

A relation that hold between two lexemes where one denotes a subclass of the other

vehicle hypernym vehicle

car hyponym car

• ontology
• taxonomy
• object hierarchy
WordNet

• A large electronic database of lexical relations

• A web-based interface
• Sets of lexical entries corresponding to unique orthographic forms,
accompanied by sets of senses associated with each form

Category Unique Forms Number of Senses

Noun 94474 116317
Verb 10319 22066
Adjective 20170 29881
Adverb 4546 5677
WordNet synset

• Synonymy is organized around the notion of a synset

• {chump, fish, fool, gull, mark, patsy, fall guy, sucker, schlemiel, shlemiel, soft
touch, mug}
• The synset is the sense associated with the WordNet entry.
• The semantic relations are relations between sysnets
 Internal Structure of Words

• Thematic roles
• Selectional restrictions
• Primitive decomposition
• Semantic fields
Thematic Roles

• Deep roles:
Houston’s Billy Hatcher broke a bat
e, x, y Isa(e, Breaking)  Breaker(e, BillyHatcher)  BrokenThing(e, y)  Isa(y,
BaseballBat)

He opened a door
e, x, y Isa(e, Opening)  Opener(e, he)  OpenedThing(e, y)  Isa(y, Door)
Thematic Roles
• Breaker and Opener are agents
• BrokenThing and OpenedThing are themes
• Some other commonly used thematic roles
• Experiencer
• Force
• Result
• Content
• Instrument
• Beneficiary
• Source
• Goal
Thematic Roles

• Thematic roles can be used as a shallow semantic language

• Can be used to determine surface realization:
AGENT > INSTRUMENT > THEME
The highest thing in the hierarchy will typically be the subject; the lowest thing will be
the object.
• Linking theory looks at the mapping between conceptual structure and grammatical
function.
Thematic Roles

• Doesn’t work for all verbs: donate, return, transfer

• It only helps for NP and PP arguments of verbs
• Differing perspective
• Amie bought the sandwich from Benson for three dollars
• Benson sold Amie the sandwich for three dollars
• Amie paid Benson three dollars for the sandwich
Selectional Restrictions

The senses of lexemes enforce selectional restrictions on their arguments

• Which airlines serve Denver
• The ServedThing is a geographical location
• Which airlines serve breakfast
• The ServedThing is a meal
This helps tell which sense of a lexeme is intended in a given context
Selectional Restrictions

Selectional restrictions occur at varying levels of specificity

In rehearsal, I often ask the musicians to imagine a tennis game.
They tell of jumping over beds they can’t imagine clearing while awake.
I cannot even imagine what this lady does all day.
Atlantis lifted Galileo from the launch pad.
Mr. Kruger lifted the fish from the water.
To diagonalize a matrix is to find its eigenvalues.
Representing Selectional Restrictions
The semantic contribution of a verb like eat

e, x, y Eating(e)  Agent(e, x)  Theme(e, y)  Isa(y, EdibleThing)

The phrase ate a hamburger would get something like

e, x, y Eating(e)  Agent(e, x)  Theme(e, y)  Isa(y, EdibleThing)

 Isa(y, hamburger)

Instead of using logical concepts, we can just use WordNet synsets: { food, nutrient }

{ hamburger, beefburger } is a hyponym of this.

Creativity and the Lexicon

We can use more word meanings that can be explicitly listed in the lexicon.
There are productive processes for creating new senses from those explicitly listed,
including

• Metaphor
• Metonymy
Metaphor

Using metaphor, we refer to, and reason about, concepts using terminology
appropriate to completely different kinds of concepts.
CORPORATION AS PERSON
• That doesn’t scare Digital, which has grown to be the worlds second-largest…
• Triton Group Ltd., a company it helped resuscitate, has begun acquiring Fuqua
shares
• But if it changed its mind, however, it would do so for investment reasons, the
filing said.
Metonymy

The use of one concept to refer to another concept closely related to it.
PRODUCT FOR PROCESS
• GM killed the Fiero because it had dedicated a full-scale factory to building the
plastic bodied car…

AUTHOR FOR WORKS

• He likes Shakespeare

PLACE FOR INSTITUTION

• The White House had no comment
Approaches to Metaphor and Metonymy

• Convention-based approaches hard-wire metaphors like CORPORATION AS

PERSON and metonyms like PRODUCT FOR PROCESS

• Reasoning-based approaches treat this as a problem for general, not necessarily

linguistic, reasoning, such as analogical reasoning.
Summary
• Lexeme
• Lexical Relations
• Homonymy
• Polysemy
• Synonymy
• Hyponymy
• WordNet
• Thematic Roles
• Selectional Restrictions
• Metaphor & Metonymy
Reference:
Books:

TEXTBOOKS
T1: Speech and Language processing an introduction to Natural Language Processing, Computational Linguistics
and speech Recognition by Daniel Jurafsky and James H. Martin
T2: Natural Language Processing with Python by Steven Bird, Ewan Klein, Edward Lopper
REFERENCE BOOKS:
R1: Handbook of Natural Language Processing, Second Edition—Nitin Indurkhya, Fred J. Damerau, Fred J. Damera

Course Link:
https://in.coursera.org/specializations/natural-language-processing

Video Link:
https://youtu.be/YVQcE5tV26s

Web Link:
https://www.tutorialspoint.com/natural_language_processing/natural_language_processing_tutorial.pdf
27