Lexical Semantics

Lexical Semantics

Semantics
Lexical semantics
Lexical semantic relations
WordNet
Word Sense Disambiguation
• Lesk algorithm
• Yarowsky’s algorithm
Lexical Semantics

Semantics : Study of meaning in language.

Meaning?
— Relationship of linguistic expression to the real world.
— Relationship of linguistic expressions to each other.

Lexical semantics : Meaning of individual word

Sentential semantics : How word meaning are combined together.

Lexical Semantics

The meaning of a word ?

Sword
Tomato
King

Referents, Extensional Definition

Lexical Semantics

Extensional Definition: of a concept or term formulates its meaning by specifying

its extension, that is, every object that falls under the definition of the concept or
term in question.

Extensional definitions are used when listing examples would give more applicable
information than other types of definition, and where listing the members of a set
tells the questioner enough about the nature of that set.

nation in world, Pine in tree, apple in fruit.

Intensional Definition:  gives the meaning of a term by specifying necessary and

sufficient conditions for when the term should be used.

In the case of nouns, this is equivalent to specifying the  properties that an object

needs to have in order to be counted as a referent of the term.

Bachelor ——> Unmarried

Telephone — —> Apparatus, sound, speech
Lexical Semantics

Lexeme: Pairing of a particular form (orthographic or phonological) with its

meaning.

For example, the lexeme BANK (noun) consists of bank and banks, but not

banker. BANKER is a lexeme of its own!

Lexicon: Finite list of lexemes

Lemma: The grammatical form that is used to represent a lexeme.

The lemma for sing, sang, sung is sing.

The specific form (e.g. sang) is called wordform.

Lemmatization: The process of mapping a wordform to a lemma.

Lexical Semantics

A word sense (or simply sense) is a discrete.

How specifically do terms relate to each other?

Here are some ways:

Hypernymy/hyponymy
Synonymy
Antonymy
Homonymy
Polysemy
Metonymy
Holonymy/meronymy
Lexical Semantics

hyponymy: A word of more specific meaning than a general or superordinate term

applicable to it. For example, spoon is a hyponym of cutlery.

hypernym: A word with a broad meaning constituting a category into which words
with more specific meanings fall; a superordinate. For example, colour is a
hypernym of red.

Hyponym Hypernym

Snake Reptile

Mango Fruit

red wine beverage

Lexical Semantics

Synonymy
(Roughly) same meaning

offspring descendent spawn

happy joyful merry

Antonymy

(Roughly) opposite meaning

happy sad

descendant ancestor

black white

up down
Lexical Semantics

Homonym — are words which sound alike or are spelled alike, but have different

meanings
Fluke: A fish, and a flatworm

The end parts of an anchor.

The fins on a whale's tail.

A stroke of luck.

Homophone – same sound

• e.g., son vs. sun, sea vs. see, sell vs. cell, their, there

Homograph – same written form

• e.g., lead (noun) vs. lead (verb) , close vs. close, minute vs. minute

Lexical Semantics
Polysemy — the coexistence of many possible meanings for a word or phrase/
Multiple related meanings.

S: (n) newspaper, paper (a daily or weekly publication on folded sheets; contains

news and articles and advertisements)

"he read his newspaper at breakfast”

S: (n) newspaper, paper, newspaper publisher (a business firm that publishes

newspapers)

"Murdoch owns many newspapers”

S: (n) newspaper, paper (the physical object that is the product of a newspaper
publisher)

"when it began to rain he covered his head with a newspaper”

S: (n) newspaper, newsprint (cheap paper made from wood pulp and used for
printing newspapers)

“they used bales of newspaper every day"

Lexical Semantics
Polysemy — the coexistence of many possible meanings for a word or phrase/
Multiple related meanings.

Bank

a financial institution.

I got a loan sanctioned from the bank.

the building where a financial institution offers services

the bank is on the Tolstoy road.

a synonym for 'rely upon'

e.g. "I'm your friend, you can bank on me”.

rely Upon is different but related, as it derives from the theme of security initiated by 1.

Lexical Semantics

Homonymy: unrelated v/s Polysemy: related meaning

S: (n) position, place (the particular portion of space occupied by something) "he put
the lamp back in its place"

S: (n) military position, position (a point occupied by troops for tactical reasons)

S: (n) position, view, perspective (a way of regarding situations or topics etc.)"consider

what follows from the positivist view"

S: (n) position, posture, attitude (the arrangement of the body and its limbs) "he
assumed an attitude of surrender"

S: (n) status, position (the relative position or standing of things or especially persons
in a society) "he had the status of a minor"; "the novel attained the status of a
classic"; "atheists do not enjoy a favourable position in American life"

S: (n) position, post, berth, office, spot, billet, place, situation (a job in an organization)
"he occupied a post in the treasury”
Lexical Semantics

Metonymy: Substitution of one entity for another related one

Ex: suit for business executive, or the turf for horse racing

The White House or The Oval Office - used in place of the President or White
House staff

• Heart - to refer to love or emotion

• Dish - for an entire plate of food

• Delhi - for centre government

• Silicon Valley - to refer to the tech industry

• Hollywood - to refer to the film industry

• New blood - used in place of new people, young generation

Word Sense Disambiguation

The task of choosing the right sense for the context.

– Classification with contextual features.

– We may be only interested in one sense

– Relying on dictionary senses has limitations in granularity and coverage.

– One of the central challenge in NLP

– Ubiquitous across most of the languages.

Usage:

Machine Translation - Correct lexical choices.

Information retrieval - resolves ambiguity in queries.

Information Extraction - accurate analysis of text.

Computationally determining which sense of a word is activated by its use in a particular context.

Word Sense Disambiguation

Knowledge Based Approaches

—Rely on knowledge resources like WordNet, Thesaurus etc.

—May use grammar rules for disambiguation.

—May use hand coded rules for disambiguation.

Machine Learning Based Approaches

—Rely on corpus evidence.

—Train a model using tagged or untagged corpus.

—Probabilistic/Statistical models.

Hybrid Approaches

—Use corpus evidence as well as semantic relations form WordNet.

Word Sense Disambiguation

Knowledge Based Approach

This airlines serves dinner This airlines serves the

in the evening flight. sector between Agra & Delhi.
— serve (Verb) — serve (Verb)
agent agent
object – edible object – sector

Requires exhaustive enumeration of:

— Argument-structure of verbs.

— Selectional preferences of arguments.

— Description of properties of words such that meeting the selectional preference criteria
can be decided.

E.g. This flight serves the “region” between Mumbai and Delhi

How to decide if “region” is compatible with “sector” ?

Word Sense Disambiguation

Knowledge Based Approach

Require a Machine Readable Dictionary (MRD).

Find the overlap between the features of different senses of an ambiguous word (sense bag) and

the features of the words in its context (context bag).

These features could be sense definitions, example sentences, hypernyms etc.

The features could also be given weights.

The sense which has the maximum overlap is selected as the contextually appropriate sense.

Word Sense Disambiguation

Lesk’s Algorithm
More like a family of algorithms which, in essence, choose the sense whose
dictionary definition shares the most words with the target word’s neighbourhood.

Steps to disambiguate word w:

1. Construct a bag of words representation of the context, B

2. For each candidate sense Si of word w:

• Calculate a signature of the sense by taking all of the words in the

dictionary definition of Si
• Compute Overlap(B, signature(Si))

3. Select the sense with the highest overlap score

Word Sense Disambiguation

Lesk’s Algorithm
Sense Bag: contains the words in the definition of a candidate sense of the ambiguous word.

Context Bag: contains the words in the definition of each sense of each context word.
E.g. “On burning coal we get ash.”

Coal
Ash
Sense 1

Sense 1
A piece of glowing carbon or burnt wood.

Trees of the olive family with

pinnate leaves, thin furrowed Sense 2

bark and gray branches.

charcoal.

Sense 2

The solid residue left when Sense 3

combustible material is A black solid combustible substance formed

thoroughly burned or by the partial decomposition of vegetable
oxidized.
matter without free access to air and under the
Sense 3
influence of moisture and often increased
To convert into ash
pressure and temperature that is widely used
as a fuel for burning

Word Sense Disambiguation

Walker’s Algorithm
Totally based on thesaurus based approach

Step 1: For each sense of the target word find the thesaurus category to which that
sense belongs.

Step 2: : Calculate the score for each sense by using the context words. A context words
will add 1 to the score of the sense if the thesaurus category of the word matches
that of the sense

E.g. The money in this bank fetches an interest of 8% per annum

Target word: bank
Clue words from the context: money, interest, annum, fetch

Sense1: Sense2:
Money Finance 1 Location 0
Interest 1 0
Fetch 0 0
Annum 1 0
Total 3 0
Word Sense Disambiguation

Machine Learning Based Approach

Supervised learning is where you have input variables (x) and an output variable
(Y) and you use an algorithm to learn the mapping function from the input to the
output.

Semi-supervised/ Partially supervised Learning where you have a large amount

of input data (X) and only some of the data is labeled (Y) are called semi-
supervised learning problems.

Unsupervised Learning is where you only have input data (X) and no
corresponding output variables. The goal for unsupervised learning is to model the
underlying structure or distribution in the data in order  to learn more about the
data

Word Sense Disambiguation

Machine Learning Based Approach

• Supervised: All data is labeled and the algorithms learn to predict the output
from the input data.

• Unsupervised: All data is unlabelled and the algorithms learn to inherent

structure from the input data.

• Semi-supervised: Some data is labeled but most of it is unlabelled and a

mixture of supervised and unsupervised techniques can be used.
Word Sense Disambiguation

Machine Learning Based Approach

Yarowsky’s Algorithm (unsupervised)

A method based on bootstrapping

Goal: Learn a classifier for a target word

Steps: 1. Gather a data set with target word to be disambiguated

2. Automatically label a small seed set of examples
3. Repeat the following for a while:
• Train a supervised learning algorithm from the seed set
• Apply the supervised model to the entire data set
• Keep the highly confident classification outputs to be the new seed set
4. Use the last model as the final model
Word Sense Disambiguation

Step 1: Disambiguating plant

Word Sense Disambiguation
Step 2 : Set initial seed

Sense A:
• plant as in a lifeform Other data

Sense B:
• plant as in a factory
Word Sense Disambiguation
Step 3 : Train a classifier (Decision-List classifier)

Note how new collocations are found for each sense

Word Sense Disambiguation
Step 3 : Change seed set

Use only the cases where classifier is highly confident

96% on binary word sense distinction, same as with supervised Methods.

Word Sense Disambiguation
Decision List Classifier
Based on ‘One sense per collocation’ property.

Nearby words provide strong and consistent clues as to the sense of a target
word.

Collect a large set of collocations for the ambiguous word.

Calculate word-sense probability distributions for all such collocations.
Calculate the log-likelihood ratio
Assuming there are only
two senses for the word.
Log (Pr(Sense-A || Collocation i) Of course, this can easily
————————————— be extended to ‘k’ senses.
Pr(Sense-A || Collocation i))

Higher log-likelihood = more predictive evidence

Collocations are ordered in a decision list, with most predictive collocations

ranked highest.