Detec%on and Es%ma%on Theory: Class Notes Ell 719

Detec%on
and Es%ma%on Theory
Class Notes
ELL 719
ELL 719: Detec%on & Es%ma%on Theory
Tenta%ve Course Outline (subject to change)
•  Introduc)on: Structure of detec%on & es%ma%on problems.

•  Simple Binary Hypothesis Tes)ng: Decision Criteria; Maximum Likelihood, Neyman-‐Pearson,
Bayes risk, Probability of Error and Min-‐Max Criteria; Receiver Opera%ng Characteris%cs;
Single and Mul%ple Observa%ons cases; Composite Hypothesis Tes%ng.
•  M’ary Hypotheses or Mul)ple Decision Theory: Mul%ple decisions; Bayes risk; Probability of
Error; General and Gaussian cases;
•  Detec)on of Con)nuous Time Signals: Detec%on of Signals in Addi%ve White Gaussian
Noise; Detec%on in Non-‐white Gaussian Noise; Signals with Unwanted Parameters (Random
phase, Random amplitude and phase).
•  Es)ma)on Theory Fundamentals: Maximum Likelihood, Bayes Cost and Minimum Variance
Es%mators; Rela%onship of Es%mators.
•  Es)ma)on with Gaussian Noise: Linear Observa%ons; Sequen%al Observa%ons; Non-‐linear
Es%ma%on; E-‐M algorithm; State Es%ma%on and Kalman Filtering.
•  Proper)es of Es)mators: Unbiased Es%mators; Efficient Es%mators; Asympto%c Proper%es.
•  Main Text: Detec%on, Es%ma%on & Modula%on Theory-‐Part 1: Harry L.Van Trees, John
Wiley; Paperback reprint, 2003.
2018 Detec%on & Es%ma%on, SP, IIT Delhi 2

Background: Pre-‐requisite
•  Signal Theory
Probability Theory, Random Variables and Random Processes
•  Signals and Systems

Representa%on of Signals and Systems. Fourier Series and
Transforms and Related Topics.
•  Basic Communica%on Theory.

Learning Outcomes
•  Be able to formulate different kinds of Signal Detec%on

Problems.
•  Be able to use sta%s%cal and mathema%cal tools to solve a
hierarchy of Detec%on Problems and design op%mum
processors.
•  Be able to formulate different kinds of parameter and signal
es%ma%on problems.
•  Be able to solve basic problems using sta%s%cal and
mathema%cal tools and design op%mal es%mators.


Introduc%on: Detec%on Theory
•  Deals with detec%ng the presence of useful signals in the
presence of noise and interference.
•  Applica%ons: Digital Communica%ons, Radar, Sonar,

Radio-‐astronomy; many others.
•  Broadly, we may look for

–  Known signals.
–  Par%ally known signals, or
–  Completely unknown and Random signals
in noise.

Examples
•  Known Signals in Noise Problems
–  Digital Communica%on via FSK:

s1 (t) = sin ω1t,s0 (t) = sin ω 0 t,0 ≤ t ≤ T

•  Transmieed over a communica%on channel, which yields
an undistorted replica, with thermal noise:

r(t) = s1 (t ) + n(t),or

r(t) = s0 (t ) + n(t),0 ≤ t ≤ T
•  Receiver: RF amplifier + Processor to decide which of the
two signals was transmieed over the %me interval.

•  Detec%on Theory Problem: Design and Evaluate the
Processor
Signals with Unknown Parameters in Noise
•  FSK problem with a random phase drii in oscillator or

medium:
r( t) = sin(ω1t + θ 1 ) + n( t),0 ≤ t ≤ T
r( t) = sin(ω 0 t + θ 0 ) + n( t),0 ≤ t ≤ T
–  θ1 and θ0:unknown, but constant phase angles.

€ –  Signal not completely known even in absence of noise.
–  This uncertainty must be taken into account while
designing the detector.

Signals with Unknown Parameters in Noise
•  Ac%ve Radar/Sonar Problem:
•  Tx. Signal: s(t) = sin ω c t,0 ≤ t ≤ T

•  Received signal when target present:
€r(t) = Vr sin(ω c (t − τ ) + θ r ) + n(t),τ ≤ t ≤ τ + T

= n(t),0 ≤ t ≤ τ ,τ + T < t ≤ ∞.
•  When target absent:
r(t) = n(t),0 ≤ t ≤ ∞
•  Three
€ unknown parameters even in the absence of noise.

€
Random Signals in Noise
•  Passive sonar detec%on: Receiver listens for noise generated

by enemy submarines: Engines, Propellers and other
machinery.
•  Acous%c signals travel through ocean to hydrophones in the

detec%on system.
•  Desired signal: Itself a sample func%on of a random process.
•  Hydrophone generates self-‐noise and picks up sea noise.

Passive Sonar Detec%on Problem

r(t) = sΩ (t) + n(t) :Signal present
r(t) = n(t) :Signal absent

•  Troposcaeer/Underwater Acous%c/Fading Channel

Communica%on Problem:
•  Transmieed signals: s1(t) and s0(t), as in FSK.
•  Received signals:
r(t) = sΩ1 (t) + n(t) :when s1(t) transmieed
r(t) = sΩ 0 (t) + n(t) :when s0(t) transmieed
•  sΩ1(t) and sΩ0(t): sample func%ons of random processes

centred around ω1 and ω0 respec%vely.

•  In such cases, the signals to be detected lack a determinis%c

component.
•  Detector Design: Must be based on the difference in the
sta%s%cal proper%es of the random processes of which sΩ1(t)
and sΩ0(t) are sample func%ons.

Mul%ple Decision Problems
•  M’ary digital modula%ons.
•  Classifica%on problems from observed features data etc.

Hierarchy: 3 levels
Known signals in •  Synchronous digital communica%ons

noise •  Paeern recogni%on problems
Signals with •  Ac%ve radar and sonar; target classifica%on

unknown •  Digital communica%on systems without phase
reference or over slowly fading channels
parameters
•  Digital comm. Over scaeer links; Passive sonar

Random Signals detec%on.
in noise •  Seismic detec%on systems; radio-‐astronomy

Classical Detec%on Theory
Defini%ons
•  Components of Decision Theory Problems:

H1 Probabilis%c
Source Transi%on Observa%on
space
H0 Mechanism

Decision

Components of Decision Theory Problems
•  Source: generates an output: One of two (or M) choices.
•  Digital Communica%ons: 1’s and 0’s of informa%on to be

transmieed.
•  Medical ECG: H1: Pa%ent had a heart aeack; H0: absent;
•  Speaker Classifica%on: Speaker German, Bri%sh or American;
Male or Female; (6 hypotheses).
•  Radar/Sonar: H1:Target present; H0: Target absent, in a given
range-‐azimuth cell.

Components of Decision Theory Problems
•  Probabilis)c Transi)on Mechanism: device that knows the source output,
but modifies it to generate a point in a suitable observa)on space
according to a probability law.
•  H1 +1 n
r Observa)on space:
+
•  H0 -‐1 one-‐dimensional
Transi%on Mechanism

Example 1.1
•  H1: r = 1 + n; H0: r = -‐1+n

•  Probability densi%es of r on the two hypotheses:


Example 1.1

Example 1.2
•  Source generates two numbers in a sequence.
H1 : r1 =1+ n1;r2 =1+ n2

H0 :r1 = −1+ n1;r2 = −1+ n2
•  Observa%on Space: 2-‐dimensional.
€
€
Example 1.2

Example 1.2

Decision Rule
•  Decision Rule: Guesses which hypotheses is true: Assigns each

point to one of the hypotheses.

•  Decision Rule mapping: Subject of Detec%on Theory.

Simple Binary Hypothesis Tests
•  Let observa%on space correspond to N observa%ons:

(r1,r2,...,rN )
•  Each observa%on: A point in an N-‐dimensional space, and
denoted by r.
•  R is generated either under H0 or H1, with a-‐priori
probabili%es P0 and P1, and known condi%onal probabili%es
pr / H (R /H 0 ) and pr / H (R /H1) respec%vely.
0 1
•  Use this informa%on to develop a suitable decision rule.
€ €
Decision Criteria
•  (i) Bayes Criterion: Assumes knowledge of P0 and P1;

•  Each of the 4 decisions assigned a cost: C00,C10,C11,C01.
•  The average cost or risk is defined as
ℜ = C00 P0 Pr(sayH 0 / H 0 true)

+C10 P0 Pr(sayH1 / H 0 true)
+C11P1 Pr(sayH
€
1 / H1true)
+C01P1 Pr(sayH 0 / H1true)

Decision Rule: Bayes Criterion
•  Decision Rule must say either H1 or H0, based on the observed
vector.
: Rule for dividing the total observa%on space Z into two
parts: Z0 and Z1.


Decision Rule: Bayes Criterion
ℜ = C00 P0 ∫ pr / H 0 (R /H 0 )dR
Z0
+C10 P0 ∫ pr / H 0 (R /H 0 )dR
Z1
+C11P1 ∫ pr / H1 (R /H1 )dR

Z1
+C01P1 ∫ pr / H1 (R /H1 )dR

Z0

Decision Rule: Bayes Test
•  It is natural to assume that the cost of wrong decisions will be
larger than the corresponding correct decision:
•  i.e., C10 > C00 and C01 > C11.
•  Bayes Test: Choose the decision regions Z0 and Z1 such that
the average risk R is minimized.


Detec%on and Es%ma%on Theory: Class Notes Ell 719

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Detec%on and Es%ma%on Theory: Class Notes Ell 719

Uploaded by

Copyright:

Available Formats

Detec%on

and Es%ma%on Theory

• Introduc)on: Structure of detec%on & es%ma%on problems.

2018 Detec%on & Es%ma%on, SP, IIT Delhi 2

• Signals and Systems

• Basic Communica%on Theory.

2018 Detec%on & Es%ma%on, SP, IIT Delhi 3

• Be able to formulate diﬀerent kinds of Signal Detec%on

2018 Detec%on & Es%ma%on, SP, IIT Delhi 4

• Applica%ons: Digital Communica%ons, Radar, Sonar,

• Broadly, we may look for

2018 Detec%on & Es%ma%on, SP, IIT Delhi 5

• FSK problem with a random phase drii in oscillator or

– θ1 and θ0:unknown, but constant phase angles.

2018 Detec%on & Es%ma%on, SP, IIT Delhi 7

• Tx. Signal: s(t) = sin ω c t,0 ≤ t ≤ T

€r(t) = Vr sin(ω c (t − τ ) + θ r ) + n(t),τ ≤ t ≤ τ + T

• Passive sonar detec%on: Receiver listens for noise generated

• Acous%c signals travel through ocean to hydrophones in the

• Desired signal: Itself a sample func%on of a random process.

• Hydrophone generates self-­‐noise and picks up sea noise.

2018 Detec%on & Es%ma%on, SP, IIT Delhi 9

Passive Sonar Detec%on Problem

r(t) = n(t) :Signal absent

2018 Detec%on & Es%ma%on, SP, IIT Delhi 10

• Troposcaeer/Underwater Acous%c/Fading Channel

r(t) = sΩ 0 (t) + n(t) :when s0(t) transmieed

• sΩ1(t) and sΩ0(t): sample func%ons of random processes

2018 Detec%on & Es%ma%on, SP, IIT Delhi 11

• In such cases, the signals to be detected lack a determinis%c

2018 Detec%on & Es%ma%on, SP, IIT Delhi 12

• M’ary digital modula%ons.

• Classiﬁca%on problems from observed features data etc.

2018 Detec%on & Es%ma%on, SP, IIT Delhi 13

Known signals in • Synchronous digital communica%ons

Signals with • Ac%ve radar and sonar; target classiﬁca%on

• Digital comm. Over scaeer links; Passive sonar

2018 Detec%on & Es%ma%on, SP, IIT Delhi 14

• Components of Decision Theory Problems:

2018 Detec%on & Es%ma%on, SP, IIT Delhi 16

• Digital Communica%ons: 1’s and 0’s of informa%on to be

2018 Detec%on & Es%ma%on, SP, IIT Delhi 17

2018 Detec%on & Es%ma%on, SP, IIT Delhi 18

• H1: r = 1 + n; H0: r = -­‐1+n

2018 Detec%on & Es%ma%on, SP, IIT Delhi 19

2018 Detec%on & Es%ma%on, SP, IIT Delhi 20

• Source generates two numbers in a sequence.

H1 : r1 =1+ n1;r2 =1+ n2

2018 Detec%on & Es%ma%on, SP, IIT Delhi 22

2018 Detec%on & Es%ma%on, SP, IIT Delhi 23

• Decision Rule: Guesses which hypotheses is true: Assigns each

2018 Detec%on & Es%ma%on, SP, IIT Delhi 24

• Let observa%on space correspond to N observa%ons:

• Use this informa%on to develop a suitable decision rule.

• (i) Bayes Criterion: Assumes knowledge of P0 and P1;

ℜ = C00 P0 Pr(sayH 0 / H 0 true)

+C01P1 Pr(sayH 0 / H1true)

2018 Detec%on & Es%ma%on, SP, IIT Delhi 26

2018 Detec%on & Es%ma%on, SP, IIT Delhi 27

+C11P1 ∫ pr / H1 (R /H1 )dR

+C01P1 ∫ pr / H1 (R /H1 )dR

2018 Detec%on & Es%ma%on, SP, IIT Delhi 28

•  Introduc)on: Structure of detec%on & es%ma%on problems.

•  Signals and Systems

•  Basic Communica%on Theory.

•  Be able to formulate diﬀerent kinds of Signal Detec%on

•  Applica%ons: Digital Communica%ons, Radar, Sonar,

•  Broadly, we may look for

•  FSK problem with a random phase drii in oscillator or

–  θ1 and θ0:unknown, but constant phase angles.

•  Tx. Signal: s(t) = sin ω c t,0 ≤ t ≤ T

•  Passive sonar detec%on: Receiver listens for noise generated

•  Acous%c signals travel through ocean to hydrophones in the

•  Desired signal: Itself a sample func%on of a random process.

•  Hydrophone generates self-‐noise and picks up sea noise.

•  Troposcaeer/Underwater Acous%c/Fading Channel

•  sΩ1(t) and sΩ0(t): sample func%ons of random processes

•  In such cases, the signals to be detected lack a determinis%c

•  M’ary digital modula%ons.

•  Classiﬁca%on problems from observed features data etc.

Known signals in •  Synchronous digital communica%ons

Signals with •  Ac%ve radar and sonar; target classiﬁca%on

•  Digital comm. Over scaeer links; Passive sonar

•  Components of Decision Theory Problems:

•  Digital Communica%ons: 1’s and 0’s of informa%on to be

•  H1: r = 1 + n; H0: r = -‐1+n

•  Source generates two numbers in a sequence.

•  Decision Rule: Guesses which hypotheses is true: Assigns each

•  Let observa%on space correspond to N observa%ons:

•  Use this informa%on to develop a suitable decision rule.

•  (i) Bayes Criterion: Assumes knowledge of P0 and P1;

•  i.e., C10 > C00 and C01 > C11.