The Full Primal Sketch

Grouping, segmentation, texture

analysis and contour linking
Gestalt Psychology

 Early 20th Century

 Wertheimer, Koffka, Köhla
 “The whole is greater than the sum of
the parts”
Long horizontal bar

Short oblique line

Marr: The full primal sketch
 Larger structures such as boundaries
and regions are made explicit using
Gestalt-like grouping rules of the
primitives in the raw primal sketch
 The Gestalt grouping rules
– The Gestalt laws of perceptual
organization
 See
– Goldstein, “Sensation & Perception”, p196-223
– Sekuler & Blake, “Perception”, p144-149
 Similarity
Proximity

Good continuation (continuity)

 YES

NO Good continuation (continuity)

Similarity
Similarity
Similarity
Similarity

The dog has no outline, yet its elements group together

and segment it from the background
Closure
Closure
Closure
Closure
Symmetry
Amodal completion
Amodal completion and good
continuation
Amodal completion and good
continuation
Amodal completion and good
continuation
Termination points can form the
starting point for grouping and
segmentation…
Common fate
Common fate
Common fate
Common fate
Common fate
Common fate
The Marroquin pattern: Spontaneous organization and
reorganization
Real world
examples…
Figure and ground
—Rubin Vase
Figure and ground
—Rubin Vase
 Figure and ground

William Lloyd Garrison

—abolitionist and social
reformer (USA)
 Figure and ground

William Lloyd Garrison

—abolitionist and social
reformer (USA)

How do we separate
objects (figures) from
their background?
Figure and ground

Colour, lightness, texture, closure, proximity…

Figure and ground

Colour, lightness, texture, closure, proximity…

Figure and ground

Colour, lightness, texture, closure, proximity…

 Figure and ground
William Lloyd Garrison
—abolitionist and social
reformer (USA)

How do we separate objects

(figures) from their background?
1. Proximity/foreground
2. Closure of occluding
boundary

There is single-cell evidence that

cells in V2 and V4 (Zhou, Friedman
and von der Heydt, 2000) are
selective for which side of the
boundary is figure and which is
ground, suggesting they might be the
neurophysiological underpinnings of
figure-ground segregation task
Figure and ground
Figure and ground
Proximity vs closure
Proximity vs closure
Proximity vs similarity
Rows or columns? Rows

Columns

Proximity Proximity

Rows

Columns Here,
proximity
Similarity wins out over
similarity
Pragnanz: A general principle
 Roughly translates as ‘good figure’
 Central law of Gestalt psychologists
– ‘Law of good figure’
– ‘Law of simplicity’
 Sates: “Every stimulus pattern is seen
in such a way that the resulting
structure is as simple as possible”
Pragnanz: A general principle
Pragnanz: A general principle
Pragnanz: A general principle

8 8
Pragnanz: A general principle
 But how do you measure simplicity?
 Compare with: lightness, contrast,
orientation, etc…
 If we understand a principle of vision,
then we should be able to program it!
 The Gestalt psychologists described
things, but did not explain them!
Some contemporary points about
grouping:
 Visual texture analysis
Some contemporary points about
grouping:
 Visual texture analysis
Some contemporary points about
grouping:
 Visual texture analysis
Some contemporary points about
grouping:
 Visual texture analysis
Some contemporary points about
grouping:
 Visual texture analysis
Some contemporary points about
grouping:
 Visual texture analysis
Some contemporary points about
grouping:
 Visual texture analysis
Texture segmentation
Texture segmentation
Texture segmentation

Amodal
completion
Texture 2
Texture 1
Some contemporary points about
grouping:
 Spatial filtering
– Cortical simple cells as oriented filters
• These provide the basic mechanisms needed
to perform texture analysis
 Contour detection
– Field, Hayes & Hess (1993)
Contour detection

Easy (with practice) Hard (even with practice)

Contour detection

Easy (with practice) Hard (even with practice)

The association field: A basis for
contour linking in V1/V2?

Strong links
The association field: A basis for
contour linking in V1/V2?

No links
A problem for Marr…
A problem for Marr…
Visual search Find the horizontal…
“Pop out”
Visual search Find the green…
“Pop out”
Visual search Find the green horizontal…
Conjunction search
Experiments…
Reaction time (seconds) A parallel search strategy
slow
fast

few many
Number of distractors
Experiments…
Reaction time (seconds)
slow
A serial search strategy
fast

few many
Number of distractors
A problem for Marr…

Much closer
to parallel
search
A problem for Marr…

Much closer
to parallel
search
A problem for Marr…

Much closer
to parallel
search
A problem for Marr…

Much closer
to parallel
search
A problem for Marr…

Much closer
to parallel
search
A problem for Marr…

Much closer
to parallel
search
A problem for Marr…
 Grouping (and parallel search) can be
driven by depth information
 Thus either:
– Depth information can get back down from
the 2-1/2D sketch to the primal sketch. Or
– Grouping and segmentation continues to
happen in the 1-1/2D sketch
 Either way, Marr’s scheme of a strict
feed-forward arrangement between
representational stages with each stage
solving different tasks cannot be correct
Have a good festive break…