Perception As Information Detection Reflections On Gibsons Ecological Approach To Visual Perception Paperbacknbsped 0367312964 9780367312961 Compress

Perception as Information Detection
This book provides a chapter-by-chapter update to and reflection on the

landmark volume by J. J. Gibson, The Ecological Approach to Visual Per-
ception (1979/2015).
Gibson’s book was presented as a pioneering approach in experimental
psychology; it was his most complete and mature description of the ecolo-
gical approach to visual perception. Perception as Information Detection
commemorates, develops, and updates each of the 16 chapters in Gibson’s
original volume. The book brings together some of the foremost perceptual
scientists in the field, from the United States, Europe, and Asia, to reflect
on Gibson’s original chapters, expand on the key concepts discussed and
relate this to their own cutting-edge research. This connects Gibson’s
classic with the current state of the field, as well as providing a new gener-
ation of students with a contemporary overview of the ecological approach
to visual perception.
This book is an important resource for perceptual scientists as well as
both undergraduates and graduates studying sensation and perception,
vision, cognitive science, ecological psychology, and philosophy of mind.
Jeffrey B. Wagman is Professor of Psychology at Illinois State University,

Normal, IL, USA. His research focuses on perception of affordances. He is
a recipient of the Illinois State University Outstanding University
Researcher Award and a Japan Society for the Promotion of Science Invita-
tion Fellowship for Research in Japan. He is an Associate Editor of the
journal Ecological Psychology.
Julia J. C. Blau is an Assistant Professor of Psychology at Central Con-

necticut State University, New Britain, CT, USA. She earned her doctorate
in ecological psychology from the University of Connecticut. Her research
focuses on the fractality of event perception, as well as the ecological
approach to film perception.
Resources for Ecological Psychology
A series of volumes edited by Jeffrey B. Wagman and
Julia J. C. Blau
[Robert E. Shaw, William M. Mace, and Michael Turvey,
Series Editors Emeriti]
Social and Applied Aspects of Perceiving Faces

Alley
Perception and Control of Self-Motion

Warren/Werthiem
Michotte’s Experimental Phenomenology

Thinès/Costall/Butterworth
Perceiving Events and Objects

Jannson/Bergstrōm/Epstein
Global Perspectives on the Ecology of Human-Machine Systems

(Volume 1)
Flach/Hancock/Caird/Vicente
Local Applications of the Ecological Approach to Human-Machine

Systems (Volume 2)
Hancock/Flach/Caird/Vicente
Dexterity and Its Development

Bernstein/Latash/Turvey
Ecological Psychology in Context

James Gibson, Roger Barker, and the Legacy of William James’s Radical
Empiricism
Heft
Perception as Information Detection

Reflections on Gibson’s Ecological Approach to Visual Perception
Wagman/Blau
Perception as Information
Detection
Reflections on Gibson’s Ecological
Approach to Visual Perception
Edited by Jeffrey B. Wagman and

Julia J. C. Blau
First edition published 2020
by Routledge
52 Vanderbilt Avenue, New York, NY 10017
and by Routledge
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
Routledge is an imprint of the Taylor & Francis Group, an informa business
© 2020 Taylor & Francis
The right of the editors Jeffrey B. Wagman and Julia J. C. Blau to be
identified as the authors of the editorial matter, and of the authors
for their individual chapters, has been asserted in accordance with
sections 77 and 78 of the Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this book may be reprinted or
reproduced or utilized in any form or by any electronic, mechanical,
or other means, now known or hereafter invented, including
photocopying and recording, or in any information storage or
retrieval system, without permission in writing from the publishers.
Trademark notice: Product or corporate names may be trademarks
or registered trademarks, and are used only for identification and
explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Names: Wagman, Jeffrey B., editor. | Blau, Julia J. C., 1982- editor.
Title: Perception as information detection : reflections on Gibson’s
Ecological approach to visual perception / edited by Jeffrey B.
Wagman and Julia J.C. Blau.
Description: New York, NY : Routledge, 2020. | Series: Resources
for ecological psychology | Includes bibliographical references and
index.
Identifiers: LCCN 2019014293 (print) | LCCN 2019016466 (ebook)
| ISBN 9780429316128 (eBook) | ISBN 9780367312954 (hardback)
| ISBN 9780367312961 (pbk.)
Subjects: LCSH: Gibson, James J. (James Jerome), 1904-1979.
Ecological approach to visual perception. | Visual perception. |
Environmental psychology.
Classification: LCC BF241.G48 (ebook) | LCC BF241.G48 P47 2020
(print) | DDC 152.14–dc23
LC record available at https://lccn.loc.gov/2019014293
ISBN: 978-0-367-31295-4 (hbk)

ISBN: 978-0-367-31296-1 (pbk)
ISBN: 978-0-429-31612-8 (ebk)
Typeset in Sabon
by Wearset Ltd, Boldon, Tyne and Wear
Contents
List of Illustrations viii

List of Contributors xi
Foreword: Resources for Ecological Psychology xiii
Preface xiv
Introduction 1
J effrey B . W agman and J ulia J . C . B lau
Part I
The Environment to Be Perceived 3
1 The Third Sense of Environment 5

E dward B aggs and A nthony C hemero
2 The Triad of Medium, Substance, and Surfaces for the

Theory of Further Scrutiny 21
T etsushi N ona k a
3 Ecological Interface Design Inspired by “The Meaningful

Environment” 37
C hristopher C . P agano A N D B rian D ay
4 Challenging the Axioms of Perception: The Retinal Image

and the Visibility of Light 51
C laudia C arello A N D M ichael T . T urvey
vi Contents
Part II
The Information for Visual Perception 71
5 Getting into the Ambient Optic Array and What We Might

Get Out of It 73
W illiam M . M ace
6 The Challenge of an Ecological Approach to Event

Perception: How to Obtain Forceful Control from Forceless
Information 90
R obert S haw A N D J effrey Kinsella - S haw
7 The Optical Information for Self-Perception in

Development 110
A udrey L . H . van der M eer A N D F . R . R uud van der W eel
8 A Guided Tour of Gibson’s Theory of Affordances 130

J effrey B . W agman
Part III
Visual Perception 149
9 Perceiving Surface Layout: Ground Theory, Affordances,

and the Objects of Perception 151
W illiam H . W arren
10 Acting Is Perceiving: Experiments on Perception of Motion

in the World and Movements of the Self, an Update 174
L . J ames S mart J r . , J ustin A . H assebroc k , A N D
M ax A . T eaford
11 Revisiting “The Discovery of the Occluding Edge and Its

Implications for Perception” 40 Years On 188
H arry H eft
12 Looking with the Head and Eyes 205

J ohn M . F rancha k
13 James Gibson’s Ecological Approach to Locomotion and

Manipulation: Development and Changing Affordances 222
Karen E . A dolph , J ustine E . H och , and O ri O ssmy
Contents vii
14 Information and Its Detection: The Consequences of
Gibson’s Theory of Information Pickup 237
B randon J . T homas , M ichael A . R iley , A N D
J effrey B . W agman
Part IV
Depiction 253
15 The Use and Uses of Depiction 255

T homas A . S toffregen
16 Revisiting Ecological Film Theory 274

J ulia J . C . B lau
References 291
Index 333
Illustrations
Figures
2.1 Water velocity 60 s after an 86 mm-long fish (Lepomis
gibbosus) passed the area 26
2.2 (a) Typical posture and movement of craftsmen during
stone bead production. (b) Examples of ellipsoidal glass
beads produced by expert (HQ) and non-expert (LQ)
craftsmen. (c) Singularity spectrum estimated for expert
(HQ) and non-expert (LQ) craftsmen 28
2.3 (a) Ventral surface of a flake detached by conchoidal
fracture. (b) Flake terminology 31
2.4 Re-fitted elongated ovate fine-grained porphyritic
phonolite cobble from Lokalalei 2C 33
3.1 A hand feeling a pair of scissors 42
4.1 (a) A diverging pencil of rays from a single reflecting point.
(b) A few converging cones show how an optical image could
be built from an infinite number of points on the object 53
4.2 The demonstration of a “retinal image” by Scheiner 56
4.3 Illumination is transparent to that which is illuminated 61
4.4 (a) Three holes drilled in a plywood box provided an
illumination shaft. (b) Rods angled obliquely directly in
front of the viewing aperture of the box 63
4.5 Photographs taken through the viewing aperture 63
4.6 Newton revisited 64
4.7 (a) Box with interior reflecting surfaces. (b) A top view
schematic of the light in the interior of the foam-lined
box. (c) A top view schematic of the light in the interior
of the box shown in (a) 65
4.8 Light reflected from spacewalkers: observations 67
5.1 Pike’s Peak, Barr Trail 76
5.2 Ambient optic array diagrams 77
6.1 The transformation of the optic array defined by a
locomotor movement 94
Illustrations ix
6.2 Illustrating the dual components of an ecosystem 101
6.3 The dual frame discrepancy hypothesis 105
6.4 Lee’s swinging room 106
7.1 A newborn baby participating in the weight-lifting
experiment 112
7.2 A newborn boy only a few hours old is studying his
hand intensely 115
7.3 A 4-month-old girl in deep concentration on the visual
motion presented on the screen in front of her 122
7.4 Accelerating looming stimulus approaching the infants’
eyes resulting in increased theta-band oscillatory activity
in the visual cortex of the extrinsic loom 126
8.1 Surfaces and affordances as interfaces 131
8.2 Affordances for support 133
8.3 Objects that afford sitting on (by humans) 134
8.4 Many animal species perceive affordances for reaching 135
8.5 Performing a given behavior creates and eliminates
affordances 137
8.6 Relationship between physics and geometry and
perception-action in standard (top) and ecological
(bottom) approaches 139
8.7 Objects that afford grasping with one hand and with
two hands 141
8.8 Perceptual experience vs. artificial measurement (M)
devices 142
9.1 The horizon ratio 154
9.2 Geometry of the ground theory 155
9.3 Ground texture as an intrinsic scale for relative
exocentric size and distance 157
9.4 Distance perception: experimental tasks, representative
data, and the results of numerical simulation based on
Equations 9.1 and 9.3 160
10.1 Depiction of the moving room used in Stoffregen and
Smart (1998) and Smart, Stoffregen, and Bardy (2002) 179
10.2 (a) Depiction of the virtual hallway stimuli used by
Warren, Kay, and Yilmaz (1996). (b) Depiction of large
screen projection of optic flow (sinusoidal) used in
Dijkstra, Schöner, and Gielen (1994) 181
10.3 Set-up and depiction of study for Littman (2011) 184
10.4 Potential alteration of perception and action when using VE 185
10.5 Depictions of optic flow conditions used in Smart et al.
(2014) 186
12.1 (a) Mobile eye tracker devised by Land (1992).
(b) Simultaneous eye and field of view recording for
tracking eye gaze 208
x Illustrations
12.2 (a) Lightweight, mobile adult eye tracker. (b) Mobile
infant eye tracker 209
12.3 Different stages of eye, head, and body rotations during
a 90° shift of gaze 215
12.4 Field of view for an average-height 8-year-old girl 217
12.5 Heat maps showing the frequency of face (top row) and
toy (bottom row) locations within the head-centered
field of view for 12- and 24-month-old infants during
fixations 218
14.1 Top: Apparatus and procedure for the overhead
reaching task. Bottom: Apparatus and procedure for the
minimum pass-through-ability task 240
14.2 (a) No perceptual intent condition. (b) Perceptual intent
condition of Experiment 1 247
15.1 A camera obscura 256
15.2 The chambered eye considered as a camera with a lens 257
15.3 Hand paintings in paleolithic rock art 263
15.4 Hand prints in development 263
15.5 The “City Churches” of Sir Christopher Wren, as seen
from 35,000 feet 266
15.6 Locating the vanishing point 272
16.1 (a) A camera receives light from a scene and focuses it,
then (b) a spinning shutter exposes each frame 276
16.2 (a) The original scene (in grayscale). (b) The negative.
(c) The positive print 277
16.3 (a) A bright light is projected through the positive print.
(b) A spinning shutter exposes, then blocks each frame
three times 278
16.4 The scanning pattern of a television or computer screen 279
16.5 (a) No blur specifies the ball (and camera) are still.
(b) Global blur specifies the camera is moving.
(c) Localized blur specifies the ball is moving 281
16.6 Typical editing structure 283
16.7 Nested event structure 286
Table
1.1 Some distinctions between the world, habitat, and
Umwelt 8
Contributors
Karen E. Adolph, Department of Psychology, New York University, New

York, USA.
Edward Baggs, Bartlett School of Architecture, University College London,
London, UK.
Julia J. C. Blau, Department of Psychology, Central Connecticut State
University, New Britain, CT, USA.
Claudia Carello, Center for the Ecological Study of Perception and Action,
University of Connecticut, Storrs, CT, USA.
Anthony Chemero, Departments of Philosophy and Psychology, University
of Cincinnati, Cincinnati, OH, USA.
Brian Day, Department of Psychology, Butler University, Indianapolis,
IN, USA.
John M. Franchak, Department of Psychology, University of California,
Riverside, CA, USA.
Justin A. Hassebrock, Department of Psychology, Miami University,
Oxford, OH, USA.
Harry Heft, Department of Psychology, Denison University, Granville,
OH, USA.
Justine E. Hoch, Department of Psychology, New York University, New
York, USA.
Jeffrey Kinsella-Shaw, Department of Kinesiology, University of Connecti-
cut and Center for the Ecological Study of Perception and Action,
William W. Mace, Department of Psychology, Trinity College, Hartford,
CT, USA.
Tetsushi Nonaka, Graduate School of Human Development and Environ-
ment, Kobe University, Kobe, Japan.
xii Contributors
Ori Ossmy, Department of Psychology, New York University, New
York, USA.
Christopher C. Pagano, Department of Psychology, Clemson University,
Clemson, SC, USA.
Michael A. Riley, Department of Psychology, Center for Cognition,
Action, & Perception, University of Cincinnati, Cincinnati, OH, USA.
Robert Shaw, Center for the Ecological Study of Perception and Action,
L. James Smart Jr., Department of Psychology, Miami University, Oxford,
OH, USA.
Thomas A. Stoffregen, School of Kinesiology, University of Minnesota,
Minneapolis, MN, USA.
Max A. Teaford, Department of Psychology, Miami University, Oxford,
OH, USA.
Brandon J. Thomas, Department of Psychology, University of Wisconsin,
Whitewater, WI, USA.
Michael T. Turvey, Center for the Ecological Study of Perception and
Action, University of Connecticut, Storrs, CT, USA, and Haskins Labo-
ratories, New Haven, CT, USA.
Audrey L. H. van der Meer, Developmental Neuroscience Laboratory,
Department of Psychology, Norwegian University of Science and Tech-
nology (NTNU), Trondheim, Norway.
F. R. Ruud van der Weel, Developmental Neuroscience Laboratory,
Department of Psychology, Norwegian University of Science and Tech-
nology (NTNU), Trondheim, Norway.
Jeffrey B. Wagman, Department of Psychology, Illinois State University,
Normal, IL, USA.
William H. Warren, Department of Cognitive, Linguistic, and Psycho-
logical Sciences, Brown University, Providence, RI, USA.
Foreword
Resources for Ecological Psychology
This series of volumes is dedicated to furthering the development of psychology

as a branch of ecological science. In its broadest sense, ecology is a multidisci-
plinary approach to the study of living systems, their environments, and the
reciprocity that has evolved between the two. Traditionally, ecological science
has emphasized the study of the biological bases of energy transactions between
animals and their physical environments across cellular, organismic, and popu-
lation scales. Ecological psychology complements this traditional focus by
emphasizing the study of information transactions between living systems and
their environments, especially as they pertain to perceiving situations of signifi-
cance to planning and execution of purposes activated in an environment.
The late James J. Gibson used the term ecological psychology to emphasize
this animal-environment mutuality for the study of problems of perception.
He believed that analyzing the environment to be perceived was just as much
a part of the psychologist’s task as analyzing animals themselves, and hence
that the “physical” concepts applied to the environment and the “biological”
and “psychological” concepts applied to organisms would have to be tailored
to one another in a larger system of mutual constraint. His early interest in
the applied problems of landing airplanes and driving automobiles led him to
pioneer the study of perceptual guidance of action.
The work of Nikolai Bernstein in biomechanics and physiology repres-
ents a complementary approach to problems of the coordination and
control of movement. His work suggests that action, too, cannot be studied
without reference to the environment, and that physical and biological
concepts must be developed together. The coupling of Gibson’s ideas with
those of Bernstein forms a natural basis for looking at the traditional
psychological topics of perceiving, acting, and knowing as activities of eco-
systems rather than isolated animals.
The aim of this series is to form a useful collection, a resource, for
people who wish to learn about ecological psychology and for those who
wish to contribute to its development. The series will include original
research, collected papers, reports of conferences and symposia, theoretical
monographs, technical handbooks, and works from the many disciplines
relevant to ecological psychology.
Jeffrey B. Wagman and Julia J. C. Blau
Preface
James J. Gibson began the preface to The Ecological Approach to Visual

Perception (1979/2015), by remarking “Vision is a strange and wonderful
business.” This is no less true in the first few decades of the 21st century
than it was in the last few decades of the 20th century. Vision continues to
be a strange and wonderful business. We hope that this book and the chap-
ters contained therein will serve as testament to this.
Since the publication of The Ecological Approach to Visual Perception
(and in large part because of it), Gibson’s ecological approach has flour-
ished, with a book series (‘Resources for Ecological Psychology’), an inter-
national society (the International Society for Ecological Psychology) and
an affiliated conference series, a dedicated journal (Ecological Psychology),
and research centers and prominent researchers based in locations around
the globe including the United States, Mexico, Brazil, the Netherlands, the
United Kingdom, Japan, South Korea, Taiwan, and Australia, among
others.
We are certain that Gibson would be pleased with how far his ecolo-
gical approach has progressed in the subsequent decades. Demonstrating
this progress is one of our primary motivations for editing this volume.
However, we are equally certain that Gibson would recognize that there is
still much work yet to be done. Inspiring others to continue to (as he put
it) “scramble through the underbrush” along with us in further developing
the ecological approach is another of our primary motivations. In short,
this book is our attempt to bring Gibson’s classic 1979 book into the 21st
century and beyond. We are especially proud that this book will be among
the first published in a revived ‘Resources for Ecological Psychology’ series
that we will be co-editing. We thank the Series Editors Emeriti (Robert
Shaw, William Mace, and Michael Turvey) for this opportunity.
This volume is dedicated to all those whose have been influenced by
Gibson’s ecological approach to (visual) perception, and who have duti-
fully (and thoroughly!) shared this influence with us. In particular, it is
dedicated to the faculty who shaped our thinking, research, and career
paths during our respective graduate educations at the Center for the Eco-
logical Study of Perception and Action at the University of Connecticut.
Preface xv
This group includes (but is not limited to) Claudia Carello, Jay Dixon,
Carol Fowler, Till Frank, Bruce Kay, Bill Mace, Claire Michaels, Bob
Shaw, Michael Turvey, and many others.
We are grateful to the chapter authors. Quite literally, this project
would not have been possible without their contributions. We also thank
the following people who graciously served as external reviewers of the
chapters: Joseph Anderson, Raoul Bongers, Felipe Cabrera, Pablo Cov-
arrubias, Matthieu M. de Wit, Judy Effken, Alen Hajnal, Ángel Jiménez,
Endre Kadar, Nam-Gyoon Kim, Kerry Marsh, Jon Matthis, Christopher
Pagano, H. A. Sedgwick, Joanne Smith, Jim Todd, and Michael Turvey.
JBW also thanks Dawn McBride and Connor Wagman, whose love and
support make all things possible.
JJCB also thanks Eric, Grayson, and Gwendolyn Blau (my incredible
family), Amy Rose Capetta (my magnificent QEF ), and Alexandra Paxton
(my brilliant sounding board).
We hope that our readers enjoy reading this collection as much as we
enjoyed editing it.
Introduction
James J. Gibson’s classic book The Ecological Approach to Visual Percep-

tion (1979/2015) was a landmark book in experimental psychology. It was
his most complete and mature description of the ecological approach to
visual perception––an approach in which animals detect complex optical
patterns that specify the behaviors that the animal can perform. It provided
a fundamentally new way to conceive of both the process of visual percep-
tion and (perhaps more importantly) what there is to be perceived. Gib-
son’s book has influenced generations of scholars across many disciplines,
but it has had its most significant and longest-lasting impact on scientists
studying visual perception, especially those who, like Gibson, do so from
an ecological perspective. As a testament to its enduring influence, the
book was reprinted in 1986 and again in 2015 (as a ‘Classic Edition’ by
Taylor & Francis). It has been translated into both German and Japanese
and, as of this writing, has been cited nearly 35,000 times in the scientific
literature.
Perception as Information Detection: Reflections on Gibson’s Ecolo-
gical Approach to Visual Perception commemorates, but also builds on
and updates each of the 16 chapters in The Ecological Approach to Visual
Perception. In this edited volume, representatives of multiple generations
of perceptual scientists from the United States, Europe, and Asia reflect on
a particular chapter of Gibson’s classic 1979 book, describe how the con-
cepts therein influenced both their respective research programs and the
development of the science more generally. We hope that the book both
serves to bridge the decades-long gap since its initial publication and brings
Gibson’s seminal work into the 21st century by providing an updated over-
view of the ecological approach to visual perception. Perception as
Information Detection: Reflections on Gibson’s Ecological Approach to
Visual Perception is both a tribute to The Ecological Approach to Visual
Perception and a reflection on the current state of the science.
Part I
The Environment to
Be Perceived
1 The Third Sense of Environment
Edward Baggs and Anthony Chemero
James Gibson begins his final book by making a distinction between two
senses in which the world surrounds the animal. The whole project of The
Ecological Approach to Visual Perception depends upon the distinction
Gibson makes between the physical world surrounding animals and the
environment surrounding them. The physical world contains everything
from sub-atomic particles to galaxies, but it does not contain meaning.
Perception does contain meaning. If the world that animals perceive is the
physical world, it is mysterious why perception is meaningful. The tradi-
tional solution to this mystery, which Gibson rejected, is that animals
make the meaning somehow. For animals like us, the traditional solution
has it that our brains construct the meaning in private, and that is where
the meaning lives. In proposing that the world that animals perceive is the
environment, not the physical world, Gibson proposes a different solution
to this mystery. Perception is meaningful, on Gibson’s account, because
there is meaning in the environment that animals perceive. Animals do not
create meaning; they discover it in the environment. Gibson’s solution
amounts to a radical rejection of the understanding of the nature of the
world and its relationship to experience, perception, and knowledge that
had been in place since the founding of modern science. Gibson’s solution
is the key move that motivates the rest of his book, and indeed the whole
of his ecological approach to psychology. The distinction has, however,
created new problems. The aim of this chapter is to argue that Gibson
should have made a distinction between not just two, but three different
senses in which the world surrounds the animal.
The distinction Gibson (1979/2015) makes is between the following:
1. The physical world: Reality is assumed to be structured in various

ways, prior to and irrespective of the existence of any animal living in
it. This structure is properly the subject matter of physics and geology.
2. The environment: Animals themselves do not perceive the world of
physics—of planets and atoms. Animals have evolved to perceive and
act at the terrestrial scale, relative to objects and surfaces that are
meaningful because they offer possibilities for action. To talk of an
6 Edward Baggs and Anthony Chemero
environment is to imply the existence of an animal that is surrounded.
The two terms are mutually codependent, and it is in this sense that
the animal and its environment constitute an animal–environment
system.
This distinction is both appropriate and intuitively appealing. But it fails to

include a further distinction: that between the evolutionary history of a
species and the personal history of a particular individual animal. These
are quite different things. We suggest that Gibson’s lumping together of
these ideas has led to no small number of confusions, and to unnecessary
controversy among proponents of ecological psychology.
It is necessary to subdivide what Gibson called the environment into
two quite distinct concepts. We propose to replace his two-way distinction
with a three-way distinction between (1) the physical world; (2) the species
habitat; and (3) the animal-specific Umwelt.
A Troublesome Observation
The additional distinction we are proposing here is not novel. Gibson
himself was quite aware that his use of “environment” referred ambigu-
ously to both the surroundings of an individual living animal (a token) and
the surroundings of an idealized member of a species (a type), and he noted
that this ambiguity could potentially cause confusion. Gibson writes
(1979/2015, p. 3):
The environment consists of the surroundings of animals. Let us observe

that in one sense the surroundings of a single animal are the same as the
surroundings of all animals but that in another sense the surroundings
of a single animal are different from those of any other animal. These
two senses of the term can be troublesome and may cause confusion.
The apparent contradiction can be resolved, but let us defer the problem
until later. (The solution lies in the fact that animals are mobile.)
As is clear here, Gibson thought he could resolve the confusion by pointing

out that animals move around. Gibson elaborates on this at the end of
Chapter 3 (Gibson, 1979/2015, p. 38):
In the course of time, each animal moves through the same paths of its
habitat as do other animals of its kind. Although it is true that no two
individuals can be at the same place at the same time, any individual
can stand in all places, and all individuals can stand in the same place
at different times. Insofar as the habitat has a persisting substantial
layout, therefore, all its inhabitants have an equal opportunity to
explore it. In this sense the environment surrounds all observers in the
same way that it surrounds a single observer.
The Third Sense of Environment 7
What Gibson says here is more or less true. For most of the objects and
surfaces around us, we can station our eyeballs relative to those surfaces at
a point of observation that was previously occupied by one of our fellows.
(There are exceptions to this: only you can see the end of your own nose
from the position of your own eyeball; Gibson, 1967a.)
But this does not really resolve the tension. There are many ways in which
the world can look different to two different members of the same species,
even if we all keep moving around in the appropriate way. This is not because
we are trapped in our own private mental world, as the traditional view has
it, but simply because we are different animals with different abilities.
Take a Chinese newspaper. Neither of the authors of this chapter knows
how to read Chinese. We might be able to recognize the general shape of
the characters and make the judgment, “That looks like Chinese writing,”
or again we might recognize someone in a photograph printed on the front
page. But we cannot read any of the headlines. The script is not meaning-
ful for us in the way that it is for someone who can read Chinese. Similarly,
the cockpit of a plane looks different to a trained pilot than it does to a
novice or to a child. The pilot can do things with the buttons and levers
that non-trained individuals cannot. Or, at an even more basic level, think
about climbing a set of stairs. The stairs look different depending on
whether you are a toddler or a long jumper or you are someone who has
just had a hip operation or you are wearing high-heeled shoes.
What’s missing, in Gibson’s elision of the two senses of environment
here, is an acknowledgment that the way the world looks to us is to some
extent a result of the way we currently are as individuals. In claiming that
we all share the same environment because we can in principle all see the
same surfaces from the same set of positions, Gibson is treating perception
as if it were an idealized process. He is suggesting that perception can be
separated from the perception–action loop that comprises the activity of a
living animal with a history of engagement and learning. This is odd,
because Gibson’s overall project throughout the book is to deny that per-
ception is a separate phenomenon in this way, and to promote the view
that perception and action are inherently one and the same process.
We suggest, then, that Gibson was right to point out the “troublesome”
ambiguity of the term “environment,” and that he was prescient in noting
that this ambiguity “may cause confusion.” Indeed, as we will show below, it
has caused confusion. The tension can be resolved, but not in the manner that
Gibson proposed. Instead, we need to make a further distinction, between:
the environment as it exists for a typical member of a species, a habitat; and
the environment as it exists for a particular living animal, an Umwelt.
It is important to note that the three senses of environment do not define
three distinct universes. Rather, the three senses are overlapping and
nested. Briefly, the habitat is the physical world considered relative to a
typical or ideal member of a species, i.e., it is a type of environment. It is a
complementary term to a species. The habitat continues to exist even when
a given animal dies. The Umwelt, meanwhile, is the physical world con-
sidered relative to a particular living individual animal, i.e., it is a token
instance of a habitat. Umwelt is a complementary term to a specific organ-
ism. When that organism dies, its Umwelt necessarily ceases to exist. (The
term Umwelt originates in the work of Jakob von Uëxkull, and refers to
the world as it appears to a given animal. See Kull, 2009, for a discussion
of the history of the term.)
We will not here attempt to provide watertight definitions of the three
concepts of world, habitat, and Umwelt. Instead we offer a set of illustra-
tive differences in Table 1.1. The upper part of Table 1.1 sets out some
Table 1.1 Some distinctions between the world, habitat, and Umwelt
Physical world Habitat Umwelt
Structure Resource/function Meaning
Scatle-free; atoms to galaxies Terrestrial scale Terrestrial scale
View from nowhere Ideal perspective for typical Has a first-person perspective
member of a species
Exists prior to any animal Exists for typical member of a Brought forth through
encountering it species development and active
exploration; enacted
No complementary term Complementary to “species” Complementary to

“organism”
Orthogonal to life Complementary to a form of life Complementary to a living

creature
Correspondence relations • ffordance-resources

A • O pportunities and
• Landscape of affordances solicitations
• Potential opportunities • Field of affordances
• Affordances as dispositional • Opportunities
properties of the habitat • Affordances as relations in
the Umwelt
Thermodynamic information; • Information-about the habitat • Information-for an

Shannon information • C onventional information (in exploring organism
the human habitat) • Saturated with linguistic
behavior (in humans)
History of pure structure, • H as a history in evolution • Has a history in

reconfigured by physical • Expanded by animal activities development
processes (niche construction) • Enriched by skill
• Target of learning acquisition
• Differentiated in learning • Location of learning
• Enriched by learning
Orthogonal to social • S hared among members of a • C eases to exist when a

processes; a set of points in species; continues to exist particular animal dies
space when a given animal dies • Places
• Behavior settings
broad distinctions between the three concepts. The lower part summarizes
the material below, in which we discuss how making the three-way distinc-
tion helps clarify the concepts of affordances, ecological information,
learning, and the social.
Some Tensions and Their Resolutions

Replacing Gibson’s two-way distinction with the three-way distinction we
propose here should allow us to re-evaluate how we think about the rest of
Gibson’s book, and indeed about the 40 years of research that has fol-
lowed. The entire program is predicated on treating “the environment” as
an unproblematic fundamental. Ecological psychologists happily state that
their unit of study is the “animal–environment system.” But this phrase
maintains the ambiguity identified above: are we talking here about a
typical member of a species residing in the environment (of that species),
or are we talking about a particular animal in its environment (its
Umwelt)?
As a result of this ambiguity a number of contradictions and controver-
sies have built up over the years. The fact that we are still wrestling with
what are supposedly the core concepts of the field so many years after the
program was instituted suggests that the basic distinctions that Gibson
made might not have been quite fine enough.
We will focus here on four key areas where the contradictions bite most
acutely. We suggest that the habitat–Umwelt distinction can help resolve
the tensions that have arisen in each of these cases. We focus on
affordances, information, learning, and the social.
Nobody Knows What Affordances Are Anymore

First, a tension has arisen in the literature over how we should think about
affordances: are they dispositional properties, relational properties, or
something else?
Gibson does not get down to defining affordances until Chapter 8 of his
book (see also Wagman, Chapter 8, in this volume). The way he initially
defines affordances there is dependent on the distinction made in Chapter 1.
“The affordances of the environment,” he writes, “are what it offers the
animal …” (Gibson 1979/2015, p. 119). Notice that “the environment” is
here treated as an unproblematic term. Indeed, the way Gibson begins
Chapter 8 seems to imply that affordances are properties of the habitat, in
our terms: they are objective features to be found “out there.” They are a
special way of describing the layout of surfaces and objects, perhaps. They
are to be understood independently of any given animal, and are therefore
not properties of the Umwelt. Thus, Gibson asks, “How do we get from sur-
faces to affordances? And if there is information in light for the perception
of surfaces, is there information for the perception of what they afford?”
Consistent with this way of describing affordances as things that are “out
there,” Turvey (1992) attempted to provide a formal definition of affordances
as dispositional properties belonging to objects, surfaces, and the like. On this
account, affordances are actualized when they come into contact with an
animal that possesses a complementary dispositional property, namely, an
effectivity. A flight of stairs has a property, “climbable,” that is actualized
when a stair-climbing animal comes along and climbs the stairs.
In a similar vein, Reed (1996a, p. 26) insists that affordances “are
aspects of the environment of all organisms, not just features of the
environment of one creature.” Reed proposes that affordances are in fact
resources, or persisting features of the habitat of a species. Because of this,
affordances are able to exert evolutionary selection pressure on popula-
tions across generations.
So that is one story about affordances: they are “out there” properties
that animals can encounter. The problem is that Gibson also suggested an
entirely different way of thinking about affordances. A couple of pages
into Chapter 8, he writes, “an affordance is neither an objective property
nor a subjective property; or it is both if you like … It is equally a fact of
the environment and a fact of behavior” (Gibson 1979/2015, p. 121).
Again, summarizing the chapter, Gibson writes, “Affordances are prop-
erties taken with reference to the observer. They are neither physical nor
phenomenal” (p. 135).
This rather complicates matters. Now it seems as if Gibson is insisting
that affordances are, after all, properties of the Umwelt of a given organ-
ism, in addition to being simply “out there” as properties of the habitat.
Attempting to demystify this neither-subjective-nor-objective quality of
affordances, Chemero (2003) proposed that affordances should be treated
not as simple “out there” properties of the habitat, but as relational prop-
erties that arise, or can potentially arise, in the encounter of an animal and
its surroundings (see also Stoffregen, 2003a). One advantage of this way of
thinking is that it allows us to talk about differences between individual
members of a species. The plane affords flying to the pilot but not to the
child because the affordance is defined relative to the individual’s skills or
abilities (Chemero, 2009).
Other theorists have expanded on this affordances-as-relations concept.
Withagen et al. (2012) argued that affordances do not merely exist as bare
possibilities for action, but can sometimes invite or solicit particular behav-
iors, as when a furniture store is arranged so as to guide customers through
it along a particular path. Erik Rietveld and his colleagues propose that we
should think of our surroundings as a “landscape of affordances,” that is,
as a whole set of possibilities that constitute our ecological niche (Rietveld
& Kiverstein, 2014). This is to be distinguished from the “field of
affordances” or the finite set of possibilities that are available to us at a
given moment, those that stand out as relevant to us (Bruineberg &
Rietveld, 2014; van Dijk & Rietveld, 2017).
It remains an outstanding question whether a single formalization of the
affordance concept can capture everything that Gibson intended it to
explain (Michaels, 2003). For the present purposes, we suggest that at least
some of the disagreement in the field can be resolved by pointing out that
affordances serve a different purpose, depending on whether we are invok-
ing them as properties of the habitat or as properties of the Umwelt. In the
former case, they are dispositional properties or resources, they exist inde-
pendently of any given animal, and they exert selection pressure. In the
latter case, they are relational properties, they depend for their existence
on the continued existence of a particular living animal, and they change
as that animal develops new abilities and skills (and, conversely, they dis-
appear as the animal “forgets” those skills or as its abilities degenerate
with age or injury).
We can have all of the features of affordances-as-resources without
rejecting the features of affordances-as-relations, and vice versa. These two
accounts of affordances are not in fact in conflict with one another. One is
proper to the habitat, the other is proper to the Umwelt.
Nobody Knows What Information Is Anymore

A second outstanding tension has arisen in the context of attempts to under-
stand Gibson’s ecological conception of information. Gibsonians have tradi-
tionally understood information as structure in energy arrays that serves a
dual role: it is both about the structure in the world that caused the pattern
in the array, and it is for an active organism, i.e., it serves to guide action in
a concrete sense (Michaels & Carello, 1981, p. 37).
This dual conception of information originates in the attempt by Gib-
son’s followers to formalize Gibson’s approach into a workable scientific
program (Turvey, Shaw, Reed, & Mace, 1981). Information, on this
approach, should be understood as the central term in a 1:1:1 relation
standing between structure in the world and structure in perception (Shaw
& McIntyre, 1974; see Chemero, 2009, p. 111). Chemero (2009) refers to
this two-way specifying relation as the symmetry principle. An important
claim is that this relation can be read in both directions (Baggs &
Chemero, 2018). Starting at one end: structure in the world specifies how
energy (light, sound waves, etc.) is structured, and this in turn specifies
what an animal perceives. Information in energy is information about the
structure in the world, and the perceptual system samples this information.
Starting at the other end: an animal is understood as an active explorer of
its surroundings. In order to survive, an animal needs to seek out places
and resources that are favorable to its ongoing existence. To do this, it
must continually move about such that it is always perceiving a set of sur-
roundings that are compatible with its needs. Perception can serve this role
because of the chain of specifying relations between perception, informa-
tion, and the world: the content of the animal’s perception specifies the
structure in surrounding energy arrays, and this structure in turn specifies
whether or not there exists structure in the world to support the animal’s
ongoing existence. As the animal moves around, it has to seek informa-
tion for its own purposes
A worry here is that this way of describing perception might be simply
too rigid. Meaning is understood as a purely external phenomenon, as
being located entirely “out there.” This seems to leave the animal with a
limited role: to be a living animal is simply to respond to structure that one
comes across. Indeed, this property of the ecological approach has proved
to be off-putting even to some working in neighboring non-representational
approaches to cognition, whom one might think would be sympathetic.
For instance, Varela et al. (1991) dismiss Gibson’s approach on the
grounds that Gibson’s conception of information relies on an unresolved
dualism: because meaning is understood to be external, they suggest, there
remains an explanatory gap between structure in the world and the biolog-
ical and phenomenological processes that allow a living animal to make
use of that structure. Gibsonians, these authors assert, are attempting to
build a “theory of perception entirely from the side of the environment”
(Varela, Thompson, & Rosch, 1991, p. 204). The suggestion is that a
theory that contains both information-about and information-for is a
theory that maintains a mind–body dualism.
One strategy for answering this critique has been to concede that
information-about is a problematic concept, because it implies meaningful
content “out there” that somehow has to be integrated into the living
perception–action processes of an exploring animal. Van Dijk et al. (2015)
argue that all we need is information-for:
“Information for” calls attention to the fact that ecological informa-

tion need not be about anything—has no “aboutness”—prior to use,
but it is for something to an active animal. It is for perceiving the
environment, for acting on affordances and the likes.
We do not endorse this strategy.

Another strategy is to abandon the symmetry principle and allow that
information can carry structure that is about more than simply physical
surfaces and their layout. Several authors have proposed that the concept
of information should be expanded so that it can account for things that
we perceive but that do not specify structure in the world in the rigid,
lawful ways demanded by the symmetry principle (Bruineberg, Chemero,
& Rietveld, 2018; Chemero, 2009; Golonka, 2015; Withagen, 2004; With-
agen & Chemero, 2009). According to these proposals, some information
is conventional information. This allows, for example, that the label on a
can of beer is informative about the can’s contents because the can exists
within an ongoing system of social practices. This notion of conventional
information means that information is in part dependent on our activities
as a community of convention-makers and convention-observers. This
seems like a necessary move if one wishes to develop an ecological
approach to deal with cultural and language-involving phenomena. To
capture the nature of linguistic phenomena, however, we will need an
account of how we create and use linguistic structures, in addition to an
account of how we recognize that objects such as beer labels have mean-
ings about things beyond what they are made of. We will recommend
below that the appropriate move is to treat language as a social phenom-
enon, i.e., as an interpersonal process not reducible to a perceptual
process.
For now, we note that another possibility exists which may allow us to
maintain all three concepts discussed here: information-about, information-
for, and the symmetry principle (for non-conventional information). The
solution is to understand information-about as a property of the habitat,
while information-for should be understood as a property of the Umwelt.
There is indeed structure in energy arrays that can potentially be used to
guide activity. A fruit pie cooling on a windowsill emits a chemical trail
into the air that can be detected by an animal with the appropriate olfac-
tory system. There is information about the pie that can be said to exist in
the habitat of animals of that type. The chemical composition of the pie is
specified in the chemicals diffused in the air, as required by the symmetry
principle and unlike in the case of the beer label. When a particular animal
does detect the chemical trail and starts to navigate up the scent gradient
toward the pie, we should understand the animal as using information for
its own pie-seeking purposes. It is both proper to say that the pie is speci-
fied in patterns in the energy array, and that information is actively sought
by a living animal.
How Do Animals Learn? How Do We Invent?

A third tension arises from a lack of clarity over the status of learning in
Gibson’s account. This tension can be traced directly to the decision made
by James J. Gibson and his wife Eleanor J. Gibson in the 1960s to divide
their research efforts: the former would focus on the senses while the latter
would focus on perceptual learning (Gibson, 1966a, p. viii). This division,
made between the Gibsons for their own convenience, has had a long-
lasting consequence: the developmental part of the ecological account has
ever since remained somewhat separate from the core perception–action
program (Rader, 2018).
To understand the tension here, it is useful to turn to a foundational
paper in perceptual learning. Gibson and Gibson (1955) argue that the tra-
ditional way of framing the learning process is incorrect. The traditional
view has it that learning about the world is building up and maintaining
an internal store of knowledge about external facts: learning is a process of
enrichment. The Gibsons propose that a more useful way of thinking
about learning is to acknowledge that what the animal comes into contact
with—the stimulus—is already richly structured, and the task that the
learner is faced with is to seek out distinguishing features in the stimulus
that allow the learner to recognize things in the world as being different
from one another: learning is a process of differentiation. The go-to
examples here are how wine experts come to recognize differences in taste
between different wines, and how workers tasked with categorizing the sex
of newly hatched chicks come to recognize male from female. In both
instances, the learning process seems to be highly implicit: skilled perceiv-
ers in these tasks may not necessarily be able to articulate precisely what it
is that allows them to carry out their categorizations. Yet they are discrimi-
nating, and they must be doing so by having learned to attend to some
variable, or set of variables, in the perceptual stimulus.
The problem here is that while what these expert perceivers are doing is
discriminating perceptual structure, not enriching an internal model, this
does not actually constitute an account of the learning process. Indeed, in
a sense, it is misleading, because from a phenomenological standpoint it
remains true that learning results in a richer experience of the world. The
pilot who has learned how to fly the plane has a richer set of behavioral
possibilities open to her, she has a richer Umwelt, even if this is arrived at
in part through learning to perceptually discriminate between the levers
and buttons on the flight control panel. We should even acknowledge here
that this enrichment is possible because of changes in the organization of
the pilot’s body and nervous system, though of course this does not imply
that the pilot has built a second, mental version of the control panel inside
her head that she is using to control the plane. There is just more in the
pilot’s Umwelt.
In recent years, several authors have developed a theory of “direct learn-
ing” which attempts to formalize the discrimination viewpoint and turn it
into a workable empirical program (Jacobs & Michaels, 2007; Jacobs,
Silva, & Calvo, 2009). The innovation here is the suggestion that the
optimum variable that an actor should be attending to, in learning to carry
out some task, is already specified in the information space of possible
solutions to the task. To become an expert in some task, the actor has to
explore this information space and seek a trajectory toward some optimum
solution. This is a powerful view of learning, but it is a view that is still
dependent on there being pre-existing structure “out there” that the animal
is able to explore. It is hard to see how this same approach might be
applied to the kind of generative behaviors that children routinely engage
in, namely, those involving language or pretend play. Children do not
merely explore a space of pre-given solutions, but invent and create.
The developmental story remains an underdeveloped part of the ecolo-
gical approach. For now we note only that the Gibsons’ dichotomy
between differentiation and enrichment accounts of perceptual learning is
not fine enough. Again it is useful to think of the differences between the
habitat and the Umwelt. In the habitat, a space of possible actions and dis-
criminations exists before any particular organism comes into existence,
and this information space may well serve as the target of learning. In the
Umwelt, meanwhile, the world comes to have shape in part because of
changes in the animal: learning does indeed result in an enrichment of
possibilities from the perspective of the living animal.
Can We Have a Gibsonian Account of the Social?

The fourth issue we wish to discuss here is the problem of reconciling Gib-
son’s approach with an account of social phenomena. Owing largely to the
work of Harry Heft, effort in this area has been focused on taking Gib-
son’s theory of perception as a starting point and trying to unite it with the
“other” ecological psychology, that of Roger Barker and his colleagues
(Barker, 1968; Heft, 2001; McGann, 2014).
Barker’s project was to attempt to classify and understand the behaviors
that a typical person engages in while carrying out their daily life. He and
his team of researchers investigated this by observing people’s actual
behavior in non-laboratory settings, namely, in a small town in Kansas
over a number of years. The central insight of this project is that a person’s
behavior is best predicted at a given instant not by what other events have
just occurred, as is implied in the methodology of the stimulus–response
psychology that was dominant at the time. Behavior is instead best pre-
dicted by the places where those behaviors occur. A given child’s behavior
in a classroom or a candy store will closely resemble that of her peers when
her peers are in the same behavior setting. To know how the child will
likely behave, it is more useful to know where she is right now than it is to
know what kinds of behaviors she has engaged in earlier in the day.
The tension here is this. Perceiving a behavior setting, say, a reading
group, as a specific instance of that behavior settings seems to require that
we as individual actors and observers in some sense stand outside that
setting, that we categorize it as an instance of “reading group” before we
are able to enter into it and participate. But this cannot be correct. Reading
groups do not exist simply as things that are furnished by the natural
world. Rather, they exist because we enact them. How can it be that such
settings are at once things that we encounter and things that we create?
In a recent paper, Heft (2018) briefly acknowledges this tension:
[A] behavior setting is an emergent property of interdependent pat-

terns of action over time among individuals and “milieu,” using Bark-
er’s terminology. From this vantage point, it can be seen that a
particular child does not enter into the behavior setting, as an indi-
vidual might enter an enclosure, but rather that the child joins a behav-
ior setting as a participant and in doing so contributes to its ongoing
functioning.
This description still does not quite resolve the tension, however. To join a
behavior setting is still to enter into something that is already ongoing.
This still leaves unexplained how it is that new behavior settings ever
emerge, as they must.
Here it is again helpful to invoke the habitat–Umwelt distinction. The
resolution lies in allowing that there is indeed structure in the habitat:
behavior settings exist as discoverable, observable units of collective behav-
ior. There are post offices and train stations, and French classes. But in the
Umwelt we do not need to perceive that we are in a given behavior setting.
Rather, the behavior settings that exist around us are what we grow into.
The Umwelt of a given individual is shaped by the places where that indi-
vidual dwells, and by the history of interactions that the individual
participates in.
Costall (1995) argued that the social does not have to be so much
incorporated into the ecological approach as demystified. This argument
remains a valuable one. In the Umwelt of a child, everything is already
social. In fact, the social comes first. The structure of the child’s surround-
ings is, prenatally, entirely constituted by and mediated through its moth-
er’s body. Following birth, the child’s surroundings are shaped by the
ongoing social practices of caregivers and community. Only through many
years of education and learning can the child come to master its settings
and take control of its interactions in a manner we are inclined to label
“adult.”
In the case of language, we are both confronted with a pre-existing set
of linguistic conventions and resources (a community of speakers of a lan-
guage), and we are enactors of that language. A child language learner
must learn to control her or his own linguistic behavior with reference to
the activities she or he engages in, progressing from babbling to vocalizing
words to skillfully using novel syntactic structures as a means of directing
the attention of self and others. It is reasonable to say that certain aspects
of a natural language are properties of the habitat: dictionaries, grammars,
prejudices about particular dialects, and ways of talking, etc. We should
not, however, jump to the conclusion that an individual’s “personal” lan-
guage is simply a property of her or his Umwelt. Language is not simply
“out there,” from the perspective of an individual speaker. It is not simply
something we perceive, but also a form of acting (Baggs, 2015). As adults,
our language is so thoroughly integrated into our self-regulation processes
and thinking that it must be something other than simply a set of “exter-
nal” resources. Take counting. When, as children, we learn to count, we
start by reproducing an auditory sequence that is modeled by more skilled
speakers around us, “one, two, three, four….” We reproduce this sequence
in interaction with other people. Later, this same sequence can serve to
structure our own activities: tracking sets of objects, telling time, meas-
uring, etc. What began as a social phenomenon has entered into the
process of self-regulated activity. In early childhood this behavior is
enacted “out loud.” As the child becomes more skilled, she or he learns to
count “in her or his head.” Language has to be “internalized,” in the sense
that what starts off as an entirely public behavior in the child comes, over
time, to be integrated into her or his self-regulation processes, in turn,
enabling new behaviors that were not previously available (Vygotsky,
1978). The problem of explaining how the social comes to enter into indi-
vidual self-regulation processes is the problem of explaining the qualitative
shift that separates the human form of life from that of other species.
The Ecological Approach as a Theory of the Structure of the

Habitat
All of this leads us to a point where we must re-evaluate exactly what
Gibson is trying to do in The Ecological Approach to Visual Perception.
Of course, in a basic sense, he is simply writing a book about visual per-
ception. But in another sense, he is attempting something much more than
that. According to Reed (1988, p. 2), Gibson concentrated for so many
years on direct perception because he believed “that a breakthrough in the
understanding of perceptual awareness and knowledge would carry in its
wake a new approach to the whole of psychology.” Gibson’s followers
have pursued this broader aim: the ecological approach as a new approach
to psychology itself. Indeed, when Gibsonians eventually set up their own
journal, they called it Ecological Psychology, the suggestion being,
perhaps, that the approach had already matured into a fully elaborated sci-
entific research program.
Gibson himself, though, never used the term “ecological psychology,”
at least not as far as we are aware, although he did contribute to a collec-
tion put together by his followers whose subtitle was ‘Towards an Ecolo-
gical Psychology’ (Shaw & Bransford, 1977). It remains unclear in what
sense it is appropriate to talk of an “ecological psychology” at all. In the
opening sentence of his Principles of Psychology, William James provides
the following definition: ‘Psychology is the Science of Mental Life, both of
its phenomena and of their conditions’ (James, 1890). Following this defi-
nition, Gibson’s Ecological Approach can perhaps be read as an unpreced-
ented account of the conditions of mental life: it is an account of the
structure of things “out there”—the structure that an animal can poten-
tially come into contact with. What it generally is not is an account of
what animals actually do when they come into contact with their
surroundings.
Another way of putting this is to say that Gibson gives an account of
the structure of the habitat. Surfaces, layout, and points of observation in
an ambient optic array: these are all idealized descriptions of the structures
available to a potential animal. Indeed, it is arguably the case that what
Gibsonians have been doing ever since is describing the structure of behav-
ior in the habitat. Take the well-studied example of braking to avoid a
c ollision (Fajen, 2005a). The aim of this research program has been to
uncover optimal control laws or strategies that a person can use to control
their braking. In other words, the aim is to characterize the information
space of a task. Warren (2006) notes that this strategy imposes a limit on
the level of predictive explanatory power that we can hope to achieve:
“researchers may need to be satisfied with theories that capture the dynam-
ical ‘deep structure’ of behavior—the morphology of attractors, repellers,
and bifurcations for a given task—rather than one that can precisely
predict individual behavior on particular occasions.”
Another way to understand this description of the methodology in terms
of “deep structure” is to appreciate that control laws reside in the habitat.
What is not included is an account of what occurs whenever a particular
living thing actually encounters some piece of worldly structure and
attempts to act with reference to it (though see Fajen, 2008, for a discus-
sion of learning in the specific case of braking to avoid a collision).
In recent years, a number of authors have been attempting to combine
Gibson’s insights with those of the enactivist approach instituted by Mat-
urana and Varela (Maturana & Varela, 1980; Varela et al., 1991); the
aim here is to establish a full-blown post-cognitivist science of the mind
(Chemero, 2009; Di Paolo, Buhrmann, & Barandiaran, 2017; Fultot,
Nie, & Carello, 2016; Hutto & Myin, 2017; McGann, 2014; van Dijk,
Withagen, & Bongers, 2015). The central insight of the enactivist
approach is that mind is a living process (Thompson, 2007). That is,
mental activity is self-producing, in the sense that the organism produces
and maintains a boundary between itself and the world; it is asymmetri-
cal in the sense that the organism does something to its surroundings
across the boundary that it has itself established; and it is normative in
the sense that the animal acts in accordance with norms that are estab-
lished, for example, by the biological need to act in an adaptive manner
(Baggs, 2018; Di Paolo et al., 2017).
The combined ecological–enactive program has yet to fully take off. It is
possible that long-standing confusion over the scope of the ecological
approach has been the problem here. It may be important to recognize that
the bulk of research in the ecological approach has been targeted at the
habitat, that is, at describing structure as it exists for an ideal member of a
species. The enactive approach, by contrast, aims to provide a theory of
the animal-in-its-Umwelt. Properly understood, there is no reason why
these two approaches cannot inform and support one another. (For a
longer version of this argument, see Baggs & Chemero, 2018.)
Can we still be realists, if we are to talk about individual animals in
their Umwelts? Gibsonians have always considered themselves to be real-
ists, in the sense that perception–action is said to be targeted directly at
structure “out there” in the world, not mediated via some mental copy of
that structure. Enactivists, by contrast, typically align themselves with con-
structivism, which is an explicitly anti-realist position, according to which
living organisms bring forth their surroundings from a subjective point of
observation (Riegler, 2005).
We believe it is possible to resolve even this contradiction. The solution
will be to pursue a dialectical version of realism (cf. Chapter 6 of Di Paolo
et al., 2017). Structure exists at the habitat level, and this structure exists
independent of any single living organism. For a living organism, mean-
while, the world is viewed from a particular perspective. In the course of
our activities we gradually uncover more of reality by a process of learning
and interaction, and at the same time, through our activities, we alter the
structure of the habitat for ourselves and those around us. The Umwelt is
not an inner copy of the habitat, but it is the habitat considered from the
point of view of the ongoing activity of a particular living organism.
Practical Ecological Interventions

The success of a behavioral science research program should ultimately be
measured by what it can be used for. The distinction we are proposing
provides a useful way of categorizing the kinds of things we might be able
to do with such a science. The habitat/Umwelt distinction implies two
ways in which the ecological approach can inform practical interventions
in everyday life. First, we can reconfigure the habitat in order to make it
easier for actors to carry out some task. Second, we can reconfigure the
animal by educating them to attend to their surroundings, their Umwelt, in
a particular way, while leaving the habitat unchanged.
We focus on a single example here: interventions designed to encourage
safer road crossing. Existing work within the ecological approach has
attempted to address this problem in the Umwelt. Lee et al. (1984) describe
a training system to teach children how to safely cross existing roads. Lee
and his colleagues set up a “pretend road” on the sidewalk alongside an
actual road, and asked children to cross it “as if crossing the adjacent road
in the face of oncoming vehicles.” The authors argue that this is a useful
training regime, and they conclude that “children should be trained in
crossing in the presence of traffic at an early age.” Here is a curious point.
The empirical program in the ecological approach has largely targeted the
habitat, as described above. When Gibsonians have attempted to apply
their insights outside the laboratory, however, they have tended to treat
the habitat as unchanging, and have instead targeted the Umwelt as the
locus of intervention. Lee et al. (1984) treat roads as if they were a natural
feature of reality to which children must be trained to adapt.
An underexplored type of intervention is in the habitat. Marshall (2018)
asks a useful question here: why are roads so much safer in Australia than
they are in the United States, as measured by fatality statistics (the fatality
rate in the US is more than twice as high)? The discrepancy appears to be
down to differences in the structure of the habitat. Partly, it is argued,
there is stricter enforcement of regulations in Australia. But the roads are
also physically designed in a different way in the two countries. In Australia
there are a great deal many more roundabouts at places where in the US there
would be a light-controlled four-way intersection. Roundabouts are inher-
ently safer because they do not give rise to opportunities for the most severe
types of inter-vehicle collisions: head-on and T-bone crashes (Rodegerdts et
al., 2010). The roads in Australia are also generally less wide, meaning that
pedestrians have less width to cross, and drivers are more cautious in their
speed because a narrower travel corridor makes greater demands on drivers
in terms of accurately controlling their position in the road.
The tools of the ecological approach are well suited to capturing why
certain features of road infrastructure are more successful, i.e., safer, than
others. More effort should be devoted to exploring interventions in habi-
tats outside the laboratory using these tools.
Conclusion
In this chapter we have argued that Gibson’s distinction between the phys-
ical world and the animal environment should be refined. The latter term
should be subdivided. We should appreciate the difference between the
species-specific habitat and the animal-specific Umwelt.
Making clear this distinction allows us to resolve several long-standing
tensions in the field. The notions of affordance and information can be
better understood if we allow that habitat resources are different from
active relational engagements, and that arrays of energy are different from
self-regulating organisms actively seeking out structured patterns. These
notions are not in conflict with one another, but are complementary.
Similarly, the phenomena of learning and social interaction can be better
understood by keeping distinct the setting of behavior and the process
through which that behavior comes to take shape over time as an indi-
vidual becomes a skillful participant in a social practice.
One last question: does this mean that we have to stop talking about
“animal-environment systems?” If the term “environment” is ambiguous,
then should we abandon it in favor of more accurate terms—“species–
habitat” and “animal-Umwelt” systems? This does not strike us as an attrac-
tive conclusion. We like the phrase “animal-environment system.” We are
used to it. It is part of the family. For the purposes of this chapter, we have
avoided talking about “the environment.” But perhaps the term need not be
problematic as long as its meaning is appropriately constrained by context.
Acknowledgments
Edward Baggs was supported by funding from the European Union’s
Horizon 2020 research and innovation program under grant agreement
No 706432. Anthony Chemero was supported by the Charles Phelps Taft
Research Center.
2 The Triad of Medium, Substance,
and Surfaces for the Theory of
Further Scrutiny
Tetsushi Nonaka
Logic works perfectly well once mankind has developed adequate language.
But logic is helpless if it has to develop this adequate language … As for the
formulation of adequate logic, there must be a language which does not
impoverish the real situation. It is terrible that in our technocratic age we
do not doubt the initial basic principles. But when these principles become
the basis for constructing either a trivial or finely developed model, then the
model is viewed as a complete substitute for the natural phenomenon itself.
(from the Kyoto Prize Lecture by Israel M. Gelfand, 1989, p. 19. Copyright
2009, Tatiana V. Gelfand and Tatiana I. Gelfand)
Charles Sanders Peirce (1955, p. 6) once remarked that every great work
of science affords some exemplification of the defective state of the art of
reasoning of the time when it was written. Perhaps one of the great ser-
vices of The Ecological Approach to Visual Perception (Gibson,
1979/2015) to psychological science was bringing to the fore a set of
fundamental assumptions which have long been, and perhaps still are,
invisible to psychologists who live amidst them. For generations, we have
been taught that the apprehension of the world depends on the percep-
tion of space and time, and that “perception begins in receptor cells that
are sensitive to one or another kind of stimulus energy” (Kandel,
Schwartz, Jessell, Siegelbaum, & Hudspeth, 2013, p. 445). Such a nar-
rative is so deep in psychology as to be almost unquestionable: Fish don’t
talk about water. But what if these fundamental assumptions are irrele-
vant to the activity of perception that orients the organs of perception
and explores the cluttered environment? What if our perceptual systems,
through evolution, are coordinated to a particular scale of nature that is
at a different level from the description provided by physics of atoms and
objects in space? What if the level of the description of the world that
our perceptual systems fit into is so unique that it deserves investigation
in its own right?
Gibson suggested that psychologists can approach the problem of per-
ception in an entirely different light, once an adequate description of the
22 Tetsushi Nonaka
environment to be perceived which does not impoverish the real situation
has been developed. In Chapter 2 of his 1979 book, “Medium, Substances,
and Surfaces,” Gibson attempted to develop a “new” description of the
environment that our perception and behavior fit into. “It will be unfa-
miliar, and it is not fully developed, but it provides a fresh approach where
the old perplexities do not block the way” (Gibson, 1979/2015, p. xi). The
aim of this chapter is two-fold: to reflect on the development of a “new”
description of the environment—the triad of medium, substances, and sur-
faces, and to highlight its significance to the study of perception and
behavior.
Useful Vision
Almost immediately after the publication of The Senses Considered as Per-
ceptual Systems (Gibson, 1966a), Gibson began working on the revision of
The Perception of the Visual World (Gibson, 1950a), which, as we now
know retrospectively, ended up in an entirely new book (Gibson,
1979/2015). In the James J. Gibson papers (#14–23–1832) at the Division
of Rare and Manuscript Collections of Cornell University Library, New
York, there is a folder entitled “Notes for Revision of Visual World,”
which contains over 50 pages of handwritten notes for the “new book”
written by Gibson in 1967. One of the notes reads as follows:
For New Book (“Useful Vision”?)
The ability to distinguish environmental substances, the forms of

matter, the material composition of the world around us, what things
are made of—the ability to distinguish among the main types and to
identify earth, clay, rock, vegetation, bark, leaves, fur, feathers, skin—
this capacity is extremely important to animals and the pickup of such
information does not depend on a single sense organ or perceptual
system. These natural substances should not be thought of chemically
but ecologically. They are important to animals for what they afford.
They are specified (1) by the texture of the surface and the pigment
color of it (even better than by the “shape” of the “object”). They are
specified (2) by their effluvia, if volatile. And they are specified (3) by
their specific gravity (density), by hardness-softness (resistance to
deformation) roughness-smoothness, and thermal conductivity.
Accordingly, they can be identified by seeing the surface, by smelling
the substance, and by palpating (“feeling”) the substance …
(Gibson, 1967b)
Although this handwritten memo was probably not intended for publica-
tion, its message is clear. A theory of visual perception, first and foremost,
needs to account for “useful vision.” It must account for our remarkably
Medium, Substance, and Surfaces 23
veridical ability to deal with the surfaces and substances, objects and
events of the environment that we are, like it or not, obliged to cope with
and use as a species and as individuals (E. J. Gibson, 1994, p. 503). What
different types of clay, rock, vegetation, bark, leaves, fur, feathers, or skin
afford to us may not be distinguishable at first glance. But they can poten-
tially be identified through the activity of perceiving—looking around, pal-
pating, listening to, or sniffing them. Unlike neural signals converted from
stimuli impinging on receptors, the activity of perceiving, involving adjust-
ments of organs, is a function of a set of meaningful properties of the
environment that are selectively attended to by animals. Naturally, an
effort to study the activity of perception thus conceived demands the
description of the functional referent of perception (Holt, 1915). Other-
wise, it would be like watching a tennis match with half the court occluded
from view. To study useful dimensions of perception, the next logical step
would be to find out the adequate level of the description of the world that
our perception and behavior fit into. An attempt to develop a “new”
description of the environment, it seems, was a natural consequence of
Gibson’s effort at understanding useful vision.
The Triad
It is likely that Gibson wrote the first draft of Part I of the new book—
“The Environment to be Perceived”—between January and November in
1971, and came up with the title of the book, An Ecological Approach to
Visual Perception (albeit beginning with “An” instead of “The”) during
this period. On January 12, 1971, Gibson (1971a) sent a tentative outline
of the new book to the editor at the Houghton Mifflin Company that was
quite different from the final version of the book: the book was then enti-
tled Everyday Visual Perception, and the Part I of the book was “A New
Theory of Perception,” for which Gibson left notes that contrast a theory
of sensation-based perception and that of information-based perception
(e.g., Gibson, 1967c).
1971 seems to have been a key year in the development of ideas that
culminated in Gibson’s (1979/2015) final book. It was when a series of
notes on affordances (Gibson, 1982a) and “Do We Ever See Light?”
(Gibson, 1971b) that later constitute the important parts of the 1979 book
were written, and the phrase “an ecological approach to visual perception”
appeared in a note (Gibson, 1971c). The use of the term “ecological psych-
ology” by Gibson is also found in the note entitled “A Preliminary List of
Postulates for an Ecological Psychology,” written in June, 1971 (Gibson,
1971d). After writing a series of notes, by November 1971, Gibson had
written up an early version of the new Part I, “The Environment to be Per-
ceived,” which included all the basic contents of the finished version of the
part, including the triad of medium, substances, and surfaces and the nine
ecological laws of surfaces (Gibson, 1971e).
24 Tetsushi Nonaka
Normal perception involves the possibility of further exploration, which
we are aware of whether or not the possibility is taken advantage of
(Gibson, 1978a). But, what makes us aware of the possibility of further
exploration in the first place? What makes us aware of the layout of the
environment in and out of sight? What makes it possible for animals to
discover the potentially meaningful features of the environment that have
not yet been taken advantage of? These are the questions that share the
same fundamental issue which cannot be resolved without restoring the
active observer to the world in a way physics never did (Gibson, 1973).
Gibson’s following thought experiment illustrates well what gets lost in the
description of the world by physics in terms of space, time, matter, and
energy: What if a wholly passive animal were in a wholly frozen world?
Or, conversely, what if an animal were in “an environment that was
changing in all parts and was wholly variant, consisting only of swirling
clouds of matter” (Gibson, 1979/2015, p. 10)? In both cases, it would be
impossible for the animal to disentangle a set of variables that are specific
to the world out there (i.e., independent of the point of observation). These
hypothetical worlds are not the environment for perceiving animals. But,
note that “in both extreme cases there would be space, time, matter, and
energy” (p. 10).
Restoration of the active observer that scrutinizes the environment
requires a fundamental reworking of the description of the world, or in
Gibson’s (1971f ) words, “the permutation of the orchard with Newton’s
apple!” The crux of this permutation was the replacement of matter and
bodies in empty space with the triad of (1) medium (the gaseous atmo-
sphere); (2) substances that are more or less substantial; and (3) surfaces
that separate the substances from the medium (Gibson, 1979/2015, p. 27).
The Medium
The recognition of the air as a medium (for terrestrial animals) allows the
distinction between potential and effective stimulation. Radiant energy,
acoustic energy, and chemical energy are propagated through the medium,
which provides the ambient sea of stimulus energy in which animals can
move about. Instead of inquiring whether one model of inferring the causes
of sensation aroused by stimuli is better than another, with the notion of
medium, we can now begin to study activity before sensations have been
aroused by stimuli, an activity that orients the organs of perception and
explores the sea of potential stimulation for the information external to the
perceiver (Gibson, 1982b, p. 398). Unlike points in space defined by an
arbitrary frame of reference, the ambient energy array surrounding each
potential point of observation is unique (Gibson, 1979/2015, p. 13). As the
observer moves from one point of observation to another, the optical
array, the acoustic array, and the chemical array are transformed accord-
ingly (p. 13). This provides the opportunities for an active observer to
move in the medium to detect invariants underlying the transforming per-
spectives in the ambient array surrounding a moving point of observation.
Among the recent advances that have furthered our understanding of
the notion of medium for perceiving animals is the insight provided by
Turvey and Fonseca (2014), whose research, probably for the first time
since Aristotle (1907), brought to light the problem of medium in haptic
perception. They hypothesized that interconnected structural hierarchies
composed of tensionally prestressed networks of our bodies that span from
the macroscale to the microscale—from muscles, tendons, and other con-
nective tissues to various micro-elastic structures such as a network of col-
lagen fibers—constitute the medium for the haptic sense organs of animals
(Turvey & Fonseca, 2014). Like the air being the medium for sound, odor,
and reverberating flux of light, despite being on the other side of the skin,
the presence of isometric tension distributed throughout all levels of inter-
connected, multiscale networks make available the opportunities for an
active perceiver to spontaneously transform the distribution of forces
throughout the tensionally integrated system in such a way as to detect the
invariant patterns that specify the source of mechanical disturbances.
Turvey and Fonseca’s (2014) rediscovery of the medium of haptic per-
ception resonates with the recent surge of interest in the mechanical basis
of information and pattern formation in a wide range of fields—
mechanobiology, soft robotics, sensory ecology, and rheology (e.g.,
Hanke, 2014; Ingber, 2006; Iwamoto, Ueyama, & Kobayashi, 2014;
Rieffel, Valero-Cuevas, & Lipson, 2010). Because the form of any struc-
ture, whether a vortex flow of water or a living tissue, is determined
through a dynamic interplay of physical forces, the distinct pattern of
forces characteristic of a mechanical disturbance may convey a physical
form of information that constrains perception and the behavior of an
agent (Ingber, 2005). One good example of this is the hydrodynamic per-
ception by aquatic animals (Hanke, 2014). Harbor seals, for instance, are
known to use their vibrissae to haptically discriminate the water move-
ments left behind by prey or predator that have passed by at an earlier
point in time, and perceive the motion path, size and shape of the object
that caused the trail (Hanke, Wieskotten, Marshall, & Dehnhardt, 2013)
(Figure 2.1). A point worth emphasizing is the fact that although the
informative patterns of water movement are there to be perceived by an
animal, there are many reasons that the animal may not attend to the
information. The harbor seal running away from the white shark may
not attend to a pattern of water movement that specifies the presence of
salmon that can be preyed upon. Near the surface of clear water during
daytime the animal may attend to optical information without taking
advantage of hydrodynamic information. The notion of medium makes
possible for us to recognize this distinction between the existing informa-
tion available to the animal and the information selectively picked up by
the perceptual activity of the animal.
26 Tetsushi Nonaka
Figure 2.1 Water velocity 60 s after an 86 mm-long fish (Lepomis gibbosus) passed

the area. Bold arrow indicates swimming direction.
Source: From Bleckmann et al. (2014): Figure 1.4a, adapted with permission from Springer-
Verlag.
According to Gibson (1979/2015, p. 13), “if we understand the notion

of medium, … we come to an entirely new way of thinking about percep-
tion and behavior.” Recent years have witnessed a growing interest in the
problem concerning the laws of variation in efficient exploratory behavior
to obtain the information about aspects of the environment relevant to the
task at hand (e.g., Boeddeker, Dittmar, Stürzl, & Egelhaaf, 2010; Nonaka
& Bril, 2014; Stephen, Arzamarski, & Michaels, 2010; Viswanathan, Da
Luz, Raposo, & Stanley, 2011). In general, exploration requires fluctu-
ations, and fluctuations increase in time. A growing body of research sug-
gests that the fluctuations in exploratory behaviors exhibit the property of
superdiffusion, where the fluctuation grows faster than normal diffusion
governed by a Gaussian probability density function (Nonaka & Bril,
2014; Stephen et al., 2010; Viswanathan et al., 2011).
Nonaka and Bril (2014) studied the exploratory movement of expert
stone beads craftsmen in India who shape a bead by a series of hammer
strikes on a stone held against the pointed tip of an iron bar (Figure 2.2).
In the field experiment, the craftsmen shaped the ellipsoidal beads made
of two different materials (carnelian stone—a familiar material, and
glass—an unfamiliar, much more fragile material) in the workshops
where they normally work. The use of the novel material must require
the acute sensitivity to the properties of the material, where the finer the
exploration, the better the probable outcome of the activities that follow.
In the exploratory tapping movement of the craftsmen during the prepar-
atory phase of the task, they found (1) the presence of long-range corre-
lations where the variance of the displacement time series of the hand
wielding the hammer grows superlinearly in time; and (2) underlying
multiplicative interactions between fluctuations at different temporal
scales indicated by the heterogeneity of scaling properties over time.
When faced with the unfamiliar condition using unusual, fragile material,
the exploratory hammer tapping movement of highly skilled experts who
were able to cope with the situation exhibited a pronounced increase in
the long-range temporal correlations. In contrast, the wielding behavior
of less skilled experts—those who could not shape the glass beads—
exhibited a significant loss of long-range correlations and reduced hetero-
geneity of scaling properties over time, which robustly discriminated the
groups with different skill levels (Figure 2.2). Alterations in multiscale
temporal structure of movement fluctuations were apparently associated
with changes in the situation differently depending on the level of exper-
tise (Nonaka & Bril, 2014).
The empirical evidence derived from this field experiment, albeit a
special case of an unusually complex skill, may well serve the purpose of
constraining the possible accounts of active touch. Traditionally, active
touch is explained by the activity of neurons that compare central motor
commands with peripheral sensory feedback during manual exploration
(Kandel et al., 2013, p. 524)—an account which exclusively focuses on
sensory receptors and the nervous system without reference to the archi-
tecture of the body where they are embedded. But the problem expert
craftsmen face is unlikely to be that of associating central and peripheral
signals. The presence of nonlinearity arising from multiplicative inter-
action across fluctuations at different timescales would greatly compli-
cate such a process, with no simple correspondence between the central
motor command, the generated movement, and the peripheral sensory
feedback that arises as a consequence of the movement. Instead, the
result is a much better fit to the alternative scenario of active touch that
takes into account the medium for the haptic perceptual system (Turvey
& Fonseca, 2014), in which efficacy of active touch depends on the
tuning of the whole system including the multiscale tensile states of the
body, the structures of which are transformed by exploratory behavior in
such a way to discriminate the invariant patterns that specify the source
of mechanical disturbances from all the other patterns that do not specify
the source (Gibson, 1966a, p. 55).
Figure 2.2 (a) Typical posture and movement of craftsmen during stone bead production. (b) Examples of ellipsoidal glass beads pro-
duced by expert (HQ) and non-expert (LQ) craftsman. (c) Singularity spectrum f(a(q)) (–1.4 ≤ q ≤3) estimated for expert (HQ)
and non-expert (LQ) craftsmen in the conditions using carnelian stone (black spheres) and glass (gray spheres) as raw mater-
ials. The vertical and horizontal bars are standard errors of the means for f(q) and a(q), respectively, of multiple realizations of
each condition.
Source: From Nonaka and Bril (2014). Adapted with permission fromthe American Psychological Association.
Substances and Surfaces
Substances refer to the portions of the environment that are in the solid or
semisolid state, which are structured in a hierarchy of nested units—such
as rock, soil, sand, mud, clay, oil, tar, wood, minerals, metal, and the
various tissues of plants and animals (Gibson, 1979/2015, p. 15). Unlike
the medium, the substances differ in all sorts of ways—in hardness, resist-
ance to flow, mass per unit volume, resistance to breaking, the tendency to
regain the previous shape after deformation, the tendency to hold the sub-
sequent shape after deformation, or the degree to which they absorb light
(p. 16). Different substances have different biochemical, physiological, and
behavioral effects on the animal. For example, the affordances of a sub-
stance for human behavior, such as making useful things, depends on these
properties of substances. The ability to distinguish among the different
substances is extremely important to us, and we often succeed in picking
up the relevant information through the process of scrutiny.
The central question is: where is the action in our scrutiny of the sub-
stances of the environment (E. J. Gibson, 1994, p. 503)? Brick, for
example, can be visually or haptically inspected. But no one has ever seen
or touched the inside of a brick (Feynman, 1985). Every time you break
the brick, you only see or feel its surface. Nevertheless, the surface of a
brick has a characteristic texture that specifies what it is made of, and can
be distinguished from other substances by seeing and touching its surface.
Likewise, “the surfaces of the substances from which primitive men fash-
ioned tools have different textures—flint, clay, wood, bone, and fiber”
(Gibson, 1979/2015, pp. 21–22).
The surface is something of fundamental importance to our perception,
especially to visual perception. It is not only because we cannot see the
inside of a thing, but also because our vision fails without the illuminated
surfaces of a thing. In fact, all we ever see is the surfaces of substances, and
the only way we see illumination is by the way of “the surface on which
the beam falls, the cloud, or the particles that are lighted” (p. 48; see also
Carello & Turvey, Chapter 4, in this volume). There is experimental evid-
ence that seeing the surfaces depends on the structure of the ambient array
which has different intensities in different directions (Gibson, Purdy, &
Lawrence, 1955). There is also evidence that the eye would be unfocusable
in homogeneous ambient light (i.e., in the unusual case where light that
surrounds a point of observation would not be different in different direc-
tions). In consequence, “the possessor of the eye could not fix it on any-
thing, and the eye would drift aimlessly” (Gibson, 1979/2015, p. 47). Our
visual system can only be adjusted to and oriented to surfaces, and this
ability relies on the fundamental fact that “all persisting substances have
surfaces, and all surfaces have a layout”—the first of the nine ecological
laws of surfaces proposed by Gibson (1979/2015, p. 19). Furthermore,
“the surface of a substance is where a mechanical action like collision is
30 Tetsushi Nonaka
located, where chemical reactions take place, where vaporization occurs,
or solution, or diffusion into the medium” (p. 86). Then, the answer to the
question raised in the beginning of the foregoing paragraph has to be as
follows: The surface is where the action is—where the activity of percep-
tion is focused, adjusted to, and oriented toward.
We pay great attention to the layouts of substantial surfaces and what
they afford. To paraphrase Reed (1996b, p. 122), for many centuries,
human beings have been chopping, cutting, tying, molding, sharpening,
honing, dyeing, shaping, etching, washing, scrubbing, brushing, sweeping,
raking, shaving, ironing, mowing, polishing, scraping, grinding, and much
more. These are operations that modify the layout of a substantial surface
so as to alter its affordance for human life—its utility or function—which
turn rough into smooth, and make available the edge or the vertices that
affords cutting or piercing. The general problem of how we perceive the
meaningful features of the substances and surfaces of the environment is
an important problem of psychology. There is, however, a lack of experi-
ments that investigate this rich behavioral repertoire involving the scrutiny
of substantial surfaces of the environment. While experimental testing of
behavior with virtual surfaces displayed in the monitor are becoming
increasingly common, it is still rare to discuss the ways in which our per-
ception and behavior differ depending on the reality of the surfaces
involved (i.e., real, material surfaces vs. virtual surfaces).
In what follows, drawing on the research on the ancient skill of stone
tool-making that takes advantage of fracture mechanics exhibited by a spe-
cific type of brittle substance, I hope to illustrate how the focus on sub-
stances and surfaces could shed light on the important problem of human
cognition, and how it could expand the intellectual horizons of ecological
psychology by connecting with other relevant developments in under-
standing human ways of life.
Flaking Stone
The earliest known evidence of the alteration of a surface layout by
humans to change its affordances is the modification of a natural, hard,
and rigid mineral material (cobbles, pebbles, and rock fragments) by means
of percussion with hard stone hammers (Režek, Dibble, McPherron, Braun,
& Lin, 2018). Thanks to its excellent durability, hard stone records very
old traces of physical actions applied to it (Pelegrin, 2005). The archeologi-
cal records clearly show that by around at least 2.6 million years ago (and
likely much earlier, e.g., Harmand et al., 2015; McPherron et al., 2010),
early human species were already habitually fracturing stones so as to alter
their utility or function—stone knapping, as archeologists call it. From the
very beginning, the aim of stone knapping was to obtain a specific layout
of surfaces—the razor-sharp edges—that afford a function which is absent
or extremely rare in the natural world: the cutting function (Roche,
lumenschine, & Shea, 2009). This is inferred from the fact that stone
B
fragments (called flakes), detached from a block of stone (called a core)
found in the old archeological sites, unequivocally display the character-
istic layout of surfaces resulting from the specific fracture mechanism
called a conchoidal fracture. This mechanism leaves razor-sharp cutting
edges and conspicuous bulbs of percussion on the fracture plane that are
unlikely to have been formed naturally (Roche, 2005; Semaw et al., 1997)
(Figure 2.3a).
Conchoidal fracture refers to the phenomenon producing a Herzian
cone. It arises from the fracture of a specific type of brittle substance—a
homogeneous and isotropic crypto-crystalline structure (e.g., flint, fine-
grained silicified sandstone) or glasses (e.g., obsidian) with no preferred
planes of weakness (Pelegrin, 2005). Conchoidal fracture requires loading
at a point near the angular edge of the block of raw material with its
Figure 2.3 (a) Ventral surface of a flake detached by conchoidal fracture. (b) Flake
terminology.
Source: (a) Adapted from “Lithic flake on flint, with its fundamentals elements for technic
description,” Copyright 2006, José-Manuel Benito Álvarez. From José-Manuel Benito
Álvarez, Wikimedia (https://creativecommons.org/licenses/by-sa/2.5/legalcode).
32 Tetsushi Nonaka
e xterior angle (called the exterior platform angle) less than 90 º (Figure
2.3b), and the to-be-flaked exterior surface (called the flaking surface)
needs to be flat or slightly convex to allow for the propagation of the
energy transmitted by the loading event (Pelegrin, 2005).
Imagine you are a prehistoric human. What would you do to obtain
precious cutting tools by fracturing stones? First, you need to look for
natural homogeneous, isotropic, and brittle material that can be fractured
conchoidally to obtain razor-sharp cutting edges, and the hammerstone
that is suitable for aimed striking. In addition, the raw material needs to
have a peculiar natural layout of surfaces with angular edges and a more
or less flat surface (e.g., cobbles with a flat surface as opposed to a convex
one) required for conchoidal fracture. Then, in order to make the most of
the precious raw material (i.e., to produce as many usable flakes as pos-
sible), you need to make sure that the specific layout of surface of the core
that allows further flake removals is preserved after each flake removal.
The foregoing is what has been found in the in-depth analysis of the
traces of stone tool-making behavior by early humans who lived near the
western margin of present-day Lake Turkana in Kenya—the archeological
site of Lokalalei 2C—around 2.34 million years ago (Delagnes & Roche,
2005; Roche et al., 1999). Surprisingly for stone fragments that are so old,
it was possible to re-fit many of these pieces found in in Lokalalei 2C back
together into an original cobble. By re-fitting the flakes, Roche and her col-
leagues examined the sequence of flake removals—the order by which
these flakes were originally removed (Delagnes & Roche, 2005). An early
stone knapper who fractured the cobble in Figure 2.4, for example,
selected a fine-grained phonolite cobble that has a flat surface (Face A) as
opposed to a highly convex surface (Face B) resulting in edges with acute
angles around the flat side of the cobble (indicated by the gray dotted line
around Face A). Roman numerals in Figure 2.4 present the order of a
series of flake removals, and the arrows show the direction of strike with a
hammerstone. Even to a novice’s eyes, it is obvious that the knapper
obtained these flakes not randomly but by following a certain set of rules.
For example, the knapper carried out all the flake removals on the flat face
(Face A) but the final attempt (V on Face B), and aimed the blows at the
edges with acute angles shown by the gray dotted line. In addition, after a
series of flake removals, the knapper switched to the opposite edge on the
same face, and alternately struck flakes from the two opposing edges
(Figure 2.4). Had flaking been carried out from one direction, the flat
surface would have been quickly lost and the remaining core would have
been wasted just after a couple of flake removals. In this example, the
knapper maintained the surface flat by alternating the direction of strike,
thus providing the opportunities for further flake removals to the very end.
It was also found that the cores and flakes in Lokalalei 2C do not show
any impact damage from failed percussions, such as might be caused by
faulty estimation of the opportunities for flaking (Delagnes & Roche,
/s /s
/s
/s
//
ŽƌĞ
ŽƌĞ
/ s
///
///
///
Ϭ ϭ Ϯ ϯ ϰ ϱ Đŵ
&ĂĐĞ &ĂĐĞ
Figure 2.4 Re-fitted elongated ovate fine-grained porphyritic phonolite cobble from Lokalalei
2C. Flaking was carried out on a large and relatively flat natural face (Face A)
from the longest available edge and a shorter adjacent edge, which are the only
portions of the perimeter of the core with suitable natural striking angles (gray
dotted line). The series of flakes were alternately struck from these two edges.
Source: From Delagnes and Roche (2005), Figure 8. Adapted with permission from Elsevier.
2005). Such systematic flake production, seemingly following a certain set

of rules (called débitage), was not the result of trying out multiple ways to
flake stones and at last succeeding in some way.
Seeing the Outcome at the Outset

How might we understand such regularity apparent in the archeological
record of flake removal? Delagnes and Roche (2005, p. 465) wrote, “the
débitage of successive series clearly hinges upon conscious planning.” But
even if such an operational sequence was consciously planned in detail, if
the outcome of each strike was unpredictable and out of control, it would
have been impossible to carry out such an orderly sequence as shown in
Figure 2.4. The French archeologist Texier (1995, p. 652), addressing this
34 Tetsushi Nonaka
point, once wrote, “if the knapper is able to organize a débitage, he has
already a good skill and knows exactly the consequences of a strike given
to such a core.” The fundamental question, then, is not so much how the
early tool-makers consciously planned the order of the core reduction
sequence, but how they foresaw the outcome of each hammerstone strike
at the outset. Without this ability to foresee and control the path of frac-
tures that would result from each strike, a phenomenon such as organized
débitage could never have been realized.
Nonaka, Bril, and Rein (2010) systematically tested the skill of
modern stone knappers to foresee and control the path of fracture that
would result from their own flaking actions. Nonaka et al.’s hypothesis
was that expert knappers could foresee and control the outcome by
tuning their actions to the lawful regularities that exist in a conchoidal
fracture. Previous conchoidal fracture experiments using a protocol in
which a steel ball is dropped on plate glass documented an invariant rela-
tion among particular variables (Dibble & Pelcin, 1995): The size of the
flake depends on the combination of two variables—the angle of the edge
and the distance between that edge and the point of the percussion (see
Figure 2.3b). The force with which the hammer strikes the core does not
affect the flake size once it is above the threshold of fracture initiation
(i.e., the minimum amount of force needed to remove a flake of a given
size). The threshold of fracture initiation depends on flake size, which in
turn depends on the striking location (Dibble & Režek, 2009). Although
they are by no means the only variables that affect the outcome of con-
choidal fracture, importantly, these variables are all under the direct
control of the knapper. Do knappers, prior to the detachment of a flake,
attend to such lawful relations so as to foresee and control the con-
sequence of a strike given to a core?
In the experiment, participants—including prominent replica crafts-
men—were asked to draw the outline of the fracture path on the surface of
a flint core expected to result from the blow they would deliver at the core
(Nonaka et al., 2010). After drawing the outline of the fracture path, the
participants were then asked to proceed to actually fracture the stone as
they had expected. The result of the experiment indicated that the task was
surprisingly difficult. Most knappers who could produce usable flakes
could not control the path of the fracture resulting from the strike given to
a core. But there were a few expert knappers who proved capable of con-
trolling the fracture path almost exactly as they had expected. Confirming
the aforementioned hypothesis, it turned out that the outline of the frac-
ture path drawn prior to the flake removal by those experts who succeeded
in the task already exhibited the lawful relation among the three variables—
the angle of the exterior edge, the distance between the point of percussion
to the exterior edge, and the flake dimensions that reflect the constraints of
conchoidal fracture (see Figure 2.3b). In contrast, no such relation
appeared in the outlines drawn by the other knappers. The result suggests
that those who are able to control conchoidal fracture are focusing their
attention to specific salient features of the layout of stone surfaces that are
lawfully related to the constraints of fracture mechanics. Simply put,
novices intended to detach rather impossible flakes, while experts intended
to detach feasible flakes from the outset.
It was further found that the experts hit the core with a lower kinetic
energy of the hammerstone at impact than the nonexperts, and that only
experts varied the kinetic energy of the hammerstone at impact in relation
to the to-be-detached flake size (Nonaka et al., 2010). Experts were aware
of the properties of a core and the mass of the hammerstone at hand in the
sense that they detached flakes without excessively overshooting the
required kinetic energy at impact. This means that those who controlled
the conchoidal fracture were tuning their action into yet another higher-
order functional relation among the relevant variables of surface layout of
the core, potential flake size, and the required kinetic energy determined by
these variables, which also depended on the mass of the hammerstone.
In summary, foresight in the control of stone flaking was shown to
depend on the continuous participation of the knapper’s behavior in the
lawful regularity of conchoidal fracture. This participation, in turn, is made
possible by the perceptual attunement of the knapper to discriminate a spe-
cific layout of surface of the core that has a consequence on the future frac-
ture event, which is observable but not easily attended to by everyone
(Nonaka et al., 2010). The path of fracture resulting in a flake is entailed by
the natural unfolding of the system of which the behavior of the knapper is a
component part (cf., Stepp & Turvey, 2010). In other words, foresight in
this ancient skill is a matter of focusing the perception and behavior of an
actor on the informative structure that specifies the inevitability of the
environmental event in which the actor takes part. This study directs us to
reconsider the primacy of the rich opportunities provided by the substances
and surfaces of the environment, toward the use of which our perceptual
and action systems continue to evolve and develop (Nonaka, 2012).
Epilogue
A “new” description of the environment to be perceived—the triad of
medium, substances, and substances that allows for both persistence and
change—provides a principled way to frame the existing possibility of
further exploration, of scrutinizing, or of looking more carefully to extract
invariants (Gibson, 1978a). Without taking this possibility into account,
the activity of perceiving would easily get confounded with the activity of
guessing which occurs in a rather atypical situation where further scrutiny
is wholly prevented. This confusion, in turn, would lead to the reduction
of the laws of perception to those of guessing.
Luminous, mechanical, or chemical energy is structured by the substan-
tial environment and becomes ambient in the medium. The ambient sea of
36 Tetsushi Nonaka
energy around each of us is usually very rich in what we call pattern and
change, which provides the inexhaustible reservoir of potentially informa-
tive invariants that lies open to further scrutiny (Gibson, 1979/2015,
p. 233). At this level of description of the environment, what has been
known tacitly is made explicit: The activity of perception is “open-ended,”
and you can keep discovering new features and details about the environ-
ment by the act of scrutiny (p. 245). Unlike guessing based on a few cues
or clues, normal perception is not based on “going beyond the data,” as
long as one can look again, or go back and look again (Gibson, 1978a).
What the triad of medium, substances, and surfaces offers us is a theory of
unlimited further discovery for perception. We will have to make a fresh
start.
Acknowledgments
I thank Tatiana V. Gelfand for giving me the permission to quote I. M.
Gelfand’s words from his Kyoto Prize Lecture as an epigraph. The writing
of this chapter was in part supported by JSPS KAKENHI Grant Numbers
JP18K12013 and JP18KT0079 from the Ministry of Education, Science,
Sports and Culture, Japan, awarded to Tetsushi Nonaka.
3 Ecological Interface Design
Inspired by “The Meaningful
Environment”
Christopher C. Pagano and Brian Day
Physics, optics, anatomy, and physiology describe facts, but not facts at a
level appropriate for the study of perception. In this book I attempt a new
level of description.
(Gibson, 1979/2015, p. xi)
James J. Gibson’s 1979 book, The Ecological Approach to Visual Percep-

tion, remains the seminal piece for the development of ecological psych-
ology. In Chapter 3, “The Meaningful Environment,” Gibson describes the
environment in which animals perceive and act. The primary aim of the
chapter is to develop an ecological nomenclature for surface layout and
objects. Gibson believed that the environment is comprised of surfaces and
mediums, which he describes in Chapter 2 of his book (see Nonaka,
Chapter 2, in this volume), rather than an environment comprised of
formal plane geometry. This distinction is very important for the develop-
ment of an ecological psychology because it describes the environment as
something that is both acted upon and acted within by an organism. In
later chapters, he will define the concept of an affordance as a mutual rela-
tionship between the organism and the environment (see also Chemero,
2003; Stoffregen, 2003a; Turvey, 1992). Chapter 3 of Gibson’s book sets
up the possibility of that mutuality by defining one half of that coupling:
the environment. In what follows we discuss some of the important ideas
in Gibson’s chapter that have been refined and extended in the decades fol-
lowing the original publication of the book. We relate these ideas to
selected topics within human factors, with an emphasis on ecological inter-
face design.
The Meaningful Environment

What is typically seen as the principal contribution of Gibson’s ecological
approach to perception is that his approach makes direct perception pos-
sible; that aspects of the world, such as its three-dimensional nature, can
be perceived without mental representation or mental inference despite the
38 Christopher C. Pagano and Brian Day
seemingly destructive mapping from the world to the two-dimensional
retina (e.g., Lombardo, 1987). This point alone, however, can be made
using traditional plane geometry and traditional physics (e.g., texture gra-
dients, optic flow, etc.). In Chapter 3, “The Meaningful Environment,”
Gibson goes beyond overcoming the destructive mapping for plane geome-
try and he sets up the argument that what is available in the stimulation
that the eye receives is not just information that is lawfully related to the
actual distances and sizes of elements in three-dimensional space. Rather,
this stimulation is lawfully related to the meaning that the terrain, inclines,
enclosures, objects, etc., hold for an organism that interacts with those sur-
faces and objects. Gibson’s push toward meaning is made clear from the
first few lines of the chapter, when he states, “If what we perceived were
the entities of physics and mathematics, meanings would have to be
imposed on them. But if what we perceive are the entities of environmental
science, their meanings can be discovered” (1979/2015, p. 28, original
emphasis). Rather than establishing a viable direct realism for perceptual
properties that can be described by formal plane geometry, which would in
itself be a daunting task, Gibson creates a direct realism for a meaningful
environment. Indeed, defining the environment as meaningful is necessary
for his direct perception. Meaning is given ontological status; it is part of
the actual surrounding world rather than existing merely within the mind
of the perceiver.
Gibson’s description of ecological reality is a fitting way to end Part I of
the book, which focuses on the environment that is coupled to the organ-
ism and perceived by the organism. The aim of Gibson’s third chapter is to
highlight that the environment, comprised of surfaces, is laid out in a way
that has inherent meaning (i.e., affordances) for the behavior of humans
and animals. This approach contrasts starkly with previous conceptions of
the environment as comprised of the abstract concepts of mathematical
space to which meaning must be added by the observer. Without a termi-
nology to describe the inherently meaningful environment to be perceived,
the development of direct perception in Part II of the 1979/2015 book
would not be possible. Gibson conceived of the visual system as the eyes in
the head on a body supported by the ground (e.g., Gibson, 1966a), but
Chapter 3 stresses that this system is embedded in an inherently meaning-
ful environment. If vision is for exploration and discovery, then the logical
starting place is to describe what it is about the environment that needs to
be discovered. Gibson provides this description of the environment in
Chapter 3.
By developing an ecological nomenclature for surface layout and
objects, Gibson establishes that the object of perception is a meaningful
environment. Gibson defines the environment in such a way that it can
serve as one half of the animal-environment mutuality that he describes in
later chapters. He goes beyond a direct perception of surface geometry by
incorporating a requisite direct perception of meaning. In later chapters,
Ecological Interface Design 39
the “meaning” of environmental surfaces described in Chapter 3 will come
to be understood as affordances (see Wagman, Chapter 8, in this volume).
It is often assumed that Gibson’s idea of direct perception made possible
his concept of affordances. However, Chapter 3 makes it clear that it is the
meaningfulness of the environment that makes direct perception possible.
Since this meaningfulness is inherent in affordances, it is actually Gibson’s
concept of affordances that makes direct realism possible. That affordances
precede direct perception is illustrated by the way some researchers have
accepted a rudimentary concept of affordances, and applied it fruitfully,
while continuing to reject the possibility of direct perception (e.g., Norman,
2013). Conversely, it does not seem possible to accept Gibson’s direct
realism and reject the possibility of affordances.
For Gibson, ecological reality consists of meaningful things to be dis-
covered. Gibson consistently uses the word “discovered” to illustrate how
meaning is uncovered in an environment, rather than having to be imposed
upon an environment by the observer. Situating the environment as inher-
ently meaningful lays the basis for a claim he makes later that value is
inherent in the animal’s relationship with the environment: “The perceiv-
ing of an affordance is … a process of perceiving a value-rich ecological
object. Any substance, any surface, any layout has some affordance for
benefit or injury to someone. Physics may be value-free, but ecology is not”
(Gibson, 1979/2015, pp. 131–132). Recognizing the environment as mean-
ingful and describing aspects of the environment (e.g., surfaces) that can be
perceived lays the groundwork for affordances. In Chapter 3, Gibson uses
the concept of affordances to discuss the opportunities that are available to
an actor in an environment. The main aim of his chapter is to offer a
description of the environment that leads logically to the concept of
affordances so that his direct realism may follow.
A Nomenclature for Surface Layout

Gibson contrasts the words used to describe a geometric layout with those
used to describe an ecological environment or habitat to move from
abstract geometry toward a surface geometry. For example, Gibson pro-
poses a terminology for a theory of surface layout that includes ground,
open environment, enclosure, detached and attached objects, edges, and
place, among others. He then picks out certain features of the environment
that are pertinent to all animals that use visual information to locomote
and otherwise act upon the environment.
Many of these features of the environment have been studied exten-
sively, such as the ground (Matthis & Fajen, 2014), stairs (Konczak,
Meeuwsen, & Cress, 1992; Warren, 1995), doorways and apertures
(Wagman & Malek, 2008, 2009; Warren & Whang, 1987), and gaps in
the support surface (Burton, 1992). Place has received much less attention.
However, recently there has been a renewed call to subject place to the
same empirical scrutiny and investigation as other environmental features
(Heft, 2018).
Gibson also touches on fire and water. He treats fire as an ecological
event, connecting it with his descriptions of other events that urge moving
from a conception of time passage to that of event passage (Gibson, 1975;
James, 1892). This shift is consistent with his call to move from describing
the environment with ecological terms rather than with geometric terms.
For Gibson, fire is simultaneously a substance and an event that has
various meanings and uses. The same is true for water, which can be con-
sidered as a surface, a medium, or a substance, endowing it with a wide
range of meanings. His discussion of fire and water illustrates that many
aspects of our environment do not have a single classification, meaning, or
value (a point Gibson makes explicitly in Chapter 3, when discussing
objects). The fact that aspects of our environment take on many meanings
and values is the perfect complement to the fact that animals can act in
many different ways. The flexibility of action is a hallmark of human and
animal agency, and its perfect complement is found in the ecological
environment. In the example of fire, Gibson states, “There is a gradient of
danger and a limit at which warmth becomes injury. The controlling of fire
entails the control of motor approach to fire and the detecting of the limit”
(1979/2015, p. 33). This type of thinking inspired Gibson’s ideas concern-
ing the margin of safety and time to contact. In fact, a field theory of such
gradients was discussed as early as Gibson and Crooks (1938).
The Margin of Safety

An important aspect of what makes the environment meaningful has to do
with hazards and the danger that they afford. In the middle of his discus-
sion regarding detached objects, Gibson inserts an aside concerning “The
Detecting of a Limit and Margin of Safety.” What Gibson is pointing out
is that every action has a limit beyond which it can no longer be completed
safely, in addition to a limit where it cannot be completed at all. Take, for
example, someone who has a maximum stepping length of 100 cm. Any
gap or obstacle in the environment longer than 100 cm will need to be
crossed with a different action than stepping. Today ecologically minded
scientists would say that the individual has an absolute critical boundary
for stepping of 100 cm. However, this person likely does not act at––or
even near––the upper limits of his or her stepping ability. Instead, this indi-
vidual has a preferred critical boundary, where any gap or obstacle in the
environment of, say, at least 80 cm, will cause him or her to alter their
action. One conception of the margin of safety refers to the region between
a person and the edge of danger, and this gap corresponds to the proximity
of danger in space and the imminence of danger in time (Gibson, 1961a).
However, there are competing ideas about the margin of safety (see
Wagman, Bai, & Smith, 2016). Mark et al. (1997) contended that a
margin of safety for one type of action refers to the difference between an
absolute critical and a preferred critical boundary for that action. Sim-
ilarly, in a study of aperture crossing, Warren and Whang (1987) used
safety margins to explain why people rotate their bodies to go through
apertures that are 1.3 x shoulder width. Wagman and Malek (2009) used
the same explanation when comparing safety margins for humans walking
under foam and hard barriers. Franchak, Celano, and Adolph (2012) argue
instead that safety margins are not about safety per se but about people
accounting for the dynamic size of the body in motion. In this view, shoul-
der width is not the appropriate measure of the absolute critical boundary
for passability through an aperture because the body oscillates from side to
side while walking, thus increasing the effective width of the body. The
concept of a margin of safety deserves further empirical investigation to
understand how actors perceive the limits of their potential actions and
avoid exceeding them to avoid injury.
Tools and the Embodied Action Schema

Gibson (1979/2015) defines an object as “a persisting substance with a
closed or nearly closed surface that can be either detached or attached”
(p. 34). Any reference to an object is to a ‘concrete’ one, and not an
‘abstract’ one. In this sense, the surface of an object becomes paramount,
because that surface is the conduit for information regarding the substance,
reflectance, color, and layout of the object. Generally, detached objects are
much more interesting for human beings and other animals who behave
similarly because they are often of a size that makes them portable and
manipulatable. Pencils, pens, headphones, watches, chairs, books, laptops,
etc. all qualify as detached objects. Gibson states that “the list of examples
(of objects) could go on without end” (p. 34), and he is right. At nearly
every moment, we use and interact with detached objects.
Gibson then begins a discussion of tools, which are a special sort of
detached object. Gibson conceptualizes tools as graspable, portable,
manipulatable, and usually rigid. Take, for instance, a wrench (spanner).
We can use a wrench to turn nuts and bolts, strike something, or throw it.
Each tool has multiple affordances. When we use tools, we incorporate the
tools as extensions of our action capabilities in our body schemas. Work
following Gibson’s has upended previous accounts of the body schema.
Traditionally, the body schema was thought of as a stable internal model
that includes the sizes, shapes, and masses of the body’s parts that is
learned early in life (Head, 1920; Iodice, Scuderi, Saggini, & Pezzulo,
2015). Head (1920) postulated that any changes to the body are compared
to a fixed body schema stored in memory. More recently, ecological scien-
tists have theorized that the body schema is simultaneously malleable and
stable, and that it is perceived in real time (Pagano & Turvey, 1998). A
body schema is malleable in that it can be adjusted due to permanent or
temporary changes made to the body or the body’s abilities. For example,
when people acquire tools (e.g., canes or artificial limbs to correct tempo-
rary or permanent changes in their physical capabilities), they calibrate to
their new action capabilities quite quickly (Day, Ebrahimi, Hartman,
Pagano, & Babu, 2017; Maravita & Iriki, 2004). Gibson states:
When in use, a tool is a sort of extension of the hand, almost an

attachment to it or a part of the user’s own body, and thus no longer a
part of the environment of the user … This capacity to attach some-
thing to the body suggests that the boundary between the animal and
the environment is not fixed at the surface of the skin but can shift.
More generally it suggests that the absolute duality of the “objective”
and “subjective” is false. When we consider the affordances of things,
we escape this philosophical dichotomy.
(1979/2015, p. 35, original emphasis)
In the only figure included in the original chapter, Gibson illustrates this
point with a drawing of a hand holding a pair of scissors with the caption
“A tool is a sort of extension of the hand … and one can actually feel the
cutting action of the blades” (p. 35; see Figure 3.1). This is used to illus-
trate how attachments to the body change the boundaries of the body and
thus alter the affordances as one interacts with the environment.
Figure 3.1 The original caption reads; “A tool is a sort of extension of the hand.
This object in use affords a special kind of cutting, and one can actually
feel the cutting action of the blades” (p. 35). This figure is also
employed as Figure 6.6 in Gibson (1966a). There its caption reads; “A
hand feeling a pair of scissors. The pressures of the metal surfaces on
the skin can be felt, but one is mainly aware of the separation and of
the movements of the handle. In use, one actually feels the cutting
action of the blades” (p. 112).
Source: From Gibson (1979/2015), Figure 3.1. Copyright 2015. Reproduced by permission of
Taylor & Francis Group, LLC, a division of Informa plc.
Rather than being a stored and relatively fixed representation, the body
schema is perceived and calibrated continuously as limbs and their attach-
ments change (Day et al., 2017; Maravita & Iriki, 2004; Pagano &
Turvey, 1998). The conception of the body schema as continuously per-
ceived from ongoing proprioceptive input runs counter to an assumption
inherent in the traditional body schema literature that proprioception does
not inform the central nervous system about the metric properties of the
body and its parts (e.g., Longo & Haggard, 2010). To reflect the fluid
nature of the body schema and the role of perceptual calibration in its
maintenance, we have proposed the term embodied action schema (Day et
al., 2017, 2019). The embodied action schema represents the body’s
current action capabilities, including any effects of a tool, such as the
lengthening of the arm to which it is attached (Cardinali et al., 2009; Day
et al., 2017, 2019; Maravita & Iriki, 2004; Sposito et al., 2012). A key
finding is that both limbs and hand-held objects are perceived through
kinesthesis via the same mechanism, with the same mechanical principles
underlying the perception of hand-held objects and the perception of the
body (Pagano & Turvey, 1998). This explains both the malleability of the
body schema and how attached objects become incorporated into the body
schema to be perceived and controlled as if a part of the body.
Incorporating a tool that extends one’s effectivities typically requires a
period of learning or adjustment before proficiency in its use can be
achieved. Recent research has shown that calibration to one’s tool-
enhanced capabilities occurs relatively quickly, though there is variability
in how much calibration is required for different conditions (Bingham,
Pan, & Mon-Williams, 2014; Bourgeois & Coello, 2012; Day et al., 2017,
2019; Fajen, 2005b; Mon-Williams & Bingham, 2007). This research has
also shown that calibration is action-specific and it involves a mapping
from embodied units of perception to embodied units of action (Bingham
& Pagano, 1998; Coats et al., 2014; Pan, Coats, & Bingham, 2014). That
calibration is action-specific means that calibrating for one action (e.g.,
reaching), will not generalize to another action that involves a different
unit of action (e.g., throwing or walking) (Proffitt & Linkenauger, 2013;
Rieser, Pick, Ashmead, & Garing, 1995; Witt, 2011; Witt, Proffitt, &
Epstein, 2010). Rather than using external metrics such as inches or cen-
timeters, the relevant units are intrinsic to the body’s scale and its action
capabilities (Cutting, 1986; Gibson, 1979/2015; Mantel, Stoffregen,
Campbell, & Bardy, 2015; Pagano, Grutzmacher, & Jenkins, 2001; Prof-
fitt & Linkenauger, 2013). Thus, what is actually being calibrated is the
mapping between intrinsic units of perception and intrinsic units of action
(Bingham & Pagano, 1998; Bingham et al., 2014; Pan et al., 2014).
The conclusions generated from the calibration research mentioned above
suggest that we move from traditional concepts of a body schema, which
implies a stable entity that is either innate or learned early in life, toward an
embodied action schema. An embodied action schema represents the
ongoing effects of constant proprioception and continuous calibration. An
embodied action schema also allows for changing action capabilities of an
actor to be perceived in real time, and stresses the similar intrinsic scaling
employed by the dual processes of perception and action (i.e., motor
control).
To this point, the focus has been on small, portable tools and their
incorporation into the embodied action schema. But humans have created
much larger tools; cars for driving, drones for surveillance, robots to
enhance our physical strength and capabilities, nuclear power plants, etc.
An area of study that ties together perception of tools and the margin of
safety is affordance-based control of action (Warren, 2006; see also Gibson
& Crooks, 1938). An example of this area of study is the control of
braking by vehicle drivers (Fajen, 2005a, 2007, 2008). In this case, action
capabilities are determined by the automobile. But, as with scissors or
other hand-held tools that Gibson mentioned in Chapter 3, the new action
capabilities are incorporated into the embodied action schema of the driver
and can be both perceived and felt by the driver. In his 1961 paper on con-
tributions of experimental psychology to safety research, Gibson mentions
that when the margin of safety is not easily detected during the operation
of automobiles or other devices, then one’s proximity to the margin of
safety must be signaled via some type of display.
Ecological Interface Design

Figure 3.1 in Gibson’s 1979 book illustrates how attachments to the body
change the boundaries of the body to alter the affordances of the agent-
environment system. The same figure was used in his 1966 book to illus-
trate that, when using hand-held objects, one feels the interaction between
the tool and environment to be at the distal tip of the tool, where the inter-
action is taking place, rather than at the skin surface that is in contact with
the proximal end of the tool. Extensive research has shown that tools can
be used as probes to explore the environment (e.g., Barac-Cikoja &
Turvey, 1991; Burton, 1992; Carello, Fitzpatrick & Turvey, 1992;
Wagman & Hajnal, 2014a). The fact that hand-held implements faithfully
convey information regarding distal objects and surfaces means that a
properly designed tool can serve as a medium for haptic perception, just as
light can serve as a medium for visual perception. In this way tools are
functionally transparent to properties of the environment (Barac-Cikoja &
Turvey, 1991; Burton, 1993; Hartman, Kil, Pagano, & Burg, 2016; Heft,
2002; Mangalam, Wagman, & Newell, 2018; Mantel, Hoppenot, & Colle,
2012). When probing a surface, one can feel both properties of the probe
and properties of the surface being probed (Carello et al., 1992). This
finding parallels what happens in vision. When perceiving visually, it is
possible to perceive both distal environmental surfaces and properties of
the light that is serving as a medium, such as whether the overall light level
is bright or dim, whether viewing is occurring in daylight sun or via arti-
ficial light at night, whether that artificial light is “warm” and yellow or
blue and “cool,” etc. Normally, however, one’s attention is focused on the
task at hand involving objects in the environment, rather than on aspects
of the illumination. Haptic probing is similar, a tool can become function-
ally transparent.
Our laboratory has investigated distal haptic perception by examining
attunement and calibration in a simulated laparoscopic surgery task (Alten-
hoff, Pagano, Kil, & Burg, 2017; Hartman et al., 2016; Long, Pagano, Sin-
gapogu, & Burg, 2016). Laparoscopic surgery, also referred to as
minimally invasive surgery (MIS), is preferable to traditional open surgery
because the incisions are much smaller and thus trauma to the body and
associated infection and recovery rates are greatly reduced. The surgeon,
however, cannot directly view his or her tissue manipulations and these
manipulations must be both performed and perceived via intervening tools.
As a result, surgeons are more likely to accidentally break or tear healthy
tissue during laparoscopic surgery than during traditional open surgery.
Our laboratory identified haptic distance-to-break (DTB), an invariant
that specifies to the observer when they are about to break a tissue. DTB is
inspired by, and is mathematically analogous to, optical time-to-contact
(TTC), which is specified by the rate of optical expansion as a surface is
approached (e.g., Lee, 1976). We investigated the ability of subjects to per-
ceive DTB through the use of a custom laparoscopic surgery simulator.
The simulator consisted of a laparoscopic surgical tool attached to a torque
motor that was directed by computer software to simulate artificial mater-
ials being pressed against, deformed by the tool, and ultimately broken if
deformed too far. As the tool is pushed inwards, the amount of force
required to push farther grows exponentially until the material fails. In one
experimental task, participants were asked to push the surgical tool as far
as they could without breaking the tissue, effectively pushing right up to
the point of breakage and stopping without actually breaking the tissue.
With only 10–15 minutes of calibration training, novice participants with
no experience in MIS were able to perform the task with a high degree of
accuracy, coming to within a few millimeters of the tissues’ break point
(Altenhoff et al., 2017; Hartman et al., 2016). Experienced surgeons with
an average of 4.8 years practicing MIS improved their ability to detect
DTB after the same brief calibration training (Long et al., 2016). Anecdo-
tally, the surgeons reported that the simulator and the task were represent-
ative of what they experienced during actual MIS. Although the expert
surgeons performed better than the novices, it was remarkable how profi-
cient the novices (undergraduate psychology students) became after a brief
period of calibration training. That such novices reached a commendable
level of expertise, and that experts had their performance significantly
improved, underscores the efficacy of calibration training for perceptual
attunement to invariants in an applied task.
Research on optical TTC and similar invariants is often criticized on the
grounds that humans and animals do not always show natural or spon-
taneous use of such invariants (see Hecht & Savelsbergh, 2004; Jacobs &
Michaels, 2007). This, however, is not a concern for the applied science of
display design. A contribution made by Ecological Interface Design (EID)
is the focus on what is meaningful in the domain application, the identifi-
cation of invariants (i.e., constraints) in the domain, and the display of
such invariants via artificial devices. If successful, such devices serve as a
medium for the distal perception of the domain. Once the invariants have
been identified and rendered in an appropriate display, users can be trained
to attune to the invariants. The fact that experienced experts do not always
use such invariants on their own accord, but demonstrate improved per-
formance once trained to do so, actually underscores the importance of
explicitly incorporating such invariants into training regimes. Training of
this nature is referred to as “the education of attention” (E. Gibson, 1963,
1969; Gibson & Gibson, 1955) to the invariant that is most relevant to the
successful completion of a given task (Hartman et al., 2016; Wagman,
Shockley, Riley, & Turvey, 2001; Withagen & Michaels, 2005a).
Designing for functional transparency in visual, haptic, and acoustic dis-
plays is a hallmark of EID. The functional transparency that is easily
achieved with hand-held tools serves as an example of what might be
achieved with more complicated devices. Thus, the challenge is to achieve
direct perception and control via artificial displays when dealing with
complex human-made systems, such as process control plants, airplane
cockpits, and hemodynamic monitors (Burns & Hajdukiewicz, 2004;
Effken, Kim, & Shaw, 1997; Hajdukiewicz & Vicente, 2004; Momtahan
& Burns, 2004). Vicente and Rasmussen (1990) stress that:
The goal of EID is to make the computer as transparent as possible,

thereby minimizing the need for inference. Just as the nervous system
and the energy medium are functionally transparent to the properties
of world that are relevant to the survival of the organism (Shaw &
Bransford, 1977), so the computer interface should be as functionally
transparent as possible to the properties of the work domain that are
relevant for effective control … The idea is to “make visible the invis-
ible” so as to create the phenomenological feeling in operators that
they are directly controlling the internal functions of the system, not
dealing with a computer intermediary.
(p. 230)
The goal is to achieve for computer-mediated visual displays the level of

functional transparency that can be achieved with haptic probing and
haptic displays. For haptic probing, the ability of the tool to act as a trans-
parent medium is determined by its material composition. The tool must,
for example, be sufficiently rigid to be useful as a probe (Burton & Cyr,
2008). In a recent magazine advertisement for a fishing rod, the manufac-
turer described the material composition of the rod (e.g., the type of
graphite in the rod and the type of cork in the handle) and then stated that
“you’ll feel every tremor” and “you’ll think it has nerves inside it.” Feeling
through the rod is critical for fly-fishing. It is critical for successfully fight-
ing the fish as it is reeled in; and it enables the user to feel, through the rod
and the line, the specific species of trout on the hook, its gender, and its
approximate age (Rosenblum, 2010). In another magazine advertisement,
Saab boasted that it had engineered its automobiles to be functionally
transparent. The advertisement reads;
Saab vs Descartes: I think therefore I am slower. I feel therefore I am

faster. Sensory cues are designed into a Saab 9-5 to enhance driver/car
interaction. The center of gravity is near the driver’s hips, where the
body first detects lateral movement. Tactile sensations of the road are
intentionally not filtered out of the steering system. Control becomes
intuitive. Driving becomes safer and more enjoyable. Car and driver
are one.
(“Saab Affiches,” 2018)
We are unsure of the accuracy of their claim regarding the hips being
where the body first detects lateral movement (and we would use the term
“haptic” instead of “tactile”), but the advertisement expresses the appro-
priate sentiment for creating artificial displays. The challenge is to achieve
this with devices such as computer-mediated displays.
In the fourth chapter of Gibson’s 1979/2015 book (see Carello & Turvey,
Chapter 4, in this volume), he claims that one does not see light as such and
one does not see the retinal image as such. Instead one sees by means of the
information that is contained in the light. EID is founded on the assumption
that if it is theoretically possible for direct perception to occur via the specifi-
city of information in the light, then it is theoretically possible for direct per-
ception to occur via artificial displays. Such displays must act as media by
similarly conveying information lawfully specific to the affordances of the
remote environment. It is important to note that perception is indirect when
a psychological process, such as inference-making or mental integration,
intervenes between the object in the world and one’s awareness of that
object (Heft, 2002). Thus, the light reaching the eye or the sound reaching
the ear does not result in indirect perception if these media are functionally
transparent, as Gibson argued they tend to be under natural viewing con-
ditions (e.g., Heft, 2002). Similarly, the existence of an intervening hand-
held tool or a computer display does not necessarily have to result in indirect
perception if these media do not cause an additional psychological process.
The challenge is to turn this into reality.
Providing users with information via a display in a way that does not
result in indirect perception has been accomplished by designing the
display so as to convey the optic flow that would be available to the
observer during natural perception and action. This has been useful for
designing displays for robot teleoperation (Gomer, Dash, Moore, &
Pagano, 2009; Mantel et al., 2012). In cases such as interfaces for process
control plants, one begins by understanding the task-relevant invariants
that exist in the dynamics of the system itself (e.g., Bennett & Flach, 2011;
Vicente & Rasmussen, 1990). That is to say, human factors practitioners
must begin, as Gibson does in Chapter 3, by understanding exactly what is
meaningful about the environment. For process control, this involves
understanding the physical constraints of the system and then displaying
the higher-order parameters (i.e., invariants) that are specific to those con-
straints. In this way, the display acts as a smart perceptual instrument to
convey higher-order information akin to TTC and DTB, rather than dis-
playing the lower-order parameters of traditional physics that Gibson
argues against in Chapter 3 (Vicente & Rasmussen, 1990). The lower-
order parameters have to be mentally integrated to form a mental model of
the system’s current state, and then mental projection must be employed to
project the current state into the future to understand how one must act to
affect coordination and control. Displaying higher order parameters similar
to TTC and DTB allows for direct perception and thus allows the display
to act as a medium on a par with light and the retina.
Another goal of EID is to convey the constraints of a complex system in an
intuitive manner. Constraints are one of the meaningful aspects of the domain
environment, and their incorporation into displays has been particularly
useful (Bennett & Flach, 2011; Borst, Flach, & Ellerbroek, 2015; Vicente &
Rasmussen, 1990). Effken and colleagues, for example, have created displays
for patient care that depict the physiological constraints imposed on hemody-
namics by fluid, force, and resistance, as well as the effects of clinical inter-
ventions on those constraints through particular combinations of medications
(Effken, 2006; Effken et al., 1997; Effken, Loeb, Kang, & Lin, 2008). Addi-
tional constraints exist between the patient and the health care practitioner,
or in more general terms, between the user and the work domain (Flach, Rey-
nolds, Cao, & Staffell, 2017). A good display for any complex system needs
to show the system’s constraints because the constraints are the targets of
effective human action (Borst et al., 2015; Effken et al., 2008).
In the Conclusion of the 1979/2015 book, Gibson discusses the use of
displays in experimental science (see Stoffregen, Chapter 15, this volume).
He writes:
The information for a certain dimension of perception or propriocep-

tion can be displayed without interference from the accompanying
information to specify the display. That is the lesson of research on
pictures and motion pictures. What is required is only that the essen-
tial invariant be isolated and set forth.
(p. 292)
Earlier, near the end of Chapter 3, Gibson speaks about “artificial objects”
or devices that display optical information. He says:
A display … is a surface that has been shaped or processed so as to

exhibit information for more than just the surface itself … images, pic-
tures, and written-on surfaces afford a special kind of knowledge that
I call mediated or indirect, knowledge at second hand.
(p. 37; original emphasis)
Gibson seems to be speaking here of static images, not of the dynamic

changing displays typically employed in EID. A “graphical” or “config-
ural” display is not necessarily an ecological display, and the type of EID
displays that have been most successful are those that contain dynamic
changing elements whose motions and relationships are specific to the
dynamic movements and constraints within the domain system (Bennett
& Flach, 2011; Burns & Hajdukiewicz, 2004; Effken et al., 1997;
Vicente & Rasmussen, 1990). Thus, visual EID displays are akin to
haptic displays, where movement and dynamics are integral (e.g., Alten-
hoff et al., 2017). For both haptic and visual displays, the invariants of
the system—the constraints that exist between component parts—are
often only revealed over time through the movements of the displayed
elements. Graphical or configural displays, such as alternative nutrition
labels proposed for grocery products, are usually little better than tradi-
tional displays (e.g., Marino & Mahan, 2005). This is because a nutri-
tion label is a static depiction of information that does not change over
time. A hemodynamic display, in contrast, changes as the patient’s con-
dition changes and it reacts to user inputs in a dynamic fashion. In addi-
tion, constraints do not exist between the separate elements displayed on
a nutrition label (e.g., sodium, total fat, vitamin C, etc.), whereas the ele-
ments of hemodynamic displays are mutually constraining (Effken et al.,
1997, 2008; Flach et al., 2017). In short, EID as an approach to display
design is most fruitful when it is applied to display information regarding
a dynamic system.
Display design and the broader study of human-machine interaction
both fall within the overarching fields of human factors and ergonomics.
The study of human factors and ergonomics begins with a basic under-
standing of human perception, cognition, motor function, and the
environment (as with Gibson’s treatment of the environment in Chapter
3). This understanding is applied to the design of artifacts that are safe
and intuitive to use. As modern humans have made all manner of tools
and continue to alter their environment, there is now, more than ever, a
need for human factors. While not widely recognized, ecological psych-
ology provides a theoretical basis for human factors, and human factors
can be considered as the real-world application of ecological psychology
(e.g., Dainoff, 2008; Dainoff & Mark, 2001; Dainoff & Wagman, 2004;
Day et al., 2019; Effken, 2006; Flach, 1990; Flach et al., 1995; Hancock
et al., 1995; Hartman et al., 2016; Mantel et al., 2012; Vicente & Ras-
mussen, 1990). EID is just one example of what ecological psychology
has to contribute to its broader field of human factors and ergonomics,
and it begins with an understanding of ‘The Meaningful Environment.’
4 Challenging the Axioms of
Perception
The Retinal Image and the Visibility
of Light
Claudia Carello and Michael T. Turvey
Visual perception can fail not only for lack of stimulation but also for lack
of stimulus information.
(Gibson, 1979/2015, p. 54)
A hallmark of James Gibson’s strategy for understanding perception was

to be very clear on the meaning of particular terms that often serve as
mutable placeholders for vague concepts. As his ecological approach was
maturing, he took pains to pin down words that had a technical veneer
but were used all too casually for the role they played in perceptual
theory. Among them, stimulus, information, and light (Gibson, 1960a,
1960b, 1961b, 1963) figure prominently in “The Relationship Between
Stimulation and Stimulus Information” (Gibson, 1979/2015, Chapter 4).
The stated goal of Gibson’s Chapter 4 was to “describe the information
available to observers for perceiving the environment” (p. 41). In his
attempts to do so, he uses another Gibsonian stylistic signature, the
careful contrast, in order to articulate what information is and what
information is not. But the particulars of information—how the optic
array is structured lawfully by surface layout and by ecological events—
were left to subsequent chapters (Chapters 5 and 6 in Gibson 1979/2015;
Mace, Chapter 5, in this volume; Shaw & Kinsella-Shaw, Chapter 6, in
this volume). The challenge taken up by Chapter 4 was the fundamental
underpinnings of two doctrines that had shaped the science of visual
perception for centuries: (1) vision begins with the retinal image; and
(2) light is visible. While the former is typically explicit, the latter is
typically implicit, though no less baneful because of it. Gibson’s logical
dismantling of these doctrines was crucial to setting up his treatment of
information. Lamentably, 40 years after The Ecological Approach to
Visual Perception, and despite the growing influence of Gibson’s many
insights on modern visual science (Nakayama, 1994), the need to parry
these historical doctrines persists.
52 Claudia Carello and Michael T. Turvey
The Inevitability of Images
Although the irrelevance of the retinal image for vision has long been
accepted wisdom among ecological psychologists, this view is not universally
embraced. Contemporary scholars still proceed as if its role in perception is
axiomatic. Image language is especially vivid in computer vision (e.g., Nixon
& Aguado, 2012), where its physiological rationale figures prominently:
“The photoreceptors sample the retinal image to produce a neural image
representation” (Ferwerda, 2001, p. 24). Neuroscience handbooks are firmly
entrenched in the mythology, providing expositions entitled “The beginnings
of visual perception: the retinal image and its initial encoding” (Yellott,
Wandell, & Cornsweet, 2011). And the textbooks we use to teach our
undergraduates cannot resist spinning fact as folklore: “The pattern of light
reaching the retina must mirror the distribution of light in the scene being
viewed. This light distribution, or retinal image as it’s called, is the raw
material on which the retina works” (Sekuler & Blake, 2002, p. 56). Light
is, indeed, patterned by “the scene,” by the substances and surfaces that are
being viewed—this is fact. But whereas Gibson sought an understanding of
how that patterning or distribution of light specifies the substances and sur-
faces that structure it, orthodox theories persist with labeling the result of
that structuring as an image—this is folklore.
Certainly, the elegant characterization of image formation owing to
Kepler (1604/1996) is irresistible for those who see a need to get a copy of
the world inside the body (see Stoffregen, Chapter 15, this volume). Light
rays diverge from every point on an object. A subset of these enters the eye
where the diverging rays are made to converge by a lens. This diverging and
converging occur at every point on the object, thereby painting a picture on
the putative retinal surface (Figure 4.1).
Gibson seemed to limit his frustration with Kepler to the allure of the
latter’s physical optics. While conceding the tidy success of the theory of
image formation applied to optical devices, such as cameras and pro-
jectors, he lamented its application to visual perception:
It works beautifully, in short, for the images that fall on screens or sur-
faces and that are intended to be looked at. But this success makes it
tempting to believe that the image on the retina falls on a kind of
screen and is itself something intended to be looked at, that is, a
picture. It leads to one of the most seductive fallacies in the history of
psychology—that the retinal image is something to be seen.
(Gibson, 1979/2015, p. 53)
Consistent with Gibson’s complaint, the astronomer’s theory of vision is

unabashed in endorsing this implication:
I say vision occurs when the image (idolum) of the whole hemisphere
of the world which is in front of the eye, and a little more, is formed
Challenging the Axioms of Perception 53
;ĂͿ ǇĞ
^ŝŶŐůĞ
ĨŽĐƵƐ
ƉŽŝŶƚ
;ďͿ
Figure 4.1 (a) A diverging pencil of rays from a single reflecting point on an object
is infinitely dense; a subset converges in a pencil of infinitely dense rays
to a single focus point on the back of the eye. (b) A few converging
cones show how an optical image could be built from an infinite
number of points on the object.
on the reddish white concave surface of the retina (retina). I leave it to

natural philosophers (physici) to discuss the way in which this image
or picture (pictura) is put together by the spiritual Principles of vision
residing in the retina and in the nerves, and whether it is to made to
appear before the soul or tribunal of the faculty of vision by a spirit
within the cerebral cavities, or the faculty of vision, like a magistrate
sent by the soul, goes out from the council chamber of the brain to
meet this image in the optic nerves and retina, as it were descending to
a lower court. For the equipment of opticians does not take them
beyond this opaque surface which first presents itself in the eye.
(Kepler, 1604/1996, De modo visionis, in Crombie, p. 338)
Two things are notable here. First, Kepler ignored the retinal anatomy that
puts the lie to the assumption of an “opaque surface” upon which an
image can form. Dissections by the physician and anatomist Herophilos
(c.335–280 bce) prompted his description of the retina as being reminiscent
of a “folded fishing net” (Swanson, 2015). The contemporary characteriza-
tion is in terms of ten layers and a dense network of blood vessels. In
neither case is there an opaque surface. Of its multiple layers, then, which
should we interpret as screen-like? Second, Kepler appreciated that getting
a retinal image is only the first step. That image must still be considered
further by additional authoritative entities. One would think that the
seduction of starting with an image, however misleading, would lose its
appeal once ensnared in the infinite regress of a succession of images, each
in need of its own viewer.
But apparently, logic goes out the window when one is smitten. Witness
the descriptions of compound eyes. As Gibson (1979/2015) pointed out,
the compound eyes of arthropods are different from the chambered eyes of
vertebrates in that they do not focus light. The cone of rays diverging from
an object is not converted into a converging cone of rays by a lens; an
image is not formed. However, those who study insect vision seem to
retain the story initiated for humans. As Gibson lamented:
Zoologists who study insect vision are so respectful of optics as taught

in physics textbooks that they are constrained to think of a sort of
upright image as being formed in the insect eye. But this notion is both
vague and self-contradictory. There is no screen on which an image
could be formed.
(1979/2015, p. 55)
Indeed, image language is routine in discussions of all kinds of eyes. One

prominent review of the evolution of eyes, for example, highlights “the
ways that the preexisting molecular and cellular building blocks have been
assembled to provide various solutions to the problem of obtaining and
transmitting an image” (Land & Fernald, 1992, p. 5, emphasis added).
Descriptions of the optical elements of compound eyes (both apposition-
types, which are common to diurnal insects and arthropods, and
superposition-types more typical of animals in dim environments, such as
nocturnal insects and long-bodied crustaceans) emphasize how they can
produce “good images” with mirrors, tubes, and analogues of two-lens
telescopes (Land & Nilsson, 2006).
On the Myth of the Retinal Image

As intimated in the preceding remarks, the retinal image has long been the
centerpiece of vision theory. It has played the theoretical role of an interme-
diary, a third entity that serves to relate two other entities, the viewer and
the environment that is viewed. No doubt, the persistence of image-centric
accounts of vision reveals great respect for the physics of image-formation.
But optics is only the set-up for the appeal of the retinal image. The tangible
payoff seems to have been provided by Descartes’ demonstration of an
actual image on the back of an eye, the very embodiment of Kepler’s “image
… formed on the reddish white concave surface of the retina … this opaque
surface which first presents itself in the eye” (noted above).
There is no denying the dominion of the retinal image in centuries of
theorizing about vision, even up to the present day. But we must emphasize
that it is an inappropriate starting point for more than reasons of logic.
Not only is there, as Gibson emphasized, no homunculus to view the
retinal image; it seems that there is no retinal image. As noted, the notion
of an image presupposes an opaque surface on which the image can appear
and be observed. However, only a dead retina is opaque (with the degree
of opacity increasing with time after death); a live retina is transparent
(Kawashima et al., 2015). The most storied evidence for a retinal image
comes from investigations of the eye of a dead animal following the eye’s
removal from its embedding capsule (a process termed enucleation) began
with Scheiner and Descartes in the early 1600s. Of singular importance to
their exposition and investigation of the image was another intuition of
Kepler’s, namely, that to comprehend the functioning of a single eye
requires two eyes—one eye (with a putative image) to be looked at and one
eye (that of the experimenter as observer illustrated in Figure 4.2d) har-
nessed to do the looking (Boring, 1942, p. 223).
Descartes detailed the essentials of such investigations in Optics: Dis-
course V (Cottingham, Stoothoff, & Murdoch, 1985). As depicted in
Figure 4.2, the back of the enucleated eye is removed and replaced by
transparent material (e.g., paper or eggshell) referred to as a “white
body,” the function of which is to allow light to pass through the eye
from front to back. The enucleated eye is then fixed so as to face illumi-
nated objects in an otherwise dark room. Viewing the white body in its
capacity as a screen at the rear of the enucleated eye, one will, in Des-
cartes’ words: “see there, not without wonder and pleasure, a picture
representing in natural perspective all the objects outside …” (Cotting-
ham, Stoothoff, & Murdoch, 1985, p. 166). For Descartes, the images so
formed in the back of the eye in the preceding way are images that “also
pass beyond into the brain” (p. 167).
It is not the case that Descartes’ authority went unchallenged. As eluci-
dated by Campbell (1817, p. 21), the surgery in question—in conjunction
with the viewer’s location behind the eye—transformed the enucleated eye-
with-window into what contemporary optics would identify as a rear-view
projector (Figure 4.2d). The philosopher Arthur Bentley appears to be the
most forthright in arguing that the status of the retinal image is more than
problematic. In an essay with the unambiguous title, “The fiction of
‘retinal image,’ ” he asserted that “Examination shows it to be a confused
mixture of analogies” (Bentley, 1954/1975, p. 268). Bentley argued that
Scheiner’s and Descartes’ procedure demonstrates no more than that “the
eyeball contains an optical system … it has next to nothing to do with the
process of vision in a living organism” (p. 271).
;ĂͿ
;ďͿ
;ĐͿ
;ĚͿ
Figure 4.2 The demonstration of a “retinal image” by Scheiner, repeated by

Descartes. (a) An intact eye presented with an object. (b) The enu-
cleated eye with retina and rear coatings scraped off. (c) A “white
body” attached to the rear of the enucleated eye. (d) The image
viewed by Descartes is not on the retina but on the “white body”
(cf. Campbell, 1817).
A core issue for Campbell and for Bentley was that an image can be
located on a translucent screen (the post-surgery state of affairs, Figure
4.2c) but it cannot be located on a transparent screen (the pre-surgery state
of affairs, Figure 4.2a).1 The foregoing has been expressed in the following
terms:
When looking through the windshield of one’s car, or the windows of

one’s office, or the lenses of one’s glasses, that which one sees is
beyond the windshield, the windows, and the lenses, and is responded
to behaviorally as such. This fundamental observation assigns the
burden of proof for the existence of a retinal image to the domain of
living vision, that is, the investigation of eyeballs that are alive not
dead.
(Turvey, 2019, p. 346)
Bentley suggests that Hermann von Helmholtz—one of the most influential

scholars in the physiology and philosophy of perception over the past 150
years—may have been responsible for popularizing what seems to be weak
evidence for the image inside a living eye. In the zealousness of his commit-
ment to a role for the retinal image, for example, Helmholtz provided a
second-hand recounting of what a colleague had told him about viewing
the retina directly (i.e., by looking through someone’s pupil). Under proper
lighting conditions, “It is even possible to see the image sometimes through
the sclerotic of the eye of a live individual, especially if he is blond and has
bright blue eyes which usually contain scant pigment in the choroid
coating” (Helmholtz, 1866/2000, p. 91).2 Such an image, Bentley noted,
was more likely simply a reflection from the cornea or the lens, so-called
Purkinje-Sanson images.
Helmholtz was convinced that his new invention, the direct ophthalmo-
scope, would provide proof of the image:
With the aid of this instrument, it is possible to look directly in the eye
from the front and see clearly not only the retina itself and its blood
vessels but the optical images that are projected on it. That this is actu-
ally the case is proved by the fact that if the eye under examination is
focused on an object that is bright enough, a distinct and sharply
defined image of it may be seen by the observer on the surface of the
retina.”
(Helmholtz, 1866/2000, p. 92)
Convoluted wording notwithstanding—he is saying the proof that you can

see an image on the retina is that you may see an image on the retina—this
report has not been added to in the 130 or so years since it first appeared.
“Reports of observation are extremely rare in the texts, considering how
notorious the fact itself is” (Bentley, 1954/1975, p. 273, note 11). It is
likely that even these rare reports are simply repetitions of Helmholtz’s
claim. Queries put to the internet with the search term “retinal image”
engender multiple instances of what should be termed images of the retina
but none of what we think of as images on the retina.3
The Persistence of Hypostasizing the Retinal Image

Although Bentley was convinced that by the end of his essay, “ ‘retinal
image’ will turn out to be just a careless use of words” (1954/1975, p. 271,
note 5), confidence in its status persists. A journal article with the forth-
right title, “How we perceive our own retina” (Kirschfeld, 2017) gives
familiar voice to the presumed narrative: “At each saccade, we take an
individual ‘snapshot’, as it were, of the optical target … If we consciously
perceive our own retina, it is like looking at such a snapshot” (p. 1905).
The twist in this new account is that the author purported to set up con-
ditions that would allow viewers to consciously perceive their own retinas:
It is possible to observe one’s own retina by illuminating the eye side-

on. In this case, light enters the eye behind the lens (‘retrolental illumi-
nation’, RLI; figure 4.3a) and the retinal vessels throw a shadow onto
the photoreceptors so that the subject perceives a wonderful view of
his own vessels, known as the entoptic image of the retina.
(p. 1905)
In the reported experiment with four participants, such an entoptic image was
produced in the left eye of the observer. Simultaneously, the right eye viewed
a cardboard with variously sized gauges that the observer used to report on
the sizes of the fovea and the blind spot in the entoptic image. The author
was happy to note that one image was not distorted relative to the other,
despite the overrepresentation of the fovea in Visual Area 1. However, the
disconnect between retinal-image-as-snapshot and image-of-the-anatomical-
retina was overlooked. The label for the specially produced image should
have been a hint: Entoptic means “originating within the eyeball.” Shadows
of the vessels of the eye are not what have been meant historically and theor-
etically by “retinal image.” The photograph of the author’s own retina, com-
plete with axes and points superimposed on the photograph to show the
location of the fovea, tellingly showed no projection of the cardboard seen by
the other eye. The claim for an image of the environment on the retina seems
not to have been substantiated on anything other than a translucent retina.
Nonetheless, vision scientists and neurobiologists have been beguiled by
all of the focusing controls that seem to point to the importance of image
quality qua image, even lamenting “special-purpose visual systems [that]
failed to provide the comprehensive neural machinery that would allow
these images to be fully exploited” (Land & Nilsson, 2006, pp. 167–168).
But what struck Gibson as important across the diversity of eye designs is
that they are well suited to registering the structure of ambient light.
Nevertheless, in both cases, in the convex eye as much as in the

concave eye, the adjacent order in the external array is preserved in the
order of stimulation at the receptor mosaic. What-is-next-to-what
remains constant; there is no shuffling or permutation of order. It is
true that the order is inverted by the concave eye and not by the
convex eye but this is a matter of no consequence.
(Gibson, 1966a, p. 165)
This is a very different take on adjacent order from that advanced by pro-
moters of the image: “Adjacent ommatidia view adjacent solid angles, so
that the image as a whole is built up from the contributions of all the
‘apposed’ ommatidia” (Land & Nilsson, 2006, p. 172). For some vision
scientists, it seems, image status can be granted to the adjacent order itself,
without requiring a landing site or a projection. As a case in point, Sekuler
and Blake (2002, p. 57) suggest that the term “image” applies to any dis-
tribution of light that preserves the spatial ordering of locations in space.
Whether or not one wants to consider the adjacencies to build up an
image, however incidentally, Gibson emphasized:
The structural properties of a retinal image are what count, not its
visible properties. Structurally, it should not be called an image at all,
for that term suggests something to be seen. The structure of a retinal
image is a sample of the structure of ambient light.
(1966a, p. 172)
The structure of ambient light is central to the other major theme of

Chapter 4 of Gibson (1979/2015), namely, his denial that perceivers actu-
ally perceive light, to which we now turn.
Do We See Light as Such?

Although the retinal image dominates orthodox theories of visual percep-
tion, the assumption that we perceive light itself is often lurking nearby.
The inexorable discussions of how the intensity and wavelength of light
affect the visual system; routine references to the visible spectrum; the psy-
chophysical dedication to measuring thresholds, constancies, and metam-
ers; and lamentations that the blind spot doesn’t respond to light, all seem
to implicate seeing light as the starting point. They are symptoms of a
failure to draw a fundamental distinction that Gibson highlighted as his
approach developed over the decades. In 1979, he summarized it succinctly
as: “the difference between receptors and a perceptual organ. Receptors
are stimulated, whereas an organ is activated. There can be stimulation of
a retina by light without any activation of the eye by stimulus informa-
tion” (1979/2015, p. 47).
This distinction is further buttressed by Gibson’s invitation to be clear
about what we mean by light. In everyday visual circumstances, there is
that which illuminates (light sources), that which is illuminated (surfaces),
and illumination (light as such). A particularly bold statement that we per-
ceive light as such was provided by Boynton, a prominent scholar in color
vision and physiological optics, who was critiquing Gibson’s then-new
claims for ecological optics: “we are not in visual contact with objects, or
edges, faces, facets, or textures. We are in contact only with photons”
(Boynton, 1974, pp. 300–301).
Gibson’s rejoinder continues to try to clarify the problem:
There is a misunderstanding of the metaphor of ‘visual contact,’ one

that goes back to Johannes Mueller … It leads to the doctrine that all
we can ever see (or at least all we can ever see directly) is light.
(1974, p. 310)
Yet the visibility of photons—that is, the seeing of illumination, the seeing
of pure non-reflected light—has been an issue of long standing (e.g.,
Baylor, Lamb, & Yau, 1979; Hecht, Schlaer, & Pirenne, 1942; Sakitt,
1972; van der Velden, 1946) and continues to be (Field & Rieke, 2002;
Koenig & Hofer, 2011; Tinsley et al., 2016).
Increasingly sophisticated methods have been dedicated to demonstrat-
ing single-photon detection by the human eye. Whether or not the results
can be considered unequivocal seems not to have undermined the charac-
terization of the goal of the refined methods. One recent investigation aptly
observes that demonstrating the response of a rod cell to an individual
photon is not equivalent to demonstrating the perception by a human
subject of a photon (Tinsley et al., 2016). Along with that admission,
however, was the frustration that limitations of previous methodologies
are to blame simply for “an inherent ambiguity about the exact number of
photons required to elicit the perception of seeing light” (p. 2).
It is instructive, for present purposes, to clarify what investigators mean
when they try “to probe the absolute limit of light perception.” The experi-
mental setting of Tinsley et al. was that of passing laser light through a
1 mm-thick beta-barium borate (BBO) crystal. When suitably illuminated,
BBO gives rise to single-photon states. The participant’s task on each trial
was to report whether a light was seen or not, and how confident the
participant felt with respect to the report. More specifically, on each trial,
the participant watched for a dim flash, which occurred at one of two
times, with both times indicated by a beep. A decision was then given as to
which beep was associated with a dim flash, and what level of confidence
should be assigned to the decision. In the 10% of trials when confidence
was high, correct decisions, averaged over participants, were about 60%
(compared to 51.6% for the 90% of trials engendering low or middling
confidence). Tinsley et al. concluded: “To our knowledge, these experi-
ments provide the first evidence for the direct perception of a single photon
by humans” (p. 6). Our reading of direct perception suggests an alternative
conclusion. With regard to the triad of that which illuminates, that which
is illuminated, and illumination as such, it is perhaps more accurate to
claim that the participants in the Tinsley et al. experiment occasionally saw
a source of illumination (that which illuminates). They did not see illumi-
nation as such. They did not see photons.
On the Invisibility of Illumination
A simple mundane phenomenon highlights that whereas light sources and
illuminated surfaces are visible, illumination is not. Consider a flashlight
held vertically in a hand with the emitted light striking the floor in an
otherwise ordinarily illuminated room (Figure 4.3). The hand with flash-
light can be so adjusted in height that one can see the emitted light at the
floor without seeing the flashlight’s bulb. That is, the illumination of the
floor (in the form of a circular disk of light if the flashlight is perpendicular
to the ground) is distinct from the source of illumination (a tungsten fila-
ment/incandescent bulb or a light emitting diode/solid state bulb). The
latter could also be seen if the flashlight were raised or tilted. What cannot
be seen is the light between the flashlight (the source of illumination) and
the floor (the thing illuminated). The illumination itself (i.e., light as such)
is not visible. Coloring or sharpening the light source does not change the
experience (Figures 4.3c and 4.3d). The illuminated surface is seen; the
illumination is not. Similar demonstrations are now common in high
school physics classes, where the light is a bright laser which only becomes
visible with the introduction of chalk dust. That the mundane phenom-
enon is nonetheless remarkable is illustrated by the puzzlement expressed
by a 4-year-old film enthusiast who demanded “Where’s the movie?!”
while looking overhead in a dark theater and gazing urgently between the
projector and the screen (J. Blau, personal communication, 2014).
Why does the assumption that light is perceived sit unexamined and
unquestioned in perceptual theory? It fits seamlessly into the narrative
developed for theories of indirect perception, namely, that our acquaint-
ance with the surrounding environment is mediated by sensations of light.
;ĂͿ ;ďͿ ;ĐͿ ;ĚͿ
Figure 4.3 Illumination is transparent to that which is illuminated. (a) The light

from a flashlight can be seen on a carpet but not in the air between the
flashlight and the floor; illumination is not visible between the observer
and the wall. (b) The view of the wall is no different when the light is
off. (c) The situation is identical for the sharp (red) beam from a laser
pointer: The spot on the floor is visible but the beam is not apparent in
front of the wall. (d) The view of the wall is no different when the laser
pointer is off.
Photoreceptors are responsive to light; they yield sensations of light and
perception of the environment is built from those sensations. In such an
account, we are in direct contact with illumination; we are in indirect
contact with that which is illuminated. In questioning the preceding char-
acterization, Gibson challenges us:
What about the opposite assertion that we never see light? It may at
first sound unreasonable, or perhaps false, but let us examine the state-
ment carefully. Of all the possible things that can be seen, is light one
of them?
(1979/2015, p. 48)
Can we test this assertion? One possible test would be provided if we could
arrange the circumstances so as to allow an observer to encounter light
when it is not illuminating any surfaces. If Gibson is right, then observers
should not experience that light; in the absence of illuminated surfaces,
they should experience only darkness.
The requisite demonstration, implemented as part of a science exhibit, has
been described by Zajonc (1993). Using details provided by him, we con-
structed the apparatus illustrated in Figure 4.4 in order to conduct some sys-
tematic investigations. The basic design entails a light source at one end of
an empty box, interior surfaces covered in light-absorbing material, and an
aperture through which to view the interior of the box. With the light turned
on, the interior of the box provides a setting in which there is illumination in
the absence of things illuminate-able. Illuminate-able objects or surfaces can
be introduced into the otherwise non-reflective box interior, either through a
small slot or by opening the hinged rear wall.
Photographs of the interior of the box were taken through the viewing
aperture for each of the following four conditions: (1) light source off, box
empty (Figure 4.5a); (2) light source on, box empty (Figure 4.5b); (3) light
source off, rod in box (Figure 4.5c); and )4) light source on, rod in box
(Figure 4.5d). The photographs reveal that the fact of light in the box is
insufficient to experience its consequence, which is restricted to Figure
4.5d. (These impressions from the photographs have been verified with a
dozen observers under experimental conditions.)4 The implication is that
illumination is visible only by way of that which is illuminated.
The latter conclusion can be taken a step further with a fifth condition:
light source on with smoke filling the box. Smoke reflects light so that it
can be seen without a rod in the box (Figures 4.5e and 4.5f ); it is a case of
Rayleigh scattering, where the reflecting surfaces are particles in the
medium. Without the particles, there would be nothing to see. The visibil-
ity of the smoke in the box at the viewing aperture (Figure 4.5f ) should be
underscored. It gives necessary evidence that the unseen light in Figures
4.5a–c is not a matter of the unavailability of light. The light is there but
invisible. Smoke renders it visible.
;ĂͿ
&ƌŽŶƚǀŝĞǁ
WůǇǁŽŽĚĞǆƚĞƌŝŽƌ
/ůůƵŵŝŶĂƟŽŶ
ƐŚĂŌ
sŝĞǁŝŶŐĂƉĞƌƚƵƌĞ Z
ZŽĚƐůŽƚ
;ďͿ dŽƉǀŝĞǁ;ůŝĚƌĞŵŽǀĞĚͿ
KďƐĞƌǀĞƌ ZŽĚŝŶƐĞƌƚĞĚ
ƚŚƌŽƵŐŚƐůŽƚ
Figure 4.4 (a) Three holes drilled in a plywood box provided an illumination shaft
(extended with a PVC pipe to secure a flashlight), a monocular viewing
aperture centered on the front wall and, to its right, an insertion slot to
allow the introduction of a thin rod. (b) Rods angled obliquely directly
in front of the viewing aperture of the box, which was lined with black,
light-absorbing foam.
Figure 4.5 Photographs taken through the viewing aperture for (a) light off/no rod;
(b) light on/no rod; (c) light off/rod present; (d) light on/rod present;
(e) when the box is filled with smoke, the beam can be seen through the
unlatched back wall; (f) the smoke as seen through the viewing
aperture.
The observer’s experience with the light box is akin to the experience of
an astronaut on a spacewalk. Light is all around but, if the spacecraft,
Earth, and the moon are out of view, nothing is seen but darkness pep-
pered with point-light sources (the distant stars). Although the astronaut’s
eyes are continuously registering photons—the sun’s light is all around—
the astronaut does not have sensations of light (Zajonc, 1993).
Colored Illumination Strictly as Such

For the purpose of generality, the light box can be used to extend the
inquiry to color vision. For example, colored acetate strips can be hung
inside the box in front of the light shaft. Colored light is thereby projected
into the box. When a sheet of white paper is affixed to the rear interior
surface of the box opposite the light shaft, a camera placed within the box
captures the colors projected on it, affirming the presence of red, green, or
blue illumination throughout the box. Observer experiences looking
through the viewing aperture are no longer unexpected: They see darkness
whether the light is red, green, or blue. But once again, an unpainted
wooden rod inserted through the slot reveals the color of the flashlight’s
acetate strip as a red, green, or blue stick is seen in an otherwise black
background.
The light box can be taken outdoors, the sun can replace the flashlight,
and we can do an outdoor analogue of Newton’s (1704/1952) original
prism experiment. As Figure 4.6 shows, the box can capture a light spec-
trum. And in agreement with the prior experiments, when the box is
Figure 4.6 Newton revisited. (a) The box was placed outdoors, slanted so that the
illumination shaft faced the sun, and an optical glass triangular prism
was aimed so as to shine down the illumination shaft. (b) With a camera
inside the box, the light spectrum (rendered here in shades of gray) can
be seen on an opaque white surface placed against the rear wall of the
box. (c) The spectrum that is visible on the rear wall is not seen at the
viewing aperture. (d–f) When a rod is introduced, however, it is easily
seen in the color of the part of the spectrum that it intersects.
closed, spectral qualities are registered by camera and observers only in the
presence of the inserted rod, that is, only in the presence of a reflecting
(illuminate-able) surface.
We are claiming that observers experience the radiant light-filled box
interior as black (i.e., as unlit) because the interior walls do not reflect light
and vision is tied to reflected light not radiant light. One might contend,
however, that observers’ experience of blackness on those occasions is
because the direction of light in the box (the direction of the illumination
shaft) is perpendicular to the direction of the viewer’s line of sight (the
viewing aperture). That is, one might contend that vision requires that illu-
mination’s direction be parallel to, not perpendicular to, the viewer’s line
of sight.
A final demonstration with the light box, however, indicates otherwise.
All of the foam walls were covered with light-reflecting paper (Figure
4.7a). In sharp contrast to the nonreflecting version of the box, the walls
;ĂͿ
;ďͿ ;ĐͿ
Figure 4.7 (a) Box with interior reflecting surfaces. (b) A top view schematic
of the light in the interior of the foam-lined box. (c) A top view
schematic of the light in the interior of the box shown in (a).
of the light-reflecting version were plainly visible at the viewing aperture.
The inserted rod was not needed to reveal the illumination of the box’s
interior. We take the implication to be that light in the box is invisible in
Figures 4.5a–c and 4.6c not because the projection of light is perpendicular
to the line of sight but because it is not reflected.
Figures 4.7b and 4.7c schematize the contrast between light’s behavior
in the non-reflecting box and the reflecting box. The consequence of scat-
tering in an enclosure, such as the box with reflecting walls, is multiple-
reflection or reverberation, an endless bouncing of light from surface to
surface, a network of convergence and divergence that is indefinitely dense.
It renders the light in a given environmental arrangement specific to the
environmental arrangement (Gibson, 1966a, 1979/2015).
For those who remain unconvinced, let us return to the situation of
astronauts in deep space. Imagine two spacewalkers who can see each
other in the surrounding blackness of space by virtue of the light each
reflects to the other’s eyes (Figure 4.8a). If we restrict our attention to the
light reflected from the face of one (Figure 4.8b), it is clear that she is
bathed in the light she reflects, including light that is reflected from her
eyes. If the other astronaut moves out of view, she sees only darkness
where he had been. She does not see the light despite still being immersed—
eyes included—in it (Figure 4.8c). The problem is not that light doesn’t
make its way into the eye of the observer where it can affect the sensory
apparatus, the problem is that there is no structured light.
Implications of the Invisibility of Light

The foregoing examples were previewed by Gibson (1979/2015):
The only way we see illumination, I believe, is by the way of that

which is illuminated, the surface on which the beam falls, the cloud, or
the particles that are lighted. We do not see the light that is in the air,
or that fills the air. If all this is correct, it becomes quite reasonable to
assert that all we ever see is the environment or facts about the
environment, never photons or waves or radiant energy.
(p. 48)
This assertion is echoed by the physicist, Zajonc: “without an object on

which light can fall, one sees only darkness. Light itself is always invisible.
We see only things, only objects, not light” (1993, p. 2).
What about the opposite scenario: What would we experience if light
itself were visible? For an illuminated environment in a transparent
medium, we would have to assume that all points of observation would be
completely filled with light. In consequence, the surrounding surfaces
would effectively be occluded by light at every point of observation (A.
Zajonc, personal communication).
;ĂͿ
йηΨΨΘ
͊ΗηΨ
;ďͿ
;ĐͿ
Figure 4.8 (a) Light reflected from spacewalkers allows each astronaut to be

seen by the other in the blackness of space where there are no other
surfaces to reflect that light. (b) Limiting the depiction to what is
reflected from the face of the astronaut on the left shows that light
is available at his or her point of observation. (c) Even when the
second astronaut is out of view, the light the first astronaut reflects
is still available at her point of observation; yet she sees nothing—
there is energy but no structure.
Do we ever see light as such? The demonstrations and experiments

reported here suggest that we do not. What we see are illuminated sur-
faces. This is a point that Gibson returned to in each of his books when he
discussed Metzger’s (1930) experiments on the Ganzfeld as well as his own
experiments (Gibson & Waddell, 1952). A homogeneous field of dim illu-
mination, such as from a fine-grained surface completely filling the visual
field or covering the eyes with homogeneous light-diffusing caps, provides
light but no structure. Observers do not perceive the surface before them:
What the observer saw, as I would now put it, was an empty medium
… The purpose of the experiment is to control and vary the projective
capacity of light. This must be isolated from the stimulating capacity
of light. Metzger’s experiment points to the distinction between an
optic array with structure and a nonarray without structure. To the
extent that the array has structure, it specifies an environment.
(Gibson, 1979/2015, p. 143)
Gibson (1966a) connected Ganzfelds to the occurrence of the special

weather conditions of a whiteout in snow plains: “whiteout provides no
information about the world because, although energy is present, structure
is absent” (p. 293). And he drew an analogy to a blackout: “Blackout pro-
vides no information about the world because energy is absent” (p. 293).
In neither case are surfaces perceived; energy alone is not enough. Echoing
Bentley’s (1954/1975) characterization of the retinal image, we would say
that casual claims of “seeing light” are another instance of “a careless use
of words.”
Summary
Our critical evaluation of the commonplace discourse on the retinal image
and the visibility of light bears on Gibson’s aim in Chapter 4: To “describe
the information available to observers for perceiving the environment.” In
our chapter, we have highlighted the historical inclination to accept the
image formulation as a fitting characterization of the light to the eye, and
the echoes in contemporary literature. As underscored, that inclination
engenders a theory of vision that is necessarily inferential: Perception is a
process of making inferences with respect to images on the retina. Corre-
spondingly, in our chapter we have highlighted the historical inclination to
accept visible light as a fitting ontological characterization of light as such,
again with contemporary echoes. That inclination leads to a theory of
vision necessarily expressed as having experiences of light quanta—that
visual experiences are first and foremost of light as such. In his Chapter 4,
in contrast, Gibson highlighted the distribution of light—its structure, its
order—as information about the surfaces and substances that structured it.
In so doing, he provided a formulation in terms of information at nature’s
ecological scale that rejects both of the storied historical implications.
Notes
1. We have not found citations of Bentley or Campbell in any of Gibson’s books
or papers. Although calls to question the literal existence of the retinal image
would likely be celebrated by Gibson, his own arguments were simply against
how the retinal image has been used conceptually.
2. Helmholtz’s footnote 1 attributes this report to A. W. Volkmann, a renowned
scholar in physiological optics.
3. Anecdotally, we have asked our own optometrist what he sees when he uses the
ophthalmoscope to peer into our eyes. He listed the features related to retinal
anatomy and health (vessels, floaters, etc.). Trying not to lead the witness too
much, we artfully encouraged him to say more. He responded, “Do you mean
do I see what you see? No, that’s just silly” (S. McKeown, personal communica-
tion, April 27, 2018).
4. Identical impressions were reported by Zajonc (1993), whose light source was a
powerful projector rather than a flashlight.
Part II
The Information for Visual
Perception
5 Getting into the Ambient Optic
Array and What We Might Get
Out of It
William M. Mace
James Gibson labored carefully over the development of the concept of the
ambient optic array. This chapter will present and elaborate Gibson’s most
mature articulation of the ambient optic array concept but with enough
history to dramatize Gibson’s constant effort to appreciate relevant empiri-
cal results, to adjust all aspects of his theorizing to one another and to keep
the system focused on the overall task of explaining perception of the
environment in a coherent way. With the development of the ambient optic
array concept in hand, the chapter will then survey what has happened in
research on vision from 1979 to the present in order to discover what
people have and have not appreciated in Gibson’s work.
The Ambient Optic Array (for Real)

At the beginning of Chapter 5 of the 1979/2015 book, Gibson says:
The central concept of ecological optics is the ambient optic array at a

point of observation. To be an array means to have an arrangement,
and to be ambient at a point means to surround a position in the
environment that could be occupied by an observer. The position may
or may not be occupied, for the present, let us treat it as if it were not.
(1979/2015, p. 58)
Thus, the optic array is a theory of the optical structure of the environ-
ment. So far as I can tell, there has never been anything quite like this, that
is, the “of the environment” part. Ecological optics says that in an environ-
ment with an atmosphere, opaque surfaces, and light, the light bouncing in
all directions will come to an equilibrium such that there will be patterned
structure that is specific to the structuring environment. The minimum
structure for Gibson is an intensity difference. Light, as streams of photons,
is not the relevant description. The relations of difference are relevant.
Light is a medium to carry structure. Structured light makes light focusa-
ble. Unstructured (uniform) light is unfocusable. A surround of homogen-
eous intensity is a Ganzfeld, and both scientists (Avant, 1965; Gibson &
74 William M. Mace
Waddell, 1952) and artists, e.g., Robert Irwin (Weschler, 2009), and James
Turrell (Adcock, 1990) have fashioned versions. In order to have accom-
modation adjustments of an eye’s lens, the adjustments have to be to some-
thing. There has to be an intensity difference created by arrangements in
the optic array that can be brought into focus. Accommodation is not
something an optical system can do in memory or imagination. There is
something in the array, structured by an environmental arrangement, to be
focused on. This (accommodative adjustment) is the first of Gibson’s list of
criteria for reality that will be presented below.
Now consider this next move of Gibson’s in presenting the structure in
the ambient optic array:
We obtain a better notion of the structure of ambient light when we

think of it as divided and subdivided into component parts. For the
terrestrial environment, the sky-earth contrast divides the unbounded
spherical field into two hemispheres, the upper being brighter than the
lower. Then both are further subdivided, the lower much more elabo-
rately than the upper and in quite a different way.
(Gibson, 1979/2015, p. 60)
The ambient optic array is divided into two parts: the upper (sky) and the
lower (earth), creating a brightness difference at the horizon. Note that this
is an extreme simplification but it is the opposite of atomism, which sim-
plifies by reduction to the smallest parts. Gibson’s simplification, the divi-
sion of the whole optic array into two parts, is a simplification that is more
like a Gestalt whole.
Simplification begins with the largest part. But unlike Gestalt entities,
this whole is not a mental organization. It is the environment that is
divided into sky and earth. It is a large containing envelope that life is
inside of (Mace, 1977). This very large containing envelope does not
require observers for its existence, although there is no specific containing
envelope without something to be contained. Minimally, to do the geome-
try, a point of observation must be declared. Both halves of the array func-
tion as background for all that is contained in the world. The sky, with
little detailed structure, is not a very determinate background, whereas the
richness of surface layout on the earth makes for a far more determinate
background for the nested components that it contains.
Now consider the subdivisions of each hemisphere––cloud configura-
tions in the sky, rich varieties of surface layout on the earth. The two
enveloping hemispheres have nested structures of adjacent patches that
project to points of observation. Gibson said that the parts of an array are
given as solid angles projecting to possible points of observation. The solid
angles are faces and facets of surfaces as well as the gaps between them
(like looking through tree leaves into a background sky). He stressed that
the perspective lines in his diagrams (see Figure 5.2) indicated differences
Getting into the Ambient Optic Array 75
in intensity of light, differences that would be preserved (invariant) over
absolute light levels. Thus, the lines in his diagrams are not rays of light.
For Gibson, the ambient optic array is a plenum. It is structurally dense,
like a jigsaw puzzle. Each “form” is a face of a surface, a facet, or even an
opening onto the sky. There are no gaps in this dense structure. The entire
set projects to a point of observation to yield a structured optic array at
that point of observation. One consequence of this initial image of a struc-
tured array is to avoid any thinking based on single points in empty space.
Motion perception, considered as a change of position of a point relative
to the retina, is not a natural topic from the standpoint of a packed optic
array. Rather, for Gibson, the changing structure of the optic array (the
disturbance of the structure) had to be fundamental and types of change,
not simple motions, need to be studied as such.
Because of this nested puzzle structure, for Gibson, the natural way to
locate some portion of the array was by inclusion (nestedness), not by
systems of coordinates. As puzzle pieces, each solid angle would be unique,
and the total arrangement at each point of observation would be unique—
in contrast to a coordinate system, in which every point is the same as
every other point. It follows from this uniqueness that the exact location in
the world of a photograph can be determined, where “exact location”
refers both to the world setting and the camera location in that setting.
Using Google Earth to zoom in on and then zoom back can provide mul-
tiple nested levels, to underscore the distinctiveness of otherwise similar-
looking locations. See Figure 5.1. Zooming back to increase the nested
structure makes the location more definite.
When Gibson described the contrast between radiant and ambient light,
and the key features of the ambient optic array, he began with an unoccu-
pied array without motion. Figure 5.2a shows Gibson’s diagrammatic optic
array of a room with one window and no person. Remember, as mentioned
above, that the lines do not indicate rays of light, but differences of intensity.
Gibson found projective geometry useful, even if limited. The geometry he
advocated was that of Euclid and Ptolemy, termed “natural perspective.”
Natural perspective is constructed from solid angles formed by the faces of
environmental surfaces projected to a point. Artificial perspective introduces
projection surfaces as slices of the solid angles. The projection surfaces of
artificial perspective contributed greatly to the mischief of picture theories of
vision that Gibson strenuously avoided. The projective geometry of the top
of the table (or stool) for the seated and standing observer in Figure 5.2c
would have the conventional texture compression differences of slant vari-
ation. Most significant for Gibson, the dotted lines in Figures 5.2a and 5.2b
represent occluded surfaces, which are not an intrinsic part of perspective
geometry. In Figure 5.2c, Gibson emphasized the occlusion difference
between optic arrays of a seated person and the same person standing. When
a point (place) in an array is occupied, then the surfaces of the viewer’s body
are added to the structure of that array. Because the body is opaque, roughly
76 William M. Mace
Figure 5.1 Pike’s Peak, Barr Trail. Used with permission, Google Earth.
Source: Google Earth Pro 7.3.2.5491. Pike’s Peak, Barr Trail. 38o 50’51.37” N 105o 01’
54.35” W elev 12792 ft eye alt 12727 ft Imagery date 8/2017.
Note: Google and the Google logo are registered trademarks of Google LLC, used with permission.
half of the optic array of a human is occluded. The edges of the eye sockets,
the nose, and some body are visible, as in the famous diagram by Ernst
Mach (see the four panels of Figures 7.1 and 7.2 in Gibson, 1979/2015).
Structure in the Optic Array

Although Gibson was enthralled by the geometry of perspective, he was
increasingly concerned by what it left out. He stressed that the geometric
notion of a station point at the convergence of solid angles does not
capture: (1) the facts of nestedness (the angles do not add up to 360°);
(2) the change of structure induced by moving observers; (3) the optical
consequences of opacity; (4) the reflectance of surfaces; (5) the spectral
reflectance (color) of surfaces; and (6) the shadow structure. The structure
of the ambient optic array must include all of these.
Invariants: The Foundation of Realism

Gibson’s core insight that allowed him to locate a basis for realism in the
optic array was that information for both change and non-change could be
Figure 5.2 Ambient optic array diagrams.
Source: From Gibson (1979/2015), Figures 5.2, 5.3, and 5.4. Copyright 2015. Reproduced by
permission of Taylor & Francis Group, LLC, a division of Informa plc.
78 William M. Mace
present and detected together. Abstractly, there could be invariance under
transformation. An early example that taught him this lesson came from
his “shadow caster” studies. These studies still were rooted in perspective
geometry and were very much within the scope of the “ground theory”
perceptual psychophysics of the program laid out in Gibson (1950a). See
p. 236 of Gibson (1979/2015). Changing shadows of a moving rigid object
can look like just that: shadows of a rigid object moving. Gibson and
Gibson (1957) showed observers the projected shadow of a rigidly rotating
patterned planar object. The dominant impression was a three-dimensional
rigid rotation of the object. The projected form stretches and compresses
in two dimensions but the observed event is constant rigid rotation in three
dimensions––a change in slant of a constant shape. Hence, invariance
under transformation. The phenomenon, but not the explanation, follows
Wallach and O’Connell (1953). Related geometry was pursued by John
Hay (1966) who showed, contra Johansson (1964), that translation, shear-
ing, stretching, and foreshortening all had to be taken into account to
capture rigidity––the constant shape.
Even though the power of the idea of invariance in the optic array
is well illustrated in rigid rotation, it is most emphatically shown in
occlusion––especially dynamic occluding edges (see Heft, Chapter 11, in
this volume). (There are 20 headings and subheadings in Gibson’s chapter
on the Ambient Optic Array. Nine of these are related to occlusion.) One
instance of occlusion occurs when a relatively uniform texture wipes out
another at a margin. This creates what Gibson called kinetic disruption in
the optic array, an instance of topological breakage. When one opaque
surface hides another, from some point of view, Gibson argued that the
mode of disappearance specifies only a change in point of view and not a
change in the existence of a hidden surface. A surface in the world that is
seen to be hidden by rotating or by another surface coming between the
observer and the hidden surface is seen to persist if occlusion is the only
change defined. For humans, the most pervasive occluding edges are the
eye sockets and face. As a head turns, it brings new texture into view in
one direction and hides texture on the trailing side. If those changes are
specific to changes in the view, and not surface existence, then it is possible
to sweep the occluding edges of eye sockets, nose, and face through the
optic array and, over time, see much of the full surround without sampling
it all simultaneously. With locomotion, one changes optic arrays. In Gib-
son’s (1979/2015) terms, the perspective structure changes. See also the
legend of Figure 5.4 of Gibson (1979/2015). New surfaces come into view,
others go out of view. The ones that come into view by uncovering are
seen to pre-exist (and not come into existence). Those that go out of view
are seen to persist (and not go out of existence). These changes are reversi-
ble and that reversibility specifies the existence of a surface separate from
the self. I can turn my head to bring into view parts of a room to my left,
turn to the right where I cannot see those parts, and then turn back to see
them again. This is the core structure in the optic array, whose full signifi-
cance has been pursued only by Gibson.
Gibson, Kaplan, Reynolds, and Wheeler (1969), and the accompanying
film, showed a variety of types of disappearance (occlusion, melting, evap-
orating, being eaten, going into the distance) of optical texture. Some
specified the destruction of a surface, others (occlusion and going into the
distance) did not. Those distinctions are crucial. Changes that specify
destruction of surfaces (e.g., evaporation, eating away, melting) are not
reversible. The hallmark of an irreversible change, presented in film or
video, is that an observer can tell if a film is being shown forward or back-
ward (Gibson & Kaushall, 1973). Aerosol spraying out of a can is one
thing. As a visible event, it is not reversible. A cloud of aerosol collecting
and going into a can is obviously impossible and looks strange when the
sprayed aerosol is shown backward.
Setting the Stage for Gibson’s Position that Structure in the

Optic Array Is Self-Warranting
The first part of Gibson (1979/2015) is about how the ambient optic array
gets its structure from the nature of the world’s surfaces, media, and illu-
mination, together with active observers themselves. When perceiving
occurs, Gibson’s position is that structure is detected and that the specifi-
city of the relation between the environment and the optic array guaran-
tees that the environment is thereby detected. Many people who describe
Gibson’s ideas would agree with this statement. Fewer people may have
wrestled with the issues in the next section, which are required to finish the
thought.
Option 1 The Ambient Optic Array and the World Are

Categorically Different
When one says that the optic array is informative, what does it mean?
Speaking colloquially, we can note that globs of substance in the environ-
ment have three dimensionality and mass. Material can be weighed. Seem-
ingly, light patterns do not have such properties. Therefore, light structure
and “objects” in the world are very different kinds of things. It presumably
is easy for most people to accept that arrangements of light can point to
(i.e., be “cues for”) material parts of the world, but surely arrangements of
light cannot be such things. Thus, there is a common distinction between
information and “things.” Gibson himself stressed that the structure in the
optic array should not be confused with what was in the environment:
Separate terms needed to be devised for physical motions and for the
optical motions that specified them, for events in the world and for
events in the array, for geometry did not provide the terms … Perhaps
80 William M. Mace
the best policy is to use the terms persistence and change to refer to the
environment but preservation and disturbance of structure to refer to
the optic array.
(1979/2015, p. 236)
And, of course, Part I (The Environment to Be Perceived) of the 1979 book

is distinct from Part II (The Information for Visual Perception).
If we were to follow the framing of Option 1, then the structure in
the optic array and the world specified by that structure are very
different kinds of things. I take this to be the most common view in
vision science. The distinction between proximal and distal, assumed by
most inferential approaches, contains this assumption. It is from this
assumption that one is faced with what Warren (2005, p. 338) and
Turvey (2019, Chapter 7) described as “Hume’s problem.” That is, if
we immediately experience the optic array, but the world that “causes”
the optic array is a different kind of thing, why would animals ever
think to hypothesize causes of their experience, and where would the
causes that animals hypothesize come from? This problem was discussed
at greater length by several of us in the early 1980s (e.g., Shaw, Turvey,
& Mace, 1982) under the rubrics “incommensurability of natural kinds”
and “intractable nonspecificity.”
Option 2 The Ambient Optic Array and the World Are Different
Perspectives on the Same Thing (Gibson)
A glimpse of an answer to “Hume’s problem,” to help develop this Option
2 comes from William James. James, like Gibson, thought that a variety of
deep puzzles originated in scholars’ lack of imagination about the richness
of experience. The foundation experiences in traditional British empiricism
were simple sensations, far from “the world” as such. Thus, the world had
to be “outside” of experience as something constructed or inferred. In con-
trast, James embraced a phenomenally rich empiricism that he called
Radical. He argued that this allowed “the world,” as content and object of
experience, inside experience to begin with. To illustrate, James asked
what the object of a mental image might be. In the case of his thinking of
“Memorial Hall,” he asked if it really is “Memorial Hall’ ” that he is
thinking of. He concludes:
[I]f I can lead you to the hall, and tell you of its history and present
uses; if in its presence I now feel my idea, however bad it may have
been, to be continued; if the associates of the image and of the felt hall
run parallel, so that each term of the one context corresponds serially,
as I walk, with an answering term of the others; why then my soul was
prophetic, and my idea must be, and by common consent would be,
called cognizant of reality.
… That percept was what meant, for into it my idea has passed by
conjunctive experiences of sameness and fulfilled intention …
Knowledge thus lives inside the tissue of experience …
(James, 1904, 539–540)
Thus, to James, “Memorial Hall” is a family of experiences sufficient to

count as the referent of the name of the building. And it remains “inside
the tissue of experience.” Two other points worth mentioning in light of
James’s quote. First, the “end” that is indicated is a family of experiences
that do not have a sharply defined end, but rather a pragmatic one. At
some point James has seen enough to be fully satisfied that he is in the
presence of Memorial Hall. We might call this “clarification to suffi-
ciency,” or “clarification until sufficient.” One never sees ALL of anything
in the world. This is consistent with how Gibson defined perceiving in
1979 (Gibson, 1979/2015, p. 244).
Notice immediately the similarity of James’s description to Gibson’s
“rules for the visual control of locomotion” (Gibson, 1979/2015 p. 222).
For example, one Gibson rule for the control of locomotion is, “To permit
scrutiny, magnify the patch in the array to such a degree that the details
can be looked at. To manipulate something graspable, magnify the patch
to such a degree that the object is within reach” (pp. 222–223). These
“rules,” in turn, are very like the descriptions Gibson used throughout his
career, from Gibson and Crooks (1938) to flying (Gibson, 1947, 1950a).
The descriptions of flying, in turn, are close to those of Langewiesche
(1944). Gibson’s descriptions are very phenomenological, although he
always used the word introspection (for phenomenology connections, see
Heft, Chapter 11, in this volume).
Automatic Tests for Reality Within the Optic Array

The major leap that allowed Gibson to realize what James partially intu-
ited was the role of invariance. By recognizing the existence and role of
invariance in the optic array, Gibson established a basis for realism that
avoided the old conundrums of incommensurability (Gibson, 1967a).
Gibson argued that his discoveries, which surely center on invariance, actu-
ally allowed for “automatic tests for reality” within the structure of the
ambient optic array. Consider the following three Gibson quotations:
A criterion for an existing surface is that it can project a visual solid

angle to some point of observation in the illuminated medium of a
cluttered environment. If it does not, it is nonexistent (unreal, imagi-
nary). An existing surface can therefore come into and go out of sight
by a shift of the point of observation. A nonexistent surface cannot.
Perception need not be sustained by a persisting sensory input, since
it consists of registering invariants and variants, not of processing
82 William M. Mace
inputs. One proof is that we perceive a layout of surfaces, only some
of which are in sight at any given moment. That is, the perceiving of
persistence depends on information not on the persistence of vision
after stimulation ceases.
(Gibson, 1977; original emphasis)
I suggest that perfectly reliable and automatic tests for reality are
involved in the working of a perceptual system … A surface is seen
with more or less definition as the accommodation of the lens changes;
an image is not. A surface becomes clearer when fixated; an image
does not. A surface can be scanned; an image cannot. When the eyes
converge on an object in the world, the sensation of crossed diplopia
disappears, and when the eyes diverge, the “double image” reappears;
this does not happen for an image in the space of the mind. An object
can be scrutinized with the whole repertory of optimizing adjustments
… No image can be scrutinized––not an afterimage, not a so-called
eidetic image, not the image in a dream, and not even a hallucination.
The most decisive test for reality is whether you can discover new
features and details by the act of scrutiny. Can you obtain new stimu-
lation and extract new information from it? Is the information inex-
haustible? Is there more to be seen? The imaginary scrutiny of an
imaginary entity cannot pass this test.
(Gibson, 1979/2015, p. 245)
If one re-reads what James said about connecting his “mental image” to
Memorial Hall by unfolding the experiential sequence to an acceptable
conclusion, it is a short (but significant) step to add Gibson’s criteria to get
to the “real world” through perceptual experience. As an example, con-
sider adding the reversibility of occlusion to what James said. Then,
James’s “soul” need not only be “prophetic,” but it can extract the invari-
ance of existing surfaces by the reversibility of the changes in the optic
array. The route to Memorial Hall, and the surfaces of the Hall itself,
consist of persisting surfaces that can be brought into view much as James
described. Those surface textures that go out of view can, reversibly, be
brought back into view if they persist. Reversibility is the guarantor of the
independent existence of the persisting surfaces. One could then add to
James’s description an allusion to the return trip to his home, a later trip
back to Memorial Hall, and so on. The reversibility of those experiences
establishes the independent reality of the surfaces.
James maintained that an object of experience can still be within experi-
ence. As important as James’s move is, it still leaves an opening for a
strong subjectivism, even if that is anathema to James. Gibson showed that
one could preserve the coherence of James’s formulation by staying within
experience, but also establish that independently existing, persisting sur-
faces, could be revealed through invariance.
Artistic Emphasis on the Ambient Optic Array
The retinal image was a bête noire for Gibson. The ambient optic array
was his alternative. It provided something overarching or surrounding to
be sampled and could be a basis for explaining what larger whole succes-
sive sampling could converge on. Without a basis for convergence (clarifi-
cation, equilibration, adjustment) larger than a single sample, processes,
such as convergence, clarification, or adjustment, are left unexplained.
Other ways to draw attention to the ambient array as a structure over-
arching retinal images can be illustrated through non-pictorial art, includ-
ing architecture.
Robert Irwin
The contrast between the retinal image and the ambient optic array may
have no better dramatization than the career of the artist, Robert Irwin
(Weschler, 2009). His work literally progressed through stages from paint-
ing pictures to obsessing over the settings for displaying pictures, to the
environments themselves, leading us out of the temptations of the retinal
image and delivering us to the ambient optic array, which is the point of
my drawing attention to Irwin’s career.
Robert Irwin was a talented conventional artist when he began his
career in the early 1950s. However, he soon established the goal of creat-
ing work that was not about anything but itself:
I began to recognize the difference between imagery and physicality,

and furthermore that for me, the moment a painting took on any kind
of image, the minute I could recognize it as having any relationship to
nature, of any kind, to me the painting went flat. … Imagery for me
constituted representation, ‘re-presentation’, a second order of reality,
where I was after a first order of presence.
(Weschler, 2009, p. 65)
In pursuit of this elusive goal, he experimented with using paintings of

lines, then tiny dots. He invented something he called his “disc” paintings
to defeat the confines of a frame, challenging himself to dissolve the frame
completely. Twelve slides of a range of Irwin’s work can be viewed on the
Pace Gallery website, www.pacegallery.com/artists/211/robert-irwin.
A major turning point in Irwin’s career occurred after an apparently
failed installation at the Museum of Modern Art in New York in 1970. He
decided that working out of his California studio was a dead end. In a
period of reflection, he made trips to the Mojave desert and mused:
The Southwest desert attracted me, I think, because it was the area
with the least kinds of identifications or connotations. It’s a place
84 William M. Mace
where you can go along for a while and nothing seems to be happening.
It’s all just flat desert, no particular events, no mountains or trees or
rivers. And then, all of a sudden, it can just take on this sort of … magical
quality. It just stands up and hums, it becomes so beautiful, incredibly,
the presence is so strong. Then twenty minutes later it will simply stop.
And I began wondering why, what those events were really about,
because they were so close to my interests, the quality of phenomena.
(pp. 163, 164)
Here are truly ambient optic arrays, structured by extended surface, sky,
and lighting at a specific time of day. Much of Irwin’s subsequent work
concentrated on finding minimal ways to heighten viewers’ awareness of
environments (both indoors and outdoors) that already exist. An example
is “Running Violet V” in a Eucalyptus Grove near the Faculty Club of the
University of California at San Diego. Because Irwin’s concerns are the
effects of a whole surround, pictures cannot capture the effects he created.
To avoid any misunderstanding, he did not allow photos of his work to be
published. He later relented. Consequently photos and video explorations
of many of his works have been made and are not hard to find online.
James Turrell
A second major artist, who surely does know something about Gibson
(having studied psychology as an undergraduate at Pomona) and who once
worked with Robert Irwin, is James Turrell (Adcock, 1990). Turrell also
has spent a career using surface layout and light to create non-picture-
based visual experiences. Many of Turrell’s works involved sharply cutting
apertures in walls and ceilings, framing the sky to create variants of what
Katz (1935) called “film color.” In these situations, the sky seems to fill the
frame but to be flush with it, as a mysterious part of the surface. The ulti-
mate frame for the sky, built by Turrell, is a sharply sculpted crater, Roden
Crater, in northern Arizona. A site for numerous arrangements for framing
light, including celestial objects, the central experience for observers is to
lie down on the floor of the crater and observe the dome-like appearance
of the framed sky. Much of Turrell’s work, shown in museums, consists of
careful arrangement of lights, sometimes to make a well-delineated, shaped
volume of light, sometimes evenly distributed to make a Ganzfeld.
For art that exploits occluding edges to control what viewers see as they
move relative to the work, see the constructions of Yaacov Agam. Internet
sources are easy to find.
Architecture
Going beyond the retinal image to the ambient optic array seemingly
would not need to stretch to artists like Irwin and Turrell and could just as
well simply cite sculpture and architecture. The architect, Michael Benedikt
is best known to ecological psychologists for pursuing and promoting a geo-
metric concept called the isovist (Benedikt & Burnham, 1985; Benedikt, in
preparation, part II). Isovist software is available at isovists.org. An isovist is
a shaped space of all surfaces visible from a given location. Draw all lines of
sight from a given location to all visible points. Gibson’s Figure 11.2 in the
1979/2015 book is essentially an isovist. Numerous measures can then be
made on an isovist, for example, minimum distance, maximum distance,
average distance, area (isovists are usually examined in two dimensions),
perimeter, perimeter2/area = jaggedness, the inverse of jaggedness gives com-
pactness. The isovist is a kind of abstraction of Gibson’s concept of a vista,
where the vista is all that is unoccluded from a given place of observation,
but the vista contains all of the nested structure, as illustrated in Figure 5.1.
An isovist is abstracted for the purpose of numerical measures.
Since Benedikt introduced his ideas to the community of ecological
psychology in 1981, the use of isovists has increased dramatically in archi-
tecture. “Space Syntax” is the name of the prime sub-field begun as early
as 1976 by Bill Hillier at University College London. See Hillier (1996) for
a full presentation that includes the use of isovists and Turner et al. (2001)
for additional analysis. There are now multiple websites showing how iso-
vists have been used in building and urban design.
Developments in Vision Science, Post 1979

What are the developments related to the ambient optic array since 1979
that we might look to? What might speak to developing the optic array
concept as distinct from the retinal image?
Research outdoors
For some years after Gibson’s 1950 vintage studies, we saw little interest
by other researchers in the ground as visual content and context. He, Ooi,
and their colleagues and students have changed that and done extensive
research using the ground as a background for perceptual judgments (e.g.,
He et al., 2004; Ooi & He, 2007). Farley Norman has contributed his
share as well (Norman et al., 2017). Hartman (1994) reported size judg-
ments of a truck parked on a hill seen against a cloudy sky compared to a
clear sky.
Jim Todd (1982) and Koenderink (1986) provided nice progress on ana-
lysis of the rigid vs. non-rigid distinction. Spröte and Fleming (2016) have
shown a more recent interest in investigating non-rigid transformations.
Their studies here were with static (albeit stereo) graphic images to see
how much might be conveyed about properties of pictured objects. Their
interest is in inferences and judgments about causal processes that led to
observed shapes, as well as material properties of the shapes.
86 William M. Mace
Mingolla and his students (Ruda et al., 2015) have used occlusion as
both an independent and a dependent variable. They have investigated the
optical information for occlusion, but also related their results to depth
perception. Tanrikulu et al. (2018) continue to examine occlusion as tied
to depth perception. Hancock and Manser (1997), using a driving simula-
tor, compared estimates of time to contact (see Smart et al., Chapter 10, in
this volume) with vehicles that simply disappeared compared to those that
“disappeared” by occlusion. The most common heirs to that work assimi-
late it to “prediction motion” (PM) tasks (Makin, 2018). Nam Gyoon Kim
(2008) examined abilities to indicate heading direction in the context of
occlusion from an undulating ground plane. Recent work using occlusion
by the Bingham lab is covered in Heft’s Chapter 11, in this volume.
Readers understand well, I imagine, that research tends to cluster
around empirical manipulations and phenomena more than the issues
framed Gibson’s way. I mentioned “prediction motion” above. In the same
vein, Gibson and Gibson (1957) tend to be extended now under the rubric
of “structure from motion” more than invariance under transformation.
See Koenderink (1986) for a critique of some “structure from motion”
work within a broad presentation of “optic flow.” Another area that Jim
Todd contributes to prominently is called “shape-from-shading.”
Alan Gilchrist (2018) has persistently examined lightness perception in
real conditions, inveighing against exclusive use of computer displays for
such research.
Seokhun Kim completed a dissertation using an optical tunnel (Kim,
Carello, & Turvey, 2016), inspired by Gibson, Purdy, and Lawrence
(1955). See Figures 9.1–9.3 in Gibson (1979/2015), where the discussion
begins on page 145. Gibson (1982c) says, “this experiment led to the
hypothesis of an optic array as contrasted with the retinal image” (p. 98).
In the optical tunnel, Gibson manipulated whether or not there was focus-
able light, and whether or not spaced partitions could be arranged to
specify a solid surface. He was manipulating what was available to an eye
rather than what was “on” the eye at a given moment.
In order to advance our knowledge of the optic array, we should look
for clues anywhere people are doing serious work related to perceiving the
environment. What Gibson can help with most is the framing vision of
what the overall project can be about in order to avoid debilitating puzzles
and blind alleys.
Research Outside Gibson, but Related

What is discussed in the next sections refers to a good bit of work that
could be seen as related to Gibson’s ambient optic array although that is
more like a whiff of Gibson. It is hard to see “Hume’s problem” taken
seriously.
Computer Graphics and Video Games
The biggest story for ambient optic array research since 1979 is in visual tech-
nology. The optic array can be manipulated as never before. The develop-
ment of computer hardware and software has not only brought dramatic new
capabilities for labs with very large budgets, but dramatic price drops have
brought extraordinary technology within the reach of a very large number of
labs. The video game industry and animation in feature films have fueled
much of the relevant media technology. Recall that for James Gibson and his
students to make a short black-and-white computer animation to investigate
occlusion (Gibson et al., 1969), they had to make a special trip to Bell Labs.
Now, artists and researchers have software tools like Python and Blender.
They have commercial software like RealFlow by Next Limit available to
them: www.nextlimit.com/realflow/latest-release/#prettyphoto/0/.
Behind this kind of work are scientists like Robert Bridson (2016), who
took on the challenge of depicting fluids. He is the developer of Naiad soft-
ware widely used in animation for feature films. See also Bridson and Batty
(2010) and the Bridson website which links a large number of papers: www.
cs.ubc.ca/~rbridson/. Adelson (2001) described forays into the study of mater-
ials, including seminal work by the Koenderink group. The Adelson group
(Xiao et al., 2016) more recently reported research on folding in fabric.
Virtual Reality
Closely related to computer graphics are virtual reality (VR) displays, some
of which are immersive and some not. Immersive virtual reality promises
to give us great purchase on the ambient optic array because it provides
optical structure that surrounds and varies with head motion. It provides a
visual environment that can be explored. Smart phone implementations
can simulate windows on another environment that can be explored. See
Scarfe and Glennerster (2015) for a review. Among ecological psycholo-
gists who have employed virtual reality are Geoff Bingham (Bingham et al.,
2001), Frank Zaal, Claire Michaels, and Reinoud Bootsma (Zaal &
Bootsma, 2011; Zaal & Michaels, 2003), Bill Warren (Warren et al.,
2017), and Jiang et al. (2018). Loomis (2016) reflected on 25 years of VR
and the topic of “presence” for the eponymous journal.
Tom Stoffregen and students (Munafo, Diedrick, & Stoffregen, 2017)
have evaluated the Oculus Rift and found that it can induce discomfort
and sickness in some people.
Plenoptic Function
The “ambient” part of Gibson’s ambient optic array has not been taken
seriously by many, except in the technology of virtual reality (see below).
One notable exception is the “Plenoptic function” of Adelson and Bergen
88 William M. Mace
(1991). This also is called the “light field” (Ng et al., 2005). Like Gibson,
the authors imagine a point of observation with light coming from all 360°
around the point. However, they presume that the proper surround for a
point is light not surfaces:
The significance of the plenoptic function is this: The world is made of

three-dimensional objects, but these objects do not communicate their
properties directly to an observer. Rather, the objects fill the space
around them with the pattern of light rays that constitutes the plenop-
tic function, and the observer takes samples from this function. The
plenoptic function serves as the sole communication link between
physical objects and their corresponding retinal images. It is the inter-
mediary between the world and the eye.
(Ng et al., 2005, p. 6)
Research like this is fashioned with computer graphics, visual physiology,

and optical devices very much in mind. Perceiving the world would come
later. Put another way, whether or not “the world” in any sense appears
naturally in the theory, or comes in outside of the theory, is in question.
Where does “the world” come from in the theory of perception? Where
does it make its appearance?
Natural Scenes
Following the work of Field (1987), people working within traditional
areas of vision research ventured to extend their work beyond simple dis-
plays to “natural scenes” (nearly always photos of scenes, not the scenes
themselves). Geisler (2008) reviews about 20 years of such work. See also
Ruderman (1994). This approach intersects with the procedures and goals
of Brunswik (1956) much more than Gibson. People are mainly interested
in efficient digital image compression and in rationalizing features of the
visual system. Thus, they ask what is statistically distinctive in a picture
that makes it look like a “natural scene.” Predictability, then, suggests
digital compression strategies that would not require storing values for
every pixel in a digital image. A recent example can be found in McCann,
Hayhoe, and Geisler (2018). They took 96 stereo photographs (of which
they could use 81) around the Austin campus of the University of Texas
for their studies of depth perception accuracy in natural scene images. Like
most vision science work, the proximal-distal distinction is taken for
granted and “Hume’s problem” is not taken seriously.
Summary of Current Research

The remaining challenge is to develop the coherent ecological approach
that Gibson charted for us. What has happened so far is that some of
ibson’s ideas have been patched on to existing programs that are taking
G
up most of the publication space in vision science (psychophysics and
neuroscience––experimental and modeling), computer graphics, human
factors, design. Each of these will play out their own agendas. We’ll see
what happens as they collide with internal contradictions or just give up
on grandiose goals and fall back to engineering practicalities.
Formal Framework for an Ecological Program

My emphasis in this chapter has been to show that invariance in the
ambient optic array allowed Gibson to bring the external world into the
optics. Robert E. Shaw has argued that the formal necessities to carry out
the ecological program are best carried out at the level of symmetries (see
Shaw & Kinsella-Shaw, Chapter 6, in this volume). The ecological
program is replete with paired concepts: animal–environment, perceiving–
acting, information–energy, observation–control, affordance–effectivity.
Shaw argues that those must be related as mathematical duals and that the
needed invariance to hold it all together is what he calls generalized action.
The duals can be composed as duals of duals, allowing for differenti-
ation without disintegration. The comprehensive structure of duals, which
Shaw calls a coalition, is an alternative to hierarchies and heterarchies. A
comprehensive list of references to this work is found in Shaw, Kinsella-
Shaw, and Mace (2019).
6 The Challenge of an Ecological
Approach to Event Perception
How to Obtain Forceful Control
from Forceless Information
Robert Shaw and Jeffrey Kinsella-Shaw
For the academic year (1969–1970), James J. Gibson invited Robert Shaw,
who was at that time an assistant professor at the University of Minnesota,
to visit Cornell University, so that he could learn more about Gibson’s eco-
logical approach to psychology. Shaw had met Gibson at Minnesota in the
summer of 1968 while taking his ecological psychology seminar at the
Center for Human Learning’s Summer Institute. It was from their lively
interactions in Gibson’s seminar that they had, so to speak, “a meeting of
minds.” Shaw’s duties at Cornell would be to help teach Gibson’s graduate
seminar and to teach the undergraduate perception course. Gibson
expressed the hope that Shaw, who had a background in mathematics and
logic, might see ways in which fundamental principles of ecological psych-
ology might be made explicit—even formalized. Shaw gratefully accepted
because he had an inkling that the fundamental significance of invariants
in Gibson’s theory of perceptual information might be amenable to a
group symmetry interpretation—a concentration of Shaw’s. The purpose
of this prologue is to impress upon the reader that everything discussed in
this chapter benefited from that year of close, almost daily, interaction
with Gibson and having been granted license by Gibson to query him
about any facts, concepts, or principles of his approach that Shaw might
need help with (Shaw, 2002).
Some Challenges to an Ecological Approach to

Event Perception
The biggest challenge to explaining Gibson’s ecological approach to event
perception is not understanding the problem he addresses but under-
standing the problem he fails to address. Although the role of ecological
information, as expressed in terms of his optic array construct, deals per-
fectly well with kinematic event properties (i.e., geometric change), it
scarcely addresses the problem of kinetic events at all (forceful change).
This is not surprising since the physical concepts needed are not generally
known outside of physics (e.g., d’Alembert’s inertial force principle; see
Shaw & Kinsella-Shaw, 2007).
An Ecological Approach to Event Perception 91
Consequently, this chapter will attempt to achieve three goals: (1) to
assay the problems that a complete theory of event perception must resolve
if progress is to be made; (2) to show that Gibson’s opening gambits on the
event perception problem were promising if incomplete; and (3) to show
how a physical concept, d’Alembert’s principle, introduced into ecological
physics by the authors (Shaw & Kinsella-Shaw, 2007), can provide a
method to solve this dimensional inhomogeneity problem.
What seems to be needed is a major innovation in the way event space-
time geometry is conceptualized. Gibson helps to clarify what this new
approach must overcome. This chapter is intended to complement Gib-
son’s event perception chapter by suggesting specifically how information
might be available for forces on which action control depends.
What Are Ecological Events?

James J. Gibson (1979/2015) tells us that to make progress on the problem of
event perception, we should begin thinking of events at the ecological scale as
being the primary realities for perceptual theory—with time and space being
abstractions from them. For Gibson, time and space are distinct orders in the
global layout of events and the structure over which they are defined. “Time”
is an abstraction from successive-order and “space” from the adjacent-order
of structures in the ecological-scaled event space. There were no competing
theories of event perception available at the time—apart from one.
Early last century, Einstein (1905/1923) built his Special Theory of Relativ-
ity on the fundamental notion of point-events (unextended points with both
spatial and temporal coordinates) and points of observation on those events
as defined at locations in space-time. The virtue of using 4D space-time geom-
etry over ordinary 3D space geometry is that dynamics is achieved intrinsic-
ally by the tracery of points in space-time (called a “world-line”) without the
need to add change “cinematographically” by sequences of frames (see Blau,
Chapter 16, in this volume). However, Einstein’s space-time (Minkowski
geometry) event theory was not what Gibson (1979/2015) had in mind, for,
given that the overall environment is stationary, he did not think relativity
theory was relevant for perception at the ecological scale. He asserted:
The optical information for distinguishing locomotion from nonloco-

motion is available, and this is extremely valuable for all observers,
human or animal. In physics the motion of an observer in space is “rel-
ative,” inasmuch as what we call motion with reference to one chosen
frame of reference may be nonmotion with reference to another frame
of reference. In ecology this does not hold, and the locomotion of an
observer in the environment is absolute. The environment is simply
that with respect to which either locomotion or a state of rest occurs,
and the problem of relativity does not arise.
(p. 68, emphasis added)
92 Robert Shaw and Jeffrey Kinsella-Shaw
As mentioned above, Gibson offers a different take on the nature of the
temporal and spatial dimensions over which Einstein’s space-time is
defined. Gibson offers the concept of successive order as a replacement for
the temporal dimension and the concept of adjacent order as a replacement
for the spatial dimension. In addition, the ecological approach also recog-
nizes that events exhibit both structural changes, or variants, and struc-
tural persistence, or invariants, over these orders.
Evidently, animals and humans have the general perceptual capability
not only to distinguish between change and nonchange but also to classify
styles of change, as well as the objects that undergo the styles of change
(Kim, Effken, & Shaw, 1995). Consequently, event perception can be
defined as the detection of information specifying a style of change that a
structure undergoes over some determinate region of space-time, or better,
over determinate adjacent places in a succession of durations. Two funda-
mental problems of event perception include, first, how one perceives
change at all, and, second, how one perceives particular styles of change as
such. A shortcoming of our field was (and still is) the lack of consensus on
terminology for discussing what it is about events that might be perceived.
Which abstract aspects of events are specified in optic array information
that are likely to be detected?
Gibson’s Principles of Event Perception

In his approach to event perception, Gibson (1979/2015, Chapter 6)
declares his focus to be on just three kinds of events:
1. Changes in surface layout include translations and rotations of an

object, collisions, deformations, and disruptions.
2. Changes of surface color or texture, significant alterations of the sur-
faces of plants and animals.
3. Changes in the existence of surface transitions of evaporations, dissi-
pation, melting, dissolving, and decay.
Dynamics is the branch of physics that studies time-dependent processes,

and includes both kinetics and kinematics. Kinetics is the branch of classi-
cal mechanics that is concerned with the relationship between motion and
causes, specifically, forces and torques. Kinematics, on the other hand, is a
branch of classical mechanics that describes the motion of points (e.g., par-
ticles), bodies (objects), and systems of bodies (groups of objects) without
considering the mass of each or the forces that caused the motion. Con-
sequently, kinematics, as a field of study, is often referred to as the “geom-
etry of motion,” while kinetics might be referred to as the “causes of the
geometry of motion.” An additional complication, Gibson notes, is that
some of these events are reversible, but many are not. Such invertibility
plays havoc with causal structure since the inverting of cause → effect to
effect → cause violates the requirement that effects cannot precede their
causes.
This fact alone would make one question whether causality has any role
to play in scientific explanation (d’Abro, 1951). For clearly, since the advent
of quantum theory with its superposition principle, information specifying
such inverted events cannot be restricted to causal structure alone. (A telling
subtitle to d’Abro’s book is “The Decline of Mechanism.”)
Traditional mechanical causation requires that for a cause to give rise to
a noncontiguous effect in a nonlocal fashion, there must be a causal chain
linking the two. In many cases, unfortunately, mechanical causal chains
may run aground because some of the mediating linkages involve nonlocal
constraints, such as least action, conservation laws, or field effects. Gibson
and his followers have recognized such nonlocal constraints as making
possible both retrospective and prospective control (Shaw & Kinsella-
Shaw, 1988; Turvey, 1992). Such processes exhibiting nonlocal control
were called ‘entelechal’ by Aristotle, and have also been called conative
processes, and recently discussed in detail by Shaw, Kinsella-Shaw, and
Mace (2019).
The Optic Array: Information for Visual Perception

It is most important to note that all three kinds of events mentioned by
Gibson involve only kinematic dimensions, i.e., space and time, without
kinetic dimensions (mass, energy, force, or momentum). There is a reason
for this emphasis on the kinematic side of dynamics. Gibson introduces a
theoretic construct, called the optic array, as an aid in characterizing the
optical information that is detected during visual perception. The optic
array is a cone of light rays defined at any point of observation (moving or
stationary) that might be occupied by an actor (but need not be). More
precisely, an optic array is a 360 º solid angle of adjacent light contrasts
concentrated at a point of observation (Figure 6.1, from Gibson,
1979/2015, p. 65).
Although, intuitively speaking, optic array geometry seems to be a
kind of projective geometry construct, it is not intended to be. Gibson
uses its projected rays only as a heuristic to facilitate communication,
and refuses, on principle, to embrace projective geometry as a general
method for ecological optics. For examples of Gibson’s use of the ray
concept, see Gibson (1966a, pp. 12, 15) where he mentions rays and uses
rays in his figures. “At every point in the illuminated medium there will
be a sheaf or “pencil” of rays converging from all directions” (p. 15).
And in Gibson (1979/2015, p. 52; emphasis added), he says: “A focused
pencil of rays consists of two parts, the diverging cone of radiant light
and the converging cone of rays refracted by the lens, one cone with
its …”. He cautions us to beware of the important distinction between
projective geometry descriptions of the optic array and what they should
Figure 6.1 The transformation of the optic array defined by a locomotor move-

ment. The solid lines represent the optic array before the observer
stands up, the dashed lines after s/he has moved. The path of locomo-
tion of the head is forward and upward by which the whole array is
transformed.
be. When speaking of the array structure, we should refer to optical

forms (projected images), not to environmental forms (shapes of objects),
and to a change in the array structure, as a transformation, not as a
motion of objects or surfaces. There are no objects or surfaces in the
optic array corresponding to those in the environment! Hence the use of
the term here is more closely allied to some kind of (as yet unknown)
“perspective” geometry than to projective geometry since abstract math-
ematics has not yet been applied to the problems of ecological optics.
Gibson admonishes us to be circumspect in treating the optic array as
ordinary geometry because the constituents of ordinary geometry, such as
point, line, or plane, lack realistic dimensions that actual material objects
possess (Pagano & Day, Chapter 3, in this volume). Points are deemed to
be “ghostly” zero dimensional, lines to have unrealistic zero width, and
areas as having no thickness at all. The lack of thickness to surface area is
inherited from its generator—the moving line segment, and the lack of
width to a line is inherited from its generator—the moving point. Since
surface opaqueness is lacking, there can be no occlusion of one surface by
another, nor the self-occlusion needed to specify an object’s shape.
For Gibson (1979/2015), there is also always some degree of recurrence
(transformational invariance) and some degree of nonrecurrence (trans-
formational variance) in the flow of ecological events. And, although
Gibson alludes to points of observation in the environment, he is careful to
explain that they are not to be thought of as being stationary, discrete,
“ghostly” points in space but rather are pauses in the movement of an
observer along a path through the environment.
Also, all lines of projection pass through a point (the point of projec-
tion) with the resultant loss of the topological property of orientability.
This loss renders the projected object’s orientation ambiguous, so that a
projective geometry cannot distinguish the front of an object from its
back (thus, precluding self-occlusion) or tell its left side from its right
side, making all surfaces unrealistically “see-through.” Because of these
impossible properties, projective geometry and its Euclidean bases
provide unrealistic models of the ecological world (Shaw & Mace,
2005).
For these reasons, one might accept that the proper use of projective
geometry is not constructing facsimiles of opaque objects but only in
describing heuristic guidelines for their placement in geometric space. For
instance, it can specify centers of objects by points and margins surround-
ing areas of light contrasts or object edges by lines. Such use is extrinsic
and post hoc at best. Gibson (1957) asserts this admonition in a paper on
optical motions and optical transformations:
The notion of point-to-point correspondence in projective geometry,

simple and powerful as it is, does not apply to the optics of events any
more than it applies to the optics of opaque surfaces, for it leaves
occlusion out of account. The fallacy lies deep in our conception of
empty space, especially the so-called third dimension of space. What-
ever the perception of space may be, if there is any such thing, it is not
simply the perception of the dimension of depth.
(p. 289)
Keeping these cautionary notes in mind, scientific prudence dictates that

we approach the problem of event perception with a degree of trepidation.
Gibson’s Event Perception

Gibson (1979/2015, Chapter 6) presents the rudiments for an ecological
approach to event perception. (For the reasons given, he preferred to call it
“an approach” rather than a theory.) Since his book is limited to vision,
we should not be surprised that his discussion concentrates on the optical
information for events. Consequently, our discussion in this exegesis is also
restricted to vision. It goes without saying that no adequate event percep-
tion theory can ignore the likelihood that all the senses act together as a
perceptual system, such that one sensory system not only complements the
others by adding redundant information but also supplements their
information with additional event information specific to its own sensory
system. In the place of cooperating sensory systems, the idea of amodal
information that lacks sense “modality” specificity has been raised (e.g.,
Michotte, Thinès, & Crabbé, 1964). More recently, some contemporary
psychologists have argued that in the place of multiple modality-specific
arrays there should be only a single general sensory system (Stoffregen,
Mantel, & Bardy, 2017). Under this view, the relative independence of the
senses under loss, namely, when one sensory subsystem is lost or weak-
ened, not all are lost and it does not compromise the global array.
There are two fundamental assumptions of the optic array: First, the
information for recognizing an event is captured through “projection”
of the physical disturbances (i.e., mechanics) into optical disturbances of
the ambient optic array. Second, the information that directly specifies
an event’s identity resides in the invariant properties of the optic array
disturbances caused by the actual event and reside at the place of obser-
vation—whether that place is occupied by an observer or not. By placing
the optic array information source outside the observer, an important
goal of ecological optics is automatically achieved. Since the place of
observation may not be occupied by an observer, no cognitive abilities
of an observer (memory, inference, mental images) can be relevant to
the make-up of the information of interest. Thus, information is sui
generis and, since there can be no mediator, it must be directly
apprehended.
Gibson (1979/2015) argues that optic array events and real-world
events are so dissimilar that they do not even deserve to be called by the
same name:
These disturbances in the optic array are not similar to the events in
the environment that they specify. The superficial likenesses are mis-
leading. Even if the optical disturbances could be reduced to the
motions of spots, they would not be like the motions of bodies or par-
ticles in space. Optical spots have no mass and no inertia, they cannot
collide, and in fact, because they are usually not spots at all but forms
nested within one another, they cannot even move. This is why I sug-
gested that a so-called optical motion had so little in common with a
physical motion that it should not even be called a motion.
(p. 101)
In short, Gibson informs us that disturbances in the optic array lack mass,
inertia, and even motion, and therefore do not resemble events in the
world involving material objects with those properties. Event perception
presumably still works because, as impoverished as the array disturbances
may be, they somehow still share common event invariants with real-world
events. The fundamental problem of information as specification is
revealed in this assertion. If so, then this way of posing the problem leaves
us facing a conundrum of how purely kinematic optical information can
specify kinetic events. And emphasizing the extreme dissimilarity of optical
array disturbances to the actual events, except for sharing invariants, as
true as it may be, seems to obscure the path to a solution.
The laws of optics and the laws of mechanics provide the bases for
determining all the invariant properties involved in each event, and must
somehow be the means by which we recognize the physical event and the
optical event as having the same referent. To be recognized as being about
the same event, the force-driven disturbances in the environment and the
forceless disturbances in the light projected from them into the optic array
must share the same invariant information. How they might do so is the
major puzzle to be addressed later in this chapter.
Specifically, we will ask how Gibson’s theory of event perception which
assumes only optic array kinematics might be expanded to include optic
array kinetics. Formally, the issue is one of dimensional inhomogeneity, a
mismatch in dimensionality between events with dimensions of mass,
length, and time versus events with only dimensions of length and time.
This problem is profoundly serious because a theory of living systems
based on a mismatch in dimensionality can never, even in principle, solve
Bernstein’s degrees of freedom problem that must be solved if a perceiving-
acting system is to be capable of adaptive interactions with the
environment.
For instance, without kinetic information, then the negative affordances
of lethal or injurious encounters with surfaces, missiles, or other life forms
could not be recognized and thus not avoided. For example, there would
be no information to distinguish the extreme danger of a charging bull
from the friendly encounter with a running child. Their difference in mass
makes the impact force from colliding with the bull extremely dangerous,
while the impact force from colliding with a small child might even be fun.
Likewise, the relative danger of stepping-off places would not be informa-
tionally distinguished from falling-off places since the difference in danger
impact due to the force of gravity would be unspecified. This is not to say
that the optic array might not register some useful kinematic information,
such as a global transformation of the optic array which specifies to the
actor that they are moving rather than some part of the environment,
which is specified by local patches of change.
The Challenge of Abstractness

One thing that makes Gibson’s theory of perception difficult to grasp is its
degree of abstractness, that, in general, information for x is not x per se
but how x fits into the layout of the environment and changes with it on
some occasions but remains unchanged on others. For instance, to begin
with, we need to explain the abstract role of the ambient optic array in
characterizing ecological information, and how being defined at the ecolo-
gical scale entails it being directly detectable.
Gibson’s introduction of the optical array as the chief information con-
struct was a brilliant theoretical move for four reasons: (1) by optically
interfacing the observer with its environment, it means the information is
automatically ecologically scaled; (2) it provides a place where the invari-
ant structure of the environment and the perspective structure of the actor
are brought together; and (3) thus allows the information specific to both
the environmental context and the perceiving actor’s situation to be
directly picked up at the same time; and, finally, (4) it objectifies the
information by locating it outside the observer and therefore assuring it is
devoid of any possible cognitive or mentalistic contributions. This is per-
force the case since no actual observer even needs to be at the point of
observation toward which the optical array is projected, thus no contribu-
tion from the observer’s memory nor inference is even, in principle, pos-
sible. It merely needs to be directly detected. And being both of the
environment and of the place where perception might occur, but need not,
the optical array is most thoroughly ecological—respecting the observer no
more than the observed. And, like affordances, whose information it may
frame, it “points both ways,” toward environment and organism alike.
An example of information about x not being x per se is the fact that a
square shape and a trapezoid shape may project the same 2D form, a trap-
ezoid, while they project different isometric invariants when each is rotated
around the same axis.
Since symmetry theory is our strategy for explaining the invariant
information specifying events as made available by the optic array, it
would be useful to provide some scientific background on symmetry group
theory. We do this next.
The Proven Importance of Symmetry Theory in Science

At the first conference on ecological optics at Cornell University in 1970,
as agreed, Shaw presented a paper entitled “The Role of Symmetry in
Event Perception,” in which he attempted to introduce symmetry principles
to ecological theory (Shaw, McIntyre, & Mace, 1974). Here is a précis of
his introduction:
Shaw began by presenting the views of Ernst Cassirer (1944) because

Gibson had cited him twice in his 1950 book (e.g., pp. 153 and 193). Cas-
sirer argued for the fundamental role of group theory in perception,
asserting that the primitive form of understanding is that of the intuitive
concept of a group. The usefulness of the group concept in contemporary
mathematics and physics offers strong support to the validity of this
insight. For instance, one of the chief functions of group theory in mathe-
matics and physics, as shown by Noether (1918), has been to describe
what properties of objects, events, or even natural laws remain invariant,
or symmetrical, across different domains, or under modification by trans-
formations (e.g., rotations and translations).
This work by Noether (1882–1935) showing group theory to be the basis of

the conservation laws has proven to be a watershed moment for physics.
Noether’s theorem in 1918 shows that there is a one-to-one correspondence
between each conservation law and a differentiable symmetry of nature. For
example, energy conservation follows from the time-invariance of physical
systems, and angular momentum conservation arises from the fact that phys-
ical systems behave the same, regardless of how they are oriented in space.
These results make Noether, arguably, the most important contributor to
the mathematical foundations of physical science in the modern era.
The Dual Role of Symmetry in Event Perception

Symmetry has a double role to play in theories that make it a highly prized
commodity in science. It is by nature a duality in that it refers equivocally
and simultaneously to both a property left invariant under a transforma-
tion (an SI) and to the invariant specifying the transformation that leaves
the property invariant (a TI) (Pittenger & Shaw, 1975). Hence if you base
your approach on a symmetry principle, your perspective on the phenom-
enon of interest is, philosophically speaking, necessarily a double
aspectism—an ontology well designed for analyzing systems founded on
duality symmetries. For example, a rotation of a circle is a symmetry
operation since both order of points and distances between them remain
invariant after the application of the operation.
As already mentioned, Shaw, McIntyre and Mace (1974) argued that
symmetry group theory provides one way for making clear what invariant
properties all events must share by virtue of being events. In an attempt to
address this problem, using the terms TI and SI, an event (E) can said to be
perceptually specified when both of these aspects of invariant information
are available to be detected, that is, when the two-variable function, E(TI,
SI), can be evaluated. For instance, an event involving a bouncing ball
might be denoted as
E(T1 = bouncing, SI = ball) = bouncing ball.
How to Build a Physically Lawful System

Two unicycle wheels are free to roam anywhere unconstrained by what the
other one is doing. But when coupled by a constraint, say, a rigid frame to
make them into a bicycle, the two wheels must act symmetrically and
follow the same course. Thus, we see that symmetry is the expression of a
constraint. For forceless, kinematic, optic array information to become
forceful, kinetic, optic array information requires the addition of con-
straints. A system that is kinematic is free to take on all possible temporal
and spatial values. This freedom allows the system to enter any states
unencumbered by which other states it might enter—so long as the next
states are successively ordered (i.e., are temporal) and adjacently ordered
(i.e., spatially ordered).
For a kinematic system to become kinetic, it must take on the symmetry
constraints it lacks. In other words, it must give up some freedom of
making changes to its state configurations. Hence a move from being
merely kinematic to being kinetic is to assume some constraints that were
missing. Which new constraints that are adopted will determine what kind
of new dynamics the refashioned system will have?
If the refashioned system is to satisfy physical laws, then it must assume
exactly the right constraints or it will be unrealistic and not physically
lawful in the usual way. But what is the “usual way?” Fortunately,
Noether (1918) answered this question for us by proving that from sym-
metries in Nature the conservation laws arise intrinsically.
If a physical system behaves indifferent to its orientation in space, then
its Lagrangian (i.e., its mechanical action) which governs it laws of motion
will be symmetric under continuous rotations, and Noether’s theorem will
dictate that the angular momentum of the system will be conserved. Sim-
ilarly, assume a physical system behaves the same, regardless of place or
time, its Lagrangian is symmetric (invariant) under continuous translations
in space and time, respectively. Then, according to Noether’s theorem, the
system will obey the conservation laws of linear momentum and energy.
(Noether’s theorems showed how symmetries and conservations are math-
ematically synonymous.)
And, finally, if the behavior of a physical system does not change upon
spatial or temporal reflection, then its Lagrangian has reflection symmetry
and time reversal symmetry, respectively. Then, according to Noether’s
theorem, a system with these symmetries means it will exhibit parity and
entropy conservation laws, respectively.
How to Build a Physically Lawful Ecosystem

You will need a toolbox with a minimal set of conceptual tools—among
them, duality, reference frame, and discrepancy. Here we provide an intu-
itive introduction to these concepts eschewing formal treatment until a
later time.
Duality
In mathematics, a duality, generally speaking, translates concepts, theorems
or mathematical structures into other concepts, theorems or structures, in a
one-to-one fashion, often (but not always) by means of an involution (e.g.,
reflection) operation: if the dual of A is B, then the dual of B is A. Such invo-
lutions sometimes have fixed points, so that the dual of A is A itself. The
clearest case of this can be seen in digraph theory where the dual is obtained
simply by reversing the arrows that couple its states (see Figure 6.2).
Gibson (1979/2015) treats affordances as dual aspects of an ecosystem
that refer mutually and reciprocally to both the organism and the environ-
ment components. He tells us:
An affordance cuts across the dichotomy of subjective-objective and

helps us to understand its inadequacy. It is equally a fact of the
environment and a fact of behavior. It is both physical and psychical,
yet neither. An affordance points both ways, to the environment and
to the observer.
(p. 121)
Gibson’s insight that an affordance provides two perspectives, one from

the organism on the environment and one from the environment on the
ŶĞĐŽƐǇƐƚĞŵĐŽŵƉƌŝƐĞƐĂŶŽƌŐĂŶŝƐŵĂŶĚŝƚƐĞŶǀŝƌŽŶŵĞŶƚ͕ĂŶĚ
K ŝŶĐůƵĚĞƐƚŚĞĂīŽƌĚĂŶĐĞƐĂŶĚĞīĞĐƟǀŝƟĞƐĚĞĮŶĞĚŽŶϬĂŶĚ͕
ŝŶĚĞƉĞŶĚĞŶƚůǇ͕ĂƐǁĞůůĂƐŝŶƚĞƌĚĞƉĞŶĚĞŶƚůǇďĞƚǁĞĞŶϬĂŶĚ͘
ͲĂīŽƌĚĂŶĐĞƐĂƌĞĚƵĂůƚŽKͲĞīĞĐƟǀŝƟĞƐ

KĂīŽƌĚĂŶĐĞƐĂƌĞĚƵĂůƚŽKĞīĞĐƟǀŝƟĞƐ
K
KĂīŽƌĚĂŶĐĞƐĂƌĞĚƵĂůƚŽKĞīĞĐƟǀŝƟĞƐ
K
KͲĂīŽƌĚĂŶĐĞƐĂƌĞĚƵĂůƚŽͲĞīĞĐƟǀŝƟĞƐ
K
Figure 6.2 Illustrating the dual components of an ecosystem.

organism, is an example of a duality (dual perspectives) that reminds us of
a quote by a major mathematician: “Fundamentally, duality gives two
different points of view of looking at the same object. There are many
things that have two points of view [agent, patient; organism, environ-
ment] and in principle they are all dualities” (Atiyah, 2007, p. 69).
The key to understanding the ecological approach is to see that, like
Noah after the flood, it construes everything as coming in pairs: organism-
environment, affordance-effectivity, information-control. These dual rela-
tionships can be conveniently summarized in a directed graph (i.e., a
digraph), as shown in Figure 6.2.
The Dual Frame Discrepancy Hypothesis: Finding Forces from

Forceless Information
The most fundamental fact to recognize is that in an ecological approach
every major concept has a dual partner—they necessarily come in dual
pairs because, by definition, there are always two points of view—one
from the organism (O) to the environment (E) and another from the E to
the O. Consequently, we begin with the organism’s frame of reference with
respect to the environment and immediately recognize that there is an
environmental frame of reference with respect to the organism. If the two
reference frames are in total agreement, such that information about x in
E-terms and information about x in O-terms are in thorough agreement,
perception, memory, and inference will reveal no discrepancies. In such
case, the O-frame and the E-frame will be dual isomorphs, and perception
and action will be perfectly coordinated.
Reference Frame
The notion of a reference frame is not the same as a coordinate system or
traditional reference system with a point origin, (0, 0, 0, 0), and metric
coordinates, (x, y, z, t). Instead a reference frame, as construed under the
ecological approach, begins with a point of view (POV) around which per-
spectives are variously organized (see Figure 6.3). The POV might be global
in being the perspectives surrounding O taken with respect to all of E—an
open vista delimited by the visual horizons alone, or something more focal,
ranging from an object and how it is situated in E being at a nearby place or
at a place some intermediate distance away, or, even most locally, being just
defined on the self alone. A reference frame is not located just by places sur-
rounding a POV or lying at various distances away but is also taken relative
to immediate-to sustained-encounters of various durations.
Most importantly, the surround is always filled by distributions of
affordances toward which actions might be taken more or less easily. The
metrics are pragmatic, being restricted to action limits, such as being easily
reachable (e.g., arm’s length, steps away), or navigable over a measured
duration (e.g., a few minutes, an hour or so, a day trip), or reachable by
locomotory treks of certain durations (e.g., walking, running, by bicycle,
car, train, etc.). The POV may also be dynamically delineated as revealed
in the “field of safe travel” surrounding automobiles or pedestrians, as
explained in Gibson and Crooks (1938).
If O and E belong to the same ecological frame, then they are mutually
and reciprocally dual (as signified by ‘<>’), but the dual relations (e.g., a<b,
b>a, a<>b) may be ordered or unordered. Here a<b means a and b are
duals but a anchors the relationship. (Analogous to the distinction between
an independent variable and its related dependent variable.) However, if
the dual relation is ordered, then we use the terms primal for the dominant
member and dual for the dominated member, where primal means
dominant (like an independent variable) and dual means dominated (like a
dependent variable). Primal denotes where the origin of the reference
system is located and dual is where the object to be related to the origin is
located. For instance, John (primal) throws the ball to Mary (dual); Mary
(dual) catches the ball that John (primal) throws. Mary (primal) then
throws the ball back to John (dual).
• O and E are dual reference frames in any ecological system (eco-

system). An ecosystem supports two major symmetries:
• O <> E: O’s perspective on E (dual) and E’s perspective on O (dual);
• O > E: O’s primal perspective on E vs E’s dual perspective on O.
• E > O: E’s primal perspective on O vs O’s dual perspective on E.
A key duality is the primal affordance an actor intends to realize and the
dual action effectivity by which it does so (e.g., catches the ball thrown,
lifts the baby down from its highchair, trims the bushes with the hedge
clippers).
Discrepancy
In general, discrepancy theory describes the deviation of a situation from
the state one would like it to be in, say, to be the dual action to some
primal affordance goal. You intend to hit the bullseye with the dart but
your throw is errant. Consequently, on the next throw you adjust the
direction of the dart ‘s release by a slight hand rotation. The kinetics of
neuromuscular control is felt directly through kinesthetic information. Put
differently, the information frame of the situated dartboard and the situ-
ated control frame of the hand holding the dart dynamically share a
common force bases, one that is rooted in visual and neuromuscular kines-
thetics (felt weight and momentum of arm, stiffness parameter, etc. in the
context of visual information about target parameters).
The Dual Frame Discrepancy Hypothesis
The work-to-be-done as specified in the primal visual information frame
must be matched by the work-actually-done in the dual neuromuscular
control frame. If not, then there is a discrepancy to be eradicated by reactive
adjustments. For the information and control frames to coalesce into the
proper ecological frame vis-à-vis the perceiving-acting cycle, a synergy com-
prising the two frames must emerge that has both the intended specificity
(goal-path accuracy) and efficacy (properly focused dynamics, or ecological
work) (Shaw & Kinsella-Shaw, 1988). Gibson formulated this idea in his
1966 book. This idea is so important and central to the ecological approach,
we should have the authority of Gibson’s own words (Gibson, 1979/2015):
There are various ways of putting this discovery, although old words must
be used in new ways since age-old doctrines are being contradicted. I sug-
gested that vision is kinesthetic in that it registers movements of the body
just as much as does the muscle-joint skin system and the inner ear system.
Vision picks up both movements of the whole body relative to the ground
and movement of a member of the body relative to the whole. Visual
kinesthesis goes along with muscular kinesthesis. The doctrine that vision
is exteroceptive, that it obtains “external” information only, is simply false.
Vision obtains information about both the environment and the self. In
fact, all the senses do so when they are considered as perceptual systems.
(p. 175)
Applying the Dual Frame Discrepancy Hypothesis
Case 1
Consider two trains standing next to each other on adjacent tracks in a
train station. On each train there is a person standing in the aisle, facing
forward, holding a full cup of coffee. Call them Bob and Alice. Unaware
that Bob is watching her through the adjacent train car windows, Alice is
lost in thought when her train jerks into motion. Bob’s train remains sta-
tionary. The sudden jerk naturally causes Alice to spill her coffee and Bob
seeing Alice’s train’s abrupt motion, even though his train remains at rest,
also spills his coffee at the same time. Why? (See Figure 6.3).
While there is no mystery regarding what caused Alice to spill her
coffee, it remains surprising that Bob being on a train at rest should spill
his coffee just from watching Alice’s minor calamity. This puzzle is instruc-
tive and solving it will make clear one way that forceless optic array
information about a forceful action can induce a forced outcome. Or,
stated differently, how can a strictly informational coupling between an
event taking place in one local reference frame somehow induce a second-
hand forceful outcome to take place in another distant reference frame?
<ĞǇĐŽŶĐĞƉƚŝƐŝŶĞƌƟĂůĨŽƌĐĞĨƌŽŵĨƌĂŵĞĚŝƐĐƌĞƉĂŶĐǇ
/ŶĞƌƟĂůĨƌĂŵĞ EŽŶͲŝŶĞƌƟĂůĨƌĂŵĞ
/сʹŵ
ŵс&
ůŝĐĞ
&сŵʹŵсŵʹŵĂсϬ
/ŶĞƌƟĂůĨƌĂŵĞ
/сʹŵĂ
ʹ
Žď
Figure 6.3 The dual frame discrepancy hypothesis. The means are depicted by
which an information coupling to a frame other than Bob’s own allows
forceless kinematic information to induce a forceful effect.
The answer is straightforward. Because of relative motion effect, Bob

mistakes the sudden motion of Alice’s train to be that of his own train.
This relative motion error causes Bob to make an unnecessary postural
adjustment response that upsets his own coffee cup. This chain of events is
depicted in Figure 6.3. Notice that this introduction of a postural force by
Bob increases the dimensions of optical information from kinematic to
kinetic, and thus is the solution sought to the dimensional inhomogeneity
problem posed earlier.
Case 2
A clever experiment done by David Lee at the University of Edinburgh
several decades ago illustrates most dramatically the reality of optical
pushes (Lishman & Lee, 1973). Assume someone is standing in a room
where the walls and ceiling are detached from the floor (Figure 6.4).
Further assume that the “room” (without the floor) swings on a very long
cable attached to a high ceiling so that it appears to glide. The result is the
room’s walls can glide but of course the room’s floor cannot.
If the room is swung toward the person (who sees the wall’s motion),
she or he will sway backwards; if the room is swung away from the person,
she or he will sway forward. Note that at no time does the wall touch the
person; thus, no mechanical force can possibly be responsible for the per-
son’s swaying. Also, since the person faces a wall that fills her or his visual
;ĂͿ ;ďͿ
K K K
>ŽĐĂůĂŶĚŐůŽďĂůŝŶĨŽƌŵĂƟŽŶĂŐƌĞĞ >ŽĐĂůĂŶĚŐůŽďĂůŝŶĨŽƌŵĂƟŽŶĚŝƐĂŐƌĞĞ
Figure 6.4 Lee’s swinging room. When local and global information agree (a), then
posture is not compromised. But when local and global information dis-
agree (b), then there is a dual frame discrepancy and posture is upset by
an optical “push.”
angle, she or he sees only the wall and nothing else in the environment,
especially not the floor. The motion of the wall projects a global optical
transformation into the person’s optic array information that specifies to
the perceiver that she or he has moved from being upright.
The information does not cause the person’s reaction—information is
forceless and, therefore, cannot be a mechanical cause. Since the
information-to-control coupling is forceless, we need an answer to this
question: By what means, metaphorically speaking, does the language of
information get translated into the language of control? The answer is
clear. The kinetics are supplied by the person’s own neuro-muscular system
whose postural equilibrium is upset by an optical push.
It is known that optical disturbances may trigger involuntary reactions
from the perceivers (Shaw & Kinsella-Shaw, 2007). Here it was found that a
movement of the wall so subtle that it goes unnoticed can still induce the
person to sway in phase with the wall’s movement. Although instructed to
stand still without moving, precise (goniometric) measurements at the person’s
ankle joint show she or he still sways in phase with the room’s movement.
d’Alembert’s Principle and Inertial Forces

We follow the arguments given in Shaw and Kinsella-Shaw (2007). In
Newton’s original analysis, his first law was based on impressed forces,
F = mA, inertial forces were omitted. Newton’s laws of motion only apply
to frames of reference in which a body remains at rest or moves uniformly
at a constant speed when no forces are impressed upon it. This is called an
inertial frame of reference. The frame itself need not be at rest—it can be
moving at a constant speed relative to another frame of reference. No iner-
tial forces are felt when a frame is inertial.
In all these cases, an optical “push” arises whenever an abrupt change
in optical structure occurs that transforms the person’s inertial frame of
reference into a non-inertial frame. Put simply: Optical pushes arise from
information specifying “frame discrepancy.” By examining its physical
foundations, we shall see what this hypothesis means.
Newton’s laws as originally formulated, however, do not apply to objects in
non-inertial frames, that is, in frames that are accelerating. But they may be
reformulated so that they do, as shown by the French physicist, Jean le Rond
d’Alembert (1717–1783). His principle states: When any object is acted on by
an impressed force that imparts an acceleration to the object, an inertial force
is produced as a reaction. Inertial forces differ from impressed forces in how
they are produced. An inertial force is created by the accelerating frame moving
out from beneath the objects it contains—temporarily leaving them behind—
until the frame of the impressed force drags them along as well. In keeping
with the Principle of Virtual Work, the resultant of this impressed force and
the inertial force is zero. In other words, when a car is at rest or moving uni-
formly at a constant velocity, no impressed forces act on it or its driver and
thus no inertial forces. However, when the driver depresses the accelerator, the
car’s motor impresses a force on the car that accelerates it. At the same time, in
reaction to the impressed force, an equal but counter-directed inertial force is
produced that acts on the driver, pushing him back against the seat.
As observed earlier, Newton’s laws only apply to objects in inertial
frames; therefore they do not apply to the accelerating car—a non-inertial
frame. But by invoking d’Alembert’s Principle, Newton’s laws can be gen-
eralized to cover this case too. Behind this principle was a simple but bril-
liant insight by d’Alembert, which is clearly revealed in four steps (for a
general discussion, see Lanczos, 1970):
First Step: We start with Newton’s Second Law of motion which

asserts that mass multiplied by acceleration equals an impressed
force, the familiar, mA = F.
Second Step: Rearrange the equation as follows: F – mA = 0.
Third Step: Define a new vector, I = –mA. This is called an inertial
force. Notice, this is a counter-force; its sign is the opposite of the
sign on the impressed force vector.
Fourth Step: We can now reformulate Newton’s Law as F + I = 0.
The third step looks a bit trivial, being nothing more than giving a new
name to the negative product of mass x acceleration. In fact, it allows the
expression of an important principle in the next step. In Newtonian mech-
anics, the concept of a system being in equilibrium entails the nullification
of all impressed forces acting on it. Static equilibrium applies to objects not
in motion. With this reformulation of Newton’s law, d’Alembert showed
us how to generalize the concept of equilibrium to objects in motion. To
make this generalization required a brilliant insight—d’Alembert had to
see that inertia itself is a force that can be included with impressed forces
to make up the total effective force of the system, i.e., effective in the sense
of summing to zero. This now allows us to extend any criterion for a
mechanical system being in static equilibrium to a moving mechanical
system being in dynamic equilibrium.
Inertial forces are experienced daily by those of us whose bodies are
carried along with a variety of accelerated frames—automobiles, trains,
buses, airplanes, swings, carnival rides, horses, or rocket ships to the
moon. The origin of these “unimpressed” forces is the tendency for objects
to resist change of their state of motion or state of rest, in accordance with
Newton’s Second Law, which asserts that a force is anything that acceler-
ates a mass, i.e.,
F = mA.
To reiterate, inertial forces differ from impressed forces in how they are
produced. An inertial force is created by the accelerating frame moving out
from beneath the objects it contains—temporarily leaving them behind—
until the train’s impressed force drags them along as well. Armed with
d’Alembert’s principle, we can now show how it is possible, at least in one
case, to transform kinematic optical array information into kinetic optic
array information.
Given dual frames of reference are involved in a situation, such as Alice
and Bob being on the two trains (or seeing both the wall and the floor
simultaneously in Lee’s room), the two frames of reference must be confus-
able by an observer (e.g., Bob). There must also be two potential energy
sources, say, A and B—A for the impressed force and B for the reactive
force—that are informationally coupled (e.g., Bob sees Alice’s train start
up and mistakes it for his own). If the shades on Bob’s train car were
pulled down, then he would have no information regarding Alice’s train
and experience no optical push. (Or, likewise, there would be no optical
push while standing in the Lee room with one’s eyes shut.)
Again, study Figure 6.3. This is how a kinematic display becomes the
control for kinetic forces. The information coupling of the two observer-
two train frames into an ecological physics field lends support to the dual
frame discrepancy hypothesis.
Conclusion
One aim in this chapter was to critically review Gibson’s approach to event
perception, as discussed in Chapter 6 of his 1979/2015 book. We stressed
the importance of event perception for having a generally adequate ecolo-
gical approach to visual perception. A second aim was to discuss the
importance of symmetry theory as a precise way to conceptualize Gibson’s
invariants approach to information. Here we followed Gibson in recogniz-
ing that successive order and adjacent order were useful replacements for
time (temporal order), an abstraction from the former, and space (spatial
order), an abstraction from the latter.
A third aim, and one we consider most significant, was to review the
problem of how forceless kinematic optic array information could also
specify forceful kinetic information so that the language of control might
somehow be a direct translation of optic array information. We argued
that it is possible to do so by means of the dual frame discrepancy hypo-
thesis. A perceived discrepancy between dual frames that should be con-
gruent causes the perceiver to make neuromuscular adjustments to
eradicate the discrepancy that manifests as a self-produced inertial force.
Then, by applying d’Alembert’s principle in the usual way, the reactive
response can be shown to be dimensionally homogeneous with a Newto-
nian force when the second law is reformulated in the manner of
d’Alembert to include inertial forces.
Our interpretation of the most fruitful aspect of Gibson’s approach to
event perception is the intrinsic duality of the affordance concept. For this
allows a natural way to have dual frames between which a discrepancy can
arise, and is the insight needed to map kinematics into kinetics, in the
context provided by Gibson’s construct of the optic array.
7 The Optical Information for
Self-Perception in Development
Audrey L. H. van der Meer and
F. R. Ruud van der Weel
According to Gibson (1979/2015), information about the self accompanies

information about the environment, and the two cannot be separated.
Information to specify the self, including the head, body, arms, and hands,
goes together with information to specify the environment. Self-perception
and environment perception are inseparable: one perceives the environ-
ment and perceives oneself at the same time. Infants spend hours looking
at their hands, and so they should, for many lessons in ecological optics
need to be learned before they realize that their hands are not ordinary
objects, but belong to themselves and can be used to touch and grasp
objects in the environment. In order to successfully reach out and catch
moving toys, infants need to develop prospective control––the ability to
perceive what is going to happen in the near future. They also need to per-
ceive objects in terms of their own bodily dimensions and action capabil-
ities, whether something is within reach or out of reach, graspable or too
big to be grasped, in terms of their affordances for action (see Wagman,
Chapter 8, in this volume).
Early Perceptuo-Motor Development

There is unity and continuity between the development of perceptual and
motor skills. Rather than assuming that the baby, under the influence of cor-
tical maturation, develops voluntary motor skills to which perception then
needs to be mapped, there is ample evidence that perceptual and motor skills
develop hand in hand. This suggests that development is best conceived as a
gradual, continuous process that begins at, or even before, birth. Acting suc-
cessfully entails perceiving environmental properties in relation to oneself
(Gibson, 1979/2015). Organisms do not perceive objects per se, but what
these objects afford for action. What any given object affords depends on
the size and action possibilities of the perceiver. However, before babies can
reach out and successfully grasp objects in the environment, they first have
to learn they have an arm––and what it can do.
Traditionally, the emergence of successful reaching and grasping is
described as a discrete step in development that suddenly appears at
Optical Information for Self-Perception 111
around four months of age (e.g., Gesell, 1928). As a result of the strong
influence of the maturation perspective, newborn babies are still usually
considered reflexive creatures, responding to physical stimuli in a compul-
sory and stereotyped manner and incapable of performing intentional
movements (Van der Meer & Van der Weel, 1995). Cortical maturation
and the resulting inhibition of reflexive movements are thought to take up
most of the first four months of a baby’s life. Until that time, movements
made by very young babies are typically dismissed as reflexive, involun-
tary, and purposeless. However, our behavioral research with neonates
indicates that newborns show voluntary control over their feeding behav-
ior, resulting in precise coordination between sucking, breathing, and swal-
lowing (Van der Meer, Holden, & Van der Weel, 2005), and that they can
move their arms in a purposeful way.
Moving a limb or the whole body in a controlled manner requires acting
together with gravity and other non-muscular forces (Bernstein, 1967;
Profeta & Turvey, 2018). Consequently, movements cannot be represented
simply as patterns of efference to the muscles, nor in any preprogrammed
context-insensitive way. Accurate control requires online regulation of
muscular activation based on perceptual information about the dynamics
of the limb movement and the external force field, as well as about the
movement of the limb relative to objects or surfaces to which it is being
guided.
Are neonates capable of such perceptuo-motor control or are their
movements to be seen as simply reflexive or due to spontaneous patterned
efference to the muscles, as is commonly believed? Next, we will describe a
series of experiments on neonatal arm movements and discuss their pos-
sible functional significance for later reaching and grasping. If we could
demonstrate that newborns take into account gravity when making arm
movements, then this would show that these spontaneous movements
cannot be characterized as stereotyped or reflexive. If we could also show
that these early spontaneous arm-waving movements were under percep-
tual control, then it is very likely that these movements have an explora-
tory function essential to establish a bodily frame of reference for action.
Waving or Reaching? The Functional Significance of Early

Arm Movements
To test whether newborn babies take account of external forces in moving
their limbs, we recorded spontaneous arm-waving movements while the
baby lay supine with its head turned to one side (Van der Meer, Van der
Weel, & Lee, 1995a, 1996). Free-hanging weights, attached to each wrist
by strings passing over pulleys, pulled on the arms in the direction of the
toes. The babies were allowed to see only the arm they were facing, only
the opposite arm on a video monitor (see Figure 7.1), or neither arm
because of occluders. The babies opposed the perturbing force to keep an
112 Audrey van der Meer and Ruud van der Weel
Figure 7.1 A newborn baby participating in the weight-lifting experiment. Despite

small weights pulling the arms away from the face in the direction of
the toes, infants opposed the perturbing force and kept the arm up and
moving in their field of view, but only if they could see the arm, either
directly or, as here, on the video monitor.
arm up and moving normally, but only when they could see the arm, either
directly or on the video monitor. Thus, newborn babies purposely move
their hand to the extent that they will counteract external forces applied to
their wrists to keep the hand in their field of view.
In order to investigate whether newborns are also able to adjust their
arm movements to environmental demands in a flexible manner, we inves-
tigated whether manipulating where the baby sees the arm has an influence
on where the baby holds the arm (Van der Meer, 1997a). Spontaneous
arm-waving movements were recorded in the semi-dark while newborns
lay supine facing to one side. A narrow beam of light 7 cm in diameter was
shone in one of two positions: high over the baby’s nose or lower down
over the baby’s chest, in such a way that the arm the baby was facing was
only visible when the hand encountered the, otherwise, invisible beam of
light. The babies deliberately changed arm position depending on the posi-
tion of the light and controlled wrist velocity by slowing down the hand to
keep it in the light and thus clearly visible. In addition, we found that the
babies were able to control deceleration of the hand in a precise manner.
For all instances where the baby’s hand entered the light and remained
there for 2 seconds or longer, the onset of deceleration (point of peak
velocity) of the hand was noted with respect to the position of the light.
Remarkably, in 70 out of all 95 cases (almost 75%), the babies started to
decelerate the arm before entering the light, showing evidence of anticipa-
tion of, rather than reaction to, the light. On those occasions where the
babies appeared not to anticipate the position of the light, more than 70%
of these occurred within the first 90 seconds after starting the experiment
or after changing the position of the light. Thus, by waving their hand
through the light in the early stages of the experiment, the babies were
learning about and remembering the position of the light. This very quickly
allowed them to accurately and prospectively control the deceleration of
the arm into the light and remain there, while effectively making the arm
clearly visible.
From an ecological perspective, the information for self-perception is
not restricted to a specific perceptual system. This brings us to the ques-
tion: Would newborn babies be able to control their arm movements by
means of sound? In order to answer this question, newborn babies
between 3 and 6 weeks of age were placed on their backs with the head
kept in the midline position with a vacuum pillow (Van der Meer & Van
der Weel, 2011). In this position, both ears were uncovered and available
for sound localization. Miniature loudspeakers were attached to the
baby’s wrists. The baby’s mother was placed in an adjacent room where
she could see her baby through a soundproof window. The mother was
instructed to speak or sing to her baby continuously into a microphone,
while the sound of her voice in real time was played softly over one of
the loudspeakers attached to the baby’s wrist. In order to hear her moth-
er’s voice, the baby would have to move the “sounding” wrist close to
the ear, and change arms when the mother’s voice was played over the
other loudspeaker. The results showed that newborn babies were able to
control their arms in such a way that the distance of the left and the right
wrist to the ear was smaller when the mother’s voice was played over
that wrist than when it was not. Further analyses showed that there were
significantly more reductions than increases in distance between wrist
and ear when the sound was on, while when the sound was off, the
number of reductions and increases in distance between wrist and ear
was about the same.
Thus, sighted newborn babies can precisely control their arms with the
help of both sight and sound. This implies that arm movements are not
simply reflexive, nor can they be explained away as excited thrashing of
the limbs. Neonates can act intentionally from the start, and they come
equipped with perceptual systems that can be used to observe the environ-
mental consequences of their actions. At the same time, actions provide
valuable information about oneself. This dual process of perceiving oneself
and perceiving the consequences of self-produced actions provides very
young infants with knowledge about themselves that is crucial for produc-
ing adaptive behavior (E. J. Gibson, 1988).
Establishing a Frame of Reference for Action
It seems plausible that the spontaneous arm waving of neonates of the kind
measured in our experiments is directed and under precise control.
Neonates will purposely counteract external forces applied to their wrists
to keep the hand in their field of view. They can also precisely control the
position, velocity, and deceleration of their arms to keep them clearly
visible. Moreover, they can direct their arms to their ears with the help of
sound. Their level of arm control, however, is not yet sufficiently developed
so that they can reach successfully for toys. Young babies have to do a lot
of practising over the first four or five months, after which they can even
catch fast-moving toys (Von Hofsten, 1983). What could be the functional
significance of neonatal arm movements for later successful reaching and
grasping?
To successfully direct behavior in the environment, the infant needs to
establish a bodily frame of reference for action (Van der Meer & Van der
Weel, 1995). Since actions are guided by perceptual information, setting
up a frame of reference for action requires establishing informational flow
between perception and action. It also requires learning about body dimen-
sions and movement possibilities. Thus, while watching their moving arms,
newborn babies pick up important information about themselves and the
world they move in––information babies need for later successful reaching
and grasping, beginning at around four or five months of age.
It is widely known that young infants spend many hours looking at their
hands (see Figure 7.2). And so they should, for infants have to learn many
lessons in ecological optics in those early weeks before they can success-
fully reach for and pick up toys in the environment. First of all, infants
have to learn that the hands belong to the self, that they are not simply
objects, but that they can be used to touch all sorts of interesting objects in
the environment. In order to successfully reach out and grasp toys, infants
also have to familiarize themselves with their own body dimensions in
units of some body-scaled or, more generally, action-scaled metric
(Warren, 1984). In other words, infants have to learn to perceive the
shapes and sizes of objects in relation to the arms and hands, as within
reach or out of reach, as graspable or not graspable, in terms of their
affordances for manipulation (Gibson, 1979/2015).
All this relational information has to be incorporated (Merleau-Ponty’s
term, see Marratto, 2012) into a bodily frame of reference for action in
those early weeks before reaching for objects develops. We have all experi-
enced this process of incorporation, namely, when learning new perceptuo-
motor skills (Tamboer, 1988). For instance, tennis rackets, skis, golf clubs,
and other extensions of the human body, such as false teeth and new cars,
first have to be incorporated into our habitual frame of reference, or
embodied action scheme (Day, Ebrahimi, Hartman, Pagano, & Babu,
2017), before we can use them to our full potential. At first, we experience
Figure 7.2 A newborn boy only a few hours old is studying his hand intensely.
those instruments as unmanageable barriers between the environment and

ourselves. However, once incorporated into our bodily frame of reference,
they increase our action possibilities considerably and are almost regarded
as our own body parts.
In this context, it is possible to speculate about the role of early arm move-
ments for distance perception in general. The late Professor Henk G. Stassen
was a mechanical engineer from Delft University of Technology in The Neth-
erlands. He specialized in man-machine systems, cybernetics, and ergonom-
ics, and was involved in designing artificial arms for babies who were born
with two stumps because of genetic disorders or because their mothers had
taken the drug thalidomide in the 1960s during pregnancy to prevent miscar-
riage. Stassen (personal communication, 1994) observed that if you fit babies
with artificial arms early, at around two to three months, they do not seem to
have any problems avoiding obstacles as soon as they learn to walk.
However, if the arms are fitted too late, the babies will have tremendous
problems perceiving affordances for navigation, and they will initially bump
into walls and obstacles when they start walking around their first birthday.
Gibson (1979/2015) suggested that the nose provides an absolute baseline for
distance perception (p. 110). Stassen’s observations would add that we per-
ceive distance in terms of our arm length, as within reach or out of reach.
During infancy, new skills are constantly appearing and bodily dimen-
sions are changing rapidly. In general, the bodily frame of reference has to
be updated during life to accommodate changes in action capabilities and
body characteristics. Sudden changes in action capabilities, as after a
stroke, show this very clearly, as do rapid changes in body size in preg-
nancy (Franchak & Adolph, 2014) and adolescence. Teenagers, for
example, can be notoriously clumsy; they undergo such sudden growth
spurts that their bodily frames of reference need to be recalibrated nearly
daily.
Successfully reaching out and grasping objects in the environment
require infants to be familiar with their own body dimensions. As infants
wave their arms while supine, they learn about their own body and its
dimensions through vision. It seems likely that a fast-growing organism
will constantly need to calibrate the system controlling movement, and
visual proprioceptive information is least susceptible to growth errors (Lee
& Aronson, 1974). This being so, our findings could have practical
implications for babies with visual deficits and for the early diagnosis of
premature babies at risk of brain damage. If early arm movements have an
important function for later reaching, then infants with signs of hypoactiv-
ity and/or spasticity of the arms should be monitored closely with respect
to retardation of developing reaching and possibly other perceptuo-motor
skills. In such cases, early intervention should concentrate on helping the
baby to explore its arms and hands, both visually and non-visually. A
simple intervention technique that could be used on babies with a visual
deficit is the use of brightly colored, high-contrast mittens, or placing
sounding proximity sensors around the baby’s wrists of the kind used in
cars for parking. Reaching out is typically the first developmental mile-
stone that blind babies fail to reach on time. The distinct sound always
accompanying that particular proprioceptive feeling when the arms move
around the face might enable the blind baby to establish a stable bodily
frame of reference for reaching based on auditory exploration of the self.
Getting Around with Light and Sound: The Role of

Prospective Control and Affordances
In order to act successfully, a baby needs not only to be able to perceive
environmental properties in relation to the self, in terms of their
affordances for action, as explained above, but the infant also needs to
control its movements by perceiving what is likely to happen next, which
requires prospective control (Lee, 1993; Turvey, 1992). To avoid colliding
with objects, the consequences of continuing the present course of action,
such as heading in a particular direction (Warren, Kay, Zosh, Duchon, &
Sahuc, 2001) or braking with a particular force (Lee, 1976, 2009), must be
perceived and evasive action taken in time. However, collisions are not
necessarily to be avoided. Getting around the environment in fact depends
on bringing about collisions of the feet with the ground, the hands with a
(moving) ball, and so on. These collisions have to be carefully controlled if
they are to secure the required propulsive force while avoiding injury to
the body. Thus, precise prospective control is needed. The where, when,
and how of the collision must be perceived ahead of time, and the body
must be prepared for it. Therefore, a crucial aspect of animal-environment
coupling concerns the pickup of predictive perceptual information. This
type of predictive information for prospective control is, in principle, avail-
able in the ambient (optic) array via tau (Lee, 2009). The variable tau (τ )
and its rate of change specify the time-to-contact between an approaching
object and any perceptual system. We will return to the concept of tau in
more detail later.
The ambient array is of fundamental importance in understanding direct
perception because it stands outside particular perceptual systems. It is the
input available to each and every one of them. “It matters little through
which sense I realize that in the dark I have blundered into a pigsty” (Von
Hornbostel, 1927, p. 83). The job of the perceptual systems is to pick up the
information in the flow field (Gibson, 1979/2015; Lee, 1980). But, are
infants capable of picking up such higher-order control information straight
from birth? One possibility is that at first, the infant’s perceptual systems are
only sensitive to lower-order variables, such as distance or velocity, and that,
with learning, the infant becomes more and more sensitive to the higher-
order variables, such as tau (Fajen, 2005b). It does not seem likely that
infants perform at random until they stumble onto the proper higher-order
variable, and thereafter “perceive directly.” Rather, merely correlated vari-
ables may be used early, and perhaps guide the search for a higher-order
informational complex (Jacobs & Michaels, 2007).
Take, for example, reaching for and intercepting a moving toy. Catch-
ing requires quite advanced timing and anticipation skills. It makes no
sense to move the hand to the place where the toy was last seen because by
the time the hand gets there, the toy will have moved further along its tra-
jectory. Thus, reaching for a moving object requires prediction of the
future location of the object, which in turn requires prospective control of
head, eye, and arm movements.
To test how prospective control of reaching develops, we investigated
reaching abilities in both full-term and preterm infants catching a toy
moving at different speeds (Van der Meer, Van der Weel, & Lee, 1994).
The toy was occluded from view by a screen during the last part of its
approach to force the infants to make use of predictive information. Babies
were tested at four-weekly intervals, between the ages of 16 and 48 weeks.
At 16 weeks, all babies showed visual interest in the toy, but none of them
were able to direct their arms to the toy to intercept it. In addition, as soon
as the toy disappeared behind the occluder, they lost interest in the toy and
seemed surprised when it reappeared at the other end. However, as soon
as the infants started reaching around 20–24 weeks of age, they anticipated
the reappearance of the moving toy with their gaze, suggesting that this
ability is a prerequisite for the onset of reaching for moving toys. From
about 40 weeks, infants showed advanced prospective control and geared
their actions of shifting gaze and moving the hand forward to certain
times, rather than distances, before the toy would reappear, consistent with
tau. In this way, infants make available the same time to catch successfully
whether the toy is moving slowly or quickly.
Our data from preterm infants show that the onset of reaching, antici-
pation of the moving toy with gaze and hand, and the switch to a more
efficient timing strategy was delayed in almost all premature babies (Van
der Meer, Lee, Liang, & Lin, 1995b), as well as their tau-coupling skills
(Kayed & Van der Meer, 2009; Lee, 1998). However, most preterms had
caught up with their full-term peers by the time they were one year of age,
corrected for prematurity. Two preterm infants who showed poorest antic-
ipation and had not switched to the time-strategy at one year of age, were
12–18 months later diagnosed as suffering from cerebral palsy, indicating
that poor development of prospective skill on the catching task might serve
as an indicator for brain damage. Some of the teenagers born preterm that
we tested on a similar task were also using a less sophisticated strategy for
timing their catching movements (Aanondsen et al., 2007).
Greater understanding of both normal and abnormal development of
use of perceptual information in prospectively guiding action might there-
fore have important diagnostic and therapeutic consequences (cf. Vaz,
Silva, Mancini, Carello, & Kinsella-Shaw, 2017). The mastering of reach-
ing and grasping normally develops very early, and it provides a founda-
tion for more specific perceptuo-motor skills based on these abilities.
Catching moving toys is such a case, requiring the pickup of predictive
information and quite advanced timing skills. Thus, if there is a problem
with basic interceptive skills, then more complex skills such as balancing
and speaking––skills that are highly dependent on prospective control––are
also likely to be affected later on in life. When studying dynamic balance
on force plates in toddlers, children, and students (Austad & Van der
Meer, 2007), the elderly (Spencer & Van der Meer, 2012), and patients
suffering from fibromyalgia and chronic fatigue syndrome (Rasouli, Stens-
dotter, & Van der Meer, 2016), we found that prospective control of
balance during gait initiation gradually improves during childhood and
middle adulthood, but then quickly deteriorates with age and for patients.
So far, we have seen that from very early on in life, babies show an
interest in their moving arms. By looking at their waving arms, they dis-
cover and learn about all the relationships essential for successful reaching
and grasping: They establish a stable bodily frame of reference for reach-
ing. Once established, the frame of reference then allows the baby to nego-
tiate actions that require taking into account properties of the environment,
such as toys moving at different speeds, requiring prospective control. Pro-
spective control demands that the affordances of the environment be per-
ceived. The properties of the environment are perceived in terms of their
affordances for action, and depend on the size, the level of development,
and the level of skill of the organism. Affordances are therefore not fixed
and, consequently, perception of affordances needs recalibrating during life
to accommodate changes in action capabilities and bodily characteristics.
This is particularly apparent during infancy, when new skills are constantly
appearing and bodily dimensions are changing rapidly (Adolph, Eppler, &
Gibson, 1993, see Adolph, Hoch, & Ossmy, Chapter 13, this volume).
Organisms do not perceive objects per se, but what these objects afford
for action. Acting successfully thus entails perceiving environmental prop-
erties in relation to oneself. When studying ducking under a barrier in
adults, nursery school children, and toddlers, we found that the affordance
of passability of a barrier is influenced not only by static variables such as
body size, but also by dynamic variables, such as speed of locomotion,
degree of motor control, and level of development. When subjects have less
control over their own vertical position in space, because they are running,
have cerebal palsy, or have only just learned to walk, a barrier affords
more cautious ducking behavior (Van der Meer, 1997b).
Interestingly, in a study where children with hemiparetic cerebal palsy
had to knock an approaching ball off a track, we found that the children
started the hitting action earlier with their affected arm, thus compensating
for the fact that this arm moves slower than the unaffected arm. Because
cerebral palsy is a congenital disorder, the children were used to the
affected arm being slower and more difficult to control, and made allow-
ances for this when initiating and continuously controlling their arm move-
ments using tau-coupling (Van der Weel, Van der Meer, & Lee, 1996).
The perceptual richness of a task also turns out to be crucial for
affordance perception, and has an effect on a child’s movement control. We
found that children with hemiparetic cerebal palsy find it easier to pronate
and supinate their forearm in an informationally rich and concrete “bang-
the-drum” task than in an abstract “move-as-far-as-you-can” task requiring
the same movement (Van der Weel, Van der Meer, & Lee, 1991). The factor
discriminating concrete and abstract tasks is the degree to which the act
required is directed to controlling physical interaction with the environment,
as opposed to producing movement for its own sake. Concrete tasks gener-
ally have greater informational support from the environment. In the con-
crete “bang-the-drum” task, the movement was controlled by visual,
auditory, and tactile information about the child’s relation to the drum, and
the attainment of the goal was readily perceptible by the child.
Perception of the environment has mostly been considered through
visual information. There is generally little research about the use of acous-
tic information for guided movement in the environment (but see Jenison,
1997; Lee, Van der Weel, Hitchcock, Matejowsky, & Pettigrew, 1992;
Russell & Turvey, 1999). Similar to vision, audition provides us with
spatial information over extended distances. Hearing may be even more
important than vision in orienting toward distant events. We often hear
things before we see them, particularly if they take place behind us or on
the other side of opaque objects, such as walls. One of infants’ first oppor-
tunities to move in the environment is by use of rotation skill in a prone
position. This skill emerges when infants are six or seven months old, and
requires them to use their arms and legs to rotate around their body axis.
Emergence of rotation skill provides the first opportunity for infants to
detect what is behind them, and to perform adequate whole-body move-
ments based on auditory information. Based on affordance research, we
investigated whether non-crawling infants would be able to use auditory
information to rotate along the shortest way to a sound source, relative to
their own position in space (Van der Meer, Ramstad, & Van der Weel,
2008). We found that in 88% of all trials, young infants were able to
rotate along the shortest way to their calling mum who was sitting behind
them. Infants chose the shortest way in 75% of the trials for the largest
angle (157.5 degrees) to 96% for the smallest angle (90 degrees), suggesting
that infants experience increased difficulty differentiating more ambiguous
auditory information for rotation. In general, use of auditory perception
for action has been a neglected research area in the ecological tradition.
Our results can contribute to the understanding of the auditory system as a
functional listening system where auditory information is used as a percep-
tual source for prospectively guiding behavior in the environment.
Toward a Developmental Neuroscience from an

Ecological Perspective
Indirect theories of perception take for granted the distinction between
sensation and perception and go on to explain the difference, for example,
when arguing whether the ability to enrich incoming sensations is innate
or learned, or a combination thereof (Michaels & Carello, 1981). The eco-
logical approach to perceptual learning and development (E. J. Gibson &
Pick, 2000), on the other hand, is based on a direct theory of perception.
Here, the information for perception is indefinitely rich and detailed and,
as a result, perceptual learning is a lifelong process of differentiating
information variables, rather than enriching bare stimulus input (Gibson
& Gibson, 1955). Traditionally, ecological psychology has opposed the
concept of enrichment, neglecting the brain in the process because that is
where the enrichment is supposed to take place and the mental representa-
tions reside. However, with the rise in neuroscience and brain imaging
techniques, the challenge for ecological psychologists is to study typical
and atypical functional brain development in a way that is consistent with
ecological theory (see also Chemero, 2009; De Wit, De Vries, Van der
Kamp, & Withagen, 2017).
Central to our developmental neuroscience approach are the ecological
concepts of prospective control and time-to-collision. For the past 25
years, we have been conducting developmental research on visual motion
perception from an ecological perspective (Agyei, Van der Weel, & Van
der Meer, 2016a). We have studied, at both the behavioral and brain level,
the visual motion paradigms of optic flow, looming, and occlusion in
infants. Our developmental studies show that the onset of self-produced
locomotion coincides with a surge in infants’ perceptual and brain devel-
opment. We show experimentally that the structure present in the optic
flow field aids participants of all ages to detect visual motion. We also
present evidence that invariants present in perceptual information are
merely reflected in the infant brain, consistent with Gibson’s (1966a)
concept of resonance. Recently, we have started to look for evidence of
vicarious use of brain tissue (Gibson, 1966a), where neurons temporarily
assemble to enable a given task.
The Visual Motion Paradigms of Optic Flow, Occlusion,

and Looming
Optic Flow
In a series of experiments, we simulated self-motion with structured optic
flow and compared it to unstructured random motion, while we measured
cortical responses to visual motion with high-density electroencephalogra-
phy (HD EEG, see Figure 7.3). Our studies show that both adults and
infants find it easier to pick up visual motion, as measured by shorter
latencies, when the information available to them is structured, as in optic
flow (Van der Meer, Fallet, & Van der Weel, 2008). We also find that at
the neural level, four-month-olds do not differentiate between simulated
forward, backward, and random visual motion, whereas older infants with
some weeks of crawling experience do (Agyei, Holth, Van der Weel, &
Van der Meer, 2015). Preterm infants at one year of age (corrected for pre-
maturity) show very little development in cortical activity in response to
visual motion (Agyei, Van der Weel, & Van der Meer, 2016b), which leads
us to suspect a dorsal stream vulnerability. As opposed to the ventral
stream that develops mainly after birth, the dorsal visual processing stream
develops during the last three months of pregnancy (Atkinson & Braddick,
2007). Preterm birth during this period seems to disturb the normal devel-
opment of the dorsal stream (e.g., Van Braeckel, Butcher, Geuze, Van
Duijn, Bos, & Bouma, 2008), possibly affecting the typical dorsal stream
functions of timing, prospective control, and visuo-motor integration.
In addition to direction of motion, we studied perception of motion
speed in a naturalistic setting where a vehicle driving down a virtual road
was simulated with optic flow (Vilhelmsen, Agyei, Van der Weel, & Van
der Meer, 2018; Vilhelmsen, Van der Weel, & Van der Meer, 2015). Adult
participants differentiated between direction and speed of motion when
they watched a road that was simulated by poles moving from near the
center of the screen and out (or in) toward the edges of the screen, creating
a realistic simulation of an optic flow field. Older infants between 8–11
Figure 7.3 A 4-month-old girl in deep concentration on the visual motion presented
on the large screen in front of her, while the corresponding electrical brain
activity (EEG) with a sensor net consisting of 128 electrodes is measured.
months with several weeks of crawling experience differentiated between

simulated backwards and forwards self-motion, but only at low driving
speeds of 17 km/h. Four-month-old infants without any locomotor experi-
ence, on the other hand, did not show any cortical evidence of being able
to discriminate between direction and speed of simulated self-motion. This
suggests that perceptual development does not happen in a vacuum, but
that perception and action develop hand in hand (Campos, Anderson,
Barbu-Roth, Hubbard, Hertenstein, & Witherington, 2000). As soon as
infants become mobile, the need for perceptual sensitivity to, for example,
whether an object is approaching on a collision course and, if so, when it
will collide, becomes much more urgent, and our brain data suggest that
perception of motion direction and speed takes several weeks of self-
produced locomotor experience to develop.
Occlusion
In most of our behavioral catching studies with infants described above,
the moving toy was occluded from view by a screen during the last part of
its approach to force the infants to make use of predictive visual informa-
tion for prospective control. Catching a toy moving at different speeds that
temporarily disappears behind an occluder before it can be caught, requires
prospective control of head, eye, and arm movements. Our catching studies
show that toward the end of the first year, most infants shift their gaze to
the other side of the occluder and start moving their hand forward at a
fixed time-to-contact before the toy had reappeared from behind the
occluder, as opposed to a fixed distance to the reappearance point (Van
der Meer et al., 1994, 1995b). This suggests that at first, infants’ percep-
tual systems are only sensitive to lower-order variables such as distance or
velocity and that, with perceptual learning, infants become more and more
sensitive to the higher-order perceptual variable of time-to-contact
captured by tau (Jacobs & Michaels, 2007; Van der Weel, Craig, & Van
der Meer, 2007).
We have just started to study brain electrical activity as a function of
perception of temporarily occluded moving objects by means of analyses in
the time-frequency domain in 8–12-month-old infants (Slinning, Ruther-
ford, & Van der Meer, 2018). Similar to Bache, Kopp, Springer, Stadler,
Lindenberger, and Werkle-Bergner (2015), we report gamma oscillations
in response to occlusion. In addition to brain data, we also collect data on
eye movements. When working with EEG, one has to find a way of “align-
ing” the enormous amount of data. Stimulus onset, a measure completely
outside the participant’s control, is traditionally used for this purpose in
research on event-related potentials. When studying timing in a visual
tracking task involving occlusion with adults (Holth, Van der Meer, &
Van der Weel, 2013), we show the importance of including behavioral
data when studying the neural correlates of prospective control. From an
ecological neuroscience perspective, we strongly recommend incorporating
behavioral measures that are actively controlled by the participant, such as
eye or reaching movements (Makeig, Gramann, Jung, Sejnowski, &
Poizner, 2009), into the EEG analysis, as this will make the brain data
clearer and easier to interpret.
Looming
How does the infant brain deal with information about imminent colli-
sions? By simulating a looming object on a direct collision course toward
infants, it is possible to investigate brain activities in response to looming
information. Looming refers to the last part of the approach of an object
that is accelerating toward the eye. To prevent an impending collision with
the looming object, infants must use a timing strategy that ensures they
have enough time to estimate when the object is about to hit them in order
to perform the appropriate evasive action. Defensive blinking is widely
considered as an indicator for sensitivity to information about looming
objects on a collision course. Infants must use time-to-collision informa-
tion to precisely time a blinking response so that they do not blink too
early and reopen their eyes before the object makes contact, or blink too
late when the object may already have made contact. For an accurate
defensive response to avoid collisions and prevent injury, development of
prospective control is important. Infants must use looming visual informa-
tion to correctly time anticipatory responses to avoid impending collisions.
We investigated the timing strategies that 5–7-month-old infants use to
determine when to make a defensive blink to a looming virtual object on a
collision course in a series of behavioral studies (Kayed, Farstad, & Van
der Meer, 2008; Kayed & Van der Meer, 2000, 2007). To time their
defensive blinks, the youngest infants used a strategy based on visual angle
analogous to the distance strategy described in the catching studies above.
As a result, they blinked too late when the looming object approached at
high accelerations. The oldest infants, on the other hand, blinked at a fixed
time-to-collision allowing them to blink in time for all the approach speeds
of the looming virtual object. When precise timing is required, the use of
the less advantageous visual angle strategy may lead to errors in perform-
ance, compared to the use of a strategy based on time-to-collision that
allows for successful performance irrespective of object size and speed.
With the presentation of a looming virtual object on a direct collision
course, we also studied the developmental differences in infants longitudin-
ally at 4 and 12 months using EEG and the visual evoked potential (VEP)
technique (Luck, 2005). The looms approached the infant with different
accelerations, and finally came up to the infant’s face to simulate an optical
collision. Measuring the electrical signal generated at the visual cortex in
response to visual looming, peak VEP responses were analysed using
source dipoles in occipital areas. Results showed a developmental trend in
the prediction of an object’s time-to collision in infants. With age, average
VEP duration decreased, with peak VEP responses closer to the loom’s
time-to-collision (Van der Meer, Svantesson, & Van der Weel, 2012).
Infants around 12 months of age with up to three months of crawling
experience used the more sophisticated and efficient time-to-collision
strategy when timing their brain responses to the virtual collision. Their
looming-related brain responses occurred at a fixed time of about 500 ms
before the optical collision, irrespective of loom speed. The use of such a
timing strategy based on a fixed time close to collision may reflect the level
of neural maturity in terms of myelinization as well as the amount of
experience with self-produced locomotion (Held & Hein, 1963). Both are
important factors required for accurate timing of evasive (and interceptive)
actions, and need to be continuously incorporated into the baby’s frame of
reference for action.
By localizing the brain source activity for looming stimuli approaching
at different speeds and using extrinsic tau-coupling analysis, the temporal
dynamics of neuronal activity in the first year of life was further investi-
gated (see Figure 7.4). Tau-coupling analysis calculated the tau of the
peak-to-peak source waveform activity and the corresponding tau of the
looms. Source dipoles that modeled brain activities within the three occipi-
tal areas of interest were fitted around peak looming VEP activity to give a
direct measure of brain source activities on a trial-by-trial basis. Testing
prelocomotor infants at 5–7 and 8–9 months and crawling infants at
10–11 months of age, we reported synchronized theta oscillations in
response to visual looming. Extrinsic tau-coupling analysis between the
external looms and the source waveform activities showed evidence of
strong and long tau-coupling in all infants, but only the oldest infants
showed brain activity with a temporal structure that was consistent with
the temporal structure present in the visual looming information (Van der
Weel & Van der Meer, 2009). Thus, the temporal structure of the different
looms was merely reflected in the brain, and not added to the brain, as
indirect theories of perception would have it. As infants become more
mobile with age, their ability to pick up the looms’ temporal structure may
improve and provide them with increasingly accurate time-to-collision
information about looming danger. Unlike young infants who treated all
the looms the same, older infants with several weeks of crawling experi-
ence differentiated well in their brain activity between the three loom
speeds with increasing values of the tau-coupling constant, K, for the faster
looms, as shown in Figure 7.4E–G.
The finding that changing patterns in the optical looming information
were reflected in the changing patterns in the neurological flow may shed
some light on the concept of resonance introduced by Gibson (1966a). The
variable tau (τ ) and its rate of change specify the time-to-contact between
an approaching object and the visual system (Lee, 2009). The same vari-
able was found to be operating in the neural flow when looming-related
activity was progressing through the infant brain. Thus, oscillatory activity
in the visual cortex was tau-coupled to the approaching looms, that is, the
change in the theta rhythm’s temporal structure was linearly correlated
with the value of tau of the looms. This, in our view, may indicate a
process of resonance in which informational and electrical flow are suc-
cessfully coupled in terms of the same variable tau via the coupling con-
stant K. However, how are these intricate processes of resonance further
organized in the infant brain?
Traditionally, it is assumed that there exists a one-to-one mapping
between brain structure and function, implying some kind of modular
organization of the brain (Fodor, 1981). In the case of our looming experi-
ments, this would involve a specific mapping procedure between the
incoming looming information and a specialized, encapsulated module in
the brain dealing with looming-related neural activity. Gibson (1966a),
however, suggested an alternative for this type of modular organization
when he introduced Lashley’s (1922) concept of vicarious use of brain
tissue, explaining that the same neural tissue can be involved in different
temporarily assembled structures suitable to a task. In other words, the
functioning of the neurons depends on the context in which they are oper-
ating. In this view, neurons can change function completely when incorp-
orated in different systems; they temporarily assemble to enable a given
task. Reed (1996b) introduced a different concept to stress the high degree
of flexibility of organization of the nervous system, namely, that of degen-
eracy. Degeneracy is the ability of elements that are structurally different
to perform the same function or yield the same output. These two concepts
express the highly flexible organization of the brain. Bullmore and Sporns
(2009) refer to this type of flexible organization as functional connectivity
as opposed to structural connectivity.
In our latest longitudinal looming results on 25 infants, we observed
both structural and functional organization principles in the infants’ brain
responses to the approaching looms (Van der Weel & Van der Meer,
2019). The location of electrical looming-related activity was stable across
all subjects and trials and occurred within a 1 cm3 area of the visual cortex.
These findings hint at a rather structural organization of brain activity in
response to looming. However, these findings may be explained by the
strict retinotopic organization of the visual system.
However, when it came to orientation of electrical looming-related
activity, the results tell an entirely different story, showing a high degree of
variability of activity that, in addition, was spread across a much larger
Figure 7.4 (A) Accelerating looming stimulus approaching the infants’ eyes result-
ing in increased theta-band oscillatory activity in the visual cortex. A
four-shell ellipsoidal head model was created for every trial and used as
a source montage to transform the recorded EEG data from electrode
level into brain source space. The results of this analysis for dipole
VCrL (visual cortex radial left, depicted in head model in light gray) are
shown for the three infant age groups in B–D. Each graph shows aver-
aged, peak-aligned source waveform (SWF) activity at dipole VCrL for
the three looms (in nanoampere, nA). Overall shape of the SWFs was
similar at the different ages, but their duration was about twice as long
in the 5- to 7-month-olds as compared to the 10- to 11-month-olds.
Note that SWF activity did not discriminate well between slow,
medium, and fast looms. Therefore, peak-to-peak SWF brain activity
was tau-coupled onto the corresponding part of the extrinsic loom to
study the temporal dynamics of neuronal activity. (E–G) Average tau-
coupling plots, tSWF vs tloom for each infant age group for the three loom
speeds, showing that crawling 10- to 11-month-olds differentiated well
between slow (in light gray), medium (in gray), and fast looms (in
black), with significantly higher values for the coupling constant, K, for
faster looms, whereas younger prelocomotor infants did not.
Source: From Van der Weel and Van der Meer (2009).
area of the visual cortex. This reveals a much more functional form of
organization with connectivity patterns emerging in various directions and
changing radically from trial to trial. With this type of flexible organiza-
tion, there is no need for a one-to-one mapping between brain structure
and function, as suggested by Fodor (1981). Instead, brain organization
can be flexible in the sense that structurally different neural tissue can be
involved in flexible temporarily-assembled structures and the functioning
of the neurons depends on the context in which they are operating. In this
view, neurons adhere to flexible principles; they temporarily assemble to
reveal the typical temporal characteristics of the approaching looms to the
brain.
The main objective of our developmental neuroscience research based
on the principles of Gibson’s ecological approach to visual perception has
always been to show that the brain is not adding to, structuring, or other-
wise enriching the incoming perceptual information, but that crucial
higher-order informational variables, about, for example, time-to-collision
via tau, are merely reflected by the brain. Our findings show that infants,
around their first birthday and after several weeks of self-produced crawl-
ing experience, clearly display in their looming-related brain activity a tem-
poral structure that is consistent with that present in the visual looming
information. Thus, invariants in the perceptual information specifying an
imminent optical collision are merely reflected in the more mature infant
brain, consistent with Gibson’s (1966a) concept of resonance. Our latest
developmental findings on looming provide evidence for degeneracy (Reed,
1996b) or vicarious use of brain tissue (Gibson, 1966a), where brain
organization is flexible, as neurons temporarily assemble to enable a given
task and change function completely when incorporated in different
systems (Van der Weel & Van der Meer, 2019).
Conclusion
What we tried to achieve by writing this chapter is to highlight the fact
that over the past 35 years, ever since we met as undergraduate students in
Amsterdam, we have been inspired and influenced by J. J. Gibson’s ecolo-
gical approach. Our developmental research on early arm movements and
interceptive timing skills emphasizes the importance of establishing a
bodily frame of reference for action as an anchor for both affordance per-
ception and prospective control of adaptive behavior. For us, Dave Lee’s
concept of tau (2009)––an example of an informational variable that can
be picked up directly––is instrumental in linking prospective control to
affordances.
The ecological approach is often accused of neglecting the brain when
explaining perception and action. We would argue here that the brain is
part and parcel of the perceptual and motor systems and therefore deserves
to play a role within ecological theory. However, departing from an
e cological approach to perception and action, the questions asked and the
answers that are considered satisfactory will be very different from those
arising from traditional perspectives. Therefore, the challenge for an ecolo-
gical or Gibsonian neuroscience is to study the brain in a way that is con-
sistent with ecological theory. Over the past 15 years, we have collected
evidence that the brain is not adding to, structuring, or otherwise enriching
the information coming in through the perceptual systems. Instead, the
(temporal) structure already present in the information appears to be
simply better reflected in the more mature infant brain after several weeks
of experience with self-produced locomotion. In our brain research, we
find the ecological concepts of resonance and vicarious function (Gibson,
1966a) and degeneracy (Reed, 1996b) increasingly useful.
8 A Guided Tour of Gibson’s
Theory of Affordances
Jeffrey B. Wagman
Although James J. Gibson contributed in myriad ways to experimental

psychology (and beyond) over a 50-year career, the concept of
“affordance” is arguably his most enduring legacy and his most influen-
tial contribution. The concept has taken hold in experimental psych-
ology, environmental psychology, human factors, communication,
neuroscience, and artificial intelligence. Moreover, the renowned philo-
sopher Daniel Dennett (2017) identified “Affordances” as the singular
scientific concept that ought to be more widely known by the general
public.
There have been multiple reviews of the empirical literature on percep-
tion of affordances (e.g., Dotov, de Wit, & Nie, 2012; Fajen, Riley, &
Turvey, 2009) as well as multiple rounds of theoretical development that
have refined the understanding of the concept (e.g., Chemero, 2003; Fajen,
2007; Rietveld & Kiverstein, 2014; Turvey, 1992; Withagen & van
Wermeskerken, 2010). Consequently, I will not attempt either here.
Rather, I will provide a guided tour of Gibson’s (1979/2015) chapter,
unpacking and explicitly connecting its contents with the theoretical and
empirical developments that have emerged in its substantial wake.
“The Theory of Affordances” is Chapter 8 in The Ecological Approach
to Visual Perception. That the chapter is included in Part II of the book
(“The Information for Visual Perception”) rather than in the subsequent
sections that more explicitly describe the process of detecting such informa-
tion reflects Gibson’s commitment that affordances are real, objective, and
ecological facts to be perceived. And Gibson gets right to this point in the
very first paragraph:
I have described the environment as the surfaces that separate substances

from the medium … But I have also described what the environment
affords … How do we go from surfaces to affordances? And if there is
information in light for the perception of surfaces, is there information
for the perception of what they afford? Perhaps the composition and
layout of surfaces constitute what they afford. If so, to perceive them is
to perceive what they afford. This is a radical hypothesis, for it implies
Guided Tour: Gibson’s Theory of Affordances 131
that the “values” and “meanings” of things in the environment can be
directly perceived.
(1979/2015, p. 119)
The short answer to Gibson’s first question is that we go from surfaces to

affordances when a point of observation in the optic array becomes occu-
pied by a perceiver (with a given set of action capabilities). Subsequently,
Gibson (1979/2015) writes, “affordances imply the complementarity of
the animal and environment” (p. 119). It might also be said that they are
the result of the complementarity between animal and environment. If
surfaces are the interface between substance and medium (see Nonaka,
Chapter 2, in this volume; Pagano & Day, Chapter 3, in this volume),
affordances are the interface between animal (qua action capabilities)
and surfaces (i.e., environment) (see Figure 8.1). Affordances are possib-
ilities for behavior emerging from relations between properties of animals
and properties of the environment. To some extent, the answer to Gib-
son’s second question is the topic of the entire book. Later, Gibson refers
to this as the “central question for the theory of affordances” (p. 132). It
may very well also be the central question for the ecological approach in
general.
What makes this hypothesis so radical is that the origin of meaning
(which has long bedeviled philosophers and psychologists) is neither phys-
ical nor psychological, but, rather, ecological. Affordances (and hence
meanings) emerge from relations between animal and environment. They
are activity-specific meanings of the surroundings (Turvey, 2013). If
Figure 8.1 Surfaces and affordances as interfaces.

132 Jeffrey B. Wagman
affordances are perceived, then meanings are perceived. For Gibson,
perception is cognitive—not because it is representational or computa-
tional, but because perception is of meaningful, complex, and emergent
relationships between perceiver and environment. Moreover, this is so
because there is complex and emergent information about such relation-
ships available in structured energy arrays.
Gibson then provides examples of affordances, starting with perhaps the
most basic of all affordances for a terrestrial animal––that of support (see
Figure 8.2). He writes:
If a terrestrial surface is nearly horizontal (instead of slanted), nearly

flat (instead of convex or concave), and sufficiently extended (relative
to the size of the animal) and if its substance is rigid (relative to the
weight of the animal), then the surface affords support.
(1979/2015, p. 119)
Perception and actualization of affordances for support and changes in

such processes with development, learning, and expertise in both typical
and atypical settings have been fertile topics of investigation in sub-
sequent decades (e.g., E. Gibson et al., 1987; Joh, Adolph, Narayanan,
& Dietz, 2007; Walter, Wagman, Stergiou, Erkman, & Stoffregen,
2017).
Returning to the theme of the emergence of affordances (and meanings),
Gibson writes:
[I]f a surface is horizontal, flat, extended, rigid, and knee-high relative

to a perceiver, it can in fact be sat upon. If it can be discriminated as
having just these properties, it should look sit-on-able … If the surface
properties are seen relative to the body surfaces, the self, they consti-
tute a seat and have meaning.
(1979/2015, p. 120)
Substance and surface properties structure reflected light (see Mace,

Chapter 5, in this volume). The structured light encountered at a point of
observation provides information about the possible relationships
between the perceiver and those substances and surfaces. Thus, when the
relationship between a surface and a perceiver is such that sitting is pos-
sible (Figure 8.3), the reflected light at a point of observation occupied by
that perceiver will provide information about this affordance (see Mark,
1987). Moreover, restrictions on the ability to explore such structure will
impair the ability to perceive such affordances (Mark, Balliet, Craver,
Douglas, & Fox, 1990, see Stoffregen, Yang, Giveans, Flanagan, &
Bardy, 2009).
Gibson also mentions affordances for climbing, falling off, getting
under, and bumping into as well as those for nutrition, manufacture, and
Figure 8.2 Affordances for support.
manipulation. Finally, he describes what are among the most complex

affordances of all—those of other animals and other people. With these
first few paragraphs, Gibson created research fodder for generations of
scientists to come.
Figure 8.3 Objects that afford sitting on (by humans).
The Niches of the Environment

“The Niches of the Environment” and “Man’s Alteration of the Natural
Environment” (the two subsequent sections) might seem to be distinct or
even unrelated, but they are inextricably linked (continuous, really) in
Gibson’s ecological approach. A niche is a way of life—a set of
affordances. Animals vary in their ways of life and in the sets of
affordances they encounter. They also vary in the complexity of their
nervous systems (and brains, if they possess one). Importantly, though,
this ought not to matter in perception of affordances. From Gibson’s
ecological perspective, a particular animal-environment relationship
lawfully structures patterned energy arrays, such that this structure is
specific to (provides information about) this relationship. With respect
to perceiving a given affordance, the details of a given animal’s nervous
system (and brain) may be irrelevant, so long as that animal can detect
the structure in that array that specifies the relevant animal-environment
relationship.
Accordingly, research has shown that species representing phyla across
the animal kingdom (including worms, spiders, mollusks, crabs, frogs,
snakes, mice, rats, hamsters, and dogs) are sensitive to affordances
(Branch, 1979; Heyser & Chemero, 2012; Jayne & Riley, 2007; Jiménez,
Sanabria, & Cabrera, 2017; Reed, 1982a; Sonoda, Asakura, Minoura,
Elwood, & Gunji, 2012; Wagman, Langley, & Farmer-Dougan, 2017)—in
many cases, in ways analogous to humans. For example, for both humans
and frogs, perception of affordances for fitting through an aperture
depends on the size of the body in motion (Franchak, Celano, & Adolph,
2012; Ingle, 1973), and for humans, dogs, rats, and hamsters, perception
of affordances for reaching depends on the length of the relevant effector
and the relative comfort of a given reaching mode (Cabrera, Sanabria,
Jiménez, & Covarrubias, 2013; Carello, Grosofsky, Reichel, Solomon, &
Turvey, 1989) (Figure 8.4).
Man’s Alteration of the Natural Environment

Why has man (sic) changed the shapes and substances of his environment?
To change what it affords him (sic) … It is a mistake to separate the natural
from the artificial as if there were two environments. It is also a mistake to
separate the cultural … from the natural environment.
(Gibson, 1979/2015, p. 122)
Just as there is a continuity between perception of affordances by human

and non-human animals, there is a continuity between perception of
affordances in the natural and built environments (see Heft, 2007;
Figure 8.4 Many animal species perceive affordances for reaching.

Source: Dog photo from Wagman et al. (2017), Figure 1. Reprinted with permission from
Springer Nature. Rat and hamster photos, courtesy of Felipe Cabrera.
ithagen & van Wermeskerken, 2010). Parts I and II of The Ecological
W
Approach to Visual Perception are devoted to developing an ecological
physics—a description of how animal-environment relationships, under-
stood as affordances, lawfully structure energy arrays such that they
provide information about those relationships to a perceiver. The con-
tinuity between natural and built environments implies that it is possible to
design environments such that they give rise to information about
affordances (see Figure 8.3). Accordingly, this has been described as an
inverse ecological physics (Flach, 1990). Such principles have been used to
design display interfaces for complex work environments such as battle-
fields, cockpits, power plants, and hospitals (see Bennett, 2017; Effken,
2006; Vicente, 2002) as well as for architectural design (e.g., Prieske,
Withagen, Smith, & Zaal, 2015; Rietveld, 2016; Sporrel, Caljouw, &
Withagen, 2017).
The concepts of niche and human alteration of the environment are
continuous in that all animal species alter their environments, thereby
constructing their own niches and creating their own affordances (see
Withagen & van Wermeskerken, 2010). Humans do so more than any
other species, especially in the creation of communication and representa-
tion systems, artifacts, dwellings, and social structures. In this way, the
continuity between the natural and built environments extends to the
cultural environment (Costall, 2012; Heft, 2007, 2017; Rietveld &
Kiverstein, 2014).
Some Affordances of the Terrestrial Environment
The Surfaces and Their Layouts

Equilibrium and posture are prerequisite to other behaviors, such as loco-
motion and manipulation … the ground is quite literally the basis of the
behavior of land animals. And it is also the basis of their visual
perception …
(1979/2015, p. 123; original emphasis)
Gibson makes two important points about affordances in the above quote,
both somewhat subtly. First, he describes behaviors (and hence,
affordances) as hierarchically nested over both space and time (Reed,
1996b; Stoffregen, 2003a; Wagman & Miller, 2003), laying the ground-
work for the description of affordances as “quicksilvery” (Chemero &
Turvey, 2007), emerging and dissolving from moment to moment as
behavior unfolds (see Figure 8.5). Second, he argues that direct contact
with a ground surface entails direct contact with information about that
ground surface. That is, direct behavior entails direct perception (Turvey,
2013).
Figure 8.5 Performing a given behavior creates and eliminates affordances.
The earth has “furniture” … It is cluttered … The solid, level, flat

surface extends behind the clutter … This is not, of course, the earth
of Copernicus; it is the earth at the scale of the human animal, and on
that scale it is flat, not round.
(Gibson,1979/2015, p. 123)
Gibson developed an ecological physics because he felt that standard
physics and geometry were inappropriate for understanding relationships
between animals and environments. Newton’s physics and Euclid’s geome-
try are useful for describing properties of abstract, imaginary, idealized,
scale-free, or closed systems. Yet, animal-environment systems are con-
crete, real, actualized, scale-dependent, and open. For Gibson, laws of
geometry and physics should not be the a priori basis from which an
understanding of perception-action achievements of animals is developed.
Rather, it should be that perception-action achievements of animals are the
a priori basis from which the laws of geometry and physics are developed
(Turvey, 2004) (see Figure 8.6).
Whereas a nearly horizontal, nearly flat, and sufficiently extended
surface affords support and unimpeded locomotion:
[a] vertical, flat, extended rigid surface such as a wall or cliff face is a
barrier to pedestrian locomotion. Slopes between vertical and horizon-
tal afford walking, if easy, but only climbing, if steep, and in the latter
case the surface cannot be flat; there must be “holds” for the hands
and feet.
(Gibson, 1979/2015, p. 124)
Accordingly, people can perceive affordances for walking up inclined sur-

faces (e.g., Fitzpatrick, Carello, Schmidt, & Corey, 1994; Kinsella-Shaw,
Shaw, & Turvey, 1992; Regia-Corte & Wagman, 2008) and for climbing
up vertical surfaces (e.g., Pijpers, Oudejans, & Bakker, 2007; Seifert,
Cordier, Orth, Courtine, & Croft, 2017).
An insufficiently extended surface affords tumbling down or falling
off. In Gibson’s words: “the brink of a cliff is a falling-off place. It is
dangerous and looks dangerous. The affordance of a certain layout is
perceived if the layout is perceived” (1979/2015, p. 124). Critically, what
is a falling-off place, and, hence, what looks like a falling-off place
depend on the animal and its action capabilities. Along these lines,
infants’ perception of affordances of surfaces depends on their experience
moving and keeping their balance in different postures in the course of
development. Between 6 and 12 months of age, infants learn to sit, then
crawl, and finally walk. Infants who are experienced sitters but novice
crawlers refuse to reach across impossibly wide gaps when tested in their
experienced sitting posture but attempt to do so when tested in their
novice crawling posture. Experienced crawling infants refuse to crawl
down impossibly steep slopes or high drop-offs but when tested in a
novice walking posture, they attempt to walk straight over the brink
(Adolph, Kretch, & LoBue, 2014; Kretch & Adolph, 2013; see Adolph,
Hoch, & Ossmy, Chapter 13, in this volume). In short, perceivers are
aware of (and become attuned to) affordances—not to metric properties,
such as distances, heights, or slopes.
Figure 8.6 Relationship between physics and geometry and perception-action in

standard (top) and ecological (bottom) approaches.
Returning to the concepts of niche, human alteration of the natural

environment, and niche construction, Gibson writes: “People have altered
the steep slopes of their habitat by building stairways so as to afford ascent
and descent. What we call the steps afford stepping, up or down, relative
to the size of a person’s legs” (1979/2015, p. 124).The first part of this
passage, of course, inspired the very first experimental investigation of
affordances. In a paradigm-establishing study, Warren (1984) found that the
boundary between riser heights that are perceived to be climbable and those
that are not occurs at body-scaled ratio (of riser-height-to-leg-length) pre-
dicted by biomechanical modeling. What is often underappreciated is that
Warren also found that the preferred riser height occurs at a body-scaled ratio
predicted by the metabolic costs of stair climbing. Follow-up work in the sub-
sequent decades focused on age-dependent changes in perception of climbabil-
ity (Konczak, Meeuwsen, & Cress, 1992) and age-independent invariants
that support such perception (Cesari, Formenti, & Olivato, 2003).
In many cases, locomotion is still afforded even when obstacles are
present. As Gibson writes: “ordinarily, there are paths between obstacles
and these openings are visible” (1979/2015, p. 132). Along these lines,
Warren and Whang (1987) found that the boundary between aperture
widths that are perceived to be pass-through-able and those that are not
occurs at body-scaled ratio (of aperture-width-to-shoulder-width) and that
optical information at a point of observation (at the eye-height of the per-
ceiver) specifies this affordance. Subsequent work has shown that percep-
tion of affordances for fitting through horizontal or vertical apertures is
better predicted by the size of the body in motion (e.g., dynamic walking
height) than by static body measurements (e.g., standing height) (Franchak,
Celano, & Adolph, 2012). Other research has investigated perception of
affordances for passing through apertures when body size is altered by an
external object (e.g., a wheelchair or athletic equipment) (Higuchi, Cinelli,
Greig, & Patla, 2006; Higuchi, Takada, Matsuura, & Imanaka, 2004) or
body growth (Franchak & Adolph, 2014; van der Meer, 1997b), and
changes in perception that accompany practice performing such behaviors
(Franchak, van der Zalm, & Adolph, 2010; Higuchi et al., 2011; Stoffre-
gen, Yang, Giveans, Flanagan, & Bardy, 2009).
The Objects
Gibson again highlights the differences between ecological physics and tradi-
tional physics by differentiating between attached and detached objects: “We
are not dealing with Newtonian objects in space, all of which are detached
but with the furniture of the earth, some items of which are attached to it
and cannot be moved without breakage” (1979/2015, p. 124). Detached
objects afford manipulation, and this is Gibson’s focus in the remainder of
this section. Graspable objects “have opposite surfaces separated by a dis-
tance less than the span of the hand” (p. 125). Subsequently, choices about
whether and how an object is reachable as well as which grasp configuration
should be used (e.g., number of digits in a single-hand grasp, whether the
object is grasped with one or two hands, or with a hand-held tool) are scaled
to the person’s anthropometric properties (Cesari & Newell, 2000; Richard-
son, Marsh, & Baron, 2007; Wagman & Morgan, 2010) (Figure 8.7).
Figure 8.7 Objects that afford grasping with one hand and with two hands.
Graspable detached objects afford manipulation. “An elongated object

of moderate size and weight affords wielding. If used to hit or strike, it is a
club or a hammer” (Gibson, 1979/2015, p. 125). The first sentence fore-
shadowed an entire program of research on perception by effortful or
dynamic touch, showing that perception of geometric and functional prop-
erties of a wielded object is constrained by how that object resists being
manipulated by muscular forces about a joint (see Carello & Turvey,
2017). The second sentence foreshadowed specific research showing that a
wielded object is perceived to afford striking-with to the extent that it facil-
itates the transference of appropriately scaled forces (Wagman, Caputo, &
Stoffregen, 2016; Wagman & Carello, 2001).
In most cases, an object that can be manipulated can be thrown—“a
graspable rigid object of moderate size and weight affords throwing”
(Gibson, 1979/2015, p. 125). Even among object manipulation tasks, throw-
ing is a skilled behavior requiring particularly tuned perception-action skills.
Part of this tuning is determining which objects can be thrown to which dis-
tances. To this end, objects that have a particular felt heaviness are perceived
to optimally afford long-distance throwing (e.g., Bingham, Schmidt, &
Rosenblum, 1989; Zhu & Bingham, 2010, 2011). The ability to strike a
target with a projectile requires additional skill. To this end, targets appear
larger to an archer when his or her form affords precision shooting than
when it does not (Lee, Lee, Carello, & Turvey, 2012).
Gibson dedicates the rest of this section to establishing the primacy of
perception of affordances as opposed to physical properties: “Orthodox
psychology asserts that we perceive these objects insofar as we discriminate
their properties or qualities … But I now suggest that what we perceive
when we look at objects are their affordances, not their qualities …”
(1979/2015, p. 125) (Figure 8.8).
Figure 8.8 Perceptual experience vs. artificial measurement (M) devices.
In Gibson’s ecological approach, the animal-environment relations that

define affordances are perceived not as a collection of discrete (lower-order)
properties but rather as an emergent, higher-order “complex particular”
(Turvey, 2015). Affordances are perceived as such, without necessitating
prior independent perception of properties of either animal or environment
(e.g., Mark, 1987; Stoffregen, 2000a, 2003a). For example, perception of
maximum jumping-reach-height is not reducible to a combination of percep-
tion of constituent lower-order affordances (i.e., maximum-jump-height plus
maximum-reach-height; Thomas, Hawkins, & Nalepka, 2017; Thomas, &
Riley, 2014; see Thomas, Riley, & Wagman, Chapter 14, in this volume).
Moreover, there is a dissociation between improvements in abilities to per-
ceive a given affordance and abilities to perceive physical properties constitu-
ent of that affordance (Higuchi et al., 2011; Mark, 1987; Thomas, Wagman,
Hawkins, Havens, & Riley, 2017; Yasuda Wagman, & Higuchi, 2014).
Relatedly, people with psychological disorders such as schizophrenia show
impaired ability to perceive affordances of a given object but not the phys-
ical properties of that object, a result that is consistent with the reduced
sense of agency in this population (Kim & Kim, 2017).
To Perceive an Object Is Not to Classify an Object

Before moving on, Gibson provides an interesting sidebar on what has
become an emerging topic of investigation in ecological psychology:
The fact that a stone is a missile does not imply that it cannot be other
things as well. It can be a paperweight, a bookend, a hammer, or a
pendulum bob … The theory of affordances recuses us from the philo-
sophical muddle of assuming fixed classes of objects …
(1979/2015, p. 126)
Gibson is stating the plainly obvious, but theoretically challenging fact

that any given object has many affordances—even for the very same
animal at the very same moment. A stone can be thrown, stacked,
pounded with, or stepped on by a given animal with appropriate action
capabilities. Alternatively, a stone can afford none of these behaviors
for a different animal with inappropriate action capabilities. This leads
to the seemingly paradoxical state of affairs in which a given object
simultaneously does and does not afford a particular set of behaviors.
However, this is only paradoxical from the perspective of traditional
physics in which a given object is a set of animal-independent prop-
erties. It is not so from the perspective of ecological physics in which a
given object is a superposition of animal-dependent properties that
become actualized in specific contexts (Turvey, 2015) (see Figures 8.5
and 8.6).
The unstated, but equally true (and equally theoretically challenging)
corollary to this fact is that a given affordance can be actualized with many
different objects. Throwing can be performed with a stone, a ball, or a
coffee mug (if so inclined). In any given situation, there is a multiplicity of
affordances—a many-to-many relationship between action capabilities and
environmental properties. At any given moment, there are multiple
affordances and multiple means by which to actualize each affordance.
Consequently, it is often insufficient for a perceiver to merely choose
whether to actualize a given affordance. Rather, he or she must choose
which affordance to actualize as well as when and how to do so. Such
choices are based not on the ability to perform an isolated behavior but
rather on the ability to perform that behavior in the context of overarching
goals and task constraints, including metabolic costs, penalties for errors,
and comfort (Comalli, Franchak, Char, & Adolph, 2013; Comalli,
Persand, & Adolph, 2017; Mark et al., 1997; Wagman, Bai, & Smith,
2016). In short, people are sensitive to hierarchically nested affordances
(Wagman et al., 2016b).
Other Persons and Animals

Gibson’s ecological approach applies not only to the understanding of per-
ception of affordances in animal-environment systems, but also in animal-
animal-environment systems (see Marsh, Richardson, Baron, & Schmidt,
2006; Richardson, Marsh, & Schmidt, 2010). A niche is a set of
affordances, and the human niche includes other people:
The richest and most elaborate affordances of the environment are
provided by other animals and for us, other people … Behavior affords
behavior and the whole subject matter of psychology and of the social
sciences can be thought of as an elaboration of this fact.
(Gibson, 1979/2015, pp. 126–127)
Accordingly, people can perceive whether another person can perform a

given behavior and, if so, how and when that person ought to do so.
Importantly, such sensitivity reflects the action capabilities of the other
person in the context of overarching goals and task constraints. In short,
people are also sensitive to hierarchically nested affordances for another
person (Cordovil, Santos, & Barreiros, 2012; Passos, Cordovil, Fernandes,
& Barreiros, 2012; Ramenzoni, Davis, Riley, & Shockley, 2010; Ramen-
zoni, Riley, Shockley, & Davis, 2008; Wagman, Stoffregen, Bai, &
Schloesser, 2018).
Continuous changes in an environmental property can lead to spon-
taneous, abrupt, and nonlinear transitions between (perception of )
affordances for another (or group) as well as for the self. For example,
when people are asked to move objects from one location to another, they
spontaneously transition from using a one-hand grasp to a two-hand grasp
to a two-person grasp as the object size increases (Richardson et al., 2007).
That is, just as in other self-organizing phenomena, individual components
of a system spontaneously coordinate behavior at a critical value to satisfy
a goal under constraints. Such results show that the principles that underlie
perception of affordances for an individual are continuous with those that
underlie perception of affordances for another person or group (see Marsh,
Richardson, Baron, & Schmidt, 2006). Gibson explicitly makes this
point here:
The perceiving of these mutual affordances is enormously complex but

it is nonetheless lawful and it is based on the pickup of the informa-
tion in touch, sound, odor, taste, and ambient light. It is just as much
based on stimulus information as the simpler perception of the support
that is offered by the ground under one’s feet.
(1979/2015, p. 127)
That is, the continuity between the perception of affordances across human
and non-human animals and in the natural and built environments extends
to the perception of affordances for the self and other. In all such cases,
higher-order, complex, and emergent patterns in structured energy arrays
provide information about affordances. For example, the ability to per-
ceive another person’s maximum reach-with-jump-height is dependent on
the detection of kinematic patterns informative about that person’s ability
to produce task-specific forces with the legs. Moreover, the ability to per-
ceive this affordance for another person improves after watching that
person (or a point light representation of that person) perform task-
relevant behaviors (e.g., walking or squatting) but not task-irrelevant
behaviors (e.g., twisting or standing) (Ramenzoni, Riley, Davis, Shockley,
& Armstrong, 2008; Ramenzoni et al., 2010). Moreover, athletes are
better attuned to information about sport-specific abilities of others than
are non-athletes but are no better attuned to information about non-sport-
specific abilities of others (Weast, Shockley, & Riley, 2011; Weast, Walton,
Chandler, Shockley, & Riley, 2014).
Places and Hiding Places

The habitat of a given animal contains places. A place is not an object with
definite boundaries but a region. The different places have different affordances
… Animals are skilled at what the psychologist calls place learning.
(Gibson, 1979/2015, p. 127)
Of course, part of place learning is learning how to get to and from a par-
ticular place. Along these lines, this passage foreshadowed work showing
that human odometry—nonvisual (kinesthetic) perception of places and their
distances—is based on detection of variables that remain invariant over
exploratory locomotion (Harrison & Turvey, 2010; Turvey et al., 2009).
The Origin of the Concept of Affordances: A Recent

History
For Gibson, the affordance concept differs from the Gestalt concepts of
“demand character,” “invitation character,” and “valence” in that “The
affordance of something does not change as the need of the observer
changes. The observer may or may not perceive or attend to the affordance
… but the affordance, being invariant, is always there to be perceived”
(1979/2015, p. 130).
However, with an argument based on industrial design, architecture,
and phenomenology, Withagen and colleagues (Withagen, Araújo, & de
Poel, 2017; Withagen, de Poel, Araújo, & Pepping, 2012) have argued that
affordances can, in fact, invite behavior but in a way that is rooted in more
permanent evolutionary, social, cultural, and personal history (e.g., social
norms) than in momentary psychological states (see Heft, 2017; Rietveld
& Kiverstein, 2014).
The Optical Information for Perceiving Affordances

The perceiving of an affordance is not a process of perceiving a value-free
object to which meaning is somehow added … it is a process of perceiving a
value-rich ecological object … Physics may be value free, but ecology is not.
(Gibson, 1979/2015, pp. 131–132)
Here, Gibson reasserts that affordances (and hence meanings and values)
emerge in the relations between animal and environment—they are inher-
ent neither in the animal nor in the environment but only in animal-
environment systems (Chemero, 2009). Affordances for climbing stairs, for
example, only emerge in a system that includes an animal, stairs, the
mutual compatibility of the two with respect to stairclimbing, and a given
occasion. Therefore, perceiving an affordance means perceiving a system
of which the animal is a part (Turvey, 2013). At first pass, this seems para-
doxical. But Gibson reminds the reader that:
[A]n affordance, as I said, points two ways, to the environment and to

the observer. So does the information to specify an affordance … to
perceive the world is to co-perceive oneself … The awareness of the
world and of one’s complementary relations to the world are not
separable.
(1979/2015, pp. 132–133)
In other words, an animal perceives a system of which it is a part because

what it perceives is inextricably tied to the surroundings, the animal itself,
its point of observation, and its movements—which are inextricably tied to
its action capabilities (Petrusz & Turvey, 2010). Relations are perceived
because the information is relational—it is also determined by the animal’s
surroundings, the animal itself, its point of observation, and its move-
ments. Though it often goes unrecognized, Gibson was the original propo-
nent of embodied, embedded, and situated cognition (Chemero, 2009;
Fultot, Nie, & Carello, 2016).
For Gibson, the complexity of the information specifying affordances is
not a stumbling block for the perceiver, nor should it be for the scientist:
[A] compound invariant is just another invariant. It is a unit and the

components do not need to be combined or associated … I have
described the invariants that enable … two or more children to perceive
the same shape at different points of observation … to perceive the
common affordance of the solid shape despite the different perspectives.
(1979/2015, p. 133)
That the same affordance can be perceived from multiple points of obser-
vation foreshadowed and inspired work on the perceptual constancy of
affordances—that perception of affordances for a given behavior reflects a
person’s action capabilities over the variety of circumstances in which that
affordance is encountered (Turvey, 1992; Wagman & Day, 2014).
Affordances for a given behavior can be perceived by means of different
anatomical components, from different points of observation, and under
different task constraints (Cole, Chan, Vereijken, & Adolph, 2013;
Wagman & Hajnal, 2014a, 2014b, 2016).
Misinformation for Affordances
Before concluding the chapter, Gibson comments on what he calls the
“misinformation for affordances.” He writes: “According to the theory
being developed, if information is picked up, perception results; if mis
information is picked up, misperception results” (1979/2015, p. 133). This
is a subtle, but profound statement about the ecological approach. In tra-
ditional approaches, perception is the result of a computational or inter-
pretive process, and perception is accurate (or inaccurate) to the degree
that the outcome of this process matches that of an artificial measuring
device (e.g., a scale, a ruler, or a protractor). In Gibson’s ecological
approach, however, perception is a lawful relationship between perceiver
and environment. Consequently, the “accuracy” of perception cannot be
evaluated, and misperception is not an error. Moreover, if perception is
primarily (or exclusively) of affordances, then the experience of the per-
ceiver is very much unlike the output of artificial measuring devices (see
Figure 8.8). Therefore, so-called ‘illusions’ do not invalidate the ecological
claim that perception is direct so much as they challenge researchers to dis-
cover lawful relationships between the information available at a point of
observation and affordances.
Gibson argues that a theory of perception should be developed from
the countless everyday successes of perception rather than from the rare
(and often artificially induced) so-called failures of perception. For an
infant who refuses to crawl across a visual cliff and an adult who walks
into a sliding glass door, in neither case is perception in error.
Affordances of the respective surfaces were (not) perceived, but this is
only because the information specifying those affordances was (not)
detected (or not present). “These two cases are instructive. In the first, a
surface of support was mistaken for air because the optic array specified
air. In the second a barrier was mistaken for air for the same reason”
(p. 134). Both perception and misperception are the detection of
information. In some (rare) cases, the information specifies one state of
affairs when another state of affairs is so. In other cases, perceivers are
not sufficiently attuned to relevant information (see Adolph, 2008;
Adolph et al., 2014; Kretch & Adolph, 2013) or are prevented from
exploring the structured energy array such that this information can be
detected (Mark et al., 1990; Stoffregen et al., 2009; Yu, Bardy, & Stof-
fregen, 2010).
Conclusion
In the concluding paragraphs of the chapter, Gibson argues that
affordances are among the most fundamental relationships between animal
and environment and play primary roles in shaping the evolution of species
and ontogenetic development of individuals:
The medium, substances, objects, places, and other animals have
affordances for a given animal … They offer benefit or injury, life or
death. This is why they must be perceived.
The possibilities of the environment and the way of life of the
animal go together inseparably …
(1979/2015, pp. 134–135)
He then returns to the central hypothesis of the theory of affordances and

perhaps of the ecological approach in general:
The hypothesis of information in ambient light to specify affordances

is the culmination of ecological optics. The notion of invariants that
are related at one extreme to the motives and needs of an observer and
at the other extreme to the substances and surfaces of a world provides
a new approach to psychology.
(p. 135)
Such a notion has indeed provided a new approach to psychology—one

that has helped to reveal the lawful bases of perception and actualization
of affordances. To a large extent, scientific psychology is the science of
agency—the ability to select, perceive, and actualize affordances appropri-
ately based on intention. Investigating perception of affordances over the
variety of circumstances in which they are encountered, and by the variety
of species that encounter them, will continue to make progress toward
Gibson’s goal of bringing scientific psychology into closer alignment with
the natural sciences (Turvey, 2013; Wagman, 2010).
Part III
Visual Perception
9 Perceiving Surface Layout
Ground Theory, Affordances, and
the Objects of Perception
William H. Warren
The Ecological Approach to Visual Perception was James Gibson’s

(1979/2015) final statement of his theory of perception, which had steadily
evolved over a lifetime of research and writing (Reed, 1988). In Chapter 9,
Gibson critiques his earlier views and almost—but not quite—embraces a
fully ecologized theory of the direct perception of surface layout. The ques-
tion that emerges from this chapter is whether the objects of perceptual
awareness are geometric properties, such as size, distance, and shape, or
ecological properties, such as the affordances of surface layout they under-
write, or both. To put it provocatively (as Gibson was wont to do), one
might ask whether affordances are perceivable but the geometric layout
per se is not (cf. Gibson, 1975).
Gibson’s theory of layout perception was prescient in many ways. His
functional analysis of optical information became central to modern
research in human and machine vision (Marr, 1982). The fundamental
importance of the ground and the horizon for layout perception has been
empirically vindicated (Bian, Braunstein, & Andersen, 2005; Sinai, Ooi, &
He, 1998). His concept of affordances has been taken up in psychology,
robotics, and design (see Hsu, 2019; Wagman, Chapter 8, in this volume).
Yet a number of vexing puzzles about perceiving surface layout persist, to
which I will suggest some resolutions here. Following Gibson’s Chapter 9,
I will focus on static monocular perception of layout in the open field.1
Reconsidering the problem of layout perception will force us to face the
question, what, exactly, is perceived?
Gibson’s Theory of Layout Perception, Updated

Chapter 9 elaborates two essential aspects of Gibson’s unfolding percep-
tual theory. First, it offers a pithy statement of what he meant by “direct
perception”: perceiving the environment by picking up optical informa-
tion, unmediated by an awareness of retinal or mental images, or by a
process of inference (Gibson, 1979/2015, pp. 139, 141). Second, it
develops this information-based account for the perception of surface
layout. These two threads are historically entangled, for it was the failure
152 William H. Warren
of the traditional cue-based account of depth perception for testing pilots
during World War II that drove Gibson to the view that perception is
direct.
Based on that experience, Gibson (1950a) rejected what he called an
air theory of perception, in which the size and distance of objects in
empty space are inferred from 2D retinal images with the aid of tradi-
tional depth cues. He replaced it with a ground theory, in which the
observer perceives not space per se but recession along the ground and
objects arrayed on the ground. Moreover, Gibson believed he had identi-
fied stimuli for geometric properties of layout such as size, distance, and
slant, in higher-order patterns of optical structure. Such observations led
him to formulate a new psychophysics of perception in place of classical
sensory psychophysics, a “stimulus-response psychology” (Gibson,
1959), in which “a percept was an automatic response” to such higher-
order stimuli (Gibson, 1979/2015, p. 141).
But in Chapter 9 he dismantles the psychophysical stimulus-response
(S-R) formulation of the 1950s. This shift began with his conception of
ecological optics and information as specificity to environmental properties
(Gibson, 1961b), and continued with a reformulation of perception as an
act of exploring, attending to, and picking up such information (Gibson,
1966a). Chapter 9 presents his mature information-based theory in which
an active agent seeks and detects information and perceives layout in the
service of goal-directed action. Some 40 years later, I want to update this
story in some, er, depth.
The Ground Surface

The foundation of layout perception in the wild, according to Gibson, is a
homogeneously textured ground surface with an explicit or implicit
horizon. This is the terrestrial context of constraint in phylogeny and onto-
geny: a continuous horizontal ground surface in a gravitational field, on
which the observer stands and other objects rest. In this context, visual
information is available to specify an object’s location on the ground
surface, distance from the observer, distance from other objects, relative
size, and slant. Even if space per se is not perceivable, spatial relations in
the ground plane are visually specified.
Consequently, if terrestrial constraints are violated, predictable errors
will ensue, as observed with floating objects, an elevated observer, and a
sloping or discontinuous ground surface. An enchanted laboratory of such
effects is The Mystery Spot, a roadside attraction in Santa Cruz, CA, where
a tilted shed has been built on a sloping hillside. These alterations to the
ground surface and the visual framework yield startling distortions of per-
ceived size and orientation (Shimamura & Prinzmetal, 1999), including
people who grow and shrink and water that flows uphill. Although such
illusions are routinely invoked to demonstrate the untrustworthiness of
Perceiving Surface Layout 153
perception (Palmer, 1999), these perceptual effects are precisely what
Gibson’s ground theory predicts.
The role of the ground and horizon is also demonstrated in more control-
led experiments. For example, Philbeck and Loomis (1997) asked standing
observers to view a target in the dark at a distance of 1–5 m, then either
close their eyes and walk to its remembered location (“blind walking”), or
verbally estimate its distance in feet (“magnitude estimation”). With targets
at eye level, distances of 1–2 m were overestimated while farther distances
were indistinguishable––whether viewing was monocular or binocular, with
a fixed or free head. With targets on the implicit ground plane, on the other
hand, judgments were close to accurate under all viewing conditions, with a
similar pattern of results for blind walking and magnitude estimation.2 Such
findings indicate that even an implicit ground plane is sufficient for percep-
tion of a target distance, whereas binocular disparity and motion parallax
are not, at least beyond a meter or two.
Moreover, if there is a discontinuity in the ground surface due to a gap,
a texture boundary, or a change in reflectance, distance judgments are sys-
tematically biased (Feria, Braunstein, & Andersen, 2003; Meng & Sedg-
wick, 2002; Sinai et al., 1998; Wu, He, & Ooi, 2007). If the experimenter
pitches the visual framework upward, the perceived size of objects resting
on the ground plane is predictably smaller, and vice versa (Matin & Fox,
1989; Stoper & Bautista, 1992)––just like the Mystery Spot. A continuous,
more-or-less homogeneous, roughly horizontal ground surface thus makes
successful distance and size perception possible.
What visual information is provided by the ground? In Chapter 9,
Gibson proposes three key invariants for the perception of exocentric
(world-centered) layout: (1) optical contact; (2) the horizon ratio; and
(3) the ground texture as an intrinsic spatial scale.
Location: Optical Contact

The exocentric location of an object on the ground surface is specified by
the point at which its base occludes the ground texture, known as optical
contact, which is invariant over viewing position. Gibson demonstrated the
effectiveness of this variable by invisibly suspending a card above the
ground surface (Gibson, 1950a, Figure 72). When viewed from the front,
the card appears to rest on the ground at the point of optical occlusion
(Epstein, 1966); if the observer moves, however, the shearing of optical
texture at its base reveals it to be floating. As first noted by Leonardo, a
cast shadow in contact with the object’s base also nails it down to a loca-
tion on the ground (Ni, Braunstein, & Andersen, 2004; Yonas, Goldsmith,
& Hallstrom, 1978). Moreover, objects piled on top of one another are
similarly located on the ground by what Meng and Sedgwick (2001) called
nested contact relations. The ground surface thus provides the fundamental
reference frame for perceiving the locations of objects in the environment.
Size: The Horizon Ratio
In his influential dissertation, Sedgwick (1973, 1986) showed that the rel-
ative size (height) of terrestrial objects is specified by the horizon ratio. As
Gibson (1979/2015) put it, the horizon intersects all objects of the same
height at the same ratio, which thus constitutes an invariant for size con-
stancy. For example, if the horizon intersects a set of telephone poles and a
tree at one-third of their height, they are all the same height (see Figure
9.1). Gibson (1950a) first showed that size judgments of a stake in an open
field, compared to a standard set before the observer (15–99 in. tall), were
accurate out to half a mile; only the variable error increased with distance.
Although this experiment did not isolate the horizon ratio, it is the only
plausible explanation of the data. Using pictorial stimuli, Rogers (1996)
subsequently demonstrated highly accurate relative size judgments based
on the horizon ratio, within certain pictorial constraints (the horizon is not
too far from the middle of the picture, the adjusted line was not more than
three times the length of the standard line).
The horizon ratio also specifies egocentric (viewer-centered) size,
relative to the observer (Figure 9.2a). The height at which the horizon
intersects an object corresponds to the observer’s eye height (E) on that
Figure 9.1 The horizon ratio. The horizon intersects all objects of the same height
at the same ratio, providing an invariant for size constancy. For
example, the telephone poles and the tree all have the same horizon
ratio of 0.36, and are thus the same size. The horizon line also corres-
ponds to the observer’s eye height on an object: each telephone pole is
thus about three eye heights tall.
;ĂͿ
�
ŚŽƌŝǌŽŶ
�
,
, с ƚĂŶ� н ƚĂŶ�
ƚĂŶ�
;ďͿ
ŚŽƌŝǌŽŶ
�
ϭ͘ϱ� с ϭ
ƚĂŶ�
Ζ с ϭ
ƚĂŶ;ϭ͘ϱ�Ϳ
Ζ

;ĐͿ
�
ŚŽƌŝǌŽŶ
�
�н�
Ζ с ϭ
ƚĂŶ;�н�Ϳ
Ζ

Figure 9.2 Geometry of the ground theory. (a) Horizon ratio specifies frontal
extent (H) in eye height units (E). (b) Declination angle (a) specifies
distance (Z) in eye height units; overestimated declination angle
(1.5a) yields perceived linear distance compression. (c) Declination
angle from raised horizon (a + e) yields perceived nonlinear dis-
tance compression
object, and thus specifies object height (H) in units of eye height. This can
be formalized as
H
__ tan a + tan g
  = ___________
  (9.1a)
E tan a
where α is the visual angle between the horizon and the base of the object
and γ is the visual angle between the horizon and the top of the object.
Note that the horizon ratio specifies not only the height but any frontal
dimension of an object, such as its width (W):
W
__ 2 tan b
______
  =   (9.1b)
E tan a
where b is half the horizontal visual angle of the object. Eye height thus
provides a body-scaled measure of an object’s frontal extent.
Contrary to the longstanding belief that perceived size depends on per-
ceived distance (size-distance invariance), the horizon ratio specifies size
independent of distance. This claim was confirmed by Haber and Levin
(2001) in an open-field experiment, which found that verbal estimates of
the size and distance of the same objects by the same observers were com-
pletely uncorrelated. Specifically, over distances of 3–100 m and vertical
sizes of 0.2–2.0 m, estimates of the distance of unfamiliar geometric shapes
accounted for zero variance in the estimates of their size, and vice versa.
The perception of size from the horizon ratio does not depend on the per-
ception of distance. This is an exemplary case of how to test whether per-
ception of one property depends on the explicit perception of another
property.
The horizon ratio holds whether the horizon is explicit (visible) or
implicit, specified by the limit of optical compression and the convergence
of ground texture and wall texture. People are highly accurate and precise
when estimating their own eye level (indicative of the perceived horizontal
or horizon) in the light, and only slightly less so in the dark (Stoper &
Cohen, 1986). When a minimal visual framework consisting of two
parallel lines or a rectangular box is pitched upward or downward, it
biases perceived eye level in the same direction by around 50%, demon-
strating both visual and gravitational influences (Matin & Fox, 1989;
Stoper & Cohen, 1989). In outdoor scenes, O’Shea and Ross (2007)
observed that sloping ground elicits a similar bias of 40% in perceived eye
level, although it saturates at +3 º to +4 º when looking at uphill slopes. As
the horizon ratio predicts, these manipulations of perceived eye level
produce corresponding biases in perceived object size (Matin & Fox, 1989;
Stoper & Bautista, 1992). Specifically, raising the perceived eye level
(hence, the implied horizon) by pitching the visual framework upward
reduces the judged vertical size of an object resting on the floor, and vice
versa.
Manipulating the observer’s effective eye height has similar effects on
size-related affordance judgments. Warren and Whang (1987) first showed
that reducing the visually specified eye height by raising a false floor
increased the perceived width of a doorway, so that a narrow aperture was
now judged to be passable. Mark (1987) likewise found an eye height
effect on the perceived vertical height that afforded sitting. Wraga (1999)
reported a similar influence of effective eye height on perceived vertical size
by using a false floor and by varying the eye height in a virtual reality (VR)
head-mounted display (Dixon, Wraga, Proffitt, & Williams, 2000). Note,
however, that the eye height manipulation only affected the perceived size
of objects between 0.2–2.5 eye heights tall (Wraga & Proffitt, 2000).
Relative Size and Distance: Ground Texture as an Intrinsic Scale
Gibson emphasized that the ground surface texture provides an intrinsic
scale for exocentric size and distance. If the surface texture is “stochasti-
cally regular,” then “equal amounts of texture” correspond to “equal
stretches of distance along the ground” everywhere in the scene. Thus, the
amount of texture covered by the base of an object provides an intrinsic
scale for size, and the amount of texture between two objects provides an
intrinsic scale for the distance interval between them. This is a potentially
powerful variable, for it provides a basis for both size and distance con-
stancy. For example, any objects that cover T texture are the same width,
while an object that covers 2T is twice as wide. This is nicely illustrated by
Figure 9.3 (from Gibson 1979/2015), in which two cylinders resting on a
tiled floor look to be the same size, for the base of each covers the width of
one floor tile.
For such a scale to be invariant over translation and rotation in the
ground plane, satisfying a Euclidean metric, the texture elements must be
symmetric (isotropy) and have a constant size over the whole surface
(homogeneity). Ordinary textures, however, are often anisotropic (e.g.,
bricks, paving stones, wood grain), undermining comparisons in different
directions. Indeed, the floor tiles in Gibson’s own figure are anisotropic,
Figure 9.3 Ground texture as an intrinsic scale for relative exocentric size and dis-
tance. Objects that cover the same amount of ground texture are the
same width, assuming the texture is homogeneous across the surface.
Thus the two cylinders appear to be the same size. Equal amounts of
texture correspond to equal stretches of distance along the ground,
assuming the texture is isotropic. That is not the case in this figure, so
the depth interval of four texture units between the cylinders is not
equal to a frontal interval of four texture units.
such that an equal amount of texture corresponds to a larger stretch of dis-
tance in depth than in the frontal dimension (see Figure 9.3). The texture
scale hypothesis predicts that manipulating this anisotropy should affect
perceived exocentric distance. In addition, Gibson’s student W. P. Purdy
(1958) proved that the egocentric distance of an object from the observer
is specified by the optical size gradient of ground texture at the object’s
base. This hypothesis predicts that manipulating the size gradient should
affect perceived egocentric distance. To my knowledge, neither of these
experiments has been done.
If ground texture is an effective scale, perceived stretches of distance in
the open field should be quite accurate, precise, and invariant over changes
in viewing distance and direction. But as we will see, there are systematic
biases in distance perception. Note that ground texture can only provide a
scale when it is visually resolvable, and common textures (e.g., grass, sand,
gravel, asphalt) become indistinct at farther distances, where the optical
density surpasses the spatial frequency threshold (around 25 cycles/degree
in daylight). This implies that perceived size over large distances depends
on the horizon ratio, and perceived distance likely depends on other
information.
Egocentric Distance: Declination Angle

Sedgwick (1973, 1986) showed that the egocentric distance (Z) of an
object resting on the ground is specified by the declination angle (a) of its
base from the horizon, in units of eyeheight (E):
Z
__ 1
_____
  =   (9.2)
E tan a
The depth cue formerly known as “height in the visual field” actually
depends on optical contact with the ground plane (Epstein, 1966; Gibson,
1950a, p. 180) and the declination from the horizon. To experimentally
test this distance information, Wallach and O’Leary (1982) used a minify-
ing lens to decrease the declination angle, yielding the predicted increase in
perceived distance, as indicated by judgments of the size of a square target.
Conversely, Ooi, Wu, and He (2001) used a base-up wedge prism to
increase the declination angle, yielding the expected decrease in perceived
distance, as measured by blind walking. There is thus converging evidence
that the effective information for egocentric distance over the ground is the
declination angle.
Slant: Texture Gradients

One of Gibson’s (1950a) early discoveries was that optical texture gradi-
ents specify the slant of a surface to the line of sight, known as optical
slant. It is not the density gradient that carries the effective information, as
Gibson thought, but the size gradient. W. P. Purdy (1958) showed mathe-
matically that, for homogeneous textures, local slant is indeed specified by
the relative optical size of adjacent elements.
However, beginning with Gibson (1950b), there is a long literature
demonstrating that perceived slant is significantly underestimated, biased
into the frontal plane (Todd, Thaler, & Dijkstra, 2005). Todd, Chris-
tensen, and Guckes (2010) showed that the effective information for per-
ceiving slant from texture is the scaling contrast (normalized difference in
optical size for the range of elements on the surface), which covaries with
slant but does not uniquely correspond to it. Thus, even though available
information specifies optical slant, in this case, the visual system appears
not to take advantage of it.
To summarize, updated evidence indicates that (1) optical contact and
cast shadows are effective information for exocentric location on the
ground, which is invariant under changes in viewing position; (2) the
horizon ratio is effective information for the frontal extent of objects over
a large range of distances, at least for objects ranging from 0.2–2.5 E; (3)
ground texture appears not to serve as an intrinsic scale for relative dis-
tance or relative size over a large range of distances; and (4) texture scaling
contrast determines perceived slant to the line of sight, which tends to be
flattened into the frontal plane.
Paradoxes of Distance Perception

We now confront the inconvenient fact that, despite what Gibson and his
followers (including me) expected, perception of distance in the open field
is distorted and inconsistent.3 These visual distortions are not dispelled by
motion parallax or binocular disparity (Foley, Ribeiro-Filho, & DaSilva,
2004; Philbeck & Loomis, 1997). Indeed, related distortions are observed
in “binocular space” with disparity alone (Hardy, Rand, & Rittler, 1951;
Luneburg, 1947), although they are presumably driven by different
variables. Consider the following paradoxes (refer to Figure 9.4, Task and
Data).
1. Egocentric distance (Figure 9.4a). If you view a target on the ground

4–20 m away, then blind walk to it, you are highly accurate (Loomis,
DaSilva, Fujita, & Fukusima, 1992; Rieser, Ashmead, Talor, &
Youngquist, 1990). But if you verbally estimate the egocentric distance
of the same target, you underestimate it by 20–30% (Foley et al.,
2004; Knapp & Loomis, 2004; Loomis & Philbeck, 2008).
2. Egocentric aspect ratio (Figure 9.4b). Perhaps, you say, action tasks
are more natural and less cognitive than magnitude estimation. Yet if
you match a frontal interval between two targets by walking toward
one of them and stopping at a point that appears to form an equilat-
eral ‘L,’ egocentric distance is also underestimated by about 30% (Li,
dĂƐŬ ĂƚĂ ^ŝŵƵůĂƟŽŶ
ϮϬ ϯϬ
;ĂͿ ůŝŶĚǁĂůŬŝŶŐ WƌĞĚ͘ǀĞƌďĂů
sĞƌďĂůĞƐƟŵĂƚĞ
ǆ Ϯϱ WƌĞĚ͘ǁĂůŬ
ĐĐƵƌĂƚĞ
ϭϱ
ϮϬ
ϭϬ ϭϱ
ϭϬ
/ŶĚŝĐĂƚĞĚĚŝƐƚĂŶĐĞ;ŵͿ
ϱ
/ŶĚŝĐĂƚĞĚŝƐƚĂŶĐĞ;ŵͿ
ϱ
<ŶĂƉƉΘ>ŽŽŵŝƐ;ϮϬϬϰͿ
Ϭ Ϭ
Ϭ ϱ ϭϬ ϭϱ ϮϬ Ϭ ϱ ϭϬ ϭϱ ϮϬ Ϯϱ ϯϬ
dĂƌŐĞƚĚŝƐƚĂŶĐĞ;ŵͿ dĂƌŐĞƚĚŝƐƚĂŶĐĞ;ŵͿ
ϰϬ &ƌŽŶƚĂůŵĂƚĐŚĞƐƚŽĞŐŽĐĞŶƚƌŝĐĚŝƐƚĂŶĐĞƐ
;ďͿ dǇƉŝĐĂůǀĞƌďĂůĞƐƟŵĂƚĞƐ;>ŽŽŵŝƐΘWŚŝůďĞĐŬ͕ϮϬϬϴͿ ϯϬ
ϯϱ 'ŝůŝŶƐŬǇ;ϭϵϱϭͿŵŽĚĞů WƌĞĚ͘ŵĂƚĐŚ
Ϯϱ ĐĐƵƌĂƚĞ
ϯϬ
Ϯϱ ϮϬ
ǆ ϮϬ ϭϱ
ϭϱ
ϭϬ
ϭϬ
ϱ
ϱ
/ŵƉƵƚĞĚƉĞƌĐĞŝǀĞĚĚŝƐƚĂŶĐĞ;ŵͿ
/ŶĚŝĐĂƚĞĚĞŐŽĐĞŶƚƌŝĐĚŝƐƚĂŶĐĞ;ŵͿ
>ŝ͕WŚŝůůŝƉƐΘƵƌŐŝŶ;ϮϬϭϭͿ
Ϭ Ϭ
Ϭ ϱ ϭϬ ϭϱ ϮϬ Ϯϱ ϯϬ ϯϱ ϰϬ Ϭ ϱ ϭϬ ϭϱ ϮϬ Ϯϱ ϯϬ
ĐƚƵĂůĞŐŽĐĞŶƚƌŝĐĚŝƐƚĂŶĐĞ;ŵͿ &ƌŽŶƚĂůŝŶƚĞƌǀĂů;ŵͿ
;ĐͿ Ϯ͘ϬϬ ϭ͘ϴ

ϭ͘Ϭŵ
ϭ͘ϴϬ
ϭ͘ϱŵ ϭ͘ϲ
ϭ͘ϲϬ
Ϯ͘Ϭŵ
ϭ͘ϰϬ ϭ͘ϰ
ϭ͘ϮϬ
DĂƚĐŚĞĚĚĞƉƚŚͬǁŝĚƚŚ
ϭ͘Ϯ
DĂƚĐŚĞĚĚĞƉƚŚͬǁŝĚƚŚ
WƌĞĚ͘ϭ͘ϱ
ϭ͘ϬϬ
>ŽŽŵŝƐ͕ĞƚĂů͘;ϭϵϵϮͿ
Ϭ͘ϴϬ
ϭ
Ϯ ϰ ϲ ϴ ϭϬ ϭϮ Ϭ Ϯ ϰ ϲ ϴ ϭϬ ϭϮ
WŚǇƐŝĐĂůĚŝƐƚĂŶĐĞŝŶŵĞƚĞƌƐ
WŚǇƐŝĐĂůĚŝƐƚĂŶĐĞ;ŵͿ
;ĚͿ ϭϲ ϭϲ
KƉĞŶĮĞůĚ WƌĞĚ͘ďŝƐĞĐƟŽŶ
ϭϱ ϭϰ
,ĂůůǁĂǇ ĐĐƵƌĂƚĞ
ϭϰ ĐĐƵƌĂƚĞ ϭϮ
ϭϯ
ϭϬ
ϭϮ
ϴ
ϭϭ
ϲ
ϭϬ
/ŶĚŝĐĂƚĞĚďŝƐĞĐƟŽŶ;ŵͿ
ϵ ϰ
ǀĞƌĂŐĞŵŝĚƉŽŝŶƚĞƐƟŵĂƚĞ;ŵͿ
ϴ ŽĚĞŶŚĞŝŵĞƌ͕ĞƚĂů͘;ϮϬϬϳͿ
Ϯ
ϳ Ϭ
ϳ ϴ ϵ ϭϬ ϭϭ ϭϮ ϭϯ ϭϰ ϭϱ ϭϲ Ϭ Ϯ ϰ ϲ ϴ ϭϬ ϭϮ ϭϰ ϭϲ
DŝĚƉŽŝŶƚ;DͿ DŝĚƉŽŝŶƚ;ŵͿ
;ĞͿ
ϯϬ
'ŝůŝŶƐŬǇ;сϮϴͿ
K͗:
ĐĐƵƌĂƚĞ
,ŽƌŝǌнϯΣ
ϮϬ
ϮϬ

Ěͻ
ϭϬ ϭϬ
Ƌ͘/͗ Ě с Ϯϴ͘ϱ
Ϯϴ͘ϱн
WĞƌĐĞŝǀĞĚĚŝƐƚĂŶĐĞ;ĚͿŝŶŵĞƚĞƌƐ
'ŝůŝŶƐŬǇ;ϭϵϱϭͿ
/ŶĚŝĐĂƚĞĚ/ŶĐƌĞŵĞŶƚĂůŝƐƚĂŶĐĞ;ŵͿ
Ϭ Ϭ
Ϭ ϭϬ ϮϬ ϯϬ ϰϬ Ϭ ϭϬ ϮϬ ϰϬ ϲϬ
WŚǇƐŝĐĂůĚŝƐƚĂŶĐĞ;ͿŝŶŵĞƚĞƌƐ WŚǇƐŝĐĂůĚŝƐƚĂŶĐĞ;ŵͿ
Figure 9.4 Distance perception: experimental tasks, representative data, and the results of numerical simulations based
on Equations 9.1 and 9.3. (a) Egocentric distance: blind walking task and verbal estimates. (b) Egocentric
aspect ratio: equilateral ‘L’ task yields linear depth compression. (c) Exocentric aspect ratio: equilateral ‘+’
task, ditto. (d) Egocentric bisection: accurate judgments. (e) Exocentric depth increments: marking off
equal intervals in depth yields nonlinear compression, simulated with Equation 9.4. Note: ‘o’ indicates
targets, ‘x’ indicates participant’s final position, arrows indicate adjustments.
Source: (a) Data from Knapp and Loomis (2004), Figure 3; (b) from Li et al. (2011), Figure 4. Adapted with permission from Springer
Nature; (c) from Loomis et al. (1992), Figure 5a. Adapted with permission from American Psychological Association; (d) from Bod-
enheimer et al. (2007), Figure 2; (e) from Gilinsky (1951), Figure 5.
Phillips, & Durgin, 2011). In other words, perceived egocentric dis-
tance is a linear function of physical distance, but compressed in depth,
with a slope of about 0.7.
3. Exocentric aspect ratio (Figure 9.4c). Well, perhaps exocentric dis-
tances between objects behave differently, thanks to the ground texture
scale. But if you attempt to adjust a depth interval between two targets
to match a frontal interval between two targets, forming an equilateral
‘+’ or ‘L,’ the depth interval is underestimated relative to the frontal
interval, increasingly so with distance, until it levels off around
30–40% at 6–10 m (Loomis et al., 1992; Loomis & Philbeck, 1999).
This presents the deepest paradox: one would think that if you can
blind walk accurately to each of these targets, you must be able to per-
ceive their spatial locations and the distances between them. Yet
physically-equal intervals in the open field are perceived as unequal
when viewed from different directions, thereby violating a Euclidean
metric for perceptual constancy (Foley et al., 2004; Toye, 1986;
Wagner, 1985).
4. Egocentric bisection (Figure 9.4d). Perhaps this is only so when com-
paring distance intervals in different directions (e.g., frontal vs depth
intervals). And happily, if you adjust a marker (Z1) to bisect the ego-
centric distance between you and a target (Z2) in the open field (setting
tana1 = 2tana2), you are highly accurate and precise—up to 300 m!
(Bodenheimer et al. (2007); Lappin, Shelton, & Rieser (2006); J. Purdy
& Gibson (1955); Rieser et al. (1990); but see Gilinsky’s, 1951, two
observers).4
5. Exocentric depth increments (Figure 9.4e). On the other hand, if you
try to mark off a series of equal increments in depth, starting near your
feet, equal intervals look progressively smaller with distance, that is,
they appear nonlinearly compressed (Gilinsky, 1951; Ooi & He,
2007). This is apparent when viewing the dashed lines running down
the middle of a highway: equal-length dashes look increasingly
compressed with distance. Another way of saying this is that perceived
incremental egocentric distance is a negatively accelerating function of
physical distance, such as a hyperbolic curve or a power law with an
exponent <1 (compare Figure 9.4e, with Figure 9.4b, center).
What’s going on here?

For over a century, researchers have sought to characterize a geometric
transformation relating objective physical space to a subjective visual
space.5 Physical space at the human scale is Euclidian, such that distance
intervals are invariant over translation and rotation. Yet the mapping
from physical to visual space plainly does not preserve metric distances—
witness the depth compression just described—and hence it is not Eucli-
dean. It has been proposed that this mapping is a hyperbolic function in
which depth is nonlinearly compressed (Gilinsky, 1951; Luneburg, 1947),
or an affine transformation in which depth is linearly compressed (Koen-
derink, van Doorn, Kappers, & Todd, 2001; Norman, Todd, Perotti, &
Tittle, 1996; Wagner, 1985). While each of these mappings captures
some results, no single geometric transformation explains all the data
(Foley et al., 2004; Norman, Crabtree, Clayton, & Norman, 2005;
Wagner, 2006).
In what follows, I will argue that there is no consistent visual space.
Rather, perception by an active agent is task-specific and information-
driven, such that judgments of different properties of layout are based on
different optical variables. Specifically, perceived egocentric distances and
exocentric depth intervals in the open field depend on the angular declina-
tion from the horizon, whereas perceived frontal intervals depend on the
horizon ratio. Consequently, perceptual judgments do not necessarily hang
together in a consistent geometry, but can be radically inconsistent (see
Bingham, Crowell, & Todd, 2004, for a similar account of reach space).
As Gibson taught us: to resolve paradoxes, follow the information.
Revisiting the Distance Paradoxes

A few years ago, Durgin and Li (2011) made a critical discovery: the decli-
nation angle of the line of gaze is actually overestimated by a factor of 1.5.
Whereas eye level is judged accurately, a declination angle of 10 º is judged
as 15 º, while a declination of 40 º is judged as 60 º (the effect appears to
saturate around 60 º). The authors measured perceived gaze declination in
three different ways (verbal estimates, bisection of horizontal and vertical
gaze, inferred from verbal slant estimates), all of which yielded the same
factor of 1.5. Why this occurs is a matter of speculation (see Durgin & Li,
2011), but the empirical finding is quite consistent. This simple observa-
tion can resolve most of the distance paradoxes: whereas perceived frontal
intervals are based on the true horizon ratio, perceived depth intervals are
based on exaggerated declination angles.
Consider the implications for egocentric distance perception. If declina-
tion angle (a) is the effective information, and gaze declination is exagger-
ated by 1.5, then perceived distance (Z’) is
Z’ 1
__ ________
  =   (9.3)
E tan(1.5a)
Egocentric distance should thus be underestimated by about 33% (a slope
of 0.67) —consistent with the reported linear depth compression. Indeed,
when Li et al. (2011) asked observers to match an egocentric distance to a
frontal extent (horizontal or vertical) in the ‘L’ task, perceived distance was
a linear function of actual distance, with slopes between 0.6 and 0.75, just
as predicted.
Importantly, exaggerating the gaze declination does not alter the visual
angles that define the horizon ratio. Thus, although an object’s perceived
istance decreases due to the (overestimated) declination angle (Equation
d
9.3), its frontal extent specified by the horizon ratio is unaffected (Equation
9.1b). Consequently, perceived intervals in depth are linearly compressed,
while perceived intervals in the frontal plane are generally accurate.6 These
two equations explain most of the distance puzzles mentioned above, as
revealed by a little numerical analysis (refer to Figure 9.4, Simulation).
1. Verbal underestimates of egocentric distance are accounted for by the

exaggerated declination angle, as just described (Figure 9.4a).
2. In contrast, frontal intervals between 2–2.5 E are perceived accurately,
based on the horizon ratio. Together, this explains egocentric aspect
ratio judgments (the ‘L’ task), in which egocentric distance (Z) is com-
pressed by 30% relative to frontal width (W) (Figure 9.4b). Specifi-
cally, perceived Z given by Equation 9.3 is matched to perceived W
given by Equation 9.1b.
3. Similarly, perceived exocentric depth intervals are based on the over-
estimated declination angles of their endpoints, explaining depth com-
pression. This accounts for the depth compression in exocentric aspect
ratio judgments (the ‘+’ task), which stabilizes around 30–40% at
8–10 m (Figure 9.4c). Specifically, a visually compressed depth inter-
val, given by the difference in exaggerated declination angles to its
endpoints (Equation 9.3), is matched to an accurately perceived width,
given by the horizon ratio (Equation 9.1b).
4. Counterintuitively, Equation 9.3 also explains accurate egocentric
bisection judgments, because exaggerated declination angles affect the
perceived distances of the target and the midpoint proportionally (set
tan(1.5a1) = 2tan(1.5a2)) (Figure 9.4d).
Some researchers have suggested that, rather than depth intervals
appearing compressed, the visual angles of frontal intervals appear
expanded (Foley et al., 2004; Li et al., 2011). However, a reanalysis of
Foley et al.’s (2004) data on verbal estimates of distances between
stakes in an open field reveals that frontal intervals were judged quite
accurately (slope of 1.0 for monocular viewing). This supports the
view that frontal intervals are accurately perceived based on the
horizon ratio (Equation 9.1b), whereas depth intervals are linearly
compressed due to the exaggerated declination angles of their end-
points (Equation 9.3).
5. The one finding that these two principles cannot explain is the non-
linear compression of perceived exocentric depth increments reported
by Gilinsky (1951), and recently replicated by Ooi and He (2007).
What might account for the nonlinearity of this task? I would like to
suggest an explanation based on the ground surface and the perceived
horizon; alternative accounts are nicely summarized by Loomis
(2014).
In a study of distance perception in VR, Messing and Durgin (2005)
observed that lowering the visual horizon yields a nonlinear expansion in
depth: perceived egocentric distance increased as a positively accelerating
function of the specified distance, for both blind walking and verbal
estimates. Conversely, raising the visual horizon (by  degrees) would
yield a negatively accelerating function for perceived egocentric
distance (Z’):
Z’ 1
__ _________
  =   (9.4)
E tan(a + )
where a is the declination angle of the target from the true horizon (Figure
9.2c). On the hypothesis that perceived depth increments are based on the
declination angles of their endpoints, this would yield a nonlinear com-
pression of the incremental depth intervals. As shown in Figure 9.4e, if the
apparent horizon is raised by only  = 3º, the predicted curve matches
Gilinsky’s data on perceived incremental distance as a function of physical
distance, and is identical to her hyperbolic curve (with her constant A =
28 m).7
Why might the horizon appear raised in Gilinksy’s incremental depth
task, but not in an egocentric distance task? First, consider that the equal-
increment procedure directs the observer’s attention to a local patch of
ground, rather than spanning the ground surface to the target and horizon
(Wu, Ooi, & He, 2004). Recall that the perception of local surface slant is
flattened into the frontal plane, perpendicular to the line of sight. The
equal-increment procedure may thus yield a frontal bias in the perceived
local slant of the ground surface (Durgin & Li, 2011; Li & Durgin, 2010;
Ooi & He, 2007; Wu et al., 2004). This in turn would raise the perceived
horizontal (Stoper & Cohen, 1989), which saturates at 3–4° in the open
field (O’Shea & Ross, 2007)––precisely accounting for Gilinsky’s nonlinear
depth compression (Figures 9.2c and 9.4e).
Ooi and He (2007) proposed a similar account of the nonlinear depth
compression based on a frontal bias in the perceived slant of the ground
surface. The main difference is that they attribute the bias to an intrinsic
curvature in the implicit ground surface, which they estimate in the
dark, while assuming that the perceived horizontal is constant (Ooi et
al., 2001, 2006; Wu et al., 2007). In contrast, the current hypothesis is
based on a known frontal bias in the perceived slant of visible surfaces,
and the known effect of a pitched visual framework on the perceived
horizontal.
To sum up the present proposal, task-specific perception of different
properties of layout is driven by different optical variables: (1) Frontal
extents between 2–2.5 E are perceived accurately based on the horizon
ratio, independent of distance. (2) The linear compression of egocentric
distances and the depth intervals between them are accounted for by over-
estimated declination angles. (3) The nonlinear compression of exocentric
depth increments results from attention to local patches of ground biasing
the perceived slope of the ground surface, hence the perceived horizon, and
the resulting declination angles.
The Visual-Locomotor Mapping

The deepest paradox remains, however: given that perceived distance is
systematically compressed in depth, why is blind walking so accurate? This
might be explained by the compensation hypothesis: a visual-locomotor
mapping undoes or compensates for the visual compression, yielding
accurate blind walking. For example, if a target at a distance of 10 m is
perceived to be 6.7 m away (Z’= 0.67 Z), the visual distance would be
mapped into a walking distance of 10 m (W = 1.5 Z’), thereby compensat-
ing for the depth compression. The same formulation would hold if per-
ceived distance is scaled in eye height units.
It is known that the relation between visually perceived distance and a
locomotor response is rapidly recalibrated. In a formative study, Rieser,
Pick, Ashmead, and Garing (1995) showed that decreasing the rate of optic
flow during walking for only 8 minutes induces adaptation, such that
participants systematically overshoot the target in a blind-walking posttest
(and vice versa). It is thus clear that the visual-locomotor mapping is
plastic and continually recalibrated. This would explain how action could
come to compensate for a linear depth compression.
However, Loomis and his colleagues (Fukusima, Loomis, & Da Silva,
1997; Philbeck, Loomis, & Beall, 1997) have made a case against the com-
pensation hypothesis. They argue that both visual distance judgments and
visually-directed action are based on perceived invariant location. For
example, in their indirect walking paradigm, the participant views a target,
then blind walks on an oblique path until instructed to turn and blind walk
to the target. Performance on indirect walking is nearly as accurate as
direct walking. The authors conclude that visually-directed locomotion
(and related tasks such as blind pointing) are guided by perceived
exocentric location, and blind walking is an accurate index of the perceived
egocentric distance to that location. Note that if this is the case, tasks
based on location should be not only accurate but also less variable and
more stable than tasks based on other properties, such as egocentric dis-
tance (see Lappin, 2016).
On the other hand, these results can also be explained by the compensa-
tion hypothesis. Suppose a target at 10 m is perceived to be 6.7 m away.
Thanks to the visual-locomotor mapping, the participant would blind walk
to the right for 5 m (sensing they had walked only 3.4 m) before being told
to turn and walk to the target (updated to be 7.5 m away). They would
turn the appropriate 68 º and walk all the way to the target (7.5 m × 1.5 =
11.2 m). Essentially, the physical path and the sensed path are similar tri-
angles, with the latter a scaled-down (0.67) version of the former. Loomis
et al. (2002) point out that a nonlinear distance compression could not be
corrected in this fashion, for it would yield systematic undershooting on
longer indirect paths. But, fortunately, egocentric distance perception is
linearly compressed, and a calibrated visual-locomotor mapping can com-
pensate for that. Even though the perceived incremental depth of highway
dashes is nonlinearly compressed, blind walking to their endpoints would
still be successful, based on a calibrated visual-locomotor mapping for ego-
centric distance.
The paradoxes of distance perception might thus be resolved by the
principle that perceived frontal intervals and depth intervals are based on
different optical variables, while the visual-locomotor mapping compen-
sates for egocentric distance compression. Once we relinquish the conceit
of a consistent visual space, and accept that perception is task-specific and
information-based, distance perception appears less paradoxical.
Affordances of Surface Layout

And yet, what are we talking about when we talk about perceiving “dis-
tance?” Distance is surely not presented to the observer in feet or meters,
although observers can learn such conventional scales. Nor is it given to
the observer in eye heights, even though the optical variables are scaled
by E. In Chapter 9, Gibson is ambivalent. On the one hand, he con-
cludes, “the dimensions of things on the ground are perceived directly”
(1979/2015, pp. 160–161), that is, we perceive the geometric properties
of spatial layout. But in almost the same breath he states that agents
need to perceive the affordances of surface layout, not the geometric
layout per se: “An affordance is a layout relative to the animal and com-
mensurate with its body … What animals need to perceive is not the
layout as such but the affordances of the layout” (p. 150). This passage
might be read in several ways: that affordances are perceived in addition
to the spatial layout, that they are not perceived by means of awareness
of the spatial layout, or that affordances are perceived but the spatial
layout is not. The implication is that we need to perceive what the
surface affords for walking, that is, the walkability or locomotor dis-
tance to a goal not geometric distance as such.
This is startlingly reminiscent of Bishop Berkeley’s early view of distance
perception:
Whoever examines what he means by saying he sees this or that thing

at a distance, will agree with me that what he sees only suggests to his
understanding that after having [advanced forward so many paces], to
be measured by the motion of his body, which is perceivable by touch,
he shall come to perceive such and such tangible ideas which have
been usually connected with such and such visible ideas.
(Essay towards a new theory of vision, 1709, Section 45)
In other words, the visually perceived distance to a thing derives from the
proprioception entailed in walking to it, that is, from the distance sensed
by the human odometer. The locomotor distance metric of the human
odometer is not well understood, and may involve the number of steps,
step length, expended energy, and/or some other quantity (Chrastil &
Warren, 2014; Turvey et al., 2009; White, Shockley, & Riley, 2013), but
for ease of exposition, I will use number of steps as a proxy for locomotor
distance, although this is known to be inadequate.
This brings us to the mutual calibration hypothesis (after Rieser et al.,
1995). In a standard calibration, a device like a ruler is calibrated to an
extrinsic standard like a platinum meter bar; in a mutual calibration, two
rulers are intrinsically calibrated together, so their scales agree with each
other. On this hypothesis, vision (declination angle) and proprioception
(locomotor distance sensed by the odometer) are calibrated together by the
achievement of an affordance (successfully walking to goals), regardless of
the metric distances involved. The resulting visual-locomotor mapping (from
declination angle to locomotor distance) guides action such as blind walking,
and the inverse mapping (from locomotor distance to declination angle)
grounds the perception of walkable distance. Pan, Coats, and Bingham
(2014) developed a similar “embodied units” theory in the context of reach-
ing, according to which feedback from successful reaching serves to calibrate
the “mapping from embodied units of perception to embodied units of
action.”
Berkeley (1709) assumed that “there is no necessary connection between
visible and tangible ideas.” But we now understand that there are lawful
relations between physical distance, optical variables, gait dynamics, and
proprioceptive variables, within a terrestrial context of constraint, which
structure information. As we walk about the world, there is massive cov-
ariation between the declination angle of a locomotor goal and the number
of steps walked to get there.8 When walking down the street, for example,
the crosswalk begins at a declination angle of 37 º and is reached in 4 steps;
the mailbox begins at 17 º and is reached in 10 steps, and so on. The decli-
nation angle is thus continually calibrated to the locomotor distance to a
goal. The same holds if the declination angle is overestimated by a factor
of 1.5, in which case a mailbox at 25 º is reached in 10 steps, etc. Once
calibrated, upon viewing a target at a declination angle of 25 º, the per-
ceived walkability (locomotor distance) is 10 steps and the observer is
equipped to blind walk to it. There is thus a mutually calibrated system of
relations among visible declination angle, tangible locomotor distance, and
perception of the affordance of walkability, bound together by successful
action. Although geometric distance underwrites this set of relations, it
drops out of the calibrated system.
This sort of continual calibration is essential to keep biological systems
tuned to the dynamic properties of the body and environment, and is much
less brittle than a hard-wired mapping. Significantly, mutual calibration
implies that optical information need only covary with a behaviorally-
relevant environmental property, not correspond to it, for the scaling con-
stants are continually tuned through action. If the visual-locomotor
relation is manipulated, by increasing the number of steps required to walk
through a given declination angle, it follows that both blind walking and
perceived walkability should adapt, although perhaps at different rates.
But if the underlying optical variable is manipulated, by using a prism to
increase the declination angle, both blind walking and perception should
immediately follow suit (see Pan et al., 2014).
Support for mutual calibration comes from the perception of distance in
VR. When an observer first dons a head-mounted display, perceived dis-
tances in the virtual environment shrink by as much as 50%, as measured
by both blind walking and verbal estimates (Loomis & Knapp, 2003; Rich-
ardson & Waller, 2007).9 However, ten minutes of sighted walking to
targets on the ground is sufficient to completely recalibrate the visual-
locomotor mapping, such that blind walking becomes accurate (Mohler,
Creem-Regehr, & Thompson, 2006; Richardson & Waller, 2007), and a
negative aftereffect is observed upon transfer back to the real world
(Waller & Richardson, 2008). In addition, visually perceived distance is
slightly but significantly rescaled, as measured by adjusting the size of a
soccer ball on the ground (Siegel & Kelly, 2017). Similar effects have been
observed in reach space, where false feedback about reaching success
induces a rescaling of both reaching and perceived distance (Pan et al.,
2014; Volcic, Fantoni, Caudek, Assad, & Domini, 2013).10
Mutual calibration can thus account for blind walking and perceived
walkability (locomotor distance). However, it cannot explain the remain-
ing paradox that verbal estimates are, at the same time, compressed in
depth. The answer, I think, is that declination angle has two separate cali-
brations, one to locomotor distance and one to meters. When making
verbal estimates, observers do not convert perceived locomotor distance
into reported number of meters. Rather, they have learned a separate cali-
bration of declination angle to meters. Although there is continual feed-
back about the actual number of steps to the mailbox over a range of
distances, this is not the case for the number of meters to the mailbox.
Consequently, the visual-meter mapping is not calibrated for the overesti-
mation of declination angle, and verbal distance judgments remain
compressed.
Distance perception in VR supports separate calibrations. Whereas
sighted walking in VR serves to recalibrate blind walking, and to a lesser
extent perceived distance, verbal distance estimates show no sign of recali-
bration (Kelly, Cherep, & Siegel, 2017). In contrast, when the declination
angle is decreased by lowering the visual horizon, both blind walking and
verbal estimates shift together (Messing & Durgin, 2005). These results
imply that the underlying variable of declination angle is separately cali-
brated to number of steps, on the one hand, and to number of meters, on
the other. Aspect ratio judgments remain compressed regardless of these
calibrations, because the visual angles specifying the frontal and depth
intervals are directly related and the distance units cancel out (Equations
9.1b and 9.3).
In sum, there is evidence consistent with the mutual calibration hypo-
thesis, with separate calibrations for conventional scales. Behavior is gener-
ally successful thanks to a mutually calibrated system of relations among
visual variables and proprioceptive variables, grounding perception in the
affordances of surface layout.
The Objects of Perception

There is a certain irony here that should not go unremarked. The problem
of layout perception has led my friend and colleague Jack Loomis, an
avowed representationalist (Loomis, 2016), to argue that we perceive loca-
tion and egocentric distance more or less veridically. At the same time, the
offspring of J. J. Gibson, who maintain that perception is direct, approv-
ingly cite Bishop Berkeley, and argue that perceived layout is distorted.
How did we get here?
The concept of direct perception has long been bound up with a some-
what naïve notion of veridicality. If the world is perceived directly, it is
argued, then perception should coincide with objective reality (Palmer,
1999), and in particular, perceived layout should be Euclidean. You may
not be able to see everything, but what you see is what you get, so the
world is as it appears to be. When it comes to spatial layout, however, the
world is decidedly not as it appears to be––at least as measured by a Eucli-
dean ruler.
One of the more consequential upshots of Gibson’s (1979/2015) eco-
logical approach is his attempt to ecologize the objects of perceptual
awareness. Why would we think that off-the-shelf Euclidean geometry or
Newtonian physics should provide an appropriate ontology for biologi-
cal agents? In Gibson’s ecological approach, the objects of perception are
behaviorally-relevant properties of the agent’s niche, such as its
affordances. A growing literature has shown that affordances are speci-
fied by body-scaled or action-scaled information (Mark, 1987; Warren,
2007; Warren & Whang, 1987), and are judged and enacted successfully
(see Wagman, Chapter 8, in this volume). Although affordances are
underwritten by the geometric properties of the environment and the
body, embedded agents may directly perceive walkable surfaces, passable
openings, reachable targets, and graspable objects by detecting such
information, without awareness of their metric sizes, distances, and
shapes.
We thus are faced with three options regarding the objects of perceptual
awareness. The first option asserts that geometric properties of surface
layout (and the agent’s body) are perceivable, and judgments about
affordances are derived from them. The distortions of perceived layout
described above, however, render this view problematic, and some evid-
ence actually suggests that affordance judgments are not based on expli-
citly perceived geometric properties. Thomas and Riley (2014; see also
Mark, 1987) obtained judgments of the reachability of an overhead target
with a stick and without a stick, as well as judgments of stick length, using
a target adjustment task. Reachability-with-stick judgments were signifi-
cantly lower than the sum of reachability-without-stick and stick-length
judgments. This result implies that participants did not base their reacha-
bility judgments on explicit perception of stick length, but directly per-
ceived the affordance.
The second option is the converse, that affordances are perceivable, but
geometric properties of spatial layout are not. This view is tempting, con-
sidering the success of action despite large distortions in perceived spatial
layout. And yet it runs counter to phenomenology, for I do seem to be
aware of the (non-Euclidean) sizes, shapes, and distances of things, not just
their affordances. Even Thomas and Riley (2014) assumed their particip-
ants could perceive stick length.
The third option claims that both affordances and spatial layout are
perceivable, but the former are perceived directly, without depending on
awareness of the latter. That would explain why actions calibrated to
body-scaled information can be successful despite perceptual distortions
and why the distortions don’t really matter. It also allows for the rescaling
of layout perception based on successful action. To support the claim that
affordances are directly perceived, future research will need to show that
judgments of affordances such as walkability and reachability are not only
different from, but more precise, more stable across tasks, and uncorre-
lated with judgments of geometric size, shape, and distance (Lappin,
2016).
Acknowledgments
Thanks to Jack Loomis, Jim Todd, Joe Lappin, Hal Sedgwick, Chris Hill,
Frank Durgin, John Rieser, Geoff Bingham, Fulvio Domini, and Jijiang He
for many happy years of head-banging over these problems. Any insights
herein must have originated with them, but of course they are not to blame
for the rest of it.
Notes
1. The view that direct perception requires a moving observer misinterprets
Chapter 9, for layout is also specified by static information within a given
context of constraint (Runeson, 1988).
2. With the lights on, performance was highly accurate under all viewing con-
ditions, for the eye-level target was mounted on a visible tripod that rested on
the visible ground.
3. This is a large literature, with results that depend on stimuli, procedure, and
test conditions. Rather than a full review, I offer my reading of what I regard
as instructive findings.
4. In indoor hallways, overestimates of the midpoint have been reported in some
experiments (Lappin et al., 2006), but not others (Bodenheimer et al., 2007).
5. There are two distinct questions here: the extrinsic geometry of visual space,
the geometric properties that are preserved by the mapping from physical to
visual space; and the intrinsic geometry of visual space, the geometric relations
among perceived properties. For the latter, see Battro, Netto, and Rozestraten
(1976); Todd, Oomes, Koenderink, and Kappers (2001).
6. This dissociation between the information for depth intervals and frontal inter-
vals differs from a dissociation between the processing of locations and extents
(Loomis, 2014; Loomis, Philbeck, & Zahorik, 2002), because the perception
of depth intervals and their endpoints are both based on exaggerated declina-
tion angles.
7. If one includes the exaggerated declination angle, so
Z’
__ 1
  = _____________
    
E tan(1.5( + ))
raising the perceived horizon by just  = 1.2 º would also approximate the data.
8. In addition, the rate of optic flow from the ground plane maps to locomotor
distance (Beusmans, 1998; Waller & Richardson, 2008). So does the time-to-
contact variable tau: if tau specifies that the lamppost is 2 seconds away, and
tau during one step is 0.5 seconds, then the lamppost is 4 steps away (Warren,
2007).
9. The extra compression in VR is still not completely understood, see Willemsen,
Colton, Creem-Regehr, and Thompson (2009).
10. The mutual calibration hypothesis differs from Proffitt’s (2006) embodied per-
ception theory, which claims, “Perception is mutable to nonvisual influences …
The apparent dimensions of surface layout expand and contract with changes
in energetic cost” and “one’s purposes, physiological state, and emotions”
(p. 110). The evidence for this claim remains controversial. Mutual calibration
occurs on a slower timescale to keep perception aligned with the affordances of
layout, which do not depend on transient intentions, needs, fatigue, or social
and emotional factors.
10 Acting Is Perceiving
Experiments on Perception of
Motion in the World and
Movements of the Self, an Update
L. James Smart Jr., Justin A. Hassebrock,
and Max A. Teaford
Movement matters. This seemingly simple idea underlies much of J. J. Gib-

son’s radical approach to perception (and action). The idea that movement
can disambiguate traditionally difficult-to-explain phenomena in percep-
tion (such as identity, size, or position) was a marked departure from more
reductionist and static image accounts of perceptual processing that relied
on memory and internal cognitive elaboration. Movements are comprised
of transformations of structure(s) over time that produce reliable changes
in energy patterns that can be detected by sensory systems. Gibson
(1979/2015) phrased it as “disturbances of structure in the ambient array”
(p. 162), noting that these changes specify what is occurring. In line with
this idea, Gibson was interested in determining if there is evidence that
suggests that the perception of changing layout is direct. In asking this
question, Gibson gave rise to a large body of research on movement per-
ception. The present chapter will briefly outline Chapter 10 (“Experiments
on the Perception of Motion in the World and Movement of the Self ”)
from Gibson (1979/2015) and then review modern techniques to study
movement perception as well as ask new questions which remain to be
answered.
The Original Questions

Traditionally, it was assumed that motion perception was simple to under-
stand and was the result of nothing more than the processing of subsequent
retinal images. However, Gibson questioned this, based upon the evidence
of invariants in static layouts, which he presented in Chapter 9 (see also
Warren, Chapter 9, in this volume). Based upon the aforementioned evid-
ence, Gibson (1979/2015) asked if there was evidence of (1) direct percep-
tion of changing layouts; and (2) direct perception of one’s own movement
relative to the environment. To do so, experiments needed to be developed
that would allow for the independent creation of changing layouts and
Acting Is Perceiving 175
would allow one to demonstrate that an organism can detect and use the
information generated from these changing layouts.
Direct Perception of Changing Layouts

Gibson (1979/2015) hypothesized that the perception of changes in
environmental layout was the result of attunement to perturbations of the
ambient optic array, which he asserted were invariants. Invariants in this
context are properties of the environment, or objects in the environment,
that are preserved in the structured patterns of energy (information) that
are available in the ambient optic array. Invariants are environmental (eco-
logical) properties, making them potentially available to any organism.
Importantly, these patterns (invariants) persist across changes in the
ambient array, as these changes are also governed by the properties of the
environment (or objects within it). Gibson (1979/2015) called these con-
sistent changes transformational invariants and suggested that organisms
are sensitive to them as well. After proposing this hypothesis, evidence
derived from a number of different paradigms that “artificially” created
these transformations was discussed (and are detailed in the next section).
Evidence for the hypothesis was also derived from research on magnifica-
tion and minification (a specific type of transformation). A study con-
ducted by Schiff, Caviness, and Gibson (1962) demonstrated that the rate
of magnification was proportional to the duration until collision (and leads
to avoidance behaviors). Later comparative research using cats, fiddler
crabs, frogs, and monkeys found comparable results, suggesting that invar-
iants rather than mental representations are being used to guide behavior
(Schiff, 1965).
The work conducted by Runeson (1977) also lends support to Gibson’s
(1979/2015) hypothesis. Runeson (1977) conducted a study using two-
body linear collisions and manipulated the motion after the collision,
which is believed to be an invariant. By systematically manipulating this
invariant, Runeson was able to create the perception of a specific type of
collision (using a pair of moving patches) as well as alter the hardness/
softness of the collision itself. Runeson and Frykholm (1983) were able to
demonstrate that invariants existed in biological motion as well, using
point-light recording of actors performing different actions. Participants in
these studies were able to reliably perceive not only the event in the record-
ing, but causal (dynamic) properties of the event as well, leading Runeson
and Frykholm to hypothesize that movement information (kinematics) is
specific to (and specifies) the causal properties (dynamics) of events and
that people are sensitive to this information.
176 L. James Smart Jr. et al.
Direct Perception of One’s Own Movement Relative to the
Environment
Gibson (1979/2015) also hypothesized that the perception of one’s own
movement was related to perturbations in the ambient optic array. One
line of evidence supporting this hypothesis came from Gibson, Olum, and
Rosenblatt (1955) who studied optic flow, namely, inflow and outflow,
both of which are invariants. Inflow (also known as expansive optic flow)
refers to when the ambient optic array expands from the center of the
visual field, specifying that one is moving toward something, whereas
outflow (also known as convergent optic flow) refers to when the ambient
optic array converges toward the center of the visual field, specifying that
one is moving away from something. Both inflow and outflow play a major
role in visual kinesthesis (co-perceiving oneself relative to the environ-
ment). At this point, Gibson (and we) understood that there were par-
ticular aspects of the ambient optic array and its movement analogue, optic
flow, that can specify object and self-motion. The persistent issue was how
to recreate this information in the absence of actual motion to allow for
experimental analyses.
Suggested Means of Studying Movement and

Self-Movement Perception
Gibson’s (1979/2015) original contention regarding retinal images was
that event perception is dependent on disturbances of structure in the
ambient array as opposed to light “prodding” and “scratching” the retinal
surface. This characterization of event perception shifts the starting point
for perception from a retinal image to information in the world, and sub-
sequently, the ambient array. As stated earlier, structural disturbances in
the ambient array were hypothesized to be transformations of invariant
properties of the world that reliably specify events. The presence of reliable
optical transformations in the ambient array allows for the direct percep-
tion of movement and self-movement due to the embedding of meaningful
structure within environmental information. In the subsequent pages, we
will reflect on these assertions while providing a modern context to Gib-
son’s (1979/2015) original ideas. Thus, we will briefly review traditional
techniques devised to study the perception of motion and events to con-
trast these methods with contemporary approaches.
Stroboscope, Moving Belts, Rotating Disks, and Projection

(Shadow)
One early tool used to study the perception of motion was the stroboscope.
The stroboscope is an instrument that generally emits brief flashes of light
and can be used to illuminate a moving stimulus at steady intervals. By
briefly illuminating a moving object at regular intervals, an observer would
experience only snapshots of the stimulus being presented. Despite only
being presented with snapshots of a stimulus, motion is often still per-
ceived. Gibson (1979/2015) argued that the perception of motion via the
stroboscopic effect ought to be considered “real” motion not simply
“apparent” motion as many early researchers suggested. Primarily, the
information in the ambient array that specifies motion is the same for con-
tinuous and intermittent events which suggests that a continuous retinal
image is not needed for event perception. That is to say, the perception of
events is not dependent on the creation of a retinal image that evolves over
time, from one identifiable point to the next, but rather, is dependent on
detecting the transformation of invariant information (not necessarily in a
successive manner).
Another method of studying event perception was the moving endless
belt which is an apparatus that allows for the presentation of various tex-
tured surfaces. The textured surfaces are moved behind a small window at
variable speeds and directions with an observer situated in front. A
paradox (the Aubert-Fleischl paradox) arises when an observer performs,
either, a pursuit tracking task (tracking the moving texture to its furthest
edge) or a fixation task (fixating on the center of the textured surface).
Namely, the speed and velocity of the textured surface are perceived as
moving more slowly during pursuit tracking than fixation (Gibson, Smith,
Steinschneider, & Johnson, 1957). This result is surprising because the
retinas of the eye are exposed to the same stimulus in either pursuit track-
ing or fixation but the perceptual experience is phenomenologically
different. Gibson (1979/2015) hypothesized that the perceptual difference
in viewing methods arise because the eyes are picking up the accretion and
deletion at the boundaries of the windowed, textured surface. The percep-
tual differences that arise in the different modes of observation can be
attributed to the movement of the background image during pursuit track-
ing and the movement of the stimulus in the retinal image.
The perception of motion has also been studied using various types of
rotating disks. The rotating disk apparatus is an apparatus that utilizes
either a color wheel or a spiral line to present circular motion to an
observer. Often, the disk is presented to an observer through a window or
some background. In the case of a color wheel, the velocity of the wheel
can be manipulated so that an observer perceives the individual compon-
ents of the wheel, as is the case at slower velocities. On the other hand,
when the wheel moves at higher velocities, observers typically perceive the
wheel as a blur (Gibson, 1954). When the disk is composed of a spiral line,
observers may perceive expansion or contraction of the disk depending on
its direction of motion. Interestingly, observers perceive the relative veloc-
ity of the rotating disk at different distances, in the case of a color wheel or
spiral line, at a constant rate even though distance and absolute velocity
share an inversely proportional relationship. Gibson (1979/2015) described
these effects, not as motion as defined by physics, but as a “shearing” of
texture at the contour of the rotating disk. It is the texture shearing at a
contour that is informative about the orbital motions of the rotating disk
since the rotating disk neither tracks across the retinal surface nor does the
retinal surface track the disk in motion.
The preceding paradigms were suggested by Gibson as a means of inde-
pendently creating specific perceptions of movement for objects or for the
self and represent truly innovative methodologies for studying movement.
The primary limitation of these methodologies is that they required a
rather elaborate and precise set of conditions to create the desired effect
and they have the unintended consequence of limiting the perceiver to
judgment-based, rather than behavioral, responses. However, studies of
visual kinesthesis have provided a means to examine both judgment and
behavior. Visual kinesthesis has been studied using a number of methods,
such as swinging or gliding rooms and optokinetic drums. These para-
digms can be used to create vivid experiences of self-movement in the
absence of actual physical movement (Gibson, 1979/2015). Gibson
(1979/2015) suggested that these experiences were the result of picking up
changes in the ambient optic array and the edges of one’s visual field
(which typically specifies movement of the self ).
The Swinging Room Paradigm

In order to demonstrate that kinesthetic information (information about
self-movement) could be derived from the ambient optic array, as well as
mechanical stimulation (e.g., somatosensory and vestibular system),
Lishman and Lee (1973) developed the swinging room paradigm. The
swinging room paradigm involves having a participant stand or walk
within an enclosure in which the walls can be moved allowing for the cre-
ation of optic flow. Interestingly, even when the participant was stationary,
moving the walls of the “room” resulted in experiencing themselves as
moving, suggesting that vision not only provides kinesthetic information
but it also dominates the other senses involved in kinesthesis. Furthermore,
it was found that the optic flow generated by the swinging room produces
systematic changes in postural sway (Lee & Lishman, 1975). Thus, these
studies were some of the first to demonstrate behavior correlates to the
detection of optic flow.
While Lee and Lishman’s studies made it clear that optic flow could
influence postural sway, the question of which aspects of flow information
were being used still remained. To determine this, Stoffregen (1985) con-
ducted a series of experiments in which radial (expansive flow) and lamel-
lar optic flow (changes in the ambient optic array indicative of sideways
movement) were presented to the central and peripheral retina. This was
achieved by using the swinging room in tangent with manipulating what
type of flow the central and peripheral retina received by changing the
direction in which the participants stood and using visual occluders. The
results suggested that both the central and peripheral retina play a role in
picking up flow but, the central retina can use optic flow (lamellar and
radial) to modulate posture to an extent (Stoffregen, 1985). And while the
peripheral retina can pick up lamellar optic flow and use it to modulate
posture, it cannot use radial flow in a similar fashion.
Technological updates to this paradigm have also benefited applied
research topics, such as motion sickness. Riccio and Stoffregen (1991)
suggested in their Postural Instability Theory that motion sickness was
the result of disruptions in the control of action in the presence of novel
or unusual motion environments. The cause of this disruption could be
the failure to detect the appropriate movement information or a lack of
ability to utilize this information (or both). Stoffregen and Smart (1998)
used a computer-controlled moving (swinging) room (Figure 10.1) to
present individuals with a complex optic flow that was not possible to
achieve with manually driven swinging rooms that tended to employ
sinusoidal movement. In particular, the optic flow presented by Stoffre-
gen and Smart had the key characteristic of being similar in nature to
optic flow that could be produced by postural sway. Stoffregen and
Smart found that this type of optical motion can induce compensatory
postural motion, as well as instability, and subsequent motion sickness.
In a follow-up study, Smart, Stoffregen, and Bardy (2002) were able to
demonstrate that resultant postural sway (as well as the experience of
vection, i.e., illusionary perception of self-movement typically induced by
visual stimuli; Hettinger & Riccio, 1992) were in fact predictive of later
motion sickness.
^ĞŶƐŽƌ
ŵŝƩĞƌ
^ĞŶƐŽƌ
ŵŽƚŽƌ
Figure 10.1 Depiction of the moving room used in Stoffregen and Smart (1998)
and Smart, Stoffregen, and Bardy (2002). The room was comprised of
3½ walls and a ceiling with the side walls on rails, so that the parti-
cipant stood on the laboratory floor.
Despite the usefulness of the swinging room paradigm, there were some
challenges that remained. One such challenge was that only the global
optic flow could be experimentally manipulated with most of the tradi-
tional physical room displays. This was problematic because outside of the
lab there is often local as well as global optic flow. Furthermore, the
realism of the environment was limited in such a paradigm, particularly in
terms of the types of motion that could be created (in most cases, it was
single axis motion). Gibson (1979/2015) noted that these challenges made
the study of visual kinesthesis harder than the study of event perception.
The need to produce a compelling optic flow to indicate a moving environ-
ment or movement through an environment was daunting to accomplish
with the technology and physical apparatus available at the time. A final
challenge that emerged from these constraints was that the direction of
influence was one-way. That is, room motion could be used to influence
personal motion, but not the converse. These challenges had the effect of
limiting the kinds of questions that could be asked, including the
following:
• How might we study the interaction of motion perception and self-

movement?
• How is the link between personal motion and environmental motion
detected and utilized?
• What happens if one fails to perceive this link?
A number of recent solutions to this issue will be discussed in the following

section.
Studying Movement Perception in the Age of Virtual

Environments
The emergence of new computer-based technologies has allowed for the
development of new paradigms that can circumvent the limitations of the
physical apparatus-based paradigms discussed earlier. These new paradigms
have allowed us to demonstrate that not only can we detect movement
information but we can also use that information to guide behaviors such as
locomotion. One such example is the work of Warren, Kay, and Yilmaz
(1996), who used virtual hallways to study optic flow. This was achieved by
projecting a hallway formed by quasi-random virtual patches that formed
the shape of a hall (walls and floor) in front of a participant while s/he
walked on a treadmill (Figure 10.2a). Oscillations that were indicative of
postural sway in six directions were superimposed onto the projected
hallway. Not only did this allow Warren and colleagues to demonstrate that
optic flow (namely, expansive flow) affected locomotion, but it also demon-
strated that motion parallax affects postural sway as well, which would not
have been possible to parse out using physical apparatus.
;ĂͿ ;ďͿ
Figure10.2 (a) Depiction of the virtual hallway stimuli (directional optic flow) used
by Warren, Kay, and Yilmaz (1996). (b) Depiction of large screen pro-
jection of optic flow (sinusoidal) used in Dijkstra, Schöner, and Gielen
(1994).
Using a similar type of manipulation Dijkstra, Schöner, and Gielen

(1994) studied optic flow in virtual environments (VE). The movement of
the participant (namely, their eyes) was captured using cameras and was
then used to render the scene they were viewing. They then sinusoidally
moved the wall and manipulated its temporal relationship with the parti-
cipants’ postural sway as well as virtual distance from the wall (Figure
10.2b). They found that both coupling (relative phase) of sway and stimu-
lus motion and the time it took to return to a stable coupling after a per-
turbation (sudden 180° shift in the stimulus) were reliably related to
parameters of the stimulus (e.g., visual expansion). Using these types of
methodologies, it has been demonstrated that there are regularities within
the ambient optic array which can be used to reliably modulate action. In
fact, it has been asserted that the information provided by optic flow and
the subsequent adaptive behavior can be characterized as a coupled
dynamic system governed by both mechanical and informational con-
straints (control laws) (see Warren, 2006, as well as Zhao & Warren,
2015, for a review of this topic).
VEs presented through head-mounted displays (HMDs) have also been
a powerful tool that allows one to examine how individuals use optical
information to guide locomotion. Fajen and Warren (2003) used HMDs in
order to study how individuals use optical information about distance and
angle to steer (themselves), avoid obstacles, and select routes. By systemati-
cally varying the parameters, the authors found that individuals’ rate of
turning was influenced by both the distance and angle of a target/obstacle
relative to the heading direction. Furthermore, their data suggested that
when locomoting, individuals’ paths are determined by targets/obstacles
rather than explicitly planning a route.
As a follow-up, Fajen and Warren (2004) used a similar task to deter-
mine what information and behavioral strategies individuals use to reach a
moving target. This was achieved by having the participants walk through
a VE and changing the information available in the environment. Over the
course of four experiments, it was found that participants used an inter-
ceptive strategy (i.e., they got in front of the target). Furthermore, this
strategy involved using egocentric direction information (i.e., the visual
direction of the target relative to their heading direction) rather than global
or local optic flow, as had been found in previous research. Taken together,
the results suggest that different types of movement information are used
for different types of tasks (e.g., locomoting to stationary targets vs moving
targets).
The true advance that VR/VE provides for the study of motion percep-
tion is that it permits the study not only of perception of motion but also
of how that perception is used to facilitate action in a dynamic, closed-
loop fashion. The importance of this innovation is that we can now look
at how motion perception is used prospectively as opposed to the compen-
satory behavioral responses that characterize much of the research
described in this chapter. It also provides a means of testing the behavior
dynamics suggested by Warren (2006). This is not a criticism of the
previous work but an acknowledgment of the potential to address new
questions. A good example of this is the work examining interceptive
action in virtual environments (Zaal & Michaels, 2003) that demonstrated
that there were key optical flow variables that provided information for
successful interception (catching fly balls). Importantly, this technology
also has the potential to allow for exploration of dynamic relations
between motor regulation and motion perception; a key example being the
research of Fajen and Warren (2003, 2004) described earlier. These studies
were able to show that participants can dynamically alter their movement
strategies in response to changes in optical flow in order to complete a
given task. While this task is still a compensatory one, it reveals a sensit-
ivity as well as an ability to exploit movement information. Littman,
Otten, and Smart (2010) performed a study to illustrate how prospective
and compensatory movement emerge in the presence of the same optical
stimuli. They had participants play or watch a first-person shooter video
game that was projected onto a large screen (1.5 × 2 m). The same stage
was used whether playing or viewing the game, so the only difference was
being dynamically linked to optical information or viewing it in an open-
loop (non-coupled) fashion. Littman and his colleagues found that postural
movement was different for passive and active trials, passive trials exhibit-
ing less overall movement and velocity. Interestingly, they found that this
difference was absent in participants who became motion sick in this
study. This revealed an important aspect of motion perception, namely,
being able to detect how (or whether) one’s own behavior influences the
available optical information. The detection of this information is essential
for kinesthesis and the prospective control of movement.
Thus, as in the original chapter, the question becomes (or remains)
“how might we study this ability to detect and utilize movement informa-
tion?” Some recent research in our lab might provide some possible
approaches to this problem. As we have presented the problem, it is not
just one of perception but of use, so the methodologies chosen need to be
able to address both aspects. An example of how this research could be
performed in an active task draws from some of the earliest studies in
vision involving transformations of the visual field using prisms (Stratton,
1897). Much of the classic research on prism adaptation was performed
with the assumption that adaptation was a perceptual phenomenon (e.g.,
Harris, 1965; Kohler, 1964; Welch, Choe, & Neinrich, 1974) rather than
a perception-action problem. Given this, the majority of this classic
research focused on perceptual reports and changes in performance (error
rate) but had little to say about actual behavior (for an interesting excep-
tion to this, see Dolezal, 1982). Recent work in the calibration literature (a
closely related literature to adaptation) has indicated that the study of cali-
bration (adaptation) should be grounded in action as one of perception’s
key functions is to aid in the appropriate regulation of behavior (Pagano
& Bingham, 1998; Pan, Coats, & Bingham, 2014). In other words, these
researchers reaffirmed that perception and action should be studied con-
currently. Thus, a means to assess whether movement information is being
detected and utilized is to create a situation where successful calibration
(adaptation) is dependent on action. This type of scenario was employed
by Littman (2011), who used a “virtual prism” paradigm in which parti-
cipants were immersed in a virtual room that had a maze on the front wall.
Participants were instructed to move alphabetically through a series of
targets to complete the maze (Figure 10.3).
What participants did not know is that Littman had altered the physics
of the virtual world to mimic prism-like distortions; in this case inverting
the yaw and pitch outputs such that if a person physically looked up and
to the left, the virtual world would present movement down and to the
right (Figure 10.4, Panel I, line B). Importantly, in contrast to physical
prism studies where the distortion is immediately apparent, the virtual dis-
tortion was only detectable (revealed) through movement, so the task
requires a behavioral rather than perceptual adjustment. Littman (2011)
found that participants could indeed adapt to the distortion, which was to
be expected––what was important is that it is one of the few studies that
actually recorded behavioral data while adapting. The analysis of the
behavior revealed that adaptation was marked by a decrease in the magni-
tude of motion over time coupled with movement strategies that become
K
D
,
/
d
z h
s

y
t
Figure 10.3 Set-up and depiction of study for Littman (2011). Participants viewed
a virtual version of the laboratory through a head-mounted display
(HMD).
more stable as they adapt. In other words, behavioral adaptation begins with
large and varied motion and eventually “settles” into a working pattern of
movement that takes into account the novel perception-action relationship
of the virtual environment. Littman was also able to show this progression
in the presence of a complex distortion (depicted in Figure 10.4, Panel III,
line F). In a similar fashion, Hartman (2018) was able to show that cali-
bration (adaptation) could occur in the presence of varying perturbations
(her manipulation can be represented by combining Panels II and III of
Figure 10.4). From a research standpoint this is promising because in VE,
physics can be altered in a number of ways. Figure 10.4 depicts some poten-
tial ways in which this perception action relationship can be altered from
spatial distortions (Figure 10.4, Panels I–III) to temporal distortions (Figure
10.4, Panel IV). Theoretically it is useful in that it allows for testing different
forms of visual information that are needed to guide behavior (one of Gib-
son’s original motivations for understanding optical flow).
/ ///

sŝƌƚƵĂůŚĞĂĚŽƌŝĞŶƚĂƟŽŶ;ΣǇĂǁͿ

&
ϲϬΣ

ĐƚƵĂůŚĞĂĚŽƌŝĞŶƚĂƟŽŶ;ΣǇĂǁͿ ĐƚƵĂůŚĞĂĚŽƌŝĞŶƚĂƟŽŶ;ΣǇĂǁͿ
/s
// '
ŚĞĂĚŽƌŝĞŶƚĂƟŽŶ;ΣǇĂǁͿ
ZĂƟŽŽĨǀŝƌƚƵĂůͬĂĐƚƵĂů

ϱƐ

ĐƚƵĂůŚĞĂĚŽƌŝĞŶƚĂƟŽŶ;ΣǇĂǁͿ dŝŵĞ;ƐĞĐͿ
Figure 10.4 Potential alteration of perception and action when using VE. Panel I:
spatial distortions, inversion (line B) and offset (line C). Panel II:
spatial gain distortions, negative gain (compression – line D) and
positive gain (expansion – Line E). Panel III: complex mapping (line
F). Panel IV: temporal distortion (modulating gain over time – line G).
Line A in all panels refers to normal (real-world) mappings.
From an applied standpoint, VE also allows us to determine what

happens when we fail to detect or use relevant movement information.
Villard, Flanagan, Albanese, and Stoffregen (2008) replicated the findings
of Stoffregen and Smart (1998) in a virtual environment and confirmed
that optic flow can be behaviorally disruptive (and lead to motion sickness)
when it does not support postural regulation. They employed the same
sum-of-sines motion profile as was utilized in the physical moving room,
demonstrating that ,like the virtual prism studies mentioned above, VE can
provide a means of logically extending this literature. One limitation of
Villard and his colleagues’ (2008) study is that it, like the original moving
room motion sickness studies, was open-loop, which relegates the particip-
ants to compensatory rather than prospective actions. To address this, VE
can again be used as one of the key aspects of the technology is that it is
designed to be interactive––behaviors of the participant have consequences
for the information available in the display and, conversely, the available
movement information can have consequences for subsequent behavior.
Specifically, the relationship between perception and action can be
manipulated in VEs. For example, Smart, Otten, Strang, Littman, and Cook
(2014) performed a conceptual replication of the work of Stoffregen and
Smart (1998) and Villard et al. (2008) by exposing participants to global
optic flow via a head-mounted display. Smart et al. (2014) manipulated both
the type of information that was available to the participants as well as the
relation between this information and their current behavior with the aim of
discovering how changes in these parameters influenced the participants’
movements. Participants were exposed to one of four types of optic flow
(movement information; see Figure 10.5):
1. Open-loop sinusoidal motion (similar to Dijkstra et al., 1994).

2. Open-loop complex motion (playback of the participant’s own motion
during baseline, approximates sum-of-sines stimuli used by Stoffregen
& Smart, 1998).
3. Closed-loop anti-phase motion (moving closer creates expansion of the
flow field, this is the real-world mapping).
4. Closed-loop in-phase motion (moving closer creates contraction of the
flow field, this is inverse of normal coupling).
Smart et al. (2014) found that not only did motion sickness rates vary
across conditions, with almost no motion sickness in the sinusoidal con-
dition and significant sickness in the in-phase condition, but there were
also behavioral differences between conditions and generally between
participants who became motion sick and those who remained well.
Similar to the Littman (2011) virtual prism study, successful postural regu-
lation involved decreased magnitude of motion over time, however, this
was coupled with more variable motion strategies over time. Those who
became motion sick seemed to be disrupted by the optic flow (evidenced by
higher magnitudes/complexity of movement) and were unable to use the
optic flow to regain stability (evidenced by rigid movement strategies over
time). This suggested a difference in the ability to recognize and exploit
movement information to guide behavior. Interestingly, anecdotal reports
by participants who became motion sick in the closed-loop (coupled) con-
ditions seemed to indicate that they were unaware that they were influen-
cing the optic flow, again suggesting that failure to detect information that
;ĂͿ ;ďͿ ;ĐͿ ;ĚͿ
Figure 10.5 Depictions of optic flow conditions used in Smart et al. (2014).

(a) Uncoupled sinusoidal motion, (b) Uncoupled complex motion.
(c) Coupled anti-phase motion. (d) Coupled in-phase motion.
specifies behavior can have detrimental effects. This observation was tested
more carefully by Cook, Hassebrock, and Smart (2018) by examining
participants’ responses to optic flow produced by other people. Cook and
his colleagues (2018) found that participants’ postural responses differed
when presented with optic flow generated by a person who became motion
sick or generated by a person who remained well. In particular, particip-
ants’ postural motion exhibited higher coupling with the optic flow from
the well person than from the motion sick person, suggesting that they
were sensitive to optical motion that could be used to successfully guide
their movements. However, like the optic flow studies mentioned earlier,
the stimuli in this case were open-loop (meaning that the participants’
actual motion had no impact on the optic flow they received). The result of
this is that while the participants perceived usable information, they were
unable to employ it to dynamically control their actions and in fact became
posturally unstable and motion sick more often than those exposed to the
optic flow from a previously motion sick person.
So where does this leave us? The studies described in this chapter
suggest that perception of motion in the world and of the self is essential
for successful interactions with our environment and, importantly, regula-
tion of our own behavior. Importantly, the newer methodologies expose
the importance of the dynamic relation between the perception of motion
and the guidance of behavior (as suggested by Warren, 2006). The research
described in this chapter also revealed that disruptions in this dynamic
coupling produce adverse outcomes, such as motion sickness. The advances
in technology described in this chapter have allowed us to ask (and answer)
more sophisticated questions about the dynamic relations between motion
in the world and of the self. These questions in some ways run counter to
what Gibson proposed in the original chapter as he warned that the study
of (visual) kinesthesis was different but often confounded with the study of
the control of movement. However, this warning in the original chapter
may reflect the technical limitations of the time that would have made
experimentally studying both the perception of motion and the control of
motion at the same time difficult. Inquiring about the nature of the
mapping between perception and action and/or our ability to distinguish
when that mapping is consequential (coupled) remain important and rel-
evant as we now operate in both natural and virtual contexts. The studies
outlined (and proposed) in this chapter can be seen as both a fulfillment of
how Gibson felt the study of motion perception needed to proceed and as
an extension of the importance he placed on movement as a way to disam-
biguate perceptual phenomena. Whether virtual or real-world, accurate
specification and guidance of action through motion perception are some-
thing that needs to be addressed as the consequences for failing to detect
movement information in either context are real. In short, movement still
matters.
11 Revisiting “The Discovery of the
Occluding Edge and Its
Implications for Perception”
40 Years On
Harry Heft
The important result … was not the perceiving of depth at an occluding

edge … depth perception requires no departure from traditional theories …
but instead the perceiving of the persistence of the occluded surface.
(Gibson, 1979/2015, p. 181)
With the above quotation in mind, I begin by emphasizing what this

chapter is not about. It is not about occlusion, but rather about the percep-
tual phenomenon of the occluding edge or dynamic occlusion. Occlusion
as such, also sometimes known as interposition or superposition in the
psychological literature, is a technique that has been utilized for centuries
by visual artists to portray the relative distance of two or more objects in a
pictorial representation. If from a specific point of observation, object B is
at least partially occluded by another object A, then B is usually perceived
to be at a greater distance from the viewer than A. The employment of
occlusion in paintings and drawings contributes to the appearance of depth
on a flat surface. In that regard, occlusion is typically referred to as a pic-
torial cue. In contrast, the phenomenon of dynamic occlusion is a visual
event; and as an event, it occurs over some duration. Minimally, it consists
of one surface progressively occluding another surface from sight: one
surface gradually goes out of sight over time at an edge of another.
Critical in the history of perceptual theory, researchers in the mid-19th
century, notably Helmholtz, began to list occlusion or superposition as one
visual cue among others for perceiving relative distance in the field of view.
Boring (1942) explicitly referred to it as one of the devices “which the
painter uses” (p. 263), noting that “[a]lthough the principle is too obvious
ordinarily to receive special mention as an artistic technic [sic], it eventu-
ally got into the lists of secondary criteria for the perception of distances,
as for example, Helmholtz’s in 1866” (p. 264, emphasis added). It is sup-
posed that because the retina is essentially a two-dimensional surface of
receptors, distance or depth is not immediately perceivable. The claim is
that the visual system uses occlusion on the retinal image as a cue for infer-
ring depth in the environment in a similar manner as an artist creates the
Revisiting “The Occluding Edge” 40 Years On 189
illusion of depth by employing occlusion on a canvas. It is worth noting
that this shift in domain of discourse from techniques that artists use to
modify surfaces to cues utilized by the visual system to construct a percept
has proceeded uncritically over decades of perceptual theorizing.
In the course of developing his ecological approach to perceiving,
Gibson abandoned the causal account of visual perception and its assump-
tion that vision begins with a projection of patterns of stimulation on the
retinal surface. Instead, he proposed that vision, when considered as a per-
ceptual system, involves detecting specifying information usually from a
moving point of observation in the ambient optic array of reflected light
(see, Mace, Chapter 5, in this volume). Perceiving entails the pickup of
information about the surface layout of the surrounding environment that
extends away from the individual. The quality of depth is not added to a
two-dimensional “image” of the environment by means of cues; instead,
extended surfaces as such are perceived.
From this alternative formulation, we can take some initial conceptual
steps to understand how it might be that the environment that is perceived
has an immediacy of, what William James called voluminousness, which is
a quality “discernible in … the original sensation of space” (James, 1890,
p. 777, emphasis in original). By immediacy, James is indicating that this
quality of voluminousness is not added to experience, but rather it is a
primordial quality of perceptual experience. Dynamic occlusion contributes
to the immediacy of voluminousness as objects are perceived to pass in
front of or behind other surfaces. It is well worth remembering in this
regard that in his early years James trained to be a painter (Bjork, 1983);
and for this reason he, unlike others writing about visual perception, was
not apt to confuse perceptual experience of the environment with percep-
tual experience of a representation of the environment created through art-
istic artifice.
Moreover, under the circumstances of dynamic occlusion, an occluded
feature of the environment is commonly experienced as being “present”
even though temporarily it may not be immediately in sight. In that light,
because perceptual processes are conventionally taken to refer to aware-
ness of what is immediately present in the field of view, the phenomenon
of dynamic occlusion compels us to reconsider what we mean by the term
“perceiving.” Dynamic occlusion reveals perceptual awareness to be
extended beyond a momentary slice of time, and critically, features of the
environment are experienced as persisting even when they are not in the
immediate field of view. A redefinition of perceiving in view of these qual-
ities of dynamic occlusion will in turn call for a reconsideration of what is
meant by remembering and anticipating. For this reason, attention to the
phenomenon of dynamic occlusion goes well beyond a consideration of a
relatively minor perceptual curiosity, but it has broad implications for
psychology theory.
190 Harry Heft
The Phenomenon of Dynamic Occlusion
If we begin somewhat simply by conceptualizing the environmental sur-
round as being composed of surfaces, objects and events––with objects
being minimally bounded entities and events being perceptible changes
over some duration of experience––then the phenomenon of dynamic
occlusion is an event relating to perceived changes among objects and their
surfaces. As an event, it cannot be perceived in an instant of time, and
therefore it cannot be displayed in a painting or captured in a photograph.
For this reason, perceptual researchers who conceptualize vision as begin-
ning with a static image on the retina, or who exclusively utilize pictures in
their work, are apt to minimize the significance of dynamic occlusion for
perceptual theory.
The occluding edge effect was studied experimentally by Gibson’s Ph.D.
student George Kaplan (1969; also see Gibson, Kaplan, Reynolds, &
Wheeler, 1969), although a few researchers and philosophers had antici-
pated it in prior decades. The effect can be produced in different ways, but
let us take the following simple case as prototypical. Consider an opaque
textured surface passing in front of another larger opaque surface with the
same texture. In the course of doing so, the leading edge of the moving
smaller surface progressively covers or occludes from view portions of the
larger surface; and as this smaller surface continues along its direction of
motion, those once occluded portions of the larger surface are progres-
sively revealed at the “trailing” edge of the occluding surface. That is, the
moving occluding surface gradually conceals and reveals portions of
the larger surface over time. In doing so, it appears to pass in front of the
larger surface.1
There are two features of this event that are important to point out.
First, the effect is only perceivable over time. This becomes clear if we
examine Kaplan’s method of producing the effect. He designed his experi-
ment using a stop action animation technique, photographing a series of
still shots as the smaller surface was gradually moved manually over (or
behind) the larger surface. The separate still shots were then run as a con-
tinuous film. As long as the animation showed the occluding surface to be
moving, the two surfaces were clearly visible with the occluding surface
appearing to be moving in front of the occluded one. But if the animated
movement of the occluding surface ceased, the appearance of two distinct
surfaces disappeared and only one homogeneously textured surface was
visible. Likewise, the effect of one surface passing in front of another was
only visible across projected frames of the film: notably, it was nowhere to
be seen in any single frame of the film.
Second, and especially significant for our purposes, the portion of the
larger surface that was progressively covered was experienced by observers
as remaining present even at those times when it was not immediately
visible. That is, the hidden portions of the surface persisted in visual
experience even when out of sight. This persistence effect is even more
striking in those cases when the moving, smaller surface is occluded progres-
sively by the larger surfaces and then reappearing at its opposite edge.
Dynamic occlusion is not the only way that an object can go out of
sight. An object can also go out of sight when it is burned, shattered,
melted, and consumed (Gibson, 1979/2015). What is different about those
events as compared to dynamic occlusion is that in those instances the
object in question goes out of existence, and this is because each of those
kinds of events are perceived as irreversible. Dynamic occlusion is critically
different because it is a reversible event. An object or surface that has
undergone dynamic occlusion is perceived as having gone behind an
occluding surface and is not presently in sight may return to the field of
view at the opposite edge of the occluding surface if it continues along its
prior direction of motion. Alternatively, if an object has gone out of sight
at an occluding edge because the observer changed the point of observa-
tion, it can be brought back into sight with the observer’s return to the
initial observation point. In both cases, the object is perceived as persisting
even when it is temporarily out of sight. What perceptual information
specifies that an object has gone out of sight rather than having gone out
of existence? Gibson proposes that it is the gradual deletion of an object’s
surface at an occluding edge. That type of transformation does not occur
in any of the above cases of irreversible events.
The Commonplace Nature of Dynamic Occlusion

Kaplan’s filmed demonstration of the occluding edge effect belies just how
commonplace it is in everyday experience because in that demonstration
the observer of the film in effect occupies a stationary viewing position.
More commonly, a perceiver is moving about in a landscape scattered with
objects, and under these circumstances there are occluding edge effects
generated through action, often in multiple locations in the visible field of
view. What is salient in such instances is that previously visible, now
occluded (hidden) objects are experienced as still being present––still “in
existence”––even when they are out of sight. Pragmatically, a hidden
surface continues to exist because if appropriate actions are taken, such as
changing one’s observation point, the surface will come back into sight.
In those environments where most of us find ourselves on a daily basis,
objects go “in and out of sight” behind other objects continuously as we
move about. Phenomenologically, they do not go “in and out of existence”
but “in and out of sight.” And even when we are occupying a stationary posi-
tion, we are aware that the parts of the room not presently in sight continue
to exist because they can be brought into view with appropriate actions.
How are we to make sense of these phenomenological claims and these
pragmatic assumptions? As already mentioned, temporarily hidden objects
can be brought back into the immediate field of view with appropriate
192 Harry Heft
actions; and awareness of this possibility is awareness of what is hidden
(Gibson, 1978b). In this respect, perceiving extends in duration beyond
what is immediately present. But this raises a further question: how
extended can perceptual awareness be?
Antecedents
Observations to the effect that objects that are presently out of sight persist
in experience can be found scattered in the 20th-century phenomenological
literature. One notable case in point is Merleau-Ponty’s (1962) treatment
in The Phenomenology of Perception (1962) of the perception of distance
or depth. Merleau-Ponty points out that “[t]raditional ideas are at one in
denying that depth is visible” and for that reason, it is assumed that “dis-
tance, like all other spatial relations, exists only for a subject who synthe-
sizes it and embraces it in thought” (p. 255). But Merleau-Ponty rejects
this account, claiming that depth, as when perceiving a three-dimensional
object, is as visible as breadth with a change in viewing position. In pas-
sages that made a deep impression on Gibson (pers. comm., 1977), he
writes that depth in that case is only “breadth seen from the side” (original
emphasis), and if I am unable to immediately perceive depth at this
moment it is because “I am simply badly placed to see it. I should see it if I
were in the position of a spectator looking on from the side” (Merleau-
Ponty, 1962, p. 255). And the side of the object that is presently out of
sight is experienced as still existing. If we adopt the perspective of perceiv-
ing as a process of detecting information with respect to a moving point of
observation, then depth is perceived as immediately as is breadth. Echoing
the remarks on the immediacy of voluminousness from William James
cited above, Merleau-Ponty continues that such considerations of depth
force us to “rediscover the primordial experience from which it springs; it
is, so to speak, the most ‘existential’ of all dimensions” (p. 256, emphasis
added).
Although Gibson does not cite Merleau-Ponty in connection with the
occluding edge, he does discuss in The Senses Considered as Perceptual
Systems (Gibson, 1966a) other instances where this phenomenon is antici-
pated. He notes that the Gestalt psychologist Koffka (1935) observed that
“he could ‘see’ the top of his table extending behind the book that lay on
it” (Gibson, 1966a, p. 204). Although Koffka attributed this phenomenon
to the dynamics of perceptual organization, Gibson avers: “insofar as his
head moved and the texture of the table was [optically] wiped and
unwiped by the edges of the book, he had information for perceiving the
table behind the book” (p. 204).
More germane to what he will later call the occluding edge, Gibson
(pp. 203–206) offers a somewhat lengthy treatment of Michotte (1963)
and his colleagues’ investigations of “kinetic occlusion.” Michotte’s
powerful demonstrations force a reconsideration of the very definition of
perception. It shows that to regard perception to be awareness of the
environment based on immediate sensory stimulation is too restrictive. Per-
ception instead is the awareness of the environment through the detection
of information, and information is most readily detected over time. In the
present case, that information is the gradual occlusion over time of a
surface at an edge. Such perceptual information specifies that the object is
going out of sight, not out of existence. Gibson (1966a) writes:
[T]his development need not be conceived as an intellectual construc-

tion of reality [e.g., attaining object permanence] from data that do
not contain the reality; it can be conceived as a process of learning to
extract information from light that does convey reality.
(p. 206)
Occluding Edge Research Post Gibson (1979)

Investigations of dynamic occlusion in the decades following Gibson
(1979/2015) can be organized with reference to three headings. One of
these, the effect of dynamic occlusion on perceived depth in two-
dimensional displays, is not our focus here; but for representative research
on this matter, see Andersen and Braunstein (1983); Braunstein, Andersen,
and Riefer (1982); Ono, Rogers, Ohmi, and Ono (1988); Rogers (1984);
Rogers and Graham, (1983); and Yoonesi and Baker (2013). We turn next
to the others.
Empirical Studies of Object Persistence

Studies examining the claim that perceivers remain aware of objects when
they are temporarily out of sight have been mostly carried out with infants.
Two methodological approaches have been employed in this body of work.
The habituation-novelty methodology long used in infant research has
been adopted here to observe infants’ reactions to events that do or do not
deviate from what should have been expected based on initial occlusion
information. For example, Moore, Borton, and Darby (1978) examined
responses when an occluded object either reappeared at a location that was
inconsistent with its initial trajectory or when the form of the object
changed upon reappearance. Five-month-old babies responded in each case
in a manner suggesting that their expectations had been violated. These
results suggest that both an object’s presence and its identity were experi-
enced as persisting even when the moving object was out of sight behind
an occluder.
A second approach has been to record eye-movements that appear to
track an object from the time of initial occlusion at an edge to its reappear-
ance from behind the occluder (e.g., Bertenthal, Longo, & Kenny, 2007;
Bremner, et al., 2005; Johnson, Amso, & Slemmer, 2003; Rosander & von
194 Harry Heft
Hofsten, 2004; von Hofsten, Feng, & Spelke, 2000). Findings indicate that
infants can track temporarily hidden objects, but at what age they are able
to do so varies as a function of factors, such as the width of the occluder.
In a recent overview, Bremner, Slater, and Johnson (2015) argue that
results employing this approach indicate that early experience of object
persistence stems from perceptual learning processes rather than innate
core knowledge, and that deletion and accretion of forms at an edge are
necessary for perceived object persistence (also see, Bertenthal, Longo, &
Darby, 2007). These investigations do not consider actions apart from eye-
movements; however, Moore and Meltzoff (1999) found that infants will
crawl across a laboratory room to retrieve an object placed under another
occluding object, and in a second study involving action, they will reach to
retrieve an object that was moved behind a nearby occluder.
What experiences early in life might contribute to infants’ awareness of
object persistence following dynamic occlusion? What opportunities might
they have had to discover that progressive deletion at an occluding edge
specifies object persistence? It is not very difficult to imagine ways in which
dynamic occlusion at an edge is a recurring event in daily life. Surely,
reversible occlusion events are available to be perceived by an infant count-
less times each day, from caregivers momentarily leaving and then return-
ing to the field of view and objects being brought into and moved out of
the field of view, to self-generated occlusion events accompanying control-
led hand and limb movements. Considered from a developmental per-
spective, object persistence with dynamic occlusion ceases to be some
arcane philosophical puzzle and can be recognized as a fact of daily life.
As for adults, in addition to the powerful phenomenology of dynamic
occlusion, Scholl and Pylyshyn (1999) showed experimentally that obser-
vers are able to track multiple objects even as they appear to pass behind
an occluder as long as they reappeared at a location consistent with their
previously viewed trajectory.
Investigations of Perceptual Continuity

Several investigations have probed the possible mechanisms that account
for the experience of continuity of objects over the course of their partial
occlusion (e.g., Palmer, Kellman, & Shipley, 2006; Sekuler & Palmer,
1992). These investigations often go beyond an analysis of occluding edge
phenomena per se. Kellman and his colleagues (e.g., Kellman, Garrigan,
& Shipley 2005; Kellman & Shipley, 1992; Shipley & Kellman, 1992)
have examined the question: how does the visual system “assemble [an]
accurate representation of the environment of reality from partial
information” (Palmer, Kellman, & Shipley, 2006, p. 514). Their work
draws inspiration from Gestalt principles, such as good continuation, and
it is anticipated by Hochberg’s earlier proposal of a schematic map
(mental representation) that integrates separate glimpses of forms (e.g.,
Hochberg, 1968, Chapter 6). Following Hochberg, Kellman and col-
leagues employ a methodology whereby partial views of forms are briefly
available to be perceived as they move behind a narrow aperture.
Palmer, Kellman, and Shipley (2006) propose two processes that allow
for the interpolation of surface areas momentarily out of sight based on
contours that are intermittently available to be perceived: one of these, per-
sistence, is a memorial process that allows “object fragments [to] remain
perceptually available for a short time after occlusion” (p. 516), and
another, position updating, is based on the perceived velocity of the visible
fragments. The idea for persistence in their model is based on the visual
icon, which has been a mainstay in perceptual theory since Sperling (1960)
and notably since Neisser’s (1967) influential book. What they fail at least
to address is that Neisser (1976) later minimized the significance of the
visual icon, as did an early promoter of information-processing models,
Haber (1983). They did so after coming to the viewpoint that the visual
icon is primarily a laboratory artifact that, at best, occurs quite rarely in
everyday circumstances. It is noteworthy that Neisser (1976) minimized
the significance of the visual icon after embracing Gibson’s perception-
action approach. In any case, it is difficult to reconcile the approach of the
Kellman group with Gibson’s because of their view of perception as a
process of mental construction.
Pan and colleagues (Pan, Bingham, & Bingham, 2013; Pan, Bingham,
& Bingham, 2017; Pan, Bingham, Chen, & Bingham, 2017) have
attempted to bring together, on the one hand, the kind of constructivist
model Kellman and colleagues have proposed with, on the other, an
ecological approach to dynamic event perception. They propose that
“[n]ormally, vision entails use of both [static] images and optic flow”
(Pan et al., 2013, p. 1639); and that continuity is provided by a stable
image structure made available in retinal displays. Unlike the Kellman
group which invokes a visual icon, they argue that stable image structure
is necessary to explain perceiving during those times when optic flow
stops. Still, the inclusion in their account of stable images captured on
the retina, which they call “embodied memory,” would seem to set this
line of research at odds with Gibson’s rejection of the retinal image for
perceiving. Presumably, they would want to distinguish embodied
memory and the retinal image. Still, Gibson resisted an account of
perceiving that was image-based.
Whether these recent proposals are necessary conceptual steps in the
development of an account of event perception, or whether they dilute and
possibly derail Gibson’s initial formulation of the occluding edge phenom-
enon and its implications for perceptual theory, remain to be seen. No
doubt some resistance to Gibson’s analysis of dynamic occlusion will stem
from questioning the plausibility of his claim that perceiving (perception-
action) extends beyond the present “moment in time.”
196 Harry Heft
How Extended Is the Present?
Whether viewed as psychological functions or as faculties, perception,
memory, and imagination have commonly been distinguished with respect
to a linear dimension of time. In this regard, perceiving has to do with
present experience, remembering with past experience, and imagining
bears on anticipated or hoped for future occurrences.2 But the occluding
edge effect, and indeed other perceptual events that transpire over some
duration, put a crimp in that all-too-neat division. In this respect, many
writing from a phenomenological perspective have pointed out that the
very notion of the present extends beyond an instant in time, that the
experienced present includes awareness of what came before and what is
likely to come next.
William James (1890) put the matter this way: “our consciousness never
shrinks to the dimensions of a glow-worm spark. The knowledge of some
other part of the stream [of awareness], past or future, near or remote is
always mixed in with our knowledge of the present thing” (p. 571, empha-
sis in original). This phenomenological description led him to claim that
the idea of a present instant in time is an abstraction based on subsequent
(post hoc) reflection about a flow of experience. Our immediate experience
is of a “specious present.”
In short, the practically cognized present is no knife edge, but a saddle-

back, with a certain breadth of its own on which we sit perched, and
from which we look in two directions into time. The unit of composi-
tion of our perception of time is a duration, with a bow and stern, as it
were—a rearward- and a forward-looking end … The experience is
from the outset a synthetic datum, not a simple one; and to sensible
perception its elements are inseparable, although attention looking
back may easily decompose the experience, and distinguish its begin-
ning from its end.
(James, 1890, pp. 574–575)
The specious present as identified by James is “a duration which is per-

ceived both as present and as temporally extended” (Le Poidevin, 2019).
On these grounds, it will be useful to distinguish between duration and
time (Bergson, 1889). Duration refers to a quality attending the experience
of events which are happenings in experience that transpire; whereas time
is a cultural convention of dividing transpiring experience (duration) into
discrete, measurable units.
The present as temporally extended both retrospectively and prospec-
tively is in evidence with dynamic occlusion. In the case of Michotte’s
tunnel effect (see Gibson, 1966a, pp. 203–206), as a moving object is per-
ceived as being gradually occluded at the tunnel’s edge, there is the experi-
ence of the “object-in-the-process-of-being occluded” that was just
previously (retrospectively) more fully visible, and that will soon (prospec-
tively) go fully out of sight. And over the duration that the moving object
is not in sight, there is an awareness of its being still present as it moves
through the tunnel because it had been gradually occluded retrospectively
as well as a prospective awareness of its anticipated gradual appearance at
the opposite end of the tunnel.
But how extended is perceiving? What is its durational breadth? Before
turning to such questions, it is necessary to distinguish, if only loosely,
between acts of perceiving and acts of reflection. It is true that this is par-
tially a definitional issue, but attention to how concepts are defined is para-
mount to theory development. Much of the confusion about perceiving,
remembering, and anticipating as distinguishable psychological processes
stems from the failure to recognize the difference between, on the one
hand, the source of experience that accompanies perceiving, and, on the
other, the source of experience that accompanies reflecting. Perceiving
typically occurs in the ongoing flow of perception-action; it is a transac-
tional engagement with immediate circumstances, however extended in
duration. In contrast, reflective acts involve stepping outside of the ongoing
flow of perception-action, even momentarily, to think reflexively about
some object or event. Following Holt (1914), whereas perceiving transpires
in the present, however extended, acts of reflecting are indexed with refer-
ence to an abstract, conceptual, temporal dimension. What we refer to as
remembering or anticipating includes an awareness that the reflected-upon
object or event is “located” in some time past or time future. This distinc-
tion between awareness that accompanies perception-action and awareness
that accompanies reflection parallels James’ (1890) well-known distinction
between knowledge by acquaintance and knowledge by description or
sometimes knowledge-about (Heft, 2003).
With this perceiving-reflecting distinction in mind, as well as James’
notion of the specious present, it might not seem so odd to attribute pro-
spectivity and retrospectivity to perceiving if perceiving is considered as
involving the pickup of information through perception-action; while, in
turn, treating memory as concerning objects and events no longer presently
available to be experienced, and anticipating as concerning objects and
events that have yet to be experienced. Indeed, this is the conceptual move
that Gibson (1979/2015) makes in Chapter 14 (see Thomas, Wagman, &
Riley, Chapter 14, in this volume). There is, on the one hand, ongoing per-
ceiving that has a prospective-retrospective durational breadth, and, on the
other, reflecting on past or future events.
From this vantage point, how do we make sense of those aspects of the
environment that are currently out of sight but can be brought into sight
with the appropriate actions, such as the parts of the room currently
behind me? Do I remember those parts of the room while only perceiving
what I can immediately see? Although that way of stating matters may at
first seem sensible, on closer scrutiny, it is fraught with difficulties. The fact
198 Harry Heft
that perceiving occurs over time, specifying “what I can immediately see”
would seem to be always slipping from my grasp. In addition, knowing
that the room behind me can be brought into view would seem to have a
place in present awareness. Indeed, I would not even be aware in the
present that I can bring it into view if it were not already there, even in the
margins (the horizon) of awareness. Likewise, it is not difficult to imagine
what one’s reaction would be if upon turning around one found an entirely
different arrangement of the room than had been there previously. Empiri-
cally, we have seen that infants exhibit surprise when a particular form
that was occluded reappears as an entirely different form (Moore, Borton,
& Darby, 1978). Perceiving would seem to be extended in duration, as is
also shown by the research literature on visually guided action (e.g.,
Gibson, 1958; Fajen, 2005b; Warren, 2006) and time-to-contact (e.g., Lee,
1976; Lee & Reddish, 1981), demonstrating the prospectivity of perceiving.
In contrast to extended perceiving, a truly past event cannot be brought
into view regardless of what actions I take, and I am aware that this is the
case. Of course, representations of past objects and events can be brought
into the present by means of, e.g., photographs and written texts. As for
the future, anticipated events cannot be transposed into my present experi-
ence regardless of how much I might want them to be. In some ways, they
are even less accessible than past events because any representation of them
is wholly speculative. The old distinctions between perception, memory,
and anticipation as referring to present, past, and future, respectively,
simply do not work once we recognize that the present as a “razor’s edge”
in time is an abstraction, and what is present refers to what is and can be
experienced by taking action.
What then can we say to the question: how extended is perceiving? Or,
more colloquially, how far ahead prospectively can an individual perceive?
And are there are durational limits to perceiving? On logical grounds
alone, there would seem to be no fixed answers to such questions if we
assume that informational structures, on the one hand, are variously
extended over time, and that perceiving by an individual, on the other,
always involves a history of perceptual learning. That is, the relative pro-
spectivity of perceiving is tied to the perceived structure of an event from
an individual’s standpoint; and also an individual’s awareness of prospec-
tive structure can change with experience. Take the case of driving a route
between two destinations. To the person who has never driven the route
before, it may likely be experienced as so many separate segments that
comprise it. At first, the duration of what can be perceived prospectively
may only be as far the next turn in the route. For a person with more
experience, the perceived structure may well encompass multiple segments
experienced as a series of higher-order extended units. And for a person
who has driven the route very many times, it may be experienced as one
extended nested set of structures that is revealed over time through action
(see below). Or to offer an example with regard to auditory events, a
s ymphony may be experienced as a multitude of segmented passages for a
newcomer to the piece, but to the conductor, it is plausible that the sym-
phony is experienced as having a unitary, if variegated, extended, nested
structure.
Pertinent to these conjectures is Eleanor Gibson’s critical assessment of
the claim that adults have an apparently wider “span of attention” as com-
pared to children. In her seminal book on perceptual learning, E. Gibson
(1969) has written:
The length of a perceptual span does not seem to distinguish the per-
ception of adults from that of children, but I believe it is the ability to
find the structure, the embedded relations constituting subordinate
units or cluster, that make a difference. It is not just a stretching out
but a making of one unit out of many smaller units.
(p. 379)
Likewise, James Gibson (1979/2015) asserts that “[p]erceiving gets wider

and finer and longer and richer and fuller as the observer explores the
environment” (p. 244). On these grounds, the question, how extended is
perceiving?, is an empirical one that can only be answered with respect to
a particular event structure and relative to the prior history of the indi-
vidual with that structure.
Perceiving a Path of Travel

Animals and humans are capable of being oriented to the habitat …
[they] can go to places in their environment that have affordances
for them.
(Gibson, 1979/2015, p. 188)
How does orientation to the habitat and knowing how to reach places
come about for the perceiver? These questions also have to do with aware-
ness of features of the environment that are not immediately in sight, and
Gibson applies his analysis of dynamic occlusion to them.
Gibson refers to the portion of the environmental layout that can be
seen “from here” as a vista. It is an extended surface layout, and in most
cases it is cluttered with various features. The visible expanse that is the
vista is limited or foreshortened in various ways. Apart from the horizon
in extended, open regions, there are regularly large surfaces or a cluster
of features that act as visual screens obstructing the view in certain
directions. However, if one travels far enough within a vista, typically a
new vista gradually emerges from behind the visual obstruction. The
successive vista that was once not in sight, progressively comes into
sight, while the previous vista goes out of sight behind. “To go from one
place to another involves the opening up of the vista ahead [in the
200 Harry Heft
process of travel] and closing in of the vista behind” (Gibson,
1979/2015, p. 189).
That portion of the path of travel where the new vista comes into view,
and reciprocally where the previous vista becomes obscured from view, is
a transition. Each transition is a reversible event in the same sense that the
progressive occlusion of an object is. A prior vista can be brought back
into view by reversing the direction of travel. The flow of information gen-
erated through way-finding then is a reversible event, and, like any event,
it is perceived over time. With this conceptualization in hand, learning a
route involves developing an awareness of a particular sequence of trans-
itions linking vistas. Way-finding is a perception-action process of opening
up successive vistas in the course of travel; and in the course of traveling a
familiar route, the perceiver is aware of what lay ahead.
Gibson continues by offering the even more radical proposal to the
effect that in the process of becoming attuned to the extended event
structure that is a path of travel, the individual develops an awareness of
the overall layout of that region of the environment. “When the vistas
have been put in order through locomotion,” Gibson writes, “the invari-
ant structure of the house, the town, or the whole habitat will be appre-
hended. The hidden and the unhidden become one environment”
(p. 189). This assertion should be understood as a logical extension of
Gibson’s more familiar explanation of the perception of invariant struc-
ture in shape perception. To explain how one perceives a table top as
rectangular when from most observation points it appears to be trapezoi-
dal, he writes:
Although the changing angles and proportions of the set of trapezoidal

projections [viewed from a moving point of observation] are a fact, the
unchanging relations among the four angles over the set [i.e., what is
invariant] are another fact, equally important, and they uniquely
specify the rectangular surface.
(p. 66)
While the invariant relations in this case may specify the table top as seen
“from all sides at once,” the invariant structure of the house or town that
is revealed through a succession of transitions and vistas may likewise
result in an awareness of that region of the layout from “everywhere at
once.”
Seeing the world at a traveling point of observation, over a long

enough time for a sufficiently extended set of paths, begins to be per-
ceiving the world at all points of observation, as if one could be every-
where at once. Each object is seen from all sides, and each place is seen
as connected to its neighbor.
(pp. 187–188)
Although this last proposal has yet to be assessed empirically, various
facets of Gibson’s approach to way-finding have been. To summarize, after
viewing a film of a route, observers demonstrated an awareness of the par-
ticular sequence of transitions making up that route, and moreover, expo-
sure only to the sequence of transitions resulted in way-finding
performance that was comparable to that after having viewed a film of the
complete route (Heft, 1983). When the continuous nature of transitions
(events) along a route was disrupted by displaying the transitions as a
series of static views, way-finding performance marginally degraded (Heft,
1983). In another study, when observers of a videotape of a route were
instructed to mark those segments in the route that are important for sub-
sequent way-finding, they initially marked the transitions in the route, as
well as prominent features (landmarks). With increasing exposure to that
route, more extended (higher-order) transition segments were selectively
identified as important for way-finding while other landmark features
within transitions were omitted (Heft, 1983). These results indicate that
the route was perceived in terms of more extended units with experience.
Finally, participants indicated that they could “learn more” and that they
would like “to explore further” after viewing continuous displays of trans-
itions as compared to a series of discrete, static views over the course of
the same transitions, suggesting the prospective value of transitions (Heft
& Nasar, 2000).
Access to a Common Public Domain

In one of the concluding sections of his chapter on the occluding edge,
Gibson takes up the issue of the possibility of public knowledge: how can
different perceivers have access to a common domain of information? This
question is even a more crucial one for psychology than Gibson acknow-
ledges here in this book which concerns visual perception. Humans are an
intensely social species. Although some social exchange and the coordin-
ation of actions are characteristic among all primates, what distinguishes
human forms of life is the degree of richness in social exchanges that make
possible joint and collective action, some of which is grounded in shared
symbolic understanding. It has been well established that collective action
based on shared knowledge is a defining characteristic of our hominin
lineage beginning at least with Homo erectus (e.g., Donald, 1991; Sterelny,
2012). Collective actions making possible group migration to widely dis-
persed locations, joint hunting, and establishing semi-permanent settle-
ments were all in evidence among our species’ immediate ancestors; and
access to a shared domain of information would seem to be crucial in such
cases. These collaborations require shared awareness (intersubjectivity) of
mutual goals in a common environment. Further, participation in joint
community practices built on a foundation of reciprocal, interpersonal
processes is essential for human development (Rogoff, 1995). Shared
202 Harry Heft
awareness of a common environment would seem to be essential for living
in human communities.
And yet a shared, intersubjective domain of awareness runs counter to
psychology’s Cartesian lineage which takes knowing to be a solitary enter-
prise. How then are cooperation, joint action, and collective intentionality
possible—arguably, the bases of community life for this intensely social
species—if awareness is at all times personal and private? One currently
fashionable approach to such issues—the “theory of mind”—claims that
young children develop intellectual abilities necessary for inferring what
others are thinking, with such inferences allowing them to adopt the per-
spective of another. Leudar and Costall (2009) have criticized this
approach which in their view results only in “profoundly intellectualizing
social interactions” (p. 19), and in doing so keeps the possibilities of a
shared awareness just as remote as ever. Positing guesswork to be operat-
ing in all instances of joint action leaves us with a view of so many sepa-
rate consciousnesses struggling at all times to coordinate actions.
The challenge here is to formulate an approach that takes the individu-
al’s perspective on the environment as its starting point, while simultan-
eously allowing for possibilities of a shared awareness that sociocultural
processes seem to require. Gibson offers an entry point for just such an
alternative based on the ecological claims that perceptual information is
available in a publicly shared ambient array and that individuals pick up
information available in that array from a moving point of observation:-
It is true that there is a different optic array for each point of observa-
tion and that different observers must occupy different points at any
one time. But observers move, and the same path may be traveled by
any observer. If a set of observers move around, the same invariants
under transformations and occlusions will be available to all. To the
extent that the invariants are detected, all observers will perceive the
same world.
(1979/2015, pp. 190–191)
A foundation for shared awareness does not require mind-reading, of any

sort, but instead what is needed to bootstrap joint and collective processes
is access to a common ground for perception-action. Because “observers
move, and the same path may be traveled by any observer,” minds con-
verge in a shared environment. Although two individuals cannot simultan-
eously occupy the same observation point, perceiving the environment
from a moving point of observation does allow for the possibility that I
can eventually adopt an observation point that you once occupied. In prin-
ciple, then, knowers have a common ground of information that is avail-
able to be perceived (Reed, 1996a). This conceptualization offers a new
starting point for addressing a question that plagued William James––
“how two minds can know the same thing” (Heft, 2002)––and as a result,
provides ecological grounds for coming to understand the basis for
coordination of joint and collective action in communities.
Moreover, the discussion of dynamic occlusion revealed that perceptual
awareness goes beyond what is seen “now” in a single instant of time, and
that a perceiver can be aware of what will come into sight with action.
That awareness extends to what may be visible from your current observa-
tion point but not from mine. The claim, then, that perceiving is extended
“in time” has potentially significant implications for developing an account
of our social existence:
To adopt the point of view of another person is not an advanced

achievement of conceptual thought. It means, I can perceive surfaces
hidden at my point of view but unhidden at yours. This means, I can
perceive a surface that is behind another. And if so, we can both per-
ceive the same world.
(Gibson, 1979/2015, p. 191; original emphasis)
Conclusion
The observation that objects persist in awareness even when they are not
immediately in sight was not a new one when Kaplan and Gibson designed
their experiment on the occluding edge. What was new, however, was their
attention to the perceptual information that specified an object going out
of sight and yet persisting, and eventually the broader implications Gibson
drew from this analysis.
It is surely unconventional in mainstream perceptual theory––if not
thoroughly paradoxical––to claim that objects not immediately in sight
can be perceived. If perception is understood to be a process of converting
sensations into percepts, as standard approaches have it, then it follows
that an object that does not presently stimulate sensory receptors cannot
possibly be perceived. One of the problems with this stance is that it
assumes that the proper way to develop an account of perceiving is to
begin with what is taken to be the simplest units of analysis. Over a
century ago that strategy was reaffirmed for many by the discovery of
individual sensory receptors. As a result, functional considerations that
might have been brought to forefront in the study of perception in the
wake of evolutionary theory took a back seat to anatomical considera-
tions. Rather than getting clear about how animals function in the
environment, how it is that animals stay in touch with the affordances of
an extended environment over time, and how perceptual systems support
those functions, the aim of much research from the days of Helmholtz is
to explain how animals overcome the apparent limitations of their
biology. Given that history, it is ironic that a number of notable func-
tional discoveries at the biological level of analysis originated with atten-
tion to perceptual experience (phenomenology).
204 Harry Heft
Ecological psychology has benefited immeasurably from attention to
phenomenology, from the analysis of optic flow to studies of affordances
(Heft, 2003), and some of its most important contributions to perceptual
theory writ large stem from that work. References to the character of per-
ceptual experience recur throughout the corpus of Gibson’s writings (e.g.,
Gibson, 1958, 1969, 1979/2015; Gibson & Crooks, 1938). His appeal to
phenomenology is especially evident in the films he created to demonstrate
his discoveries and proposals (Gibson, 1955, 1968; Gibson & Kaushall,
1973).
Among the perceptual phenomena examined by Gibson, dynamic occlu-
sion has arguably received the least attention in the years following the
publication of The Ecological Approach to Visual Perception. Its sweeping
implications for perceptual theory and for social theory have barely been
realized.
Notes
1. For a demonstration of this effect, see Gibson’s film, “The Change from Visible
to Invisible: A Study of Optical Transitions,” available at: www.youtube.com/
watch?v=1qQLtIICXoE
2. Imagining can also be with reference to occurrences “out of time” (fantasy).
12 Looking with the Head and Eyes
John M. Franchak
Although The Ecological Approach to Visual Perception (Gibson,

1979/2015) is perhaps best known for the theories of affordances and
direct perception, Gibson’s theory of visual exploration with the eye-head-
body system was no less revolutionary. The evolution of his thought on
visual exploratory behavior can be traced through his major works. In
Perception of the Visual World, Gibson (1950a) considered the
consequences of head and body movements on visual experience, and
emphasized the role of the body in stabilizing vision. In The Senses
Considered as Perceptual Systems, Gibson (1966a) defined the visual
perception system as extending beyond the eyes to involve active
exploration with the eyes, head, and body. The culmination of these ideas
appears in The Ecological Approach to Visual Perception: Gibson argued
that the purpose of a whole-body visual system is for exploring one’s
surroundings. Gibson dissociated what people are asked to do in typical
laboratory studies, what he termed “looking at a page or a picture,” from
what people actually do in real life, that is, “looking around”:
The perceptual capacities of the organism do not lie in discrete ana-

tomical parts of the body but lie in systems with nested functions. Even
so, it might be argued, one surely looks with the eyes even if one does
not see with the eyes. But looking with the eyes alone is mere looking
at, not looking around. It is the scanning of an object, a page of print,
or a picture. One also looks with the head, not just the eyes, more
exactly with the head-eye system …
(p. 205, emphasis in original)
This characterization of visual exploration was at odds with the dominant

paradigm in vision science, in which the eyes scan restricted, two-
dimensional stimuli while the head and body remain still. Today, screen-
based tasks that present shapes, photographs, or movies to passive
observers are still far more common compared to natural tasks in which
active observers explore the environment with unfettered body movement.
Gibson dismissed studies of “looking at” for three reasons, each of which
206 John M. Franchak
has been influential in guiding contemporary research on naturalistic
looking with the eyes and head.
First, Gibson claimed that studies using such simplistic, two-dimensional
displays lead researchers to place too much emphasis on the influence of
stimulus features on gaze behavior: “It is certainly a fallacy to assume that
a saccadic movement is a response to a ‘stimulus’ on the periphery of the
retina that brings it to the fovea” (p. 212). Gibson rejected the notion of
stimulus-driven, involuntary eye movements in favor of a functional
approach in which eye movements support the perception of ecologically-
relevant aspects of the environment and facilitate action performance.
Indeed, contemporary work using both screen-based and natural tasks
have consistently found that observer’s tasks and goals are more influential
in determining where people look compared to the appearance of visual
targets (Tatler, Hayhoe, Land, & Ballard, 2011).
Second, Gibson emphasized that what we see is not the bounded and
fleeting visual field, “a patchwork of colors something like a picture” (p.
206), but the visual world, which is stable, unbounded and “consists of
familiar surfaces and objects one behind another” (p. 206). Importantly,
Gibson claimed that “the awareness of the observer’s own body in the
world is a part of the experience” (p. 207) of the visual world. An
observer’s experience of the real world is active: Visual exploration
supports interacting with the environment and, in turn, the observer’s
movements affect what is seen. In contrast, an observer’s experience of
looking at photographs or screen-based displays is mediated (see
Stoffregen, Chapter 15, and Blau, Chapter 16, in this volume). When
visually exploring an image, the observer indirectly perceives what is
represented but directly perceives the representational medium itself. The
observer is aware of being seated looking at an image, not of being present
in the visual world that is represented by the image or video. Moving the
head does not change the observer’s perspective within the photograph,
but merely changes where the photograph is within the field of view.
Third, Gibson claimed that studies of “looking at” give undue
prominence to eye movements, which are only one part of an integrated
perceptual system: “One sees the environment not with the eyes but with
the eyes-in-the-head-on-the-body-resting-on-the-ground” (p. 205). Studying
the nested movements that permit looking around is not possible with a
seated observer who looks at a computer screen. Naturalistic studies of
adults’ visual exploration show that eye, head, and body movements are
coordinated to pick up information in the environment (Land, 2004).
Moreover, infant studies show that developmental changes in the motor
skills underlying visual exploration fundamentally alter what infants see
(Franchak, Kretch, & Adolph, 2018; Kretch, Franchak, & Adolph, 2014).
In this chapter, I will review empirical work that has expanded on these
three main points. Each section will describe work with adults as well as
infant and child research to highlight how differences in the developing
Looking with the Head and Eyes 207
eye-head-body system are relevant for understanding visual exploration. I
will begin with a discussion of the methodological innovations that were
necessary to carry out such work.
Measuring Visual Exploration

Whereas research in affordances followed shortly after the 1979 publica-
tion of The Ecological Approach to Visual Perception (e.g., Warren, 1984),
the first studies of naturalistic visual exploration did not appear until the
late 1990s (Land & Furneaux, 1997; Land, Mennie, & Rusted, 1999).
One reason for the delay was the need for technology for studying looking
around rather than mere looking at—mobile eye tracking. Why does visual
exploration require specialized technology to study when other ecological
aspects of visual perception, such as affordances, do not? The relevant
behaviors for studying affordance perception are gross motor movements—
reaching for objects, stepping on risers, leaping over gaps—that can be
scored by human observers or recorded with a basic video camera. In con-
trast, accurately scoring where a participant is looking varies from
extremely difficult to utterly impossible depending on the complexity of
the environment and the required precision of the measurement when
relying on a basic third-person video recording. A few observational
studies have successfully scored gaze behavior in naturalistic (but con-
strained) tasks from a third-person perspective, such as when adults glance
down while descending stairs (Rosenbaum, 2010).
Because the eyes are small and move extremely quickly—up to 700°/s
(Land, 2006)—high-speed eye tracking devices with specialized cameras
are required to record their movements, and thus to determine gaze direc-
tion. Various eye tracking devices have been created dating back to the
start of the 20th century; however, most systems—which I will refer to as
stationary eye trackers—require a seated observer, a fixed display, and
(often) immobilization of the head (for a review, see Tatler & Land, 2015).
Gibson noted the contribution of high speed stationary eye trackers in dis-
covering microsaccades, extremely quick movements of the eye during fix-
ations. However, Gibson recognized that stationary eye movement
recording was insufficient to study whole-body visual exploration.
Although modern stationary systems can record eye movements with
incredible spatial accuracy (0.02°–0.5°) and temporal resolution
(500–2000 Hz), the observer is required to sit in place and eye movements
can only be measured within the boundaries of a computer display.
Studying whole-body visual exploration requires mobile eye trackers
that can track eye movements in an observer who is free to move the head
and body. The first mobile eye trackers were developed in the 1940s to the
1960s but did not allow for the study of natural behavior because the large
devices occluded parts of the observer’s view and were too cumbersome to
allow natural movement (Tatler & Land, 2015). In the early 1990s, Land
(1992) developed a portable, head-mounted eye tracker that simultan-
eously recorded videos of the observer’s eye and the first-person field of
view (Figure 12.1).
Eye position is calibrated with gaze locations within the first-person
(i.e., head-centered) field of view video. The field of view video records the
changes in viewpoint that result from shifts of head and body position;
thus, mobile eye trackers can truly measure looking around in natural
tasks. Land and his colleagues used this new technique to study visual
exploration during driving (Land, 1992; Land & Lee, 1994), playing
sports (Land & Furneaux, 1997), and while completing everyday activities,
such as making a cup of tea (Land et al., 1999).
Technological advances have led to several improvements in mobile eye
tracking. Whereas early mobile eye tracking required manual coding of eye
position for every video frame, later systems automate this process using
computer vision algorithms to detect the eye (D. Li, Babcock, & Parkhurst,
2006). Although the spatial (0.5°–2°) and temporal (30–120 Hz) resolution
of mobile eye trackers is modest compared to stationary eye trackers, it is
sufficient for most natural tasks. Miniaturization of video and electronics
components continues to make eye tracking devices more portable and less
obtrusive to conduct studies “in the wild” as opposed to in the lab (Figure
12.2a). Moreover, the development of lightweight mobile eye trackers
(Figure 12.2b) allowed for the first recording of real-world visual explora-
tion in freely-moving infants (Franchak, Kretch, Soska, & Adolph, 2011;
Franchak, Kretch, Soska, Babcock, & Adolph, 2010) and children (Fran-
chak & Adolph, 2010).
Integration of mobile eye tracking with other technologies has opened
up new avenues of research. Simultaneous eye and motion tracking allows
;ĂͿ ;ďͿ
Figure 12.1 (a) Mobile eye tracker devised by Land (1992). (b) Simultaneous eye
and field of view recording for tracking eye gaze.
Source: From Land (1992). Reprinted by permission from Springer Nature.
;ĂͿ ;ďͿ
Figure 12.2 (a) Lightweight, mobile adult eye tracker. (b) Mobile infant eye
tracker.
Source: (a) Photo courtesy of Jason Babcock, (b) Photo courtesy of Chaun Luo.
researchers to precisely measure how eye, head, hand, and body move-
ments are coordinated in everyday tasks. For example, Franchak and Yu
(2015) studied infants’ and caregivers’ naturalistic play to measure eye and
head alignment to toys while reaching. Adults adjusted the speed of their
reaches depending on visual alignment—when they aligned their eyes
directly on toys they reached more quickly but slowed their reaches when
toys were in the periphery. In contrast, infants did not systematically
coordinate eye movements with reaching speed. Matthis and colleagues
(Matthis, Yates, & Hayhoe, 2018) combined mobile eye tracking with full-
body inertial sensing to examine visual guidance of locomotion while
hiking outdoors and found that walkers altered their gaze allocation to
account for the different task demands associated with walking over
different terrains (e.g., rocky trail vs uniform dirt path).
Scoring where observers look is challenging for mobile eye tracking
research compared with stationary eye tracking research. With stationary
eye trackers, the locations of gaze targets in a stimulus are the same for
every participant. For example, calculating how often participants look at
a face in a photograph means defining which pixels comprise the face, and
then calculating how frequently participants’ gaze coordinates fall on those
pixels. In contrast, in mobile eye tracking the locations of gaze targets in
the field of view are unique to each participant, depending on how each
participant chose to orient the field of view by moving the body and head
(Franchak, 2017). For example, scoring how often infants look at care-
givers’ faces in a naturalistic interaction means scoring where faces are in
each infant’s field of view at every moment—each infant will have their
caregiver’s face in view in different locations at different times. Con-
sequently, researchers manually score where each individual participant
looked frame-by-frame. The time-intensive scoring means that sample sizes
in mobile eye tracking studies are typically smaller than in comparable sta-
tionary eye tracking studies and might also contribute to why the size of
the mobile eye tracking literature is still modest.
One promising solution is automated computer vision detection of
targets of interest. For example, researchers have developed systems to
detect where faces (Frank, Simmons, Yurovsky, & Pusiol, 2013), hands
(Bambach, Franchak, Crandall, & Yu, 2014), and objects (Yu & Smith,
2013) are located in the field of view for each participant to automatically
score looking behavior. However, accuracy varies and is typically worse
compared with human coding because detection depends on visual clutter
in the scene and the discriminability of targets. Another solution is to use
eye trackers embedded in virtual reality headgears; manual coding of
looking is unnecessary because the location of gaze targets must be known
to render the virtual environment (Diaz, Cooper, Kit, & Hayhoe, 2013).
Yet another promising method is linking the mobile eye tracking reference
frame to that of a motion capture system to automatically detect gaze with
respect to body movements (Matthis et al., 2018).
Task-Driven Visual Exploration

Gibson dismissed the idea that eye movements are directed toward targets
based on stimulus features. Instead, Gibson stressed the importance of
visual exploration for supporting the observer’s behavioral agenda.
Moving the eyes and head to look around is an exploratory behavior that
should depend on what information is relevant to the observer’s task.
Indeed, early studies by Buswell and Yarbus demonstrated that different
instructions when looking at artwork influenced how observers directed
their eye movements in both space and time (Buswell, 1935; Yarbus,
1967).
Despite early work highlighting the role of observers’ tasks and goals,
stimulus-driven theories of visual attention have been prominent in
modeling eye movements in screen-based tasks (Borji & Itti, 2013; Itti,
Koch, & Niebur, 1998; Itti & Koch, 2001). Such bottom-up saliency
models predict that observers make eye movements to targets that “pop
out” from the surrounding scene based on their appearance—brightness,
color, and movement. One reason that bottom-up approaches have con-
tinued to thrive is that many screen-based studies fail to provide the
observer with a clearly defined task. Typically, observers are instructed to
simply view the photograph or video. Assigning a task, such as identifying
the location where a photograph was taken, reduces the influence of sali-
ency on eye movements (Smith & Mital, 2013).
Task influences on visual exploration are even more striking when
observers are free to engage in everyday actions that involve movement.
For example, when making a cup of tea (Land et al., 1999) or a peanut
butter and jelly sandwich (Hayhoe, Shrivastava, Mruczek, & Pelz, 2003),
most eye movements are directed toward task-relevant objects (e.g., the tea
kettle or the jelly jar). Irrelevant objects and the “background” of a scene
are rarely looked at even when salient. Even when visually salient, irrele-
vant objects are inserted in the background, they are rarely fixated
(Rothkopf, Ballard, & Hayhoe, 2007). Instead, varying instructions about
which tasks to treat as obstacles and which to approach altered looking
behavior. Even in a real-world search task—where target “pop out” should
be more likely to drive eye movements—bottom-up factors had little influ-
ence unless observers were explicitly given instructions to expect that the
search target would look different compared with distractors (Foulsham,
Chapman, Nasiopoulos, & Kingstone, 2014).
The pickup of visual information about task-relevant targets shows not
just spatial but also temporal coordination with actions. The eyes shift to
objects just before the hand reaches, such as looking at a knife before
reaching toward it, or the eyes may monitor ongoing events, such as when
watching a tea kettle as it is being filled with water (Hayhoe et al., 2003;
Land et al., 1999). Less frequently, observers may break from looking at
the current, in-use object to “look ahead” at an object that will be relevant
a few seconds in future, such as glancing at the soap dispenser while
approaching the sink (Pelz & Canosa, 2001).
Task-specificity and timeliness of visual exploration are also evident in
infants’ naturalistic behavior. While infants and caregivers played together
with a set of toys, infants most often look at toys they are playing with and
spend less time looking at caregivers (Franchak et al., 2018; Yu & Smith,
2013). Infants coordinate the timing of eye and hand movements during
natural reaching: Infants keep their eyes fixed on toys as they guide reach-
ing movements and look away either just before or just after the hand
makes contact (Franchak et al., 2011). From 9–24 months of age, infants
increasingly orient their heads to center toys in view, possibly because
looking with aligned eyes and head facilitates visual guidance of reaching
and object manipulation (Franchak, Smith, & Yu, under review).
Awareness of the Self and Others

The second problem in laboratory tasks that Gibson recognized is the
mediated experience of looking at a scene in a photograph (in modern
tasks, a computer screen). Even when looking at a “natural image”, such
as a photograph of an outdoor scene, the observer’s experience is not of
being present in that scene but of being seated in a chair in a laboratory
room looking at a photographic representation. Gibson stressed that the
visual consequences of observer movement are important for self-
perception. However, when viewing a photograph in a laboratory study,
moving the head changes the photograph’s location in the observer’s visual
field rather than shifting the observer’s viewpoint within the photograph.
Additionally, when perception is mediated, the observer cannot act on a
scene but must passively view it.
Indeed, agency—having the ability to act on the environment—has con-
sequences on how people visually explore. Foulsham and colleagues (2011)
tested this directly by comparing active observers and passive observers.
Active observers’ gaze was recorded with a mobile eye tracker while they
walked across campus to get a cup of coffee. Passive observers sat in the
lab and watched scenes recorded from the active observers’ point of view
displayed on a screen. Whereas active observers had agency—they could
choose how to move their bodies and heads to interact with and explore
their surroundings—passive observers could only see the scenes. Active and
passive observers’ eye movements differed: Active observers clustered their
gaze tightly within the head-centered field of view, presumably because
they were able to use their heads to center targets of interest. In contrast,
passive observers who could only move their eyes distributed gaze more
broadly around the image. Active observers more often looked at or
slightly below the horizon and toward the path as they walked, but passive
observers who did not need to guide locomotion looked less often at areas
in the scene that were relevant for walking. In a similar comparison
between active and passive observers that used a different task (making a
cup of tea), both groups of observers looked more at task-relevant objects
compared to task-irrelevant objects (Tatler et al., 2013). However, active
observers looked even more at task-relevant objects compared to passive
observers because they needed to manipulate those objects to make the cup
of tea. A second condition that removed the active observer’s need to
manipulate objects (walking through the room to memorize the objects)
eliminated the difference; both active and passive observers looked equally
at task-relevant objects.
Social looking, that is, looking at the hands, faces, and bodies of people,
differs between natural and screen-based tasks because looking at people
who can engage in social interaction versus images of people who cannot
interact with the observer are fundamentally different experiences. When
watching videos of people, adult observers frequently look at faces, specifi-
cally at the eyes to monitor others’ gaze (Birmingham, Bischof, & King-
stone, 2008). However, looking at a video recording of a stranger is
different than looking at a stranger in real life because the real-life person
can recognize eye contact as a social gesture that signals a desire to
interact. Consequently, observers are less likely to look at a real person in
a laboratory room compared to a video of another person displayed on a
computer screen (Laidlaw, Foulsham, Kuhn, & Kingstone, 2011). In the
study comparing active and passive observers walking to get a cup of
coffee, passive observers were more likely than the active observers to look
at nearby people in the scene (Foulsham et al., 2011).
Similar parallels are seen when comparing screen-based versus natural
tasks of infant social looking. Face-looking increases over the first year of
life, with 12-month-old infants spending as much as 50% of the time
looking at faces while watching videos (Franchak, Heeger, Hasson, &
Adolph, 2016; Frank, Vul, & Johnson, 2009). However, in real-life loco-
motor play, infants spend only 4% of the time looking at caregivers’ faces
and 15% of the time looking at caregivers’ hands and bodies but spend
38% of the time looking at toys (Franchak et al., 2018). Even when care-
givers spoke to infants, infants responded only 8% of the time by looking
at the caregiver’s face (Franchak et al., 2011). When infants and caregivers
sat across from one another at a table, infants spent 12% looking at care-
givers’ faces but 60% of the time looking at toys (Yu & Smith, 2013).
Low rates of face-looking are likely due to two factors. First, adult’s
faces are often outside infants’ field of view, especially when infants tilt
their heads down to look at toys (Franchak et al., under review), unlike in
screen-based tasks in which faces are placed in easily viewed locations in a
display. Indeed, when playing on the floor, infants look more often at
sitting caregivers’ faces compared with standing caregivers’ faces (Franchak
et al., 2011; Franchak et al., 2018). Second, infants’ preoccupation with
guiding locomotion and playing with toys means that faces are competing
with other task-relevant locations (toys, furniture, distant locations). No
such competition exists in screen-based tasks when infants passively view
photographs or videos. A real-world study of infants being carried through
a hallway created a passive viewing situation more similar to a screen-
based tasks, and found high rates of infants looking at people and faces
(significantly more than caregivers whose eye movements were also
recorded) (Kretch & Adolph, 2015).
A Nested Perceptual System for Looking Around

Gibson emphasized that a nested visual system—the eyes within the head
within the body—is coordinated to visually explore one’s surroundings.
Eye, head, and body movements serve two complementary functions in
visual exploration: (1) shifting gaze from one location in the world to
another; and (2) stabilizing gaze on a desired target while compensating
for movement of the target, the observer, or both.
Saccades and head movements are tightly coordinated to shift gaze from
one location in the environment to another. Saccades are rapid movements
of the eyes to shift the fovea—the location of greatest visual acuity—from
one place to another within the head-centered field of view. The eyes can
rotate approximately 50° along the horizontal axis within the head to look
at extremely eccentric targets (Land, 2006). However, such extreme eye
rotations are seldom observed in natural tasks. Both infants and adults
keep the eyes relatively centered in their orbits to look where the head is
pointed (Bambach et al., 2014; Bambach, Smith, Crandall, & Yu, 2016;
Einhauser et al., 2007; Foulsham et al., 2011). That observers rarely make
extreme eye movements indicates that much of visual exploration is
accomplished with head movements. Indeed, laboratory studies find that
head movements accompany eye movements to prevent extreme rotations
of the eye, even when the eye can fixate the target without moving the head
(Guitton & Volle, 1987; Oomen, Smith, & Stahl, 2004). One reason that
observers may choose to minimize eye rotation is that it is efficient for
visual exploration (Tatler, 2007). Maximum rotation of the eyes in a direc-
tion precludes them from rotating further in that direction, which could be
detrimental when tracking targets (especially ones that move unpredict-
ably). Continual, compensatory head movements that re-center the eyes
within the head mean that the eyes are free to quickly move in any direc-
tion at a moment’s notice.
Eye and head movements are also coordinated to stabilize gaze. The
vestibular-ocular reflex (VOR) is a rapid counter-rotation of the eyes that
compensates for rotations of the head. VOR happens concurrently with
head movements; feed-forward information about future head movement
can inform when and how to counter-rotate the eye (Land, 2004). Gibson
noted the importance of VOR, “It compensates for the turning of the head.
Thus, it is a nonmotion of the eyes relative to the environment, a posture,
like fixation” (p. 210, emphasis in original). Just as body postures like
sitting and standing involve compensatory movements to keep the body
stable on a support surface, compensatory eye movements keep gaze stable
with respect to the environment and keep the observer from experiencing
the visual world “swinging” around as the head and body move. Gibson
recognized that the term “reflex” is inaccurate because compensatory eye
movements are in fact voluntary. VOR is automatic but it can be sus-
pended, such as when the eyes and head move in synchrony to make a
large (>50°) shift of gaze (Guitton & Volle, 1987).
Studies of whole-body visual exploration show that eye, head, and body
movements are tightly coordinated to achieve both gaze shifts and stabili-
zation from moment to moment, but coordination patterns are flexibly
assembled to meet the demands of various tasks. Land (2004) measured
the timing between eye, head, and trunk rotations during the tea-making
task. Figure 12.3 shows a typical 90° shift of gaze from one location in the
world to another. The eyes, head, and trunk all begin to move together in
step 1. At step 2, the eyes have stopped moving relative to the head while
the head and trunk continue to rotate. At step 3, the eyes have reached the
desired target, so they counter-rotate to compensate for continued rota-
tions of the trunk and head. At step 4, the head has reached its desired
position, so it must counter-rotate along with the eyes to stabilize gaze on
the target while the trunk finishes its rotation at step 5.
Other studies have investigated coordination between eyes, head, and
hands. In a block-copying task that required looking back and forth
between a model and a workspace, Pelz and colleagues (Pelz, Hayhoe, &
Loeber, 2001) discovered task-dependent synergies between the timing of
eye, head, and hand movements as participants repetitively checked the
d d d d d
ϭ Ϯ ϯ ϰ ϱ
Figure 12.3 Different stages of eye, head, and body rotations during a 90° shift of
gaze from one location in the world to the target “T.”
Source: From Land (2004). Reprinted by permission from Springer Nature.
model while manipulating objects in the workspace. Unlike laboratory

investigations of eye-head coupling that only found head movements
accompanying large shifts of gaze (20°), in the block-copying task head
rotation varied widely during gaze shifts (from 1–10°) meaning that eye-in-
head position also varied. Other work using a similar task found that head
movements were sometimes dissociated from gaze direction to orient
toward future reaching targets that the eyes had not yet fixated (Smeets,
Hayhoe, & Ballard, 1996).
Moving the head is important for minimizing extreme eye movements,
however, head movements themselves should also be minimized to make
visual exploration efficient because head movements are effortful. Obser-
vers must choose whether to move the head to align the eyes toward a
target or whether to view a target in the periphery without moving the eyes
and head. During naturalistic locomotion, observers avoid unnecessary
head movements because moving the head to look down at the ground
surface is energetically costly and shifts the visual field away from the
destination ahead. When walking on a flat, uniform surface, observers look
down less often compared to when traversing a more difficult, uneven
surface (‘t Hart & Einhauser, 2012). When observers do look down
toward the terrain, they do so in a way to maximize the efficiency of gait
planning (Matthis et al., 2018). The difficulty of the terrain informs visual-
motor planning; walkers can manage to look longer ahead (four steps in
advance) when traversing easy terrain, but on difficult terrain observers
look closer to the moment of stepping (two steps ahead). Watching the
current step or looking only one step ahead is seldom done because cor-
rections at that stage disrupt the gait cycle. Comparing children aged
4–8-years-old to adults in an obstacle navigation task revealed that chil-
dren timed obstacle fixations similarly to adults—both groups fixated obs-
tacles two or three steps ahead (Franchak & Adolph, 2010). Similar scaling
of obstacle fixations to steps suggest that children, like adults, avoid
looking down at their feet while traversing obstacles and prefer to look
ahead. Even 14-month-old infants—while walking over bridges spanning a
precipice—only glanced at bridges at the start of each trial and seldom
looked down at the bridge while traversing it (Kretch & Adolph, 2017).
Decisions about whether to look at targets in the environment may
depend on the interaction between motor and informational costs of
looking. In laboratory investigations of obstacle navigation, adult particip-
ants fixated obstacles in advance 78% of the time while repeatedly walking
up to and over a single obstacle trial after trial (Patla & Vickers, 1997).
However, when obstacle navigation was embedded in another task—
searching for objects around a room in a scavenger hunt—adults fixated
obstacles in advance only 32% of the time (Franchak & Adolph, 2010).
Turning the head down to look at obstacles would interfere with searching
around the room, so participants chose instead to rely on peripheral
information about obstacles. Children fixated obstacles significantly more
often (59% of the time), possibly because children’s shorter stature meant
that obstacles on the ground were easier to view without tilting the head
down. Figures 12.4a and 12.4b show the visual fields for a typical 8-year-
old girl compared to a 20-year-old woman. Because of the difference in
height, the lower bound of the field of view is closer to the feet for the
child compared to the adult, meaning that the child has greater oppor-
tunity to view obstacles in her or his path.
Just as differences in height affect visual access to the environment,
different body postures affect how infants see the world. Kretch and col-
leagues (Kretch et al., 2014) measured the visual fields of infants while
crawling versus walking (Figures 12.4c and 12.4d). The differences in
viewpoint are striking: Walking infants can see more of the world ahead,
whereas crawlers see more of the floor. A quarter of the time, crawling
infants see only the floor and nothing of the world ahead. These postural
effects on visual access to the world have functional consequences. When
navigating obstacles, 14-month-old infants fixated 72% of obstacles while
walking compared with 90% of obstacles while crawling (Franchak et al.,
2011), presumably because obstacles are more likely to be in view while
crawling than walking. When caregivers showed infants an attractive toy,
walking infants were more likely than crawling infants to look at the toy,
suggesting that crawling infants were unable to compensate for their
restricted view (Kretch et al., 2014).
Social looking also depends on infants’ body posture. Infants are less
likely to look at caregivers’ faces while crawling compared to while sitting
or standing upright (Franchak et al., 2018). The effect is specific to looking
at caregivers’ faces; looks to caregivers’ bodies did not differ by posture
because bodies were accessible, even in crawlers’ view. Changes in infants’
posture over development might alter what infants see in everyday life. In-
home studies using wearable head cameras find that the presence of faces
;ĂͿŚŝůĚ ;ďͿĚƵůƚ
;ĐͿƌĂǁůŝŶŐŝŶĨĂŶƚ
;ĚͿtĂůŬŝŶŐŝŶĨĂŶƚ
Figure 12.4 Field of view for an average-height 8-year-old girl (a) compared

with an average-height 20-year-old woman (b). The obstacle (gray
rectangle) is in the field of view for the shorter child but not for
the taller adult. Field of view measurements for a crawling infant
(c) compared to a walking infant (d) show that crawlers see more
of the ground and less of the world ahead compared with
walkers.
Source: A and B from Franchak and Adolph (2010). Adapted with permission of Elsevier.
C and D from Kretch et al. (2014). Adapted with permission from John Wiley & Sons.
in infants’ field of view declines over the first two years of life (Jayaraman,
Fausey, & Smith, 2015). Indeed, young infants spend nearly 50% of the
time held up off the ground (Franchak, 2019), which provides a good
vantage point for face-looking (Kretch & Adolph, 2015). As infants learn
to sit, crawl, and walk, they spend more time down on the floor in posi-
tions that are less conducive to face looking.
Differences between postures place constraints on what infants can see,
but within each posture infants can choose how to orient their heads to
determine what is in view. When sitting across from caregivers and playing
with toys, 12-month-old infants try to keep both toys and faces in view
simultaneously, which results in neither type of target being well-centered
in view (Franchak et al., under review). By 24 months, infants center toys
in view at the expense of caregivers’ faces. This is true of face and toy loca-
tions both while those targets are fixated as well as when they are not;
Figure 12.5 shows the differences in the locations of faces and toys in
head-centered field of view of 12- and 24-month-old infants during fixa-
tions. Biasing toys in view might help infants control manual actions with
toys and deal with unpredictable movement of toys in the field of view.
These effects may be task dependent. Whereas 12-month-olds did not bias
their view to favor toys over faces in the seated play task, they did bias
toys over faces in a locomotor play task (Luo & Franchak, in preparation).
Possibly, viewpoint bias depends on the relative locations of different
targets in the environment.
ϭϮŵŽŶƚŚƐ ϮϰŵŽŶƚŚƐ
&ĂĐĞƐ
dŽǇƐ
Figure 12.5 Heat maps showing the frequency of face (top row) and toy (bottom
row) locations within the head-centered field of view for 12- and
24-month-old infants during fixations. White indicates low frequency
while increasingly dark shades of gray indicate higher frequency.
Dashed black circles indicate a 15° radius around the center of view.
For 12-month-olds, neither toys nor faces were often within the middle
of the field of view. For 24-month-olds, toys were frequently in the
center of view but faces were not.
Looking Ahead
To summarize, Gibson argued that studies of looking at photographs in
the laboratory cannot inform on the real-life behavior of looking around
the world. The advent of mobile eye tracking allowed researchers to collect
empirical data that support this claim. First, naturalistic studies of eye
tracking across a variety of tasks show that people use the eyes and head
to select visual targets that support their goals and actions within a task;
physical appearance of stimuli bears little on where people look. Second,
awareness of the self in the visual world—as an agent who actively engages
with the environment and other social agents—leads to different decisions
about where to look in natural tasks compared with mediated studies of
looking at images or videos by passive observers. Third, the coordination
of eyes, head, and body to select where to look is flexible, task-specific,
and a ubiquitous part of natural behavior that cannot be studied in screen-
based tasks. What adults see depends on how they choose to orient the
nested visual system; what infants see depends on how they are able to
orient the nested visual system. Efficient visual exploration depends on
deciding how to coordinate visual exploration amidst the informational
and motor demands of other ongoing actions. It is important to note that
the studies cited in each of the above sections could easily have been
applied to support any of these three points; nearly every study of
naturalistic visual exploration shows the importance of task, self, and
whole-body exploration.
Given these limitations of screen-based tasks, which have been
consistently highlighted by 20 years of mobile eye-tracking research, it is
important to consider the role that screen-based tasks should play in visual
perception research as Gibson did 40 years ago. The advantage of screen-
based tasks is the ability to show multiple observers the identical stimulus
and to be able to manipulate the qualities of those stimuli. Psychophysics
and physiological studies of vision would be impossible without the level
of control provided by screen-based tasks. However, this comes at the
expense of generalization because observers cannot move their bodies or
interact with their surroundings. Undoubtedly, the results of screen-based
tasks generalize to sedentary activities like watching television and, to a
lesser extent, using a computer (even interacting with computer software
with a mouse or touchscreen is an active task). Beyond this, generalization
is not guaranteed. At the broadest level of analysis, screen-based studies
are concordant with natural tasks in showing that task factors outweigh
target appearance in influencing where people look (e.g., Smith & Mital,
2013). But at a more detailed level, comparisons between active and
passive observers find differences in visual exploration even when watching
the same “stimulus” (e.g., Foulsham et al., 2011).
More work is needed to test which aspects of screen-based tasks
generalize to natural tasks. Eye tracking integrated with virtual reality
provides a means by which researchers can study whole-body visual
exploration while maintaining similar experimental control (and the ability
to replicate) screen-based tasks in three dimensions. For example, visual
search time is comparable in both 2D (screen) and 3D (virtual reality)
search tasks, but those in 3D learn spatial associations better and use fewer
fixations to complete the task (C.-L. Li, Pilar Aivar, Kit, Tong, & Hayhoe,
2016).
However, the laboratory and naturalistic paradigms are about more
than methodological differences. Widespread acceptance of the screen-
based paradigm is rooted in the theoretical commitment to what Gibson
termed the “sequence theory” of visual perception. In this view, visual
perception is based on a series of discrete retinal images, so studying
observers’ perception of images on a screen is an appropriate
methodological choice. But Gibson argues that this theory cannot account
for our perception of a visual world that is persistent and stable in both
space and time:
The error was to suppose in the first place that perception of the
environment is based on a sequence of discrete images. If it is based
instead on invariance in a flow of stimulation, the problem of
integration does not arise. There is no need to unify or combine
different pictures if the scene is in the sequence, specified by the
invariant structure that underlies the samples of the ambient array.
The problem of explaining the experience of … what I would now call
the surrounding environment is a false problem. The retinal image is
bounded, to be sure, and the foveal image has even smaller bounds,
but the ambient array is unbounded.
(pp. 221–222, emphasis in original)
In this view, attempting to study visual perception by providing the retina

with a stimulus image is misguided because awareness of the world comes
through the observers’ active exploration of the environment over time.
This exploration—scanning with the eyes within the head to bring the
fovea across different areas on the environment or moving the head to
bring objects in and out of the visual field—is a series of complex
transformations of optical information. Invariance in such transformations,
Gibson claims, carries the information for the stable, unbounded visual
world that we do perceive.
Looking ahead, we must test how observers’ active visual exploration
supports perception of a stable visual world. Rather than asking how an
image or video compels the observer to look, we should ask how the
observer actively looks with the eyes, body, and head to bring objects in
and out of view and how observers choose to shift the visual field to
maintain contact with their surroundings. Maintaining awareness of the
visual world can be thought of as the most basic “task” of visual
exploration. The coordination of eyes and head for this “super-ordinate
task” might provide the basis from which observers adapt visual
exploration to fit the many specific “sub-tasks” of everyday life (e.g.,
domestic tasks, social interactions, locomotion) that have already been
studied.
13 James Gibson’s Ecological
Approach to Locomotion and
Manipulation
Development and Changing
Affordances
Karen E. Adolph, Justine E. Hoch, and
Ori Ossmy
Action Is Integral to Perception

One of James Gibson’s profound insights was that perception and action
are functionally linked. Indeed, in a book purported to be about percep-
tion, every chapter of The Ecological Approach to Visual Perception is
permeated with references to action.
Perception-Action Reciprocity
In Gibson’s (1979/2015) view, perception and action are interdependent:
“We must perceive in order to move, but we must also move in order to
perceive” (p. 213). The primary function of perception is to guide action
adaptively. Perceiving possibilities for action (or “affordances” in Gibson’s
terminology) requires humans and other animals to detect relations
between the self and the world (see Wagman, Chapter 8, in this volume).
Reciprocally, motor actions generate perceptual information. Exploratory
movements such as looking with the head and eyes or feeling with hands,
whiskers, or antennae are intended to “forage” for perceptual information
(see Franchak, Chapter 12, in this volume). Performatory actions (e.g.,
catching a ball), and spontaneous movements (e.g., fidgeting) are not
intended to generate perceptual information, but do so nonetheless.
In everyday activities, perception and action are seamlessly intertwined.
Tiny swaying movements while sitting or standing yield information about
postural stability and body position relative to the environment. Locomo-
tion generates information about the body’s dimensions, abilities, and loca-
tion relative to the layout of surfaces. Grasping an object produces
information about the acting limb relative to the acted-on object. More-
over, the coupling between perception and action creates an ongoing flow
of information, blurring the boundary between remembering and planning
Gibson’s Ecological Approach to Locomotion 223
(see Thomas, Riley, & Wagman, Chapter 14, this volume). Every move-
ment creates perceptual information about its consequences and about
what to do next. Thus, action ties perception to a history of the recent past
and provides a roadmap into the near future.
Basic Actions
Gibson highlighted four types of basic actions: (1) postural actions,
(2) exploratory actions, (3) locomotion, and (4) manipulation. Each type
plays a prominent role in his theory. Gibson devoted many pages, and in
the case of visual exploration and locomotion/manipulation, whole chap-
ters to their discussion. In Gibson’s treatment, the four types of basic
actions are interconnected. Posture and exploration, for example, support
and guide locomotion and manipulation.
Postural actions are the foundation for every other kind of action (see
Reed, 1982b). Posture provides a stable base for moving the torso, head,
and eyes during visual exploration and for moving the limbs during loco-
motion and manipulation. Postural actions also keep the animal oriented
to gravity and the medium (air, water, or ground) in or on which it moves.
Exploratory actions are movements intended to generate information
for perceptual systems (E. J. Gibson & Pick, 2000). Exploration can guide
upcoming actions, aid knowledge acquisition, or support playful activity
(E. J. Gibson, 1988). Turning the head creates motion parallax and optic
flow, and brings new parts of the self and environment into view. Wielding
an object or palpating a surface creates torques at the joints and deforma-
tion and stretching of flesh and skin, thereby revealing information about
object and surface properties.
Locomotion involves moving the whole body through the environment.
It can be accomplished with a tremendous variety of means. Trotting dogs
create forces against the ground with their limbs; swimming fish sweep
their tails back and forth through the water; and flying insects beat their
wings against the air (Dickinson, Farley, Full, Koehl, Kram, & Lehman,
2000). Even during a single observation, animals exhibit a variety of means
of locomotion (Adolph & Berger, 2015). Human infants, for example,
produce multiple patterns of interlimb coordination when crawling.
Manipulation involves acting on objects and surfaces in the environ-
ment. Humans normally do it with their hands, but when their hands are
occupied, adults can use a hip to bump open a door or their neck and chin
to hold a package. People without hands can learn to use their feet to
thread a needle, play the piano, light a cigarette, or make a sandwich.
Non-human animals use necks, beaks, and mouths to manipulate and
transport objects.
In contrast to typical approaches (e.g., Schmidt & Lee, 2011), Gibson’s
ecological approach to locomotion and manipulation is not merely about
the well-studied human activities of walking and reaching. Instead,
224 Karen E. Adolph et al.
ibson’s theory is broad enough to cover a wide range of animals with a
G
wide range of perception-action systems performing the wide range of
behaviors that the species evolved to perform. His theory is robust enough
to explain perceptual control of action in a wide range of environments. It
applies to natural, uncultivated environments but also generalizes to the
designed world of manufactured artifacts and built environments—actions
like sitting on chairs, using binoculars, driving cars, landing planes, and
using tools such as hammers (see Pagano & Day, Chapter 3, in this
volume). However, because Gibson’s primary focus was on perceptual
control of functional action, his approach is less suited to explain many
popular topics in human perception research such as visual processing,
color perception, covert attention, face perception, and optical illusions.
Indeed, many classic and current perceptual phenomena—especially those
arising from artificial laboratory tasks—are not well handled by his theory.
Exploration, Development, and Changing Affordances

Despite Gibson’s broad treatment of perceptual guidance of action, a critical
factor is conspicuously absent from The Ecological Approach—development.
Contributions of the Susan Linn Sage Professor

The omission of development is striking because Gibson’s wife and valued
colleague, Eleanor Gibson, was a renowned developmental scientist,
member of the National Academy of Sciences, fellow of the American
Academy of Arts and Sciences, and author of Principles of Perceptual
Learning and Development (1969). Years before The Ecological Approach
was published, Eleanor Gibson was considered a world expert in percep-
tual learning and development (Adolph & Kretch, 2015). As J. Gibson
wrote in the preface of his book:
Above all there is the Susan Linn Sage Professor of Psychology at

Cornell who worked very hard on this book, even if she did not write
it. She is married to me, and we share responsibility for important
decisions. Any errors in this book that remain are her fault as much
as mine.
(p. xii)
By 1979, when The Ecological Approach was published, many methods

were available to study perceptual and motor development in infants. In
fact, two decades before its publication, Eleanor Gibson devised one of the
most famous paradigms in developmental psychology—an illusory drop-
off dubbed a “visual cliff ”—to test depth perception and affordance per-
ception in infant humans and other animals. The apparatus is a
glass-covered table with a patterned surface flush beneath the glass on the
“shallow” side and far beneath the glass on the “deep” side (Adolph &
Kretch, 2012; E. J. Gibson, 1991; E. J. Gibson & Walk, 1960). Although
the visual cliff is discussed in several chapters of The Ecological Approach,
development is not.
Development and Changing Affordances

Albeit absent from The Ecological Approach, development is integral to
Gibson’s proposal that perception functions to detect affordances for action.
Animals’ bodies and environments develop, and these changes alter the
body-environment relations that make particular actions possible or not.
Gibson recognized that affordances for locomotion and manipulation differ
across species. Lightweight water bugs with hydrophobic legs can walk on
water, but heavy animals or insects without hydrophobic legs cannot.
However, affordances also differ depending on each animal’s level of devel-
opment. An 18-cm drop-off (the height of a typical stair) is an impossibly
high cliff for a newly walking infant. With developmental changes in leg
length, muscle strength, and balance control, it becomes a challenging but
navigable obstacle for a more accomplished toddler, a big step for a child, a
trivial stair for a young adult, and a challenge again for an elderly adult.
Indeed, development creates new affordances for locomotion and
manipulation (for reviews, see Adolph & Hoch, 2019; Adolph & Robin-
son, 2015; E. J. Gibson, 1988). Before infants can sit or stand, much of the
world is out of sight and out of reach. Head-mounted eye tracking shows
that in a prone position, infants primarily see the ground in front of their
hands. They cannot see objects across the room or lift their head to see
their caregiver’s face. Object manipulation is difficult because infants must
prop themselves on one hand to grasp and explore objects with the other
hand. After infants can sit and stand, the whole room comes into view. In
a sitting or standing position, both hands are free to manipulate objects,
facilitating learning about object properties. The ability to walk opens up a
new world of affordances. Compared with experienced crawlers, novice
walkers spend more time in motion, take twice as many steps, and travel
three times the distance. Given their upright posture and propensity to
move, walkers go to more places, access more distal objects, carry objects
more frequently, and bring objects to their caregivers. In short, walkers
move, see, explore, and interact more (Adolph & Tamis-LeMonda, 2014).
What Is Learned? Exploratory Actions, Information “Pickup,”

and Perception of Affordances
Gibson acknowledged the importance of learning and exploration through-
out The Ecological Approach. As the explorer learns to discriminate or, in
his terminology, “pick up” the relevant information that specifies an
affordance, exploratory activities become increasingly efficient. But Gibson
was confused about whether infants must learn to perceive affordances of
drop-offs and other basic features of the environment. He wrote: “the
basic affordances of the environment are perceivable … without an exces-
sive amount of learning” (p. 134). In fact, Gibson thought the perception
of basic affordances was a better candidate for nativism than abstract
knowledge: “As for innate versus learned perception, it is much more sens-
ible to assume an innate capacity to notice falling-off places in terrestrial
animals than it is to assume that they have innate ideas of mental concepts
of geometry” (p. 151). Indeed, if any knowledge were innate, it should be
knowledge that ensures an animal’s survival—“don’t fall off a cliff!”
Nevertheless, Gibson’s assumption was wrong. Human infants do not have
an innate capacity to perceive a large drop-off as a falling-off place
(Adolph & Kretch, 2012). Neither do kittens, puppies, infant monkeys,
and other animals that cannot walk at birth. But Gibson knew this too. He
also wrote, “Perceiving the meaning of an edge in the surface of support,
either a falling-off edge or a stepping-down edge, seems to be a capability
that animals develop. This is not abstract depth perception but affordance
perception” (p. 160). It seems that at the time of Gibson’s writing, he did
not fully appreciate the contributions of the Susan Linn Sage Professor.
In the years since The Ecological Approach was published, develop-
mental researchers have learned much about age- and experience-related
changes in perception of affordances for locomotion and manipulation.
Thus, for the remainder of this chapter, we focus on examples of develop-
mental research conducted from an ecological point of view. Many find-
ings support Gibson’s ideas about mature visual guidance of basic actions.
However, work with human infants also reveals a protracted process of
learning to perceive affordances and guide actions adaptively as infants
acquire locomotor and manipulative skills.
Affordances for Locomotion

The Ecological Approach outlined several important functions of vision
for guiding locomotion: keeping balance, controlling collision, steering,
navigating a cluttered environment, and coping with variations in the
ground surface. However, Gibson did not acknowledge that visual guid-
ance of locomotion improves with development. Moreover, Gibson’s focus
on vision made short shrift of other sources of information. Haptic
information, for example, is important for keeping balance, and haptic
exploration is critical for detecting variations in surface friction and rigid-
ity (for a review, see Adolph & Joh, 2009).
Maintaining Balance
While upright, the body is always in motion. Even during quiet stance, the
body sways within the base of support. To maintain balance, a sway in
one direction must be met by a compensatory sway in the opposite direc-
tion. As Gibson supposed, optic flow is important for balance control.
When visual, haptic, vestibular, and muscle-joint information for body
sway are in conflict, visual information trumps the rest. Adults, for
example, adjust their standing posture in response to simulated optic
flow in a “moving room” (Lee & Lishman, 1975). Forward and back-
ward movements of the room’s side walls create a lamellar flow structure
in the visual periphery. The optic texture elements streaming in parallel
along the sides of the field of view simulate the visual information for
body sway (like the false perception of self-motion when an adjacent car
or train starts moving). In response, adults induce compensatory sways
in the opposite direction (see Shaw & Kinsella-Shaw, Chapter 6, in this
volume; Smart, Hasselbrock, & Teaford, Chapter 10, in this volume).
Older children (3–6 years) do likewise. Compensatory responses in tod-
dlers are so strong that they often step, stagger, and fall (for a review, see
Adolph & Berger, 2015). Although precrawlers do not respond to
peripheral lamellar flow in the moving room, after 15 days of
experimentally-boosted locomotor experience propelling themselves
around in baby go-carts, their responses are more similar to those of
infants who are independently mobile.
Of course, in everyday settings, vision is not the only source of informa-
tion for postural sway. As Gibson recognized, multiple sources of informa-
tion redundantly specify the body’s position in space. Indeed, merely the
light touch of a toddler’s hand resting on a horizontal surface provides
haptic information for postural stability (for a review, see Adolph &
Berger, 2015). Although touching a surface does not mechanically support
infants’ weight, it reduces postural sway, and walking experience improves
toddlers’ ability to benefit from a light touch.
Controlling Collision
In addition to lamellar flow, postural sway and locomotion also create
radial optic flow. As Gibson suggested, optic texture elements streaming
outward from a central point of expansion specify the direction of heading,
and can guide locomotion toward a goal, or around an obstacle (Warren,
1998). The rate of change in the expansion of optic flow specifies the time
to contact, so collisions can be softened or avoided (Lee, 2009). Gibson
pointed out that an approaching obstacle (e.g., a ball on a collision course
with the observer’s face) expands in the observer’s field of view. It also
hides more and more of the background vista until the obstacle fills the
field of view. In contrast, a receding obstacle (e.g., a car speeding away)
progressively reveals more and more of the background scene. Likewise, as
an observer approaches an obstacle, more and more of the background is
occluded, but as an observer approaches an aperture, more and more of
the background is revealed inside the opening.
Long before infants are independently mobile, they distinguish an
approaching obstacle from an aperture. They blink their eyes and press
their heads backward in response to a looming obstacle but not in response
to a looming aperture (for a review, see Adolph & Berger, 2015). Infants
also take the path of the approaching obstacle into account and distinguish
objects on a head-on collision course from those that will pass safely to
one side (Schmuckler, Collimore, & Dannemiller, 2007). However,
younger infants use information about the size of the visual angle created
by the approaching object; they wait until the visual angle is a certain size
and as a consequence they blink too late to protect their eyes from objects
on a fast collision course (Kayed & van der Meer, 2007). In contrast, older
infants use information about the time to contact. Because this information
is available earlier in the object trajectory, they can respond to faster accel-
erations (see van der Meer & van der Weel, Chapter 7, in this volume).
After infants become mobile, they slow down and turn their bodies
while approaching apertures (for reviews, see Adolph & Berger, 2015;
Adolph & Robinson, 2015). But people of all ages and sizes, from infants
to elderly adults, attempt to squeeze through apertures too narrow to fit
their bodies. Pregnant women adjust their judgments to their growing
bellies, but they also attempt to navigate apertures that are slightly too
small. Apparently, attempting to fit is more compelling than the penalty of
entrapment. Unlike adults, however, infants repeatedly attempt to navigate
apertures so small that they can only wedge an arm or leg into the
opening.
Coping with a Cluttered Environment and Variations in Terrain

Locomotion gets an animal from one place to another, but the everyday
environment is rarely free of obstacles. The landscape is littered with hills,
holes, stairs, and ledges. The ground can be slippery, sloping, rigid, or
unstable. Thus, locomotion requires coping with variations in the terrain.
For Gibson, a particularly illustrative example is the difference between a
cliff and a step. A cliff is a drop-off that is large relative to an animal’s size
and capabilities. A step is a drop-off that is small relative to the animal’s
size and capabilities. Moreover, “a falling-off edge is dangerous, but a
stepping-off edge is not” (p. 149). The Gibsons disagreed about whether
the ability to differentiate cliff from step develops. Indeed, both Gibsons
told their students about a family trip to the Grand Canyon in which
James Gibson insisted that their young children would naturally avoid the
drop-off, but Eleanor Gibson worried that perceiving risk at the edge
requires learning (Adolph & Kretch, 2012).
In fact, human infants and other altricial animals require weeks to months
of locomotor experience before they can tell the difference between safe and
risky ground (for reviews, see Adolph & Hoch, 2019; Adolph & Kretch,
2012; Adolph & Robinson, 2015). When babies first begin crawling and
walking, they plunge right over the edge of a cliff. On a visual cliff, the glass
surface ensures infants’ safety. On an actual cliff, infants are harnessed or an
experimenter catches them when they fall. With each week of crawling and
walking experience, infants’ judgments become increasingly accurate, until
they can discern the difference between gradations of step and cliff (0–90 cm)
within 1 cm of accuracy. Similarly, novice crawlers and walkers fall repeat-
edly over the brink of impossibly steep slopes. Infants’ judgments improve
with locomotor experience until they can distinguish possible from imposs-
ible slopes (0–50°) within 2° of accuracy. Experienced infant walkers even
calibrate their perception of affordances to take experimentally-induced
changes in their bodies into account—lead-weighted shoulder packs or
Teflon-soled shoes. Likewise, experienced crawlers and walkers precisely
detect possibilities for locomotion across bridges and ledges varying from
0–60 cm in width.
However, infants show no evidence of transfer over the developmental
transition from crawling to walking. The same experienced crawlers who per-
ceive affordances with exacting precision, repeatedly attempt impossibly large
drop-offs and steep slopes when they face the precipice as novice walkers.
More generally, learning to perceive affordances does not transfer from
earlier developing postures to later developing ones. The same infants who
precisely perceive affordances for reaching over gaps (0–90 cm) in an experi-
enced sitting posture repeatedly attempt impossibly large gaps in a novice
crawling posture. Whereas experienced cruisers accurately detect affordances
for cruising over gaps (0–90 cm) in a handrail they hold for support, they do
not perceive the impossibility of cruising over gaps in the floor beneath their
feet. Novice walkers also step straight into gaps in the floor.
Thus, infants show separate learning curves for sitting, crawling, cruis-
ing, and walking and no evidence that learning is faster the next time
around (for a review, see Adolph & Robinson, 2015). Why no transfer?
Why does sitting experience only teach infants to perceive affordances for
sitting, crawling experience only teach infants to perceive affordances for
crawling, and so on? New modes of action create new affordance relations.
The body-environment relations for crawling over a precipice, for example,
are entirely different from those for walking. They involve different body
parts producing different forces in different configurations. The explora-
tory actions that generate information for affordances are also different.
For each posture in development, infants must learn all over again to
generate, detect, and use perceptual information about body-environment
relations to perceive affordances for balance and locomotion.
Exploration in the Service of Locomotion

On Gibson’s account, perceptual information is needed to guide
locomotion adaptively, and exploration is needed to generate the required
information. Like locomotion and other performatory actions, exploratory
actions also must be planned and controlled. Because exploration in the
service of locomotion plays out in a temporal and spatial sequence as
animals approach an obstacle or destination, guiding the current
exploratory action requires perceptual information obtained by prior
exploration (Adolph, Eppler, Marin, Wiese, & Clearfield, 2000).
Moreover, exploratory movements have different costs in terms of time,
effort, and potential risk (Kretch & Adolph, 2017). Information collected
from peripheral vision is “free” because the eyes pick up visual information
in the course of locomotion. However, it takes effort to move head and
eyes to look at an obstacle. Adults minimize costs by using peripheral
vision whenever possible. Head-mounted eye tracking shows that adults
can step up, down, and over small obstacles using peripheral vision alone
(Franchak & Adolph, 2010). But on rough terrain, adults continually look
at the ground, and direct their gaze to the foothold two steps ahead
(Matthis, Yates, & Hayhoe, 2018). By dint of their smaller stature, infants
and children see the ground closer to their feet through peripheral vision.
Although infants and children direct their gaze to obstacles more fre-
quently than do adults, they can also navigate small obstacles using only
peripheral vision (Franchak, Kretch, Soska, & Adolph, 2011).
Haptic exploration is more costly than looking, and a search for
alternative methods by testing various options is more costly still. Whereas
visual exploration can occur from a distance, haptic exploration and testing
alternatives require proximity to the potentially risky surface. To explore by
touching, observers must modify their gait as they approach the obstacle,
interrupt locomotion by stopping at the edge, and maintain balance while
poking out a “feeler” (e.g., hand or foot) to probe the surface. The more
closely the exploratory action approximates the locomotor action, the more
useful the information, but the greater the risk of injury.
Infants minimize costs by ramping up exploratory actions based on
information obtained from previous exploratory movements (Adolph et
al., 2000; Kretch & Adolph, 2017). For example, as infants approach a
bridge spanning a precipice, head-mounted eye tracking shows that their
first sight of the obstacle is through peripheral vision. If the bridge is wide,
they run straight across. But if the bridge is narrow, they direct a quick
glance at the obstacle. If this glance suggests further information is needed,
infants slow down and engage in more costly haptic exploration by poking
a foot out to feel the bridge or take tiny steps with their feet just over the
edge. If haptic information suggests crossing to be safe, infants carefully
inch their way to the landing platform. But if crossing still feels risky, they
test alternative strategies by shifting positions (e.g., from standing to squat-
ting, crawling, backing, and sitting) before they discover a suitable course
of action. If infants cannot find an alternative, they avoid going.
As Gibson noted, animals misperceive affordances for locomotion when
they fail to detect the relevant information from a distance: Birds fly into
glass windows and people trip over unseen impediments on the ground.
Visual information can reveal upcoming changes in surface layout such as
elevations, slopes, or the narrowing of the path, and thereby instigate the
appropriate sequence of exploratory movements. However, information
about variations in friction and rigidity can only be detected through direct
contact with the ground surface (Adolph & Joh, 2009; Adolph & Robinson,
2015). For example, the visual information available from a shiny Teflon
surface, rippling waterbed, or lumpy foam pit is not sufficient to alert the
observer that the surface may not afford locomotion. Shine is not a reliable
cue for slip, and lumps are not a reliable cue for deformability because the
properties relevant for locomotion only emerge when the supporting limbs
contact the ground surface (Adolph & Joh, 2009). Without additional haptic
information, non-human animals and people of all ages step straight onto
slippery, squishy, or flimsy surfaces—and fall (Adolph & Berger, 2015).
Affordances for Manipulation

In Gibson’s view, understanding perceptual control of manual actions is
especially challenging because “the uses of the hands are almost unlimited”
(p. 224). The “five-pronged squirming protrusions” at the end of the
human arm can be used to grasp objects, open lids, eat food, and caress a
child (p. 214). The Ecological Approach gives equal attention to manipu-
lation of natural objects (e.g., rocks and leaves) and manufactured artifacts
(e.g., scissors, cabinets, blankets). Gibson’s ideas about affordances for
manipulation inspired a wave of research in human factors and product
design (Norman, 2013; see Pagano & Day, Chapter 3, in this volume). In
particular, Gibson highlighted the use of tools to extend manual capacities.
While in use, the tool becomes an extension of the body; when it is put
down, it reverts to being an object. Thus, Gibson argued that tool use blurs
the boundary between body and environment.
However, Gibson’s insights about manual action gave short shrift to
development; he did not discuss the foundational role of posture in sup-
porting manipulation; and he over-emphasized the role of visual kinesthe-
sis in reaching. Finally, Gibson’s approach to prospective control—using
perceptual information to plan future actions—works best when the neces-
sary information is currently specified in the ambient array (e.g., catching a
ball). His approach is less tenable for multi-step actions that require
higher-order planning (when initial actions required to accomplish the end
goal are not specified by currently available perceptual information).
Posture Provides the Foundation for Manipulation

Gibson noted that locomotion can support manual activity—many trips
toward objects terminate in reaching and grasping, and locomotion is often
used to transport objects. But posture is even more critical than locomo-
tion to support manipulation (for reviews, see Adolph & Berger, 2015;
Adolph & Hoch, 2019; von Hofsten, 2003). Lifting an arm to reach
displaces the body’s center of mass. To avoid disruptions to balance, the
torso must anticipate the arm movement. More generally, the postural and
manual systems must collaborate so that actions can be timed and
controlled prospectively. For infants, this is a difficult task. Without suffi-
cient coordination and strength to stabilize the body, infants’ heads flop
over and their torsos collapse as they try to initiate a reach from a sitting
position. Infants can reach with one hand only after they learn to prop
themselves up on one arm in a prone or tripod position, and they can reach
with two hands free only after they learn to sit independently.
Exploring an object in hand is also dependent on posture. While sitting
with both hands free, infants can explore an object by fingering the surface
to generate information about shape and texture; they can rotate it and
transfer it between hands to reveal its three-dimensional form; and they
can alternate between mouthing and looking to exploit the sensitivity of
the mouth and tongue. But while prone, such exploratory procedures are
hampered because one hand is occupied with propping up the head and
chest. While supine, mouthing predominates because infants’ arms are too
weak to hold the object in front of their eyes.
Planning a Reach in the Light and in the Dark

As Gibson noted, vision plays an important role in reaching. The sight of a
desirable object provides an impetus to reach. Neonates and older infants
extend or flap their arms more frequently in the presence of a visually
salient object than when no object is present (for reviews, see Adolph &
Berger, 2015; von Hofsten, 2003). The sight of the hand is also attractive
to young infants. Neonates turn their head to keep their hand in view. The
early tendency to coordinate eyes and arms and to extend the arms toward
looked-at objects likely provides a developmental basis for later visual-
manual exploration and visually guided reaching. But to what extent does
the concurrent sight of the object and hand guide a reach? Of course,
Gibson was correct that looking at the hand as it approaches an object
causes “the optical minification of the squirming silhouette of the hand …
while optical magnification specifies flexion of the arm” (1979/2015,
p. 113). However, Gibson overemphasized the role of visual kinesthesis—
seeing where the body parts are and what they are doing relative to things
in the environment—in guiding a reach to a target. In his words, “reaching
is an elongation of the arm-shape and a minification of the five-pronged
hand-shape until contact occurs” (p. 224), but that is not the way that
reaching typically happens—even in infants.
Like Gibson, the first researchers to study the development of reaching
conceived it as a process of continually matching the sight of the hand to
the sight of the target (for reviews, see Adolph & Berger, 2015; Adolph &
Robinson, 2015; Smitsman & Corbetta, 2010). Indeed, at first blush, the
primacy of visual kinesthesis in infant reaching seems a reasonable suppo-
sition. Infants’ first reaches are jerky and inefficient with multiple changes
in direction and speed; over development, the reaching path becomes
straighter and smoother. Thus, researchers inferred that jerky reach traject-
ories result from online corrections and overcorrections as infants continu-
ously track and adjust the location of the hand relative to the target. But
infants do not need to see their hands to reach. Infants reach for objects in
the light at the same age that they reach for glowing objects in the dark,
and early attempts at reaching are jerky and erratic regardless of the light-
ing. Jerky infant reach trajectories are likely due to the biomechanical and
dynamic characteristics of arm movements, not a product of visual guid-
ance and correction.
Although normally not necessary for online guidance of reaching, sight
of the hand and target facilitates more precise manual actions (Berthier &
Carrico, 2010). Seeing hand and target, for example, improves infants’
control of their hand trajectory as they reach for a tiny morsel of food.
Moreover, getting the hand to the object only solves part of the problem of
object prehension. After the target is located and the arm is on its way,
grasping the object requires configuring the hand relative to object prop-
erties. Visual kinesthesis facilitates grasping an object as Gibson proposed.
Infants use concurrent visual information about hand and object to adjust
their hand to the object’s shape, size, and orientation (for reviews, see
Adolph & Berger, 2015; Adolph & Robinson, 2015). Very young infants
bump their hand into the object before opening their fingers to grasp it.
Older infants retrieve objects with a whole-handed power grip, and later
use a precision grip (index finger and thumb) to grasp small objects. By
10–12 months of age, infants adjust the space between finger and thumb—
grip aperture—as their hand approaches the object. For large or unwieldy
objects, infants use visual information to decide whether two hands are
needed, and if so, they scale the space between their hands to the object’s
size. When objects are irregularly shaped, infants aim their hands closer to
the center of mass. Eventually, children’s manual actions are so precise that
they switch from a one- to a two-handed grasp when the size of an object
exceeds the size of their grip aperture. As Gibson noted,
Long before the child can discriminate one inch, or two, or three, he
can see the fit of the object to the pincer like action of the opposable
thumb. The child learns his scale of sizes as commensurate with his
body, not with a measuring stick.
(1979/2015, p. 224)
Planning for Future Manual Actions

Gibson argued that optical information specifies both current and impend-
ing events—events that begin in the present and will continue into the near
future (e.g., imminent contact with a looming object). Gibson also argued
that such information provides the basis for controlling action prospec-
tively (e.g., dodging a thrown rock or catching a thrown ball). Empirical
evidence of two different kinds suggests that the ability to use visual
information to anticipate the future location or orientation of a moving
object develops in infancy (for review, see von Hofsten, 2003). First,
infants successfully intercept a moving target by putting their hand into the
object’s path or appropriately orienting their hand to grasp a rotating rod.
Second, if the trajectory of a moving object abruptly changes, infants move
their hand to where the object should have been rather than to its new,
unanticipated location.
For manual actions involving moving objects and/or moving observers,
timing is critical. Younger infants attempt to catch a moving target by
“chasing” it with the near hand as it moves past and then “catching” it
with the far hand as it moves into the far side of reaching space (Fagard,
Spelke, & von Hofsten, 2009). Older infants lift their far hand to intercept
the object as it approaches. Moreover, the visual information that infants
use to intercept moving objects changes over development. Younger infants
use information about object distance or speed to time their reach (Kayed
& van der Meer, 2009). Older infants perceive the time to contact the
object. Information for time to contact is more efficient because it ensures
sufficient time to lift the arm and orient the hand (see van der Meer & van
der Weel, Chapter 7, in this volume).
When the infant is moving and the target is stationary, the timing of
postural, visual, and manual systems must be precisely coordinated. For
example, to retrieve a target while spinning 360° on a motorized chair,
infants must first turn head and trunk to visually locate the object, then
plan arm movements to ensure that their hand arrives at the object’s loca-
tion, and finally shape the hand to intercept and grasp the object before
they spin past. Infants and adults share a common spatiotemporal sequence
of postural-visual-manual coordination, but infants perform this sequence
more slowly than adults (Rachwani, Golenia, Herzberg, & Adolph, in
press). For infants, greater postural control predicts faster planning of
manual actions.
Some manipulative actions involve multiple steps (e.g., pick up fork,
spear bite of broccoli, and bring fork to mouth). Although Gibson did not
explicitly address multi-step actions, his approach can explain such behav-
iors. Each step in the sequence generates information for the next step, so
the multi-step action unfolds in an ongoing stream of perceptual informa-
tion. Older infants, for example, fit a square block into a square aperture
by first grasping the block and then smoothly aligning the block as it
approaches the aperture so that the fit is seamless and uninterrupted (for a
review, see Lockman, Fears, & Jung, 2018). Younger infants do not pre-
adjust the orientation of the block during approach; instead, they bring the
block to the aperture and then wiggle the block back and forth to orient
the edges appropriately. Children show similar age-related improvements
in pre-orienting rods to fit into slots.
However, in some cases, infants (like adults) show evidence of their
intentions for the second step in the sequence as they are performing the
first step (Claxton, Keen, & McCarty, 2003). Infants place a block atop a
block tower more slowly than they chuck a block into a bin. But remark-
ably, when the block’s destination is the tower, the initial reach for the
block is also slower than when the block’s destination is the bin (Chen,
Keen, Rosander, & von Hofsten, 2010; Claxton et al., 2003). This
intention-influenced initial action is not well explained by Gibson’s pro-
posals for visually controlled manipulation.
Some actions stretch so far into the future that information about the
end goal is not immediately available to perception. For example, in tasks
involving “end-state comfort,” observers must alter the form of their initial
action to take the end goal into account (for a review, see Wunsch,
Henning, Aschersleben, & Weigelt, 2013). When flipping an upside-down
glass to fill it with water, adults initially grip the glass with their thumb
pointing down—an atypical grip—so that after flipping it, the thumb
points up in a position comfortable for filling the glass. In contrast, young
children flip the water glass with an initial thumb-up grip and end in an
awkward thumb-down position. Similarly, when grasping tools such as
spoons or hammers, infants and preschoolers always use a typical over-
hand grip. But when the handle points away from the dominant hand, an
initial overhand grip leaves the hand in an awkward position for eating or
hammering (Comalli, Keen, Abraham, Foo, Lee, & Adolph, 2016;
McCarty, Clifton, & Collard, 1999). More generally, children do not show
adult-like planning for end-state comfort for most tasks—turning handles,
rotating dowels, and so on—until 9–12 years of age. Because Gibson’s
theory relies on perceptual information that is immediately available, he
does not address planning for end-state comfort.
Conclusion: Learning to Learn

Perceptual-motor learning permeates Gibson’s Ecological Approach. In Gib-
son’s view, animals need not learn to associate perceptual information about
the world with appropriate actions. He argues that association learning is
not the primary mechanism for perception of affordances for locomotion,
manipulation, or any other action. Indeed, forming associations would be
maladaptive because affordances depend on relations between body and
environment, not fixed facts about the body or environment. Simple associ-
ation learning cannot work because the status of the body and the environ-
ment are always changing. Instead, animals learn to detect information for
affordances in real time in their moment-by-moment, situation-by-situation
interactions with the world. Thus, perceptual-motor learning involves learn-
ing to perform and hone exploratory movements that generate information
for affordances, and learning to differentiate and use the specifying informa-
tion to plan and guide actions adaptively—a process of learning to learn.
Given Gibson’s emphasis on affordances, the exclusion of development
from The Ecological Approach is a serious omission. Development alters
affordances. Developmental changes in the body (e.g., the ability to walk)
and the environment (e.g., the new vistas engendered by an upright
posture) change the available perceptual information and enable new
opportunities for action. Moreover, development ensures learning to learn.
Rapid body growth and dramatic changes in motor skills during infancy
and early childhood discourage reliance on simple associations or fixed
facts about the body or environment. Yesterday’s body is different from
today’s body. A poor crawler last week is a more proficient crawler a week
later. Solutions that worked earlier in development may not hold at later
points in development. Thus, infants and young children are forced to
learn in the midst of rapid and dramatic development change. The flux of
development creates exactly the type of flexible and adaptive perceptual-
motor learning that Gibson proposed.
Acknowledgments
Work on this chapter was supported by the National Institute of Child
Health and Human Development (R01-HD033486 and R01-HD086034)
to Karen Adolph. We are grateful to Jennifer Rachwani and members of
the NYU infant action lab for their insightful comments.
14 Information and Its Detection
The Consequences of Gibson’s
Theory of Information Pickup
Brandon J. Thomas, Michael A. Riley, and
Jeffrey B. Wagman
Chapter 14 of Gibson’s (1979/2015) landmark book is a wide-ranging and

critical chapter that provides an overview of the theory of information
pickup, which includes many of the most radical aspects of his ecological
approach to visual perception. While several consequences of the theory of
information pickup are made explicit, perhaps the most significant con-
sequence is an implicit one. Here and elsewhere, Gibson was critical about
the often implicit assumptions that psychologists make about the relation-
ship between the organism and the environment. He felt that such assump-
tions have far-reaching consequences for theories of perception and, more
generally, for theories of knowing. Gibson questioned many of these
assumptions and, in doing so, redefined the problem of perception in his
theory of information pickup. Specifically, Gibson proposed new answers
to the questions of what is perceived, how it is perceived, and why it is
perceived.
Traditional cognitive theory assumes that animals are fundamentally
separate from their environment. In such a description, the animal has
tenuous contact with the environment provided only by the inherently
meaningless stimulation patterns impinging on the sense organs. The
animal must interpret those patterns (and infer a meaningful environment)
using knowledge obtained either by previous experience or by genetic
inheritance.
Moreover, in traditional approaches, classic physical and geometrical
variables (e.g., length, width, distance) are considered to be the relevant
environmental properties to be perceived. The stimulation patterns imping-
ing on isolated sensory receptors (e.g., retina, cochlea, mechanoreceptors)
are ambiguous, at best, about such properties, because those patterns can
only specify the momentary states of the receptors and sense organs them-
selves. Thus, these dual assumptions create an epistemic gap between
meaningless (or at best, ambiguous) stimulation of the sense organs and
the meaningful experience of the animal. In large part, this epistemic gap
exists because the descriptors of the environment (borrowed from physics
and geometry) and the patterns of activation in sense receptors (borrowed
from physiology) are insufficient to explain the experience of the world
238 Brandon J. Thomas et al.
(a uniquely psychological phenomenon). Traditionally, psychologists have
seemingly needed to look to cognitive or neurophysiological processing
(e.g., representations, cues, inferences) to fill this epistemic gap and subse-
quently explain how psychological phenomena are (almost always) unam-
biguous and meaningful (see Wagman, 2010). Furthermore, this gap leads
inevitably to circular explanations of perception, because an environment
that is disconnected from the organism and unspecified by patterns of
stimulation leaves cognition itself as the basis by which cognition occurs.
Gibson’s theory of information pickup turns this problem on its head by
reconsidering ontology and shifting the level of analysis from (within) the
isolated animal to the animal-environment system (see Part I in this
volume). For Gibson, animal and environment are reciprocal. A funda-
mental consequence of this reciprocity is the dissolution of the epistemic
gap between animal and environment. Perception is not the process of
inferring a meaningful environment from meaningless stimulation. Rather,
it is an ongoing process of detecting meaningful information that yields
meaningful experience.
The theory of information pickup further proposes that the relevant
aspects of the environment to be perceived are not geometric and physical
properties such as lengths, widths, distances, angles, or colors. Rather, they
are ecological properties, such as surfaces, substances, places, objects,
events, and affordances. In addition, rather than ignoring the potential for
meaningful structure in energetic media (e.g., ambient light), the theory of
information pickup explicitly focuses on such structure—on its invariance
over transformation. In a radical departure from traditional descriptions of
perception, Gibson’s theory of information pickup proposes that patterns
in energetic media relate unambiguously to (i.e., they specify) properties of
the animal-environment system. Moreover, the notion of passive and local-
ized sense organs is discarded. Instead, the theory of information pickup
reconsiders how patterns in energetic media are detected by active percep-
tual systems that extend across the entire body, even encompassing objects
temporarily attached to the body such as implements or tools.
The theory of information pickup collapses the epistemic gap that tradi-
tionally necessitates the invocation of cognitive processing. By developing
a new ontology, the ecological approach changes the course of psycho-
logical inquiry. Perception as “a keeping-in-touch with the world” rather
than “an appearance in the theater of consciousness” provides a new
foundation upon which to build psychological theory. In his Chapter 14,
Gibson (1979/2015) explicated these defining features of the theory of
information pickup and mused on some of the broader implications of his
theory. We will touch on some of the significant developments in the
theory of information pickup that have occurred since the release of the
book and connect the theory to some of our own work.
Information and Its Detection 239
What Is Perceived?
Traditional theories assume conventional physical and geometric variables
(distances, angles, colors, etc.) are the objects of perception. Rather than
relying on this animal-independent description of the environment, Gibson
argued for an animal-dependent description in terms of ecological physics,
where the relevant variables become places, objects, substances, and events.
These constitute the relevant features of the environment for an organism
in its niche. Moreover, the environment is perceived in animal-referential
terms—that is, in terms of affordances. Affordances are opportunities for
behavior that emerge from relations between the organism and the
environment (Stoffregen, 2003a).
Though affordances are not an explicit topic in Gibson’s (1979/2015)
Chapter 14 (see Wagman, Chapter 8, in this volume), they are the corner-
stone of his ecological physics, and consideration of the theory of
affordances highlights many of the important aspects of his theory of
information pickup. Specifically, Gibson proposed that affordances are not
computed by mentally combining lower-order properties of animal and
environment. Rather, they are perceived directly by detecting higher-order
patterns in ambient energy arrays—the information specifying affordances.
Consequently, affordances ought to be perceived as such—as emergent
complex particulars (Turvey, 2015)—rather than as collections of discrete
lower-order properties.
Thomas and Riley (2014) investigated that hypothesis for the percep-
tion of affordances for reaching overhead and for passing through an aper-
ture when the perceiver’s action capabilities were augmented by a
hand-held implement. Based on Gibson’s theory of information pickup, the
authors hypothesized that under such circumstances perceived affordances
would not be reducible to a combination of the relevant metric properties
of the person and the object. Participants gave reports about two
affordances: (1) overhead reach-with-ability of the stick; and (2) pass-
through-ability of an aperture when holding the stick such that it extended
from the side of the body (see Figure 14.1, top). The stick increased
maximum reaching height and decreased minimum passing width, respec-
tively. Participants reported both perceived and remembered affordances
(with the stick present and absent, respectively) for the two behaviors. In
the remembered affordance conditions, the stick was removed from view,
but participants made reports as if they were using it. They also reported
the length of the stick in each case.
Thomas and Riley (2014) found that participants overestimated the length
of the stick and underestimated its effect on performing the reaching and
passing through tasks (Figure 14.1, bottom). Furthermore, perceived (or
remembered) affordances were not equivalent to a combination of reports
about the metric properties of the person and the object. These results
support the notion that affordances are perceived as such (i.e., as emergent,
Figure 14.1 Top: Apparatus and procedure for the overhead reaching task.
Average perceived/actual ratios when participants reported ability to
reach with stick while it was present, while it was absent, with remem-
bered stick length added to reach-ability without the stick (i.e., addi-
tive model), and reach-ability without the stick (right). Bottom:
Apparatus and procedure for the minimum pass-through-ability task.
Average perceived/actual ratios when participants reported ability to
pass through the aperture with stick while it was present, while it was
absent, with remembered stick length added to pass-through-ability
without the stick (i.e., additive model), and pass-through-ability
without the stick (right).
Source: From Thomas and Riley (2014). Adapted with permission from American Psycho-
logical Association.
complex particulars) rather than inferred, constructed, or computed by com-

bining perceptions of lower-order properties (e.g., stick length-plus-maximum
reaching-height or stick-length-plus-shoulder-width).
Similarly, Mark (1987) asked participants to judge the maximum step-
on-able-ness and sit-on-able-ness of surfaces while wearing blocks on their
feet. Participants also made judgments about the height of the blocks. Per-
ception of both affordances rapidly recalibrated to the new abilities to step
and sit, but perception of block height was inaccurate and did not improve
across trials. That is, affordance perception was more accurate than—and
seemingly independent of—block height perception. Consistent with Gib-
son’s theory of information pickup, perceivers are (and become increas-
ingly) attuned to the information specifying affordances (not isolated
geometric or physical properties). Information about such affordances need
not be inferred, constructed, or computed. It need only be detected.
The Information for Perception

Gibson’s redefinition of information for perception is perhaps the most
prominent aspect of his theory, but it is also one of the most counterintui-
tive. Information is traditionally discussed within the context of Shannon
and Weaver’s (1949) communication framework, in which a noisy and
transient signal (i.e., a representation) is transmitted between a sender and
a receiver. Applied to perception, the environment is the sender, the organ-
ism is the receiver, and the signal is the pattern of activation in the sense
receptors. However, this conceptualization of information is inappropriate
for the theory of information pickup, for a number of reasons. First, per-
ception is not a communicative act. The environment does not send signals
to the perceiver. Rather, the information is detected (and generated) by the
activities of the perceiver. Second, the information for perception is not
transient. Rather, it exists indefinitely and is inexhaustible. Perception is a
continuous act with no beginning and no end. Finally, and perhaps most
critically, the information for perception is not coded or represented. It is
not arbitrarily related to its source. Rather, it is specific (i.e., lawfully
related) to the animal-environment system.
Gibson’s discussion of information was largely focused on the concept
of the ambient optic array, which is structured light reflected from surfaces
of the terrestrial environment that surrounds all points of observation.
Moving through the environment simultaneously generates variance and
invariance in the optic array (i.e., optic flow) which specifies both environ-
mental surfaces and the perceiver’s relation to such surfaces. Lee (1976)
used this concept to develop an informational variable specifying time to
contact (τ), which can be used to avoid or control collisions with surfaces
or objects.  is specified by the ratio of an object’s visual angle and its
visual angle’s rate of change. The derivative of τ with respect to time (֗ )
specifies the deceleration required to avoid a collision. Since information
about time-to-contact is important for performing so many common
behaviors (locomotion, interception, etc.), optic flow and  in particular
have inspired a great deal of research in the subsequent decades, with 
serving as a paradigmatic example of an informational variable that speci-
fies a behaviorally relevant animal-environment relation.
The framework of information-based control advanced Gibson’s
concept of information, suggesting that movements are continuously scaled
to information via some control law (Warren, 2006). The control law
guides movements so as to produce a task-specific value of the informa-
tional variable. For example, catching fly balls can be accomplished by
detecting the optical acceleration of the ball on a two-dimensional visual
plane (Chapman, 1968). The control law is to move so as to “zero out”
the optical acceleration, which will bring the fielder to the landing spot of
the ball.
More recently, Fajen (2007) adapted the information-based control
approach to accommodate perception of affordances (what he called
affordance-based control). Fajen’s motivation for affordance-based control
was that information-based control accounts do not necessarily posit infor-
mational variables that specify action capabilities, so information must be
described in arbitrary, extrinsic units. However, by expressing informa-
tional variable as a ratio of relevant physical properties and relevant
action-related properties, behavioral capabilities are specified and informa-
tion is expressed in intrinsic units. Control laws then become a matter of
moving to keep certain actions possible, by staying within the boundary
between possible and impossible actions. Thus, in the case of catching a fly
ball, the catchableness of a fly ball is specified by a ratio of optical acceler-
ation and maximum running speed, and the control law is to move so that
the fly ball remains catchable (Oudejans, Michaels, Bakker, & Dolné,
1996). Furthermore, information that specifies action boundaries allows
agents to effectively calibrate their actions (Fajen, 2005b).
Another substantial advance in Gibson’s concept of information is Stof-
fregen and Bardy’s (2001) global array hypothesis. Perceptual systems that
operate independently of one another present a theoretical quagmire for
the theory of information pickup, since information detected by separate
perceptual systems would need to be combined (see also Stoffregen,
Mantel, & Bardy, 2017). Since the animal-environment system structures
multiple energetic media simultaneously, Stoffregen and Bardy argued that
the information for perception exists only in emergent, higher-order pat-
terns across these energy arrays—what they call the global array. They
argued that perception involves detection of these global patterns. While
the perceptual systems could be considered anatomically distinct, they,
nonetheless, work as a single functional system responsible for maintaining
the contact between animal and environment. Whereas Gibson argued that
the concept of sense organs was inappropriate for understanding percep-
tion, Stoffregen and colleagues argue that concept of separate perceptual
systems is equally inappropriate. In their approach, information in the
global array is detected by a single perceptual system (Stoffregen et al., 2017).
Furthermore, Witt and Riley (2014) used the extended global array in
an attempt to reconcile the ecological approach to perception-action with
action-specific perception, which suggests that the environment is perceived
in terms of the ability to act within it (Witt, 2011). Thus, the perception of
the environment depends on both physical properties and the capacity of
the organism to behave with respect to them. For example, golfers perceive
the size of a hole in terms of their ability to putt the ball into it (Witt,
Linkenauger, Bakdash, & Proffitt, 2008). The two approaches are similar,
but since action-specific perception posits the use of putatively internal
factors (i.e., emotions and skill level) and is concerned with perceptual
reports of discrete, animal-neutral properties (e.g., distance), it appears to
be inconsistent with direct perception. However, by extending the global
array to include invariants within interoceptive arrays (which are internal
chemical arrays that potentially specify physiological states), the two the-
ories become potentially compatible. For example, White, Shockley, and
Riley (2013) found that perceived distance traveled is specified by a multi-
modal informational variable that scales optically specified distance by the
effort necessary to traverse that distance (operationalized as the amount of
O2 consumed to travel the distance). Thus, the information for perceived
distance was a higher-order pattern defined across exteroceptive (optical)
and interoceptive (chemical) media.
The Concept of a Perceptual System

Gibson (1966a; 1979/2015) discarded the notion of five senses as tradi-
tionally conceived and offered, instead, his theory of active, functionally
defined perceptual systems. One of the aims of this new description was to
frame perception as an activity—an obtaining of information rather than a
receiving of stimulation. Because information can only be detected by act-
ively sampling energy distributions, a description of sensory receptors
alone is insufficient for understanding perception. Perceiving requires
detecting invariance in energetic media over transformations, including
(and especially) those created by the movements of the perceiver. Gibson
referred to perceptual systems as “continuous input-output loops” but
even this description does not do justice to what Gibson had in mind. For
example, in this new view, the visual system includes—in addition to the
eyes—the head, neck, torso, legs, and even the hands. The inclusion of
these seemingly unrelated anatomical components underscores the notion
that seeing is itself an activity that includes subordinate actions like orient-
ing, locomoting, magnifying, and scanning. The particular anatomical
components that comprise a perceptual system vary depending on the
property of the environment that the animal intends to perceive and the
behavior that the animal intends to perform. As a result, work on this
aspect of the theory of information pickup has largely been directed at
understanding the task specificity and anatomical independence of percep-
tual systems. We now turn to that empirical work.
Smart Perceptual Instruments

Consonant with Gibson’s notion of information pickup, Runeson (1977)
suggested that perception be conceived as the application of what he
termed “smart” perceptual mechanisms (i.e., smart perceptual instru-
ments). Carello, Fitzpatrick, Domaniewicz, Chan, and Turvey (1992; see
also Bingham, 1988) outlined four fundamental features of smart percep-
tual instruments. Such instruments are smart because they capitalize on
special circumstances of the situation; determinate because the information
detected is lawfully related to properties of the animal and environment;
scaled because measurements of a given property are commensurate with
that property; and soft because the instruments themselves are spontan-
eously, flexibly, and temporarily assembled to accomplish a given
measurement task.
Carello et al. (1992) tested these features of smart perceptual instru-
ments in the context of perception by dynamic touch—the effortful move-
ment of objects attached to the body, such as wielding and hefting, leading
to structured deformation of tissues sensitive to mechanical energy fluxes.
Participants were asked to perceive the length of unseen rods by holding
them horizontally. The anatomical components that were used to hold the
rod (one hand, two hands, knee), the application of force from the environ-
ment (downward and upward forces applied on the rods by implements),
and the inertial properties of the rods were manipulated within and across
experiments. Consistent with their hypotheses, perceived rod length was
specific to the invariant inertial properties of the rods (determinate) and
commensurate with actual rod length (scaled), regardless of both the par-
ticular rods used in the set and the anatomical components used to hold
the rod (smart and soft).
Because perceptual systems are functionally defined and perception is
specific to information, the ability to perceive a given property is—within
limits—independent of particular anatomical components. Many different
measurement instruments can be assembled using a given set of anatomical
components, and many different anatomical components can be assembled
into a given measurement instrument. Wagman and Hajnal (2014a, 2014b,
2016) investigated this latter aspect of smart perceptual instruments in a
series of studies (see also Withagen, 2004; and Withagen & Michaels,
2005a, 2005b). They found that participants can perceive whether an
unseen inclined surface affords standing on when they explore that surface
with a rod, regardless of whether that rod is held in one hand or the other,
or in both hands in multiple grip configurations (Wagman & Hajnal,
2014a). Moreover, participants could perceive this affordance when they
explored the surface with a rod attached to anatomical components not
typically used for exploratory wielding, such as the foot or even the head
(Wagman & Hajnal, 2014b, 2016).
Such flexibility of perception not only occurs within a given perceptual
modality, but also extends across modalities. A given object, event, or
affordance simultaneously structures multiple energy arrays (cf. Stoffregen
& Bardy, 2001). In some cases, structure in any one of those arrays may
be sufficient to specify that object, event, or affordance. For example, the
length of an unseen object can be perceived by viewing that object, wield-
ing that object, or by listening to the sounds produced when that object is
dropped to the floor. Despite the anatomical, physiological, and functional
differences between touch and audition, there is evidence that, in each case,
perception of length is constrained by the object’s inertial properties (see
Carello, Anderson, & Kunkler-Peck, 1998; Carello, Wagman, & Turvey,
2005). Such properties have the potential to lawfully structure both the
patterns of mechanical energy fluxes when the object is wielded and the
patterns of vibrations when the (unsupported) object strikes a surface and
are therefore potentially informative about length in each case.
Wagman and Abney (2012, 2013) further tested this possibility using a
transfer of recalibration protocol. They reasoned that if information about
a given property is simultaneously available in multiple energy arrays, feed-
back about perception of length via touch ought to transfer to the percep-
tion of length via audition and vice versa. Participants judged the length of
a set of unseen rods either by listening to the rods fall onto a wooden
surface or wielding the rods about their wrist. When feedback about length
was provided for either perception by audition or perception by touch,
improvements in perception transferred across perceptual modalities.
Wagman and Abney (2012, 2013) argued that the modality-independence
of perception is the same in kind as the anatomical-independence of
perception.
The Dynamics of Perceptual Instruments: Soft-Assembly as

Coordinative Structures
Understanding how the perceptual systems are coordinated so as to act-
ively exploit invariants from the ambient flux of energy is analogous to the
degrees of freedom problem in motor control (Bernstein, 1967; see Profeta
& Turvey, 2018). Namely, how do organisms compress the high-
dimensional cellular, muscular, and skeletal units into low-dimensional
systems that are appropriate for performing some particular goal-directed
behavior? Though a great deal of work has been done to address this issue,
we will focus on the work that particularly highlights the theoretical devel-
opment of perceptual systems.
Fitch, Tuller, and Turvey (1982) proposed a theory of coordinative
structures to account for the degrees of freedom problem in perception and
action (see also Latash, 2008; Latash, Scholz, & Schöner, 2002; Riley,
Richardson, Shockley, & Ramenzoni, 2011). Like smart perceptual instru-
ments, coordinative structures are temporarily (i.e., softly) assembled col-
lections of otherwise potentially independent cellular, muscular, and
skeletal units. Coordinate structures are functional because the units are
organized for the purpose of stabilizing task performance, flexible because
they are not strictly dependent on any particular aspects of anatomy, com-
pensatory because the units reciprocally compensate for fluctuations in the
environment and within the units themselves, and context-sensitive since
the role of the coordinative structure as a whole or of any individual unit
within the structure can change, depending on task constraints.
Understanding perceptual systems as coordinative structures requires an
appropriate analytic framework. The dynamical systems framework natur-
ally accommodates a conceptualization that treats perceptual systems as
synergistic—as macroscopic systems that are circularly linked to the com-
ponent level of cellular-muscular-skeletal units. The perceptual system can
stabilize into a finite number of low-dimensional regimes to perform a task
or achieve a goal-directed action (Haken, 1977). This macroscopic, func-
tionally organized system is both comprised of and recruits component
level units, based on Haken’s slaving principle. This conceptualization of
behavior presents a solution to the degrees of freedom problem, because
the macroscopic system is a low-dimensional, self-organizing system that
emerges and constrains the high-dimensional, component level system.
Thus, we can collapse across the many degrees of freedom at the com-
ponent level and study the simpler patterns observed at the macroscopic
level.
Warren’s (2006) behavioral dynamics model of perception-action nicely
captures the notion of a perceptual system in the parlance of dynamical
systems. The organism and environment are treated as a pair of coupled
dynamical systems, both comprised of a vector of state variables that
define the current state of the entire system. The two systems are coupled
through both the external forces from the environment and the informa-
tion detected by the organism. These coupled dynamical systems comprise
the component level of the system. The macroscopic level is that of a goal-
directed action (i.e., behavioral dynamics), and it both emerges from the
component level while also enslaving it. The behavioral dynamics can
themselves be modeled as a dynamical system with its own stable and
unstable regimes.
Gibson’s perceptual systems are captured by the behavioral dynamics
model. The organism and environment are mutually coupled through per-
ception and action. This component-level coupling gives rise to and is
enslaved by the macroscopic behavioral dynamics that function to solve
perception-action goals. For example, a number of studies on dynamic or
effortful touch have found that the intention to perceive different object
properties results in exploratory wielding movements with different
dynamical signatures that subsequently generate and detect information to
perceive the intended property (Riley, Wagman, Santana, Carello, &
Turvey, 2002; Stephen, Arzamarski, & Michaels, 2010). Riley et al. asked
participants to wield unseen rods with no specified intent to perceive any
particular property, or with the intent to perceive length, width, height, or
grasp position by dynamic touch (all of which require directing attention
to different aspects of the inertia tensor). They analyzed the 3-D wielding
movements of participants as they performed the task. They found that the
deterministic structure of wielding movements differed depending the per-
ceptual intent of the participant, suggesting that the behavioral dynamics
were differentially assembled to perform the function of the task (see
Figure 14.2). The difference in intention modulates the structure of the
low-dimensional, higher-order pattern that emerges.
In this example, the component levels are the effectors responsible for
the exploratory movements and the physical properties of the rods. These
components are coupled by the information that arises between the inter-
action of the effectors and the rods during wielding and also by the phys-
ical forces involved in that wielding. The behavioral dynamics of
ϱϰϴ
EŽŝŶƚĞŶƟŽŶ
Ǉ
Ϯϳϰ
ũ
ǌ ǆ
ϭ
;ĂͿ ϭ Ϯϳϰ ϱϰϴ
ŝ
ϱϳϮ
/ŶƚĞŶƟŽŶ
Ǉ
Ϯϴϲ
ũ
ǌ ǆ
ϭ
;ďͿ ϭ Ϯϴϲ ϱϳϮ
ŝ
Figure 14.2 (a) No perceptual intent condition and (b) perceptual intent condition of
the Experiment 1. Left panels show 3-D plots of exploratory wielding in
a single trial. Right panels show recurrence plots for the y coordinate.
Source: From Riley et al. (2002). Reprinted with permission from Sage.
wielding-to-perceive a given property emerge as a low-dimensional dynam-
ical system that enslaves the component level systems.
The concept of perceptual systems continues to develop under the eco-
logical approach to perception and action. Perception is a coordinated
activity. The behavioral dynamics of perceptual systems reflect functional,
softly assembled couplings between information and movement, and
between animal and environment. These couplings and the resulting
dynamics are constrained by the particulars of a task and by the perceiver’s
intentions and purpose.
The False Dichotomy Between Present and Past Experience

In traditional accounts, perception begins with the impinging of ambigu-
ous stimulation patterns on the sensory organs and ends with an inference
about the external source of the stimulation. In such accounts, this process
occurs only when sensory energy activates the sense receptors. When
sensory energy no longer activates the sense receptors, contact with the
object no longer occurs by means of perception. Instead, such contact can
occur only by means of memory. For Gibson, the momentary stimulation
of the sense receptors is merely incidental to detection of information and,
thus, to perception. Invariance emerges only in the context of transforma-
tion, and such transformations can only occur over space and time. Thus,
perception is a continuous act that necessarily occurs over time. As pointed
out by Gibson (1966b) and later by Michaels and Carello (1981), although
separation of the past, present, and future in subjective experience has
intuitive appeal, this separation has no empirical or theoretical basis. The
boundaries are fuzzy at best. For Gibson, perceiving “goes on;” it is a con-
tinuous activity and the signposts of experience are ecological events (see
Blau, Petrusz, & Carello, 2013), not the passage of time, per se (see Shaw
& Kinsella-Shaw, Chapter 6, in this volume).
Some recent work has empirically demonstrated the continuity between
memory and perception. Wagman, Thomas, McBride, and Day (2013)
tested whether remembered affordances are scaled to action capabilities in
the same way as perceived affordances. Participants reported maximum
reaching height when they expected to step on a stool and reach with their
hand and when they expected to pick up a stick and reach with the stick.
Moreover, they performed these tasks both when each object was visible
and after each object had been removed from view. Perception of
maximum reaching height scaled to reaching ability (and impeding changes
in reaching ability) both when such objects were present and after the
objects had been removed from view. In other words, “remembered
affordances” scaled to action capabilities in the same way as perceived
affordances (see also Thomas & Riley, 2014).
What are the consequences of the theory of information pickup for what
a person will remember about the environment? The theory of information
pickup suggests that there is a lawful relationship between properties of
the environment, information, and the perception of those properties. This
notion places constraints on what can be remembered. Gibson argued that
exploratory behaviors of perceptual systems are organized so as to detect
the information that specifies the property-to-be-perceived. From this per-
spective, the only difference between perception and memory is that objects
of perception can be further scrutinized and explored while objects of
memory can no longer be scrutinized by perceptual systems. Therefore, the
limiting factors of experience are the informational constraints of the
ambient stimulus flux and the ecological physics of the environment.
Withagen (2004) differentiated between specifying variables and non-
specifying variables. Non-specifying variables can be detected but do not
specify the to-be-perceived property of the environment (the term “non-
specifying” should not be taken to mean that those variables are not
related in a one-to-one fashion to any property, however). If informational
variables about two different properties of the environment just so happen
to correlate with each other in a specific context, then a perceiver might
incidentally detect information about both properties when he or she
intends to perceive one of those properties. Several studies have demon-
strated that people detect information that does not specify a to-be-
perceived property under various task constraints (Jacobs & Michaels,
2001; Jacobs, Runeson, & Michaels, 2001; Runeson & Vedeler, 1993;
Withagen & van Wermeskerken, 2009). These studies provide evidence
that non-specifying informational variables can be detected when multiple
informational variables happen to situationally relate to the perceived
property. Michaels and de Vries (1998) referred to the joint constraints of
natural law and the arbitrary or non-arbitrary constraints of a given scen-
ario as a “task ecology,” a concept that depicts the particular circum-
stances within which information is detected. For example, if the
information for the heaviness and length of a set of objects is correlated,
then a perceiver might incidentally detect information that specifies heavi-
ness when perceiving length, provided that the resulting perception is
accurate enough to satisfy current task constraints. Thus, the capacity to
remember might depend on the task ecology of perception.
Thomas and Riley (2015) tested this hypothesis by evaluating the influ-
ence of informational constraints on remembered properties of an object.
Participants reported the length and heaviness of a set of rods when the
information for these two properties either co-varied or not in separate
conditions. Participants wielded rods that were either visible (visual, geo-
metric information was available in addition to haptic-based information)
or occluded (haptic-based information only) and did so with the intention
to perceive rod reach-with-ability. Then, after the objects had been
removed from the hand and from view, participants were asked to report
the remembered heaviness of the rod via a magnitude estimation task.
Because of the nature of the rod set used in these experiments, the visual,
geometric information about reach-with-ability was uncorrelated with
heaviness in the visible condition. However, haptic-based information
about reach-with-ability was correlated with heaviness in the occluded
condition. Therefore, the task ecology either afforded remembering rod
heaviness or not, based on the constraints of the rod set. Participants were
able to remember proportional differences in heaviness when haptic
information was detected. However, participants were unable to remember
proportional differences in heaviness when visual information was avail-
able and, presumably, attended to (i.e., picked up). Thus, it seems that
what was remembered about the object was determined by what the per-
ceiver intended to perceive about the object together with the incidental
correlations of the information for that property with other properties. In
other words, remembering can be incidentally accomplished with the use
of non-specifying information.
By reconsidering the distinction between perception and memory, it is
possible to redefine the problem of memory. If there is no principled reason
to assume that the separation between perception and memory occurs at
the loss of sensory contact, then we must redefine this distinction or
discard it. This is Gibson’s strategy in his theory of information pickup:
Study psychology at the level of the animal in the environment and attenu-
ate the epistemic gap, rendering cognitive processing unnecessary for
explaining our experience of the environment. This theoretical develop-
ment of ecological remembering is in its nascent stages, but the work above
provides some avenues for future development.
A New Approach to Knowing

The final section in the chapter on information pickup is what Gibson calls
“A New Approach to Knowing.” Traditionally, perceiving is considered a
momentary activity, occurring only as long as an energetic link between
stimuli and receptors exists. Knowing, on the other hand, is thought to be
more permanent, lasting indefinitely. Consequently, Gibson’s reconceptu-
alization of perception as a continuous act puts perception on relatively
equal footing with knowing. Just as the theory of information pickup
closes the epistemic gap between animal and environment, it also closes the
conceptual gap between perception and knowledge. For Gibson, both per-
ceiving and knowing require detecting information in the context of trans-
formations over space and time. They are different in degree but not in
kind. Perception is cognitive, not because it requires cognition but, because
it yields knowledge of meaningful, complex, and emergent relationships
between perceiver and environment (what might otherwise be called “con-
cepts”). For Gibson, knowing is an extension of perceiving.
Recently, several scholars (Bruineberg, Chemero, & Rietveld, 2018;
Chemero, 2009; Rietveld & Kiverstein, 2014; Withagen, Araújo, & de
Poel, 2017) have attempted to explain how this epistemic link can be
extended to aspects of the “sociomaterial environment” that are not “sen-
sorily present.” In other words, to use one of their examples, how can a
perceiver know that there is beer in a can on the table, when only the can
(and not the beer inside it) structures the ambient light reflecting to the
point of observation occupied by the perceiver? Moreover, how can this
knowledge be explained without invoking representations or “higher-order
cognition?” To deal with this, Bruineberg and colleagues differentiated
between two types of information. Lawful ecological information describes
regularities in an ecological niche between energetic structure from a point
of observation and an aspect of the environment. Alternatively, general
ecological information describes regularities in the ecological niche
between aspects of the (sociocultural) environment. General ecological
information pertains to aspects of the environment that tend to occur
together due to constraints, such as complex causal structures, convention,
norms, or habits. In their analysis, a combination of lawful and conven-
tional constraints allows the ambient light at a point of observation to
carry information about the presence of beer (in the can). They then rede-
fine affordances as relations between aspects of the (sociocultural) environ-
ment and abilities and argue that this redefinition allows ecological
psychology to account for problems of so-called higher-cognition.
To conclude the chapter, Gibson described three aids to comprehension
that make the abstracting and the extracting of invariants easier. He dis-
cussed knowing mediated by instruments, knowing mediated by descrip-
tions, and the use of pictures. Much of the subsequent work on knowing
by instruments has included perceptual tools, such as a hand-held rod or
an enactive torch (a vibrotactile sensory substitution device) (Burton, 1993;
Favela, Riley, Shockley, & Chemero, 2018; see Wagman & Chemero,
2014). For Gibson, “lower-level” instruments are continuous with the
kinds of displays that can give rise to affordances for the control of
complex work environments (e.g., battlefields, cockpits, power plants, hos-
pitals). In all of these cases, the instruments can support direct perception
of affordances (see Pagano & Day, Chapter 3, in this volume). Relatedly,
Dotov, Nie, and Chemero (2010) found that a smoothly operating
participant-tool-system exhibits structured variability (1/f or pink noise)
characteristic of coordinated behavior in many complex physical, biologi-
cal, and cognitive systems. When the efficacy of the participant-tool-system
is disrupted, however, this structure is reduced.
There has been less subsequent ecological work on knowing mediated
by descriptions or pictures (but see Stoffregen, Chapter 15, and Blau,
Chapter 16, in this volume). For Gibson, descriptions (both written and
spoken) themselves are abstractions. They are “information that has been
put into words” that cannot be explored or scrutinized in the way that
information (in the form of structured energy) can. As Gibson, puts it “the
invariants have already been extracted.” In discussing the limitation of
descriptions, Gibson remarks that there are always “wordless facts” that
are not (easily) captured in a verbal description. The marked difficulty in
expressing such wordless facts is consistent with our claim that affordances
are perceived as emergent, complex particulars rather than as collections
of discrete properties. A complex particular is difficult to put into words.
A discrete property is not.
Conclusion
Most contemporary theories of psychology create an epistemic gap that
separates reality from experience. Ambiguous and passive stimulation of
sense receptors by meaningless physical variables must necessarily be
enriched by cognitive mediation to become meaningful to the agent. Gib-
son’s theory of information pickup, and his ecological approach more
broadly, sidestep this issue by making different assumptions about what is
perceived, how perception occurs, and why perception occurs. Gibson
posited the animal-environment system as the fundamental unit of analysis
and described perception of the environment in terms of ecological physics,
rather than in terms of the abstract physics of Newton and the imaginary
geometry of Euclid. He shifted the focus from sensory stimulation to
information in ambient energetic media and redefined the senses as percep-
tual systems that actively and continuously detect such information. Also,
he rescaled the time course of perception, beyond the presence or absence
of sensory stimulation to the boundaries of ecological events (see Blau et
al., 2013).
Since the release of Gibson’s (1979/2015) book, the theory of informa-
tion pickup has continued to be validated and expanded. Ecological
physics has withstood considerable empirical scrutiny. Perceptual systems
have been shown to exhibit anatomical and modality independence, and
advances in the understanding of perceptual systems as smart perceptual
instruments have occurred in tandem with advances in the understanding
of perception-action as the soft assembly of coordinative structures. Bur-
geoning work has begun to demonstrate the efficacy of the theory of
information pickup for understanding memory as well as other functions
of higher-order cognition. With some of the implications yet unexplored
and more still to be realized, Gibson’s ecological approach to psychology
continues to provide fruitful lines of scientific inquiry even after 40 years.
Part IV
Depiction
15 The Use and Uses of Depiction
Thomas A. Stoffregen
James Gibson cared deeply about theoretical issues relating to the nature,
creation, and perception of images, pictures, and other depictions. He wrote
about these issues throughout his career and devoted an entire section of his
most mature theoretical statement to depiction. In this chapter, I point out
that depictions have been of great significance in the behavioral sciences. I
note that behavioral science research continues to rely on measurements of
experience and behavior relative to depictions and that, therefore, theoretical
issues relating to the nature and status of depictions are important for our
evaluation of much current research. A related issue is whether observers
(correctly) perceive the fact of depiction, or (erroneously) believe that they
are observing “the real thing” (i.e., that which is depicted), rather than a
depiction. My discussion leads me to consider the affordances of animal-
depiction systems. I argue that animal-depiction systems and animal-
undepicted systems (to use an unwieldy term) have markedly different
affordances, and that both humans and non-humans readily differentiate
these affordances. I claim that the fact of depiction is specified, and that
exploratory activity generates information in the global array (Stoffregen &
Bardy, 2001; Stoffregen, Mantel, & Bardy, 2017) that permits direct percep-
tion of the affordances of animal-depiction systems.
Depictions in the History of Behavioral Science

The camera obscura is an enclosed space that is kept in darkness except for
a small aperture in one wall. Light entering through this aperture projects
onto the opposite wall, forming an inverted image of the outside world.
Before 1600, cameras obscura were often very large: An entire room could
be a camera (Figure 15.1). Observers, standing inside the camera, could
directly observe the projected image (Lindberg, 1968). Paper or canvas
could be attached to the wall, making it simple to trace the projected
image. The camera obscura was used as an aid to painting (e.g., Hockney,
2001). Thus, for centuries both scientists and artists (who, often, were the
same people) were accustomed, in very literal, practical senses, to the idea
of the projected image as something that could be seen, in itself.
256 Thomas A. Stoffregen
Figure 15.1 A camera obscura, in which light enters through a hole. The figure
illustrates the projection of an image that could be viewed by a human
observer.
From Antiquity, the aperture through which light entered the camera
obscura was a hole (such devices still do exist; small versions are called
pinhole cameras; Wade & Finger, 2001). At the beginning of the 16th
century Johannes Kepler fitted a camera obscura with a convex glass lens,
which had the effect of making the projected image both brighter and
sharper. Kepler’s camera + lens system bore a physical resemblance to the
(stationary) chambered eye, which has a lens that refracts and focuses
light, a chamber through which the light passes, and a “far wall”, the
retina, onto which an inverted image is projected (Figure 15.2). Like many
of his contemporaries, Kepler had extensive experience looking at the pro-
jected images inside cameras obscura. Combining this common experience
with the physical resemblance between the camera + lens and the cham-
bered eye, Kepler claimed that the chambered eye functions as a camera
obscura, and that perception of the illuminated environment consists of
seeing (or looking at) the image that exists on the retina (Lindberg, 1976).
Kepler’s theory (i.e., that the eye yields an image on the retina) was con-
firmed by dissecting the eye of an ox and scaping the back until the tissue
was so thin that the projected image could be seen by a person looking at
the back of the eye (Gal & Chen-Morris, 2010; see Carello & Turvey,
Chapter 4, in this volume).
Kepler’s analysis was the origin of the “picture theory” of perception
(Gibson, 1979, pp. 52–55; Ronchi, 1957; Stoffregen, 2013; Wade &
Finger, 2001; cf. Cutting, 1988). James Gibson rejected the picture theory.
The Use and Uses of Depiction 257
Figure 15.2 The chambered eye considered as a camera with a lens. This figure
was used by Descartes, and was inspired by Kepler (Lindberg,
1976). Note the human observer, who views the projected image.
He claimed that perception of the illuminated environment does not entail

or constitute seeing (or looking at) the retinal image. Yet, Gibson shared
with Kepler the belief that the perception of pictures, such as drawings
made in cameras obscura, is important for our understanding of percep-
tion. Gibson argued that perceiving the illuminated environment is not like
perceiving pictures, but that the perception of pictures—paintings, draw-
ings, and the like—is a type of perception that must be explained in any
general theory of perception.
For Kepler and his intellectual descendants, seeing the projected image
inside a camera obscura was seeing, so that if the scientist understood per-
ception inside a camera, then the scientist understood perception, in
general. In part, for this reason, behavioral scientists have, for centuries,
treated depictions as substitutes for physical reality, with the implication
that research on the perception of pictures should generalize to all
perception.
There is not a clear boundary between “still pictures” and “moving pic-
tures.” For example, until the 19th century, cameras did not have shutters.
Light entered the camera continuously, such that the projected image was not
static, but dynamic (Stoffregen, 2013). That is, prior to the adoption of shut-
ters in the 19th century, cameras projected optic flow, rather than static
images. In the cinema, and in virtual reality systems, moving pictures actually
comprise a sequence of still pictures. These appear as continuous movement
only when presented in rapid succession. Vision does not have the same struc-
ture: Eyes do not have shutters and, consequently, have neither “frames” nor
“frame rates.” Indeed, it is likely that for vision, truly static patterns of stimu-
lation are both unnatural and vanishingly rare, such that they have little or
nothing to do with visual function. Gibson (1979/2015) made this point in
his chapter on “motion pictures” (p. 281; see also Lee, 1980).
Depictions in Contemporary Behavioral Science

Within the ecological approach to perception and action, the use of still
pictures in behavioral experimentation has declined, but moving depictions
continue to be used. Examples include digital depictions (e.g., Plumert,
Kearney, & Cremer, 2004) and interactive depictions, such as those pre-
sented in head-mounted displays (e.g., Bruggeman, Zosh, & Warren, 2007;
Fajen, 2005a). Gibson was acutely aware of the difficulties and issues
involved in using depictions (both static and dynamic) as substitutes for
physical reality:
[D]rawings are particularly inappropriate objects with which to begin

a study of the perception of form. A drawing is a human production
never found in a natural environment; it is complicated by being a
thing with which [people] communicate with one another; it is not a
simple presentation to sense-organs but a representation of a substitute
object. As a stimulus for perception, it is convenient …
(Gibson, 1951, p. 404, emphasis in the original)
To behold a motion picture is thus similar in important ways to

observing the ordinary happenings of life. But it is also radically dis-
similar in other ways that are just as important. Both need to be
understood.
(Gibson, 1979/2015, p. 282)
One of the central ideas of Kepler’s theory remains true (implicitly) in
much contemporary research: That looking at a depiction is representative
of looking, in general. One way to determine whether depictions are
appropriate sources of stimulation in behavioral science research is to ask
whether observers can (and do) differentiate depictions from the rest of
physical reality. If depictions are perceived as such, then it may be that
perception of depictions may not be representative of perception, in
general.
Depiction and Belief

Behavioral scientists often study participants’ responses to depictions. The
results of such studies typically are not interpreted as informing us about
the perception of depictions, as such, but about perception, in general. It
seems to me that this practice implies one of two claims (Stoffregen, 1993;
Stoffregen, Bardy, Smart, & Pagulayan, 2003). First, it may be that obser-
vers (whether human or otherwise) do not differentiate depictions from
“that which is depicted.” That is, it may be that “visual awareness” of
depictions is equivalent to visual awareness of non-depicted reality. Alter-
nately, it may be that observers differentiate depictions from “that which
is depicted” but that, for some reason, this differentiation is not important
for our understanding of perception. This second possibility appears to be
consistent with Gibson’s view, as reflected in the quotation from Gibson
(1979/2015, p. 282) above. As noted by Gibson, we need to know how
perception of depictions is “similar in important ways to observing the
ordinary happenings of life,” but we also need to know how the percep-
tion of depictions is “radically dissimilar” from perception of non-depicted
physical reality. This need should motivate a program of empirical testing
to directly address questions about how and when the perception of depic-
tions differs from the perception of non-depicted physical reality (e.g., Stof-
fregen, Bardy, Merhi, & Oullier, 2004). In this section, I examine these
two claims.
Do people believe that depictions are real? That is, do they confuse
depictions with the things that are depicted? I use belief as a proxy for
Gibson’s term, awareness. Researchers and developers often assert that
digital depictions are perceived as reality, that is, that observers do not dif-
ferentiate depictions from the depicted. A common term used to express
this idea is presence, which is loosely defined as the subjective awareness of
being in the depicted setting (e.g., the fictional planet, Zorgon), rather than
the actual, physical setting (e.g., one’s basement). Presence is equivalent to
what Anderson (1996) called “the illusion of reality” (Shapiro & McDon-
ald, 1992; Stoffregen, 1993; Stoffregen et al., 2003).
The concept of presence often carries considerable intuitive plausibility,
but it is not without its problems. One problem is that, in most cases, crea-
tors of depictions actually do not intend for observers to believe that they
truly are in the depicted settings or situations. Artists (painters, play-
wrights, film directors, video game designers, and so on) do not try to fool
us, a fact of which Gibson was aware (1979/2015, p. 269; cf. Goffman,
1974). Stoffregen et al. (2003) distinguished between the (accurate) percep-
tion of realism and the (inaccurate) perception of reality. The fact that a
person has the subjective experience, “this is realistic,” actually implies
awareness of the fact that “this” is not “the real thing,” but instead is
something (e.g., a display) that has been created to resemble, in certain,
limited ways, something else. To emphasize the contrast, when we meet a
friend on the street, we may make any of a wide number of remarks about
their appearance (e.g., “you look good today”), but it is unlikely that we
ever will say to our friend, “you look realistic today.”
A second, logically separate problem is that the attempt to deceive cannot
actually succeed, or can succeed only within very narrow limits. Gibson
(1979/2015) knew that the “illusion of reality” is less than an illusion: It is a
falsehood. That is, any general assertion that perceivers truly are unable to
distinguish depictions from the depicted reality is false: “the notion of an
image that is literally and actually indistinguishable from the reality is a
myth” (Gibson, 1971g, p. 33). And, “no matter how faithful, how lifelike,
how realistic a picture becomes, it does not become the object pictured”
(p. 33). Directly, “there is no illusion” (Gibson, 1979/2015, p. 271). And,
most emphatically, “No painter and no photographer should ever strive to
give viewers the feeling that they are looking at a real place, object, person,
or event. There is no need to do so. In any case, the effort is bound to be a
failure” (p. 269; cf. Berger, Gonzalez-Franco, Ofek, & Hinckley, 2018).
Gibson (1979/2015) described a simple experiment in which the parti-
cipant stood looking at a photograph. The experimenter asked questions
about distances to things in the photograph, such as “How far away from
you is the elm tree?”. Participants understood these questions and gave
their best estimates. But they could also answer when Gibson asked, “How
far away from you is the picture?”. That is, participants could attend to
both (or either) the depicted layout (i.e., in the photograph) and the unde-
picted layout (i.e., the photographic print, on a wall, in a room). Attending
to the contents of a depiction generally requires a “suspension of disbe-
lief,” that is, a choice, on the part of the observer, to “play along” with the
intentions of the person who created the depiction (e.g., Goffman, 1974).
Gibson (1979/2015) claimed that perception of depictions was paradox-
ical and that it entailed simultaneous but incompatible perception of phys-
ical reality (of the depiction, as such and its physical substrate) and of the
(absent) contents of the depiction. This cannot be: Paradox is not permit-
ted under physical law. A resolution to the apparent paradox is that per-
ception is never actually of the “things depicted” as distinct from the fact
of depiction (Stoffregen, 1997). The fact that, by voluntarily suspending
disbelief we can attend to the contents of a depiction does not imply that
we are unable to differentiate the fact of depiction.
Typically, the discussion about pictures and other depictions has focused
upon the conscious awareness of the viewer; hence, the title of Gibson’s
(1979/2015) Chapter 15, “Pictures and visual awareness.” From the per-
spective of the artist, this focus makes sense. It makes sense also in the
context of theories of perception that are, themselves, focused on the idea
that perception is a mental process that ends with conscious awareness. By
contrast, The Ecological Approach to Perception and Action emphasizes
relations between perception and the control of behavior. Conscious
awareness can be important, but there are other methods by which we may
determine whether perceivers differentiate depictions from that which is
depicted.
If people (and other observers) differentiate depictions from that which
is depicted, then we are obliged to ask whether the perception of depic-
tions differs from the perception of that which is depicted in ways that are
important for our understanding of general theories of perception and
action. In the following sections, I address this question in the context of
the behavioral opportunities of, and behavioral responses to, depictions.
The Affordances of Depictions

Gibson’s analysis of depiction was not carried out in a vacuum. He took
part in a spirited debate. Participants in the debate included behavioral sci-
entists (e.g., Cutting, 1988; Ittelson, 1996) and scholars from the arts (e.g.,
Gombrich, 1972). Much of the debate included two assumptions. First,
that depictions primarily were things to be looked at. Second, that the
principal desideratum was an explanation of the viewer’s conscious experi-
ence. Artists typically seek to influence the viewer’s conscious experience,
and behavioral scientists often have evaluated perception through reports
of conscious experience. To be sure, we are all familiar with what it feels
like to view great art, and many readers can recall participating in experi-
ments in which the “stimuli” were pictures and other depictions (e.g.,
Necker cubes, or tachistoscopic presentations). But the existence of depic-
tions is not (and never has been) limited to art, or to behavioral science. In
addition to asking about what depictions are, we can ask what they are
for, what functions they serve or enable. I consider it significant that the
terms, afford and affordance do not appear in Chapter 15 (Gibson,
1979/2015).
Shapiro and McDonald (1992, p. 102) noted, “Real events sometimes
demand action, while obviously fictional events can usually be enjoyed
vicariously.” That is, the affordances of depictions, and of “the depicted”
often differ qualitatively. In his Chapter 8, Gibson (1979/2015) presented
his theory of affordances, that is, of opportunities for action that arise in
animal-environment systems. To illustrate the concept, Gibson offered a
list of affordances. His list did not include or refer to depiction. Here, to
illustrate my claim that depictions have affordances, and that these
affordances often differ qualitatively from those found in animal-
environment systems that do not include depictions, I now list what I take
to be some affordances of animal-depiction systems. The categories are not
intended to be exhaustive, or mutually exclusive.
Aesthetic Experience
This category comprises most of what we refer to as art; painting, sculp-
ture, photography, cinema, and other creations whose primary purpose is
to give rise to some type of aesthetic awareness (e.g., of “beauty”). Aes-
thetic art is functional only as part of an artist-observer system.
To Communicate Personal Information

Gibson knew “a picture is a record” (1979/2015, p. 278). For thousands
of years, depictions have been created with the purpose of communicating
information about the identity and individuality of particular people. In
simple terms, I refer to portraiture, whether in drawing or painting, sculp-
ture, photography, or other media. Portraits may or may not be “artistic,”
but with or without artistry they afford communication of the facial fea-
tures, economic status, and (depending upon the skill and penetration of
the artist) the character of the person or persons depicted. Portraits are
functional only as part of sitter-artist-observer systems.
A sub-category of portraiture is the self-portrait, in which the artist and
the subject are the same person. Self-portraits afford the preservation and
communication of information about the self. This sub-category includes the
pictorial likeness (e.g., the many self-portraits of Rembrandt van Rijn, or
Edward Hopper). But it also includes very different “records of the self.” At
some primal level, any depiction is a record of its creator. Non-pictorial
records of the self are among the earliest surviving depictions. A vivid
example is “hand paintings,” made by placing one’s hand on a wall and
blowing pigment onto the wall, creating an outline figure of the hand (Figure
15.3). It is worth noting that this simple affordance for recording and com-
municating the existence of the self was discovered and actualized by early
humans on every continent, except Antarctica (Chippindale & Tacon, 1998).
Paleolithic hand paintings demonstrate that detection and actualization
of affordances for recording and communicating the existence of the self
precede most other forms and uses of depiction. Certainly, they precede all
conventions of drawing, of linear perspective, and so on. It is significant
that the detection and exploitation of affordances for recording and com-
municating one’s own existence are primal not only in the history of our
species but also in the development of the individual: Children learn about
affordances of depictions for recording their own existence and activity,
and often proudly announce their understanding of this function (“I made
it!”) long before they “learn how to draw” (Figure 15.4).
Figure 15.3 Hand paintings in paleolithic rock art. These are “self-portraits” in the
sense that they record (and communicate) the existence of the self.
Source: Australian Hand Painting, from Bradshaw Foundation, available at: www.bradshaw
foundation.com.
Figure 15.4 Hand prints in development.

To Influence the Behavior of Non-Human Animals
This category comprises two sub-categories: depictions created by
humans, and depictions created by non-human animals. Perhaps the best
example of the first sub-category is the perennial scarecrow. Sadly, the
scarecrow is of such low fidelity that crows rapidly differentiate the
depiction (which cannot harm them) from the depicted (e.g., a farmer
with a shotgun). Scarecrows are functional (to the extent that they are
functional, at all) only as part of systems that comprise humans (e.g.,
farmers) and animals that compete with humans for food (e.g., crows).
More successful examples are found in hunting. Decoys, that is, depic-
tions of ducks in the illuminated environment, and duck calls, that is,
depictions of duck vocalizations in the acoustic environment, are widely
understood to be successful in the sense of encouraging living ducks to
land (and, thereby, render themselves targets for hunters; M. T. Stoffre-
gen, personal communication, April 17, 2018).
The far more common sub-category is depictions created by non-human
animals to influence the behavior of other non-humans. In the illuminated
environment, a good example is mimicry in certain species of octopus, who
can vary the coloration of their own bodies so as to blend into the back-
ground, rendering themselves less visible to predators (e.g., Anderson,
Mather, & Wood, 2013). This mimicry is functional only as part of an
octopus-predator system. Many species have discovered (either as indi-
viduals, or across many generations) that various forms of imitation or
mimicry afford influencing the behavior of other species, or of conspecifics.
To Encourage Economic Action

I refer to depictions used in advertising (which are far more common than
“art” depictions). Advertising exists because advertisers believe that it can
influence behavior, that is, because advertisers believe that advertising
affords economic action. Advertising is functional only as part of an
advertiser-consumer system.
To Deceive for Commercial Gain

This category includes forgeries in art (i.e., paintings that afford prices
associated with artists who did not actually create them). It also includes
fakes in consumer products, such as fake wristwatches, or deliberately mis-
labelled wine (e.g., Rothwell & Atlas, 2010). A good forger can deceive, at
great profit, and these deceptions can succeed for many years. There is not
a categorical distinction between forgeries and evident depictions, that is,
between the belief in reality and the awareness of depiction (cf. Davies,
1985). But to search for such a categorical distinction is to miss the point.
The actual issue is the ability, the willingness, and the skill of the observer,
user, and/or purchaser to explore, to conduct “tests for reality” (Gibson,
1979/2015, p. 292).
To Influence Social and/or Political Action

Animal-depiction systems can also have social affordances on a large scale,
what we might call societal affordances. History testifies to the ability of
depiction-society systems to influence social movements. For example,
Jacques-Louis David’s (1793) painting, La Mort de Marat, was used to
influence public opinion during the French Revolution, while Picasso’s
Guernica played an important role in galvanizing public outrage over the
1937 bombing of the Spanish town. Photography has a long tradition of
influence in social movements. The documentary photographs of Lewis
Hine contributed to the adoption of child labor laws in the United States.
Similarly, the photographs of William Henry Jackson were instrumental in
convincing the US government to preserve the natural wonders of the Yel-
lowstone region in a national park. David, Picasso, Hine, and Jackson all
understood—and stated, explicitly—that their depictions could be used to
achieve goals that had nothing to do with depiction, as such. That is, they
understood some of the affordances of depictions for social change. In this
context, depictions are functional only as part of advocate-society systems,
or propagandist-proletariat systems.
Opportunities to influence events in society and politics are more
abstract than types of affordances that typically have been subject to con-
trolled experimental research. But it can be argued that, as with
affordances for aesthetic appreciation, they fit within the logic of the
affordance concept. Indeed, if the ecological approach is to succeed as a
general theory of perception-action, then it must offer an affordance-based
account of “higher cognition” (Bruineberg, Chemero, & Rietveld, 2018;
Heft, 2017; Reed, 1996b; Stoffregen, 2000a, 2000b, 2003a, 2003b, 2004).
To Communicate Quantitative Information

The principal constituent of this category is data graphics. Depictions
afford the recording and communication of quantitative information (e.g.,
Tufte, 1983). It is possible that the number of “statistical graphics”
exceeds the number of “art” paintings. Data graphics are functional only
as part of author-reader systems.
To Communicate Geographical Information

Maps are depictions that enable or optimize certain actions. A good
example is road maps. These provide information about the layout of
drive-able surfaces. For drivers (or helpful passengers), road maps afford
navigation, that is, action in the driving environment relative to things that
Figure 15.5 The original caption reads; Long dismissed as a simple connect-the-

dots game, this plate was finally identified as the “City Churches” of
Sir Christopher Wren as seen from 35,000 feet.
Source: Plate from McCaulay (1978). Copyright ©1978 by David McCaulay. Reprinted by
permission of Houghton Mifflin Publishing Company. All rights reserved.
are not immediately discernible through the perceptual system, such as a

distant destination. Contemporary maps (e.g., on cell phones) often are
updated with traffic information in real time. Readers who use such maps
can judge for themselves whether they provide useful opportunities for
action. Maps are functional only as part of mapmaker-map user systems.
Figure 15.5 gently mocks the use of maps to convey geographical informa-
tion about the architectural legacy of Sir Christopher Wren (1632–1723),
a prominent British architect.
To Enable or Optimize Control

The lowly speedometer is a depiction that, in a driver-automobile system
affords maintenance of desired speed. (It is worthwhile noting that the
speedometer exists because metric speed is not specified in available stimu-
lation.) In aviation, “artificial horizon” displays depict an aircraft’s
dynamic orientation relative to the Earth’s horizon and, thereby, improve
a pilot’s ability to control aircraft orientation and trajectory relative to the
ground (e.g., Rolnick & Bles, 1989). By affording better control in pilot-
aircraft systems, these displays help pilots to avoid the grim fate known as
“controlled flight into the terrain.” At much more abstract levels, Ecolo-
gical Interface Design yields real-time depictions that afford more rapid
and more accurate control of complex socio-technical systems, such as
nuclear power plants (e.g., Vicente, 2002). These depictions are functional
only as part of controller-controlled systems. Depictions used in medical
settings often afford life-saving interventions, as part of provider-patient
systems (e.g., Vicente, Kada-Bekhaled, Hillel, Cassano, & Orser, 2003).
Finally, consider icons on the screens of interactive (i.e., touch-based)
devices. These icons are graphical renderings whose function is to com-
municate something about what a user can do with software applications.
The concept of affordances has been widely adopted in the interface design
community (e.g., Stoffregen, Bardy, & Mantel, 2006; Stoffregen & Mantel,
2015; Torenvliet, 2003). The affordances of these graphical (i.e., nonver-
bal) depictions for communication are of special relevance for persons with
intellectual disabilities (e.g., Kumin, Lazar, Feng, Wentz, & Ekedebe,
2012). Systems comprising intellectually disabled persons and tablet com-
puters achieve functionality that profoundly enriches many lives.
To Study Behavior
Participant-depiction systems afford a great deal of behavioral science
research. Behavioral scientists have been busily, creatively, industriously
exploiting this affordance for several centuries. Participant-environment
systems that do not include depictions afford different behavioral science
research. Whether, and how, these two types of affordances may overlap is
subject to empirical tests which, thus far, have been far too rare.
Looking at art is an ancient activity, giving rise to conscious experience
that can be profound. However, depictions have uses that extend far
beyond aesthetic appreciation. Animal-depiction systems have a very wide
range of affordances; affordances that are routinely actualized in ordinary
life. The utterly common exploitation of these diverse affordances suggests
that, in general, we accurately differentiate animal-depiction systems from
animal-environment systems that do not include depictions. In The Ecolo-
gical Approach to Perception and Action, accurate differentiation arises
from direct perception which, in turn, entails the claim that affordances
relating to depictions may be specified. It is to that topic that I now turn.
Specification and Exploration

If awareness is related to detected information (Reed, 1996b), and if obser-
vers are not fooled by depictions, then it is likely that observers (human or
otherwise) generally differentiate the affordances of animal-depictions
systems from the affordances of systems that do not include depictions
(Stoffregen et al., 2003; Stoffregen et al., 2017). An important question,
then, is whether (and how) depictions are specified. More formally, the
question is whether the affordances of animal-depiction systems are
specified.
The specification of affordances is a critical topic for The Ecological
Approach to Perception and Action and one that, to date, has received
little formal analysis (e.g., Mantel, Stoffregen, Campbell, & Bardy, 2015;
Stoffregen, 2000a, 2000b, 2003b). Analyses of specification, that is, formal
analyses of specificational parameters that are available to perceivers,
typically have focused upon parameters that exist in individual forms of
ambient energy. In 1966, Gibson proposed that information can exist as
emergent, higher-order patterns that extend across different forms of
ambient energy. He gave as examples higher-order patterns that emerge
out of relations between stimulation of the haptic and vestibular systems.
Stoffregen and Bardy (2001) argued that such emergent, higher-order pat-
terns are not merely possible and do not exist in a merely occasional sense.
Rather, they claimed that emergent, higher-order patterns extending across
multiple forms of ambient energy are, in principle, the only type of pat-
terns that meet the criteria for lawful, 1:1 specification of the animal-
environment system. The set of patterns extending across multiple forms
of ambient energy was named by Stoffregen and Bardy the global array.
The global array consists of patterns that extend across multiple forms of
ambient energy and which exist solely as emergent, higher-order relations
between multiple forms of ambient energy.
Gibson (1979/2015, p. 132) argued that information (specificational
parameters in ambient energy) can exist as higher-order, invariant rela-
tions between other, lower-order invariants: “A unique combination of
invariants, a compound invariant, is just another invariant. It is a unit,
and the components do not have to be combined or associated.” Para-
meters of the global array are nothing more and nothing less than com-
pound invariants. They differ qualitatively from the lower-order,
constituent invariants that they comprise. This qualitative difference
means that compound invariants do not exist in, and cannot be derived
from, the lower-order component invariants that are found in individual
forms of ambient energy, such as the optic array, or the acoustic array.
Parameters of the global array emerge from relations between patterns in
“single-energy arrays”: they are irreducible. In the context of depictions,
scientists (and scholars in the arts) commonly focus on patterns that exist
in single-energy arrays. For example, paintings typically are considered
to structure only the optic array. This common focus does not negate the
existence of emergent, higher-order patterns in the global array. The
historical fact that scholars have not considered the possible existence of
higher-order patterns extending across multiple single-energy arrays does
not mean that such patterns do not exist, or are not important for
perception:
The theory of psychophysical parallelism assumes that the dimensions
of consciousness are in correspondence with the dimensions of physics
and that the equations of such correspondence can be established is an
expression of Cartesian dualism. Perceivers are not aware of the
dimensions of physics. They are aware of the dimensions of the
information in the flowing array of stimulation that are relevant to
their lives.
(Gibson, 1979/2015, p. 293)
The global array is not ambient to the massless, geometric points of the
conventions of linear perspective (i.e., to pure kinematics); rather, it
intrinsically is ambient to and provides information about the full
dynamics (i.e., both kinematics and kinetics) of animal-environment
systems, including animal-depiction systems. The physical activity of
animals (including small, subtle movements, such as changes in gaze used
in viewing a painting) simultaneously (but differently) alters the structure
of multiple forms of ambient energy.
Specification of the animal-environment system in the global array has
important consequences for our ability to differentiate animal-depiction
systems from other types of animal-environment systems. As one example,
observer movement relative to a flat screen (e.g., a projection surface for
cinema or virtual reality) simultaneously alters the structure of optics and
of gravitoinertial force, thereby yielding patterns in the global array that
are different from patterns generated by observer movement relative to a
three-dimensional layout (Stoffregen, 1997). This point was made origin-
ally by Gibson (1966a, 1979/2015), and remains valid. The same point
applies, in slightly subtler form, to the consequences of observer movement
relative to three-dimensional depictions. Similarly, touching of depictions
(e.g., a painting of a baseball) yields relations between haptics and optics
that differ qualitatively from touching of the depicted object (e.g., a base-
ball). It can be argued that for active, unconstrained observers, depiction is
specified in the global array and, therefore, that it is always possible to dif-
ferentiate depiction from the corresponding physical reality (Stoffregen et
al., 2003). It is for this reason that depictions can be differentiated from
undepicted reality: The fact of depiction is specified. This argument applies
not only to depictions in the illuminated environment but also to depic-
tions in other single-energy arrays (e.g., acoustics, as in recorded or synthe-
sized sound, including speech). It applies equally to “multimodal”
simulations (see Figures 9 and 10 in Stoffregen et al., 2017). The fact that
experimental participants can perceive depictions (i.e., can recognize the
thing or activity that is being depicted) and can control depiction-based
systems does not imply that the information detected and used under
experimental conditions is the same as in the outside world.
A central premise of The Ecological Approach to Perception and Action
is that perceptual information is not imposed but, rather, is obtained. In
1966, Gibson devoted an entire chapter to this claim (1966a, Chapter 2),
and the obtaining of information was the principal subject of Part 3 of
Gibson (1979/2015). Our physical interaction with the environment causes
changes in the stimulation of receptors. The nature of changes in stimu-
lation is a reciprocal function of relations between the nature of an organ-
ism’s movement and the nature of the environment, that is, of the
animal-environment system.
One common use of skilled movements for the generation and obtaining
of information about affordances is in the exploratory activity of infants.
Infants use particular movements to differentiate the affordances for loco-
motion of surfaces (e.g., Adolph, Eppler, Marin, Wiese, & Clearfield,
2000). The literature on infants’ manipulation of handheld objects (e.g.,
Bushnell & Boudreau, 1993), can be re-interpreted in terms of exploratory
actions generating information about affordances of the infant-toy system,
as can recent studies relating infants’ control of gaze in the learning of new
words (Rader, 2018; Rader & Zukow-Goldring, 2012).
In adults, early qualitative examples include Solomon and Turvey
(1988), who showed analytically that active exploration of objects in iner-
tial space generates dynamic patterns of stimulation that are sufficient for
perception of an affordance for reaching. Riley, Wagman, Santana,
Carello, and Turvey (2002) reported that the patterns of movement
selected in manual wielding differed as a function of task instructions, that
is, as a function of which properties of the animal-environment system
participants were attempting to detect. Mantel et al. (2015; cf. Bingham &
Stassen, 1994), demonstrated analytically that particular types of embodied
movement generate patterns in the global array that carry information
about particular affordances. Specifically, they demonstrated that changes
in the optic array (such as might be presented to an otherwise stationary
observer) were intractably ambiguous with respect to the affordance that
participants were asked to perceive (i.e., whether a virtual object was
depicted as being within reach). That is, they showed that self-generated
exploratory activity was essential for the generation (and, therefore, the
pickup) of information in the global array.
Many classical illusions yield the intended conscious awareness only
when exploratory activity is restricted. The Ames room can be understood
as a three-dimensional depiction of an oddly-shaped room. The illusion
occurs when observers fail to differentiate the depiction from common,
rectilinear rooms (e.g., Runeson, 1988). The illusion works only when the
point of observation is fixed, whether through restraint of the head (e.g., a
chin rest), or of a camera. Movement of the point of observation rapidly
generates information about the actual geometry of the room, thereby
allowing observers to detect the depiction for what it is. The analysis of
Mantel et al. (2015) might be extended to the Ames room situation. We
could examine emergent patterns extending across optics and gravito
inertial force, generated by self-controlled motion of the head, that differ
as a function of the three-dimensional geometry of the displays. Such an
analysis could predict particular observer movements that should optimize
specification of the peculiar physical geometry of the physical Ames room
(cf. Riley et al., 2002). Similarly, in Mark, Balliett, Craver, Douglas, and
Fox (1990), the elimination of ordinary standing body sway was sufficient
both to prevent participants from detecting and learning about changes in
affordances for sitting. Body sway simultaneously (but differently) alters
the position and movement of the body relative to the illuminated environ-
ment, the acoustic environment, the gravitoinertial force environment, and
so on, thereby generating higher-order, emergent patterns in the global
array that appear to be sufficient for the perception of maximum sitting
height. While looking at a painting, sway likely generates patterns in the
global array that are unique to the two-dimensional geometry of the paint-
ing. As has been shown with manual wielding, the complex dynamics of
body sway vary as a function of changes in the goals of perception (e.g.,
Palatinus, Kelty-Stephen, Kinsella-Shaw, Carello, & Turvey, 2014; cf.
Riley et al., 2002), and so I predict that task-specific variations in sway
likely optimize global array parameters that specify particular aspects of
the animal-painting system.
Depiction Beyond the Illuminated Environment

Gibson (1979/2015) focused upon depictions in the illuminated environ-
ment; drawings, photographs, and the like. Yet depictions are found in
many environments. Depiction in the acoustic environment is as old as
in the illuminated environment. Just as some invertebrates create depic-
tions in the illuminated environment (Anderson et al., 2013), others
animals create depictions in the acoustic environment (e.g., Boppré,
Vain-Wright, & Wickler, 2017). Vocal mimicry is practiced by humans
(e.g., impersonations, which can be understood as vocal depictions of
other people), and many other species (e.g., Powys, Taylor, & Probets,
2013). Similarly, consider “sound effects”, that is, sounds that are
intended to resemble things other than their actual sources. A well-
known example is the “thunder sheet”, a large section of sheet metal
that, when struck with a mallet, yields a sound that resembles thunder.
Online sources provide fascinating descriptions of techniques used by
acoustic professionals to depict various events (www.epicsound.com/
sfx/). Sound design is a recognized field in various media, including
cinema, radio (Stoffregen, 1997), and the theater (e.g., Kaye, LeBrecht,
& Budries, 2016). Depiction extends also to the domain of aerosols (e.g.,
synthetic chemicals that “smell like” natural substances) and solutes
(e.g., synthetic flavors, such as “imitation vanilla”). A great deal of effort
and money has been invested in the attempt to find or create solutes that
(during ingestion) cannot be differentiated from sugar, and on substances
that have the same “mouth feel” as various comestibles. A complete
theory of depiction, and of affordances relating to depiction, must
include depictions beyond the illuminated environment.
Conclusion
In his Chapter 15, Gibson (1979/2015) covered many topics, only some of
which are addressed in the present chapter. For example, Gibson wrote
about how children learn to draw (pp. 262–266), and about conventions
of drawing (pp. 273–277). Figure 15.6 satirizes some of these conventions
(cf. Gibson’s Figures 15.3, 15.5, and 16.1). The development of skills of
depiction, and conventions that guide the creation of depiction are
important issues. Gibson’s insights on these topics have been given insuffi-
cient consideration.
Consistent with his peers, Gibson (1979/2015) defined depiction in
terms external to the observer, and candidate definitions were evaluated in
terms of conscious experience, or “visual awareness.” I have reviewed the
practice, in behavioral science research, of using depictions as substitutes
for un-depicted reality which, historically, is based upon the claim that
Figure 15.6 The original caption reads; Locating the vanishing point. This whimsi-
cal satire demonstrates that the conventions of linear perspective are
conventions, not laws.
Source: Plate from McCaulay (1978). Copyright ©1978 by David McCaulay. Reprinted by
permission of Houghton Mifflin Publishing Company. All rights reserved.
looking at the world is meaningfully related to looking at images. I have
suggested that an essential step in our understanding of depictions must be
to consider them in terms of the animal-environment system, with primary
emphasis on affordances that exist in such systems. I state that the
affordances of animal-environment systems that include depictions differ
from the affordances of animal-environment systems that do not include
depictions. My claims are developed from Gibson’s arguments that depic-
tions are differentiated from the corresponding, undepicted reality, and
that depictions have an uncertain relevance to general theories of percep-
tion and action. I do not propose that depictions can never be used in
experimental research on perception and action. Rather, I claim that the
fact of depiction is specified in global array patterns that are available to
research participants and that, depending upon the hypotheses being
tested, this fact may have consequences for the interpretation of experi-
mental data, and for any attempt to use such data to evaluate general the-
ories of perception-action. I have further suggested that conscious
awareness of depictions, while important, is a subset of a much larger class
of behaviors that are afforded in animal-depiction systems, and that a
general understanding of depictions will require us to examine the full
range of these affordances.
16 Revisiting Ecological Film Theory
Julia J. C. Blau
In the final chapter of his (1979/2015) text, Gibson laid out a plan for an
ecological theory of film perception. It should, he argued, be understood as
a presentation of a changing optic array. The information present in the
optic array displayed in a film is analogous to that encountered by a typical
observer in a natural scene: Overall optical expansion is generated by the
camera moving forward just as surely as it is generated by the organism
moving in the same way. As such, the audience would view these events as
if they were occupying the same position in space as the camera and have a
sense of being present in the depicted events.
However, Gibson (1979/2015) struggled to reconcile this egocentric
view with the inability on the part of the audience to in any way shape
those events. If the perceptual system is about moving to perceive, then
what do we say about an event that cannot be moved by the observer?
Moreover, he pointed out that “no one is ever wholly deceived” (p. 287)
into believing the events are actually happening in front of them––they do
not run from the cinematic monster (cf. Stoffregen, 1997), and they do not
feel as though their own head has turned when the camera has panned. I
believe it is possible to reconcile Gibson’s intuition––that the information
presented in a film is lawful––with the concerns about realness by drawing
some distinctions between perceiving depictions and perceiving depicted
events, as well as deepening the understanding of the technology involved.
Imagine that you are watching a film. The heroine draws her sword and
places the blade at the base of the villain’s throat; he grimaces but does not
back away; instead, the two glare angrily at each other. Assuming the film-
maker has done their job, you might be having a complex emotional
response to this moment. Her anger is righteous, and you empathize with
it. The villain probably deserves to die, but you are impressed by his
bravery and hope she shows mercy. Thunder rolls in the background, and
you feel a sense of dread.
There is a great deal to unpack here. First, the technology that produced
the film presents a depiction of events. That technology evolved, through
trial-and-error, to make the viewer perceive motion where there (techni-
cally) is none. The heroine draws her sword, and you perceive her arm
Revisiting Ecological Film Theory 275
move through the action as seamlessly as if she were in front of you.
Second, the story is having an emotional impact on you, specifically of an
empathetic type. You fear for the villain’s life, even as you understand and
are moved by the heroine’s fury. That emotional impact has been deliber-
ately crafted through the careful selection of which events will be presented
and in what fashion. And finally, you have noticed that the weather in the
scene is stormy, and since storms have preceded violence several times in
the film, you sense that things are not looking good for the villain. The
thoughtful use of consistent visual and auditory symbols has given you a
deeper insight to the scene than you might have had otherwise.
Because of the broad applicability of the ecological approach—and,
more importantly, parsimony—I contend that the ecological theory of per-
ception can explain the psychological phenomena present in all three of
these cases.
The Technology of Film

Proponents of the ecological perspective might be forgiven if they have a
somewhat uneasy relationship with the motion picture camera and projec-
tion apparatus––after all, it has served (ill-fittingly) as a metaphor for the
visual system to the detriment of the field. However, that metaphor is
based on a misunderstanding of the way cameras work and the process by
which they were designed.
Film cameras were not designed with a thorough understanding of how
human perception functions. Rather, filmmakers and inventors fiddled
with the technology until they got something that worked. In fact, as soon
as they got something that was watchable, they began using it to produce
films for an increasingly insatiable audience. Others have provided better
and more detailed histories of the development of the film camera (e.g.,
Anderson, 1996), but for our purposes, we need only a rough under-
standing of the technology and its development to understand why films
are able to do what they do.
The Technology of Recording

During filming, actors perform a scene (or, in ecological language, they
enact an event). Light reflected from the people and objects in the scene
reaches the aperture in the front of the camera (Figure 16.1a). Much like
the iris of the chambered eye, this aperture only allows through the light
that strikes it directly. The light passes through a lens that focuses the light
on a small, rectangular opening in a metal plate. The metal plate keeps the
rest of the photosensitive film (a thin sheet of plastic coated in chemicals)
from being exposed (Figure 16.1b).
Positioned between the lens and the metal plate is a rotating disk (called
a shutter) in the shape of a half-circle. The shutter rotates such that a full
276 Julia J. C. Blau
;ĂͿ ;ďͿ
Figure 16.1 (a) A camera receives light from a scene and focuses it, then (b) a spin-
ning shutter exposes each frame.
revolution takes 1/24th of a second. When the shutter is not covering the
opening in the metal plate, the film is held still, and the film is exposed.
The light from the scene strikes part of the film that is positioned directly
behind the rectangular opening. When the film is later treated with a
different chemical, the light causes a chemical reaction that darkens that
portion of the film—the more light that strikes the film, the more it
darkens. In color films, three layers of photosensitive film are pressed
together. Each has chemicals that are sensitive to different parts of the light
spectrum: red, green, and blue. The red layer would respond to differing
levels of red light, for example; the process is otherwise the same.
When the shutter spins to cover the opening, the light no longer reaches
the film. Using perforations on the sides of the film, the gears in the camera
advance the film to the next section (called a frame) ready to be exposed
when the shutter uncovers the rectangle once more. In this way, the film
does not smoothly and continuously move through the camera; rather, it
stops and starts 24 times per second. Importantly, the shutter is open and
exposing the film for a non-instantaneous amount of time. The film is
receiving light for about 1/48th of a second, meaning each frame is sam-
pling the optic flow for that entire time. Any movement during that
window will be recorded as such (more on this below).
The technology of film, film viewing, and filmmaking has changed––in
some ways drastically––since the publication of Gibson’s 1979 text. The
vast majority of independent filmmaking and even a large portion of pro-
fessionally made films are recorded by digital cameras instead of film. That
said, a digital camera works very similarly to a film camera. Instead of
photosensitive film, the light is focused on a computer chip which converts
the amount of light into a digital print (stored in binary: 1s and 0s). Instead
of a spinning shutter, the camera turns the sensitivity of the chip on and
off in the same pattern: record for 1/48th of a second, stop recording for
1/48th of a second, repeat. These data are then stored on either magnetic
tape (like you might find in a VHS tape) or (more commonly) in the hard
drive of the camera.
The Technology of Playback

The photosensitive film that has been exposed during recording is called a
negative because there is a tonal reversal of light: Where the light was the
brightest in the photographed scene, the film is the darkest; where the light
was the darkest, the film is transparent (see Figure 16.2b). From the neg-
ative, a positive print is made. The negative is fed with a second reel of
photosensitive film through a device that shines a bright light through the
negative and onto the film. Where the negative is dark (i.e., where the
brightest light was in the original scene), a shadow is cast on the film.
Where the negative is transparent (i.e., where the light was the dimmest in
the original scene), no shadow is cast, and the light gets through. Once
again, the light causes the film to darken.
The positive print looks like the original scene: Areas of light in the ori-
ginal scene are now transparent, and areas of dark in the original scene are
now dark (see Figure 16.2c). The positive print is fed through a projector
(Figure 16.3a), where a metal plate with a rectangular hole keeps the film
in the right location to illuminate the image. The projector shines a bright
light through the film: Where the film is transparent, the light passes
through, and where the film is dark, it casts a shadow on the screen.
Just as in the camera, a spinning shutter is positioned between the light
source and the metal plate. This shutter, however, is divided into six
sections––three that block the light and three that allow the light through
(Figure 16.3b). The shutter still spins a full revolution in 1/24th of a
second, but this results in the image being projected onto the screen three
;ĂͿ ;ďͿ ;ĐͿ
Figure 16.2 (a) The original scene (in grayscale); (b) the negative; (c) the positive
print.
;ĂͿ ;ďͿ
Figure 16.3 (a) A bright light is projected through the positive print; (b) a spinning
shutter exposes then blocks each frame three times.
times, for 1/144th of a second, with a 1/144th of a second of no light

between each. When the third shutter blade is covering the image, the film
advances to the next frame. Put simply, the projector presents 24 frames
per second, but each frame is presented three times, for a total of 72 flashes
of light per second with momentary darkness in-between each. In other
words, for half of the time, the theater is in total darkness.
Digital projectors are far more complicated. While there are many
different types of digital projectors, the end result (i.e., the light on the
screen) is the same for all of them. In general, the digital frame that was
recorded by the camera is converted to three black and white images and
sent (via a complicated series of mirrors) to three light sources. White light
is fed through a prism and split into red, green, and blue light. These
images are like the positive print of the film camera in that the dark por-
tions cast a shadow on the screen (blocking, say, green light), and the
transparent sections allow the light through. The three images are recom-
bined using a prism, and the result is projected onto the screen. The pre-
sentation of images is ultimately the same as the film projector: 72
presentations of 24 images, with darkness in-between each.
Televisions and computer screens work very differently from these pro-
jectors. Philo Farnsworth, the inventor of the television, was reportedly
inspired by watching a farmer plowing his field––going back and forth
methodically down the rows until the entire field was plowed. Like the
farmer, the screen presents the light of the images in rows––the number of
rows depends on the screen, with more rows providing a crisper image (the
first TVs had 525 rows, while modern TVs have upwards of 2,160). Unlike
the farmed field, these rows are not perfectly horizontal but instead go
back and forth diagonally downwards until they reach the bottom of the
screen (Figure 16.4). The screen can either be progressive (i.e., only one
Figure 16.4 The scanning pattern of a television or computer screen. The screen

presents the light of the image back and forth (black arrows) from the
top of the screen to the bottom. In a progressive screen, it subsequently
presents the image in the alternate rows (gray dashed arrows). In an
interleaved screen, the alternate rows are presented simultaneously.
signal bouncing back and forth at a time) or interleaved (i.e., two signals
bouncing back and forth, one from the first image, one from the second
image). To make things more complicated, the TV typically presents 30
images per second1 but presents each image twice, once on each of the two
alternate lines.
Broadly, there are two important differences between screens (either TV
or computer) and projectors (film or digital) that will be at issue in our dis-
cussions of film perception. First, for screens there is no full darkness: the
screen is always projecting some light. Second, unlike the projector, there
is never a full image presented on screens––only one or two small lines.
Flicker Fusion and Perception of Motion

When considering how humans observe films, it is important to draw a firm
distinction between two perceptual phenomena: illusion of motion and
flicker fusion. The former is perception of smooth motion from a series of
rapidly-presented still photographs. Flicker fusion (Simonson & Brozek,
1952) or the illusion of succession (Barsam & Monahan, 2004/2016) is the
perception of a continuous stream of light, even though the light is actually
flickering rapidly. Early films did not quite work this out, and they would
have continuous motion but not continuous light. In other words, the
heroine would seem to draw her sword smoothly, but the image would
appear to be flickering (which is why films are sometimes called “flicks”).
Film theory frequently conflates these two phenomena, and typical
explanations rest on some version of persistence of vision. For example,
this quote from a contemporary Introduction to Film textbook describes
persistence of vision as:
the process by which the human brain retains an image for a fraction
of a second longer than the eye records it. You can observe this phe-
nomenon by quickly switching a light on and then off in a dark room.
Doing this, you should see an afterimage of the objects in the room.
(Barsam & Monahan, 2004/2016, pp. 47–48)
Leaving aside the conflation of retinal afterimage and cortical processes for
the moment, this passage is also supposed to be an explanation for why we
can see motion in a series of still photographs. Rather than pointing out
the fallacy in such an explanation––after all, Anderson and Anderson
(1993) have already provided a logical deconstruction of the topic––I will
instead present a more ecological take on the two concepts.
Flicker fusion is a physiological phenomenon, roughly equivalent to
making sure there is enough light in the room to stimulate the eye’s rods
and cones (i.e., above the absolute threshold). That is, it is not about
detecting information for affordances or events, rather it is about the pre-
sentation rate of light exceeding the minimum temporal threshold of our
retinal cells. While our perceptual system as a whole does not sample dis-
cretely (it samples the optic flow continuously), individual neurons are
limited in their ability to continuously fire. After being stimulated by light,
there is a refractory period during which––no matter how much stimu-
lation is presented to it––a retinal cell cannot fire. As long as light reaches
the cell before this refractory period is over, the cell cannot tell the differ-
ence between continuous and discontinuous light.
Flicker fusion happens at roughly 50 flashes per second, well below the 72
flashes per second used by film projection (Berry & Meister, 1998; Simonson
& Brozek, 1952). There is a flash of light, the retinal cell fires in response,
and by the time it is ready to fire again, more light has been presented. Or in
other words, the retinal cells are firing as often as they possibly can––with or
without constant presentation of light. When presented with continuous light,
the retinal ganglion cells do not fire all at once; some fire while others are in
their refractory period. When presented with flickering light, however, they
synchronize with the flicker (Berry, Warland, & Meister, 1997).
Perceiving motion while watching a series of purportedly still photo-
graphs is a far more interesting and far more––forgive the pun––illuminating
phenomenon. It is true that there is no actual motion on the screen. But to
be fair, there is no actual heroine, no sword, no villain either. However,
the information for all of these is presented. The traditional explanations
of persistence of vision make two (largely incorrect) assumptions. The first
is that the eye (or the cortex, depending on the theorist) is replicating––and
holding onto––an image to compare to the next incoming image. The
second is that because the images presented are individually still, all motion
perception must be an inference.
The first assumption is patently false. If a complete image were required
for persistence, then the television (or computer) screen would not work. A
full image is never presented, only a thin line (or two). Unless you are willing
to posit two different visual mechanisms––one for theater projection and one
for television––this cannot be. Additionally, the eye is never passive, even
when sitting purportedly still in a movie theater. If film motion perception
required point-by-point comparison of an incoming image to the one captured
just previously, those points would have to be positioned in the exact same
part of the retina: an impossible task for a continuously moving eye.
The second assumption would require that the shutter on a camera be
open for an instantaneous length of time. Or in other words, if the shutter
is open for even a fraction of a second, the photoreceptive film will capture
light from that entire time period. If there is movement, optic flow will be
captured in the image (see Figure 16.5). Research demonstrates that even a
portion of the optic flow is sufficient to specify the motion of the self or of
objects in the scene (Bardy, Warren, & Kay, 1999). Why would perception
of motion in film be any different? Perceiving motion in film, then, is an
illusion in the truly Gibsonian sense of the word (Turvey, Shaw, Reed, &
Mace, 1981) in that motion is lawfully specified, and so we see motion.
The camera is sampling the optic flow that has to have been generated
lawfully. As Stoffregen (1997) explained:
If the optic array is a product of physical law, then its structure cannot
bear a paradoxical relation to physical reality, any more than physical
law can produce a paradox. Similarly, if films are created in accord-
ance with physical law, then they cannot structure optic arrays in ways
that are incompatible or paradoxical.
(p. 166)
Figure 16.5 (a) No blur specifies the ball (and camera) are still. (b) Global blur
(global optic flow) specifies the camera is moving. (c) Localized blur
(localized optic flow) specifies the ball is moving.
The last part of this quote brings up an interesting point: Not all film or
television is produced in accordance with physical law. Digital animation,
computer generated imagery (CGI), stop-motion animation, and even the
humble cartoon are all presenting artificially-generated events that are not
necessarily subject to physical law.
Take stop-motion animation: small figures are made out of clay, and
positioned in front of a still camera which takes a photograph, the clay
figure is moved a very tiny amount, another photograph is taken. This
process repeats until the photos can be strung together and presented
through a film projector. The clay figure appears to move!
As this technology developed, the filmmakers learned very quickly that
the movements of the clay figures appeared jerky, particularly when they
were supposed to be moving at higher speeds (Brostow & Essa, 2001). The
solution was to introduce artificial motion blur (see Figure 16.5), which
ecological psychologists would call optic flow. When filming a moving
subject, the slight trace of light left by that motion on film is lawfully gen-
erated and so motion is specified. When the motion is artificially generated,
however, the same cannot be said (the clay figure is still when its photo is
taken). To make the motion appear fluid, you have to specify that the char-
acter continues to move during the time that the shutter on the camera is
open, or in other words, you have to introduce blur (Andreev, 2010; Dai
& Wu, 2008; Kawagishi, Hatsuyama, & Kondo, 2003).
Editing
After the actors have completed their scene, but before the audience can
watch a finished film, editing takes place. Typically, scenes are filmed from
many different angles, many different times, and the best bits are strung
together into a coherent narrative. Putting it so simply, however, may give
the false impression that this task is simple. Editing the movie often takes
far longer than filming it did (Singleton, 1991)—up to six months for films
without special effects and CGI, longer for those with them. Deciding what
to include, what to exclude, and even what is missing and needs to be re-
filmed is an incredibly difficult art.
Orientation to a Scene
The most fundamental question of editing is: Why does discontinuous film-
making (as opposed to continuous filmmaking, where the entire story is
captured in one shot) work at all? In our scene with the heroine and the
villain, the camera jumps instantaneously across a cut (see Figure 16.6).
Why does this not bother the audience? If we argue that the audience per-
ceives these events by virtue of the lawful relationship between the pre-
sented information and the events to be perceived, then the audience
should feel as though they have been instantaneously transported from
Figure 16.6 Typical editing structure. (a) Two shot (also known as an establishing
shot); (b) over-the-shoulder from the villain’s perspective; (c) over-the-
shoulder from the heroine’s perspective. The camera does not move
continuously from one position to the next, but rather makes a discon-
tinuous jump (at the edit point).
kneeling next to the villain, to standing next to the heroine—and yet, they
do not. In fact, viewers frequently fail to notice editing altogether (Ander-
son, 1996; Murch, 2001).
The ecological approach has an answer to this question: Editing works
because we are oriented in the Gibsonian sense. Murch (2001) is deeply
mistaken when he asserts that “[n]othing in our day-to-day experience
seems to prepare us for such a thing.” Orientation is fundamental to daily
perception and fairly clearly demonstrated in the way that films are edited.
The passage where Gibson (1979/2015) describes the process of orienta-
tion even sounds like it could be a page out of a film textbook:
When the vistas have been put in order by exploratory locomotion, the
invariant structure of the house, the town, or the whole habitat will be
apprehended. The hidden and the unhidden become one environment
… One is oriented to the environment. It is not so much having a
bird’s-eye view of the terrain as it is being everywhere at once. The
getting of a bird’s-eye view is helpful in being oriented, and the
explorer will look down from a high place if possible.
(p. 189)
Filmmakers call this process “establishing” rather than orienting (see

Figure 16.6), but the idea is the same (Anderson, 1996). Filmmakers will
start scenes with a wide view of the relevant layouts––even going so far as
to show a view of an entire city (a bird’s-eye view) or the outside of the
building in which the next scene takes place––before moving the camera
closer to the characters. Once the observer is oriented, the camera can
occupy any position, and––provided the orientation is not violated––the
observer will be able to detect the relevant information for the events.
Selection of Events: The World Line
In a well-made film, events are presented in a realistic, empathy-generating
fashion. This is true even for movies set in non-realistic settings (e.g.,
fantasy or science fiction genres). Gibson (1979/2015) argued that our
impression of realism in films comes from the movements of the camera
mimicking “natural movements of the head-body system” (p. 285). He
suggested we detect the information in films the same as when we encoun-
ter it as actor/perceivers. When the camera pans, it specifies that the head
has turned. Or in other words, we view films from an egocentric per-
spective, seeing the events as our own as if we were silent onlookers.
Because of this, Gibson suggested that the subjective camera should not
be so neglected. The subjective camera is when the camera acts as a char-
acter in the story, occupying space and engaging in events as that character
would. If Gibson’s assertion that the presented events are perceived as ours
were correct, the subjective camera should make the experience more
engaging, more realistic. But Gibson overlooked one important fact: The
subjective camera has been neglected because it does not work!
There have been many attempts over the years to use the subjective
camera effectively. (For relatively recent examples, see “The Ghost in the
Machine” [a 2012 episode of the show, Bones] or Hardcore Henry [the
2015 action film]).2 However, there are good reasons that it has never
caught on. Watching a film made with a subjective camera is awkward,
uncomfortable, often motion sickness-inducing. It feels forced and arti-
ficial. As Stoffregen (1997) points out, with rare exceptions, films are
limited to optical and auditory information. Without kinesthetic feedback,
the information presented by the film will always be incomplete (see also
Stoffregen & Bardy, 2001). The presented information may specify that a
head has turned, but it can never specify that your head has turned. The
subjective camera makes that discrepancy very stark, and so the events
cease to be realistic.
The failure of the subjective camera suggests that films are typically not
attempting to present an egocentric view to the audience. Instead, films take
advantage of our ability to adopt an allocentric view (Mark, 2007). The eco-
logical perspective is very clear on this point: Orientation allows us to under-
stand the layouts and surfaces we are not currently directly witnessing:
The theory asserts that an observer can perceive the persisting layout
from other places than the one occupied at rest. This means that the
layout can be perceived from the position of another observer. The
common assertion, then, that “I can put myself in your position” has
meaning in ecological optics and is not a mere figure of speech. To
adopt the point of view of another person is not an advanced achieve-
ment of conceptual thought.
(Gibson, 1979/2015, p. 191)
In other words, because of the fully public nature of information, we are
capable of perceiving the events and affordances of other people (Creem-
Regehr, Gagnon, Geuss, & Stefanucci, 2013; Mark, 2007), even when
those people are presented in kinematic displays (e.g., Stoffregen, Gorday,
Sheng, & Flynn, 1999).
I agree with Stoffregen (1997; Chapter 15, in this volume) that what we
perceive when we watch a movie is information specifying a depiction. I
further argue that well-made films are a depiction of someone else’s
events—more specifically, a subset of someone else’s world line. A world
line (Kugler, Turvey, Carello, & Shaw, 1985) is a collection of events and
affordances that are meaningful to a specific person—a collection that,
taken together, constitutes the complete narrative of that person’s life.
At any given moment, an organism is surrounded by a broad array of
information specifying an impossibly large number of events and
affordances (see Shaw & Kinsella-Shaw, Chapter 6, in this volume;
Wagman, Chapter 8, in this volume). An organism does not attend to this
entire infinite set, however, they selectively attend to those events directly
relevant to their current intentions and needs. As such, the selection of
attended-to events says something about the internal state of the person
attending to them (Shaw, McIntyre, & Mace, 1974). From this perspective,
event perception is a reflection of the organism (Bingham, 2000) or, for
these purposes, the character. If the villain—instead of paying attention to
the sword at his throat—is paying attention to the set of keys tied to the
heroine’s belt, that tells the audience something about the character’s state
of mind.
I assert that an immersive narrative is created by presenting a coherent
(and coherently nested) series of events and affordances, meaningful to a
particular entity, usually a character within the story. Admittedly, not all
films are told from the perspective of one person. When they are not, they
are often told from the perspective of a group of people or switch between
distinct subsets of characters. In such cases, the presented world line is the
events and affordances they share. Regardless, the events garner empathy
in precisely the same way that other people do in non-cinematic life: We
experience secondhand sadness in dramatic films and secondhand embar-
rassment in cringe comedy. We do not run from the monster that is, after
all, not threatening us, but we urge the character on screen to do so. We
understand that the events are theirs.
Nested Event Structure: Event Selection

The job of an editor is to choose those events that specify the story (the
narrative) being told. A brief narrative need not—indeed, should not—
include all the events included in a character’s world line, but it should not
deviate from those events. The world line of a particular character is a
subset of the infinite set of all events, and a story must be a subset of the
Figure 16.7 Nested event structure. A person or character’s world line is entirely

contained within the infinite set, the narrative is entirely contained
within the world line.
character’s world line (Figure 16.7). The editor (working with the director)
selects those events that will give us the desired insight into a character,
building empathy while advancing the plot (the narrative set).
The phrase “killing your babies” refers to times during the editing
process when the editor must cut out a beloved scene. This could be a
scene in which the acting was particularly good, or a shot was particularly
beautiful, or the dialogue was particularly amusing. These scenes will last
until the very last stage of editing, surviving cut after cut. But every time
the editor and director watch the movie, that scene brings the rhythm of
the movie to a screeching halt. They may not even be sure why: They just
know that it does, and so the scene is removed.
“Killing your babies” is an interesting phenomenon that offers insight
to the process of selecting which events to present in a film. From an ecolo-
gical perspective, I believe it is prompted by a need to fix incoherent
nesting. For example, an editor might cut a scene where the villain uses
technology that did not exist at the time of the movie (i.e., the scene vio-
lates the specified infinite set) or a scene between the captain of the hero-
ine’s guard and his wife (i.e., the scene violates the specified world line
because it is neither the heroine nor the villain’s event). Understanding the
last kind of incoherent nesting––violating the story set––is a more subtle
art, one that sets professional filmmakers apart from amateurs.
Which events need to be presented to specify a story? Trouble arises
whenever the filmmakers include an unnecessary event or exclude a neces-
sary one. The problem with the latter is obvious: If the audience does not
have enough information to understand the sequence of events, they will
be lost. The difficulty with the former is less obvious. After all, in non-
cinematic life, events irrelevant to our current needs are literally happening
all the time. Why would it be problematic in a cinematic context to show
too much?
The answer lies in the fact that the audience has an unspoken contract
with the filmmakers. The filmmaker promises to show all information
needed to specify the relevant events, and the audience promises to believe
that (Gibson, 1979/2015; Proffitt, 1977; Willats, 1990). Including an event
without specific relevance to that narrative leaves the audience frustrated.
Chekhov’s (1889) oft-used trope perhaps best exemplifies this concern:
“One must not put a loaded rifle on stage if it isn’t going to go off. It is
wrong to make promises you don’t mean to keep” (p. 163). Put another
way, if you include a gun and it does not go off, you have essentially put in
an open parenthesis (i.e., the beginning to an event) and never a closing
parenthesis (i.e., the end of the event). A gun makes this incompleteness
particularly obvious because it is such a salient object, but any event
without later relevance will have precisely the same effect on the audience.
If the event in question is not absolutely essential to the understanding of
the overarching narrative, it ought to be removed.
Symbolism: Nonspecifying Constraints

In our scene, the heroine raises her sword to the villain’s throat, there is a
crash of thunder, which elicits a feeling of dread: Something bad is about
to happen. Were the scene taking place on a bright sunny day, it would not
provoke the same feelings. What is happening?
Symbols and symbolism3 can be described as the arbitrary assignment of
meaning to something, such as objects, events, or sounds. The connection
between the event (say, a storm) and the meaning (say, a violent emotion)
is purely coincidental outside of stories; and yet, in stories, we come to rely
on a storm as a signal for violent emotions, on owls as a signal for wisdom,
and so on. Gibson (1979/2015) believed the ecological account should be
able to handle “the whole realm of social significance for human beings”
(p. 120; see also Bruineberg, Chemero, & Rietveld, 2018), and so an eco-
logical account of literary symbolism is needed.
The concern with embracing literary symbolism in an ecological per-
spective is that it is not strictly lawfully based. As a result, they begin to
sound like cues, from which a higher cognitive process would build a
representation of meaning. This directly contradicts the ecological
approach, which insists on a lawful structuring of the array, allowing for
information detection, rather than information processing.
However, I do not believe understanding human sensitivity to literary
symbolism has to contradict this view, particularly when the perspective
does not wish to loan intelligence to any organism. Absent such a loan, an
organism cannot know a priori which information will be lawful and
which will be informative but nonspecifying (see Withagen, 2004). More-
over, our own infinite set’s constraints can change––today I drove to work
at speeds that would have been impossible a century ago. We need to be
adaptable, which means being sensitive to new regularities. There are
experimental examples of perceivers attuning to information gradients that
are artificially linked to an environmental property: Participants quickly
learn to navigate a maze blindfolded using vibrotactile feedback on the
tongue (Chebat, Maidenbaum, & Amedi, 2015; Chebat, Rainville, Kupers,
& Ptito, 2007) or to estimate distances using the frequency of sound
(Maidenbaum et al., 2014). In such cases, Bruineberg et al. (2018) sug-
gested we are using general ecological information, or a “regularity in the
ecological niche between aspects of the environment, x and y, such that the
occurrence of aspect x makes the occurrence of aspect y likely” (p. 7). Or
in other words, any sufficiently regular connection is enough to constrain
behavior. Literary symbols are an example of informative but nonspecify-
ing variables.
Filmmakers are deeply aware of the tendency of humans4 to attend to
regularities in our informational surround. They pick and choose which
information is presented with surgical care. Literary symbolism arises
when coincidence is made regular: If every single time a circle is present on
the screen something horrifying happens (as is the case in the 2002 film,
The Ring), then circles will begin to elicit fear in the audience. If the color
red is present every single time the supernatural world imposes on the
natural one (as is the case in the 1999 movie, The Sixth Sense), then the
color red will have the audience looking around for the ghost. In the speci-
fied infinite set of the movie, those regularities are meaningful.
The Sound of Films

Gibson’s (1979/2015) text did not address the auditory world of films
(understandably, given it was a text on visual perception), and in general,
this is an under-researched area. Several perceptual puzzles in this domain
are worthy of investigation. In the early days of filmmaking, technology
did not allow for the sound of the action to be recorded simultaneously
with the visual.5 This led to “silent films,” where you could not hear the
dialogue of the actors. This is a misnomer, however, as they were anything
but silent (Geduld, 1975). Before sound technology advanced to the point
of synchronized sound, theaters would either play gramophone records of
popular music or hire piano players to play live music during the film. Or
in other words, there was a near-immediate realization that films could not
actually be silent; watching in silence is uncomfortable. This is in keeping
with the idea that watching films is replicating the experience of viewing
someone else’s events. Provided you are close enough to observe their
events, you would both see and hear them; not having access to that
information gradient when you feel as though you should causes
discomfort.
Sound Balancing
Once synchronized sound was perfected (in the early 1900s), an entire
branch of technical expertise was created: the sound editor (or later, sound
engineer or sound designer). These technicians quickly realized that simply
placing a microphone in the center of the room will not result in a sound-
track worth listening to. Films selectively present visual events to specify
the narrative; the same is true for auditory events.
In non-cinematic life, we selectively attend to relevant affordances speci-
fied by the auditory array. We will not notice the sound of the fan while
we are listening to our colleague speak. A microphone is not capable of
such selective attention, and a speaker will play the sound of the fan as
loudly as the sound of the words, making the latter inaudible. To replicate
the experience of viewing someone else’s events, sound must be balanced.
That is, it must have the most important sounds be the loudest and the
least important be the quietest.
Sound Effects
In addition to dialogue and music, the majority of films include sound
effects. Sound effects are sounds that are not recorded during filming but
are added later (in post-production). There are several different kinds of
sound effects, but here I would like to focus on background (sometimes
called ambience or walla) and Foley effects.
Background sound effects are the incidental sounds that surround us but
to which we do not attend. We attend to the sound of the villain pleading
for his life but not the sound of the birds chirping and the nearby guards
talking quietly to each other. Typically, the source of these sounds is either
not present (no birds are seen) or ambiguous (it is not clear which guards
are speaking). While proper sound balancing would ensure that these
sounds would be quiet, it would also make sure they were not absent. A
scene without background noise feels artificial, as if it were filmed in a
tin can.
Foley effects (named after Jack Foley, their inventor) are sounds that
link up with a specific event happening on the screen (e.g., a sword being
drawn from its scabbard, or rain falling). Oddly, not only are they are not
recorded with the scene, they are often created by recording an entirely
different event. In actuality, a metal sword being drawn from a leather
scabbard makes very little noise. However, a film will feel wrong without a
noise to punctuate that moment, so a recording of a kitchen knife being
drawn across a hammer head will stand in. Carello, Wagman, and Turvey
(2005) suggest that Foley effects have some “shared properties” (p. 100)
with the sounds they are meant to mimic, and this may be so, but then
why would recording the actual sound (say, rain falling) create unrealistic
sounds (recorded rain sounds like popping), while recording an entirely
different event (salt falling on tinfoil) creates a more realistic sound?
Both background and Foley sound effects are somewhat puzzling from a
perceptual standpoint. Background effects are only noticeable when they
are absent; in a sense, they need to be there so they can disappear. Foley
effects often only sound realistic when they are recorded in a fully unrealis-
tic fashion. The psychology behind this is not well understood, and given
the profound effect they have on whether a film “works” or not, more
research is certainly warranted.
Conclusion
A great deal of work remains to be done to complete an ecological
accounting of film perception. I argue that the beginnings of that account
should start with the findings of filmmakers. Filmmakers are scientists,
whether they intend to be or not, and they have been performing experi-
ments on the human perceptual system for generations. As ecological psy-
chologists, we should at least attempt to understand their findings.
Whether in the context of perceiving depictions (see Stoffregen, Chapter
15, in this volume) or perceiving natural scenes, what “works” for film
offers insights into human perception—should we choose to listen.
Notes
1. Computer screen image presentation rates (called the refresh rate) are often far
higher, which is important for video game programmers and scientists trying to
present stimuli very quickly.
2. Examples from video games are far more common, likely because those allow
you to actually manipulate events. However, cut scenes (over which the player
has no control) almost exclusively switch to a third-person point of view.
3. I am using the literary terms “symbolism” and “symbols” quite broadly here.
The psychological principles here described would apply similarly to any similar
terms and so the distinctions are superfluous to this discussion. I chose to use
these terms (as opposed to terms used in psychological literature such as “sign”)
because these are the terms typically used in literature and film criticism.
4. This applies to other organisms as well, but filmmakers are not terribly con-
cerned with the ability of, say, a dog in this respect.
5. This was for a variety of reasons but mostly because of the difficulty in synchro-
nizing the different mechanisms being used for visual and auditory recording
and playback (Ulano, 2009) and quality of recording (Crafton, 1999).
References
Aanondsen, C. M., Van der Meer, A. L. H., Brubakk, A.-M., Evensen, K. A. I.,
Skranes, J. S. Myhr, G. E., & Van der Weel, F. R. (2007). Differentiating pro-
spective control information for catching in at-risk and control adolescents.
Developmental Medicine and Child Neurology, 49, 112–116.
Adcock, C. (1990). James Turrell: The art of light and space. Berkeley, CA: Univer-
sity of California Press.
Adelson, E. H. (2001). On seeing stuff: The perception of materials by humans and
machines. In B. E. Rogowitz & T. N. Pappas (Eds.) Proceedings of the SPIE, Vol.
4299 Human Vision and Electronic Imaging VI (pp 1–12). doi:10.1117/12.429489.
Adelson, E. H., & Bergen, J. R. (1991). The plenoptic function and the elements of
early vision. In M. Landy, & J. A. Movshon (Eds.), Computational models of
visual processing (pp. 3–20). Cambridge, MA: MIT Press.
Adolph, K. E. (2008). Learning to move. Current Directions in Psychological
Science, 17, 213–218.
Adolph, K. E., & Berger, S. E. (2015). Physical and motor development. In M. H.
Bornstein & M. E. Lamb (Eds.), Development science: An advanced textbook (7
ed., pp. 261–333). New York: Psychology Press.
Adolph, K. E., Eppler, M. A., & Gibson, E. J. (1993). Crawling versus walking
infants’ perception of affordances for locomotion over sloping surfaces. Child
Development, 64, 1158–1174.
Adolph, K. E., Eppler, M. A., Marin, L., Wiese, I. B., & Clearfield, M. W. (2000).
Exploration in the service of prospective control. Infant Behavior & Develop-
ment, 23, 441–460.
Adolph, K. E., & Hoch, J. E. (2019). Motor development: Embodied, embedded,
enculturated, and enabling. Annual Review of Psychology, 70, 26.1–26.24.
Adolph, K. E., & Joh, A. S. (2009). Multiple learning mechanisms in the develop-
ment of action. In A. Woodward & A. Needham (Eds.), Learning and the infant
mind (pp. 172–207). New York: Oxford University Press.
Adolph, K. E., & Kretch, K. S. (2012). Infants on the edge: Beyond the visual cliff.
In A. Slater & P. Quinn (Eds.), Developmental psychology: Revisiting the classic
studies. London: Sage Publications.
Adolph, K. E., & Kretch, K. S. (2015). Gibson’s theory of perceptual learning. In
H. Keller (Ed.), International encyclopedia of the social and behavioral sciences
(2 ed., vol. 10, pp. 127–134). New York: Elsevier.
Adolph, K. E., Kretch, K. S., & LoBue, V. (2014). Fear of heights in infants?
Current Directions in Psychological Science, 23, 60–66.
292 References
Adolph, K. E., & Robinson, S. R. (2015). Motor development. In L. Liben & U.
Muller (Eds.), Handbook of child psychology and developmental science (7th
ed., vol. 2 Cognitive Processes, pp. 113–157). New York: Wiley.
Agyei, S. B., Holth, M., Van der Weel, F. R., & Van der Meer, A. L. H. (2015).
Longitudinal study of perception of structured optic flow and random visual
motion in infants using high-density EEG. Developmental Science, 18, 436–451.
Agyei, S. B., Van der Weel, F. R., & Van der Meer, A. L. H. (2016a). Development
of visual motion perception for prospective control: Brain and behavioral studies
in infants. Frontiers in Psychology, 7, 100.
Agyei, S. B., Van der Weel, F. R., & Van der Meer, A. L. H. (2016b). Longitudinal
study of preterm and full-term infants: High density EEG analyses of cortical
activity in response to visual motion. Neuropsychologia, 84, 89–104.
Altenhoff, B. M., Pagano, C. C., Kil, I., & Burg, T. C. (2017). Haptic distance-to-
break in the presence of friction. Journal of Experimental Psychology: Human
Perception and Performance, 43, 231–244.
Andersen, G. J., & Braunstein, M. L. (1983). Dynamic occlusion in the perception
of rotation in depth. Perception & Psychophysics, 34, 356–362.
Anderson, J. D. (1996). The reality of illusion: An ecological approach to cognitive
film theory. Carbondale, IL: Southern Illinois University Press.
Anderson, J. D., & Anderson, B. (1993). The myth of persistence of vision revis-
ited. Journal of Film and Video, 5, 3–12.
Anderson, R. C., Mather, J. A., & Wood, J. B. (2013). Octopus: The ocean’s intel-
ligent invertebrate. Portland, OR: Timber Press.
Andreev, D. (2010). Real-time frame rate up-conversion for video games: or how
to get from 30 to 60 fps for free. In ACM SIGGRAPH 2010 Talks, 16. ACM.
Aristotle. (1907). De anima. In R. D. Hicks (Ed.), Aristotle de anima with trans-
lation, introduction and notes. Cambridge, MA: Harvard University Press.
Atiyah, M. (2007). Duality in Mathematics and Physics, lecture notes from the
Institut de Matematica de la Universitat de Barcelona (p. 69). Barcelona: Univer-
sity of Barcelona.
Atkinson, J., & Braddick, O. (2007). Visual and visuocognitive development in
children born very prematurely. Progress in Brain Research, 164, 123–149.
Austad, H., & Van der Meer, A. L. H. (2007). Prospective dynamic balance control
in healthy children and adults. Experimental Brain Research, 181, 289–295.
Avant, L. (1965). Vision in the Ganzfeld. Psychological Bulletin, 64, 246–258.
Bache, C., Kopp, F., Springer, A., Stadler, W., Lindenberger, U., & Werkle-
Bergner, M. (2015). Rhythmic neural activity indicates the contribution of atten-
tion and memory to the processing of occluded movements in 10-month-old
infants. International Journal of Psychophysiology, 98, 201–212.
Baggs, E. (2015). A radical empiricist theory of speaking: Linguistic meaning
without conventions. Ecological Psychology, 27, 251–264.
Baggs, E. (2018). A psychology of the in between? Constructivist Foundations, 13,
395–397.
Baggs, E., & Chemero, A. (2018). Radical embodiment in two directions. Synthese.
doi:https://doi.org/10.1007/s11229-018-02020-9.
Bambach, S., Franchak, J. M., Crandall, D. J., & Yu, C. (2014). Detecting hands in
children’s egocentric views to understand embodied attention during social inter-
action. Proceedings of the 36th Annual Meeting of the Cognitive Science
Society, 36.
References 293
Bambach, S., Smith, L. B., Crandall, D. J., & Yu, C. (2016). Objects in the center:
How the infant’s body constrains infant scenes. Paper presented at the IEEE 6th
Joint International Conference on Development and Learning and Epigenetic
Robotics.
Barac-Cikoja, D., & Turvey, M. T. (1991). Perceiving aperture size by striking.
Journal of Experimental Psychology: Human Perception and Performance, 17,
330–346.
Bardy, B. G., Warren, W. H., & Kay, B. A. (1999). The role of central and periph-
eral vision in postural control during walking. Perception & Psychophysics, 61,
1356–1368.
Barker, R. G. (1968). Ecological Psychology. Stanford, CA: Stanford University
Press.
Barsam, R., & Monahan, D. (2004|2016). Looking at movies: An introduction to
film. (5th ed.). New York: W. W. Norton & Company.
Battro, A. M., Netto, S. P., & Rozestraten, R. J. A. (1976). Riemannian geometries
of variable curvature in visual space: Visual alleys, horopters, and triangles in big
open fields. Perception, 5, 9–23.
Baylor, D. A., Lamb, T. D., & Yau, K.-W. (1979). Responses of retinal rods to
single photons. Journal of Physiology, 288, 613–634.
Benedikt, M. (in preparation). Architecture beyond experience. University of Texas
at Austin.
Benedikt, M., & Burnham, C. (1985). Perceiving architectural space: From optic
arrays to isovists. In W. H. Warren, Jr. & R. E. Shaw, Persistence and change:
Proceedings of the first international conference on event perception. (pp.
103–114). Mahwah, NJ: Lawrence Erlbaum Associates.
Bennett, K. B. (2017). Ecological interface design and system safety: One facet of
Rasmussen’s legacy. Applied Ergonomics, 59, 625–636.
Bennett, K. B., & Flach, J. M. (2011). Display and interface design: Subtle science,
exact art. London: Taylor & Francis.
Bentley, A. F. (1954/1975). The fiction of “retinal image.” In A. F. Bentley & S.
Ratner (Eds.), Inquiry into inquiries: Essays in social theory (pp. 268–285).
Boston, MA: Beacon Press.
Berger, C. C., Gonzalez-Franco, M., Ofek, E., & Hinckley, K. (2018). The uncanny
valley of haptics. Science Robotics, 3, eaar7010.
Bergson, H. (1889). Time and free will: An essay on the immediate data of con-
sciousness. New York: Dover Publications.
Bernstein, N. (1967). The coordination and regulation of movements. London: Per-
gamon.
Berry II, M. J., & Meister, M. (1998). Refractoriness and neural precision. Journal
of Neurosciences, 18(6), 2200–2211.
Berry, M. J., Warland, D. K., & Meister, M. (1997). The structure and precision of
retinal spike trains. Proceedings of the National Academy of Sciences of the
U.S.A., 94, 5411–5416.
Bertenthal, B. J., Longo, M. R., & Kenny, S. (2007). Phenomenal permanence and the
development of predictive tracking in infancy. Child Development, 78, 350–363.
Berthier, N. E., & Carrico, R. L. (2010). Visual information and object size in
infant reaching. Infant Behavior and Development, 33, 555–566.
Beusmans, J. M. H. (1998). Optic flow and the metric of the visual ground plane.
Vision Research, 38, 1153–1170.
294 References
Bian, Z., Braunstein, M. L., & Andersen, G. J. (2005). The ground dominance
effect in the perception of 3-D layout. Perception & Psychophysics, 67,
815–828.
Bingham, G. P. (1987). Dynamical systems and event perception: A working paper,
Part II. Perceiving Acting Workshop Review, 2, 4–7.
Bingham, G. P. (1988). Task-specific devices and the perceptual bottleneck. Human
Movement Science, 7, 225–264.
Bingham, G. P.(1995). Dynamics and the problem of visual event recognition. In
R. Port, & T. van Gelder (Eds.), Mind as motion: Dynamics, behavior and cog-
nition (pp. 403–448). Cambridge, MA: MIT Press.
Bingham, G. P. (2000). Events (like objects) are things, can have affordance prop-
erties, and can be perceived. Ecological Psychology, 12, 29–36.
Bingham, G. P., Bradley, A., Bailey, M., & Vinner, R. (2001). Accommodation,
occlusion, and disparity matching are used to guide reaching: A comparison of
actual versus virtual environments. Journal of Experimental Psychology: Human
Bingham, G. P., Crowell, J. A., & Todd, J. T. (2004). Distortions of distance and
shape are not produced by a single continuous transformation of reach space.
Perception & Psychophysics, 66, 152–169.
Bingham, G. P., & Pagano, C. C. (1998). The necessity of a perception–action
approach to definite distance perception: Monocular distance perception to guide
reaching. Journal of Experimental Psychology: Human Perception and Perform-
ance, 24, 145–168.
Bingham, G. P., Pan, J. S., & Mon-Williams, M. A. (2014). Calibration is both
functional and anatomical. Journal of Experimental Psychology: Human Percep-
tion and Performance, 40, 61–70.
Bingham, G. P., Rosenblum, L., & Schmidt, R. (1995). Dynamics and the orienta-
tion of kinematic forms in visual event recognition. Journal of Experimental
Psychology: Human Perception and Performance, 21, 1473–1493.
Bingham, G. P., Schmidt, R. C., & Rosenblum, L. D. (1989). Hefting for a
maximum distance throw: A smart perceptual mechanism. Journal of Experi-
mental Psychology: Human Perception and Performance, 15, 507–528.
Bingham, G. P., & Stassen, M. G. (1994). Monocular egocentric distance informa-
tion generated by head movement. Ecological Psychology, 6, 219–238.
Bingham, G. P., & Wickelgren, E. (2008). Events and actions as dynamically
molded spatiotemporal objects: A critique of the motor theory of biological
motion perception. In T. Shipley & J. Zacks (Eds.), Understanding events:
From perception to action (pp. 255–286). New York: Oxford University
Press.
Birmingham, E., Bischof, W. F., & Kingstone, A. (2008). Gaze selection in complex
social scenes. Visual Cognition, 16, 341–355.
Bjork, D. W. (1983). The compromised scientist: William James in the develop-
ment of American psychology. New York: Columbia University Press.
Blau, J. J. C., Petrusz, S., & Carello, C (2013). Fractal structure of event segmenta-
tion: Lessons from reel and real events. Ecological Psychology, 25, 81–101.
Bodenheimer, B., Meng, J., Wu, H., Narasimham, G., Rump, B., McNamara, T.
P., … Rieser, J. J. (2007). Distance estimation in virtual and real environments
using bisection. Paper presented at the 4th Symposium on Applied Perception in
Graphics and Visualization.
References 295
Boeddeker, N., Dittmar, L., Stürzl, W., & Egelhaaf, M. (2010). The fine structure
of honeybee head and body yaw movements in a homing task. Proceedings of
the Royal Society of London B: Biological Sciences, 277, 1899–1906.
Boppré, M., Vane-Wright, R. I., & Wickler, W. (2017). A hypothesis to explain
accuracy of wasp resemblances. Ecology and Evolution, 7, 73–81.
Boring, E. G. (1942). Sensation and perception in the history of experimental
psychology. New York: Appleton-Century-Crofts, Inc.
Borji, A., & Itti, L. (2013). State-of-the-art in visual attention modeling. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 35, 185–207.
Borst, C., Flach, J. M., & Ellerbroek, J. (2015). Beyond ecological interface design:
Lessons from concerns and misconceptions. IEEE: Transactions on Systems,
Man, and Cybernetics, 45, 164–175.
Bourgeois, J., & Coello, Y. (2012). Effect of visuomotor calibration and uncer-
tainty on the perception of peripersonal space. Attention, Perception, & Psycho-
physics, 74, 1268–1283.
Boynton, R. M. (1974). The visual system: Environmental information. In E. C.
Carterette & M. Friedman (Eds.). Handbook of perception (pp. 285–308). New
York: Academic Press.
Branch, G. M. (1979). Aggression by limpets against invertebrate predators.
Animal Behaviour, 27, 408–410.
Braunstein, M. L., Andersen, G. J., & Riefer, D. M. (1982). The use of occlusion to
resolve ambiguity in parallel projections. Perception & Psychophysics, 31, 261–267.
Bremner, J. G., Johnson, S. P., Slater, A. M., Mason, U., Foster, K., Cheshire, A.,
& Spring, J. (2005). Conditions for young children’s perception of object traject-
ories. Child Development, 113(76), 1029–1043.
Bremner, J. G., Slater, A. M., & Johnson, S. P. (2015). Perception of object persist-
ence: The origins of object permanence in infancy. Child Development Perspec-
tives, 9, 7–13.
Bridson, R. (2016). Fluid simulation for computer graphics (2nd ed.). New York:
CRC Press.
Bridson, R., & Batty, C. (2010). Computational physics in film. Science, 330,
1756–1757.
Brostow, G. J., & Essa, I. (2001). Image-based motion blur for stop motion anima-
tion. In Proceedings of the 28th Annual Conference on Computer Graphics and
Interactive Techniques (pp. 561–566). ACM.
Bruggeman, H., Zosh, W., & Warren, W. H. (2007). Optic flow drives human
visuo-locomotor adaptation. Current Biology, 17, 2035–2040.
Bruineberg, J., Chemero, A., & Rietveld, E. (2018). General ecological information
supports engagement with affordances for ‘higher’ cognition. Synthese, 1–21.
Bruineberg, J., & Rietveld, E. (2014). Self-organization, free energy minimization,
and optimal grip on a field of affordances. Frontiers in Human Neuroscience,
8, 599.
Brunswik, E. (1956). Perception and the representative design of psychological
experiments. Berkeley, CA: University of California Press.
Bullmore, E., & Sporns, O. (2009). Complex brain networks: Graph theoretical
analysis of structural and functional systems. Nature Reviews Neuroscience, 10,
186–198.
Burns, M., & Hajdukiewicz, J. R. (2004). Ecological interface design. New York:
CRC Press.
296 References
Burton, G. (1992). Nonvisual judgment of the crossability of path gaps. Journal of
Experimental Psychology: Human Perception and Performance, 18, 698–713.
Burton, G. (1993). Non-neural extensions of haptic sensitivity. Ecological Psych-
ology, 5, 105–124.
Burton, G., & Cyr, J. (2008). Movements of the cane prior to locomotion judg-
ments: the informer fallacy and the training fallacy versus the role of expecta-
tion. In O. Y. Chebykin, G. Z. Bedny, & W. Karwowski (Eds.) Ergonomics and
psychology: Developments in theory and practice. New York: CRC Press.
Bushnell, E. W., & Boudreau, J. P. (1993). Motor development and the mind: The
potential role of motor abilities as a determinant of aspects of perceptual devel-
opment. Child Development, 64, 1005–1021.
Buswell, G. T. (1935). How people look at pictures: A study of the psychology of
perception in art. Chicago: University of Chicago Press.
Cabrera, F., Sanabria, F., Jiménez, Á. A., & Covarrubias, P. (2013). An affordance
analysis of unconditioned lever pressing in rats and hamsters. Behavioural Pro-
cesses, 92, 36–46.
Campbell, J. (1817). On vision. Annals of Philosophy, 10, 17–29.
Campos, J. J., Anderson, D. I., Barbu-Roth, M. A., Hubbard, E. M., Hertenstein,
M. J., & Witherington, D. (2000). Travel broadens the mind. Infancy, 1,
149–219.
Cardinali, L., Frassinetti, F., Brozzoli, C., Urquizar, C., Roy, A. C., & Farne, A.
(2009). Tool-use induces morphological updating of the body schema. Current
Biology, 19, R478–R479.
Carello, C., Anderson, K. L., & Kunkler-Peck, A. J. (1998). Perception of object
length by sound. Psychological Science, 9, 211–214.
Carello, C., Fitzpatrick, P., Domaniewicz, I., Chan, T., & Turvey, M. T. (1992).
Effortful touch with minimal movement. Journal of Experimental Psychology:
Human Perception and Performance, 18, 290–302.
Carello, C., Fitzpatrick, P., & Turvey, M. T. (1992). Haptic probing: Perceiving
the length of a probe and the distance of a surface probed. Perception & Psycho-
physics, 51, 580–598.
Carello, C., Grosofsky, A., Reichel, F. D., Solomon, H. Y., & Turvey, M. T.
(1989). Visually perceiving what is reachable. Ecological Psychology, 1, 27–54.
Carello, C., & Turvey, M. T. (2017). Useful dimensions of haptic perception: 50
years after the senses considered as perceptual systems. Ecological Psychology,
29, 95–121.
Carello, C., Wagman, J. B., & Turvey, M. T. (2005). Acoustic specification of
object properties. In J. Anderson & B. Anderson (Eds.), Moving image theory:
Ecological considerations (pp. 79–104). Carbondale, IL: Southern Illinois Univer-
sity Press.
Cassirer, E. (1944). The concept of group and the theory of perception. Philosophy
and Phenomenological Research: A Quarterly Journal, 5, 1–36.
Cesari, P., Formenti, F., & Olivato, P. (2003). A common perceptual parameter for
stair climbing for children, young and old adults. Human Movement Science, 22,
111–124.
Cesari, P., & Newell, K. M. (2000). Body-scaled transitions in human grip configu-
rations. Journal of Experimental Psychology: Human Perception and Perform-
ance, 26, 1657–1668.
Chapman, S. (1968). Catching a baseball. American Journal of Physics, 36, 868–870.
References 297
Chebat, D. R., Maidenbaum, S., & Amedi, A. (2015). Navigation using sensory
substitution in real and virtual mazes. PloS ONE, 10, e0126307.
Chebat, D. R., Rainville, C., Kupers, R., & Ptito, M. (2007). Tactile–‘visual’ acuity
of the tongue in early blind individuals. Neuroreport, 18, 1901–1904.
Chekhov, A. (1889). Personal letter to playwright Aleksandr Semenovich Lazarev.
In L. Goldberg (1976). Russian literature in the nineteenth century: Essays. Jeru-
salem: Magnes Press, Hebrew University.
Chemero, A. (2003). An outline of a theory of affordances. Ecological Psychology,
15, 181–195.
Chemero, A. (2009). Radical embodied cognitive science. Cambridge, MA: MIT
Press.
Chemero, A., & Turvey, M. T. (2007). Complexity, hypersets, and the ecological
perspective on perception-action. Biological Theory, 2, 23–36.
Chen, Y. P., Keen, R., Rosander, K., & von Hofsten, C. (2010). Movement plan-
ning reflects skill level and age changes in toddlers. Child Development, 81,
1846–1858.
Chippindale, C., & Tacon, P. S. C. (1998). The archeology of rock art. New York:
Cambridge University Press.
Chrastil, E. R., & Warren, W. H. (2014). Does the human odometer use an extrin-
sic or intrinsic metric? Attention, Perception, & Psychophysics, 76, 230–246.
Claxton, L. J., Keen, R., & McCarty, M. E. (2003). Evidence of motor planning in
infant reaching behavior. Psychological Science, 14, 354–356.
Coats, R. O., Pan, J. S., & Bingham, G. P. (2014). Perturbation of perceptual units
reveals dominance hierarchy in cross calibration. Journal of Experimental Psych-
ology: Human Perception and Performance, 40, 328–341.
Cole, W. G., Chan, G. L. Y., Vereijken, B., & Adolph, K. E. (2013). Perceiving
affordances for different motor skills. Experimental Brain Research, 225, 309–319.
Comalli, D. M,, Franchak, J., Char, A., & Adolph, K. (2013). Ledge and wedge:
Younger and older adults’ perception of action possibilities. Experimental Brain
Research, 228, 183–192.
Comalli, D. M., Keen, R., Abraham, E., Foo, V. J., Lee, M. H., & Adolph, K. E.
(2016). The development of tool use: Planning for end-state comfort. Develop-
mental Psychology, 52, 1878–1892.
Comalli, D. M., Persand, D., & Adolph, K. E. (2017). Motor decisions are not
black and white: Selecting actions in the “gray zone”. Experimental Brain
Research, 235, 1793–1807.
Cook, H. E., Hassebrock, J. A., & Smart, L. J. (2018). Responding to other peo-
ple’s posture: Visually induced motion sickness from naturally generated optic
flow. Frontiers in Psychology, 9(1901), 1–9. doi:10.3389/fpsyg.2018.01901.
Cordovil, R., Santos, C., & Barreiros, J. (2012). Perceiving children’s behavior and
reaching limits in a risk environment. Journal of Experimental Child Psychology,
111, 319–330.
Costall, A. (1995). Socializing affordances. Theory & Psychology, 5, 467–481.
Costall, A. (2012). Canonical affordances in context. AVANT. Pismo Awangardy
Filozoficzno-Naukowej, 85–93.
Cottingham, J., Stoothoff, R., & Murdoch, B. (1985). The philosophical writings
of Descartes, Vol. 1: Optics. Cambridge: Cambridge University Press.
Crafton, D. (1999). The talkies: American cinema’s transition to sound, 1926–1931
(Vol. 4). Oakland, CA: University of California Press.
298 References
Creem-Regehr, S. H., Gagnon, K. T., Geuss, M. N., & Stefanucci, J. K. (2013).
Relating spatial perspective taking to the perception of other’s affordances: Pro-
viding a foundation for predicting the future behavior of others. Frontiers in
Human Neuroscience, 7, 596.
Crombie, A. C. (1996). Science, art and nature in medieval and modern thought.
London: Hambledon Press.
Cutting, J. E. (1986). Perception with an eye for motion. Cambridge, MA: MIT
Press.
Cutting, J. E. (1988). Affine distortions of pictorial space: Some predictions for
Goldstein (1987) that La Gournerie (1859) might have made. Journal of Experi-
mental Psychology: Human Perception & Performance, 14, 305–311.
d’Abro, A. (1951). The rise of the new physics: Its mathematical and physical the-
ories (2nd ed.). New York: Dover.
Dai, S., & Wu, Y. (2008). Motion from blur. In Computer Vision and Pattern
Recognition, 2008. CVPR 2008, 1–8. Evanston, IL: Northwestern University
Press.
Dainoff, M. J. (2008). Ecological ergonomics. In O. Y. Chebykin, G. Z. Bedny, &
W. Karwowski (Eds.), Ergonomics and psychology: Developments in theory and
practice (pp. 1897–1900). Boca Raton, FL: CRC Press.
Dainoff, M. J., & Mark, L. S. (2001). Affordances. In W. Karwowski (Ed.) Inter-
national encyclopedia of human factors (Vol. II, pp. 3–28), London: Routledge.
Dainoff, M., & Wagman, J. B. (2004). Implications of dynamic touch for human
factors/ergonomics: Contributions from ecological psychology. In Proceedings of
the Human Factors and Ergonomics Society Annual Meeting, 48, 1319–1320.
Davies, R. (1985). What’s bred in the bone. New York: Viking.
Day, B., Ebrahimi, E., Hartman, L. S., Pagano, C. C., & Babu, S. V. (2017). Cali-
bration to tool use during visually-guided reaching. Acta Psychologica, 181,
27–39.
Day, B., Ebrahimi, E., Hartman, L. S., Pagano, C. C., Robb, A. C., & Babu, S. V.
(2019). Examining the effects of altered avatars on perception-action in virtual
reality. Journal of Experimental Psychology: Applied, 25(1). doi:10.1037/
xap0000192.
Delagnes, A., & Roche, H. (2005). Late Pliocene hominid knapping skills: The case
of Lokalalei 2C, West Turkana, Kenya. Journal of Human Evolution, 48,
435–472.
Dennett, D. (2017, January 1). What scientific term or concept ought to be more
widely known? Edge. Retrieved from www.edge.org/response-detail/27002.
De Wit, M. M., De Vries, S., Van der Kamp, J., & Withagen, R. (2017).
Affordances and neuroscience: Steps towards a successful marriage. Neuro-
science & Biobehavioral Reviews, 80, 622–629.
Diaz, G., Cooper, J., Kit, D., & Hayhoe, M. M. (2013). Real-time recording and
classification of eye movements in an immersive virtual environment. Journal of
Vision, 13, 1–14.
Dibble, H. L., & Pelcin, A. (1995). The effect of hammer mass and velocity on
flake mass. Journal of Archaeological Science, 22, 429–439.
Dibble, H. L., & Režek, Ž. (2009). Introducing a new experimental design for con-
trolled studies of flake formation: Results for exterior platform angle, platform
depth, angle of blow, velocity, and force. Journal of Archaeological Science, 36,
1945–1954.
References 299
Dickinson, M. H., Farley, C. T., Full, R. J., Koehl, M. A. R., Kram, R., & Lehman,
S. (2000). How animals move: An integrative view. Science, 288, 100–106.
Dijkstra, T. M. H., Schöner, G., & Gielen, C. C. A. M. (1994). Temporal stability
of the action-perception cycle for postural control in a moving visual environ-
ment. Experimental Brain Research, 97, 477–486.
Di Paolo, E., Buhrmann, T., & Barandiaran, X. (2017). Sensorimotor life: An enac-
tive proposal. Oxford: Oxford University Press.
Dixon, M. W., Wraga, M. J., Proffitt, D. R., & Williams, G. C. (2000). Eye
height scaling of absolute size in immersive and nonimmersive displays. Journal
of Experimental Psychology: Human Perception and Performance, 26,
582–593.
Dolezal, H. (1982). Living in a world transformed: Perceptual and performatory
adaptation to visual distortion. Cambridge, MA: Academic Press.
Donald, M. (1991). The origins of modern mind: Three stages in the evolution of
culture and cognition. Cambridge, MA: Harvard University Press.
Dotov, D. G., de Wit, M. M., & Nie, L. (2012). Understanding affordances:
History and contemporary development of Gibson’s central concept. AVANT.
Pismo Awangardy Filozoficzno-Naukowej, 28–39.
Dotov, D. G., Nie, L., & Chemero, A. (2010). A demonstration of the transition
from ready-to-hand to unready-to-hand. PLoS ONE, 5, e9433.
Durgin, F. H., & Li, Z. (2011). Perceptual scale expansion: An efficient angular
coding strategy for locomotor space. Attention, Perception, & Psychophysics,
73, 1856–1870.
Effken, J. A. (2006). Improving clinical decision making through ecological inter-
faces. Ecological Psychology, 18, 283–318.
Effken, J. A., Kim, N.-G., & Shaw, R. E. (1997). Making the constraints visible:
Testing the ecological approach to interface design, Ergonomics, 40, 1–27.
Effken, J. A., Loeb, R. G., Kang, Y., & Lin, Z-C. (2008). Clinical information dis-
plays to improve ICU outcomes. International Journal of Medical Informatics,
77, 765–777.
Einhauser, W., Schumann, F., Bardins, S., Bartl, K., Boning, G., Schneider, E., &
Konig, P. (2007). Human eye-head co-ordination in natural exploration.
Network: Computation in Neural Systems, 18, 267–297.
Einstein, E. (1905/1923). Zur Elektrodynamik bewegter Körper, Annalen der
Physik, 17, 891; English trans. On the Electrodynamics of Moving Bodies by
George Barker Jeffery and Wilfrid Perrett (1923).
Epstein, W. (1966). Perceived depth as a function of relative height under three
background conditions. Journal of Experimental Psychology, 72, 335–338.
Fagard, J., Spelke, E. S., & von Hofsten, C. (2009). Reaching and grasping a
moving object in 6-, 8-, and 10-month-old infants: Laterality and performance.
Infant Behavior and Development, 32, 137–146.
Fajen, B. R. (2005a). Calibration, information, and control strategies for braking
to avoid a collision. Journal of Experimental Psychology: Human Perception and
Performance, 31, 480–501.
Fajen, B. R. (2005b). Perceiving possibilities for action: On the necessity of cali-
bration and perceptual learning for the visual guidance of action. Perception, 34,
717–740.
Fajen, B. R. (2007). Affordance-based control of visually guided action. Ecological
Psychology, 19, 383–410.
300 References
Fajen, B. R. (2008). Perceptual learning and the visual control of braking. Percep-
tion & Psychophysics, 70, 1117–1129.
Fajen, B. R., Riley, M. A., & Turvey, M. T. (2009). Information, affordances, and
the control of action in sport. International Journal of Sport Psychology, 40,
79–107.
Fajen, B. R., & Warren, W. H. (2003). Behavioral dynamics of steering, obstacle
avoidance, and route selection. Journal of Experimental Psychology: Human
Perception and Performance, 29, 343–362. doi:10.1037/0096-1523.29.2.343.
Fajen, B. R., & Warren, W. H. (2004). Visual guidance of intercepting a moving
target on foot. Perception, 33, 689–715. doi:10.1068/p5236.
Favela, L. H., Riley, M. A., Shockley, K., & Chemero, A. (2018). Perceptually
equivalent judgments made visually and by haptic sensory-substitution devices.
Ecological Psychology. https://doi.org/10.1080/10407413.2018.1473712.
Feria, C. S., Braunstein, M. L., & Andersen, G. J. (2003). Judging distance across
texture discontinuities. Perception, 32, 1423–1440.
Ferwerda, J. A. (2001). Elements of early vision for computer graphics. IEEE Com-
puter Graphics and Applications, 21, 22–33.
Feynman, R. (1985). Surely you’re joking Mr. Feynman! Adventures of a curious
character. New York: W. W. Norton & Company.
Field, D. J. (1987). Relations between the statistics of natural images and the
response properties of cortical cells. Journal of the Optical Society of America A,
4, 2379–2394.
Field, G. D., & Rieke, F. (2002). Mechanisms regulating variability of the single
photon responses of mammalian rod photoreceptors. Neuron, 35, 733–747.
Fitch, H. L., Tuller, B., & Turvey, M. T. (1982). The Bernstein perspective: III.
Tuning of coordinative structures with special reference to perception. In A. S.
Kelso (Ed.), Human motor behavior (pp. 271–281). New York: Routledge.
Fitzpatrick, P., Carello, C., Schmidt, R. C., & Corey, D. (1994). Haptic and visual
perception of an affordance for upright posture. Ecological Psychology, 6,
265–287.
Flach, J. M. (1990). The ecology of human-machine systems I: Introduction. Eco-
logical Psychology, 2, 191–205.
Flach, J. M., Hancock, P., Caird, J., & Vicente, K. (1995). Global perspectives on
the ecology of human-machine systems, Hillsdale, NJ: Lawrence Erlbaum Asso-
ciates.
Flach, J. M., Reynolds, P., Cao, C., & Staffell, T. (2017). Engineering representa-
tions to support evidence-based clinical practice. Proceedings of the 2017 Inter-
national Symposium on Human Factors and Ergonomics in Health Care, 6,
66–73.
Fodor, J. A. (1981). Representations: Philosophical essays on the foundations of
cognitive science. Cambridge, MA: The MIT Press.
Foley, J. M., Ribeiro-Filho, N. P., & DaSilva, J. A. (2004). Visual perception of
extent and the geometry of visual space. Vision Research, 44, 147–156.
Foulsham, T., Chapman, C., Nasiopoulos, E., & Kingstone, A. (2014). Top-down
and bottom-up aspects of active search in a real-world environment. Canadian
Journal of Experimental Psychology, 68, 8–19.
Foulsham, T., Walker, E., & Kingstone, A. (2011). The where, what, and when of
gaze allocation in the lab and the natural environment. Vision Research, 51,
1920–1931.
References 301
Franchak, J. M. (2017). Using head-mounted eye tracking to study development. In
B. Hopkins, E. Geangu, & S. Linkenauger (Eds.), The Cambridge encyclopedia
of child development (2nd ed., pp. 113–116). Cambridge: Cambridge University
Press.
Franchak, J. M. (2019). Changing opportunities for learning in everyday life: Infant
body position over the first year. Infancy, 24, 187–209.
Franchak, J. M., & Adolph, K. E. (2010). Visually guided navigation: Head-
mounted eye-tracking of natural locomotion in children and adults. Vision
Research, 50, 2766–2774.
Franchak, J. M., & Adolph, K. E. (2014). Gut estimates: Pregnant women adapt to
changing possibilities for squeezing through doorways. Attention, Perception, &
Psychophysics, 76, 460–472.
Franchak, J. M., Celano, E. C., & Adolph, K. E. (2012). Perception of passage
through openings depends on the size of the body in motion. Experimental Brain
Research, 223, 301–310.
Franchak, J. M., Heeger, D. J., Hasson, U., & Adolph, K. E. (2016). Free-viewing
gaze behavior in infants and adults. Infancy, 21, 262–287.
Franchak, J. M., Kretch, K. S., & Adolph, K. E. (2018). See and be seen: Infant-
caregiver social looking during locomotor free play. Developmental Science, 21,
e12626.
Franchak, J. M., Kretch, K. S., Soska, K. C., & Adolph, K. E. (2011). Head-
mounted eye tracking: A new method to describe infant looking. Child Develop-
ment, 82, 1738–1750.
Franchak, J. M., Kretch, K. S., Soska, K. C., Babcock, J. S., & Adolph, K. E.
(2010). Head-mounted eye-tracking in infants’ natural interactions: A new
method. Paper presented at the Proceedings of the 2010 Symposium on Eye
Tracking Research and Applications, Austin, TX.
Franchak, J. M., Smith, L. B., & Yu, C. (under review). Developmental changes in
how head orientation structures infants’ visual experiences.
Franchak, J. M., van der Zalm, D. J., & Adolph, K. E. (2010). Learning by doing: Action
performance facilitates affordance perception. Vision Research, 50, 2758–2765.
Franchak, J. M., & Yu, C. (2015). Visual-motor coordination in natural reaching
of young children and adults. Proceedings of the 37th Annual Meeting of the
Cognitive Science Society. Austin, TX: Cognitive Science Society.
Frank, M. C., Simmons, K., Yurovsky, D., & Pusiol, G. (2013). Developmental
and postural changes in children’s visual access to faces. In M. Knauff, M.
Pauen, N. Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th Annual
Meeting of the Cognitive Science Society (pp. 454–459). Austin, TX: Cognitive
Science Society.
Frank, M. C., Vul, E., & Johnson, S. P. (2009). Development of infants’ attention
to faces during the first year. Cognition, 110, 160–170.
Fukusima, S. S., Loomis, J. M., & Da Silva, J. A. (1997). Visual perception of ego-
centric distance as assessed by triangulation. Journal of Experimental Psych-
Fultot, M., Nie, L., & Carello, C. (2016). Perception-action mutuality obviates
mental construction. Constructivist Foundations, 11, 298–307.
Gal, O., & Chen-Morris, R. (2010). Baroque optics and the disappearance of the
observer: From Kepler’s optics to Descartes’ doubt. Journal of the History of
Ideas, 71, 191–217.
302 References
Geduld, H. M. (1975). The birth of the talkies: From Edison to Jolson. Blooming-
ton, IN: Indiana University Press.
Geisler, W. S. (2008). Visual perception and the statistical properties of natural
scenes. Annual Review of Psychology, 59, 167–192.
Gelfand, I. M. (1989). Two archetypes in the psychology of man: The lecture notes
for Kyoto Prize. Retrieved from www.israelmgelfand.com/talks/two_archetypes.
pdf.
Gesell, A. L. (1928). Infancy and human growth. New York: Macmillan.
Gibson, E. J. (1963). Perceptual learning. Annual Review of Psychology, 14,
29–56.
Gibson, E. J. (1969). Principles of perceptual learning and development. New
York: Appleton-Century-Crofts.
Gibson, E. J. (1988). Exploratory behavior in the development of perceiving, acting
and the acquiring of knowledge. Annual Review of Psychology, 39, 1–41.
Gibson, E. J. (1991). An odyssey in learning and perception. Cambridge, MA: MIT
Press.
Gibson, E. J. (1994). An odyssey in learning and perception. Cambridge, MA: MIT
Press.
Gibson, E. J., & Pick, A. D. (2000). An ecological approach to perceptual learning
and development. New York: Oxford University Press.
Gibson, E. J., Riccio, G., Schmuckler, M. A., Stoffregen, T. A., Rosenberg, D., &
Taormina, J. (1987). Detection of the traversability of surfaces by crawling and
walking infants. Journal of Experimental Psychology: Human Perception and
Gibson, E. J., & Walk, R. D. (1960). The “visual cliff ”. Scientific American, 202,
64–71.
Gibson, J. J. (Ed.) (1947). Motion picture testing and research. Report No. 7. Army
Air Forces Aviation Psychology Program Research Reports.
Gibson, J. J. (1950a). Perception of the visual world. Boston: Houghton Mifflin.
Gibson, J. J. (1950b). The perception of visual surfaces. The American Journal of
Gibson, J. J. (1951). What is a form? Psychological Review, 58, 403–412.
Gibson, J. J. (1954). The visual perception of objective motion and subjective
movement. Psychological Review, 61, 304–314.
Gibson, J. J. (1955). Optical motions and transformations as stimuli for visual per-
ception. State College, PA: Psychological Cinema Register.
Gibson, J. (1957). Optical motions and transformations as stimuli for visual per-
ception. Psychological Review, 64, 288–294.
Gibson, J. J. (1958). Visually controlled locomotion and visual orientation in
animals. British Journal of Psychology, 49, 182–194.
Gibson, J. J. (1959). Perception as a function of stimulation. In S. Koch (Ed.),
Psychology: A study of a science (Vol. I, pp. 456–473). New York: McGraw-
Hill.
Gibson, J. J. (1960a). The concept of the stimulus in psychology. American Psy-
chologist, 15, 694–703.
Gibson, J. J. (1960b). The information contained in light. Acta Psychologica, 17,
23–30.
Gibson, J. J. (1961a). The contribution of experimental psychology to the formula-
tion of the problem of safety: A brief for basic research. In L. W. Mayo (Ed.).
References 303
Behavioral approaches to accident research. New York: Association for the Aid
of Crippled Children.
Gibson, J. J. (1961b). Ecological optics. Vision Research, 1, 253–262.
Gibson, J. J. (1963). The useful dimensions of sensitivity. American Psychologist,
18, 1–15.
Gibson, J. J. (1966a). The senses considered as perceptual systems. Boston, MA:
Houghton Mifflin.
Gibson, J. J. (1966b). The problem of temporal order in stimulation and percep-
tion. The Journal of Psychology, 62, 141–149.
Gibson, J. J. (1967a). New reasons for realism. Synthese, 17, 162–172.
Gibson, J. J. (1967b). For new book (“Useful Vision”?). Unpublished manuscript,
James J. Gibson papers, #14–23–1832. Division of Rare and Manuscript Collec-
tions, Cornell University Library.
Gibson, J. J. (1967c). Outline of an “uncompromising” new book on Vision (Note
for the Introduction). Unpublished manuscript, James J. Gibson papers,
#14–23–1832. Division of Rare and Manuscript Collections, Cornell University
Library.
Gibson, J. J. (1969). The change from visible to invisible: A study of optical trans-
itions. State College, PA: Psychological Cinema Register.
Gibson, J. J. (1971a). Everyday visual perception (tentative outline). Unpublished
manuscript, James J. Gibson papers, #14–23–1832. Division of Rare and Manu-
script Collections, Cornell University Library.
Gibson, J. J. (1971b). Do we ever see light? Unpublished manuscript, retrieved
from www.trincoll.edu/depts/ecopsyc/perils/.
Gibson, J. J. (1971c). On the distinction between objects and substances. Unpub-
lished manuscript, retrieved from www.trincoll.edu/depts/ecopsyc/perils/.
Gibson, J. J. (1971d). A preliminary list of postulates for an ecological psychology.
Unpublished manuscript, James J. Gibson papers, #14–23–1832. Division of
Rare and Manuscript Collections, Cornell University Library.
Gibson, J. J. (1971e). The environment to be perceived: Part I of an ecological
approach to visual perception. Unpublished manuscript, James J. Gibson papers,
#14–23–1832. Division of Rare and Manuscript Collections, Cornell University
Library.
Gibson, J. J. (1971f ). Nov 4 discussion of Part I of MS. Unpublished manuscript,
Gibson, J. J. (1971g). The information available in pictures. Leonardo, 4, 27–35.
Gibson, J. J. (1973). The consequence of assuming that there is a persisting
environmental layout along with a changing environmental layout. Unpublished
manuscript, James J. Gibson papers, #14–23–1832. Division of Rare and Manu-
script Collections, Cornell University Library.
Gibson, J. J. (1974). A note on ecological optics. In E. C. Carterette & M. Fried-
man (Eds.). Handbook of perception (pp. 309–312). New York: Academic Press.
Gibson, J. J. (1975). Events are perceivable but time is not. In J. T. Fraser & N. Law-
rence (Eds.), The study of time II (pp. 295–301). New York: Springer-Verlag.
Gibson, J. J. (1977). A tentative formula for the perception of persistence and
awareness of reality. Unpublished manuscript, December 1977. Available at
https://commons.trincoll.edu/purpleperils/1972-1979/a-t entative-formula-
for-the-perception-of-persistence-and-awareness-of-reality/.
304 References
Gibson, J. J. (1978a). The theory of further scrutiny. Unpublished manuscript,
Gibson, J. J. (1978b). The perceiving of hidden surfaces. In P. Machamer & R.
Turnbull (Eds.), Studies in perception. Columbus, OH: Ohio State University.
Gibson, J. J. (1979/2015). The ecological approach to visual perception: classic
edition. New York: Psychology Press. (originally published in 1979).
Gibson, J. J. (1982a). Notes on affordances. In E. Reed & R. Jones (Eds.), Reasons
for realism: Selected essays of James J. Gibson (pp. 401–418). Hillsdale, NJ:
Lawrence Erlbaum Associates.
Gibson, J. J. (1982b). The myth of passive perception: A reply to Richards. In E.
Reed & R. Jones (Eds.), Reasons for realism: Selected essays of James J. Gibson
(pp. 397–400). Hillsdale, NJ: Lawrence Erlbaum Associates.
Gibson, J. J. (1982c). A history of the ideas behind ecological optics: Introductory
remarks at the workshop on ecological optics. In E. Reed & R. Jones (Eds.).
Reasons for realism: Selected essays of James J. Gibson (pp. 90–101). Mahwah,
NJ: Lawrence Erlbaum Associates, Inc.
Gibson, J. J., & Crooks, L. E. (1938). A theoretical field analysis of automobile
driving. American Journal of Psychology, 51, 453–471.
Gibson, J. J., & Gibson, E. J. (1955). Perceptual learning: Differentiation or enrich-
ment? Psychological Review, 62, 32–41.
Gibson, J. J., & Gibson, E. J. (1957). Continuous perspective transformations and
the perception of rigid motion. Journal of Experimental Psychology, 54,
129–138.
Gibson, J. J., Kaplan, G., Reynolds, H., & Wheeler, K. (1969). The change from
visible to invisible: A study of optical transitions. Perception & Psychophysics,
5, 113–116.
Gibson, J. J., & Kaushall, P. (1973). Reversible and irreversible events. State
College, PA: Psychological Cinema Register.
Gibson, J. J., Olum, P., & Rosenblatt, F. (1955). Parallax and perspective during
aircraft landings. The American Journal of Psychology, 68, 372–385.
Gibson, J. J., Purdy, J., & Lawrence, L. (1955). A method of controlling stimu-
lation for the study of space perception: The optical tunnel. Journal of Experi-
mental Psychology, 50, 1–14.
Gibson, J. J., Smith, O., Steinschneider, A., & Johnson, C. (1957). The relative
accuracy of visual perception of motion during fixation and pursuit. The Amer
ican Journal of Psychology, 70, 64–68.
Gibson, J. J., & Waddell, D. (1952). Homogeneous retinal stimulation and visual
perception. American Journal of Psychology, 65, 263–270.
Gilchrist, A. L. (2018). To compute lightness, illumination is not estimated, it is
held constant. Journal of Experimental Psychology: Human Perception and Per-
formance, 44, 1258–1267.
Gilinsky, A. S. (1951). Perceived size and distance in visual space. Psychological
Review, 58, 460–482.
Goffman, E. (1974). Frame analysis. New York: Harper & Row.
Golonka, S. (2015). Laws and conventions in language-related behaviors. Ecolo-
gical Psychology, 27, 236–250.
Gombrich, E. H. (1972). Art and illusion (2nd ed.). Princeton, NJ: Princeton
University Press.
References 305
Gomer, J. A., Dash, C., Moore, K. S., & Pagano, C. C. (2009). Using radial
outflow to provide depth information during teleoperation. Presence: Teleopera-
tors and Virtual Environments, 18, 304–320.
Guitton, D., & Volle, M. (1987). Gaze control in humans: Eye-head coordination
during orienting movements to targets within and beyond the oculomotor range.
Journal of Neurophysiology, 58, 427–459.
Haber, R. N. (1983). The impending demise of the icon: A critique of the concept
of iconic storage in visual information processing. Behavioral and Brain Sciences,
6, 1–11.
Haber, R. N., & Levin, C. A. (2001). The independence of size perception and dis-
tance perception. Perception & Psychophysics, 63, 1140–1152.
Hajdukiewicz, J. R., & Vicente, K. J. (2004). What does computer-mediated
control of a thermal-hydraulic system have to do with moving your jaw to
speak? Evidence for synergies in process control. Ecological Psychology, 16,
255–285.
Haken, H. (1977). Synergetics: An introduction. Nonequilibrium phase trans-
itions and self-organization in physics, chemistry, and biology. Berlin: Springer-
Verlag.
Hancock, P., Flach, J., Caird, J., & Vicente, K. (1995). Local applications of the
ecological approach to human-machine systems. Hillsdale, NJ: Lawrence
Erlbaum Associates.
Hancock, P. A., & Manser, M. P. (1997). Time-to-contact: More than tau alone.
Ecological Psychology, 9, 265–297.
Hanke, W. (2014). Natural hydrodynamic stimuli. In H. Bleckmann et al. (Eds.),
Flow sensing in air and water. Berlin: Springer-Verlag.
Hanke, W., Wieskotten, S., Marshall, C., & Dehnhardt, G. (2013). Hydrodynamic
perception in true seals (Phocidae) and eared seals (Otariidae). Journal of Com-
parative Physiology A, 199, 421–440.
Hardy, L. H., Rand, G., & Rittler, M. C. (1951). Investigation of visual space: The
Blumenfeld alleys. AMA Archives of Ophthalmology, 45, 53–63.
Harmand, S., Lewis, J. E., Feibel, C. S., Lepre, C. J., Prat, S., Lenoble, A. et al.
(2015). 3.3-million-year-old stone tools from Lomekwi 3, West Turkana, Kenya.
Nature, 521, 310–315.
Harris, C. S. (1965). Perceptual adaptation to inverted, reversed, and displaced
vision. Psychological Review, 72, 419–444.
Harrison, S. J., & Turvey, M. T. (2010). Place learning by mechanical contact.
Journal of Experimental Biology, 213, 1436–1442.
Hartman, A. M. (1994). Cloudy skies: Information for perception of objects near
horizons. Ecological Psychology, 6, 289–296.
Hartman, L. S. (2018). Perception-action system calibration in the presence of
stable and unstable perceptual perturbations. All Dissertations.2144. Retrieved
from https//tigerprints.clemson.edu/all_dissertations/2144
Hartman, L. S., Kil, I., Pagano, C. C., & Burg, T. C. (2016). Investigating haptic
distance-to-break using linear and nonlinear materials in a simulated minimally
invasive surgery task. Ergonomics, 59, 1171–1181.
Hay, J. C. (1966). Optical motions and space perception: An extension of Gibson’s
analysis. Psychological Review, 73, 550–565.
Hayhoe, M. M., Shrivastava, A., Mruczek, R. E. B., & Pelz, J. B. (2003). Visual
memory and motor planning in a natural task. Journal of Vision, 3, 49–63.
306 References
He, Z. J., Wu, B., Ooi, T. L., Yarbrough, G., & Wu, J. (2004). Judging egocentric dis-
tance on the ground: Occlusion and surface integration. Perception, 33, 789–806.
Head, H. (1920). Studies in neurology, Vol. 2. London: Oxford University Press.
Hecht, H., & Savelsbergh, G. (2004). Time-to-contact. Amsterdam: Elsevier Science.
Hecht, S., Schlaer, S., & Pirenne, M. H. (1942). Energy, quanta, and vision.
Journal of General Physiology, 25, 819–840.
Heft, H. (1983). Way‑finding as the perception of information over time. Popula-
tion and Environment: Behavioral and Social Issues, 6, 133–150.
Heft, H. (2001). Ecological psychology in context: James Gibson, Roger Barker,
and the legacy of William James’s radical empiricism. Mahwah, NJ: Lawrence
Erlbaum Associates.
Heft, H. (2002). Restoring naturalism to James’s epistemology: A belated reply to
Miller & Bode. Transactions of the Charles S. Peirce Society, 38, 557–580.
Heft, H. (2003). Affordances, dynamic experience, and the challenge of reification.
Heft, H. (2007). The social constitution of perceiver-environment reciprocity. Eco-
Heft, H. (2017). Perceptual information of “an entirely different order”: The “cul-
tural environment” in The Senses Considered as Perceptual Systems. Ecological
Heft, H. (2018). Places: Widening the scope of an ecological approach to perception–
action with an emphasis on child development. Ecological Psychology, 30, 99–123.
Heft, H., & Nasar, J. (2000). Evaluating environmental scenes using dynamic
versus static displays. Environment & Behavior, 32, 301–322.
Held, R., & Hein, A. (1963). Movement-produced stimulation in the development
of visually guided behaviour. Journal of Comparative and Physiological Psych-
ology, 56, 872–876.
Helmholtz, von H. (1866/2000). Handbuch der physiologischen Optik, Dritter
Brand. (J. P. C. Southall Trans., as Helmhotz’s treatise on physiological optics,
Vol. III, pp. 91–92). Bristol: Thoemmes.
Hettinger, L. J., & Riccio, G. E. (1992). Visually induced motion sickness in virtual
environments. Presence: Teleoperators and Virtual Environments, 1, 306–310.
Heyser, C. J., & Chemero, A. (2012). Novel object exploration in mice: Not all
objects are created equal. Behavioural Processes, 89, 232–238.
Higuchi, T., Cinelli, M. E., Greig, M. A., & Patla, A. E. (2006). Locomotion
through apertures when wider space for locomotion is necessary: Adaptation to
artificially altered bodily states. Experimental Brain Research, 175, 50–59.
Higuchi, T., Murai, G., Kijima, A., Seya, Y., Wagman, J. B., & Imanaka, K.
(2011). Athletic experience influences shoulder rotations when running through
apertures. Human Movement Science, 30, 534–549.
Higuchi, T., Takada., H., Matsuura, Y., & Imanaka, K., (2004). Visual estimation
of spatial requirements for locomotion in novice wheelchair users. Journal of
Experimental Psychology: Applied, 10, 55–66.
Hillier, B. (1996). Space is the machine: A configural theory of architecture. Cam-
bridge: Cambridge University Press.
Hochberg, J. E. (1968). Perception (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.
Hockney, D. (2001). Secret knowledge: Rediscovering the lost techniques of the
old masters. New York: Penguin Putnam.
Holt, E. B. (1914). The concept of consciousness. New York: Macmillan.
References 307
Holt, E. B. (1915). The Freudian wish and its place in ethics. New York:
Henry Holt.
Holth, M., Van der Meer, A. L. H., & Van der Weel, F. R. (2013). Combining
findings from gaze and electroencephalography recordings to study timing in a
visual tracking task. NeuroReport, 24, 968–972.
Hsu, J. (2019). Machines on mission possible. Nature Machine Intelligence, 1,
124–127.
Hutto, D. D. & Myin, E. (2017). Evolving enactivism: Basic minds meet content.
Cambridge, MA: MIT Press.
Ingber, D. E. (2005). Mechanical control of tissue growth: function follows form. Pro-
ceedings of the National Academy of Sciences of the U.S.A., 102, 11571–11572.
Ingber, D. E. (2006). Cellular mechanotransduction: Putting all the pieces together
again. FASEB Journal, 20, 811–827.
Ingle, D. (1973). Spontaneous shape discrimination by frogs during unconditioned
escape behaviour. Physiological Psychology, 1, 71–73.
Iodice, P., Scuderi, N., Saggini, R., & Pezzulo, G. (2015). Multiple timescales of
body schema reorganization due to plastic surgery. Human Movement Science,
42, 54–70.
Ittelson, W. H. (1996). Visual perception of markings. Psychonomic Bulletin &
Review, 3, 171–187.
Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature
Reviews Neuroscience, 2, 194–203.
Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention
for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 20, 1254–1259.
Iwamoto, M., Ueyama, D., & Kobayashi, R. (2014). The advantage of mucus for
adhesive locomotion in gastropods. Journal of Theoretical Biology, 353, 133–141.
Jacobs, D. M., & Michaels, C. F. (2001). Individual differences and the use of non-
specifying variables in learning to perceive distance and size: Comments on McCon-
nell, Muchisky, and Bingham (1998). Perception & Psychophysics, 63, 563–571.
Jacobs, D. M., & Michaels, C. F. (2007). Direct learning. Ecological Psychology,
19, 321–349.
Jacobs, D. M., Runeson, S., & Michaels, C. F. (2001). Learning to visually perceive
the relative mass of colliding balls in globally and locally constrained task ecolo-
gies. Journal of Experimental Psychology: Human Perception and Performance,
27, 1019–1038.
Jacobs, D. M., Silva, P. L., & Calvo, J. (2009). An empirical illustration and
formalization of the theory of direct learning: The muscle-based perception of
kinetic properties. Ecological Psychology, 21, 245–289.
James, W. (1890). The principles of psychology. New York: Henry Holt and
Company.
James, W. (1892). The stream of consciousness. In Psychology, Chapter XI. Cleve-
land and New York: World Publishing Company. Retrieved April 26, 2018 from
http://psychclassics.yorku.ca/James/jimmy11.htm.
James, W. (1904). A world of pure experience. Journal of Philosophy, Psychology,
and Scientific Methods, 1, 533–543.
Jayaraman, W., Fausey, C. M., & Smith, L. B. (2015). The faces in infant-
perspective scenes change over the first year of life. PloS ONE. doi:10.1371/
journal.pone.0123780.
308 References
Jayne, B. C., & Riley, M. A. (2007). Scaling of the axial morphology and gap-
bridging ability of the brown tree snake, Boiga irregularis. Journal of Experi-
mental Biology, 210, 1148–1160.
Jenison, R. L. (1997). On acoustic information for motion. Ecological Psychology,
9, 131–151.
Jiang, Y., O’Neal, E. E., Yon, J. P., Franzen, L., Rahimian, P., Plumert, J. M., &
Kearney, J. (2018). Acting together: Joint pedestrian road crossing in an immer-
sive virtual environment. ACM Transactions on Applied Perception, 15(2),
Article 8.
Jiménez, Á. A., Sanabria, F., & Cabrera, F. (2017). The effect of lever height on the
microstructure of operant behavior. Behavioural Processes, 140, 181–189.
Joh, A. S., Adolph, K. E., Narayanan, P. J., & Dietz, V. A. (2007). Gauging possib-
ilities for action based on friction underfoot. Journal of Experimental Psych-
Johansson, G. (1964). Perception of motion and changing form. Scandinavian
Journal of Psychology, 5, 181–208.
Johansson, G. (1973). Visual perception of biological motion and a model for its
analysis. Perception and Psychophysics, 14, 201–211.
Johansson, G., von Hofsten, C., & Jansson, G. (1980). Event perception. Annual
Review of Psychology, 31, 27–63.
Johnson, S. P., Amso, D., & Slemmer, J. A. (2003). Development of object con-
cepts in infancy: Evidence for early learning in an eye-tracking paradigm.
PANAS, 100, 10568–10573.
Latash, M. L. (2008). Synergy. New York: Oxford University Press.
Latash, M. L., Scholz, J. P., & Schöner, G. (2002). Motor control strategies
revealed in the structure of motor variability. Exercise and Sport Sciences
Reviews, 30, 26–31.
Kandel, E. R., Schwartz, J. H., Jessell, T. M., Siegelbaum, S. A., & Hudspeth, A. J.
(Eds.). (2013). Principles of neural science. New York: McGraw-Hill Education.
Kaplan, G. (1969). Kinetic disruption of optical texture. The perception of depth at
an edge. Perception & Psychophysics, 6, 193–198.
Katz, D. (1935). The world of colour. (R. B. MacLeod & C. W. Fox, Trans.)
London: Kegan Paul, Trench, Trubner & Co.
Kawagishi, Y., Hatsuyama, K., & Kondo, K. (2003, July). Cartoon blur: Nonpho-
torealistic motion blur. In Computer Graphics International, 2003. Proceedings
(pp. 276–281). IEEE.
Kawashima, W., Hatake, K., Kudo, R., Nakanishi, M., Tamaki, S. et al. (2015).
Estimating the time after death on the basis of corneal opacity. Journal of For-
ensic Research, 6(1), 1–5.
Kaye, D., LeBrecht, J., & Budries, D. (2016). Sound and music for the theatre: The
art and technique of design (4th ed.). New York: Routledge.
Kayed, N. S., Farstad, H., & Van der Meer, A. L. H. (2008). Preterm infants’
timing strategies to optical collisions. Early Human Development, 84,
381–388.
Kayed, N. S., & Van der Meer, A. L. H. (2000). Timing strategies used in defensive
blinking to optical collisions in 5- to 7-month-old infants. Infant Behavior &
Development, 23, 253–270.
Kayed, N. S., & Van der Meer, A. L. H. (2007). Infants’ timing strategies to optical
collisions: A longitudinal study. Infant Behavior & Development, 30, 50–59.
References 309
Kayed, N. S., & Van der Meer, A. L. H. (2009). A longitudinal study of prospec-
tive control in catching by full-term and preterm infants. Experimental Brain
Research, 194, 245–258.
Kellman, P. J., Garrigan, P. B., & Shipley, T. F. (2005). Object interpolation in
three dimensions. Psychological Review, 112, 586–609.
Kellman, P. J., & Shipley, T. F. (1992). Visual interpolation in object perception.
Current Directions in Psychological Science, 1, 193–199.
Kelly, J. W., Cherep, L. A., & Siegel, Z. D. (2017). Perceived space in the HTC
vive. ACM Transactions on Applied Perception, 15, 1–17.
Kepler, J. (1604/1996). Astronomiae Pars Optica (The optical part of astronomy).
In A. C. Crombie (1996). Science, art and nature in medieval and modern
thought. London: Hambledon Press.
Kim, N-G. (2008). Dynamic occlusion and optical flow from corrugated surfaces.
Kim, N. G., Effken, J., & Shaw, R. (1995). Perceiving persistence under change
and over structure. Ecological Psychology, 7, 217–256.
Kim, N. G., & Kim, H. (2017). Schizophrenia: An impairment in the capacity to per-
ceive affordances. Frontiers in Psychology, 8. doi:10.3389/fpsyg.2017.01052.
Kim, S., Carello, C., & Turvey, M. T. (2016). Size and distance are perceived inde-
pendently in an optical tunnel: Evidence for direct perception. Vision Research,
125, 1–11.
Kinsella-Shaw, J. M., Shaw, B., & Turvey, M. T. (1992). Perceiving ‘walk-on-able’
slopes. Ecological Psychology, 4, 223–239.
Kirschfeld, K. (2017). How we perceive our own retina. Proceedings of the Royal
Society, B, 284, 1904–1909.
Klein, F. C. (1872/2008). A comparative review of recent researches in geometry.
arXiv preprint arXiv:0807.3161.
Knapp, J. M., & Loomis, J. M. (2004). Limited field of view of head-mounted dis-
plays is not the cause of distance underestimation in virtual environments. Pres-
ence: Teleoperators & Virtual Environments, 13, 572–577.
Koenderink, J. J. (1986). Optic flow. Vision Research, 26, 161–180.
Koenderink, J. J., van Doorn, A. J., Kappers, A. M. L., & Todd, J. T. (2001).
Ambiguity and the ‘mental eye’ in pictorial relief. Perception, 30, 431–448.
Koenig, D., & Hofer, H. (2011). The absolute threshold of cone vision. Journal of
Vision, 11, 1–24.
Koffka, K. (1935). The principles of Gestalt psychology. New York: Harcourt-Brace.
Kohler, I. (1964). The formation and transformation of the perceptual world.
Psychological Issues, 3, 1–173.
Konczak, J., Meeuwsen, H. J., & Cress, M. E. (1992). Changing affordances in
stair climbing: The perception of maximum climbability in young and older
adults. Journal of Experimental Psychology: Human Perception and Perform-
ance, 18, 691–697.
Kretch, K. S., & Adolph, K. E. (2013). Cliff or step? Posture-specific learning at the
edge of a drop-off. Child Development, 84, 226–240.
Kretch, K. S., & Adolph, K. E. (2015). Active vision in passive locomotion: Real-
world free viewing in infants and adults. Developmental Science, 18, 736–750.
doi:10.1111/desc.12251.
Kretch, K. S., & Adolph, K. E. (2017). The organization of exploratory behaviors
in infant locomotor planning. Developmental Science, 20, e12421.
310 References
Kretch, K. S., Franchak, J. M., & Adolph, K. E. (2014). Crawling and walking
infants see the world differently. Child Development. doi:10.1111/cdev.12206.
Kugler, P. N., Turvey, M. T., Carello, C., & Shaw, R. E. (1985). The physics of
controlled collisions: A reverie about locomotion. In Persistence and change:
Proceedings of the First International Conference on Event Perception (pp.
195–222). Hillsdale, NJ: Erlbaum.
Kull, K. (2009). Umwelt and modelling. In P. Cobley (Ed.), The Routledge hand-
book of semiotics. (pp. 43–56). New York: Routledge.
Kumin, L., Lazar, J., Feng, J. H., Wentz, B., & Ekedebe, N. (2012). A usability
evaluation of workplace-related tasks on a multi-touch tablet computer by adults
with Down syndrome. Journal of Usability Studies, 7, 118–142.
Laidlaw, K. E. W., Foulsham, T., Kuhn, G., & Kingstone, A. (2011). Potential
social interactions are important to social attention. Proceedings of the National
Academy of Sciences of the U.S.A., 108, 5548–5553.
Lanczos. C. (1970). The variational principles of mechanics (4th ed.). Mineola,
NY: Dover Publications.
Land, M. F. (1992). Predictable eye-head coordination during driving. Nature,
359, 318–320.
Land, M. F. (2004). The coordination of rotations of the eyes, head and trunk in
saccadic turns produced in natural situations. Experimental Brain Research, 159,
151–160.
Land, M. F. (2006). Eye movements and the control of actions in everyday life.
Progress in Retinal and Eye Research, 25, 296–324.
Land, M. F., & Fernald, R. D. (1992). The evolution of eyes. Annual Review of
Neuroscience, 15, 1–29.
Land, M. F., & Furneaux, S. (1997). The knowledge base of the oculomotor
system. Philosophical Transactions of the Royal Society of London B, 352,
1231–1239.
Land, M. F., & Lee, D. N. (1994). Where we look when we steer. Nature, 369,
742–744.
Land, M. F., Mennie, N., & Rusted, J. (1999). The roles of vision and eye move-
ments in the control of activities of daily living. Perception, 28, 1311–1328.
Land, M. F., & Nilsson, D-E. (2006). General-purpose and special-purpose visual
systems. In E. Warrant & D-E. Nilsson (Eds.) Invertebrate vision (pp. 167–210).
Cambridge: Cambridge University Press.
Langewiesche, W. (1944). Stick and rudder. New York: McGraw-Hill.
Lappin, J. S. (2016). Identifying spatiotemporal information. In J. W. Houpt & L.
M. Blaha (Eds.), Mathematical models of perception and cognition (Vol. I, pp.
121–165): London: Psychology Press.
Lappin, J. S., Shelton, A. L., & Rieser, J. J. (2006). Environmental context influ-
ences visually perceived distance. Attention, Perception, & Psychophysics, 68,
571–581.
Lashley, K. S. (1922). Studies of cerebral function in learning: IV. Vicarious func-
tion after destruction of the visual areas. American Journal of Physiology, 59,
44–71.
Lee, D. N. (1976). A theory of visual control of braking based on information
about time-to-collision. Perception, 5, 437–459.
Lee, D. N. (1980). The optic flow-field: The foundation of vision. Philosophical
Transactions of the Royal Society of London B, 290, 169–179.
References 311
Lee, D. N. (1993). Body-environment coupling. In U. Neisser (Ed.), The perceived
self: Ecological and interpersonal sources of self-knowledge (pp. 43–67). Cam-
bridge, MA: Cambridge University Press.
Lee, D. N. (1998). Guiding movement by coupling taus. Ecological Psychology,
10, 221–250.
Lee, D. N. (2009). General tau theory: Evolution to date. Perception, 38, 837–858.
Lee, D. N., & Aronson, E. (1974). Visual proprioceptive control of standing in
human infants. Perception & Psychophysics, 15, 529–532.
Lee, D. N., & Lishman, J. R. (1975). Visual proprioceptive control of stance.
Journal of Human Movement Studies, 1, 87–95.
Lee, D. N., & Reddish, P. (1981). Plummeting gannets: A paradigm for ecological
optics. Nature, 293, 293–294.
Lee, D. N., Van der Weel, F. R., Hitchcock, T., Matejowsky, E., & Pettigrew, J. D.
(1992). Common principle of guidance by echolocation and vision. Journal of
Comparative Physiology A, 171, 563–571.
Lee, D. N., Young, D. S., and McLaughlin, C. M. (1984). A roadside simulation of
road crossing for children. Ergonomics, 27, 1271–1281.
Lee, Y., Lee, S., Carello, C., & Turvey, M. T. (2012). An archer’s perceived form
scales the “hitableness” of archery targets. Journal of Experimental Psychology:
Le Poidevin, R. (2019). The experience and perception of time. In Edward N . Zalta
(Ed.), The Stanford encyclopedia of philosophy, summer 2019 ed.). Retrieved from
https://plato.stanford.edu/archives/sum2019/entries/time-experience/
Leudar, I., & Costall, A. (2009). Introduction: Against ‘theory of mind.’ In I.
Leudar & A. Costall (Eds.), Against theory of mind (pp. 1–15). Basingstoke: Pal-
grave Macmillan.
Li, C.-L., Pilar Aivar, M., Kit, D. M., Tong, M. H., & Hayhoe, M. M. (2016).
Memory and visual search in naturalistic 2D and 3D environments. Journal of
Vision, 16, 1–20.
Li, D., Babcock, J., & Parkhurst, D. J. (2006). openEyes: A low-cost head-mounted
eye-tracking solution. Proceedings of the 2006 Symposium on Eye Tracking
Research and Applications.
Li, Z., & Durgin, F. H. (2010). Perceived slant of binocularly viewed large-scale surfaces:
A common model from explicit and implicit measures. Journal of Vision, 10, 13–13.
Li, Z., Phillips, J., & Durgin, F. H. (2011). The underestimation of egocentric dis-
tance: Evidence from frontal matching tasks. Attention, Perception, & Psycho-
physics, 73, 2205–2217.
Lindberg, D. C. (1968). The theory of pinhole images from antiquity to the thir-
teenth century. Archive for History of Exact Sciences, 5, 154–176.
Lindberg, D. C. (1976). Theories of vision from Al-Kindi to Kepler. Chicago:
University of Chicago Press.
Lishman, J. R., & Lee, D. N. (1973). The autonomy of visual kinaesthesis. Percep-
tion, 2, 287–294.
Littman, E. (2011). Adaptation to simultaneous multi-dimensional distortions.
(electronic thesis or dissertation). Miami University, OH. Retrieved from https://
etd.ohiolink.edu/.
Littman, E. M., Otten, E. W., & Smart, L. J. (2010). Consequence of self versus
externally generated visual motion on postural regulation. Ecological Psych-
ology, 22, 150–167.
312 References
Lockman, J. J., Fears, N. E., & Jung, W. P. (2018). The development of object
fitting: The dynamics of spatial coordination (Vol. 55). Amsterdam: Elsevier.
Lombardo, T. J. (1987). The reciprocity of perceiver and environment. Hillsdale,
NJ: Lawrence Erlbaum Associates.
Long, L. O., Pagano, C. C, Singapogu, R. B., & Burg, T. C. (2016). Surgeons’ per-
ception of soft tissue constraints and distance-to-break in a simulated minimally
invasive surgery Task. Proceedings of the Human Factors and Ergonomics
Society 2016 Annual Meeting (pp. 1598–1602). Washington, DC, September
19–23, 2016.
Longo, M. R., & Haggard, P. (2010). An implicit body representation underlying
human position sense. Proceedings of the National Academy of Sciences of the
U.S.A., 107, 11727–11732.
Loomis, J. M. (2014). Three theories for reconciling the linearity of egocentric dis-
tance perception with distortion of shape on the ground plane. Psychology &
Neuroscience, 7, 245–251.
Loomis, J. M. (2016). Presence in virtual reality and everyday life: Immersion
within a world of representation. Presence: Teleoperators and Virtual Environ-
ments, 25, 169–174.
Loomis, J. M., DaSilva, J. A., Fujita, N., & Fukusima, S. S. (1992). Visual space
perception and visually directed action. Journal of Experimental Psychology:
Loomis, J. M., & Knapp, J. M. (2003). Visual perception of egocentric distance in
real and virtual environments. In L. J. Hettinger & M. W. Haas (Eds.), Virtual
and adaptive environments (pp. 21–46). Mahwah, NJ: Erlbaum.
Loomis, J. M., & Philbeck, J. W. (1999). Is the anisotropy of perceived 3-D shape
invariant across scale? Perception & Psychophysics, 61, 397–402.
Loomis, J. M., & Philbeck, J. W. (2008). Measuring spatial perception with
spatial updating and action. In R. L. Klatzky, B. MacWhinney, & M. Behr-
mann (Eds.), Embodiment, ego-space, and action (pp. 1–43). New York:
Psychology Press.
Loomis, J. M., Philbeck, J. W., & Zahorik, P. (2002). Dissociation between loca-
tion and shape in visual space. Journal of Experimental Psychology: Human Per-
ception and Performance, 28, 1202–1212.
Luck, S. J. (2005). An introduction to the event-related potential technique. Cam-
bridge, MA: MIT Press.
Luneburg, R. K. (1947). Mathematical analysis of binocular vision. Princeton, NJ:
Princeton University Press.
Luo, C., & Franchak, J. M. (in preparation). Infant eye-head alignment to objects
and faces during free play.
Mace, W. M. (1977). James J. Gibson’s strategy for perceiving: Ask not what’s
inside your head, but what your head’s inside of. In R. E. Shaw & J. Bransford
(Eds.), Perceiving, acting, and knowing (pp. 43–65). Hillsdale, NJ: Lawrence
Erlbaum Associates, Inc.
Maidenbaum, S., Hanassy, S., Abboud, S., Buchs, G., Chebat, D. R., Levy-Tzedek,
S., & Amedi, A. (2014). The “EyeCane”, a new electronic travel aid for the
blind: Technology, behavior & swift learning. Restorative Neurology and Neuro-
science, 32, 813–824.
Makeig, S., Gramann, K., Jung, T. P., Sejnowski, T. J., & Poizner, H. (2009). Linking
brain, mind and behavior. International Journal of Psychophysiology, 73, 95–100.
References 313
Makin, A. D. J. (2018). The common rate control account of prediction motion.
Psychonomic Bulletin & Review, 25, 1784–1797.
Mangalam, M., Wagman, J. B., & Newell, K. M. (2018). Temperature influences
perception of the length of a wielded object via effortful touch. Experimental
Brain Research, 236, 505–516.
Mantel, B., Hoppenot, P., & Colle, P. (2012). Perceiving for acting with teleoper-
ated robots: Ecological principles to human–robot interaction design. IEEE
Transactions on Systems, Man, and Cybernetics: Part A: Systems and Humans,
42, 1460–1475.
Mantel, B., Stoffregen, T. A., Campbell, A., & Bardy, B. G. (2015). Exploratory
movement generates higher-order information that is sufficient for accurate per-
ception of scaled egocentric distance. PLoS ONE, 10, e0120025.
Maravita, A., & Iriki, A. (2004). Tools for the body (schema). TRENDS in Cogni-
tive Sciences, 8, 79–86.
Marino, C. J., & Mahan, R. P. (2005). Configural displays can improve nutrition-
related decisions: An application of the proximity compatibility principle.
Human Factors, 47, 121–130.
Mark, L. S. (1987). Eyeheight-scaled information about affordances: A study of
sitting and stair climbing. Journal of Experimental Psychology: Human Percep-
tion and Performance, 13, 361–370.
Mark, L. S. (2007). Perceiving the actions of other people. Ecological Psychology,
19, 107–136.
Mark, L. S., Balliett, J. A., Craver, K. D., Douglas, S. D., & Fox, T. (1990). What
an actor must do in order to perceive the affordance for sitting. Ecological
Mark, L. S., Nemeth, K., Gardner, D., Dainoff, M. J., Paasche, J., Duffy, M., &
Grandt, K. (1997). Postural dynamics and the preferred critical boundary for vis-
ually guided reaching. Journal of Experimental Psychology: Human Perception
and Performance, 23, 1365–1379.
Mark, L. S., Shaw, R., & Pittenger, J. (1988): Natural constraints, scales of ana-
lysis, and information for the perception of growing faces. In T. Alley (Ed.),
Social and applied aspects of perceiving faces (pp. 11–49). Hillsdale, NJ:
Erlbaum.
Marr, D. (1982). Vision. San Francisco, CA: W. H. Freeman.
Marsh, K. L., Richardson, M. J., Baron, R. M., & Schmidt, R. C. (2006). Contrasting
approaches to perceiving and acting with others. Ecological Psychology, 18, 1–38.
Marratto, S. L. (2012). The intercorporeal self: Merleau-Ponty and subjectivity.
Albany, NY: SUNY Press.
Marshall, W. E. (2018). Understanding international road safety disparities: Why
is Australia so much safer than the United States? Accident Analysis & Preven-
tion, 111, 251–265.
Matin, L., & Fox, C. R. (1989). Visually perceived eye level and perceived eleva-
tion of objects: Linearly additive influences from visual field pitch and gravity.
Vision Research, 29, 315–324.
Matthis, J. S., & Fajen, B. R. (2014). Visual control of foot placement when
walking over complex terrain. Journal of Experimental psychology: Human Per-
ception and Performance, 40, 106–115.
Matthis, J. S., Yates, J. L., & Hayhoe, M. M. (2018). Gaze and the control of foot
placement when walking in natural terrain. Current Biology, 28, 1224–1233.
314 References
Maturana, H. R., & Varela, F. J. (1980). Autopoiesis and cognition: The realiza-
tion of the living., London: D. Reidel.
McCann, B. C., Hayhoe, M. M., & Geisler, W. S. (2018). Contributions of monoc-
ular and binocular cues to distance discrimination in natural scenes. Journal of
Vision, 18(4):12, doi:10.1167/18.4.12.
McCarty, M. E., Clifton, R. K., & Collard, R. R. (1999). Problem solving in infancy:
The emergence of an action plan. Developmental Psychology, 35, 1091–1101.
McCaulay, D. (1978). Great moments in architecture. Boston: Houghton
Mifflin Co.
McGann, M. (2014). Enacting a social ecology: Radically embodied intersubjectiv-
ity. Frontiers in Psychology, 5, 1321.
McPherron, S. P., Alemseged, Z., Marean, C. W., Wynn, J. G., Reed, D., Geraads,
D., et al. (2010). Evidence for stone-tool-assisted consumption of animal tissues
before 3.39 million years ago at Dikika, Ethiopia. Nature, 466, 857–860.
Meeuwsen, H. J., & Cress, M. E. (1992). Changing affordances in stair climbing:
The perception of maximum climbability in young and older adults. Journal of
Meng, J. C., & Sedgwick, H. A. (2001). Distance perception mediated through
nested contact relations among surfaces. Perception & Psychophysics, 63, 1–15.
Meng, J. C., & Sedgwick, H. A. (2002). Distance perception across spatial discon-
tinuities. Perception & Psychophysics, 64, 1–14.
Merleau-Ponty, M. (1962). The phenomenology of perception. London: Routledge
& Kegan Paul.
Messing, R., & Durgin, F. H. (2005). Distance perception and the visual horizon in
head-mounted displays. ACM Transactions on Applied Perception (TAP), 2,
234–250.
Metzger, W. (1930). Optische Untersuchungen im Ganzfeld II [Optical investiga-
tions in the whole field]. Psychologische Forschung, 13, 6–29.
Michaels, C. F. (2003). Affordances: Four points of debate. Ecological Psychology,
15, 135–148.
Michaels, C. F., & Carello, C. (1981). Direct perception. Englewood Cliffs, NJ:
Prentice Hall.
Michaels, C. F., & de Vries, M. M. (1998). Higher order and lower order variables
in the visual perception of relative pulling force. Journal of Experimental Psych-
Michotte, A. (1963). The perception of causality (T. R. Miles & E. Miles, Trans.).
London: Methuen.
Michotte, A., Thinès, G. O., & Crabbé, G. (1964). Les compléments amodaux des
structures perceptives. Louvain: Institut de psychologie de l’Université de Louvain.
Mohler, B. J., Creem-Regehr, S. H., & Thompson, W. B. (2006). The influence of
feedback on egocentric distance judgments in real and virtual environments. Pro-
ceedings of the 3rd Symposium on Applied Perception in Graphics and Visuali-
zation (pp. 9–14). ACM.
Momtahan, K., & Burns, C. M. (2004). Applications of ecological interface design
in supporting the nursing process. Journal of Healthcare Information Manage-
ment, 18, 74–82.
Mon-Williams, M., & Bingham, G. P. (2007). Calibrating reach distance to visual
targets. Journal of Experimental Psychology: Human Perception and Perform-
ance, 33, 645–656.
References 315
Moore, M. K, Borton, R., & Darby, B. L. (1978). Visual tracking in young infants:
Evidence for object identity or object permanence? Journal of Experimental
Child Psychology, 25, 183–198.
Moore, M. K., & Meltzoff, A. N. (1999). New findings on object permanence: A
developmental difference between two types of occlusion. British Journal of
Developmental Psychology, 17, 563–584.
Munafo, J., Diedrick, M., & Stoffregen, T. A. (2017). The virtual reality
head‑mounted display Oculus Rift induces motion sickness and is sexist in its
effects. Experimental Brain Research, 235, 889–901.
Murch, W. (2001). In the blink of an eye: A perspective on film editing. Holly-
wood, CA: Silman-James Press.
Nakayama, K. (1994). James J. Gibson; An appreciation. Psychological Review,
101, 329–335.
Neisser, U. (1967). Cognitive psychology. Englewood Cliffs, NJ: Prentice-Hall.
Neisser, U. (1976). Cognition and reality. San Francisco: W. H. Freeman.
Newton, I. (1704/1952). Opticks or a treatise of the reflections, refractions, inflec-
tions & colors of light. New York: Dover Publications.
Ng, R., Levoy, N., Brédif, M., Duval, G., Horowitz, M., & Hanrahan, P. (2005).
Light field photography with a hand-held plenoptic camera. Stanford Tech
Report CTSR 2005–02.
Ni, R., Braunstein, M. L., & Andersen, G. J. (2004). Perception of scene layout
from optical contact, shadows, and motion. Perception, 33, 1305–1318.
Nixon, M. S., & Aguado, A. S. (2012). Feature extraction & image processing for
computer vision. Amsterdam: Elsevier.
Noether, E. (1918). Invariante Variationsprobleme. Nachricht der Königliche Ges-
ellschaft des Wissens zu Göttingen, Math-phys. Klasse (1918) (pp. 235–257).
English Reprint: physics/0503066. doi:org/10.1080/00411457108231446, 57.
Nonaka, T. (2012). What exists in the environment that motivates the emergence,
transmission, and sophistication of tool use? Behavioral and Brain Sciences, 35,
233–234.
Nonaka, T., & Bril, B. (2014). Fractal dynamics in dexterous tool use: The case of
hammering behavior of bead craftsmen. Journal of Experimental Psychology:
Nonaka, T., Bril, B., & Rein, R. (2010). How do stone knappers predict and
control the outcome of flaking? Implications for understanding early stone tool
technology. Journal of Human Evolution, 59, 155–167.
Norman, D. (2013). The design of everyday things. (Rev. ed.). New York: Basic
Books.
Norman, J. F., Adkins, O. C., Dowell, C. J., Shain, L. M., Hoyng, S. C., &
Kinnard, J. D. (2017). The visual perception of distance ratios outdoors. Atten-
tion, Perception & Psychophysics, 79, 1195–1203.
Norman, J. F., Crabtree, C. E., Clayton, A. M., & Norman, H. F. (2005). The per-
ception of distances and spatial relationships in natural outdoor environments.
Perception, 34, 1315–1324.
Norman, J. F., Todd, J. T., Perotti, V. J., & Tittle, J. S. (1996). The visual percep-
tion of three-dimensional length. Journal of Experimental Psychology: Human
Ohno, H., Rogers, B. J., Ohmi, M., & Ono, M. E. (1988). Dynamic occlusion and
motion parallax in depth perception. Perception, 17, 255–266.
316 References
Ooi, T. L., & He, Z. J. (2007). A distance judgment function based on space per-
ception mechanisms: Revisiting Gilinsky’s (1951) equation. Psychological
Review, 144, 441–454.
Ooi, T. L., Wu, B., & He, Z. J. (2001). Distance determined by the angular decli-
nation below the horizon. Nature, 414, 197–200.
Ooi, T. L., Wu, B., & He, Z. J. (2006). Perceptual space in the dark affected by the
intrinsic bias of the visual system. Perception, 35(5), 605–624.
Oomen, B. S., Smith, R. M., & Stahl, J. S. (2004). The influence of future gaze ori-
entation upon eye-head coupling during saccades. Experimental Brain Research,
155, 9–18.
O’Shea, R. P., & Ross, H. E. (2007). Judgments of visually perceived eye level
(VPEL) in outdoor scenes: Effects of slope and height. Perception, 36,
1168–1178.
Oudejans, R. R., Michaels, C. F., Bakker, F. C., & Dolné, M. A. (1996). The relev-
ance of action in perceiving affordances: Perception of catchableness of fly balls.
879–891.
Pagano, C. C., & Bingham, G. P. (1998). Comparing measures of monocular dis-
tance perception: Verbal and reaching errors are not correlated. Journal of
Pagano, C. C., Grutzmacher, R. P., & Jenkins, J. C. (2001). Comparing verbal and
reaching responses to visually perceived egocentric distances. Ecological Psych-
ology, 13, 197–226.
Pagano, C. C., & Turvey, M. T. (1998). Eigenvectors of the inertia tensor and per-
ceiving the orientation of limbs and objects. Journal of Applied Biomechanics,
14, 331–359.
Palatinus, Z., Kelty-Stephen, D., Kinsella-Shaw, J., Carello, C., & Turvey, M. T.
(2014). Haptic perceptual intent in quiet standing affects multifractal scaling of
postural fluctuations. Journal of Experimental Psychology: Human Perception
& Performance, 40, 1808–1818.
Palmer, E. M., Kellman, P. J., & Shipley, T. F. (2006). A theory of dynamic
occluded and illusory object perception. Journal of Experimental Psychology:
General, 135, 513–541.
Palmer, S. E. (1999). Vision science. Cambridge, MA: MIT Press.
Pan, J. S., Bingham, N., & Bingham, G. P. (2013). Embodied memory: Effective
and stable perception by combining flow and image structure. Journal of Experi-
Pan, J. S., Bingham, N., & Bingham, G. P. (2017). Embodied memory allows
accurate and stable perception of hidden objects despite orientation change.
1343–1358.
Pan, J. S., Bingham, N., Chen, C., & Bingham, G. P. (2017). Breaking camouflage
and detecting targets require optic flow and image structure information. Applied
Optics, 56, 6410–6418.
Pan, J. S., Coats, R. O., & Bingham, G. P. (2014). Calibration is action specific but
perturbation of perceptual units is not. Journal of Experimental Psychology:
Passos, P., Cordovil, R., Fernandes, O., & Barreiros, J. (2012). Perceiving
affordances in rugby union. Journal of Sports Sciences, 30, 1175–1182.
References 317
Patla, A. E., & Vickers, J. N. (1997). Where and when do we look as we approach
and step over an obstacle in the travel path? Neuroreport, 8, 3661–3665.
Peirce, C. S. (1955). The fixation of belief. In J. Buchler (Ed.), Philosophical writ-
ings of Peirce. New York: Dover Publications.
Pelegrin, J. (2005). Remarks about archaeological techniques and methods of knap-
ping: Elements of a cognitive approach to stone knapping. In V. Roux & B. Bril
(Eds.), Stone knapping: The necessary conditions for a uniquely hominin behav-
ior (pp. 35–52). Cambridge: McDonald Institute for Archaeological Research.
Pelz, J. B., & Canosa, R. (2001). Oculomotor behavior and perceptual strategies in
complex tasks. Vision Research, 41, 3587–3596.
Pelz, J. B., Hayhoe, M. M., & Loeber, R. (2001). The coordination of eye, head,
and hand movements in a natural task. Experimental Brain Research, 139,
266–277.
Petrusz, S. C., & Turvey, M. T. (2010). On the distinctive features of ecological
laws. Ecological Psychology, 22, 44–68.
Philbeck, J. W., & Loomis, J. M. (1997). Comparison of two indicators of per-
ceived egocentric distance under full-cue and reduced-cue conditions. Journal of
Philbeck, J. W., Loomis, J. M., & Beall, A. C. (1997). Visually perceived location is
an invariant in the control of action. Perception and Psychophysics, 59,
601–612.
Pijpers, J. R., Oudejans, R. R., & Bakker, F. C. (2007). Changes in the perception
of action possibilities while climbing to fatigue on a climbing wall. Journal of
Sports Sciences, 25, 97–110.
Pittenger, J., & Shaw, R. (1975). Aging faces as viscal-elastic events: Implications
for a theory of non-rigid shape perception. Journal of Experimental Psychology:
Human Performance and Perception, 1, 374–382.
Plumert, J. M., Kearney, J. K., & Cremer, J. F. (2004). Children’s perception of
gap affordances: Bicycling across traffic-filled intersections in an immersive
virtual environment. Child Development, 75, 1243–1253.
Powys, V. Taylor, H., & Probets, C. (2013). A little flute music: Mimicry, memory,
and narrativity. Environmental Humanities, 3, 43–70.
Prieske, B., Withagen, R., Smith, J., & Zaal, F. T. (2015). Affordances in a simple
playscape: Are children attracted to challenging affordances?. Journal of
Environmental Psychology, 41, 101–111.
Profeta, V. L., & Turvey, M. T. (2018). Bernstein’s levels of movement construc-
tion: A contemporary perspective. Human Movement Science, 57, 111–133.
Proffitt, D. R. (1977). Demonstrations to investigate the meaning of everyday
experience (Doctoral dissertation, ProQuest Information & Learning).
Proffitt, D. R. (2006). Embodied perception and the economy of action. Perspec-
tives on Psychological Science, 1, 110–122.
Proffitt, D. R., & Linkenauger, S. A. (2013). Perception viewed as a phenotypic
expression. In W. Prinz, M. Beisert, & A. Herwig (Eds.), Action science: Founda-
tions of an emerging discipline. Cambridge, MA: MIT Press.
Purdy, J., & Gibson, E. J. (1955). Distance judgment by the method of fractiona-
tion. Journal of Experimental Psychology, 50(6), 374.
Purdy, W. P. (1958). The hypothesis of psychophysical correspondence in space
perception. Dissertation Abstracts International, 19, 1454–1455. (University
Microfilms No. 1458–5594.)
318 References
Rachwani, J., Golenia, L., Herzberg, O., & Adolph, K. E. (in press). Postural, visual,
and manual coordination in the development of prehension. Child Development.
Rader, N. d. V. (2018). Uniting Jimmy and Jackie: Foundation for a research program
in developmental ecological psychology. Ecological Psychology, 30, 129–145.
Rader, N. de V., & Zukow-Goldring, P. (2012). Caregivers’ gestures direct infant
attention during early work learning: The importance of dynamic synchrony.
Language Sciences, 34, 559–568.
Ramenzoni, V. C., Davis, T. J., Riley, M. A., & Shockley, K. (2010). Perceiving
action boundaries: Learning effects in perceiving maximum jumping-reach
affordances. Attention, Perception, & Psychophysics, 72, 1110–1119.
Ramenzoni, V. C., Riley, M. A., Davis, T., Shockley, K., & Armstrong, R. (2008).
Tuning in to another person’s action capabilities: Perceiving maximal jumping-
reach height from walking kinematics. Journal of Experimental Psychology:
Human Perception & Performance, 34, 919–928.
Ramenzoni, V. C., Riley, M. A., Shockley, K., & Davis, T. (2008). An information-
based approach to action understanding. Cognition, 106, 1059–1070.
Rasouli, O., Stensdotter, A.-K., & Van der Meer, A. L. H. (2016). TauG-guidance
of dynamic balance control during gait initiation in patients with chronic fatigue
syndrome and fibromyalgia. Clinical Biomechanics, 37, 147–152.
Reed, E. S. (1982a). Darwin’s earthworms: A case study in evolutionary psych-
ology. Behaviorism, 10, 165–185.
Reed, E. S. (1982b). An outline of a theory of action systems. Journal of Motor
Behavior, 14, 98–134.
Reed, E. S. (1988). James J. Gibson and the psychology of perception. New Haven,
CT: Yale University Press.
Reed, E. S. (1996a). The necessity of experience. New Haven, CT: Yale University
Press.
Reed, E. S. (1996b). Encountering the world: Toward an ecological psychology.
New York: Oxford University Press.
Regia-Corte, T., & Wagman, J. B. (2008). Perception of affordances for standing
on an inclined surface depends on height of center of mass. Experimental Brain
Research, 191, 25–35.
Režek, Ž., Dibble, H. L., McPherron, S. P., Braun, D. R., & Lin, S. C. (2018). Two
million years of flaking stone and the evolutionary efficiency of stone tool tech-
nology. Nature Ecology & Evolution, 2, 628–633.
Riccio, G. E., & Stoffregen, T. A. (1991). An ecological theory of motion sickness
and postural instability. Ecological Psychology, 3, 195–240.
Richardson, A. R., & Waller, D. (2007). Interaction with an immersive virtual
environment corrects users’ distance estimates. Human Factors, 47, 507–517.
Richardson, M. J., Marsh, K. L., & Baron, R. M. (2007). Judging and actualizing
intrapersonal and interpersonal affordances. Journal of Experimental Psych-
Richardson, M. J., Marsh, K. L., & Schmidt, R. C. (2010). Challenging the egocen-
tric view of coordinated perceiving, acting, and knowing. In B. Mesquita, L. F.
Barrett, & E. Smith (Eds.), The mind in context (pp. 307–333). New York: The
Guilford Press.
Rieffel, J. A., Valero-Cuevas, F. J., & Lipson, H. (2010). Morphological communi-
cation: Exploiting coupled dynamics in a complex mechanical structure to
achieve locomotion. Journal of the Royal Society Interface, 7, 613–621.
References 319
Riegler, A. (2005). The constructivist challenge. Constructivist Foundations, 1, 1–8.
Rieser, J. J., Ashmead, D. H., Talor, C. R., & Youngquist, G. A. (1990). Visual
perception and the guidance of locomotion without vision to previously seen
targets. Perception, 19, 675–689.
Rieser, J. J., Pick, H. L., Ashmead, D. H., & Garing, A. E. (1995). Calibration of
human locomotion and models of perceptual-motor organization. Journal of
Rietveld, E. (2016). Situating the embodied mind in a landscape of standing
affordances for living without chairs: Materializing a philosophical worldview.
Sports Medicine, 46, 27–932.
Rietveld, E., & Kiverstein, J. (2014). A rich landscape of affordances. Ecological
Riley, M. A., Richardson, M. J., Shockley, K., & Ramenzoni, V. C. (2011). Inter-
personal synergies. Frontiers in Psychology, 2, 38.
Riley, M. A., Wagman, J. B., Santana, M. V., Carello, C., & Turvey, M. T. (2002).
Perceptual behavior: Recurrence analysis of a haptic exploratory procedure. Per-
ception, 31, 481–510.
Roche, H. (2005). From simple flaking to shaping: Stone-knapping evolution
among early hominins. In V. Roux & B. Bril (Eds.), Stone knapping: The neces-
sary conditions for a uniquely hominin behavior (pp. 35–52). Cambridge:
McDonald Institute for Archaeological Research.
Roche, H., Blumenschine, R. J., & Shea, J. J. (2009). Origins and adaptations of
early Homo: What archeology tells us. In F. E. Grine, J. G. Fleagle, & R. E.
Leakey (Eds.), The first humans: Origin and early evolution of the genus homo
(pp. 135–150). New York: Springer.
Roche, H., Delagnes, A., Brugal, J. P., Feibel, C., Kibunjia, M., Mourre, V., &
Texier, P. J. (1999). Early hominid stone tool production and technical skill 2.34
Myr ago in West Turkana, Kenya. Nature, 399, 57–60.
Rodegerdts, L., Bansen, J., Tiesler, C., Knudsen, J., Myers, E., Johnson, M., et al.
(2010). Roundabouts: An informational guide (2nd ed.). Washington, DC:
Transportation Research Board.
Rogers, B. J. (1984). Dynamic occlusion, motion parallax, and the perception of
3-D surfaces, Perception, 13, A46.
Rogers, B. J., & Graham, M. E. (1983). Dynamic occlusion in the perception of
depth structure. Perception, 12, A15.
Rogers, S. (1996). The horizon-ratio relation as information for relative size in pic-
tures. Perception & Psychophysics, 58, 142–152.
Rogoff, B. (1995). The cultural nature of human development. New York: Oxford
University Press.
Rolnick, A., & Bles, W. (1989). Performance and well-being under tilting con-
ditions: The effects of visual reference and artificial horizon. Aviation, Space, and
Environmental Medicine, 60, 779–785.
Ronchi, V. (1957). Optics: The science of vision (E. Rosen, Trans.). New York:
New York University Press. (Original work published in 1955).
Rosander, K., & von Hofsten, C. (2004). Infants’ emerging ability to represent
occluded object motion, Cognition, 91, 1–22.
Rosenbaum, D. A. (2010). Walking down memory lane: Where walkers look as
they descend stairs provides hints about how they control their walking behav-
ior. The American Journal of Psychology, 122, 425–430.
320 References
Rosenblum, L. D. (2010). See what I’m saying: The extraordinary power of our
five senses. New York: W. W. Norton.
Rothkopf, C. A., Ballard, D. H., & Hayhoe, M. M. (2007). Task and context
determine where you look. Journal of Vision, 7, 1–20.
Rothwell, J., & Atlas, R. [Directors] (2010). Sour Grapes. Screenplay by Al
Morrow. London: Met Film Production.
Ruda, R., Livitz, G., Riesen, G., & Mingolla, E. (2015). Computational modeling
of depth ordering in occlusion through accretion or deletion of texture. Journal
of Vision, 15(9), 1–23. doi:10.1167/15.9.20.
Ruderman, D. L. (1994). The statistics of natural images. Computation in Neural
Systems, 5, 517–548.
Runeson, S. (1977). On the possibility of “smart” perceptual mechanisms. Scan-
dinavian Journal of Psychology, 18, 172–179.
Runeson, S. (1988). The distorted room illusion, equivalent configurations, and the
specificity of static optic arrays. Journal of Experimental Psychology: Human
Runeson, S., & Frykholm, G. (1983). Kinematic specification of dynamics as an
informational basis for person-and-action perception: Expectation, gender recog-
nition, and deceptive intention. Journal of Experimental Psychology: General,
112, 585–615.
Runeson, S., & Vedeler, D. (1993). The indispensability of pre-collision kinematics in
the visual perception of relative mass. Perception & Psychophysics, 53, 617–633.
Russell, M. K., & Turvey, M. T. (1999). Auditory perception of unimpeded
passage. Ecological Psychology, 11, 175–188.
Saab Affiches. (2018, April 17). Retrieved from http://saabactu.blogspot.
com/2010/08/saab-affiches-saab-vs-bahaus.html.
Sakitt, B. (1972). Counting every quantum. Journal of Physiology, 223, 131–150.
Scarfe, P., & Glennerster, A. (2015). Using high-fidelity virtual reality to study per-
ception in freely moving observers. Journal of Vision, 15(9), 1–11.
doi:10.1167/15.9.3.
Schiff, W. (1965). Perception of impending collision: A study of visually directed
avoidant behavior. Psychological Monographs: General and Applied, 79, 1–26.
Schiff, W., Caviness, J. A., & Gibson, J. J. (1962). Persistent fear responses in
rhesus monkeys to the optical stimulus of ” looming”. Science, 136, 982–983.
Schmidt, R. A., & Lee, T. D. (2011). Motor control and learning: A behavioral
emphasis (5th ed.). Champaign, IL: Human Kinetics.
Schmuckler, M. A., Collimore, L. M., & Dannemiller, J. L. (2007). Infants’ reac-
tions to object collision on hit and miss trajectories. Infancy, 12, 105–118.
Scholl, B. J., & Pylyshyn, Z. W. (1999). Tracking multiple items through occlusion:
Clues to visual objecthood. Cognitive Psychology, 38, 259–290.
Sedgwick, H. A. (1973). The visible horizon: A potential source of visual informa-
tion for the perception of size and distance. (Doctoral dissertation), Cornell
University.
Sedgwick, H. A. (1986). Space perception. In K. R. Boff, L. Kaufman, & J. P.
Thomas (Eds.), Handbook of perception and human performance, Vol. 1:
Sensory processes and perception (pp. 21.21–21.57). New York: Wiley.
Seifert, L., Cordier, R., Orth, D., Courtine, Y., & Croft, J. L. (2017). Role of route
previewing strategies on climbing fluency and exploratory movements. PloS
ONE, 12(4), e0176306.
References 321
Sekuler, A., & Palmer, S. (1992). Perception of partly occluded objects: A microge-
netic analysis. Journal of Experimental Psychology: General, 121, 95–111.
Sekuler, R., & Blake, R. (2002). Perception. New York: McGraw-Hill.
Semaw, S., Renne, P., Harris, J. W., Feibel, C. S., Bernor, R. L., Fesseha, N., &
Mowbray, K. (1997). 2.5-million-year-old stone tools from Gona, Ethiopia.
Nature, 385, 333–336.
Shannon, C., & Weaver, W. (1949). The mathematical theory of information.
Urbana, IL: University of Illinois Press.
Shapiro, M. A., & McDonald, D. G. (1992). I’m not a real doctor, but I play one
in virtual reality: Implications of virtual reality for judgments about reality.
Journal of Communication, 42, 94–114.
Shaw, R. E. (2002). Theoretical hubris and the willingness to be radical. Ecological
Shaw, R. E., & Bransford, J., (Eds.) (1977). Perceiving, acting, and knowing:
Towards an ecological psychology. Hillsdale, NJ: Lawrence Erlbaum Associates.
Shaw, R. E., Flascher, O., & Mace, W. M. (1996). Dimensions of event perception.
In W. Prinz & B. Bridgeman (Eds.) Handbook of perception and action (Vol. 1,
pp. 345–395). London: Academic Press.
Shaw, R., & Kinsella-Shaw J. (1988). Ecological mechanics: A physical geometry
for intentional constraints. Human Movement Science, 7, 155–200.
Shaw, R. E., & Kinsella-Shaw, J. M. (2007). Could optical ‘pushes’ be inertial
forces? A geometro-dynamical hypothesis. Ecological Psychology, 19, 305–320.
Shaw, R. E., Kinsella-Shaw, J. M., & Mace, W. M. (2019). Affordance types and
tokens: Are Gibson’s affordances trustworthy? Ecological Psychology, 31, 49–75.
Shaw, R. E., & Mace, W. M. (2005). The value of oriented geometry for ecological
psychology and moving image art. In J. Anderson & B. Anderson (Eds.). Moving
image theory: Ecological considerations. (pp. 38–48). Carbondale, IL: Southern
Illinois University Press.
Shaw, R. E., Mark, L. S., Jenkins, H., & Mingolla, E. (1982). A dynamic geometry
for predicting growth of gross craniofacial morphology. In A. Dixon & B. Sarnat
(Eds.), Factors and mechanisms influencing bone growth (pp. 423–431). New
York: Liss.
Shaw, R. E., & McIntyre, M. (1974). Algoristic foundations to cognitive psych-
ology. In W. Weimer, & D. Palermo (Eds.), Cognition and symbolic processes.
Hillsdale, NJ: Lawrence Erlbaum Associates.
Shaw, R. E., McIntyre, M., & Mace, W. (1974). The role of symmetry in event
perception. In R. B. Macleod & H. L. Pick, Jr. (Eds.), Perception: Essays in
honor of James J. Gibson. (pp. 276–310). Ithaca, NY: Cornell University Press.
Shaw, R. E., & Pittenger, J. B. (1977). Perceiving the face of change in changing
faces: Implications for a theory of object perception. In R. Shaw & J. Bransford
(Eds.) Perceiving, acting and knowing: Toward an ecological psychology (pp.
103–132). Hillsdale, NJ: Lawrence Erlbaum Associates.
Shaw, R. E., & Pittenger, J. B. (1978). On perceiving change. In H. Pick & E. Salt-
zman (Eds.), Modes of perceiving and processing information. (pp. 187–204)
Hillsdale, NJ: Lawrence Erlbaum Associates.
Shaw, R. E., Turvey, M. T., & Mace, W. M. (1982). Ecological psychology: The
consequence of a commitment to realism. In W. Weimer & D. Palermo (Eds.),
Cognition and the symbolic processes (Vol. II, pp. 59–226). Hillsdale, NJ: Law-
rence Erlbaum Associates.
322 References
Shimamura, A. P., & Prinzmetal, W. (1999). The Mystery Spot illusion and its
relation to other visual illusions. Psychological Science, 10, 501–507.
Shipley, T. F., & Kellman, P. J. (1992). Perception of partly occluded objects and
illusory figures: Evidence for an identity hypothesis. Journal of Experimental
Siegel, Z. D., & Kelly, J. W. (2017). Walking through a virtual environment
improves perceived size within and beyond the walked space. Attention, Percep-
tion, & Psychophysics, 79, 39–44.
Simonson, E., & Brozek, J. (1952). Flicker fusion frequency: background and
applications. Physiological Reviews, 32, 349–378.
Sinai, M. J., Ooi, T. L., & He, Z. J. (1998). Terrain influences the accurate judg-
ment of distance. Nature, 395, 497–500.
Singleton, R. S. (1991). Film scheduling, or, how long will it take to shoot your
movie? New York: Lone Eagle.
Skinner, B. F. (1977). The experimental analysis of operant behavior. Annals of the
New York Academy of Sciences, 291, 374–385.
Slinning, R., Rutherford, C., & Van der Meer, A. L. H. (2018). Perception of
occlusion of moving objects in young infants: A high-density EEG study. Poster
presented at ICIS Philadelphia 2018, 21st Biennial International Congress on
Infant Studies.
Smart Jr, L. J., Otten, E. W., Strang, A. J., Littman, E. M., & Cook, H. E. (2014).
Influence of complexity and coupling of optic flow on visually induced motion
sickness. Ecological Psychology, 26, 301–324.
Smart, L. J., Stoffregen, T. A., & Bardy, B. G. (2002). Visually induced motion
sickness predicted by postural instability. Human Factors, 44, 451–465.
Smeets, J. B. J., Hayhoe, M. M., & Ballard, D. H. (1996). Goal-directed arm move-
ments change eye-head coordination. Experimental Brain Research, 109, 434–440.
Smith, T. J., & Mital, P. K. (2013). Attentional synchrony and the influence of
viewing task on gaze behavior in static and dynamic scenes. Journal of Vision,
13, 1–24.
Smitsman, A. W., & Corbetta, D. (2010). Action in infancy: Perspectives, concepts,
and challenges. In J. G. Bremner & T. D. Wachs (Eds.), The Wiley-Blackwell
handbook of infant development (2nd ed., Vol. 1, pp. 167–203). Chichester:
Wiley-Blackwell.
Solomon, H. Y., & Turvey, M. T. (1988). Haptically perceiving the distances
reachable with hand-held objects. Journal of Experimental Psychology: Human
Perception & Performance, 14, 404–427.
Sonoda, K., Asakura, A., Minoura, M., Elwood, R. W., & Gunji, Y. P. (2012). Hermit
crabs perceive the extent of their virtual bodies. Biology Letters, 8, 495–497.
Spencer, L. M., & Van der Meer, A. L. H. (2012). TauG-guidance of dynamic balance
control during gait initiation across adulthood. Gait & Posture, 36, 523–526.
Sperling, G. (1960). The information available in brief visual presentations. Psycho-
logical Monographs: General and Applied, 74, 1–29.
Sporrel, K., Caljouw, S. R., & Withagen, R. (2017). Gap-crossing behavior in a
standardized and a nonstandardized jumping stone configuration. PloS ONE,
12(5), e0176165.
Sposito, A., Bolognini, N., Vallar, G., & Maravita, A. (2012). Extension of per-
ceived arm length following tool-use: Clues to plasticity of body metrics. Neu-
ropsychologia, 50, 2187–2194.
References 323
Spröte, P., & Fleming, R. W. (2016). Bent out of shape: The visual inference of
non-rigid shape transformations applied to objects. Vision Research, 126,
330–346.
Stephen, D. G., Arzamarski, R., & Michaels, C. F. (2010). The role of fractality in
perceptual learning: Exploration in dynamic touch. Journal of Experimental
Stepp, N., & Turvey, M. T. (2010). On strong anticipation. Cognitive Systems
Research, 11, 148–164.
Sterelny, K. (2012). The evolved apprentice: How evolution made humans unique.
Cambridge, MA: MIT Press.
Stoffregen, T. A. (1985). Flow structure versus retinal location in the optical
control of stance. Journal of Experimental Psychology: Human Perception and
Stoffregen, T. A. (1993). “Natural”, “real”, and the use of non-physical displays in
perception-action research. International Society for Ecological Psychology
Newsletter, 6, 4–9.
Stoffregen, T. A. (1997). Filming the world: An essay review of Anderson’s The
reality of illusion. Ecological Psychology, 9, 161–177.
Stoffregen, T. A. (2000a). Affordances and events. Ecological Psychology,
12, 1–28.
Stoffregen, T. A. (2000b). Affordances and events: Theory and research. Ecological
Psychology, 12, 93–107 (reply to commentaries).
Stoffregen, T. A. (2003a). Affordances as properties of the animal-environment
system. Ecological Psychology, 15, 115–134.
Stoffregen, T. A. (2003b). Affordances are enough: Reply to Chemero et al. (2003).
Stoffregen, T. A. (2004). Breadth and limits of the affordance concept. Ecological
Stoffregen, T. A. (2013). On the physical origins of inverted optic images. Ecolo-
gical Psychology, 25, 369–382.
Stoffregen, T. A., & Bardy, B. G. (2001). On specification and the senses. Behavi-
oral and Brain Sciences, 24, 195–213.
Stoffregen, T. A., Bardy, B. G., & Mantel, B. (2006). Affordances in the design of
enactive systems. Virtual Reality, 10, 4–10.
Stoffregen, T. A., Bardy, B. G., Merhi, O., & Oullier, O. (2004). Postural responses
to two technologies for generating optical flow. Presence, 13, 601–615.
Stoffregen, T. A., Bardy, B. G. Smart, L. J., & Pagulayan, R. J. (2003). On the
nature and evaluation of fidelity in virtual environments. In L. J. Hettinger & M.
W. Haas (Eds.), Virtual and adaptive environments: Applications, implications,
and human performance issues (pp. 111–128). Mahwah, NJ: Lawrence Erlbaum
Associates.
Stoffregen, T. A., Gorday, K. M., Sheng, Y.-Y., & Flynn, S. B. (1999). Perceiving
affordances for another person’s actions. Journal of Experimental Psychology:
Stoffregen, T. A., & Mantel, B. (2015). Exploratory movement and affordances in
design. Artificial Intelligence for Engineering Design, Analysis and Manufac-
turing, 29, 257–265.
Stoffregen, T. A., Mantel, B., & Bardy, B. G. (2017). The senses considered as one
perceptual system. Ecological Psychology, 29, 165–197.
324 References
Stoffregen, T. A., & Smart Jr, L. J. (1998). Postural instability precedes motion
sickness. Brain Research Bulletin, 47, 437–448.
Stoffregen, T. A., Yang, C. M., Giveans, M. R., Flanagan, M., & Bardy, B. G.
(2009). Movement in the perception of an affordance for wheelchair locomotion.
Stoper, A. E., & Bautista, A. (1992). Apparent height as a function of pitched
environment and task. Paper presented at the Association for Research in Vision
and Ophthalmology, Sarasota, FL.
Stoper, A. E., & Cohen, M. M. (1986). Judgments of eye level in light and in dark-
ness. Perception & Psychophysics, 40, 311–316.
Stoper, A. E., & Cohen, M. M. (1989). Effect of structured visual environments on
apparent eye level. Perception & Psychophysics, 46, 469–475.
Stratton, G. M. (1897). Vision without inversion of the retinal image. Psycho-
logical Review, 4, 463–481.
Swanson, L. W. (2015). Neuroanatomical terminology. New York: Oxford Univer-
sity Press.
Tamboer, J. W. I. (1988). Images of the body underlying concepts of action. In O.
G. Meijer & K. Roth (Eds.), Complex movement behaviour: The motor-action
controversy (pp. 439–462). Amsterdam: Elsevier Science Publishers.
Tanrikulu, O., Froyen, V., Feldman, J., & Singh, M. (2018). When is accreting/
deleting texture seen as in front? Interpretation of depth from texture motion.
Perception, 47, 694–721.
Tatler, B. W. (2007). The central fixation bias in scene viewing: Selecting an
optimal viewing position independently of motor biases and image feature distri-
butions. Journal of Vision, 7, 1–17. doi:10.1167/7.14.4.
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011). Eye guidance
in natural vision: Reinterpreting salience. Journal of Vision, 11, 1–23.
Tatler, B. W., Hirose, Y., Finnegan, S. K., Pievilainen, R., Kirtley, C., & Kennedy,
A. (2013). Priorities for selection and representation in natural tasks. Philo-
sophical Transactions of the Royal Society B, 368, 20130066.
Tatler, B. W., & Land, M. F. (2015). Everyday visual attention. In J. M. Fawcett,
E. F. Risko, & A. Kingstone (Eds.), The handbook of attention. Cambridge, MA:
MIT Press.
Texier, P.-J. (1995). The Oldowan assemblage from NY 18 site at Nyabusosi
(Toro-Uganda). Comptes Rendus de l’Académie des Sciences, Paris, 320, II a,
647–653.
‘t Hart, B. M., & Einhauser, W. (2012). Mind the step: Complementary effects of
an implicit task on eye and head movements in real-life gaze allocation. Experi-
mental Brain Research, 223, 233–249.
Thomas, B. J., Hawkins, M. M., & Nalepka, P. (2017). Perceiver as polar planime-
ter: Direct perception of jumping, reaching, and jump-reaching affordances for
the self and others. Psychological Research, 82, 655–674.
Thomas, B. J., & Riley, M. A. (2014). Remembered affordances reflect the funda-
mentally action-relevant, context-specific nature of visual perception. Journal of
Thomas, B. J., & Riley, M. A. (2015). The selection and usage of information for
perceiving and remembering intended and unintended object properties. Journal
of Experimental Psychology: Human Perception and Performance, 41, 807–815.
doi:10.1037/xhp0000050.
References 325
Thomas, B. J., Wagman, J. B., Hawkins, M., Havens, M., & Riley, M. A. (2017).
The independent perceptual calibration of action-neutral and-referential environ-
mental properties. Perception, 46, 586–604.
Thompson, E. (2007). Mind in life: Biology, phenomenology, and the sciences of
mind. Cambridge, MA: Harvard University Press.
Tinsley, J. M., Molodtsov, M. I., Prevedel, R., Wartmann, D., Espigule-Pons, J.,
Lauwers, M., & Vaziri, A. (2016). Direct detection of a single photon by
humans. Nature Communications, 7(12172), 1–9.
Todd, J. T. (1982). Visual information about rigid and nonrigid motion: A geo-
metric analysis. Journal of Experimental Psychology: Human Perception and
Todd, J. T., Christensen, J. T., & Guckes, K. C. (2010). Are discrimination thresh-
olds a valid measure of variance for judgments of slant from texture? Journal of
Vision, 10(2), 1–18.
Todd, J. T., Oomes, A. H. J., Koenderink, J. J., & Kappers, A. M. L. (2001). On
the affine structure of perceptual space. Psychological Science, 12, 191–196.
Todd, J. T., Thaler, L., & Dijkstra, T. M. H. (2005). The effects of field of view on
the perception of 3D slant from texture. Vision Research, 45, 1501–1517.
Torenvliet, G. (2003). We can’t afford it! Interactions, 10, 13–17.
Toye, R. C. (1986). The effect of viewing position on the perceived layout of space.
Perception & Psychophysics, 40, 85–92.
Tufte, E. R. (1983). The visual display of quantitative information. Cheshire, CT:
Graphics Press.
Turner, A., Doxa, M., O’Sullivan, D., & Penn, A. (2001). From isovists to visibility
graphs: A methodology for the analysis of architectural space. Environment and
Planning B: Planning and Design, 28, 103–121.
Turvey, M. T. (1992). Affordances and prospective control: An outline of the onto-
logy. Ecological Psychology, 4, 173–187.
Turvey, M. T. (2004). Space (and its perception): The first and final frontier. Eco-
Turvey, M. T. (2013). Ecological perspective on perception-action: What kind of
science does it entail? In W. Prinz, M. Beisert, & A. Herwig (Eds.), Action science:
Foundations of an emerging discipline (pp. 139–170). Cambridge, MA: MIT Press.
Turvey, M. T. (2015). Quantum-like issues at nature’s ecological scale (the scale of
organisms and their environments). Mind and Matter, 13, 7–44.
Turvey, M. T. (2019). Lectures on perception: An ecological perspective. New
York: Routledge.
Turvey, M. T., & Fonseca, S. T. (2014). The medium of haptic perception: A
tensegrity hypothesis. Journal of Motor Behavior, 46, 143–187.
Turvey, M. T., Romaniak-Gross, C., Isenhower, R. W., Arzamarski, R., Harrison,
S., & Carello, C. (2009). Human odometer is gait-symmetry specific. Proceed-
ings of the Royal Society of London B: Biological Sciences, 276, 4309–4314.
Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace, W. M. (1981). Ecological laws
of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9,
237–304.
Ulano, M. (2009). Moving pictures that talk: Part 1: How is it possible? CAS
Journal. Retrieved July 3, 2018 from www.filmsound.org/ulano/talkies.htm.
Van Braeckel, K., Butcher, P. R., Geuze, R. H., Van Duijn, M. A. J., Bos, A., &
Bouma, A. (2008). Less efficient elementary visuomotor processes in 7- to
326 References
10-year-old preterm-born children without cerebral palsy: An indication of
impaired dorsal stream processes. Neuropsychology, 22, 755–764.
Van der Meer, A. L. H. (1997a). Keeping the arm in the limelight: Advanced visual
control of arm movements in neonates. European Journal of Paediatric
Neurology, 4, 103–108.
Van der Meer, A. L. H. (1997b). Visual guidance of passing under a barrier. Early
Development and Parenting, 6, 149–157.
Van der Meer, A. L., Fallet, G., & Van der Weel, F. R. (2008). Perception of struc-
tured optic flow and random visual motion in infants and adults: A high-density
EEG study. Experimental Brain Research, 186, 493–502.
Van der Meer, A. L. H., Holden, G., & Van der Weel, F. R. (2005). Coordination
of sucking, swallowing, and breathing in healthy newborns. Journal of Pediatrics
and Neonatology, 2, 69–72.
Van der Meer, A. L. H., Ramstad, M., & Van der Weel, F. R. (2008). Choosing
the shortest way to mum: Auditory guided rotation in 6- to 9-month-old infants.
Infant Behavior and Development, 31, 207–216.
Van der Meer, A. L. H., Svantesson, M. & Van der Weel, F. R. (2012). Longitud-
inal study of looming in infants with high-density EEG. Developmental Neuro-
science, 34, 48–501. doi:10.1159/000345154.
Van der Meer, A. L. H., & Van der Weel, F. R. (1995). Move yourself, baby!
Perceptuo-motor development from a continuous perspective. In P. Rochat (Ed.),
The self in infancy: Theory and research (pp. 257–275). Amsterdam: Elsevier
Science Publishers.
Van der Meer, A. L. H., & Van der Weel, F. R. (2011). Auditory guided arm and
whole body movements in young infants. In P. Strumello (Ed.), Advances in
sound localization (pp. 297–314). Vienna: InTech.
Van der Meer, A. L. H., & Van der Weel, F. R. (2013). Combining findings from
gaze and electroencephalography recordings to study timing in a visual tracking
task. NeuroReport, 24, 968–972.
Van der Meer, A. L. H., Van der Weel, F. R., & Lee, D. N. (1994). Prospective
control in catching by infants. Perception, 23, 287–302.
Van der Meer, A. L. H., Van der Weel, F. R., & Lee, D. N. (1995a). The functional
significance of arm movements in neonates. Science, 267, 693–695.
Van der Meer, A. L. H., Van der Weel, F. R., & Lee, D. N. (1996). Lifting weights
in neonates: Developing visual control of reaching. Scandinavian Journal of
Van der Meer, A. L. H., Van der Weel, F. R., Lee, D. N., Laing, I. A., & Lin, J.-P.
(1995b). Development of prospective control of catching moving objects in
preterm at-risk infants. Developmental Medicine and Child Neurology, 37,
145–158.
Van der Velden, H. A. (1946). The number of quanta necessary for the perception
of light in the human eye. Ophthalmologica, 111, 321–331.
Van der Weel, F. R., Craig, C. M., & Van der Meer, A. L. H. (2007). The rate of
change of tau. In G.-J. Pepping & M. A. Grealy (Eds.), Closing the gap: The sci-
entific writings of David N. Lee (pp. 305–365). Mahwah, NJ: Lawrence Erlbaum
Associates.
Van der Weel, F. R. & Van der Meer, A. L. H. (2009). Seeing it coming: Infants’
brain responses to looming danger. Naturwissenschaften, 96, 1385–1391.
doi:10.1007/s00114-009-0585-y.
References 327
Van der Weel, F. R., & Van der Meer, A. L. H. (2019). Infants’ brain responses to
looming danger: Degeneracy of neural connectivity patterns. In press in the
special issue on “Gibsonian neuroscience” for Ecological Psychology.
Van der Weel, F. R., Van der Meer, A. L. H., & Lee, D. N. (1991). Effect of task
on movement control in cerebral palsy: Implications for assessment and therapy.
Developmental Medicine and Child Neurology, 33, 419–426.
Van der Weel, F. R., Van der Meer, A. L. H., & Lee, D. N. (1996). Measuring dys-
function of basic movement control in cerebral palsy. Human Movement Science,
15, 253–283.
Van Dijk, L., & Rietveld, E. (2017). Foregrounding sociomaterial practice in our
understanding of affordances: The skilled intentionality framework. Frontiers in
Psychology, 7, 1969.
Van Dijk, L., Withagen, R., & Bongers, R. M. (2015). Information without
content: A Gibsonian reply to enactivists’ worries. Cognition, 134, 210–214.
Varela, F. J., Thompson, E., & Rosch, E. (1991). The embodied mind: Cognitive
science and human experience. Cambridge, MA: MIT Press.
Vaz, D. V., Silva, P. L., Mancini, M. C., Carello, C., & Kinsella-Shaw, J. (2017).
Towards an ecologically grounded functional practice in rehabilitation. Human
Movement Science, 52, 117–132.
Vicente, K. J. (2002). Ecological interface design: Progress and challenges. Human
Factors, 44, 62–78.
Vicente, K. J., Kada-Bekhaled, K., Hillel, G., Cassano, A., & Orser, B. A. (2003). Pro-
gramming errors contribute to death from patient-controlled analgesia: Case report
and estimate of probability. Canadian Journal of Anesthesiology, 50, 328–332.
Vicente, K. J., & Rasmussen, J. (1990). The ecology of human-machine systems II:
Mediating direct perception in complex work domains. Ecological Psychology,
2, 207–249.
Vilhelmsen, K., Agyei, S. B., Van der Weel, F. R., & Van der Meer, A. L. H.
(2018). A high-density EEG study of differentiation between two speeds and
directions of simulated optic flow in adults and infants. Psychophysiology.
doi:10.1111/psyp.13281.
Vilhelmsen, K., Van der Weel, F. R., & Van der Meer, A. L. H. (2015). A high-
density EEG study of differences between three high speeds of simulated forward
motion from optic flow in adult participants. Frontiers in Systems Neuroscience,
9, 146.
Villard, S. J., Flanagan, M. B., Albanese, G. M., & Stoffregen, T. A. (2008). Pos-
tural instability and motion sickness in a virtual moving room. Human Factors,
50, 332–345.
Viswanathan, G. M., Da Luz, M. G., Raposo, E. P., & Stanley, H. E. (2011). The
physics of foraging: An introduction to random searches and biological encoun-
ters. Cambridge: Cambridge University Press.
Volcic, R., Fantoni, C., Caudek, C., Assad, J. A., & Domini, F. (2013). Visuomo-
tor adaptation changes stereoscopic depth perception and tactile discrimination.
Journal of Neuroscience, 33, 17081–17088.
Von Hofsten, C. (1983). Catching skills in infancy. Journal of Experimental Psych-
Von Hofsten, C. (2003). On the development of perception and action. In K. J.
Connolly & J. Valsiner (Eds.), Handbook of developmental psychology (pp.
114–140). London: Sage.
328 References
Von Hofsten, C., Feng, Q., & Spelke, E. (2000). Object representation and predic-
tive action in infancy. Developmental Science, 32, 193–205.
Von Hornbostel, E. M. (1927). The unity of the senses. Psyche, 7, 83–89.
Von Uëxkull, J. (1992). A stroll through the worlds of animals and men: A picture
book of invisible worlds. Semiotica, 89, 319–391. (Original work published 1934.)
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological
processes. Cambridge, MA: Harvard University Press.
Wade, N. J., & Finger, S. (2001). The eye as an optical instrument: From camera
obscura to Helmholtz’s perspective. Perception, 30, 1157–1177.
Wagman, J. B. (2010). What is responsible for the emergence of order and pattern
in psychological systems? Journal of Theoretical and Philosophical Psychology,
30, 32–50.
Wagman, J. B., & Abney, D. H. (2012). Transfer of recalibration from audition to
touch: Modality independence as a special case of anatomical independence.
589–602.
Wagman, J. B., & Abney, D. H. (2013). Is calibration of the perception of length
modality-independent? Attention, Perception, & Psychophysics, 75, 824–829.
Wagman, J. B., Bai, J., & Smith, P. J. (2016). Nesting in perception of affordances
for stepping and leaping. Attention, Perception, & Psychophysics, 78, 1771–1780.
Wagman, J. B., Caputo, S. E., & Stoffregen, T. A. (2016a). Hierarchical nesting of
affordances in a tool use task. Journal of Experimental Psychology: Human Per-
ception & Performance, 42, 1627–1642.
Wagman, J. B., & Carello, C. (2001). Affordances and inertial constraints on tool
use. Ecological Psychology, 13, 173–195.
Wagman, J. B., & Chemero, A. (2014). The end of the debate over extended cogni-
tion. In T. Solymosi & J. Shook (Eds.), Neuroscience, neurophilosophy, and
pragmatism: Understanding brains at work in the world (pp. 105–124). Basing-
stoke: Palgrave Macmillan.
Wagman, J. B., & Day, B. M. (2014). Changes in context and perception of
maximum reaching height. Perception, 43, 129–144.
Wagman, J. B., & Hajnal, A. (2014a). Task specificity and anatomical independ-
ence in perception of properties by means of a wielded object. Journal of Experi-
Wagman, J. B., & Hajnal, A. (2014b). Task specificity and anatomical independ-
ence in perception of properties by means of a wielded object. Journal of Experi-
Wagman, J. B., & Hajnal, A. (2016). Use your head! Perception of action possibil-
ities by means of an object attached to the head. Experimental Brain Research,
234, 829–836.
Wagman, J. B., Langley, M. D., & Farmer-Dougan, V. (2017). Doggone
affordances: Canine perception of affordances for reaching. Psychonomic Bul-
letin & Review, 24, 1097–1103.
Wagman, J. B., & Malek, E. A. (2008). Perception of affordances for walking
under a barrier from proximal and distal points of observation. Ecological
Wagman, J. B., & Malek, E. A. (2009). Geometric, kinetic-kinematic, and inten-
tional constraints influence willingness to pass under a barrier. Experimental
References 329
Wagman, J. B., & Miller, D. B. (2003). Nested reciprocities: The organism-
environment system in perception-action and development. Developmental Psy-
chobiology, 42, 317–334.
Wagman, J. B., & Morgan, L. L. (2010). Nested prospectivity in perception: Per-
ceived maximum reaching height reflects anticipated changes in reaching ability.
Psychonomic Bulletin & Review, 17, 905–909.
Wagman, J. B., Shockley, K., Riley, M. A., & Turvey, M. T. (2001). Attunement,
calibration, and exploration in fast haptic perceptual learning. Journal of Motor
Behavior, 33, 323–327.
Wagman, J. B., Stoffregen, T. A., Bai, J., & Schloesser, D. S. (2018). Perceiving
nested affordances for another person’s actions. The Quarterly Journal of
Experimental Psychology, 7, 790–799.
Wagman, J. B., Thomas, B. J., McBride, D. M., & Day, B. M. (2013). Perception
of maximum reaching height when the means of reaching are no longer in view.
Wagner, M. (1985). The metric of visual space. Perception & Psychophysics, 38,
483–495.
Wagner, M. (2006). The geometries of visual space: New York: Psychology Press.
Wallach, H., & O’Connell, D. N. (1953). The kinetic depth effect. Journal of
Experimental Psychology, 45, 205–217.
Wallach, H., & O’Leary, A. (1982). Slope of regard as a distance cue. Perception
& Psychophysics, 31, 145–148.
Waller, D., & Richardson, A. R. (2008). Correcting distance estimates by interact-
ing with immersive virtual environments: Effects of task and available sensory
information. Journal of Experimental Psychology: Applied, 14, 61–72.
Walter, H., Wagman, J. B., Stergiou, N., Erkmen, N., & Stoffregen, T. A. (2017).
Dynamic perception of dynamic affordances: Walking on a ship at sea. Experi-
mental Brain Research, 235, 517–524.
Wang, S., Baillargeon, R., & Paterson, S. (2005). Detecting continuity violations in
infancy: A new account and new evidence for covering and tube events. Cogni-
tion, 95, 129–173.
Warren, W. H. (1984). Perceiving affordances: Visual guidance of stair climbing.
683–703.
Warren, W. H. (1995). Constructing an econiche. In J. Flach, P. Hancock, J. Caird,
& K. Vicente (Eds.), Global perspectives on the ecology of human-machine
systems, Hillsdale, NJ: Lawrence Erlbaum Associates.
Warren, W. H. (1998). Visually controlled locomotion: 40 years later. Ecological
Psychology, 10(3–4), 177–219.
Warren, W. H. (2005). Direct perception: The view from here. Philosophical
Topics, 33, 335–361.
Warren, W. H. (2006). The dynamics of perception and action. Psychological
Review, 113, 358–389.
Warren, W. H. (2007). Action-scaled information. In G. J. Pepping & M. L. Grealy
(Eds.), Closing the gap: The scientific writings of David N. Lee (pp. 253–268).
Mahwah, NJ: Erlbaum.
Warren, W. H., Kay, B. A., & Yilmaz, E. H. (1996). Visual control of posture
during walking: Functional specificity. Journal of Experimental Psychology:
330 References
Warren, W. H., Kay, B. A., Zosh, W. D., Duchon, A. P., & Sahuc, S. (2001). Optic
flow is used to control human walking. Nature Neuroscience, 4, 213–216.
Warren, W. H., Rothman, D. B., Schnapp, B. H., & Ericson, J. D. (2017). Worm-
holes in virtual space: From cognitive maps to cognitive graphs. Cognition, 166,
152–163.
Warren, W. H., & Shaw, R. E. (1985a). Events and encounters as units of analysis
for ecological psychology. In W. H. Warren & R. E. Shaw (Eds.), Persistence
and change: Proceedings of the First International Conference on Event Percep-
tion. (pp. 1–27). Hillsdale, NJ: Lawrence Erlbaum Associates.
Warren, W. H., & Shaw, R. E. (Eds.) (1985b). Persistence and change: Proceedings
of the First International Conference on Event Perception. Hillsdale, NJ Law-
rence Erlbaum Associates.
Warren, W. H., & Whang, S. (1987). Visual guidance of walking through aper-
tures: Body-scaled information for affordances. Journal of Experimental Psych-
Weast, J. A., Shockley, K., & Riley, M. A. (2011). The influence of athletic experi-
ence and kinematic information on skill-relevant affordance perception. The
Quarterly Journal of Experimental Psychology, 64, 689–706.
Weast, J. A., Walton, A., Chandler, B. C., Shockley, K., & Riley, M. A. (2014).
Essential kinematic information, athletic experience, and affordance perception
for others. Psychonomic Bulletin & Review, 21, 823–829.
Welch, R. B., Choe, C. S., & Neinrich, D. R. (1974). Evidence for a three-
component model of prism adaptation. Journal of Experimental Psychology,
103, 700–705.
Weschler, L. (2009). Seeing is forgetting the name of the thing one sees. Berkeley,
CA: University of California Press. (original work published 1982).
Weyl, H. (1952). Symmetry. Princeton, NJ: Princeton University Press.
Wheeler, J. (2000). Geons, black holes, and quantum foam (Rev. ed.). New York:
W. W. Norton & Company.
White, E., Shockley, K., & Riley, M. A. (2013). Multimodally specified energy
expenditure and action-based distance judgments. Psychonomic Bulletin &
Review, 20, 1371–1377.
Willats, J. (1990). The draughtsman’s contract: How an artist creates an image. In
H. Barlow, C. Blakemore, & M. Weston-Smith (Eds.), Images and understanding
(pp. 235–255). Cambridge: Cambridge University Press.
Willemsen, P., Colton, M. B., Creem-Regehr, S. H., & Thompson, W. B. (2009).
The effects of head-mounted display mechanical properties and field of view on
distance judgments in virtual environments. ACM Transactions on Applied Per-
ception (TAP), 6(2), 8.
Withagen, R. (2004). The pickup of nonspecifying variables does not entail indirect
perception. Ecological Psychology, 16, 237–253.
Withagen, R., Araújo, D., & de Poel, H. J. (2017). Inviting affordances and agency.
New Ideas in Psychology, 45, 11–18.
Withagen, R., & Chemero, A. (2009). Naturalizing perception: Developing the
Gibsonian approach to perception along evolutionary lines. Theory & Psych-
ology, 19, 363–389.
Withagen, R., de Poel, H. J., Araújo, D., & Pepping, G. J. (2012). Affordances can
invite behavior: Reconsidering the relationship between affordances and agency.
New Ideas in Psychology, 30, 250–258.
References 331
Withagen, R., & Michaels, C. F. (2005a). The role of feedback information for
calibration and attunement in perceiving length by dynamic touch. Journal of
Withagen, R., & Michaels, C. F. (2005b). On ecological conceptualizations of per-
ceptual systems and action systems. Theory & Psychology, 15, 603–620.
Withagen, R., & van Wermeskerken, M. (2009). Individual differences in learning
to perceive length by dynamic touch: Evidence for variation in perceptual learn-
ing capacities. Perception & Psychophysics, 71, 64–75.
Withagen, R., & van Wermeskerken, M. (2010). The role of affordances in the
evolutionary process reconsidered: A niche construction perspective. Theory &
Witt, J. K. (2011). Action’s effect on perception. Current Directions in Psycho-
logical Science, 20, 201–206.
Witt, J. K., Linkenauger, S. A., Bakdash, J. Z., & Proffitt, D. R. (2008). Putting to
a bigger hole: Golf performance relates to perceived size. Psychonomic Bulletin
& Review, 15, 581–585.
Witt, J. K., Proffitt, D. R., & Epstein, W. (2010). When and how are spatial per-
ceptions scaled? Journal of Experimental Psychology: Human Perception and
Witt, J. K., & Riley, M. A. (2014). Discovering your inner Gibson: Reconciling
action-specific and ecological approaches to perception–action. Psychonomic
Bulletin & Review, 21, 1353–1370.
Wraga, M. J. (1999). The role of eye height in perceiving affordances and object
dimensions. Perception and Psychophysics, 61, 490–507.
Wraga, M. J., & Proffitt, D. R. (2000). Mapping the zone of eye-height utility for
seated and standing observers. Perception, 29, 1361–1383.
Wu, B., He, Z. J., & Ooi, T. L. (2007). Inaccurate representation of the ground
surface beyond a texture boundary. Perception, 36, 703–721.
Wu, B., Ooi, T. L., & He, Z. J. (2004). Perceiving distance accurately by a direc-
tional process of integrating ground information. Nature, 428, 73–77.
Wunsch, K., Henning, A., Aschersleben, G., & Weigelt, M. (2013). A systematic
review of the end-state comfort effect in normally developing children and in
children with developmental disorders. Journal of Motor Learning and Develop-
ment, 1, 59–76.
Xiao, B., Bi, W., Jia, X, Wei, H., & Adelson, E. H. (2016). Can you see what you
feel? Color and folding properties affect visual–tactile material discrimination of
fabrics. Journal of Vision, 16(3), 34, doi:10.1167/16.3.34.
Yarbus, A. L. (1967). Eye movements and vision. New York: Plenum.
Yasuda, M., Wagman, J. B., & Higuchi, T. (2014). Can perception of aperture passa-
bility be improved immediately after practice in actual passage? Dissociation between
walking and wheelchair use. Experimental Brain Research, 232, 753–764.
Yellott, J. I. Jr., Wandell, B. A., & Cornsweet, T. N. (2011). The beginnings of
visual perception: The retinal image and its initial encoding. In Handbook of
physiology: The nervous system. New York: Wiley Online Library.
Yonas, A., Goldsmith, L. T., & Hallstrom, J. L. (1978). Development of sensitivity
to information provided by cast shadows in pictures. Perception, 7, 333–341.
Yoonesi, A., & Baker, C. L. (2013). Depth perception from dynamic occlusion in
motion parallax: Roles of expansion-compression versus accretion-deletion.
Journal of Vision, 13, 1–6.
332 References
Yu, C., & Smith, L. B. (2013). Joint attention without gaze following: Human
infants and their parents coordinate visual attention to objects through eye-hand
coordination. PLoS ONE, 8, e79659. doi:10.1371/journal.pone.0079659.
Yu, Y., Bardy, B. G., & Stoffregen, T. A. (2010). Influences of head and torso
movement before and during affordance perception. Journal of Motor Behavior,
43, 45–54.
Zaal, F. T. J. M., & Bootsma, R. J. (2011). Virtual reality as a tool for the study of
perception-action: The case of running to catch fly balls. Presence: Teleoperators
& Virtual Environments, 20, 93–103.
Zaal, F. T. J. M., & Michaels, C. F. (2003). The information for catching fly balls:
Judging and intercepting virtual balls in a CAVE. Journal of Experimental Psych-
Zajonc, A. (1993). Catching the light: The entwined history of light and mind.
New York: Bantam Books.
Zhao, H., & Warren, W. H. (2015). On-line and model-based approaches to the
visual control of action. Vision Research, 110, 190–202.
Zhu, Q., & Bingham, G. P. (2010). Learning to perceive the affordance for long-
distance throwing: Smart mechanism or function learning? Journal of Experi-
mental Psychology: Human Perception & Performance, 36, 862–875.
Zhu, Q., & Bingham, G. P. (2011). Human readiness to throw: The size–weight
illusion is not an illusion when picking the best objects to throw. Evolution and
Human Behavior, 32, 288–293.
Index
Page numbers in bold denote tables, those in italics denote figures.
absolute critical boundary 40–2 anticipation 35, 113, 117–18, 198

action: relationship with perception arm movements 111, 112, 113–14,
182, 184–6, 187, 222–3, 248; basic 115, 116–19, 123, 128, 233–4
actions 221 artistic representation 83–5, 188–9,
active touch 27 211, 255, 258, 260–2, 263, 264
adaptation 167, 183–4 audition: auditory information 116,
adjacent order 58–9, 91–2, 109 119–20, 245; “silent” films 288–9;
affordance-based control 44, 242 sound effects 289–90; sound
affordances: perception of by animals balancing 289–90
135, 136; changing 224–6; for awareness 17, 47, 146, 151, 169,
climbing 7, 10, 132, 138, 140, 146; of 171–2, 189, 192–4, 196–203, 206,
depictions 261, 262, 263, 264–5, 266, 211–12, 219–20, 259–61, 264,
267; in design 136, 145; for fitting 267, 270, 272, 273; see also
through 135, 140; for locomotion presence
136, 138, 140, 145, 225–6, 229–31,
270; for manipulation 133, 136, behavior setting 8, 15–16
140–1, 231; of the medium 24–5, 26, behavioral dynamics 246–8
27, 28; nested 136, 143–4; of other Bentley, Arthur 55–7, 68
people 133, 143–4; for passability Bernstein’s degrees of freedom problem
119; as relations 8–11, 37–9, 41–2; 97, 245–6
remembered 239, 240, 248–50; as bodily frame of reference for action
resources 8, 10–11; of substances 111, 114, 115, 116, 118, 124, 128
29–30; for support 132, 133, 138, body schema see embodied action
144, 147; of surfaces 29–30, 151, 156, schema
168–72; for throwing 141, 143; of
tools 30, 31, 32, 33 calibration 43–5, 183–4; mutual
agency 40, 142, 148, 212, 274 169–71; recalibration 171, 245
air theory of perception 152 camera obscura 255, 256, 258
allocentric view 284 chambered eye 53–4, 256–7, 275
ambient light 29, 58–9, 74–5, 144, 148, changing surface layout 30, 31, 32, 33,
238, 251 34–5
anatomical independence of perception compound eyes 54
242–5, 252 computer graphics 87–9
animal-environment system 6, 9, 20, continuity of objects 194–5
37–8, 89, 117, 134, 136, 138, 142–3, controlling collision 226–8; see also
146, 238, 241–2, 252, 261–2, time to contact
267–70, 273 coordinative structure 245–6, 252
334 Index
d’Alembert’s principle 90–1, 106–9 egocentric view 274, 284; see also
declination angle 155, 158, 164–7, awareness; presence
169–71 Einstein, Albert 91–2
degeneracy 127–9 electroencephalography (EEG) 121,
depiction: in the acoustic environment 122, 123–4, 126, 127
271; affordances of 261–2, 263, embodied 43, 146, 169, 195, 270;
264–5, 266, 267; apparent paradox action schema 41, 42, 43–4
of 260; in behavioral science 85, 255, enactive approach 18
256–7, 258–9; of events 261, 271, enucleated eye 55, 56
274, 285, 282, 290; of information environment: cluttered 21–2, 228–9;
49; in smell 271; specification of human alteration of 30, 31, 32, 33,
267–71; as substitute for physical 34–5; of individual 6–7, 8, 9–10, 20;
reality 82, 206, 211, 255, 260, 272; as large life-containing envelope 74;
in taste 271 as meaningful 12, 23–4, 30; of
depth: compression 163–7; cues 152, 164 species 6–7, 8, 9–10, 20; vs. physical
Descartes 47, 54, 55, 56, 257 world 8, 12
development 8, 13–14, 110, 116, events: ecological 23, 40, 51, 84, 91–3,
118–21, 122, 123–4, 128, 138, 94, 95, 188, 190, 201, 211, 238–9,
147–8, 194, 201, 206, 212–13, 215, 244, 248, 252, 275; information for
216, 217–18, 222, 224–6, 229, 282–3, 288–90; irreversible 79, 191;
231–6, 262, 263, 272; see also nested structure 198–9, 285, 286,
learning 287; past-present-future divide
dimensional inhomogeneity 91, 97, 105 196–9, 233; perception of 90–1, 92,
direct perception 17, 37–9, 46–9, 60, 95–7, 109, 175–7, 180, 195–6,
117, 136, 151, 171, 174–6, 205, 243, 284–5; reversible 78–9, 191, 194,
251, 255, 267 200; symmetry in 98–9
display 30, 44, 46–9, 86–8, 108, 136, exploration 8, 24–5, 26, 27, 35, 38,
156, 170, 181, 184–5, 190, 193, 195, 182, 219–21, 223–6, 229–31, 267,
206–7, 251, 258, 260, 266–7, 271, 270; see also visual exploration
274; see also depiction eye height 140, 154–6, 167–8
distance perception 115, 168, 171; eye trackers 207, 208–9, 210
distance constancy 157–8; paradoxes
of 159, 160, 163–6, 168–9 flicker fusion 279–80
distance to break (DTB) 45, 48 foresight see anticipation
dual frame discrepancy hypothesis 102, functional transparency 44–7
104, 105–6, 108–9
duality 42, 99–100, 101, 102–3, 109 Ganzfeld 29, 67–8, 73–4, 84
dynamic occlusion 78–9, 82, 86–7, Gibson, Eleanor 13, 199, 224, 228
188–92, 194–6, 199–200, 202–4; geometric properties 39–40, 76, 90,
commonplace nature of 191–2, 194; 141, 151–2, 156, 164, 168–70, 172,
historical antecedents to 192–3 237–9, 241, 249–50, 269
global array 242–3, 255, 268–71, 273,
ecological interface design (EID) 37, 284
44–50, 267 ground surface 136, 152–3, 157,
ecological neuroscience 120–1, 122, 166–7, 215, 226, 231
123, 128–9 ground theory of perception 78, 151–3,
ecological optics 59, 73, 93–4, 96, 98, 155
110, 114, 148, 152–3, 284
ecological physics 91, 108, 136, 137, habitat 6–7, 8, 9–11, 13, 15–20, 39,
138, 139, 140, 143, 239, 249, 252 139, 145, 199–200, 283
editing 282, 283, 284–5, 286, 287 haptic information 226–7, 230–1, 250;
education of attention 46 see also haptic perception
egocentric distance 158–9, 163–8, 171; haptic perception 25–7, 29, 44–9
see also declination angle head movements 213–15
Index 335
Helmholtz, Hermann von 57, 188, 203 113–14, 117, 120, 123, 132, 145,
higher-order variables 35, 48, 117, 123, 193–4, 198–200, 225–6, 228–9,
128, 142–4, 152, 198, 201, 239, 235–6, 270–1; see also development
242–3, 247, 251–2, 268, 271; see light box demonstrations 63–5
also invariant light: as illumination 29, 44–5, 59–63,
horizon ratio 153, 154–5, 156, 158–9, 67, 176, 233, 255, 256, 258; as such
164–7 11, 25, 47, 59–61, 64, 67, 73–5, 79,
human factors 37, 48–50, 89, 130, 231 93, 144, 148, 278; as stimulation 55,
Hume’s problem 80, 86, 88 58–9, 62, 86, 88, 112, 176, 277, 280;
hydrodynamic perception 25 information in 9, 47–8, 52, 95, 97,
130, 132, 189, 193, 238, 241, 251,
illusions 147, 179, 189, 224, 270–1, 275, 276, 281–2; invisibility of 51,
279, 281; Mystery Spot 152–3 61, 62, 66, 112–13; see also images
“illusion of reality” see presence locomotion 91, 168, 180, 181, 182,
images: image formation 52, 53, 54–5, 209, 215, 222–3, 228, 231;
56, 57, 68, 82, 94, 255, 256–7, 258, developing 235–6; and distance
277–9, 280, 281; entoptic image 58; perception 168; exploration with and
mental image 80, 82, 96, 151; see in service of 78, 81, 94, 119, 136,
also depiction; displays; retinal image 145, 200, 212–13, 229–30, 283; self-
information 151–2, 164, 158; about produced 121–2, 124, 128–9; see
132–4, 136, 144–5; -based control also affordances
241–2; body-scaled and action-scaled looming 121, 123–5, 126, 127–8, 228,
172; common domain of 201–3, 285; 234; see also time to contact
conventional 8, 12–13; higher-order
152; lawful vs. general ecological 251, maintaining balance 118, 138, 225–7,
287–8; mechanical 25–9; optical 230, 232
151–3, 159, 170; pickup 225, 238–43, margin of safety 40–1, 44
248–50, 252; specifying 52, 66, 68, meaning 5, 7, 8, 12–13, 23–4, 30,
152, 189, 191–4, 282–5; specifying vs. 37–40, 46, 48, 131–2, 145–6, 176,
non-specifying 238, 241, 249–50; vs. 226, 237–8, 250, 252, 272, 284–5,
stimulation 59; see also invariant; see 287–8
also higher-order variables minimally invasive surgery (MIS) see
insect vision 54 laparoscopic surgery
interposition see dynamic occlusion motion blur 281, 282
invariant 34–6, 45–9, 75–81, 89–92, motion perception 75, 120; changing
96–100, 109, 121, 128, 140, 146, layouts 174–5; facilitating action
148, 153–9, 164, 167, 174–7, 200, 175; in films 279, 280–1, 282; means
220, 251, 268, 283; over of studying 174–80, 182, 187; of self
transformation 25, 27, 145, 202, 174, 176, 183
238, 241, 243–5, 248; see motion sickness 179, 183, 185–6, 187,
also higher-order variables; see 284
also information moving room see swinging room
Irwin, Robert 74, 83–4
natural scenes 88
Kepler, Johannes 52, 53, 54–5, 256–7, nestedness: of affordances 136, 143–4;
258–9 of environment 7, 29, 74–6, 85, 153;
kinetics vs. kinematics 90, 92–3, 97, of events 198–9, 285, 286, 287; of
100, 103, 105–6, 108–9, 269 perceptual system 205–6, 213–14,
knowing 198–9, 202, 237, 250–1 215, 216, 217–18, 219
niche 8, 10, 134–6, 139, 143, 172, 239,
Lagrangian 100 251, 288
language 13–14, 16–17, 21
laparoscopic surgery 45 obstacle navigation 40, 115, 140,
learning 7, 8, 9, 13–16, 18–20, 43, 181–2, 215, 216, 217, 225, 227–30
336 Index
occlusion 75, 95, 121, 122, 123, 153; 176–7, 189–90, 195, 220, 256–8,
see also dynamic occlusion 280–1
optic array 51, 68, 86, 90, 92, 93, 94,
96–8, 100, 104, 106, 108–9, 131, saccades 58, 207, 213
202, 274, 281; ambient 17, 73–5, sense organ vs. perceptual system 21–2,
76–7, 87, 89, 117, 175–6, 178, 241, 27, 104, 189, 203, 206, 238, 242–6,
268; artistic emphasis on 83–5; 247, 248–9, 252, 280
difference from world 79–81; realism size perception: size constancy 152–4;
in 81–2; structure in 76–7, 78–80, size-distance invariance 156; see also
147, 181, 189, 270, 274, 281 horizon ratio
optic flow 38, 48, 86, 167, 121, 176, smart perceptual instruments 48,
178–82, 185–7, 195, 204, 223, 227, 243–5, 252
241, 258, 276, 280–2 soft assembly 244–5, 248, 252
optical contact 153, 159 space: information 14–15, 18; physical
optical devices 52, 88 8, 24, 40, 91–2, 95–6, 109, 140, 152,
optical slant 158–9; see also texture 164, 170, 227, 270; and time 21, 91,
optics 37, 52, 54–5, 59, 73, 89, 93–8, 93, 100, 136, 210, 220, 248, 250;
110, 114, 148, 152, 269–70, 284 visual 164, 168
orientation 199, 282, 283, 284 static image see retinal image
outdoors research 85–6 stone knapping 30, 31, 32, 33, 34–5
structured light 51, 58–9, 66, 67, 68,
persistence 35, 80, 82, 92, 188, 191, 73, 132, 241; see also light
193, 195; empirical studies of 193–4; subjective camera 284; see also presence
of vision (in films) 280–1 substances (human alteration of) 30,
photons 59–60, 64, 66, 73 31, 32, 33, 34–5
pinhole camera see camera obscura superdiffusion 26
places 6, 8, 11, 15–16, 92, 97, 145, superposition see dynamic occlusion
148, 199, 225–6, 238–9 surface layout 30, 35, 37–41, 51, 74,
planning 33, 215, 222, 229; for 84, 92, 151, 168–72, 189, 199, 231
reaching 232–3; for manual actions swinging room 178, 179–80, 181, 185,
233–5 105–6, 178–80, 227
plenoptic function 87–8 symbolism 275, 287–8
posture 28, 106, 138, 225, 227, 229; as symmetry: group theory 90, 98–100,
foundation for manipulation 136, 109; principle 11–13
223, 231–2, 236; as point of
observation 216, 217, 218; and task ecology 249–50
stabilization 179, 214 task: blind walking 153, 158, 163,
preferred critical boundary 40–1, 44 166–71; bisection 163–5; aspect ratio
presence 87, 259–61, 274; see also 159–63, 165; depth increment 163,
orientation 166–7; visually-directed action 167;
projection angles 75 magnitude estimation 153, 159
prospective control 93, 110, 113, tau see time to contact
116–24, 128, 183, 231 technology of film: in recording 275–7;
in playback 277–9
reaching and grasping (development of) tensegrity 25–7
110–11, 114, 116–18, 123; see also texture 29, 78–9, 82, 92, 152–3, 156,
development 177–8, 190, 192, 227; as an intrinsic
reference frame 100, 102–4, 153, 210 scale 22, 157; for size and distance 75,
resonance 121, 125, 128–9 157–8, 163; for slant 158–9; gradient
retina 38, 48, 52, 53, 55, 56, 57–9, 75, 38, 158; isotropy 157; homogeneity
178–9, 188, 206, 237, 280; see also 157; scaling contrast 159
retinal image time and space as abstractions 91, 109
retinal image 47, 49, 51–2, 53, 54–5, time to contact 40, 45–8, 86, 117,
56, 57–9, 68, 83–6, 88, 151–2, 174, 123–5, 198, 227–8, 234, 241; tau
Index 337
117–19, 123, 125, 127–8; tau- vista 85, 102, 199–200, 227, 236, 283;
coupling 118–19, 125–7; time to see also environment
collision 116–17, 120, 122–5, 128 visual cliff 147, 224–5, 229
tools 29–30, 32, 34, 41–50, 140, 224, visual evoked potential (VEP) 124–5
231, 235, 238, 251 visual exploration 205–8, 213–14, 215,
Turrell, James 74, 84 219, 221, 223, 230; measuring see
eye trackers; social looking 212–13;
Umwelt 6–11, 13–16, 18–20
task-driven 210–11; whole body 214,
vicarious function 121, 127–9 220
video games 87, 290n2 visual-locomotor mapping 167–70
virtual environment see virtual reality
virtual reality 87, 156, 166, 170–1, way-finding 199–201
180–7, 210, 219–20, 258, 269 world line 91, 284–7

Perception As Information Detection Reflections On Gibsons Ecological Approach To Visual Perception Paperbacknbsped 0367312964 9780367312961 Compress

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Perception As Information Detection Reflections On Gibsons Ecological Approach To Visual Perception Paperbacknbsped 0367312964 9780367312961 Compress

Uploaded by

Copyright:

Available Formats

Perception as Information Detection

This book provides a chapter-­by-chapter update to and reflection on the

Jeffrey B. Wagman is Professor of Psychology at Illinois State University,

Julia J. C. Blau is an Assistant Professor of Psychology at Central Con-

Social and Applied Aspects of Perceiving Faces

Perception and Control of Self-­Motion

Michotte’s Experimental Phenomenology

Perceiving Events and Objects

Global Perspectives on the Ecology of Human-­Machine Systems

Local Applications of the Ecological Approach to Human-­Machine

Dexterity and Its Development

Ecological Psychology in Context

Perception as Information Detection

Edited by Jeffrey B. Wagman and

ISBN: 978-0-367-31295-4 (hbk)

List of Illustrations viii

1 The Third Sense of Environment 5

2 The Triad of Medium, Substance, and Surfaces for the

3 Ecological Interface Design Inspired by “The Meaningful

4 Challenging the Axioms of Perception: The Retinal Image

5 Getting into the Ambient Optic Array and What We Might

6 The Challenge of an Ecological Approach to Event

7 The Optical Information for Self-­Perception in

8 A Guided Tour of Gibson’s Theory of Affordances 130

9 Perceiving Surface Layout: Ground Theory, Affordances,

10 Acting Is Perceiving: Experiments on Perception of Motion

11 Revisiting “The Discovery of the Occluding Edge and Its

12 Looking with the Head and Eyes 205

13 James Gibson’s Ecological Approach to Locomotion and

15 The Use and Uses of Depiction 255

16 Revisiting Ecological Film Theory 274

Karen E. Adolph, Department of Psychology, New York University, New

This series of volumes is dedicated to furthering the development of psychology

James J. Gibson began the preface to The Ecological Approach to Visual

James J. Gibson’s classic book The Ecological Approach to Visual Percep-

1. The physical world: Reality is assumed to be structured in various

This distinction is both appropriate and intuitively appealing. But it fails to

The environment consists of the surroundings of animals. Let us observe

As is clear here, Gibson thought he could resolve the confusion by pointing

Physical world Habitat Umwelt

Structure Resource/function Meaning

Scatle-free; atoms to galaxies Terrestrial scale Terrestrial scale

No complementary term Complementary to “species” Complementary to

Orthogonal to life Complementary to a form of life Complementary to a living

Correspondence relations •  ffordance-resources

Thermodynamic information; • Information-about the habitat • Information-for an

History of pure structure, • H as a history in evolution • Has a history in

Orthogonal to social • S hared among members of a • C eases to exist when a

Some Tensions and Their Resolutions

Nobody Knows What Affordances Are Anymore

Nobody Knows What Information Is Anymore

“Information for” calls attention to the fact that ecological informa-

We do not endorse this strategy.

How Do Animals Learn? How Do We Invent?

Can We Have a Gibsonian Account of the Social?

[A] behavior setting is an emergent property of interdependent pat-

The Ecological Approach as a Theory of the Structure of the

Practical Ecological Interventions

For New Book (“Useful Vision”?)

The ability to distinguish environmental substances, the forms of

Figure 2.1 Water velocity 60 s after an 86 mm-long fish (Lepomis gibbosus) passed

According to Gibson (1979/2015, p. 13), “if we understand the notion

This book provides a chapter-by-chapter update to and reflection on the

Perception and Control of Self-Motion

Global Perspectives on the Ecology of Human-Machine Systems

Local Applications of the Ecological Approach to Human-Machine

List of Illustrations viii

1 The Third Sense of Environment 5

7 The Optical Information for Self-Perception in

8 A Guided Tour of Gibson’s Theory of Affordances 130

12 Looking with the Head and Eyes 205

15 The Use and Uses of Depiction 255

16 Revisiting Ecological Film Theory 274

Correspondence relations • ffordance-resources

Thermodynamic information; • Information-about the habitat • Information-for an

History of pure structure, • H as a history in evolution • Has a history in

Orthogonal to social • S hared among members of a • C eases to exist when a

Figure 2.1 Water velocity 60 s after an 86 mm-long fish (Lepomis gibbosus) passed

The goal is to achieve for computer-mediated visual displays the level of

Figure 4.2 The demonstration of a “retinal image” by Scheiner, repeated by

Figure 4.3 Illumination is transparent to that which is illuminated. (a) The light

Figure 4.8 (a) Light reflected from spacewalkers allows each astronaut to be

Dynamics is the branch of physics that studies time-dependent processes,

Figure 6.1 The transformation of the optic array defined by a locomotor move-

The notion of point-to-point correspondence in projective geometry,