Artificial Intelligence

Text Book
Artificial Intelligence A Modern Approach – Second
Edition – Stuart Russell and Peter Norvig – Prentice
Hall India ( PHI ).

Reference Book
Artificial Intelligence – Elaine Rich, Kevin Knight-
Second Edition – Tata McGraw Hill.
Lecture 1
 Introduction
Homo sapiens—man the wise—because our
intelligence is so important to us.

 Humans - tried to understand how we think and act

—that is, how our brain works.

AI, is concerned with not just understanding but also

building intelligent entities.
Introduction To AI
Concerned with the design of intelligence in an
artificial device.

The term was coined by McCarthy in 1956.

Two Key terms in the definition – Intelligence and

Artificial Device.
 AI is Fascinating…..
AI – ranked - most interesting and fastest-growing
fields.

Already generating over a trillion dollars a year in

revenue.

AI expert Kai-Fu Lee predicts that its impact will be

“more than anything in the history of mankind.”

 Moreover, the intellectual frontiers of AI are wide

open.
AI is Fascinating……
general (learning, reasoning, perception, and so on)
to the specific, such as ……
playing chess,
proving mathematical theorems,
writing poetry,
driving a car, or
diagnosing diseases.
AI is relevant to any intellectual task; it is truly a
universal field
 What is AI?
 What is Intelligence ? 4 schools of thoughts !!

 School 1 - defined intelligence in terms of fidelity to human

performance - Think like human.

 School2 - an abstract, formal definition of intelligence called

rationality—loosely speaking, doing the “right thing.” - Act
Rationally

 School3 - consider intelligence to be a property of internal thought

processes and reasoning – Think Rationally.

 School3 - focus on intelligent behavior, an external characterization –

Act like a human.
What is Intelligence – Two Dimensions – Four
Combinations
Human Vs Rational.
Thoughts Vs Behavior.
What is Intelligence – Behavior
Aspect
A system with intelligence is expected to behave as
intelligently as a human.
 A system with intelligence is expected to behave in
the best possible manner.
What is Intelligence – Thought
Process
Are we looking at the thought process or reasoning
ability of the system?
Or are we only interested in the final manifestations
of the system in terms of its actions?
What is Artificial Intelligence
Thinking Humanly : The Cognitive Modeling
approach
Understanding human thought and an effort to build
machines that emulate the human thought process.
This view is the cognitive science approach to AI.
introspection—trying to catch our own thoughts as
they go by;
psychological experiments—observing a person in
action;
 brain imaging—observing the brain in action.
 Thinking Humanly : The Cognitive
Modeling approach
 Allen Newell and Herbert Simon, who developed GPS, the “General
Problem Solver” (Newell and Simon 1961).

 Not content merely to have their program solve problems correctly.

 They were more concerned with comparing the sequence and timing
of its reasoning steps to those of human subjects solving the same
problems.

 cognitive science - computer models from AI + experimental

techniques from psychology - testable theories - human mind.
Acting Humanly : The Turing Test Approach
 Acting Humanly
natural language processing to communicate
successfully in a human language;

knowledge representation to store what it knows

or hears;

automated reasoning to answer questions and to

draw new conclusions;

machine learning to adapt to new circumstances

and to detect and extrapolate patterns.
 Acting Humanly
computer vision and speech recognition to
perceive the world;
 robotics to manipulate objects and move about.
Thinking Rationally : The “laws of thought”
approach
studies of ideal or rational thought process and
inference.

emphasis on the inferencing mechanism, and its

properties.

How the system arrives at a conclusion, or the

reasoning is very important.
Thinking Rationally : The “laws of thought”
approach.
The soundness and completeness of the inference
mechanisms are important – Logic – Syllogisms.
Eg; Socrates is a man, all men are mortal, so Socrates
is mortal.
Logic knowledge of the world that is certain.

We simply don’t know the rules of, say, politics or

warfare in the same way that we know the rules of
chess or arithmetic.

The theory of probability fills this gap, allowing

rigorous reasoning with uncertain information.
 Acting Rationally : The rational agent
approach
Deals with building machines that act rationally.

The focus is on how the system acts and performs,

and not so much on the reasoning process.

An agent is just something that acts; operate

autonomously, perceive their environment, persist,
adapt to change, and create and pursue goals.

A rational agent is one that acts rationally, that is, in

the best possible manner.
 Typical AI problems
Common Tasks.

Expert Tasks.
 Common Tasks
Recognizing people, objects.

 Communicating (through natural language).

Navigating around obstacles on the streets.

Other Tasks that are routinely done by humans and

animals.
 Expert Tasks
Medical diagnosis.

 Mathematical problem solving.

 Playing games like chess.

Tasks that performed by skilled specialists.

Common Tasks Vs Expert Tasks
Common Tasks – Easily mastered by humans, tough
for Computers.

Expert Tasks – Easily mastered by Computers, tough

for Humans.
 Intelligent Behavior
Perception involving image recognition
and computer vision
Reasoning
Learning
Understanding language involving natural
language processing, speech processing
Solving problems
Robotics
 Practical Impact of AI
AI components are embedded in numerous devices
e.g. in copy machines for automatic correction of
operation for copy quality improvement.
 AI systems are in everyday use for identifying credit
card fraud.
for advising doctors, for recognizing speech and in
helping complex planning tasks.
Intelligent tutoring systems that provide students
with personalized attention
 Approaches to AI
Strong AI
Weak AI
Applied AI
Cognitive AI
 Strong AI
Aims to build machines that can truly reason and
solve problems.
should be self aware and their overall intellectual
ability needs to be indistinguishable from that of a
human being.
Excessive optimism in the 1950s and 1960s concerning
strong AI has given way to an appreciation of the
extreme difficulty of the problem.
 Strong AI maintains that suitably programmed
machines are capable of cognitive mental states.
 Weak AI
creation of some form of computer-based artificial
intelligence that cannot truly reason and solve
problems.

Can act as if it were intelligent.

Weak AI holds that suitably programmed machines

can simulate human cognition.
 Applied AI
Aims to produce commercially viable "smart"
systems.

 Eg:A Security system that is able to recognise the

faces of people who are permitted to enter a
particular building.

Applied AI has already enjoyed considerable success.

 Cognitive AI
computers are used to test theories about how the
human mind works.

Example : Theories about how we recognize faces and

other objects, or about how we solve abstract
problems.
 Limits of AI Today
Today’s successful AI systems operate in well-defined
domains and employ narrow, specialized knowledge.

Common sense knowledge is needed to function in

complex, open-ended worlds.

Such a system also needs to understand unconstrained

natural language.

These capabilities are not yet fully present in today’s

intelligent systems.
 What can AI systems do
In Computer vision, the systems are capable of face
recognition
 In Robotics, we have been able to make vehicles that
are mostly autonomous.
In Natural language processing, we have systems that
are capable of simple machine translation.
 Today’s Expert systems can carry out medical
diagnosis in a narrow domain
 Speech understanding systems are capable of
recognizing several thousand words continuous
speech
 What can AI systems do
Planning and scheduling systems had been employed
in scheduling experiments with the Hubble
Telescope.

The Learning systems are capable of doing text

categorization into about a 1000 topics

 In Games, AI systems can play at the Grand Master

level in chess (world champion), checkers, etc.
What can AI systems NOT do yet
Understand natural language robustly (e.g., read and
understand articles in a newspaper)

 Surf the web.

 Interpret an arbitrary visual scene.

 Learn a natural language.

 Construct plans in dynamic real-time domains .

 Exhibit true autonomy and intelligence

 AI History
Aristotle (384-322 BC) developed an informal system
of syllogistic logic, which is the basis of the first
formal deductive reasoning system.
Early in the 17th century, Descartes proposed that
bodies of animals are nothing more than complex
machines.
Pascal in 1642 made the first mechanical digital
calculating machine.
In the 19th century, George Boole developed a binary
algebra representing (some) "laws of thought."
 AI History
Charles Babbage & Ada Byron worked on
programmable mechanical calculating machines.
Gottlob Frege, Bertram Russell, Alfred North
Whitehead, and Kurt Gödel built on Boole's initial
logic concepts to develop mathematical
representations of logic problems.
In 1943 McCulloch & Pitts developed a Boolean circuit
model of brain. They wrote the paper “A Logical
Calculus of Ideas Immanent in Nervous Activity”,
which explained how it is possible for neural networks
to compute.
 AI History
Marvin Minsky and Dean Edmonds built the SNARC
in 1951, which is the first randomly wired neural
network learning machine (SNARC stands for
Stochastic Neural-Analog Reinforcement Computer).
It was a neural network computer that used 3000
vacuum tubes and a network with 40 neurons.
In 1950 Turing wrote an article on “Computing
Machinery and Intelligence” which articulated a
complete vision of AI.
 AI History
In 1956 a famous conference took place in
Dartmouth. The conference brought together the
founding fathers of artificial intelligence for the first
time. In this meeting the term “Artificial Intelligence”
was adopted.
Between 1952 and 1956, Samuel had developed several
programs for playing checkers.
In 1956, Newell & Simon’s Logic Theorist was
published. It is considered by many to be the first AI
program. In 1959, Gelernter developed a Geometry
Engine.
 AI History
In 1961 James Slagle (PhD dissertation, MIT) wrote a
symbolic integration program, SAINT. It was written
in LISP and solved calculus problems at the college
freshman level.
In 1963, Thomas Evan's program Analogy was
developed which could solve IQ test type analogy
problems.
In 1963, Edward A. Feigenbaum & Julian Feldman
published Computers and Thought, the first
collection of articles about artificial intelligence.
 AI History
In 1965, J. Allen Robinson invented a mechanical
proof procedure, the Resolution Method, which
allowed programs to work efficiently with formal logic
as a representation language.
In 1969 the SRI robot, Shakey, demonstrated
combining locomotion, perception and problem
solving.
In 1974: MYCIN demonstrated the power of rule-
based systems for knowledge representation and
inference in medical diagnosis and therapy.
 AI History
In the 1980s, Lisp Machines developed and marketed.
Around 1985, neural networks return to popularity.
In 1988, there was a resurgence of probabilistic and
decision-theoretic methods .
 AI History
The 1990's saw major advances in all areas of AI like
machine learning, data mining
intelligent tutoring,
case-based reasoning,
multi-agent planning, scheduling,
uncertain reasoning,
natural language understanding and translation,
 vision, virtual reality, games, and other topics.
 Famous AI system
ALVINN-Autonomous Land Vehicle In a Neural
Network.
 The system drove a car from the East Coast of USA to
the west coast, a total of about 2850 miles.
Out of this about 50 miles were driven by a human,
and the rest solely by the system.
 Famous AI system
Deep Blue
In 1997, the Deep Blue chess program created by IBM,
beat the, then world chess champion, Gary Kasparov.
Machine translation
Autonomous agents
In space exploration, robotic space probes
autonomously monitor their surroundings, make
decisions and act to achieve their goals.
eg : NASA’s Mars Rovers.
 Internet agents. – to monitor users' tasks, seek
needed information, and to learn which information
is most useful
 Depth First Search (DFS)
Expands the deepest node in the current frontier.

nodes expanded, - dropped from the frontier, - “backs

up” to the next deepest node -unexplored successors.

BFS – Graph Search – FIFO

DFS – Graph search – LIFO

A LIFO queue means - most recently generated node

is chosen for expansion.
 DFS
Alternative Implementation- a recursive function
that calls itself on each of its children in turn.

Graph-search version - avoids repeated states and

redundant paths - complete in finite state space.

The tree-search version, is not complete.

In infinite state spaces, both versions fail if an infinite

non-goal path is encountered.
 DFS
Both versions are non optimal.

explore the entire left subtree even if node C is a goal

node.

If node J were also a goal node, then depth-first

search would return it as a solution instead of C.

Depth-first search is not optimal.

Time Complexity – O(bd)
Space Complexity – O(bm)
 Why DFS?
So far, DFS - no clear advantage over BFS

 so why DFS?

The reason is the space complexity

assuming that nodes at the same depth as the goal

node have no successors.

DFS requires 156 kilobytes instead of 10 Exabyte at

depth d = 16 - a factor of 7 trillion times less space !!!
 Backtracking DFS
A variant of DFS - backtracking search - still less memory.

Backtracking - only one successor generated - at a

time – not all successors.

 partially expanded node - remembers which

successor to generate next.

In this way, only O(m) memory is needed rather than

O(bm).
 A* Algorithm – Time
Complexity
For problems with constant step costs, the growth in
run time as a function of the optimal solution depth d is
analyzed in terms of the absolute error or the relative
error of the heuristic.

The absolute error is defined as ∆ ≡ h ∗ − h, where h ∗ is

the actual cost of getting from the root to the goal, and

the relative error is defined as ǫ ≡ (h∗ − h)/h∗

Time complexity = O(b^εd)

 SMA* Algorithm
SMA∗ is complete if there is any reachable solution—
that is, if d, the depth of the shallowest goal node, is
less than the memory size (expressed in nodes).

It is optimal if any optimal solution is reachable;

otherwise, it returns the best reachable solution.

SMA∗ is forced to switch back and forth continually

among many candidate solution paths, only a small
subset of which can fit in memory. (This resembles
the problem of thrashing in disk paging systems.)
 SMA*
Then the extra time required for repeated
regeneration of the same nodes means that problems
that would be practically solvable by A∗, given
unlimited memory, become intractable for SMA∗.

memory limitations can make a problem

intractable from the point of view of
computation time.
The effect of heuristic accuracy on
performance
Characterize the quality of a heuristic is the effective
branching factor b ∗ .
 total number of nodes generated by A∗ for a
particular problem is N - solution depth is d, then b ∗
is the branching factor that a uniform tree of depth d
would have to have in order to contain N + 1 nodes.
Thus,
 The effect of heuristic accuracy on
performance
For example, if A∗ finds a solution at depth 5 using 52
nodes, then the effective branching factor is 1.92.

A well designed heuristic would have a value of b ∗

close to 1, allowing fairly large problems to be solved
at reasonable computational cost.
 Summary
Before an agent can start searching for solutions, a
goal must be identified and a well defined problem
must be formulated.
 A problem consists of five parts: initial state, actions,
goal test, path cost, solution.
 Search algorithms - judged on the basis of
completeness, optimality, time complexity, and space
complexity
 Searching with Non Deterministic
Actions
Uninformed / Informed Search technique - Assumed
That The Environment Is Fully Observable And
Deterministic .
Partially Observable Environment – Percepts -
Narrow Down The Set Of Possible States - Easier To
Achieve Goals.

Non deterministic Environment - Percepts - Which

Of The Possible Outcomes Of Its Actions Has
Actually Occurred.
 Searching with Non Deterministic
Actions
Both Cases - Future Percepts - Cannot Be Determined In
Advance - The Agents Future Actions - Depend On Those
Future Percepts.

Solution To A Problem - Not A Sequence - contingency

Plan – or - Strategy
Searching with Non Deterministic Actions – The
erratic vaccum world
Searching with Non Deterministic Actions – The
erratic vaccum world
We Introduce Nondeterminism, The Suck action
works as follows:

When applied to A dirty square - the action cleans

the square and - sometimes cleans up dirt in an
adjacent square too.

When applied to a clean square the action sometimes

deposits dirt on the carpet
Searching with Non Deterministic Actions –
The erratic vaccum world
Eg: Erratic vacuum world.

 suck action - state 1- leads to a state in the set {5, 7}.

Eg: If we start in state1 there is no single sequence Of

actions that solves the problem instead we need a
contingency plan:
Searching with Non Deterministic Actions –
The erratic vaccum world

Solutions for nondeterministic problems can contain

nested if-then-else Statements - they are trees rather
than sequences.

Many problems in the real physical world are

contingency problems because exact prediction is
impossible.
 AND-OR Search Trees
In a deterministic environment - only branching is
introduced by the agents own choices or node in each
state. We call these nodes OR Node.

In the vacuum world at an OR node the agent

chooses Left or Right or Suck.

In a nondeterministic environment branching is also

introduced by the environment’s choice of outcome
for each action – AND Nodes
 AND-OR Search Trees
The suck action in state1 leads to a state in the set {5,
7}.
 AND-OR Search Trees
A Solution for an AND-OR Search problem is a
subtree that:

Has a goal node at every leaf

Specifies one action at each of its OR nodes.
Includes every outcome branch at each of its AND
nodes.
Imperfect Real-Time Decisions
The minimax algorithm generates the entire game
search space.

alpha–beta algorithm - prune large parts of it.

However, alpha–beta still - search all the way to

terminal states.

This depth - not practical - moves must be made in a

reasonable time.
Imperfect Real-Time Decisions
Claude Shannon’s paper Programming a Computer
for Playing Chess (1950) - programs should cut off
the search earlier and apply a heuristic evaluation
function.
 Imperfect Real-Time Decisions -
Evaluation functions
An evaluation function returns an estimate of the
expected utility - as the heuristic functions.

An inaccurate evaluation function will guide an agent

toward positions that turn out to be lost.

How exactly do we design good evaluation functions?

 How to design Evaluation
Functions?
evaluation function should order the terminal states
in the same way as the true utility function: states
that are wins must evaluate better than draws, which
in turn must be better than losses.

the computation must not take too long! (The whole

point is to search faster.)

for nonterminal states, the evaluation function

should be strongly correlated with the actual chances
of winning.
 How to design Evaluation
Functions
Most evaluation functions work by calculating
various features of the state.

Eg: in chess, we would have features for the number

of white pawns, black pawns, white queens, black
queens, and so on.

The features, taken together, define various

categories or equivalence classes of states..
How to design Evaluation Functions
Eg: One category contains all two-pawn vs. one-pawn
end games.

suppose 72% lead to a win (utility +1); 20% to a loss

(0), and 8% to a draw (1/2).

 expected value: (0.72 × +1) + (0.20 × 0) + (0.08 × 1/2) = 0.76.

The expected value can be determined for each

category.
too much experience to estimate all the probabilities
of winning.
 How to design Evaluation Functions
Mathematically, this kind of evaluation function is
called a weighted linear function because it can be
expressed as.

For chess, the fi could be the numbers of each kind of

piece on the board, and

the wi could be the values of the pieces (1 for pawn, 3

for bishop, etc.).
How to design Evaluation Functions
assigning the value 3 to a bishop ignores the fact that
bishops are more powerful in the endgame. – Use
Machine Learning to learn weights !!

Cutting off search - modify ALPHA-BETA-SEARCH -

call the heuristic EVAL function when it is
appropriate to cut off the search.

Use Depth limited / IDS – with IDS – Move Ordering

– More Robust !!