(Winsberg 2019 SEP)

“In its narrowest sense, a computer simulation is a program that is run on a

computer and that uses step-by-step methods to explore the approximate behavior of a
mathematical model. Usually this is a model of a real-world system (although the
system in question might be an imaginary or hypothetical one). Such a computer
program is a computer simulation model. One run of the program on the computer is a
computer simulation of the system. “

SEP AI

“These answers all assume that AI should be defined in terms of its goals: a
candidate definition thus has the form “AI is the field that aims at building …” The
answers all fall under a quartet of types placed along two dimensions. One dimension
is whether the goal is to match human performance, or, instead, ideal rationality. The
other dimension is whether the goal is to build systems that reason/think, or rather
systems that act. The situation is summed up in this table:”

“A huge part of AI’s growth in applications has been made possible through
invention of new algorithms in the subfield of machine learning. Machine learning is
concerned with building systems that improve their performance on a task when given
examples of ideal performance on the task, or improve their performance with
repeated experience on the task. Algorithms from machine learning have been used in
speech recognition systems, spam filters, online fraud-detection systems, product-
recommendation systems, etc. The current state-of-the-art in machine learning can be
divided into three areas”

1. Supervised Learning: A form of learning in which a computer tries to learn a

function f given examples, the training data T, of its values at various points in
its domain T={⟨x1,f(x1)⟩,⟨x2,f(x2)⟩,…,⟨xn,f(xn)⟩}.A sample task would be
trying to label images of faces with a person’s name. The supervision in
supervised learning comes in the form of the value of the function f(x) at
various points x in some part of the domain of the function. This is usually
given in the form of a fixed set of input and output pairs for the function. Let h
be the “learned function.” The goal of supervised learning is have h match as
 closely as possible the true function f over the same domain. The error is
usually defined in terms of an error function, for instance, error=∑x∈Tδ(f(x)
−h(x)), over the training data T. Other forms of supervision and goals for
learning are possible. For example, in active learning the learning algorithm
can request the value of the function for arbitrary inputs. Supervised learning
dominates the field of machine learning and has been used in almost all
practical applications mentioned just above.
2. Unsupervised Learning: Here the machine tries to find useful knowledge or
information when given some raw data {x1,x2,…,xn}. There is no function
associated with the input that has to be learned. The idea is that the machine
helps uncover interesting patterns or information that could be hidden in the
data. One use of unsupervised learning is data mining, where large volumes of
data are searched for interesting information. PageRank, one of the earliest
algorithms used by the Google search engine, can be considered to be an
unsupervised learning system that ranks pages without any human supervision
(Chapter 14.10, Hastie et al. 2009).
3. Reinforcement Learning: Here a machine is set loose in an environment where
it constantly acts and perceives (similar to the Russell/Hutter view above) and
only occasionally receives feedback on its behavior in the form of rewards or
punishments. The machine has to learn to behave rationally from this
feedback. One use of reinforcement learning has been in building agents to
play computer games. The objective here is to build agents that map sensory
data from the game at every time instant to an action that would help win in
the game or maximize a human player’s enjoyment of the game. In most
games, we know how well we are playing only at the end of the game or only
at infrequent intervals throughout the game (e.g., a chess game that we feel we
are winning could quickly turn against us at the end). In supervised learning,
the training data has ideal input-output pairs. This form of learning is not
suitable for building agents that have to operate across a length of time and are
judged not on one action but a series of actions and their effects on the
environment. The field of Reinforcement Learning tries to tackle this problem
through a variety of methods. Though a bit dated, Sutton and Barto (1998)
provide a comprehensive introduction to the field.