PART I - PROBABILITY Chapter 1.

Introduction to Probability
1.1 Examples. The need for probability
(a) Clinical trial of a new drug. Old drug has 82% success rate. New drug claims a 90% success rate. Try drug on 100 patients, investigate the success rate. 1. Need for a reasonable number of patients. 2. Different groups give different results. need for theory. 3. How should groups be chosen? (b) Betting: combination of odds. Odds against A: a 1. Odds against B: b 1. Wish to bet on A or B winning. The odds needed are Similarly: health studies and measures of risk. (c) Opinion Polling Access current political view of the electorate, e.g. By-election, 2 candidates.
[simplify: no ``don't knows'']

(a +1)( b +1) 1 ( a + b + 2)

Choose say, 100 from the electorate. Problems: possibility of ``bias'' inherent error (uncertainty) of sampling Need the concept of a random sample. The same features appear in many other examples, e.g., (d) Accident studies: before/after remodelled road junction. (e) Quality control: incoming batch of fireworks, requires accepting or rejecting. Select a random sample to test. Probability theory is that branch of mathematics that treats those aspects of systems that have random or haphazard features.

1.2 Experiment and Event

In each example in Section 1.1 something done e.g. drug given to patient, horse race, sample drawn from the electorate has an uncertain result e.g. number cured, winner, identity of sample members. These are called the EXPERIMENT or TRIAL and EVENT, OUTCOME or RESULT. Experiments are considered to be repeatable, but at any repetition we do not know what the result will be. We do know the set of all possible outcomes.

DEFINITION: THE SAMPLE SPACE This is the set of all possible outcomes of the experiment. EXAMPLES: (1) (2) Experiment: toss a coin 5 times and count the number of heads. Sample Space: {0, 1, 2, 3, 4, 5 }. Experiment: As (1) above, but record the event in order. Sample Space: all 25 5-tuples of $H$ and $T$.
i.e. HHHHH HHHHT HHHTH

TTTTT

Both these sample spaces are finite. There are other possibilities. (3) Experiment: Toss a coin repeatedly, record the number of T's preceding the first H.
e.g. TTH 2 TTTTTH 5

Sample space: {0, 1, 2, } (4) Experiment: Record the duration (in seconds) of the next telephone call through the UKC switchboard. Sample Space: Set of all real numbers.

Notes on Samples Spaces (SS)

In example 4 the SS is continuous, all the others are discrete. Usually an experiment involving measuring has a continuous SS and one involving counting a discrete SS. For most practical purposes we ignore the fact that strictly all SS's are discrete because of the finite precision with which numbers are recorded. The SS of an experiment is not necessarily unique: there may well be more than one way to describe the outcomes. (c.f. earlier examples on coin tossing.) The elements of a SS form a set of outcomes. Notation: Experiment E and Sample Space S Definition: An event is any subset of a sample space. Event is even; exceeds 3; is 2 or 5; contains no pair of consecutive H's; is HTTTH; has 2nd element equal to T. as 1, above. between 100 and 200;
60;

Examples of events SS 1. number of head 2. the 5-tuple 3. Number of T's preceding first H 4. Measurement

> 500.

VENN DIAGRAMS

Ways of illustrating SS's, outcomes and events, and their relationships. The SS S is represented by a rectangle. The outcomes in S are represented by points in the rectangle. Events are collections of outcomes, so are represented by collections of points, i.e. by regions. Events are subsets of S. They are identified by drawing a closed curve around the relevant points.

EXAMPLE E: Toss 4 coins, record sequence.

Event A: 3 or 4 H Event B: exactly 2 H Event C: 2nd and 3rd result H The general configuration for 3 events is like this:

When using Venn diagrams to prove relationships between events, always draw them as above, to include all possible intersections.

The 4 coins example (earlier) is a special case: A and B do not overlap If C occurs then one of A and B must occur. This can be illustrated like this:

But usually the general form is used with some regions being recognised as empty. Events of special importance

1. {x}$, where x is a single element of S, i.e. an outcome.

DEFINITION: An event comprising exactly one outcome is often termed a simple event; any other event is a compound event.

2. S itself -- the certain event.

3. 0 or , a dummy event containing no outcomes, the impossible event. 1.3 Relations between sets Notation: Sets: A, B, , E1, B3 Elements: x, y, xA: x is contained in A. xA: x is not contained in A. A B: A is contained in B. UNION of sets: The union of A and B (A B) consists of all elements in A or B or both. INTERSECTION of sets: The intersection of A and B (A B), consists of all elements in both A and B. COMPLEMENT of a set: The complement of a set ( A ) consists of those elements not in A. These three are the basic relations, all other can be derived by using combinations of them.

1. It is often convenient to define extra notation. e.g A B: in A but not in B

x ( A B ) ( x A) and ( x B )

i.e. x ( A B ) , hence A B = A B 2. Extensions to more than two events (care needed)

A B C = ( A B) C = A ( B C )

Similarly A B C = ( A B ) C = A ( B C ) But ( A B) C A ( B C ) Compare with the algebraic use of + and 2 3 4 2+3+4 2 3+4 DEFINITIONS 1. Events A1,A2,,An are mutually exclusive if no two can occur simultaneously, i.e. if Ai Aj = for al i j 2. Events A1,A2,,An are mutually exhaustive if at least one is certain to occur, i.e. if A1 A2 An = S Several important results are concerned with events that are mutually exclusive and exhaustive. 1.4 Probability A number attached to each event arising out of an experiment. The ``more likely'' the event the large the probability. Notation: Event A has probability denoted by P(A) Pr(A). e.g. Pr(total of 7 when two dice are thrown). Consider an experiment performed n times; suppose that event A occurs on nA of these. The frequency of A is nA. The relative frequency (r.f.) of A is

nA . n

The r.f. gives an idea of how likely A is to occur, but it cannot be equal to Pr(A) because in a further n trials nA will be different. If just one further trial is performed,

nA n +1 nA changes to A or n n +1 n +1
Calibration of probability scale: 0 Impossible fair coin

1 certain

Axioms of Probability A1 A2 A3 Notes For any event A, 0 Pr(A) 1. For the event S, Pr(S) = 1. For the two events A and B satisfying A B = Pr(A B) = Pr(A) + Pr(B).

1) The basic idea of probability is for practical purposes linked to relative frequency but
it is not the same thing. For any event, the r.f. can and does vary; the probability is fixed.

2) Mathematically, the theory of probability develops from the axioms, without

reference to the interpretation. e.g. the same axioms apply to probability regarded as a measure of degrees of belief (subjective probability) in Bayesian statistics.

1.5 Numerical assessment of probabilities

Three main ways of doing this: symmetry limiting relative frequency subjective (degrees of belief) SYMMETRY This can be applied only when the experiment is symmetrical with respect to its outcomes, which are all known to be equally likely. Calculation: If there are n outcomes and if all are equally likely, each has probability 1 n If an event A comprises a outcomes then a Pr( A) = n No. of outcomes favourable to event So Pr(event) = Total no. of outcomes Applications: mainly to games of chance sampling problems genetics Probability calculations reduce to counting problems.

LIMITING RELATIVE FREQUENCY Relative frequencies cannot be used directly to define probabilities. We need to define Pr(A), where A is some event based on the experiment E. Let E be performed n times, and let the number of occurrences of A be nA. Let S n = consider a graph of Sn against n.

nA and n

The graph ``settles down'' to a r.f. of 0.5. The intuitive idea is that as n gets large Sn gets close to some constant defined as Pr(A). (The convergence is not straightforward). This is the Limiting Relative Frequency definition of probability. IMPORTANT DISTINCTION: For any finite n, Sn is not the probability of A. It may be thought of as an estimate of Pr(A). SUBJECTIVE PROBABILITY These represent degrees of belief. e.g.1. Party X will win the election e.g.2. The 11.09 to London today will be late. This extends the concept of probability to events which do not have a long run frequency interpretation. Closely related to ideas of consistent betting behaviour. Such ideas are becoming very widely used in certain areas of statistics (Bayesian methodology).

SUMMARY - Chapter 1
The subject of probability relates to experiments whose results are uncertain but must be one of a set of outcomes, the sample space. Events are sets and can be manipulated (, , A etc) To each event (E) is attached a number, its probability, denoted by P(E) or Pr(E).

Probabilities satisfy certain conditions -- the axioms -- on which the development of probability theory is based. The axioms of Probability are: A1 For any event A, 0 Pr(A) 1. A2 For the event S, Pr(S) = 1. A3 For the two events A and B satisfying A B = Pr(A B) = Pr(A) + Pr(B). Several interpretations of probability obey the system of axioms.