Contents Features of this Teat i Preface wit 1 Experiments, Models, and Probabilities Gelling Started with Probability Z Lt Set Theory g 1.2 Applying Set Theory to Probability 7 1.3 Probability Axioms a 1.4 Conditional Probability 15 Partitions and the Lea af Tolal Probability 18 Independence 24 1.7 Marusp 27 Problems 29 equential. Experiments 35 1 Tree Diagrams 35 2.2 Counting Methods 40 2.3 Independent Trials 49 24 Reliability Analysis 52 2.5 Marian 58 Problems 57 3 Discrete Random Variables 62 Definitions 62 Probability Mass Function 65 Families af Discrete Random Variables 68 Cumulative Distribution Function (CDF) 17 Averages end Bupected Value 80 Functions of a Rendom Variable 86 Bepected Value of a Derived Random Variable 90 Variance and Standard Deviation 93 MATLAB 99 Problems 106xiv CONTENTS { Continuous Random Variables 118 4.1 Continuous Sample Space 118 4 The Cumulative Distribution Function 12t 4.3 Probability Density Function 123 4.4 Bvpected Values 128 4.5 Families of Cowlinuous Random Variables 132 4.6 Gaussian Random Variables 158 4.7 Delta Functions, Miwed Random Variables U5 4.8 Marian 152 Problems 154 Multiple Random Variables 162 5.1 Joint Cumulative Distribution Function 163 5.2 Joint Probability Mass Function 166 5.3 Marginal PMF 169 5.4 Joint Probability Densily Function 1H 5.5 Marginal PDE 17 5.6 Independent Random Variables 178 3.7 Expected Value of a Function of Two Random Variables 5.8 Covariance. Correlation and Independence 5.9 Bivariate Gaussian Random Variables 5.10 Multivariate Probabilily Models 5.11 Marias Problems Probability Models of Derived Random Variables 6.1 PMF of a Function of Two Discrete Random Variables 6.2 Funetions Yielding Continuous Random Variables 6.3 Funetions Yielding Discrete or Mixed Random Variables 6.4 Continuous Functions of Two Continuous Random Variables 6.5 PDF of the Sum of Two Random Variables 6.6 MaTuaR Problems % Conditional Probabitity Models 4.1 Conditioning a Random Variable by an Event Conditional Expected Value Given an EventCONTENTS av Conditioning Two Random Variables by an Event Conditioning by a Random Variable Coniitional Expected Value Given a Random Variable Bivariate Gaussian Random Variables: Conditional PDFs 17 MaviaR Problems 8 Random Vectors Vector Notation Jndependent Random Variables and Random Vectors Funetions of Random Vectors Expected Value Veetor and Correlation Matria Gaussian Random Vectors ‘MATLAB Problems 9 Suins of Random Variables 306 9.1 Expected Values of Sums 306 9.2 Moment Generating Functions 310 9.3 MGF of the Sum of Independent Random Variables 314 9.4 Random Sums of Independent Random Variables 317 9.5 Central Limit Theorem #21 9.6 Mavuaw 528 Problems 331 10 The Sample Mean 337 10.1 Somple Mcan: Expected Value and Variance 10.2 Deviation of a Random Variable from the Expected Value 939 10.3 Laws of Large Numbers 343 10.{ Point Hstimates of Model Parameters 345 10.5. Confidence Intervals 352 10.6 Marvan 558 Problems 360 If Hypothesis Testing 366 11.1 Significence Testing 507 11.2. Binary Hypothesis T 11.3. Multiple Hypothe sting 310 9841 Experiments, Models, and Probabilities Get 1g Started with Probability The title of this book is Probability and Stochastic Processes, We say and hear aud read the word protahifity and lis relatives (possible. probable, probably) tn many contexts, Within the realm of applied mathematics, the meaning of profability is a question that has occupied maihenutticians, philosophers, scientists, and social scientists for hundreds of years Byeryoue accepts that the probability of au event isa uumber betwoen () and 1. Some prople interpret probability as a physical property: (like mass or voime or temperature) that can be measured. This is tempting wheu we talk about the probability that a. coin flip will come ap heads. This probability is closely related to the nature of the coin, Fiddling around with the coin eau alter the probability of heads Another interpretation of probability relates to the knowledge that we have about something. We might assign a low probability to the truth of the statement, Fé is rainang now in Phoenir, Arizona, because we know that Phoenix is in the desert Howover, our knowledge changes if we learn that it was taining au hour ago in Phoenix. This knowledge would cause us to assign a higher probability to the truth of the siaement, Lf ts raining now in Phoenix Both views are useful when we apply probability theory to practical problems. Whichever view we take, we will rely on the abstract mathematics of probability, which consists of definitions, axioms, and inferences (Hhearems) that follow from the axioms. While the structure of the subject conforms to principles of pure logic, the: terminology is not entirely abstract, Tastead, it reflects the practical origins of proba prodicted with we started studying physics. There we said that if we do the same thing in the again — send a space shuttle inte orbit, for example ity theory, which was devloped to deseribe phenomena that cannot be tainty. The point of view is different from the uno we toole when same way over snd aver2 CHAPTER 2 EXPERIMENTS, MODELS, AND PROBABILITIES the result will always be the same. ‘To predict the result, we Lave to take account of all relevant, facts The mathematies of probability begins when the situation is so coniplex that we just can’t replicate everything important exactly, like when we fabricate and twst an integrated eiteuit. Tn this case, repetitions of the same procedure yield different resulis, The situation is not totally chaotic, however, While each outcome may he unpredictable, there are consistent patterns to be observed when we repeat the procedure a large umber of Limes, Uuiderstanding these pattenus helps euglnesrs establish test procednires to ensure that a factory meets «quality objectives. Tn thi repeatable procedure (making aud texting a chip) with mpredictable outeomes (the quality of individual chips), the probability isn nambor between () and 1 that states the proportion of times we expect a cortain thing to happen, such as the proportion of chips that pass a test As an introduction to probability and stochastic processes, this book serves three purposes: # Ik introduces students tu the logic of probability theory. # It helps students develop intuition into how the theory relates te practical situations # Tt teaches students how to apply probability theory to solving engineering problems, To exhibit the logic of the subject, we show clearly in the text three catezories of theoretical material: definitions, axioms, and theorems. Definitions establish the logic of probability cheory, und axioms are facts that we accept without proof, Theorems are consequences that follow logically from definitions and axioms. Fach theorem has a proof that refers to definitions, axioms, and other theorems, Al- though there are dozens of definitions and theorems, there ary only jhree axioms of probability theory. ‘These thiee axioms are the foundation on which the entire subject rests, To meet our goal of presenting the logic of the subject, we could. set out the material as dozens of definitions followed by three axioms followed by dozens of theorems. Each theorem would be accompanied by a complete proof, While rigorous. this approach wonld completely fail to mect. onr second aim of conveying the intuition necessary to work on practical problems, ‘To address this goal, we augment the pirely mathematical material with a large number of examples of practical phenomena that can be analyzed by means of probability theory. We also interleave definitions and theorems, presenting some theorems with complete proofs, presenting others with partial proofs, and omitting some proofs altogether. We find that most engineering students study probability with the aim of using it to solve practical problems, and we eater mostly 10 this goal, We also encourage students te take an interest in the logic of the subject — it is very elegant — and, wo feel that the material presented. is sufficient to enable these students to fll in te gaps we Therefore, as you read this book you will find « progression of definitions, axioms, aye left in the prooks. theorems, more comments designed te contribute to your understanding of the theory. We also include brief quizzes that you should try ta solve as you read the book. Each one cfnitions, and more theorems, all interleaved with examples and41 SET THEORY 3 will help you decide whether you lave grasped the material presented just before the quiz. The problems at the end of each chapter give you more practice applying. the material introduced in the chapter. ‘They vary considerably in their level of difficulty. Some of them take you more deeply into the subject than whe examples and quizzes do. 1.1 Set Theory We deal with uncertainty every day, Pople tnd machines use probability theory to deseribe uncertainty about whether an event oceurs and to make decisions in the presence of this uncertainty. ‘The matheuuities of probability relies on elements the theory of sets reviewed in this section. The mathomatieal basis of probability is the theory of sets. Most people who study probability have already encountered set theory and are fariliar with Suck tert as set, element, union, interseetion, and complement, For them, the following paragraphs will review material-already learned and introduce the notation and terminology we tse here. For people who have no prior acquaintance with sets, this, material introduces basic definitions and the properties of sets that are important in the study of probability A set ip a collection of things, We use capital letters to denote sets. The things that together make up the'set are clemenis, When we use mathennatical notation nally use small letters. Thus we can ha bol Sp ig an clement af sot A.” The symbol ¢ is the opposite of €. Thus ¢ ¢ A moans “eis not an element of set A.” fo refer to set elements, we 1 set A with elements x,y. and 2, The sy! denotes set inclusion, Thus e € A means Tr is essential when working with sets to have a definition of each get. The definition allows someone to consider anything conceivable and determine whether that thing is an plement of the set. There are many ways to define # set. One way is simply to name the elements: {Rutyers University, New York University, the planet Moreury}, — (L1) Note that in stating the definition, we write the name of the set on one side of = and the definition in curly brackets { } ou the other side of =. Th follows that *the planet closest to the Sum € A® isa true statement: Tt is also true thot “Bill Clinton ¢ A” Another way of writing the set is to give a rule for tosting something to determine whether it is a inomber of the set B = {all Ritgers juniors who, weigh more than 170 ponnds} i) hu engineering, we offen use mathematical miles for generating all of the elements of the set 1,2,3.4.5} (13)4 CHAPTER 2 EXPERIMENTS, MODELS, AND PROBABILITIES This notation tells us to form a set by performing the operation to the left of the vertieal bar, |, on the mmbers to the right of the bar. Therefore, C = {1,4,9, 16,25} (ua) Some sets Lave an infinite number of elements, For example D= {z"|#=1,2,3,...} The dors tell us to continue the sequence to the left of the dots, Since there is no aumber to the right of the dots, we continme the sequence indefinitely, forming an infinite set containing all perfect squares except 0, The definition of D implies that Ide D and 10 ¢ D, In addition to ser inclusion, we also have the notion of » subset, whieh describes a wwlatiouship between twa sets, By definition, A is a subsel of B if every aeuber of A is also a member of B. We use the symbol < to denote subset. Thus Ac B is & subset of the set BL ip mathematical notation for the statement “the set 2 Using the definitions of sets Cand Din Equations (1.38) and (1.5), we observe that Oc DE T= {all positive integers, negative integers, and 0} (16) it follows that CC T.and DT. ‘The definition of set equali Bois {= Bifand only if BCA and ACR. This is che mathematical way of stating that A and B are identical if and only if every element of A is an element of J and every element of B is an element of A. This definition implies that a set is unaffected by the order of the elements in a. definition, For example, {0,17,46} = {17.0.46} = {46,0, 17} are all the same set. ‘Tu work with sets mathematically it iy necessary to define a umiversal set, This is the sct of ll things that we could possibly consider in a given context. Tn any study, all sot operations relate to the universal set for that study. The members of tho universal set include all of the clements of all of the sets in the study. We will use the letter § to denote the universal vet, Bur example, the universal get for A conld be S = {all amiversities in the United States, all planets}. The universal set for @ could be S = £ = {0,1,2,...}. By definition, every set is a subset of the universal set. That is, for any set ¥, X c The null set, whieh is also important, may seem like it is not a set at all. By definition it has no elements. The notation for the nal set is 2, By definition @ is a subset of every set. For any set A, ec A. B It iy customary to efor to Venn diagrams to display relationships among sets. By convention, the region rectangle is the universal set S. Closed surfaces within this roctangle denote sets. A enclosed by tle ta Veur diageum depicting, the relutionship A CL is shown on the Toft44 SETTMEORY 5 When we do set algebra, we form new sets from existing sets, There are three oper- ations for doing this: union, intersection, and complement. Union and intersection onibine two existing sets to produce a third set. ‘The complement operation forms ‘The notation aud definitions follow The union of sets A aud B is the set of all elements 4UB that axe either in A or in B, ot in both A und B is denoted by AUB. In this r AU Big the comploto shaded aron. Formally, a new set from one existing, n€ AUB ifand only ife cA ore B. ‘The set operation union corresponds to the logical operation, The éntersertion af two sets A and ig the set of all Ane elements that are contained both in Avand B. The intersection is denoted by AB. Another r ion a for intersection is AB. Formally, the definition 19.2 if and only if e © A and Lt The set operation intersection corresponds to the log= eal “and” function, 1 The complement of a set A, denoted by A’, is the set of all elements in $ that are not in A. The complerent of $ is the mull set 2, Formally, & 2 € A® fund only if ¢ A In working with probability we will offen refer to two important properties oF cc + the definitions, lvctions of sets. Here Aceollection of sets Al,...,Ap i8 rmutwally exelusine if A and only if The wore disjoint is somotimes wi mutually exclusive6 CHAPTER 2 EXPERIMENTS, MODELS, AND PROBABILITIES Acollcetion of sets / if aud only if «An is collectively exhaustive UAn = 8. (La) Ay Li the definition of coltectively exlaustive, we used the soe tation 4, U Ag Ue Uy for the union of N for ay tig ++ tq, wwe will use w shorthand for unious und inversectious of 1 sets: ‘hat cumbersome wo- sets. Just as 7% y 2) is a shorthand Ua (1.9) (1.40) Wo will see that eollections of sets that are both nmutually exchisive and colle. tively exhaustive are sufficiently useful to merit a definition. A collection of sets Ay,...,Ay is a partition. if it Ar | Ad) As | AG) poth wutually exclusive aud vollectively exhanstive, ‘Front the definition of set operations, we can derive many important relationships between sets and other sets derived from them, One example is ANBCA (uy To prove that this is trne, if iS necessary to show that far A (VA. then it is also true that © A, A proof that two sets are equal, for example, X = ¥, requires two separate proofs y be complicated to show. As we ser in the following theorem, this can Theorem 1.1 De Morgan's taw mtates all three baste operations: (Au'By mB Proof There ate two parts to the proof © To show (AUB) C A'TBS, supp AUB)’, That implies x ¢ AUB, Henes, ef Aud © 7 B, whieh together imply « ¢A° aud vee BF That is, 2 CAPO BE@ CHAPTER 2 EXPERIMENTS, MODELS, AND PROBABILITIES repeatable experiment. An experiment consists of a procedure and observations. There is tmeertainty in what will be observed: otherwise, performing the experiment would be unnseessary. Some examples of experiments inehicle 1. Flip acoin, Did it land wi Wh heads or tails facings up? Walk to a bus stop. How long do yon wait for the arrival of a bus? Give a lecture, Mow many students are seated in the fourth row? 4. Transmit one of a collection of waveforms over a channel. What waveform arrives at the receiver? Transmit one of a collection of waveforms aver channel. Which waveform does the receiver identify us the transmitted waveform? For the most part, we will analyze models of actnal physical experiments. We create models becaitse real experiments generally are too complicated to analyze. For example, to describe allof the factors affecting your waiting time at-a bus stop. you may consider © ‘The time of day. (Is it mush hour?) © The speed of each ear that passed by while you waited © ‘Dhe weight, Lorsepower, and gear ratios of cue kind of bus used by the bus company © The psychological profile aud work schedule of each bus driver. (Some drivers drive faster than others.) © ‘The wtatus of all roud construction within 100 miles of Uke bus stop. Tt should be apparent that it would bo difficntt to analyze the effect of each of Whese factors ou the likelilioud that you will wait less than five miautes fora bus. Cousequently, it is necessary to study a mode? of the esperiment that captures the important part of the actual physical experiment, Since we will focus ou the model of the experiment almost exchisively, we often will use the word experiment to refer to the model of an experiment. Example 1.2 ‘An experiment consists of the following procedure, observation, and model © Procedure: Moniter activity at @ Phonesmart store © Observation: Observe which type of phone (Apricot or Banana) the next customer purchases, © Model: Apricots and Bananas are equally likely. The result of each purchase is unrelated to the results of previous purchases. ‘As wo have said, an experiment consists of both a procedure and observations. It is important to understand thot two experiments with the stuue proceduze but with different observations are different experiments. For example, consider those two experiments:12 APPLYING SET THEORY TO PROBABILITY 9 Example 1.3 Monitor the Phonesmart store until three customers purchase phones. Observe the sequence of Apricots and Bananas Example 1.4 Monitor the Phonesmart store until three customers purchase phones. Observe the number of Apricots. These two experiments have the same procedure: monitor the Phonesmart store until three customers purchase phones. ‘They are different experimonts because they require different observations. We will describe models of experimen set of possible experimental outcomes. Tn the context of probability, we meuning te the word outcome. in terms of a give precise Definition 1.1 ‘Outcome An outcome of an experiment is any possible observation of that experiment. Tmplicit in the definition of an ontcome is the notion that each outcome is distin guishuble fro every other outcome. As a result, we define the universal set of all possible enteamies. Tn probability terms, we call this miversal set the saraple space Definition 1.2 ‘Sample Space The sample space of on eeperiment is the finest-qratn, mutual tively echmustive set of ail possible outcomes. exclusive, vollec- The finest-grain property simply means that all possible distinguishable outcomes are identified separately. "The requirement. (hat outcomes he umbually exchisive says that if one outcome occurs, then no other outcome also occurs. For the set of 8 to be collectively exhaustive, aust be onteos in the sample space ory outcome of the experin Example 1.5 # The sample space in Example 1.2 is 5 — {a,b} where a is the outcome “Apricot sold," and b is the outcome “Banana sold.’ © The sample space in Example 1.3 is [aria aab, aba, abl, bac, bas, Bh, bb) (uta) # The sample space in Example 1.4 is S$ — {0.1.2.3} Example 1.6: Manufacture an integrated circuit and test it to determine whether it meets quality objectives. The possible outcomes are “accepted” (11) and “rejected” (r). The sample space is $ ~ {a,"}.1.2 APPLYING SET THEORY TO PROBABILITY 7 To show ASABE ¢ (ALE)*, suppose x © ASA BS. In this caso, # CA and a cB Equivalently x ¢ A anda ¢ B so that x ¢ AUB. Hence, @ © (AUB) Example 1.1 Phonesmart offers customers two kinds of smart phones, Apricot (A) end Banana (2) It is possible to buy a Banana phone with an optional external battery F. Apricot customers can buy a phoné with an external battery (£2) or an extra memory card (C’) or both. Draw, a Venn diagram that shows the relationship among the items A,B,C and E available to Phonesmart customers. Since each phone is either Apricot or Banana, A and B form a partition, Since the external battery 2 is available for both kinds of phones, £ intersects both A and B, However, since the memory card Cis available only to Apricot customers, CC A. A Venn diagram representing these facts is shown on the right, Quiz 1.1 Gerlaudas olfers customers two kinds of pizza crust, Tuseau (1’) u 0 and Neapolitan (N). Th addition, each pizza may have mushe soouss (AM) or cious (Q) as described by the Veun diagram al right, Tor the sets specified below, shade the corresponding region of the Venn diagram. (a) (>) Num (eh Noe (@) ene 1.2. Applying Set Theory to Probability Probability is based on a repeutable experiment that co a procedure and observations. An outcome is an obseevation, An event is a set of outcomes, ‘The mathematics we study is a branch of measure theory. Probabiliny is a muanber that describes a ses. The higher che number, the more probability there is, In this seuse probabilicy is like a quantity that measures: a plysical phenomenon; for example, a weight or a temperature. Fowever, it is not necessary to think ahout probability in physical ters. We can do all the unath abstractly, just ax we defined, sets and set operations in the:previons paragraphs without any reference to physical phenomena, unarely for engineors, the language of probability (ineluding the word pm! ability itself) makes us think of things that we experience. The basie model is a10 CHAPTERL EXPERIMENTS, MODELS, AND PROBABILITIES et Algebra Probability Set Event Universal set Sample space Element Outcome Table 1.2 The terminology of set theory and prabobility Jn common speech, au event is Something that occurs. In an experiment, we may say that an event aceurs when a. certain phenomenon is observed. To define an event mathematically, we must identify afl outeomes for which the phenomenon is observed. ‘That is, for euch outcome, either the particular event occurs or it does not. In probability terms, we define an event in terms of the outcomes in the sample space. Definition 1.3 Event An event is.a set of outcomes of an experiment, ‘Table 1.1 relates the terminology of probability to set theory. All of this may seen so Simple thal it is boring, While this is true of the definitions themselves, applying them is a different matter. Defining the sample spare and its cutecmes, are key elements of the solution of any probability problent, A probability problem: arises from some practical sitnation that ean he modeled as an experiment. To work on the problew, it is necessary to define Lhe experiment carefully aud then derive the sample space. Getting this right is a hig step toward solving the problem Example 1.7 Suppose we roll a six-sided die and observe the number of dots on the side facing upwards. We can label these outcomes / = 1,...,G where ¢ denotes the outcome that i dots appear on the up face. The sample space is 5 = {1.2,...,6}. Each subset of Sis an event. Examples of events are # The event F; = (Roll 4 or higher} = (4.5.6) 21.6} = {14}. © The event (The roll is even} = © By — {The roll is the square of an integer Example 1.8 Observe the number of minutes a customer spends in the Phonesmart store. An out: come T is a nonnegative real number. The sample space is $ — {T|T > 0}. The event “the customer stays longer than five minutes is {7'’ > 5}1.3 PROBABILITY AXIOMS — LL Example 1.9 Monitor three customers in the Phonesmart store, Classify the behavior as buying () if a customer purchases a smartphone. Otherwise the behavior is no purchase (r)). An outcome of the experiment is a sequence of three customer decisions. We can denote each outcome by a three-letter word such as bu indicating that the first and third customers buy a phone and the second customer does not. We denote the event that customer i buys a phone by B, and the event customer i does not buy a phone by Ni. The event Be — {nbn,nbb.tin, bbb}. We can also express an outcome as an intersection of events 1 and. For example the outcome bib = W1.No Ly, Quiz 1.2 Monitor thrce consseutive packets goin packet header, each packet can be classified as either video (n) if it was sent from a Youtub lotters (each letter is through a Internet router, Based an the server or as ordinary data (d). Your observation is # sequence of three ther v or d). For example, iwo video packets followed by one data packet corresponds to vvd. Weite the elements of che following sors A = {second packet is video}: By — {second packer is data}, 4a = {oll packets are the same}. Ry = {vidleo and date alternate} Ay = {one oF uiore video packets}, —_Lly = {two or uiore data packets} For each pair of events Ay and By, Ag aud By, aad so on, identify whether the pair of events is either inu{ually exclusive or cullectiyely exhaustive or both. 1.3 Probability Axioms A probability wodel assigns a uimaber between 0 aud 1 to ever event, The probability of the anion of mutually exclusive events the sum of the probabilities of the events ia the union. Thus far our model of an experiment consists uf a procedure and observations. This Jonds to a set-theory representation with a + $), outeames (s that are vents (A that are seis of clanents). To complote mple space (aniversal anents of $), and assign 9 probahility P| With respect to our physical idea of the experiment, the probability of an event is the proportion of the time that event is observed in a large number of rans of the expotiment. ‘This is the relative frequency notion of probability. Mathematically, this is expressed in the following axioms the model, to every event, A, in the sample space. Definition 1.4 ‘Axioms of Probability A probability measure Pl.) isa function that maps cvents in the sample space to real nicnbers such that12 CHAPTER. EXPERIMENTS, MODELS, AND PROBABILITIES Axiom 1 For any event A, P|A] > 0. Axiom 2 P(S) Axiom 3 For any @untuble collection A of nnutually exolusive events P[ArU AgU---] = PAs) + P[Ag} + We will build our entire theory of probability on these three axioms. Axioms: 1 aud 2 simply establish a probability axa number bejween O aud 1. Axiom 3 states that the probability of the 1mion of nmtually exclusive events is the sim of the individual probabilities, We will use this axiom over and over in developing the theory of probability and in solving problems. Tn fact, it is really all we have to work with. Everything else follows from Axiom 3. To uso Axiom 3 to solve a practical problem, we will learn in Section 1.5 to analyze a complicated event as the events whose probabilities we can calenlate. Then, we will add the probabilities of the mntually exclusive events to find the probability af the complicated event we are interested in. A useful extension of Axiom 3 applies to the union of two umtually exclusive events. union of mutually exelnsi Theorem 1.2 For mutually exclusive events Ay and As, P[A, UAa] = P[A\] +P Aa] Although it may appear that Theorem 1.2 is a. trivial special case of Axiom 3, this is not so. In fact, a simple proof of ‘Theorem 1.2 may also vse Axiom 2! If you are curious, Problem 1.3.13 gives the first steps toward a proof. Ib is a simple matter tovextend Theorem 1.2 to any finite union of mutually exclusive sets, Theorem 1.3 PAS AU AYU U Ay and AA, = @ fori gj. then Pla Yea Tn Chapter conrespouds to the idea of relative fequeucy. ‘The correspondence relers toa se- 0, we show that the probability measure established by the axioms qnential experiment consisting of repetitions of the hasic experiment. We refer te cach repetition of the experiment as a trial. 4u these n trials, WVa(n)} is the number hat ovent A oecurs. The telative frequency of A is the fraction N4(n)/n. of times Theorem 10.7 proves that lin... Na(m)/n — Pll1.3 PROBABILITY AXIOMS — 13 Here we list some properties of probabilities that follow direetly from the three axioms, While we do not supply the proofs, we sug some of these theore st that students prove at least ns in order to gain experienee working with the axioms, Theorem 1.4 The probability measure Pl] satisfies (a) Piz} = 0. fy) Plas —PiAl (c) For ony A and B (not necessarily mutually exclusive), PIAUB) =P lal +P |B) —PlAng). (d) ACB, then P[Al < PB). Another éonsequence of te axionis can be expressed as the following, Uheorern: Theorem 1.5 The probability of an coent B= {51,2 the outcomes contained in the vent: )- EP Hs] = s18m} ts the surn of the probabilities of PIF Proof Hach outcome si is an event (a set) with the single element definition are mutually exclusive, 73 ean be expressed sets: Since outcomes hy rion of m mutmally exclusive sa} Ue Ue} (Ls) with (s)} {aj} = 2 fori £9. Applying Theorem 1.3 with By = (sip yields in SPR ay Comments on Notation We use the notation P[}| to indicate the probability of an event, ‘The expression in tho square brackets is an event. Within tho eontoxt of one experiment, PLA) ean be viewed as a function that transforms event ‘oa number between 0 and 114 CHAPTERL EXPERIMENTS, MODELS, AND PROBABILITIES Note that {4,} is the formal notation for a sot with the single element s,. For convenience, we will sometimes write P[s;) rather that the more complete P[{s:}] to denote the probability of this outcome. We will also abbreviate the uotation for the probability of the intersection of two: events, P(A 1B]. Sometimes we will write it as P[A, B] and sometimes as P[AB] Thus by definition. P|AN 8) = P_A,B) = PIAL). Equally Likely Outcomes A large mamber of experiments have a sample space $= (81.2...) im which our knowledge of the practical situation leads us to believe that uo one outcome is suy more likely than any other. Lu these exp rqually likely. In such a case, the axioms of probability imply that every outcome tats probability 1/r, iments we say that the 1 outcomes are Theorem 1.6 For an experiment equally likely, S=(stece.4n) in which caeh outcome 8, is sith sarapte sp Pil=i/n 1<8 0. In most oxperiments, PUL) = 0 means thet it is certain that JJ never occurs. lu Us ease, ib is illogieal to speak of the probability of A given that Bacents, Note that P[.| B] is a respectable. probability measure relative to @ sample-space that consists of all the outcomes in. B_ This means that, P[A\B] has properties corresponding to the three axioms of probability Theorem 1.7 A conititional probabitity measure PLA|B) hus the following propertios that corre- spond to the arions of probatality. Axion L: PIA)B] > Aviom 2: P|B|B] = 1 Aviom #: IfA= Al UAy U-+ with Ag A, = @ fori # j, then PAIS) = PAB] + P Aol) + You should be nble to prove these statements using Definition 1.5. Example 1.12 With respect to Example 1.11, consider the:a priori probability model Plrr]=0.01, Plra]=001, Plar|=001, P al = 0.97, (1.16) Find the probability of A — “second chip rejected” and 1 — “first chip rejected.” Also find the conditional probability that the second chip is a reject given that the first chip is a reject. We saw in Example 1.11 that A is the union of two mutually exclusive events (outcomes) rand ar, Therefore, the a priori probability that the second chip is rejected is PIA] = Pieri +P far] = 0.02 (ayy14 CONDITIONAL PROBABILITY — 17 This is also the a priori probability that the first chip is rejected P[B] =P [rr] +P fra] — 0.02. (ts) The conditional probability of the second chip being rejected given that the first chip is rejected is, by definition, the ratio of PAB] to P'BI, where, in this example P[AR] = P [both rejected] = P [rr] = 0.01 (1.19) Thus P(AB! |A\B] — = — 0.01/0.02 = 0.5. (1.20; PlAiB| = ra L ( 1.20) The information that the first chip is a reject drastically changes our state of Knowledge about the second chip. We started with near certainty, P[A) ~ 01.02, that the second chip would not fail and ended with complete uncertainty about the quality of the second chip, P/A|B] ~ 0.5 Example 1.13 Shuffle a deck of cards and observe the bottom card. What is the conditional probability that the bottom card ie the ace of clubs given that the bottom card is a black card? The sample space consists of the 52 cards that can appear on the bottom of the deck Let A denote the event that the bottom card is the ace of clubs. Since all cards are equally likely to be at the bottom, the probability that a particular card, such as the ace of clubs, is at the bottom is P1A] — 1/52, Let Li be the event that the bottom card is a black card. The event B occurs if the bottom card is one of the 26 clubs or spades, so that P[B] = 26/52. Given B, the conditional probability of A is P[AB] _ PA] PIB) PIB] 36752 The key step was observing that Av = A, because if the bottom card is the ace of clubs, then the bottom card must be a black card. Mathematically, this is an example of the fact that A 3 implies that AB — A PAIL) = (1.21) Example 1.14 Roll two fair four-sided dice, Let 1 and Xz denote the number of dots that appear on die 1 and die 2, respectively. Let A be the event X, > 2. What is P[AJ? Let 2 denote the event Vz > Xj. What is P[R]? What is P/A|B)? Xa We begin by observing that the sample space has 16 ele- ments corresponding to the four possible values of Xi and the same four values of Since the dice are fair, the outcomes are equally likely, each with probability 1/16. We draw the sample space as a set of black circles in a two-dimensional diagram, in which the axes represent the events y and Xx. Each outcome is a pair of val ues (X;,Xy). The rectangle represents A. It contains 12 ‘outcomes, each with probability 1/16PARTITIONS AND THE LAW OF TOTAL PROBABILITY — 19 Figure 11 Tn this esiinple of Theorem 1.8, the partition is BE = {B1, Ba, Bs, Bi} and C)~ ANB, for i= 1,...,4. Tt ahould be appazont that A= C\ UC: JCSUC, Example 1,16 Continuing Example 1.15, let B, = {outcomes with i heads). Each B, is an event containing one or more outcomes. For example, By — {ttth, thi, thtt, httt} contains four outcomes. The set B = | Bo, Pi, Bs, By, Pa} is a partition. Its members are mutually exclusive and collectively exhaustive. It is not a sample space because it lacks the finest-grain property. Learning that an experiment produces an event 13; tells you that one coin came up heads, but it doesn’t tell you which coin it was ‘The experiment in Example 1.15 and Example 1.16 refers to a “toy problem,” ou thut is easily visualized but isn’t something wo would do in the course of our professional work. Mathematically, however, it is equivalent to many real engi neering problans. For example, observe x pair of modems trausuitting four bits from one computer to another. For each bit, obsorye whether the receiving modem detects the bit correctly (¢) or makes an error (¢). Or test fonr integrated cirenits. For each one, observe whether the circiit is acceptable (a) or a reject (r). Tn all of these examples, the sample space contains 16 four-letter words formed with an alphabet containing two letters. If we are interested oly in the mumber of times one of the letters occurs, it fs sulficient to refer only Lo the partition B, which does not coutain all of the iufurutiow about the experiinent but does comin all of the information we need. The partition is simpler to deal with than the sample has fewer members (there are five events iu the partition aud 16 outcomes in the sample space). The simplification is much more significant when the complexity of the experiment is higher, For example, in testing 20 cireuits the ample spac. has thors. while the corresponding, partition has only 2 members space because18 CHAPTER 1 EXPERIMENTS, MODELS, AND PROBABILITIES To find P/ A], we add up the probabilities of outcomes in A, so P{A| = 12/16 = 3/4. The triangle represents B. It contains six outcomes. Therefore P[B] = 6/15 = 3/8 The event AB has three outcomes, ), (2, 4), (3,4), so PLAB} 3/16. From the definition of conditional probability. we write PAB) _ 1 PR, 2 PLA) = (1.22) We can also derive this fact from the diagram by restricting our attention to the six ‘outcomes in J? (the conditioning event) and noting that thrae of the six outcomes in B (one-half of the total) are also in A Quiz 1.4 Monitor three consecutive packets going through an Thternet. ronier. Classify each one as either video (v) or data (d), Your observation is a sequence of three letters (cach one is cither 1 or). For example, three video packets correspond to. nv The outcomes vev and ddd each have probability 0.2 whereus each of Uke other ontcomes rnd, vdeo, vdd, dv, dud, and dd has: probability 0.1. Connt che number of video packets Ny in the three pac also ealeulate the following probabilities: ‘ol have observed. Describe in words and ts (a) PNY = (c) Pifwed} |My (a) Pid} [Ny =2 (e) P[Ny = (f) PiNy SUN = J] 1.5 Partitions and the Law of Total Probability A partition divides the sample space into mutually exchisive sets, ‘The law of total probability expresses the probability of an event as the sun of the probabilities of outcomes that are in the separate sets of a partition Example 1.15 Flip four coins, a penny, 2 nickel, a dime, and a quarter. Examine the coins in order (penny, then nickel, then dime, then quarter) and observe whether each coin shows a head (it) or a tail (/). What is the sample space? How many elements are in the sample space? The sample space consists of 16 four-letter words, with each letter either fi or ', For example, the outcome H1hh refers to the penny and the nickel showing tails and the dime and quarter showing heads. There are 15 members of the sample space20 CHAPTER 1 EXPERIMENTS, MODELS, AND PROBABILITIES We observed in Section 1.3 that the entire theory of probability is based om a. inion of mutually exelnsive events. The following theorem shows haw to nse a partition to represent an event as a wnion of nmtnally exclusive events. Theorem 1.8 For a partition B= {By, Bo...) and any event A in tie sample space, tet Cy = ANB), For i ¢j, the events O, and C, are mutually exclusive and Ques. Figue Ll isa picture of Theoan L8. Example 1.17, In the coin-tossing experiment of Example 1.16, let A equal the set of outcomes with less than three heads: A= (rit, fette, tHuttetibit dtl, feast, etht, Atle tilife tthe Ant} (1.28) From Example 1.16, lot Fj — {outcomes with 7 heads}. Since (y.....Fs} is 3 par- tition, Theorem 1.8 states that (AO B)U(ANB)U(AN BULAN BS) UCAD Ba) (1.24) In this example, B,C A, for #=0.1,2. Therefore AND, = L, for i= 01,2. Also, for i= Band i = 4, AMR, = @ so that A = By UR, U Ra, 2 union of mutually exclusive sets. In words, this example states that the event "less than three heads" is the union of events “zero heads,” “one head,” and “two heads.” We advis» you to make sure you understand Theorem LS and Example 117 Many practical problems use the mathematical technique contained in the theorem Forexample, find the probability that there are Uuree or more bad eircults in u bates that comes from 9 fabrication machine. ‘The following theorem sefers to a pactition (23), 29,...,B,)} aud any event, A. It states chat we can ffnd the probability of A by adding the probabilities of the paris of A that are in the separate components of the event space Theorem 1.9 Por ony event A, and partition (2), Ba...., Bra} Pia = Sane) Proof The proof Follow directly from Theorem 1.8 and Thecren 1.3. Tr thin case, te mutually exchasive sets are C, = (9 Bi}PARTITIONS AND THE LAW OF TOTAL PROBABILITY 24 Theorem 1.9 is often used when the sample space can be written in the form of a table, In this table, the rows and colimns each represent a partition. ‘This method is shown in the following example. Example 1.18 A company has 2 made! of email use. It classifies all emails as either long (i), if they are over 10 MB in size, or brief (3). It also observes whether the email is just text (0), has attached images (i), or has an attached video (v), This model implies an experiment in which the procedure is to monitor an email and the obsarvation consists of the type of email, f, j, or 0 and the length, 7 or b. The sample space has six outcomes: S = {if bi. 1i.bi.tv,bo}, In this problem, each email is classifed in two ways: by length and by type. Using J. for the event that an email is long and B for the event that a email is brief, {.L, 13} is a partition. Similarly, the text (11), image (J), and video (V’) classification is a partition {71,7}. The sample space can be represented by a table in which the rows and columns are labeled by events and the intersection of each row and column event contains a single outcome. The corresponding table entry is the probability of that outcome. In this case, the table is Tor ov E\03 042 0.15 B\02 008 0 For example, from the table we can read that the probability of a brief image email is, Phi] = P[BI]— 0.08. Note that (7.1. V} isa partition corresponding to { By. By. Ba} in Theorem 1.9. Thus we can apply Theorem 1.9 to find the probability of a long email P= P [LT + P[Lf| + P|LV] = 0.57. (1.26) Law of Total Probability In many applications, we begin with information about conditional probabilities andl use the law of total probability to caleulate unconditional probabilities Theorem 1.10 Law of Total Probability Por @ partition {By Bz.-...Bya} with PB] > 0 for alli, Pid y P[A\B) PLB, Proof This follows {rom Thearem 1.9 aud the identity: PAB, direst consequence of the definition af conditional probability PIAIB|P[Bi), whieh isa The nsefulness of the result can be sven in the next example22 CHAPTER 1 EXPERIMENTS, MODELS, AND PROBABILITIES Example 1.19 ‘A company has three machines Hi, Lis, and By making 1 k() resistors. Resistors ‘within 50 0 of the nominal value are considerad acceptable. It has been observed that 80% of the resistors produced by ly and 90% of the resistors produced by [ty are acceptable. The percentage for machine Be is 60%. Each hour, machine #) produces 3000 resistors, By produces 4000 resistors, and s produces 3000 resistors. All of the resistors are mixed together at random in one bin and packed for shipment. What is ‘the probability that the company ships an acceptable resistor? Let A = {resistor is acceptable}, Using the resistor accuracy information to formulate a probability model, we write P[A|Bi|= 0.8, P[A|Bs]-09, P/A|Bi] — 06 (1.27) The production figures state that 3000 + 4000 + 3000 = 10,000 resistors per hour are produced. The fraction from machine 3) is P[B), = 3000/10,000 = 0.3. Similarly, P[Pa] — 04 and P|Bs] ~ 0.3. Now it is a simple matter to apply the law of total probability to find the acceptable probability for all resistors shipped by the company: P [A] =P [ALB PR] +P [Al Ba] P [Ba] + P [A|Ba) P [Ba (1.28) (0.8)(0.3) + (0.9) (0-4) + {0.6)(0.3) = 0.78. (1.29) For the whole factory, 76% of resistors are within 50 © of the nominal value. Bayes’ Theorem When we have adve ev iuférmation about PLA] aud aeed to ealeulate PLZ]A], wwo refer to tho following formula Theorem 1.11 Bayes’ theorem P[BiAl FUIBIP ea [4 Proof [Ag] 2)» (8) Pa” Pl PIB] (1.30) Bayes’ Uworeu is a siuple cousequeuce of une definition of eouditional probability Tt has a name berause it is extremely nnseful for making inferences ahout phenomena that cannot be observed directly. Sometimes these inferences are described as “rea- soning about causes when we observe effects.” For example, let: {FBy.-.++ Bn) be a partition that ineludes all possible states of something that interests us but thatPARTITIONS AND THE LAW OF TOTAL PROBABILITY — 23 we cannot observe directly (for example, the machine that made a particular resis tor). For each possible state, Bj, we know the prior probability P[Byj and P[A Bi) the probability that an event A oceurs (the resistor meets a quality criterion) if B; is the actual state. Now we observe the actual event (either the resistor passes or falls a test), and we ask about the thing we are interested in (the machines that might have produced the resistor), That is, we use Bayes’ theorem to find P[A,|A]. P/Bo|A],....P{Bml4]. Tn performing the calenlations, we nse che law of toval probability to calculate the denominator in Theorem LUI, Thus for state Li, Pl ALB) P LB SE, PAB) PLR] P(B|A) = (4.31) Example 1.20 In Example 1.19 about a shipment of resistors from the factory, we learned that: © The probability that 2 resistor is from machine By is P[Us| = 0.3. © The probability that a resistor is acceptable — i.e.. within 50 value — is P/] = 0.7 © Given that a resistor is from machine 135, the conditional probability that it is acceptable is P| Bs] — 0.6. What is the probability that an acceptable resistor comes from machine By? of the nominal Now we are given the event A that 2 resistor is within 50 9 of the nominal value, and we need to find PB;|A|. Using Bayes’ theorem, we have P [ALBy) PLB, P [AAA (1.32) (Ball P Al Since all of the quantities we need are given in the problem description, our answer is P [Bs |A] = (01.6)(0.8) (0.78) = 0.28. (1.33) Similarly we obtain P'By|A) = 0.31 and P[Ba| A] = 0.46, OF all resistors within 50 2 of the nominal value, only 23% come from machine Ly (even though this machine produces 30% of all resistors), Machine 2, produces 31% of the resistors that meet the 50 0 criterion and machine By produces 46% of them Quiz 15 Monitor customer hehayior in the Phonesmart store. Classify the hehavior as br ing (8) if a customer purchases a smartphone, Otherwise the bebuvior is uo par chase (N'). Classify the time @ customer is in the store as long (.) if the customer stays more than three minutes: otherwise classify the aimonnt of time as rapid (R). Based on experience with many customers, we use the probability model PIN] =0.7, PIL] =0.6, PINE] . Find the following probabilities: (a) PIBUL| (>) PIVUL (6) PINUB] (@) PILE24 CHAPTER 1 EXPERIMENTS, MODELS, AND PROBABILITIES 1.6 Independence ‘Two events sre inidependent if observinuy one event doas not change the probability of observing the other event Definition 1.6 ‘Two Independent Events Events A oni B ore independent if and only if P[AB] =P iA]? |B When events A sud B have nonzero probabilities, the following formulas are equiv- alent to the definition of independent events: P[A\B] =P | J), Pll = Py] (1.34) ‘To interpret independence, consider probability as a description of our knawledge Of the: result of the experiment. P{A] describes our prior knowledge (before the experiment is performed) that the onteome is incinded in event A. The fact thar the outcome is in B is partial information about the experiment, P|A\B] reflects our knowledge of A when we learn that B occurs. P[A\B) = P[A] states that Tearing that G occurs does not change our information about A, It is in this sense that the events are independent. Problem 1.6.11 asks the reader to prove that if A and B are independent, then Ani B® are also independent, ‘The logic behind this conelusion is that if learning that event B occurs docs not alter the probability of event A, then learning that B does not vecur alo should not alter the probability of A, Keep in mind that independent, and mutually exclusive are not syn- onyms. In some contexts these words can have similar meanings, but this is mot the case in probability. Mutually exclusive events A and B have no ottteomes in contmon and therefore P/AJ3] = 0. In most situations independent events: are not imttnally exclusive! Exceptions occur only when P[4] = 0 er P[B] = 0. When we have to calenlate probabilities, knowledge that events A and B are mutually exclusive is very lwipful. Axiom 3 enables us to add their probabilities te obtain the probability of the union, Knowledge that events C and D are independent is also very useful, Definition 1.6 enables us to multiply their probabilities w obtain the probability of the intersection. Example 1.21 Suppose that for the experiment monitoring three purchasing decisions in Example 1. each outcome (a sequence of three decisions, each either buy or not buy) is equally likely. Ave the events Bs that the second customer purchases a phone and .V, that the second customer does not purchase 2 phone independent? Aro the events I?) and Bs independent? Each element of the sample space $ — {Ubb, bbi..bnb, bin, nbb nbn. nb, nnn} has16 INDEPENDENCE — 25 probability 1/8. Each of the events By = {bbb,bom.nbb,nbn} and {tnb, bon, nb, nnn} 1.35) contains four outcomes, so P|B: P[Na] = 4/8. However, Be Na = @ and P[ByNo] = 0. That is, Be and No are mutually exclusive because the second cus- tomer cannot both purchase a phone and not purchase a phone. Since P| Jt»N»| 4 PAs] P/Na], Bo and Vo are not independent. Learning whether or not the event Ba (second customer buys a phone) occurs drastically affects our knowledge of whether or not the event Ny (second customer does not buy a phone) occurs. Each of the events By = {hnn.bub,hbn,bbb} and By = {hbn.bbb,nbn.nbb) has four outcomes, so P[B,| = P Bz] = 4/8 = 1/2. In this case, the intersection B, 1 By = {bbn, bbb} has probability P[B, Bo] = 2/8 = 1/4. Since P|B,B,, = P|B,|P[Bz|, events B, and Bs are independent. Learning whether or not the event By (second customer buys a phone) oceurs does not affect our knowledge of whether or not the event 2, (First customer buys a phone) occurs. Tn this example we have analyzed a probability model to determine whether two events are independent. In many practical applications we reason in the opposite direction. Our knowledge of an experiment leads us to assume thal certain pairs of events are independent. We thea use this knowledge to build a probability model for the experiment Example 1.22 Integrated circuits undergo two tests. A mechanical test determines whether pins have ‘the correct spacing, and an electrical test checks the relationship of outputs to inputs We assume that electrical failures and mechanical failures cccur independently. Our information about circuit production tells us that mechanical failures occur with prob- ability 0.05 and electrical failures occur with probability 0.2. What is the probability model of an experiment that consists of testing an integrated circuit and observing the results of the mechanical and electrical tests? To build the probability model, we note that the sample space contains four outcomes: $= {(mn, ea), (ma, er), (mr en), (mr, er)} (1.36) where mm denotes mechanical, « denotes electrical, a denotes accept, andy denotes reject. Let A/ and E denote the events that the mechanical and electrical tests are acceptable. Our prior information tells us that P[A/, = 0.05, and PIE" = 0.2. This implies P[Mf] ~ 0.05 and P/E} — 0.8, Using the independence assumption and Definition 1.6, we obtain the probabilities of the four outcomes: P[(ma.ea)] = P [ME] = P[M|P TR] = 0.95 x 0.8 = 0.76, (1.37) BP [(ma,er)] = PLM EY) =P (M|P |") = 0.95 x 0.2 = 0.19, (4.38) P [(mr.ca)] = P [M°X) = P (At*]P [z] = 0.05 0.8 = 0.04, 1.39) P [(nr,er)) =P [MSR =P [Mo PE) = 0.08 x 02=0.01 (1.40)26° © CHAPTER 1 EXPERIMENTS, MODELS, AND PROBABILITIES Thus far, we have considered independence as a property of a pair of events, (Often we consider larger sets of independent events. For more than two events to he tidependent, the probability model bas to meet a set of conditions, To define mtual independence, we begin with three sets. Defi Aa, ond J n 1.7-——Three Independent Events 4; are mutually independent if anil on (a) Ay ond Ay are independent (b) Ay and Ay ave independent, fe) Ay nnd Ay are independent (d) P[Ay 9 Ay Ag] = Ply] B[Aa] PLAg| ‘Phe final condition is a simple extension of Definition 1.6. ‘The following example shows why this condition is insufficient, to gnarantee that, “everything is independent of everything else,” the ides ub the heart of independence, Example 1.23 In an experiment with equiprobable outcomes, the partition is S — {1,2.3,4}, Pls] — 1/4 for all s ©'S. Are the events Ay = [1.3.4], Az — (2,3.4}, and Ay — & mutually independent? These three sets satisfy the final condition of Definition 1.7 because A, Ag /12 and Aa As} P [AIP [49] P [Ay] =0. (a) However, A, and Ay are not independent because, with all outcomes equiprobable, P [At 9 Ag] = P[{3,4}] = 1/2 4 P [Ai] P[ B/A x 3/4. (1.42) Hence the three events are not mutually independent. ‘The definition of un arbitrary uumber of mutually independent events is an extension of Definition 1.7 Defini Uf > 3. the even n 1.8 More than Two Independent Events A Aayecy An are mutually independent if arid only if (a) all collections of m —1 events chosen from Ay, Ag,...Aq are mtitually inde pendent, (0) PIAL Aas++ 7 An] = P’At! PLAS *--P[Aytr MATLAB 27 This definition and Example 1.23 show us that when v7 > 2 it is a complex matter to determine whether or not m events are mutually independent. On the other hand, if we know that 7 events are mutually independent, it is a simple matter to determine the probability of Ue intersection of any subset of the m events. Just multiply the probabilities of the events in the subser. Quiz 1.6 Monitor two consecutive packets going through a router. Classify each one as video (v) if it was seur Hom a Youtube server or as ordliziry data (d) otherwise, Your observation is a sequence of two letters (oither 1» or d). For exampl packels conespouds to ve, ‘The two packets are iudependent and the probability that any one of them is a video packet: is 0.S. Denote the identity of packet # by Ch UE packet is a video packet, then Ci =v; otherwise, C, = d. Count the number Ny of vido packets in the two packets yon have ohscrved. Determine whether the following pairs of ovents aro independent (a) {Ny = 2} and (Ny = 1} (b) {Ny 2 Up and {Cy =v} (©) {Cz =e} and {C1 =a} (4) {Cz =o) and {My is even} . two video 17) MATLAR ‘The MATLA® programniing environiitent can be used for studying probebility models by performing numerical calculations, sinulat- ing experiments. and drawing graphs. Sinmlations make extensive awe of the MatLab raudoi number generator rand. In addition to introducing aspects of probability theory, each chaptor of this book coneludes with a section that uses MATLAB to demonstrate with nimoriral examples the concepts presented in the chapter All of the MATLAB progr the companion website, On the other hand, the MVTLAR see are not essential 10 understanding the theory. You ean use this text to learn probability without using MATLAS us in this book éan be downloaded from ions Engineers studied and applied probability theory long before the invention of MAT- LAB. Nevertheless, MarLAs provides a convenient programming environ for solving probability problems and for building models of probabilistic systems. Ver sions of Maras. including a low-cost student edition, are available for most, com puter sysiems. Al the end of euch chapter, we include a MATLAB section (like tis ome) that introduces ways that MATLAn can be applied to the concepts and problems of the chapter. We assume you already have some fanuiliarity with the basies of running Mart.aa. Tf yon do not, we encourage you to investigate the built-in tutorial, books dedicated to MaTtAR, and various Web resonress,PROBLEMS — 29. Finally, we note that MATLAB’s random numbers are only seemingly unpredictable, In fact, MAaTtiAR maintains a seed valne that determines the subsequent “ numbers that will be rerurned. This seed is controlled by the rng fimetion; s=rng saves the current seed and mg(s) restores a previously suved seed. Initializing the random number generator with the same seed always generates the same sequence: clon Example 1.26 osm >> poerandi (90,1, 12) eo 78 80 80 72 02 68 66 77 78 80 88 >> engl); >> S0erandi (50,1,12) oo 76 80 8) 72 92 88 86 77 78 59 58 When you mu a simulation that uses rand, rng seed is initialized sequence of rand values when you are debugging your simulation, iL normally doesn't water bow the However, it can be instructive to se the same repratahle Quiz 17 The uumiber of ehicuctes sina tweet is equally likely to be any integer between 1 and 110. Sinmlate an experiment that generates 1040 tweets and counts the mimber of “loug” tweets that Lave over 120 characters, Repeat this experiment 5 times Problems Difficul Easy Moderate Difficult, Experts Only Jontinuing Quiz 1.1, write Ger- 1.1.36 Ricardo’s offers customers two kinds Jnnda’s entive menu in words (supply prices of pizea erust, Roman (J2) and Neapolitan if yon wish). (N). All pizzas: have cheese but not all pix zas have tomato sance, Roman pizzas ean have tomato sauce or they can he white (1); Neapolitan pizzas always have Lomato 1.1.20 For Gerlanda’s pizza in Quiz 1.1, an swe these questions: (a) Ate Nand M autually exelusive? (b) Are JV, 2, and Mf eéllvetively ehaus tive? (c) Are?’ and O mutually exclusive this condition in words. State (d) Does Gerkunda’s make Tuscan pizzas with mushrooms and onions! (e} Does Gerlanda's make Neapolitan plz zas that have neither mushrooms nor sauce. It is possible to order » Roman pi shrones (M) added. A Neapolitan pizza can contain mushrooms or onions (OQ) tbr both, iu addition to the tamato sauce and Draw a Vi the relationship amoug the ingredionts M, OU, and W i pizeria. with m ches nv ddiagvamn that shows the meni of Ricardo’ 1.2.16 A hypothetical wii transmission can take place at any of three speeds30 CHAPTER 1 EXPERIMENTS, MODELS, depending on the condition of the radio channel between a laptop and an access point, The speeds are liigh (fr) st 54 Mb/s, medium (m) at 11 Mb/s, and low (f) at 1 Mb/s. A user of the wi-8 connection can transmit a short signal corresponding to-a mouse clice (e), ort long signal eortespond- ing'to a tweet (4). Consider the of uouitoring wil the transmission speer! and the length. observation Is a two-lett ple, i high-spend. mote periment ignals and observing An word, for exam Tidy transmission (a) What ts the sample space of the exper imont? (b) Let Ay be the event “medium speed connection.” What axe the outcomes in Ay? (c) Tet Ag be the event ‘What are the outcomes in As? Smonse elie (a) Let Aa be the sh speed connection or low speed connection. What sre che outeomes in Ay? (e} Are Ay and Ay mutually exclu (F) Are Ay. Ag, and Ay collectively exhius: tive? 1.2.20 An integrated creuit factory Tins three machines X,Y, and Z ‘Test ono in- tegrated cinrait procheed hy each machine: ther a cirewit insaeceptatile (a) or it fil (J), Aa observation is test results entresponding to. the ciret from machines X.Y, and Z, respeetively For example, aaj is the observation that the circuits from X and ¥ pass the teat and the cixeuit from Z fails tho tost (a) What a sequence of three the clements of the sample ie: expurtenent? (by) What are the elements of the sets Zp = [eenit from Z falls} Xa = (cineuit from X is accoptable} (e}, Ate Zir and X4 mutually exclusive? (d) Ate Zp and X, tive? ollectively exhaus AND PROBABILITIES (o} What are the elements of the sets © = {more than one eirenit neceptable} , D = {at least two civenits fail) (©) Are © and D mutually exelusive? (g) Are C and P collectively exhaustive? 1.2.3 Shufile a deck of cards and turn over the first card, What is the sample space of this experiment? How many ontoomes are in the event that the first card is a heart? 1.2.4 Find out tho birthday (mouth and day but not year) of a ranciomly chosen pers son. What is the sample space of the ex periment? How iany outcomes are in the event that the person is bora ia duly? 1.25 sity Phe sample space of an experiment of all undergradnstes at Give four examples of partitions, 1.2.6 ‘Lhe samp! consists af the measnireel wsistimces of two resistors, Give four examples of partitions, 13.1% Find P[B] in each ease (4) Rvents A and Dare a partition and Pla] =6PiB (b) For events 4 and B, PAW) = Pla] and PIA 5] = (c} Forevents A and 8, PIAL 8) PIB sce of an experiment Pal- 1.3.26 You roll two fair sic-alded dice; one dle is ved, the other is white, Let be the event that the red dio rolls. Let W, be the event that the white die rolls (a) What is PUSH}? (ob) What is the P[Sy] that the sum of the two rolls is &? 13.3% You coll two fair six-sided dice. Vind the probability P[Js| that the atec- ute value of the differenve of the dice is 3 1.3.48 Indicate whether each staternent is true or false, (a) TPA] = 24’), then PLA] =1 (b) Por all A and 2, PIA] < PA) PB](c) ELA} < PIB), then PLAB) < P{B (@) PAB) = PIA), chen Pal > PIB 1.3.56 Computer programs are classified hy the le execution time gil of tho coureo code and by the Progimms with mone than lines in the souree code are big (B). Programs with < 190 fines are lithe (Z) ast programs () run in loss thom (1 see onds. Si w programs (IV) quire at least (.1 secouels, Motitor a program exceuted by a computer. Observe the length of che some code and the ran time, ‘The protr ability model for this experiment contain the following information: P[LF] = 003. P[BF] = 0.2, and PIBW] = 0.2. What is the sample space of the experiment? Caleu- late tie following probabilities: P|’), PB) and PIU Bl 13.6% ‘There are: two types of cellular phones, handheld phones (A) that vou mobile phones (Af) Oat are vehicles riounted Phen calls con be classified by the traveling speed of the user as fast (F) or slow (WP). Monitor a cellular phone call and observe the type of telephone and the speed of the user. ‘The probability model for this experiment, has the follow. ing information: PIF] = 0.5, P[MF] = 0.2 PLAT] =0.1. What is the sample space of the experiment? Pind the following pecba bilitios B/W), PIAL), aad D/A] 1.3.76 Shuffle-a docs of card the first card. What is the probability that the first card sa heart el tiien ower 1.3.86 You have a six-sided die that you roll once and observe the number of di facing upwards: What is the sample space’ What is the probability of each sample out come? What is the probability of 2, che event that the roll is even 1.3.96 A student's scexe oni a L0-poiit quiz is equally likely to be any integer between (andl 10. Whet is the probability of ana which requires the student to get a scare of 9 or moro? Whst is the probability the student gets an # by getting less than 4? PROBLEMS — I 13.10% Use ‘Theorem 1.4 to prove the fol. lowing. facts: (a) P[AUB] > PLA (b) PIAUR) > PIR] () PIAnB] < Pla] (a) Plana) < Pi 1.3.11 duction the union bouruk For any eallection Use Theorem 14 to prow of events Ary... Pai UAU--U A] < OPA ot 1.3.12 probability, prove Plo 13.13 of prob bility and the fiet that Plz] = 0, prove Theorem 1.3. Hint: Define A, = B, for f= hem and Ai =2 fori >m 1.3.14¢¢ For cach fact stated in Theo vent 14, determine which vf the three ioms of probahility are needed to prove the fact 1.4.16 Mobile telephones perform handagf fas they move from cell to cell During. a call, a telephone either performs zero hand off (Hy), one handoff (1), or more chan ue haudolf (Hz). In addition, each call cither long (L), if it lasts more than three minutes, or briet (22). The following table describes the probabilities of the possible types of calls Using only the three axions of Using, the three axior Hy Hy Hs Tfpod 01 02 B04 04 04 What is the probability that a brief enll will have no handoffs? (b) What is the probability that a eal with, one haneloff will be Tong? (¢) What is the probability that a fong eall will have one or more Landofis? 1.4.2 roll once the roll is You have @ six-sided die that you Let R, denote’ the eveut that Let Gl, denote the event, that32 CHAPTER the roll is greater than j. Let # denote the event that the roll of the die is evens simnbered. What is P{Ry|Gi], the conditional probability that 3 is rolled given that the cull is greater haw 1? (b) What is the conditional probability that 6 is rolled given that the roll is greater than 32 (¢} What is P\Gs{ 2}, the conditional prob: ability that the roll is greater than 9 Je voll is even? sven that t (a) Given that, the roll is greater than 3, what is the conditional protability hat the roll is even? 1.43 a shuffled deck of ree earides 2, ard. Tat Cy denote the event that eard 4 is picked Let £ denote the event that the card cho- seu isa even-numbered card (8) What is P[@:|F), the probability that the 21 picked given that sn even: You have aad 4. You draw one numbered card is chosen? (b) What is the conditional probability that an eveusmunbered card is picked! ssivon thut the 2 is picked? 1.4.48 Phonesmart is having a sale on Bax nanas. If you buy one Banana at full price you got a-secand at half price, When eon: ples come in to buy a pair of phones, sales of Apricots and Bananas are equally likely, Mowover, given that the first phone sold is. Banana, the second phone is twiee ws Whely t be a Risnane rather than an Apri cot. What is the probability that a couple buys-a pair of Bananas’? 1.4.58 Tho bvuie riiles of gonotios werd dis overed in mid-1800s by Mendel, who founel that cach chs as wh ‘actersstic of a pea plant. such ler the seeds were green oF yellow ch In his pas plants, Mendel foune is determined by tw genes, one from parent that yellow speds were a dominant trait over seeds, A yy pea with two yellow genes yellow seeds, a gy pea with two seces sive genes hae groen soede; a hybrid gy or yo EXPERIMENTS, MODELS, AND PROBABILITIES pea has yellow soeds, In one of Mendel’s periments, he started with a parental gen eration in which half de pew plants were y and half the planta were gg. The two groupa ‘wore exossbied so that each poa plant in the first generation was gy. Iu the second gon eration, each pea plant was equally. Wely to inherit ay or a g gene from each oneration parent, What i the probability PLY] that a randomly chosen pea plant in the second generation hs vellow seeds? 1.4.68 From Problem 14.5, whi conditional probability of yy. that a pea plant has two dominant gones given the event ¥ that it has yellow seeds? is the 1.4.78 You lave a Shlfled deck of three cards: 2, 4, three cans. and 4, and you deal out the Let B, denote the event that ih card dealt ig even mmbered (a) What is P1zy|2), the probability the second card is cven given that the Brst card is event (b) What is the conditional probability that the first two cards ane even given that the third card is even (@) Lot ©, represent the event that the éth card dealt is odd numbered. What is P[Fa\Q1), the conditional probability that: the second card is even given that the first eae a (a) Whar, that, the second card is odd given hat the first eaed Is odd” is the conditional probability 1.4.8) Deer ticks can carry both Lyme dis- ease and lun (AGE). In it was found that L6Y case, 10% had HGB, and that 10% of the ticks uit had either Lyme divease or TGR carried hath diseases an granulocytic elrlichiosis study of Licks in tho Midwest, earried Lyme dis- (a) What is the probability P[Li) that a tick carries both Lyme disease (L) aud HCE (H)? (bh) What is the conditional probabtity that o fick has HGE given that it hes Lyme d1.5.18 Given the model of handoffs and call Jongths ia Problem tl.1, (a) What i¢ the probability P{Mfu) thot phone makes no Tnandlof fe? (b) What i the probability a call is brief? (ec), What is the probability a call is long or there are at least two handofis? 1.8.2© For the telephone wage model of Pxample 1.78, let B,, denote the event tht «call is billed for mm minnses. To generate & phoue bill, observe the duration of the call in intoger minutes (rounding up). Charge for M minutes Mf = 1,2... iF the wxact duration 1 is AL — 1 < (-< Af A mote camplete probability model shows that for m= 1,2... the probability of each event B, P [Bal =al a)” * where a= 1 (0.57)" = 0.171. (a) Classify a call e& Jong, L, if the call lasts more than three minutes. What is PIF? (b) Whas is the probability that a esll sill be billed for mine minutes ar less? 1.5.3) Suppose a cellular equally likely ta make zero handoils (Hy) one handoff (Hh), or more than one hand- Off (7Ta). Also, 1 caller is either om foot (F) a vehicle (V). telephone is with probability 5/12 or (8) Given the preceding information, find three wavs to fill in the followlag probs ability vable: th Mh th F C (b) Suppose we also learn that 4/4 of all callers are on foot making calls with no Frandoffs and thot 1/6 of all eallens are cwhicle users making, call with » single Dandofl, Given these additional facts find all possible ways to Gill in the table of probabilitie PROBLEMS — 23 A and # to be st 1.6.16 Is it possible for dependent events vet satisfy: 1.6.26 Events A and B ace equiproba- ble, mutually exchiiye, and independens What is Pil] 1.6.36 Av Phonesmart store, each phone Id is twice as Likely to be an Apricot a a Banana. Also each phone sale is indepo dent of any other phone sale. Lf you monitor the sale of two phones, what is the probabil ity that the two phones sold are the same? Use a Venn diagram in whieh the areas are proportional to their prob- 4 and abili that are independent, es 10 illustrate two events 1.6.58 Ln au experiment, ond are iu tually exclusive eveuits with probabilities PA] = 1/4 and P(A] — 1/8. (4) Find PLAN Bi]. PAVE), PAB, and PLA WBF), (0) Are A ane B independent? 1.6.6 dependent ovents with probabilities P{C! = 5/8 and P{D] = 3/8. Th am excperitnent, @ and 1 are in. (a) Determine the probabilities PICA 7] DIE A, and PICT nD" (0) Ave C* and L° independent? 1.6.7 tually exclusive events with probabilities [AUB] =5/8 and PIA = 3/8. (a) Find P(B), PIA BY), ond PLA 89} (b) Are A amd JF independent? Tn an experiment, 4 and 7 are mu- 16.8% In an experiment, C. and 9 tare independent events with probabilities PIC D| = 1/3, and PC] = 1/2. (a) Bind PL), PICO DY), and PIC D4] (o) Find P[C UD) and BIC UD (c) Are C and D® independent? 1.6.98 In an experiment with equiproba ble ontcomes, the sample space is = {1.2.3.4} and Pis) = 1/4 for all s € $ Pind (luce events in $ that ave pairwise in dependent but are not independent. (Net34 CHAPTER Pairwise indepoudeat events meet the first three conditions of Deainition 1.7) 1.6.10) One ¢ (Continiation of Problem 1.5 f Mendes most significant result the conclusion thet genos determin. ing. diferent, characteristics are tramanit ted independently. In pea plants, Mendel fouud that round peas (r) are a dom nant trait over wrinkled peas (w). bonded trossbred a gromp of (¢95 ny) pens with a group of (wie.gg) eas. To this nottion rr denotes a pea with two “round” genes and ww denoics a pee with two “wein- Hed" genes. The first generation wore ¢i ther (ru, ug). (rw. oy). ver. ug), er (urs gy) plants with both hybrid shape and hybrid Breeding among the first gener~ ation yielded sccond-genoration plants: in which genes for cach characteristic equtally likely to be either dominant or re What is the probability P[Y] thas fa sccond-gencration pea plant has yellow seeds? What is the probability: P[R] that a second-generation plant fas ronud peas” Are i and Y independent events? How color EXPERIMENTS, MODELS, AND PROBABILITIES many visibly different kinds of pea plants woutk! Mendel observe in the second gene ation? What are the probabilities of each of these kinds? 16.116 For independent events A and B. prove that (a) Aud B® are independent (p) AS and B are Independent () AS and BE are independent 1.6.129 Use Venn diagram in which the event areas are proportional to their proba bilities co illustrate three events A, B. C that are independent. 1.6.13 Use a Veni diagram in whieh e nvens arein proportion to their probabilities to illustrate events 4, B, and C that are pairwise independent but not independent L719 Following Quiz 13, use Marian, bat not the to genermte fa vector T of 200 independent test seoras snch that al] scores between 1 and 100 are ‘equally likely ane randi fiction,2 Sequential Experiments Many applications of probability refer vo sequential experiments in which the pro: cedure cousists of many actions perforined in sequence, with am observation taken after each action. Bach nection in the procedure together with the outcome a rwedl as a separate experiment model. Tn analyzing sequential experiments we reicr to the separate experiments Gated with it can be vi ty ith its own probabi iu the sequence as subexperiments, 2.1 Tree Diagrams ‘Troe diagrams display the outcomes of the subexperiments in a Sequential experiment, ‘The labels of the branches are probabilities and conditional probabilities. The probability of an outeame of the entire experiment is the product of the probabilities of branchos going from the root of the tree to 4 leaf. Many experiments consist of a sequence of swherperiments. The procedure fol- lowed far each subexperiment. may depend on the results of the previous subexper- iments, We often Gud it useful to use a type of graph referred to asa free diagram to represent the sequence of subexperiments. To do so, we assemble the outeotn ofeach subexperiment into sets in a partition. Starting at the coot of the tree,! we represent each event in the partition of the first subexperiment asa branch and we label the branch with ¢ The events in the partition of the second snbexperiment appear as branches growing from every node at the eud of the fast subexperiment, ‘The labels of the branches probability of the event, Hach branch leads to a node. {Unik Miologieal trees, whic grow from the ground up. pro ait, Somo of them havo th abilities usually grow from Teft 10 roots on top ane! looves on the bet 3536 CHAPTER2 SEQUENTIAL EXPERIMENTS of the second subexperhment are the conditional probabilities of the events in the soeand subexperiment, We continue the procedure taking the remaining snbexper- iments in order. The nodes at the end of the final subexperiment are the leaves of the tree. Each leaf corresponds to an outcome of the entire sequential experiment. The probability of each outcome is the product of the probabilities and conditional probabilities on the pati from the root to the leaf We usually label each leaf with a name for the event and the probability of the event ‘This is a complicated description of a simple procedure as we see ia the following five examples Example 2.1 For the resistors of Example 1.19, we used A to denote the event that a randomly chosen resistor is “within 50 [2 of tho nominal value.” This could mean “acceptable.” We use the notation N’ ("not acceptable”) for the complement of A. The experiment of testing 2 resistor can be viewed as a two-step procedure. First we identify which machine (Fi, By, of Bs) produced the resistor. Second, we find out if the resistor is acceptable. Draw a tree for this sequential experiment. What is the probability of choosing a resistor from machine Ry that is not acceptable? This two-step procedure is shown in the oo tree on the left. To use the tree to find the probability of the event 2), a nonacceptable resistor from machine Ha, we start at the left and find that the iia probability of reaching Bp is P[By] 0.4. We then move to the right to Js \ and multiply P{By) by PLA’ Ry] = 0.1 to obtain P[BN] = (0.4)(0.1) = 0.04 sroperty of all tree diagrams that represent ties! om the branches leaviug any ode add We observe in this example a general sequential experiiuents, The probabi up to 1. This is a consequence of the law of total probahility and the property of conditional probabilities that corresponds to Axiom 3 (Theorem 1.7), Moreover, Aviom 2 implies that the probabilities of all of the leaves add up to 1 Example 2.2 Traffic engineers have coordinated the timing of two traffic lights to encourage a run of green lights. In particular, the timing was designed so that with probability .8 a driver will find the second light to have the same color as the first. Assuming the first light is equally likely to be red or green, what is the probability P[Gy! that the second light is green? Also, what is [IV], the probability that you wait for at least one of the first two lights? Lastly, what is P[G,|f%), the conditional probability of a green first light given a red second light?21 TREE DIAGRAMS = 37 The tree for the two-light experiment is shown on the left. The probability that the second light is green is Ds Gree va a P [Go] =P [GiGal + P |AiGol , =o4-01-05. (21) UR—aeKih 4 The event HT that you wait for at least cone light is the event that at least one light is ved W ={RG.UG, RU RRs} (22) The probability that you wait for at least one light is PW] =P [RL Gy] +P [Gi Aol +P [RR] =0.1 40.1 +04 = 0.6 (23) An alternative wey to the same answer is to observe that IV’ is also the complement of ‘the event that both lights are green. Thus, PW =P (G1G2)"| =1 —P GiGa| = 0.6. (2a) To find PIG, |Ry), we need PUy) = 1— P[Go| = 0.5, Since P[G Ry) = 0.1, the conditional probability that you have a green first light given 2 red second light is PIG Ry _ out M1 Ral 2 02. 2.5) PGR = “pray ~ 05 ‘ Example 2.3 Suppose you have two coins, one biased. one fair, but you don't know which coin is which. Coin 1 is bissed. It comes up heads with probability 3/4, while coin 2 comes up heads with probability 1/2. Suppose you pick a coin at random and flip it, Let C; denote the event that coin 7 is picked. Let Jd and 1 denote the possible outcomes of the flip. Given that the outcome of the flip is a head, what is P[(| W, the probability that you picked the biased coin? Given that the outcome is a tail, what is the probability P [C/T] that you picked the biased coin? First, we construct the sample tree on the Weft. To find the conditional probabilities, we see ws Pica a Pica = Pt Plow PG +P a From the leaf probabilities in the sample tree, y/8 8 PICU = Feri 521 TREE DIAGRAMS — 39 (a) Switch (b) Do Not Switch Figure 2.1 Troe Dingrams for Monty Hall Monty opens door 2 (event R,), you switch to door 3 and then Monty opens door 3 to roveal a goat (event C'). On the other hand, if the car is behind door 2, Monty raveals the goat behind door 3 and you switch to door 2.and win the car. Similarly, if the car is behind door 3, Monty reveals the goat behind door 2 and you switch to door 3 and win the car. For always switch, we see that PC]=P [FRAC] + P [Ay Ra = 2/8. (27) If you do not switch, the tree is shown in Figure 2.1(b). In this tree, when the car is behind door 1 (event H) and Monty opens door 2 (event Fz), you stay with door 1 and then Monty opens door J to reveal the car, On the other hand, if the car is behind. door 2, Monty will open door 3 to reveal the goat. Since your final choice was door 1, Monty opens door 1 to reveal the goat, For do not switch, PIC] = Pi RoC] + Pl BxC) = 1/8. Thus switching is better; if you don't switch, you win the car only if you initially guessed the location of the car correctly, an event that occurs with probability 1/2. If you switch, you win the ‘car when your initial guess was wrong, an event with probability 2/3. Note that the two trees look largely the same because the key step where you make 2 choice is somewhat hidden because it is implied by the first two branches followed in the tree Quiz 2.1 Inu cellular phone system, a mobile phone inst be paged to receive a plione cal However, paging attempts don’t always siiceead hecan: receive the paging signal clearly, Consequently, the systent will page a phone np to three times before giving wp. TF the results of all paging attempts are independent and a single paging attempt succeeds with probability 0.8, sketch a probability tree for this experimen and find the probability P[F] that the phone receives the paging signal clearly, the mobile phone may not38 CHAPTER2 SEQUENTIAL EXPERIMENTS Similarly, PIC, Lys yer PIC |r) = (26) ‘As we would expect, we are more likely to have chosen coin 1 when the first flip is heads, but we are more likely to have chosen coin 2 when the first flip is tails. ‘The next example is the "Monty Hall” game, a famous problem with a solition that many regard as counterintuitive. ‘Tree diagrams provide a. clear explanation of the answer. Example 2.4 Monty Hall tn the Menty Hall game, 2 new car is hidden behind one of three closed doors while a goat is hidden behind each of the other two doors. Your goal is to select the door that hides the car. You make 2 preliminary selection and then a final selaction, The game proceeds as follows: © You select a door. © The host, Monty Hall (who knows where the car is hidden), opens one of the two doors you didn't select to reveal a goat. © Monty then asks you if you would like to switch your selection to the ether unopened door. © After you make your choice (either staying with your original door, or switching, doors), Monty reveals the prize behind your chosen door. Te maximize your probability P[C] of winning the car, is switching to the other door either (a) a good idea, (5) a bad idea or (c) makes no difference? To solve this problem, we will consider the “switch” and “do not switch" policies separately. That is, we will construct two different tree diagrams: The first describes what happens if you switch doors while the second describes what happens if you do not switch First we describe what is the same no matter what policy you follow. Suppose the doors are numbered 1, 2, and 3. Let H, denote the event that the car is hidden behind door i. Also, let's assume you first choose door 1. (Whatever door you do choose, ‘that door can be labeled door 1 and it would not change your probability of winning, ) Now let A, denote the event that Monty opens door / that hides a goat. If the car is behind door 1 Monty can choose to open door 2 or door 3 because both hide goats. He chooses deor 2 or door 3 by flipping a fair coin, If the car is behind door 2, Monty opens door 3 and if the car is behind door 3, Monty opens door 2, Let C denote the event that you win the car and G the event that you win a goat. After Monty opens one of the doors, you decide whether to change your choice or stay with your choice of door 1. Finally, Monty opens the door of your final choice, either door I or the door you switched to. The tree diagram in Figure 2.1(3) applies to the situation in which you change your choice. From this tree we leam that when the car is behind door 1 (avent Hi) and40 CHAPTER2 SEQUENTIAL EXPERIMENTS 2.2. Counting Methods In all applications of probability theory it is important to under stand the sample space of an experiment. The methods in 1 section determine the number of outcomes in the sample space of a sequential experiment Understanding the sample space is a key step in formulating aud solving a prob- ability problem. ‘To begin, it is offen nseful ta know the number of ontcames in the sample space. This number ean be enormous as in the following simple example. Example 2.5: Choose 7 cards at random from a deck of 52 different cards. Display the cards in the order in which you choose them. How many different sequences of cards are possible? The procedure consists of sever: subexperiments. In each subexperiment, the obser vation is the identity of one card. The first subexperiment has 52 possible outcomes corresponding to the 52 cards that could he drawn. For each outcome of the first subex. periment, the second subexperiment has 51 possible outcomes corresponding to the 61 remaining cards. Therefore there aro 52 x51 outcomes of the first two subexperiments. The total number of outcomes of the seven subexperiments is D2 x OT Ks x Ab = OTA. 2TA TRA (28) Although many practical experiments arom determining, the Size of a Sample space all follow from the fundamental principle of connting in Theorem 2.1 complicated, the techniques for Theorem 2.1 An experiment consists of two subexperiments, Tf one 9 ahd the other suubexperiment has n outcomes, then the experiment has nk otteomes, aperiment has k outcornes Example 2.6 There are two subexperiments, The first subexperiment is “Flip a coin and observe either heads Hor tails T.”" The second subexperiment is “Roll a six-sided die and observe the number of spots.” It has six outcomes, 1,2,...,6, The experiment, “Flip a coin and roll a die,” has 2 x G = 12 outcomes: (7,1). (7,2), 8), (HA), (HD). (HB CD. 2) 03. GN 5), C6 Generally, if an experimeut B has |: subexperiments By,....£ where Ey has %,22 COUNTING METHODS — 41 outcomes, then E has []!_, n; outcomes. In Example 2 distinguishable objects. In general, an ordered sequence of i clistingnish: is called a k-permutation, We will use the notation (1), to denote the umnber of possible k-pemmmitations of n distinguishable objects. To find (7). suppose we have n distinguishable objects, and the experiment is to choose a sequence of f of these objects. There are 1 choices for the first object, —1 choices for the second object we chose an ordered sequence of seven objects out of a set of 52 able objects etc. Therefore, Ue total uuuber of possibilities is (m= n(n — Yom 2)o(M— Te (2.9) Multiplying the right side by (1 — &)!/(n — )! yields one next theorem, Theorem 2.2 The number of k-permutations dfn distinguishable objects is nla ’t 5 % nl (me = nlm — (n= 2)0o(n Gm Sampling without Replacement Sampling without replacement, corresponds to a sequential experiment in which the sample space of each subexperimont depends an the onteomes of previous subex- periments. Choosing objects randomly from a collection is called sampling. and the chosen objects are known as a sample. A k-permatation isa type of sample ob- tainer] hy specific rules for selecting ahjects from the collection, Tn particular, once we choose an object for a k-perumtation, we remov the object from the collection and we cannot choose ft again. Consequently, this procedure is called sampling without replacement. Different omtcomes in» K-pormutation ave distinguished by the order in which ob- ale. By contrast, in many practical problems, we are coucemed, only with the identity of the objects in a sample, not their order. For example, in many card games, only the set of eatds received by a player is of interest, ‘The cets arrive in a sau order in which they arrive is irrelevant. Example 2.7 Suppose there are four objects, A, B, C, and D, and we define an experiment in which the procedure is to choose two objects without replacement, arrange them in alphabetical order, and observe the result. In this case, to observe A) we could choose A first or PD first or both A and 7) simultaneously. The possible outcomes of the experiment are Als, AC. AD, BC, BD, and CD. Jn contrast to this example with six outcomes, the ext example shows that the A-pormutation corresponding to an exporimont in which the observation is the se- quence of two letters has 4/2! = 12 outcomes.42 CHAPTER2 SEQUENTIAL EXPERIMENTS Example 2.8 Suppose there are four objects, A, 8, (, and D, and we define an experiment in which the procedure is to choose tan objects without replacement and observe the result The 12 possible outcomes of the experiment are AB, AC’, AD, BA, BC, BD, CA. €B.CD, DA, DB, and DC. In Example 2.7, each onteome i a subset of the outcomes of a k-permutation. Each subset is Wo use the notation (") to denote the number of this number are (), we perform the following two snbexperiments to assemble a. k-permutation of n distingnishable objects alled 4 -combinution, We want te find the number of k-combinations combinations. The words for fi chuose k,” the snmuber of k-combinations of » objects. Lo find 1. Choose a k-combination out of the n objects, 2, Choose a K-permutation of the & objects in the k-combination Theorem 2.2 tells ns that the number of outcomes of the combined experiment is (n)e. The first subexperiment has (}) possible outcomes, the number we have to derive, By Theorem 2.2, the second experiment has (/), = k! possible outcomes. ince there are (nr), possible outcomes of he combined experiment (i) Reatranging the terms yields our next roantt 10) Theorem 2.3 The number of ways to choose k objects oul uf n distinguishable objects is ny (ms at hk) RE Ae We enoonnter (1!) in other mathematical sindies. Sometimes it i called a, binomial coufficient because it appears (as the eoefBcient of ay"*) in the expansion of the hinomial (2° + y)". Tn addition, we observe that (2) =(.24) en) The loge behind this identity is that choosing & out of n elements to be part of a subset is equivalent to choosing n —k elements to be excluded from the subser In most contexts, (‘t) iy undefitied except for integers n aud & with 0 < k 0, we define al () — Lag Ba Ode i 0 otherwise.22 COUNTING METHODS — 43 This definition captures the intuition that give, say, n = 33 objects, there are zoo ways of choosing k = —5 objects, zero ways of choosing k = 8.7 objects, and veto ways of chonsing k ~ 87 objects. Although this extended definition may seem. unnecessary, and perhaps even silly, it will make wnany formulas in later chapters more concise and easier for students to grasp. Example 2.9: © The number of combinations of seven cards chosen from a deck of 52 cards is (2) 52x BL x AO = 183,781.560. (2:12) 7 IxSK which is the number of 7-combinations of 52 objects. By contrast, we found in Example 2.5 674,274.182.400 7-permutations of 52 objects. (The ratio is T= 5040) © There are 11 players on a basketball team. The starting lineup consists of five players. There are (\") — 462 possible starting lineups. © There are ({:"’) = 10° ways of dividing 120 students enrolled in a probability course into two sections with 60 students in each section, A baseball team has 15 field players and ten pitchers. Each field player can take any of tha cight nonpitching positions. The starting lineup consists of one pitcher and eight field players. Therefore, the number of possible starting lineups is. N= (8) (2) = 61,350. For each choice of starting lineup, the manager must submit to the umpire a batting order for the 9 starters. The number of possible batting orders is NV x 9! = 23,351,328,000 since there ate V ways to choose the 9 starters, and for each choice of 9 starters, there are !! ~ 362.880 possible batting orders. Example 2.10 There are four queens in a deck of 52 cards. You are given seven cards at random from the deck, What is the probability thet you have no queens? Consider an experiment in which the procedure is to select seven cards at random from a set of 52 cards and the observation is to determine if there are one or more queens in the selection. The sample space contains if = (°2) possible combinations of seven cards, each with probability 1/H. There are xg ~ (°-") combinations with no queens. The probability of receiving no queens is the ratio of the number of outcomes with no queens to the number of outcomes in the sample space. Hyo/ Hf — 0.5504, Another way of analyzing this experiment is to consider it as a sequence of seven subexperiments. The first subexperiment consists of selecting a card at random and observing whether it is a queen, If it is 2 queen, an outcome with probability 4/52 (because there are 52 cutcomes in the sample space and four of them are in the event {queen}), stop looking for queens. Otherwise, with probability 48/52, select another card from the remaining 51 cards and observe whether it is 2 queen. This outcome of this subexperiment has probability 4/51. IF the second card is not a queen, an outcome44 CHAPTER2 SEQUENTIAL EXPERIMENTS with probability 47/51, continue until you select a queen or you have seven cards with no queen. Using (2; and Ny to indicate a “Queen” or “No queen” on subexperiment i ‘the tree for this experiment is oa. The probability of the event ‘Vz that no queen is received in your seven cards is the product of che probabilities of the branches leading to N; 18/52) x (7/ x (12/46) = 0.5504 (2.13) Sampling with Replacement Cousider selecting an object from a collection of objects, seplacine the selected abject, and repeating the proc object before malking anothe ss several times, ¢ ch time replacing the colected election. We refer to this situation as sampling with replacement, Bach selection is tho procedure of a subexperisnont, The suibexperi- ments are referred to as independent trials, In this section we consider the mumber of possible outecmes that result fom sampling with rephicement. In Lhe next soe= tion we dotive probability models for for experiments that specify sampling with replacement Example 2.11 There are four queens in a deck of 52 cards. You are given seven cards at random from the deck. After receiving each card you return it to the deck and receive another card at random. Observe whether you have not received any queens among the seven cards you were given, What is the probability that you have received no queens? outcomes, There are 48 The sample space contains outcomes with no queens, The ratio is (48/52) — 0.5710, the probability of receiving no queens. If this experiment is considered as a sequence of seven subexperiments, the tree looks the same as the tree in Example 2.10, except that all the horizontal branches have probability 48/5 and all the diagonal branches have probability 4/52 Example 2.12 A laptop computer has USB slots A and 12. Each slot can be used for connecting a memory card (1m), a camera {c) ora printer (p). It is possible to connect two memory cards, two cameras, or two printers to the laptop. How many ways can we use the two USB slots? This example corresponds to sampling twa times with replacement from the set {m. rp} Let cy denote the outcome that device type wis used in slot A and device type y is used in slot B. The possible outcomes are S = {mirn.me,mp, em. ee, ep. pir, pes pp} « The sample space S contains nine outcomes22 COUNTING METHODS = 45 ‘The fact that Example 2.12 has nine possible onteomes should not be surprising, Sinee we were sampling with replacement, there were always three possible out comes for eacl of the subexperiments to attach a device toa USB slot, Hence, by the fundamental theorem of counting, Example 2.12 must have 3 x 3 — 9 possible outcomes, Tn Example 2.12. me and em are distinct outcomes. This result gene urally whea we want to choose with: rable objects. The experiment consists of a sequence of n in the sample space of cach subexperi- th replncement a sample of n objects placement a sauple of m objects out of a collection of yn distingui identical subexperiments with m outcome ment. Fence there are a" ways to choose Theorem 2.4 Given m distinguishable objects, there are m” ways to choose with replacement an omtered sample of n objects Example 2.13 There are 219 ~ 11124 binary sequences of length 10. Example 2.14 The letters A through Z can produce 26? = 456,976 four-letter words Sampling with replacement corresponds to performing n repetitions of an iden. perhnent. Usiug ey to denote the outcome of the éth subexperiment, the result for n repetitions of the subexperiment is « sequence ay... tical sub Example 2.15 A chip fabrication facility produces microprocessors, Each microprocessor is tested to determine whether it runs reliably at an acceptable clock speed. A subexperiment to ‘test a micropracessor has sample space Sas = {0,1} to indicate whether the test was a failure (0) or a success (1), For test 1, we record 1, — 0 oF xr, = | to indicate the result. In testing four microprocessors, the observation sequence, sri.raira:ta, is one of 16 possible outcomes: g— $0000, 0001, 9010, O11, 100, OLOL, 0140, DLL 1000, 1001, 1010, 1011, 1100, 1101, 1110, 111 Note that we can thinls of the observation sequence .r:,,...2 as the result of ompling with replacement. n leutical ies from a sample space Syjy. For sequences of ubexperiments, we can express Theorem 2.4 a846 CHAPTER2 SEQUENTIAL EXPERIMENTS Theorem 2.5 Porn repetitions of u subeeperiment with sample space Saur = {80 i}, the sample space S of the sequential experiment has m'* eutcanes Example 2.16 There are ten students in a probability class. Each earns a grades = Say = {A202} We use the notation xr, to denote the grade of the ith student. For example, the grades for the class could be nig2-+- tw = OBBACF BACF (2.44) The sample space Sof possible sequences contains 4!" = 1,048,576 outcomes. ly Example 2.12 and Example 2.16, sepeuting a subexperimont 1 tines and record= ing the observation consists of constricting a word with n letters. In genet repetitions of bhe same subexperiment consists of choosing yymbols from the alpha het {50 1}. Tn Example 2.15, 2 — 2 and we have a hmary alphabet with syuubols so = Oand #1 = L. A more challenging problem than finding the number of posstble combinations: of m objects sampled with replacement from a set of n objects is to calewlate the number of observation sequences such that each object appears. a specified mauber of times, We start with the case in which each subexperiment is a trial with sample space 8 1 i — (0.1} indicating failuse or success, Example 2.17 For five subexperiments with sample space Say = (0. 1}, what is the number of obser- vation sequences in which () appears ry — 2 times and | appeats ny — 8 times? The 10 fivetetter words with 0 appearing twice and 1 appearing three times are {00111 01011, 01201, 04110, LOOLL, LOLOL, 10110, 11001, 11010, LLL00} Example 217 determines the number of ouleomes iat the sample space of ant experiment with five suibexperiments by listing all of the outcomes. Even in this simple exauple it is not a saple matter to deteriaine all of tle outeomes, aud it most pract the sample the other hand, the coumting methods covered in this chapter provide formas for quickly calculating the munber of outcomes in a sample space Tn Fxample 2.17 each ontcome corresponds to the position of three ones in a five-lettor binary word. That is, each outcome is completely specified by choosing three positions that contain 1. There are (3) = 10 ways to choose three positions al applications of probability there are far more then ten ontcomes in pace of am experiaent and listing (hem all is out of the question. Ou ina word. More generally, for Iongth n binary words with nj 1's, we choose (7 slots to hold a L22 COUNTING METHODS = 47 Theorem 2.6 The number of observulion sequences for n subeiperiments with sunple space S = {0.1) uxth appearing no mes and 1 appearing ny =n —no times is (,") ‘Theorem 2.6 can be generalized to subexperiments with 1m > 2 elements in the sample space. Porn trials of a subexperinent with saniple space Sms = {o0..8m1}, we want to Bind the number of outcomes in which s9 appears ny times, 9) appear ny times, and 50 on, Of course, there wre no stich outcomes tmles No ~---+ rma =. The notation for the number of outcomes is It is referred to as the multinomial coefficient. ‘To derive a formula for the muulti- nomial coefficient, we gonoralize the logic used in deriving the formula for the bi- nomial coeflicient. With n subexperiments, representing the observation sequence by mslots; we first choose vo positions in the observation sequence bo hold sq, then ny positions to hold s1, and so on. ‘The details can be found in the proof of the Dollowing theorem: Theorem 2.7 Por w repetitions of a sutexperiment with sample space S = {8,-.-.4ma}s the nuraber of Length r= ny +++ My—1 observation sequences with ; appearing ', bimwes ts _,)> Start with n empty slots and perform the following sequence nt ) nolny!-+ Maal Proof Let = ( of subexpesiments: Subexperiment Procedure 0 Tabel nin SWois W A 1 Label 1 slots os sa m1 Tabel the remaining tun slots ak fina form sabexperinent 0. After ry slots have been labeled, there ent 1. After subexperiment j— A, yy Fe) ny slots have alveady boon filled, leaving ("~' There are (2°) wavs to are ("5") ways to perform subexperia ) ways to perform subexperiment j From the fundamental counting principle w-Qesler Cra) al (n= ne)! (119 == ya)! = Gn najinal (n= na — najind "Ga aw M1} at48 CHAPTER2 SEQUENTIAL EXPERIMENTS Canceling the commen factors, wo obtain the formula of the theorens, Note that a binomial coefficient is the special case of the multinomial eoetficient for wu alphabet with ri =2 symbols. du particular, for w= 19 +101, Ord = CD eit) Lastly, in the same way that we extended the definition of the binomial eoeffi- cient, we will employ an extended definition for the multinomial coefficient Definition 2.2 Multinomial Coeff For an integer n> 0, we define ! Tyee iyi _ J nolaatss Mma! me € {0,1 n}, m— 0 otherwise. Example 2.18 In Example 2.16, the professor uses a curve in determining student grades, When there are ten students in a probabilit class, the professor always issues two grades of A, three grades of B, three grades of C and two grades of F. How many different ways can the professor assign grades to the ten students? In Example 2.16, we determine that with four possible grades there are4!! = 1,048 ways of assigning gracles to ten students. However, now we are |imited to choosing ily =2 students to receive an A, 111 = 3 students to receive @ 13, 12 = 3 students to receive aC and ng — 4 students to receive an F. The number of ways that fit the curve is the multinomial coefficient Cronin) = (33.82) = a 00. f Quiz 2.2 Consider a binary code with 4 bits (0) or 1) in each code word. An example of a code word is 0110, (a) Mow many different code words are there? (b) How many code words have exactly two zeroes? (c) How many code words hegin with a zero? (a) Tra consiant-ratio binary code, each code word has N bits. In every word, AF of the NV bits ave L aud the other VW — M1 bits are 0. How ana words are in the code with V = Sand M = 32 y dilferent code23° INDEPENDENT TRIALS 49 2.3 Independent Trials Independent trials are identival subexperiments in a yequential ox. periment. The probability models of all the snbexperiments are identical and independent of the outcomes of previous subexperi- ments. Sampling, with replacement is one category of experiments with independent trials We now apply thé counting methods of Section 22 to derive probability models for experiments consisting of independent repetitions of a subexperiment. We start with a simple sui meni in sehich there are two ontcomes: a sucess (1) ocenrs with probahility p; otherwise, a failure (0) oceurs with probability 1 —p. ‘The results of all crials- of the subexperiment are mutually independent. An onteome of the complete experiment is @ sequence of stuccesses and failures denoted by a sequence of oues a . For example, 10101... is an alternating sequence of successes and faihires. Let Bran; denoie the event mp failures and ny successes in ‘To find P [Ey ,); we first consider, an example m= my forty trial Example 2.19 What is the probability P[2,s] of two failures and three successes in five independent trials with success probability p. To find P[#».], we observe that the outcomes with three successes in five trials are 11100, 11010, LLOOL, 10110, 10101, 10011, OLLL0, 01101, 01011, and OLLI, We note that the probability of each outcome is a product of five probabilities, each related to one subexperiment. In outcomes with three successes, three of the probabilities are p and the other two are 1 ~ p. Therefore each outcome with three successes hes probability (1 — p)?p* - From Theorem 2.6, we know that the number of such sequences is (3). To find P|[Fa.j], we add up the probabilities associated with the 10 outcomes with 3 successes, yielding P [Fal (2.18) Tn general. for — ng +m independent trials we observe tha @ Rach ontcome with np failures and 1 sticcesses has probability (1 — p)"®p! © Chere ace ( ) outcomes that have 1p failures aud vy successes, Therefore the prok ability of m) successes in nt independent trials is the sum of (!!) terms, each with probability (1— p)"p"! = (1 — py" pt Theorem 2.8: The probability of vy failures anil ry, successes int = rq + my independent trials ts P [Ensue (*)a = pyre" = ("Ja —pyrepne23 INDEPENDENT TRIALS SL An ontcome of the experiment consists of a sequence of n siubexperiment out comes: In the probability tree of the experiment, each node has m branches and braneh + has probability p,. The probability of an oatcome of the sequential expert ment is just the product of the rt branch probabilities on a path from the root of the tree to the leaf representing the outcome. For example, with n — 5, the ontcome sosasasesy Occurs with probability popapapemi. We want to fiud the probability of the event Birocomans = {80 OCS Hy TIMES, S4y 4 DECUES Moy a times} Note that the notation 2 i pliew that the exporinient cousists of a « quence of m= ny fe++b ty trials, ‘Ta ealenlate PLE i]. We observe that the probahility of the outcome Soo ay 1 dmb (2.28) duchy POPE PL 24) Next, we observe that any other experimental outcome that is a reordering of the preceding sequence has the same probability because on each path throwgh the tee to such an outcome there are 7; occurrences of sj. As a result BP Bory optgony | = MN DE + Bh where M, the mumber of sin outcomes, is the multinomial coefficient (,.”,.,) of Definition 2.2. Applying Theorem 2.7, we have the following theorem ‘Theorem 2.9% A subemperiment has sample space Seay — {50)--.)8m-a} with Pls] = pa No + ss + th —1 tdependent trials, the probability of nm; occurences of s;, #=0.L.:,m—1) t8 n \ P [Broan ( ni gf LE jemand = ai stg a) 20 Pa Example 2.22 A packet processed by an Internet router carries either audio information with probability 7/10, video, with probability 2/10, or text with probability 1/10. Let Ey,y,. denote the event that the router processes a audio packets, v video packets, and ¢ text packets in ‘a sequence of 100 packets. In this case, cal) Gi) a) (ca) ae PlE qos =(50 CHAPTER2 SEQUENTIAL EXPERIMENTS ‘The second forma in this theorem is the result of unultiplying the probability of ng failures inn trials by the mumber of outcomes with ny failures, Example 2.20 In Example 1.19, we found that a randomly tested resistor was acceptable with proba- bility P[4) = 0.78. If we randomly test 100 resistors, what is the probability of 7), the event that resistors test acceptable? Testing each resistor is an independent trial with a success occurring when a resistor is acceptable. Thus for 0 0.07 2.5 MATLAn ‘Two or three lines of MATLAB code are sufficient to simulate an arbitsary number of sequential trials, ‘We recall from Section 1.7 that rand (1m)

> X-rend(100,60)<0.6;| The MATLAB code for this task appears on the left. The >> Yesum(x.1) 100 x 60 matric X has j, jth element LG, 5)=0 (tails) or XC, j)=1 (heads) to indicate the result of fp i of subexperiment j. Since Y sums X across the first dimension, (3) is the number of heads in the jth subexperiment, Each Y¢3) is between () and 100 and generally in the neighborhood of 50. The output of a sample run is shown in Figure 2.4 Example 2.26 Simulate the testing of 100 microprocessors as described in Example 2.23. Your output should be a 4 1 vector X such that X, is the number of grade i microprocessors, Yebiptest.a The first line generates a row vector of random grades Geceil(4erand(1,100));| for 1100 microprocessors, The possible test scores are in the vector T. Lastly, K+hist(G,T) returns a histogram Xehist (G,7) vector X such that XCj) counts the number of elements GG) that equal TC) Note that "help hist" will show the variety of ways that the hist function can be called. Morever, X-hist(G,T) does more than just count the number of elements of @ that equal each element of T. In particular, hist (G,T) creates bins centered around each T(j) and counts the number of elements of @ that fall into each bin Note that in Marian all variables are assumed to be matrices. Th writing MArLaa code, X muy be an ye x 1m matzix, an n> L column vector, a Lx m row vootor, or a 1 T scalar. Tn Maran, we write (4,5) to index tho i, jth clement By contrast, in this text, we vary the notation depending on whether we have aPROBLEMS 57 sealar X, or a vector or matric X. In addition, we use X,,) to denote the i, jth element variable, Quiz 2.5 Thus, X and X (ina MATLAB code fragment) may botl retor to the same The flip of a thick coin yields heads with probability 0.4, tails with probability 0.5, or lands on ity edge with probability 0.1 should be a 3x1 vector X such that Xi, 3 of beads, tails, aud edge. Problems Difficulty Easy 2.L1e Suppose sou Hip a coin twice, On any Hip, the coin comes up heads with prot ability 1/4, Use fy and 7) to denote the result of Hip 4 (a) What is the probability, P[/M| 172), that the first flip is heads given that the sec ‘ond flip is heads? (b) What is the probability that the first Hip ia howds ned the seemd lip is tala? 2.1.20 For Feample 22. suppase PIGi] = 1/2, Pie 3/4, and P\Ga) Ra] = 1/8 Find PIG: 1}, andl LC Ga). 2.1.3 At the end of regulation time, ie ‘kethall team ix trailing hy ane point and a player Yoes to the line for two free tarows. AC the player wakes exactly one free throw the game goes we. The prcba bility that the frst fee throw is good is 1/2. However, if the first attempt is good the player relaxes nd the xeccnd attempt ts good with probability player misses the Hist attempt, the added pressure reduces the success probability: to 1/1. What isthe probability that the game goes lilo, overtinne? /4. However. if the 2.1.de You have two biased coins. Coin A comes up tieads with probability L/4, Coin B comes up heads with probability 3/4 Tawever, you are not aure whieh is which 0 vou chose coin ranidomly sind you Bip it. IF the flip is heads, yom gmess that the fipped coin is B; otherwise, vou guess that the flipped coln is A. What is the probabil- ity P{C] that your gues is comect? Sintulate 100 thick coin fips. Your output , and X3 are the mumiber of occurrences Moderate 2.1.59 Suppose that for the general popula tion, 1 in 5000 people carries the human im mumodeficiency viras (HEY). «A test for the presence of HIV yields either a positive (+) be negative (-} response, Suppose the test gives the correct ansiwer 99% of the time What is P[=|/7], the conditional probabil ity that @ person tests negetive given that the person docs have the HIV virus? What is PLH|1], the conditional probability that arrandomly chosen person has the ITV virus, given that the person tests positive? Difficult Experts Only 2.1.6 A machine produces photo detecta: in pairs. ‘Teste thay the Hrst photo dutector is acceptable with probability 3/5, When the Stat photo detector is accept ble, the seeond photo detector 1s accept- able with probability 4/2. Lf the lirst photo Jotector ie defective, the second photo tector is weceptable with probability 2 (@) Pind the probability that exactly one phote detector La pair is acceptable. (0) Pind the probability thet beth photo detectors in a pair are defective 2.1.78 You have two biased coins, Co comes up heads with probability 1/4. C DB comes up beads with probability Tow a0 you flip each 3/4 + which ig which ence, choosing the frst coin randomly, Use A and Ti to dente the result of flip’, Let A, be the event that coin A was flipped first, Let By be the event that coin 2 was lipped first, What i P[Ay As)?58 CHAPTER2 Ase Hi and Hy independent” Explain your answer 2.1.8 A particular birth defeet of the heart a newborn infant gill aye the de: fact D with probability PID) = 10~*. in the gencval rxam ofa newbom, a particular Tear arrhythmia A occurs with probability 0.99 nfania with the defict, Tosever the arthythmia also appears with probabil: ily O.L in infants without the defect. When the arrhythunia ts present, # lab test for defect is perforined. The result of the test ia either positive (event T~) or negn tive (event T~}. Ina newhorm with the de- foet, the lab test is positive with probabil ity p = 0.8901 Th newhorn without the defect, the Tab test is neyative with probability p = 0.900 If the arrhythmia is present and the test is positive, (lien heart surgery (event 11) is perfonned, (a) Given the arytunia 4 is preseat, what is the probability the infant has ue de- fioct D? ipendent. from test 10 test (b) Given chat an infamt bas the defect; what is the probability PLD} that Ihoart sungery iv performed? Given that the Infant does not have tlhe defect, what is the probability q=PIT P| thatan unnecessary heart surgery is performed? (d) Find the probability P/M] ¢hot an in- fant has heart surgery. performed for the arrythmia (c) Given that heart surgery is performed. what is the probability thut the new: the defect? born does not hav 2.1.98 Suppore Dagwood (Blondie’s huss band) wants to eat a sane goonadict. Dagwood decides to let the frp Of acoin dewermine whether he eats. Using ich but needs ta fan unbiased coin, Dagwood will postpone the diet (and go dinectly to the rafrigerator) if either (a) he flips heads on his first fip or (b) he fips tails on the first flip but then proceeds to get two heads out of the next three flips. Note that the first dip is no! SEQUENTIAL EXPERIMENTS counted in the attempt to win two of thee and that Dagwood never performs any un necessary flips. Tet 72, be the event that Dagwood flips heads on try j, Let T, be the event that tails occurs on Hip 4 (0) Draw the tree for this experiment, L bel the probabilities of a (b) What are Pixs) aud PZ)? (6) Lot 2 be the eveut that Dogwood miust diet, What is P[D]? What ie Pi7T,)D)? (a) Are Hy and Hz independent events? 21.108 The quality of each pair of photo detectors prodnced by the machine in Prob- Jem 2.1.6 is independent of the quality of very other pair of detectors, (a) What is the probability of finding no good detestorn ina collection of'n pairs produced hy the machine? (b) How many pairs of detectors must the machine produce to reach a probability of 0,99 that there will be at Teast one aeceptable photo detector? 21.118 In Steven Steogata’s New York Times blog http: //opizioaator bloge nytimes .con/2010/04/25/chances~are/ Tref=opinion, the following problem was posed to highlight the eonfusing character of conditional probabilities Before quing on vacation for a week, you ask your apacsy frien ta water your viling plant. Without waier, the plant has a 90 percent chance of dying. ven with proper watering, it has « 20 percent chance of dir ing. And He probability that gor friend twill forget to. water if a8 30 percent. Ce) What's the chance iat your plant will sur- vive the werk? (b) Jf at's dead when you return, what's the chance that your friend forgot to wader it? (¢) If yon friend forgot to water i, what's the chance Wil be dead uden you return Solve parts (a), (b) and (c) of this problem, 21.128 Each time @ fisherman casts his line, a fish is caught with probability p, in- dependent. of whether @ fish. Is caught on y other cast of the line, ‘The fisherman will fich all day until a fish is caught audthen he will quit and go home, Let @; de- note: the event that on cast ¥ the fisherman catches a fish, Draw the tree for this exper= iment and find PIC)), P[C,], aud P|Cy) as functions of p. 2.2.16 On each tun of the knob, a gum Dall machine is equally likely to dispense red, yellow, green or blue gumball, indepeu- dent from turn to turn. After eight tums, whats ix the: probability Pls YG] that you have received 2 red, 2 yellow, 2 green and 2 blue gumba 2.2.26 A Stachnrst candy package contains }2 individual candy pieces. Fach piece is equally likely Lo be berry, orange, lemon, or cheuy, independent of all other picces. (a) What is the probability thet a Star- burst paekage has only berry or cherry piven ond znre orange or lemon pinces? (b) What is the probability that a Sta Thnirst package Inns no: cherry pieces? (¢) What is the probability P[F)] that all twelve pieces of your Starburst are the same Havor? 2.2.36 Your Starburst candy hes 12 pieces, three pieces of each of four flavors: herny orange. and cherry, arranged in a vandom order in the pack. You draw the rst threo pieces from tbe pack. (a) What is the sante Flavor Temon. wrobability they are all the (b) What is the probability they axe all di ferent flavors? 2.2.88 Your Starburst exndy hus 12 plore three pieces of each of four flavors: berry Jemou, orange, and cherry, arranged ia 4 random order in the pac. You draw the first fonrr pioous From the pack (is) What is the probability P[FA] they ane all che same flavor? What is the probability P[Fy] they are all different Biers? () (ec) What is the probability P [Fy] that your Starburst hes exactly tivo pieves of each of two diffrent flavors? PROBLEMS 59 2.2.5% In a gamo of rummy, you are dealt a seven-card hand. (a) What isthe probability P[A hand has only ted eards | that your (b) What is the peobability PUA] that your hand has only face cards? (c) What is the probability Pld; F] that your rnd hus ony red face cards? (The face cards ace jack, queen, snd king.) 2.2.6 In a game of poker, you are dealt a five-eawd hand. (a) What is the probability P[Rs] that your hand has only red cast? (b} What isthe how akind? probability of a “full with three-ofa-kind andl te 2.2.7@ Consider a inary code with 5 bits (0-0 2) lu each code word, Au example of a code word is 01010, How many differ ont cade words are there? How many code words hava exaetly three 0's? 2.2.8% Consider a language containing four letters: A, B.C, D. How many threeletter swords can you form in this language? Tow many four-letter words can you form ifeach letter 2.2.96 On an American League baschalt team with 15 field players and 10 pitchers the manager selects a starting Eneup with S field players, 1 pitebor, and 1 designated hitter. ‘The lineup specifies the players for these positions aud the pesitions In a be ing order for the 8 ficld players and lesig- rusted hitter. If the designated hitter must Tse chosen among all Hie field players, how many possible starting lineups are there? .ppears only once in cach word? 2.2.10 Suppose that in Problem 2.2.9, the designated hitter ean be chosen from among, all the players. Tow many possible starting lineups ace there? 2.2.11 At a casino, the only gan berless roulette. On w spin of the wheel the ball lands in @ space with color red (7), -n (g), or block (4), ‘The wheel has 19 ved spaces, 1) green spaces and 2 black spaces.60 CHAPTER (a) 1a 40 spins of the whedl, find the prob- ability of the event A= {19 reds. 19 greens, and 2 blacks} (8) Ta 40 spins of th ability of Gis wheel, find the probs 19 greens} (c) The only bets allowed are red and green. Given that you randomly choose to bet red or green, what is the proba bility p that your bet is a winner? 2.2.12 centers, A basketball team has three pure four pure forwards, four pure guares, and one swingman who ean play vither ard or forward. A pure position player can play only the designated posi tion. If the coach must start lineup with fone conter, two Forwards, and two guards, how many possible Tneups ean the ecach choose” 2.2.13) An instant lottery kicket consists of a collection of boxes covered with gray wax. Hor a subset of the boxes, the gray wax. hides a special mark, I a player seratehes olf the correet number of the wwarked boxes {and no boxes without the mark), then that ticket is a winner, Design an instant lontery game in which a player seratches five boxes and the probability that a ticket is a winmer is approximately 0.01 2.3.16 Consider a binary code with 5 (0.0r 1) in each code word. An example of code word is 01070, Tn eseh cade word, # Dit is a zeco with probability 0.5, indepen dent of any other bit bits (a) What is the probability of the code ‘word OOLIT (b) What ig the probability (hat, word contains exactly thn a code 23.20 7 NBA championships over approximately 50 Thus it may soem roasonable to a sume that ins given year the Celtics sin the Litle with probability p= 16/30 = 0.32, independent of any other year, Given such 2 model, what would be the probability ¢ Boston Celtics have won 16 SEQUENTIAL EXPERIMENTS of the Celtics winning eight straight cham- pionships beginning in 19597 Also, what Would he the probability of the Celtics wine ning the title in 10 out of 11 years, starting in 10597 Given your answers, de you trust this simple probability model? 23.3% Suppose each day that you drive to work a traffic light that you encounter is ther green with probability 7/16, sod with probability’ 7/16, or yellow with probability V/R, independent of the status of the light oa any other day. If over the couse of five days. G, Y. and #t denote the mum times the light is found to be seven, yellow, or red, respectively, whust is the probability that PIG =2.Y = 1, R= 2]? Also, what is the probability P{G er of 2.3.40 lu a game beweeu two equal teams, the home team wins with probability p> 1/2. Ta a best of three playo series, a team, with the home advantage has a game at home, followed by a game away, followed by a home game if necessary. The seri over as suo as One team wins {wo game What is P[H], the probability that the team wwith the home advantage wins the series? Ts the home advantage increased by playing three-game series rather than a oue-gamie playoff? ‘That is, is it true that PIA] > p for all p > 1/2? 2.3.50 A collection of field. goal kickers are divided lute groups 1 aud 2, Group i lies i kickers, On any kick, « kicker from group twill kick a field goal with proba: bility 1/(/—1). independent of the outcome Of any otter kicks, (a) A Wicker id selectod at random from among. all the kickers and stteripts one ficld goal, Let A be te event that a field goal is Wicked. Find PLR] (b) ‘Iwo kickers are solocted at random; K. is the event that kicker y kicks a field goal. Are KX, and Na independent? (oc) A Kicker i selected at random and at= tempts 10 field goals. Tet M he the number af misses. PIT = 5)2.4.18 A particular operation has six com ponents. Each component has & failure probebility q. independent of any other eomp. A succena oth of the following conditions: © Compononts 1, 2, and 9 all work, or component 4 works J operation requires © Component 6 or component 6 woul Drawn block diagram fir this aperntion similar to those of Figure 2.2 en page 53. Derive a forimula for the protiability PM that the operation is succosstul 2.4.28 We wish to modify the collular tole phone coding system in Example 2.27 in order to reduce the number of errors, In particular, if there are two o three zeroes in the received sequence of 3 bits; we will say that a deletion (event 9) occurs Othe erwise, if ab least 4 zeroes are receiver, the recelver decides a zero was sent, or if at least 4 ones are received, the seceiver decides « one was sent, We say that an crror oceu if ( was sent and the receiver decides j #4 was sent, For this modified protocol, what is the probability P[£| of au error? What is the probability PLD] of a deletion’? 2.4.3 Suppose a LO-digit phone aumber is transmitted by a cellular phone using foxr auch digit, using the model of binary symbol errors and deletions given in Problem 24.2. Let C denote the umuber of bits sent correctly, D the num bor of deletions, and £ the uumbor of ex wors, Find PIC +6, Dd. d, and ¢ 24.4) Consider the devier in Prob: Tam 2.4.1. Suppose ee can replace anyone component with an ultrareliable component that has a fallure probability of g/2 = 0.05. Which component should wo replace? 29.1 tvinls of the experinn Your ov tors C binary symbobs for ¢) few all ¢ Build a MATLAG simulation of 30 nt of Example 2.3. nny shold be a pair of 50% 1 yee nd H. Fer the ith trial. 4, will PROBLEMS 61 zocord whether it was heads (H, = 1) or tails (Hi = 0), and C, © {1.2} will rocord which coin was picked. 2.5.28 Following Qui suppos communication link hes different probabilities for transmitting Gand | Wher a | is sont, it is received as i 0 with, probability 0,01. When a 0 is sent, itis r= ceived as a 1 with probability 0,03, Each bit in a packer is still equally likely to bea Dor 1, Packets have been coded such that if five or fewer bits are received ia error, then the packet can be decoded. Simulate the transmission of 100 packets, each contsine ing 100 bits, Connt the number of packe decoded correctly. the 2.8.38 For a failure probability q = 0.2, simulate 100 trials of the siz-component test oF Problem 24.1. Te were found to work? Perform 10) repetitions ef the 100 telals, What do you learn from 10 repetitions of 100 trials compared to a simuleted experiment with 100 trials? many devices 2.5.49 Write « Marian finetion Necountequal (G,) thet duplicates the action of bist (G,1) in Example 2.26, Hint: Use ndgria, 2.5.5 simulation to Ssolve” Problen In thie problem, we use a MATLAB Qa. Recall hat @ particular operation has six eompo- uewts, Each component bas @ fallure prob- ability q independent of any: other compor nowt. Tho operation is sucessful if both © Components 1. 2, and 3-all work, or component 4 work. © Component 5 oF component 6 works. With g = 0.2, simulate the replacement of @ component with an ulleareliable compe nent. For cach replacement of a regular component, pertorm 100 trials: Ave 100 trinls sufficient to decide which component should be replaced?3 Discrete Random Variables 3.1 Definitions A random variable assigns numbers to outcomes in the sample space of an experiment, Chapter 1 defines a probability model, It hegins with a physical model of an experiment, An experiment consists of a procedure and observations. ‘The set, of all possible observations, §, is the sample space af the experiment. S is the beginning of the mathenttical probability model. In addition to S, the muathenuatical model Includes a rule for assigning numbers between i)-and 1 to sets A in S. ‘Thus for every AC S, the model gives usa probability PLA), where 0< Pl] < L Tn this chapter and for most of the remainder of this book, we examine probability models that assign ummabers to the outcomes in the sumple space, When we observe one of these numbers. we refer to the observation as a random variable. Ta our for example, X notation, the name of a sandora variable is always a capital letter, The set of possible values of X is the range of X. Since we often consider more than one random variable at a time, we denote the range of a random variable by the letter $ with a subseript that is the name of the ig the range of random variable X, $y is the range of ran forth. We use Sy to donote the range of X because the X ig analogous to S, the set of ‘ble outcomes of A probability model always begins wit an experiment. Each random variable is related directly to this experiment. ‘There are three types of relationships. fandom variable, Thus $y m variable 3°, and 60, ot of all possible values of n experiment 1. The random variable is the observation. Example 3.1 The experiment is to attach a photo detector to an optical fiber and count the number of photons arriving in a oné-microsecond time interval. Each observation 6234 DEFIWITIONS — 63 is a random variable X. The range of X is Sx = {0,1.2,...}. In this case, Sx, the range of X', and the sample space S are identical ‘The randow variable is a fuuction of Ue observation. Example 3.2 The experiment is to test six integrated circuits and after each test observe whether the circuit is accepted (a) or rejected (r). Each observation is a sequence of six letters where each letter is either a or r. For example, sy = aeraza, The sample space consists of the 64 possible sequences. A random variable related to this experiment is .V, the number of accepted circuits. For outcome ss, N = 5 circuits are accepted. The range of N is Sy = {0.1...-.6)] ¢ random variable is a function of another random variable Example 3.3 In Example 3.2, the net revenue 2 obtained for 2 batch of six integrated circuits is $5 for each circuit accepted minus $7 for each circuit rejected. (This is because for each bad circuit that goes out of the factory, it will cost the company $7 to deal with the customer's complaint and supply a good replacement circuit:) When .V circuits are accepted, 6—V’ circuits are rejected so that the net revenue Fis related to V by the function R=g(N BN — (6 —N} = BN — Ad dollars, (3) Since Sw = {0.....6}, the range of Tis Sie (42,90, -18,—6,6, 18, 30} ‘The revenue associated with ss = aaraaa and all other outcomes for which 2 is (5) 18 dollars (3.3) If we have a probabilily model for the Integrated cireult experiment ia Exum ple 1 ase that probability model to obtain a probability: model for the random variable, ‘The remainder of this chapter will develop methods to charac terize probability medels for random variables. We observe thet in the preceding csumples, the value of a random variable eau always. be derived from the outcome of the underlying experiment. This is not a coincidence, The formal definition of a random variable reflects this fact64 CHAPTER DISCRETE RANDOM VARIABLES Definition 3.1 Random Variable Arandom variable consists of wn experiment with « probability measure Pl) de~ jfined on-n sample space § and a function that assigns o real number to cach auteome in the sample space of the experiment, This definition acknowledges that a random variable is the result of an underlying exporiuent, but it also permits us to separate the experiment, in particular, tke observations, from the process of assigning, numbers to ontcomes. As we saw in Example 3.1, the assignment aay be implicit in the definition of he expecinent, or it may require further analysis Jn some definitions of experiments, the procedures contain variable parameters, Tn these experiments, there can be values of the parameters for which it is inn possible to perform the observations specified in the experiments. In these cases, the experiments do not produce random variables. We refer to experiments with paramoter settings that do not produce random variables as improper experiments. Example 3.4 The procedure of an experiment is to fire a rocket in a vertical direction from Earth's surface with initial velocity V km/h. The observation is 7’ seconds, the time elapsed until the rocket returns to Earth. Under what conditions is the experiment improper? At low velocities, V’, the rocket will return to Earth at @ random time T' seconds that depends on atmospheric conditions and small details of the rocket's shape and weight. However, when V > 0* = 40,000 km/hr, the rocket will not return to Earth. Thus, the experiment is improper when V >" because it is impossible to perform the specified observation. On oceasion, it is important to identify the mundont variable by the function X(s) thar maps the sample outcome s to the corresponding value of the random variable X, As needed, we will write (X lo emphasize that there iy a set of sample points $< $ for which X(s) =x. That is, we have adopted the shorshand notation {x ry = (8 2 8|X(s) = (aay Here are some more random variables: © A, the nimber of students asleep in the next probability Ieetitre: re in the next hour: © C, the number of texts you rece! © M4, the mumber of mninntes you wait until the next Lext-arrives, Random variables A ond Care discrete random variables. The possible valies of these random variables form @ countable set. ‘The underlying experiments have ample spaeos that are discrete. The random variable AF ean he any nomogative real number, It is a continuous random variable. Its experiment has a continuous