INM2016 Software Reliability and Measurement Winter term, 2012 Coursework assignment: Problem 2

General instructions
Recall that the coursework assignment is distributed as three problems. All the coursework together is worth 30% of the module mark, 10% for each problem. You must follow the general instructions distributed previously with Problem 1. Please read them again before you proceed. These questions refer to the definitions of dependability measures, the techniques for reliability testing and the inference methods for estimating steady-state reliability. They require you to apply to practical examples the concepts and formulas explained in the lectures.

Question 1 (3 marks; expected size of answer: between one half and one page)
This question concerns the basic use of the exponential reliability function. You are told that a piece of software has a failure rate () equal to 0.0012 per hour. For this software: what is the expected number of failures in one year of operation? what is the probability of functioning for one year without ever failing? what is its MTTF, in hours? How many months is it? what is this softwares probability of functioning without failure for a time equal to its MTTF? if it restarts in 2 minutes (on average) after every failure, what is its average availability?

For each answer, show your steps for obtaining the numerical answer.

Question 2 (4 marks; expected size of answer: between one and three pages)
This question applies the methods explained in the lecture about assessing steady-state reliability. You need to use the following formulas from the document Guidelines for Statistical Testing (available through Moodle), pp 88 and 89. Background details, if you were to need them, can be found in the body of that document. Estimating the probability of failure on demand. Bayesian procedure The required measure is the probability of failure on demand, . Given T independent test runs with r failures; given a prior distribution with density f(): ........... "Beta" prior distribution with parameters (a,b), mean posterior distribution: posterior expected value of : a : a+b is "Beta" with parameters (a+r, b+T-r) E(| R=r, T))= a+r a+b+T

Notes for using the above formulas: in the above formulas, the pfd is called or or (the Greek letter theta). The Beta distribution is a form of distribution often used in Bayesian reliability file: IN2016_11-12_CWproblem2_v01.doc p 1 of 3

assessment. For the purpose of this exercise, you do not need to know exactly how it is calculated, since we will reason about the mean value of a products pfd, which is given by the formulas above. You are testing a software product for which the best time variable for reliability is the number of demands. It has been estimated that when installed with the whole expected user base and in continuous use it will receive 50,000 demands per year. Your management has decided that it can only be distributed for installation when the results of testing are such that one can expect less than 50 failure per year of operation. You are asked to apply a Bayesian inference process. The prior distribution of the pfd is a Beta distribution function with parameters a=2, b=18 (its probability density function is shown below).

What is the expected value of the softwares pfd? (By the way, its median is about 0.085. Can you tell from the graph how many modes this distribution has, at approximately what values of pfd?) Using this expected value of the pfd as if it were the true pfd 1, what number of failures would you expect in one year of operation? In the first 200 test cases, there are no failures. What is the expected value of the probability of failure per demand? Again using this mean as if it were the true pfd, what number of failures would you expect in one year of operation? You keep testing, and by the time you reach 1000 test cases, 2 failures have occurred. Again, what is the expected value of the probability of failure per demand, and if you use this mean as if it were the true pfd, what number of failures would you expect in one year of operation? You keep testing, and no more failures are observed in many tests; what total number of tests do you need to reach, without any more failures occurring, for you to be able to tell the management that your prediction matches their requirement on expected number of failures per year of operation? Supposing no failure at all had been observed (that is, not even the 2 failures in the first 1000 tests), after how many tests would your prediction have matched the requirement?

For information: this is not an exact procedure, but it is know to err on the side of pessimism (often a good first approximation) given the reliability functions that apply to software (exponential distribution of time to failure or geometric distribution of number of demands before failure.

file: IN2016_11-12_CWproblem2_v01.doc

p 2 of 3

Question 3 (3 marks; expected size of answer: between a half and three pages)
You manage software product reliability acceptance testing for a new product, a fire alarm that senses the temperature in a room and the concentration of smoke particles in the air, and sounds a siren when either measure reaches a danger level. The product has other features, including a remote monitoring mode of operation in which the fire alarm automatically calls a pre-programmed telephone number if the siren sounds or if the battery is running low. This feature is popular for protecting homes during holiday absences. Your company has a reputation for high reliability and is trying to expand its export market, so it is important that the product when shipped be highly reliable and safe. A quick analysis identifies two ways for this system to fail: not sounding the siren when it should sound it; or sounding it when it should not. Indicate: which of these failures you would count in order to produce a safety assessment of the system and which ones you would count in order to produce a reliability assessment what is another way of failing that would affect safety and another way of failing that would not affect safety.

Since the system operates in essentially two different situations there is a fire or there is no fire you decide to specify two separate testing subprofiles for these two conditions. You ask one of your subordinates to prepare the subprofile for the no fire condition. He has first to prepare a profile of mode of use (remote monitoring or not), temperature, and amount of particles in the air. He reports as follows: on average, customers will use remote monitoring for about 15 days in any one year (mostly in the Summer or in the 25th December - 1st January period). So, a fraction of 15/365 of all test cases will be in remote monitoring mode for each test case, I will choose the temperature by sampling the distribution of temperatures observed over the last 10 years, hour by hour, by my local weather service (that is: I will take the records 10 * 362 * 24 distinct values and for each test case pick one at random, all of them with equal probability)

Indicate two ways in which this way of building the test profile violates the rules we have given, and how you could correct these flaws.

file: IN2016_11-12_CWproblem2_v01.doc

p 3 of 3