2280 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO.
12, DECEMBER 2000
Optimal Production Control in a Discrete
Manufacturing System with Unreliable Machines and
Random Demands
Youyi Feng and Houmin Yan
Abstract—We consider a production control problem in a manu-
facturing system with a failure-prone machine and a stochastic de-
mand. The objective is to minimize a discounted inventory holding
and backlog cost over an infinite planning horizon. The optimal
production control of continuous, stochastic manufacturing sys-
tems with a failure-prone machine and a constant demand has been
considered in Akella and Kumar [1]. However, the problem of op-
timal production control for discrete stochastic manufacturing sys-
tems with uncertain demands remains open. In this paper, we in-
vestigate a case where the exogenous demand forms a homoge-
neous Poisson flow. Primarily, we show that the optimal produc-
tion control for such a system is of the threshold control type. In
addition, the explicit form of production control policy and the ob-
jective functions are provided. Numerical examples are included to
demonstrate the results obtained in the paper and to compare with
the one in Akella and Kumar.
Index Terms—Optimal control, Poisson demand, production
planning and scheduling, stochastic manufacturing systems,
threshold control policies.
I. INTRODUCTION
B EGINNING with Kimemia and Gershwin [11], Akella
and Kumar [1], and Bielecki and Kumar [3], there has
been considerable interest in the problem of optimal production
planning and control of stochastic manufacturing systems.
Kimemia and Gershwin tackled a multimachine flowshop with
unreliable machines without internal buffers. Kimemia and
Gershwin demonstrated that a near-optimal control can be char-
acterized by some threshold values (known as hedging point).
Akella and Kumar dealt with a single-machine, single-part-type
system with linear holding and backlog cost. They assumed
that the machine capacity is a birth-death process, i.e., the
time between successive machine failures and repair times
are modeled as exponentially distributed random variables.
They further assumed that the demand is a constant, and the
state and control variables are continuous. They obtained an
explicit solution to the problem and verified the optimality
Manuscript received January 13, 1999; revised October 25, 1999 and Feb-
ruary 4, 2000. Recommended by Associate Editor, Q. Zhang. This research is
partially supported by Grant RP3930033 of the National University of Singa-
pore, and an Earmarked Grant from the Research Grant Council under Grant
CUHK351/96E.
Y. Feng is with the Research Group, Enron Corp., Houston, TX 77002 USA
(e-mail: youyifeng@enron.com).
H. Yan is with the Department of Systems Engineering and Engineering
Management, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong
(e-mail: yan@se.cuhk.edu.hk).
Publisher Item Identifier S 0018-9286(00)10662-2.
0018–9286/00$10.00 © 2000 IEEE
of the resulting threshold control policy. In particular, they
showed that the optimal production policy is characterized by
a critical number, known as the threshold level or the hedging
point. The optimal policy requires the machine to produce at
full capacity if the inventory level is lower than the critical
number, to produce nothing if the the inventory level is higher
than the critical number, and to produce as much as the demand
if the inventory level is equal to the critical number. Bielecki
and Kumar [3] dealt with a similar problem with a long-run
average cost objective. It has been demonstrated as well in [3]
that the threshold production control policy remains optimal.
Explicit analytical solutions of the stochastic failure-prone
manufacturing system have so far been obtained only in Akella
and Kumar [1] and Bielecki and Kumar [3]. With the assump-
tions of continuous failure-prone manufacturing systems and
constant demand, a number of researchers attempted to solve
more general problems than [1] and [3]. This line of research
has resulted in the existence and partial characterization of op-
timal production policies. For example, Van Ryzin et al. [16],
and Bai and Gershwin [2] provided an approximation of op-
timal feedback controls in the case of manufacturing systems
consisting of two or three machines in tandem. Hu and Xiang
[10], Hu et al. [9], and Liberopoulos and Hu [13] extended the
Akella and Kumar [1] to the case of non-Markovian machine
capacity process. For the long-run average cost objectives, Shar-
ifnia [19] extended Bielecki and Kumar [3] to the case of more
than two machine states. Liberopoulos and Caramanis [12] fur-
ther demonstrated that the Sharifnia’s approach applies when
the machine transition rates depend on the production rates. In
addition, multiple product systems, especially systems require
either setup costs and/or set up times, were also under considera-
tion (see Gershwin [7], Gershwin et al. [8], Caramanis et al. [5],
Connolly et al. [6], and Sethi and Zhang [18]). For a complete
survey in optimal control of stochastic manufacturing systems,
we refer the reader to the book by Sethi and Zhang [18] and a
recent survey paper by Sethi et al. [17]. Note that these results
are based on the relatively restrictive assumptions of constant
demand rate and continuous production systems.
However, it is well documented that demand uncertainty is
the one of major factors affecting the decision making in pro-
duction planning and control [14] and [20]. In addition, many
manufacturing functions are discrete in nature. Hence, taking
into account a discrete failure-prone stochastic manufacturing
system with the uncertain demand will not only reveal theoret-
ical properties and managerial implications but also open the av-
enue of applications. Recently, Perkins and Srikant [15] studied
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2281

continuous failure-prone manufacturing system with uncertain to ensure a stable system, i.e., the average capacity is no less
demand with a long-run average cost criterion. They demon- than the average demand
strated that hedging point policy is optimal with bursty demand.
In this paper, following the work of Akella and Kumar [1], (2)
we consider an optimal inventory/production control problem in
a stochastic manufacturing system where the machine capacity At the same time, we define the capacity and demand gap as
process is a birth–death process. In addition, we assume that the . By definition, .
demand is uncertain. Differing from Perkins and Srikant [15], We use to denote system states, where is
we consider a discrete manufacturing system with a discounted the inventory state, and is the machine state for
cost criterion in which the demand, the capacity of production, , i.e.,
and the processing time per unit are random variables. Specif-
ically, we show that the optimal threshold control policy can if the system is up
be specified by a critical inventory level. Moreover, we demon- if the system is down.
strate that this optimal threshold control policy minimizes costs
for all feasible production control policies over all the levels of Our aim is to minimize the operational cost on average dis-
initial inventory and machine states. counted over the infinite horizon, regardless of the initial inven-
The rest of paper is organized as follows: Section II intro- tory levels and the operational status of the underlying machine.
duces the system model and notations. An optimality condi- To this end, we can write the discounted cost over an infinite
tion is also provided at the end of the section. We evaluate horizon as
the threshold production control policies in Section III. In Sec-
tion IV, we first identify the optimal threshold production con-
trol policy and then demonstrate that this optimal threshold pro-
duction control policy remains optimal for all feasible control (3)
policies investigated. Section V provides numerical examples.
Final, we conclude the paper in Section VI. where is the level of inventory at under control
, and , respectively;
II. PRELIMINARIES is the discount factor.
The objective is to find an admissible capacity allocation, or
In this section, we provide background and notations. We will production control , which minimizes total inventory and
also characterize the optimality of the system with functional backlog costs over time. Note that an admissible control
equations which suffice to identify the optimal production con- requires that: i) is measurable in , ii) for
trol policies. , and iii) for . Specifically, the objective is
to minimize the discounted cost (3) for any initial state of the
A. Model Description and Notations
system , over , i.e.,
The underlying manufacturing system is subject to random
machine breakdowns. When the system is up, the machine can (4)
produce the product at a rate up to , where is the ma-
chine capacity. The time duration of making a unit of the product
is exponentially distributed. When the machine is down, the B. Optimality Conditions
system can produce nothing. When the machine recovers from The recent development in the point process intensity control
the breakdown, it will continue the incomplete work on the unit sheds new light on the problem of the production planning
left from the last breakdown. Because of the memoryless prop- and scheduling of stochastic manufacturing systems. In such
erty, the remaining processing time on the unit is stochastically systems, the production output has been considered as a flow of
equivalent to that of being processed from scratch. We further point process; the process is controllable by adjusting the pro-
assume that the durations of up and down periods are exponen- duction rate in the range of . In what follows, we will first
tially distributed with rates of and , respectively. The de- establish the specialization of the Hamilton–Jacobi–Bellman
mand process forms a homogeneous Poisson flow with a con- (HJB) equation for the intensity control of point processes
stant rate . The holding and backlog cost per unit of item over (see Brémaud [4]). The HJB equation is available when no
per unit of time are and , respectively. Therefore, we can interference is allowed during the middle of production. This
write the cost function as follows: point is interpreted by viewing the following relation satisfied
by an arbitrary feasible policy . For a feasible policy ,
if
(1) assuming that it takes rate at state
otherwise.

In addition, we require that any feasible control policy satisfies

the condition that once the production starts with a given rate,
no adjustment on the rate be allowed until either the machine is
broken down or the current unit is completed. Like Akella and
Kumar [1], we assume that the system has a sufficient capacity
2282 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

Denote and as the threshold control policy and its

objective function associated with integer , respectively. Thus,
when , we have

(6)

and when
The last equation is equivalent to

(7)

Furthermore, when , the production is stopped owing to

the failure of the machine; for all levels of inventory, we have,
An analogous relation for is available as well. The pre-
ceding method can be used to decompose the integral as in Bré-
maud [4] so as to yield the following theorem. (8)
Theorem 2.1: In light of the inventory and backlog cost pa-
rameters defined in (1), a feasible production control policy is
A. Cost Functions for a Threshold Control Policy
optimal if and only if its cost function satisfies the fol-
lowing Hamilton–Jacobi equations: when In order to obtain explicitly, we will show that the
conditions of (6)–(8) together suffice to identify the .
1) Characteristic Equations and Homogeneous Solutions to
(6)–(8): In general, the costs of having a positive inventory
and being backlogged differ. Such a difference also appears in
(6)–(8). Removing these costs in the above equations results in
the homogeneous equations
and when

(9)
and

Proof: The situation that the state variable is continuous (10)

is fully discussed in Sethi and Zhang [18, Th. 3.2, pp. 37–42].
Their argument is also available to the discrete state variables. Because of the disappearance of the cost term , it is possible
We omit the reiteration of the proof and refer to their book. for us to treat both inventory levels similarly. The characteristic
solutions to the system of these simultaneous difference equa-
III. THRESHOLD CONTROL POLICIES AND COST FUNCTIONS tions can be rewritten in terms of the operator
This section consists of the main body of the study and con-
tains many insights observed from the analyzes of the threshold
control policy. First, let us determine the expected cost for a spe-
Using this operator, the system of (9) and (10) is then rewritten
cific threshold control policy (Akella and Kumar [1], Gershwin
as
[7], and Sethi and Zhang [18, p. 31]). The discrete version of the
threshold control policy can be written as

if and when the machine is up (11)

(5) and
otherwise
(12)
where is the parameter required by a threshold control policy.
This is known as the threshold level or the hedging point. In respectively. Thus, the characteristic equation of the foregoing
other words, the system under the threshold control policy pro- system is
duces at its maximum speed only when the inventory level is less
than the threshold level. Note that the inventory level is bounded
by the threshold level except for the initial stages.
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2283

where is the determinant of matrix . The above determi- Hence, when , and , only is likely to be pre-
nant of matrix can be transformed into the following equa- sented. That is
tion:
(16)

But some of the above coefficients are mutually dependent in the

sense that one will determine the other. To reduce the number
(13) of undetermined coefficients or, in other words, to identify the
independent coefficients, we substitute, for instance, (16) into
We will show that the solutions to (13) are all real and positive. (6) and (7). It can be verified by balancing the coefficients of
Moreover, we will demonstrate that there exists one and only on both sides of the resulting equations that
one solution to (13) that is greater than one.
Lemma 3.1: There are three positive solutions to (13). De-
note these solutions as . Then,
.
Proof: Set aside

The second equation is obvious since is the solution to the

characteristic equation (13). Analogously, similar relations hold
between each pair of coefficients . That is

It is readily seen that and

when . Since , there exists at least one
real root in the interval . However, by factoring

All these coefficients will be used to obtain the general homo-

geneous solutions of (6) and (8) for the positive and the negative
As a result of substituting into the left-hand inventory levels, respectively.
side of the preceding equation
C. Particular and General Solutions
We now focus on the particular solutions to the system of the
equations comprising (6) and (7). For a clear presentation, we
will separately discuss the positive inventory and the negative
It is then implied that there exists a real root of in inventory levels in the ongoing analysis. Taking into account
and another root in . Hence, denote the negative inventory case first, we use the method of undeter-
as the largest root of the mined coefficients to derive a particular solution to the system.
middle root and the smallest root, Observe that (6) and (8) are equivalent to
respectively. The Lemma follows.
As a result, these solutions to (13) will be exploited to estab-
lish the homogeneous solutions to (9) and (10).
and
B. Homogeneous Solutions
To identify the homogeneous solutions to the system of
(9) and (10), the method of undetermined coefficients can be
employed. Note that these solutions are specified by a critical
number , which is arbitrary. However, for the purpose of The right-hand side of both equations are linear to . There-
optimality, we merely consider now. When , fore, we assume that the particular solutions have the forms of
the solutions can be set aside in terms of , , and

(14)
and Substituting into the system simultaneously and com-
paring the coefficients, we get following equations:
(15)

However, for and can no longer be used to structure

the solutions since inserting them will result in the exponential (17)
growth in the cost values for the linear increase of backlogs.
2284 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

The system of linear simultaneous equations of (17) consists of and for

four unknowns and four equations. Solving (17) yields

if

(18)

if
These will be substituted explicitly into the expressions for the (22)
particular solution to (6) and (8), .
Analogously, when , we will be able to derive When , the obvious boundary conditions from (6)–(8)
a particular solution to (6) and (8) which we also denote as are
. As a matter of fact, it can be obtained from
the previous analysis by replacing with . Therefore, for
, we achieve a particular solution to the system of simul- (23)
taneous difference equations (6) and (7) as well. To sum up, we and
display the results of the foregoing analysis in the following for-
mulas:
(24)

Another equation, which also crosses the level of , is (6)

if
in

if
(19) (25)

Equations (23)–(25) can be simplified by algebraic manipu-

if lations and by making use of the properties of the solutions
and . As a result, we have
if
(20)

Equations (19) and (20) together represent a particular solution

(26)
to the system of (6)–(8) with regard to the positive and negative
inventory levels, respectively. With them, we are now able to
construct a general solution to the system of (6)–(8).
The procedure to identify a general solution to the system of
(6)–(8) depends on the boundary conditions and is more com-
plicated for than for . First and foremost, we take
account of the solutions related to . Note that a general
solution to the system of (6)–(8) is the sum of a general ho- (27)
mogeneous solution and a particular solution. Hence, a general
and
solution to the system of (6) and (8) is written as follows: for

(28)
if
In view of (26) and (28), we further simplify (27). That is

if
(29)
(21)
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2285

Thereafter, (26), (29), and (28) have comprised a system of si-
multaneous linear equations with regard to the three unknown
variables . Solving it yields
D. The Values of Cost Function at the Levels Above
When , (7) and (8) are valid when (6) is disabled.
This makes the way of presenting the cost function in the levels
higher than different from that of those lower than . To ob-
tain the cost function for these levels, observe the characteristic
equation of the system of simultaneous difference equations (7)
and (8)

(30)
(34)
By factoring in the right-hand side of (34), we easily get the
solutions to (34) as

(31)
Thus, the general homogeneous solution to the system of (7) and
(8) is represented by

(32) Analogously, substituting into the homoge-

neous system of (7) and (8) and adopting the method of unde-
termined coefficients again yields
It is worth noting that the coefficients and are irrelevant
to the critical number . But it will be shown that and
are dependent on . In fact, they can be identified explicitly in
view of (7). To see this, consider that, when , (7) becomes

To identify the solution to the system of (7) and (8), a particular

solution to the system needs to be found. To this end, let
which is equivalent to

Substituting these expressions back into (7) and (8), we get the
following simultaneous equations:

In terms of the first and third equations, which form a subsystem

of two simultaneous linear equations with two unknowns, we
have
Since are constants in terms of (30) and (31), can be
determined solely

Analogously, we have by resolving

(33) the other two linear equations with being sub-
stituted by their values. In summary, the general solutions of
From (33), it is clear that is the function of . Conse- ; can be represented as
quently, can be identified by from (26). Therefore, given
a threshold control policy of a critical level , we are able if
to obtain its cost functions with regard to the inventory levels
below or upon . For the levels , the cost functions can
if
be determined in a different manner. This is the next task of our
analysis. (35)
2286 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

Because of the boundary condition, when , both (21) and Note that determining for differs from and looks
(35) must coincide. That is easier than for . Considering (22) in and the
properties of , we have

This equation leads to

Now (22) is applicable only for . Hence, in view of

(22), the cost function for is identified explicitly.
Solving the preceding system of simultaneous linear equations Analogously, (35) is applicable for while the
with and unknowns, we get coefficients are to be determined in another way.
The boundary conditions can be set when . This is to
establish

(36)

and

(37)

Thus, the cost function ; in (35) is well

defined in terms of in (36) and (37). Solving the preceding system yields and . As a result
Thus far, we have obtained the cost function for a given
threshold control policy of . We conclude this section
with the following theorem.
Theorem 3.2: For a given threshold control policy associated
(38)
with an integer , the cost function of this policy can be de-
termined uniquely by (21), (22), and (35) where the coefficients and
of ; and are given, respectively,
by (30)–(32), (33), (36), and (37).

E. The Cost Functions of Negative Critical Numbers (39)

Although we have so far derived the cost functions for the
positive critical numbers, the methodology can be employed to Substituting back (38) and (39) into (35), we are able to express
identify the threshold policies with a nonpositive number. For , explicitly.
simplicity, we outline the procedure. To this end, we assume that
the underlying threshold control policy is associated with a neg- IV. OPTIMALITY OF THE OPTIMAL THRESHOLD
ative integer . Thus, when , (22) can be employed to CONTROL POLICY
identify with a determined . But now (21)
is no longer useful in identifying . However, it A. Preliminary Results
is analogous to (35) to identify , . In this section, we will discuss some preliminary results
Because the optimal threshold control policy is never a nega- which are useful for the optimality analysis. Part of these,
tive critical number, we are not interested in specifying the cost which look somehow technical, are essentially linked with the
function for all levels. Let us show briefly how to determine insights of the system. We will try to interpret their applications
, . in sequence.
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2287

Lemma 4.1: sions for it as well. This is the contribution of the following
lemma.
(40) Lemma 4.4: Let temporarily. Then

Proof: See the Appendix.

Lemma 4.1 is very useful in optimality analysis. Meanwhile,
it implies an interesting fact as a limit of the production capacity.
Corollary 4.2: When the production capacity is going to
infinity

(41)
(43)
Proof: In view of (40)
Proof: See the Appendix.
In the following lemma, we present an equality which will be
useful in optimality analysis.
Lemma 4.5: For
Since

it is obvious that (41) holds.

Corollary 4.2 will be employed to reveal an important fact
that increasing the capacity of production would make the zero-
threshold control policy optimal. (44)
The next lemma is crucial for consistency. Note that the co-
efficient is policy-dependent, or more specifically, is the Proof: See the Appendix.
function of . For sake of exposition, we denote it by In particular, when , the preceding lemma can be sim-
whenever the dependency should be emphasized. For instance, plified as follows.
is used in the following lemma. Corollary 4.6:
Lemma 4.3:

(42) (45)

Proof: See the Appendix. Proof: In terms of (44)

Remember that given the zero threshold control policy, the
cost function , can be identified by the last part
of the foregoing section. This is the first approach. However,
its cost functions can also be specified at least in form by the
second approach. That is the one specializing the cost functions
for positive critical numbers at . Lemma 4.3 discloses
that both approaches yield identical cost functions. As a matter
of fact, the first approach yields that

In terms of the second approach

As a result of Lemma 4.3, they are identical.

Similarly, it is legitimate to consider the consistency of
. We show that both approaches yield identical expres- The last equality is just (45).
2288 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

B. Identification of the Optimal Threshold Control Policies where the function is defined as
We will show, at first, that the optimal threshold control policy
cannot be of a negative integer.
Lemma 4.7: The optimal threshold control policy cannot be
associated with a negative integer .
Proof: Recall the expressions for the cost function of a
threshold control policy with a negative number . In par-
Then is the minimum of .
ticular, for
Proof: This is straightforward from Lemma 4.9.
We are able to assert that exists finitely. This is actually
owing to a limit since .
However, the figure
if

is negative. It is thus implied that when is large enough

if
For any Corollary 4.12: if and only if

(46)

This inequality implies that the policy associated with cannot Proof: In view of Corollary 4.6
be optimum since obviously the policy defined by per-
forms better than it, at least at all backlog levels.
As a result, we will be no longer interested in those threshold
control policies associated with negative critical numbers. Anal-
ogously, in their deterministic and continuous manufacturing Hence, if and only if . However, if
system, Akella and Kumar [1] pointed out the same result. and only if
To identify the optimal threshold control policy, we need the
following concept of unimodal function.
Definition 4.8: A function for a set of integers,
, is called unimodal if there exists a such that when This is equivalent to (46).
and is nonincreasing and when , On the basis of further analysis from Corollary 4.12, we
and is nondeceasing. would like to point out two possible situations where the
Lemma 4.9: is convex in . In other words, zero-threshold control is optimal.
is nondecreasing in . Corollary 4.13: When the capacity of production is large
Proof: See the Appendix. enough, irrespective of the backlog cost coefficient , the zero-
Lemma 4.10: is a unimodal function of . threshold control is optimal. In addition, when the backlog cost
Proof: It is easily checked out that coefficient is small enough, the zero-threshold control is op-
timal as well.
Proof: The last result is readily obtained from (46) since

Since is increasing for

, there exists at most an such that
and . Hence, when is But the first result is a bit complicated. This is because
negative, and, when is nonnega-
tive. In other words, is the minimum of , in particular,
when is decreasing and when
is nondecreasing. Therefore, is unimodal and the lemma
follows.
Corollary 4.11: Suppose that is the maximum of
integers such that

where the last equality is due to (41) of Lemma 4.2. Hence, when
is large enough, (46) takes effect.
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2289

In the next section, we shall show that the optimal threshold TABLE I
control is the optimum of all feasible control policies. SYSTEM PARAMETERS FOR TESTING EXAMPLES

C. Discounted Value Criterion

This section will focus on ensuring that the cost function
satisfies the discount cost criterion. In other
words, we will ensure the HJB equation for the model. To this
end, it is essential to ensure that is the unimodal func-
tion of with the minimum equal to . Once again, for sim-
plicity, we suppress the subscript of in this section.
Lemma 4.14: is a decreasing function of .
Proof: See the Appendix.
Lemma 4.15: is an increasing function of .
Proof: See the Appendix.
Because of Lemmas 4.14 and 4.15, the cost function
is unimodal with respect to the inventory level . This is suf-
ficient for the to satisfy the HJB equation.
When the inventory is below the optimal critical level ,
we need to produce the product at the maximum rate to meet the
demand; when the inventory is above or at the critical level, the
production stops. The function of the optimal threshold control Fig. 1. Cost functions.
policy is readily done just for the HJB equations. To see it, let
“argmin ” stand for the minimizers of over all feasible sets. and deterministic demand model obtain a reasonable approxi-
We will ensure that satisfies jointly the HJB mation of the discrete and stochastic nature of manufacturing
equations in the following lemma. systems when the systems capacity is close to the demand.
Lemma 4.16: The cost function of the optimal threshold con-
trol policy does satisfy the HJB equation. Moreover, the optimal VI. CONCLUSIONS
threshold control policy is optimal among all admissible control In this paper, we consider a production control problem in a
policies. manufacturing system with a failure-prone machine and a sto-
Proof: It is obvious from Lemmas 4.14 and 4.15 that chastic demand. Unlike Akella and Kumar [1], we consider the
problem of optimal production control for discrete stochastic
manufacturing systems with uncertain demands. Specifically,
if we investigate the case of the exogenous demand forming a ho-
otherwise. mogeneous Poisson flow. It has been demonstrated that the op-
timal production control remains a threshold production con-
Therefore, satisfies the HJB equation in The- trol type. In addition, the explicit form of the production con-
orem 2.1 trol policy and the objective functions are provided. Numer-
ical experiments are carried out to demonstrate the policy, and
V. EXAMPLES its difference to the continuous model. Further research direc-
tions include extending the model to the cases of multiple ma-
In this section, we use the results obtained above to solve two chines/products and average cost objectives.
sample problems. The same sets of data are also applied to the
Akella and Kumar model. However, the mean of demand and APPENDIX
the mean machine capacity are used for these required param-
eters. Note that, in the Akella and Kumar model, the demand A. The Proof for Lemma 4.1
and machine capacities are considered as deterministic param-
Recalling that is the largest solution of the characteristic
eters. The systems parameters for these two examples are listed
equation system of (6) and (7), we get
in Table I.
First, we depict the cost functions and in
Fig. 1. The figures indicate that both functions obtain the min-
imum at 32. In Tables II and III, we list the optimal threshold Note that . Hence, the left-hand the side of preceding
values and the optimal cost functions for different initial values. equation is positive and less than one since .
Comparing with the continuous model of Akella and Kumar, it Considering
is not difficult to find that the results obtained by the continuous
model are lower bounds of ours. Note that the threshold value
significantly differs from Akella and Kumar when the capacity
and demand gap is large (refer to the definition of the capacity
and demand gap on page 101). This suggests that the continuous
2290 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

TABLE II
COMPUTATIONAL RESULTS OF CASE 1, ( = 33:33)

TABLE III
COMPUTATIONAL RESULTS OF CASE 2, ( = 12:5)

we have B. The Proof for Lemma 4.3

It is readily verified that (42) is equal to

However the equation

can be transformed into, by simple algebraic manipulations, the

following equations: by substituting

into the right-hand side of (42). However, by employing the ex-

pressions (31) and (32), respectively, we can simplify

As a result of combining Lemma 4.1 and the above equations

It is therefore easily checked out that the last inequality is equiv-

alent to the second inequality of (40). For the first inequality of
(40), note that
This expression for can be employed to sim-
plify (42) further. As a result
entailing that

Dividing both sides of the last inequality by results in the first

inequality of (40).
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2291

Note that

and

Consequently, we get

We then substitute the last equality into the left-hand side of (47)
and get

This completes Lemma 4.3.

C. The Proof of Lemma 4.4

In view of Lemma 4.3, (43) is equal to

The proof is completed.

D. The Proof of Lemma 4.5

Substituting the values of into the right-hand side
(47) of (44), we get

We simplify the summation in the left-hand side of (47) as fol-

lows:
2292 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

Also, note that we have repeatedly made use of the following

relations in the above derivations: for

Therefore, the lemma follows.

E. The Proof of Lemma 4.9

Observe that for

In view of (44), we can simplify

as:

To show that is an increasing function of , consider

It is readily seen that the last equation is just (44). Note that the
second-to-last equation has employed the following relation:

Consequently, it is found that is strictly increasing and so

is convex. The proof is completed henceforth.
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2293

F. The Proof of Lemma 4.14 The last of the preceding derivations ensures that for
. Henceforth, the
For
Lemma follows.

G. The Proof of Lemma 4.15

Proof: We will investigate two different cases.

Case 1 : We begin with simplifying the expressions for
and . Note that associated with can be
rewritten, respectively, as, after some algebraic manipulations

The last inequality, which finishes this case, is due to the defi-
nition of . Now consider another case of . Note
2294 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

The foregoing expressions for can be employed in we will see it is an increasing function of . As a matter of
simplifying fact, for

where

Furthermore, it is readily seen that , so that for

entailing immediately that

This completes the justification for .

Case : The now has a simpler form than
that of

It is induced that

while the first inequality is due to the definition of .

Furthermore

where the inequality is also due to the definition of .

Therefore, in combination
Substituting these expressions into the following relation, we
get

where

Setting
FENG AND YAN: OPTIMAL PRODUCTION CONTROL IN A DISCRETE MANUFACTURING SYSTEM 2295

Utilizing Corollary 4.12, it is found that as shown in the equation In particular, . On the other hand, when
at the top of the page. Thus

implying that for all . It

is then ensured that

and the case of is proven virtually.

This completes the proof.

We are ensuring that REFERENCES

[1] R. Akella and P. R. Kumar, “Optimal control of production rate in a
failure-prone manufacturing system,” IEEE Trans. Automat. Contr., vol.
31, pp. 116–126, 1986.
[2] S. Bai and S. B. Gershwin, “Scheduling manufacturing systems with
work-in-process inventory,” in Proc. 29th IEEE Conf. Decision and
Control, Honolulu, HI, 1990, pp. 557–564.
[3] T. R. Bielecki and P. R. Kumar, “Optimality of zero-inventory policies
for unreliable manufacturing systems,” Operations Res., vol. 36, no. 4,
pp. 532–541, 1988.
[4] P. Brémaud, Point Processes and Queues, Martingale Dynamics. New
In fact, since , the quantity is York: Springer-Verlag, 1981.
strictly increasing at and goes to as . Therefore, [5] M. Caramanis, A. Sharifnia, J. Q. Hu, and S. B. Gershwin, “Devel-
there should exist a unique such that for opment of a science base for planning and scheduling manufacturing
system,” in Proc. 1991 NSF Design and Manufacturing Systems Conf.,
The University of Texas at Austin, Austin, TX, Jan. 9–11, 1991, pp.
27–40.
[6] S. Connolly, Y. Dallery, and S. B. Gershwin, “A real-time policy for per-
forming setup changes in a manufacturing system,” in Proc. 31st IEEE
Conf. Decision and Control, Tucson, AZ, Dec. 1992.
[7] S. B. Gershwin, Manufacturing Systems Engineering. Englewood
and for Cliffs, NJ: Prentice-Hall, 1994.
[8] S. B. Gershwin, M. Caramanis, and P. Murray, “Simulation experience
with a hierarchical scheduling policy for simple manufacturing system,”
in Proc. 27th IEEE Conf. Decision and Control, Austin, TX, 1988, pp.
1841–1849.
[9] J. Q. Hu, P. Vakili, and G. X. Yu, “Optimality of hedging point policies
in the production control of failure-prone manufacturing systems,” IEEE
As a result, when Trans. Automat. Contr., vol. 39, pp. 1875–1880, 1994.
[10] J. Q. Hu and D. Xiang, “Structural properties of optimal production con-
trollers in failure prone manufacturing systems,” IEEE Trans. Automat.
Contr., vol. 39, pp. 640–642, 1994.
[11] J. G. Kimemia and S. B. Gershwin, “An algorithm for the computer
control of production in flexible manufacturing systems,” IIE Trans.,
vol. 15, pp. 353–362, 1993.
[12] G. Liberopoulos and M. Caramanis, “Production control of manufac-
turing systems with production rate-dependent failure rates,” IEEE
Trans. Automat. Contr., vol. 39, pp. 889–895, 1994.
[13] G. Liberopoulos and J. Q. Hu, “On the ordering of hedging points in
a class of manufacturing flow control models,” IEEE Trans. Automat.
Contr., vol. 40, pp. 282–286, 1995.
2296 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 45, NO. 12, DECEMBER 2000

[14] S. Nahmias, Production and Operations Analysis, 3rd ed. Chicago, IL:
Irwin, 1997.
[15] J. R. Perkins and R. Srikant, “Failure-prone production systems with
uncertain demand,” Boston Univ., 1998. Working Paper.
[16] G. J. Van Ryzin, X. C. Lou, and S. B. Gershwin, “Production control for
a tandem two-machine system,” IIE Trans., vol. 25, pp. 5–20, 1993.
[17] S. P. Sethi, H. Yan, H. Zhang, and Q. Zhang, “Asymptotic optimality
of hierarchical controls in stochastic manufacturing systems: A review,”
Univ. Texas at Dallas, 1998. Working paper.
[18] S. P. Sethi and Q. Zhang, Hierarchical decision making in stochastic
manufacturing systems. Cambridge, MA: Birkhäuser, 1994.
[19] A. Sharifnia, “Production control of a manufacturing system with
multiple machine states,” IEEE Trans. Automat. Contr., vol. 33, pp.
620–625, 1988.
[20] E. A. Silver, D. F. Pyke, and R. Peterson, Inventory Management and
Production Planning and Scheduling, 3rd ed. New York: Wiley, 1998.
Houmin Yan received the B.S. and M.S. degrees
from the Department of Automation, Tsinghua Uni-
versity, in 1982 and 1985, respectively, and the Ph.D.
degree from the Faculty of Management, University
of Toronto in 1993. He was an Engineer with CIM
Group, Semiconductor Manufacturing Operations,
Digital Equipment Corporation, Hudson, Mass. in
1991. He has been with the Department of Systems
Engineering and Engineering Management, the
Chinese University of Hong Kong as an Assis-
tant/Associate Professor since Jan. 1994. His main
research areas are operations management, stochastic models, simulations, and
supply chain management. He has published in journals such as Operations
Research, Production and Operations Management, Discrete Dynamic Event
Systems: Theory and Applications, Journal of Classification, Journal of
Optimization: Theory and Applications, International Journal of Production
Economics, and IEEE TRANSACTIONS ON AUTOMATIC CONTROL.

Youyi Feng received the Ph.M. and Ph.D. degrees

from the Department of Industrial Engineering and
Operations Research, Columbia University, in 1991
and 1995, respectively. He obtained the B.S. degree in
1982 and the M.S. degree in 1985, respectively, from
the Department of Mathematics, Nanjing University.
From 1994 to 1995, he worked as a finance analyst
for Baring Security, Inc. and Lasser Marshall, Inc. in
New York subsequently. He jointed National Univer-
sity of Singapore as an Assistant Professor in 1995
for the Department of Information System, School
of Computing. His research interest includes yield management, stochastic in-
ventory/production systems, supply chain management and e-commerce. His
research works have been published in Management Science, Operations Re-
search, IIE Trans. and Ann. of Operations Research.