Identification and Control of Dynamical Systems Using Neural Networks KUMPATI S. NARENDRA retiow, tee, Ano KANNAN PARTHASARATHY ation and contol of mines aan ‘tnd contr State and dynamic bark propugtonwthods forthe a parameters are acessed, Inthe model hat ae Ir recurrent network ae ercnerted i ove ‘mfguraton and hence here rel ned to stad them in ie {ashon Simulation eats eval tht the ateaton and adaptive entrlwhemes soe ae practical eal Bai coed ‘dentions are nrdced throughout the paper and theortalqoe- tons which have tobe addrened ae alee dsb 1. Ivrmopuerion |ATHEMATICAL systems theory, which has in the past five decades evolved into a powerful scientific discipline of wide applicability, deals with the analysis and synthesis of dynamical systems. The best developed. aspect of the theory treats systems defined by linear op- cerators using well established techniques based on linear algebra, complex variable theory, and the theory of oF dlinary linear differential equations. Since design tech- niques for dynamical systems are closely related to their stability properties and since necessary and sufficient con- ditions forthe stability of linear time-invariant systems have been generated over the past century, well-known, ‘design methods have been established for such systems, In contrast to this, the stability of nonlinear systems can be established for the most par only on a system-by-sys- tem basis and hence itis not surprising that design pro- cedures that simultaneously meet the requirements of st bility, robustness, and good dynamical response are not currently available for large classes of such systems In the past three decades major advances have been ‘made in adaptive identification and contol for identifying and controlling linear time-invariant plants with unknown, Parameters. The choice of the identifier and conttoler structures is based on well established results in linear systems theory. Stable adaptive laws for the adjustment ‘of parameters in these eases which assure the global st bility of the relevant overall systems are also based on, Properties of linear systems as well as stability results that ‘ort war supponed by te National eens Founda det gat EE S137 andy Sol Nana bosons ener Cota 1791 dic in oath Bp of Hs Egg ake TERE Lop Number 53588 are well known for such systems [1]. In this paper our ierest is in the identification and control of nonlinear {dynamic plants using neural networks. Since very few re sults exist in nonlinear systems theory which can be di rectly applied, considerable care has to be exercised inthe statement of the problems, the choice of the identifier and controller structures, as well asthe generation of adaptive laws for the adjustment of the parameters. ‘Two classes of neural networks which have received considerable attention in the area of artificial neural net works in recent years are: 1) multilayer neural networks and 2) recurrent networks. Multilayer metworks have proved extremely successful in pattern recognition prob Jems [2}-[S} while recurrent networks have been used in associative memories as well as for the solution of opti- ‘mization problems [6}-19}. From a systems theoretic point of view, multilayer networks represent static nonlinear maps while recurrent networks are represented by nonlin- car dynamic feedback systems. In spite of the seeming diflerences between the two classes of networks, there are compelling reasons to view them in unified fashion. In fact, it isthe conviction of the authors that dynamical ele ‘ments and feedback wil be increasingly used in the future, resulting in complex systems containing both types of net- works. This, in turn, will necessitate unified treatment ‘of such networks. In Section TIT of this paper this view. Point is elaborated further. ‘This paper is written with three principal objectives. ‘This first and most important objective isto suggest iden tification as well as controller structures using neural net- ‘works for the adaptive control of unknown nonlinear dy ‘namical systems. While major advances have been made in the design of adaptive controllers for lineer systems ‘with unknown parameters, such controllers cannot be used forthe global control of nonlinear systems. The models suggested consequently represent a fist step in this direc tion. A second objective is to present a prescriptive ‘method for the dynamic adjustment of the parameters based on back propagation. The term dynamic back prop- agation is introduced in this context. The third and final objective isto state clearly the many theoretical assump: tions that have to be made to have well posed problems Block diagram representations of systems commonly used in systems theory, as well as computer simulations, are included throughout the paper to illustrate the various concepts introduced. The paper is organized as follows: 1045-9227/90/0300.0004801.00 © 1990 IEEESection I deals with basic concepts and notational details used throughout the paper. In Section I, multilayer and recurrent networks are treated in a unified fashion. Sec- tion IV deals with static and dynamic methods forthe ad Justment of parameters of neural networks. Identification ‘models are introduced in Section V while Section VI deals With the problem of adaptive control. Finally, in Section VII, some directions are given for future work TI, Preuisunanies, Basic CONCEPTS, AND NOTATION In this section, many concepts related to the problem of identification and contol are collected and presented for easy reference, While only some of them are directly used in the procedures discussed in Sections V and VI, all of, ‘them are relevant fora broad understanding of the role of ‘neural neeworks in dynamical systems. A. Characterization and Identification of Systems System characterization and identification are funda- ‘mental problems in systems theory. The problem of char- acterization is concemed with the mathematical represen {ation ofa system; a model of a system is expressed as an ‘operator P from an input space ‘U into an output space and the objective is t0 characterize the class @ to which P belongs. Given a class @ and the fact that Pe 6, the problem of identification is to determine a class © C and an element P € & so that P approximates P in some desired sense. In static systems, the spaces U and Y are subsets of and {", respectively, while in dynamical systems they are generally assumed to be bounded Leb- ‘esgue integrable functions on the interval [0, T} or {0, =). In both eases, the operator P is defined implicitly by the specified input-output pairs. The choice ofthe class of identification models ®, as well asthe specific method used 10 determine B, depends upon a vanity of factors, which are related to the accuracy desired, as well as an- lytical tractability. These include the adequacy of the ‘model Po represent P, its simplicity, the ease with which itcan be identified, how readily it can be extended if i does not satisfy specifications, and finally whether the P chosen isto be used off line or on line. In practical appli- cations many ofthese decisions naturally depend upon the prior information that is available concemming the plant to be identified 1 Identfcation of Static and Dynamic Systems: The problem of pattem recognition is a typical example of identification of static systems. Compact sets U, Care ‘mapped into elements € /";(i = 1, 2, "+" .) inthe ‘output space by a decision function P. The elements of U, sdenore the pattern vectors corresponding to class 3. In ‘dynamical systems, the operator defining a given plant is implicitly defined by the inpot-ourput pars of time functions u(#), (1), €[0, TI. In both cases the objec tive isto determine P so th {9 - yl) = JP) - POO] se a) for some desired ¢ > 0 and a suitably defined nom (de noted by Il-[}} on the output space. In (1), P(u) = § de wed notes the output of the identification model and hence 'y de is the error between the output generated by P and the observed output y. A more detailed statement of the identification problem of dynamical systems is given, in Section IL 2. The Weierstrass Theorem and the Stone-Weier- strass Theorem: Let C({a, b}) denote the space of con- tinuous real valued functions defined on the interval [4 5] with the norm of fe C({a, 6]) defined by [sl = sup {| £00] "6 fa, 2} ‘The famous approximation theorem of Weierstass states that any function in C({a, b]) can be approximated ar- bitrarly closely by a polynomial. Alternately, the set of polynomials is dense in Cif, b]). Naturally, Weier rast's theorem and its generalization to multiple dimen- Sions finds wide aplication im the approximation of con- tinuous functions f:°* —> [ using polynomials (¢.2.. pater recognition). A generalization of Weierstrast's theorem due to Stone called the Stone-Welerstas theo- rem can be used asthe stating point forall the approxi Ihation procedures for dynamical systems Theorem: (Stone-Weicrstass (10): Let be a com- pact metric space. If isa subalgebra of CCL, which Contains the constant functions and separates points of L then is dense in CCU, Tn the problems of interest us we shall assume that the plant P to be identified belongs to the space @ of bounded, continuous, time-invariant and causal operators [11] By the Stone- Weierstrass theorem. if 6 satisfies the conditions of the theorem, model belonging to can be chosen which approximates any specified operator Pe © 'A vast literature exists on the characterization of non- Tinea functionals and includes the classic works of Vol tema, Wiener, Bare, and Urysohn. Using the Stone~ ‘Weierstrass theorem it can be shown that a given nonin- cat functional under certain conditions ca be represented by a corresponding series such as the Voltera series or the Wiener series. In spite of the impressive theoretical work that these represent, very few have found wide ap Dlicatin inthe identification of large classes of practical honlinar systems. In this paper out interest i mainly on fepresenatons which permit on-line identification snd contol of dynamic systems in tems of finite dimensional nonlinear difernce (or diferent) equation. Such non- Tinear models are well known in the systems iterate and ate considered inthe following subsection B, Input-State-Output Representation of Systems ‘The method of representing dynamical systems by vee- tor differential or difference equations is currently. well established in systems theory and applies toa fae Taree class of systems. For example, the differential equations HO 2 ay = afar ae 280) = PLC), w(t] @) yt = ¥Lx(9)]where x(1) 2 EC, a2. oo7 yO an) & [ag (2), mC, 22> 5 ayCOYT and yO) © [yD 9200s ‘Yn 1)1? represent a p input m output system of oF der m with u(2) representing the inputs, x,(t) the state variables, and y,(r) the outputs ofthe system. and ¥ are static nonlinear maps defined as 15" x 1 — Band ViN" > BM The vector x(1) denotes the state of the system at time rand is determined by the state at time f ‘ 0 determines the step size. In the following discussions the constant vector of plant parameters {ay we ucts 8,117 will be denoted by p and that of the fon = Gans Bae + Bail” by B. Lea time-invariant plants which are controllable can be shown o be stabilizable by linear state feedback. This fact hasbeen used to design adaptive contollers for such plant. For example, if an upper bound on the onder of the plants known, the contol input can be generated 3s a linear combination ofthe past values of the input and Output respectively. 1 (K) represents the contro param ler vector it can be shown that a constant vector 8" ex- iss such that when &(K) = 8 the plant together with the contoler has the same input-outpt characteristics a the reference model. Adaptive algonthns for adjusting 8(K) ina stable fashion are row well known and have the gen eral form shown in (8) 4. Nonlinear Systems: The importance of contllabil ity and obserabilty in the formation ofthe identi tion and contol problems for linear systems is evident from the discussion in Section H-C-3. Other well-known results in linear systems theory are also called upon to Choose a reference model as well as a suitable paramet ization ofthe plant and to assure the existence of u desired controler. In recent years a numberof authors have ad ‘resed issues such a conuollability, obserabilty, fed back stabilization, and observer design for nonlinear sy tems [13}-[16}. In spite of such attempts constructive procedures, similar 1 those available for linear systems, 4o not exist for nonlinear systems. Hence, the choice of idemicaion and controller models for nonlinear plants is formidable poblem and succesfl identification snd contol has depend upon several strong assumptions re garding the input-output behavior ofthe plant. For ex ample, if a SISO system is represented by the equation {G), we shall assume that the sate of the system can be reconstrvcted from n measurements of the input and ou put. More precisely, y,(k) = WLG)], 3k + 1) YEG. uD I} yk tn i) = HLL SOL). WE} wk # DD) ue + n= 21) Yield nonlinear equations in unknowns x(&) if w(k), Suk +n — 2), ¥p(B) 20 (kn = 1) are specified and we shall assume that for any sctof values ‘of w(k) in a compact region in ‘U,& unigue solution 10 the above problem exits. This permits identfation pro czdures to be proposed for aonlincar systems along lines ‘similar to those inthe linear case "Even when the function ¢ is known in (3) and the state vector is accessible, the determination of u(.) fo the plant tohavea desired trajectory isan equally dificult problem Hence, forthe generation of the contol input, the exis tence of suitable inverse operators have to be assumed. If 4 controller structure is assumed to generate the input 14(.), further assumptions have to be made to assure the existence ofa constant control parameter vector to achieve the desied objective. All these indicate that considerable ‘progress in nonlinear contro theory wil be needed to ob- {ain rigorous solutions to the identification and contrel problemsIn spite ofthe above comments, the linear models de- scsbed in Section I-C-3 motivate the choice of strictures for identifiers and controllers in the nonlinear case. It is jn these structures that we shall incorporate neural net- works as described in Sections V and VI. A variety of considerations discussed in Section III reveal that both ‘multilayer neural networks as well as recurrent networks, which are currently being extensively studied, will feature as subsystems inthe design of identifiers and controllers for nonlinear dynamical systems. IL. Murmiaver aND RECURRENT Networks: The assumptions that have to be made to assure well posed problems using models suggested in Sections V and VI are closely related to the properties of multilayer and recurrent networks. In this ection, we describe briefly the two classes of neural networks and indicate why a unified treatment of the two may be warranted to deal with more complex systems in the future. A. Multilayer Networks A typical multilayer network with an input layer, an ‘output layer, and two hidden layers is shown in Fig. 2. For convenience we denote this in block diagram form as shown in Fig. 3 with three weight matrices IV", Wand W” and a diagonal nonlinear operator T' with identical sig- ‘modal elements [i.e., y(4) = 1 = e"/1 + e°*] fol- lowing each of the weight matrices. Each layer ofthe net- ‘work can then be represented by the operator Niu] = rp%u (9) and the input-output mapping of the mulilayer network an be represented by y= Mal = rrr LW al] = NNN Lal (10) In practice, mutilayer networks have been used success flyin paem recognition problems [2}-SI. The weights of the network W', W?, and W° are adjusted as described in Section TV to minimize a sutblefeton of the enor between the output yof the network and a desired opt sg: This eels in he mapping fection Na] wealzed by the network, mapping vectors into corresponding ouput classes. Geoerlly a discontinuous mapping such as + nearest neighbor fle i used atthe last stage to map the input sets nto pints inthe range space corresponding to outpot classes. From a systems theostic point of view, tullilyer network an be consiered as versatile nonin: Gar maps with the elements of he weight mates 25 pa ‘ameter. Inthe following sections we shall we the terms rights" and “parameters interchangeably B. Recurrent Networks Recurrent networks, introduced in the works of Hop- field [6] and discussed quite extensively in the literature, provide an alternative approach to pattem recognition One version of the network suggested by Hopfield con- HE eH eH Fig. 3. A lok dag eprsettin of eer mt, sists ofa single layer network Ny, included in feedback Configuration, witha time delay (Figs. 4 and 5). Such a network represents a disrete-ime dynamical system and an be deseibed by XPD EN (AH) x(0) = 5. Given an initial value ro, the dynamical system evolves to-anequilum sate if) is suitably chosen, The set of inital conditions inthe neighborhood of x which con- ‘verge tothe same equilibrium state f then identified with that state. The tem "associative memory” is used to de- seribe such systems. Recently, both continuous-time and diseretetime recurrent networks have een studied with constant inputs {17}. The inputs rather than the iil conditions represent the patterns to be classified inthis case. Inthe continuous-time case, the dynamic system in the feedback path has diagonal transfer matix with identical elements 1/(s + a) along the diagonal. The system is then represented by the equation Ee nar + Nix] +7 any so that x(1) Fs the state of the system at time f, and the constant vector 1 € "is the input. © A Unified Approach In spite ofthe sceming differences between the two ap- proaches to patter recognition using neural networks, it is lear that a close relation exists between them. Recur- rent networks with oF without constant inputs are merely nonlinear dynamical systems and the asymptotic behavior Of such systems depends both on the initial conditions a5. well as the specific input used. In both cases, this depends. critically on the nonlinear map represented by the neural network used in the feedback loop. For example, when ‘no input is used, the equilibrium state of the recurrent network in the discrete case is merely the fixed point of the mapping Nj. Thus the existence of a fied point, the conditions under which itis unique, the maximum num- ber of fixed points that can be achieved in a given network are all relevant to both multilayer and recurrent networks. Much of the current literature deals with such problems [18} and for mathematical tractabilty most of thembt Fig 4, Te Hopld atv “PEE> Fig. 5. Block diagram representation of the Hope stor sume that recurrent networks contain only single layer networks (ie.,.N;[.))- As mentioned earlier, inputs when they exist are assumed to be constant. Recently, two layer recurrent networks have also been considered (19] and ‘more general forms of recurrent networks ean be con- structed by including mullayer networks in the feedback loop (20). In spite of the interesting ideas that have been Presented in these papers, our understanding of such sys {ems is still far from complete. Inthe identification and control problems considered in Sections V and VI, mul tilayer networks are used in cascade and feedback config trations and the inputs to such models are functions of time. D. Generaliced Newral Networks From the above discussion, it follows that the basi ele- iments in a multilayer network is the mapping Ny[.] = PLW".}, while the addition ofthe time delay element z~' in the feedback path (Fig. 5) results in @ recurrent net- work, In fact, general recurrent networks can be con- structed composed of only the basic operations of 1) de lay, 2) summation, and 3) the nonlinear operator W,[.] In continuous-time networks, the delay operator is re- placed by an integrator. In some cases (asin (11) mul: tiplication by a constant is also allowed. Hence such net- ‘works are nonlinear feedback systems which consist only of elements Nj[.]. in addition to the usual operations found in linea systems. Since arbitrary linear time-invariant dynamical systems cean be constructed using the operations of summation, Fig 6. () Represeton 1b) Represent 2.) Repetto 3, ‘multiplication by a constant and time delay, the class of nonlinear dynamical systems that can be generated using generalized neural networks can be represented in terms of transfer matrices of linear systems i.e., W(z)] and nonlinear operators N[.J. Fig. 6 shows these operators connected in cascade and feedback in four configurations which represent the building blocks for more complex systems. The superscript notation is used in the figures to distinguish between different multilayer networks in any specific representation From the discussion of generalized neural networks, it follows that the mapping properties of N,[. and come: {quently N{. ](as defined in (10)) play a central roe in all, analytical studies of such networks. It has recently been shown in [21], using the Stone- Weierstrass theorem, that a two layer network with an arbitrarily large number of ‘nodes in the hidden layer can approximate any continuous function fe C(*, ©") over a compact subset of "This Provides the motivation to assume thatthe class of gen- eralized networks described is adequate to deal with a large class of problems in nonlinear systems theory. In fact, all the structures used in Section V and VI for the Construction of identification and controller models are ‘generalized neural networks and are closely related tothe configurations shown in Fig. 6. For ease of discussion in the rest ofthe paper, we shall denote the class of functions generated by a network containing N layers by the symbol TU aye Such a network has iy inputs, iy outputs and (iW ~"T) sets of nodes in the hidden layers, each con- taining i, f, © ++, jy nodes, respectively. IV. Back PROPAGATION IN STATIC AND D¥NaMic Systems In both static identification (c.g., pattem recognition) and dynamic system identification of the type treated in this paper, ifneural networks are used, the objective is to determine an adaptive algorithm or rule which adjusts the parameters of the network based on a given set of input— ‘output pais. Ifthe weights of the networks are considered as elements of a parameter vector 8, the learning process involves the determination of the vector 8* which opti 'mizes a performance function J based on the output exor. ‘Back propagation isthe most commonly used method forthis purpose in static contexts. The gradient of the per- formance function with respect to @ is computed as VJ and @ is adjusted along the negative gradient as 0 = yom — Vd |p ham where m, the step size, is suitably chosen constant and aon denotes the nominal value of 8 at which the gradient is Computed. In this section, a diagrammatic representa- tion of back propagation is first introduced. Following this, a method of extending this concept 1o dynamical sys- ‘ems is described and the term dynamic back propagation, is defined. Prescriptive methods for the adjustment of weight vectors are suggested which can be used in the identification and control problems ofthe type discussed in Sections V and VI Inthe early 1960's, when the adaptive identification and control of linear dynamical systems were extensively studied, sensitivity models were developed to generat the Pantal derivatives ofthe performance criteria with respect, to the adjustable parameters of the system, These models, were the first to use sensitivity methods for dynamical systems and provided a great deal of insight into the nec- essary adaptive system structure [22]-[25}. Since concep- tually the above problem is identical to that of determin: ing the parameters of neural neworks in identification and, ‘contol problems, itis clear that back-propagation can be extended to dynamical systems as well A. A Diagrammatic Representation of Back Propagation In tis section we introduce a diagrammatic represen- tation of back propagation. While the diagrammatic and algorithmic representation are informational) equiv lent, thei computational eficiency is different since the former preserves information about topological and geo- metic relations. In particular, the diagrammatic represen tation provides a better visual understanding ofthe entire process of back propagation, lends itself to modifications Which are computationally more efficient and suggests novel modifications of the existing structure to include other fonctional extensions Inthe thre layered network shown in Fig. 2,” {u, tu] denotes the input patter vector while y* Lois yo 27+ JD is the ouput vector, 2 2 [0 yy « epland 27 © [2 1%] ae the outputs at the fst and, the second hidden layers, respectively {w) byes {Wicdexpand {wh Jeg ae the weight matrices associated with thé three layers as shown in Fig. 2. The ecto De F7, Fe /" and Ye 2” are ax shown in Fig. 2 with 1(T,) = bs 424) = 2 and 765) = yr where F, sand J, ate elements of 0, 2 and respeatively. If yf = by ‘Yer "5 Ya] 8 the desited output vector, the out A crtor vector fora given input pattern w is defined as ¢ 1 ~ yu. The performance ctterion J is then defined as s-D4 here the summation i carried out over all pater ina Biven set 5. Ifthe input pattems ate assumed to be pre Sented at each instant of time. the performance criterion J may be interpreted as the sum squared error over an Jneral of time. Its this interpretation which is found to be relevant in dynamic systems. In the latter case, the inpets and ouputs are time sequences and the pet mance erteron J has the form (1/7) Bie,-rey es where Tis x suitably chosen integer. While stitly speaking the adjustment of the parame- ters should be carted out by determining the gradient of Jin parameter space. the procedure commonly followed isto adjust tat every instant based onthe eor at thal instant and a small step size. 1 represents typical parameter, 3¢/48, has to be determined to compute the [radient a e*(Be/@0,). The back propagation method is 4 convenient method of determining this gradient Fig. 7 shows the diagrammatic representation of back propagation forthe three layer network shown in Fig. 2 ‘The analytical method of deriving the gradient is wel ‘known inthe literature and will not be repeated here. Fi. T merely shows how the various components of the gr dient are realized. In our example, its seen that sighals tu, and z and 7 (D), 7 (2), and y°(5), a8 well a the efor vetor, are used inthe computation ofthe gradient (were (2) is the derivative of (x) with respect to), 4m, pa, and np muliplications are nesded to compute the paral derivatives with respect the elements of W", 1, and W respectively. The structure of the weight matrices in the network used to compute the derivatives is scen 0 ‘identical wo that in the original network while the signal flow i in the opposite direction, justifying the use ofthe term “back propagation.” For furher details regarding the diagrammatic representation, the reader is refered 10 {26} and [27]. The advantages of the diagrammatic rep: resentation mentioned earlier are evident from Fig. 7. “More relevant to our purpose is that the same represen- tation can be readily modified forthe dynamic ease. Tn fact, the diagrammatic representation was used exten: sively in all the simulation studies desribed in Sections Vand Vi B. Dynamic Back Propagation Ina causal dynamical system the change in a parameter at time k will produce a change in the output y(1) fo all. 1 = k, For example, given a noalinear dynamical system x(k +1) = (14), w(k), 8]: yD) = WEE) | where @ isa parameter, u is the input and x is the state vector {defined in (3), the partial derivative of y(k) with respect to 8 can be obtained by solving the linear state equations 2k + 1) = A(R)e(A) + 0(R), 2fb) = 0 wk) = CUE)2() (a2) where 2(k) ok ax(4)/00 € *, ACK) = (bk) € BCR) ©", WE) = dy(k) /20 © 5" and CR)Fig. 7. Aes or back ropagson. = ¥,(h) © BI". &, and, are Jacobian matrices and the vector #, represents the partial derivative of & with respect to 6. Equation (12) represents the linearized equa ‘ions of the nonlinear system around the nominal mec tory and input. If A(k), (hk), and C(k) can be com puted, wk), the partial derivative of y with respect 10 0 fan be obtained as the output of a dynamic sensitivity ‘model. In the previous section, generalized neural networks were defined and four representations of such networks with dynamical systems and multilayer neural networks Connected in series and feedback were shown in Fig. 6 Since complex. dynamical systems can be expressed in terms ofthese four representations, the back-propagation method can be extended to such systems ifthe paral de- rivative of the outputs with respect to the parameters can bbe determined for each of the representations. Inthe fol- lowing we indicate briefly how (12) ean be specialized to these four cases. In all cases it is assumed thatthe partial derivative of the output of a multilayer neural network with respect to one of the parameters can be computed using static back propagation and can be realized as the ‘output of the network in Fig. 7. Ih representation 1, the desired output y,(k) as well as the error e(k) © y(k) — yu(k) are functions of time. Representation 1 isthe simplest situation that can arise in dynamical systems. This is because elk) _ aylh) _ yy.) 90 4 a, where 6, atypical parameter of the network W. Since 407/94, can be computed at every instant using Static back propagation, de(k)/@d, canbe realized as the outpt ofa {Syaamiea! system W7(z) whose inputs ae the paral de rivatives generated. Tn representation 2, the determination of the gradient is rendered more complex bythe presence of neural network NIS1¢0, isa ypical parameter of N the partial desvative ‘e(k) /00, is computed by static back propagation. How. ever, iis a typical parameter of N’ ay, 35 ~ Fan, 8; Since 3v/29, can be computed using the method de setibed in representation | and dy, /30 can be obrained by static back propagation, the product of the two yield the partial derivative of the signal y, with respect to the pa rameter 8, Representation 3 shows a neural network connected in feedback with a transfer matrix W(2). The input to the ‘nonlinear feedback system isa vector u(k). If, isa typ- jal parameter of the neural network, the aim isto deter- rine the derivatives 8y,(k) /86, fori'= 1,2, +++ , mand all k = 0, We observe here for the first time a situation ‘not encountered earlier, in that 8y,()/89, i the solution of a difference equation, ic., ,(k) /30, is affected by its fon past values (3) ANL0] yyy B® , NLU aw) 39, + a9, In (13), 9/3, a vector and ON ]/Ov and aN 01/36, are the Jacobian matrix and a vector, respectively. which ft evaluate around the nominal aectory. Hence it rep- tesens linearized diference equation inthe variables 3, /30,. Since 9N[0}/80 and NL 01/39, ean be com puted at every instant of time, the desired partial deriva- tives can be generated a the output of a dynamical system shown in Fig. 8a) (the bar notation 4y/ 0 suse in (13) to distinguish between ay/ 30, and aN{ 0/36). Inthe final representation, the feedback system is pre- ceded by a neural network N The presence of N* does not afect the computation of the partial derivatives ofthe Output with respect tothe parameters of N"- However, if 6, is atypical parameter of Nit ean be showin that dy/20, can be abained as By _ aN! fo} | aN"[u] 0 we | +m or alternately it can be represented a the output of the ‘dynamical system shown in Fig. 8(b) whose inputs can be ‘computed at every instant of time. Tall the problems of identification and control that we will be concerned with in the following sections, the ma- trix W(z) is diagonal and consists only of elements ofthe form 2" (ie.. delay of d, units). Funher since dynamic ‘back propagation is considerably more involved than static back propagation, the structure of the identification ‘models is chosen, wherever possible, so thatthe latter ean be used. The models of back propagation developed here can be applied to general control problems where neural ‘networks and linear dynamical systems are interconnected jn arbitrary configurations and where static back propa {gation cannot be justified. For further details the reader isrcv in epesematon referred to [27]. A paper based on [27] but providing de- tails concerning the implementation of the algorithms in practical applications is currently under preparation, V. IoexnricaTion [As mentioned in Section II, the ability of neural net works to approximate large classes of nonlinear Functions sulficienty accurately make them prime candidates for use in dynamic models for the representation of nonlinear plants. The fat that static and dynamic back-propagation methods, as described in Section IV, can be used for the adjustment of their parameters also makes them attractive in identifiers and controller. In this section four models. forthe representation of SISO plants are introduced which can also be generalized to the multivariable case. Follow- ing thi, identification models are suggested containing ‘multilayer neural neqworks as subsystems, These models, ate motivated by the models which have been used in the adaptive systems literature for the identification and con- ‘tol of linear systems and can be considered as their gen= cralization to nonlinear systems, A. Characterization ‘The four models of discrete-time plants introduced here can be described by the following nonlinear difference equations Model I: y,(k + 1) Bean + elu(t),u(k 1) alk ms 1) Model I: ys(k + 1) Arp pk De ap = 1] +B ame - 0) Model I: y,(k + 1) = FU yl = De yyy (k= +] + alu(8), wk 1). alk =m 1] Model 1V: y(k + 1) Flys. Ip gpl = + a(k) uk = 1). 0+ alk =m + 1)) (a) where [1(k), yp(K)] represents the input-output pair of the SISO plant at time k, and m =n. The block diagram representation of the various models are shown in Fig. 9. ‘The functions f:5" + in Models Mand Il and JoE8"" 5) i Model IV, and g: 5 — Fin (14) are ‘assumed tobe differentiable functions of their arguments Inall the four models, the output of the plant atthe time + 1 depends both on its past m values'yp(k 1) (i = 0,1, +m = 1) as well as the past m values of the input u(k ~ j )(J = 0, 1,+** m1), The dependence fon the past values y,(K — /) is linear in Model I while in Mode Ul the dependence on the past values of the input 1u(K = j )is assumed to be linear. In Mode! III the non linear dependence of y,(k + 1) on y,(k ~ {) and w(k ~ 4s assumed to be separable. IL is evident that Model IV in which y,(k + 1) isa nonlinear funetion of y,(& ~ i) and u(k — j) subsumes Models I-Il. If @ general non linear SISO plant can be described by an equation of the {orm (3) and satisfies the stringent observability condition, discussed in Section I-C-4, itcan be represented by such 4 model. In spite of its generality, Model IV is, however, analytically the least tractable and hence for practical ap” plications Some of the other models are found to be more attractive. For example, as will be apparent in the fllow- ing section, Model ITs particularly suited for the contol problem. From the results given in Section II, it follows that under fairly weak conditions on the function f andor gin (14), mutayer neural networks can be consructed to ap proximate such mappings over compact sts, We shall a sume for convenience that f and/or g belong to a known, 1486 IU. e4, im the domain of interest, so thatthe plant cam be represented by a generalized neural network as discussed in Section II. This assumption motivates the ‘choice of the identification models and allows the state ‘ment of well posed identification problems. In panticula the identification models have the same structure as the plant but contain neural networks with adjustable param: Leta nonlinear dynamic plant be represented by one of the four models described in (14). If such a plant isto be ‘identified using input-output data, it must be further as sumed that it has bounded outputs for the class of per- ‘missible inputs. This implies that the model chosen to represent the plant also enjoys this propery. In the case ‘of Model I tis implies thatthe roots ofthe characteristic equation 2° — agi?! = += = gaz = ay = Olle inthe interior of the unit cirele. In the other three cases no such simple algebraic conditions exist, Hence the study of the stability properties of recurrent networks contain- ing multilayer networks represents an important area of researchi. 9 Repent HD pan.) Ma Ml 0 Mo “The models described thus fa are for the representation of discrete-time plants. Continuous-time analogs ofthese ‘models can be described by differemial equations, 3s Stated in Section II. While we shall deal exclusively with discrete-time systems, the same methods also carry over to the continuous time case B. Identification ‘The problem of identification, as described in Section UC, consists of setting up a suitably parameterized iden- tification model and adjusting the parameters of the model to optimize performance function based on the error be- tween the plant and the identification model outputs. Since the nonlinear functions inthe representation of the plant are assumed to belong to.a known class 3U yy, i the domain of interest, the structure ofthe identification riodel is chosen to be identical to that of the plant. By assumption, weight matrices ofthe neural networks inthe identification model exist so that, forthe same initial con- ditions, both plant and model have the same output for ‘any specified input. Hence the identification procedure consists in adjusting the parameters of the neural net- ‘works in the model using the method described in Section TV based on the error between the plant and model out puts, However, as shown in what follows, suitable pre Cautions have to be taken to ensure thatthe procedure re- sus in convergence ofthe identification model parameters to their desired values. 1. Parallel Hdentfication Model: Fig. 10(a) shows a plant which can be represented by Model I with n = 2 tnd m = I. To identify the plant one can assume the structure of the identification model shown in Fig, 10(a) and described by the equation Spl +1) ioSA(K) + &x5y(K — 1) + N[a(AD][As mentioned in Section H-C-3, this is referred to as a parallel model. Identification then involves the estimation ofthe parameters das well as the weights ofthe neural network using dynamic back propagation based on the er- ror e(K) between the model output §,(k) and the actual ‘ouput ¥, (4). From’ the assumptions made eaflier, the plant is bounded-input bounded-output (BIBO) stable in the pres fence of an input (in the assumed class). Hence, all the signals in the plant are uniformly bounded, In contrast to this, the stability of the identification model as described here with a neural network cannot be assured and has to be proved. Hence if a parallel model is used, thee is no {guarantee that the parameters will converge of thatthe ‘output error will tend to zero, In spite of two decades of ‘work, conditions under which the parallel model param- ters will converge even inthe linear case are at present unknown. Hence, for plant representations using Models, T-V, the following identification model, known as the series-parallel model, is used. 2. Series-Parallel ‘Model: In contrast t0 the parallel ‘model described above, in the seres-parallel model the ‘output ofthe plant (rather than the identification model) is fed back into the identification model as shown in Fi, 10(b). This implies that in this case the identification model has the form Sk +1 Garp (k) + aayp(k = 1) + N[u(K)] ‘We shall use the same procedure with all the four models described earlier. The series-parallel identification model, corresponding to plant represented by Model IV has the form shown in Fig. 11. TDL in Fig, 11 denotes a tapped delay Tine whose outpot vector has for its elements the delayed values ofthe input signal. Hence the past values ‘of the input and the output of the plant form the input vector to a neural network whose output J,(k) core sponds to the estimate of the plant output at any instant ‘of time . The seriespaallel model enjoys several advan- sin a Fig 1, Idemifeaion of soln lets wing merlot tages over the parallel model. Since the plant is assumed, to be BIBO stable, all the signals used inthe identification procedure (i.e.. inputs to the neural networks) are hounded. Further, since no feedback loop exists in the ‘model, static back propagation can be used 10 adjust the parameters reducing the computational overhead substan- Aially. Finally, assuming that the output error tends to a small value asymptotically that yp (Kk) 5y(), the se- ries-parallel model may be replaced by a parallel model without serious consequences. This has practical impl cations if the identification model is to be used off line, In view of the above considerations the series parallel ‘model is used in all the simulations in this paper. . Simulation Results In this section simulation results of nonlinear plant identification using the models suggested earlier are pre- sented. Six examples are presented where the prior infor- mation available dictates the choice of one of the Models I-IV. Each example is chosen to emphasize a specific point. In the first five examples, the seres-paralel model is used to identify the given plant and static back-propa- gation is used to adjust parameters ofthe neural networks. AA final example is used to indicate how dynamic back propagation may be used in identification problems. Due to space limitations, only the principal results are pre- sented here, The reader interested in further details is re ferred to (27]-(29}.iH e nets Fig, 12, Example: (2) Ops ofthat and enifiction model whee Sagan tp a 0") pn pa at oon 1. Example 1: The plant to be identified is governed by the difference equation JHE + 1) = 0.35,(4) + 0.6y,(8 ~ 1) + Flak] (as) ‘where the unknown function has the form fu) = 0.6 sin (ae) + 0.3 sin (3xu) + 0.1 sin (xu), From (13), itis clear thatthe unforced linear system is asymptotically sta ble and hence any bounded input results ina bounded out Dut In onder to identity the plant, «series parallel model sovemed by the difference equation Spl + 1) = O.39(K) + 0.64(k ~ 1) + N[u(K)] was used. The weighs in the neural network were ad justed at every instant of time (7, ~ 1) using static back propagation. ‘The neural network belonged to the class ‘TTisosos and the gradient method employed a step size of = 0.25. The input wo the plat and the model wes a sinusoid w(K) = sin (2xk/280). As seen from Fig. 12(). the output of the model follows the output of the plant almost immediately but fil to do so when the adaptation process is stopped at k = $00, indicating tat the ident- Fiation of the plant is not complete. Hence the ienifi- cation procedure was continued for 50 000 time steps Using a random input whose amplitude was uniformly dis: tnbuted in the interval [1,1] atthe end of which the Adaptation was terminated, Fig, 12(6) shows the outputs ‘ofthe plant and the trained model. The nonlinear function in the plant inthis eae is fun] = w+ 0.34" = 0.4u ‘As can be seen from the figure, the idemication eror is ‘small even when the input i changed to a sum of two sinusoids u(k) = sin (2k /280) + sin (224/25) atk 250, 2. Example 2: ‘The plant to be identified is described by the second-order diference equation alk +1) = FL aC). p(k = 1D] + wk where FL%(). (k= 0) yolk) yp ~ 1) Lyp(k) + 2.5] T+ 908 + ye) (16) ‘This corresponds to Model Il, A series-parallel identifier of the type discussed earlier is used to identify the pla from input-output data and is described by the equation Spl +1) = WL y6(4), 99k — D)] + a(R) 7) whore Nis neural network with N e903. The iden tiation process involves the adjustment of the weights (of N using back propagation Some prior information concerning the input-output boehavior ofthe plant is needed before identification can be undertaken, This includes the number of equilibrium states of the unforced system and thir stability proper: ties, the compact set ‘U to which the input belongs and ‘whether the plant output is bounded for this elas of in- puts, Alb itis assumed thatthe mapping N can approx: imate fover the desired domain 4. Equilibrium states of the unforced system: The ‘equilibrium states of the unforced system yp(k +1) = $13, (4), ¥pk ~ 1)] with fas defined in (16) are (0, 0) fand (2, 3)/ respectively, inthe state space, This implies that while in equilbeium without an input, the ouput of the plant is either the sequence {0} or the sequence {2}. Further, forany input | w(4)| = 5, te output ofthe plant {s uniformly bounded for intial conditions (0, 0) and (2, 2) and satisfies the inequality | y9 (4) = 13. ‘Assuming different intial conditions in the state space and with zero input, the weights of the neural network ‘were adjusted so thai the error e(k-+ 1) = yy(k + 1) — MLop(k). 9y(b — 1D) is minimized. When the weights converged to constant values, the equation J, (k 1) = NL 5, Ck), 5y(k— 1)] was simulated fr ini conditions Within a radius of 4. "The identified system was found to have the same trajectories asthe plat forthe same iit conditions. The behavior of the plant andthe identied mode! for different initial conditions are shown in Fig. 13. Temust be emphasized here that im practice the initial con ditions of the plant cannot be chosen atthe diseretion of the designer and must be relized only by using diferent inputs tothe plan. , Identification: While the neural network realized above ean be used in the identification model, a separateFig. 13. Example 2: Behavior of he unos system. a) Asta am simulation was carted out using both inputs and outputs and a series-parallel model. The input w(K) was assumed. {o be an i.i.d, random signal uniformly distributed in the interval [~2, 2] and a step sizeof 9 = 0.25 was used in the gradient method. The weights in the neural network ‘were adjusted at intervals of five steps using the gradient of Ei ,-ce°(J). Fig. 14 shows the outputs ofthe plant land the model after the identification procedure was te minated at k = 100 000, 3. Example 3: In Example 2, the input is seen to occur linearly in the difference equation describing the plant. In this example the plant is described by Model IIT and has the form, vel) 1+ 9p(k) which corresponds to fL.¥9(4)] = yy(k)/C1 + ya(K)?) and g[u(k)] = 0 (A) in (14). A seres-paralel identifi cation model consists of two neural networks Ny and Ny belonging to 9U my ,1 and can be described by the diffe ng an, ence equation Sok +1) = NL 9909] + NelwO)) ‘The estimates fand g are obtained by using neural net- works Nand N,. The weights in the neural networks were adjusted at every instant of time using a step size of 9 = 1.1 and was continued for 100 000 time steps. Since the {input was a random input in interval [~2, 2]. & approx- imates g only over this interval. Since this in turn results in the variation of yp over the interval (~10, 10], f ap- proximates f over the later interval. The functions f and as well as f and g over their respective domains are shown in Fig. 15(a) and (b). In Fig, 15(c), the outputs of the plant as well a the identification model for an input u(k) = sin (2xk/28) + sin (2rk/10) are shown and are seen tobe indistinguishable 4. Exanple 4: The same methods used for identifica tion of plants in examples 1-3 can be used when the un- known plants are known to belong to Model IV. In this yk +w) Fig 4 Example 2: Oupus fhe plat and he idenifenton mode example, the plant is assumed to be of the form sok +1) =F[%9(8) 99k = Ds gk = 2), w(k), w(k = 1D] where the unknown function f has the form anisms ~ 1) + e429] = EAL — Tt In the dentifegtion mode, a neural network N belonging ‘to the class 93.20, 10,1 is used to approximate the function {Fig 16 shows tbe ouput ofthe plant and the mote ‘rhe ‘theidelifeaion procedure Was cated out for 100 00 steps using a random input ial uniformly die tre inthe imenal [=r Tfand sep ie of > 1.25. As mentioned ear, during the ideation po- ese sence parle) model i wed, ot after the et fenton proces is teminatd. he performance of the tovel is ude sing paral move In Fg. 16, the Input tothe plant andthe dented rode ven by wd) = sin (24k/250) for kz 500 and w(k) = 0.8 in (3ek/250 0.3 sin (20k /25) fork > 50, "Example 3: tn ths ecample i shown thatthe samme methods used to det S190 plans ea be sed 10eaftlole y/(er*) f and it cxtimate Ny + + aul and ite etimate N, + Fig 18. Etamle 3: Plt of he nts fad J) Pats of te "acns gf.) Op of te la se ean mae wand iy Fie. 16, Example 4 enisto f Model V identify MIMO plants as well. The plant is described by the equations salt) yak +] _ | Et) al peas al- yult rat) |* La) 1+ ypalk) ‘This corresponds to the multivariable version of Model IL. The series-paralel identification model consists of wo. neural networks N" and N* and is described by the equa tion sulk +1) [eae ato) [ne | Spalk +1) ~ LLG). yt] * Lanta. ‘The idemifcation procedure was carried ou fr 100 000 time steps using a step size of» = 0.1 with random inputs ty (and ug (X) uniformly distbuted in the interval 1, 1}. The responses ofthe plant and the identifeation mode for a vector input {sin (224/25), cos (2rE/25)]! are shown in Fig. 17 ‘Comment: In examples 1, 3.4, and 5 the adjustment ofthe parameters was carried ou by computing the gra dient of e*(k) at instant & while in example 2 adjustments ‘were based on the gradient of an eror function evaluated over an interval of length 5. While fom a theoretical point of view itis preferable to use a larger interval to define the enor function, very litle improvement was observed inthe simulations. This accounts forthe fact that in ex- amples 3,4, and 5 adjustments were based on the iosan- taneous rather than an average eror signal ‘6. Example 6: In examples 1-5, aseries-paralleliden- tfcation model was used and hence the parameters ofthe neural networks were adjusted using the Satie hack-prop- gation method. In this example, we consider simpleo Pig. 17. Example 5: Response ofthe pt an the enicton mode iy » hhh ai [| HV | \| | | Fig. 1, ale 6: 0) Ong of lam eteaon model fe ‘edWial rae lth first order nonlinear system which is identified using the dynamic back-propagation method discussed in Section TV. The nonlinear plant is described by the difference equation spk +1) = 0.8, (8) + F400] where the function fi] = (u ~ 0.8)a(u + 0.5) i un Known, However, i is assumed that f ean be approxi ated tothe desired degree of accuracy by a multilayer neural network “The identification model used is described by the dif ference equation p(k +1) = 0.85,(K) + MUD] and the neural network belonged t0 the clas 9800 “The model chosen coresponds to representation | in Se tion IV {refer to Fig. 6@). The objective isto adjust a total of 261 weighs in the neural network so that ¢(K) 2 9,(k) = yp(k) 0 asymptotically. Defining the per- formance enerion to be minimized as J = (1/27) Ehjurss e1(/). the partial derivative of J with respect toa weight 6, inthe neural network can be computed as (21/28,) = (A/T) Eheraras eC) (Be(4)/38,). The ‘quantity (9e(7)/04, canbe computed ina dynamic fash jon using the method discussed in Section IV and used in the following rule to update 8 deli) kT ok-TH1- eli) Trdion where i the step size in the gradient procedure, Fig, 18(a) shows the outputs ofthe plant and the iden tification model when the weights in the neural network were adjusted after an interval of 10 time steps using a step size of = 0.01. The input to the plant (and the model) was u(k) = sin (2rk/25). In Fig. 18(b), the function f(1) = (u ~ 0.8)u(u + 0.5), as well as the function realized by the three layer neural network after 50 000 steps for w © [=1. 1], are shown. As seen from the figure, the neural network approximates the given function quite accurately VI. Costnot oF DYNAMICAL SYSTEMS 'As mentioned in Section Tl forthe sake of mathemat ‘cal tractability most ofthe effort during the past two dec fades in the model reference adaptive contol theory has been directed towards the control of linear time-invariant plants with unknown parameters. Much of the theoretical Work inthe late 1970's was aimed at determining adaptivelaws for the adjustment of the control parameter vector Ck) which would result in stable overall systems, In 1980 (30]-(33}, it was conclusively shown that for both dis- erete-time and continuous-time systems, such. stable adaptive laws could be determined provided that some prior information concerning the plant transfer function ‘was available. Since that time much ofthe esearch inthe area has been directed towards determining. conditions whieh assure the robusiness of the overall system under diferent types of perturbations. In contrast to the above, very little work has been re= Ported on the global adaptive contro of plants described by nonlinear difference or differentia equations. It is the control of such systems that we are primarily inter ested in this section. Since, in most problems, very little theory exists to guide the analysis, one of the aims is to indicate precisely how the nonlinear control problem dif. fers from the linear one and the nature ofthe theoretical ‘questions that have to be answered. Algebraic and Analytic Pars of Adaptive Control Prob- ems: In conventional adaptive control theory, two stages are generally distinguished in the adaptive process. Inthe first, referred to as the algebraic part, itis frst shown thatthe controller has the necessary degrees of freedom toachieve the desied objective. More precisely, if some prior information regarding the plant is given, its shown, that a controller parameter vector 8* exists for every valve of the plant parameter vector p, so thatthe output of the controlled plant together with the controler approaches the output ofthe reference model asymptotically. The an- lytic part of the problem is then to determine stable adaptive laws for adjusting 6() so that lim. @(k) = {9 and the output error tends to 2er0 Direct and Indirect Control: For over 20 years, two distinct approaches have been used [1] to control a plant adaptively. These are 1) direct conirol and 2) indirect onirol. In direct control, the parameters ofthe controler are directly adjusted to reduce some norm of the output error, In indirect contol, the parameters of the plant are ‘estimated asthe elements of a vector (XK) at any instant ‘kand the parameter vector (k) ofthe controler is chosen assuming that (A) represents the tre value p ofthe plant parameter vector. Even when the plant is assumed to be linear and time invariant, both direct and indirect adap- tive control result in overall nonlinear systems. Figs. 19 and 20 represent the structure of the overall adaptive sys- {em using the two methods forthe adaptive control of a linear time-invariant plant (1) A. Adaptive Control of Nonlinear Systems Using Newral Networks For a detailed treatment of direct and indirect control systems the reader is referred to [1]. The same approaches Which have proved successful fr linear plants can also be attempted when nonlinear plants have to be adaptively controlled, The structure used for the identification model as well as the controller are strongly motivated by those ig 19. Di apie cont a Fi 20, Indiet apne son used in the linear ease, However, in place of the linear agains, nonlinear neural networks are used Methods for identifying nonlinear plants using delayed values of both plant input and output were discussed in the previous section and Fig. 11 shows a general ident fication model. Fig. 21 shows a controller whose output is the contol input to the plant and whose inputs are the 1). Once the plant has been identified tothe desired level of accuracy, control action can be initiated so that the output ‘ofthe plant follows the output ofa stable reference model must be emphasized that even ifthe plant has bounded. ‘outputs for bounded inputs, feedback control may result {in unbounded solutions. Hence, for on-line contro iden- tification and control must proceed simultaneously. The time intervals T, and T., respectively, over which the identification and control parameters are to be updated hhave to be judiciously chosen in such a case Five examples, in which nonlinear plants are adaptively controlled, ate included below and illustrate the ideas dis- ‘cused eatlier. As inthe previous section, each example is chosen to emphasize a specific point 1. Example 7: We consider here the problem of con- twolling the plant discussed in example 2 which is de- seribed by the difference equation y+ 1) =F [vole yolk — D] + WK) where the function Flop). yolk = DY kook — Loglk) +25 = Wk — DL yl) +25] (yg) T+ ya(k) + ype — 1) is assumed to be unknown. A reference model is de- Serbed by the second-order diference equation Ya(k + 1) = O.6yq(k) + O.2¥—(k — 1) + r(k) where r(A) is a bounded reference input. Ifthe ouput error e,(k) is defined as e-(k) © yp() — yn ()s the aim ‘ot controls 10 determines bounded conta input w(K) such that lima e,(4) = 0, Ite function 1.1 ia (19) is Known, it follows direty that at stage k,w(X) can be Computed from a knowledge of j,(k) and its past values u(k) FL p(s yolk ~ 1)] + 0.639(8) + 0.2y4(k = 1) + r(R) (20) resulting in the error difference equation e,(k + 1 0.6e-(k) + 0.2e,(k — 1). Since the reference model is asymptotically stable, it follows that fim e-() = 0 for arbitrary inital conditions. However, since. is un~ known, it i estimated on line as fas discussed in example using # neural network N and the series-parallel method. ‘The control input to the plant at any instant k is com- puted using {in pace of fas (k) = NL ¥9(H). 99k = 1D] + 0-648) 4 O.2y(k = 1) + rk) (ay ‘This results i the nonlinear difference equation afk +1) = FL yO). 398 — D] — NLC), 99k =) + 0.6y,(k) + 0.2y,(k = 1) + (A) (22) goveming the behavior of the plant. The structure of the Overall system is shown in Fig. 23. In the first stage, the unknown plant was identified off line using random inputs as described in example 2. Fol- lowing this, 21) was used to generate the control input. ‘The response of the controlled system with a reference inpot r(k) = sin (2xk/25) is shown in Fig, 24(b). In the second stage, both identification and control were implemented simultaneously using different values of 7, and T., The asymptotie response of the system when iden- Uiication and contol start at k= O with T= T. = Lis shown in Fig. 25(a). Since itis desirable to adjust the ‘contol parameters a a slower rate than the identification ‘parameter, the experiment was repeated with T; = 1 and T= 3and is shown in Fig. 25(). Since the identification process is not complete for small values of k, the control ean be theoretically unstable. However, this was not ob- ‘served inthe simulations. Ifthe contro is initiated a ime[NARENDRA AND PARTHASARATAY: IDENTIFICATION AND CONTROL OF DYNAMICAL SYSTEMS x Fig 23, Example 7: Since of he vel stem, H\ f\ ie. 2, Etample 7) Response fr no cant aan () Reon or PDS 25) ith coat — eaten) @ o 25 uml) Rape en nnd att = 0 wh = Ousing nominal values ofthe parameters ofthe neural The simulations reported above indicate that for stable network with T, 10, the output ofthe plant was and eficient on-line contro, the idemtification must be su seen to increase in an unbounded fashion as showin in Fig. ficiently accurate before control action is initiated and 26. hence 7; and 7, should be chosen with care.LLY Pip. 26 Etample 7: Repose when com is nine a k = 0 ih 2. Example 8: The unknown plant in this case corre- sponds to Model II and can be described by a difference equation of the form yk +1) =F Uv. p(k =D). oo yp = + 1] + Zouk. msn I is assumed that the parameters 8,(j = 0, 1, +++, m = 1) are unknown, but that By is nonzero with a known sign. The specific plant used in the simulation study was Syp(K)yp(k = 1) T3 3H) + 31) 2) + u(k) + 08u(k ~ 1) (23) ok +1) ‘The output of the stable reference model is described by Yolk + 1) = 0.22yq(K) + O.64yQ(K — 1) = 05 yq(k — 2) + r(8) where r is the uniformly bounded reference input. The responses ofthe reference model andthe plant when r(k) = u(k) = sin (22k /25) are shown in Fig. 27. While the ‘output of the reference model is also a sinusoid of the same frequency, the response of the plant is seen to con- tain higher harmonies. It i assumed that sgn Bo =F land that Bp = 0.1. This enables a projection type algo rithm tobe used inthe identification procedure so thatthe estimate By of By satisfies the inequality By = 0.1, The Control input any instant of time fis generated as (8) = 5 [ALC p(k = 1.29] = Byu(k ~ 1) + 0.32y,(k) + O.64y,(k ~ 1) = O.Syp(k ~ 2) + r(8)] (24) In Fig. 28, the plant i identified over a period of 50 000 time steps using an input which is random and distributed ig. 27. Example Repos of te eee mel anh plan when uniformly over the interval [-2, 2]. AC the end of this, Interval, the control is implemented as given in (24). The response of the plant as well as the reference model are shown in Fig. 28. In Fig. 28a) the reference input is r(k) in (2k /25), while in Fig. 28(b) the reference input is r(R) = sin (2xk/25) + sin (2zk/10). In both cases the control system is found to perform satisfactorily. Since the plant is identified over a sufcienty long time with a ‘eneral input, the parameters Bp and 8 are found to con- verge to 1.005 and 0.8023, respectively, which are close to the true values of 1 and 0.8. In Fig. 29 the response ofthe controlled plant to ref erence input r(k) = sin (22k /25) is shown, when iden- {ication and control are initiated at k = 0. Since the input is no suficiently general, Bo() and By (k) tend to values 4.71 and 3.59 50 that the asymptote values ofthe param: eer errors are large. In spite ofthis, the ouput ror is, seen to tend to zero for values of k greater than 9900. This, example reveals that good control may be possible with: ‘out good parameter identification 3. Example 9: In this case, the plant is described by the same equation as in 23) with 0.84(k — 1) replaced by 1-lu(k ~ 1) and the same procedure is adopted as in example 8 t0 generate the contro input. It is found that the output error is bounded and even tends to zero while the control input grows in an unbounded fashion (Fig. 30), This is a phenomenon which is well known in adaptive ‘control theory and arises due to the presence of zeros of the plant transfer function lying outside the unit circle. In the present context u(k) + I.lu(k~ 1) ean be zero even as u(k) = (~I.1)! tends to & in an oscillatory fashion ‘The same phenomenon can also occur in systems where the dependence of yon w in nonlinear. 4. Example 10: The control ofthe nonlinear multivari able plant with two inputs and two outputs, discussed in ‘example 5, is considered in this example and the plant is sescribed by (18). The reference model is linear and is described by the difference equations pes? | [os Seip] [| swt +1 Lor -08!Lsate] * LaceyWhere rj und ry ate bounded reference inputs. The plant is identified as in example 5 and control is initiated afer the identification process is complete. The responses of the plant as compared tothe reference model forthe sane inputs are shown in Fig. 31. The improvement in the te ponses, when the neural networks in the identification ‘model are used {0 generate the control input tothe plant are evident from the Figure. The outputs of the controled plant and the reference model are shown and indicate that, ‘the output err is almost zero 5. Exanple 11: In examples 7-10, the output of the plant depends Tinearly on the control input. This makes ‘the computation ofthe later relatively straightforward, In this example the plant is described by Model Il and hasfren eayts |. reconleneas), wor 7 ry a wo = rnd “ | /\ a i \ z. laeth NAN AY the form yp(k +1) wl) ayy Tr ath) which was identified successfully in example 3. Choosing the reference model as yal + 1) = O.6yq(K) + F(R) the aim once again is to choose u(k) 50 that tim. L¥s(40 = yq (ADI = 0 IEF Lye] = g/l + 99) and gC] 'v, the control input in thi case f chosen as a u(k) AL yp ()] + 0.6y,(4) + r(K)] (25) a ey where f and g°" are the estimates of f and g~'. respec- ‘Fig. 32. Example 11; Structte of the overall system, tively. The estimates f and g are oblained as deserted carer using neural networks Nyand Ny. Since (a has been realized asthe output of a neural network N, the The determination of N, was carried out over 25 000 time weights ofa neural network N.€ 50} yq (Shown in Fig. steps using a random input uniformly distributed in the 32) can be adjusted so that N,{N.(7)] = ras r(k) varies interval {~4, 4] and a step size of » = 0.01. Since the over the interval (~4, 4]. The range ~4, 4] was chosen plant nonlinearities fand g as well as ¢~' have been es- for r(d) since this assures thatthe Input to the Menil~ timated using neural networks Nj, Nyx and N, respec cation model varies over the same range for which the tively. the contol input tothe plant can be determined estimates fand gare vali. In Fig. 33,N,L(7)] is plots using (28). The outpit of the plant to a reference input ted against rand is seen to be unity overthe enti range. r(k) = sin (2xk/25) + sin (2zk/10) i shown in FigTeter) | sd) tad ry ig. 3, Example: Outputs fhe elrence model and plat wane 1 etc conor (0) up fe ference ode a pla with ‘eehack sn 34(a) when a feedback controller is not used the response. with @ controller is shown in Fig. 34(6). The response in Fig. 34(b) is identical to that ofthe reference model and is almost indistinguishable from it. Hence, from this ex- ample we conclude that it may be possible in some cases, to generate a control input to an unknown plant so that almost perfect model following is achieved, VIL. Comments ano Conctusions: In this paper models for the identification and control ‘of nonlinear dynamic systems are suggested. These models, which include multilayer neural networks as well as linear dynamics, can be viewed as generalized neural networks. Inthe specific models given, the delayed values, of relevant signals inthe system are used as inputs to mul Uilayer neural networks. Methods for the adjustment of Parameters in generalized neural networks are treated and the concept of dynamic back propagation is introduced in this context to generate partial derivatives of a perfor- mance index with respect to adjustable parameters on lin. However, in many identifiers and controllers itis shown that by using series parallel model, the gradient can be ‘oblained with the simpler static back-propagation method. The simulation studies on low order nonlinear dynamic systems reveal that identification and control using the ‘methods suggested can be very effective. There is every ‘eason to believe thatthe same methods can also be used successfully for the identification and control of multiva lable systems of higher dimensions, Hence, they should find wide application in many areas of applied science. Several assumptions were made conceming the plant ‘characteristics in the simulation studies to achieve satis- factory identification and control. For example, in all cases the plant was assumed to have bounded outputs for the class of inputs specified. An obvious and important extension of the methods in the future willbe to the eon trol of unstable systems in some compact domain in the state space. All the plants were also assumed to be of re: ive degree unity (i.c., input atk affects the output at + 1), minimum phase (i. no unbounded input les in the mul space of the operator representing the plant) and Models II and Il used in conttol problems assumed that, inverses of operators existed and could be approximate. Future work will attempt to relax some oF all of these assumptions. Further, in all cases the gradient method is used exclusively forthe adjustment of the parameters of the plant. Since it is well Known that such methods canlead to instability for large values of the step size n, i is essential that efforts be directed towards determining sta ble adaptive laws for adjusting the parameters. Such work is currently in progress. ‘A number of assumptions were made throughout the pa- per regarding the plant to be controlled for the methods to prove successful. These include stability propenies of recurrent networks with multilayer neural networks inthe forward path, controllability, observabilty, and idemtfi- ability of the models suggested as well asthe existence of nonlinear controllers to match the response of the refer- lence model. At the present stage of development of non- linear control theory, few constructive methods exist for checking the validity ofthese assumptions in the context of general nonlinear systems. However, the fact that we are dealing with special classes of systems represented by generalized neural networks should make the develop” ‘ment of such methods more traciable. Hence, concurrent ‘theoretical research in these areas is needed to justify the ‘models suggested in this paper, AcKNOWLEDOMENT ‘The authors would like to thank the reviewers and the associate editor for their careful reading of the paper and their helpful comments. RePeRENCES [UT KS. Narnia and ALM. Annawan, Sable Adapive Stems Englewood Clie I: Pence al 18 (2) D'or, tenets onset et tsngison of spoken ant tienen" TERE rms ert Speck, Sin Prensa. 0 Setno pp. ete ly 188 1s) RP Goran ad J Seca "Lenn ctscabon of sonar {args tings muse pal ren IEEE Tran Aw 14) 171 Sujnowsh and CR: Rosenberg, “Pall pets hat ey te poounce Eapish text amples St wae Tr Fp. 15-168, 15) B Widow, RG. Winer, an R.A Bair, “Layer eta ets Ie patito aE Ton ch, Sa! Pro 161 1.1 Hopbeld, "Nesrai aetworks ane physical systems with evergect pro Na head So 0890) ‘Gert Nol 32 pp iste 1) D.W. Tank and 3.5. Hope stem: An AD convener gal Scion Sess net po [ring cei EEE Try Spat Man C3bcms el CAS sovsTyp SoS, iy 1986 (9) H Ratt and F Winter “New ewan for ong comma esto ie toy NT ir an, Svan, el Aa 111 GG 5, Neen, “ein fo Hien Ch tech ep €T.38 Apr 971s ako soma, ow 112} Une an. Sides, Thor and Praca Recrelen ‘ieaon."Cambge MA MELT Pree. 198 ua, SN Sigh and We. Hugh "Decoupling class of nonlinear 11) Ean Te ct of ep omy, EE Col Spar Mag ol 8002-99 26°30. 118] A. uid. A. Xveser, Gon Glo a. Momo. “Noni ‘comping via edback A iret geomet aprach EEE 1161 $5 Sng and Ando, Adan como fea 9 tena. TREE Tran. ama Com ol th 90 Te pp U7] Fide "Genealain of bask propapton wo weuet wots" P Reset vl 9,019; pp. 2239-2238 Now 198. us) HL AUN Mice! aad. Prod." Gaetve aaa yn Theis ofa cin of man weeks: EEE Pn Coea Sl 1091 Kon, Shen waite memo, Man Cert wl Sv a0 ps9 60 anes 988 (20) K°3" Naren ad Ke Partai, New eto and yea Salsyatina. Pan 1A pment apron 1 Hops sewer emer Sy Sie Dep tal Eg Ya Une, New 1211 Homi M: Suhel He Whe, “Maer odfrwan ‘ewons a talesal spposinaac Dep, Goran, Un {F Calfoma Sn Diet CA tscuson pap, Dep Booms 22) RS Narendra and Mori, Multipart set ona NC pp 13h tay Fe “Ma of vy gi he vegan {LFS pp SIS 1St8 10h (0 Le ae tdi, “Opinii fea (2518 Ga Ed, Sem Stn Aa echar Popes Bowden, Huhiso ad ons. 1058 : (26) B'S Nivnia and Patty, A dngrammac pect: tow af tuck propagation. Center fo Sy. Se Depo Ett ton RES Ace Pty il pga pa incl Eng. Yale Unvery, New Haven CT, ech ep. 505, [ne] K's. Neen sn K,Panhsarhy, Nevnl pewors an yu ‘ayn Reticence ap) KS. Reena nd K.Fanhsarhy, Neualnetwors an yu alguemn Par: Conta Comer Sy Sry Dope Bee Eng Yale Users, New Haven, CT ech rep 80, May 1989. (0) RS Re Hn ad Sa, te ane ‘no! 25, pp. t-408 Jone 98 [a1 5°5"Mone, "lll sy perms atv ens" IEEE 132) G7" Gonduin, B, Ramage. aed PE Canes, “Dore tme Pt vo ema i sere appa 5. Sarda 6 a 198t and he MS. tnd PD eres rom Ha ‘int He yin the Deparien f Engine vAglied Sceace at Ya Send en ae Frofestrn 0H He wss he Charman of the Bepariment of Etna! Engng om 198810 1987 an carne he Drs o the Cente fr Sytem Science [Yat Meise arf merous ch poletont ae ars of ‘ae thn che ahr ofthe bnksFequncy Doman Creat Jorabolse Say couthor) Taylor patted by Acar Pe in 1973 Sale Ape Sse (eothor A.M Ansa): an egmaing Automated con conto, AL Tha pa ‘ithe y ene all 089 He alo the sir of he nk Apt ‘ton of tgpive Cnt codon RW. Monet poised by Ac See ti a pean earn Sr Cea Me 1m Adapive Camo feb Ke Onega a P. Dora) ch wil be abit toy sapien log atomata ad the conte of ‘SSence andthe TEE (UK) He war the spon fhe 1972 rnin Taylor Award ofthe IEE Systems, Man, nd tert Sot and the (Gong Aacty Het Paper Ava of IEEE Como ystems So fj Kannan Parthasarathy was born n nda om A SP fee ete racened he Beech degre Esc eepaonng fom he Ind inte of “echology Mara, 1980 ad the MS. de> © BeetaSM mi dee cil cpeeig Frc tpn eles doctored Yale tread lenin yates wears a tii apton to acapve coma podem. ie Purtuanahy ihe pen ot he Com stu High Tecnology Grate Selah fr the ee 1989-1850,