A Distributed Computing Model Based on Multiserver Ji Hua Xie Li Department of Computer Science Nanjing University Nanjing, 2100093, P.R.

China email: postcomp@nju.edu.cn(Subject: To Ji Hua) [ Abstract ] : With the development of distributed system, there are many resources coexisted in the network environment. To a special kind of service, there may exist some relationships among many servers, such as equalservice, superservice and subservice. In this paper, we define this phenomenon as Multiserver system. Besides, We discuss the problems in Multiserver system in the view of Open Distributed Processing(ODP), and propose a basic open model about Multiserver system. Based on the open model, we also have implemented a prototype system(OMM) on the top of PVM(parallel virtual machine). With description of the architecture and behavior of the components in OMM, we can make an distributed system more transparent, interoperable and fault-tolerated. [ Keywords ] : Multiserver, Open Distributed Processing, Service, Service Relation. 1. Introduction Client/Server is an important paradigm in current distributed systems, in which a client object can invoke a server object located in other machines usually through the remote procedure call(RPC) mechanism. With the development of distributed system, many resources coexist in network environment, and to a special service, there may be some relationships among many servers. Here we define this phenomenon as Multiserver system: Multiserver_System.'=<Client, abstract(server)>, abstract(server) refers to a kind of service, which includes two components, one is the set of servers, another is the relationship between the servers. These kinds of relationships among servers will be discussed in detail in section 3. Figurel outlines the difference between the traditional client/server system and the Multiserver system. The problems arising from Multiserver system are mainly about the mechanisms needed to manage the related servers, such as how we can dynamically obtain the service .with transparency and fault tolerance, in other words, how we make Multiserver system be an open environment[l-81.

Client's Viewpoint

Client's Viewpoint

(@ ~

!
servers

Figurela Traditional Client/ServerModel Figurelb Multiserver o d e l M The rest of the paper is organized as the following: section 2 presents the issues related to Multiserver in the view of Open Distributed Processing(ODP); Section 3 and section 4 describe a basic open model about Multiserver system and a prototype system(OMM) on the top of PVM(parallel virtual machine) implemented by our research group; finally we make a conclusion in section 5.

2. ODP & Multiserver There is an increasing trend towards achieving more connection among existing computer systems. The main obstacle to reach this aim is the lack of common view of the states of the whole system. Open Distributed Processing(ODP) is the trend of distributed computing, which aims to solve the software interface problem to create distributed applications that can operate in an open, heterogeneous and autonomous environment to achieve portability, transparency and interoperability. ISO RM_ODP currently under development provides an ODP system framework, and uses five viewpoints to abstract the ODP systems. It also proposes conceptions and structure rules of the corresponding viewpoint languages to specify the viewpoints, but the details about how to construct the open system is not given[9-13l. To the Multiserver system, we should first identify four critical problems so as to make it suit to the point of ODP. Transparency of locating and invoking a server. Transparency is an important feature of an open system. To achieve transparency, Multiserver system should provide a mechanism to mask the details of the location and invocation of a server. -k Dynamic selection and Binding of Services. In a Multiserver system, the number of available services constantly changes because new services are added and old ones remove& Therefore, users and applications on such an environment should be able to choose the service which to be used at run-time. As services are dynamically selected, dynamic binding of interfaces is necessary, because applications cannot be bound to services at the time of compilation and linking, and static binding cannot support system evolution either which is inevitable in a large open system. Interoperability. To achieve interoperability among the servers in a Multiserver systems, a protocol is needed to deal with the relationships and the information exchange among the servers. Fault Tolerance. To be a fault-tolerated Multiserver system, the management of service and its relationships is needed so that a client will not fail to obtain service just because a certain server is crashed. 3. Open Model for Multiserver To identify the problems mentioned above, an open model for Multiserver is proposed. But some basic conceptions should be defined first. 3.1 Basic Definitions Service: In a Multiserver system, service is defined by a server's unique identifier and the set of operations provided by the server, e.g., file server, print server. Informally we define a service as following: Service:=Object_ID U Operations

Operations:={opl U op2,... U opi...}, i > 1 RPC & Interface : Remote Procedure Call is a paradigm for communication between

machines over a network, it provides the system or an application programmer a level of abstraction above the underlying message stream. Instead of sending and receiving messages, the programmer calls the remote procedure and receives return values. This frees the programmers froln dealing with the specifies of the distributed system, while providing a familiar interface. The syntax and semantics of the remote call are similar to the local. RPC mechanism is an important middleware in distributed system[14-151. In this paper, it is assumed that, in a Multiserver system, each Service Instance is a RPC server, which provides some methods to user applications; RPC Interface is the interface of RPC methods, which can be defined as follows: with a RPC method being F(Xl: cr 1,x2: cr 2,...Xn: cr n)--, (yl: p 1,Y2: p 2,'"Ym: p m) RPC_Interface :=(F, cT 1, ~ 2 , ' " , ~ n, p 1, p 2 , ' " , p m) F the method's name ; 1, cr 2 , ' " , e n " the type of input parameters; P 1, p 2, '", p m " the type of return parameters, In a Multiserver system, every Service Instance has one or more RPC_Interface Service Interface: Service Interface includes the ID of one server and the RPC Interfaces provided by the server, it can be defined as follows: Service_Interface: = { ID, Set(RPe_Interface) } Set(RPC_Interface) := {RPe_Interface 1,...,RPC_Interfacei,... }
Super/Sub Service: Super/Sub service defines that if servicei is a Sub service of servicej,

it means servicei can be used in place of servicej, but conversely, it's not true. The terms super and sub service are used here with their object-oriented meaning. Thus a service X is a subservice of service Y, if: X has all of the operations defined for Y(and possibly more), "k for all of these common operations: -- the input parameters for X are in a subtype relation to the input parameters for Y(i.e. X's input parameters can take a wider range of values than Y's),
-- the result parameters for Y are in a subtype relation to the result parameters for Y(i.e. X's result parameters can be from a narrower range of values than Y's). The notion of Subservice is equivalent to the program refinement concept of weakening pre-conditions and strengthening post-conditions. Superservice is simply the inverse of Subservice. EqualService: EqualService defines that two service are potentially substitutable, which means there are the same servers in the Multiserver system. In other words, two services, X and Y, can be regarded as equivalent if X is a Subservice of Y and Y is a Subservice of X. Service Instance: Service Instance refers to an active object, which provides some kind of service. In Multiserver system, Service Instance corresponds to a process or a thread. Service relation & related Servers: Service relation defines, in a Multiserver system, the relationships between service and a set of service instance, which may be the Equal, Super or Sub relationships; Related Servers defines the set of service instances, which have some

service relation, its definition is as following: {Service_Relation:={Service, related_servers{Service_Instance} } } ~ {Equal, Super, Sub} related servers(Service_Instance):={ Service__Instancel,... Service__Instancei,... }, i >t 1 Service Domain: With a large distributed system, many servers exist in the network environment, there has the requirement organize servers among a set of Service Domain. Service Domain is a set of components which share a common attribute or are managed by the same management agents. The domain identifies the sphere of influence of management. A domain could range from a physical boundary containing the hardware components of a computer systems to the boundary round a set of logical resources under the control of a single administrator. Domains allow components and resources which are related from a management point of view to be structured for a management purposes. 3.2 Open Model for Multiserver In this section , we presents the open distributed computing model for Multiserver, which aims to provide client the dynamic, transparent and fault tolerant service. Fig.2 outlines the architecture of the model. We give the details of the model later i.......................................................................................
/

@
Figure2 An Open Model for MultiServer

Client Interface: In the open model, a unified client interface is defined. By using the client interface, one client can invoke a server with a certain server identifier or with the RPC interfaces. The request for one service will be processed by the kernel in our open model; In this paper, client interface is identified as two sets of APIs, static invoking mechanism and dynamic mechanism, which will be detailed in section 4. Server Interface: Server interface is used by one server to register or withdraw its service by providing the description of its service interface. In the server interface, one server can join into the Mulfiserver system in the plug and play method without influencing the client's behaviour, so one client can obtain a service without understanding the details of the wanted servers;. Service Manager: To achieve the dynamic selection and binding of a service with transparency and fault tolerance, service manager plays a major role in providing a solution to those problems. It facilitates service selection, dynamic binding of clients and servers and run-time type checking by providing a canonical description of a service and related services in order to be able to replace one type with a compatible one. Commonly, service manager contains two repositories: service interface repository and service relation repository, and it also provides a set of functions to maintain these repositories, i.e. adding/deleting a service entry, matching the relationship between two service description,

et al. Binder: Binder is an important component in the open model. It provides the binding function for one client. When receiving a binding request from the client interface or from the service manager, the binder will find the proper binding parameters, i.e. the server identifier by searching the service repository, then use the server identifier to bind the server, so that one client can invoke the server correctly. Trader: when a special service can not be found in the local service domain, the trader plays the role of forwarding the request to other service domain by providing the wanted service interface description. Fig. shows the behaviors of traders.
] "" '",..

'":::::~':':"

~ ~nee~r~oi!:7om ) ain

~ ~nee~o;!: main) drYo

Fig. 3 The protocol between domains 4. Case Study We have implemented a prototype system(OMM) based on the open model on the top of PVM(Parallel Virtual Machine). Figure4 gives the architecture of the OMM system.

Management Dynamic service Sta Interface Inte~ace | Ace otherdomains --~ Trader IBinder] Servicemanager ~ Se~ -'Q ' Q -- Int ServiceInterface Servicerelation ] e I Repository IIRepository I
i

Reque~ otherdomains

(3roupColrununicatton& RPC Tid-Based Pvmmessage-passing

Fig.40MM Architecture

4.1 P V M PVM is a software system that permits a heterogeneous collection of UNIX computers networked together to be viewed by a user's program as a single parallel computer. PVM is the mainstay of the Heterogeneous Network Computing research project, a collaborative venture between Oak Ridge National Laboratory and the University of Tennessee[16-19]. The overall objective of the PVM system is to enable such a collection of computers to be used cooperatively for distributed and parallel computation. The main principles upon which PVM is based include the following:

User-Configured host pool: The application's computational tasks execute on a set of machines that are selected by the user for a given run of the PVM program. The host pool may be altered by adding and deleting machines during operation Process-based computation: The unit of parallelism in PVM is a task(often but not always a UNIX process), an independent sequential thread of control that alternates between communication and computation. No process-to-process mapping is implied or enforced by PVM; in particularm multiple tasks may execute on a single processor. PVM used a task identifier(TID) to address tasks and groups of tasks within a virtual machine. The TID contains four fields. Naturally, TIDs are intended to be opaque to the application and the programmer should not attempt to predict their values or modify them without using functions Explicit message-passing model: Collections of computational tasks, each performing a part of an application's workload using data-, functional-, or hybrid decomposition, cooperate by explicitly sending and receiving messages to one another. Message size is limited only by the amount of available memory. Group Communication: PVM provides a set of primitives supporting group communication, i.e. pvm_joingroup, pvm_lvgroup, pvm_bcast, pvm_recv. In OMM model, Traders is organized as groups for the management among service domains. 4.2 Tid-based RPC PVM is a message-passing based system, not supporting RPC mechanism. We think remote procedure call is essential for distributed computing. However, in traditional client/server model, every service instance has not a global identifier, so a client needs to know the server's IP address, procedure number, et. al, before calling the service, which leads to less service transparency. In OMM model, a tid-based RPC mechanism is proposed and implemented. User applications can export their RPC service by a set of primitives provided by OMM. We adopt object oriented method to locate and invoke a server object, and each server object is identified in the global service space by its PVM task tid, which can be used to locate a task over a network environment. PVM allocates an opaque tid for each server, any server is a task in PVM. By using Tid as object reference, we can get a Tid-based server space, which is object oriented. 4.3 Static service interface With static service binding, one client can bind to the specific server in compile time. In this mechanism, the client should understand the details about the tid of a certain server. Most of the current mechanism support this static service binding, it can be used by the client who wants to obtain the service directly. In OMM, a static service interface is defined as supplying an API to user applications, and this interface is useful when a client has understood the wanted server. 4.4 Dynamic service interface In the Multiserver system, providing not only a static binding mechanism but also a dynamic binding mechanism is necessary. Compared with static binding mechanism, dynamic service binding does not need a client to obtain the tid of one server ahead the compile time, while it can dynamically search the wanted service via service interface in the service space, and bind to the proper server. Through a dynamic service interface defined in OMM system, we can do better about portability, transparency, fault tolerance in the distributed environment. 8

4.5 Service Interface Repository: it is the repository of the service interface information in the Multiserver system. When a client request a service dynamically, this component provides the binder the needed information oftid and RPC_Services. 4.6 Service Relation Repository: it is the repository of the available server relationships among the related servers. When a specific server cannot satisfy the client's service request, this repository will provide the information about service manager to find an proper server by the service interface information. 4.7 Service Management Interface: it provides the API of service enrolling, service deleting, service relation adding and service relation removing, service querying, i.e., _service_export(), _service_relation_add(), _service_delete(), _service_query(). 4.8 System's Behavior: There are some important components in OMM which defined as Fig.2, i.e., binder, trader and service manager, which maintain the service and service relationship information in the Multiserver system, deal with the client's service request and the request from/to other service domains. Their behaviors can be identified as the following: First, if a client request a service i via a tid, and this server cannot satisfy the client's request for some reasons, e.g., heavy loading, network node crashing, et al, the server manager will automatically search the service relation repository then match the related server, and find a specific server who can substitute the service i. But if there are no suitable server in local domain, the request of trader is invoked to search a server in other service domains. These steps are shown as following: If Client.invoke[Service(i).tid] != True then { while(related_to(Service(i)) !=NIYLL & (Client.request[Service(i)] != True))

{
If (Service(j) ~ related_to(Service(i))) then If { { { Service(j) is EqualService of Service(i)} U {Service(j) is Subservice of Service(i)} } n {Some other attributes, eg,load information} } then Client.invoke[Service(j).tid] else j++;

}}
else { service_manager.request to
trader(rpc_mterface), . ..... }

Second, if a client task requests a service via rpc_interface, the service manager will search and match in the service repository, return the tid of the available and proper server, before the client uses the Tid to invoke the binder to bind to a selected server. The following is shown the steps: Client.request(Service.rpc_interface) result=FALSE while(servicerepository !=NULL)&(result !=TRUE)

if(S ervice_manager, match =TRUE) then { return(tid); result=TRUE) } Client.request to binder(tid); Client.invoke(tid);
4 . 9 0 M M API We are coming to explain the API provided by the OMM system. int _service_invoke(int sid, char* operation, char* in, char* out) It is used to invoke a server identified by the tid, operation is the name of the method provided by the server, in is the address of input parameter and the return value's is out. int _service_export(char** Service_Interface) It is used to export its service interface, which consist of the attributes and its rpc interfaces. The client can use the server only after the server has exported its service. void _service_run0 The server object uses it to wait the client's request after the _service_export0 function is called. int _service_import(char** Service_Interface) If a client only knows the wanted service interface, without knowing how to invoke the server, it may turn to this primitive. The return value is the server object reference rid, and with it, the client can invoke the specific server by using _service_invoke0. int _service_cancel(int sid) A system administrator can use it to cancel a server instance, and this server cannot be used later. If there is an entry in service relation repository, it will also be canceled. int _service_relation_add(int sid 1, int sid2, int relation) It may add an entry in service relation repository about two server objects, e.g., equal, super or sub relationship. int _service_relation_delete(int sidl, int sid2, int relation) It is used to delete an entry in the service relation repository about two server objects. int _service_relation_query(int sidl,int sid2) To query in the service relation repository about the relationship between two server objects, it can be used. 5. Conclusion This paper proposes the conceptions of the Multiserver system and discusses the issues related to this system in the open distributed processing point of view. To get the transparent dynamic service with fault tolerance, an open model for Multiserver is set up. Besides, we have implemented a prototype system OMM based on the open model in the SUN OS4.1.2, SCO UNIX for MPX and PVM 3.37 environment. Next work is to perfect the prototype system further and to do research work further on the Multiserver system. [ References ]

1. A Albanese, "Service negotiationamong network managemententities", Computer Communications, 10

vol. 16, 2, 1993. 2. Richard M. Adler, "Distributed Coordination Models for Client/Server Computing", IEEE Computer, April, 1995. 3. Friedemann Mattern, et al., "Framework for the organization of cooperative services in distributed client-server systems", Computer Communications, vol. 15, 4, 1992. 4. Magdi N Kamel and Nabil N Kamel, "Federated database management systems: Requirements, issues and solutions", Computer Communications, vol 15, 4, 1992. 5. Dr. Morris Sloman, "A FrameWork For Distributed Systems Management", IFIP 1988. 6. Y Ni and A Goscinski, "Trader Coperation to enable object sharing among users of homogeneous distributed systems", Computer Communications, Vol 17, 3, 1994, 218-229. 7. V Tschammer, et al., "Cooperative management in open distributed systems", Computer Communications, vol 17, 10, 1994. 8. Debajyoti Mukhopadhyay, "An approach to realizing interoperability using APPC", Computer Communications, vol 16, 11, 1993. 9. ISO/IEC JTC/SC21, "Basic Reference Model & Open Distributed Processing", 10. P.F.Linington , 11. A.Herbert, International IFIP Workshop on Open Distributed Processing, "The Challenge of ODP", Berlin, Oct, Part1 to 4, 1993.

"Introduction to Open Distributed Processing Basic Reference Model", 1991. International IFIP Workshop on Open Distributed International IFIP

Processing, Berlin, Oct, 1991. 12. M.Bearman and K.Raymond, "Federating Traders:An ODP adventure", Workshop on Open Distributed Processing, Berlin, Oct, 1991. 13. "Object Management Architecture Guide 2.0", OMG TC Document 92.11.

14. AndrewS. Tanenbaum,Distrinuted Operating System,Prentice Hall, EnglewoodCliffs,N.J. 07632. 15. YushengLiu and Doan B Hong, "OSIRPC Model and Protocol",ComputerCommunication,Vol 17, 1, 1994. 16. "PVM: Parallel Virtual Machine a User's Guide and Tutorial for Networked Parallel Computing", The MIT Press, Cambridge, Massachusetts, London, England. 17. G.A. Geist "PVM-aplatform for portable Distributed Computing", 15-21, HPC'94, Singapore. 18. Song JianJian, Choo Heng Ken, "Developmentof a process Migration Facility on Top ofPVM", 114-123, HPC'94, Singapore. 19. WentongCai, Hung KhoonTan, "ParallelDiscrete Simulationusing PVM", 345-353, HPC'94, Singapore.

11