12th IFIP/IEEE IM 2011: Application Session

A Novel Distributed Call Admission Control Solution based on Machine Learning Approach

Abul Bashar, bashar-a@email.ulster.ac.uk Gerard Parr, gp.parr@ulster.ac.uk Sally McClean, si.mcclean@ulster.ac.uk Bryan Scotney, bw.scotney@ulster.ac.uk School of Computing and Info. Engg. University of Ulster Coleraine, UK BT52 1SA

Detlef Nauck, detlef.nauck@bt.com Research and Technology British Telecom, Adastral Park Ipswich, UK IP5 3RE

IM 2011: 12th IFIP/IEEE International Symposium on Integrated Network Management

Abstract
The advent of IP-based Next Generation Network (NGN) and its guaranteed QoS promise has attracted significant attention from both service providers and subscribers. However, to fulfil the said promise, there is a need to provide effective Call Admission Control (CAC) based QoS provisioning solutions which are autonomous, intelligent and scalable. This paper proposes a novel distributed and scalable call admission control solution which operates at the network edges. The key ingredient of our solution utilises Machine Learning (ML) approach for estimating network QoS metrics based on the model which is learned in an offline mode. More specifically, it uses a prominent ML technique, Bayesian Networks (BN), for learning the behaviour of an edge-to-edge router pair, for estimation of end-to-end delay to make localised admission control decisions. A thorough mathematical model is presented of such a distributed system through the use of joint probability distribution formulation and the concept of Multiple Entity Bayesian Network (MEBN). Further, we evaluate our proposed solution through a simulated setup in Opnet and provide prediction accuracy results to measure the performance of the BN models. We also discuss the implementation issues of our approach in terms of complexity, scalability and interpretability. We conclude with a comparative study of our distributed approach to a centralised approach, focussing on performance accuracy and system complexity. Even though this paper extends our previous work on a similar subject, it does provide a significant and distinct contribution to this research domain.

978-1-4244-9220-6/11/$26.00 2011 IEEE

867

Motivation : NGN and its Challenges

IP-based, over WDM

Fixed, wireless & mobile

Call Admission Control function for QoS

NGN: ITU-T recommendation, Guaranteed QoS, Converged services Reduces: CAPEX and OPEX Challenges: Complex, heterogeneous, unpredictable Qos Provisioning: Call Admission Control (CAC) at network edges Problems with existing CAC: analytically intractable, non-scalable Machine Learning for CAC: Autonomic, Scalable and Predictive solutions Our contribution: Distributed CAC based on MEBN for NGN Slide 2
The rapid increase in the deployment of Next Generation Network (NGN) (which was proposed by ITU-T) has posed challenge to the network management systems to devise cutting edge solutions for their effective monitoring, decision making and administration. The NGN was designed to provide converged services with guaranteed QoS and also promised enormous savings to the network and service providers by reducing their CAPEX and OPEX [1]. However, to fulfil such a promise it is necessary to develop autonomic, scalable and predictive network QoS management solutions which can cater to such highly complex, dynamic and unpredictable networks [2]. Call Admission Control (CAC) schemes are widely accepted to be a means of providing desired level of QoS in a communication network. The key function of a CAC system is to make decisions to accept or reject a new call request, based on whether this new call can be supported with the desired QoS [3]. However, as the number of services, their classes and size of the network grows, the CAC problem becomes difficult and intractable to solve through conventional analytical methods. Machine learning (ML) is one approach which is capable of addressing the issues raised above. These techniques have the ability to learn the system behaviour from past data and estimate future behaviour based on the learned system model [4] [13]. They use automated and intelligent data analysis techniques to build models from network management datasets, for prediction and decision-making to realise QoS management objectives. Significant amount of research has been done to apply ML techniques for automated QoS provisioning in heterogeneous networks. Our contribution to this domain is to provide a theory, implementation and evaluation of a distributed admission control mechanism which is based on the concept of Multiple Entity Bayesian Networks (MEBN). The remainder of this paper is structured as follows. We first provide a survey of related work and state our research objectives in Slide 3. Slides 4 and 5 present the basics of ML-based CAC and BN theory. Further, a mathematical treatment of a distributed CAC system is presented in Slide 6. Subsequently, we provide the details of the simulation setup and present our simulation results (Slides 7-9). Finally, we conclude the paper by summarising key findings and pointing areas for further research (Slide 10).

868

Related work and Research Objectives

Existing Approaches : ML-based CAC for various networks
Neural Networks (in CDMA Cellular networks) Reinforcement Learning (in Wireless Cellular networks) Support Vector Machines (in UMTS networks) Genetic Algorithms (in Wireless Mesh Networks) Bayesian Networks (in NGN)

Drawbacks of Existing Approaches

Implemented on single network element : Stand-alone solutions Centralised solutions : Multiple element solutions are not distributed

Our proposed objectives

To compare and contrast ML-based CAC solutions Study pros and cons of centralised and distributed solutions Bayesian Network solution: feasibility study and performance issues Slide 3
CAC is a major preventive technique to guarantee QoS to various class-based services, as recommended by the ITU-T for NGN [5]. Several ML techniques have been applied to implement CAC solutions in various types of networks. We present the summary of the latest research work in this domain. Neural networks (NN) and Adaptive Critic Design (ACD) have been used for admission control decision making through selflearning approach in CDMA cellular networks [6]. Reinforcement Learning (RL) in conjunction with stochastic approximation has been used for QoS provisioning, which maximizes the network revenue subject to several predetermined QoS constraints [7]. Support Vector Machine (SVM) based CAC algorithm utilises service vector and network vector to predict admission state for making call admission decisions [8]. Genetic algorithm (GA) based approximation approach was utilised to guarantee QoS in a differentiated priorities handoff scenarios for wireless mesh networks [9]. Bayesian Network (BN) based CAC (proposed by the authors) has been found to be effective in providing QoS to converged services in a NGN environment by accurately predicting congestion through packet loss measurements [10]. We conclude that the majority of the existing work either show the validity of their approach on a single network element or provide centralised solutions in case of multiple network elements. We extend the work presented in our previous paper [11], wherein we had compared the performance of two prominent ML approaches (NN and BN) for single network element CAC decision making by utilising the predictions of delay and packet loss QoS metrics. In this paper, we provide a significant and distinct contribution (compared to our earlier work [11]) by attempting to answer the following research questions. What are the prominent ML-based CAC solutions and how do they compare to each other? Is there a significant difference between the performance of centralised and distributed CAC approaches and what are the relative merits and demerits? Is Bayesian Network approach a feasible solution to CAC problem in a distributed environment and what are the implementation issues?

869

Basic theory of ML-based CAC

Network Topology with a bottleneck ML-based CAC at the edge router

CAC is generally implemented at network edges Input

Traffic Descriptors (Peak rate, Average rate, Burst duration, Service Class) Qos Metrics (Packet loss, Delay, Jitter) System State (Link Bandwidth, Buffer occupancy)

Output
Admission Decision (Admit or Reject) Estimation of Qos Metrics (Packet loss, Delay, Jitter)

Operation
Trained offline and then used for online decision-making Key Performance measure: Prediction accuracy

Slide 4
CAC mechanisms usually reside at the edge of the networks, generally between the access and the core network. It is obviously wise to implement such a mechanism at the edge so that the core network (which might have bottlenecks) is protected from excess traffic burden, resulting in violation of desired QoS objectives. Such a scenario is illustrated in the above figure, where the sources (S0-S3) are requesting access (from ingress router, IR) to establish connections to transmit their traffic. Since there is a bottleneck link between the core router (CR) and egress router (ER), some calls need to be rejected to maintain the desired QoS. The above scenario was studied in our previous paper [11]. An input to an ML-based CAC system consists of call/flow requests and the output is a decision to either admit or deny the call. The call request consists of its characteristics defined in terms of traffic descriptors and desired QoS during its unknown call duration in the network. Parameters like peak rate, average rate, maximum burst duration and type of application, are provided by the call initiator and are termed as traffic descriptors. In the absence of such information, they can even be measured through network monitoring tools. The standard Qos metrics comprise of packet loss, average delay or delay variation (jitter). In addition to these inputs, a set of system state parameters like available link bandwidth and buffer occupancy are also provided as inputs to CAC system. The output of the ML module (BN in our case) can be either the final accept/reject decision, or the value of any QoS metrics like packet loss, delay or jitter. Based on the choice of inputs and outputs, the ML module is trained offline with a set of data which is observed in the system for a desired period of time. The training data set consists of cases where both the inputs and outputs are known. In the online mode, the trained model has to output the estimate of the target parameter. When a new request comes in, the traffic descriptors and the system state information is given as an input to the ML module. The output of the ML module, which is the predicted QoS, is compared to the existing system QoS and an admission decision is made based on some predefined rules. It is obvious that the overall CAC performance is dependent on the prediction accuracy of the model. Prediction accuracy depends on how well the model estimates the unknown output and it is dependent on the diversity of cases used for training it.

870

Bayesian Network Representation

BN is a probabilistic graphical model, a mapping of physical system variables into a visual and intuitive model Directed Acylic Graph structure : using nodes and arcs Encodes conditional independence relation among system random variables Defined mathematically using joint probability distribution formulation Inference feature : Repeated use of Bayes rule to estimate unobserved nodes based on evidence of observed nodes Slide 5
A Bayesian Network (shown above) is a probabilistic graphical model that represents a set of random variables and their conditional dependencies via a directed acyclic graph (DAG). Essentially it encodes the joint probability distributions of all the random variables (represented as nodes). The fundamental assumption is that there is a useful underlying structure to the problem being modelled which can be captured with a BN. This means that not every node in the BN is connected to every other node. In such a case, a BN gives a compact representation of probability distribution in computationally tractable way. Let us consider a BN composed of

n nodes, X 1 to X n . A particular instance in the joint distribution is generally represented

as P ( X 1 = x1 , X 2 = x 2 , ...., X n = x n ) . For the sake of simplicity we represent the joint distribution as P ( x1 , x 2 ,...., x n ) . Now, using the chain rule of probability we can factorise joint probabilities as:

P ( x1 , x 2 ,...., x n ) = P ( x1 ) P ( x 2 | x1 ),...., P ( x n | x1 , x 2 ,....x n 1 ) = P ( xi | x1 ,..., xi 1 )

i

(1)

As pointed out earlier that not every node in the BN is connected (i.e. dependent on) another node, we can reduce equation (1) to:

P ( x1 , x 2 ,...., x n ) = P ( xi | Parents ( X i ))
i

(2)

where Parents ( X i ) {xi | x1 ,..., xi 1 }. This means that the value of a particular node is conditional only on the values of its parents. The conclusion is that fewer parents mean a sparse BN (few arcs and more independencies) and computationally compact representation of joint probabilities (which defines the system under consideration).

871

Distributed Bayesian Network Formulation

Multiple edge router topology for distributed CAC study Three edge router pairs (IR0-ER0, IR1-ER1 and IR2-ER2) Three BN models for each pair (BN0, BN1 and BN2) Multi Entity Bayesian Network (MEBN) : Relation between multiple BNs We developed the mathematical formulation using joint probability distribution Note: The MEBN figure shown above is a speculated behavioral model Slide 6
Multiple Router Model: To advance our previous work, we consider a scenario which has multiple bottleneck links. Such a scenario is shown above, where we have two bottleneck links (CR0-CR1 and CR1-CR2). It needs to be noted that, we will apply the BN model not on the bottleneck links, but between a pair of edge routers located on the edge network. As such, we will now have three edge router pairs (IR0-ER0, IR1-ER1 and IR2-ER2) and their behaviour will be modelled by individual BN models. We assume that each BN makes decisions independent of the other BNs. Mathematical formulation: The BN mathematical model derived in the previous slide is extended to suit our requirement of distributed BN scenario using the Multiple Entity Bayesian Network (MEBN) concept [12]. We define an entity to be our BN model for a single edge router pair (namely IR x ER x , where x = 1,2,...N ) . Let us assume the domain has N entities, named Y1 to Yn . Now, the joint probability distribution of such a domain will be:

P( y1 , y 2 ,..., y N ) = P( y1 ) P( y 2 | y1 ),....,P( y N | y1 , y 2 ,.... y N 1 )

= P( y j | y1 ,..., y j 1 ) ,
j

where j = 1,2,..., N

(3)

We know from Eq. (2) (refer Slide 5),

P ( y j ) = P ( x ij | Parents ( X ij ))
i

where X ij is the i th node of BN corresponding to the j th entity in the MEBN. Finally applying the reasoning that, not every entity is dependent on another, we can define the joint probability as:

P( y1 , y 2 ,...., y N ) = P( y j | Parents (Y j )) ;
j

where Parents (Y j )

{y

| y1 ,..., y j 1

(4)

In the special case when all the entities are independent (i.e. absence of child-parent relationships), Eq. (4) becomes:

P( y1 , y 2 ,...., y N ) = P( y j ) = P( x ij | Parents ( X ij )) j j i Eq. (5) characterises a special case of MEBN and is the mathematical basis for our simulation experiments.

(5)

872

Experimental Setup Details

Network Topology in OPNET Topology definition

Parameter
Sources Destinations Ingress Routers Egress Routers Core Routers

Source Traffic definition

Parameter
Flow generation rate (flows/sec) Average flow duration (sec) Packet generation rate (packets/sec) Packet size (bits) Type of service

Value
S0, S1, S2 D0, D1, D2 IR0, IR1, IR2 ER0, ER1, ER2 CR0, CR1, CR2

Value
5 2.0 Exponential (4) Exponential (1024) Best Effort

Slide 7

Network Topology: The network topology (as shown above) for our CAC experiments was simulated using Opnet Modeler [14]. The three sources (S0, S1, S2) connect to three destinations (D0, D1, D2) via the edge and core router network. The links between core routers (CR0-CR1 and CR1-CR2) are the bottleneck links and they play an important role in deciding the QoS between the source and destinations. Traffic Characteristics: The sources generate traffic which is modelled as being self-similar. It is a type of traffic which is characterized by the same fractal properties that are present in the traffic generated by many of todays Internet applications. The values of traffic parameters like, flow arrival rate, average flow duration, packet generation rate, packet size and type of service are shown in the slide above. Simulation details: The simulation was run (with multiple seeds) over a simulation time of 3000 minutes to capture data for offline training purpose. The statistics were captured at an interval of 60s. The statistics used in our study were: aggregate incoming Traffic (traffic descriptor), Queue occupancy (system state), queuing Delay and packet Loss (QoS metrics). For the purpose of model interpretability, we discretise these statistics into five levels (VLO, LO, MED, HI, VHI) of equal widths. It is to be noted that BN is capable of accepting discrete as well as continuous inputs. Discretisation of the data offers the advantage of being easily understood by the network manager. At the same time, it is known that BN have some limitations when dealing with continuous data, which we wished to avoid in our research work.

873

Results for Distributed Scenario

BN for S0-D0 pair (BN0)

BN for S1-D1 pair (BN1)

BN for S2-D2 pair (BN2)

CPT Size (for 3 BNs) = 280 *3 = 840

BN0 80
Delay Prediction Accuracy

BN1

BN2

79 78 77 76 75 74 0 500 1000 1500 2000 2500 3000

Prediction accuracy variation of individual BNs has similar behaviour and the maximum value of it is 79.3% (BN2), for training data size of 3000 cases
Number of Training Cases

Delay Prediction Accuracy for individual BN

Slide 8
The training data obtained from the earlier step (see Slide 7) is used to learn the BN models. The BN models were built using the GUI of Hugin Researcher [15]. This involves two steps. Structural Learning: The structure of the BN can be constructed manually by the subject expert or through structure learning algorithms: PC (Peter & Clark) and NPC (Necessary Path Condition) algorithms. The basic idea of these constraint-based algorithms is to derive a set of conditional independence and dependence statements (CIDs) by statistical tests among the nodes of the BN. NPC algorithm allows the system modeller to incorporate domain knowledge and hence enhances the structure of the BN. Parameter Learning: The Conditional Probability Tables (CPTs) or parameters can be specified, based on the knowledge of the domain expert, by the process of parameter elicitation. The past data may also be used as the basis for learning the parameters using efficient algorithms. The Expectation Maximization (EM) algorithm is particularly suitable for batch parametric learning, while Adaptation algorithms are useful for sequential parameter updates. We used EM algorithm in a batch learning mode to estimate the CPTs, since our current objective is to use an offline learned model which is assumed to be static (and sufficient) for our experimental purposes. The BNs for each of the edge to edge pairs (S0-D0, S1-D1 and S2-D2) are shown above (see Slide 8 for BN0, BN1 and BN2) and it is observed that they have same structure, but different CPTs (not shown due to space limitations). We experimented with various training set sizes and observed the prediction accuracy. These results were obtained using the process of 10-fold cross validation, where the training data is partitioned into training and test sets and the prediction accuracy is averaged over 10 such iterations. As shown in the slide above, the prediction accuracy of Delay node increases as a function of training data set size. We have plotted the prediction accuracy for three BNs (BN0 for source S0, BN1 for source S1 and BN2 for source S2). It is observed that they follow a similar trend with an average prediction accuracy of about 77 % (for the three BNs). The maximum prediction accuracy was found to be 79.3%. The total size of all the CPTs was found to be 840 (for memory requirement comparison purpose).

874

Results for Centralised Scenario

CPT size for (combined BN) = 1340 This is about 60% more memory requirement than the distributed scenario

Centralised BN for all edge-to-edge pairs

Delay Prediction Accuracy

Centralised BN 92

Maximum prediction accuracy of 90.5% for training data size of 3000 cases. This is about 11% more than the distributed scenario

89

86

83 0 500 1000 1500 2000 2500 3000

Number of Training Cases

Delay Prediction Accuracy for centralised BN

Slide 9
The training data obtained from the earlier steps (see Slide 7) was similarly used for constructing the BN model for the centralised scenario. However, here the BN nodes were slightly different, because our interest was to estimate the aggregate delay and aggregate packet loss for the overall network. To achieve this objective, we considered the following nodes for the aggregate BN.

Sn_Traffic where n = 0,1,2 (used as it is from the collected data) Sn_Queue where n = 0,1,2 (used as it is from the collected data) Total_Loss = S0_Loss + S1_Loss + S2_Loss (combined node from 3 data sets) Total_Delay = S0_Delay + S1_Delay + S2_Delay (combined node from 3 data sets)

After the new nodes were defined for the centralised scenario, we constructed a single BN model with the same algorithms (refer Slide 8) as for the distributed scenario. The centralised BN model is shown in the slide above. It can be seen that it has a different structure (as compared to the distributed model) and obviously different CPTs. The structure was partly influenced by the domain knowledge and as such has a more readable and intuitive structure. It basically tells us that the traffic from the sources has an influence on the queue behaviour and in turn the queues affect the total delay and total packet loss in the network. We did a similar analysis of observing the delay (Total_Delay) prediction accuracy with the increase in the training data size. It was found that the prediction accuracy increases with training data size and reaches saturation at about 2000 training cases. The average value of prediction accuracy was about 88% with a maximum of 90.5%. Also we calculated the total CPT size which was found to be 1340. Based on these results, we arrived at the following comparative observations:

Centralised scenario had an improved prediction accuracy of about 11% as compared to the distributed
scenario.

However, there was an increase in the size of the CPTs of about 60% when compared to the distributed scenario.

875

Conclusions and Future Work

Conclusions and Discussion
Presented and evaluated the BN based CAC framework Mathematical treatment, joint probability distributions, MEBN concept Compared the centralised and distributed CAC approaches Even though we presented performance results for offline learned BN model, our approach is flexible to cater for online operation

Future Work
Sensing the potential for BN based CAC, we plan to extend our work to incorporate further improvements First priority is to evaluate the distributed approach in an online mode with a mesh topology shown in this figure To study complexity of BN algorithms in terms of CPU, memory needs To study the performance of CAC in terms of call blocking probability and network resource utilisation

Slide 10
Based on the problem formulation, implementation details and results obtained from the previous slides, we put forward the conclusions of our study. The original contribution of our work was to design and implement a CAC solution based on the Bayesian Network approach. To the best of our knowledge no similar work exists in the literature. The mathematical model for distributed scenario (using the MEBN concept) was another novel contribution of this paper.

Based on the comparison of centralised and distributed BN models, we conclude that distributed model is better overall. Even though the prediction accuracy of the centralised model is higher than the distributed model, the centralised model has higher memory requirement too (in terms of the CPT size). Since we are interested in models which are scalable, we plan to study the distributed model in more detail later. Even though we showed the performance results for the BN model that was trained in an offline mode, we must emphasise that our overall approach has the ability to operate in online mode. This means that the solution can improve its internal knowledge with training whilst in operation mode with new incidents so that more accurate predictions are made in future.

. Future work will also involve a thorough evaluation of our BN approach by considering the following points. We plan to extend our evaluation of the BN based CAC by working on the mesh topology of core routers shown in the slide above, with bottleneck links connected to router CR1. Also we need to study the effect of offered network load on call blocking probability in an online scenario.

It is planned to seek input and collaboration from our industrial partner BT, for practical implementation of our solution.

876

Acknowledgement & References

The authors would like to acknowledge the support of University of Ulster for providing Vice Chancellors Research Studentship (VCRS) and IU-ATC (http://www.iu-atc.com) for funding the joint work with British Telecom. In particular, we wish to acknowledge the support from BT Adastral Park for organising the Research Internship.

Slide 11

References
[1] General overview of NGN, ITU-T Recommendation Y.2001, Dec. 2004. [2] R. Boutaba, J. P. Martin-Flatin, J. L. Hellerstein, R. H. Katz, G. Pavlou, L. Chin-Tau, Recent advances in autonomic communications [Guest Editorial], In IEEE Journal on Selected Areas in Communications, vol. 28, no. 1, pp. 1-3, Jan. 2010. [3] C. Yun and H. Perros, QoS control for NGN: A Survey of Techniques, In Journal of Network and Systems Management, vol. 18, no. 4, pp. 447-461, Feb. 2010. [4] E. Alpaydin, Introduction to Machine Learning, MIT Press, 2004. [5] Resource and admission control functions in next generation networks, ITU-T Recommendation Y.2111, Nov. 2008. [6] D. Liu, Y. Zhang, H. Zhang, A self-learning call admission control scheme for CDMA cellular networks, In IEEE Transactions on Neural Networks, vol.16, no. 5, pp. 1219-1228, Sep. 2005. [7] F. R. Yu, V. W. S. Wong, V. C. M. Leung, A new QoS provisioning method for adaptive multimedia in wireless networks, In IEEE Transactions on Vehicular Technology, vol. 57, no. 3, pp. 1899-1909, May 2008. [8] P. Guo, M. Zhang, Y. Jiang, J. Ren, Policy-based QoS control using call admission control and SVM, In Proc. of 2nd International Conference on Pervasive Computing and Applications (ICPCA 2007), pp. 685-688, Jul. 2007. [9] B. Rong, Y. Qian, K. Lu, R. Q. Hu, M. Kadoch, Mobile agent based handoff in wireless mesh networks: architecture and call admission control, In IEEE Transactions on Vehicular Technology, vol. 58, no. 8, pp. 4565-4575, Oct. 2009. [10] A. Bashar, G. P. Parr, S. I. McClean, B. W. Scotney, D. Nauck, Learning-based call admission control framework for QoS management in heterogeneous networks, In Proc. of Springer LNCS CCIS series, 2nd International Conference on Networked Digital Technologies (NDT 2010), vol. II, pp. 99-111, Jul. 2010. [11] A. Bashar, G. P. Parr, S. I. McClean, B. W. Scotney, D. Nauck, Machine Learning based call admission control approaches: A comparative study, in Proc. of IEEE/IFIP 6th International Conference on Network and Service Management (CNSM 2010), Oct. 2010. [12] K. B. Laskey, MEBN: A Language for first-order Bayesian knowledge bases, Artificial Intelligence, vol. 172, no. 2-3, pp. 140-178, Feb. 2008. [13] J. Qi, F. Wu, L. Li, H. Shu, Artificial intelligence applications in the telecommunication industry, in Expert Systems, vol. 24, no. 4, pp. 271-291, Sep. 2007. [14] Opnet Modeler 16.0, http://www.opnet.com [15] Hugin Researcher 7.3, http://www.hugin.com

877