This document proposes using network traffic analysis and linear programming techniques to automatically reconfigure virtual network topologies in IP/MPLS over DWDM networks. It involves developing probes to collect traffic data, using machine learning to predict future traffic patterns, and using optimization algorithms to determine optimal virtual link configurations based on the predictions. A prototype application will be demonstrated in a testbed at C-DOT to closely model real networks and prove the approach. The goal is to evolve the prototypes into commercial products to optimize various networks in the country.
This document proposes using network traffic analysis and linear programming techniques to automatically reconfigure virtual network topologies in IP/MPLS over DWDM networks. It involves developing probes to collect traffic data, using machine learning to predict future traffic patterns, and using optimization algorithms to determine optimal virtual link configurations based on the predictions. A prototype application will be demonstrated in a testbed at C-DOT to closely model real networks and prove the approach. The goal is to evolve the prototypes into commercial products to optimize various networks in the country.
This document proposes using network traffic analysis and linear programming techniques to automatically reconfigure virtual network topologies in IP/MPLS over DWDM networks. It involves developing probes to collect traffic data, using machine learning to predict future traffic patterns, and using optimization algorithms to determine optimal virtual link configurations based on the predictions. A prototype application will be demonstrated in a testbed at C-DOT to closely model real networks and prove the approach. The goal is to evolve the prototypes into commercial products to optimize various networks in the country.
Introduction With exponential growth in data and multimedia traffic, communication networks worldwide are migrating to the more efficient IP/MPLS over DWDM networks. The combination of IP/MPLS and DWDM transport network technologies provide a compelling choice for meeting the exploding network traffic demands. IP/MPLS over DWDM network retain the advantages of simultaneous hosting of multiple virtual network topologies and also are able to support the upcoming Network Function Virtualization (NFV) architectures efficiently. Problem Statement Similar to contemporary Virtual Private Networks (VPNs), service providers overlay virtual network topologies over physical optical transport topologies by connecting IP/MPLS routers through virtual links in order to organize traffic in their multi-layer IP/MPLS networks efficiently. These virtual links may in turn be directly mapped to coloured wavelengths in the underlying optical networks. Traffic forecasting in such networks is generally based on static IP/MPLS topologies. In static topologies, virtual links are usually overprovisioned to cater for deviations from the forecasted traffic, thus leading to inefficient use of networks resources and increase in cost of ownership. In order to cater to overflow traffic once the pre- defined ’utilization threshold’ of an existing link is exceeded, virtual network topologies are adapted by creating additional virtual links and/or deleting some existing ones. However, such techniques are suitable for networks with deterministic traffic patterns like traditional voice networks. The proliferation of new types of high bandwidth services like content delivery, video distribution, data centre interconnection etc. cause not only changes capacity demand in existing virtual links, but more importantly they cause intra-day variations in direction of the network traffic. Additionally, with the advent of Software Defined Networking (SDN) and Network Function Virtualization (NFV) the traffic patterns in networks are becoming increasingly on-demand and dynamic. Predictability traffic patterns and therefore the reconfiguration of virtual network topologies at the network and transport levels become too complex for manual reconfiguration. Alternate network resource optimization mechanisms based on Linear Programming, not dependent on human intervention, need to be evolved for the contemporary and future communication networks. 57 Proposed Solution Traffic data records the time and duration of a communication. Network Traffic Analysis (NTA) is a methodology for examining this data to determine the detailed shape of the communication streams, the identities of the communicating end points, and what can be established about their location in the network. The data may be sketchy or incomplete. Simply knowing what typical communication patterns look like can be used to infer information about a particular observed communication. The proposed solution is to use traffic analysis to develop algorithmic automation techniques for dynamic reconfiguration of virtual network topologies. Because most of many basic and fundamental optimization problems on communication networks are categorized into linear programming problems, this proposal focuses on using Linear Programming techniques for determining optimal virtual network topologies based on statistical traffic flows. When network operators or service providers design and control their networks in practical environments, in most cases they first formulate an optimization problem that corresponds to the desired communication networks with required parameters, and then they solve the problem by running linear programming solvers on computing systems. Such techniques are a refinement over the prevalent approach of context free adaptation of virtual link capacities based on bandwidth alone. They do so by also taking into account end-to-end monitoring of a multitude of source-destination traffic parameters using deep packet inspection at the network edge. The techniques evolved through this proposal are then proposed to be embodied as a software system that works as follows. Traffic monitoring data is collected at regular intervals using network probes built into IP/MPLS edge routers. Every edge router in the network collects sets of data samples to all other destination routers in the network. These source-destination sample sets are continuously collected and stored as time series data in a common large distributed data repository. The time series is periodically retrieved for each source-destination pair by a software module which summarizes the collected data into modelled data representing traffic pattern between that origin destination pair for the current period. Summarized data over a large number of previous time periods is retrieved by another prediction software module which uses machine learning techniques to generate source-destination traffic matrix for the next time period. The newly generated traffic matrix becomes input to an analysis software module which uses sophisticated mathematical algorithms involving combination of statistics, graph theory, neural networks, linear programming etc. to decide whether the current virtual network topology needs to be changed and how. Depending on results of the analysis module, actionable information is derived which is used to automatically reprogram virtual link topology at the IP/MPLS and/or the wavelength level depending on granularity of the refinement required between an origin-destination pair. 58 Figure 2 illustrates a broad overview of the proposed approach. Main Optimization Algorithm MPPLAB Output Analysis/ Control/ Reconfiguration Variables DWDM control IP/ MPLS Control Input generation for Main Algorithm Prediction/ Stat Module Data Models Data Mining Probe Probe Probe IP/MPLS Network Layer DWDM Optical Layer Virtual Links Figure .2: Overview of the proposed approach Network traffic sampling can additionally be used for analysing multiple security use cases such as detection of suspicious activity to investigation of activity that might be an incident, to post breach forensics. Consequently, implementing and embedding software that understands normal traffic patterns and operations to detect and investigate abnormal ones using network traffic analysis can be used in building secure routers. Such secure routers have the capability to enhance network security much beyond the aggregate security achievable by secure links alone in the network. Project Deliverables • Prototype Network Traffic Analysis and Virtual Topology Reconfiguration Application coupled to MPPLAB 59 • Network Probes for traffic collection & classification • Test Bed at C-DOT for proving the above reconfiguration application Project Timelines Year 1 Year 2 Year 3 SN Activity# Quarter! 01 02 03 04 05 06 07 08 09 10 11 12 1.0 Development of Network Traffic Analysis (NTA) Application 1.1 Study of Application approaches 1.2 Requirement specification for application 1.3 Model & Algorithm creation 1.4 Coding and Testing of Application 1.5 Phase 1 demonstration of traffic analysis application on standard compute servers 1.6 Algorithms optimization & code parallelization for scalability 1.7 Phase 2 demonstration of traffic analysis application on scaled computing systems 2.0 Development of Network Probes 2.1 Identification of changes in indigenous routers 2.2 Requirement specification for software probes 2.3 Probe software development 2.4 Testing of probes 2.5 Collection of sample data 2.6 Testing with Phase 1 traffic analysis application 2.7 Testing with Phase 2 traffic analysis application 3.0 System Proving on Test Bed 3.1 NKN connectivity setup 3.2 Load Tester Development 3.3 Production of Routers 3.4 Production of DWDM systems 3.5 Test Bed creation 3.6 Application testing with IP/MPLS Routers and DWDM systems 60 3.7 Demonstration of test bed Conclusion A new approach to automate the reconfiguration of network topologies for IP/MPLS over DWDM networks is proposed. The proposal envisages creating an embodiment of the concept in a test bed to be set up at C-DOT which closely models the behaviour of real packet networks. The project involves development of network probes, novel optimization algorithms and analysis software which will be demonstrated in the testbed as a unified system. The outcome of this research program will be a set of prototype deliverables. On successful completion, the prototype deliverables will be evolved by C-DOT into a commercial grade product(s) which will be usable for optimizing private, service provider, public and strategic networks in the country.
The Development and Study of The Methods and Algorithms For The Classification of Data Flows of Cloud Applications in The Network of The Virtual Data Center