Professional Documents
Culture Documents
Draft 3
Draft 3
Draft 3
This is to certify that the Project Work titled Building Disaster recovery capabilities has been successfully completed at MphasiS Ltd.by Dinesh Kumar, under my guidance, in partial fulfillment of the Post Graduate Diploma in Industrial Management at National Institute of Industrial Engineering (NITIE), Mumbai. I wish him a bright and prosperous future.
Acknowledgement ................................................................................................................................... 3 Executive Summary ................................................................................................................................. 3 Company Overview.................................................................................................................................. 3 Applications ......................................................................................................................................... 4 BPO ..................................................................................................................................................... 4 Infrastructure Services ......................................................................................................................... 4 Payments Solutions.............................................................................................................................. 5 Project Overview ..................................................................................................................................... 6 Need for the project ............................................................................................................................ 6 Objectives of the project ...................................................................................................................... 6 Key Deliverables .................................................................................................................................. 6 Literature Survey ..................................................................................................................................... 6 BCP Planning Methodology .................................................................................................................. 6 Building Disaster recovery capability in Service desk business .................................................................. 7 Identification of critical requirements .................................................................................................. 7 Defining Maximum Tolerable Period of Disruption ............................................................................... 7 Defining Recovery Time Objectives ...................................................................................................... 7 Identification of Alternate Site ............................................................................................................. 8 Identification of resource and infrastructure requirements .................................................................. 8 BCM Response ..................................................................................................................................... 8 Training and testing ........................................................................................................................... 11 Maintenance and updating ................................................................................................................ 11 Identification and prioritization of possible risks in Disaster Recovery strategy ...................................... 11 Risk Identification .............................................................................................................................. 11 Risk Prioritization ............................................................................................................................... 16 Analysis of current recovery strategies .................................................................................................. 18 Account A .......................................................................................................................................... 18
Acknowledgement
This project is the result of two months of work whereby I have been accompanied and supported by many people. It is a pleasant aspect that I have now the opportunity to express my sincere thanks to all of them. I would like to express my gratitude to all my professors, my institute, team members & the company. Their generous help and support enabled me to complete this study within the stipulated time period. I am very obliged to National Institute of Industrial Engineering (NITIE)& MphasiSLtd.for giving me the opportunity to work on this project & would like to specially thank Dr. V. B. Khanapuri (Associate Professor, NITIE) ,Mr. MakarandVaze ( Project Manager, MphasiS Ltd.) and Mr. RitieshSethi (Associate Vice President,MphasiS Ltd.) who provided guidance, help and cooperation at every stage of the project. I would like to thank my colleagues and friends who have directly or indirectly provided me assistance and helped to materialize this project. However I take the responsibility for all my shortcomings. I, in all sincerity, hope that my efforts will be appreciated.
Executive Summary
The Mphasis India BPO Service Delivery Centre provides Service Desk support to various corporate clients. In order to ensure that the Service Desk services are available at all times, the BPO has developed a disaster recovery plan to provide basic Service Desk service in the event of a disaster. The objective of the project is to analyze the disaster recovery strategies for four accounts so as to improve the current disaster recovery strategies. The project expected identification and prioritization of possible risks and analysis of recovery strategies of four corporate accounts.
Company Overview
3
MphasiS (BSE: 526299, NSE: MPHASIS), a unit of Hewlett-Packard Co., is an information technology services company based in Bangalore, India. MphasiS is certified with ISO 9001:2008, ISO/IEC 27001:2005 and is assessed at CMMI v 1.2 Level 5.It is the sixth largest IT company in India with more than 38,000 employees as of 2010. The company has 29 offices in 14 countries with delivery centers in India, Sri Lanka, China, North America and Europe. MphasiS is engaged in providing information technology services to its customers around the world. MphasiS provides integrated solutions that include business process outsourcing, infrastructure technology, and application services. The application service offered by the company includes application development as well as applications maintenance and support services. The markets served by the company are financial services & insurance, healthcare, manufacturing, government, transportation, communications, and consumer & retail industries.
Applications
MphasiS is uniquely positioned to cater to the global strategic outsourcing demand and have integrated changing environments into Applications service portfolio. Full range of application services helps to invest more strategically in core business activities, while improving ROI from tight IT budgets.
BPO
MphasiS, a leader in providing end-to-end Business Process Outsourcing services, provides high quality, value-added voice and transaction-based services to Fortune 500 companies worldwide. Being the early entrant in the BPO space, they are experts in providing strategy, solutions, and services to solve complex business issues and achieve results. By coupling their Business Process Outsourcing services and in-depth industry specific knowledge, they provide business focused solutions tailored to clients strategic goals.
Infrastructure Services
MphasiS Infrastructure services have the distinguished blend of classic HP pedigree clubbed with skilled scalable workforce and firm focus on service delivery quality. Whether you are looking to reduce costs, improve business-to-IT alignment, drive innovation, or reduce risks, MphasiS offers a comprehensive suite of remote Infrastructure management services. MphasiS helps optimize clients infrastructure through a mature global delivery model, standardized processes, adoption of IT Infrastructure Library (ITIL) framework and global service network, robust private network with built in redundancies. This network enables secure connection with clients 24x7x365.
Payments Solutions
MphasiS offers a gamut of services such as consulting, application development, modernization, integration, and testing in payments arena to clients worldwide. The array of solutions from MphasiS for wholesale payments, retail payments, PCI DSS compliance, and other value added services are well designed to help clients reduce processing costs, retain investments, streamline processes, transform applications, identify and deploy new channels thereby increasing their revenues streams.
Applications
BPO
Infrastructure Services
Remote Monitoring Center
Payments Solutions
Retail & Wholesale Payments
Applications Development
Industry Offerings
Applications Management
Service Offerings
Workplace Services
Business Practices
Network Services
Technology Practices
Security Services
Consulting Services
Solutions
Project Overview
Need for the project
Business continuity plans are not taken up very seriously and disaster recovery strategies were not defined with proper rationale for many BPO accounts of the Mphasis. This is so because disaster situation had not been faced since the inception of most of the accounts. This project aims to identify the possible risks in the disaster recovery strategies and to recommend the improvements in the disaster recovery strategies of the four accounts.
Key Deliverables
List of the possible risks with corrective actions Prioritized list of the identified risks Risks in the current recovery strategies of the four account Recommendations to avoid the risks associated with current recovery strategies of the four accounts
Literature Survey
BCP Planning Methodology
- Infrastructure Requirements: Following are the supporting infrastructure required for an agent to perform his job o Seats o Desktops and laptops o VOIPS (Voice over Internet Protocol) o Network / LAN, power points o Landline, mobile, fax, printer - Application requirement o DW software: It is the software used to maintain the status of the agents and route the calls to the agents. o EKMS: Enterprise Knowledge Management System contains all the supporting documents for the agents to find the solutions to the problems faced by caller.
BCM Response
Project BCP Team
The Project BCP team required to respond to an incident and subsequently initiate recovery procedures are as follows: y
Project/Account Owners
Project owners are the personnel responsible for all the decisions made regarding the project. These personnel are to be informed when the BCP is to be invoked, as they are to take the key decisions during the incident and the recovery phases. y
Project/ Account BCM Coordinator(s)
BCM Coordinators are the personnel responsible for authorizing activities, communications and response teams that may be necessary for supporting the incidents in consultation with IMT and Project Owners. They aid the Project Owners in determining whether a disaster should bedeclared and which Teams should be mobilized. They also Report progress and problems as required. y
Recovery Team
The Recovery Team consists of resources that are critical to the delivery of the project including team members, team leads and project managers. Their role is to initiate the recovery of critical activities identified during the Business Impact Analysis (BIA). The details of this team are depicted the call tree in Table below and referenced in Table 2 : Recovery Team Call Tree
Project Team Detail Role Project Owner Name Contact Details Backup Team Member Name of the Backup team person Contact Details
Contact No. BCMS Coordinator Name of BCMS Coordinator Office: Contact No. Home: Contact No. Cell Phone: Contact No. Recovery Team Refer Table 2: Recovery Team Call Tree for more details
Contact No. Office: Contact No. Home: Contact No. Cell Phone: Contact No.
BCMS Coordinator
Name of Coordinator
BCMS
Member 1
of
project
Member 2
of
project
member
10
Risk Identification
Following major risks with the recovery strategy were identified:
11
the customer and it is defined in SLA. In later case, it is calculated by comparing the revenue loss or penalty to the expected profit from the account.
1
Factors 0-4 hours` 4 hours 24 hours
10
15
1 day - 3 days
3 days - 7 days
1 week - 2 weeks
Number of billable hours lost Primary Operational impact Any other financial Impact Total Operational Impact
This operational impact could be compared with the expected profit from the account or the profit above minimum profit margin required. MTPoD could be defined for each service separately.
would be difficult to identify the infrastructure requirementsas it would depend on the number of agents. If infrastructure requirements are not defined clearly there are chance to raise request for more or less than requirement. Corrective action: Number of agents required to meet the service level requirements defined in RTOshould be clearly defined. It could be derived from the peak level requirements and the service level of the RTO. For example, if 100% service level is required and the peak level agents requirement is 45 then 45 seats needs to be booked. And if 33% service level is required then 15 seats needs to be reserved and it should be scaled to 45 within MTPoD. Seats Required = (No of agents in peak hours) x ( service level requirement) One or two extra seats may need to be added for operations manager and assistant operations manager.Other infrastructure requirements could be defined using the number of agents as follows: Desktops = No of agents Landlines = 1 or 2 for operation manager and assistant operations manager Printers = 1 No. of ports = Total number of seats VOIP = Total number of seats LAN = Total number of seats Number of power points = 2 x (number of agents) + 2 (Each agent requires two power points while manager requires only one) Applications: Internet explorer, DW, Avaya interaction center, EKMS
13
Mphasis network should be available at the alternate site. As the Mphasis has offices at Mumbai and Pune, they can act as alternate site for each other. Mumbai-Pune: Distance = 160 kms Travel time by road = 3 to 4 hrs Travel time by train = 3hrs
14
If team members are not aware of the recovery process or not practiced the BCP procedure they may find it difficult to execute it at the time of disaster. Corrective action: Regular exercise should be done for the BCP procedures and escalation process should be practiced regularly. Currently it is practiced annually or semi-annually. Quarterly tests could be taken to find out the understanding of the disaster recovery procedures. If tests results are not satisfactory training needs to be given for that member.
15
Risk Prioritization
Operations Managers and assistant operations manager were asked to identify each risk as high, medium or low. Based on the following description: High Risk:Risks that could lead to failure of recovery or could lead to exceed the MTPoD.These risks could lead to serious business loss. Medium Risk : Risks that causes large delays and lead to exceed RTO and sometimes MTPoD. These could cause the business loss and could affect the service levels. Low Risk: Risks that could cause operational problems and could lead to delays in RTO. Business is not affected but could cause delays and interruption in recovery. Following values were assigned to the three levels: High :3
Medium : 2 Low :1
Averages are calculated for each risks and categorization is done as follows: 1 to 1.5 : Low 1.5 to 2.5 : Medium 2.5 to 3 : High
Sl. No.
Risks
R1 R2 R3 R4 R5 R6 R7 R8 Average
Risk category
2.9
High
2.3
Medium
16
Resource requirements are not quantified Alternate site is quite far Alternate site is in the same city Alternate contact is not defined in the BCP team Unavailability of BCP copies at the time of disaster Unawareness and inadequate training of BCP BCP is not updated with latest contacts Availability of infrastructure requirements at alternate site has not been verified
2.4
Medium
1.4
Low
1.6
Medium
1.9
Medium
1.8
Medium
2.4
Medium
1.6
Medium
10
2.8
High
Following list of prioritization is created from the average score: 1) MTPoD is not defined 2) Availability of infrastructure requirements at alternate site is not verified 3) Resource requirements are not quantified 4) Unawareness and inadequate training of BCP 5) RTO is not defined 6) Alternate contact is not defined in the BCP team 7) Unavailability of BCP copies at the time of disaster 8) Alternate site is in the same city 9) BCP is not updated with latest contacts 10) Alternate site is quite far 17
Account A
Account Details: Following are the important details about the account: Number of Agents= 96 approx. Approx. no of calls = 22,000 per month Contract is with HP and HP deals with customer
Current Recovery strategy: For US/Canada and EMEA, the alternative recovery site is Tower 4, Magarpatta, Pune. Strategy is to have the calls routed through a DR skill to a specific defined set of employee ids. 3 agents would be creating the ticket number with the customer details and problems. 18
Tickets would be sent to the Mumbai site when it is up. Risks with the strategy: No business impact analysis is done and MTPoD and RTO are not defined: No plans if the disaster effects the functioning of Mumbai site for more than 24 hrs. Plan has been left to discussion at the time of disaster. Three agents would not be enough to handle peak hour volumes which requirearound 40 agents. Recommendations: Business impact analysis must be done to define MTPoD and RTO or it shouldbe agreed upon with HP. Based on RTO and service level agreements alternate site and recovery requirements should be set.
Account B
Account Details: Number of Agents= 102 approx. Approx no of calls = 24,000 per month New account, started in Oct, 2010 BCP plan is yet to be signed off Current Recovery strategy: MTPoD is 120hr according to contract RTO is set as 90 hrs 30 % of services need to be rendered within 24 hrs Alternate site 1: Tower CC4, Magarpatta, Pune Alternate site 2: Infinity Park, Mumbai Number of seats required = 30
19
LAN (30), VOIP (30) , Power Points (58), Laptops (2), printer (1),etc are required Risks with current strategy: RTO of 90 hrs is not defined with the service level requirements. 90 hrs that is 75% of MTPoD is quite large. No rationale behind 30 seats requirements. It would not be possible to handle peak hour call volume which requires 45 agents. Recommendations: RTO must be defined with the service level requirements. RTO is generally less than 50% of the MTPoD. 45 seats are required if 100% service level needs to be met because peak hour requirements is 45 agents.
If 30 seats are used then what service level could be met should be defined and agreed upon.
Account C
Account Details: Number of Agents= 125 approx. Approx no of calls = 27,000 per month BCP plan is yet to be signed off although the account is quite old Current Recovery strategy: RTO = 9 hrs Call transferred to HP, Budapest and HP would handle the calls Risks with current strategy: MTPoD is not defined Alternate site is not defined in India Recovery requirements have not been identified Recommendations: 20
BIA must be done to define the MTPoD Alternate site could be Mumbai. Mangalore agents who handles only mails and chats can be trained to manage the voice calls in case of disaster Service level should be defined with RTO
Account D
Account Details: Number of Agents= 66 approx. Approx no of calls = 19,000 per month Standard format is not used for BCP Current Recovery strategy: Calls will be routed to Pune EON Kharadi and as many agents as possible will be moved to Pune EON Kharadi. Critical Service Desk support is to be provided within 1 hour Basic Service Desk services would be available within 4 hours Risks with current strategy: MTPoD is not defined Alternate site is in the same city Recovery requirements have not been identified Recommendations: BIA must be done to define the MTPoD Alternate site could be Mumbai Requirements should be defined
21
Miscellaneous
Following observations are common to all the accounts: Total number seats required at a recovery site which is an alternate site to the accounts which are located at same site should be compared with available seats.If there is significant difference in the requirements and availability, alternate sites should be adjusted. Tests for the awareness of the BCP should be conducted quarterly and training should be conducted if the results are not satisfactory
22