Business Continuity

&
Disaster Recovery
No DRP and often little motivation for a
DRP.

The level of effort and\or cost required to

create DRP can cause this project to have a
low priority.

“Nice to have" or "just insurance that will not

be used ".
What happened on
August 14 , 2003?
th
Disasters happen every day...its a fact! 
• Disasters cost money so
why suffer by being
unprepared?
• Organizations that survive
typically have:
• management foresight
• tested procedures
• processes
• back-up facilities
• Business Continuity
Planning (BCP)
Downtime Costs Money
Percentage Downtime Per Year (7x24x365)
Availability Days Hours Minutes Cost$ *
95% 18 6 0 $250M
99% 3 15 36 $51M
99.9% 0 8 46 $5,003,312
99.99% 0 0 53 $504,136
99.999% 0 0 5 $47,560
99.9999% 0 0 1 $9,512

Numbers assume $5B yearly revenue run rate.

* Oracle calculated costs and is not associated with the Standish Group Report
Business Continuity Planning vs.
Disaster Recovery Planning
• Both are directed at recovery of operations
• Business Continuity Planning is directed at the recovery and
resumption of business activities across the entire enterprise
• Disaster Recovery Planning is usually directed at the recovery of
information technology systems and business applications, including
corporate data
• BCP addresses Processes, People and Property
• Based on how your business and systems works created, then
tested, and redefined and finally implemented and tested on a
periodic basis.
Business Continuity Planning Phases
• Typically three phases
• Pre-Planning
• Planning
• Post-Planning
• Critical success factor
• Cost is always an issue
• Executive ownership is critical
• Must be a business priority
Phase One: Pre-Planning
• Project initiation and
management
• Establish a need
• Executive management
ownership
• Time and budget allocation
• Risk evaluation and control
• Events and environment issues
• Facilities and process evaluation
• Cost benefit analysis
• Impact analysis
• Disruption and disaster scenarios
• Critical business functions
• Recovery time analysis
 There may be network bandwidth
Phase Two: Planning issues if more than one customer
declares a disaster at a single time
also geographic area issues.
• Strategies
• Alternative organizational recovery
• Operations and information systems
• Recovery time objectives

• Emergency response and operations

• Procedures of stabilization
• Operations center
• Emergency command and control

• Developing and implementing the

plan
 Where do you start?

 The first step is to create a DR team and this includes an:

1.Executive sponsor. Authority to declare a disaster?
2.DR coordinator.
3.Team leaders (there will be several groups and possibly subgroups).
4.Team members.

Primary or backup for position, with every position

having more than one person assigned this to
minimize people as a single point of failure.
Understand the business goals and objectives

• Critical systems?
• Key processes and applications?
• Dependencies on other systems?

Dependencies on other systems and user interface, and the sensitivity

of the data.
 Attempt to quantify their impact relative to the overall business
goals: BIA
• Impact of the failure.
• Probability of failure.
• Estimated incidents (failures).
• Cost of mitigation.
Oracle BCM Business Flow
Disaster Recovery - Business Continuity Planning

Start
A
Does current
Global IT Business N DR plan Y Y
DR plan require Approval received?
passes tests?
Operations modification?
A A
Review changes made
to Global IT
environment (1) Y N N
C
Establish a
multidiscipline team (2) Multi-disciplined Multi-disciplined Multi-disciplined
Identify Business
Disaster Recovery Disaster Recovery Disaster Recovery
Continuity / Disaster
Recovery team Planning Team Planning Team Planning Team
members (3) Identify within current Modify DR plan as Modify new DR plan to
plan areas that require necessary & re-test address reviewers
additional work to plan (11) concerns (14)
mitigate new risk (7) Determine if
Multi-disciplined Develop new DR plan modifications to plan
B
Disaster Recovery (8) requires additional
Planning Team testing (15)
C
Perform business risk
Multi-disciplined
assessment to Multi-disciplined
determine current risk / Disaster Recovery
Disaster Recovery Planning Team
future risk profile (4)
Document & Planning Team
Submit DR plan to Plan requires
communicate business Determine what testing additional testing due
Senior Management for Y
risk assessment results has to be performed on to modifications?
approval (12)
& risk profile to Global DR plan (9)
IT Senior Management C
Team (5) B
Review current
Global IT Senior N
Disaster Recovery plan
Multi-disciplined Management
to determine if new risk
profile is mitigated Disaster Recovery Multi-disciplined
Planning Team Review new / changed
within current DR plan Disaster Recovery
(6) DR plan (13)
Test DR plan (10) Planning Team
Re-submit to Senior
Management for
approval (16)

B
 Documentation:
Progress.
Phase Three: Post-Planning Scheduled Flow.
• Awareness and training of Stakeholders
• Create organizational awareness
• Enhance skills
• Maintaining and exercising
• Coordinate plan exercises
• Evaluate and document exercise results
• Develop process to maintain the plan
• Report results clearly and concisely
• Coordination and communication
• Communication with media, families,
suppliers
Independent of the underlying
• Crisis coordination with first responders,
local authorities systems being recovered.
 Technology?
Match the Tools to the Business Needs

Wks Days Hrs Mins Secs Secs Mins Hrs Days Wks

Recovery Point Recovery Time

Sync. Clustering
Replication
Async. Remote
Replication Replication
Online
Tape or Disk Restore Tape
Backup Restore

prioritization and a cost-benefit analysis

BC/DR Must Address Every Component
• Network Infrastructure
• Data Storage – online, near-line and off-line
• Application servers and their offspring

Any component down = the entire system is un-usable

Clients

Load Balancer
Web Cache Application
Java Server Tier
Clusters
Database
Tier

Only as Good as the Weakest Link

Network Infrastructure

• Wide Area Traffic Manager to direct client traffic to proper site

• Network load balancer to distribute incoming requests
• Dedicated, fast link between sites
• Influences production database performance
• Redundant components and paths
• Network paths to the site and within the site

• Note: Routine maintenance including backups

should still be performed at the site.
BC/DR Techniques for Data Storage

• Snapshots – frequent, within an array, FC, temporary

• Mirrors – frequent, in a different array, FC, temporary
• Replicas – synchronous or async, remote or local, FC or IP, temporary
or semi-permanent
• Near-Line Disk – infrequent, x-platform, FC or IP, BI copy, DLM, or
staging for backup
• Tape Backup – infrequent, FC or IP, required best practice for DR
Application Availability with Local Clustering

Server 1 Server 2
Instance ‘A’ Instance ‘B’

Database

Protects from local server failures

Depends on shared available storage
 Wide Area Clustering
• Extends local clustering model to several sites
• Requires data mirroring or replication

Cleveland
Columbus
Cincinnati
Sandusky
Wide Area Clustering
Site Migration

Failover

Replication
 3. Transferring Data back to Production
Machines:
• Production will need to shift from a hot site back to a
permanent location.

• A process needs to be defined to manage this

migration.

• Synchronize the machines to a specific point in time.

• It should also be noted that this is one of the more

difficult tasks to test.
Key Steps to Success
• Conduct a Business Impact Analysis
• Identify which processes are truly critical and cost of BC
• Prioritize investments in people and technology
• Plan and Implement
1.Hardware Issues.
2.Networking issues.
3.Software issues.
• Test, test, test!!!
Nothing should be assumed or left to chance. Plan for the worst case.
• Support an external validation effort of the DRP exercise
• Review the business continuity plan when the business process
changes
Decision Factors

• Prioritize business functions

• Work with business units for business continuity to determine IT
disaster planning levels
• Determine level of acceptable risks
• Distance for secondary site
• Hot versus cold site/Warm site
• Mirror data versus backups
• Redundant servers with failover versus build new server at time of disaster
 Summary

 The DRP is a living document that is refined over several iterations

and update over time.

 No matter how good it is it probably will fail during the first

execution.

 The key is to continue to improve the plan so that will work if and
when it is ever needed.