High Level steps for DR Architecture

PS (Public Storage) has hosted their Production (US-Central1 region) and DR (US-EAST4 region)
Environment in GCP. Keeping this in mind, the drafted design provides disaster recovery and
restoration using GCP’s native backup and DR service.

IN this document, we will cover the restoration of below resources:

Virtual Machines

Global Load balancers

Regional Load balancers

Cloud Storage

Expected RTO (Recovery Time Objective):

Expected RPO (Recovery Point Objective):

Subnet Range Subnet Name Restoration required
PROD(US-CENTRAL1)
172.20.0.0/24 ps-sb-prod-anc Yes
172.20.1.0/24 ps-sb-prod-domain-infra No
172.20.2.0/24 ps-sb-prod-corp-db No
172.20.3.0/24 ps-sb-prod-pci-db No
172.20.4.0/24 ps-sb-prod-pci-app Yes
172.20.5.0/24 ps-sb-prod-pci-web Yes
172.20.6.0/24 ps-sb-prod-financial Yes
172.20.8.0/26 ps-sb-prod-pan-fw No
172.20.7.0/24 ps-sb-prod-member-svr Yes
 172.20.15.0/24 ps-sb-dev-app Yes

172.20.48.0/24 ps-sb-prod-api Yes

172.20.49.0/26 ps-sb-prod-api-pan-fw No

172.20.64.0/24 ps-sb-qa-api Yes

172.20.65.0/26 ps-sb-non-prod-api-pan-fw No
172.20.66.0/24 ps-sb-stage-api Yes

172.20.80.0/24 ps-sb-transit No
172.20.81.0/26 ps-sb-transit-pan-fw No

172.20.96.0/24 ps-sb-mgmt No
172.20.97.0/26 ps-sb-mgmt-pan-fw No

172.20.112.0/24 ps-sb-prod-egress No
172.20.113.0/26 ps-sb-prod-egress-pan-fw No
DR(US-CENTRAL1)
172.20.0.0/24 ps-sb-dr-anc Yes
172.20.129.0/24 ps-sb-dr-domain-infra No
172.20.130.0/24 ps-sb-dr-corp-db No
172.20.131.0/24 ps-sb-dr-pci-db No
172.20.4.0/24 ps-sb-dr-pci-app Yes
172.20.5.0/24 ps-sb-dr-pci-web Yes
172.20.6.0/24 ps-sb-dr-financial Yes
172.20.136.0/26 ps-sb-dr-pan-fw No
172.20.7.0/24 ps-sb-dr-member-svr Yes
172.20.15.0/24 s-sb-dr-dev-app Yes

172.20.48.0/24 ps-sb-dr-api Yes

172.20.177.0/26 ps-sb-dr-api-pan-fw No

172.20.192.0/24 ps-sb-dr-mgmt No
172.20.193.0/26 ps-sb-dr-mgmt-pan-fw No

172.20.208.0/24 ps-sb-dr-transit No
172.20.210.0/26 ps-sb-dr-transit-pan-fw No

172.20.224.0/24 ps-sb-dr-egress No
172.20.226.0/26 ps-sb-dr-egress-pan-fw No

High level steps for the DR process.

Palo Alto Route Changes

Change in Virtual Router.

 Connect to DR PAN Firewall on 172.20.210.6 using AD credentials.

 Network>Virtual Routers>default>Static Routes
 Delete the current route named ‘route-to-prod’
 Create new routes mentioned below.

172.20.0.0/20 Ethernet1/5 172.20.145.1

172.20.48.0/20 Ethernet1/4 172.20.177.1

Virtual Machines

 Confirm the health status of Google compute Engine in US-Central1(Iowa) from GCP cloud
status portal (https://status.cloud.google.com/regional/americas)
 Make sure, there are no such service interruption in US-East4(Northern Virginia)
 Pause/stop the AD replication between production and DR.
 Pause/stop the PCI DB replication between production and DR.
 Disconnect the VPN tunnels by disabling the BGP on both production and DR gateway.

Project: ps-shared-host-prj-a6
Hybrid Connectivity > VPN > ha-prod-to-dr-vpn-t2
Edit the BGP session, disable BGP Peer

Repeat the same process for secondary tunnel.

Hybrid Connectivity > VPN > ha-vpn-tunnel1

Edit the BGP session, disable BGP Peer

 Connect to the DR PAN Firewall (172.20.210.6), Update the below static routes.
 Network>Virtual Routers>default>static Routes

Name: route-to-prod
Destination: 172.20.0.0/17
Interface:
Value:

Once done, commit to the change.

 Reach out to robert.shepherd@zayo.com for changes being carried out in Velo orchestrator.
This would involve the forwarding the property and Glendale traffic towards DR sdwan
firewall instead of production sdwan firewall.

 Spin up the VM using the Snapshot which is taken using Backup and DR Google cloud
Service.

 Go to Backup and DR Management Console and click on the Backup and Restore Tab 
Select the Restore.

Once you're In Restore window search for server, and click on that say next
Select the Snapshot as shown in the screenshot  and click on Mount Button.
Once you are in the Mount window, select Mount as New GCE INSTANCE.

Note: If we want to spin a new VM in another Project or in DR Region then you can select the --
Mount as New GCE INSTANCE.

If you want to restore in the Same VM then select  Mount TO EXISTING GCE INSTANCE
 Select the or Change the Project Name, which Project we have to Build the VM using
Snapshot.
 Change the Region as per your requirement and Zone
 Please select the Service account of the target Project and paste in the Service account box.
Select the Network and Subnet.

Internal IPV4 IP Section if you want to select.

 After restoring the VM’s, make sure, the relative applications are now pointing to the correct
set of DB’s.

Global Load Balancers

As the name suggested, these are global in nature and do not restrict themselves to any
region, so in case of US-Central1 being unavailable, we would need to change the backend
for the External (Global) load balancers.

Regional Load balancers

The internal (Regional) load balancers would be available in DR with the same name and IP
address assigned to it but the backend would be dummy instance groups which would need
to be replaced by correct instance groups, once we restore the production instances in DR.

Cloud Storage

Buckets(multi-regional): No changes required, will be available even if US-Central1 goes

down.
Buckets (Regional): Create another bucket in us-east4 with a different name and run the
below command to copy over all the data. Make sure, the user has enough access role to be
able to run below commands on both the buckets. (Please refer IAM)
gs cp gs://bucket1 gs://bucket2