DB2 HADR Pacemaker Luis Garmendia

Luís Garmendia
High availability
 An outage is any disruption in the ability of the
database solution to serve user applications.
Unplanned outages
 Examples of unplanned outages include:
• The failure of one component of the system, including
hardware or software failure.
• Invalid administrative or user application actions such
accidentally dropping a table that is needed for
business-critical transactions.
• Poor performance due to suboptimal configuration, or
inadequate hardware or software..
Indice
 Introducción a HA Clusters
 Pacemaker
 Prerequisites and installation of Pacemaker
 Db2cm. The db2 cluster management utility
 Configuring a clustered environtment using
db2cm
 Maintaining a Pacemaker cluster domain
 References
High availability
 Planned outages
• Upgrade. Upgrading your software or hardware
can sometimes require a partial or a full outage.
https://www.ibm.com/support/producthub/db2/docs/content/SSEPGG_11.5.
0/com.ibm.db2.luw.admin.ha.doc/doc/c0006354.html
High availability strategies
System monitoring
• Collecting statistics about the components of your
solution to facilitate workload balancing or detecting
that components have failed.
Load balancing
• Transferring some workload from an overloaded
component of your solution to another (Db2 purescale)
Maximizing performance
Minimizing the impact of maintenance
• (automated maintenance )
Database Logging
• Linear logging
High availability through
redundancy
Redundancy
Having secondary copies of each component.
At system level:
 Uninterrupted or backup power supplies
 Multiple network fibers between each component
 Bonding or load balancing of network cards
 Multiple hard drives in a redundant array
 Clusters of CPUs
Db2 High Availability Disaster
Recover (HADR):
Redundancy
Db2 High Availability Disaster
Recover (HADR):
Redundancy
At database level having two databases
• A primary database that normally processes all or
most of the application workload;
• A secondary database (standby database) that can
take over the workload if the primary database fails
High availability through Failover
Failover
• Transferring all workload from a failed component of
your solution to a secondary component.
Idle Standby (HADR)
• A primary system processes all the workload while a
secondary or standby system is idle.
• In an high availability disaster recovery (HADR) setup,
you can have up to three standbys and you can
configure each standby to allow read-only workloads.
Mutual Takeover (purescale)

In this configuration, there are multiple systems, and each
system is the designated secondary for another system.
High availability through clustering
 A cluster is a group of connected

machines that work together as a single
system.
 When one machine in a cluster fails,
cluster managing software transfers the
workload of the failed machine onto other
machines.
Heartbeat monitoring
To detect a failure on one machine in the cluster,
failover software can use heartbeat to confirm
availability.
Heartbeat monitoring involves system services
that maintain constant communication
between all the machines in a cluster.
If a heartbeat is not detected, failover to a
backup machine starts.
Virtual IP address takeover

• When there is a failure on one machine in the
cluster, cluster managers can transfer
workload from one machine to another by
transferring the virtual IP address from one
machine to another.
• This workload transfer is transparent to the
end-client application, which continues to use
the same IP address to connect to the
database, unaware that the physical
machine's IP address maps has changed.
Virtual IP address takeover
Luís Garmendia
Cluster Managing Software
The Db2 High Availability Feature enables

integration between IBM Db2 server and
cluster managing software.
 Clustering software automates the

failover of resources, such as processes,
applications, storages, and IP addresses.
 Clustering software monitors the health of
the network, hardware, and software
processes, detects and communicates any
fault, and automatically fails the service
and associated resources over to a
healthy host in the cluster
 Some cluster Managing Software are:

• IBM PowerHA SystemMirror for AIX
• Microsoft Cluster Service for Windows
operating systems (MSCS)
• LifeKeeper for Linux/Windows
• Tivoli SA MP.
• Tivoli System Automation is designed to provide
a clustering in a heterogeneous environment.
• Pacemaker
Db2haicu (old) vs db2cm (new)
DB2 High Availability Instance Configuration Utility
(db2haicu) is a text-based utility that can be used to
configure and administer your highly available
databases in a clustered environment.
Pacemaker is included in db2 v11.5.4, and uses

db2cm instead of db2haicu to configure the cluster.
Luís Garmendia
Db2 cluster manager utility
db2cm
Db2cm command syntax
db2cm -create -cluster
Creates a new Pacemaker cluster domain with two hosts and
the public Ethernet resource.
The following parameters are mandatory:
-domain domain-name
Specifies the name of the Pacemaker domain
-host host-name
Specifies the name of the host. This is also used as a host in
the HADR cluster.
-publicEthernet public-Ethernet-device-name
Specifies the name of the public network adapter
Create HADR Database resources
Create the HADR database resources.

/home/db2inst1/sqllib/adm/db2cm -
create -db SAMPLE
-instance db2inst1
Create VIP resources
(Optional) Create the VIP resources for the newly

created database.
sqllib/adm/db2cm -create -primaryVIP

<IP address>
-db SAMPLE
–instance db2inst1
CRM STATUS
db2cm -delete
-cluster
db2cm -delete -cluster
-publicEthernet public-Ethernet-device-name -host host-name
db2cm -delete -publicEthernet eth1 -host host1
-instance instance_name -host host-name-of-the-instance
db2cm -delete -instance db2inst1 -host host1
-db database-name -instance local-instance-name
db2cm -delete -db employee_db -instance db2inst1
-primaryVIP -db database-name -instance local-instance-name
db2cm -delete -primaryVIP -db employee_db -instance db2inst1
-standbyVIP -db database-name -instance local-instance-name
db2cm -delete -standbyVIP -db employee_db -instance db2inst1
-qdevice
Removes the quorum device daemon from the corosync cluster nodes.
db2cm -delete -qdevice
If quorum device is no longer being utilized by other clusters, as root:
systemctl stop corosync-qnetd rm -rf /etc/corosync/qnetd/nssdb
db2cm
-disable
Disable automation for all Pacemaker resources of all Db2 instances in
the Pacemaker domain.
db2cm -disable
-enable
Enable automation for all Pacemaker resources of all Db2 instances in
the Pacemaker domain.
db2cm -enable
-list
Display the cluster configuration and status information.
db2cm -list
-dump
Export cluster information pertaining to the local host to a compressed file
in the local directory. Use this for problem determination
db2cm -dump ¡
db2cm
-export
Backup the cluster configuration to a file which can be used with the -
import option to quickly redeploy the cluster on the same set of hosts.
db2cm -export /tmp/backup.conf
-import
Restore the cluster configuration from a previously saved configuration
generated by the -export option.
db2cm -import /tmp/backup.conf
-copy_resources resourceAgentsPath -host host-name
Copies resource agent scripts from the specified path to the resources
path on the specified host.
db2cm -copy_resources
/tmp/20191231_Db2HADR_Pacemaker_Beta1/Db2AgentScripts/
-host host1
-help
Print usage information for db2cm.
db2cm -help
Luís Garmendia
Pacemaker
 Pacemaker is an open-source, high availability
cluster manager software integrated with Db2
Advanced Edition and Db2 Standard Edition on
Linux
 It provides high availability and disaster recovery
capabilities for on-premises deployments and
non-container cloud environments, such as
Amazon Web Service (AWS).
Pacemaker
 Db2 Version 11.5 Mod Pack 4

includes Pacemaker as a cluster manager
for automated fail-over to HADR standby
databases.
 Suported in
• RedHat Enterprise Linux (RHEL) 8.1
• SuSE Linux Enterprise Server (SLES) 15
SP1
RA
Corosync
 Communication layer
 Pacemaker utilizes the Corosync Cluster
Engine, an open source group communication
system software, to:
1. Provide a consistent view of cluster topology,
2. Ensure reliable messaging infrastructure so
that events are executed in the same order in
each node
3. To apply quorum constraints.
Pacemaker Resources
 Resources are set of Db2 defined entities

where states are to be monitored, started,
or stopped. This includes:
1. The Db2 member process,
2. HADR capable databases,
3. Ethernet network adapters,
4. Virtual IP addresses.
Pacemaker Constraints
 Constraints are rules setup during cluster

creation to augment the behavior of processes:
• Location constraint - where resources can run.
• Ordering constraint - the order of certain
resource actions must occur in.
• Co-location constraint - the dependency of the
location of one resource on another.
 Constraints Examples
 https://www.ibm.com/support/knowledgecenter/SSEPGG_11.5.0/com.ibm.db2.luw.admin.ha.doc/doc/c_pacemaker_base_component.html
1. Intance resource db2_draping1_gerry_0 prefers to run on

the draping1 host:
location prefer-db2_draping1_gerry_0
db2_draping1_gerry_0 100: draping1
2. The database resource will only run if the Ethernet network

adapter eth1 is healthy
location loc-rule-db2_gerry_gerry_SAMPLE-eth1-
talkers1 db2_gerry_gerry_SAMPLE-clone \ rule -
inf: db2ethmon-eth1 eq 0
3. The database resource starts before the primary

VIP resource does.
order order-rule-db2_gerry_gerry_SAMPLE-
then-primary-VIP Mandatory:
db2_gerry_gerry_SAMPLE-clone:start
4. The following co-location constraint ensures that

the primary VIP is running on the same host as the
primary HADR database.
•colocation db2_gerry_gerry_SAMPLE-
primary-VIP-colocation inf:
db2_gerry_gerry_SAMPLE-primary-
VIP:Started db2_gerry_gerry_SAMPLE-
clone:Master
RA
Pacemaker Base Component

Resource set
A group of resources under the effect of a specific
constraint.
Resource model
Pacemaker resource model for Db2 refers to the
pre-defined relationship and constraints of all
resources.
The resource model is created as part of the
cluster setup using the db2cm utility with the -
create option.
RA
Pacemaker Base Component

 Resource agents
 Resource agents in Pacemaker are the Db2 user
exits which are a set of shell scripts developed
and supported by Db2 to perform actions on the
resources defined in the resource model.
RA
Pacemaker Resource agents
 Resource agents
 A total of three resource agents are provided,
copied at
/usr/lib/ocf/resource.d/heartbeat
 db2ethmon
The resource agent to monitor the defined
Ethernet network adapter.
This is at host level.
RA
Pacemaker Resource agents
 Resource agents
 db2inst
The resource agent to monitor, start, and stop
the Db2 member process.
This is at the Db2 instance level.
 db2hadr
The resource agent to monitor, start, and stop
individual HADR-enabled databases.
This is at the Db2 database level.
RA
Pacemaker Cluster domain leader
 One of the nodes in the cluster will be elected as

the Domain Leader (also known as the
Designated Controller (DC) in Pacemaker terms)
where the Pacemaker controller daemon residing
on the DC will assume the role to make all cluster
decisions.
 A new domain leader will be elected if the current
domain leader's host fails.
RA
Networks in a Pacemaker cluster
 To configure elements of your Pacemaker cluster

domain that are related to networks, you can
use Db2 cluster manager (db2cm) utility to add
a physical network to your cluster domain.
 A physical network is composed of:
1. network interface cards,
2. IP addresses,
3. subnet masks.
RA
 Network interface cards

• A network interface card (NIC) is hardware that
connects a computer (also called a cluster node)
to a network.
• When you use db2cm to add a physical network
to your cluster domain, you specify the name of
the NIC as well as the host name of the
computer to which the NIC belongs to.
RA
 IP addresses
• An Internet Protocol address (IP address) is a
unique address on a network, for example:
129.30.180.16.
 Subnet masks
• A network can be partitioned into multiple logical
subnets using subnet masks
• When you use db2cm to add an IP address to
your cluster domain, you have an option to
specify the subnet mask for the IP address.
RA
 Virtual IP address (VIP)

• Configuring a VIP resource via db2cm is
supported for both primary and standby.
• Network adapters that are in different subnets
cannot be added to the same network.
• A VIP address cannot failover between different
subnets or virtual local area networks.
RA
Quorum devices support on

Pacemaker
 A quorum device helps a cluster manager make cluster
management decisions when the cluster manager's
normal decision process does not produce a clear
choice.
 When a cluster manager has to choose between multiple
potential actions, the cluster manager counts how many
cluster domain nodes support each of the potential
actions, then it chooses the action that is supported by
the majority of cluster domain nodes. If exactly the same
number of cluster domain nodes supports more than one
choice, then the cluster manager refers to a quorum
device to make the choice.
RA

Pacemaker
 Two-node quorum
• This is the default mechanism. There is no tie-
breaker mechanism and is therefore prone to
split-brain scenario. It is not recommended for
production environments.
 QDevice quorum
• QDevice is similar to a Network IP tiebreaker in
the sense that it requires an external resource
accessible by all hosts in the
current Pacemaker cluster.
RA

Pacemaker
 QDevice quorum
• The quorum decision logic require the resource
to be located on a separate host similar to a
majority quorum requirement.
• However, it is considerably simpler in terms of
setup as the host with the QDevice does not
need to be part of the Pacemaker quorum. In
short, the QDevice is the best blended quorum
solution that combines reliability and simplicity.
RA

Pacemaker
RA

Pacemaker
 QDevice quorum
• Facts about QDevices:A quorum device acts as the third-party
arbitration device for the cluster. Its primary use is to allow a cluster
to sustain more node failures than standard quorum rules allow.
• As seen in the image above, it is a Corosync daemon process --
QDevice daemon (corosync-qdevice) running on each node in the
cluster. It provides a configured number of votes to the quorum
subsystem based on a third-party arbitrator's decision.
• This third-party arbitrator is a separate Corosync QNet daemon
(corosync-qnetd) running on a separate host (not part of the
cluster) and contributes to the deciding vote of the corosync-qdevice
logic that ultimately decides the surviving side in a split-brain
scenario..
RA

Pacemaker
 QDevice quorum
• Both QDevice daemon and QNet daemon are provided
with different software packages and must be installed
separately. The former must be installed on each host in
the cluster (Host1 and Host2 in the image above), while
the latter is only required on a separate host not part of
the cluster (Host3 in the image above).
• The QNet daemon can be used as the arbitrator for
another cluster (in the above image, two clusters sharing
the same host with qnetd process) given that all clusters
have a unique name.
RA

Pacemaker
 Majority quorum (not supported yet)
• The Majority Quorum avoids split-brain
scenarios by adding a third node to the cluster
for arbitration. In a split-brain scenario,
whichever side successfully acquires the third
node is the surviving side.
• The difference between QDevice and Majority
Quorum is that the third node is fully integrated
into the cluster.
RA

Pacemaker
The QDevice quorum is the recommended quorum
mechanism for Db2.
 This requires a third host to install the
corosync-qnetd software (available in the public
IBM site) to act as the arbitrator.
 The host itself is not required to be part of the

cluster and does not require the Db2 server to
be installed.
RA

Pacemaker
Table 1. The advantages and disadvantages of each quorum type
Quorum type Advantages Disadvantages
Two Node •Simplest setup •Potential split-brain
•No additional hardware or scenario leading to dual
software configuration primary phenomenon
QDevice •More reliable than Two- •Requires a TCP/IP accessible
Node quorum host from the primary and
•No need to include the third standby hosts.
host as part of the cluster
•No need to include the
full Pacemaker cluster
software stack on the third
host. Only one Corosync
RPM is needed
Majority •More reliable than Two- •Need to include the

Node quorum full Pacemaker cluster stack
installed and configured on
the third host
Luís Garmendia
Before Installing pacemaker
The following steps are only required to run once on any one of the
hosts by root. There is no need to run them in both hosts.
1) Connect to one of the Red Hat linux hosts, desde una terminal:
ssh inst491@10.250.0.1 Password: ibm2blue
ssh inst491@10.250.0.9 Password: ibm2blue
Db2 en: docker exec –it server /bin/bash
Correspondencia entre /home/inst491/hello-
docker/serverA/data y en docker /database
Db2cm en /home/inst491/hello-
docker/serverA/data/config/db2inst1/sqllib/adm
Pacemaker libraries en: /home/inst491/hello-
docker/serverA/data/tmp
su root Password: sinensia
Before Installing pacemaker
/etc/hosts
Add hosts short names pacemaker1 y pacemaker9 in /etc/hosts
(psswd de root: sinensia)
Before
After
:
Users and groupid
 users must ensure that the instance owner,

fenced user, other users and their associated
groups are created
groupadd db2iadm1
useradd -g db2iadm1 -m -d
/home/db2inst1 db2inst1
Ping
•On ip-10-250-0-1, ping ip-10-250-0-9 works.
•On ip-10-250-0-9, ping ip-10-250-0-1 works

Passwordless secure shell (SSH)
 Passwordless secure shell (SSH) for root and

instance user IDs
Passwordless secure shell (SSH)
 Passwordless SSH for both root and instance

user must be configured between the HADR
nodes
ssh ip-172-31-15-79 -l root ls
ssh ip-172-31-15-79.us-east-2.compute.internal -l root ls
ssh ip-172-31-10-145 -l root ls
ssh ip-172-31-10-145.us-east-2.compute.internal -l root ls
Local storage
 Ensure the following local storage, for

example /tmp, on each node for all cluster
related software (excluding space required
for Db2 server, database, log files and so on):
• 50 MB for cluster storage RPMs and extracted files
• 200 MB for full installation of cluster related software
• At least 1GB in /var to store cluster software log files
KornShell
 KornShell
 In addition to packages reported by
the db2prereqcheck -l command, KornShell
(ksh) is required for the Db2 resource agents
for Pacemaker.
On RHEL systems, use yum to install:

yum install ksh
On SLES systems, use zypper to install:

zypper install ksh
Prerequirites for an integrated
solution using Pacemaker
 Cluster software
 Db2 supports Pacemaker as an integrated
cluster manager solution.
 However, this is only the case if
the Pacemaker and its software stack being
used are supplied by Db2 directly and
corresponds to a specific Db2 release and
configured using the new db2cm utility.
 Db2 high availability disaster recovery
(HADR)
• Ensure that both HADR databases exist on
different systems.
• Ensure all HADR databases are started in their
respective primary and standby database roles,
and that all HADR primary-standby database
pairs are in peer state.
• Ensure that users are using one of the following
HADR synchronization modes: SYNC or
NEARSYNC.
 Db2 high availability disaster recovery
(HADR)
• Configure hadr_peer_window for all HADR
databases to a value of at least 120 seconds.
• Disable the Db2 fault monitor.
Luís Garmendia
Download Pacemaker
 The Pacemaker version supported by Db2 must

be downloaded from the public IBM
website: Db2 Automated HADR with Pacemaker.
https://idaas.iam.ibm.com/idaas/oidc/endpoint/default/authorize?scope=openid&response_type=code&state=202
10110-f29b0b3f-53d3-44a8-9df9-5c59094acc2a-d3ee2&redirect_uri=https%3A%2F%2Fidaas-
us01a.ice.ibmcloud.com%2Fidaas%2Fmtfim%2Fsps%2Fidaas%2Flogin%2Fibmid%2Fredirect&client_id=CICPro
dClient1
Download Pacemaker
Db2 Automated HADR with Pacemaker.
Download Pacemaker
Db2 Automated HADR with Pacemaker.
Installing Pacemaker
1 As root on the first host, extract the tar file in the /tmp folder.
cd /tmp
tar -zxf
Db2_v11.5.4.0_Pacemaker_20200418_RHEL8.1_x86_64.tar.gz
2 Verify that the following packages are installed. The output may vary slightly
for different architectures and Linux distributions. All packages should include
the db2pcmk text in the output.
For example:
[root@ip-172-31-15-79 RPMS]# rpm -q corosync
corosync-3.0.3-1.db2pcmk.el8.x86_64
[root@ip-172-31-15-79 RPMS]# rpm -q pacemaker
pacemaker-2.0.2-1.db2pcmk.el8.x86_64
[root@ip-172-31-15-79 RPMS]# rpm -q crmsh
crmsh-4.1.0-0.db2pcmk.el8.noarch
1 As root on the first host, extract the tar file in the /tmp folder.
cd /tmp
tar -zxf
Db2_v11.5.4.0_Pacemaker_20200418_RHEL8.1_x86_64.tar.gz
2 Installing pacemaker 1/2

Configuring a clustered
environtment using db2cm
2 Installing pacemaker 2/2
Comprobar que pacemaker está instalado
Crear conexion ssh passwordless
Copy db2cm to sqllib/adm
3 Copy the db2cm utility from the cluster software directory into the
instance sqllib/adm directory:
Cp
/tmp/Db2_v11.5.4.0_Pacemaker_20200418_RHEL8.1_x86_64/Db2/d
b2cm
/home/db2inst1/sqllib/adm
chmod 755 /home/db2inst1/sqllib/adm/db2cm
Copy resource agents
4 Copy the resource agent scripts (db2hadr, db2inst, db2ethmon)

from /tmp/Db2agents into /usr/lib/ocf/resource.d/heartbeat/ on
both hosts
/home/db2inst1/sqllib/adm/db2cm -copy_resources
/tmp/Db2_v11.5.4.0_Pacemaker_20200418_RHEL8.1_x86_64/Db2ag
ents -host ip-172-31-10-145
/home/db2inst1/sqllib/adm/db2cm -copy_resources
/tmp/Db2_v11.5.4.0_Pacemaker_20200418_RHEL8.1_x86_64/Db2ag
ents -host ip-172-31-15-79
Copy resource agents
Copy the tools (db2ethmon, db2hadr, db2inst) have been copied
Cat db2ethmon
Cat db2hadr
Cat db2inst
Installing Qdevice quorum
The QDevice quorum requires a third host

accessible via a TCP/IP network by the other hosts
in the cluster.
However, the third host itself does not need to be
configured as a part of the cluster.
There is no need to have the DB2 or Pacemaker
software installed. The only requirement is to
install a corosync-qnetd RPM on it.
Procedure
1.On the primary and standby hosts, ensure the corosync
qdevice package is installed with the following command:
rpm -qa | grep corosync-qdevice
If it is not installed, install it using the following command on RHEL
systems:
dnf install /<path_to_RPMS_provided>/corosync-
qdevice*
For SLES systems, use the following command:
zypper install --allow-unsigned-rpm
/<path_to_RPMS_provided>/corosync-qdevice*
2.On the third host, install the Corosync QNet software with the
following command on RHEL systems:
dnf install /<path_to_RPMS_provided>/corosync-qnetd*
For SLES systems, use the following command:
zypper install --allow-unsigned-rpm
/<path_to_RPMS_provided>/corosync-qnetd*
3 As a root user, run the following db2cm command to setup the
QDevice from one of the cluster nodes
The db2cm command requires a passwordless SSH to be configured

between the node that it is going to run on and the node that will host
the QDevice.
./db2cm -create -qdevice <hostname>

4 Run the following corosync command on the
primary and standby hosts to verify that the
quorum was setup correctly.
corosync-qdevice-tool -s
5 Run the following corosync command on the

QDevice host to verify that the quorum device is
running correctly.
corosync-qnetd-tool -l
Public cloud vendors: AWS
Configuring a Db2 clustered environment
with Pacemaker using the db2cm utility is also
applicable to cloud environments.
Setting up two-node HADR with fencing on
Amazon Web Services (AWS)
https://www.ibm.com/support/pages/node/6327425
RA
Pacemaker Architecture
 https://clusterlabs.org/pacemaker/doc/en-
US/Pacemaker/2.0/html/Pacemaker_Explained/_pacemaker_archite
cture.html
Luís Garmendia
db2cm
Db2 cluster manager utility (db2cm) is a utility
that you can use to configure and administer
your highly available databases in a clustered
environment with Pacemaker.
$INSTANCE/sqllib/adm/db2cm
db2cm
Db2cm command syntax
Creates a new Pacemaker cluster domain with two hosts and
the public Ethernet resource.
The following parameters are mandatory:
-domain domain-name
Specifies the name of the Pacemaker domain
-host host-name
Specifies the name of the host. This is also used as a host in
the HADR cluster.
-publicEthernet public-Ethernet-device-name
Specifies the name of the public network adapter
For example:
./db2cm -create -cluster -domain MyPCSDomain
-host host1 -publicEthernet eth0
-host host2 -publicEthernet eth2
db2cm –create -instance
3 Create the instance resource model by running the
following commands:
db2cm -create -instance db2inst1 –host ip-172-31-15-79

db2cm -create -instance db2inst1 –host ip-172-31-10-145
Check on both hosts that the DBM CFG has been

updated:
db2 get dbm cfg | grep -i cluster

Cluster manager = PACEMAKER
crm status
Check the status
5 Create a new database. For example, a database called SAMPLE.
Docker exec –ti server /bin/bash
Db2sampl
Then configure HADR on that new database.

Setup log archive:
mkdir /database/config/home/db2inst1/archive
Db2 update db cfg for sample using logarchmeth1
‘DISK:/database/config/home/db2inst1/archive/’
Db2 update db cfg for SAMPLE using LOGINDEXBUILD ON
inmediate
Db2 backup db SAMPLE
Update HADR configuration
Copy the backup and restore at the standby
Scp SAMPLE.0… IP:/home/db2inst2
Db2 restore db sample from . Taken at
 5.4 Set the HADR-specific configuration parameters
On the primary and standby instance:
"UPDATE DB CFG FOR dbname USING
HADR_LOCAL_HOST hostname
HADR_LOCAL_SVC servicename
HADR_SYNCMODE syncmode"
Open the HADR ports (as root)
Enable HADR
Send SAMPLE Backup to pacemaker9
scp file.txt username@to_host:/remote/directory/

In pacemaker9:
Poner log Archivo
DROP DATABASE SAMPLE
RESTORE DATABASE SAMPLE
HADR db cfg configuration
db2 start hadr on db SAMPLE as standby
db2 start hadr on db SAMPLE as primary

Create HADR DB cluster resources
6 Create the HADR database resources.

/home/db2inst1/sqllib/adm/db2cm -create -db SAMPLE
-instance db2inst1
7 (Optional) Create the VIP resources for the newly created database.
sqllib/adm/db2cm -create -primaryVIP <IP address>

-db SAMPLE
–instance db2inst1
CRM STATUS
8 Verify the cluster again using crm status.
CRM STATUS
crm status before DB resources
See status with db2cm -list
See status with db2cm -list
Disable Pacemaker
db2inst2@dw_pacemaker_1]~>sudo ~/sqllib/adm/db2cm
-disable
Cluster manager is disabled.

Enable Pacemaker
db2inst2@dw_pacemaker_1]~>sudo ~/sqllib/adm/db2cm
-enable
Luís Garmendia
Maintaining a Pacemaker cluster
domain
https://www.ibm.com/support/knowledgecenter/SSEPGG_11.5.0/com.ibm.db2.luw.a
dmin.ha.doc/doc/r_maintaining_cluster_domain.html
 User initiated takeover
Follow the procedure to initiate a user takeover.
 User initiated takeover by force
Follow the procedure to initiate a user takeover by force. Expect
the Pacemaker cluster to reintegrate the old primary as the new
standby.
 Add a HADR database resource to the resource model
Perform the following procedure to create a new database resource
to an existing database in the instance.
./db2cm -create -db SAMPLE -instance db2inst1
domain
 Delete an existing HADR database resource from the resource
model
This procedure is mandatory when dropping an HADR enabled
database from the instance. Perform this procedure only after the
database is dropped.
./db2cm -delete -db SAMPLE -instance db2inst1
 Associate a primary VIP with an existing HADR database of an

instance
Follow the procedure to associate a primary VIP with an existing
HADR database of an instance.
./db2cm -create -primaryVIP 9.21.107.154 -db SAMPLE -instance db2inst1
 Disassociate a primary VIP with an existing HADR database of

an instance
Follow the procedure to disassociate a primary VIP with an existing
HADR database of an instance. hardware, can be restored.
./db2cm -delete -standbyVIP -db SAMPLE -instance db2inst1
domain
 Associate a standby VIP with an existing HADR database of an
instance for read-on-standby
Follow the procedure to associate a standby VIP with an existing HADR
database of an instance for read-on-standby.
./db2cm -create -standbyVIP 9.28.232.70 -instance db2inst1 -db SAMPLE
 Disassociate a standby VIP with an existing HADR database of an
instance
Follow the procedure to disassociate a standby VIP with an existing
HADR database of an instance.
./db2cm -delete -standbyVIP -db SAMPLE -instance db2inst1
 Remove all resources related to the public Ethernet adapter device
on a host in the resource model
Follow the procedure to remove all resources related to the public
Ethernet adapter device on a host in the resource model.
./db2cm -delete -publicEthernet eth1 -host ignited1
domain
 Remove all resources related to an instance in the resource model
Follow this procedure to keep the cluster intact but have all resources
(instance, database, Ethernet) along with all constraints removed.
./db2cm -delete -instance
 Remove an automated HADR cluster with Pacemaker
Follow the procedure to remove an automated HADR cluster
with Pacemaker.
./db2cm -delete -cluster
 Backup cluster configuration information
The following procedure can be used to save a valid cluster
configuration to a backup file.
db2cm -export <file path>
 Restore from a saved Pacemaker cluster configuration
[
Backup/Restore
The resulting backup configuration cannot be

deployed on a new set of hosts where any of the
following details are different from the original cluster:
 Hostnames
 Domain name
 Interface names
 Instance names
 Database names
 Primary/Standby virtual IP addresses
 Qdevice host
[
Backup/Restore
 Backup/restore the cluster configuration:

Example:
db2cm -export /tmp/backup.conf
root@jesting1]$ crm status
ERROR: status: crm_mon (rc=102): Error: cluster is not
available on this node
Converting a Tivoli SAMP cluster to
a Pacemaker cluster
Procedure
1.As an instance owner, backup the existing Tivoli SA MP cluster configuration by exporting it to an xml file using the
following command:
db2haicu -o exportedFile.xml
2.As an instance owner, delete the Tivoli SA MP resource model on both the primary and standby hosts using the
following command:
db2haicu -delete
3.As an instance owner, validate that the HADR databases pairs are in PEER state. For each database, run:
db2pd -db dbname -hadr | grep HADR_STATE
4.Install Pacemaker
5.Ensure that the db2hadr, db2inst and db2ethmon agent scripts have been copied
to /usr/lib/ocf/resource.d/heartbeat on all hosts in the cluster.
6.As a root user, run the db2cm command to create the cluster. For example:
/db2cm -create -cluster -domain hadom -host ip-172-31-15-79 -publicEthernet eth1 -host
ip-172-31-10-145 -publicEthernet eth1
7.As a root user, run the db2cm command to create the instance resources for the two HADR hosts. db2cm -create
-instance <instance name> -host <hostname in primary site>
db2cm -create -instance <instance name> -host <hostname in DR site>
8.As a root user, run the db2cm command for each HADR database pair to create the HADR database resources.
db2cm -create -db <database name> -instance <instance name>
9.As a root user, run the db2cm command to create the primary VIP resource for the specified database. For example:
db2cm -create -primaryVIP <IPv4 Address> -db <database name> -instance <instance
name>
10.Validate that the resource model for each HADR database.
[root@talkers1~]# crm status
.
Converting a Pacemaker cluster

Tivoli SAMP cluster
1.Backup the current Pacemaker resource model in case it is required to
restore at a later time.
As a root user, run the following:

cp /var/lib/pacemaker/cib/cib.xml
$HOME/cib_backup.xml
2.Delete the current Pacemaker resources and cluster. Run the following
command:
3.Create the Tivoli SA MP resource model.
•If a backup of a previous Tivoli SA MP configuration is to be restored using
a db2haicu XML file, run the following command first on the standby
instance and then on the primary instance:
db2haicu -f exportedFile.xml
•To create the resource model from scratch, run the db2haicu on the
standby instance and follow the on-screen instructions
Luís Garmendia
Testing: Shut down standby
Restart standby
Restart standby
Testing: Shut down primary
Restart nodo1
Restart nodo1
Testing: Takeover
[db2inst2@dw_pacemaker_1]~>date; db2 takeover hadr

on db SAMPLE;
Luís Garmendia
Configure a VIP Resource
The "VIP resource" is a single TCP/IP address that will be
shared between the nodes in the Pacemaker HADR cluster
and remote clients will (should) be able to connect to the
HA database using this address only, regardless of which
node in the cluster is the current primary.
Configure VIP
[root@dw_pacemaker_1]/root#
/home/db2inst2/sqllib/adm/db2cm -create -primaryVIP
192.168.56.249 -db SAMPLE -instance db2inst2
Test a client connection in a failover
with VIP
 In the case of a failover, Pacemaker will (should)
automatically transfer the VIP address to the
new primary server.
Establish a remote connection
Catalog node with VIP resource, i.e. TCP/IP address 192.168.56.249
List connections in node 1
[db2inst2@dw_pacemaker_1]/home/db2inst2>db2 list applications

Auth Id Application Appl. Application Id DB # of
Name Handle Name Agents
-------- -------------- ---------- ------------------------------
- -------- -----
DB2INST2 db2bp 7054 192.168.56.1.54490.200811085242 SAMPLE 1
Failover
Failover
Failover
The client must reconnect

Remote reconnection
ALTERNATE SERVER
[db2inst2@dw_pacemaker_1]/home/db2inst2>db2 update
alternate server for database SAMPLE using hostname
192.168.56.249 port 51000
[db2inst2@dw_pacemaker_2]/home/db2inst2>db2 update
alternate server for database SAMPLE using hostname
192.168.56.249 port 51000
[
ALTERNATE SERVER
In the remote client:

[db2dwtst@itchy:~]$ db2 list db directory
...
Database 2 entry:
Database alias = DWPMSAMP
Database name = SAMPLE
Node name = DWPM1
Database release level = 14.00
Directory entry type = Remote
Alternate server hostname = 192.168.56.249
Alternate server port number = 51000
FAILOVER WITH VIP and ACR
The ACR (Automatic Client Reroute) configuration
is updated on all nodes in the Pacemaker HADR
cluster first (using the previously configured VIP
resource).
Any in-flight transactions that were active at the time
of the failover were lost.
CONCLUSIONS
 1. Pacemaker installation is a simple matter

 2. Pacemaker adds the High Availability
functionality to DB2 HADR clusters, and it does
that well
 3. Coupled with ACR, Pacemaker enables
uninterrupted client connections to DB2 HADR
clusters
References
• DB2 Documentation
https://www.ibm.com/support/producthub/db2/docs/content/SSEPGG_11.5.0/com.ibm.db2.luw.admin.ha.doc
/doc/c_integrated_solution_pacemaker.html
https://www.ibm.com/support/producthub/db2/docs/content/SSEPGG_11.5.0/com.ibm.db2.luw.admin.ha.doc
/doc/c_pacemaker_intro.html
https://www.ibm.com/support/knowledgecenter/SSEPGG_11.5.0/com.ibm.db2.luw.wn.doc/doc/c_ha_enhance
ments_11-5-5.html
https://www.ibm.com/support/knowledgecenter/SSEPGG_11.5.0/com.ibm.db2.luw.admin.ha.doc/doc/r_maint
aining_cluster_domain.html
150
References
• DB2 Documentation
https://www.ibm.com/support/knowledgecenter/SSE
PGG_11.5.0/com.ibm.db2.luw.admin.ha.doc/doc/c_i
ntegrated_solution_pacemaker.html
https://www.ibm.com/support/knowledgecenter/SSE
PGG_11.5.0/com.ibm.db2.luw.admin.ha.doc/doc/c
_prereq_pacemaker.html
151
References
• Pacemaker Documentation
https://clusterlabs.org/pacemaker/doc/en-
US/Pacemaker/2.0/html/Pacemaker_Explained/_pac
emaker_architecture.html
152
References
https://access.redhat.com/articles/3172731
https://www.linuxjournal.com/content/ahead-pack-
pacemaker-high-availability-stack
https://slideplayer.com/slide/8067194/
153
References
IBM support
?mhsrc=ibmsearch_a&mhq=Pacemaker
?mhsrc=ibmsearch_a&mhq=Pacemaker
154
References
Triton Consulting
https://www.triton.co.uk/automating-hadr-failovers-with-pacemaker/
E-book
https://marketing.triton.co.uk/acton/attachment/15128/f-0f9509bd-5858-4e0c-af2a-
3bde214d825e/1/-/-/-/-/DB2%20HADR%20Automation%20with%20Pacemaker.pdf
155

DB2 HADR Pacemaker Luis Garmendia

Uploaded by

Copyright:

Available Formats

You might also like

DB2 HADR Pacemaker Luis Garmendia

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DB2 HADR Pacemaker Luis Garmendia

Uploaded by

Copyright:

Available Formats

Luís Garmendia

Mutual Takeover (purescale)

 A cluster is a group of connected

Virtual IP address takeover

The Db2 High Availability Feature enables

 Clustering software automates the

 Some cluster Managing Software are:

Pacemaker is included in db2 v11.5.4, and uses

Create the HADR database resources.

(Optional) Create the VIP resources for the newly

sqllib/adm/db2cm -create -primaryVIP

 Db2 Version 11.5 Mod Pack 4

 Resources are set of Db2 defined entities

 Constraints are rules setup during cluster

1. Intance resource db2_draping1_gerry_0 prefers to run on

2. The database resource will only run if the Ethernet network

3. The database resource starts before the primary

4. The following co-location constraint ensures that

Pacemaker Base Component

Pacemaker Base Component

Pacemaker Resource agents

Pacemaker Resource agents

Pacemaker Cluster domain leader

 One of the nodes in the cluster will be elected as

Networks in a Pacemaker cluster

 To configure elements of your Pacemaker cluster

Networks in a Pacemaker cluster

 Network interface cards

Networks in a Pacemaker cluster

Networks in a Pacemaker cluster

 Virtual IP address (VIP)

Quorum devices support on

Quorum devices support on

Quorum devices support on

Quorum devices support on

Quorum devices support on

Quorum devices support on

Quorum devices support on

Quorum devices support on

 The host itself is not required to be part of the

Quorum devices support on

Majority •More reliable than Two- •Need to include the

 users must ensure that the instance owner,

•On ip-10-250-0-1, ping ip-10-250-0-9 works.

•On ip-10-250-0-9, ping ip-10-250-0-1 works

 Passwordless secure shell (SSH) for root and

 Passwordless SSH for both root and instance

 Ensure the following local storage, for

On RHEL systems, use yum to install:

On SLES systems, use zypper to install:

 The Pacemaker version supported by Db2 must

2 Installing pacemaker 1/2

4 Copy the resource agent scripts (db2hadr, db2inst, db2ethmon)

The QDevice quorum requires a third host

The db2cm command requires a passwordless SSH to be configured

./db2cm -create -qdevice <hostname>

5 Run the following corosync command on the

db2cm -create -instance db2inst1 –host ip-172-31-15-79