Technical Documentation

2.1.1.2 Create entries in /etc/fstab for new mounts Before you start with the OS portion
of this procedure, you should ensure that the backboot and backroot devices are mounted to the file
system as /var/backup and /var/backup/boot/efi. These mount points should already exist in the file
/etc/fstab similar to the example below:
## Sample SLES 11 entries for HANA System Backup using EXT3 file system
/dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part4 /var/backup/boot/efi vfat umask=0002,utf8=true 0 0
/dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part5 /var/backup ext3 acl,user_xattr 1 1

Listing 1: Example SUSE 11 fstab entries with ext3

Note
The hexadecimal portion of the value of /dev/disk/by-id/
scsi-3600605b0038ac2601a9a1f01cc74cf23-partx will be different for every individ-
ual drive and installation. We recommend to read the contents of /etc/fstab before and
copy only the value for the stated partitions for all new backup partitions. Pay particular
notice to rename the partition to the correct partition created!
## Sample SLES 12 entries for HANA System Backup using EXT4 file system
UUID=b68c3b74-0ece-4e69-8961-6c1f61b05cad /var/backup ext4 acl,user_xattr 1 1
UUID=AFE0-328F /var/backup/boot/efi vfat umask=0002,utf8=true 0 0

Listing 2: Example SUSE 12 fstab entries with ext4

Note
The hexadecimal portion of the value of /dev/disk/by-uuid/ will be different for every
individual drive and installation. We recommend to read the contents of /etc/fstab before
and copy only the value for the correct partitions found in /dev/disk/by-uuid/ for all new
backup partitions. Pay particular notice to rename the partition to the correct partition
created!

## Sample RHEL 6 entries for HANA System Backup using EXT4 file system
UUID=c605201a-04bc-47a8-bbc4-b6808ee98fe1 /var/backup ext4 defaults 1 2
UUID=FF50-7B37 /var/backup/boot/efi vfat umask=0077,shortname=winnt 0 0

Listing 3: Example Red Hat 6 fstab entries with ext4

Note
The hexadecimal portion of the value of /dev/disk/by-uuid/ will be different for every
individual drive and installation. We recommend to read the contents of /etc/fstab before
and copy only the value for the correct partitions found in /dev/disk/by-uuid/ for all new
backup partitions. Pay particular notice to rename the partition to the correct partition
created!

2.1.2 Drive Partitions

Starting with version 1.8.80-10 of the the Lenovo Solution installation media, it will create two (2) extra
partitions on the first drive (sda). Each partition has a specific label and purpose for the system backup
and restore. Along with the original partition labels of: hanaboot, hanaroot and hanaswap, we also create
those called: backboot and backroot for this purpose. The correlation of these labels to the appropriate
devices can be found by listing the symbolic links in the directory /dev/disk/by-label.
This paper will continue to use the labels to showcase a backup and recovery concept. An example
partition layout for systems is shown below. The first device is partitioned into several physical and

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 10
1.12.121-16 © Copyright Lenovo, 2016
 Technical Documentation

logical partitions and named with a label, a simple identifier, and a Universally Unique Identifier (UUID)
. Only the UUID is promised to remain connected to the proper partition as it was created.

Attention
It is important to remember that if a disk is ever replaced due to a failure, that none of these
identifiers would remain the same after the disk exchange. You must modify the appropriate
files, such as /etc/fstab accordingly after such an exchange.

Partition /dev/disk/by-label /dev/disk/by-id /dev/disk/by-uuid

/dev/sda1 hanaboot scsi-{33-hexadecimal-number}-part1 hexadecimal number
/dev/sda2 hanaroot scsi-{33-hexadecimal-number}-part2 hexadecimal number
/dev/sda3 swap scsi-{33-hexadecimal-number}-part3 hexadecimal number
/dev/sda4 backboot scsi-{33-hexadecimal-number}-part4 hexadecimal number
/dev/sda5 backroot scsi-{33-hexadecimal-number}-part5 hexadecimal number

Table 2: Sample Linux Partition Naming Examples in SLES 11

Attention
Pay special attention on systems installed earlier than version 1.8.80-10, or systems installed
manually using the SAP HANA specifications. These systems may have been installed with
no extra partitions or even others that are used for other auxillary file systems unrelated to
SAP HANA. If this is the case, then you should be certain to create new backup partitions
elsewhere and determine a way to backup and save off the data in these auxillary partitions.
The backup and recovery of these drives is not part of this document, but similar rules can
be applied.

2.1.3 Boot Loader

The server can use two different methods to boot. The default method is using the Unified Extensible
Firmware Interface (UEFI). According to Wikipedia10 , the Unified Extensible Firmware Interface is a
specification that defines a software interface between an operating system and platform firmware. The
second method is using the legacy method: Basic Input / Output System (BIOS), which was a typical
way to boot SAP HANA on eX5 based systems due to the limitations of the OS available at that time.
Linux requires a boot loader that understands the specific boot method. Today, three options are avail-
able: Grand Unified Bootloader (GRUB) , GRUB2 and Linux Loader (LILO) . The way a server boots;
and, subsequently, installs the boot loader determines some of the system partitioning and file system
layout of the installed server. In addition, each Linux distributor installs their version of GRUB or LILO
differently. Although it is possible to use both methods to boot and install the Lenovo Solution server,
this document will only cover the steps necessary to create a restore image using the UEFI11 boot mech-
anism with either the GRUB, GRUB2 and the LILO boot loaders. If you are using the Legacy Boot
option of System eX5 or X6 servers, you will need to become familiar with how each distribution handles
the boot procedure with the Legacy BIOS boot option as this is not part of this documentation.
EFI Linux Loader (ELILO) is the interface that SUSE Linux Enterprise Server (SLES) 11 uses to talk
to the LILO boot loader. The Linux installation will place the boot loader under the directory /boot/efi.
The configuration file for ELILO can be found in /etc/elilo.conf. Red Hat Enterprise Linux (RHEL)
6 uses GRUB and the Linux installation will place the boot loader under the directory /boot/efi.
10 http://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface
11 Unified Extensible Firmware Interface

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 11
1.12.121-16 © Copyright Lenovo, 2016
 Technical Documentation

The configuration file for GRUB can be found in /boot/grub/menu.conf or /boot/grub/grub.conf,

depending on the version of GRUB. SLES 12 uses GRUB2 which places the boot loader under /boot
and the respective configuration file under /boot/grub/grub.cfg. These directories are important to
remember when a separate partition has been made by the OS to store these files. Both SLES 11 and
RHEL12 6 create separate partitions for the /boot directories. SLES 12 does not.

2.1.4 Backup Procedure

In order to perform a simple backup and restoration of the OS, excluding the SAP HANA executable,
configuration, data or logs, you need to run a few commands in Linux in order to set up a working copy
of the OS. What we will explain here is a method of copying the Linux file system that is contained on
two partitions of the first hard drive.
Using the Linux command rsync, you are able to intelligently copy a file system from one partition to
another quickly and with little effort. This tool can also be set up in nightly cron schedules to happen
automatically and semi-automate the process of taking a backup image of the OS. As seen in the general
concept is that the user uses rsync to copy the contents of the root (/) and boot (/boot/efi) directories
from their original partitions onto two newly created partitions on the same hard drive.
In order to perform an initial backup run as root the following commands. The initial backup will take a
long time as it is copying the entire file system under the hanaroot partition into the backroot partition.
Subsequent executions of the rsync command will be shorter as it is intelligent enough to only copy what
has changed between calls of the command.
As the system administrator (root) run:
start_stamp=$(date +%s)
# Begin backup of root file system
rsync -aAXxv --delete / /var/backup --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/boot/efi*,/run/*,/mnt/*,/media/*,\
/lost+found,/var/backup/*,/sapmnt/*,/var/lib/ntp/proc/*,/etc/fstab}
middle_stamp=$(date +%s)
echo "Root file system completed in $( echo \
"(${middle_stamp}-${start_stamp})/60"| bc ) minutes $( echo "(${middle_stamp}-${start_stamp})%60"| bc ) seconds"

Listing 4: Example rsync command

If the hanaboot partition exists, then you should run this command as well:
# Begin backup of /boot/efi file system
rsync -aAXxv --delete /boot/efi/ /var/backup/boot/efi/
end_stamp=$(date +%s)
echo "Boot file system completed in $( echo \
"(${end_stamp}-${start_stamp})/60"| bc ) minutes $( echo "(${end_stamp}-${start_stamp})%60"| bc ) seconds"

Listing 5: Example rsync command

2.1.5 Correcting the backup fstab

After each time the rsync command has completed, the root file system has now been copied exactly
from / into /var/backup/. In order to boot from the backup partition backroot, we want to switch the
partition labels (or ids) from hana* to the back* labelled partitions. The hana* partitions should now
be mounted as the file system /var/backup in order to restore from the backed up image in the case of
a recovery.
12 Red Hat Enterprise Linux

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 12
1.12.121-16 © Copyright Lenovo, 2016
 Technical Documentation

We recommend to slightly modify the message of the day (motd) so that you can visually see that you
are using the backup image. Since this is also copied on top of any previous images, it is best to use a
symbolic link to keep both the backup and original motd file.
touch /etc/motd.{bak,orig}
echo "## !!!!! T H E B A C K U P M E S S A G E !!!!! ##" > /etc/motd.bak
cat /etc/motd >> /etc/motd.{bak,orig}
rm /etc/motd
ln -s /etc/motd.orig /etc/motd
Listing 6: Creating a copy of the motd file

After every rsync run, the fstab needs to be adopted as shown here. We recommend to create a copy
of the original and backup so that you can easily switch between the two after a call to rsync. You can
copy the original file /etc/fstab to /etc/fstab.orig and create a new copy called /etc/fstab.bak.
touch /etc/fstab.{bak,orig}
echo "## !!!!! T H E B A C K U P F S T A B F I L E !!!!! ##" > /etc/fstab.bak
cat /etc/fstab >> /etc/fstab.{bak,orig}
rm /etc/fstab
ln -s /etc/fstab.orig /etc/fstab
Listing 7: Example SLES primary fstab file

Thereafter, you can change the SUSE Linux Enterprise Server entries in /etc/fstab.bak to:
## Adding entries for HANA System Backup
/dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part2 /var/backup ext3 acl,user_xattr 1 1
/dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part1 /var/backup/boot/efi vfat umask=0002,utf8=true 0 0
/dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part3 swap swap defaults 0 0
/dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part4 /boot/efi vfat umask=0002,utf8=true 0 0
/dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part5 / ext3 acl,user_xattr 1 1

Listing 8: Example SLES backup fstab file

Notice that the entries /dev/disk/by-id will be different on your system. The mountpoints need to be
changed as shown above.
On Red Hat Enterprise Linux the entries in /etc/fstab.bak should be changed to:
## Adding entries for HANA System Backup
LABEL=backroot / ext3 acl,user_xattr 1 1
LABEL=backboot /boot/efi vfat umask=0002,utf8=true 0 0
LABEL=hanaswap swap swap defaults 0 0
LABEL=hanaroot /var/backup ext3 acl,user_xattr 1 1
LABEL=hanaboot /var/backup/boot/efi vfat umask=0002,utf8=true 0 0
Listing 9: Example RHEL backup fstab file

Notice that only the labels have changed.

After these files have been created, you will also need to recreate a symbolic link under the /var/backup
directory to point the files to their backup representations as follows:
rm /var/backup/etc/fstab
cd /var/backup/etc
ln -s fstab.bak fstab
Listing 10: Changing files for backup partition

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 13
1.12.121-16 © Copyright Lenovo, 2016
 Technical Documentation

2.1.6 Add boot loader entry for backup partition

ELILO installed systems SLES 11 uses ELILO to boot the Linux OS. After the fstab file has
been modified, create a backup entry in the ELILO boot menu (/etc/elilo.conf) by copying the whole
subsection identified by the label=linux statement.
On RHEL replacing the label and root values with the value backup and backroot partition ID. On
SLES the according scsi-<id>-part<X> has to be changed to fit the <id> and partition <X> on the
given system. It is important to modify the string

###Don't change this comment - YaST2 identifier: Original name: name###

on these installs. Otherwise, YaST will not see this option in the boot list for ELILO and may not present
it to you during boot.
## Adding Restore entry to UEFI Boot menu
image = vmlinuz-3.0.76-0.11-default
###Don't change this comment - YaST2 identifier: Original name: backup###
label = backup
append = "resume=/dev/sda3 splash=silent transparent_hugepage=never
intel_idle.max_cstate=0 processor.max_cstate=0 showopts "
description = "Backup of SAP HANA Platform Edition Image"
initrd = initrd-3.0.76-0.11-default
root = /dev/disk/by-id/scsi-3600605b0038ac2601a9a1f01cc74cf23-part5
Listing 11: Example UEFI Configuration for Primary Partition

If you update the kernel, you will also need to update the lines image = and initrd = in this file for the
backup entry.
After changing the elilo.conf run
elilo --verbose

to update the boot loader. The intention is that you will be able to start up the backup partition in
order to copy the saved state in the backup partition over top of the primary partition.

GRUB installed systems RHEL 6 uses GRUB by default to load the Linux OS as well as SLES
installs on Systems eX5 hardware. In systems installed using the GRUB boot loader edit the contents of
/boot/grub/grub.cfg (RHEL), or /boot/grub/menu.lst (SLES), and copy the section for the primary
partition to edit it as the new backup partition.
This is a copy of the default boot line with the title, root and kernel lines changes to match the
partition used for the backup partition.On RHEL replacing the label and root values with the value
backup and backroot partition ID. On SLES the according scsi-<id>-part<X> has to be changed to
fit the <id> and partition <X> on the given system.
title Backup of SAP HANA Platform Edition Image
root (hd0,<PARTITION NR, see below>)
kernel /boot/vmlinuz-2.6.32-431.el6.x86_64 ro root=LABEL=backroot
KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 crashkernel=auto
processor.max_cstate=0 intel_idle.max_cstate=0
transparent_hugepage=never SYSFONT=latarcyrheb-sun16
rd_NO_LUKS rd_NO_LVM rd_NO_DM rd_NO_MD rhgb quiet

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 14
1.12.121-16 © Copyright Lenovo, 2016
 Technical Documentation

initrd /boot/initramfs-2.6.32-431.el6.x86_64.img
Listing 12: Example GRUB Configuration for Primary Partition

On SLES use the command:

yast2 bootloader

to update the boot loader, on RHEL:

grub-install /dev/sda

Note
The partition number for a GRUB installed partition is based on the device syntax of (←-
,→device[,partmap-name1part-num1[,partmap-name2part-num2[,...]]]). The syntax
(hd0) represents using the entire disk of the first device, for example sda, while the syn-
tax (hd0,1) represents using the second partition of the device, for example sda2. Notice
that GRUB identifies the first partition on the first device as (hd0,0) or (hd0) for short.

Note
In our example, we presume that the hanaroot partition is (hd0,1) and the backroot←-
,→ partition is (hd0,4).
Append or change these lines in /var/backup/etc/grub.conf. Here, we exchange the meanings of the
hanaroot and backroot partitions. When booting into this kernel, the hanaroot is the partition to be
restored, and the backroot is the default partition to be booted. The title, root and kernel lines are
changed to match the partition used for the backroot partition. We should also change the parameter
default in the header subsection to point to the Restore image (usually the subsection number 2) rather
than the original SAP HANA image.
default=2
title Restore from SAP HANA Platform Edition Backup Image
root (hd0,<PARTITION NR, see above>)
kernel /boot/vmlinuz-2.6.32-431.el6.x86_64 ro root=LABEL=backroot
KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 crashkernel=auto
processor.max_cstate=0 intel_idle.max_cstate=0
transparent_hugepage=never SYSFONT=latarcyrheb-sun16
rd_NO_LUKS rd_NO_LVM rd_NO_DM rd_NO_MD rhgb quiet
initrd /boot/initramfs-2.6.32-431.el6.x86_64.img
Listing 13: Example GRUB Configuration for Backup Partition

In systems installed using the GRUB boot loader (by default all Red Hat based installs and SUSE installs
on System eX5 hardware), edit the contents of /boot/grub/grub.cfg (RHEL), or /boot/grub/menu.lst
(SLES), and copy the section for the primary partition to edit it as the new backup partition.
This is a copy of the default boot line with the title, root and kernel lines changes to match the
partition used for the backup partition.On RHEL replacing the label and root values with the value
backup and backroot partition ID. On SLES the according scsi-<id>-part<X> has to be changed to
fit the <id> and partition <X> on the given system.
title Backup of SAP HANA Platform Edition Image
root (hd0,<PARTITION NR, see below>)
kernel /boot/vmlinuz-2.6.32-431.el6.x86_64 ro root=LABEL=backroot
KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 crashkernel=auto
processor.max_cstate=0 intel_idle.max_cstate=0
transparent_hugepage=never SYSFONT=latarcyrheb-sun16

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 15
1.12.121-16 © Copyright Lenovo, 2016
 Technical Documentation

rd_NO_LUKS rd_NO_LVM rd_NO_DM rd_NO_MD rhgb quiet

initrd /boot/initramfs-2.6.32-431.el6.x86_64.img
Listing 14: Example GRUB Configuration for Primary Partition

On SLES use the command:

yast2 bootloader

to update the boot loader, on RHEL:

grub-install /dev/sda

Note
The partition number for a GRUB installed partition is based on the device syntax of (←-
,→device[,partmap-name1part-num1[,partmap-name2part-num2[,...]]]). The syntax
(hd0) represents using the entire disk of the first device, for example sda, while the syn-
tax (hd0,1) represents using the second partition of the device, for example sda2. Notice
that GRUB identifies the first partition on the first device as (hd0,0) or (hd0) for short.

Note
In our example, we presume that the hanaroot partition is (hd0,1) and the backroot←-
,→ partition is (hd0,4).
Append or change these lines in /var/backup/etc/grub.conf. Here, we exchange the meanings of the
hanaroot and backroot partitions. When booting into this kernel, the hanaroot is the partition to be
restored, and the backroot is the default partition to be booted. The title, root and kernel lines are
changed to match the partition used for the backroot partition. We should also change the parameter
default in the header subsection to point to the Restore image (usually the subsection number 2) rather
than the original SAP HANA image.
default=2
title Restore from SAP HANA Platform Edition Backup Image
root (hd0,<PARTITION NR, see above>)
kernel /boot/vmlinuz-2.6.32-431.el6.x86_64 ro root=LABEL=backroot
KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 crashkernel=auto
processor.max_cstate=0 intel_idle.max_cstate=0
transparent_hugepage=never SYSFONT=latarcyrheb-sun16
rd_NO_LUKS rd_NO_LVM rd_NO_DM rd_NO_MD rhgb quiet
initrd /boot/initramfs-2.6.32-431.el6.x86_64.img
Listing 15: Example GRUB Configuration for Backup Partition

GRUB2 installed systems SLES 12 and RHEL 7 use GRUB2 by default to load the Linux OS. The
configuration is different than GRUB installations. In the file /etc/grub.d/40_custom, you should add a
section that describes the backup partition. The partition number for a GRUB installed partition is based
on the device syntax of (device[,partmap-name1part-num1[,partmap-name2part-num2[,...]]])←-
,→. The syntax hint-*=hd0 represents using the entire disk of the first device as a GUID Partitioned
device. Notice that GRUB2 identifies the fourth GUID partition on the first device as hd0,gpt4. The
long string is the UUID description of the partition that can be found in /dev/disk/by-uuid. These all
must be changed to that of your installed system as well as the versions for vmlinuz and initrd. They
must match the version of the kernel to be booted by the backup.
menuentry 'Restore From Backup - SLES12' --class sles12 --class gnu-linux --class ←-
,→gnu --class os $menuentry_id_option 'gnulinux-simple-80d8705f-58f4-44b5-9866-3←-
,→e5bfd0ec75e' {

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 16
1.12.121-16 © Copyright Lenovo, 2016
 Technical Documentation

load_video
set gfxpayload=keep
insmod gzio
insmod part_gpt
insmod ext2
set root='hd0,gpt4'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt4 --←-
,→hint-efi=hd0,gpt4 --hint-baremetal=ahci0,gpt4 80d8705f-58f4-44b5-9866-3←-
,→e5bfd0ec75e
else
search --no-floppy --fs-uuid --set=root 80d8705f-58f4-44b5-9866-3←-
,→e5bfd0ec75e
fi
echo 'Loading Linux 3.12.51-52.31-default ...'
linuxefi /boot/vmlinuz-3.12.51-52.31-default root=UUID=80d8705f-58f4←-
,→-44b5-9866-3e5bfd0ec75e ro resume=/dev/sda5 splash=silent transparent_hugepage←-
,→=never intel_idle.max_cstate=0 processor.max_cstate=0 nohz=off showopts
echo 'Loading initial ramdisk ...'
initrdefi /boot/initrd-3.12.51-52.31-default
}
Listing 16: Example Grub2 Configuration for backup partition on SLES 12

In order to make this change permanent, the command grub2-mkconfig -o /boot/grub2/grub.cfg

should be run. After setting you should be able to reboot and see a new menu entry in GRUB with the
text of the menuentry above.

2.2 Restoring the Operating System Image

In case of a planned or unplanned system outage, the wish to recover the last known good backup of the
root and boot file system partitions that have been copied on to the backup partitions is possible. In the
case of a hard drive failure where the backup partitions have been lost, the copies stored on an external
storage must be recopied into the backup partitions after the hard drive failure has been resolved by the
hardware support team. After that, the restore can take place as described here.
Restart the machine and boot the backup OS. While booting, select the created boot option for the
backup partition from the list given by the ELILO boot loader menu. By default there is no menu
congigured, but if you press the TAB key while you see the text ELILO Booting:... you will be given
the options you can choose. The newly created option of "backup" should be visible. If not, rerun the
elilo --verbose command in the original OS and restart.
The GRUB boot loader menu is shown by default (see 2: Sample GRUB boot loader screen on page 18.
You can use the arrow keys to select the newly created option "backup". This should be done only after
checking that the boot loader menu in the backup partition has been properly updated according to the
directions in 2.1.5: Correcting the backup fstab on page 12 above.

Backup and Restore Guide Lenovo® Systems Solution™ for SAP HANA® 17
1.12.121-16 © Copyright Lenovo, 2016