2015 International Conference on Data and Software Engineering
Distributed Replicated Block Device (DRDB)
Implementation on Cluster Storage Data Migration
Mardhani Riasetiawan, Ahmad Ashari
Department of Computer Science and Electronics
Faculty of Mathematic and Natural Sciences, UGM
Yogyakarta, Indonesia
mardhani@ugm.ac.id, ashari@ugm.ac.id
Abstract— Data Center systems are required to have high
availability in order to meet the continuing needs of the user. If the
server or application in which a failure or require maintenance, the
virtual machine will be migrated to another server in the cluster
are still available. The role of shared storage is very important here
to keep a virtual machine that can continue the work and do not
lose data on the destination server. This study seeks to implement
virtualization at the server cluster system uses a virtual machine on
Proxmox VE and Distributed Replicated Block Device (DRBD) as
shared storage. Implementation is done by using two nodes, and
made a comparison with two other nodes in the cluster system that
does not use shared storage. Shared storage works by
synchronizing and replication of virtual machine data that can then
migrate online. The use of shared storage will affect virtual
machine performance, especially on the speed of the disk during the
process of sending and receiving data, and the availability of
services. Measurement of downtime during the migration to test the
success of the system. Testing the TCP connection is made to ensure
the network throughput and connection test results compared to
test disk performance.
Keywords—DRDBt; shared storage; virtualization; cluster;
virtual machine
I. INTRODUCTION
Architecture of a Data Center must meet several requirements
in order to maximize utilization with a minimum cost. Data
Center takes its availability at all times and all needed with the
shortest possible waiting time in case of failure or when
maintenance is required. Data Center that meets these
requirements is said to have a high availability (High Available).
Increased utilization means more workload to be performed to
make the resources and technologies needed increased so that the
system can continue to work. At the time of low hardware
utilization can also spend unnecessary [1]. The service quality
Data Center uncontrolled, business growth may not correspond to
what was planned [2]. Each time a new application is run need
servers and new storage space. In many cases, there will be
referred to as server sprawl where server utilization is low, the
lack of disk space, and system management costs to be high.
978-1-4673-8430-8/15/$31.00 ©2015 IEEE
Irwan Endrayanto
Department of Mathematic
Faculty of Mathematic and Natural Sciences, UGM
Yogyakarta, Indonesia
endrayanto@ugm.ac.id
Server consolidation is needed to resolve this problem, one
alternative is virtualization. Since the decade of the 60s was the
idea or the notion of virtualization technology. In the server
virtualization, each application is made to run on each virtual
machine and virtual machines can be run together in a single
physical machine [3]. In addition, virtual machines can be
migrated from one server to another with minimal waiting times
for applications that are running in it.
Virtualization on the server but can create a dynamic IT
infrastructure that is flexible and can also make the quality of
service as required by the cost of resources and technology
guaranteed [4]. Currently, the large-scale enterprise systems,
virtualization is the best choice. Some observers projecting in the
next decade, the number of physical or virtual servers will grow
10-fold, the amount of information that is managed by Data
Center growing 50-fold, the number of files that are processed
Data Center grown 75-fold. Additionally, in 2015 an estimated
7.9 Zettabyte data will be created and replicated. This will
certainly have an impact on the increasing energy and cost
required [5].
In virtualization there are various virtualization platform that
can be used. The platforms are usually paid, but there are also
open source. The advantage of using an open source platform
that will not burden the user from the cost side. Conducted a
study of 3 pieces of virtualization platforms, namely Proxmox
VE, VMWare ESX, and OpenStack [6]. This study aims to
measure the performance of virtual server resources in terms of
overhead and linearity. Results of these studies is to Proxmox,
which is open source software, overhead and linearity values
lower than any other platform. This is because Proxmox using
virtual machines with OpenVZ, which is container-based
virtualization, so that operation can be shared between the host
kernel (Proxmox VE) and guest (virtual machine). Therefore,
virtual machines with OpenVZ running lighter.
Based on the data that has been mentioned before, there
should be research on the performance and availability of
resources generated by the virtualization technology on the open
source virtualization platform that can be implemented and used
93
 2015 International Conference on Data and Software Engineering
on small-sized systems. This system can be used by individuals
and small companies that do not require sophisticated resources
or large.
II. VIRTUALIZATION DATA CENTER
There are several studies that have been done on the topic of
virtualization Data Center. Garnieri did research on a company
server implementation [7]. Research aims to design and
implement a server infrastructure that can optimize the utility of
hardware resources by minimizing downtime when performing
server maintenance and flexible to the workload. To optimize the
utility he designed the server virtualization using VMWare
Infrastructure 3 with features analyzed were Vmotion and
Distributed Resource Scheduler (DRS). He managed to make the
system server virtualization with Vmotion and tested by sending
an ICMP packet and checking the md5 checksum hash values
using the file sent to a virtual machine that performed the
migration. ICMP packets are sent to test the availability of vm,
while checking the hash to determine the effect, if done migration
vm conditions. While DRS feature is used to test load balancing
on the system.
In contrast to Chang, et al, the research the implementation of
small private-cloud computing and services (PSCCS) with the
aim of the system can be used for small and medium-scale
enterprise, school, and social groups [8]. Implementation using
the operating system Openfiler on five private small cloud server
(PSCS) is different, using vSphere ESX / ESXi Server, Hyper-V,
Proxmox VE, Ubuntu Enterprise Server and Xen, as well as
using the technique iSCSI storage on TCP / IP. The storage
technique known to reduce the burden on each physical host.
They found that Proxmox VE has a level higher hardware
utilization and efficient in terms of energy.
The research that applying virtualization to the information
system in an ICT Center [9]. In these implementations to test the
performance of web server. Then, the performance of the web
server in a virtualized environment and physical environment.
Results of the performance measurement is a web server, a web
server running on the virtual environment can serve as a web
server running on the physical environment. Some statistics show
a decline in performance with the average performance of the
current decline in the performance of web servers running on the
physical environment. Even so the system can save financing of
servers and network devices of up to 48.11% for the financing of
server hardware and 35, 134% of the financing network hardware
when using virtualization infrastructure at the Data Center.
Other studies on Proxmox has tested the performance of the
Ceph-Data-node cluster with nodes are geographically dispersed
[10]. The aim of this study was to analyze the scalability and
performance of Ceph to the availability of services. Performance
testing is done by first testing the connections between nodes of
which there is a firewall. After the testing the performance of
storage on each node to determine if it does not contain a storage
performance of virtual machines and contains one or more virtual
machines. Results from these studies is the discovery bottleneck
on the network by using a firewall.
Several studies have been described in the literature review
above is an example of the implementation of server
virtualization. Server virtualization is a technology that is still
growing and has the potential implementation of which is still
widespread. Research by the author is another example of the
implementation of server virtualization. Motivation authors is to
investigate the performance of the software is open source which
can be used for individual or small company. As far as the
literature that has been done, no studies have reported on the
design and implementation of data center virtualization with
Proxmox VE hypervisor software and shared storage using
DRBD. As has been analyzed that the software Proxmox VE has
the advantages of other software [6]. Therefore, it is expected in
addition to minimize the costs, but still produced a good
performance.
III. RESEARCH WORKS
Cluster system can only migrate a virtual machine with
offline mode. It means that the virtual machine must be switched
off first and then do the migration process. It is intended that the
virtual machine does not experience a loss or corruption of data
during the migration process. Besides the length of the migration
process depends on the amount of data and virtual machine
configuration files to be transferred and replicated to the
destination machine, it caused quite a long downtime. Therefore,
this system can not be used on systems with high availability.
By using shared storage virtual machines can migrate online,
that she could do a live migration at a time to migrate to storage.
With this method, the virtual machine can immediately resume
work on the destination machine without damage or loss of data.
The function of the shared storage that is it will be a real-time
replication of data from a virtual machine from one node to
another node in the cluster. As a result, when the necessary
process of migrating the system does not experience significant
downtime, therefore it is possible to do a live migration. From
concept mentioned above will require the development of virtual
cluster with shared storage with the hope to increase the
availability of services on a virtual machine.
In this study will explain the steps in designing and
implementing the Virtual Data Center to improve service
availability and optimize the performance of network hardware
utilization and disk. Solutions to problems of poor availability
cluster systems by using shared storage with functionality for
syncing and replication of virtual machine data in it. To reduce
downtime when a node failure or necessary maintenance, virtual
machines can be moved to another node in the cluster. The
system is expected to perform live migration to another node.
The system can then be tested and analyzed to find the system
performance of the network and disk devices. In this study the
implementation of the Virtual Data Center performed using
nested virtualization.
94
 2015 International Conference on Data and Software Engineering

This study will be simulated on the virtual private network by cluster system in the guest hypervisor as seen in Figure 1. Then
applying the selection system components approaching the actual do the installation process DRBD as shared storage. The latter
physical system. The components used is a server machine that is process makes container OpenVZ and KVM virtual machine on
connected to the external network through the switch. Server Proxmox guest web interface according to the specifications.
serves as a hypervisor host machines (L0) with hypervisor
Proxmox, hereinafter in this study is referred to as the host
Proxmox. Proxmox host contains two guest hypervisor (L1)
which also uses Proxmox hypervisor as virtual machines with the
type of KVM, hereinafter in this study is called guest Proxmox.
Proxmox election because the open source hypervisor that can
reduce system costs.
In addition to use a KVM hypervisor so that the guest can use
technology VirtIO (para-virtualization) as the network and disk
devices, and can be election 'host' for the type of CPU. VirtIO use
will create a buffer in the system to improve performance. Host
CPU type is selected so that the guest hypervisor has the same
CPU technology with Proxmox host, so as to improve
performance. Selection of the guest Proxmox disk used for the
type of raw disk size larger. The size of the disk in accordance
with specified during installation and does not use cache to
reduce traffic bus system and copying data. Two guest hypervisor
will be made into the cluster system. Inside the guest virtual Figure 1. Proxmox Implementation
machine hypervisor will be placed OpenVZ and KVM for
testing. OpenVZ virtual machine use because it has a small
overhead [6], so that it can be used on this system with
performance approaching the physical system. However, KVM is A. Cluster System Develpoment
also used to compare the length of time the migration.
At each node needs to be added to the hostname configuration
Guest hypervisor designed to have identical specifications file / etc / hosts so that the node can recognize other nodes. To
that can be implemented together with the shared storage. Shared create a cluster on proxmox command is used on the primary
storage is selected that supports DRBD cluster with two nodes. node, the node A, is 'pvecm create <namacluster>'. Then the
Each node installed with the configuration of the primary / second node, the node B, add itself to the cluster with the
primary. DRBD use the local data storage that does not require command 'pvecm add <hostnamenodeA>'. After entering the
additional data storage devices. Only data storage on each node password node A, then automatically the node B to join the
plus a new partition to differentiate between data Proxmox virtual cluster.
machine data. DRBD devices require support on a Linux system
B. Installing DRBD
to work replicating and distributing data. Logical Volume can be
used as a device support because he himself is a block device. DRBD installations necessary to perform the command 'apt-get
Partition for virtual machine data will then be applied using the install drbd8-utils'. At the time of this research, a new Proxmox
Logical Volume Manager for storing virtual machine data and supports DRBD for version 8. Before configuration, the storage
testable performance. In a system without using shared storage need to be prepared for DRBD device. The system is assumed to
cluster for testing the performance of the disc also use the same have the partition / dev / vda1 to be used as a block device. There
configuration, but the testing is done on the data partition are two DRBD configuration files that need to be added to each
Proxmox because the virtual machine files are placed together node contained in the directory /etc/drbd.d.
with data Proxmox. Both Host Proxmox, guest Proxmox, and The first configuration is global_common.conf configuration
virtual machines are placed on the network with the same subnet. shown in Figure 1. Configuration consists of the global and
common parts. On the Common composed into several sections.
IV. IMPLEMENTATIONS Part of the startup configuration timeout settings. Net part is the
This section describes the steps implementing the Virtual configuration of security settings and the handling in the event of
Data Center is a cluster system with shared storage. On the server a split brain. Part syncer a treatment configuration
host machine be installed Proxmox VE bare metal hypervisor, so synchronization.
it does not require an operating system underneath. Two Guest
Proxmox made via the web interface Proxmox host, the node A
and node B, according to the study design and hardware
specifications that have been mentioned before. Once the virtual
machine is created, the installation process is done by creating a

95
 2015 International Conference on Data and Software Engineering

• rate used to establish the replication bandwidth,

whichever is smaller than the total bandwidth
• verify-alg used to verify the contents of the data using
algorithms md5.
The second configuration file is r0.res as seen in the figure 3.
This configuration defines the resources used image. This
configuration defines the resources used for DRBD on each node.
In the configuration file there are some parameters that should be
added. The following explanation of these parameters:
• the device resources used block device applications
• disk device at low-level blocks that are used to store and
retrieve data
• address describing each node's IP address and the port
used is 7788
• Used meta internal disk

Figure 2. Configuration
In the configuration file there are some parameters that are added
as seen in the Figure 2. The following explanation of these
parameters:
• usage-count is not used for the statistical calculations
DRBD
• Resources replicated synchronously, which uses
protocol C
• WFC-timeout is the time it takes to wait for a
connection from another node at start up Figure 3. Configuration on r0.res
degr-WFC-timeout is the time it takes to wait for DRBD is run with the command '/etc/init.d/drbd start'. Meta
another node connection while only the remaining one data created with the command 'drbdadm create-md r0'.
node Command 'drbdadm - --overwrite-data-of-peer primary r0' is
• Become-primary-on valuable both because it takes both needed to change the node A as the primary node. DRBD will
nodes to process the data write then begin replicating and distributing data from node A to node
allow-two-primaries create a primary role on both nodes B, until the process is completed, the node B is also the primary
node. Furthermore, the web interface is added storage newly
• cram-hmac-alg is the algorithm used to authenticate a created virtual machine so that data can be stored on the storage.
peer node, use sha1 algorithm and password used on the
Assessment of the effect of this migration process is used as a
line shared-secret after-sb-0pri been discard-zero-
benchmark of success in meeting the availability.The result on
changes so that when split-brain and there is no primary
system testing migration time shown OpenVZ which has a range
node, synchronization of data has changed on disk
of values of 22 to 64 seconds. The average migration time is 37
• after-sb-discard-secondary 1pri selected so that when seconds. This value is greater than the time KVM migration
the split-brain and left one node, canceled any change which has a range of values from 10 to 12 seconds with an
from the secondary node average of 11.15 seconds, as seen in Table 1.
• after-sb-2pris been disconnected so that when split-brain
and a second node claimed as a primary node, it was
decided connections to prevent data corruption

96
 2015 International Conference on Data and Software Engineering

V. CONCLUSIONS AND DISCUSSION

In this study, has successfully implemented a Virtual Data
Center system uses a virtual machine on Proxmox. The system
Table 1. OpenVZ Migration Test
runs on the virtualization nested and can work as in the actual
Test Times (s) Packet Loss system. The system consists of a two-node cluster with DRBD
1 58 3 act replicate and distribute virtual machine data. The test results
2 27 11 influence the use of shared storage during migration shows the
3 33 18 system successfully migrating live virtual machines. The system
4 24 10 also managed to maintain the integrity of the data so as to
5 24 10 minimize the risk of damage or loss of data.
6 63 26
7 24 8 ACKNOWLEDGMENT
8 64 31
The research has been conducted in Computer System and
9 42 27
Network Labs, Department of Computer Science and Electronics,
10 22 8
Faculty of Mathematic and Natural Sciences, Universitas Gadjah
11 64 34
Mada, Indonesia. The research is part of Cloud.wg.ugm.ac.id
12 29 12
research activities, with researcher members, Arif Akbar.
13 60 31
14 26 10 REFERENCES
15 23 9
16 23 9
[1] Graupner, S., Kotov, V., dan Trinks, H., 2002, Resource-Sharing and
17 26 11 Service Deployment in Virtual Data Centers, Prosiding International
18 28 14 Conference on Distributing Computing Systems Workshops, Vienna
19 29 13 [2] Khanna, G., Beaty, K., Kar, G., dan Kochut, A., Application performance
20 64 26 management in virtualized server environments. Prosiding Network
Operations and Management Symposium (NOMS 2006), Vancouver.
[3] Singh, A., Korupolu, M., dan Mohapatra, D., 2008, Server-Storage
KVM can be seen on the output value of the duration of the Virtualization: Integration and Load Balancing in Data Centers, Prosiding
downtime, so the OpenVZ takes delivery of ICMP packets to see ACM/IEEE Conference on Supercomputing, Austin.
the number of packets lost for the comparison of virtual machine [4] White, R., dan Abels, T., 2004, Energy Resource Management in the
availability. The number of lost packets indicate container Virtual Data Center, Prosiding International Symposium on Electronics
inactive. ICMP packet delivery is done through a computer on and the Environment, Washington.
the same network with the system. The number of packets lost [5] Geng, H., 2015, Data Center Handbook, John Wiley & Sons, New Jersey
can be calculated from the difference between packets [6] Arfriandi, A., 2012, Perancangan, Implementasi, dan Analisis Kinerja
transmitted and received packets. In Table 1 shows the number of Virtualisasi Server Menggunakan Proxmox, VMware ESX, dan Openstack,
Tesis, Fakultas Teknik, Universitas Gadjah Mada, Yogyakarta.
packets lost between 3 and 34 pack with an average of 16
[7] Garnieri, M., 2010, Desain dan Implementasi Virtualisasi Server di PT
packets. This package is quite large amount because the package Thiess Contractors Indonesia, Skripsi, Fakultas Matematika dan Ilmu
delivery is done at intervals of 1 second. However calculating the Pengetahuan Alam, Universitas Gadjah Mada, Yogyakarta
duration of downtime can not be done accurately. At the output [8] Chang, B.R., Tsai, H.F., Chen, C.M., Lin, Z.Y., dan Huang, C.F., 2012,
of KVM, known to cause downtime migration process with an Assessment of Hypervisor and Shared Storage for Cloud Computing
average duration of 11.65 milliseconds. This test showed that Server, Prosiding International Conference on Innovation in Bio-Inspired
when the container and the migration of virtual machine is Computing and Applications, Kaohsiung.
inactive for a while. In the process known KVM migration [9] Hatta, P., 2014, Analisis dan Implementasi Virtualisasi Data Center di ICT
Center FKIP UNS, Tesis, Fakultas Teknik, Universitas Gadjah Mada,
downtime duration of less than 1 second, therefore it can be said Yogyakarta.
the system successfully execute a live migration.
[10] Malanik, D., dan Jasek, R., 2014, The Performance of the Data-Cluster
Based on the CEPH Platform with Geographically Separated Nodes,
Prosiding International Conference on Mathematics and Computers in
Sciences and in Industry, Varna

97