Professional Documents
Culture Documents
Redhat El6.3 and Centos 6.3 Installation Configuration Guide
Redhat El6.3 and Centos 6.3 Installation Configuration Guide
Date: 22/04/14
Version: 1.7
Author: OMN
Table of Contents
1 Introduction....................................................................................................................6
1.1 Objective.................................................................................................................................................... 6
1.2 Audience.................................................................................................................................................... 6
1.3 How This Document Is Organised........................................................................................................... 7
1.4 References................................................................................................................................................ 7
2 Overall Steps..................................................................................................................8
3 Commissioning Hardware.............................................................................................9
3.1 Configuring RAID Controller.................................................................................................................... 9
3.2 Setting BIOS Date and Time................................................................................................................... 11
5 Post-install OS Configuration.....................................................................................17
5.1 Updating BIOS......................................................................................................................................... 17
5.2 Updating Firmware................................................................................................................................. 17
5.3 Updating Smart Array Controller Firmware.......................................................................................... 17
5.4 Installing Additional Platform Rpms..................................................................................................... 17
5.5 Editing /etc/hosts File............................................................................................................................. 18
5.6 Identifying Physical LAN Devices......................................................................................................... 19
5.7 Via Running configure-os-post-install.sh.............................................................................................19
5.7.1 Running the Shell Script........................................................................................................................ 19
5.7.2 Checking the Outcome via check-configure-post-os-install.sh...............................................................20
5.7.3 Post-OS-Install Configuration in Detail................................................................................................... 20
5.7.3.1 Disable Prelinking (/etc/prelink.conf)................................................................................................... 20
5.7.3.2 Disable updatedb Cronjob (/etc/cron.daily/mlocate.cron)....................................................................20
5.7.3.3 Disable Some Daemon Processes (/etc/init.d)....................................................................................20
5.7.3.4 Modify /etc/sysctl.conf......................................................................................................................... 20
5.7.3.5 Modify the Number of inodes.............................................................................................................. 21
5.7.3.6 Add Limits to the Number of Open Files and Procs (/etc/security/limits.conf).....................................22
11 Glossary........................................................................................................................41
List of Tables
Table 1: Document Change History.................................................................................................................. 5
Table 7.1: Kickstart Parameters...................................................................................................................... 34
Table 8.1: Default Partitions for SMS Router, SMS Hub, BMS........................................................................36
Table 8.2: Default Partitions for SMS Router, SMS Hub, BMS - Cassandra with 10 disks layout...................36
Table 8.3: Default Partitions for Standalone Assure........................................................................................37
Table 8.4: Default Partitions for Standalone Care........................................................................................... 37
Table 8.5: Default Partitions for Co-located Assure/Care................................................................................38
Copyright Notice
Copyright © Openmind Networks Limited, 2014. All rights reserved. The copyright in this document is owned
by Openmind Networks Limited ("Openmind Networks", "OMN"). This document may not be reproduced, in
whole or in part, in any form without the express consent of Openmind Networks in writing.
Information contained in this document is proprietary and confidential to Openmind Networks. That
information, irrespective of form, must not be used other than for the purposes for which it is disclosed to the
recipient and must not under any circumstances be disclosed to any third party without the express consent
in writing of Openmind Networks. Certain Trade Marks referred to in this document are the property of
Openmind Networks, the rights of owners of other Trade Marks referred to in this document are hereby
acknowledged.
Although Openmind Networks uses all reasonable efforts to ensure the accuracy and completeness of this
document, no warranty or representation whatever is given by Openmind Networks in respect of it and any
use of or reliance on any of the information contained herein is entirely at the risk the person so acting.
Openmind Networks shall have no liability whatsoever in respect of any use of or reliance on any of such
information.
1.3 02/01/14 Tom Suzuki Replaced with ext4. Updated Section 3.1 for Smart Array
configuration and Section 5.7.2.6 for
/etc/security/limits.conf
1.4 03/03/14 Tom Suzuki Updated limits.conf values to cater for Cassandra-based
message store.
1.5 03/03/14 Tom Suzuki Updated to add a new item for /etc/sysctl.conf
1.6 09/04/14 Tom Suzuki Updated to add a note on NetworkManager to disable.
1 Introduction
1.1 Objective
This document describes the method and the procedure to follow in installing one of these operating systems
for the Openmind Networks' Traffic Control product suite.
• RedHatEL6.3 x86_64 (64-bit) with required 32-bit libraries
• CentOS6.3 x86_64 (64-bit) with required 32-bit libraries
1.2 Audience
The document is intended for all stakeholders involved in design, integration, approval and implementation of
the OMN product. It is assumed that the installation engineer is familiar with computers/a Linux operating
system.
Input Notes: Notes for assisting input is adhered in Italic, smaller font size.
TBD: The space that requires input is marked TBD with yellow highlighter.
Updated Areas: Updated paragraphs, sections, etc. are marked abc with sky blue highlighter.
1.4 References
2 Overall Steps
This section illustrates the overall steps to complete OS software installation on the Openmind's product
platform.
There are three ways to complete the step of installing operating system software for Traffic Control:
• Full Manual method (No Kickstart script used, entirely manual installation)
• Script-driven method - semi-automatic (With Kickstart script, disc partitions are manually configured)
• Script-driven method - fully automatic (No manual intervention required)
Whichever method you may choose to follow, it is recommended that OS installation and configuration be
completed on one server first. Once you have one server done correctly, it is much easier to replicate the
same task across the rest of the servers.
For deployment of four or more cluster nodes, the script-driven method enables a concurrent installation
process using multiple DVDs and USB keys.
3 Commissioning Hardware
This section describes the method of commissioning the hardware platform.
It is assumed that description in this document is based on HP Proliant DL380 G8 server with P420i Smart
Array as the hardware platform.
IMPORTANT
Whatever the logical drive configuration it may be, make sure that the subsequent logical drive
configuration is reflected in partition layout in the Kickstart script and configure-os-post-install.sh you are
going to use.
Example 1:
2 Bay 2 Bay 6
3 Bay 3 Bay 7
4 Bay 4 Bay 8
NOTE:
Standard configuration.
Example 2:
1 Bay 1 Bay 1
2 Bay 2 Bay 2
3 Bay 3 Bay 3
4 Bay 4 Bay 4
5 Bay 5 Bay 5
NOTE:
Use of Bays may vary from deployment to deployment.
Example 3:
1 Bay 1 Bay 2
2 Bay 3 Bay 4
NOTE:
ASSURE/CARE co-located server configuration.
Steps Notes
1 Turn on the server. Watch the self hard drive check on
the front green LED.
2 Press any key when prompted to do so to invoke menu
options. This allows the system to call the menu options for
iLO, then Smart Array Controller.
3 In the next screen, wait until a message "HP Smart Array
Controller XXXXX: Initializing ..." appears after an iLO option.
Right after this, a list of function keys appears for menu
options. Press F8 immediately to go into the Smart Array
Controller configuration menu.
4 Configure the logical drives in the menu. Make sure that you Make sure that hard drives are
delete any logical drive that may exist from manufacturer's paired across bays/boxes.
assembly.
5 After configuring all logical drives, select the menu for marking
the primary boot volume. When a list of configured logical
drives appears, select the first logical drive and save.
6 View the logical drives. If all is OK, exit the Smart Array Exiting the configuration menu, the
Controller configuration menu. system prompts several menu
options.
7 To set BIOS date and time, press F9 when prompted.
NOTE: If you have to use an external DVD drive, go into [F11: Boot Options] and select the USB key drive
option. The default boot order does not seem to take effect if the first choice (CDROM) drive is absent.
The agreed hardware bill of materials must be rack-mounted and cabled according to the installation
documents accompanying the hardware and the agreed "System Architecture Description" project's system
architecture document.
4.1.1 Prerequisites
• DVD drive (internal or external on the server)
• 4x USB 2.0 port available on the server
• Up to 3 USB 2.0 key drives formatted in FAT32
• Up to 3 OS DVDs
• A pocket-sized USB hub (optional but highly recommended)
configure-os-post-install.sh
check-configure-post-os-install.sh
setup-bonded-interfaces.sh
install-omn.sh
gen_cdi.sh
genlicence and genfeaturelicence binaries matching the version of libtbx of the Q-release
A text file containing a list of features licensed for the customer (i.e. cdi-extract, without commented
lines)
ii. software/<Q-release>/BASE/ to contain OMN BASE rpm files (Q-release is 13-Q1, for example)
iii. software/<Q-release>/PATCHES/ to contain OMN PATCH rpm files (Q-release is 13-Q1, for
example)
iv. software/<Q-release>/3rd-party/ to contain 3rd-party software (e.g. JDK, MySQL, HP firmware)
The device name “sda1” may vary. You can find out the correct device name by checking the virtual
consoles pressing Alt+F1, Alt+F2, Alt+F3 and Alt+F4 in succession, one of which will display a USB
connection event with the device name. If you get a message that the script is not found, simply unplug and
re-plug the USB key drive, then one of the virtual consoles will indicate the device name for the USB key
drive.
Remove the DVD when Kickstart OS installation from the media is completed successfully. The Reboot
button should be presented on the screen if auto-reboot is disabled in the Kickstart script. If auto-reboot is
enabled, a GUI login screen should be presented after the reboot. Whichever it is, log in as root to proceed.
4.2.1 Prerequisites
• DVD drive (internal or external on the server)
• OS DVD
compat-libstdc++-296.i686
cyrus-sasl-lib.i686
db4.i686
gamin.i686
glib2.i686
glibc.i686
keyutils-libs.i686
krb5-libs.i686
libacl.i686
libattr.i686
libcom_err.i686
libgcc.i686
libgssglue.i686
libgtop2.i686
libjpeg.i686
libjpeg-turbo.i686
libpng.i686
libselinux.i686
libstdc++.i686
ncurses-libs.i686
nspr.i686
nss.i686
nss-softokn-freebl.i686
nss-softokn.i686
nss-util.i686
openldap.i686
openssl098e.i686
openssl.i686
popt.i686
readline.i686
sqlite.i686
zlib.i686
SELINUX=disabled
5 Post-install OS Configuration
This section describes post-install OS configuration.
1. Prior to updating the Smart Array Controller firmware, shut down the server.
2. Take out half of the RAID1 physical drives.
3. Power on the server, and verify that the server starts up fine.
4. Shut down and put all the physical drives back in.
5. Power on the server and verify that the server starts up fine.
6. If any update is desired, install the Smart Array Controller firmware update.
7. Repeat the physical redundancy check above.
On a node of a multi-node cluster, the hostname is assigned to its Internal LAN (samson LAN) with an
incrementing number. This makes it easier to identify the node when looking at channel logs and traces.
# Internal VLAN
10.100.192.36 smsc01
10.100.192.37 smsc02
10.100.192.38 smsc03
10.100.192.39 smsc04
10.100.192.40 smsc05
# STPs
10.164.17.71 stp1a
10.164.17.72 stp1b
10.164.18.71 stp2a
10.164.18.72 stp2b
The hostname command should return the hostname, and /etc/sysconfig/network should have the
HOSTNAME variable set to the hostname assigned to the Internal LAN.
[omn@smsc01 ~]$ cat /etc/sysconfig/network | grep HOSTNAME
NETWORKING=yes
HOSTNAME=smsc01
IPV6INIT=no
:
:
where <device> is eth0, eth1, eth2, etc. Whilst this command is run, the identified port keeps flashing its
LED steadily. Once visually checked, press Ctrl+C on the keyboard to return to the prompt.
Check that the file /etc/udev/rules.d/70- persistent-net.rules defines the persistent device name
mapping.
The script makes a backup copy of the existing files above by adding .bak suffix prior to editing them.
5. Visually check the changes comparing files between before and after the changes, also viewing the
log file from the script, /root/configure-os-post-install-<hostname>.log"
#/usr/bin/updatedb -f "nfs,.......
1. Check the amount of inodes available using the df command and note the device names.
df -i
Here is an example:
[root@tc1 ~]# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda6 5799936 209310 5590626 4% /
/dev/sda1 51200 48 51152 1% /boot
/dev/sdb1 100007936 44615 99963321 1% /apps
/dev/sdc1 100007936 287 100007649 1% /data
3. Run the following to re-create the file system with one hundred million inodes - if it's needed,
i.e. if the original number of inodes is less then one hundred million inodes.
4. Bear in mind that you must use the device names on your node that corresponds to /apps, /logs
and /data partitions respectively. Example commands are given below:
mkfs.ext4 /dev/cciss/c0d2p1 -N 100000000 -L /data
mkfs.ext4 /dev/cciss/c0d1p1 -N 100000000 -L /apps
mkfs.ext4 /dev/cciss/c0d3p1 -N 100000000 -L /logs
The procedure may take more than 15 minutes per partition and all data on the partitions will be lost.
5. Mount the partitions when all is done.
mount /data
mount /apps
mount /logs
5.7.3.6 Add Limits to the Number of Open Files and Procs (/etc/security/limits.conf)
Add the following lines to increase the maximum number of open flies and processes:
From 13Q3 onwards with RHEL6.3/CentOS6.3, some additional items are set to cater for Cassandra-based
message store.
* soft memlock unlimited
* hard memlock unlimited
* soft as unlimited
* hard as unlimited
The limit changes take effect on the next shell you start and onwards, but not the currently open shells.
Verify them by running ulimit -a on the new shell as omn.
omn% ulimit -a
NOTE:
Although these values relate to each individual server, nofiles and nproc values vary subject to the number
of cluster nodes and the maximum number of open ESME connections. It is advisable to adjust these
values manually afterwards.
id:3:initdefault:
1. Create the /etc/init/samson.conf file for Upstart as root on each node, which contains the following
lines. Depending on the adjacent LAN switch, you may need to add a sleep of 30 to 60 before the
“start on stopped rc ...” statement.
console output
respawn
respawn limit 5 120
exec /apps/omn/scripts/samson.sh
2. To activate the changes and start the samson process, run the following commands as root:
root# stop samson
root# status samson
root# start samson
root# ps –fu omn
Also update the /etc/fstab file of the server for these disk partitions to have the noatime option set
instead of defaults, as follows:
It is assumed that all Ethernet LAN interfaces are full-duplex Gigabit Ethernet at least for the Internal VLAN.
Using FastEthernet on the Internal VLAN is not supported since the intra-cluster communication requires the
bandwidth of Gigabit Ethernet.
2. It is assumed that the USB key drive is mounted with the exec option. Check the mount command
output. If it is mounted with noexec option, just copy the script to /root.
3. To execute the bonding configuration script on the server, check the command-line syntax first.
Comments in the script also helps.
root# cd <script location>; ./setup-bonded-interfaces.sh --help
6. If your Kickstart script used a temporary Ethernet interface, put it back to factory default to disable it
(Refer to the one in vanilla).
7. Bounce the interfaces.
root# service network restart
8. Check the bonded interfaces. The output should be consistent with the parameters supplied in the
script. Refer to Appendix for details.
NETWORKING=yes
NETWORKING_IPV6=no
IPV6INIT=no
HOSTNAME=<short hostname>
GATEWAY=<gateway ip>
[root@smsc01 ~]# for n in `seq 153 164`; do ssh 172.21.202.$n sestatus ; done
root@172.21.202.153's password:
SELinux status: disabled
root@172.21.202.154's password:
SELinux status: disabled
:
[root@smsc01 ~]# for n in `seq 153 164`; do ssh 172.21.202.$n /etc/init.d/iptables status
; done
root@172.21.202.153's password:
Firewall is stopped.
root@172.21.202.154's password:
Firewall is stopped.
:
[root@smsc01 ~]# for n in `seq 153 164`; do ssh 172.21.202.$n chkconfig –-list iptables ;
done
root@172.21.202.153's password:
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off
:
[root@smsc01 ~]#
2. Execute ping between each deployed server, both via Internal LAN and via External LAN. In a multi-
node deployment, it is a good idea to do this in a script file. For example,
#!/bin/bash
for n in `seq 153 164` ; do echo "###### 172.21.202.${n} #####" ; ping -c 3 172.21.202.$
{n} | grep packet ; done
1. Edit and tailor the appropriate interface file in /etc/sysconfig/network-scripts/ifcfg-ethX for the
deployment. For example:
2. To make static IP routes persistent, create a static route configuration file in /etc/sysconfig/network-
scripts. The filename is route-<interface> (route-eth7 for eth7), which contains address, netmask,
gateway and device name for each static route for the interface. For examle:
3. Run the following commands to make the changes permanent. For example, if the device is eth7,
root# cd /etc/sysconfig/network-scripts
root# ifdown eth7; ifup eth7
4. Run the netstat -rn command to verify the IP route table. For example:
NOTE: The line for the secondary NTP server should not be included if there is only a single NTP
server.
2. To ensure that time does not go backwards, edit the file /etc/sysconfig/ntpd to add the option -x to
the OPTIONS flag. For example:
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"
SYNC_HWCLOCK=yes
3. To immediately synchronise the clock, run the ntpupdate command as root, which will set the date
and time to that of the NTP server specified.
root# ntpdate -u <ip address of NTP server>
6. Restart ntpd.
root# service ntpd restart
7. To check where your time is being synchronised from, use this command:
root# ntpq -p
remote refid st t when poll reach delay offset jitter
==========================================================================
*ntp.telecoms-company.com LOCAL(0) 11 u 754 1024 377 0.256 0.048 0.019
If the jitter value shown is not equal to 4000.00, then this means that synchronisation has been
successful.
There is also a file called /etc/ntp/ntpservers listing the NTP servers which the server can use. In the
system boot sequence, the NTP service may try to connect to the NTP servers in this file in the order
they appear. By default, clock.redhat.com and clock2.redhat.com are the first two entries in this file.
The boot program will report that NTP synchronisation failed when it fails to connect to
clock.redhat.com, even though the NTP synchronisation would succeed with the NTP server(s)
specified in /etc/ntp.conf.
It is best to remove these redhat.com entries from this file.
8. Check on which run-levels the ntpd service will be active by the chkconfig command.
root# chkconfig --list ntpd
If it is not ON for run-level 3 and 5, run the command to update the activation run-levels.
#version=RHEL6.3
install
cdrom
lang en_US.UTF-8
keyboard us
timezone Europe/Dublin
rootpw --iscrypted
$6$Hrnktaxxc0SkfWO7$9Rtdk4rJOTYCKI4iWcqLNHrqgIK3u.oFXLqt/EIgm4WrGBv1/vsyDpi8U54.tl1wk.kBA
nYgrME920b170VsF0
authconfig --enableshadow --passalgo=sha512 –enablefingerprint
%packages
@Base
@Core
@base
:
:
vsftpd
%end
install N/A
cdrom
timezone <local_time_zone>
reboot N/A
8.2 SMS Router, SMS Hub, Standalone BMS - Cassandra with 10 disks
Table 8.2: Default Partitions for SMS Router, SMS Hub, BMS - Cassandra with 10 disks layout
Physical Logical Hard Disc Partition FS Type On Disc Size (MB)
Drives Drive Size
2x 146GB 2 146GB /logs ext4 sdc Fill the max allowed size
6x 1TB 3 3TB /assuredb ext4 sdd Fill the max allowed size
2x 146GB 2 146GB /logs ext4 sdc Fill the max allowed size
12x 1TB 3 6TB /caredb ext4 sdd Fill the max allowed size
6x 1TB 3 3TB /caredb ext4 sdd Fill the max allowed size
6x 1TB 4 3TB /assuredb ext4 sde Fill the max allowed size
11 Glossary
Terms Description
AO Application Originated
AS Application Server
DA Destination Address
DPC Destination Point Code
GT Global Title
HLR Home Location Register
Terms Description
IP Internet Protocol
MO Mobile Originated
MO-FSM Mobile Originated Forward Short Message
MT Mobile Terminated
MT-FSM Mobile Terminated Forward Short Message
NI Network Indicator
OA Originator Address
PC Point Code
Terms Description
PLMN Public Land Mobile Network
rc Routing Context
Req Request
SN Subscriber Number
SNMP Simple Network Management Protocol
Terms Description
TC Traffic Control
TT Translation Type
VLR Visited Location Register
VT Virtual Terminal
WAN Wide Area Network