About Booting From PXE

C H A P T E R
Booting from PXE

This chapter includes the following sections:

About Booting from PXE, page 3-1 Configuring Boot over PXE, page 3-2
About Booting from PXE

This section describes the Cisco BoIB using PXE and includes a high-level overview of the boot process followed by a detailed step-by-step configuration procedure. This section includes the following topics:

Hardware Setup for PXE Boot Process Overview of the PXE Boot Process
Hardware Setup for PXE Boot Process

The hardware setup used for the following example consists of one IB host booting RHEL3 and one server running RHEL3 providing DHCP, TFTP, and NFS services. These services could be provided by a single RHEL3 node or separate network nodes connected either to the IB or Ethernet network. This document describes both cases. If some of the services are provided by Ethernet node, an Ethernet gateway is required. The Ethernet gateway configuration is described in the Configuring the Ethernet Gateway section on page 3-8.
Overview of the PXE Boot Process

This section contains an overview of the PXE boot process. See Figure 3-1 for a sample hardware setup. The system follows the following procedure to boot over PXE:
1.
The host BIOS reads the configuration from nonvolatile memory to determine the boot device order. In order to boot over IB using PXE, the first entry for the boot order should be PXE by default. Use the tvflash -o command to set the boot order without entering the BIOS, as described in Configuring the Booting Host, page 3-5. The host BIOS sends a DHCP request to a PXE-enabled DHCP server to obtain initial configuration and boot parameters, IP address, and boot loader filename.
2.
Cisco SFS Boot over InfiniBand User Guide for Linux OL-12368-01
3-1
Chapter 3 Configuring Boot over PXE
Booting from PXE
3. 4. 5. 6. 7. 8.
The host BIOS loads the boot loader from TFTP server and starts it. Boot loader reads its configuration file from TFTP server and parses it to get boot options, Linux kernel, and initrd filenames. The boot loader loads the Linux kernel and executes it, passing some options from the configuration file. The Linux kernel performs the initialization procedures and loads the initrd image from the TFTP server as a temporary root file system before mounting the root file system. The initrd image startup script starts and loads the IB stack and IPoIB driver. This process changes the root file system to an NFS volume. The Linux initialization continues with the NFS root file system, using the IPoIB interface until it is up and running.
Configuring Boot over PXE

This section describes how to perform PXE boot with the BoIB feature. To perform a PXE boot with the BoIB feature (BoIB-PXE), you must perform the steps described in the following sections:

Verifying Prerequisites, page 3-2 Performing Hardware Setup, page 3-3 Checking the HCA Firmware Version, page 3-4 Configuring the Booting Host, page 3-5 Configuring the DHCP Server, page 3-6 Configuring the TFTP Server, page 3-7 Configuring the Boot Loader, page 3-7 Configuring the Ethernet Gateway, page 3-8
Verifying Prerequisites
The following requirements must be met to perform a boot over PXE:

A host with a Cisco HCA. A server with a PXE-enabled Cisco HCA. The boot server provides DHCP, TFTP, and NFS services. These services could be provided by a single node or separate network nodes connected either to the IB or Ethernet network. This document describes both cases. If some of the services are provided by an Ethernet node, an Ethernet gateway is required. A server switch, such as an SFS 3001 Server Switch or an SFS 3012R Server Switch, with an optional Ethernet gateway.
Cisco SFS Boot over InfiniBand User Guide for Linux
3-2
OL-12368-01
Chapter 3
Booting from PXE Configuring Boot over PXE
Performing Hardware Setup

To perform the hardware setup, perform the following steps:
Step 1
Install the HCAs into your host and server. For details, see the Cisco SFS InfiniBand Host Drivers User Guide for Linux.
Note
The HCAs listed in Table 3-1 support the BoIB feature. To verify that your HCA supports BoIB, enter the tvflash -i command and verify that the word Cougarcub or Lioncub appears in the output.
HCAs and their HCA Codenames in Output
Table 3-1
Cisco Product ID SFS-HCA-X2T7-A1 SFS-HCA-X2T7R-A1 SFS-HCA-X2S7-A1 SFS-HCA-X2S7R-A1 SFS-HCA-E2T7-A1 SFS-HCA-E2T7R-A1 SFS-HCA-E2S7-A1 SFS-HCA-E2S7R-A1
Step 2 Step 3
HCA Codename in Output Cougarcub Cougarcub - RemoteBoot Cougarcub Cougarcub - RemoteBoot Lioncub Lioncub - RemoteBoot Lioncub Lioncub - RemoteBoot
Install and connect all required hardware. Connect the HCA to an IB port on your server switch. Figure 3-1 shows a sample hardware setup for the boot process.
3-3
Booting from PXE
Figure 3-1
Sample Boot over PXE Hardware Setup
Boot Server PXE-enabled firmware on HCA RHEL3 Connect the boot server either directly to InfiniBand or to an Ethernet network, but use the same IP subnet as the boot host. In case of an Ethernet-based boot server, an Ethernet gateway is used to bridge the IP subnet.
Services Optional Ethernet router Use an Ethernet gateway to bridge the IP subnet Connect optional Ethernet gateway directly to Ethernet host or to optional Ethernet router DHCP, TFTP, and NFS services
SFS 3012R Switch The first port of the HCA is connected to the InfiniBand switch
* These services could be provided by a single node or separate network nodes connected either to the InfiniBand or Ethernet network. - If some of the services are provided by an Ethernet node, an Ethernet gateway is required.
Boot Host
Diskless station with InfiniBand HCA RHEL3
Checking the HCA Firmware Version

To verify that your HCA provides the BoIB feature, enter the /usr/local/topspin/sbin/tvflash -i command from the host CLI. If .Boot appears in the description of your HCA, the HCA includes the BoIB feature. If your HCA does not run BoIB firmware, update the HCA. For instructions, see the Cisco SFS InfiniBand Host Drivers User Guide for Linux. The following example shows how to check the HCA firmware version:
host1# /usr/local/topspin/sbin/tvflash -i HCA #0: MT23108, Cougar Cub, revision A1 Primary image is v3.5.000 build 3.2.0.118, with label 'HCA.CougarCub.A1.Boot' Secondary image is v3.5.000 build 3.2.0.106, with label 'HCA.CougarCub.A1.Boot' Vital Product Data
3-4
181862
OL-12368-01
Chapter 3
Product Name: Cougar cub P/N: MHXL-CF128-T E/C: C-00 S/N: MT0420F00221 Freq/Power: N/A Checksum: Ok Date Code: N/A
Note
The firmware is available on the BoIB ISO image. For instructions on how to upgrade your firmware, see the Cisco SFS InfiniBand Host Drivers User Guide for Linux.
Configuring the Booting Host

The host drivers provide a feature, tvflash -o, that allows you to change the default boot option without using the boot ROM itself. If you decide to set the default option without using the automatic tvflash -o option, you can do this manually. The procedure is BIOS-dependent. Set the boot device order so that the BIOS selects boot over PXE to boot. This step requires you to enter the motherboard setup program, find the menu item specifying the boot order, and change it accordingly, as required. To configure the booting host, perform the following steps:
Step 1
Display the current default boot option by entering the tvflash -o command.
host1# /usr/local/topspin/sbin/tvflash -o auto_upgrade=yes boot_enable_port_1=yes boot_enable_port_2=yes boot_service_scan=yes boot_type=well_known boot_saved_port=0 boot_saved_ioc_num=0 boot_saved_dgid=0000:0000:0000:0000:0000:0000:0000:0000 boot_saved_service_name=0000:0000:0000:0000
Step 2 Step 3
Locate the boot_type field. In this example, the boot_type is set to well_known, so the boot hosts try the WKSN option by default. Change the default boot option to PXE, if the default boot_type is not PXE, by entering the tvflash -o boot_type=pxe command. Set the value to pxe. You need to use a boot ROM that is PXE-capable.
host1# /usr/local/topspin/sbin/tvflash -o boot_type=pxe Writing [============================================================] host1# /usr/local/topspin/sbin/tvflash -o auto_upgrade=yes boot_enable_port_1=yes boot_enable_port_2=yes boot_service_scan=yes boot_type=pxe boot_saved_port=0 boot_saved_ioc_num=0 boot_saved_dgid=0000:0000:0000:0000:0000:0000:0000:0000 boot_saved_service_name=0000:0000:0000:0000
3-5
Booting from PXE
Configuring the DHCP Server

To configure the DHCP server, perform the following steps:
Step 1
Verify that the DHCP server supports PXE extensions. The following example shows how to configure a Linux DHCP server. Include a section describing the IP subnet of the booting host in the DHCP server configuration /etc/dhcpd.conf file. The minimum required information is as follows:
ddns-update-style none; subnet 40.0.0.0 netmask 255.255.0.0 { next-server 40.0.0.101; filename "/pxelinux.0"; range dynamic-bootp 40.0.0.50 40.0.0.100; always-broadcast on; }
The important options are next-server, filename, and always-broadcast.

The next-server and filename options specify the TFTP server and filename of the initial boot loader. The always-broadcast option must be turned on. As part of the DHCP over IB specification, and as a requirement for the IB DHCP client to acquire the IP address, turning on the always-broadcast option causes the DHCP server to broadcast the DHCP OFFER reply, instead of unicast. IP addresses and ranges can be changed to match the target setup.
Note Step 2
The broadcast option must be specified even if the DHCP server is on the Ethernet network.
(Optional) Limit the DHCP server to specific IP interfaces by including the following statement in the /etc/sysconfig/dhcpd file: DHCPDARGS=interface In this example, the DHCP server is active only on the ib0 interface.
host1# /etc/sysconfig/dhcpd DHCPDARGS="ib0 lo0"
Step 3
Activate the DHCP server after the configuration files are prepared. To start the server, use the service dhcpd start command as root.
host1# # service dhcpd start Starting dhcpd: [ OK ]
Step 4
Make the DHCP configuration persistent across reboots by using the chkconfig command.
host1# chkconfig --list dhcpd dhcpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Step 5
Verify that the DHCP server is running.

host1# ps -ef | grep dhcp root 4069 1 0 12:06 ? 00:00:00 /usr/sbin/dhcpd ib0 lo0
3-6
OL-12368-01
Chapter 3
Configuring the TFTP Server

This section describes how to configure the TFTP server.
Note
The steps in this section assume you are using the Linux tftp-server package as a TFTP server. In this procedure, the setup tftp service is handled by the xinetd daemon. The corresponding xinetd configuration file must be edited in order to enable the service. The steps that follow show a sample of the /etc/xinetd.d/tftp file:
Step 1
Enable TFTP service. Edit the xinetd.d file to enable TFTP service with root directory /tftpboot. All file paths must be relative to this directory.
service tftp { socket_type protocol wait user server server_args disable per_source cps flags }
= = = = = = = = = =
dgram udp yes root /usr/sbin/in.tftpd -s /tftpboot no 11 100 2 IPv4
Step 2
Activate the configuration by restarting the inetd daemon.

# service xinetd restart Stopping xinetd: Starting xinetd: [ [ OK OK ] ]
The TFTP service is now active and the initial boot loader, Linux kernel, and initrd image can be placed under the TFTP root directory.
Configuring the Boot Loader

This section describes how to configure the boot loader. Only the PXE boot loader is used. The filename is pxelinux.0.
Note
The boot loader used in this example is from syslinux distribution http://syslinux.zytor.com. To configure the boot loader, perform the following steps:
Step 1
Copy the boot loader file to the tftpboot directory stated in the DHCP configuration file. The boot loader expects to find its configuration file on the TFTP server under the pxelinux.cfg directory. Create /tftpboot/pxelinux.cfg directory and create a configuration file. The boot loader configuration could be for a specific host, for a group of hosts, or for all hosts.
Step 2
3-7
Booting from PXE
To create individual configuration files, the filename must match the corresponding host IP address as a 32-bit hex number. For example, for a host with IP address 40.0.0.99, the filename would have to be 28000063. To create a single configuration file for use by all hosts (as in the example below), the filename must be default. The following example shows how to configure the boot loader to load the Linux kernel with the filename linux and initrd image filename initrd from the /tftpboot directory on the TFTP server. The example also shows how some kernel parameters are passed to the Linux kernel.
serial 0 38400 default rhel3 prompt 1 timeout 100 label rhel3 kernel linux append root=/dev/cciss/c0d0p3 initrd=initrd console=ttyS0,38400
Configuring the Ethernet Gateway

This section illustrates how to configure the Ethernet gateway. If you have configured your services so that DHCP, TFTP, or NFS servers are on the Ethernet network, an Ethernet gateway must be configured to bridge the IP subnet. The following sample configuration is for an Ethernet gateway located in an SFS 3012R Server Switch:

The Ethernet gateway is installed in slot 6. One bridging group is configured between Ethernet port 5 on the gateway and the second IB port, which is internal. No VLAN tagging is configured and the default partition key (P_KEY) is used on the IB side. The broadcast forwarding must be enabled in order for the DHCP server to work across the gateway.
SFS-3012R# configure SFS-3012R(config)# bridge-group 2 subnet-prefix 40.0.0.0 16 SFS-3012R(config)# bridge-group 2 broadcast-forwarding SFS-3012R(config)# interface ethernet 6/5 SFS-3012R(config)# bridge-group 2 SFS-3012R(config)# interface gateway 6/2 SFS-3012R(config-if-gw-2/2)#)# bridge-group 2 SFS-3012R(config-if-gw-2/2)#)# exit
3-8
OL-12368-01

About Booting From PXE

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

About Booting From PXE

Uploaded by

Copyright:

Available Formats

C H A P T E R

Booting from PXE