Professional Documents
Culture Documents
Vmware Commands Guide
Vmware Commands Guide
Vmware Commands Guide
This guide has been compiled by the consultants & trainers at Taupo Consulting and is based upon their personal experiences with
the VMware ESX Server product. The information in this guide is not verified or sanctioned by VMware Inc and we encourage our
website visitors to use www.vmware.com/vmtn as their primary source of VMware product information. We are of course delighted
if you find our shared experience documented in this guide of use in your environment.
We are experimenting with different layouts of this help guide, currently a standard HTML table with border. If you have any
suggestions, additions or corrections we would be more than happy to receive your emails on vmware@b2v.co.uk. Thanks for
visiting our site!
The version of VMware ESX Server included with Virtual Infrastructure 3 has a number of brand new command line
commands!
Boot Process
/etc/lilo.conf
LILO is the boot loader (LILO=LInux LOader) used for VMware ESX server. If you are new to Linux, then remember that
Windows has a boot loader too, it's called NTLDR. The Linux version used as the service console in ESX Server 2.x is based
upon a modified version of Red Hat Linux 7.2.
LILO is the only supported boot loader for ESX, so don't replace it with any other Linux loader, e.g. GRUB.
The lilo.conf file is the configuration text file that defines how the Linux OS will boot. If you are familiar with Windows, then
this file is similar to BOOT.INI. However, in contrast to the Windows file, the lilo.conf text file is compiled into a binary file,
and it is that binary file which is actually used by LILO at boot time.
Here is a sample section of a lilo.conf file. You can see the initrd line which specifies the ramdisk image that the boot
loader uses to load the Linux service console kernel. The Linux kernel image name is vmnix and many VMware administrators
use the term vmnix when referring to the service console.
image=/boot/vmlinuz-2.4.9-vmnix2
label=esx
root=/dev/sda2
initrd=/boot/initrd-2.4.9-vmnix2.img
read-only
append="mem=272M cpci=0:*;1:*;2:*;4:*;12:;16:*;"
If you are troubleshooting the APPEND line, then use vmkpcidivy tool. You should not have to revert to manually editing
this file. If you ever do edit this file, then you need to write those changes into the boot sector by running /sbin/lilo . If you
are unsure the right changes will be made, you can do a trial run with the command /sbin/lilo -t .
The pci device mask specified in the append line of lilo.conf is actually an include, not a mask out. The important thing to note
is that the append line defines the physical PCI bus hardware that is visible to the service console.
ESX manages allocation of PCI devices between service console and VMkernel with the expectation of the boot loader being
LILO.
You can also view PCI device allocation using the MUI, whilst logged in as root. This is found under Startup Options in the
Options tab of the MUI as shown below.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 2 of 55
Alternatively, you could use the legacy MUI web interface using the URL
http://esxserver/pcidivy
The LILO boot loader has a boot prompt as well, displayed rather inconspicuously below the red text menu. It is at this boot
prompt that you can supply additional boot parameters. You may wish to restrict LILO from accepting such user-entered boot
parameters unless a password is entered.
password=<password>
restricted
If you only enter the password line to the file, then a password would be required to boot the system, if you also have the
restricted option then you would only need the password for making boot modifications. In the LILO boot menu, any option
that requires a password has a "P" next to the image name and any option with the restricted option has an "R" next to the
image name.
If it is a concern that the /etc/lilo.conf file contains a password stored in clear text, the file should be secured using
permissions that only allow root access, i.e. rwx------. You can implement this with the chmod command and the 600 numeric
to represent rw.
Boot Order
LILO instructs the BIOS to load the service console kernel, e.g. /boot/initrd-2.4.9-vmnix2.img This kernel obeys what is stored
in the file /etc/inittab.
/etc/inittab
This file is read by the Linux init process during boot and specifies the run level to be used by the service console. The line in
this file that states the run level will look something like this.
id:3:initdefault:
The run level that the service console uses is run level 3, which specifies full multi-user mode. The init process then works
through the start up scripts in the appropriate directory. For run level 3, this directory would be
/etc/rc.d/rc3.d
The file also starts up the virtual terminals on the service console, mingetty tty2 through mingetty tty5.
The mingetty process is a manager of virtual terminals for Linux; it is a minimal version of universal getty found in UNIX. It
does not support to connections of serial port connected terminals and is therefore "lighter" than getty and performs the
majority of most terminal needs. In the past, when UNIX was deployed on large machines and dumb terminals were connected
using serial connections, the getty service was used. Nowadays, almost nobody connects to a Linux machine by the serial port,
and for that reason it was decided to lighten getty, adopting a "minimum getty" in many distributions of Linux.
/etc/rc.d/rc.local
A start up text configuration file most commonly edited when we need to share the service console's physical NIC (pNIC) with
the VMkernel.
This may be required when a server has only 2 physical NICs, but we really want 3, so we can dedicate 1 NIC to VMotion. To
do this we add the following lines to the end of the rc.local file.
You can use the insmod utility to load driver modules either by explicitly stating the path and module file or by just the module
name and insmod will locate the correct one. In the example above, the actual driver file is
/lib/modules/2.4.9-vmnix2/misc/vmxnet_console.o
If we do need to do this, then we need to decide which network functions should share a physical NIC (pNIC), for example
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 3 of 55
So, how you share your pNICs will depend on how much management traffic there is in relation to VM traffic as well as how
often VMotion operations are likely to occur.
If you need to VLAN tag the service console traffic when using the vmxnet_console module, then you just add the VLAN ID
number after the device name in rc.local. For example, to place the service console on VLAN number 105, we would modify
the insmod line to read
/etc/rc.d/rc2.d/
This directory contains the start-up scripts for run level 2.
/etc/rc.d/rc3.d/
This directory contains the start-up scripts (logical links) for run level 3. Run level 3 is used most of the time as it is command
line full multi-user mode. The start up scripts all start with the letter "S" and the following 2 digits indicate the start up order.
The "K" scripts in this directory are shutdown scripts. The S scripts we are interested in for understanding the ESX server boot
process are shown below:
S00vmkstart
S10network
S11vmware
S12syslog
S55sshd
S56xinetd
S91httpd.vmware
By looking at the script titles we can guess what some of them do, e.g. S55 starts the secure shell daemon (putty in now!),
S56 starts xinetd which amongst other things handles remote console sessions and then S91 starts, which gives us an Apache
web server, known to us as simply as the MUI. If you would like to add your own scripts, you can place them anywhere in this
start-up order. For example, if you wanted a script to start after xinetd but before the MUI, you could label it something like
"S60custom".
A neat trick if you are looking to temporarily disable a start up script is to rename the file from capital "S" to lowercase "s".
chkconfig --list
This service console tool displays a table showing which daemons are enabled for the run levels for the Linux service console.
The following is a snip output of the chkconfig --list command
If we wanted to change a service so that it is enabled for a particular run level, then we can use chkconfig –level.
The above command would turn on ntpd for run level 1, this would not affect the run levels that ntpd was already set for. So in
this example, the ntpd run levels would be
If we just want to turn on a daemon for the current run level we can just type the name of the service we want to
enable/disable with on or off as a parameter. So to turn on nfs daemon for the current run level (whatever that may be) you
would type:
chkconfig nfs on
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 4 of 55
If you are not sure what runlevel you are currently in, just use the command runlevel and the current runlevel will be
displayed.
To avoid unnecessarily rebooting an ESX server after making certain configuration changes, we can frequently just restart
the appropriate daemon. For example we could restart the Apache web server for the MUI with the command:
S12syslogd
The centralised logging system. When ESX is running, both the service console and VMkernel log messages through it.
The /etc/rc3.d/S12syslogd file is actually a logical link to the executable file in /etc/rc.d/init.d/syslog
logger
This is a great tool for creating manual entries in the log file.
logger -i -t username "This test message will appear in the service console log file!"
So now we could examine the last few lines of the service console log file to see our new entry:
tail /var/log/messages
If you were setting up logging from the service console of one ESX server to a centralised log server, then this would be a
great way of testing that the centralised logging was working as expected.
/etc/ssh/
This directory contains the "Secure Shell" configuration files. The service console has both a secure shell client and a secure
shell server (daemon).
# /etc/init.d/sshd restart
It is important to use the full path to the ssh daemon to do this. An easier way to do this is by using the service command
The configuration of the SSH server daemon is stored in the text file /etc/ssh/sshd_config. An important setting in this file is
PermitRootLogin=Yes/No. You can quickly check this with a grep on the file.
If you do edit the file, make sure you restart the service for the changes to take effect.
ssh
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 5 of 55
We can use it to gain a command line session with a remote host, typically the service console of another ESX Server. In the
following example, we are logged on to the service console of ESX server "esx01" and we are opening a command line session
with the service console of ESX server "esx02".
Once you have established an ssh session with another host, the known_hosts file on your server is populated.
~/.ssh/known_hosts
The text file ~/.ssh/known_hosts stores the RSA keys for known hosts. This file is in the hidden subdirectory .ssh, found in
every users' home directory. Note this file is maintained on a per-user basis. The ~ (tilda) character in the path above denotes
a variable corresponding to the currently logged on users' home directory.
The .ssh subdirectory is not created until you make an outbound ssh or scp connection to another host.
If you rebuild one of your ESX hosts, when you try to reconnect to it over ssh you may be prevented from connecting, if the
known_hosts file has cached the old key. In the following command, we examine the contents of the known_hosts file (we've
truncated the length of the key here!)
ssh-keygen
Generate a public & private key set for the ESX Server.
ssh-keygen -t dsa
/etc/xinetd.conf
This is the configuration file for xinetd, the eXtended InterNET services daemon.
Originally the inetd daemon helped in controlling network connections to a computer. When a request arrives at a TCP/UDP
port that is managed by inetd, the request is forwarded to a program called tcpd (/usr/sbin/tcpd). Then tcpd decides, in
accordance with the rules contained in the hosts.{allow, deny} files whether or not to grant the request. If the request is
allowed, then the the corresponding server process (e.g. ftp) can be started. This mechanism is also referred to as
tcp_wrapper.
xinetd provides access control capabilities similar to the ones provided by tcp_wrapper.
The daemon itself is stored in /usr/sbin/xinetd This launches the daemons that are bound to it on demand.
vmware-authd
This is the authentication daemon. This daemon authenticates users of the management interface (MUI) and remote consoles
using the username/password database defined in /etc/passwd. This service binds via the xinetd daemon and so the
configuration file that specifies the listening port is
/etc/xinetd.d/vmware-authd
This text file contains the settings for the VMware remote access authentication daemon. This file specifies the TCP:902 port
used by remote console.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 6 of 55
If this port was changed here, it must also be changed in the file /etc/vmware/config. Any changes must also be reflected
in the remote console client settings and VirtualCenter.
If we wanted to add Kerberos off-box authentication for MUI access, then its in the pluggable authentication module
configuration file that corresponds to this daemon that we would make a change. This file is found at
/etc/pam.d/vmware-authd
We would need to change the current "auth required" to "auth sufficient" and add a last line of "auth required" using the
Kerberos authentication module. Modification may be required to the /etc/krb5.conf, /var/kerberos/krb5kdc/kdc.conf for
server locations and /etc/hosts to resolve these server IP addresses.
S11vmware
This is a file in the /etc/rc3.d/ directory and performs the following actions:
This S11vmware file is actually a logical link file to the actual script which is stored in the file /etc/rc.d/init.d/vmware
vmware-serverd
This daemon runs on demand in the service console to provide information to any VMware service that needs it. This performs
actions in the service console on behalf of the VMware Remote Console and the web based MUI. It is started at boot time to do
any VM autostarts. This process is replaced with vmware-ccagent if VirtualCenter is installed.
vmware-ccagent
This daemon runs as the replacement for vmware-serverd that is installed when the server is managed by VirtualCenter. The
vmware-ccagent process is automatically installed on an ESX host simply by adding the host to a VirtualCenter farm, i.e. the
process upgrade is transparent to the VC administrator.
If the automatic install of this component fails, it can be installed manually by copying the appropriate RPM package from the
VirtualCenter server to the ESX host which is to be VC-managed.
to the ESX host and then from the command line run
The most likely reason you would need to do this manual method is when the VC server is on a separate subnet from the ESX
host and there is a firewall in-between. Even if TCP:902 is open between the subnets, some dynamic ports are temporarily
required for this vmware-ccagent install.
If you are running ESX Server version 2.5.2 with VirtualCenter 1.3, you will no longer see the process vmware-ccagent.
The original process name vmware-serverd remains even after adding the ESX host to a VirtualCenter farm.
If you are running ESX Server version 3 with VirtualCenter 2 (not released yet!) then you'll see something completely
different.
S91httpd.vmware
This script starts the Apache web server which provides the ESX Server MUI. Configuration is stored in
/usr/lib/vmware-mui/apache/conf/httpd.conf
This process communicates with vmware-serverd for backend data. Remember a refresh in the browser is only a refresh to
Apache, to get new data, click on the refresh button to get new kernel data. Remember if the httpd.vmware service starts and
then stops immediately, check your service console disk space.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 7 of 55
The HTML files for the MUI can be found in the following path
/usr/lib/vmware-mui/apache/htdocs/vmware/en
snmpd
This is the Master SNMP daemon in the service console. SNMP services in ESX Server are comprised of the Master SNMP Agent
and the VMware SNMP SubAgent. If you are like me and don't use SNMP everyday, then a great resource for understanding
this simple, but powerful protocol can be found at http://www.dpstele.com/layers/l2/snmp_l2_tut_part1.html.
The Master SNMP agent (snmpd) can be replaced with the HP Insight Agent or Dell OpenManage as required.
vmware-snmpd
This is the VMware SNMP SubAgent daemon.
snmpsetup.sh
This script sets up a new snmpd.conf file which allows you to see VMware ESX Server MIB items. The normal use of this script
would be to run:
# snmpsetup.sh default
Stopping agents.
Stopping snmpd: [FAILED]
Stopping vmware-snmpd: [FAILED]
Setup finished.
Restarting agents.
Starting snmpd: [ OK ]
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 8 of 55
You could then enable the Master SNMP Agent for required run-levels with
chkconfig snmpd on
Then enable the VMware SNMP SubAgent for required run-levels with
chkconfig vmware-snmpd on
Also note, that if you are configuring snmp entirely from the command line, then you will also need to update the
file /etc/vmware/config to include the text
serverd.snmpdconf.subagentenabled = "TRUE"
/etc/snmp/snmp.conf
This is the configuration file for the Master SNMP Agent.
The following is the default contents of this file after ESX has been installed.
vmware-snmptrap
snmpwalk
Used to walkthrough SNMP mibs. -M – use MIBSDIR -m all use mibs list instead of default mibs list.
SNMP Receiver
A utility to display SNMP traps. MIBs can be loaded into this.
lsmod
Lists the device driver modules loaded for the service console Linux. So we will see the service console dedicated network card
module, local SCSI adapter module and even USB modules.
If a module has a tainted value of 1, this denotes the driver is not covered under the GNU license. The same information that
lsmod produces can also be found by inspecting the file /proc/modules. We would do this with a tool such as cat. For
example:
# cat /proc/modules
There is a different command which lists the driver modules that the VMkernel is using which is called vmkload_mod and can
also be found in this guide.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 9 of 55
init 0
Instructing a halt.
init 1
Instructing run level 1
esx 1
If this is entered at the LILO boot prompt we can get a root shell. We are instructing the vmnix kernel to execute at run level 1
(single user mode).
esx 3
Again at the LILO boot prompt, this time, we are instructing the vmnix kernel to execute at run level 3 (the default). If we
suspected the run level was wrong we can use this to get back up and running without having to revert to booting Linux on its
own.
linux rescue
Used when you boot the ESX server with a Red Hat Linux CD.
chroot
Change root directory to new directory specified as a parameter.
shutdown
Brings down the system in an orderly way. This will execute the kill scripts for the current run level, which should be 3 (full
multi-user), i.e. the scripts which start with the letter 'K' in the directory /etc/rc3.d/ will be executed in order.
linux -s
At the LILO boot loader, the default options are
If we use the cursor key at the LILO screen to select one of the three default choices, the boot prompt (displayed below the
menu) changes to reflect this. This allows us to augment the boot command with an option switch.
boot: linux –s
In this case, the –s instructs Linux to boot in single user mode. A critical security point here is that in single user mode, Linux
automatically logs on as root! Once in single user mode if we wish to continue into multi-user mode then we type either exit or
CTRL-D. To restrict access to single user mode, check the "restricted" parameter in the configuration file /etc/lilo.conf.
RPM Utilities
rpm
As ESX service console is based on modified Red Hat Linux, we can use the RPM package installation method. The following
command switch (-qa) lists the rpms installed in the service console.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 10 of 55
# rpm -qa
mailcap-2.1.6-1
setup-2.5.7-1
basesystem-7.0-2
bdflush-1.5-17
chkconfig-1.2.24-1
cracklib-2.7-12
db2-2.4.14-7
etc!.....
If we are only interested in the VMware rpms, then we can just pipe the output of rpm -qa command into the grep search tool.
VMware-mui-2.5.0-11548
VMware-esx-2.5.0-11548
VMware-perftools-2.5.0-11548
VMware-ccagent-esx-2.5.0-11343
If we then want to find out more information on an individual RPM package, we can use the rpm -qi option to query a package
which reports the file version, vendor, license and description.
If we then want to know what files are included in the rpm package, we can use query with the list option to see the files
inside. For example, to see the files
rpm2cpio
If you are wanting to extract a single file from a RPM package but you don't want to install the RPM, then this is the tool for
you. Probably best if you copy the RPM to a temp directory so when you extract the RPM you can then navigate the directory
structure created in that temp directory to find the file or files you need.
i = Restore archive
d = Create landing directories
m = Create previous file modification times
v = verbose
ifup
Used to bring up a network interface. For example, to bring the eth0 interface up, we would enter:
# ifup eth0
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 11 of 55
ifdown
Used to take a network interface down. For example
# ifdown eth0
If we wish to take the interface down and then up again, we can separate these two commands with a semicolon to run the
commands consecutively.
ifconfig
Prints a list of the network interfaces. If you are used to using ipconfig in Windows operating systems, this is a very similar
tool, as such it is a quick way of finding out the IP address and MAC address of the service console NIC. The tool can not only
report on the interfaces by can perform some actions as well, such as taking the interface on or offline.
ifconfig eth0
ifconfig eth0 up
ifconfig lo down
mii-tool
Media Independent Interface tool. This tool can be used to force the service console network to a particular speed or duplex.
# mii-tool -F
Doesn’t work correctly with some network cards, including Intel 1000 Pro copper NICs.
The semicolon separating the two commands in the above example can be used to separate any two command line entries
when you wish the commands to be executed sequentially. In the Windows command line, the same thing can be achieved by
the separator "&&".
/etc/init.d/network restart
/etc/nsswitch.conf
This is the name service switch configuration file. If you need to modify the order of how names in the service console are
resolved, this is the place to make the change. You can view and edit this conf file as usual.
There will be a number of lines to this file, but the one you are likely to be interested in will start "hosts:" as shown:
In the above example, the name service will use the /etc/hosts file, then NIS+ and then the DNS name server specified in
the /etc/resolv.conf file.
/etc/hosts
This is the host name resolution lookup file, just like Windows has in the %windir%\system32\drivers\etc directory. The ESX
server MUST have an entry for itself in its own hosts file. This file should be correctly populated during the installation of ESX
Server. Here is a sample hosts file from the service console
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 12 of 55
Notice that each line has a 3rd column which specifies an alias.
hostname
This utility displays the service console hostname. There are some useful switches to this command
and
dnsdomainname
This utility will report the domain name if the following are set
This tool does not appear to let you set the DNS domain name.
/etc/resolv.conf
The text configuration file contains the DNS name server settings, i.e. the IP addresses of the DNS servers that the service
console should use for host name resolution. This file does not need to be present if you are not using DNS.
search taupoconsulting.net
nameserver 192.168.1.150
/etc/sysconfig/network
This text configuration file contains the service console hostname and default gateway IP address.
NETWORKING=yes
HOSTNAME=esx1
GATEWAY=192.168.1.1
/etc/sysconfig/network-scripts/ifcfg-eth0
The ifcfg-eth0 configuration file contains the IP address, subnet mask and device name for the service consoles network
connection. Specifically, the file contains the IP configuration for interface eth0, typically the only network interface the service
console has.
DEVICE=eth0
BOOTPROTO=static
IPADDR=192.168.1.51
NETMASK=255.255.255.0
ONBOOT=yes
/etc/sysconfig/network-scripts/network-functions
This file is found in Red Hat Linux VMs and may require editing if there is a problem obtaining a DHCP address in the guest OS.
A VMware knowledge base article exists (977) which describes this fully, but the following text may require editing in this
configuration file:
check_link_down () {
return 1;
}
Note this update only relates to Linux Guest operating systems inside a VM, this is not a setting required for the ifcfg-eth0
file in the service console.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 13 of 55
route
This command modifies or prints the routing table in the service console.
netconfig
This is the Red Hat Linux network configuration setup program. If you need to reconfigure the service console network
settings, e.g. change IP address or default gateway etc then this is a great quick way to achieve that without resorting to
directly editing the configuration files where these settings are stored. Simply enter netconfig without any parameters.
This utility will update the following IP configuration files for you
/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network
/etc/issue
File which shows ESX and vmnix version
uname
If you specify this command with the -a switch, an output similar to the following is seen:
/proc/net/NICfamily/eth0.info
A text file that can be checked to see what the service console NIC is doing. The speed of the console NIC specified in the file
modules.conf can be confirmed by this file.
netstat
This command displays the currently active network connections.
netstat --inet -n -p -e
User Administration
id
Displays the user ID (UID) for the currently logged on user, or if the command is supplied with a parameter, can be used to
display the UID of a named user.
id robin
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 14 of 55
This output tells us that the user robin has a UID of 508, a primary group membership of robin and secondary group
membership of techsupport.
alias
Great for adding your own command line shortcut commands. For example, HP-UX administrators may be very used to just
typing "lsf" to list the contents of a directory. Now this is just "ls -F" but we want the short way of typing it
The above command alias will not however persist to another login session. To have that alias available to you on next login,
you would need to add this text to your .bashrc file in your home directory.
To make the alias available to all users on the system, you could add the alias definition to the file /etc/bashrc, which is
referenced by the users' /home/<user>/.bashrc file, like an include.
If you just type alias without parameters, you will see a list of the aliases you have defined.
passwd
Used to change the password of the currently logged on user (use the command with no parameters) or for changing the
password of a named user account (supply the user name as a parameter).
passwd <user>
Remember that passwords are not stored in the /etc/passwd file, but in the file /etc/shadow
If you are ever needing to reset an unknown root account password, then it is this utility you would run after booting into Linux
single user mode.
adduser
This is just a symbolic link (shortcut!) to the useradd utility.
useradd
This command adds a user and so updates the /etc/passwd file. So the following command:
useradd sally
would add a user called sally. We could equally have created a service console user by using "Users and Groups" in the Options
tab of the MUI. We can set more than the basic properties of a user account with some additional switches. The following
command
would add a user called robin who is a member of the techsupportusers group and has a home directory /home/robin and will
receive the Linux bash shell at login.
The service console is a modified version of Red Hat Linux (RHL), and by default in RHL, when a user account is added, a group
is created of exactly the same name and has only the user account as a member. This feature is called User Private Groups
(UPG) and is discussed in more detail on the RedHat documentation website found here.
So, now that we know about UPGs, looking again at the command above, the command adds a user called robin whose
primary group (-g) is called robin and other group (-G) membership is techsupport
We can add additional parameters to the useradd command to more fully specify the account.
In the above example the users’ primary group is Finance and the shell is specified. In this case the shell is /bin/false which
is a bogus shell which would prevent interactive logon by this user. By default in the service console, the shell assigned to
users is the BASH shell - specified as /bin/bash (BASH stands for Bourne-Again SHell). It appears the only other Linux shell
that is shipped with the service console is csh (the C shell).
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 15 of 55
groupadd
Adds a group to /etc/group
groupadd esxadmins
In the above example, a new group called esxadmins is created and therefore a new line appears in /etc/group.
gpasswd
The best tool for adding users to groups, which updates the /etc/group configuration file. The following command adds the
user greg to the esxusers secondary group.
usermod
This command is used to modify a user.
Be very careful with this command if you intend to use it to modify a users' group membership. When used with –G to set the
users group membership, it is not adding the user to a group but is actually setting the list of secondary groups a user
belongs to. Therefore in the following example if bill had secondary group list of esxusers and sqladmins, then after entering:
then bill would only have a secondary group of techsupport and nothing else! We would have overwritten the entry in
the /etc/group file that listed bill as a member of esxusers and sqladmins. This is why the command gpasswd is so much
clearer.
It is good to use the id command to check what groups a user is a member of, before and after the user modification
operation to ensure you have got it right.
groupmod
This command is used to modify a group, typically to rename it.
su
When it used without parameters, we are specifying to switch to the user root. However, we can use the su command to switch
shell to any user account. In the first example, we are logged in as the user kevin and we are switching to user ali.
In this second example, we are switching from being logged on as a user called sara to being logged on as root. Notice to
switch to root, we don't need to specify a username.
[sara@esx1host sara]$ su -
Password:
[root@esx1host root]#
If we restrict the built-in user account root from logging in over the SSH protocol, then we are forcing remote users to
authenticate as themselves and then su to run privileged commands if need be, thus leaving a decent audit trail. The downside
being that those users would still know the root account password.
If you would like to restrict the use of the su command, then we can limit it to the members of a specific group called wheel.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 16 of 55
This group is defined in the /etc/group file by default and it's membership can be modified by root. In order to limit su to the
wheel group members we need to modify a configuration file called /etc/pam.d/su
There is a single line in this file that needs to be uncommented to limit the use of su. The line is shown below as it appears it
that file, all that is required is the removal of the # symbol at the start of the line.
sudo
Allows delegation of administration in terms of certain commands that normally only a particular user can execute (usually
root). So if the user ali had been given the authority to run vmkfstools, then sudo would be used like:
The vmkfstools command would then run under the security context of the root user. The superb feature of this tool is that the
user ali does not need to know or supply the root password to be able to run the delegated command. Further, we can keep an
audit trail of when sudo was invoked.
visudo
This is just the vi text editor, but it automatically opens and locks for exclusive edit, the /etc/sudoers file. The point of
visudo is to ensure we always edit the right file as the location of the sudoers file differs between nix distributions, but this
command is constant and will utilise the right sudoers file for the distribution being used.
But a great benefit of using visudo over vi, is that it performs some basic syntax checking for us!
/etc/sudoers
The text file that contains the sudo users and the rules that apply to them. The first "ALL" relates to all machines (useful if this
is a network wide file). Otherwise, this could be the hostname of the one machine we are trying to run the command on. In the
following example we are allowing the user "alistair" to run the kill command, commands in /usr/bin and commands
in /usr/sbin/alistair
The best source I've found so far on detailed use and background of sudo can be found at
http://aplawrence.com/Basics/sudo.html
/etc/group
This file contains a list of the security groups defined in the service console. We don't normally directly edit this file, but we use
the user administration
kirsten:x:505:kirsten
esxusers:x:507:kirsten,flagship
flagship:x:508:flagship
vpxuser:x:511:
adminaccount:x:512:
JohnSmith:x:513:
This may look like a list of users, but it is a list of groups. As the service console (vmnix) is a modified version of Red Hat
Linux, the Linux security configuration is the same as Red Hat. One feature of Red Hat not found in all Linux distributions is
that of the user private group (UPG). Whenever you create a user, a group of the same name is created also and the user is
made a member. The format of the file is:
groupname:x:user1,user2
so when we see groups like JohnSmith:x:513 we can assume the 513 is the UID for the user JohnSmith and this is his UPG.
/etc/passwd
This file contains a list of users defined on the server. When we add a user account to ESX server (with either the MUI or a
command line tool such as useradd) we are adding to this text file.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 17 of 55
ali:x:500:500:Alistair Sutherland:/home/ali:/bin/bash
sara:x:501:501:Sara Daniels:/home/sara:/bin/bash
janice:x:502:502::/home/janice:/bin/bash
andy:x:503:503::/home/andy:/bin/bash
username:x:userID:groupID:fullname:homedirectory:shell
vipw
Launches vi text editor and opens the /etc/passwd file.
/etc/shadow
This text file contains the user accounts' encrypted passwords.
ali:$1$tkSdSEQD$x8pXvtDZ3Xta6zza9lKqh.:12733:0:99999:7:::
sara:$1$c4jofyxg$8zjaMTXWhW2hniTXKUt7V/:12733:0:99999:7:::
If a user account has been disabled with the usermod command, a "!" will be placed in front of the encrypted password in this
file.
/etc/skel/
This is the skeleton directory; new home directories are populated with copies of the files stored in here.
NIS
Network Information Service, formerly known as Yellow Pages.
NIS is a network lookup service which consists of databases and processes. It works where a NIS master server stores the
source files for the maps such as
/etc/passwd
/etc/group
/etc/hosts
A NIS master serves a NIS domain. You can have multiple NIS servers for a domain, but only 1 is the master, other NIS
servers host read-only copies, i.e. they are slaves. NIS databases are in DBM format.
NIS client machines are those which get their configuration from the NIS Master. A NIS client runs the process ypbind.
ypserv
ypbind
The NIS client runs this process.
yp-tools
The collection of ypset, ypwhich, ypcat
/var/log
This directory stores key log files for both the service console and the VMkernel.
Of note are the vmkernel, vmkwarning & messages file logs. These logs can be viewed with the more, cat, head and tail
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 18 of 55
command line tools. We can also access these logs via the MUI via the following link in the Options tab.
If you use the sudo tool to run a command under a different security context then the log file /var/log/secure will contain the
audit trail for such activity. Check the file /etc/syslog.conf for logging settings.
You can use less /var/log/logfile and then use SHIFT-f to enable dynamic update as new data is delivered to that file.
/etc/syslog.conf
This configuration file defines the system logging settings.
local6.* /dev/tty3
lsof
List open files Pipe the results into grep to check for open ports
pam
Pluggable Authentication Module. This allows ESX server to use off-box authentication sources, e.g. Active Directory,
eDirectory/NDS, LDAP directories.
free
Shows free memory in the service console. The "-m" switch specifies to display the results in megabytes. If the service console
is low on memory, you can increase the amount of physical RAM it gets using either the MUI (Options tab, Startup Profile) or
the command line tool vmkpcidivy. Here is the output of running free -m
Given these results, I would be thinking about either running fewer VMs, disconnecting unused devices from VMs, stopping any
unnecessary applications or increasing service console RAM.
fdisk
This is the standard Linux disk partitioning tool. As an ESX administrator you shouldn't need to use this for partition creation,
but it's great for viewing the partition table. It gives great information that augments the output of the vdf -h command.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 19 of 55
Looking at the above output of the fdisk command, the last two partitions are for the VMkernel. Partitions of type "fc"
correspond to the VMKcore dump partition. Partitions of type "fb" are VMFS volumes.
If you wanted to create a new VMFS volume from the service console command, then you could use fdisk to create the custom
partition type.
fdisk /dev/sdf
makefs
If you are creating a new ext3 partition in the service console, then you should use fdisk to create the partition and then use
makefs to create the ext3 file system on the partition; the process of creating the block groups and inodes.
In the following example, we have added a 2nd disk to the service console (appearing as SCSI disk "b" i.e. /dev/sdb). By using
fdisk we have created a primary partition. Now, to create the file system we use makefs
e2label
To label the ext3 file system you have just created, you can use the e2label command.
e2label
du
Disk usage. Great for finding out which folders are using disk space
du –h /home/ali/vmware
du –h ~
du –s summary
df
Command to list disk partitions with their capacities and free space statistics. We normally use this command with the -h
switch to indicate human readable.
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 2.0G 640M 1.2G 34% /
/dev/sda1 45M 12M 31M 27% /boot
/dev/sda7 2.0G 33M 1.8G 2% /home
none 93M 0 93M 0% /dev/shm
/dev/sda8 2.0G 33M 1.8G 2% /tmp
/dev/sda6 2.0G 226M 1.6G 12% /var
/dev/sda5 9.8G 2.9G 6.5G 31% /vmimages
//win2k/share 137G 75G 61G 55% /root/class
vdf
Print disk partitions with knowledge of VMFS partitions (type FB) with human readable switch.
This is a great tool to run when first diagnosing an ESX server. The results of this command tell us whether the server was
partitioned correctly and if any partitions are constrained for disk space.
# vdf -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 2.0G 640M 1.2G 34% /
/dev/sda1 45M 12M 31M 27% /boot
/dev/sda7 2.0G 33M 1.8G 2% /home
none 93M 0 93M 0% /dev/shm
/dev/sda8 2.0G 33M 1.8G 2% /tmp
/dev/sda6 2.0G 226M 1.6G 12% /var
/dev/sda5 9.8G 2.9G 6.5G 31% /vmimages
//win2k/share 137G 75G 61G 55% /root/class
vmhba0:0:0:10 48G 15G 33G 31% /vmfs/vmhba0:0:0:10
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 20 of 55
When troubleshooting, make this your first command to run. You will be able to review if each partition for the service console
and the VMkernel has enough disk space. Just take a quick look down the "Avail" column and if you see a zero there's likely a
problem right there, or just look at the USE% column.
dd
Disk dump utility common to Linux. This can be used to copy a file while converting and formatting. This can be a quick and
dirty way of making an ISO CD-ROM image. This could be done in the service console with
This tool can be used to create an additional swap file. For example, if we did not allocate a big enough swap partition for the
service console during ESX installation, we can create one now in a file of 64MB.
If we did add a swap file, we would need to make sure it is started when ESX starts. Therefore, an entry in the file system
table /etc/fstab would be needed as this file describes the local and remote file systems to mount at boot. The total amount
of service console swap space is the sum of the swap partition and any swap files that are active.
mkswap
A command that must be run against a newly created service console swap file in order to activate it. Think of creating a swap
file with the dd command is like creating a partition, then mkswap is like formatting that partition. The swapon command then
enables the swap space when you need it.
swapon
Enables swap file for service console.
swapoff
Disables swap file for service console.
/proc/swaps
A text file that can be checked to see what swap the service console is using. The output contains a priority which shows which
swap device will be used first before the other(s). Useful to determine if swap space is getting used and if there is more than 1
swap. Remember this is vmnix (service console) swap, not VMkernel. The VMkernel swap is in one or more files on a VMFS
volume (hence the strong recommendation that even when using a SAN, a vmfs volume is created on direct attached storage
to allow local swap).
File Commands
touch
When used with a non-existent filename, this tool creates an empty file of name filename.
# touch newfile
However, this can be used to touch an existing file and update its last modified or last accessed attributes. This could be
scripted if required. Be careful and avoid running touch against any file stored on a VMFS volume, as there appears to be a
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 21 of 55
problem there. Remember that not all Linux tools are modified for VMFS awareness.
The VMFS is not an ext3 partition. but the directory /vmfs in the service console provides mount points to the VMkernel-
mounted VMFS volumes.
cat
This command is used frequently to view the contents of a text file, exactly as the command type in DOS or Windows
command line. So to view a view we could enter
# cat /etc/vmware/netmap.conf
We can also use this tool to create text files quickly at the command line, by entering the text and then using the key sequence
CTRL-D to write to file. In the following example, we create a new bare-minimum vmx file at the command line.
guestOS = "winxppro"
config.version = "6"
virtualHW.version = "3"
CTRL-D
echo
echo blah > file
Writes the text following echo command to file. This could be good for quickly creating files
Another great use of this technique is to make changes to the ESX server configuration via the /proc hierarchy, e.g. changing
the number of shares for a VM
would change the VM CPU shares to 2500. However such a change would only exist for the duration of the world created for
that VM. After the VM is powered off this in memory structure is lost. To make such a change persistent, we would need to add
the line
sched.cpu.shares = "2476"
head
By default, the head command prints the first 10 lines of the specified file. We can choose how many lines we want instead of
10 by specifying the –n switch. This is good for looking at the file /proc/vmware/vmhba:x:x:x/0:0 with the –n 22 switch. Also
good for using with the file command to determine whether a virtual disk is in ESX format or COW format.
The “–“ is crucial to making the above command work. For an ESX virtual disk we would expect to see something like standard
input: x86 boot sector.
tail
Prints the last 10 lines of the specified file. Just like the head command, there is a –n switch that can be specified to list the
last n lines of the named file.
If you are using this to view the last few entries in a log file, you can use the -f switch to "follow" changes as they happen to
the file.
sort
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 22 of 55
sort /etc/vmware/vm-list
sort –g –k 2 scores.txt
grep
Group regular expression, used to string search the files or command outputs. You can use grep –i to indicate search with
case insensitive.
or the output of a command can be piped directly into grep, for example the output of all running processes in the service
console could be searched for the string "vmware"
cut
This utility is great for stripping out unnecessary data from a file or command output. For example, if we were viewing the
contents of a file and we wished just to view a particular piece of the file, we could use something like:
cmp
This is a file compare utility which is useful for comparing two files.
find
The find utility is used much in the same way as many Windows people used the DIR command. If you know roughly what files
you are looking for, then this is the tool. The ls tool simply lists, whereas the find tool will find according to one or more
criteria, a common one being find files modified in the last day using the –mtime switch as shown in the table.
vi
We can't talk about the command line without talking about vi. This is the simple but powerful text editor in Linux and UNIX.
People tend to love it or hate it. Either way, it's nearly always there in any *nix implementation and just by memorising a few
commands you can be up and running with it. If you can use Windows Notepad, you can use vi!
vi filename
The first thing that throws you is that to enter text into your file, you need to press "i" for Insert mode. You can then enter
your text just as any other text editor. When you are done with text entering, just press the Escape (Esc) key to come out of
insert mode. If you are happy with your file, then we need to Write & Quit (wq). To enter commands in this command line
editor, rather than having menus, we have a command prompt in the application. To reach the vi command prompt, simply
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 23 of 55
enter ":" - the colon character which will automatically place your cursor at the bottom of the session. Here you can enter the
"wq" command to write and quit the editor. That's it!
SHIFT ZZ Quit the editor and save any changes made - just a fast way of doing ":wq"
Esc key Exits the current mode, e.g. out of insert mode back to view mode.
These commands are just extra if you have the inclination to learn!
/ search - if you entered /failed then the cursor would move to the first instance of "failed in the text
$ jumps to the end of the opened file
yy copy - it's y for yank!
dd delete a line (cut) if you precede this with a number e.g. 8dd, then it would delete 8 lines
p paste
%s/old/new/g substitute any occurrences of the world "old" with the world "new"
There are some great web sites which document the features of vi in superb depth, one of them is the staff site at University of
Washington which helped me. Their site is at http://staff.washington.edu/rells/R110/
nano
Another text editor, more friendly but you should use –w to avoid word wrap.
wc
Word count utility.
wc filename
setup
Allows changing of NIC, region, firewall, mouse, keyboard.
authconfig
sysntv
mouseconfig
netconfig
ls
ls -a
List files in a directory including hidden (also known as dot files due to their prefix) files.
ls -dl */
List directories in long format (does not display files). Could add as a shell alias, say lsd.
ls -ltr
If you are interested in knowing where on the disk files are stored, based on their inode, use the -i switch.
ls -lia
ll
This command is exactly the same as entering ls –al. The "ll" command is in fact an alias to the ls command with the -al
switch. You can confirm this by entering the alias command.
less
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 24 of 55
Scrollable command line, great for piping large output into. The big difference between less and more is that you can scroll up
or down in the file you are viewing.
more
Exactly the same as DOS and Windows, also great for piping large output into. For example, to view the contents of a file one
screen at a time
more /etc/ssh/sshd_config
ls -al |more
chown
Changes file ownership. If only 1 user name is specified then the user ownership is set only and the group ownership is left
unchanged as shown in the example below.
However if you wish to reset both the user owner and group owner, then rather than having to use chown and then chgrp
straight after it, you can set user and group ownership in one operation by specifying the user owner and group owner
separated by a colon as in the example shown.
chgrp
Changes the group owner for a file, leaving the user owner unchanged. In the following example, we have a virtual machine
configuration file w2k.vmx which has been created by the user bill. By default, the permissions on the vmx file will be that the
owner is the user bill, and the group owner is the group called bill. Remember in Red Hat Linux we have user private groups -
every user account has a corresponding group of the same name!
#ll
-rwxr-wr-- bill bill w2k.vmx
Now we are going to change the group owner of the file to the group called vmadmins.
So, in a full file listing, when you see 2 names, e.g. bill vmadmins, the first name is the user owner and the second name is the
group owner. In Red Hat Linux, we have something called user private groups, which means that for each user account, there
is a group account of the same name. So if you see a file owner and group owner as the same name, these are not the same
security principals, one is the user account, the other is a group of the same name.
chmod
The chmod utility is used to change file permissions and so is similar to cacls.exe found in Windows. We can use either letters
or numeric equivalency when setting permissions with chmod. We can set permissions for 3 security principals, the user, the
group and others (ugo). If you are from a Windows background then don’t confuse “o” with owner.
When we look at a file listing using ls -al the file & directory permissions are shown on the left.
In the above example, the file has 3 permissions described in the -rwxr-xr-- string. These are:
rwx for the User owner - in our example above, this is the Linux user 'ali'
r-x for the Group owner - in our example above, this is the Linux group 'vmadmins'
r for all Others - permission for any other user who is neither the user or group owner.
In this first chmod example, we are going to change the permissions on the file.txt by removing the read & execute permission
for the user owner of the file and we are also going to remove the read permission for the group owner of the file.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 25 of 55
Note that using + or – indicates we are adding to or removing from the existing permissions. If we wish to reset the permission
we use “=” to explicitly set the object permissions, overwriting anything that was already set.
Sometimes you will see a chmod command using 'a' to specify all (user, group & other), so we could quickly set read
permissions by
A more common way to set permission is using chmod is using numeric equivalent values (4,2,1 for r,w,x) and permutations
thereof.
Watch for chmod commands with 4 digits, e.g. chmod 0754. This refers to additional attributes as described below.
Sticky bit
When the sticky bit (t) is set on executable files, it tells Linux to keep the application in memory. The reason for this is to
improve load times for other users who wish to run the same executable. This relates to the multi-user nature of UNIX/Linux.
Given the speed of memory and disk access nowadays the need to keep applications in memory is much less important and so
the sticky bit isn't needed so much.
When the sticky bit is turned on for a directory, users can have read and/or write permissions for that directory, but they can
only remove or rename files that they own.
If you see a "t" in a file or directory permission, this indicates the sticky bit is set. You can turn on the sticky bit with the chmod
tool and specify "t".
chmod +t /directory
You can then view the directory with ls -al and note that the executable permissions indicator bit is shown as a "t" showing
that the directory has the sticky bit set.
The Set User ID bit is used on an executable file, so that when it is run, it is run under the security context of the file owner
and not the current user who launched that executable. So, if I have an executable file whose owner is 'root' and it has the
setuid bit set, then when I run this application as a normal user, that application would still run under 'root' privilege.
To set the UID bit, we use chmod with the "s" indicator. In the following example, the Perl script called listswitch.pl is has a
user owner 'ali' and a group owner 'vmadmins'. Once the user id bit is set on this file, whoever launches the executable will not
in fact be the owner of the process, the user 'ali' will be the process owner.
You may have already been using a program with setuid set and not even known about it! The sudo command is owned by root
and has the setuid bit set. You can check if the setuid bit is set by inspecting the file permissions
Just like SUID, setting the SGID bit for a file sets your group ID to the file's group while the file is executing. So again, we use
the chmod tool with 's' but this time we set it on the group permission.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 26 of 55
The group id bit is a great feature to enable easier management of permissions on the files in that directory. When the group id
bit is set on a directory, any files or subdirectories created in that directory will automatically have their group ownership set to
the same as the parent directory!
As we have seen above, to set any of these 3 attributes, we can use the 't' and 's' indicators. However, often we set
permissions with chmod using numerical values like 777 to represent rwx. When setting user id, group id or sticky bits using
chmod and numerical values, we use a 4th digit preceding the usual 3 used with chmod. That digit is set using the following:
So if we want to set a file with permission -rwxr-xr-x and set the user ID bit we could use the following:
# chmod 4755
which would result in a new file permission of -rwsr-xr-x. Notice the "x" of the user permission is now an "s" indicating the
setuid bit is set.
umask
Another permissions feature you may encounter is that of umask. This is set on a directory and acts as a permissions template
filter whereby default permissions on new objects are set based on what the umask removes from the standard permissions.
The most frequently used umask is 022, this would take away the write permission for the group owner and others in a
permission list, i.e. full permission equals 777, corresponding to read(4), write(2) and execute (1).
vmkpcidivy
A VMware tool. This is used to divide up the RAM and physical PCI resources in a server between the service console and the
VMkernel. Either operating system can be assigned a PCI card or the PCI card is shared between the two operating systems.
For example, a typical ESX server would have a division of physical PCI resources as:
The vmkpcidivy tool is stored in the directory /usr/sbin/vmkpcidivy. This tool asks a series of questions and should be used
with the –i switch for interactive mode. To assign a PCI card to either operating system, we use the 3 characters c, v & s.
To run, we just type vmkpcidivy -i If you add a new NIC, SCSI or fibre channel PCI card to your physical server, you should
boot the server into Linux and run the vmkpcidivy command. This way you can correctly assign the PCI card to the right
operating system and also allows you to check that the new PCI card has not changed your existing PCI assignment. Once you
have saved your changes, restart the server and boot ESX Server normally. This command is also used to refreshnames and –
q vmhba_devs For example, if I had a SAN LUN of vmhba1:0:25 and lets say I removed the VMFS from this LUN and now
wished to use it from the service console, I’d run
# vmkpcidivy -refreshnames
and then would run vmkpcidivy again this time with the query switch (-q)
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 27 of 55
# vmkpcidivy -q vmhba_devs
to find out what device name the service console was going to use for this LUN, e.g. vmhba0:0:0 /dev/sda
A very useful feature of this tool is the ability to create a new profile. This adds a new boot option to the LILO boot menu that
will have its own allocation of memory and PCI devices. If you are unsure about the changes you are making, then create a
new profile e.g. esx (modified)
This tool is no longer required in VMware ESX 3.0 as all PCI hardware is now assigned to the VMkernel!
vmkchdev
This is a little known utility that is very useful. The following use of the command
vmkchdev -L
lists the PCI devices and reports whether they are assigned to VMkernel or the service console. We can also get this
information from running vmkpcidivy, but if we only want a quick report of which device is owned by which OS, then this is
great. Notice also that the PCI device ID is reported which is very helpful where we have more than one device of the same
name, e.g. you could have 2 dual port Intel ethernet cards.
vmkfstools
The vmkfstools utility is the tool for managing virtual disks. Remember that to copy a file into a VMFS could have an adverse
affect on other VMs with virtual disks on the same LUN. We always want to avoid using file copy tools to populate a VMFS.
Copy operations will update the volume in 16k blocks causing unnecessary SCSI reservations to update vmfs metadata.
The switches that can be used with the command are listed below:
Remember that the vmfs parameter always goes last on this command parameter set for vmkfstools. This can be confusing for
the beginner as the source and target order is different for imports and exports.
If we want to simply list the files on a vmfs volumes we use the -l switch.
vmkfstools –l /vmfs/vmhba0:0:0:8
vmkfstools –l <vmfs-metadatalabel>
If we use the command with the lh switch we get the results in human readable format. Notice that file sizes are shown
rounded with the "G" symbol.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 28 of 55
Name: Local (public) Capacity 48G, 33G avail, file block size 1.0M
Permission Uid Gid Attr Bytes Last Modified Filename
rw------- 0 0 swap 1.2G Apr 26 12:30 SwapFile.vswp
rw------- 0 0 disk 2.0G Apr 26 14:07 ad1-win2000server.vmdk
rw------- 0 0 disk 2.0G Apr 27 15:21 ad2-win2000adv.vmdk
rw------- 0 0 disk 2.0G Apr 27 08:41 Clone of ad2-win2000adv.vmdk
To create a new VMFS volume, we use the -C switch. In the following example, we are creating a VMFS volume on LUN16 on
host bus adapter 1, typically the fibre channel adapter.
If someone has created a VMFS volume with an illegal character in the volume label, you may have problems removing that
volume in the MUI. If this is the case, just overwrite the VMFS volume by creating a new volume over the top of the badly
named one using the -C switch.
To create a new empty virtual disk on a VMFS volume we use the -c switch
This command would create a new virtual disk (monolithic) on the specified VMFS volume. Remember it is always better to use
the VMFS name as this will not change even if your hba hardware does.
To import a virtual disk into the VMFS we use vmkfstools with the -i switch. This will take a virtual disk in sparse (COW) format
into monolithic format without causing excessive SCSI reservations on the LUN holding the target VMFS.
As always with this command, the parameter specifying the VMFS location is always the last parameter.
If you just wish to view the properties of a VMFS volume, you can use the -P switch to print the volume properties. You can
use either the logical name for the vmhba partition or the VMFS volume label.
cos-rescan.sh
This script calls vmkfstools command with the -s switch. This is meant to be safer that directly executing vmkfstools -s as
some pre-checks are made.
vmware-cmd
A command line tool to perform VM operations, such as power on and off or connect/disconnect devices. This tool always
requires the full path to the configuration file of the VM you wish to manipulate. This tool is found in /usr/bin
There is no man page for this tool and --help doesn't yield anything beyond simply entering the command without
parameters. Some additional information is visible if you enter
vmware-cmd -h
The first thing we can look at is to registering and un-registering a VM. We use the "-s" switch to indicate we performing a
server operation, as opposed to VM operation.
The next use of this command is to list the VMs on the server. However, this will only list the registered VMs, i.e. the VMs
which are listed in the file /etc/vmware/vm-list
# vmware-cmd –l
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 29 of 55
/home/vmware/vm1/vm1.vmx
/home/vmware/vm2/vm2.vmx
/home/alistair/vmware/alisrv1/alisrv1.vmx
/home/andy/vmware/andysolaris/andysolaris.vmx
Next we are looking at connecting or disconnecting a device. Typically this will be for the connection of IDE CD-ROM ISO files
or floppy image files.
To perform power operations we unsurprisingly use the start and stop parameters. A stop operation type can be soft, try
soft or hard. A stop hard is the last resort and equivalent to a forced VM power off. Here is an example of starting and then
soft stopping a VM.
If we wish to query the current heartbeat value for a VM, the getheartbeat parameter does the trick. Remember though, that
in order to draw any meaning from this, we should query the heartbeat twice to prove the value is in fact increasing! For
example,
If we want to determine simply if the VM is powered on or not, then we can use the getstate
To find out the VMID (also known as the world ID) of a VM, we can use the getid parameter. The VMID is analogous to
process ID (PID) but is the unique ID that the VMkernel is using for the Virtual Machine Monitor. The VMID of a VM is normally
a 3 digit number greater than 100.
For every VM that is running with a VMID in the VMkernel, there are a parallel set of management processes running in the
service console. These processes are there to allow operators interact with the VM, for example, power on and off, gain remote
console access and to maintain the per-VM logging in the file vmware.log. To find the parent process ID (PID) of the
management processes that correspond to a VM, we can use the getpid parameter.
Both the VMID and PID remain unchanged while the VM is running. Once the VM is powered off, those IDs are removed and
the VM will more than likely get a new VMID and PID the next time it is powered on.
We can also use this tool to answer questions such as the commit of a REDO file to virtual disk:
0) OK
Select choice. Press enter for default <0> : 0
selected 0 : OK
vmkdump
This is used to manage the VM kernel core dump partition. We can change the partition used if required. This tool is also
needed if the core dump partition had been removed; because ESX expects it to be there when starting up, so we need to tell
ESX that it has gone.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 30 of 55
Remember the vmkcore partition does not have a mount point in the service console and is not specified as ext3. We can use
the fdisk -l command to view where the core dump partition is in relation to the disk layout.
vmkload_mod
Allows viewing with the –l switch, loaded and unloaded VMkernel modules. This command differs from lsmod which lists the
modules loaded for the service console. This is a very good way of differentiating what modules the kernel is using versus the
ones used by vmnix.
# vmkload_mod -l
vm-support
A great built-in tool which collects all configuration files on an ESX host and builds a tar archive that can be sent to VMware
support so they can have a complete picture of your system to assist in the troubleshooting effort.
A useful function of this tool is to list running VMs using the -x switch.
[root@esx1 root]#
Watch out for the creation of empty subdirectories of the name "vm-support.<pid-of-process>" in the directory where you run
this tool with the -x switch. It is safe to delete these directories.
vmware
This command can be used to add ESX and ESX SMP serial numbers using the command line.
This command can also be used to display the ESX server version and patch level vmware -v would return something like:
Devices
/etc/modules.conf
This file lists the device driver modules that will be loaded by the service console. The equivalent file for identifying the
modules loaded by the VMkernel is /etc/vmware/vmkmodule.conf The primary reason for examination or changes to this file is
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 31 of 55
to view or configure the service console network interface, eth0. This file not only lists eth0 (as discussed in network section)
but in fact this text file sometimes describes the devices that are assigned to the service console. Here is a sample
modules.conf:
Notice the options available for network cards in this file. If we cannot use the mii-tool to force NIC speed and duplex, then
we can remove the comment character ("#") from the appropriate options line in the this file. If there are problems with the
interface eth0 disappearing after a rescan SAN operation, ensure that the Ethernet alias definitions above eth0 (i.e. eth1, eth2
etc.) in modules.conf are commented out.
modinfo
This tool takes a service console driver and displays the options it supports. For example
modinfo e1000
Would produce a list of flow control settings for the Intel gigabit NIC.
modinfo cciss
Would produce the file details and version of the HP Smart Array controller.
insmod
Insert module. This command loads a device driver module.
You are only likely to encounter this command if you decide to share your service console physical network card with the
VMkernel, when it is used to load the vmxnet_console device driver module.
modprobe
/etc/vmware/vmkmodule.conf
This file lists the device driver modules that the VMkernel will load. This is the VMkernel equivalent to the service console
modules.conf file. Notice that it is vmkmodule.conf and not plural, as is the case with the equivalent service console file!
megaraid.o
nfshaper.o
tcpip
qla2200_604.o
lspci
Great tool for listing pci devices. Could be used to demo what the VM is presenting to the guest OS.
You may wish to examine /proc/pci also in order to correctly identify PCI devices and their slot configurations. One point to
note is that when you are faced with PCI slot numbers is that not all hardware vendors number their slots in a straight forward
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 32 of 55
left to right configuration. Make sure you know your slot numbers and their layout!
lsusb
A tool to list USB devices.
Gives out way more info that is actually required. Remember that USB devices cannot be presented to virtual machines in ESX
Server. If you wish to use a USB device in ESX, then you will have to use a USB over IP device and install the appropriate
driver software into your guest OS for this. The most common USB over IP device is AnywhereUSB from Digi. Details can be
found at www.digi.com/products/usb. A company called Keyspan also produce a similar device, details at www.keyspan.com
kudzu
A Red Hat tool to detect and configure hardware. However, be careful using this tool with VMware ESX if you are making
changes to network or HBA PCI resources.
dmesg
/etc/vmware/netmap.conf
This text file maps ESX virtual switch names to device names. It is a network map configuration file as opposed to the more
generic devices map config file (devnames.conf).
network0.name = "SecuredGigabit"
network0.device = "vmnic2"
network1.name = "VirtualSwitch1"
network1.device = "vmnet_1"
network2.name = "InternetSwitch"
network2.device = "vmnic1"
Remember that
A bond can be in one of three modes, out-mac (default), out-ip and standby
out-mac A VM virtual NIC is assigned to a pNIC in the bond and it uses only that
out-ip A VM TCP conversation is placed on an available pNIC
standby A VM will only use one NIC until a failure, then the other is used. There is no point in having more that 2 NICs in a
bond in this mode.
/etc/vmware/devnames.conf
This text file maps device names (example above) to modules and their PCI addresses. Note that the devnames.conf file
contains SCSI devices and NIC devices.
/etc/vmware/vmware-devices.map
Appears to be like a hardware compatibility list. Watch out for creating your own device map, devices.local
The /etc/vmware/vmware-devices.map file contains a list of devices supported by ESX Server. This release includes support for
a local version of this file, /etc/vmware/vmware-devices.map.local. Modify the vmware-devices.map.local to select different
device drivers. This file is not modified during an ESX Server upgrade, preserving your customizations. The vmware-
devices.map.local is read when the VMkernel is loaded:
Any changes to the vmware-devices.map.local file require a reboot, or at least an unload/reload of the VMkernel to take effect.
Entries in the vmware-devices.map.local files are used in addition to the entries in the vmware-devices.map file. The vmware-
devices.map.local file does not need to mirror the vmware-devices.map file.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 33 of 55
Any vmware-devices.map.local file entries that correspond to the vmware-devices.map file entries supercede the vmware-
devices.map file entries.
/etc/vmware/config
Contains some useful settings.
libdir = "/usr/lib/vmware"
dhcpd.fullpath = "/usr/bin/vmnet-dhcpd"
authd.fullpath = "/usr/sbin/vmware-authd"
authd.client.port = "902"
loop.fullpath = "/usr/bin/vmware-loop"
vmware.fullpath = "/usr/bin/vmware"
control.fullpath = "/usr/bin/vmware-cmd"
serverd.fullpath = "/usr/sbin/vmware-serverd"
wizard.fullpath = "/usr/bin/vmware-wizard"
serverd.init.fullpath = "/usr/lib/vmware/serverd/init.pl"
serverd.vpxuser = "vpxuser"
serverd.snmpdconf.fullpath = "/etc/snmp/snmpd.conf"
snmp.enable = "TRUE"
prefvmx.useRecommendedLockedMemSize = "TRUE"
autoStart.defaultStartDelay = "240"
If this file is missing or corrupted then you will get some very weird behaviour, for example the inability to power on any
virtual machine and no vmware.log file being created. If the root file system should become full, certain files can be corrupted,
make sure this isn't one of them. I've seen this file truncated a number of times. To check if this may be your problem, try
then it could be that the /etc/vmware/config file is truncated, corrupt or simply missing. Normally, simply copying this file
from another server will normally restore the server to normal operations.
/etc/vmware/hwconfig
Loads of information in this text file. Useful for finding which nic is in which team. If using alongside devnames.conf and
netmap.conf use the following command
/dev/fd0
How to address the floppy disk drive.
/dev/sda
Denotes a SCSI device in the service console.
So the first SCSI disk would be sda, the second would be sdb and so on.
/proc/vmware/pci
Bus:Sl.F Vend:Dvid Subv:Subd Type Vendor ISA/irq/Vec P M Module Name Spawned bus
000:00.0 8086:3590 1028:016e Host/PCI Intel C
000:02.0 8086:3595 0000:0000 PCI/PCI Intel 001 C
000:03.0 8086:3596 0000:0000 PCI/PCI Intel 004 C
000:04.0 8086:3597 0000:0000 PCI/PCI Intel 007 C
000:05.0 8086:3598 0000:0000 PCI/PCI Intel 010 C
000:06.0 8086:3599 0000:0000 PCI/PCI Intel 013 C
000:29.0 8086:24d2 1028:016e USB Intel 11/ 16/0x69 A C
000:29.1 8086:24d4 1028:016e USB Intel 10/ 19/0x71 B C
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 34 of 55
findnic
This tool is also known as “The VMkernel Network Card Locator”. It locates a physical NIC in an ESX server by using pings. It is
left to the operator to check which NIC is sending the echo requests by either unplugging network cables or inspecting
flickering lights on the NIC or the switch. The useful feature of this tool is we can ping based on the device name that the
VMkernel uses to access the NIC. We should remember that each physical NIC in an ESX server that is allocated to the
VMkernel does not itself have an IP address, therefore in to perform an ICMP echo request, we need to temporarily give that
NIC an IP; this is specified as the first IP parameter in the command, the second IP parameter being the ICMP destination.
The above command will send ICMP echo requests to 192.168.1.3 every 5 seconds. We could also use the –f switch which
would flood ping.
/proc
The volatile /proc directory hierarchy that can be treated as a file system but is actually held in RAM. We can interrogate the
files and directories in /proc to find out some great information about the running of the service console.
/proc/vmware
The volatile /proc/vmware directory hierarchy that can be treated as a file system but is held in RAM. We can interrogate the
files and directories in /proc/vmware to find out some great information about the running of the VMKernel.
/proc/vmware/sched/cpu
A text file snapshot of CPU scheduling. If you cat this file you can gather some very useful information, including which CPU a
world is running on, the processor affinity, cpu min & max values, shares, runtimes etc. This information is presented in
tabular format which can be great for comparing what resources different VMs are running with. The sample output shown
below has been modified in width to fit onto this page.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 35 of 55
If you determined that one of these VMs temporarily needed more CPU shares, you could increase CPU shares on the fly at the
command line, just by using echo to input a value into the shares file for that VM.
/proc/vmware/sched/ncpus
This is an in-memory file displaying the number of processors (ncpus) in the ESX server. This is a very useful file to inspect
when you are unsure how many physical processors you have and if hyperthreading is enabled.
# cat /proc/vmware/sched/ncpus
4 logical
2 physical
You can also get the same information from the top three lines of esxtop.
watch
This is a fantastic utility that polls whatever command you supply it with an displays a running changing status. For example,
we could use the command
to obtain a dynamic view of memory usage by the VMkernel. If you are viewing lots of output but can't see what is actually
changing between refreshes, we can use the -d parameter to specify display differences, thus highlighting changes between
refreshes.
/proc/vmware/vm
Every virtual machine running has a VMID, also known as a World ID (WID). A world is the software entity created in the
VMkernel that runs the virtual machine. To put it another way, every Virtual Machine Monitor (VMM) has a unique world ID
assigned for the duration it is powered on. It is analogous to a process ID in any other operating system. We can view the
VMID of a VM from the Status Monitor tab of the MUI.
<pic>
When we use the tool esxtop we get presented with two columns that look identical, VCPUID and WID. The VCPUID is the ID
number of the virtual processor of that VM. This number will be the same as the world ID of the VMM, indicated in the WID
column. Where things get interesting is when we have a VM with 2 virtual CPUs, i.e. we are using virtual SMP (symmetric
multiprocessing). In this case, a VM gets two VCPUIDs, but is still only 1 world. So the output of esxtop when you have a vSMP
VM would be similar to
In the above example, it can be seen that there are two VCPUIDS (164 & 165) that correspond to the same world ID (164).
/proc/vmware/vm/xxx/disk/vmhba
You can obtain disk queuing activity from this file for each VM. However, you will likely find that the width of the data produced
is too wide for your screen and you get a horrible line-wrap.
To avoid this and view the data in a more sensible fashion, pipe the output of the command into the less command with the -S
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 36 of 55
switch as shown:
It might also be a good idea to use the watch command on this file, as the disk queue length will be constantly changing and
when you cat the file, you may only be sampling the queue while its on zero!
/proc/vmware/scsi/
A directory which contains subdirectories for each host bus adapter (hba).
/proc/vmware/net
To view the status of the virtual Ethernet switches in vmkernel and obtain bandwidth measurements, we can inspect or sample
values from the /proc/vmware/net directory. The subdirectories of /proc/vmware/net will correspond to each virtual Ethernet
switch defined in the VMkernel.
These directories are labelled using names vmnic, vmnet and bond. If you wish to reconcile a vmnic number to the virtual
Ethernet switch name exposed in the MUI, then inspect /etc/vmware/netmap.conf.
In the subdirectories of each virtual switch (e.g. /proc/vmware/net/vmnic0 ) you will find files that correspond to per-virtual
MAC address of each VM attached to that VM.
smbclient
This is a redirector type tool to view and connect to SMB (Microsoft networking) hosts. Before updating the /etc/fstab file
with remote file system information, check first using smbclient that the share is visible. The following was produced with
To create a mount point to a Microsoft share is very straightforward. Remember, we are allowing the service console to access
a remote file system. This is not related to what virtual machines are doing. Further, we need to be careful if we are
attempting to do any file operations due to potential limits with 2GB file sizes.
Alternatively, if you just want to map temporarily to a Microsoft host and not have to modify fstab, then use smbclient
interactively as shown:
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 37 of 55
NFS
Network File System (NFS) is provided by rpc.nfsd and would normally be launched by an nfs script in /etc/rc.d. To start
using NFS to mount directories on other ESX or Linux servers, we can use the following steps:
1. Change the ESX Server which is to be the NFS server, to use medium security (using the MUI is easiest for this).
2. Add an entry to the file /etc/exports on the NFS server by either using vi text editor (/vmimages *) or use the exportfs
command
3. Check that the NFS client IP address or hostname is not excluded by the server file /etc/hosts.deny
4. It is up to you if you explicitly allow the NFS client by adding the NFS client to /etc/hosts.allow on the NFS server
7. Mount remote export directory (on the NFS server) from the NFS client with the command
a. mount –t nfs nfsserver:/export localdir/localmount
showmount
This command is used by a NFS client to see what directories are being exported by a NFS server.
showmount –e nfsserver
This command can be specified with the hostname name or IP address of the NFS server holding the exported directories.
exportfs
The exportfs command allows you to selectively export or unexport directories without restarting the various NFS services.
nfsconfig
NIS
NIS stands for Network Information Service. This was formerly called Yellow Pages (YP).
Amongst other things, NIS can ensure that the numeric user IDs are unique across the organisation. This is because numeric
user IDs are used in NFS, so we can have a mistaken identity situation as user id 515 on a nfs client will not be the same as
user id 515 on a nfs server.
vmware-mount.pl
Just like the Windows utility to mount virtual disks when they are powered off so you can check what’s in them. This will mount
ext3 and vfat as read/write but NTFS as read only. To find out what file systems are in the virtual disk, use the –p switch.
vmware-mount.pl –p /vmfs/VMFS-VOL1/win2k3.vmdk
--------------------------------------------
VMware for Linux - Virtual Hard Disk Mounter Version: 1.0 build-9638
Copyright 1998 VMware, Inc. All rights reserved. -- VMware Confidential
--------------------------------------------
If we actually want to mount a partition then we need to be specific and create a directory (or use an existing) to be our mount
point.
mkdir /myntfs
vmware-mount.pl /vmfs/VMFS-VOL1/win2k3.vmdk 1 –t ntfs –o ro /myntfs
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 38 of 55
This command will tie up the console window hence you’ll need to spawn a new window first to navigate to /myntfs to view the
contents.
mount
Mount file system This command can be used with the –t switch to specify file system type, e.g. nfs, smbfs or iso9660
mount /mnt/cdrom
We can also use mount to gain access at the service console to an ISO image or floppy disk image using the following mount
syntax:
If you want to make your own ISOs then you can use the utility mkisofs (not included in the service console) to select the files
and create the ISO file, then use the cdrecord utility to write to device.
umount
Un-mount file system. Note it is u-mount and not unmount!
smbmount
A mount –t smbfs passes control across to this utility. We can use this utility directly if we prefer for mounting SMB host file
systems.
/etc/fstab
This is the file system table. This file describes the partitions and storage that the service console can access and how. The first
column is the device name, the second is the mount point.
If we have smb mount points defined in the fstab file, then this file could end up with user credentials in it. The fstab file is
readable by everyone so this would not be good. We can place the credentials for the smbmount in a hidden secured file in our
home folder eg. /root/.smbcreds
echo username=user > .smbcreds echo password=pass >> .smbcreds chmod 600 .smbcreds
Then in the /etc/fstab file we substitute the username and password for credentials=/root/.smbcreds. Therefore the whole
line in the fstab would be
The noauto option specifies that this mount point should not be automatically mounted at boot. The administrator will mount
and umount this as is required.
The 0 0 at the end of the line specifies backup pass and fsck pass
The backup pass flag relates to backup methods and generally you won't be backing up remote mounts in the service console.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 39 of 55
The fsck pass flag is a kind of dirty flag. If a file system were un-cleanly closed, then fsck would check that flag on next boot
and fix any errors found. Again, in the ESX service console, it's likely your mount points will be remote and will employ their
own file system checks.
Therefore, your custom entries in /etc/fstab will typically terminate with the text noauto 0 0
vmfs_ftp
Binary equivalent of ftp client, both vmfs_ftp and ftp are stored in /usr/bin.
lynx
This is a text mode web browser. Not that crazy, but a useful tool! Try it with this web page if you need command line help!
wget
An http file get utility. For example, if you were at the command line and you needed the fix script for VMtools install for SuSe
Linux you could use:
wget http://woody.linif.org/vmconffix.sh
Shell
~/.bashrc
A hidden file that extends shell script for the BASH shell. This is a hidden file that is found in each users home directory.
~/.bash_history
Another hidden file which stores the previously entered commands by the user, i.e. a command history.
~/.bash_logout
A hidden script file that executes when a user logs out. The default content of this file is
# ~/.bash_logout
clear
/etc/bashrc
System-wide bash shell settings.
set mode
If you are a dedicated fan of the vi text editing tool then you bring it's functionality to the command line interpreter with this
environment setting.
history
This command lists the commands you have previously entered along with a numeric index ID for each one.
To re-use one of your previous commands, just enter an exclamation mark followed by the numeric ID of the command you
wish to re-use. For example, here we are using the history to view the commands and then re-using one by its numeric ID.
[ali@esx1 ali]$ !2
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 40 of 55
The history command is dependent upon the ~/.bash_history file being present and populated with data.
A great variation on this is just to use CTRL-R at the command line. This brings up a searchable command history which is very
powerful...try it out!
clear
The clear command clears the terminal of existing output and returns the cursor to the first line.
This is the equivalent of the CLS command found in MS-DOS and the Windows command prompt. A great shortcut way of
doing a clear is CTRL-L, what is very cool, is that if you are in the middle of typing a command you can do a CTRL-L and the
screen is cleared but your command line is still maintained!
sum
Prints the 16-bit checksum and size of the specified file.
md5sum
Prints the MD5 checksum for the file supplied as a command parameter. VMware publish md5 checksums for all their
downloads on the www.vmware.com website to allow the end user to confirm that the file had not be corrupted in download.
date
If we are checking the time and date of our ESX Service Console, then the date command is very useful. Just entering the
"date" command returns what the service console thinks the current date is.
If the date is incorrect and you wish to reset it you would enter the command with the -s switch and specify date in dd/m/yyyy
format.
Once you have set the date, you will want to ensure that the hardware clock matches your newly entered date. We can do this
with the hwclock command described below.
hwclock
We can use this command to synchronise the server hardware clock with the date we set in the service console. If you enter
the command with no parameters then the value of the hardware clock is displayed.
# hwclock
If we want to synchronise the hardware clock with the service console date and time, we use the following:
# hwclock -systohc
cal
Display calendar for current month or set of months. The following command displays 3 months, current month and the month
before and after.
# cal -3
March 2006 April 2006 May 2006
Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa
1 2 3 4 1 1 2 3 4 5 6
5 6 7 8 9 10 11 2 3 4 5 6 7 8 7 8 9 10 11 12 13
12 13 14 15 16 17 18 9 10 11 12 13 14 15 14 15 16 17 18 19 20
19 20 21 22 23 24 25 16 17 18 19 20 21 22 21 22 23 24 25 26 27
26 27 28 29 30 31 23 24 25 26 27 28 29 28 29 30 31
30
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 41 of 55
Surprisingly useful!
/etc/vmware/vm-list
This is an auto-generated text file listing the VMs on the ESX Server. This file should not be manually edited and should match
up with output of the command
vmware-cmd –l
When you register a VM, it is automatically added to this file. The order in which VMs appear listed in the MUI is dependent
upon the order in which the VMs are listed in this file.
Process Management
PID
PID stands for Process ID. Every running process has a process ID that is valid for the length of process execution. Use the ps
command to view the service console processes and their associated PIDs.
PPID
PPID stands for Parent Process ID, which is the PID of the process that launched that process.
ps
Show running processes in the service console.
ps –A ps –eaf
ps –eaf |grep vmware-serverd
ps –efw
The -f switch
is useful as the “w” indicates wide format, so we can see the full directory path to the vmx file.
Another good option is the H option to show the process hierarchy in a similar way to pstree.
ps -eH
which might keep Solaris people happy as we don't have the ptree utility in Linux.
pstree -h
init-+-crond
|-gpm
|-httpd---3*[httpd]
|-keventd
|-khubd
|-4*[kjournald]
|-klogd
|-5*[mingetty]
|-scsi_eh_0
|-snmpd
|-sshd---sshd---bash---pstree
|-syslogd
|-vmfs_flush
|-vmklogger
|-vmkstatus---sleep
|-vmware-ccagent---vmware-ccagent
|-5*[vmware-vmx-+-vmware-mks]
| |-2*[vmware-vmx]]
| `-vmware-vmx---vmware-vmx]`-xinetd
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 42 of 55
You can display this process hierarchy with process ID numbers (PID) using the -p switch. To specify that the utility lists the
processes with their command line arguments the -a switch should be used
# pstree -ap
renice
Change process priority. If there were many VMs running concurrently, it may be necessary to temporarily raise the priority of
the MUI in order to complete administrative tasks
To reset the PID of these processes back to their defaults, use renice again to set the priority to zero.
pidof
Finds the PID (process ID) of a named process.
# pidof vmware-authd
bg
This is used to place a process in the background. For example, if we started a process at the command line that was time
consuming and we wanted to work on something else, we could suspend that process with the CTRL-Z key sequence, and then
place that suspended process in the background using this command. For example:
$ sleep 900
CTRL-Z
[1]+ Stopped
$ bg 1
$ jobs
Now the job will be running in the background. If you want to start a process in the background just add a "&" to the end of
the command.
fg
The foreground command, used to bring a background job back to the foreground for processing. For example:
nohup
If you run a process in the background and then log out, your process will be terminated. However, if you use the nohup
command as a prefix to launching your command in the background, then your process will continue to execute in the
background until it terminates.
You still need to launch the process in the background when using the nohup command, i.e. after your command you need
an "&" character. For example:
&
We can configure processes to run in the background by adding a “&” suffix when launching the program.
# sleep 10 &
If we know a process will take a while and we need the interactive command prompt back, then it's easier to launch the
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 43 of 55
process this way, rather than start it, CTRL-Z and using bg to place it in the background.
jobs
List the process jobs running in the background.
Disk.MaxLUN
This is a VMkernel parameter setting.
By default this value is set to 8, which means the VMkernel scans LUNs 0 to 7 on start up. So, if we what to scan up to LUN
number N we must set Disk.MaxLUN to N+1. If you change this setting away from the VMkernel default, then the following file
is created (or modified if it exists already).
/etc/vmware/vmkconfig
The safest way to update this parameter is by using the MUI in the Options tab, Advanced Settings. If you wish to inspect this
value in the command line you can
# cat /etc/vmware/config/Disk/MaxLUN
Disk.MaskLUNs
This parameter controls LUN visibility, again this is in the /etc/vmware/vmkconfig file. LUN masking is only supported on fibre
channel HBAs. This overrides the Disk.MaxLUN setting.
vmhba0:0:4,6-255 would scan 0,1,2,3,5 i.e. skip 4 and skip 6 through 255
vmhba0:0:3,4,9-255 would scan 0,1,2,5,6,7,8 i.e. skip 3 & 4 and skip 9 through 255
If you have multiple paths to LUNs you wish to mask, you will need to supply a mask that masks LUNs on all available paths to
those LUNs.
/etc/vmware/vmkconfig
This is a text file which stores VMkernel configuration, like a Windows INI file or Windows registry key. The important fact to
note is that this file does not exist until you make an edit away from the default setting.
If you want to remove a setting from this file, either remove the offending line from this text file by manually editing it, or you
can use the MUI (Options Tab, Advanced Settings) and enter a value of "" (i.e. two quotation marks). Unfortunately as of ESX
2.5 just deleting the existing value in the MUI will not work.
vmkfstools -s
Used to re-scan for new LUNs on specified host bus adapter. Supply this command with the vmhba name of the HBA you wish
to rescan, for example:
vmkfstools -s vmhba0
However, this has been known to cause problems in the past, hence the development of a script called cos-rescan.sh to help.
WWN
SAN devices are identified by a world wide name, a unique 64-bit address. Remember we can use the perl script wwpn.pl to
determine quickly what the WWN is for the installed FC hba.
LUN
A LUN identifies individual units of storage behind a SCSI ID. A LUN could be a single disk, a RAID1 volume, a RAID5 volume
or a logical partition of a RAID volume. For administrators of ESX Server, a LUN is simply a unit of storage that is presented
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 44 of 55
from SAN.
Zoning
Zoning is either hard (switch port) or soft (WWN controlled).
LUN Masking
LUN Masking is a disk array feature that controls which LUNs are presented to which WWNs. The term selective presentation is
also used, particularly in HP kit.
/proc/scsi/driver/number
The WWN for an adapter would be found in this file.
Disk.SupportSparseLUN
The setting Disk.SupportSparseLUN should =1 as LUNs may be discontiguous. For example if there are disk volumes at LUNs
0,1,2 and 6 then we want to be sure that after LUN 2, the VMkernel storage driver does not stop scanning. We want the LUN
scanning to reach the last LUN specified in DiskMaxLUN parameter, regardless of whether the visible LUN numbers are
contiguous or not.
Disk.RetryUnitAttention=1
This setting tells ESX server to retry SCSI commands as vendor specific status codes may have been received and ESX may
think the volume is present but not accessible when in fact its just a message that cache has been upgraded.
vmkmultipath
Allows SAN multipath maintenance from the command line, instead of MUI, Options, Storage Management.
free
A simple Linux utility to display available free memory in the service console.
Frustratingly, this doesn't have a -h switch for human readable as the df tool does, so we need to specify -k, -m or -g for
kilobytes, megabytes and gigabytes respectively.
esxtop
This is the VMware version of top and provides CPU, memory and disk configuration real-time information just like the service
console utility top does, but this time we only see the data relating to the VMkernel so we see worlds instead of processes as
resource consumers.
6:38pm up 2 days, 4:59, 17 worlds, load average: 0.00, 0.00, 0.00, 0.00
PCPU: 1.26%, 0.00% : 0.63% used total
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 45 of 55
top
Shows the running processes in the service console and lists the top consumers of CPU time. So if someone is running
something silly in the service console like a DOOM network server!
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
18894 root 10 0 1024 1020 824 R 0.9 0.5 0:00 top
1 root 8 0 504 492 436 S 0.0 0.2 0:04 init
2 root 8 0 0 0 0 SW 0.0 0.0 0:00 keventd
3 root 19 19 0 0 0 SWN 0.0 0.0 0:00 ksoftirqd_CPU0
4 root 9 0 0 0 0 SW 0.0 0.0 0:00 kswapd
5 root 9 0 0 0 0 SW 0.0 0.0 0:00 kreclaimd
6 root 9 0 0 0 0 SW 0.0 0.0 0:00 bdflush
7 root 9 0 0 0 0 SW 0.0 0.0 0:00 kupdated
12 root 9 0 0 0 0 SW 0.0 0.0 0:01 kjournald
88 root 9 0 0 0 0 SW 0.0 0.0 0:00 khubd
283 root 9 0 0 0 0 SW 0.0 0.0 0:00 kjournald
284 root 9 0 0 0 0 SW 0.0 0.0 0:00 kjournald
285 root 9 0 0 0 0 SW 0.0 0.0 0:00 kjournald
655 root 9 0 0 0 0 SW 0.0 0.0 0:00 vmfs_flush
785 root 9 0 0 0 0 SW 0.0 0.0 0:00 scsi_eh_0
1092 root 9 0 472 460 396 S 0.0 0.2 0:00 vmklogger
1198 root 9 0 23252 22M 2292 S 0.0 12.1 7:17 vmware-ccagent
The sample output shown above is static, but the actual output of the tool is continually changing as the processes are running.
vmstat
Don’t get confused by this command, it is a Linux command, not a VMware command. This is meant to view Linux processes,
memory and paging. We generally use the vmstat command with 2 numeric parameters, the first parameter is how frequently
the tool should run (specified in seconds). The second parameter specifies how many times the tool should run e.g.
vmstat 2 3
would run the tool every 2 seconds for 3 times and then exit, producing an output similar to the following:
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 46 of 55
Under the swap heading in the output are the column headers "si" and "so" which correspond to swapped-in and swapped-out.
The general rule is that if r is consistently greater than the number of physical processors in the box, then the system will be
slow. However, given that the ESX Server service console can only use physical CPU0, the service console rule should be if r is
consistently greater than 1, the service console will be slow, directly impacting your ability to manage the ESX server. Poor
performance could manifest itself as poor MUI or remote console performance.
There is something to be aware of in the Linux service console about swap. If the service console runs out of swap, then the
survival instincts of the Linux kernel kick in! Linux will kill off other processes at random to keep itself alive, watch for this
should your MUI go down, don't just re-start it, check why it stopped by checking service console RAM using the free
command and the vmstat command.
man cpu
Great info on using the /proc/vmware file system to change share allocation and processor affinity.
/proc/vmware
This is not a "real" directory, but is an in-memory volatile view of what is happening inside the VMkernel. By examining the
contents of the files in the various directories within /proc/vmware, we can gain valuable information real-time information. A
great start is to look in the file meminfo
cat /proc/vmware/meminfo
In a subdirectory called /vm there are subdirectories for each vm labelled by number How do you find out what number
corresponds to what VM? Well, you can either just look in the "Status Monitor" tab in the MUI, or go to the command line and
run B2V's listworlds.pl script or search each vmware.log file for each VM and locate it's worldid:
Some other examples of great info that can be extracted are outlined in the following table:
/proc/vmware/sched/cpu
vmkusage
This VMware-supplied tool generates web page usage reports on the ESX server as an extension to the MUI. This was not
automatically installed in previous versions of ESX.
vmkusagectl install
to install the utilisation web pages & setup a cron job vmkusage –graph to generate graph images.
Sometimes this tool loses track of which VMs it should be charting. When this happens try using
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 47 of 55
vmkusage -rescan
vmkusage –regroove
can be used to to wipe the charting data database and start logging stats again.
If you are doing specific troubleshooting, the vmkusage tool has a hyper switch which enables sampling data at a much higher
frequency, but logging is only recorded every 1 minute. It is recommended you only use hyper on non-production systems and
even then only for specific troubleshooting.
vmkusage - hyper
There is another feature of vmkusage which allows generating of text performance reports at the command line.
There is a tool called vmktree which many customers like to use with this tool, it can be found at
http://tihlde.org/~larstr/vmktree/
hstatus
This is a web page like the MUI accessible via
http://esxserver/hstatus
Not sure if we need to be logged in for this to work, but vmkusage does appear to be required. We get loads of output on this
page, similar to running a number of command line tools. This is part of what looks like the legacy web interface to ESX server,
i.e. it doesn’t look as cool as the MUI of ESX 2.x.
This legacy web output is not supported in the current release and could contain erroneous information.
crontab
Lists the cron jobs scheduled for the user when used with the –l (list) switch. For Windows users, think of cron as being like
"Scheduled Tasks".
When customers install vmkusage they see a message stating that a cron job has been added, however crontab will not list
the new job as it is not added under the context of the root user id.
Perl Scripts
vmsnap.pl
Backup script to backup a virtual disk whilst the virtual machine is still running. This is achieved by the script creating a REDO
file and then a REDO of the REDO file, thereby reducing the changing data and allowing a straight copy of the non-changing
file.
We supply the script the vmx file as a parameter to this script which will be read to find out where the virtual disk is.
If we use this script with the -l switch we are stating that the resulting snapshot should be stored on the local server. The
default path will be /vmimages/backup?
If we choose to store the archive on another server, then the snapshot still occurs locally, but once complete, the archive is
copied automatically using scp to the target archive server.
Remember the backups produced are crash consistent. This means when you restore a snapshot image, the OS will start a file
system that was not cleanly shut down, and is therefore consistent with an OS that has just crashed.
In the following example, the -l switch has been used to specify the backup only occurs locally.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 48 of 55
[root@esx1 root]#
Some users have run into problems with vmsnap.pl reporting that their virtual disk is already in REDO mode and cannot
be snapshot backed up. Be sure to check that previous snapshot backups ran successfully and there was sufficient disk space.
vmres.pl
This script is the restore tool for vmsnap.pl
wwpn.pl
This tool is used to determine the world wide name (WWN) for a hba (fibre channel adapter). The same information can be
found in the MUI under Storage Management, but this is nice and quick if you've already got a command line open.
If you run the script with the -v switch for verbose output, you also get the WWN information for the storage processors on
SAN as well as the WWN for the ESX server hba.
The utility is called wwpn because it is listing the worldwide port name for your fibre channel hba. What can be confusing is
that there is a WWN for the node.
pbind.pl
Create or delete HBA persistent bindings.
vmware-mount.pl
This is used to mount a virtual disk in the service console while the virtual machine is not running. For example, if a
configuration file in the guest OS virtual disk is preventing a virtual machine from booting correctly, you could use this utility to
mount that virtual disk in another working VM.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 49 of 55
vmAddRedo.pl
This script adds a REDO file to an existing VM. When we add a REDO file to VM, all disk write operations are directed to the
REDO file; in other words it is a disk write delta file. The VM can be left in REDO mode if required, but its most common use for
freezing the virtual disk file of a VM while it is running so it may be backed up.
In the following example, we add a REDO file to the powered-on VM called SUNone. As shown, the script parses the
configuration file and then creates a REDO file of the same name as the virtual disk but with a file extension .REDO. The key
point here is that this can be performed against a running VM.
The REDO file is 16MB in size at creation and then grows further 16MB blocks as disk write operations dictate. The REDO file
will not exceed the size of the original virtual disk. This is a delta file, not a REDO log, so a disk can only be 100% different
from the original.
vmCommit.pl
This script commits a REDO file back into the original virtual disk file.
vmware-config-tools.pl
This script is used to reconfigure the VMware Tools installed inside a Linux VM.
cos-rescan.sh
This script can be used instead of vmkfstools -s.
vmware-config-mui.pl
This script will regenerate the MUI SSL certificates. If you ever change an ESX Server hostname, you'll need to run this script.
See the section on SSL for more details.
/usr/sbin/vmkstatus
This Perl script runs on virtual terminal 1 (tty1) of the ESX server and provides the default console screen. This script is bound
to tty1 in the initialisation table /etc/inittab. This script can be run at anytime from the command line and an output similar
to the following would be seen.
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 50 of 55
http://esx1.taupoconsulting.net/
To get direct shell access to the Service Console, you may press
Alt-F2 to switch to a virtual terminal where you may log in.
http://www.vmware.com/support/
File Management
cp
Linux file copy tool.
cp source-file destination-file
cp -a
cp -l creates a link and is an alternative to using the ln utility to create a hard link.
scp
Secure copy tool, used to copy files from one Linux host to another. For example if we are copying a virtual disk in COW
(sparse) format from the service console of one ESX server to the /vmimages directory on another, then this should do the
trick.
If you want to copy files from your Windows PC to the service console, e.g. you've just used your CD burning software to
create an ISO file and now you want it up in /vmimages, then you could use the Windows freeware pscp which comes from the
authors of the SSH client PuTTY. This Windows command line utility can be found at
http://www.chiark.greenend.org.uk/~sgtatham/putty/
Alternatively, you could use a Windows GUI tool such as WinSCP, if using the command line is not your preference. This tool
can be found at http://winscp.net/eng/index.php
A further option available to you is the free tool called Veeam, available from www.veeam.com. The advantage of this tool is
that is extremely fast at file transfers, a typical transfer of 30 minutes could be be cut to 5 minutes! Try it and see!
mv
File move tool. When we rename a file, we just move the file to a new name.
mv old-filename /new-dir/new-filename
ln
Create link. An ESX Administrator shouldn't generally need to create links, but it is useful to understand them, particularly
when looking at the contents of the /vmfs directory in the service console. Also, if you ever change time zones you'll need to
use this tool.
The purpose of the link is to allow you to access a file or directory that is located another directory by using a file in the current
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 51 of 55
directory. For example, if you wanted to access the IP configuration file /etc/sysconfig/network-scripts/ifcfg-eth0 and
you wanted to simply access this file using a filename in your home directory using a simple name like "ipconf", you could
create a link to it using the ln utility.
# ln /etc/sysconfig/network-scripts/ifcfg-eth0 ipconf
You can confirm this has worked when you perform an ls -al as the number shown after the file permissions indicates the
number of hard links to the same inode, in the following example, the link count is 2.
There are actually 2 types of link that can be created, hard and soft. What we've just done above is a hard link. A hard link is
where you have 2 file names either in the same or different directories which point at the same data on disk. As the two file
names are linked directly to same file data and file attributes, if for example you change the permissions on one of the files,
you are changing the other file as they are pointing at exactly the same file on disk - known as an inode.
A soft link, more commonly known as a symbolic link, is where you create a pointer file to the real file that contains the data.
In many ways this is like a shortcut file in Windows - i.e. a LNK file. To create a symbolic link we still use the ln utility, but with
the -s switch.
# ln -s /etc/sysconfig/network-scripts/ifcfg-eth0 ifconflink
If you now do a ls -al on the directory where you created the symbolic link, we get something like the following:
Notice that in the file description, the first byte of the file permissions, the "l" indicates that the file is in fact a symbolic link.
The best definition I've found so far for the exact differences between a hard and a soft link can be found at
http://linuxgazette.net/105/pitcher.html. Thanks to Lew Pitcher for publishing this great article.
rm
This is the equivalent of the MS-DOS or Windows command DEL. In other words, this tool deletes files.
# rm testfile
rm: remove `testfile'? y
If you need to remove all the files in a directory then we could use recursion with the -r switch
# rm -r /olddata/
shred
If you are ultra paranoid about others recovering your deleted files, then you can shred them. This utility overwrites the file 25
times (by default) with random data thus making any attempts to recover the data extremely difficult. The file itself won't be
deleted unless you specify the -u switch as well.
# shred secretfile -u
mkdir
Utility to create directories in the service console file system. In the first example we are creating a new directory called iso in
the existing directory /vmimages.
mkdir /vmimages/iso
You can create multiple directories at the same time using this tool simply by supplying multiple directory parameters
separated by the space character as shown:
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 52 of 55
rmdir
Remove directory.
wall
This tool sends a broadcast message to everybody who is logged into a terminal of the service console. This can be good for
notifying other administrators of your intended actions!
SSL
Connections to the MUI and Remote Console are secured using SSL.
/etc/vmware-mui/ssl/mui.crt
/etc/vmware-mui/ssl/mui.key
/etc/vmware/ssl/rui.crt
/etc/vmware/ssl/rui.key
If you rename your ESX host, you may wish to update the SSL certificates to reflect the new name and be consistent with the
hostname. It is possible to regenerate the SSL certificates using the command vmware-config-mui.pl.
To regenerate the remote console certificate, move the existing certificates to a temporary location and then go into the MUI,
Security Settings and choose allow unencrypted remote console sessions, click OK and then OK. Test this has worked. Then go
back into the MUI and switch the security settings back to High or enforce encryption on remote console sessions in custom. You
will now have new rui files in /etc/vmware/ssl directory.
You can find the ESX build version number from the top line of the MUI or in the file /proc/vmware/version or run the service
console command line tool vmware -v
ESX 2.5.5? ?
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 53 of 55
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 54 of 55
ESX 1.5.1
ESX 1.5.0
When ESX server is upgraded it is recommended that you upgrade the VM tools. If the upgrade of VMware tools still yields the
same tools version number as before the upgrade, re-attempt a re-install of the VMware tools whilst logged onto the guest
operating system as a local administrator, as opposed to a domain administrative account.
VMware ACE
VMware Converter
Virtual Machine Importer Version 2.0 Build 30557 - 2nd October 2006
Virtual Machine Importer Version 1.5 Build 18430 - 29th November 2005
Virtual Machine Importer Version 1.0 Build 12997 - 7th April 2005
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008
B2V Guide to VMware ESX Server Page 55 of 55
Free Virtualisation
http://www.b2v.co.uk/b2vguide2vmware.htm 1/24/2008