Professional Documents
Culture Documents
Kernel Howto
Kernel Howto
Table of Contents
The Linux Kernel HOWTO...............................................................................................................................1
Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com............................................................................1
1. Introduction..........................................................................................................................................1
2. Quick Steps − Kernel Compile............................................................................................................1
3. Loadable Modules................................................................................................................................1
4. Cloning of Linux Kernels....................................................................................................................1
5. Important questions and their answers.................................................................................................1
6. Patching the kernel...............................................................................................................................1
7. Tips and tricks......................................................................................................................................2
8. Linux Kernel Textbooks and Documents............................................................................................2
9. Kernel Files Information......................................................................................................................2
10. Other Formats of this Document........................................................................................................2
11. Appendix A − Creating initrd.img file...............................................................................................2
12. Appendix B − Sample lilo.conf.........................................................................................................2
13. Appendix C − GRUB Details And A Sample grub.conf...................................................................2
14. Appendix D − Post Kernel Building..................................................................................................2
15. Appendix E − Troubleshoot Common Mistakes...............................................................................2
1. Introduction..........................................................................................................................................3
2. Quick Steps − Kernel Compile............................................................................................................4
2.1 Precautionary Preparations...............................................................................................................4
2.2 Minor Upgrading of Kernel..............................................................................................................4
2.3 For the Impatient...............................................................................................................................4
2.4 Building New Kernel − Explanation of Steps..................................................................................5
2.5 Troubleshooting.................................................................................................................................9
2.6 Post Kernel Building.........................................................................................................................9
3. Loadable Modules...............................................................................................................................9
3.1 Installing the module utilities..........................................................................................................10
3.2 Modules distributed with the kernel................................................................................................10
3.3 Howto Install Just A Single Module ?.............................................................................................10
4. Cloning of Linux Kernels.................................................................................................................11
5. Important questions and their answers...............................................................................................12
5.1 What does the kernel do, anyway?..................................................................................................12
5.2 Why would I want to upgrade my kernel?.......................................................................................12
5.3 What kind of hardware do the newer kernels support?....................................................................12
5.4 What version of gcc and libc do I need?..........................................................................................12
5.5 What's a loadable module?..............................................................................................................12
5.6 How much disk space do I need?.....................................................................................................12
5.7 How long does it take?.....................................................................................................................12
6. Patching the kernel.............................................................................................................................13
6.1 Applying a patch..............................................................................................................................13
6.2 If something goes wrong..................................................................................................................13
6.3 Getting rid of the .orig files..............................................................................................................14
6.4 Other patches...................................................................................................................................14
7. Tips and tricks....................................................................................................................................15
7.1 Redirecting output of the make or patch commands.......................................................................15
7.2 Conditional kernel install.................................................................................................................15
7.3 Kernel updates.................................................................................................................................15
8. Linux Kernel Textbooks and Documents..........................................................................................16
i
The Linux Kernel HOWTO
Table of Contents
The Linux Kernel HOWTO
9. Kernel Files Information...................................................................................................................16
9.1 vmlinuz and vmlinux......................................................................................................................16
9.2 Bootloader Files..............................................................................................................................16
9.3 Message File....................................................................................................................................17
9.4 initrd.img.........................................................................................................................................17
9.5 bzImage...........................................................................................................................................17
9.6 module−info....................................................................................................................................17
9.7 config..............................................................................................................................................17
9.8 System.map.....................................................................................................................................17
System.map.....................................................................................................................................18
What Are Symbols?.........................................................................................................................18
What Is The Kernel Symbol Table?................................................................................................18
What Is The System.map File?........................................................................................................19
What Is An Oops?...........................................................................................................................19
What Does An Oops Have To Do With System.map?....................................................................20
Where Should System.map Be Located?........................................................................................20
What else uses the System.map.......................................................................................................21
What Happens If I Don't Have A Healthy System.map?................................................................21
How Do I Remedy The Above Situation?.......................................................................................22
10. Other Formats of this Document......................................................................................................22
10.1 Acrobat PDF format......................................................................................................................23
10.2 Convert Linuxdoc to Docbook format..........................................................................................24
10.3 Convert to MS WinHelp format...................................................................................................24
10.4 Reading various formats...............................................................................................................24
11. Appendix A − Creating initrd.img file............................................................................................25
11.1 Using mkinitrd...............................................................................................................................25
11.2 Kernel Docs...................................................................................................................................26
11.3 Linuxman Book.............................................................................................................................26
12. Appendix B − Sample lilo.conf......................................................................................................28
13. Appendix C − GRUB Details And A Sample grub.conf................................................................29
14. Appendix D − Post Kernel Building...............................................................................................31
15. Appendix E − Troubleshoot Common Mistakes............................................................................32
15.1 Compiles OK but does not boot.....................................................................................................32
15.2 The System Hangs at LILO...........................................................................................................32
15.3 No init found..................................................................................................................................32
15.4 Lot of Compile Errors....................................................................................................................33
15.5 The 'depmod' gives "Unresolved symbol error messages"............................................................33
15.6 Kernel Does Not Load Module − "Unresolved symbols" Error Messages...................................34
15.7 Kernel fails to load a module.........................................................................................................34
15.8 Loadable modules..........................................................................................................................34
15.9 See Docs.........................................................................................................................................35
15.10 make clean...................................................................................................................................35
15.11 Huge or slow kernels...................................................................................................................35
15.12 The parallel port doesn't work/my printer doesn't work..............................................................35
15.13 Kernel doesn't compile.................................................................................................................35
15.14 New version of the kernel doesn't seem to boot..........................................................................36
15.15 You forgot to run LILO, or system doesn't boot at all.................................................................36
ii
The Linux Kernel HOWTO
Table of Contents
The Linux Kernel HOWTO
15.16 It says `warning: bdflush not running'.........................................................................................37
15.17 I can't get my IDE/ATAPI CD−ROM drive to work...................................................................37
15.18 It says weird things about obsolete routing requests...................................................................37
15.19 ``Not a compressed kernel Image file''........................................................................................38
15.20 Problems with console terminal after upgrade to Linux v1.3.x...................................................38
15.21 Can't seem to compile things after kernel upgrade......................................................................38
15.22 Increasing limits...........................................................................................................................38
iii
The Linux Kernel HOWTO
Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com
v5.3, 31 March 2003
This is a detailed guide to kernel configuration, compilation, upgrades, and troubleshooting for ix86−based
systems. Can be useful for other architectures as well. This document is kept small & simple, so that even
non−technical "home computer users" will be able to compile and run the Linux Kernel.
1. Introduction
2. Quick Steps − Kernel Compile
• 2.1 Precautionary Preparations
• 2.2 Minor Upgrading of Kernel
• 2.3 For the Impatient
• 2.4 Building New Kernel − Explanation of Steps
• 2.5 Troubleshooting
• 2.6 Post Kernel Building
3. Loadable Modules
• 3.1 Installing the module utilities
• 3.2 Modules distributed with the kernel
• 3.3 Howto Install Just A Single Module ?
1. Introduction
You compile Linux kernel for one of following reasons:
Note: This document is kept small & simple, so that even non−technical "home computer users" will be able
to compile and run the Linux Kernel!
1. Introduction 3
The Linux Kernel HOWTO
Kernel re−compile is required in order to make the kernel very lean and which will result in FASTER
operating system . It is also required to support any new devices.
For minor upgrades : This step may save you time, if you want to reuse the old settings. Whenever you
install the kernel, generally you put the config file in /boot. So, you can use the existing version of config file:
Or another method is − you can copy the .config file from your old linux kernel source tree to new kernel tree.
or one other method is − you can use "make oldconfig" which default all questions based on the contents of
your existing ./.config file.
NOTE: If you do not have lot of disk space in /usr/src then you can unpack the kernel source package on any
partition where you have free disk space (like /home). Because kernel compile needs lot of disk space for
object files like *.o. For this reason the /usr/src/linux MUST be a soft link pointing to your source directory.
Note: Below 'bash#' denotes the bash prompt, you should type the commands that appear after the 'bash#'
prompt. Below are commands tested on Redhat Linux Kernel 2.4.7−10, but it should work for other
distributions with very minor changes. It should also work for older kernel versions like 2.2, 2.0 and 1.3. It
should also work for future or newer versions of kernel (with little changes − let me know).
• Note: You can have many kernel images on your system. By following the steps below you do not
overwrite or damage your existing kernel. These steps are very safe and your current kernel will be
intact and will not be touched.
1. Unpack the sources: Login in as 'root' throughout all these steps. Mount Redhat linux cdrom and
install the linux kernel source rpm
bash$ su − root
bash# cd /mnt/cdrom/RedHat/RPMS
bash# rpm −i kernel−headers*.rpm
bash# rpm −i kernel−source*.rpm
bash# rpm −i dev86*.rpm
bash# rpm −i bin86*.rpm
(The bin86*.rpm and 'as86' is required only for OLDER Linux systems like Redhat 5.x. Get Intel
assembler 'as86' command from dev86*.rpm on cdrom or from bin86−mandrake , bin86−kondara ).
Also make sure that /usr/src/linux is soft link pointing to proper unpacked source.
bash# cd /usr/src
bash# ls −l # You should see that /usr/src/linux is soft link pointing to source
lrwxrwxrwx 1 root root 19 Jan 26 11:01 linux −> linux−2.4.18−19.8.0
drwxr−xr−x 17 root root 4096 Jan 25 21:08 linux−2.4.18−14
drwxr−xr−x 17 root root 4096 Mar 26 12:50 linux−2.4.18−19.8.0
drwxr−xr−x 7 root root 4096 Jan 14 16:32 redhat
If it is not a soft link then do rename /usr/src/linux to /usr/src/linux−2.4.yy and create a soft link.
Or another method is − you can copy the .config file from your old linux kernel source tree to new
kernel tree
or one other method is − you can use "make oldconfig" which default all questions based on the
contents of your existing ./.config file.
3. Clean : Before doing mrproper below, you may want to backup the .config file.
bash# cd /usr/src/linux
bash# cp .config .config.save
bash# make clean
bash# make mrproper # Must do this if want to start clean slate or if you face lot
4. Configure:
♦ Start X−windows with 'startx'. If you are not able to start X−window then see next step
below.
If you find scrambled display, then use different terminal emulators like vt100,
vt102, vt220 or ansi. The display will be scrambled and will have garbage
characters in cases where you use telnet to login to remote linux. In such
cases you should use the terminal emulators like vt100, vt220.
For example:
bash# export TERM=vt220
bash# export TERM=ansi
The "make xconfig" or "make menuconfig" brings up a user friendly GUI interface. And "make
config" brings up command−line console mode interface. You can load the configuration file from
/usr/src/linux/.config (dot config file. Note the dot before config). Click on button "Load
Configuration from File".
♦ VERY IMPORTANT !!! : Select proper CPU type − Pentium 3, AMD K6, Cyrix, Pentium
4, Intel 386, DEC Alpha, PowerPC otherwise kernel compile will fail and even if it compiles,
it will not boot!!
♦ Select SMP support − whether single CPU or multiple CPUs
♦ Filesystems − Select Windows95 Vfat, MSDOS, NTFS as part of kernel and not as loadable
modules. (My personal preference, but you are free to pick your own option).
♦ Enable the Loadable kernel modules support! With this option you can load/unload the device
drivers dynamically on running linux system on the fly. See the Modules chapter at Loadable
Modules.
Save and Exit "make xconfig". All the options which you selected is now saved into configuration file
at /usr/src/linux/.config (dot config file).
5. Dep : And now, do −
6. Give a unique name to your new Kernel: You can give a name to your kernel, so that it is unique
and does not interfere with others.
bash# cd /usr/src/linux
bash# vi Makefile
bash# cd /usr/src/linux
bash# man nohup
bash# nohup make bzImage &
bash# man tail
bash# tail −f nohup.out (.... to monitor the progress)
This will put the kernel in /usr/src/linux/arch/i386/boot/bzImage
8. LOADABLE MODULES: Now, while the 'make' is cranking along in the previous step "Do make",
you should bring up another new xterm shell window and follow these steps: This step is required
# Redirect outputs such that you do not overwrite the nohup.out which is still running...
bash# nohup make modules 1> modules.out 2> modules.err &
bash# make modules_install # Do this, only after the above make command is successful
This will copy the modules to /lib/modules directory. See the Modules chapter at Loadable Modules.
9. Now go to Lunch or Bed : Since both the make windows are cranking along, and now, you can go to
lunch (chitchat, have nap) or go to bed (have nice Linux dreams in sleep) and when you wake up and
come back the system is ready! You can check with command 'less nohup.out' to see the log of
output.
bash# cd /usr/src/linux
bash# less nohup.out
bash# less modules.err
bash# less modules.out
10. bzImage: After bzImage is successful, copy the kernel image to /boot directory. You must copy the
new kernel image to /boot directory, otherwise the new kernel MAY NOT boot. You must also copy
the config file to /boot area to reflect the kernel image, for documentation purpose.
# You MUST copy the config file to reflect the corresponding kernel image, for documentatio
bash# cp /usr/src/linux/.config /boot/config−<your_kernelversion_date>
# Example: cp /usr/src/linux/.config /boot/config−2.4.18−19.8.0−26mar2001
NOTE : If you are planning to use the initrd in LILO or GRUB then you may want to build initrd and
place it in /boot/initrd*.img. See the Appendix A at Creating initrd.img file.
11. Configure GRUB or LILO : There are two options for boot loading under Redhat Linux − GRUB
and LILO.
Configure GRUB: GRUB is recent and much better tool than LILO and it is my first preference to
use GRUB. LILO is an older technology. GRUB differs from bootloaders such as LILO in that "it can
lie to MS Windows and make MS Windows believe that it's installed on the first partition even if it's
not!!". So you can keep your current Linux system where it is and install Windows on the side. See
the Appendix C − GRUB details and sample grub.conf file.
Configure LILO: LILO is older tool and see the Appendix B − Sample lilo.conf to configure LILO.
(see also http://www.linuxdoc.org/HOWTO/LILO−crash−rescue−HOWTO.html)
12. Reboot the machine and at lilo press tab key and type 'myker' If it boots then you did a good job!
Otherwise at lilo select your old kernel, boot and re−try all over again. Your old kernel is still
INTACT and SAFE at say /boot/vmlinuz−2.0.34−0.6
13. If your new kernel 'myker' boots and works properly, you can create the boot disk. Insert a blank
bash# cd /usr/src/linux
bash# make bzdisk
Optional − You can also build RPM packages of kernel, in case you want to install the new image on
several machines.
15. Clean: Optional − make clean (If you want to free up disk space)
2.5 Troubleshooting
Having any problems? See the troubleshooting chapter.
3. Loadable Modules
Loadable kernel modules can save memory and ease configuration. The scope of modules has grown to
include filesystems, ethernet card drivers, tape drivers, printer drivers, and more.
Loadable modules are pieces of kernel code which are not linked (included) directly in the kernel. One
compiles them separately, and can insert and remove them into the running kernel at almost any time. Due to
its flexibility, this is now the preferred way to code certain kernel features. Many popular device drivers, such
as the PCMCIA drivers and the QIC−80/40 tape driver, are loadable modules.
2.5 Troubleshooting 9
The Linux Kernel HOWTO
bash# modprobe loop
bash# insmod loop
bash# lsmod
You can set the PATH which the insmod searches in /etc/modules.conf.
insmod inserts a module into the running kernel. Modules usually have a .o extension; the example driver
mentioned above is called drv_hello.o, so to insert this, one would say `insmod drv_hello.o'. To
see the modules that the kernel is currently using, use lsmod. The output looks like this:
blah# lsmod
Module: #pages: Used by:
drv_hello 1
`drv_hello' is the name of the module, it uses one page (4k) of memory, and no other kernel modules
depend on it at the moment. To remove this module, use `rmmod drv_hello'. Note that rmmod wants a
module name, not a filename; you get this from lsmod's listing. The other module utilities' purposes are
documented in their manual pages.
This can be especially handy with filesystems. You may not use the minix or msdos filesystems frequently.
For example, if I encountered an msdos (shudder) floppy, I would insmod
/usr/src/linux/modules/msdos.o, and then rmmod msdos when finished. This procedure saves
about 50k of RAM in the kernel during normal operation. A small note is in order for the minix filesystem:
you should always configure it directly into the kernel for use in ``rescue'' disks.
You can compile just a single module file (say like foo.o) and install it. For this simply edit the Makefile and
change the SUBDIRS to add only those directories you are interested.
For an example, if I am interested in installing only fs/autofs module, then I do the following :
cd /usr/src/linux
cp Makefile Makefile.my
vi Makefile.my
# And comment out the line having 'SUBDIRS' and add the
# directory you are interested, for example like fs/autofs as below :
#SUBDIRS =kernel drivers mm fs net ipc lib abi crypto
SUBDIRS =fs/autofs
# Save the file Makefile.my and give −
make −f Makefile.my modules
# This will create module autofs.o
Learn more about Makefile and make. See the manual for GNU make at
• http://www.gnu.org/manual/make.
• University of Utah Makefile http://www.math.utah.edu/docs/info/make−stds_toc.html
• University of Hawaii Makefile http://www.eng.hawaii.edu/Tutor/Make
• In Linux − man make
• In Linux − info make
Get familiar with the Makefile which makes the modules. The Makefile has module line like
The patsubst function has the syntax $(patsubst pattern,replacement,text). It uses the percent symbol (%) the
same way pattern rules do − as a string which matches in both the pattern and the replacement text. It searches
the text for whitespace−separated words that match the pattern and substitutes the replacement for them.
This makefile includes shell functions as well as standard make functions. The syntax for a shell function is
$(shell command). This returns the output of the shell function (stripping new lines).
There is certainly more to the kernel's operation than this, but these basic functions are the most important to
know.
If this troubles you, and you happen to have a faster machine around to compile on, you can build on the fast
machines (assuming you give it the right parameters, that your ulilities are up−to−date, and so on), and then
transfer the kernel image to the slower machine.
So, continuing with the example above, let's suppose that you have `patch46.gz' in /usr/src. cd to
/usr/src and do a `zcat patch46.gz | patch −p0' (or `patch −p0 < patch46' if the patch
isn't compressed). You'll see things whizz by (or flutter by, if your system is that slow) telling you that it is
trying to apply hunks, and whether it succeeds or not. Usually, this action goes by too quickly for you to read,
and you're not too sure whether it worked or not, so you might want to use the −s flag to patch, which tells
patch to only report error messages (you don't get as much of the ``hey, my computer is actually doing
something for a change!'' feeling, but you may prefer this..). To look for parts which might not have gone
smoothly, cd to /usr/src/linux and look for files with a .rej extension. Some versions of patch
(older versions which may have been compiled with on an inferior filesystem) leave the rejects with a #
extension. You can use `find' to look for you;
prints all files who live in the current directory or any subdirectories with a .rej extension to the standard
output.
If everything went right, do a `make clean', `config', and `dep' as described in sections 3 and 4.
There are quite a few options to the patch command. As mentioned above, patch −s will suppress all
messages except the errors. If you keep your kernel source in some other place than /usr/src/linux,
patch −p1 (in that directory) will patch things cleanly. Other patch options are well−documented in the
manual page.
The most frequent problem that used to arise was when a patch modified a file called `config.in' and it
didn't look quite right, because you changed the options to suit your machine. This has been taken care of, but
one still might encounter it with an older release. To fix it, look at the config.in.rej file, and see what
remains of the original patch. The changes will typically be marked with `+' and `−' at the beginning of the
line. Look at the lines surrounding it, and remember if they were set to `y' or `n'. Now, edit config.in, and
change `y' to `n' and `n' to `y' when appropriate. Do a
and if it reports that it succeeded (no fails), then you can continue on with a configuration and compilation.
The config.in.rej file will remain, but you can get delete it.
If you encounter further problems, you might have installed a patch out of order. If patch says `previously
applied patch detected: Assume −R?', you are probably trying to apply a patch which is below
your current version number; if you answer `y', it will attempt to degrade your source, and will most likely
fail; thus, you will need to get a whole new source tree (which might not have been such a bad idea in the first
place).
To back out (unapply) a patch, use `patch −R' on the original patch.
The best thing to do when patches really turn out wrong is to start over again with a clean, out−of−the−box
source tree (for example, from one of the linux−x.y.z.tar.gz files), and start again.
will take care of it for you. Versions of patch which use # for rejects use a tilde instead of .orig.
There are better ways to get rid of the .orig files, which depend on GNU xargs:
How common are the patches not in the standard distribution? You will probably hear of them. I used to use
the noblink patch for my virtual consoles because I hate blinking cursors (This patch is (or at least was)
frequently updated for new kernel releases.). With most newer device drivers being developed as loadable
modules, though, the frequecy of ``nonstandard'' patches is decreasing significantly.
image = /usr/src/linux/arch/i386/boot/bzImage
label = new_kernel
to the end of your LILO configuration file, you can choose to run a newly compiled kernel without touching
your old /vmlinuz (after running lilo, of course). The easiest way to tell LILO to boot a new kernel is to
press the shift key at bootup time (when it says LILO on the screen, and nothing else), which gives you a
prompt. At this point, you can enter `new_kernel' to boot the new kernel.
If you wish to keep several different kernel source trees on your system at the same time (this can take up a lot
of disk space; be careful), the most common way is to name them /usr/src/linux−x.y.z, where
x.y.z is the kernel version. You can then ``select'' a source tree with a symbolic link; for example, `ln −sf
linux−1.2.2 /usr/src/linux' would make the 1.2.2 tree current. Before creating a symbolic link
like this, make certain that the last argument to ln is not a real directory (old symbolic links are fine); the
result will not be what you expect.
The vmlinux is the uncompressed built kernel, vmlinuz is the compressed one, that has been made bootable.
(Note both names vmlinux and vmlinuz look same except for last letter z). Generally, you don't need to worry
about vmlinux, it is just an intermediate step.
The kernel usually makes a bzImage, and stores it in arch/i386/boot, and it is up to the user to copy it to /boot
and configure GRUB or LILO.
the .b files are "bootloader" files. they are part of the dance required to get a kernel into memory to begin
with. You should NOT touch them.
The 'message' file contains the message your bootloader will display, prompting you to choose an OS. So DO
NOT touch it.
9.4 initrd.img
See the Appendix A at Description of initrd.img file.
9.5 bzImage
The bzImage is the compressed kernel image created with command 'make bzImage' during kernel compile.
9.6 module−info
This is created by utils/modlist.
9.7 config
Everytime you compile and install the kernel image in /boot, you should also copy the corresponding config
file to /boot area, for documentation and future reference. Do NOT touch or edit these files!!
ls −l /boot/config−*
−rw−r−−r−− 1 root root 42111 Sep 4 2002 /boot/config−2.4.18−14
−rw−r−−r−− 1 root root 42328 Jan 26 01:29 /boot/config−2.4.18−19.8.0
−rw−r−−r−− 1 root root 51426 Jan 25 22:21 /boot/config−2.4.18−19.8.0BOOT
−rw−r−−r−− 1 root root 52328 Jan 28 03:22 /boot/config−2.4.18−19.8.0−26mar2003
9.8 System.map
System.map is a "phone directory" list of function in a particular build of a kernel. It is typically a symlink to
the System.map of the currently running kernel. If you use the wrong (or no) System.map, debugging crashes
is harder, but has no other effects. Without System.map, you may face minor annoyance messages.
ls −ld /boot/System.map*
lrwxrwxrwx 1 root root 30 Jan 26 19:26 /boot/System.map −> System.map−2.4.18−19.
−rw−r−−r−− 1 root root 501166 Sep 4 2002 /boot/System.map−2.4.18−14
−rw−r−−r−− 1 root root 510786 Jan 26 01:29 /boot/System.map−2.4.18−19.8.0
How The Kernel Symbol Table Is Created ? System.map is produced by 'nm vmlinux' and irrelevant or
uninteresting symbols are grepped out, When you compile the kernel, this file 'System.map' is created at
/usr/src/linux/System.map. Something like below:
From http://www.dirac.org/linux/systemmap.html
System.map
There seems to be a dearth of information about the System.map file. It's really nothing mysterious, and in the
scheme of things, it's really not that important. But a lack of documentation makes it shady. It's like an
earlobe; we all have one, but nobody really knows why. This is a little web page I cooked up that explains the
why.
Note, I'm not out to be 100% correct. For instance, it's possible for a system to not have /proc filesystem
support, but most systems do. I'm going to assume you "go with the flow" and have a fairly typical system.
Some of the stuff on oopses comes from Alessandro Rubini's "Linux Device Drivers" which is where I
learned most of what I know about kernel programming.
Humans, on the other hand, do not appreciate names like c0343f20. We prefer to use something like size_t
BytesRead. Normally, this doesn't present much of a problem. The kernel is mainly written in C, so the
compiler/linker allows us to use symbol names when we code and allows the kernel to use addresses when it
runs. Everyone is happy.
There are situations, however, where we need to know the address of a symbol (or the symbol for an address).
This is done by a symbol table, and is very similar to how gdb can give you the function name from a address
(or an address from a function name). A symbol table is a listing of all symbols along with their address. Here
is an example of a symbol table:
c03441a0 B dmi_broken
System.map 18
The Linux Kernel HOWTO
c03441a4 B is_sony_vaio_laptop
c03441c0 b dmi_ident
c0344200 b pci_bios_present
c0344204 b pirq_table
c0344208 b pirq_router
c034420c b pirq_router_dev
c0344220 b ascii_buffer
c0344224 b ascii_buf_bytes
You can see that the variable named dmi_broken is at the kernel address c03441a0.
1. /proc/ksyms
2. System.map
Every time you compile a new kernel, the addresses of various symbol names are bound to change.
/proc/ksyms is a "proc file" and is created on the fly when a kernel boots up. Actually, it's not really a file; it's
simply a representation of kernel data which is given the illusion of being a disk file. If you don't believe me,
try finding the filesize of /proc/ksyms. Therefore, it will always be correct for the kernel that is currently
running..
However, System.map is an actual file on your filesystem. When you compile a new kernel, your old
System.map has wrong symbol information. A new System.map is generated with each kernel compile and
you need to replace the old copy with your new copy.
What Is An Oops?
What is the most common bug in your homebrewed programs? The segfault. Good ol' signal 11.
What is the most common bug in the Linux kernel? The segfault. Except here, the notion of a segfault is much
more complicated and can be, as you can imagine, much more serious. When the kernel dereferences an
invalid pointer, it's not called a segfault −− it's called an "oops". An oops indicates a kernel bug and should
always be reported and fixed.
Note that an oops is not the same thing as a segfault. Your program cannot recover from a segfault. The kernel
doesn't necessarily have to be in an unstable state when an oops occurs. The Linux kernel is very robust; the
oops may just kill the current process and leave the rest of the kernel in a good, solid state.
An oops is not a kernel panic. In a panic, the kernel cannot continue; the system grinds to a halt and must be
restarted. An oops may cause a panic if a vital part of the system is destroyed. An oops in a device driver, for
example, will almost never cause a panic.
When an oops occurs, the system will print out information that is relevent to debugging the problem, like the
contents of all the CPU registers, and the location of page descriptor tables. In particular, the contents of the
EIP: 0010:[<00000000>]
Call Trace: [<c010b860>]
To help us use this cryptic oops output, Linux uses a daemon called klogd, the kernel logging daemon. klogd
intercepts kernel oopses and logs them with syslogd, changing some of the useless information like c010b860
with information that humans can use. In other words, klogd is a kernel message logger which can perform
name−address resolution. Once klogd tranforms the kernel message, it uses whatever logger is in place to log
system wide messages, usually syslogd.
To perform name−address resolution, klogd uses System.map. Now you know what an oops has to do with
System.map.
Fine print: There are actually two types of address resolution are performed by klogd.
System.map and is therefore not relevant to this discussion, but I'll describe it briefly anyhow.
Suppose you load a kernel module which generates an oops. An oops message is generated, and klogd
intercepts it. It is found that the oops occured at d00cf810. Since this address belongs to a dynamically loaded
module, it has no entry in the System.map file. klogd will search for it, find nothing, and conclude that a
loadable module must have generated the oops. klogd then queries the kernel for symbols that were exported
by loadable modules. Even if the module author didn't export his symbols, at the very least, klogd will know
what module generated the oops, which is better than knowing nothing about the oops at all.
There's other software that uses System.map, and I'll get into that shortly.
1. /boot/System.map
2. /System.map
3. /usr/src/linux/System.map
System.map also has versioning information, and klogd intelligently searches for the correct map file. For
• Somewhere during the 2.5.x series, the Linux kernel started to untar into linux−version, rather than
just linux (show of hands −− how many people have been waiting for this to happen?). I don't know if
klogd has been modified to search in /usr/src/linux−version/System.map yet. TODO: Look at the
klogd srouce. If someone beats me to it, please email me and let me know if klogd has been modified
to look in the new directory name for the linux source code.
• The man page doesn't tell the whole the story. Look at this:
Apparently, not only does klogd look for the correct version of the map in the 3 klogd search directories, but
klogd also knows to look for the name "System.map" followed by "−kernelversion", like System.map−2.4.18.
This is undocumented feature of klogd.
A few drivers will need System.map to resolve symbols (since they're linked against the kernel headers
instead of, say, glibc). They will not work correctly without the System.map created for the particular kernel
you're currently running. This is NOT the same thing as a module not loading because of a kernel version
mismatch. That has to do with the kernel version, not the kernel symbol table which changes between kernels
of the same version!
and ps :
and many other pieces of software like dosemu require a correct System.map.
System.map does not match actual kernel Not a fatal error, but can be annoying to see everytime you do a ps
ax. Some software, like dosemu, may not work correctly (although I don't know of anything off the top of my
head). Lastly, your klogd or ksymoops output will not be reliable in case of a kernel oops.
• /boot/vmlinuz−2.2.14
• /boot/vmlinuz−2.2.13
Then just rename your map files according to the kernel version and put them in /boot, like:
/boot/System.map−2.2.14
/boot/System.map−2.2.13
Now what if you have two copies of the same kernel? Like:
• /boot/vmlinuz−2.2.14
• /boot/vmlinuz−2.2.14.nosound
The best answer would be if all software looked for the following files:
/boot/System.map−2.2.14
/boot/System.map−2.2.14.nosound
System.map−2.2.14
System.map−2.2.14.sound
ln −s System.map−2.2.14.sound System.map # Here System.map −> System.map−2.2.14.sound
This document is published in 14 different formats namely − DVI, Postscript, Latex, Adobe Acrobat PDF,
LyX, GNU−info, HTML, RTF(Rich Text Format), Plain−text, Unix man pages, single HTML file, SGML
(Linuxdoc format), SGML (Docbook format), MS WinHelp format.
• http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f
or ALT+f within the web−browser.
You can also find this document at the following mirrors sites −
• You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML
formats from − ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other−formats/ and
http://www.linuxdoc.org/docs.html#howto
• Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and
http://www.linuxdoc.org/docs.html#howto
• Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto
Single HTML file can be created with command (see man sgml2html) − sgml2html −split 0
xxxxhowto.sgml
• Translations to other languages like French, German, Spanish, Chinese, Japanese are in
ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any
help from you to translate to other languages is welcome.
The document is written using a tool called "SGML−Tools" which can be got from −
http://www.sgmltools.org Compiling the source you will get the following commands like
Or you can use Ghostscript command ps2pdf. ps2pdf is a work−alike for nearly all the functionality of
Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files.
ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a
special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the
makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.
The ld2db.sh is not 100% clean, you will get lot of errors when you run
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may
need to put closing tag < /Para> for each < Listitem>
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp
files.
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use
'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert
menu press 'x'.
You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in
ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located
through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly
than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you
• Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost
gv howto.ps
ghostscript howto.ps
You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron
Web browser or any of the 10 other web browsers.
You can read the latex, LyX output using LyX a X−Windows front end to latex.
You can avoid this if you build your scsi drivers right into the kernel, instead of into modules. (Many persons
recommend this).
su − root
ls −l initrd−2.4.18−19.8.0custom.img
−rw−r−−r−− 1 root root 127314 Mar 19 21:54 initrd−2.4.18−19.8.0custom.img
cp ./initrd−2.4.18−19.8.0custom.img /boot
See the following sections for the manual method of creating an initrd image.
Some of the following descriptions may be difficult to understand without knowledge of kernel modules
explained in Chapter 42. You may want to come back to it later.
Consider a system with zero IDE disks and one SCSI disk containing a LINUX installation. There are BIOS
interrupts to read the SCSI disk, just as there were for the IDE, so LILO can happily access a kernel image
somewhere inside the SCSI partition. However, the kernel is going to be lost without a kernel module [See
Chapter 42. The kernel doesn't support every possible kind of hardware out there all by itself. It is actually
divided into a main part (the kernel image discussed in this chapter) and hundreds of modules (loadable parts
that reside in /lib/modules/) that support the many type of SCSI, network, sound etc., peripheral devices.] that
understands the particular SCSI driver. So although the kernel can load and execute, it won't be able to mount
its root file system without loading a SCSI module first. But the module itself resides in the root file system in
/lib/modules/. This is a tricky situation to solve and is done in one of two ways: either (a) using a kernel with
preenabled SCSI support or (b) using what is known as an initrd preliminary root file system image.
The first method is what I recommend. It's a straightforward (though time−consuming) procedure to create a
kernel with SCSI support for your SCSI card built−in (and not in a separate module). Built−in SCSI and
network drivers will also autodetect cards most of the time, allowing immediate access to the device−−they
will work without being given any options [Discussed in Chapter 42.] and, most importantly, without your
having to read up on how to configure them. This setup is known as compiled−in support for a hardware
driver (as opposed to module support for the driver). The resulting kernel image will be larger by an amount
equal to the size of module. Chapter 42 discusses such kernel compiles.
The second method is faster but trickier. LINUX supports what is known as an initrd image ( initial rAM disk
image). This is a small, +1.5 megabyte file system that is loaded by LILO and mounted by the kernel instead
of the real file system. The kernel mounts this file system as a RAM disk, executes the file /linuxrc, and then
only mounts the real file system.
Start by creating a small file system. Make a directory /initrd and copy the following files into it.
On my system, the file initrd/bin/insmod is the statically linked [meaning it does not require shared libraries.]
version copied from /sbin/insmod.static−−a member of the modutils−2.3.13 package. initrd/bin/sash is a
statically linked shell from the sash−3.4 package. You can recompile insmod from source if you don't have a
statically linked version. Alternatively, copy the needed DLLs from /lib/ to initrd/lib/. (You can get the list of
required DLLs by running ldd /sbin/insmod. Don't forget to also copy symlinks and run strip −s {lib} to
reduce the size of the DLLs.)
Now copy into the initrd/lib/ directory the SCSI modules you require. For example, if we have an Adaptec
AIC−7850 SCSI adapter, we would require the aic7xxx.o module from /lib/modules/{version}/scsi/aic7xxx.o.
Then, place it in the initrd/lib/ directory.
The file initrd/linuxrc should contain a script to load all the modules needed for the kernel to access the SCSI
partition. In this case, just the aic7xxx module [ insmod can take options such as the IRQ and IO−port for the
device. See Chapter 42.]:
#!/bin/sash
aliasall
Now double−check all your permissions and then chroot to the file system for testing.
Your lilo.conf file can be changed slightly to force use of an initrd file system. Simply add the initrd option.
For example:
boot=/dev/sda
prompt
timeout = 50
compact
vga = extended
linear
image = /boot/vmlinuz−2.2.17
initrd = /boot/initrd−2.2.17
label = linux
root = /dev/sda1
read−only
Notice the use of the linear option. This is a BIOS trick that you can read about in lilo(5). It is often necessary
but can make SCSI disks nonportable to different BIOSs (meaning that you will have to rerun lilo if you move
the disk to a different computer).
Always give a date extension to the filename, because it tells you when you built the kernel, as shown below:
Now give −
bash# lilo
bash# lilo −q
You must re−run lilo even if the entry 'myker' exists, everytime you create a new bzImage.
Given below is a sample /etc/lilo.conf file. You should follow the naming conventions like ker2217 (for
kernel 2.2.17), ker2214 (for kernel 2.2.14). You can have many kernel images on the same /boot system. On
my machine I have something like:
boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
default=firewall
image=/boot/vmlinuz−2.2.14−5.0
label=ker2214
read−only
root=/dev/hda9
image=/boot/vmlinuz−2.2.17−14
label=ker2217
read−only
root=/dev/hda9
#image=/usr/src/linux/arch/i386/boot/bzImage
# label=myker
# root=/dev/hda7
# read−only
image=/boot/bzImage.myker.11feb2001
label=myker11feb
root=/dev/hda9
read−only
image=/boot/bzImage.myker.01jan2001
label=myker01jan
root=/dev/hda9
read−only
image=/boot/bzImage.myker−firewall.16mar2001
label=firewall
root=/dev/hda9
read−only
• http://www.tldp.org/HOWTO/Linux+Win9x+Grub−HOWTO/intro.html
• GNU GRUB http://www.gnu.org/software/grub
• Redhat Manual.
• Multiboot−with−GRUB minihowto
• Grub Manual
Edit the file /etc/grub.conf to make entries for the new kernel. See the sample file below:
splashimage=(hd0,8)/boot/grub/splash.xpm.gz
title Windows 98
hide (hd0,0)
hide (hd0,1)
unhide (hd0,2)
rootnoverify (hd0,2)
chainloader +1
makeactive
• Please see the video card manual which is usually shipped with the PC. You should look for a
"Technical Specifications" page.
• Please see the monitor's manual and look for a "Technical Specifications" page.
If you are using latest version of Linux (2.4 or later) and inside KDE/GNOME desktop click on
Start−>"System Settings"−>Display.
You can configure the Video card and monitor by using these commands:
bash$ su − root
bash# man Xconfigurator
bash# /usr/bin/X11/Xconfigurator −−help
bash# /usr/bin/X11/Xconfigurator
bash# /usr/bin/X11/Xconfigurator −−expert
See also:
bash# man xf86config
bash# /usr/bin/X11/xf86config
If your card is not detected automatically, then you can use the −−expert option and select the "Unlisted card".
If your monitor is not listed then select the generic monitor type SVGA 1024x768.
If you are using latest version of Linux (2.4 or later) and inside KDE/GNOME desktop click on
Start−>"System Settings"−>Soundcard Detection.
bash$ su − root
bash# man sndconfig
bash# /usr/sbin/sndconfig
Then start X−window 'KDE desktop' with 'startx' command. Click on 'K
Start−>ControlCenter−>SoundServer−>General−>Test Sound'. This should play the test sound. Then click on
'K Start−>MultiMedia−>SoundMixer−>SoundVolumeSlider' and adjust the sound volume.
Network card configuration: If you are using latest version of Linux (2.4 or later) and inside KDE/GNOME
desktop click on Start−>"System Settings"−>Network.
• Use /sbin/linuxconf
• Or use KDE control panel
• Refer to HOWTO docs on 'Networking' at http://www.linuxdoc.org
Configure Firewall and IP Masquerading : For Linux kernel version 2.4 and above, the firewall and IP
Masquerading is implemented by NetFilter package. Hence in kernel config you should enable Netfilter and
run the Firewall/IPMasq script. Download the scripts from Firewall−IPMasq scripts , main page of Netfilter is
at http://netfilter.samba.org. Related materials at firewalling−matures and Netfilter−FAQ.
For kernel version below 2.4 you should install the firewall rpms from rpmfind.net or firewall.src.rpm.
Configuration of other devices: Refer to HOWTO docs relating to your devices at http://www.linuxdoc.org
Solution: You did not create initrd image file. See the Appendix A at Creating initrd.img file. Also, you must
do 'make modules' and 'make modules_install' in addition to creating the initrd image file.
Reason: Probably you did not set the BIOS to pick up the proper Primary Master IDE and Secondary Slave
IDE hard disk partition.
Solution: Power on the machine and press DEL key to do setup of the BIOS (Basic Input Output system).
Select the IDE settings and set proper primary hard disk partition and slave drives. When the system boots it
looks for the primary IDE hard disk and the Master Boot Record partition. It reads the MBR and starts loading
the Linux Kernel from the hard disk partition.
The problem is that you did not set the "root=" parameter properly in the /etc/lilo.conf. In my case, I used
root=/dev/hda1 which is having the root partition "/". You must properly point the root device in your
lilo.conf, it can be like /dev/hdb2 or /dev/hda7.
The kernel looks for the init command which is located in /sbin/init. And /sbin directory lives on the root
partition. For details see −
See the Appendix C − GRUB details and sample grub.conf file and see the Appendix B − Sample lilo.conf.
If this problem persists, then try menuconfig instead of xconfig. Sometimes GUI version xconfig causes some
problems:
bash$ su − root
bash# man depmod
bash# depmod
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/linear.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/multipath.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/raid0.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/raid1.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/raid5.o
Reason: You did not make modules and install the modules after building the new kernel with "make
bzImage".
Solution: After you build the new kernel, you must do:
bash$ su − root
bash# cd /usr/src/linux
bash# make dep
bash# make clean
bash# make mrproper
bash# nohup make bzImage &
bash# tail −f nohup.out (.... to monitor the progress)
bash# make modules
bash# make modules_install
The step given below may not be required but is needed ONLY FOR EMERGENCIES where your
/lib/modules files are damaged. If you already have the /lib/modules directory and in case you want replace
them use the −−force to replace the package and select appropriate cpu architecture.
For new versions of linux redhat linux 6.0 and later, the kernel modules are included with kernel−2.2*.rpm.
Install the loadable modules and the kernel with
This is only for old versions of redhat linux 5.2 and before. Boot new kernel and install the loadable modules
15.6 Kernel Does Not Load Module − "Unresolved symbols" Error Messages 34
The Linux Kernel HOWTO
You can find out how much memory the kernel is using by taking the total amount of memory in your
machine and subtracting from it the amount of ``total mem'' in /proc/meminfo or the output of the
command `free'.
Then there are the names. Linux 2.2 names the printer devices differently than previous releases. The upshot
of this is that if you had an lp1 under your old kernel, it's probably an lp0 under your new one. Use
`dmesg' or look through the logs in /var/log to find out.
It tends to disturb people when it's suggested that their hardware has problems. Well, I'm not making this up.
There is an FAQ for it −− it's at http://www.bitwizard.nl/sig11.
In the following example, / is /dev/hda1, and the filesystem which holds /usr/src/linux is
/dev/hda3, normally mounted at /usr. Both are second extended filesystems. The working kernel image
in /usr/src/linux/arch/i386/boot is called bzImage.
The idea is that if there is a functioning bzImage, it is possible to use that for the new floppy. Another
alternative, which may or may not work better (it depends on the particular method in which you messed up
your system) is discussed after the example.
First, boot from a boot/root disk combo or rescue disk, and mount the filesystem which contains the working
kernel image:
mkdir /mnt
mount −t ext2 /dev/hda3 /mnt
If mkdir tells you that the directory already exists, just ignore it. Now, cd to the place where the working
kernel image was. Note that
Place a formatted disk in drive ``A:'' (not your boot or root disk!), dump the image to the disk, and configure it
for your root filesystem:
cd /mnt/src/linux/arch/i386/boot
dd if=bzImage of=/dev/fd0
rdev /dev/fd0 /dev/hda1
cd /
You should now be able to reboot your system as normal from this floppy. Don't forget to run lilo (or
whatever it was that you did wrong) after the reboot!
As mentioned above, there is another common alternative. If you happened to have a working kernel image in
/ (/vmlinuz for example), you can use that for a boot disk. Supposing all of the above conditions, and that
my kernel image is /vmlinuz, just make these alterations to the example above: change /dev/hda3 to
/dev/hda1 (the / filesystem), /mnt/src/linux to /mnt, and if=bzImage to if=vmlinuz. The
note explaining how to derive /mnt/src/linux may be ignored.
Using LILO with big drives (more than 1024 cylinders) can cause problems. See the LILO mini−HOWTO or
documentation for help on that.
If your CD−ROM drive is the only device on a particular IDE interface, it must be jumpered as ``master'' or
``single.'' Supposedly, this is the most common error.
Creative Labs (for one) has put IDE interfaces on their sound cards now. However, this leads to the interesting
problem that while some people only have one interface to being with, many have two IDE interfaces built−in
to their motherboards (at IRQ15, usually), so a common practice is to make the soundblaster interface a third
IDE port (IRQ11, or so I'm told).
This causes problems with older Linux versions like 1.3 and below. in that versions Linux don't support a
third IDE interface. To get around this, you have a few choices.
If you have a second IDE port already, chances are that you are not using it or it doesn't already have two
devices on it. Take the ATAPI drive off the sound card and put it on the second interface. You can then
disable the sound card's interface, which saves an IRQ anyway.
If you don't have a second interface, jumper the sound card's interface (not the sound card's sound part) as
IRQ15, the second interface. It should work.
#include <linux/xyzzy.h>
Normally, there is a link called linux in /usr/include to the include/linux directory of your
kernel source (/usr/src/linux/include/linux in the typical system). If this link is not there, or
points to the wrong place, most things will not compile at all. If you decided that the kernel source was taking
too much room on the disk and deleted it, this will obviously be a problem. Another way it might go wrong is
with file permissions; if your root has a umask which doesn't allow other users to see its files by default, and
you extracted the kernel source without the p (preserve filemodes) option, those users also won't be able to
use the C compiler. Although you could use the chmod command to fix this, it is probably easier to
re−extract the include files. You can do this the same way you did the whole source at the beginning, only
with an additional argument:
Note: ``make config'' will recreate the /usr/src/linux link if it isn't there.