Add a System Call in Linux

OS Project 1 Summer 2013

GROUP 2
Syed Shabbir Haider (110594)
Ahad Tanvir (110606)
Kaywan Rauf (110935)

Add a System Call in Linux

OS Project 1 Summer 2013

About System Calls:

System calls are the routines written inside the Linux kernel. System calls provide a general interface between a user application
and the kernel. Whenever a user application requires OS services, it cannot directly invoke the kernel for that particular
service. For this purpose, system calls are invoked via API. Basic component of a Linux operating system is kernel. Its free and
open source, hence Linux kernel can be modified and customized depending upon different developing and designing needs. In
this project we will add a system call in the Linux kernel. This will be a simple Hello world system call, which is pretty
useless, but its main purpose is to get to know the basics of adding a custom system call in Linux kernel.

Add a System Call in Linux | [Pick the date]

Following are the steps for adding a system call in the Linux kernel:

Get the source of your kernel

You can use any Linux distribution, and any version of the kernel for this purpose. If you are interested in downloading the
kernel version of your Linux distribution, first check the version of your kernel by typing the following command in the
terminal
uname r
this will display you the kernel version of your Linux distribution, Im using Ubuntu Linux 12.04, and it displays me the kernel
version 3.2.46

sudo apt-get source kernel

Add a System Call in Linux | [Pick the date]

to download the source for the particular version of your kernel distribution type this command in the terminal

The source of that particular kernel will begin to download, it will take some time depending upon the speed of your internet
connections.
You can also go to www.kernel.org to download the kernel of particular version or the latest stable kernel from the kernel
archives. As I mentioned if youre not interested in working on the particular kernel version of your Linux distribution, you can
download the latest stable kernel or any version that you wish. The most recent distributions of Linux such as Ubuntu,
Backtrack etc. have a big kernel, so if you are using any recent version of Linux then it is advisable to download a smaller
version of kernel.
Tip: If you want to use a kernel other than your Linux distribution, try to choose a kernel which is smaller in size, because
bigger the kernel more it will take to compile it.

Add a System Call in Linux | [Pick the date]

Once you downloaded your kernel source, next step is to extract it. For this purpose go to your home folder or root directory,
you would see a zipped file named linux-source-x.y.z.tar.bz (where x, y , z indicates the version such as linux-source3.2.0),
Right click on the file and then click Extract here. I have extracted this source in the home folder(root directory) for my
convenience so that I can easily access it, you may extract it anywhere.

After you extract, you would be able to see a folder named linux-source-x.y.z (where x, y , z indicates the version such as
linux-source3.2.0),
Now the source of the kernel is available to you, you would be able to browse and view the files and make changes.
You can also extract this zipped file by typing the following command in the terminal,
tar xvzf linux-source-x.y.z.tar.bz(where x, y , z indicates the version such as linux-source3.2.0),

In the folder linux-source-x.y.z (where x, y , z indicates the version such as linux-source3.2.0), go the file
arch/x86/kernel/syscall_table_32.S, and add the following line

Add a System Call in Linux | [Pick the date]

Add the system call to the system call table:

Add a System Call in Linux | [Pick the date]

.long sys_hello

syscall_table_32 is the system call table which consists of assemble instructions in which record of all the system calls in
the system is kept. You can assume it as an array having consecutive locations, on each location theres an entry of the system
call. We have the entry of our system call at the end of this table, if we add another system call it will be added in the end as

well and so on(just like filling the consecutive locations of an array). sys_call_table is the base pointer and points to the start of
the array, that is, to the zero entry of an array in other words. If a user-space program invokes the sendmssg system call, the
number passed is 345. The dispatcher routine adds this number to the sys_call_table base and arrives at the three forty fifth
entry that holds the address of sys_sendmmsg.
Define Macros associated with the system call:
In the linux-source-x.y.z folder go the file arch/x86/include/asm/unistd_32.h, you would notice that a number is been
defined with each system call, that number is called macro. Macro is a constant associated with each system call which
indicated the location of the system call in the system call table or in other words it simply defines the system call number.
Add the following line
#define __NR_hello 349

#define NR_syscalls 350

NR_syscalls is another macro constant, which saves the total number of system calls present in the system. In my system there
were total 350 system call, by adding one it the total number of system call becomes 351( NR_syscalls indicates 350 because it
begins counting from 0, so from 0-350, total of 351 system calls in the system).
Caution:
Number of system call might vary from version to version in the Linux kernel, so be careful when you are defining your system
call and the macro associated with it and incrementing the NR_syscalls constant.

Add a System Call in Linux | [Pick the date]

This is the definition of our system call, and also increment the value of the NR_SYSCALLS by 1, since we are adding one
system call

Add a System Call in Linux | [Pick the date]

Now go the file arch/x86/include/asm/unistd_64.h and add the definition of you system call,
#define __NR_hello 312
__SYSCALL(__NR_hello, sys_hello)

Now to the file include/linux/syscalls.h, add the prototype of the system call.
asmlinkage long sys_hello(void);

Add a System Call in Linux | [Pick the date]

(Look at the arrows in the figures)

Add a System Call in Linux | [Pick the date]

Now in the folder linux-source-x.y.z (where x, y , z indicates the version such as linux-source3.2.0), create a new folder and
name that folder hello. In that folder create a file named hello.c, and add the following code to that file.

#include <linux/kernel.h>
asmlinkage long sys_hello(void)
{
printk("Hello world\n");
return 0;
}

obj-y := hello.o
This will create the object file of the system call to be execute it. Kernel will use this file to build the system call. Now go the
folder linux-source-x.y.z (where x, y , z indicates the version such as linux-source3.2.0), and look for a make file. Open that
file and edit the following line to the text shown in the image below
core-y
:= kernel/ mm/ fs/ ipc/ security/ crypto/ block/

Add a System Call in Linux | [Pick the date]

This is our system call which kernel will execute. asmlinkage is itself is afunction which is must for the kernels otherwise the
system call wont work, and kernel wont be able to distinguish that either it is a user program or a kernel program. You can
see printk statement in the body of that function. printk is just like printf or cout which displays the output, but printf and
cout display output on the console window, unlike these two statements printk will write the output of the system call to the
kernel log. On execution , the system call returns a long integer, indicating the status of the system call, for example if system
call returned 1, then that means that system call was terminated prematurely, or if it returns -1 it means system call does not
exist( kernel was unable to locate it in the system call table), if it returns 0, that means it executed normally as it should.
Now in the same directory (hello folder), create another file, and name it Makefile, and add the following text to it.

10

Add a System Call in Linux | [Pick the date]

11

This will make sure that our hello system call is in hello directory and it is is also compiled and build when we will compile
our kernel.

Now we have added the system call, we need to compile our kernel and test this system call.
Compiling the kernel:
For compiling the kernel following three packages must be installed on your system.
gcc compiler (latest version)
ncurses development kit
system packages(updates)
For installing gcc compiler type the following command in the terminal
sudo apt-get install gcc

sudo apt-get install libncurces5-dev

Now run the updates by typing the following command to the terminal
sudo apt-get update && sudo apt-get upgrade
Now we are ready to compile the kernel, for compiling the kernel type the following to the terminal
sudo make
That will take some time depending upon the size of your kernel. Normally it takes 45-50 minutes to compile a smaller or
medium kernel but in my case it took more than one and a half an hour!

Add a System Call in Linux | [Pick the date]

For installing ncurses development packages type the following in the terminal

12

Add a System Call in Linux | [Pick the date]

Something like that will be displayed on the screen while compiling the kernel

13

Tip:
If you have multi-core, multi-processor machine then you can reduce the compile time of the kernel by making use of cores
available. For compiling on different cores type the following to the terminal

sudo make jX (where X is the number of cores available)

This will reduce the compile time by the factor (compile time/number of cores).
(Note: You might not be able to interact with your programs or applications during the execution of this command, because this
command make a high use of the processor and its cores, so your interactive programs might become slow or not responding)
After successful compilation of your kernel, next step is to install the newly modified kernel to your system.
For installation type the following command in the terminal.
sudo make install_modules install
This will install the modified kernel and all the necessary modules. One you have installed your new kernel, reboot your system
with this new kernel.

#include
#include
#include
#include

<stdio.h>
<linux/kernel.h>
<sys/syscall.h>
<unistd.h>

#define __NR_hello 312 //349 if you are running a 32bit kernel and following my
tutorial
long hello_syscall(void)
{
return syscall(__NR_hello);
}

Add a System Call in Linux | [Pick the date]

After rebooting run the following test program to check either our system call is working fine or not.

14

int main(int argc, char *argv[])

{
long int a = hello_syscall();
printf("System call returned %ld\n", a);
return 0;
}

The output of the program should be:

Add a System Call in Linux | [Pick the date]

System call returned 0

15

As shown in the image below:

Add a System Call in Linux | [Pick the date]

As I mentioned earlier that you would not be able to see the output of the system call Hello World on the console
window, the output of that system call is written in the kernel log.
To view the kernel log type the following command in the terminal:

16

dmesg

Add a System Call in Linux | [Pick the date]

Output will be displayed as shown in the image.

17

Add a System Call in Linux | [Pick the date]

Most probably, the output of the system call will be written at the end of the log. In the image you would notice that Hello
world appeared two times. This is because I ran my test.c program twice or in other words I invoked the hello system call two
times, that means every time you invoke the system call, its output is going to be written on the kernel log. Thats why Hello
word appeared twice on the log can be seen.

18