Chapter 2: Threads

In a traditional operating system, each process has an address space and a single thread of control. Nevertheless, there
are frequently situations in which it is desirable to have multiple threads of control in the same address space running
in quasi-parallel, as though they were separate processes (except for the shared address space). This chapter is
intended to provide more details about these light weight processes called threads.

I) Why are threads useful?

To figure out the importance of threads, let us consider the following examples:

Example 1: Let’s consider a word processor. Such a process comprises multiple activities, such as typing, formatting,
typing errors check, automatic saving on disk, etc. All these activities operate on the same document. So they should
necessarily go on at once. Among these activities, some can block from time to time (automatic saving waiting for a
block to be actually written on the disk for example). Thus, considering the process as having a single thread of
control leads to the fact that if one of the activities blocks, then the whole process will be blocked. But why not
switching to the next activity within the process, instead of blocking the whole process? Since there is substantial
computation and substantial I/O in the process, why not making them to overlap in order to speed up the application?
On a multiprocessor system, activities going on at the same time within the process can be launched on separated
CPUs, thus ensuring a real parallelism.

This is why a word processor can necessarily be made up of multiple threads.

Fig: A word processor with three threads

Example 2: Let’s consider a dedicated server for a World Wide Web site. Requests for pages come in and the
requested page is sent back to the client. At most web sites, some pages are more commonly accessed than other
pages, and then are generally kept in main memory to reduce disk accesses. Doing so is referred to as caching. If the
server process (the one receiving client requests, processing them and sending back responses to clients) is made up of
a single thread of control, when it will start a disk read for example, the CPU may just stay idle, and only one request
could be processed at a time. But receiving a new client request and processing the previous one (reading from disk
for example) can be carried out at once, to satisfy more clients at a time and enhance the performances of the system
(since it is a dedicated server). This is why the server thread can be organized using threads as follows:

There is a thread called dispatcher, whose work is to read incoming requests from the network; and many other
threads, called worker threads, whose job is to actually process requests. Thus, the web server can be organized as
follows: The dispatcher reads and examine incoming requests from the network. After examining the request, it
chooses an idle (i.e. blocked) worker thread, hands it the request and wakes it up. When the worker wakes up, it starts
servicing the request. When a worker blocks on the disk operation, another thread is chosen to work, possibly the
dispatcher or another worker.
 Fig: A multithreaded web server

From the examples described above, one can figure out the following reasons for which threads are useful:

Main reason: in many applications, multiple activities are going on at once. Some of these may block from time to
time. By decomposing such an application into multiple sequential threads, the programming model becomes
simpler. This argument also holds for having parallel processes in the computer system, instead of thinking about
receiving interrupts, switching from one process to another, etc. So, as the system programming model becomes
simpler by considering parallel processes within the system, the process programming model also becomes simpler by
considering a process in which many activities are going on at once as made up of multiple sequential threads that
can run in quasi-parallel. Compared to processes, there is a new element added with threads: the ability for all the
threads in the process to share the process address space and all of its data among themselves. This ability is
essential, especially to those threads can run in quasi-parallel (switching becomes faster than context switch).

The second reason is that, since they are lighter weight than processes, they are easier (i.e. faster) to create and destroy
than processes.

The third reason for having threads is that in some circumstances, having such entities lighter weight than processes
can ameliorate performances of our system. In fact, when there is substantial computing and also substantial I/O
within the process, threads allow these activities to overlap, thus speeding up the application.

Another very important reason is the fact that on multiprocessors, real parallelism can be ensured by using threads.

II) Classical thread model

The process model is based on two independent concepts: resource grouping and execution. Sometimes, it is useful
to separate them; this is where threads come in. So, as far as the execution is concerned, a process has the concept of
thread of execution (usually shortened to just thread) to refer to the execution stream.

What threads add to the process model is to allow multiple executions to take place in the same process environment,
to a large degree independent of one another. Allowing multiple threads in the same process is referred to as
multithreading. So a process can be made up of one or many threads of control.

Fig: a) three processes each with one thread b) one process with three threads.

When a multithreaded process is run on a single CPU system, the threads take turns running. The CPU switches
rapidly back and forth among the threads, providing the illusion that the threads are running in parallel. As far as
resources are concerned, different threads in a process are not as independent as different processes. All threads in a
process have the same address space, so they share the same global variables. But not all resources are shared by all
the threads in the same process. In fact, each thread has its own program counter that keeps track of which
instruction to execute. It also has other registers to hold its current working variables, a stack which contains the
execution history. The following table represent some process items that are shared among all its threads and some
that are private to each thread.

There is no protection between threads because it is impossible and even not necessary.

Like a traditional process, a thread can be in any one of several states: running, blocked, ready, or terminated. The
transitions between thread states are the same as transitions between process states.

III) Operations on threads

The following are some operations that are mostly performed on threads:

Creation: when multithreading is present, processes normally start with a single thread present. This thread has the
ability to create new threads by calling a library procedure, for example, thread_create, with a parameter typically
specifying the name of a procedure for the new thread to run.

Termination: when a thread has finished its work, it can exit by calling a library procedure, for example,
thread_exit.

Wait for a (specific) thread to exit: In some thread systems, one thread can wait for a specific thread to exit by
calling a procedure, for example, thread_join. This procedure blocks the calling thread until the specified thread has
exited.

Releasing the CPU: the thread_yield procedure allows a thread to voluntarily give up the CPU to let another thread
run. This call is important because there is no clock interrupt to actually enforce multiprogramming as there is with
processes.

Other calls allow one thread to wait for another thread to finish some work, for a thread to announce that it has
finished some work, and so on.

IV) Case study: POSIX threads

here is a list of some POSIX threads procedure calls.

V) Threads implementation

There are two main ways to implement multithreading: in user space and in kernel space. A hybrid
implementation is also possible.
 1- Implementing multithreading in user space (management provided by a run-time system)

Here, threads are referred to as user level threads. The threads package is put entirely in user space, and the kernel
knows nothing about them. A run-time system is then provided for every process in user space to manage its threads.
As far as the kernel is concerned, it is managing ordinary, single-threaded processes. With this approach, threads are
implemented by a library. The general structure of this implementation is given on the following figure:

When threads are managed in user space, each process has its own private thread table to keep track of its threads, and
which is managed by the run-time system, which includes a thread scheduler. Unlike the process table found in the
kernel, this table keeps track only of the per-thread properties.

Advantages:

- user-level threads package can be implemented on an operating system that doesn’t support threads.
- Thread creation, termination, and switching from one thread to another can be very fast, because they are just
performed by library calls in user mode, and the kernel doesn’t intervene.
- This implementation also allows each process to have its own customized scheduling algorithm, which is very
important in some environments.
So all in all, user-level threads provide better performance.

Disadvantages: Despite their better performance, user-level threads have some major problems among which we
have:

- the problem of how blocking system calls are implemented, to avoid blocking the whole process when one of its
threads blocks.
- The problem of how to handle a page fault when it happens with a thread within a process.
- The problem of how to avoid a currently running thread from hogging the CPU, since the implementation allow
the CPU to only be released by the running thread, using the thread_yield library call.

Assignment: Describe some alternatives that can be used to overcome these problems, but by still keeping the threads
package in user space.

2- Implementing threads in kernel space (management provided by the OS kernel)

Here, threads are referred to as kernel-level threads. The run-time system is no more needed, and there is no thread
table in each process. Instead, the kernel has a thread table that keeps track of all threads in the system. All thread
management (creation, termination, etc.) is now done though kernel calls. In addition, the kernel also maintains the
traditional process table to keep track of processes. Here, all calls that might block a thread are implemented as system
calls. When a thread blocks, the kernel can run either another thread from the same process or a thread from a different
process. This implementation is depicted on the following figure:
Advantages: No problem with blocking calls, page faults, and no thread can decide to hog the CPU.

Disadvantages:

- greater cost of creating and destroying threads in the kernel. An alternative solution is to recycle threads.
- Cannot be implemented on an operating system that doesn’t support threading.

Discussion: what to do when a process creates a child process? Which parent’s threads should the child process have?
How to handle a signal sent to a process? Should it be forwarded to all its threads?`

3- Hybrid implementation (multiplexing user-level threads on kernel-level threads)

To combine the advantages of user-level threads with kernel-level threads, the hybrid approach is used. It consists of
using kernel-level threads and then multiplexing user-level threads on some or all of the kernel threads. With this
approach, the programmer can determine how many kernel threads to use and how many user-level threads to
multiplex on each one. This model gives the ultimate in flexibility. The hybrid implementation is presented on the
following figure: