Professional Documents
Culture Documents
Multitasking From The Linux Command Line
Multitasking From The Linux Command Line
Prioritization
by Chris Jean
Did you know that you aren’t limited to working on one thing at a time while on a Linux command line? You can
actually “minimize” a program that you are in, get back to the command line, and then return to the program
whenever you’d like.
When you run a program or script on the Linux command line (from now on referred to as the shell), you are
creating a new job. For those that are used to GUI environments, each of these jobs is somewhat like a window
on the desktop. Just as you can have multiple windows and switch between them, the shell is capable of
managing multiple jobs and allows you to switch between them.
There is a lot to cover, so let’s start simple by describing what states a shell job can be in.
Jobs States
Foreground Jobs
Up to now, you may have only worked with foreground jobs in the shell. Foreground jobs are ones that don’t
return you to the command prompt until you exit out of the job or the job finishes.
If you run “ping yahoo.com” from the shell, the ping process will become a foreground job and will not return
you to the command prompt until you stop it.
Just in case you didn’t know, you can stop the ping process by pressing Ctrl+c. Unlike in many applications that
you may be used to, Ctrl+c does not mean “copy” in a Linux shell. Instead of copying text, this signals the
foreground process to terminate by sending it the SIGINT (Signal Interrupt) signal.
An example of a foreground job that won’t return you to the command prompt until it is finished is
the zip command. Imagine that you have a large amount of files inside a folder called “stuff”. If I run “zip -r
stuff.zip stuff” and don’t know about job management, I won’t have access to the shell
until zip finishes its task and returns me to the prompt.
Background Jobs
Putting a job in the background allows the job to run exactly as if it were in the foreground except that it does not
receive user input. If a command produces output and is put in the background, it will still produce the output.
Why would putting a job in the background be helpful then you may wonder. There are a number of reasons, and
they will become clear in the examples below.
Stopped Jobs
There is technically a third state that a job can be in. Jobs that aren’t in the foreground yet are also not in the
background have been stopped.
Like background jobs, stopped jobs do not receive user input. Unlike background jobs, stopped jobs do not
produce output. As a matter of fact, they don’t do anything. They have essentially been put on hold and won’t do
any processing until they are resumed or killed.
It may seem like stopped jobs are the odd man out, but they are actually the key to getting started with
multitasking on the Linux shell. I use stopped jobs every day, and I couldn’t imagine using the shell without them.
Imagine that you are in your favorite shell text editor working on a program or modifying config files, and you
can’t remember what the IP address you need is but you could easily look it up with dig. You could easily open
up another shell connection and run the command, but it would be quicker to simply stop the editor, run the
command quickly, copy the text, and bring your editor to the foreground.
The following is a sequence that illustrates how this example would look on your shell.
96.125.165.12
[chris@local ~]$ fg %1
The key to stopping a running job is the Ctrl+z key combination. Again, some of you may be used to Ctrl+z as
the shortcut to undo, but in the Linux shell, Ctrl+z sends the SIGTSTP (Signal Tty SToP) signal to the
foreground job. When you press this key combination, the running program will be stopped and you will be
returned to the command prompt.
You may notice the “%1” after fg. The “1″ is the job number that is to be brought to the foreground. The job
number is listed in brackets when the job was stopped. These numbers are crucial when you have multiple jobs
in stopped states. At any time, you can run “jobs” to get a list of the current jobs.
I stop jobs every day, and it greatly reduces the number of active shell sessions I have running. Sometimes I
have stopped jobs that I set up ahead of time when I first access the shell. For example, I may open up a MySQL
session, connect it to the database of the site I’m working on, and then stop it. Any time I need to run a query, I
can load up the job by running “fg 1“, run the query, and then stop the job again.
Stopping jobs is great, but what if we want to be returned to the command prompt while still allowing the job to
run? Sending a job to the background will do exactly that.
As I talked about in my post about finding files, the locate command is extremely helpful. Sometimes you need to
manually run the updatedbcommand in order for the locate database to update. The problem is that
the updatedb command can take a very long time to run, sometimes it can take more than an hour.
I’ll use the example of the updatedb command to show how you can take a foreground process and put it in
the background so that you can free up the shell.
[chris@local ~]$ sudo updatedb
1. I’m back at the command prompt and can do anything I’d like while the job works merrily away in the
background.
2. Notice how I used “%1″ with bg just as I did with fg. Both commands accept a job number to tell it which
job is to be sent to the foreground/background.
3. After running bg, a line similar to when I pressed Ctrl+z was printed showing the command and job
number. However, there is also the addition of the “&” at the end. We’ll get into the reason for that in the next
section.
When a process that is running in background finishes, you will receive a message the next time your command
prompt refreshes. The prompt refreshes when you run another command or simply hit Enter to get a new
prompt.
Continuing from the previous example, I receive output similar to the following when the updatedb command
finishes:
[chris@local ~]$
[1]+ Done sudo updatedb
[chris@local ~]$
As with all the other job-related output, you will be notified of the job number and the command.
Now that you see the value of having jobs run in the background, you might wonder if you can just start the
command in the background to begin with. In fact, you can!
Remember that odd ampersand “&” after the command in the output of the bg command? The ampersand
means that the job is running in the background. You can run a command with the ampersand at the end to
automatically put the job in the background.
Going back to the updatedb example again, we could have simply run the following to have the job in the
background and get our command prompt back immediately:
[chris@local ~]$ sudo date
...
[chris@local ~]$ sudo updatedb &
Now that you know the fundamentals of managing jobs, it’s time to get more advanced and tell you how to
terminate jobs and modify their execution priority.
Terminating Jobs
Sometimes a process goes rogue and either won’t respond to input, is consuming massive amounts of
resources, or both. At times like this, it’s very important to know how to kill a job which will cease the program’s
execution and free up its used resources.
The command used to kill processes is appropriately called kill. The kill command can accept either process
IDs or job numbers (the job number must be preceded by a percent sign, “%”).
If my updatedb job from before fails to respond, the job has a job number of “1″, and I want to tell the job to
stop, I could run the following command:
[chris@local ~]$ sudo kill %1
[chris@local ~]$
Note that you don’t have to always run the kill command with sudo, you just have to ensure that whatever
privilege level you are running the killcommand at has the ability to control that process. In other words, if you
are trying to kill a process run with sudo, you’ll need to use sudo to kill it as well.
Prioritizing Jobs
Linux does a great job of prioritizing tasks automatically; however, what Linux thinks is best isn’t always what you
want. Basically, Linux always wants to make a process finish as soon as possible, which means that you get a
nice bit of lag when you try to do your work as a CPU or disk hungry process churns away. Fortunately, there are
tools that will let you tell Linux what you want.
First, let’s deal with managing a process’ use of CPU time. If you are going to run a command that will eat up
large amounts of CPU cycles and you want to make sure that this process doesn’t keep you from doing your
work, you can run the command with nice to give the command a lower-than-default priority.
[chris@local ~]$ sudo date
[1] 16312
[chris@local ~]$
As before, a junk sudo command was run first to take care of the password prompt.
Running the command in nice lowers the processes CPU priority and will result in smoother CPU performance
for your other applications.
What if you already ran a command whose process is now greedily chewing away at your CPU resources?
That’s where the renice command comes into play.
With renice, you can change the CPU priority of an existing process.
[chris@local ~]$ sudo date
[1] 16324
[chris@local ~]$ sudo renice 10 16324
[1] 16324
[chris@local ~]$ sudo ionice -c3 -p16324
[chris@local ~]$
If we’d rather just lower the Best Effort class priority (defaults to 4) for the command so the process isn’t limited to
idle IO periods, we can run the following:
[1] 16324
[chris@local ~]$ sudo ionice -c2 -n7 -p16324
[chris@local ~]$
Again, the Real Time class should not be used as it can prevent you from being able to interact with your system.
You may wonder where you can get the process ID if you don’t know it, can’t remember it, or didn’t start the
process (an automated script may have launched it). You can find process IDs with the ps command.
For example, if I had an updatedb program running in the background, and I wanted to find its process ID, I can
run the following:
Another useful command is “ps -u user” (replace “user” with your username) which lists all processes your
user has access to. Basically, these are processes that your user or some process using your user’s permissions
started.
Another really helpful use of ps is “ps aux“. This command will list all the system’s processes, which user
owns the process, and even detail the resource usage of the process. This is very helpful for finding processes
that you don’t know enough about to find using other methods.
Keep in mind that many scripts execute from inside shell environments. This means that you can manage the
script’s job using the tools I’ve given you here.
If you look in the /etc/cron.daily folder, you’ll find many scripts that execute once a day. These scripts
can create a huge load on a system while they execute. If around the clock performance is crucial, you can
prioritize the shells that the scripts run in which causes all the commands run inside that shell to follow the same
prioritization rules.
This all sounds very complex, but it’s actually very simple.
Let’s open up the /etc/cron.daily/mlocate file (pick another file if you don’t have this one). At the top,
you should see a line like the following:
#!/bin/bash
This shebang line tells Linux what interpreter is used to execute the script. My example script uses the bash shell
as the interpreter. Other scripts may use the sh shell instead of the bash shell, but that won’t make any
difference.
renice 10 $$
ionice -c3 -p$$
The $$ variable is a special variable that contains the shell’s own process ID. The two commands above cause
the CPU priority to lower to 10 and changes the IO priority to class 3 (Idle).
Add this type of customization to your resource hungry scripts, and your system will perform more smoothly
around the clock. Keep in mind that this also can result in the scripts taking much longer to complete.
You can use those same commands above using the $$ variable to change the CPU and IO priority of shells
that you open in order to lower the impact you have while working on the system. This can be a smart thing to do
if you need to compile software or run other intensive processes on production machines.
Important Commands
Here’s a list of all the commands, key combos, and other important bits that I went over with a brief description
for each. It’s like the word list at the end of children’s books.