Case Study 1:

AMOEBA
History of Amoeba

 Amoeba originated at the Vrije Universiteit, Amsterdam,

The Netherlands in 1981 as a research project in distributed
and parallel computing. It was designed primarily by
Andrew S. Tanenbaum and three of his Ph.D. students. By
1983, an initial prototype, Amoeba 1.0, was operational.

 Starting in 1984, a second group was set up. This work used
Amoeba 3.0, which was based on RPC. Using Amoeba 3.0,
it was possible for clients in Tromso to access servers in
Amsterdam transparently, and vice versa.
Research Goals

 The primary goal of the project was to build a

transparent distributed operating system.
 An important distinction between Amoeba and
most other distributed systems is that Amoeba
has no concept of a “home machine”.
 A secondary goal of Amoeba is to provide a
testbed for doing distributed and parallel
programming.
 The Amoeba System Architecture

Processor pool File

server Print server
X-terminals
The Amoeba Microkernel

Client Server
Thread

Microkernel

Process management
Memory management
Communication
I/O
The Amoeba Servers

 Amoeba is based on the client-server model.

 Probably the most important server is the file server, known as
the bullet server.
 Another important server is the directory server, also known as
the soap server.
Objects and Capabilities

 The basic unifying concept underlying all the Amoeba servers

and the services they provide is the object.
 Each object is managed by a server process.
 Objects are named and protected by capabilities.
A capability in Amoeba

Bits 48 24 8 48

Server port Object Rights Check

Capability

 When the server creates an object, it generates a capability.

 If aport
client wants to create
objecta restricted capability, go through the
following.
1111 1111 C (random number)
 On subsequent operations, the client must present the
capability to identify the object.
Restricted capability

New rights mask

Capability
0000 0001
port object 1111 1111 C

Exclusive OR
One-way function

Restricted capability
port object 00000001 f(C XOR 00000001)
 Standard Operations

Age Perform a garbage collection cycle: starts a new garbage collection cycle to get rid of old objects
that are no longer accessible.
Copy Duplicate the object and return a capability for the copy: it is a shortcut that makes it possible to
duplicate an object without actually transferring it. Without this operation, copying a file would
require sending it over the network twice: from the server to the client and then back again.
Detroy Destroy the object and reclaim its storage: deletes the object

Getparams Get parameters associated with the server: allow the system administrator to read and write
parameters that control server operation. For example, the algorithm used to choose processors can
be selected using this mechanism.
Info Get an ASCII string briefly describing the object

Restrict Produce a new, restricted capability for the object

Setparams Set parameters associated with the server: same as Getparams

Status Get current status information from the server

Touch Pretend the object was just used: tells the server that the object touched is still in used.
Process Management in Amoeba

 A process is an object in Amoeba. When a process is created,

the parent process is given a capability for the child process.
The child can be suspended, restarted, signaled, or destroyed.
 Process management is handled at three different levels
in Amoeba.
 1. At the lowest level are the process servers, which are
kernel threads running on every machine.
 2. At the next level up we have a set of library
procedures that provide a more convenient interface for
user programs.
 3. Finally, the simplest way to create a process is to use
the run server, which does most of the work of
determining where to run the new process.
 Process Descriptor

Architecture = 386
Capability for exit status
Segment descriptors
Thread 1 Thread 2 Thead 3
PC1 PC2 PC 3
SP1 SP2 SP 3
Text Shared data 1 2 3 Stacks
SP1 SP3
PC1 SP2
Private data
PC2

PC3

Segments
Library procedures

 exec: to do an RPC with the specified process server asking it

to run the process.
 getload: returns information about the CPU speed, current
load, and amount of memory free at the moment.
 stun: to suspend a child process. Two kinds of stuns are
supported: normal and emergency.
Threads

 When a process starts up, it has one thread.

 During execution, the process can create additional threads,
and existing threads can terminate.
 Three methods are provided for threads to synchronize:
signals, mutexes, and semaphores.
 All threads are managed by the kernel.
Memory Management in Amoeba

 Amoeba has an extremely simple memory model.

 A process can have any number of segments it
wants to have, and they can be located wherever it
wants in the process’ virtual address space.
 Segments are not swapped or paged, so a process
must be entirely memory resident to run.
 Each segment is stored contiguously in memory.
Mapped segments
Process virtual
address space Memory segments

D
S S T

T
Communication in Amoeba

 Amoeba supports two forms of communication: RPC, using

point-to-point message passing and group communication.
Remote Procedure Call

 RPC Primitives:
1. get_request – indicates a server’s willingness to listen on a
port.
2. put_reply – done by a server when it has a reply to send.
3. trans – send a message from client to server and wait for the
reply.
  
 Group Communication in Amoeba

CreateGroup Create a new group and set its

parameters
JoinGroup Make the caller a member of a group
LeaveGroup Remove the caller from a group
SendToGroup Reliably send a message to all members
of a group
ReceiveFromG Block until a message arrives from a
roup group
ResetGroup Initiate recovery after a process crash
The Amoeba Reliable Broadcast
Protocol

1. The user process traps to the kernel, passing it the

message.
2. The kernel accepts the message and blocks the user
process.
3. The kernel sends a point-to-point message to the
sequencer.
4. When the sequencer gets the message, it allocates the
next available sequence number, puts the sequence
number in a header field reserved for it, and broadcasts
the message (and sequence number).
5. When the sending kernel sees the broadcast message, it
unblocks the calling process to let it continue execution.
System Structure
Application programs

A A A

Kernel S Kernel Kernel

Sequencer enabled Sequencer disabled

Broadcast network
 An example

Sequencer machine
A
M25 B
A A Last=24
A M25
M M25
M
Last = 24 Last=24history S
M25
M25

Request
for 24 A
C

Last=23 M25

buffered
Sender’s action for sending

 The sender sends a message to the sequencer and starts a timer:

 (a) the broadcast comes back before the timer runs out. (normal
case).
 the sender just stops the timer.
 (b) the broadcast has not come back before the timer expires.
(either the message or the broadcast has been lost).
 the sender retransmits the message.
if the original message is lost, no harm is done.
if the sender missed the broadcast, the sequencer will
detect the retransmission as a duplicate and tell the
sender everything is all right.
(c ) The broadcast comes back before the timer
expires, but it is the wrong broadcast. This occurs
when two processes attempt to broadcast
simultaneously.
 If message A gets to the sequencer first, and is
broadcast. A sees the broadcast and unblocks its
application program. However, B sees A’s
broadcast and realizes it has failed to go first. B
will accept A’s broadcast and wait.
Sequencer’s action

 If a Request for Broadcast arrives:

 (a) check to see if the message is a
retransmission. If so, inform the sender that the
broadcast has been done.
 (b) if the message is new, assign the next
sequence number to it, and increment the
sequencer counter by 1.
 The message and its identifier are stored in a
history buffer, and the message is then
broadcast.
Sender’s action for receiving

 When the sender receives a broadcast:

 (a) if the sequence number is 1 higher than the most recent one
(normal case). No broadcast has been missed.
 (b) if the sequence number is more than 1 higher (a broadcast
has been missed), the sender will send a message to the
sequencer asking for the lost broadcast.
Management of the history buffer

 If the history buffer fills up, if the sequencer knows that

all machines have received broadcasts, say, 0 through
23, correctly, it can delete these from its history buffer.
There are several mechanisms to discover this
information:
 (a) each Request for Broadcast message sent to the
sequencer carriers a piggybacked acknowledgement, k,
meaning that all broadcasts up to and including k have
been correctly received.
 (b) the sequencer can broadcast a Request for Status
message asking the number of the highest broadcast
received in sequence.
 Two methods for doing reliable
broadcasting

2 2 1 1 2 2

1 2 1 S 2 B
A S B A

2 2 2 2
1 1

1. Message sent to the sequencer 1. A broadcast M

2. The sequencer broadcasts it 2. S broadcasts Accept
 In method 1, each message appears in full on the network
twice. Each user machine is interrupted only once.
 In method 2, the full message appears only once on the
network. Each machine is interrupted twice.
Summary of the protocol

 This protocol allows reliable broadcasting to be done on an

unreliable network in just over two messages per reliable
broadcast. Each broadcast is indivisible, and all applications
receive all messages in the same order, no matter how many
are lost.
Fault tolerance

 The protocol is also fault tolerant.

 When a processor crashes, sooner or later, some
kernel will detect that the crashed machine are not
being acknowledged. The kernel will mark the
crashed processor as dead and initiates a recovery.
 In phase 1, one process is elected as coordinator.
 In phase 2, the coordinator rebuilds the group and
brings all the other processes up to date.
 The protocol is also fault tolerant

coordinator coordinator Sequencer dies

(a) 40 43 41 44 40 X
0 1 2 3 4 5

coordinator Dead sequencer

(b) 40 43 41 44 40 X
0 1 2 3 4 5

new sequencer

(c ) 44 44 44 44 44 X
0 1 2 3 4

(a) The sequencer crashes (b) A coordinator is selected (c) Recovery

 How does the coordinator get any message it has
missed if the sequencer has crashed?
 The solution lies in the value of k. If k is 0 (non-
fault tolerant), only the sequencer maintains a
history buffer.
 If k >0, k+1 machines maintain an up-to-date
history buffer. If k machines crash, there is still one
left to supply the coordinator with any messages it
needs.
 How to implement?
 In method 2, when the sequencer sees a
message, M, that was just broadcast, it does
not immediately broadcast an Accept
message. Instead, it waits until the k
lowest-numbered kernels have
acknowledged that they have seen and
stored it. Now k+1 machines have stored M
in their history buffers.
The Fast Local Internet Protocol

 Amoeba uses a custom protocol called FLIP for actual

message transmission. The protocol handles both RPC and
group communication and is below them in the protocol
hierarchy.
 FLIP is a network layer protocol.
 A

RPC Group

FLIP layer
 The Bullet Server

Create Create a new file; optionally commit it as

well
Read Read all or part of a specified file
Size Return the size of a specified file
Modify Overwrite n bytes of an uncommitted file
Insert Insert or append n bytes to an
uncommitted file
Delete Delete n bytes from an uncommitted file
 The Directory Server
Create Create a new directory
Delete Delete a directory or an entry in a directory
Append Add a new directory entry to a specified
directory
Replace Replace a single directory entry
Lookup Return the capability set corresponding to a
specified name
Getmasks Return the rights masks for the specified entry
Chmod Change the rights bits in an existing directory
entry
 The Replication Server
 The Run Server
 The Boot Server
 The TCP/IP Server
 Disk server
 I/O server
 A time-of-day server
 A random number server
 Swiss Army Knife server