Department of Computer Science

COMP40451 Cyber Security

Lecture 9
Operating Systems
Security (1)
Dr Xiaoqi Ma
Outline
1 Principles of operating systems security

2 Memory and address protection

3 File protection

4 Summary
Operating Systems Concepts (1)
 An operating system (commonly abbreviated to OS) is an interface between hardware
and user
 An OS is responsible for the management and coordination of activities and the sharing
of the resources of the computer
 Operating systems offer a number of services to application programs and users
Operating Systems Concepts (2)
 Applications access these services through application programming interfaces (APIs) or
system calls
 Common contemporary operating system families include
 Unix-like systems (BSD, Darwin (Mac OS X), Linux, SunOS (Solaris/OpenSolaris), Android)
 Windows NT series (XP/Vista/7/8/10)
Operating System Functions
OS Security Principles
 An operating system must be capable of distinguishing between requests which should be
allowed to be processed, and others which should not be processed
 An operating system with a high level of security will also offer auditing options
 An operating system should take care of security of network services
 Internal security is especially relevant for multi-user systems
Operating Systems Integrity
 OS generic integrity policy: Users must not be able to modify the operating system
 Two competing security requirements:
 Users should be able to use (invoke) the operating system
 Users should not be able to misuse the operating system
Memory Protection Methods
 Fence
 Relocation
 Base/bounds registers
 Segmentation
 Paging
Fence (1)
 The simplest form of memory protection was introduced in single-user operating systems
to prevent a faulty user program from destroying part of the resident portion of the
operating system
 A fence is a method to confine users to one side of a boundary
 In one implementation, the fence was a predefined memory address, enabling the
operating system to reside on one side and the user to stay on the other
Fence (2)

Source of image: Pfleeger et al. Security in Computing, 5th Ed. Prentice Hall, 2015.
Fence (3)
 Another implementation used a hardware register, often called a fence register,
containing the address of the end of the operating system
Fence (4)

Source of image: Pfleeger et al. Security in Computing, 5th Ed. Prentice Hall, 2015.
Relocation: The Problem
 If the operating system can be assumed to be of a fixed size, programmers can write their
code assuming that the program begins at a constant address
 However, it also makes it essentially impossible to change the starting address
Relocation: The Solution
 Relocation is the process of taking a program written as if it began at address 0 and
changing all addresses to reflect the actual address at which the program is located in
memory
 In many instances, this effort merely entails adding a constant relocation factor to each
address of the program. That is, the relocation factor is the starting address of the
memory assigned for the program
Base/Bounds Registers (1)
 Fence registers provide a lower bound (a starting address) but not an upper bound
 An upper bound can be useful in knowing how much space is allotted and in checking for
overflows into “forbidden” areas
 A variable fence register is generally known as a base register
 The second register, called a bounds register, is an upper address limit, in the same way
that a base or fence register is a lower address limit
Base/Bounds Registers (2)

Source of image: Pfleeger et al. Security in Computing, 5th Ed. Prentice Hall, 2015.
Base/Bounds Registers (3)
 With a pair of base/bounds registers, a user is perfectly protected from outside users
 Erroneous addresses inside a user’s address space can still affect that program because
the base/bounds checking guarantees only that each address is inside the user’s address
space
Base/Bounds Registers (4)
 We can solve this overwriting problem by using another pair of base/bounds registers,
one for the instructions (code) of the program and a second for the data space
 Then, only instruction fetches (instructions to be executed) are relocated and checked
with the first register pair, and only data accesses (operands of instructions) are relocated
and checked with the second register pair
Base/Bounds Registers (5)

Source of image: Pfleeger et al. Security in Computing, 5th Ed. Prentice Hall, 2015.
Segmentation (1)
 Segmentation involves the simple notion of dividing a program into separate pieces
 Each piece has a logical unity, exhibiting a relationship among all of its code or data
values
Segmentation (2)
 Segmentation was developed as a feasible means to produce the effect of the equivalent
of an unbounded number of base/bounds registers
 Segmentation allows a program to be divided into many pieces having different access
rights
 Each segment has a unique name. A code or data item within a segment is addressed as
the pair ⟨name, offset⟩, where 𝑛𝑎𝑚𝑒 is the name of the segment containing the data item
and offset is its location within the segment (that is, its distance from the start of the
segment)
Segmentation (3)

Source of image: Pfleeger et al. Security in Computing, 5th Ed. Prentice Hall, 2015.
Segmentation (4)
 The operating system must maintain a table of segment names and their true addresses in
memory
 When a program generates an address of the form ⟨name, offset⟩, the operating system
looks up 𝑛𝑎𝑚𝑒 in the segment directory and determines its real beginning memory
address
 To that address the operating system adds offset, giving the true memory address of the
code or data item
Segmentation (5)

Source of image: Pfleeger et al. Security in Computing, 5th Ed. Prentice Hall, 2015.
Paging (1)
 One alternative to segmentation is paging
 The program is divided into equal-sized pieces called pages, and memory is divided into
equal-sized units called page frames
 Each address in a paging scheme is a two-part object, this time consisting of ⟨page,
offset⟩
Paging (2)
 The operating system maintains a table of user page numbers and their true addresses in
memory.
 The page portion of every ⟨page, offset⟩ reference is converted to a page frame address
by a table lookup
 The offset portion is added to the page frame address to produce the real memory address
of the object referred to as ⟨page, offset⟩
Paging (3)

Source of image: Pfleeger et al. Security in Computing, 5th Ed. Prentice Hall, 2015.
File Protection
 All-none protection
 Files are either public or private
 Group protection
 It focuses on identifying groups of users who have some common relationship
 When creating a file, a user defines access right to the file for the user, for other members of the same
group, and for all other users in general
 Individual permissions
 It involves using a name, a token, or a secret
Summary
1 Operating systems concepts and security principles
2 Memory and address protection
3 File protection