UE20CS501

Computer Systems for Programmers

Unit-05: Virtual Memory

Santhosh Kumar V
Department of Computer Science & Engineering
Computer Systems for Programmers
Unit-05: Virtual Memory

Topic:
Dynamic Memory Allocation,
Garbage Collection, Memory bugs.

Santhosh Kumar V
Department of Computer Science & Engineering
05/28/2024 UE20CS501 2
Overview
CS:APP 9.9.1
Memory / RAM
• Heap management is an important
component that affects program
0xfffffffc
-
User stack 0x80000000

performance int x, y[10];

main()
-
• Need to balance: {
int a, b[10]; Memory-mapped
Regions f o r shared

– Speed & performance of

libraries
int *c;
c = malloc(10);
allocation/deallocation func();
-

}
– Memory utilization (reduce Heap

wasted areas)
Uninitialized Data
(.bss)
I n i t i a l i z e d Data

– Ease of usage by the

(.data) 0x10000000
-

programmer Code ( . t e x t )

0x00400000
-
 Dynamic Memory Allocation
int x, y[10]; Memory
Application invisible to
main() Kernel virtual memory
user code
Dynamic Memory Allocator {
User stack
int a, b[10]; (created at runtime)
Heap int *c; %rsp
c = malloc(10); (stack
• Programmers use dynamic func(); pointer)

memory allocators (such as } Memory-mapped region for

shared libraries
malloc) to acquire virtual
memory (VM) at run time.
– for data structures whose size brk
is only known at runtime Run-time heap
(created by malloc)
• Dynamic memory
Read/write segment Loaded
allocators manage an area from
(.data, .bss)
of process VM known as the
Read-only segment executable
the heap. (.init, .text, .rodata) file
0x400000
Unused
0
C Dynamic Memory Allocation
Functions from st d l i b . h
• void *malloc(size_t s i z e ) Memory / RAM
– Allocates size bytes and returns a pointer to the -
0xfffffffc

• void
block* c a l l o c ( s i z e _ t nmemb, s i z e _ t s i z e ) User stack 0x80000000
– Allocates nmemb*size bytes, sets the memory to 0,
returns a pointer to the block
-
• void f re e ( v o id * p t r ) function
Memory-mapped
– Frees the block at address ptr (returned by malloc/calloc), Regions f o r shared
libraries
returns it to the system for re-use by subsequent malloc calls
... malloc
i n t main() { Heap allocates:
i n t num; scores[3] 0x0006de4c
printf("How many s t u d e n t s ? \ n " ); scores[2]
scanf("%d", &num); scores[1] 0x0006de48
scores[0]
Uninitialized Data 0x0006de44
/ / cast “ ( i n t * ) ” from vo id * not necessary i n C
(.bss)
i n t *scores = malloc(num * s i z e o f ( i n t ) ) ; 0x0006de40
I n i t i a l i z e d Data
(.data) 0x10000000
/ / can now access scores[0] . . scores[num-1];
free(scores); / / deallocate -
return 0; Code ( . t e x t )
}
0x00400000
-
OS & the Heap
• The OS kernel maintains the brk
pointer Memory / RAM
– Virtual address of the top of the heap -
0xfffffffc

– Per process (threads share the heap) User stack 0x80000000

• brk pointer is updated via a system -

call (see Linux example below) Memory-mapped

Regions f o r shared
libraries
– #include <unistd.h>
– void * s b r k ( i n t p t r _ t increment); -

• Increments the brk pointer (up or down) brk p t r .

and returns the old brk pointer on success Heap

Uninitialized Data
– Newly allocated memory is zero-initialized (.bss)
I n i t i a l i z e d Data

• malloc/free allow the reuse of blocks (.data)

-
0x10000000

allocated on the heap with sbrk Code ( . t e x t )

0x00400000
intptr_t is a signed integer type that will match the size of pointers (32- or
64-bits) -
Dynamic Memory Allocation
• Allocator maintains heap as collection of variable sized
blocks, which are either allocated or free
• Types of allocators
– Explicit allocator: application allocates and frees space
• E.g., malloc and free in C

– Implicit allocator: application allocates, but does not free space

• E.g., new and garbage collection in Java

• simple explicit memory allocation

 How malloc (1) works?
• The C-library implementation will provide
an implementation to manage the heap Memory / RAM
0xfffffffc
• At startup, the C-Library will allocate an -
User stack 0x80000000
initialize size of the heap via sbrk
-

Memory-mapped
= sbrk( 1 << 20) ; / / 1 MB Regions f o r shared
– h e a p _ in i t libraries

– void *heap_init;
- new brk

o r i g brk
Uninitialized Data
(.bss)
I n i t i a l i z e d Data
(.data) 0x10000000
-
Code ( . t e x t )

0x00400000
-
 10.11

A First Look at malloc (2)

• The C-library implementation will provide
an implementation to manage the heap Memory / RAM

• At startup, the C-Library will allocate an -

0xfffffffc

initialize size of the heap via sbrk User stack 0x80000000

• Subsequent requests by malloc (or new) -

will give out portions of the heap Memory-mapped
Regions f o r shared

• Calls to free or delete will reclaim those libraries

memory areas -
brk

Heap
allocated
free allocated
free
allocated alloc
free Hea allocated
p
Uninitialized Data
(.bss)
I n i t i a l i z e d Data
(.data) 0x10000000
-
Code ( . t e x t )

0x00400000
-
A First Look at malloc (3)
• The C-library implementation will provide Memory / RAM
0xfffffffc
an implementation to manage the heap -

0x80000000
• At startup, the C-Library will allocate an
User stack

initialize size of the heap via sbrk -

• Subsequent requests by malloc (or new) Memory-mapped

Regions f o r shared

will give out portions of the heap

libraries

-
new brk
• Calls to free or delete will reclaim those Heap
o ld brk
memory areas

Heap
allocated
free allocated
free
allocated alloc
Hea
• If there is not enough contiguous free heap
free allocated
p
Uninitialized Data
(.bss)
memory to satisfy a call to malloc/new I n i t i a l i z e d Data
(.data)
then the library will use sbrk to increase
0x10000000
-

the size of the heap Code ( . t e x t )

– When no memory exists, an exception or NULL pointer will 0x00400000

-
be returned and the program may fail
Allocators and Garbage Collection
• An allocator will manage the free Memory / RAM
0xfffffffc
-
space of the heap User stack 0x80000000

• Types:
-
– Explicit Allocator: Requires the Memory-mapped

programmer to explicitly free memory

Regions f o r shared
libraries

when it is no longer used -

brk
Heap
• Exemplified by malloc/new in C/C++

Heap
allocated
free allocated

– Implicit Allocator: Requires the allocator free

allocated
free Hea
alloc
allocated
p
to determine when memory can be
Uninitialized Data
(.bss)

reclaimed and freed (i.e., known as

I n i t i a l i z e d Data
(.data) 0x10000000

garbage collection) -
Code ( . t e x t )

• Used by Java, Python, etc. 0x00400000

-
Allocator Requirements
• Arbitrary request sequences
– No correlation to when allocation and free Memory / RAM

requests will be made -

0xfffffffc

• Immediate response User stack 0x80000000

– Cannot delay a request to optimize allocation

strategy -

• Use only the heap Memory-mapped

Regions f o r shared
libraries
– Any heap management data must exist on the
-
heap or be scalar (single & not arrays) variables brk
Heap
• Align blocks

Heap
allocated
free allocated
– Allocated blocks must be aligned to any type of free
allocated
Hea
alloc
free allocated
data p
Uninitialized Data
(.bss)
• Previously allocated blocks may not be I n i t i a l i z e d Data
(.data)
moved -
0x10000000

– Once allocated the block cannot be altered by Code ( . t e x t )

the allocator until it is freed 0x00400000

-
 Allocation Example (Conceptual) #define SIZ sizeof(size_t)

Free word Show 8-byte words as squares

Allocations are double-word aligned
Allocated word
Allocated block Free block
(4 words) (2 words)

p1 = malloc(4*SIZ)

p2 = malloc(5*SIZ)

p3 = malloc(6*SIZ)

free(p2)

p4 = malloc(2*SIZ)
Allocator Goals
• Maximize throughput (i.e., fast allocation / deallocation)
• Maximize memory utilization
– Take as little memory as possible from the OS with sbrk
The .g., ays
– We need a formal definition of peak memory utilization
•
se no n allo
E

U(k) = max { P(i) for i = 1, .., k

goa eed cat } / H(k)
we

Hk
alw

• P(i) = memory allocated after i malloc/free requests

ls a
re a kee
• H(k) = total heap size (allocated/free) after k requests Pk

(Monotonically non-decreasing.)
t o p tra block
to e new

k (~time)
dds ck s w
(co of fr ith s
nfli ee
ctin bloc r
g)wit
he
ks
b
Fragmentation
• Poor memory utilization is caused by fragmentation
• Sections of memory (that are free) but can’t be used to store data, and
cannot be used to satisfy allocation requests.
• Two kinds
– External: Many small fragments of free space between allocated blocks
– Internal: When payload of is smaller than the block size allocated
• Often used when fixed size "chunks" are allocated
• Notice: There may be enough total free memory for a request but not
contiguous free memory
External Fragmentation

allocated free

Internal Fragmentation

all ocat ed allo cate d

 Internal Fragmentation
• For a given block, internal fragmentation occurs if payload is smaller
than block size

Block

Internal Internal
Payload
fragmentation fragmentation

• Caused by
– Overhead of maintaining heap data structures
– Padding for alignment purposes
– Explicit policy decisions
(e.g., to return a big block to satisfy a small request)
• Depends only on the pattern of previous requests
– Thus, easy to measure
 External Fragmentation #define SIZ sizeof(size_t)

• Occurs when there is enough aggregate heap memory, but no single free block is large
enough
• Caused due to alloc/free pattern leaving “holes(free mem)” between allocated blocks
p1 = malloc(4*SIZ)

p2 = malloc(5*SIZ)

p3 = malloc(6*SIZ)

free(p2)

p4 = malloc(7*SIZ) Yikes! (what would happen now?)

• Amount of external fragmentation

depends on the pattern of future requests
– Thus, difficult to measure
Implementation Issues
• How do we know how much memory to free given just a
pointer?

• How do we keep track of the free blocks?

• What do we do with the extra space when allocating a structure

that is smaller than the free block it is placed in?

• How do we pick a block to use for allocation -- many might fit?

• How do we reuse a block that has been freed?

 Knowing How Much to Free
• Standard method
– Keep the length (in bytes) of a block in the
word preceding the block.
• Including the header
• This word is often called the header field or p0
p0 = malloc(4*SIZ)
header
48

– Requires an extra word for every allocated block size Payload Padding
(aligned) (for alignme
block free(p0)
Keeping Track of Free Blocks
• Method 1: Implicit list using length—links all blocks
Need to tag
Unused each block as
32 48 32 16 allocated/free

• Method 2: Explicit list among the free blocks using pointers

Need space
32 48 32 16 for pointers

• Method 3: Segregated free list

– Different free lists for different size classes

• Method 4: Blocks sorted by size

– Can use a balanced tree (e.g. Red-Black tree) with pointers within
each free block, and the length used as a key
Free Block Management Unused
32 48 32 16
• Allocated blocks are the
programmer's to manage Header alloc ated ( 1 )
or f r e e ( 0 )
and need not be tracked Block Size 00a

pointer
explicitly Payload
returned from

• We must manage free lists malloc ()

heap_start
to make new allocations Padding/Footer

• Implicit free lists:

– Scan through both allocated free free

allocated and free blocks to Implicit Free List

find an appropriate free
block to allocate
32 48 32 16
• Explicit free lists: heap_start
– Maintain explicit list of free
blocks with each storing
information to find the next allocated free free free

(other) free block(s)

free_list
Explicit Free List
Implicit Free List
• For each block we need both size and allocation status
– Could store this information in two words: wasteful!
• Standard trick
– When blocks are aligned, some low-order address bits are always 0
– Instead of storing an always-0 bit, use it as an allocated/free flag
– When reading the Size word, must mask out this bit
1 word Unused
32 48 32 16

Size a a = 1: Allocated block

a = 0: Free block
Format of
allocated and Payload
Size: total block size
free blocks
Payload: application data
(allocated blocks only)
Optional
padding
Detailed Implicit Free List Example Double-word
aligned

End
Unused Block
Start
of 16/0 32/1 64/0 32/1 8/1
heap

heap_start heap_end
1 word

Allocated blocks: shaded

Size a a = 1: Allocated block
a = 0: Free block
Free blocks: unshaded
Headers: labeled with “size in words/allocated bit”
Payload
Size: total block size Headers are at non-aligned positions
 Payloads are aligned
Payload: application data
(allocated blocks only)
Optional
padding
Keeping Track of Free Blocks
• Method 1: Implicit list using length—links all blocks
Need to tag
Unused each block as
32 48 32 16 allocated/free

• Method 2: Explicit list among the free blocks using pointers

Need space
32 48 32 16 for pointers

• Method 3: Segregated free list

– Different free lists for different size classes

• Method 4: Blocks sorted by size

– Can use a balanced tree (e.g. Red-Black tree) with pointers within
each free block, and the length used as a key
Explicit Free Lists

Allocated (as before) Free

Size a 32 48 32 16
Size a
Next
Payload and Prev
padding

Size a Size a

• Maintain list(s) of free blocks, not all blocks

– The “next” free block could be anywhere
• So we need to store forward/back pointers, not just sizes
– Still need boundary tags for coalescing
– Luckily we track only free blocks, so we can use payload area
Explicit Free Lists
• When a block is free we can 32 48 32 16
use some portion of the block
to store explicit pointers to Header alloc ated ( 1 )
or f r e e ( 0 )
Block Size 001
"other" free blocks
Payload pointer
– Could use a simple returned from

doubly-linked list or some malloc ()

other data structure heap_start Padding/Footer

• Increases minimum size block allocated free free

(and potential internal
fragmentation for small
allocations) free_list
Block Size Block Size 000
• We can return the blocks in 000
Prev. Free Blk

"any" order (more on the next Prev. Free Blk

Next Free Blk Next Free Blk

slide) Empty
Empty

Padding/Footer
Padding/Footer
Implicit Memory Management: Garbage Collection

• Garbage collection: automatic reclamation of heap-

allocated storage—application never has to free
void foo() {
int *p = malloc(128);
return; /* p block is now garbage */
}

• Common in many dynamic languages:

– Python, Ruby, Java, Perl, ML, Lisp, Mathematica

• Variants (“conservative” garbage collectors) exist for C

and C++
– However, cannot necessarily collect all garbage
Garbage Collection
• How does the memory manager know when
memory can be freed?
– In general we cannot know what is going to be used in
the future since it depends on conditionals
– But we can tell that certain blocks cannot be used if there
are no pointers to them

• Must make certain assumptions about pointers

– Memory manager can distinguish pointers from non-
pointers
– All pointers point to the start of a block
– Cannot hide pointers
(e.g., by coercing them to an int, and then back again)
Classical GC Algorithms
• Mark-and-sweep collection (McCarthy, 1960)
– Does not move blocks (unless you also “compact”)
• Reference counting (Collins, 1960)
– Does not move blocks (not discussed)
• Copying collection (Minsky, 1963)
– Moves blocks (not discussed)
• Generational Collectors (Lieberman and Hewitt, 1983)
– Collection based on lifetimes
• Most allocations become garbage very soon
• So focus reclamation work on zones of memory recently allocated
• For more information:
Jones and Lin, “Garbage Collection: Algorithms for
Automatic Dynamic Memory”, John Wiley & Sons, 1996.
 Memory as a Graph

• We view memory as a directed graph

– Each block is a node in the graph
– Each pointer is an edge in the graph
– Locations not in the heap that contain pointers into the heap are called
root nodes (e.g. registers, locations on the stack, global variables)
Root nodes

Heap nodes reachable

Not-reachable
(garbage)

A node (block) is reachable if there is a path from any root to that node.
Non-reachable nodes are garbage (cannot be needed by the application)
 Mark and Sweep Collecting
• Can build on top of malloc/free package
– Allocate using malloc until you “run out of space”
• When out of space:
– Use extra mark bit in the head of each block
– Mark: Start at roots and set mark bit on each reachable block
– Sweep: Scan all blocks and free blocks that are not marked
root Note: arrows
here denote
Before mark memory refs, not
free list ptrs.

After mark Mark bit set

After sweep free free

Assumptions For a Simple Implementation

• Application
– new(n): returns pointer to new block with all locations cleared
– read(b,i): read location i of block b into register
– write(b,i,v): write v into location i of block b

• Each block will have a header word

– addressed as b[-1], for a block b
– Used for different purposes in different collectors

• Instructions used by the Garbage Collector

– is_ptr(p): determines whether p is a pointer
– length(b): returns the length of block b, not including the
header
– get_roots(): returns all the roots
Mark and Sweep (cont.)
Mark using depth-first traversal of the memory graph
ptr mark(ptr p) {
if (!is_ptr(p)) return; // do nothing if not pointer
if (markBitSet(p)) return; // check if already marked
setMarkBit(p); // set the mark bit
for (i=0; i < length(p); i++) // call mark on all words
mark(p[i]); // in the block
return;
}

Sweep using lengths to find next block

ptr sweep(ptr p, ptr end) {
while (p < end) {
if markBitSet(p)
clearMarkBit();
else if (allocateBitSet(p))
free(p);
p += length(p);
}
Conservative Mark & Sweep in C
• A “conservative garbage collector” for C programs
– is_ptr() determines if a word is a pointer by checking if it
points to an allocated block of memory
– But, in C pointers can point to the middle of a block
ptr
Header

• So how to find the beginning of the block?

– Can use a balanced binary tree to keep track of all allocated
blocks (key is start-of-block)
– Balanced-tree pointers can be stored in header (use two
additional words) Head Data
Size
Left: smaller addresses
Right: larger addresses
Left Right
Topics
• Explicit free lists
• Segregated free lists
• Garbage collection
• Memory-related perils and pitfalls
Memory-Related Perils and Pitfalls

• Dereferencing bad pointers

• Reading uninitialized memory
• Overwriting memory
• Referencing nonexistent variables
• Freeing blocks multiple times
• Referencing freed blocks
• Failing to free blocks
C operators
Operators Associativity
() [] -> . left to right
! ~ ++ -- + - * & (type) sizeof right to left
* / % left to right
+ - left to right
<< >> left to right
< <= > >= left to right
== != left to right
& left to right
^ left to right
| left to right
&& left to right
|| left to right
?: right to left
= += -= *= /= %= &= ^= != <<= >>= right to left
, left to right

• ->, (), and [] have high precedence, with * and & just below
•Unary +, -, and * have higher precedence than binary forms

Source: K&R page 53

C Pointer Declarations: Test Yourself!
int *p p is a pointer to int

int *p[13] p is an array[13] of pointer to int

int *(p[13]) p is an array[13] of pointer to int

int **p p is a pointer to a pointer to an int

int (*p)[13] p is a pointer to an array[13] of int

int *f() f is a function returning a pointer to int

int (*f)() f is a pointer to a function returning int

f is a function returning ptr to an array[13]

int (*(*f())[13])()
of pointers to functions returning int

x is an array[3] of pointers to functions

int (*(*x[3])())[5]
returning pointers to array[5] of ints

Source: K&R Sec 5.12

Dereferencing Bad Pointers

• The classic scanf bug

int val;

...

scanf(“%d”, val);
 Reading Uninitialized Memory
• Assuming that heap data is initialized to zero
/* return y = Ax */
int *matvec(int **A, int *x) {
int *y = malloc(N*sizeof(int));
int i, j;

for (i=0; i<N; i++)

for (j=0; j<N; j++)
y[i] += A[i][j]*x[j];
return y;
}
Overwriting Memory
• Allocating the (possibly) wrong sized object

int **p;

p = malloc(N*sizeof(int));

for (i=0; i<N; i++) {

p[i] = malloc(M*sizeof(int));
}
Overwriting Memory
• Off-by-one error

int **p;

p = malloc(N*sizeof(int *));

for (i=0; i<=N; i++) {

p[i] = malloc(M*sizeof(int));
}
 Overwriting Memory
• Not checking the max string size
char s[8];
int i;

/* reads “123456789”
from stdin */
gets(s);

• Basis for classic buffer overflow attacks

Overwriting Memory
• Misunderstanding pointer arithmetic

int *search(int *p, int val) {

while (*p && *p != val)

p += sizeof(int);

return p;
}
 Overwriting Memory
• Referencing a pointer instead of the object
it points to
int *BinheapDelete(int **binheap, int *size) {
int *packet;
packet = binheap[0];
binheap[0] = binheap[*size - 1];
*size--;
Heapify(binheap, *size, 0);
return(packet);
}
Referencing Nonexistent Variables
• Forgetting that local variables disappear
when a function returns
int *foo () {
int val;

return &val;
}
Freeing Blocks Multiple Times
• Nasty!

x = malloc(N*sizeof(int));
<manipulate x>
free(x);

y = malloc(M*sizeof(int));
<manipulate y>
free(x);
Referencing Freed Blocks

• Evil!
x = malloc(N*sizeof(int));
<manipulate x>
free(x);
...
y = malloc(M*sizeof(int));
for (i=0; i<M; i++)
y[i] = x[i]++;
Failing to Free Blocks (Memory Leaks)
• Slow, long-term killer!

foo() {
int *x = malloc(N*sizeof(int));
...
return;
}
Failing to Free Blocks (Memory Leaks)
• Freeing only part of a data structure
struct list {
int val;
struct list *next;
};

foo() {
struct list *head = malloc(sizeof(struct list));
head->val = 0;
head->next = NULL;
<create and manipulate the rest of the list>
...
free(head);
return;
}
05/28/2024 UE20CS501 51
Dealing With Memory Bugs
• Debugger: gdb
– Good for finding bad pointer dereferences
– Hard to detect the other memory bugs
• Data structure consistency checker
– Runs silently, prints message only on error
– Use as a probe to zero in on error
• Binary translator: valgrind
• allocating the wrong size, using an uninitialized pointer,
accessing memory after it was freed, overrunning a buffer, and
so on.
– Powerful debugging and analysis technique
– Rewrites text section of executable object file
– Checks each individual reference at runtime
• Bad pointers, overwrites, refs outside of allocated block
• glibc malloc contains checking code
– setenv MALLOC_CHECK_ 3
 THANK YOU

Santhosh Kumar V.
Department of Computer Science & Engineering

05/28/2024 UE20CS501 53