you're in the.

right place! This

zine, has Comics

•.. 5 minutes later...

not so complicated.
X want to learn
more, nowf J
by 5ulia Evans
hHps: //jvns.ca.
iui+ier. com/bOrk,
 v "Table o? contents v
Unix
perm iss ions...H Sockets............... IO virtual memory... 17-

unix. domain
/ proc................5 Sockets............... 11 shared libraries... IB

system calls.... 6 processes........ 17 copy on write....IA

Signals............ T’ •threads................13> page -faul+s....... 20

■File descripters.8 ■floating point..... 14 mmap..............21

man page
pipes.................. 9 -File buffering..... IS sec+> ons........... 22

16
 Unix permissions H
There, are. 3 things (jou Is -1 File.txt" shows you permissions.
can do +o a -file Here's how to interpret-the output:

rw- rw- r-- bork sWf

read xt/rrVe execute bork (user\ staff (group^ ANYONE
can can can
read £ write read £ write read

File permissions are. \Q- bits 110 in binarij is b Se+ui3 aFFects

So rw - r— r executables
setg,d
V / user ^roop oJl = I l O 1 OO ioO Sis -I /bin/ping
OQp 11 o 11 o i oo
= 6 H H rw| r-x r-x root" root
sticky r" w x ru x ru >
this means ping always
For Files:
runs as root
r = can read chmod 6MH tile-txt-
W = can write setgid does 3 different
X - c-crn execute means change the
unrelated things far
For directories/A s approximately. permissions -to executables, directories,
and regular -files.
r - can list Files rw- r-- r--
W = can create Files
Simple!
\ - can cd into i access Files
 OA cxmoL^in^ direc-torcj: /proc
Everc^ process on Linux
has cl PID CprocessTDl
like. Hl-
in /proc./HZ., there's a.
lot of VERy USEFUL
infornncd-ion o<bowt
process H2_.
Directory with eveay -file
the process has openf
Run $ Is -I /proc/HZ/fd
+0 see -Vhe list of Piles for-
process H2.
These sym links are also ma^cc
Z you can use fhero +o recover
deleted filesV
/proc/Ptf)/crndline
cotnmand l'ne
arguments the process
Was started with
/proc /PlD/enxfirori
cx.ll of +he process s
environment variables
/proc /PlD /st-ack
The kernel’s current" stack
for+he process. Useful if
it’s stuck, in a system call.
List of process’^ memory
maps. Shared libraries,
heap 1 anonymous maps, etc.
5
/proc/PlOfexC
Symlink +o +he process's binary
mayic.: works even if +he
binany has been de let ad f
/proc/PIO /status
Ts the proyracn running or
asleep? How mucin memory
is it usiny? And much more;
. % t t m ,,
oind : more 2
‘1» 1<1 z **
Look od"
nnan proc
•for' more information 1
System coA\s 6

Tf- uou've ever used
* /MIX
you've used signals
rthe tinker you j
'' process CJl^f^Pj£^Z fSCamenWfanA
Every siynoA has a. default
action, which is one of:
' Snore
El Kill process
I3H) kill process AND
make core dump file
0 slop process
II
|c? resume process
S i anal s 7
4Xe Linux kernel sends you can send signals
processes signals in ^ourself with the kill
lo+s of situations System call or command
/+hatpTpe\
Sl6ll\ir Ctrl-C”1 various
/yourcKIdl closed? SlbTERtl kill ( levels of
1 terminated/
----------C.’ eS^*X. sioKiLL foil -q j “die"
-------------- C!nshrucf i oqj
—----- — SIOjHOP kill -HuP
often interpreted as
yfauit^/->' “feloo<d confi'q", e.g. nginx
Vour proyra.fr can set Signals can be hard
Custom handlers "for to handle correctly since
almost ancy signal they can happen at
(-------K /okvJT'iiX . ANY tinne
J (clean up and \ %_.
00^h^db^a^signJ^
exceptions'. precess
S16STOP g SlbklLL r XsuRPRise-!
can't be ianored (another signal! J 'y
3“t
SIOkILLed L~~\
 ■File descriptors
Unix systems use Isof (list openfiles'! will Vite descriptors Can
integers to track Show you a process's open files refer to :
*♦ files on disk
open f iles______ $lsof -p 42H1*~.P)P ***[*
-+ pipes
^j^open too.bet) FO NAME
Sockets (network connections'!
k 0 /dev/pts/t-hjl
process XT L"77—XTkwnel "♦terminals (like xterm^
H /okacjTWs 1 /dev/pts/tKji
2 pipe.-. iqi7q 2 devices (your speaker? Zdev/^uf)
1 -File *7 for / 3 /home / bark /awesome txt L0J» M?£f (e''en+fd< inottfu,
, teu- J signalfd,epoll{ etc.) 3J
& /imp /
+hese integers are called PVFRyTMIlKO on Unix ■">
■File descriptors tD is for file descriptor
2^ \iS ft. file, but lots ot things arc

Let's see how some simple (almost every process

VjKen you read or write
Python code works under has 3 standard FDs’-
+o a file /pipe /^twork the hood:
sVdin -> 0
Python:
you do that using a file f - open rfile.txt") Sfdouf ■* 3-
descriptor f- read lines t-! s+derr -> 2
XjJ connect fo Behind the scenes*.
\p^§oogle.co<n J read from sfd m
Y_____ /oM fd isVj Hj] means
l<jj <is 4
Python Z os read fronn the file
f wriFf program______ _________ fl
^rend PrS^perewe *“>
descriptor 0"
6BT / HfrP/i.d __/£
l^+o fd »S y (done f) ( file »4 /xft6 contents? J Could be a pipe, or file or terminal
Sometimes c^o<-> want to send 0. pipe is a. pair
when is does
■Hie. output of one process of 2 magical
file descriptors wri+el®, "hi"),
to the. input of another
P @ WC can read it •
P’P® pipe
inpuT ou+pat-
4 Xs | vjc -ft rea.d ( ©
yJS\
S3 "hi"
X S3 -files? stdin ) stdout Pipes are one */ay.—p
0 pipe You can,-t write to

/-inuK creates a buffer- what if your named pipes

foreacK pipe. target process dies? $ rnktifo nny-pipe

^S\ buffer This lets 2. unrelated

/wc\
© fc^SJ"5 ® processes communicate
through a. pipe f
If doc+a. cjets wri-H-en -to -Hne pipe.
■faster ‘than it's read, fhe buffer If wc dies, the pipe JL. fi open (./mu -pipe')
1ST f.wr.teC“M\n"')
will -fill up. Q|j;»g;;;| Will close and Is will be
sent S16PIPE. By default
When fhe buffer is full, writes
SlOPIPF fernnina+es your- f-open(./my-pipe^
+o Will block (waif) until the,
process. f.read lined e- “hi!
reader reads. This is normal £■ ok<!!/
 Sockets
Unix, systems Have
are complicated an API called He
Hvercx HTTP library uses AF-IMET7-
Uhat's that?
sockets under -the. hood
AF-IMFT means basically
curl awesome.coni “internetsocket”: it lets you
Python-, requests.yetC'yo connect to other computers
On the internet usina their
IP address.
The main alternative is
A F- UNIX (“uniK domain socket A
■For connecting to programs
on the same computet-.
io
here's u/hat yettiny
a cat picture uiitt\ +hg
socked API looks like/-
© Create a socket-
-fd= SOCkeKAF-IMET, SOCK-STREAM
(5) Connect to an IP/port
Connectttd 4 I2.IS.IH.I5-So')
@ Hake a. request"
Uirite (fd, “bFT /cat. pry HTTP/II
(S) Read the response
cat-picture =■ read tfd ... t
?> kinds ot internet"
(AF-iNFYl sockets •.
SOCk_ STRIATA = TCP
f J.L*
curl uses this
SOCk_ D&RAH - UftP
dijionA uses this
SOCk_ RAV/ — just let me
T Send IP packets,
pmg uses J W;H ji^pienvent
my own protocol.
 Unix domain sockets
Unix domain socket's thec^ let 2 programs There are 2 kinds of
are. -Files. uniX domain sockets:
On -the same computer
Communicate. rT~~v-Tlike TCP? Lets
$-File, mgsock.sock
Docker uses Unix domain Y°u send a
Socket" continuous
socket's, for example ! stream of bytes.
■HieFile's permissions foKfiET /contained*'feques+’
{docVaprarrT^I1^ Uj j! Lets
process
determine u/ho can send you send discrete
chun ks of data
data to the Socket". Here you go!^ 1 9 a □ n □

oukoin¥a£e 1 Q.dvan+03 e 2. advantage S

Lets ujod use File permissions UDP sockets aren’t always You can send a File
reliable (even on the same descriptor over a unix
io restrict access to HTTP/
computer). domain socket.
database services?
Unix domain datagram Useful uitea handlma untrusted
chmoJ 600 secret. sock inpu-l Files! J
sockets arg reliable!
And thej won't reorder packets?
This is why Docker uses f----------------- ________
< *=7 here's a file A LLL’.T"'
a Unix domain socket.© nn rl can send data/1 downloaded '
nAvr-un evil] /permistofll r-jj-i proSs(and d'llj process! sketchy.com T 1^771/
•licJ \container/ yderned rlrzJ ^arrive Sandboxed
process k----------J x-------- > Linux process
 12
Pit) USER and GROUP ENVIRONMENT
VARIABLES
SIGNAL
HANDLERS
Jln/ho are you)
*r i*-n
(jrunn i n as ? J like PATH I you r—< /l ignoreX
reporting for b=^\si&TERH !J
Can set them with:
dut<^ F
^Julift-T^^i p-.-| f"L shot
1 env A=val ./program
(down safelyl)

WoRkiN G PARENT PIO COMMAND LINE" OPEN FILES

DIRECTORY tfASsXplOl lini+^S ARGUMENTS Everc^ open file has
*s everyone's
ancestor" See. them in an offset.
Relative paths L/btahX
E3PIO IH"7
are relative. to the /proc /PlO/cmdline fl've reaJ JOOO |
Working directargf T liil'T bytes of that one)
chdir changesit. £0P'O

IMEMORV THREADS NAMESPACES

CAPABILITIES
heap', stack'. = rrri^aln in the hosr\
sometimes one p-r-i JI have "A (network namespace]
shared libraries'. ^\ap_ ptrace;
the program's binarcl'-
Sometimes LOTS
T have rv»j own\ /7\
iGell 1 have LnamespacVr
mnnaoed -files' ICAP-SVS-ADHii^Io]
process
 threads 13
Threads lef a process threads in the same and theig share code
do nnang di-f-ferent’ things process share memorij calculate- p*
ah the same time.

IZ1I write some' -Find— big-prime-number /

00
digits ot" TF
to Ox I1AH1O m me but each -thread has its
thread 1
own stack and th eg can
uh oh! that's wl be run bg ditferent CPUs
X was putting m
at the same time
prime numbers. thread 2.
|CPU ij |CPU 2]
tr thread primes -thread
sharing memory can cause problems U/hg use threads instead
(race conditionsf) oh starting a neu/ process?
-»■ a thread takes less time
memory
to create.
TT T’m going to
add 1 to that -> sharing data between threads
number" V is very easy. But it's also
2 easier to make mistakes
With threads.
o ./UfR0M6. you weren't supposed
Result: 24 Should be.25< ■fe CHA<J6 E +ha+- da-t
 rloatinq point" ih
ex double is 6H bits V/eird double ar'drhme+ic
SiOjO exponent -Fraction
4 4 4
IOOIIOII IOOIIOII IOOIIOII IOOIIOII 752+n n-752 . f+be next number a-Fter
IOOIIOII IOOIIOII IOOIIOII IOOIIOII 1 2S2is 2s2<L)
F-1023 , ,
±2 X l.frac ' + «SH ~ । 4— ithe next number aF+er
That means there are 1 is l.y
2feM doubles.
n 2ooo_ ■ x- -x z
The biggest one is about Z — intimtc^ 4— inf inity is a double
2lo*s inWjj- infinity nan = -not a number.

doubles get" -Farther Javascript only has ^S-Tdoubles are scar^ \

apart as theg get bigger Y 1 ^d+heir arithmetic 1
doubles (no integers F)
is weird f J
between 2n and 2 there
> 2**5*
are always 251 doubles, /tkeg're verij logicalt |
4007144254740492
evenly spaced. just understand how I __
theij work and don't
> 2**5* *1.
that means the next double 1 use integers over 2s3 / A
4007194254740492
after 2fe0 is 26%feM^2b0 \in Javascript w J
I57 Same number ! uln ok 1

1 ? 1_________________
Crz*/ Pr 1 nted som e +exf |
\T/ but it didn't appear
l^on the screen.u»hy??j
/timeto learn about)
-flushing
3 kinds of buffering
(defaults vary by library) '-'
CD Mone. This is the.
default -for stderr.
©Line buffering.
(write after newline). The
default for terminals.
©"full" buffering.
(write in big a unks)
"The default for files
and pipes.
ile bufferinq
On Linux, you write to
■piles$ terminals witK
the System call
v wriVe *
[O] ^/please write “1w cats'*'!
pfotets\to file »1 Istdout) J
(oka^f^Jcl
Linux,
-flushing
To force your I/O library
to write everything it
has in its buffer right
now, call flusht
Fn^/l'll call urit(TD
ftWftan j please
15
I/O libraries don't
always call write,
when you print.
printf C*T V Cats");
|q) °*/Uft,b "For a aeuiline\
prirrtf L^eforeac+odJlu writing]
This is called buffering and
it helps save on syscalls.
when it's useful to flush
—♦ when writing an
interactive, prompt t
Python example.:
printC* password •- -flush = True)
—►when you're, writing to a
pipe. / socket
p-r> /no seriously, actually'
J
Write to that pipe
 memory a.UoG<xVion lfe
your program ha.5 your memory allocator Thing? 2 : Ask the OS
memory (malloc') is responsible tor more memory?
■for 2. -things.
1 ra used
binary
BM Yl/A nfcg&MIIWWBTOKiwwgl □ -Free
ThiwC? 1: keep track, of rrr-i ooy°h no!I'm being ousked}
1 ' 3Mg> stack
what memory is used/-free. “ ? for MO HB and I don't J

ra used ma oc
587 MB heap
CO-7 MQ n r
? ° ~rree
___ /can I have"")
the heap is what* YA vhh WMf/M____ WffitUOh
] C_z j^lMB more ?/ p~r|
your allocator
manages ____ malloc Q^ere you
V/////////ZZ//7SZ1

your memory malloc tries to till in malloc isn't =ma^icn

allocator's inter-face unused space when you ask
it’s jusl" afunctionf
tor memory
malloc (size-t size)
allocate size bytes of your Jean I have 512 \ you can always:
memory X return a pointer to it. code. (gbytes of memory?J
•free (void* pointer)
-* use a different malloc
mark, the memory as unused library like jemalloc
(and maybe give back to the OS). malloc
reallocCvoid* pointer, size-t size) or tcmalloc (easy!)
wA w/K YA uzai w/zw,
ask for more /less memory for pointer.
1 1 vk11/m/m, -►implement your own
CalloC (size.t members, size-t size)
allocate array + initialize to 0. malloc (harder}
your new/ memory qp
 viriuou memor
physical memory has ?ver<^ program has its
lyxir computer ho3
oun virtual address space.
addresses, like
physical memory
0-2GB . |00 ^0x~l21S^o"—» ^pup^ies^
but when your program
program
references an address
•ike- Ox 5cfe9a2a2?
jTTj00 ban^nas^)
25
io I 5 । i~ '“’’T| —------- * J that s not a physical
memory address I
program
2
Io iMtft
1+s a virtual address.

Linux keeps a mapping -from when your program everu fime. you switch
virtual memory pages to which process is running,
accesses a virtual address
physical memory pages called
Linux needs to switch
the page table rTTl/lm accessing)
the page table
'>9C" is o^£^^«"es
bdVPKZiooo 7
cpu ~~ ~ ~- rrrn.yhere's the address
l^chunk. of memory ] bigger ।__ ooKT'll look thai up in"X
15] 7 page fable and \ Crd ^process 21S0's page fable]
MMU V^en e'-“iess fhe right)
P1D virtual addr physical addr "m€moru\P5'‘iCa’' addfe« / /Thanks,I’ll uscX rrri
1171 0x20000 0x11200 0 managemenf^-’-------- - — |+ha+ now f /*b21
unj^- '■
2310 0x2000 0 0x22 ^000 ------------- :------ S MMU
2310 0x21000 0x1788’000 hardware
Shared libraries 18
 C opuonvjr iVe.
Copying all that" memory
-the cloned process
new/ processes using every time we fork
has EXACTLY the same
the. fork^ or cloned Would be slow and a.
memorcy waste of RANI
—> same hexip
/often processes calH
—* Same stack. Exec right after fork,
fl. child process that's ~* Same memory maps which means they
CL copy of the caller don't use the parent
if the parent has 368 of process's memory
memor^/ +he child will too. basically at all ¥ j
parent CKild
So Linux lets them share Linux does this by \when a process tries
physical RAM and only giving bo-Hn the processes to write to a shared
identical page tables. memory address:
copies the mernorcy
when one of the_m © there s a 5 page faultr
Q) Linux makes a copy
ram of the page (. updates
1 I'd like to change -the page Table
5 \that memory CJ tne process continues,
blissfully ignorant
J I'll make ytH?| pr but it marks every page,
own copy* J—*'~L^
as read only. process
 pa.<^e f aulfs 20
Linux process has what happens during
Some pages are marked
a page -table a. page fault?
as either
-♦ the MMU sends an
* pa^e. table. # *read or\lg interrupt
vir’hjotl nfiCmor^ physical memory ★ not resident in memorcy ~* your program stops running
address address
-» Linux kernel code to handle
Ox IR72.2000 Ox 1422000 when you trig -to access the page fault runs
Ox 11724000 Ox 1422000 <xpage that’s marked
Ox 1524000 not in memory __o»{ -T'" tix -Hie problem and!
"not resident in memory,
Ox ItHMOOO 0x4a0oo read onia liiJ f let your program keep /
it triaqers a I page, fault f Linux ( runninq

nof resident in memory

US<Jalley moans the ©run out erf- RAM (y)When a program tries to access
da+a. is on disk y g ^wiiiwmnnwww* % f the memorg.there's al J?*9® I
disk-. HMMMiM • fault;
virtual memory @ Linux.saves some RAH <3) |^3t*S tiwe rnove- some's
. ................... data to disk. LI data. back to RAMTj
RAK-M
in ram on ^isk. disk-prow Virtual memora wuunuf/uw/MM ll~l
(?) mark those pages as ram h i
"not resident in memory' © if this happens a lot, your
Having some virtual memorcj
in the page table program gets VE~Ry SLO^ _
that is actuality on disk is virtual memory * —I
rrn ll'm always waiting for data.)
hour swap and mnnapwork. RAM ymw^miimwa—i LJ (to be moved in I out of RAM/
 mmap
VjhaVs nnmap -For? load Files lazily
iwitb mmap
want to work. wiFW
Vlhen you mmap a. file, it
Y' J <*■ ^E’RV MR6E FILE /
A 1 but if won't Fit / yets mapped into your
memory J program^ memory.
4. ' « <__
Could but nothing is AC. WALL'/
\ 4"rij mm op fJ read info RAH until you
try to access the. memory,
6nmop = “ memory map")/\ (how it works: page faults?)
u
now -to mnnap
in Pyfbon
imporf" mmap
4= open (“HU&B. txt ""i
mm: mfnap.mmapff.f;|enoO, o)
^~+his won't read -Hie
Z । O OT File from disk. J
1^1..__ ___
Finishes ~instantly,
print (mm C-iooo-.-f)
this^will read onl<5
the last 1000 bytes▼

sharing bit^-File^ dynanruC linking (Xnonymous memory maps

U/i-VVa mmot-p
bj)'2_2T' we all want to
Pjj Ej~J~ead the some file^
(no problem mmop
Even if 10 processes
mtnap a. file, if will only
be. read info memory
woncew
uses rnmcip
-» not from a File
Eh- fl- need to
Ijjse Ii^c.so 6j (memoruj set to 0 by default)
program
“* With MAP. SHARED, you can
______ C standard libraru
Use them to share memory
(you too eh9. no problenj)
With a Sub process f
~L always mmap, so L [ [J j
that -File (s probably / dynamic
loaded into memoryJ linker”
kalread^-_________ s
 ncicin paqe sec+ions
man pages are split up man paae sections
info 8 sections
(T) programs (2) System calls
© (2) ®
Iman grep $ man senof ile.
$ man 2. read $ man Is iman pdra.ce
(^) C functions
means ''get me the man page. devices
■for read from section l". 3> man printr i man null
1 man fopen
There's both for /dev/null docs
—►a program called "read (5^ "File formats (S) games
-» and a system ca.ll called read '
$ man sudoers not super useful-
so for /etc/sudoers Iman si
$> man 1 read S man proc is niu favourite
files in /proc ? J 1,
from that section
gitfes you o- different" man p°-9e "from
miscellaneous Sysadmin programs
J man 1 read explains concepts?
If you don't specify a section, man will Sman 7 pipe
$ man apt
look, through all the sections £ show 1 man chroot
i man 7 symlink
the first one if finds.
VJcxrvV Vo learn more?
I Inic^hlty recommend
THE LINUX
"this book:----------------- PROGRAMMING
Every chapter is a readable.; INTERFACE
short (usually 10-20 pages)
explanation of CL Linux system.
I used it as a reference
constantly when wrifiny
this z.ine.
1 V it because even though
its huge ant comprehensive^
(IS00 pages , fhe chapters
are short and self - cordainecl
and it’s very easy -bo pick if
Up and learn something.
 love
•find more aue&ome zines at”
-> Wizardz.ines.com 4-