Nov 30, 2023

🎯Backend Communication Design Patterns

⚡Request-Response Model
1. Client sends a Request
2. Server parses the Request
- Server should know what a Request is, where does it start, where does it end
- Not as cost efficient as you think it is
3. Server processes the Request
- Knowing what the Request method and actually processing it are completely
different things
- Serialization and deserialization of Request is part of processing but the
definition of whether it is processing or parsing is subjective to some extent.
4. Server sends a Response
5. Client parses the Response and consumes the output

❓ Where is it used
Everywhere basically.
1. Web => HTTP, DNS, SSH
2. RPC
3. SQL and DB Protocols
4. APIs (REST/SOAP/GraphQL)

💀 Anatomy of a Request/Response
A Request structure is defined by both the client and the server. It has a boundary and is
defined by a protocol and message format:
GET /HTTP/1.1
Headers
<CRLF>
 BODY

Both client and server have to understand this structure (via code, e.g. http library) and
agree to it and be able to process it into an object that your backend language can
understand.

Same things are applicable for the Response as well.

⬆️ Sending an image to a Server with Request-Response

1. Send the entire image in one request
Has problems with network restrictions, can’t send too much data.

2. Send the image in chunks

Can be resumed in case a crash happens, just have to send the chunks which haven’t
been received yet.

Doesn’t always jive well with all use-cases.

For example,
1. Notification service
2. Chatting application
You will have to use Polling in these use-cases but this will cause a lot of network
congestion.

3. Very long requests

4. What if the client disconnects?
It has no knowledge of whether the server has responded or not.

What a complete Request-Response flow involves:

ℹ️INFO:
Do note that the Request has to be first constructed and serialized into binary at
the Client-side before it is transmitted across the network. Same goes for the
Response at the Server-side.
🧑‍💻Demo curl command to showcase how Request-Response works:
Command: curl --trace output.txt http://google.com

== Info: Trying 142.250.76.78:80...

== Info: Connected to google.com (142.250.76.78) port 80

ℹ️INFO:
Sending Header
=> Send header, 73 bytes (0x49)

ℹ️INFO:
Request Method, Protocol used and version of Protocol used are listed in the request
header along with the User Agent
0000: 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a GET / HTTP/1.1..
0010: 48 6f 73 74 3a 20 67 6f 6f 67 6c 65 2e 63 6f 6d Host: google.com
0020: 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 63 75 ..User-Agent: cu
0030: 72 6c 2f 38 2e 34 2e 30 0d 0a 41 63 63 65 70 74 rl/8.4.0..Accept
0040: 3a 20 2a 2f 2a 0d 0a 0d 0a : */*....

ℹ️INFO:
Started receiving Response, first we receive the response headers. First header is
301 status code that redirects us to “http://www.google.com” but redirection is not
supported by “curl” so we continue receiving further Response headers (which is
also an interesting behaviour about the Request-Response model)
<= Recv header, 32 bytes (0x20)
0000: 48 54 54 50 2f 31 2e 31 20 33 30 31 20 4d 6f 76 HTTP/1.1 301 Mov
0010: 65 64 20 50 65 72 6d 61 6e 65 6e 74 6c 79 0d 0a ed Permanently..
<= Recv header, 34 bytes (0x22)
0000: 4c 6f 63 61 74 69 6f 6e 3a 20 68 74 74 70 3a 2f Location: http:/
0010: 2f 77 77 77 2e 67 6f 6f 67 6c 65 2e 63 6f 6d 2f /www.google.com/
0020: 0d 0a ..

ℹ️INFO:
Continue receiving further Response headers
<= Recv header, 40 bytes (0x28)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 xt/html; charset
0020: 3d 55 54 46 2d 38 0d 0a =UTF-8..
<= Recv header, 245 bytes (0xf5)
0000: 43 6f 6e 74 65 6e 74 2d 53 65 63 75 72 69 74 79 Content-Security
0010: 2d 50 6f 6c 69 63 79 2d 52 65 70 6f 72 74 2d 4f -Policy-Report-O
0020: 6e 6c 79 3a 20 6f 62 6a 65 63 74 2d 73 72 63 20 nly: object-src
…

ℹ️INFO:
After so many Response headers, we finally receive the data of the Response which
contains the HTML responsible for displaying that “The document has moved to
http://www.google.com”
<= Recv data, 219 bytes (0xdb)
0000: 3c 48 54 4d 4c 3e 3c 48 45 41 44 3e 3c 6d 65 74 <HTML><HEAD><met
0010: 61 20 68 74 74 70 2d 65 71 75 69 76 3d 22 63 6f a http-equiv="co
0020: 6e 74 65 6e 74 2d 74 79 70 65 22 20 63 6f 6e 74 ntent-type" cont
0030: 65 6e 74 3d 22 74 65 78 74 2f 68 74 6d 6c 3b 63 ent="text/html;c
0040: 68 61 72 73 65 74 3d 75 74 66 2d 38 22 3e 0a 3c harset=utf-8">.<
0050: 54 49 54 4c 45 3e 33 30 31 20 4d 6f 76 65 64 3c TITLE>301 Moved<
0060: 2f 54 49 54 4c 45 3e 3c 2f 48 45 41 44 3e 3c 42 /TITLE></HEAD><B
0070: 4f 44 59 3e 0a 3c 48 31 3e 33 30 31 20 4d 6f 76 ODY>.<H1>301 Mov
0080: 65 64 3c 2f 48 31 3e 0a 54 68 65 20 64 6f 63 75 ed</H1>.The docu
0090: 6d 65 6e 74 20 68 61 73 20 6d 6f 76 65 64 0a 3c ment has moved.<
00a0: 41 20 48 52 45 46 3d 22 68 74 74 70 3a 2f 2f 77 A HREF="http://w
00b0: 77 77 2e 67 6f 6f 67 6c 65 2e 63 6f 6d 2f 22 3e ww.google.com/">
00c0: 68 65 72 65 3c 2f 41 3e 2e 0d 0a 3c 2f 42 4f 44 here</A>...</BOD
00d0: 59 3e 3c 2f 48 54 4d 4c 3e 0d 0a Y></HTML>..
== Info: Connection #0 to host google.com left intact
Dec 1, 2023
⚡Synchronous-Asynchronous Communication
Can I do work while I wait?

Synchronous I/O

Caller sends a Request and the Caller is now blocked.

No interactions can now be done on the caller-side.

For example, if you click a button it now goes to the backend, it is now waiting for a
response, not even the button will give any kind of response until the Request has been
responded to.

Receiver responds and the Caller will now be unblocked.

Caller and Receiver are in “sync”.

Asynchronous I/O

Caller sends a Request. However a Caller can work until it gets a Response.

But how does it know it has gotten a Response?

1. Check if the Response is ready
- Polling, i.e. spam “Are you Ready?” signals to the Receiver. Also called epoll.
- node.js uses epoll.
2. Receiver calls back when it is done
- Response is added to a Completed queue/stack from where the Caller will
understand that it has gotten a response).
- Windows uses the Completion queue logic.
3. Spin up a new thread that blocks
- When a blocking operation like reading a file is encountered, create a new
thread that will do this task and let it be blocked till it gets a response while the
main thread continues execution.
- Nice trick to perform async execution.
Example of an OS asynchronous call (NodeJS)

ℹ️INFO:
We use callbacks here (or Promises/async-await in modern Node.js or futures in
C++).

In the context of the Request-Response model, Synchronicity is a client property

because the “Can I work while I wait?” kind of questions are from the perspective of a
client and not the Server.

Most modern client libraries are asynchronous.

For browsers, it is the event loop that is responsible for checking whether any callbacks
have been received and continue allowing the client to do their tasks on the interface.

🧑‍💻 What the heck is the event loop anyway? | Philip Roberts | JSConf EU

▶️
Synchronous communication Asking someone a question in a meeting
You need a live response then and there itself.

Asynchronous communication ▶️ Sending an email to someone

The response can come back at any time.

Asynchronous Backend Processing

The whole Backend is synchronous in nature.

But the Client is calling backend handlers asynchronously, how do we manage this?
● Use Queues.
● When a Request to execute a handler is received, the backend puts this Request
into a Queue and responds back. It does not guarantee an immediate Response
because it may already be servicing some other Request.
● Because of this the Client gets immediately unblocked as it receives a Job ID or a
Promise that will contain the response of its Request once it is done being
processed by the Server.

There is a lot more detail to this that we will discuss later from the backend side of
things.

So in conclusion, in order to accomplish async processing in a request-response model,

we basically did two Request-Response flows, one to send the Request and receive the
“Promise” and the second one to receive the real Response from the Server.

Asynchronous Commits in Postgres

First understand how regular commits work.

All your transaction’s changes are saved to a data structure called a “Wall” which is
stored in-memory.

When you call commit, all the changes in the “Wall” are committed to the Database
which is stored on disk. This is a synchronous operation and the user is blocked from
the DB till the write to disk is completed successfully.

So the async version of this just removes the blocking section of the execution, we
immediately return to the user that the transaction has been committed before it is
written to disk and hope that it is committed. Ofcourse there are a lot of problems with
this and you can easily end up with Dirty Reads and so on. But it is asynchronous in
nature.
 Asynchronous Replication

One master replica for committing writes and worker replicas for performing reads.

When a write is done, you can either write to all worker replicas and only then commit to
master replica (all the while the Client is blocked).

Or you can write to the master replica and return to the Client and asynchronously
update all the data in the worker replicas.

Also note that the commit in the Master replica can itself be an asynchronous commit
as discussed before, but they are different levels of asynchronicity.

🧑‍💻Async Demo with Node.js

⌨️Code:

✅Output:
 ⚡Push
I want it as soon as possible

❓ Why
Because Request-Response isn’t always ideal.

For example,
Client wants real-time notification from the backend.
● A user just logged in
● A message has just been received.

For these use-cases the Request-Response model doesn’t scale that well.

This is where the Push model would be the perfect candidate.

If the Server knows something, it would push it to the Client.

❓ What is a Push
Client connects to a Server
Server sends data to the Client

Client doesn’t have to Request anything (unidirectional data stream from the Server).
The protocol must be bidirectional (but this is debatable).

Used by RabbitMQ, the moment the Queue receives a result, it pushes to the Clients
(subscribers) of the Queue.
➕ Pros
● Real time

➖ Cons
● Clients must be online.
The Client must be connected to the Server for the Push to work.
● Clients might not be able to handle the Push.
- What if the Server is pushing tons of notifications to the Client?
It has no idea whether the Client can support this load or not.This is why Kafka
didn’t implement Push like RabbitMQ but instead moved to Long-Polling.
● Requires a bidirectional protocol.
● Polling is preferred for light-weight Clients so the Client makes a Request when it
is available to receive a Response.
 🧑‍💻WebSocket Demo for Push event
⌨️Server Code:
const http = require("http");
const WebSocketServer = require("websocket").server
let connections = [];

// Create a raw http server (this will help us create the TCP which will
then pass to the websocket to do the job)
const httpserver = http.createServer()

// Pass the httpserver object to the WebSocketServer library to do all

the job, this class will override the req/res
const websocket = new WebSocketServer({"httpServer": httpserver })

// Listen on the TCP socket

httpserver.listen(8080, () => console.log("My server is listening on
port 8080"))

// When a legit websocket request comes listen to it and get the

connection .. once you get a connection thats it!
websocket.on("request", request=> {

const connection = request.accept(null, request.origin)

connection.on("message", message => {
// Someone just sent a message tell everybody that message
// Using the source port of the client's connection as the unique
identifier for the client, neat trick!
connections.forEach (c=>
c.send(`User${connection.socket.remotePort} says: ${message.utf8Data}`))
})

connections.push(connection)
// Someone just connected, tell everybody about it
connections.forEach (c=>
c.send(`User${connection.socket.remotePort} just connected.`))

})
⌨️Client code:
let ws = new WebSocket("ws://localhost:8080");
ws.onmessage = message => console.log(`Received: ${message.data}`);
ws.send("Hello! I'm client")

We will run the client code on the browser’s Console to behave as a new Client. And we
can start multiple browsers to simulate multiple Clients.

Now you can chat between the various browser sessions basically via the “ws.send”
command.
Dec 2, 2023

⚡Short Polling
Request is taking a while, I’ll check with you later

This is what people usually refer to when they are talking about Polling.

❓ Why
Because Request-Response isn’t always ideal.

For example,
A request takes a long time to process.
● Uploading a YouTube video.
The backend wants to send a notification.
● A user just logged in.
● In this case, the Request-Response model does not make much sense, whereas
the Polling or Push model works better in this case.

❓ What is Short Polling

Client sends a Request.
Server responds immediately with a handle which is a unique identifier for the backend
to recognize this Request.

Server continues to process the Request. It can do this processing at any time and in
any way, it can maintain the reference of this Request in any way too, like using a Queue,
writing to disk, storing it in-memory, etc.

Client now uses the handle to check for the Request’s status.
It will Poll for the status.

Multiple “short” Request-Response flows will constitute Polls.

(short = short duration between each Request from the Client).
So at a granular level, this model is Request-Response in nature but the overall design is
based on asynchronous execution via polling which includes Request-Response.

Server doesn’t respond when the Response is ready, instead it responds with the
Response when it is ready and the Client has polled for the Response.

➕ Pros
● Simple.
● Good for long running requests.

➖ Cons
● Too chatty
Most of the Polls are just empty responses from the Server saying that the
Response is not ready.
● Network congestion is too high.
 ● Wasted backend resources
Because even these network calls to the backend to respond to the Poll requests
from the Clients will add up and end up taking valuable time that the backend
could have used to actually process these Requests.

🧑‍💻Demo Job Status: Check if a job is done with progress %

⌨️Server Code:

Client Request (with cURL):

Submit Job: curl -X POST http://localhost:8080/submit
Example response: job:166707044819
GET Job Status: curl http://localhost:8080/checkstatus?jobId=job:166707044819
Example response: JobStatus: 40%. Repeat till the job is completed.
 ⚡Long Polling
Request is taking a while, I’ll check with you later
But talk to me only when it’s ready!

Used by Kafka when a Client wants to consume a topic, it will make a Long Polling
request.

❓ Why
Because of the cons of Short Polling.

❓ What is Long Polling

Client sends a Request.

Server responds immediately with a handle.

Server continues to process the Request.

Client uses that handle to check for status.

Server DOES NOT reply until it has a Response to send to the Client.
So we have asynchronous execution with the help of the handle, we can disconnect
when needed and we are less chatty.

It is called Long Polling because the Poll request itself can take a long time to be
responded to as it's waiting for the Server to actually have a Response.
Some variations of this model also have timeouts (time for when polling should be
retried), like server timeouts, client timeouts, etc.

In the context of Kafka:

A Kafka Consumer asks its subscribed Kafka Topic on whether there are any Kafka
Messages for it to consume?
If there are no Messages, then the Topic will simply not respond.
The Push model did not work with Kafka because the Clients often were not able to
handle the load of the pushes that the Servers/Topics were making.

➕ Pros
● Less chatty and backend friendly.
● Clients can still disconnect.

➖ Cons
● Not real-time.

Why?
If the Topic got any messages in the time it took for the Client to get a Response
to its previous Poll request, then these messages would not be part of the
Response and the Client wouldn’t know about them.
 So in essence, the Client has delayed data and it is not real-time.

This is a trade-off for the Long Polling model but if real-time data is not your
priority then this is not an issue at all.

🧑‍💻Demo Job Status: Check if a job is done with progress %

Same code as before with just a few changes.
 ℹ️INFO:
We wait till the “checkJobComplete” method returns true in a while loop for the
“checkstatus” endpoint. This essentially blocks the client and creates a Long Poll
request.

“checkJobComplete” method is a promise-based method that will return false when

the job progress is less than 100 and true otherwise with the “resolve” method.

But note that the return of “false” is wrapped inside a “setTimeout” function with 1
second of time gap. This allows Node to essentially breathe and allows it to
asynchronously execute the “updateJob” method as well to update the progress.

Interesting behaviour of Node you can learn through this. If you don’t give that 1
second time gap, then “checkJobComplete” will immediately resolve to false and
proceed with the next iteration of the while loop in the “checkstatus” endpoint
which will again call “checkJobComplete” which will immediately resolve to false
and this continues on and the Event Loop never becomes free to execute the
“updateJob” method which will increase the Job’s progress.

When you run this code and the cURL requests, you will find that upon sending a Poll
request to the “checkstatus” endpoint, the terminal (Client) basically blocks and will now
wait till the Server has responded to the “checkstatus” request.
Dec 3, 2023
⚡Server Sent Events
One Request, a very, very long response
(The response doesn’t really have an end)

❓ Why
Because Request-Response isn’t ideal for notifications from the backend.
Client wants real-time notifications from the backend about certain events.

Push can be used here but it is restrictive because the Client has to be able to handle the load
that’s being pushed by the Client.

Server Sent Events work with HTTP and the Request-Response model as well.

❓ What is Server Sent Events

(A Response has a start and an end)

Client sends a Request (it is a special kind of Request).

Server sends logical events as part of the Response.

The Client would have a parser that understands any Server Sent Event as an object-type.

Server never writes the end of the Response.

For any event like a user logging in, the Server would send a Server Sent Event as a Response to
the Client, the Client would know that a Response has been received and parse it accordingly to
understand what is the event that has occurred. This same thing repeats for any event that is
required to be notified to the Client.

It is still a Request but it is an unending Response.

The Server is writing an Event and sending it but it is not ending the Response.

➕ Pros
● Real-time.
The moment the Server gets the information, it sends the Event to the Client.
● Compatible with Request-Response (HTTP)

➖ Cons
● Clients must be online.
● Clients might not be able to handle the Events being sent.
As can be seen, these are the same cons as with the Push model.
● Polling is preferred for light-weight clients.
● HTTP/1.1 problem (to be discussed more in HTTP section)
○ Can only establish 6 TCP connections with most browsers.
○ In HTTP/1.1, you can only send one Request per connection. So if you
have an active connection in which the Server is responding to and the
Request-Response has not ended yet, then no more Requests can be sent
from the Client on this connection.
 ○ Then how can you get your other data from the Server like the HTML, CSS,
etc. For that you have the remaining 5 connections. But what if all 6
connections are being used for SSE? Then no more connections can be
made to any domain as there are no connections left.
○ The only workaround for this is using HTTP 2 which allows you to stream
data in the same connection.

🧑‍💻Demo Job Status: Check if a job is done with progress %

⌨️Server Code:

⌨️Client Code:
ℹ️INFO:
The Client requests for the SSE on the “stream” endpoint.

Notice that the “stream” endpoint handler, sets the “Content-Type” Response
Header to “text/event-stream” and sends a “message” using the “send” method
without actually ending the Request.

The “send” method sends a new message every second to the Client and the Client
is configured to execute “console.log” whenever a message is received on this
Request.

Every Response body that is sent by the Server on this Request would be interpreted
as a different stream as long as you start the Response body with the word “data:
” and end with “\n\n”.

How is this Response parsed? The browser does this for you.
For this you just have to wrap the Request at the Client-side in an “EventSource”
object as shown above. This will allow you to capture each message that is received
on this “stream” via the “onmessage” event handler.

When you execute the Client code you will see that the entire thing is one Response
and there is an “EventStream” tab in the Network view for this Request where all
the streams of this Response are visible.
 ⚡Publish Subscribe
One Publisher, Many Readers

❓ Why
Because the Request-Response model is really bad for Requests that will take a long time to
respond to and in this case, we are talking about situations where the Request has to be
processed by multiple Services before the Response can be generated.

For example, uploading a YouTube video with basic Request-Response model:

Here the Client has to wait while the uploaded video goes through the Upload Service,
the Compress Service, the Format Service, the Notification Service before the final
confirmation is received and till then the Client is blocked.

If at any point, one of the Services fails, then the entire flow is lost and all data is
essentially lost, which is really bad!
❓ What is Pub-Sub

The Client uploads a video to the Upload Service. The Upload Service then returns a
response to the Client that the video has been uploaded, so the Client is immediately
unblocked.

The Upload Service then saves the raw mp4 video to the corresponding Message Queue
(also called Broker) topic/channel.

Because something is now available on the “Raw mp4 Video” topic, the Compress
Service which is a Consumer/Subscriber to this topic/channel can consume it and
process it further to generate the Compressed video which it adds to the “Compressed
video” channel.

From here it is picked up by the Format Service channel which then generates videos of
different quality formats and sends them to the corresponding video quality topic from
where they can be further processed by other Services and sent to other topics before
the final output is ready which is done via the Notification Service which in.

How the “picking up” of a Message from a Topic/Channel is done is up to the

implementation of the backend system. It can be a Push which is what RabbitMQ does
or it can be a Long Poll which is what Kafka does (here, “Compress Service” will Long
Poll the “Raw mp4 Videos” Topic) or it can be something else entirely too.

In the above example, the Upload Service was purely a Publisher, whereas the Compress
Service, the Format Service are both Publishers as well as Subscribers and the
Notification Service is purely a Subscriber.

➕ Pros
● Scales with multiple receivers.
● Great for microservices.
● Loose coupling.
● Works even while clients are not running (so the upload is completed even when
the Client has closed the browser tab).

➖ Cons
● Message delivery issues (Two Generals Problem)
○ How do you know that the Subscriber actually got the message?
○ What if it got the message twice?
○ Very difficult problem to solve.
● Complexity added because of the broker infrastructure.
● Network congestion
○ Atleast in the case of Polling as multiple Subscribers keep on Polling the
subscribed channels.

🧑‍💻RabbitMQ Demo: Publish a job and have a worker pick them up

Create a cloud RabbitMQ instance using CloudAMQP (Advanced Message Queue
Protocol).

Get the AMQP URL of the instance and insert it in the code for the “amqpServer”
variable.
⌨️Publisher Code:

ℹ️INFO:
Replace “amqpServer” with the cloud instance’s AMQP URL.
What the code does:
- Create a connection to the server, then create a channel.
- Assert that a queue in this channel called “jobs” exists.
- Send to the “jobs” queue, a stringified JSON of the specified “msg” variable
which takes in the first command line argument, i.e. “process.argv[2]” and
puts it in the JSON as:
{
“number”: process.argv[2]
}
 - Close the channel and the connection.

Run the Publisher code as:

node publisher.js 107

Response: Job sent successfully 107

Can see the Dashboard of the Queue on the CloudAMQP instance where a new
publish would be visible.

⌨️Subscriber Code:
ℹ️INFO:
Replace “amqpServer” with the cloud instance’s AMQP URL.
What the code does:
- Create a connection to the server, then create a channel.
- Assert that a queue in this channel called “jobs” exists.
- Write a consumer method for the “jobs” queue that will parse the input
message into a JSON and print out the “number” field of the input JSON that
was sent.
- If the input number is 7 then send an ACK message on this channel, which
would indicate to the channel that this message has been consumed and it
should be dequeued from the channel.
- Do not close the channel or the connection.

Run the Subscriber code as:

node consumer.js

Response:
Waiting for messages…
Received job with input 107

Note: Because we are not acknowledging this message, it never gets dequeued from
the queue, so while a message is not being consumed by a Consumer and you
connect to the same queue, you will still receive the old message that was
submitted to the queue, even though this message was “almost” consumed by a
previous Consumer. This is because the previous Consumer never acknowledged it.
Dec 5, 2023
⚡Multiplexing vs Demultiplexing
HTTP/2, QUIC, Connection Pool, MPTCP

Multiplexing
Merge multiple TCP requests into a single connection.

Demultiplexing
The reverse basically. Demerge the single merged TCP connection into multiple
requests that will be sent to multiple Clients.

❓Where is it used
Multiplexing with HTTP/2

When you make multiple HTTP 1.1 requests at the same time, the browser collects
these requests and sends them to the HTTP/1.1 Server using different
channels/connections.

We have discussed this before too with Server Sent Events and how there’s a limitation
of 6 connections per Client.

Whereas HTTP/2 uses multiplexing in order to send a single request to the server as
shown below:

Multiplexing HTTP/2 on the Backend

Here we have a reverse proxy like Envoy or nginx at the backend whose receiver side
(frontend) is h1 (HTTP/1.1) and its sender side (backend) is h2 (HTTP/2).
The reverse proxy will receive independent HTTP/1.1 requests and multiplex them to
send a single HTTP/2 connection to the backend server.

This allows the system to have more throughput.

But the server now has to spend extra resources and time in order to parse this
multiplexed request to open the actual requests that were received, so it requires more
backend resources.

Demultiplexing HTTP/1.1 on the Backend

Pretty similar explanation as the previous section just in reverse.

Consequently this provides lesser throughput on the backend but also consumes a
lower amount of backend resources.

ℹ️INFO:
Having a separate connection for each Request is preferred in terms of backend
design because each connection would have its own rules, flow control, congestion
control, etc. from a networking point of view.

However you also have to consider the trade-off with the higher throughput you are
getting with the multiplexed approach. Just a thing to look out for while you are
designing your systems.
 Connection Pooling

The backend has a resource to connect to (usually it’s the database but it can be
anything else too).

We spin up an X amount of connections to this resource, 4 for the above example.

When a Request comes to the backend that requires access to this resource, it will just
allocate a free connection to the Request and let the processing complete and return
the Response.

This continues until there is a freely available connection. And in the above example, by
the time the “RED” Request comes in, there is no available connection, so it will be
blocked till one of the connections becomes free.

ℹ️INFO:
This is kind of a fancy way of implementing Demultiplexing, as a single Request gets
demultiplexed into multiple connections to the resource.

There are other benefits of doing Connection Pooling as well like the overhead of
connecting to the database (or resource) is moved to the start of the backend
server and is not done on a per Request basis, which makes DB queries much more
efficient as Connections are already established.

Similar to this, HTTP/1.1 browser basically has a Connection Pool of 6 channels,

where each channel can contain one Request. Exhaust all these channels, i.e. 6
connections and any further Request will be blocked until one of the previous
Requests get a Response.
Example: Use the Long Polling demo code to submit a job and while it's in progress
submit atleast 5 “fetch” Requests to the “checkstatus” endpoint. Now there are 6
active Requests and any further Requests will not be serviced. If you try to
submit a new job, your Request will be pending even though the “submit” endpoint
is supposed to respond immediately. This is because the Request got blocked at the
Client-side itself!
Dec 7, 2023
⚡Stateless vs Stateful
Is state stored in the backend?

Stateful
Stores state about Clients in its memory.
The backend depends on the information being there in order to function properly. This
is a very big distinction from just storing information in memory.

Stateless
Client is responsible to “transfer the state” with every Request.
The backend can store information but can also operate as intended even if the data
was lost.

ℹ️INFO:
Even a Stateless app can have an information store like a database (obviously it
will), but the point is that the application (backend) itself is independent of any
information that is stored within it (in-memory).

So the architecture overall would be stateful, however the app itself would be
stateless because you can start up a new instance of the app and it would still work.

💡Litmus Test for Stateless Backends:

Can you restart the backend during idle time while the client workflow continues to
work?

Basically this question asks the following:

While the backend is not doing any operation, can you restart the backend and all
Clients that were connected to it previously can still operate the same as if nothing
has changed.

This would tell you if there is some state that the backend was relying on to
function as required.
 Stateful Backend

The user logs in to the website, the backend generates session data (ID and stuff) S1
and returns it to the user and stores the session data LOCALLY.

The user then requests for their profile with the cookie S1 attached to the Request. The
backend on receiving this Request, checks if S1 is in memory so that it authenticates
the user’s credentials and returns their profile data in the Response. Key point is that
session data is fetched from the backend itself and not the database.

Stateful Backend breaks in functionality:

Now that the backend has restarted and the sessions are empty, the user on requesting
their profile would just be redirected to the login page again.
ℹ️State & Replication
Sometimes you may still be able to view the profile page because in practice,
applications are replicated in order to ensure scalability and all that good stuff.

So it is possible that the backend instance that got restarted and lost its session
data is not the same backend instance where your new profile Request went to and
this instance still has the user’s session data and the login is still valid.

This redirection to backend instances is handled by a Load Balancer.

However in the world of system design, this might not always be a good thing as you
would need your instances to be consistent with each other, which is called
“stickiness”, so you would need sticky sessions and sticky load balancing in order to
maintain this consistency.

Stateless Backend

I think this image does a good job of summarizing everything.

 ⚡Stateless vs Stateful Protocols
Protocols can be designed to store states.

TCP ▶️ Stateful
● It sends segments in sequence, whose order is stored at both the Sender and the
Receiver, i.e. state.
● There’s state machines that reside at both Sender and Receiver that ensures the
sequence of segments is maintained for the transfer.
● There’s also plenty of other states involved here that’s required for flow control,
congestion control, etc.
● If any of these states are lost then you are essentially back to square one and
reset the connection.

UDP ▶️ Stateless
It stores information but it doesn’t necessarily have to be there.

DNS and UDP

When a DNS Client sends a UDP datagram, you don’t send it on a connection (because

✅
UDP is connectionless), you simply have the source and destination IP address and the
local port of the Client where the Response should be received and just do it

When the DNS Server responds back how does the DNS Client know what the datagram
represents?
It also sends the queryID in the UDP Request to the port on the DNS Client that sent it
so that the DNS Client can identify what this datagram is. And from here, the OS can
use the destination port of the request to map it to the application that’s the source of
the Request.

If the application actually dies, then there’s no one to accept this datagram at the
destination port in the DNS Client. So you can argue that the DNS Client is stateful,
however the protocol being used, UDP, is independent of this, so it is stateless.

QUIC and UDP

QUIC (Quick UDP Internet Connections) operates on connections just like TCP does but
the communication protocol it uses is UDP which is stateless.
So with each Request, each Client sends the connectionID in order to identify which
connection this datagram belongs to.

ℹ️Protocol Stack??
You can build a stateless protocol on top of a stateful one and vice-versa.

For example, HTTP is built on top of TCP.

HTTP is stateless, Clients send Requests, Servers send Responses, Clients receive the
Responses. If a Server dies and a new instance is back up, HTTP does not change its
behaviour and continues to function as is, a hallmark of stateless behaviour!

If the TCP connection breaks, then HTTP blindly creates another one.

But we do need to transfer the state which is why we actually have to use cookies in
the first place!

This is also what the phrase “transfer the state” means. We have states that we
want to store, but as per design decisions and tradeoffs, we set some aspects to be
stateless and some to be stateful, so if a stateless component is involved then the
Client that’s using it has to “transfer the state” to another component which is
stateful.

Complete Stateless System

● Incredibly rare.
● This means that state has to be carried with every Request.
● You need a backend service that relies completely on the input of the Request,
there is no dependence on any other piece of information or another service as
well.
○ Example: a system to check if an input parameter is a prime number.
● JWT (JSON Web Token) is completely stateless but that has its own tradeoffs:
○ What if a token gets stolen and I want to mark it as invalid?
■ There’s no persistent storage for the token and because of which
that token can be used till whenever it expires by any actor.
■ That’s why JWT has the concept of refresh tokens and access
tokens and all the principles associated with them, like longer
lifespan refresh tokens, shorter lifespan access tokens and so on.
■ This also has problems like what if someone stole the refresh
token?
■ That’s why you also need to have TLS in your system.
Dec 8, 2023
⚡Sidecar Pattern
Thick Clients, Thicker Backends

💡Every protocol requires a library

For example, when you use HTTP in Python, you use the HTTP library and so on and so
forth for every other protocol even if it is built into the language.

When a Client sends an HTTP request to the Server, the HTTP library at the Client-side
parses the Request into HTTP format and sends it across the network.

When it is received by the Server, the HTTP library at the Server-side kicks in and parses
the HTTP format into a format that can be used by the Server for processing (probably
JSON).
Another example of the same:

ℹ️INFO:
As you can see in the TLS example, the libraries at the Client and Server-sides do
not always have to be the same.

🎯Changing the library is hard

Once you use a library, your app is entrenched with that library.

An app and all its libraries “should” be the same language.

Doesn’t necessarily have to be the same because you can have intermediate API
packages to use libraries from other languages but most of the time you will be using
libraries from the same language.

Changing the library also requires testing.

There could be breaking changes, you have to maintain backwards compatibility.

Adding features to the library is hard.
Because of all this, microservices suffer.

💡What if we delegate communication?

What if we let a proxy which has the rich library in it take up the task of communicating
with the application?
In this case, the Client has a thin library that never changes and is only responsible for
talking to the proxy.
For example, the Client will only have the HTTP/1.1 library (h1) installed.

The proxy can choose whatever library, language, etc. it wants to communicate with the
actual backend/application.

This is where the sidecar design pattern comes into the picture!

⚡Sidecar Design Pattern

ℹ️INFO:
For this discussion,
Client🟰 Frontend application
Server🟰 Backend application
Basically both components contain business logic.

Basic Principle: All HTTP requests received at the Client and Server would be redirected
to the Client and Server sidecar proxies respectively. The proxy as well as the
Client/Server are applications running on the same machine, so they communicate with
each other via loopback (127.0.0.1).
Basically Client talks with the Sidecar proxy in HTTP/1.1, so it can be a thin Client. The
sidecar proxy has all the jazzy stuff like HTTP/2, TLS library, etc. and it communicates
with the Server.

This Request will be received by the Server sidecar Reverse Proxy which also
understands the format of Request being sent and it also knows that all requests are
actually intended to be sent to the Server which also has HTTP/1.1 only.

What you achieve here is that the communication is completely isolated from the
business logic/application!

So whoever owns the two sidecar proxies at the Client and Server ends will need to
upgrade their libraries and protocols, however the actual application or business logic
remains the same.
Another example with HTTP/3 but no change in application code:

⁉️Sidecar Examples
● Service Mesh Proxies
Examples: Linkerd, Istio, Envoy

● Sidecar Proxy Container

● Must be Layer-7 (Application Layer) Proxy

➕ Pros
● Language agnostic (polyglot)
Example: Java application can talk to Rust backend because that’s being handled
by the proxies.

● Protocol upgrade is managed separately.

● Security is also managed separately, and is consistent throughout the application

because everything will go through the proxy.
 ● Tracing and monitoring, the core reason is that everything goes through the
proxy. Can place tracing IDs on each request and you can achieve full traceability.

● Service Discovery, using a centralized DNS. All of the proxies will talk to each
other via a centralized DNS which will enable Service Discovery directly.

● Caching, the Sidecar pattern can do caching, but it has to be implemented

ofcourse.

➖ Cons
● Complexity
How to debug if something goes wrong? There’s just so many components now!

● Latency
Two more hops have been added to every single Request, they are local hops, but
Request serialization, transfer, deserialization, parsing and processing will take a
significant chunk of time when you have many, many services involved with your
application, which you usually do with microservices.
Dec 11, 2023

🎯 Protocols
What to take into account when designing a protocol?

Some of the protocol properties will be discussed. It is not an exhaustive list and what
matters in the end is the performance and whether it is applicable for your use-case.

❓What is a protocol
A system that allows two parties to communicate.

A protocol is designed with a set of properties/rules that are to be followed by both

parties.

These rules are designed based on the purpose of the protocol.

Example: TCP, UDP, HTTP, gRPC, FTP, etc.

⚡Protocol Properties
★ Data Format
○ Text-based
■ Plain-text
■ JSON
■ XML
○ Binary
■ protobuf
■ RESP (Redis Serialization Protocol)
■ h2
■ h3
★ Transfer Mode
○ Message-based: Discrete messages will be sent. Each message has a
start and an end. If the message is too large then it will be broken into IP
packets.
 ■ UDP
■ HTTP
○ Stream: Stream of data will be sent instead of discrete messages.
■ TCP
■ WebRTC
○ This is why TCP built on top of HTTP adds an overhead for
communication. The Client and Server both have to read the TCP stream
which is just a collection of bytes and figure out where each HTTP
message starts and ends and also why new alternatives to TCP are being
proposed.
★ Addressing System
○ DNS Name (Layer-7)
○ IP (Layer-3)
○ MAC (Layer-2)
★ Directionality
○ Bidirectional (TCP)
○ Unidirectional (HTTP)
○ Full/Half Duplex
★ State
○ Stateful
■ TCP
■ gRPC
■ Apache Thrift
○ Stateless
■ UDP
■ HTTP
★ Routing
○ Proxies
○ Gateways
Closely related to how addressing is handled, whether proxies and gateways are
involved in the transfer of information from source to destination.
★ Flow & Congestion Control
○ TCP (has both)
○ UDP (no control)
★ Error Management
○ Error codes
○ Retries, timeouts
Dec 12, 2023
⚡OSI Model
Open Systems Interconnection Model

❓Why do we need a communication model

Agnostic Applications
● Without a standard model, your application must have knowledge of the
underlying network medium.
● This means that for each underlying network medium you have to author
different versions of your apps so that it works on WiFi vs Ethernet vs LTE vs
Fiber because there’s no standard model for handling the network medium.

Network Equipment Management

● Without a standard model, upgrading network equipment becomes difficult.
● Because of no standard model, the network equipment (like routers) is highly
coupled to the network medium and isolation is absent in general.
● So if you try to upgrade the network equipment, then chances are high that
something else will break and that will have to be upgraded as well and so on.
● However with the presence of standardization, the equipment can simply be
upgraded without worry for the medium assuming that the upgraded equipment
is still supported for these mediums.

Decoupled Innovation
● Innovations can be done in each layer separately without affecting the rest of the
model.
● The layers of the OSI Model are designed with clear answers for “Separation of
Concerns” and “Principle of Cohesion”.
● So although there’s a clear sequence between each layer like the Physical Layer
is followed by the Data Link Layer, there’s separation between the two, such that
changes in one layer will not affect the other and this allows for decoupled
innovation.

❓What is the OSI Model

7 Layers: Each Layer specifies a specific networking component.

1️⃣Layer 7: Application - HTTP/FTP/gRPC

Where we (backend engineers) work.

2️⃣Layer 6: Presentation - Encoding, Serialization

Where the requests and responses get encoded, serialized, deserialized, decoded.

3️⃣Layer 5: Session - Connection establishment, TLS

4️⃣Layer 4: Transport - UDP/TCP

● The next important thing for backend engineers after Layer 7.
● We configure our application to be a Layer 4 app, it means that we are aware of
what protocols are being used at Layer 4, how these protocols communicate and
so on.
● Visibility into ports, source as well as destination ports.
● Information is sent as segments in TCP and datagrams in UDP.

5️⃣Layer 3: Network - IP
● A packet that’s destined for an IP address.
● There’s no idea of ports here, just destination and source IP addresses.

6️⃣Layer 2 - Data Link - Frames, MAC Address, Ethernet

● We send frames to MAC addresses over a network medium like Ethernet, WiFi,
etc.

7️⃣Layer 1 - Physical - Electric Signals, Fiber or Radio Waves

Read the example below for a complete understanding!

💡OSI Layers: Example

Example sending a POST request to an HTTPS webpage

1. Sender’s Perspective

Layer 7: Application
A POST Request with JSON data is to be sent to the HTTPS server.
Layer 6: Presentation
This layer serializes the JSON to flat byte strings. Because JSON is only understood by
your application, for the network it has to be bytes, so this layer does that task.

Layer 5: Session
This layer requests to establish a TCP connection and TLS (if applicable, which is yes in
this example).

Layer 4: Transport
● Sends “SYN” (Synchronize) Request segment to the target at port 443 (HTTPS).
● First “data” that is being sent is the “SYN” Request.
● This will establish a connection with the 3-Way TCP Handshaking method
(SYN-ACK-SYN).

Layer 3: Network
The “SYN” Request is placed in an IP Packet/s and it adds the source and destination IP
addresses.

Layer 2: Data Link

Each packet goes into a single frame which adds the source and destination MAC
addresses.

Layer 1: Physical
Each frame becomes a string of bits which is converted into either a radio signal (WiFi),
electric signals (Ethernet) or light (Fiber).

ℹ️NOTE from Hussein:

Take all of this with a grain of salt, it's not always cut and dry.

2. Receiver’s Perspective

Receiver computer receives the POST Request the other way round.

Layer 1: Physical
● Radio, electric or light signals are received and converted into digital bits.
● This gives you intuition into how a sender can send data via WiFi and a receiver
can receive it via Fiber, the Physical Layer shoulders that responsibility of
 converting the signal into digital bits and vice versa irrespective of the network
medium.

Layer 2: Data Link

The bits from Layer 1 are assembled into frames.

Layer 3: Network
The frames from Layer 2 are assembled into IP packets.

Layer 4: Transport
● The IP packets from Layer 3 are assembled into TCP segments.
● Deals with congestion control, flow control and retransmission in the case of
TCP.
● If the Segment is “SYN”, we don’t need to go to the further layers as we are still
just processing the connection request (we will hit the Session layer for just
establishing the connection but any further communication is not done).

Layer 5: Session
● The connection session is established or identified.
● We only arrive at this layer when necessary, i.e. the 3-way handshake is done.

Layer 6: Presentation
● Deserialize the flat byte strings back to JSON for the app to consume.
● The agnostic nature of the OSI model is present here too, the deserialization logic
can be completely different from the serialization logic.
● The serialization could have been in JavaScript and the deserialization can be in
Python or C# or whatever, the point is that it does not matter. This layer will take
responsibility for doing it!

Layer 7: Application
The application understands the JSON POST request and your backend server’s
Request Receive Event is triggered and your business logic basically kicks in.
📈Few Helpful Diagrams
ℹ️INFO:
What the above diagram is trying to illustrate:
The serialized bytes are shoved into segments, a segment is shoved into packets, a
packet is shoved into frames.

⁉️What do Routers and Switches do then

These networking devices look into the data received based on which they take routing
decisions.

Switch
● A Switch can connect different subnets together, and even connect hosts on the
same subnet.
● The Switch understands where to send a Frame based on its destination MAC
address, so it can be used to connect different subnets together and prevent
unnecessary transmission of data to unrelated subnets as the Switch will
perform the appropriate Routing.
● So a Switch operates at the Data Link layer, i.e. it is a Layer 2 device.
● It also has to read the MAC address, so there is intelligence involved here, unlike
a Hub which would just broadcast the frame to every connected node.
● Once it is done with the lookup, it will retransmit the data essentially to the next
node on the path.

Router
Can function the same as a Switch if the data is being sent through the same subnet.
But for it to actually route, it needs the IP address, i.e. it is a Layer 3 device.
Example Request where the Request goes to a Switch, then a Router before reaching
the Server:

Example Request where the Request goes to a Layer 4 Proxy/Firewall then a Layer 7
Load Balancer/CDN and then goes to the Backend Server:
 ● A Firewall blocks certain applications from sending data or blocks certain
unwanted packets to come through the network.
● So it’s clear that the Firewall would have to look at the IP addresses as well as
the ports (in order to identify applications). However, some Firewalls can even go
up to the Application layer.
● That’s why this is a Layer 4 Firewall which is called a Transparent
Firewall/Proxies.
● It’s transparent because everything up to Layer 4 is basically public, it’s never
encrypted and is basically available to everyone.
● This is where your ISPs (governments) can place firewalls (transparent) to
prevent you from accessing certain websites and applications.
● To go to Layer 3, you would have to provide the security certificate for
authenticating the data which you don’t have as an external entity, that’s why
Layer 3 onwards is not transparent.
● VPNs are Layer 3 technologies in one of their implementations, as a VPN can
place an IP packet in another IP packet in order to mask it.

ℹ️INFO:
This is not the only way for VPNs to be implemented, there’s others that are Layer 2
and so on.

● Load Balancers and CDNs

○ A Load Balancer’s basic logic boils down to routing rules.
○ If a Request is received at the “/images” path, then send the Request to
these nodes.
○ This concept of a “path” is an Application Layer concept, so that’s why all
Load Balancers and Content Delivery Networks (CDNs) are Layer 7
devices.
○ Now you can see how much extra work an LB has to do as compared to a
Firewall/Proxy.
○ A new session also has to be created between the LB and the actual
Backend Server which adds even more overhead to the time it takes.

😡Shortcomings of the OSI Model

● The OSI Model has too many layers which can be hard to comprehend.
It is very easy to argue about which layer does what.
 ● Simpler to deal with Layers 5-6-7 as a single layer called Application Layer.
TCP/IP Model is something that does exactly this.

💡TCP/IP Model
Much simpler than the OSI Model, it has just 4 layers.
1️⃣The Application Layer is composed of Layers 5, 6 and 7 of the OSI Model.
2️⃣Transport (Layer 4 of OSI)
3️⃣Internet (Layer 3 of OSI)
4️⃣Data Link (Layer 2 of OSI)

ℹ️INFO:
The Physical Layer is not officially covered in the model.
Dec 13, 2023

🎯 Internet Protocol (IP)

⚡IP Building Blocks
Understanding the IP Protocol
An IP Packet is just some data with a source IP address and a destination IP address.
No information about ports or anything from the higher layers is required.

❓What is an IP Address
● It is a Layer 3 property.
● Can be set automatically or statically.
● It has two sections: Network and Host.
● It consists of 32 bits or 4 bytes in IPv4.

Network vs Host 🟰 Subnet 🤔🤔🤔🤔🤔

● IPv4 addresses are of the format:
a.b.c.d/x
where,
■ a, b, c, d, x are integers
■ “x” 🟰
Number of network bits
■ “a”, “b”, “c”, “d” 🟰 Integer representation of the Host bits

● Example: 192.168.254.0/24
○ The first 24 bits (3 bytes) are network and the remaining 8 are for the host.
○ This means we can have 2^24 (16777216) networks and each network
can have 2^8 (255) hosts.

Subnet Mask
● “192.168.254.0/24” is also called a subnet.
● The subnet has a mask which is “255.255.255.0”
 ● Subnet mask is used to determine whether the IP address that I am about to
send a packet to is in the same subnet or not.
● Based on the response to this question, two paths open up…

Default Gateway
● Most networks consist of hosts and a Default Gateway.
● When Host A wants to talk to Host B:
○ If both are in the same subnet, then just use the MAC address for
host-to-host communication.
○ If they are not in the same subnet, then send it to someone that knows
where to route this packet to. Usually this someone is the router or a
gateway which will be there in every network.
● The Gateway has an IP address and each Host should know its Gateway.

💡Example: Host 192.168.1.3 wants to send a packet to 192.168.1.2

ℹ️Why does this work?

● The “&” symbol is the Bitwise AND operator.
● This mechanism is how a Host determines whether the destination Host is in
the same Subnet as itself.
● Analyze the Subnet Mask, byte-by-byte.
● Each “255” is 8 bits filled with just 1.
● If you logically AND “A” with 1, you get “A” as the output.
 ● So in this case, the first 3 bytes are returned as is.
● The last byte of the Mask is just 0.
● If you logically “AND” “A” with 0, you get 0 as the output.
● So in this case, the last byte will be 0 regardless of the input.

Conclusion:
A&1 🟰A
A&0 🟰0
a.b.c.d & 255.255.255.0 = a.b.c.0

💡Example: Host 192.168.1.3 wants to send a packet to 192.168.2.2

Once the packet is sent to the Default Gateway, it again becomes the same as the
previous scenario.

Now the packet is sent from the Gateway router to the destination machine. The source
MAC address has changed from the source machine’s to that of the Gateway router.

This is where every possible attack happens.

ℹ️ARP Poisoning
What if a machine pretends to be a machine that is part of a subnet’s Gateway
router?
In the context of the above example, what if a 4th machine with some IP address is
able to successfully send packets to the Gateway router with IP address,
192.168.1.100 without actually being part of the subnet.

How does the Gateway router become able to send the packet to the destination
machine even though it’s part of a different subnet?
Because it has two IP Addresses, one for each subnet!
In the above example, when machines from subnet A send packets to it, it receives them
with IP address 192.168.1.100 and when it is sending packets to subnet B, it sends
them with IP address 192.168.2.100.
This way it is a part of both subnets!

ℹ️ Backend and DB in different subnets

What would happen in this case?
Each DB query would go through the Gateway Router of the Backend application.
This means that there’s a possibility of congestion at the Router, as it gets too many
requests and cannot send the packets out to the DB subnet immediately.

This is why you may end up with scenarios where your DB query from the backend
takes so much time to just reach the DB.

This also gives you insight into how networking comes into play when designing a
backend system.

A possible fix for such a scenario would be to place a high performance Switch
between the two instead of a Router but that’s more of a network engineering
thing. However just knowing the basics of this would help you interface with your
networking team better!
Dec 15, 2023

⚡The IP Packet
Anatomy of the IP Packet
The IP Packet has two sections, “Header” and “Data”.
The “Header” section is 20 bytes long but it can go up to 60 bytes if options are enabled.
The “Data” section can go up to 65536 bytes (64KB):
- The IP Packet has 16 bits to address the “Data” section, so 2^16 bits of data are
allowed.

This 20 bytes of Headers is basically the cost of doing business because it’s not your
data, it’s just what allows communication to happen correctly.

🤡IP Packet to the Backend Engineer

This is not what it looks like but it’s convenient to look at it this way.

😔What IP Packet actually looks like

4 bytes in each row and the diagram shows how the first 5 rows are arranged in order to
give the 20 bytes of the “Header” section mentioned before.
The “IHL” field which is set to 5 by default defines how many rows the “Header” section
can contain. This parameter is set to a value beyond 5 (max 15 because 15 x 4 bytes =
60 bytes) in order to enable certain options when required.

⚡IP Header Fields

Version - 4 bits
The IP Protocol version

● 4 bits to specify the version of the IP Protocol.

● Right now we just have two versions, 4 (IPv4) or 6 (IPv6).
● Seems a bit of an overkill right now but it is provisioned so that we can go up to
2^4 = 16 versions of the IP protocol.

Internet Header Length (IHL) - 4 bits

Defines the Options length

● As mentioned earlier, the value determines how many rows would the IP Header
have where each row is 4 bytes (32 bits).
● The minimum value for this is 5 and the maximum value for this is 15.

Total Length - 16-bits

Data + Header

● Has 16 bits, 2^16 bits = 64KB of Data + Header section data can be addressed.
● Defines the total length of the IP Packet which contains both data as well as
header’s length.

Fragmentation - Jumbo Packets - 32 bits

Fields involved: Identification (16 bits), Flags (3 bits), Fragment Offset (13 bits)

By default an IP Packet should fit nicely into a single Frame.

 ℹ️Maximum Transmission Unit, MTU
The MTU is a field attached to a network connected device like a router which
defines the maximum size of a packet that it can accept.

It is configurable depending on the network medium being used and capacity of

network connected devices being used for the transmission of data.

Standard value is 1500 bytes.

Usually the MTU size configured for the system will ensure that this is the case.
But if it’s a jumbo Packet, then it is possible that an IP Packet doesn’t fit into a single
Frame.

In this case, there’s two options which can be better summarized via ChatGPT.

🤖 ChatGPT Time
Question
I know that when the IP Packet size exceeds the MTU, the packet is to be
fragmented if the "Don't Fragment" bit is not set. However, where is all this
determined? Is it in the Client that's sending the Packet, is it at some intermediate
router or is it at the destination?

Answer

The process of determining whether IP packets need to be fragmented due to

exceeding the Maximum Transmission Unit (MTU) is typically a collaborative effort
involving both the sender (client) and the routers along the path to the destination.
Let me break down the process for you:

1. Sender (Client):
● The sender generates IP packets and includes the "Don't Fragment" (DF) bit in
the IP header.
● If the DF bit is set, it indicates that the sender does not want the packet to
be fragmented.
 2. Intermediate Routers:
● As the packet travels through the network, it may encounter routers.
● Each router along the path has information about the MTU of the next hop
(the next segment of the network that the packet will traverse).
● Routers have a process called "Path MTU Discovery" to determine the smallest
MTU along the path to the destination.
● If a router determines that the packet size exceeds the MTU of the next hop,
and the DF bit is set, the router cannot fragment the packet.
● Instead, the router should send an ICMP (Internet Control Message Protocol)
"Destination Unreachable - Fragmentation Needed" message back to the
sender.

3. Sender's Reaction to ICMP Message:

● Upon receiving the ICMP message, the sender can take appropriate action.
● It might choose to reduce the size of the packets it sends, adjust the MTU
setting, or allow fragmentation.

So, in summary, it's a combination of the sender's decision to set the DF bit, the
intermediate routers' ability to determine the MTU of the next hop and generate
ICMP messages when necessary, and the sender's response to those ICMP messages.
The ultimate goal is to ensure that packets are delivered without fragmentation
when possible, and fragmentation occurs only when necessary and feasible.

Assume that we ended up with Fragmentation, now the Fragmentation might not
necessarily be received in order at the Receiver.

Now the Receiver has to assemble these Fragments back together and this is one of the
most dangerous things security-wise because people can fake fragmentation and the
fragmentation will incur cost at the receiver’s end as well.
There’s also other problems that have to be handled like Fragment loss, retransmission
of lost Fragments and so on.

Because of all this, most often fragmentation is just skipped and the client sends an IP
Packet/s that is compatible with the MTU size that is discovered with “Path MTU
Discovery”.

Now onto the actual parts of the packet:

● Identification: ID of the Fragment
 ● Flags: Consists of 3 bits which specify the following flags:
○ First bit is Reserved.
○ Don’t Fragment
Whether any intermediate Router can fragment this flag if the MTU size is
exceeded by the Packet.
○ More Fragments
■ The last Fragment will have this set to 0 and all other Fragments
will have this set to 1.
■ Used to indicate whether any more Fragments are left to be sent or
not.
● Fragmentation Offset
○ It is used to indicate the starting position of the data in the Fragment in
relation to the start of the data in the original Packet.
○ This information is used to reassemble the data from all the Fragments
(whether they arrive in order or not).

🧑‍💻Reference: https://packetpushers.net/ip-fragmentation-in-detail/
ℹ️ Fragments vs Frames?
Really silly question that I wanted to clear up because I got lost as well.

Fragmentation happens at the Network Layer and a Fragment is a fragment of an IP

Packet, so it too is an IP Packet at the end of the day.

A Frame is what the Data Link Layer uses to address the data being sent/received
and it contains the Source and Destination MAC addresses. So assuming
fragmentation has taken place, each IP Packet Fragment would be placed in a
Frame at the Data Link Layer before being transmitted via the Physical Layer.

Time To Live (TTL) - 8 bits

How many hops can this Packet survive?

Why do we need this?

● Because Packets can take different routes to end up at the same destination.
● But in doing so, it is possible that they may end up in an infinite loop and there’s
no way of tracking this because IP protocol is not stateful.
 ● This is where TTL comes in.

It is a single byte that is a counter.

When a Packet is sent by the Client, the Client estimates how many hops this Packet
should take to reach the destination and fills this value in the TTL field of the Packet.

For each new Router that this Packet visits, the Router will subtract 1 from the TTL
field value.
If a Packet reaches a Router and the TTL value is 0, then the Router will drop the
Packet and we incur Packet loss which will be informed to the Client via an ICMP
message of type “Destination Unreachable” (read further for more explanation on
ICMP).

You can see how this is the trick being used by IP to “transfer the state” as discussed
before.
It explicitly doesn’t store a state but the IP Packet bypasses this restriction by doing the
subtraction logic above.

Normally the default value set for TTL is more than enough for regular transmission but
sometimes some issues can happen in which case the Client will increase the TTL’s
value.

Protocol - 8 bits
What protocol is inside the data section?

Why do we need this?

Why can’t we just send random data and figure out the protocol from the data’s
contents itself?
● Because this method is way more efficient.
● In this method, each Router does not have to parse the Data in order to figure out
the protocol and infer it from the Header itself which is just 20 bytes.
● Based on the Protocol, the Router can decide whether to pull/read the Data or
not.
● This method also allows Protocol-wise blocking to be enabled at the granularity
level of a Router which is also very helpful.
🧑‍💻List of IP Supported Protocol Numbers:
https://en.wikipedia.org/wiki/List_of_IP_protocol_numbers

Source & Destination IP Addresses - 32 bits each

● The most important pieces of metadata in a packet.

● Where are you coming from and where are you going?

Spoofing IP Addresses
● Your source IP address can be spoofed, i.e. every Packet that goes out from your
machine will have its source IP address changed to something else.
● But this is not as simple as it sounds because the first Router that each Packet
will hit is your ISP’s Router and they can detect this suspicious behaviour.
● They can also reassign the source IP address to the original value because they
are the ones that gave you your IP address in the first place.
● This is why spoofing is not an easy task to do.
● And even if you spoof how do you receive responses back? Because the source
IP address being sent in the Packets is not your actual IP address, so that’s
another hurdle.

ECN - Explicit Congestion Notification - 2 bits

ℹ️Congestion & Packet Loss

Simple explanation of what congestion means:
IP Packets start to be dropped.

❓Why does this happen

● Each Router has a Buffer that is used for storing Packets.
● At a time, one Packet is picked from this Buffer and sent towards its
destination IP address.
● If there are too many Packets being received at a Router then this Buffer fills
up and once the Buffer size has been exceeded, it would have nowhere to
store these newly received Packets.
● This results in those Packets being dropped, i.e. Packet loss.
 ● This packet loss doesn’t even result in an ICMP message being sent.
● So the Client has to figure out on its own via retransmission timers that run out if
a Packet has dropped and it hasn’t received an ACK (Acknowledgement) yet.
● This is what happens when there is congestion and it is not really ideal.
● So we need better congestion control.
● For this we have the ECN.
● When a Router’s buffer is almost filled and it receives a Packet, it sets the ECN
appropriately indicating the congestion status.
● This ECN status gets bubbled down to the Receiver.
● Now the Receiver knows that some of the Routers that the Packets took
experienced congestion, so it will inform the Client via an acknowledgment that
congestion is happening.

Just by adding 2 bits, the Client and Server both got to know of the congestion status
and can take steps to remedy this like taking alternate routes, etc.
 ⚡ICMP
Internet Control Messaging Protocol
Designed for sending control signals:
● Host unreachable, port unreachable, fragmentation needed
● Packet expired (infinite loop in routers)

Layer 3 Protocol - so only works with IP addresses, it has no meaning with port
addresses.
“Port unreachable” is a valid message because ICMP is still operating from Layer 3.

PING and traceroute use it.

Doesn’t require listeners or ports to be opened in order for an ICMP message to be

received.
● This sounds innocuous but think about it a bit.
● Everything requires a port for a message to be received on, however ICMP does
not have that restriction.
● It just requires that the receiver machine has ICMP-enabled.

ℹ️ INFO:
ICMP is also enclosed within an IP Packet, don’t forget!
Everything is just IP Packets when you are dealing with Layer 3 stuff.

“Although ICMP messages are contained within standard IP packets, ICMP

messages are usually processed as a special case, distinguished from normal IP
processing.”
- Ref: Wikipedia
 ICMP Header

It’s a 4 byte message.

1️⃣Type - 8 bits

🟰
Used for representing the signal’s type, like “Destination unreachable”, etc.
Have a possible 2^8 255 unique codes.

2️⃣Code - 8 bits
Used for representing the signal’s code like “Port unreachable”, etc.
It’s basically like the subtype of the ICMP message.

🧑‍💻ICMP Types & Codes in use:

https://www.ibm.com/docs/en/qsip/7.4?topic=applications-icmp-type-code-ids

3️⃣Checksum - 16 bits
Used for verifying whether the ICMP message is corrupted or not.

ℹ️Disabling ICMP
Some firewalls block ICMP for security reasons.
This is why PING doesn’t work everytime.

Disabling ICMP can also cause real damage with connection establishment.
 ● An example of this is that you will now have to use fragmentation for sure
because the other option with jumbo packets involved sending an ICMP
message back to the Client which isn’t possible anymore.

TraceRoute

Can you identify the entire path your IP Packet takes?

Clever use of TTL.

How does it work?

● When TTL becomes 0 but the message is undelivered, ICMP delivers a
“Destination Unreachable” message back to the Client.
● The “Source IP” field in the Response IP Packet would tell you where the IP
Packet was dropped.
● This is the trick used by TraceRoute in order to determine the path that an IP
Packet takes to reach a destination.

Procedure:
● Start with TTL set to 1 and send an ICMP message out of type “Echo”. It will
make exactly one hop, be dropped and in response the Client will receive the first
Router’s IP address via ICMP message.
● Just repeat this with each value of TTL and you will essentially trace the entire
route that the Packet takes.
● When the Packet actually reaches the destination, it will reply with an ICMP
message of type “Echo Reply”.

ℹ️NOTE:
Doesn’t always work as the path changes and ICMP might be blocked.

As you can imagine, when TTL = 3, the IP packet may take one path and when it is
at 4 or some higher value, it may take another path and so on.

Because of this the results from TraceRoute are not always reliable.
It is possible that a Router may act as a Switch which is a Layer-2 device in order to
route to the next Router in the Path, this is possible when they are in the same
network, i.e. MAC addresses are known.

In this case, the TTL won’t be deducted by 1 even though the Packet has made 1
hop.

This is a LIE because it’s the Frame that has completed one hop and not the Packet,
the Packet was never decapsulated from the Frame.

Refer to the section below for more info!

🤖 ChatGPT Time
Question
Would a Switch only know the MAC addresses of the devices that are connected to it
in the same network?

Answer

Yes, that's correct. A switch operates at the data link layer (Layer 2) of the OSI
model, and it builds a MAC address table to forward frames efficiently within a local
network. A switch learns the MAC addresses of devices connected to its ports by
examining the source MAC address of incoming frames.

Here's how it works:

1. Learning Phase
● When a frame enters a switch port, the switch looks at the source MAC
address of the frame.
● It checks its MAC address table to see if it already knows which port
corresponds to that MAC address.
● If the MAC address is not in the table, the switch adds an entry associating
the MAC address with the incoming port.

2. Forwarding Phase
● When the switch needs to forward a frame to a specific device, it consults its
MAC address table.
 ● If the destination MAC address is in the table, the switch forwards the frame
only to the port where that device is connected.
● If the destination MAC address is not in the table or the entry has expired,
the switch may flood the frame to all ports (except the incoming port),
hoping to reach the correct device. Once the correct device responds, the
switch learns its MAC address and updates its table.

This process ensures that the switch builds an efficient MAC address table over
time, allowing it to forward frames only to the necessary ports and reducing
unnecessary traffic on the network.

It's important to note that a switch operates within a single broadcast domain, and
it primarily works at Layer 2. For communication between different IP subnets or
networks, a router is typically used. Routers operate at the network layer (Layer 3)
and use IP addresses for routing decisions.
Dec 27, 2023

⚡User Datagram Protocol - UDP

● Layer 4 Protocol (Transport)
Built on top of IP

● Ability to address processes in a host using ports.

Multiple requests can be sent to the same IP address for different applications
which are running on different ports.

● Simple protocol to send and receive data.

No management or congestion control or anything like that has to be
implemented which makes it simple.

● Prior communication is not required but this is a double-edged sword obviously.

● Stateless, as no knowledge is stored on the host.

A datagram is sent from source to destination and that’s that, no information is
stored on the source.

● 8-byte header Datagram.

Which is quite small. Recall that the IP header is atleast 20 bytes large.

UDP Use Cases

1. Video Streaming
Don’t care if some of the video frames are lost as it will only marginally lower the quality
of the video. This is because UDP doesn’t guarantee delivery.

2. VPN
What a VPN does is it takes your IP Packet and places it inside another IP Packet, the
first IP Packet will be the only thing your ISP sees and the second IP Packet will contain
the actual request you are sending.

All IP Packets will go to the VPN server which will unpack it and actually do the job of
transmitting your IP packet and sending the response back to you.
If the second IP packet itself is using TCP underneath for communication and if your
connection to the VPN server, i.e. the first IP packet is also TCP then it will be too much
load and management becomes really complex and you run into something called the
TCP Meltdown.

ℹ️ TCP Meltdown
A TCP meltdown occurs when you stack one transmission protocol on top of another,
like what happens when a TCP tunnel is transporting TCP traffic inside it.

3. DNS
4. WebRTC

Multiplexing and Demultiplexing with UDP

The requests from various applications at the source are sent to a destination by
multiplexing into UDP datagrams, i.e. multiple requests in the same connection.

The UDP datagrams are demultiplexed at the destination to the corresponding

application.

Each Request will have the source and destination IP address as well as port address,
so the application that sent the Request and the application that must receive the
Request are also identified.

Example:
App1 running on port 5555 at 10.0.0.1 wants to communicate with AppX running on
port 53 at 10.0.0.2
 ⚡UDP Datagram
UDP Header is 8 bytes.

It is placed into an IP packet in the “Data” field.

Ports are 16 bit (0 to 65535).

ℹ️ Max addressable processes

If you spin up more than 65535 addressable processes (the actual number is lesser
than this because some addresses are reserved) on your machine, then you won't be
able to establish new UDP (or TCP) connections because there are no more unique
port numbers to assign.

Anatomy of UDP Datagram:

16 bits each for storing source and destination port which identifies the process that’s
the Request’s source and destination on their respective machines.

Length 🟰
Length of “Data” field.
Checksum 🟰
Checksum of entire UDP datagram, for checking corruption of any fields.

➕ Pros
● Simple protocol.
● Header size is small so the datagrams are also small.
● Uses less bandwidth.
● Stateless.
● Consumes less memory because there’s no state stored in the server/client.
 ● Low latency - but this is because there’s no handshake, ordered delivery,
retransmission or guaranteed delivery.

➖ Cons
● No acknowledgement.
● No guaranteed delivery.
● Connectionless: anyone can send data without prior knowledge.

ℹ️ UDP Flood Attack

Server has to check all UDP datagrams irrespective of the source because it stores
no information about past UDP datagrams as it is stateless.

Because of this an attacker can flood a server with UDP datagrams and this will
result in a Denial-of-Service attack (DoS).

ℹ️ DNS Flood Attack

Assuming the attacker can spoof their IP address, they’ll spam DNS queries at the
DNS Server but the source IP address will be the target’s IP address.

The DNS Server is scalable and will be able to reply to all of these. But the
responses goes to the target’s machine. The target machine’s OS has to process all
these responses even though it never sent the DNS requests as DNS uses UDP, so this
results in a DoS of the target’s IP address.

● No flow control.
● No congestion control.
● No ordered packets

● Security: can be easily spoofed.

[TCP cannot be spoofed]
🧑‍💻UDP Server in C
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <arpa/inet.h>

void main(int argc, char **argv){

if(argc != 2) {
printf("Usage: %s <port>\n", argv[0]);
exit(0);
}

int port = atoi(argv[1]);

int sockfd;
struct sockaddr_in si_me, si_other;
char buffer[1024];
socklen_t addr_size;

sockfd = socket(AF_INET, SOCK_DGRAM, 0);

memset(&si_me, '\0', sizeof(si_me));

si_me.sin_family = AF_INET;
si_me.sin_port = htons(port);
si_me.sin_addr.s_addr = inet_addr("127.0.0.1");

bind(sockfd, (struct sockaddr*)&si_me, sizeof(si_me));

addr_size = sizeof(si_other);
recvfrom(
sockfd, buffer, 1024, 0,
(struct sockaddr*)& si_other, &addr_size
);
printf("[+]Data Received: %s", buffer);
}
The key thing to notice with this code is that you “bind” the address to the port to get a
socket (which is just a file descriptor) and after this you can start receiving datagrams
on this socket. There is no concept like listening for connections like with TCP as you
will see later on because UDP has no concept of connections.
How to send a request to this socket?
./a.out 4444 [Running compiled C program with 4444 as the input which is the port]
nc -u 127.0.0.1 4444 [Netcat command tool with u flag for UDP and port as 4444]

🧑‍💻UDP Server in node.js

import dgram from 'dgram';

const socket = dgram.createSocket("udp4");

socket.bind(5500, "127.0.0.1");
socket.on("message", (msg, info) => {
console.log(`My server got a datagram ${msg}, from:
${info.address}:${info.port}`);
});

The code is obviously a lot more simpler because all the abstraction gets taken care of
by the ‘dgram’ library and you don’t have to write any looping structure to accept multiple
UDP datagrams.

The C code above will simply accept a single datagram and terminate after printing the
data received. However this node.js code attaches an “on message” callback to the
socket which would keep on running till the process is terminated, so it can accept
datagrams till the program is terminated.
Jan 8, 2024

⚡Transmission Control Protocol - TCP

Alternative Layer-4 Protocol to UDP.

It has the ability to control the transmission unlike UDP which sprays out datagrams like
a firehose, this is dangerous as it can be uncontrolled and this is also one of the reasons
some firewalls actually block UDP datagrams.

It introduces the concept of connections which is analogous to a session which means

that there’s knowledge shared between sender and receiver, i.e. state which is stored on
both the client and the server. To build this state/connection, we require a handshake
between the two parties.

For all this management it requires 20 bytes of headers which can go up to 60 bytes. So
this means we now have a minimum of 20 bytes for the IP header and 20 bytes for the
TCP header, so 40 bytes has to be spent per request atleast just to send what is
essentially metadata. Worst case, you can have 60+60=120 bytes of headers.

TCP Use Cases

1. Reliable communication
Chatting applications make sense here, whereas video streaming applications don’t
need to be reliable.

2. Remote shell
3. Database connections
Obviously you don’t want your DB queries or your remote sessions to be unreliable or
tampered with.

4. Web communications
5. Any bidirectional communication
 ⚡TCP Connection
Connection is a Layer-5 concept (Session).

It is an agreement between the client and the server which is performed via a
handshake. A connection must be created in order to send data.

It is because of this handshake and this policy of dropping packets from

“un-handshaked” sources, that TCP cannot be spoofed.

A connection is identified by 4 properties:

● Source IP & Source Port
● Destination IP & Destination Port

These 4 properties are hashed by the OS (client and server both) and stored in a hash
table called the TCP Connection table as the key. This hash key which is sometimes
called the socket or file descriptor will now be used to store session data in the table
that the OS will maintain of all TCP connections, do note that this is in heavy overlap
with the Session Layer, i.e. Layer 5. This is also the way the OS determines whether a
connection exists or not, just look for it in the connection table. The TCP Connection
table is stored in-memory. So the more the number of TCP connections on a machine,
the more memory will be used in order to maintain the TCP Connection table.

It requires a 3-way handshake which will be explained later.

Segments are sequenced and are sent in order.

Segments have to be acknowledged and lost segments are retransmitted which is
determined with the help of ordering and sequencing.

The concept of retransmission is what makes TCP reliable.

Multiplexing and Demultiplexing with TCP

Pretty similar to how UDP works with multiplexing and demultiplexing.

The Sender multiplexes all its apps into TCP connections.

The Receiver demultiplexes the TCP segments to their corresponding app based on
connection pairs.

Sometimes apps can close or shut down without cleanly closing the TCP connection
which can cause leakage and which can eventually cause problems as with all leakages.

1️⃣Connection Establishment ⏩ TCP 3-Way Handshake

App1 on 10.0.0.1 wants to send data to AppX on 10.0.0.2

App1 sends “SYN” to AppX to synchronize its sequence numbers, i.e. where does my
sequence numbers start from and associated metadata.
AppX sends “SYN/ACK” (basically, “SYN” + “ACK”) to synchronize its sequence
numbers.

App1 sends “ACK” to AppX’s “SYN”

This is the three-way handshake for TCP.

As you can see, the source IP + port and destination IP + port becomes the file
descriptor for the connection.

Sender sends the “SYN” IP packet first, then it receives a “SYN/ACK” packet in response
and then Sender sends an “ACK” packet to the Receiver to acknowledge the Receiver’s
“SYN”.

Fun fact: WhatsApp has reached atleast 3 million concurrent TCP connections.

The bottleneck in this system in terms of scalability will be the CPU cycles spent to
compute the hash for each request (source IP + port, destination IP + port has to be
hashed each time to perform any look-up) and the memory of the machine because the
TCP connection table is stored in-memory, so in the end as connection numbers grow,
more and more memory gets used.

2️⃣Sending Data

App1 sends data to AppX.

App1 encapsulates the data in a segment and sends it.
AppX acknowledges the segment.

In this example, the data that App1 is sending is to execute the “ls” command on AppX.

ℹ️ Can App1 send a new segment before the ACK of the old segment arrives?
YES!

But there’s a limit to this which is determined by the capacity of the Receiver
which is managed via a mechanism called Flow Control and it is also determined by
the capacity of the intermediate routers via which the segments are sent which is
managed via a mechanism called Congestion Control.

3️⃣Acknowledgement

App1 sends segments 1, 2 and 3 to AppX.

AppX acknowledges all of them with a single “ACK3”.
ℹ️ INFO
You can acknowledge multiple messages with a single ACK response!

ACK n, where n = number of requests being acknowledged

4️⃣Lost Data
App1 sends segments 1, 2 and 3 to AppX
Segment 3 is lost, AppX acknowledges only 2 segments
App1 resends Segment 3
AppX acknowledges all 3 segments now
 5️⃣Closing Connection - 4-Way Handshake
App1 wants to close the connection.
App1 sends FIN, AppX ACK’s it.
AppX sends FIN, App1 ACK’s it.
Four-way handshake.

ℹ️Potential Attack
If I know the source IP + port and destination IP + port of a TCP connection and the
last sequence number (guessing works I suppose), then I can generate the hash
key for the connection. Using this hash key, I can send a FIN request (along with the
last sequence number) to one of the two machines in order to close the connection
even though I don’t own either of the machines.

Once the handshake is complete, the file descriptor is fully cleared from the Receiver of
the first FIN (AppX’s side), whereas it is still retained in the Sender of the first FIN
(App1’s side) in a time-wait state.

This is done because it is possible that there could be segments being transmitted from
AppX which could be received eventually. So if the file descriptor gets cleared at App1’s
side and then some segments are received, then those segments would be dropped as
there is no hash key in the TCP connection table that matches these segments’ hash.
This is not really desirable, so we retain the entry at AppX’s side until the retransmission
timer (or some timer of that sort) expires.
Jan 12, 2024
⚡TCP Segment
TCP Segment header is 20 bytes and can go up to 60 bytes.
TCP Segments slide into an IP packet as “data”.
Ports are 16 bits (0 to 65535)

Anatomy of TCP Segment:

Source and Destination Port: 4 bytes (2 bytes / 16 bits for each)

Sequence Number: 1 byte

This is the Sequence Number of the Segment.
2^32 Segments are possible in a sequence of a TCP Connection.

ℹ️Running out of Sequence Numbers

Once you run out of Sequence Numbers, the Sequence Number is set back to zero
and it is treated as a new Sequence.

There are various methods in existence to handle this roundabout situation.

Acknowledgement Number: 1 byte

This would contain the Sequence Number of the Segment that you are acknowledging.
This would only be considered valid if the ACK flag is set.

ℹ️Why are Acknowledgement and Sequence Number two separate fields?

Why can’t we just have one field whose definition changes on the basis of whether
the ACK flag is set or not?

This is because a TCP endpoint can acknowledge a Segment while sending data to
the same endpoint in the same Segment, so they both are different fields.

Window Size: 2 bytes

● Flow Control Window Size.

● As a Receiver, this field is used to indicate to the Sender what is the maximum
window size of data that I can process at a time.

● 2^16 = 65kB which is quite low so there’s another way to provision atleast 1GB of
Window Size.

Flags: 9 bits

SYN: Synchronize, used in 3-way handshaking for initiating a TCP connection.

FIN: Finish, used in 4-way handshaking for closing a TCP connection.

RST: Reset, reset the connection

PSH: Push, you are pushing data in this TCP Segment.

ACK: Acknowledgement, used to acknowledge a TCP Segment that’s received (ACK

Number is used here).

URG: Urgent, the Segment is Urgent (whatever that means)

ECE: ECN-Echo
CWR: Congestion Window Reduced
These two flags are used for Congestion Control.
 ℹ️ Explicit congestion notification
TCP supports ECN using two flags in the TCP header.
The first, ECN-Echo (ECE) is used to echo back the congestion indication (i.e., signal
the sender to reduce the transmission rate).

The second, Congestion Window Reduced (CWR), to acknowledge that the

congestion-indication echoing was received.

NS: Nonce Sum

It is an experimental flag used to help protect against accidental or malicious
concealment of marked packets from the sender.

Maximum Segment Size (MSS)

● Segment Size depends on the MTU of the network.

● Usually 512 bytes can go up to 1460 bytes.

● Default MTU on the Internet is 1500 bytes.

● This results in an MSS of 1460 bytes, because of 20+20 = 40 bytes of headers of

IP + TCP.

● An MTU with Jumbo frames goes up to 9000 bytes or more.

● MSS can be larger in such cases.

🧑‍💻TCP Server in C
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <arpa/inet.h>

#define PORT 4444

void main(){

int sockfd;
struct sockaddr_in serverAddr;

int newSocket;
struct sockaddr_in newAddr;

socklen_t addr_size;
char buffer[1024];

sockfd = socket(AF_INET, SOCK_STREAM, 0);

printf("[+]Server Socket Created Sucessfully.\n");
memset(&serverAddr, '\0', sizeof(serverAddr));

serverAddr.sin_family = AF_INET;
serverAddr.sin_port = htons(PORT);
serverAddr.sin_addr.s_addr = inet_addr("127.0.0.1");

bind(sockfd, (struct sockaddr*)&serverAddr, sizeof(serverAddr));

printf("[+]Bind to Port number %d.\n", PORT);

listen(sockfd, 5);
printf("[+]Listening...\n");

newSocket = accept(sockfd, (struct sockaddr*)&newAddr,

&addr_size);

strcpy(buffer, "Hello");
send(newSocket, buffer, strlen(buffer), 0);
printf("[+]Closing the connection.\n");
}

How to send a request to this socket?

nc 127.0.0.1 4444 [ netcat command tool ]
ℹ️ SOCK_STREAM
Specifying that you want TCP, SOCK_DGRAM = UDP

ℹ️ listen(sockfd, 5);
Function that does the actual TCP handshake.

Binding is not the same as listening for TCP.

In UDP, you can bind a socket to a port to pick up any incoming datagrams on that
port.

Whereas, in TCP, you have to “listen” for connection requests on the binded socket
as well as “accept” the connection request.

The second parameter to this function is the backlog which specifies the size of the
queue of connection requests that can be held by this socket. Once a connection
request has been accepted, it goes from the queue to your application.

This introduces a new problem, what if you can’t accept the connection
requests as fast as possible and the queue becomes full?
Such requests won’t be handshaked and the sender will not receive a SYN/ACK,
because of this the retransmission logic of TCP will kick in, this is why a lot of
multi-threading, larger queue sizes and other methods are used to do this faster.

ℹ️ accept()
Accept the request that is at the front of the connection queue.

Only when you accept a request, do you get a socket that you can actually send and
receive data from.

This is the main bottleneck when it comes to TCP performance, accepting requests
as fast as possible. A lot of work goes into this.

You can loop and accept requests but then you are limited by the speed of the loop
execution. So multithreading comes into the picture here, the main thread accepts
 the connection and then sends the created socket to a new thread to complete its
work and now the main thread is free to receive a new connection.

🧑‍💻TCP Server in Node.js

import net from 'net';

const server = net.createServer( socket => {

console.log("TCP Handshake successful with " + socket.remoteAddress
+ ":" + socket.remotePort);
socket.write("Hello Client");
socket.on("data", data => {
console.log("Received data " + data.toString());
});
});

server.listen(8800, "127.0.0.1");

Look at how much simpler this is.

The callback inside the “createServer” method is only called when the TCP Handshake
is complete (from the listen method) and basically is equivalent to the “accept” method
in C.

After this, the socket can write to the Client and can also listen for specific events like
“on data” from the Client via the event handler attached which in this case is just
printing the data.

Note: Always listen on an address.

How to send a request

nc 127.0.0.1 8080
Jan 19, 2024

⚡Transport Layer Security - TLS

● It basically involves adding a layer of security on top of IP/TCP.

● TLS replaced SSL in 1999.

● Which layer does TLS belong to is debatable, but one can say it's typically in
Layer 5 (Session). But by its logic, it can be implemented on any Layer of the OSI
Model.

⚡HTTPS
HTTPS = HTTP + TLS
Runs at port 443.

● The main difference between here is that there’s a key that exists at both Sender
and Receiver and a Symmetric Key algorithm is used for encryption.
 ● A handshake that happens between the two parties is responsible for this key to
be exchanged.

● All further requests and responses are encrypted and decrypted using this key
because of which the data is safe from 3rd parties like the router that your packet
goes through and any possible intruders.

🎯Why TLS?
How does the key get to both parties?
The key exchange requires an Asymmetric Key algorithm (PKI, Public Key
Infrastructure).

ℹ️Asymmetric Key Algorithms

What’s the difference?
Asymmetric Key algorithms require two keys, one for encryption and another one
for decryption unlike Symmetric Key algorithms.

Why are Symmetric Key algorithms even used?

It looks like Asymmetric Key algorithms won’t have the issue of exchanging the key
between Sender and Receiver as they both can be separate keys.

However in exchange for this, Asymmetric Key algorithms are much slower in
comparison to Symmetric Key algorithms.

Symmetric algorithms use the XOR operator which is significantly faster than the
exponential components that Asymmetric algorithms use.

TLS also authenticates the Server, i.e. how does the Client verify that the Server is
actually who they claim to be?
This is done with the help of a Certificate that the Server responds with which verifies
their authenticity.

😔
TLS also handles Extensions like SNI, 0RTT, preshared keys, etc. which is a very vast
topic to discuss but not right now
⚡TLS 1.2 ▶️ RSA Algorithm
● For the key exchange in this, the RSA algorithm is used.
[There is an option in TLS 1.2 to use Diffie Hellman which is better and an even
better algorithm called Elliptic Curve Diffie Hellman]

● The RSA algorithm uses a Public key (coloured in red below) and a Private key
(coloured in blue below).

● The Public key will be shared by the Server and the Private key will never be
shared.

● The whole exchange between a Client & Server is illustrated below:

Once a TCP connection has been opened between a Client and the Server:

● The Client sends a TLS hello signal to the Server.

● In this request, the Client mentions that it has to encrypt incoming data to send,
for which it will use some algorithm, say RSA for the key exchange and SHA for
the data encryption.
 ● In response, the Server will send a TLS hello signal to the Client which will
contain the Server’s certificate, i.e. its Public key.

● This request from the Server instructs the Client to generate the “pre-master”
(shown in yellow colour above) or essentially the input to generate the Symmetric

🤔🤔
key to be used for the encryption and send it to the Server. But shouldn’t this be
counter-intuitive for security

● Well, there’s a brilliant trick to this:

The pre-master is encrypted using the RSA Public key and sent to the Server via a
TLS “Change cipher, Fin” signal.

● The data in this signal is encrypted and cannot be understood by a third party as
decrypting it requires the RSA Private key which only the Server has. All an
intruder can understand is that this signal contains the encrypted symmetric key
because that’s what a TLS “Change cipher, Fin” signal contains.

● Once the signal has reached the Server, it can use the RSA Private key to decrypt
the signal and get the decrypted pre-master which it can use to generate the
Symmetric key.

● Now both Client and Server have the Symmetric keys and no one in the middle
could catch hold of it!

● In response, the Server responds with a TLS “Change cipher, fin” signal.

● Now any data the Client has to send, it can first encrypt it using the Symmetric
key and then send it and the Server can decrypt it using the same Symmetric key
that it has.

And as discussed before, if only the asymmetric keys were used for data encryption,
then it would be really time consuming and that’s why symmetric keys are used.

ℹ️NOTE:
Private keys are 1 of 1. A machine has only 1 private key.

Symmetric keys are session based, they change on every new session.
 ℹ️Cons of this approach: Forward Secrecy
Say I am an ISP that has all the encrypted data exchanged between a Server and a
Client.

But what if I get the private key?

How can that happen though?
Because of bugs in the libraries used to implement cryptographic logic. Back in the
day, there was a bug in the library that used RSA because of which you could
determine where the RSA private key was loaded in the server’s memory. So
attackers exploited other vulnerabilities to leak parts of the server’s memory which
would contain the private keys.

Now I can basically decrypt every single “Change cipher, fin” message that I have
stored to obtain the Symmetric keys used for that session between the two parties,
which would mean I have decrypted every single message!

That is what Forward Secrecy means, RSA does not have Forward Secrecy, i.e. in the
future if someone knows the private key, then they can decrypt every single
message that has ever been received on that machine.

In order to counter this, private keys are also refreshed in a fixed duration of
time, say 3 months. So that even if the keys are leaked, only 3 months of data
would be leaked.

⚡Diffie Hellman Key Exchange Algorithm

Key Idea: To generate the Symmetric key, you need two Private keys and one Public key.

One private key is generated by the Client and one is generated by the Server and it is
kept with them, only the Public key is shared, note that the public key is two numbers in
this case.
To illustrate the code of this a bit better:

If you have private key x and you combine it with the public key (g,n) via the equation:
g^x % n
The resulting output is unbreakable and can be shared.

Same goes for the case if you combine private key y and the public key (g,n).
g^y % n
These can’t be easily broken no matter how much time you put into it.

If you combine all 3 keys, you get the final output:

If you are the Server, then the Client will send you, (g^x % n) and you will have y:
(g^x % n) ^ y = g^xy % n

If you are the Client, then the Server will send you, (g^y % n) and you will have x:
(g^y % n) ^ x = g^xy % n

In both cases, you get the same output which will be your Symmetric key!

⚡TLS 1.3

Client generates the combined key of (g, n, x) and sends it to the Server.

The Server which has the y will combine it with the input to get (g, n, x, y) which is the
Symmetric Key.
Now the Server combines (g, n, y) and sends it to the Client so that it too can decrypt the
message.

Now the Client can combine it with the x that it has to get (g, n, x, y) which is again the
Symmetric Key!

Now both parties finally have the key and regular Symmetric key encryption-decryption
can proceed from here!

💡TLS Summary
1. Vanilla HTTP - No security over the network

2. HTTPS - HTTP + TLS

3. TLS 1.2 Handshake

Requires two round trips: first one to determine the key-exchange algorithm to use and
the second one to actually exchange the keys.

4. Diffie Hellman
Requires a single handshake only!
Is much more secure than RSA because of there being two private keys on two different
devices.

5. TLS 1.3
Because of there being no certificate handshake, a roundtrip is avoided which makes it
faster and moreover the certificate file is significantly large in TLS anyways because of
which even more time is saved in TLS 1.3!

The number of roundtrips can even be zeroif the Server and Client have communicated
with each other in the past. This requires concepts like0RTT and pre-shared keys and
both parties agreeing to these terms.
Jan 22, 2024
⚡HTTP/1.1
HTTP Request Anatomy:

Method
GET, POST, PUT, etc.

Path
Where to send this Request to at the Receiver, also includes all path and query Request
parameters.
There’s a server-set limit for the length of the overall URL, so that has to be kept in mind
always.

Protocol
Which version of HTTP is being used.

Headers
Includes all relevant metadata about the request and the sender.

Body
Request data (GET request will have an empty body)
 ℹ️NOTE:
Host is always the domain name and not the IP address because multiple hosts can
be hosted on a single IP address but they will always need a different domain name.

HTTP Response Anatomy:

Protocol
Same as before.

Code
Response status code.
2XX OK Request series
3XX Redirection Request series
4XX Bad Request series
5XX Server Error Request series

Code Text
The status code’s meaning in text.
Basically a redundant field which is why this field is not there in HTTP 2 and above.
⚡HTTP + HTTPS Flow

open, close = Opening and closing of TCP connection because HTTP is built over TCP
after all.

Handshake = TLS Handshake which involves the key exchange of the Symmetric Key as
discussed before.
Always encrypt the requests with the Symmetric Key at the Sender’s end which is
decrypted via the same Symmetric Key at the Receiver’s end.

HTTP 1.0

For every new Request, open a new TCP connection, when a Response is received, close
the TCP connection.

No Headers are required in an HTTP 1.0 Request (this includes the host as well).

ℹ️NOTE:
An explanation for this design is that in the 90s, RAM was limited and keeping a TCP
connection alive equals consuming more memory because of the TCP connection
table. So to save on memory, the connections were closed, but it’s not a hindrance
in the modern era.
➖ Cons
● Slow
The problem with this design is the opening and closing of the TCP connection
as it overall takes more time (more handshakes).

● Buffering
The concept of a data stream is not supported here, like how Server Sent Events
function with HTTP 2.0. So a Response cannot be sent in parts and only
complete Responses can be sent.

● No multi-homed websites
Because the HOST header is optional, the Client never sets the Host header, this
means that the Client is sending an HTTP Request to an IP address, and so a
single IP address can only serve one domain’s Requests.

HTTP 1.1

● Persistent TCP Connection

○ Uses the Keep-Alive HTTP Response Header which tells the Client to keep
the Connection alive.
○ This Header was present in the HTTP 1.0 as well but it was not enforced.
 ○ This allows HTTP 1.1 to have lower latency and lower CPU usage because
of much lesser TCP connections being opened and closed.

● Proxying and Multi-homed websites

Because of the HOST field being mandatory in HTTP 1.1, the same problem as
HTTP 1.0 doesn’t exist and we can have multi-homed websites (same IP address,
different domain names) and proxies (one IP address with a domain name
pointing to another IP address).

HTTP 1.1 Pipelining

The problem with this concept is that there’s no guaranteed order with either the
Request being received or the Response being received at the Sender’s end. So how will
the Sender recognize which Response belongs to which Request.

Because of this HTTP Pipelining is disabled with HTTP 1.1 by default.

You could just assign an ID to each Request and the Server can respond with this ID
when it is sending a Request so that the Client can understand it, but it is not an HTTP
feature. This will be something that you are developing at the Application Layer.
HTTP 2
● Built by Google initially and called SPDY.

● HTTP Headers as well as the body are compressed.

HTTP Header compression was disabled in HTTP 1.1 because of a vulnerability.

● Introduced the idea of multiplexing

○ That is, sending multiple HTTP Requests at the same time on the same
connection.
○ Each Request will be tagged with a unique ID which makes this possible.

● Protocol ossification.

🤖 ChatGPT Time
Question
What does protocol ossification mean in the context of HTTP 2.0?

Answer

Protocol ossification refers to the phenomenon where a network protocol becomes

difficult to change or evolve over time due to its widespread deployment and the
strict expectations set by middleboxes (like firewalls, NATs, proxies, and various
security appliances) that inspect and often interfere with network traffic.

In the context of HTTP/2, protocol ossification is a concern because prior to

HTTP/2, the HTTP protocol had been largely text-based, and middleboxes had been
built with the assumption that they could read and sometimes modify HTTP
headers.

With the introduction of HTTP/2:

Binary Framing Layer: HTTP/2 introduced a binary framing layer that encapsulates
all messages in binary format. This makes it harder for middleboxes to inspect and
manipulate HTTP headers, as they would need to understand the new binary
protocol.

Multiplexing: HTTP/2 allows multiple requests and responses to be sent

concurrently over a single TCP connection. This is different from HTTP/1.x, where
each request/response pair would typically use a separate TCP connection or be
processed in a sequential order (HTTP/1.1 with pipelining). Middleboxes that were
designed with the older model in mind may not handle the new concurrent streams
correctly.

Encryption with TLS: As mentioned earlier, most implementations of HTTP/2 use TLS
encryption. Encryption prevents middleboxes from inspecting the content of the
data packets, thus limiting their ability to apply rules based on the packet content.

Header Compression: HTTP/2 uses HPACK for header compression, which is a

mechanism that reduces overhead but also obscures header information from
middleboxes.

Because of these changes, there was a risk that the deployment of HTTP/2 would be
hindered by middleboxes that were not updated or configured to support these new
features. Middleboxes that enforced policies based on the old HTTP/1.x behavior
could potentially block or break HTTP/2 connections.

To mitigate this, the designers of HTTP/2 included specific features to work around
potential ossification:

ALPN in TLS: The Application-Layer Protocol Negotiation (ALPN) extension in TLS

is used during the handshake to negotiate the use of HTTP/2, allowing both the
client and server to agree on using the protocol before any HTTP/2 traffic is
sent. This helps prevent issues with middleboxes that don't recognize HTTP/2.

Fallback to HTTP/1.x: If there are problems establishing an HTTP/2 connection,

clients and servers can fall back to HTTP/1.x, ensuring that communication is still
possible even if some middleboxes do not support the newer protocol.

The concept of protocol ossification is important to consider in the design of new

protocols or protocol features, as it can significantly impact the deployability and
future evolution of the protocol.

HTTP over QUIC (HTTP 3)

● Replaces TCP with QUIC (UDP with Congestion Control).
● Supports all HTTP 2 features.
● It does this without Head-of-Line blocking.
● More will be discussed later.
Jan 23, 2024
⚡WebSockets
Bidirectional communication on the web

The WebSocket protocol is designed to work within the existing HTTP infrastructure.

TCP is not suitable for bidirectional real-time communication as TCP has a lot of
overhead involved with its connections and cannot maintain low-latency and persistent
connections between two parties.

We first open a TCP connection between two parties.

We then perform a WebSocket handshake between the two.

Now the connection is “WebSocket-y” or bi-directional and is no longer following the

Request-Response model. Both parties can send messages to each other at any time in
any order.

⚡WebSocket Handshake, ws:// or wss://

wss = WebSocket Secure

First you send a GET 1.1 UPGRADE request to the Server in order to initiate the WS
Handshake.

Note that the “Upgrade” header is being used.

In response you receive a 101 Response which stands for Switching Protocols.
Once this is done, the handshake is complete and the communication protocol is now
WebSockets.
Once this is done, regular bidirectional communication can take place.

ℹ️Sec-WebSocket-Protocol
The “Sec-WebSocket-Protocol” header is used by the Client to request which
WebSocket sub-protocols to use for communication.

The values specified in the “Sec-WebSocket-Protocol” header are not fixed and can
be any string that represents a sub-protocol identifier agreed upon by the client and
server. These sub-protocol identifiers are used to select a common application-level
protocol on top of the basic WebSocket protocol.

For example in the above response header, “Sec-WebSocket-Protocol” contains only

“chat” which means that the Server will talk with the Client only on the “chat”
sub-protocol.

ℹ️Sec-WebSocket-Key
The “Sec-WebSocket-Key” header is used to ensure that only the parties that have
the relevant key can communicate with each other.

It is a base64-encoded value that the client randomly generates for each

connection. This key serves as part of a security mechanism to prevent certain
types of attacks on the WebSocket protocol, such as cache poisoning and
cross-protocol attacks.

The Sec-WebSocket-Accept header value is used in order to validate this security

mechanism.
The Sec-WebSocket-Key is not used for encryption or any ongoing security of the
WebSocket connection itself; instead, it's solely for validating the handshake
process. After the handshake, the data frames sent over the WebSocket connection
can be optionally encrypted using TLS if the WebSocket connection was established
over wss:// (WebSocket Secure).

WebSocket Use-cases
Chatting
Live feed
Multiplayer gaming
Showing client progress/logging

➕ Pros
● Full duplex (no polling).
● HTTP compatible.
● Firewall friendly because it is a standard and itself is built on top of HTTP, so it
works on port 80 itself and no way port 80 will be blocked by a firewall.

➖ Cons
● Proxying is tricky. Because you have to terminate the connection at layer 7 which
is not easy.
● Layer 7 load balancing is challenging, a WebSocket connection has to be kept
open for hours on end sometimes, but it is possible that any device on the
routing path be it intermediate routers or the device itself can fail and just drop
the connection. For this the WS standard includes PING and PONG messages
which are sent periodically to keep the connection alive.
● Stateful (HTTP wasn’t stateful), and it’s difficult to horizontally scale stateful
connections.
ℹ️How are WebSockets stateful?
1. Unique Socket Instance: Each WebSocket connection corresponds to a
unique socket instance on both the client and server sides. This allows the
server to maintain context and send messages back to the correct client at
any time.

2. Session State: The server can associate state with each WebSocket
connection, such as user identity, conversation state, or game progress. This
state persists across the messages sent over the WebSocket, enabling more
complex interactions than stateless HTTP could easily support.

3. Server Push Capability: Unlike HTTP, where the client must initiate all
requests, WebSocket servers can push unsolicited messages to clients. This
requires the server to keep track of active connections and their state to
know where and when to send messages.

ℹ️When to use WebSockets?

Rule of thumb: Do you absolutely need bidirectional communication?

Long polling, SSE are also good alternatives as they are compatible with HTTP and
have lesser overhead as they don’t require extra headers. But note that they aren’t
actually bidirectional, which is why that rule of thumb is there.

🧑‍💻Demo: Repeat: Push Model with WebSockets in Node.js

This is the same code as before but now you have the additional knowledge of what the
WebSocket layer is doing by providing a bi-directional protocol for Push to work.
Server Code:
const http = require("http");
const WebSocketServer = require("websocket").server
let connections = [];

// Create a raw http server (this will help us create the TCP
connection which will be then passed to the WebSocket to do its job)
const httpserver = http.createServer()

// Pass the httpserver object to the WebSocketServer library to take

over, this class will override the req/res from HTTP
const websocket = new WebSocketServer({ "httpServer": httpserver })

// Listen on the TCP socket

httpserver.listen(8080, () => console.log("My server is listening on
port 8080"))

// When a legit WebSocket request comes, i.e. a GET UPGRADE request,

// listen to it and get the connection ..
// once you get a connection that’s it!
websocket.on("request", request => {
const connection = request.accept(null, request.origin)
connection.on("message", message => {
// Someone just sent a message, push that message to everybody
// Using the source port of the client's connection
// as the unique identifier for the client, neat trick!
connections.forEach (c =>
c.send(`User${connection.socket.remotePort} says:
${message.utf8Data}`)
)
})

connections.push(connection)

// Someone just connected, tell everybody about it

connections.forEach (c=>
c.send(`User${connection.socket.remotePort} just connected.`)
)
})
Client Code:
Run the below code in the console of browser windows, each window equals a new
client.

// Note that the URL starts with "ws://" in the case of WebSockets
let ws = new WebSocket("ws://localhost:8080");
ws.onmessage = message => console.log(`Received: ${message.data}`);
ws.send("Hello! I'm client")
Jan 29, 2024
⚡HTTP/2
Recap: Multiple requests can be sent concurrently in the same TCP connection. Each
request has a stream ID attached to it that helps distinguish it from other requests in the
same connection.

The Server can respond in any order (TCP ofcourse requires segments to be ordered but
at layer 7 atleast, it’s unordered) and if it's a multithreaded server then it can even
respond concurrently. It all depends on the design.

➕ Pros (discussed before)

● Multiplexing over Single Connection (saves resources), no longer restricted to
just 6 Connections/Requests concurrently.
● Compression of headers as well as data (HTTP/1.1 could only compress data).
● Server Push.
● Protocol ossification.

➖ Cons
● TCP head-of-line blocking
○ When you send a request, TCP will label the segments in order.
○ This is a desired behaviour as that’s how TCP maintains ordering.
○ But in the context of multiple requests in the same stream, which HTTP/2
allows, request number 10 might not have anything to do with request
number 5, so their delivery order doesn’t matter but TCP would still order
them.
 ○ And if even one Packet from one Request got dropped or is received out
of order, then all the requests in the same connection would not be
processed by the Server because to TCP it looks like all Segments of the
same Request haven’t been received, so it must wait till all of them are
received before the Server can process them.
○ Everything at the head of the line at the Server gets blocked and that’s
what “head-of-line blocking” means.
○ Note: If Packets are received in sequence, then the Server will
continuously process each one and if a complete Request has been
received, then it will acknowledge it with a Response, even if other
Requests from the same connection haven’t been received.
● The Server Push model of sending resources never picked up steam.
○ Server pre-sends resources to the Client by making assumptions on what
they might need.
○ Lots of overhead.
○ Breaks Request-Response model of the web.
○ The Server’s assumptions were not smart enough, Client could already
have the resources cached but the Server would not account for this
before pushing the resources unnecessarily and other oversights like
these.
● High CPU usage.
○ The multiple Requests being sent concurrently in the same connection
comes at a price.
○ Where does a Request start and end when received at the Server? There
could be multiple Requests in the same Request received at the
Server-side, what about the stream IDs that have to be checked and
verified?
○ So a lot of parsing is required Server-side to deserialize a Request body
and its headers out of the stream that is received at the Server side and
that’s where we incur the high CPU usage.

Understanding these tradeoffs is the job of a backend engineer and choosing the best
protocol as per your use-cases is the key insight here.

For example, if your clients have the need of sending multiple Requests in the same
connection, only then it makes sense to use HTTP/2 over HTTP/1.1 as that will give you
a performance boost (no longer restricted to just 6 requests concurrently) in exchange
for heavier load at the backend. However if that is not the case, then why even use
HTTP/2, just go for HTTP/1.1 and incur lower backend load.
⚡HTTP/3
HTTP over QUIC Multiplexed Streams

● First QUIC was built to replace TCP and then HTTP/3 was built on top of QUIC in
order to solve HTTP/2 shortcomings.

● Like HTTP/2, QUIC uses streams as well. But QUIC uses UDP instead.

● Because QUIC uses UDP, you lose all the features that TCP provides like flow
control, congestion control, etc. However this gives you the freedom to develop
whichever features are required at a granular level, giving more flexibility and an
overall better design that’s suited for your use-case.

● QUIC is a stateful protocol that is built on top of UDP which is stateless

Example of how HTTP/3 Streams have no HOL Blocking:

If we lose packet 3 which is part of “stream2”, it would be out of order delivery for TCP,
but because QUIC uses UDP, these are QUIC datagrams and not TCP segments which
do not have ordered delivery as a requirement. Now the Server will process all
datagrams and be able to process the remaining requests when they are received which
are streams 3 and 4 while stream 2 will end up waiting till packet 3 is received.

And thusly, we have no head-of-line blocking with QUIC!

➕ Pros
● Merges connection setup and TLS into one handshake.
Why couldn’t we do this before?
○ Because TCP was built before security was ever a need.
○ So TLS was built as a separate concept and the TCP handshake had to be
independent of the TLS handshake.
○ QUIC was newly built with security in mind, which allowed this merger of
handshakes to happen.
● Has congestion control at stream level
○ If a stream is sending a lot of packets and starving other streams, then
that can be controlled with the help of this.
● Connection migration
○ Because UDP is stateless, the connection ID for a connection has to be
sent with every packet.
○ Suppose you migrated to a new connection, this can happen when you
change from WiFi to hotspot, go from 4G to 5G, etc.
○ You will have a new IP address, so all your requests will have a different
source IP, so again a handshake has to be done in order to establish a new
connection.
○ But because UDP requires you to send the connection ID with every
request, the Server would be able to recognize that this is the same Client
which has shifted to a new network interface, so it will migrate that
connection over such that your new IP address will be recognized without
any new connection setup.
○ Obviously this has a con in that a potential attacker can hijack a
connection by making use of a connection ID between two other parties.
The connection ID has to be in plaintext and not encrypted for the system
to work but this also makes it susceptible to a potential attack.

➖ Cons
● Takes a lot of CPU with the parsing logic.
● UDP could be blocked by enterprise systems/proxies, etc.
● QUIC does not allow for fragmentation because of severe security and spoofing
concerns, so the DF flag is always set in a QUIC IP Packet, but if there’s no
fragmentation then every IP packet must fit in the MTU. But then how would you
send a large QUIC datagram?
ℹ️Why wasn’t “HTTP/2 over QUIC” designed?
Because of the header compression algorithm that HTTP/2 came with, HPACK.

HPACK was designed because of the security flaws with the existing header
compression algorithm.

HPACK is static replacement logic, where a static byte is assigned to a header value
and the Client and Server essentially have a look-up table where they send this byte
in the packet (which is encrypted) and then the Receiver performs a lookup using
the byte in its lookup table to obtain the value of the actual header content.

The problem with HPACK in turn is that it relies on the ordered delivery of packets
which TCP guarantees and this is obviously not there in QUIC, so that doesn’t work.

To overcome this, QPACK was designed for HTTP/3.

Jan 30, 2024
⚡gRPC
Taking HTTP/2 to the next level

● Creating new protocols to communicate has to bring with it new client libraries
that are compatible with the new protocol’s rules. This adds a lot of headache in
order to adapt communication accordingly.

● gRPC which uses protocol buffers (protobuf) for specifying Message Format has
a single client library already made for most of the popular languages.

● It exposes APIs to the Client and uses HTTP/2 underneath but it is completely
abstracted away from the Client.

gRPC Modes

● Unary RPC (Request Response model)

● Server Streaming RPC (video streaming from Server)
● Client Streaming RPC (uploading large file to Server)
● Bidirectional Streaming RPC (both Client and Server talk to each other)

🧑‍💻Demo: To-Do App with gRPC

todo.proto
This file defines the protobuf structure used for sending a message from a gRPC Client
to a gRPC Server.

syntax = "proto3";

package todoPackage;

message voidNoParam {}
// Structure for representing a To-Do Item sent between Client &
Server
// The " = 1", " = 2" markers on each element identify the unique
field
 // number that field uses in the binary encoding.
// Field numbers 1-15 require one less byte to encode than higher
numbers
message TodoItem {
int32 id = 1;
string text = 2;
}
// Structure for containing an array of TodoItem type objects
message TodoItems {
repeated TodoItem items = 1;
}

service Todo {
// Create a new TodoItem with the Item's text as input
// For convenience sake, instead of taking text as input,
// we take in TodoItem as input (with id as -1 or as needed)
rpc createTodo(TodoItem) returns (TodoItem);
// Takes no input and returns TodoItems
// protobuf doesn't allow an rpc with no parameters to be
declared
// For this reason we have the voidNoParam message type which
// is literally empty
rpc readTodos(voidNoParam) returns (TodoItems);
// stream is a reserved keyword in protobuf which informs that
// a single item will be transmitted at a time but there will
be more
// items being sent on the same "stream"
rpc readTodosStream(voidNoParam) returns (stream TodoItem);
}

This proto file has to be compiled into the relevant language being used so that it can be
understood by the Client/Server platform, so compiling the proto file into js/cpp/etc.
Files.
server.js
@grpc/proto-loader is basically the protobuf compiler that gets you the JS files out of
the proto files that have been provided to it.
const grpc = require("grpc");
const protoLoader = require("@grpc/proto-loader")
const packageDef = protoLoader.loadSync("todo.proto", {});
const grpcObject = grpc.loadPackageDefinition(packageDef);
const todoPackage = grpcObject.todoPackage;

const server = new grpc.Server();

// Communication between Client and Server will be unsecured, i.e.

plaintext
// If you want it to be secured, use grpc.ServerCredentials.useSsl()
but
// that requires SSL certificate to be passed and so on
server.bind("0.0.0.0:40000",
grpc.ServerCredentials.createInsecure());

// Get the service from the proto file and assign it to a JSON object
which
// contains the definition of each rpc method
server.addService(todoPackage.Todo.service, {
"createTodo": createTodo,
"readTodos" : readTodos,
"readTodosStream": readTodosStream
});

// Start listening on the specified host address and port

server.start();

const todos = []

// This is how gRPC Service RPC methods are defined

// call = What the gRPC Client sent
// callback = Method that the Server can use to send back a Response
function createTodo (call, callback) {
const todoItem = {
"id": todos.length + 1,
 "text": call.request.text
}
todos.push(todoItem)
// first param = size of payload, setting it to null makes it be
// auto-calculated
callback(null, todoItem);
}

// Use the “call” object from the Client to write each item
separately back
// to the Client, at which point the “on data” event will be
triggered at
// the client-side which can be read below
function readTodosStream(call, callback) {
todos.forEach(t => call.write(t));
call.end();
}

// Send the entire "todos" list to the Client via the "callback"
method
// "readTodos" proto expects "TodoItems" type as Response, so our
response
// should also match the same format, field name is "items" and its
value
// must be the list of "TodoItem" objects
function readTodos(call, callback) {
callback(null, {"items": todos})
}

client.js
const grpc = require("grpc");
const protoLoader = require("@grpc/proto-loader")
const packageDef = protoLoader.loadSync("todo.proto", {});
const grpcObject = grpc.loadPackageDefinition(packageDef);
const todoPackage = grpcObject.todoPackage;

const text = process.argv[2];

// Create an object of Todo service
// Basically the client object is connected to the gRPC Server
const client = new todoPackage.Todo(
"localhost:40000",
grpc.credentials.createInsecure()
);

// Call "createTodo" method of the gRPC Server by passing it the

input
// that is specified in the proto file
client.createTodo({
"id": -1,
"text": text
}, (err, response) => {
console.log("Received from server " + JSON.stringify(response))
})

// Read the "todos" list from the Server and print each item
client.readTodos(null, (err, response) => {
console.log("read the todos from server " +
JSON.stringify(response))
response.items.forEach(a => console.log(a.text));
})

// While reading a stream return type, you will receive a "call"

object on
// which you can attach event handlers as done below with the "on
data"
// and "on end" event
const call = client.readTodosStream();
call.on("data", item => {
console.log("received item from server " + JSON.stringify(item))
})

call.on("end", e => console.log("server done!"))

➕ Pros
● Fast and compact - because it is built on top of HTTP/2 and is in binary.

● One client library - owned by gRPC Team and managed by the community.

● Communication modes as specified before (client/server/bidirectional

streaming).

● Cancel request with HTTP/2, not an easy thing to accomplish.

○ The Client must know the ID of the Request.
○ In HTTP/2, the Client can use the streaming ID and send this cancel
Request to the Server.
○ The Server must also be capable of understanding this cancel Request
and actually cancel the referenced Request that could already be in the
midst of being processed.
● protobuf which is a powerful format creating schemas.

➖ Cons
● Schema - a con of adding protobuf to the system because of which for even a
simple or non-relevant use-case you will have to create proto files and maintain
schemas.

● Thick Client - Because of the thick client library, there can be bugs in it which can
result in various issues, specially security issues.

● Proxies - Difficult to implement with gRPC but it has been done. nginx as a layer 7
LB proxy is used.

● Error handling - no native error handling is provided with gRPC.

● No native browser support for gRPC - because gRPC uses a lot of low-level
HTTP/2 APIs which are not necessarily exposed by the browser. This can be
implemented with the help of proxies but it is certainly not easy.

● Timeouts (pub/sub) - The underlying TCP connection can die at any time due to
timeouts and if a TCP connection dies, then all the gRPC streams that are active
on it will have to be reestablished.
Feb 5, 2024
⚡WebRTC
Web Real-Time Communication

❓ Why
Because we need a standardized method that provides an API for sharing media
between two parties with efficiency and low latency, i.e. peer-to-peer communication
API for sharing media (audio and video) with good performance.

It enables rich communication between browsers, mobile and IoT devices.

💡 Overview
1. A wants to connect to B.
2. A finds out all possible ways the Internet can connect to it.
3. B finds out all possible ways the Internet can connect to it.
4. Both parties also collect information about the media (media formats), security
(params, algos, methods, encryption keys, etc.) that they themselves
support/have along with a bunch of other information which forms their Session
Description Protocol (SDP).
5. A and B signal all the collected session information via some means.
6. Using some mechanism both A and B send their SDP data to the other party.
7. Once this is done, A can connect to B via the most optimal path without requiring
any SDP information.

ℹ️What is WebRTC doing here then?

WebRTC doesn’t provide a mechanism for the transfer of the SDP data, this can be
done in any way.

The goal is that the session data must be exchanged between the two parties.

😆
The mechanism can be anything, it can be WebSockets (standard), QR codes,
WhatsApp message ( ), etc.
WebRTC simply provides you the mechanism for continuing communication between
two parties that are already connected.

But ofcourse it does this in a more efficient and performant way as compared to the
other methods that you may use for actually establishing the connection.

What are candidates? Like A1, A2, A3, B1, B2, B3.
They are the public interfaces for A and B to communicate via the Internet.
There can be multiple candidates for a device because there can be multiple public IPs
at the same time:
A1 can be the public IP address of A.
A2 can be the NAT traversed public IP address of A.
And so on.

The three questions listed:

- How can people reach me?
- Security Parameters?
- Media Options?

These are answered by A and B both and are exchanged with the other party via some
offline communication mechanism as discussed before and not via the Internet
mechanism displayed in the diagram above.
Once the information is exchanged and the most optimal path at the time can be
identified and selected for connection establishment directly.

⚡ Network Address Translation (NAT)

Client wants to communicate with Google which is at “4.4.4.4:80” and the Client’s
private IP address is “10.0.0.2”. This is done with the help of a GET request to the
destination.

Subnet masking is done to determine whether both are in the same network.
In this example, they are not in the same network because of which communication
outside the Client’s network has to be done.
So the Client will contact the Gateway for further help by telling it to send this GET
request packet to the destination. The Gateway is the router that the Client is
connected to.

This Request is sent to the Gateway with the help of an ARP Request (more info below)
which resolves the router’s IP address to its MAC address.

The Gateway checks that the destination IP is not itself, so it has to send the packet to
the actual destination. But the source IP of this packet is the Client’s private IP address
which will not make sense when transmitted outside the network. So the Gateway which
has a public IP address (in this example, its “5.5.5.5”), replaces the source IP of the
packet to its public IP address as well as changes the source port to a new one
(selected randomly).

And while doing this, it adds a new entry to the NAT Table which looks as follows:

This basically means,

The Client with IP address 10.0.0.2 on port 8992 wants to communicate with the server
with IP address 4.4.4.4 on port 80 and the Gateway has assigned an external IP address
of 5.5.5.5 and an external port of 3333 to this Client. This table entry will enable the
Gateway to translate the external IP and external port combination on any received
packet back to the internal IP and internal port combination of the Client.
When the Server receives the Request, it will send a Response back to the Gateway as it
has no idea of the internal network.

The Gateway receives the packet and examines via the NAT Table that it is not the
intended recipient and translates the External IP and External Port address to the
Internal IP and Port addresses and sends the Response to the actual Client.

🤖 ChatGPT Time
Question
What is ARP?

Answer

The Address Resolution Protocol (ARP) is a network protocol used to map an Internet
Protocol address (IP address) to a physical machine address that is recognized in the
local network. In most cases, this physical machine address is the Media Access
Control (MAC) address, which is a unique identifier assigned to network interfaces
for communications on the physical network segment.

ARP operates within the link layer of the OSI model and is used primarily over
Ethernet. When a device on an IP network wants to communicate with another
device, it needs to send a packet to the target device's MAC address. However, IP
packets encapsulated within Ethernet frames use IP addresses to identify sending
and receiving hosts. ARP provides the necessary translation by resolving IP addresses
to MAC addresses.

Here's how ARP typically works:

1. ARP Request: When a host needs to communicate with another host on the
same local network and does not know the target host's MAC address, it
broadcasts an ARP request packet to all devices on the network. This packet
contains the sender's IP and MAC addresses, as well as the IP address of the
target host. The packet does not contain the target's MAC address, which is
what the sender is trying to find out.

2. ARP Reply: All devices on the local network segment receive the ARP
request, but only the device with the IP address that matches the target IP
address in the ARP request will respond. This device sends an ARP reply
 packet back to the sender, providing its MAC address.

3. ARP Cache: The sender receives the ARP reply and can now send the IP
packet to the target host's MAC address. Additionally, the sender will store
the IP-to-MAC address mapping in its ARP cache for a certain period to avoid
having to broadcast an ARP request for subsequent communications with the
same host.

The ARP process is critical for the proper functioning of IPv4 networks. IPv6, the
successor to IPv4, replaces ARP with a similar mechanism called Neighbor Discovery
Protocol (NDP), which operates within the ICMPv6 scope and provides several
improvements over ARP.

It's important to note that while ARP is a widely used and necessary protocol, it can
be vulnerable to certain types of attacks, such as ARP spoofing or ARP poisoning,
where an attacker sends forged ARP messages onto the network. This can lead to
the attacker intercepting data frames on the network, which is why network
security measures are important to mitigate such threats.

⚡ NAT Methods
There’s other implementation methods than the below 4, each method can also be
implemented in different ways but for basic understanding this should be enough.

Address Restricted NAT

Port-Restricted NAT
Symmetric NAT (does not work with WebRTC pretty much and WebSockets would be
better)

One-to-One NAT (Full-Cone NAT)

Packets to “external IP: external port” on the Router always map to “internal IP: internal
port” without exceptions.
With One-to-One NAT, all the incoming Requests shown above would be accepted.

Address Restricted NAT

Packets to “external IP: external port” on the router always map to “internal IP: internal
port” as long as the source address from the packet matches the table (regardless of
port).

Basically, allow a packet if we have communicated with the same host before.
In terms of the NAT table this would translate to, only allow a packet through, if its
source IP address matches one of the “Dest IP” fields in the NAT table.
Port Restricted NAT
Packets to “external IP: external port” on the router always map to “internal IP: internal
port” as long as the source address and the source from the packet matches the table.

Basically, same as Address Restricted NAT but now we also verify that the “Dest Port”
field matches along with the “Dest IP” field that we were already checking. So source IP
and source port of the packet must match a “Dest IP” and “Dest Port” pair in the NAT
table.

Symmetric NAT

Requires pair matching from both sides.

Destination IP of the packet must be equal to the External IP in the table.

Destination Port of the packet must be equal to the External Port in the table.
Source IP of the packet must be equal to the Destination IP in the table.
Source Port of the packet must be equal to the Destination Port in the table.

Only when these 4 fields match, is the packet allowed to pass through.

Because of this restricted nature, Symmetric NAT does not work with WebRTC and
WebSockets are used in this case which is a downgrade for sure. So it is preferable that
Symmetric NAT is not used in the first place.
⚡ Session Traversal Utilities for NAT (STUN)
● The basic task of STUN is “Tell me my public IP address/port through NAT”.
● This basically just involves taking Layer 4 data, sending a packet that contains
my source IP and port and then sending it back to me.

● Works for Full-Cone, Port/Address Restricted NAT.

● Doesn’t work for Symmetric NAT.

○ Because if we try to get our own IP and port via the STUN utility, then a
request would be sent with an IP and port pair.
○ But this cannot be repeated as a pair is unique for a machine, so such a
Request cannot be sent.

● STUN Server port 3478, can be run on any other port as well just like any
application.
● STUN Server port 5349 for TLS, basically secured STUN.

● Cheap to maintain, because it’s a very simple collection of utilities at the end of
the day.

STUN Workflow

STUN Request
 ● Client requests for its public IP and port via STUN (written as STN in the
diagram).
● Request reaches Gateway which replaces the source IP and port with the public
IP and port and sends it to the destination which is the STUN Server.
● At this point a new entry has been made in the NAT table of the Gateway which is
shown above.

STUN Response

● The STUN Server reads the source IP and port of the Request IP Packet and puts
that information into the data field of the response IP packet and sends this
Response to the Client.

● The Gateway receives the Response packet, does a NAT table lookup, verifies its
entry and then replaces destination IP and port with the Client’s actual internal IP
and port.

● When the Client receives the packet, it can read the packet’s data in order to
obtain its public IP and port.

● Note: This is also why STUN can be encrypted, the STUN Server will simply use
TLS while sending the Response.
Note: If either party was using address restricted NAT then this connection request
would fail. So there must exist some prior communication between the two parties
using some other mechanism as was discussed in the overview.

This shows why Symmetric NAT doesn’t work with WebRTC.

When the Client tries to establish a connection with the Server using its NAT, which the
Server obtained via STUN and exchanged with the Client via some mechanism, the
Server recognizes that the NAT is valid, however it is only valid for the source IP and port
pair that corresponds to the STUN Server and not the Client’s source IP and port pair
(because 4 fields have to match in Symmetric NAT).

So the Client’s Request gets rejected.

Feb 6, 2024
⚡ Traversal Using Relays around NAT (TURN)
● In the case of Symmetric NAT, we use TURN.
● It’s just a server that relays packets.

● TURN default server is at port 3478 and TLS-based TURN is at port 5349.

● It is relatively expensive to maintain and run.

Basically, the TURN Server is used as an intermediate packet forwarder whose sole task
is to enable communications between both parties.

Obviously, both parties will have a Symmetric NAT safe connection established with the
TURN Server so that their communication can happen without any issues.

⚡Interactive Connectivity Establishment (ICE)

● ICE collects all available candidates (local IP addresses, reflexive addresses -
STUN ones and relayed addresses - TURN ones).
○ Why are local IP addresses relevant? Because both parties can also be on
the same network in which case, why would NAT even be involved.
 ● These are called ICE candidates.

● All the collected addresses are then sent to the remote peer via SDP.

⚡Session Description Protocol (SDP)

● A format that describes ICE candidates, networking options, media options,
security options and other stuff.

● It’s not really a protocol but it’s a format.

● It is the most important concept in WebRTC.

● The goal is to take the SDP generated by a user and send it somehow to the other
party (as mentioned in the overview).

● Also note that generating the SDP is a time consuming process as the entire
network has to be scanned in order to determine the ICE candidates.
Feb 7, 2024
SDP Example

Follows the format of:

Variable Name=Variable Value
(Variable Name must be a single character)

You can also see a couple of ICE candidates in the above image.

⚡SDP Signaling
Somehow send the SDP that we just generated to the other party we wish to
communicate with.

Signaling can be done via a Tweet, QR Code, WhatsApp, WebSockets, HTTP Request. It
doesn’t matter, just somehow get that large string to the other party.
🧑‍💻WebRTC Demo
1. A wants to connect to B, for this demo, it will be two browsers, Browser A and
Browser B.

2. A creates an “offer”, it finds all ICE candidates, security options, media options
and generates the SDP.
 The “offer” is basically A’s generated SDP.

3. A will set the offer as its local description.

4. A signals the offer somehow to B (using WebSockets, etc).

5. B will set A’s offer as its remote description.

6. B creates the “answer” which is basically B’s generated SDP.

7. B sets the answer as its local description and signals the “answer” to A
somehow.

8. A sets the answer as its remote description.

9. Connection is created and the data channel is exchanged.

Browser “A” Console Code:

// This just exists on your browser by default, lc = local connection

const lc = new RTCPeerConnection();

// Create the data channel from which communication will happen

const dc = lc.createDataChannel("channel");

// Self-explanatory event handlers for the data channel

dc.onmessage = e => console.log("Just got a message " + e.data);
dc.onopen = e => console.log("Connection opened");

// Upon finding a new ICE candidate, print the entire local

description, i.e. A's SDP
lc.onicecandidate = e => console.log("New ICE Candidate! Reprinting
SDP: " + JSON.stringify(lc.localDescription));

// Create the offer, i.e. set A's SDP

// During this process, it will find all the ICE candidates which
trigger the above event handler multiple times
lc.createOffer().then(o => lc.setLocalDescription(o)).then(a =>
 console.log("offer set successfully"));

Here is the output of this code:

offer set successfully

New ICE Candidate! Reprinting SDP:

{"type":"offer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0
1223286171082717073 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
84:23:7D:2D:79:B9:E2:3E:58:07:91:DC:9F:6D:C8:A5:91:D9:EA:48:62:0A:00:
3D:B0:A4:27:E8:FA:03:EB:38\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application
9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4
0.0.0.0\r\na=candidate:0 1 UDP 2122187007
96f08512-484e-458f-925c-f92193f5c2db.local 51093 typ
host\r\na=sendrecv\r\na=ice-pwd:f4eda4ab6fd41ed04d5d528da45db660\r\na
=ice-ufrag:3a86c799\r\na=mid:0\r\na=setup:actpass\r\na=sctp-port:5000
\r\na=max-message-size:1073741823\r\n"}

New ICE Candidate! Reprinting SDP:

{"type":"offer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0
1223286171082717073 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
84:23:7D:2D:79:B9:E2:3E:58:07:91:DC:9F:6D:C8:A5:91:D9:EA:48:62:0A:00:
3D:B0:A4:27:E8:FA:03:EB:38\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application
9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4
0.0.0.0\r\na=candidate:0 1 UDP 2122187007
96f08512-484e-458f-925c-f92193f5c2db.local 51093 typ
host\r\na=candidate:1 1 UDP 2122252543
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 51094 typ
host\r\na=sendrecv\r\na=ice-pwd:f4eda4ab6fd41ed04d5d528da45db660\r\na
=ice-ufrag:3a86c799\r\na=mid:0\r\na=setup:actpass\r\na=sctp-port:5000
\r\na=max-message-size:1073741823\r\n"}

New ICE Candidate! Reprinting SDP:

{"type":"offer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0
1223286171082717073 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
84:23:7D:2D:79:B9:E2:3E:58:07:91:DC:9F:6D:C8:A5:91:D9:EA:48:62:0A:00:
3D:B0:A4:27:E8:FA:03:EB:38\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application
9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4
0.0.0.0\r\na=candidate:0 1 UDP 2122187007
96f08512-484e-458f-925c-f92193f5c2db.local 51093 typ
host\r\na=candidate:1 1 UDP 2122252543
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 51094 typ
host\r\na=candidate:2 1 TCP 2105458943
96f08512-484e-458f-925c-f92193f5c2db.local 9 typ host tcptype
active\r\na=sendrecv\r\na=ice-pwd:f4eda4ab6fd41ed04d5d528da45db660\r\
na=ice-ufrag:3a86c799\r\na=mid:0\r\na=setup:actpass\r\na=sctp-port:50
00\r\na=max-message-size:1073741823\r\n"}

New ICE Candidate! Reprinting SDP:

{"type":"offer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0
1223286171082717073 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
84:23:7D:2D:79:B9:E2:3E:58:07:91:DC:9F:6D:C8:A5:91:D9:EA:48:62:0A:00:
3D:B0:A4:27:E8:FA:03:EB:38\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application
9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4
0.0.0.0\r\na=candidate:0 1 UDP 2122187007
96f08512-484e-458f-925c-f92193f5c2db.local 51093 typ
host\r\na=candidate:1 1 UDP 2122252543
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 51094 typ
host\r\na=candidate:2 1 TCP 2105458943
96f08512-484e-458f-925c-f92193f5c2db.local 9 typ host tcptype
active\r\na=candidate:3 1 TCP 2105524479
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 9 typ host tcptype
active\r\na=sendrecv\r\na=ice-pwd:f4eda4ab6fd41ed04d5d528da45db660\r\
na=ice-ufrag:3a86c799\r\na=mid:0\r\na=setup:actpass\r\na=sctp-port:50
00\r\na=max-message-size:1073741823\r\n"}

New ICE Candidate! Reprinting SDP:

{"type":"offer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0
1223286171082717073 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
84:23:7D:2D:79:B9:E2:3E:58:07:91:DC:9F:6D:C8:A5:91:D9:EA:48:62:0A:00:
3D:B0:A4:27:E8:FA:03:EB:38\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application
9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4
0.0.0.0\r\na=candidate:0 1 UDP 2122187007
96f08512-484e-458f-925c-f92193f5c2db.local 51093 typ
host\r\na=candidate:1 1 UDP 2122252543
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 51094 typ
host\r\na=candidate:2 1 TCP 2105458943
96f08512-484e-458f-925c-f92193f5c2db.local 9 typ host tcptype
active\r\na=candidate:3 1 TCP 2105524479
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 9 typ host tcptype
active\r\na=sendrecv\r\na=end-of-candidates\r\na=ice-pwd:f4eda4ab6fd4
1ed04d5d528da45db660\r\na=ice-ufrag:3a86c799\r\na=mid:0\r\na=setup:ac
tpass\r\na=sctp-port:5000\r\na=max-message-size:1073741823\r\n"}

Now we can copy one of the SDPs to the other browser console, i.e. B and write the
following code:

const offer =
{"type":"offer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0
1223286171082717073 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
84:23:7D:2D:79:B9:E2:3E:58:07:91:DC:9F:6D:C8:A5:91:D9:EA:48:62:0A:00:
3D:B0:A4:27:E8:FA:03:EB:38\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application
9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4
0.0.0.0\r\na=candidate:0 1 UDP 2122187007
96f08512-484e-458f-925c-f92193f5c2db.local 51093 typ
host\r\na=candidate:1 1 UDP 2122252543
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 51094 typ
host\r\na=candidate:2 1 TCP 2105458943
96f08512-484e-458f-925c-f92193f5c2db.local 9 typ host tcptype
active\r\na=candidate:3 1 TCP 2105524479
2d65039d-859c-43a3-bcee-a1bef41a15ad.local 9 typ host tcptype
active\r\na=sendrecv\r\na=end-of-candidates\r\na=ice-pwd:f4eda4ab6fd4
1ed04d5d528da45db660\r\na=ice-ufrag:3a86c799\r\na=mid:0\r\na=setup:ac
tpass\r\na=sctp-port:5000\r\na=max-message-size:1073741823\r\n"};

// Create a remote channel where the initiator, Browser A will send

data on
const rc = new RTCPeerConnection();

rc.onicecandidate = e => console.log("New ICE Candidate! Reprinting

SDP: " + JSON.stringify(rc.localDescription));

// On data being sent by the initiator (A), it will be sent on the

data channel, attach an event handler to save A's data channel at B
as follows:
rc.ondatachannel = e => {
// Note: Channel name could be anything, here "dc" is used, just
has to be a property of "rc"
rc.dc = e.channel;
// Now that the data channel has been set, event handlers can be
placed on this data channel like the "onopen" and "onmessage" ones we
did at browser A
rc.dc.onmessage = e => console.log("new message from client: " +
e.data);
rc.dc.onopen = e => console.log("Connection opened");
}

// Set the remote description of B to be the "offer"

rc.setRemoteDescription(offer).then(a => console.log("Offer Set"));

// Create the answer and set it as the localDescription of B

// This would trigger the "onicecandidate" event handler as it did
with A
rc.createAnswer().then(a => rc.setLocalDescription(a)).then(a =>
console.log("Answer Created"));

Now we can copy the answer that was generated by Browser B to Browser A and finish
the connection setup as:

const answer =
{"type":"answer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0
 6266682524487062443 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
EB:39:77:ED:48:62:F6:5E:7C:BA:EF:86:53:C2:FD:78:8B:F7:1F:6B:E8:E3:96:
59:77:DC:3C:8D:81:F5:D6:9F\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application
9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4
0.0.0.0\r\na=candidate:0 1 UDP 2122187007
c442ffa9-090c-4b68-bad5-03dccefd93b6.local 63011 typ
host\r\na=candidate:1 1 UDP 2122252543
2fdac0fa-39c6-4b28-8571-e357194dc6db.local 63012 typ
host\r\na=candidate:2 1 TCP 2105458943
c442ffa9-090c-4b68-bad5-03dccefd93b6.local 9 typ host tcptype
active\r\na=candidate:3 1 TCP 2105524479
2fdac0fa-39c6-4b28-8571-e357194dc6db.local 9 typ host tcptype
active\r\na=sendrecv\r\na=end-of-candidates\r\na=ice-pwd:c46effc0062a
ffe58866ce3d6676d98b\r\na=ice-ufrag:b9c3438a\r\na=mid:0\r\na=setup:ac
tive\r\na=sctp-port:5000\r\na=max-message-size:1073741823\r\n"}

lc.setRemoteDescription(answer);

You should see “Connection opened” being printed to the console.

I got a problem when I tried to run this with Firefox:

WebRTC: ICE failed, add a STUN server and see about:webrtc for
more details

It could also be about “add a TURN Server”

You can use the free TURN and STUN Servers from Open Relay
(https://www.metered.ca/tools/openrelay/) and add it to your configuration as:

const configuration = {
"iceServers": [{ "url": "stun:stun2.1.google.com:19302" }]
};

const pc = new RTCPeerConnection(configuration);

However I was not able to find a TURN server that quickly and just tried it out on Edge
and it worked as intended so I just moved on :(
Now you can send data on the data channel from the initiator (A) to the other party (B)
as:
dc.send("Yo what up");

This would print the following at B’s console with the help of the “onmessage” event
handler on the data channel:
“New message from client: Yo what up”

B can send a message to A with the help of the data channel that is attached to “rc” as:

rc.dc.send("Good, how u doing");

And this would print the following at A’s console with the help of the “onmessage” event
handler on the data channel:
“Just got a message Good, how u doing”

ℹ️How to share media? ▶️ Media API

This was just a simple message passing mechanism, how about for sharing media?

You use the Media API.

“getUserMedia” method can be used to access the microphone and video camera.

“RTCPConnection.addTrack(stream)”
The only major difference in this case, is to add the media to the WebRTC channel
via the “lc.addTrack” method and everything else should follow the same logic.

Tutorial Link: https://web.dev/articles/webrtc-basics

➕ Pros
● P2P is great as that is the fastest path to communicate between two parties
without having to go through a 3rd party. This gives you low latency for high
bandwidth content like media.
● Popular and standardized API because of which we don’t have to build our own.
➖ Cons
● Maintaining STUN and TURN servers, expensive because of the work involved,
the cost to maintain a public IP, the cost to have a server up all the time. Discord
ended up creating their own TURN and STUN servers.
● P2P falls apart in the case of multiple participants, for example, a Discord voice
channel where there’s 10 people, then that means each person has to be
connected to every other person, i.e. 10 * 9 = 90 connections just for that voice
channel alone.

ℹ️onIceCandidate and addIceCandidate

How to maintain the connection as new candidates come and go?

“onIceCandidate” tells the user that there is a new candidate even after the SDP
has already been created. The candidate is signaled and sent to the other party
“somehow”. The other party uses “addIceCandidate” to add it to its SDP.

⚡Set custom TURN and STUN Servers

Same concept as I mentioned before:
const iceConfiguration = {
iceServers: [{
urls: 'turn:turnserver.company.com:3478',
username: 'optional-username',
credentials: 'auth-token'},
{
urls: "stun:stun.services.mozilla.com",
username: "test@mozilla.com",
credential: "webrtcdemo"
}
]}

const pc = new RTCPeerConnection(iceConfiguration);

Create your own STUN and TURN servers using COTURN which is an open-source
initiative!
Feb 8, 2024
⏪Recap: HTTPS over TCP with TLS 1.2

3 Roundtrips
Roundtrip 1: TCP 3-Way Handshake
Roundtrip 2 & 3: TLS 4-Way Handshake
Roundtrip 2: Deciding algorithm for handshake, certificate sending from Server
Roundtrip 3: Key Exchange, uses asymmetric key exchange algorithm

⏪Recap: HTTPS over TCP with TLS 1.3

2 Roundtrips only. This is why TLS 1.3 is always preferred.

Roundtrip 1: TCP 3-Way Handshake

Roundtrip 2: TLS Handshake
No algorithm selection, no certificate transfer, only key exchange happens.

⏪Recap: HTTPS over QUIC (HTTP/3)

 ● Does everything (TCP + TLS handshake) in the same roundtrip.
● QUIC was built with TLS in mind which wasn’t the case for TCP.
● Uses TLS 1.3 so only a single roundtrip is required for setting up TLS which when
coupled with QUIC, results in a single roundtrip overall.

⏩Not a Recap: HTTPS over TCP with TLS 1.3 0RTT

This uses TCP, so we still have the 3-way handshake that QUIC avoids.

It relies on an existing connection between the Client and the Server and the Client uses
the pre-shared key to encrypt and send the data along with the TLS handshake ”Client
Hello” message.

The Server, if it recognizes the pre-shared key can decrypt the received data, generate
the response, encrypt it and then send it to the Client along with the “Server hello”
message and then the Client can send the “FIN” message to complete the TLS
handshake.

⏩Not a Recap: HTTPS over QUIC with TLS 1.3 0RTT

Same as before, but now instead of TCP, we are using QUIC.

So the QUIC and TLS handshaking can happen together but because of the 0RTT
concept involving the pre-shared keys mentioned above, data can be sent along with the
QUIC handshake.

This results in there literally being 0 round trips conducted to set up the connection, as
the only handshake here happens in sync with data transmission, so there’s no overhead
of setting up a connection.

0RTT is a really difficult thing to implement and only a few sources have successfully
implemented it, a major one being Cloudflare.

Feb 12, 2024

🎯Backend Execution Patterns
🥊Process vs Thread
⚡Process
● A set of instructions.
● Has an isolated memory.
● Has a Process ID (PID).
● A process has to be scheduled in the CPU in order for it to be executed.

⚡Thread
● A light-weight Process.
● So it is also a set of instructions.
● But the difference is that it shares its memory with its parent process.
● It also has an ID.
● It also has to be scheduled in the CPU in order for it to be executed.

⚡Ways of Implementing a Backend

1. Single Threaded Process
● One process with a single thread which is simple to write as you don’t have to
deal with multiple threads.
● Example: Node.js

2. Multi-Processes
● App has multiple processes.
● Each process would have its own memory.
● This approach takes advantage of multicore systems, as a system can
simultaneously execute the same number of instructions as the number of cores
it has.
● This approach also consumes more memory as each process would have its
own isolated memory.
● Examples: Nginx, Postgres
ℹ️Redis Backup Routine (COW Mechanism)
The whole Redis cache is maintained in memory. So Redis provides an option for
users to save a snapshot of the current cache to disk. This runs in a periodic time
interval (RDB snapshot every five minutes or more seems to be the standard).

But how would you continue providing the cache services if the process is already
being blocked by the backup process?
You create a new process and this new process will manage the work of writing to
disk, while the main process will continue serving requests.

But won’t this allow the snapshot to be inconsistent? Because the cache can be
updated as the backup is ongoing.

Yes, this is why the child process should have the main process’ entire memory
copied but that’s a very expensive operation because the entire memory can be
very large in practice.

To avoid this, the OS’ Copy-on-Write (COW) mechanism is used by Redis. The backup
process will copy the relevant page from the main process when it is modified by a
user operation, basically on a write. This is why this mechanism is called
Copy-on-Write.

3. Multi-Threaded
● One process, multiple threads.
● All threads will have the shared memory, so they will have to compete with each
other for memory access (mutex locks come into the picture here).
● This approach takes advantage of multi-cores as each thread can be slotted to
execute different parts of the same process at the same time.
● It overall requires less memory than the multi-process approach.
● Will have to deal with race conditions, i.e. the locking stuff.
○ Even incrementing a variable can result in a race condition between
threads.
○ So that’s why locks, latches exist.
● Examples: Apache, Envoy, SQL Server (can disable this in SQL Server).

How many is too many?

 ● If a process/thread enters the wait state (because of IO or some event), then it
gets kicked out of the CPU so that a new process/thread can get CPU time.
● This is a context switch and there’s an associated context switch time that the
CPU takes to clear the old process’ data out and load the new process’ data in.
● Because of this overhead of switching the in-execution process/thread, having
too many processes/threads is not always a good thing.
● Multiple cores certainly help with this as you can schedule multiple threads at the
same time and context switching won’t be a major overhead.
● Rule of Thumb: Number of Cores = Number of Processes
○ If your processes exceed the number of cores, then the processes are just
waiting in order to get CPU time and you will keep incurring context switch
time as an overhead.
○ If your processes are less than the number of cores, then the cores are not
being utilized to their max efficiency and there will be many empty cycles
where the CPU core is doing nothing.

Feb 14, 2024

❓How Connections are Established
SYN/Accept Queues
 ● Server listens on an “address:port”.
● Client connects.
● Kernel does the handshake creating a connection.
● The Backend process “Accepts” the connection.
● The Kernel of the Server is the one that creates the socket and it also has two
queues “SYN” and “Accept”.
● When the Client sends a SYN, the Kernel adds it to the “SYN” Queue and replies
with “SYN/ACK”.
● The Client replies with ACK.
● Kernel finds the SYN matching the ACK in the “SYN” Queue.
● Kernel finishes the connection setup to get the full connection.
● Kernel removes the SYN from the “SYN” Queue.
● Kernel adds the full connection to the “Accept” Queue.
● The Backend process (on the same machine where the Kernel did all this)
accepts the connection and it is removed from the “Accept” Queue.
● A file descriptor is created for the newly created connection that the backend can
use for communication.

ℹ️NOTE
You can see how TCP is a stateful protocol as it has to maintain these queues for its
functioning, among many other things.

The size of these queues is called the Backlog.

You can now understand just how much CPU is involved in setting up even a single
TCP connection!

ℹ️Socket vs Connection
Socket = What we listen on for a Connection

We can have 100s of Connections on a single Socket.

Socket = “Address:Port”
Problems with Accepting Connections
● Backend doesn’t accept fast enough.
○ This can be because the Queues are full, which can be because the CPU is
already overloaded with several tasks or there can be several other
reasons for this because there’s just so many moving parts, the network is
congested, a TLS issue, packet delivery fails, packet corruptions, etc..
● Clients who don’t ACK
○ This is related to an attack called SYN flooding.
○ In this, the Client spams SYN requests to the Server but doesn’t ACK any
of the SYN/ACK requests received.
○ Because of this the SYN queue of the Server fills up and any legitimate
Clients trying to connect to the Server will not be able to store their SYN
requests in the Queue as the existing elements won’t be dequeued till their
corresponding SYN/ACK is received.
● Small backlog is a problem as it can get filled really quickly but a large backlog
can lead to complications in the overall design and performance.

🤖 ChatGPT Time
Question
What is NIC?

Answer

A Network Interface Card (NIC) is a hardware component that allows a computer or

other device to connect to a network. NICs can be used for both wired and wireless
connections and are essential for enabling devices to communicate over a network
such as a local area network (LAN), wide area network (WAN), or the internet.

NICs typically come in the form of an expansion card that can be inserted into a
computer's motherboard, although they can also be built into the motherboard itself
as an integrated network interface. For laptops and some compact or custom-built
computers, NICs might come in the form of a USB adapter or be embedded within
the machine's circuitry.

Here are some key points about NICs:

● Physical Layer Connection: NICs operate at the physical layer of the OSI
model, providing the physical connection to the network medium, which
could be copper cables (for Ethernet), fiber optics, or radio waves (for
 Wi-Fi).

● Data Link Layer Functionality: NICs also operate at the data link layer,
where they handle the communication of data frames between a local device
and the network. Each NIC has a unique MAC address burned into it, which is
used to identify the device on the local network segment.

● Speed and Duplex Settings: NICs come with various speed capabilities, such
as 10/100 Mbps, 1 Gbps, or even 10 Gbps and higher for high-performance
networking. They also support different duplex modes, enabling either
half-duplex (one-way communication at a time) or full-duplex (two-way
communication simultaneously) operation.

● Interface and Drivers: NICs provide a physical interface (such as an RJ45 port
for Ethernet or an antenna for Wi-Fi) for connecting to the network. They
require drivers, which are software components that allow the operating
system to communicate with the hardware.

● Advanced Features: Some NICs offer advanced features such as offloading

(processing some of the network tasks to reduce CPU load), Quality of Service
(QoS) settings, VLAN support, and more.

NICs are a critical part of networking infrastructure, and their reliability and
performance can significantly impact the overall network experience. Whether for
personal computers, servers, or specialized network devices, NICs serve as the
gateway for network communication.

🤖 ChatGPT Time
Question
What is the SYN flood attack and how is it resolved?

Answer

A SYN flood attack is a type of Denial-of-Service (DoS) attack that exploits part of
the normal TCP three-way handshake to overwhelm a targeted server with TCP
connection requests, but the attacker does not complete the handshake. The goal
of the attacker is to consume enough server resources to make the system
unresponsive to legitimate traffic.

Here's how a SYN flood attack works:

1. The attacker sends a flood of TCP SYN (synchronize) packets to the target
server's IP address, often with a spoofed source IP address.
2. The server receives the SYN packets and allocates resources to handle the
new connections by sending back a SYN-ACK (synchronize-acknowledge)
packet for each connection attempt.
3. The attacker either does not respond or sends additional SYN packets with
different spoofed source IP addresses.
4. The server waits for the final ACK (acknowledge) packet to complete the TCP
handshake, which never arrives. These half-open connections accumulate,
consuming server resources and potentially leading to legitimate requests
being ignored or dropped.

To mitigate and resolve SYN flood attacks, several strategies can be implemented:

1. SYN Cookies: This is a technique where the server does not allocate
resources for the connection until the client's ACK packet is received.
Instead, it sends back a SYN-ACK with a specially crafted sequence number
(cookie) that encodes the connection information. If the client responds with
the correct ACK packet (which includes the cookie), the server can then
allocate resources and establish the connection.

2. Firewalls and Intrusion Prevention Systems (IPS): Properly configured

firewalls or IPS devices can detect and filter out malicious SYN packets based
on various attributes such as traffic volume, frequency, or anomalies in the
packet headers.

3. Ratelimiting: Limiting the number of SYN packets accepted per second can
help prevent the server from being overwhelmed. This can be done at the
firewall, router, or even at the server level.

4. Load Balancers: Distributing incoming traffic across multiple servers can help
absorb the impact of a SYN flood attack, reducing the strain on any single
server.
 5. TCP Stack Hardening: Operating systems can be configured to handle SYN
floods more efficiently, such as increasing the size of the SYN queue,
reducing the SYN-RECEIVED timer, or enabling aggressive TCP SYN flooding
protections.

6. DDoS Mitigation Services: These services can detect and mitigate large-scale
DDoS attacks, including SYN floods, by filtering traffic before it reaches the
target server.

7. Hybrid Solutions: A combination of on-premises and cloud-based DDoS

protection can provide a more robust defense against SYN flood attacks.

It is essential to have a comprehensive security strategy that includes both

proactive measures to reduce the risk of SYN flood attacks and reactive measures to
mitigate them if they occur. Regularly updating and patching network devices and
servers is also crucial to protect against known vulnerabilities that attackers could
exploit.

Feb 15, 2024

❓Reading and Sending Receive Buffers
Send and Receive Buffers

● The Kernel actually has two more queues called the Send and Receive buffers.
● When a Client sends data to the Server on a connection, the Server’s Kernel puts
the data in the Receive queue.
● Kernel ACKs that received data and updates the Flow Control Window by
deducting the amount of data that has been received.
 ○ This is possibly done with a delay, so that a bunch of bytes can be
acknowledged with a single ACK signal, instead of spamming ACKs for
every byte received. Note that for this, delayed acknowledgement has to
be enabled. This mechanism is part of a collection of intelligent algorithm
checks done by the Kernel (OS) to save on time and resources.
○ Why is the Window Size reduced, shouldn’t it be increased now that the
data has been received?
■ Because the Server hasn’t actually accepted the data, it is still in the
Receive Queue.
■ So if the Window size is say 1000 bytes and the data (including all
additional metadata) that has been received is 500 bytes, then as
long as the Server hasn’t accepted it, it can only receive 500 more
bytes of data in the Flow Control Window.
■ That is why this update is done.
● The Backend app calls read to copy data from the Receive buffer into its process’
memory (this is an expensive operation as this design choice leads to copy times
adding up really quickly and there are proposals in place to implement zero copy
principles).
● This is when the backend starts processing the raw bytes that it just copied to its
memory.
● Assuming that TLS is being used, first it will decrypt the bytes received using the
symmetric key that the kernel has.
● Now it will read the raw decrypted byte-wise data as per the HTTP headers
specified and get the actual HTTP data.
● Now that the Request has actually been parsed, the backend will check whether
an event with the associated Request is wired in which case it will call that
specific event handler.
● Note that all of this is done by the HTTP (or HTTPS) library that has been
imported and used in your backend, you just never knew all of this was going on
behind the scenes.

ℹ️The read operation is blocking?

The read operation from the backend process to the Receive Queue is a blocking
operation, so if the Queue is empty, then the backend is blocked.

If this process is to be done asynchronously, then there’s more advanced concepts

like io_uring in Linux that allows you to do this.
There’s also “epoll” in which case, when the backend process requests for data and
its not there, then it would immediately fail and then the backend process has
basically subscribed to the relevant Socket file descriptors is alerted by the Kernel
only when there’s data on the Socket file descriptors that the backend process has
subscribed to.

Send Buffers
● When you call “response.end()” or something to that effect in your backend
application, the response is sent to the Send Queue with the send() method.
● Ofcourse do remember that the libraries that you are using wil first encrypt the
data with TLS, serialize the data to bytes, etc. before it is sent to the Send Queue.
● So the response data is copied from the process’ memory to the Send Queue.
● Now the responsibility of actually sending the Response is handed off to the
Kernel.
● So the Response isn’t actually immediately sent, the Kernel actually lets the data
sit in the Send Queue till enough data has accumulated (Maximum Segment
Size) so that it can fit into a Segment.
● This is done for efficiency reasons and is done with the help of Nagle’s Algorithm
but note that this also introduces latency in the overall process because of which
Nagle’s Algorithm is often not preferred.
● Eventually the Response data will be sent and the Client will acknowledge it with
an ACK.
● Now the data will be removed from the Send Queue.
● So the Send Queue is basically a Sliding Window used by TCP for managing data
being sent outside.

Problems with Reading and Sending

● The backend doesn’t read fast enough because of which the Receive Buffer will
be full.
● The Server’s Kernel will not be able to receive any more data.
● The Client will not be able to send the data to the Server as the Server keeps
dropping the packets and the Client will slow down and end up getting blocked
basically.
Feb 16, 2024
⚡Ways of Implementing a Backend
Listener, Acceptor, Reader

Listener: Who is listening for incoming connections

Acceptor: Who is accepting the connections and the data
Reader: Who is reading the data that has been received from the subscribed to file
descriptors.
These 3 tasks can be done by the same process, i.e. the backend. However you can play
around with these roles and assign them to different processes/threads to get different
styles in which different backends are architected.

1. Single Listener/Single Worker Thread

Example: Node
Single threaded application that takes care of all 3 roles, listener, acceptor and reader.

Problem:
● If there’s too many connections, then there’s a lot of “chatting” going on by the
process (Node) and it might not be able to handle the load and slow down.
● Which means the OS/CPU will be overloaded with tasks that are associated with
this process and all Clients will not get serviced in a fair manner.

Solution:
● Spin up more Node processes that would serve the backend and would make the
best use out of multicore systems.
● Each Node process would still be its own Listener, Acceptor and Reader but it
would be better equipped to handle increased load.

🤖 ChatGPT Time
Question
Why spinning up multiple Node.js processes can help handle higher load from
Clients?

Answer

Spinning up multiple Node.js processes can help handle higher load from clients due
to the following reasons:

1. Single-Threaded Nature: Node.js operates on a single-threaded event loop.

While this is efficient for I/O-bound tasks, it means that only one operation
can be processed at a time per process. If a Node.js process is handling a
CPU-intensive task, it can block the event loop and delay the processing of
other incoming requests.

2. Utilizing Multiple Cores: Modern servers and computers have multi-core

CPUs. Running multiple Node.js processes allows the application to take
 advantage of multiple cores, effectively parallelizing the workload and
increasing the capacity to handle concurrent requests.

3. Avoiding Bottlenecks: By having multiple processes, you can distribute

incoming client connections across these processes. This distribution helps
prevent any single Node.js process from becoming a bottleneck, leading to
more efficient handling of high loads.

4. Fault Tolerance and Reliability: Multiple processes can provide fault

tolerance. If one process crashes due to an error, the other processes can
continue to serve client requests. This setup improves the overall reliability
of the application.

5. Load Balancing: When using multiple processes, a load balancer (either at

the operating system level or through a reverse proxy) can distribute
incoming requests across the different Node.js instances. This ensures that
the load is evenly spread and that no single instance becomes overwhelmed.

Node.js provides a built-in module called cluster that simplifies the creation of
child processes (workers) that run simultaneously and share the same server port.
The cluster module allows you to spawn a pool of worker processes that can
handle more tasks in parallel, improving the application's capacity to handle high
traffic loads.

In summary, spinning up multiple Node.js processes helps to utilize system resources

more effectively, providing better performance and reliability when handling higher
loads from clients.

2. Single Listener/Multiple Worker Threads

Single Listener & Acceptor but multiple Readers, so it is a multi-threaded application.
Example: memcached (in-memory store like Redis)
Note: The blue orbs in the diagram are the Client connections.

● N number of worker threads (Readers) will be spun beforehand, this usually

matches the number of cores on the system.
● When the main thread accepts a connection, it hands off that connection to one
of the Worker threads and now the Worker thread will do the job of processing it
while the main thread can resume its task of listening for more Client
connections from the Receive queue of the Kernel.
● Also note that a form of load balancing is happening here as connections are
sent to one of several Worker threads.

Problem: No true load balancing

● It is true that some form of load balancing happens as the main thread hands off
work to the worker threads.
● But depending on the nature of the connection, some worker threads can end up
doing way more work than the other ones.
 ● For example, if one of the connections is gRPC while the other one is HTTP/1.1,
then clearly the gRPC will incur a lot of traffic as well as work required whereas
the HTTP connection is much simpler and lighter in comparison. So there’s an
imbalance in the work done by the respective worker threads, which means
improper resource utilization.

3. Single Listener/Multiple Worker Threads with Message Load Balancing

Example: RAMCloud (super high-speed storage for large-scale datacenter applications).

Single Listener, Single Acceptor and Single Reader, so everything happens in a single
process. So what do the threads do?

The threads receive the actual ready to process requests/messages. Listening,

Accepting and Reading the request is done by the main process itself, the actual
processing of requests is done by the multiple threads that are spun up. These are
actual worker threads now as they have no other tasks than to simply execute the actual
work, aka true load balancing.

Problem: Main process does a lot of work

To avoid this we can spin up more instances of the main process, but then all the “main
processes” would have to listen on the same port among other things.There’s a lot of
complications that get added and it's not an easy thing to accomplish, but it is possible.

Feb 17, 2024

4. Multiple Acceptor Threads on a Single Socket

Example: nginx (no longer does this by default though)
● Has a single Listener which is the main process and will have the actual listener
Socket.
● All threads of this Listener will also have access to the Socket because of the
shared memory space.
● Each thread calls accept on the same socket object but only one thread can hold
a socket at a given time (mutex).
● So nginx implements the mutex locking mechanism which ensures that only one
acceptor thread will have access to the socket at any given time.
● Once the connection is accepted, the Socket can be let go and the Reader will
take over from the Acceptor.
● This design introduces a lot of wait in the system but if your application involves
getting a lot of accept requests from Client, like it is in the case of nginx, this is
actually a feasible design.
5. Multiple Listeners, Acceptors and Readers with Socket Sharding
● Trying this on Node.js would fail because Node.js doesn’t allow two different
processes to listen on the same socket.
● But in lower level languages like C, Rust, etc. the “SO_REUSEPORT” option allows
you to accomplish this.
● This is done with the help of separate Listen and Accept Queues at the OS Level.
● Whenever a process requests for a Socket to listen on, it will receive a unique
Socket identifier even though that Socket (address and port combination) is
already being held by some other process.
● So the unnecessary waiting for mutex logic is avoided.
● The old problem of true load balancing pops back in again, as the number of
connections held by a process will be load balanced, however the load on each
process may not be load balanced because of the differences in connection
types.
● This problem can be sidestepped by adding another layer of worker threads and
the original layer of processes just does the listening and accepting. But this will
make your architecture quite complex, but the performance tradeoff can be
worth it depending on your application.
⚡Backend Idempotency
The idea that your Request can be repeatable without making any changes to the state
of the backend, no side effects basically.

Example: Request, POST /comment

Implementation 1:
Takes comment and appends it to the table.
NOT IDEMPOTENT
● To explain why consider a scenario where a user/proxy retries the same
Request? This doesn’t mean just posting the same comment, it means the user is
attempting to make the same Request (maybe via DevTools or something).
● In this scenario, the backend should actually recognize that this is the same
Request and it has already been processed before and therefore this new
Request is a duplicate and should be dropped, then it would have been
Idempotent.
● Comments are harmless but imagine this scenario for financial transactions.

Summary
● In conclusion, an idempotent Request can be retried without affecting the
backend.
● Easiest implementation for this is to send a RequestID with every request and
this can be used by the backend for identification of each Request.
○ If RequestID has been processed already, then return without executing
anything.
○ Also known as Idempotency Token.
○ Has the overhead of maintaining the lookup table.

Idempotency for HTTP

● GET is idempotent.
● POST isn’t so we try to make it as idempotent as possible.
● Browsers and proxies treat GET Requests as idempotent.
● This is why you should never use a GET request to write something to the
backend and must also design your backend controllers as such to block these
Requests.
⚡Nagle’s Algorithm
Client-side Delay

● Combine small segments and send them in a single one.

● The client waits till the Maximum Segment Size (MSS) is reached before sending
the segment.
● This avoids the overhead of sending header data (atleast 40 bytes, TCP + IP) for
just a few bytes of data.
● However, this introduces a lot of delay at the client-side even though every other
metric would be performing just as intended.
● This is why a lot of tools like cURL have outright disabled this algorithm as in the
modern era, bandwidth is of a lesser concern as compared to performance.
● NOTE: This logic executes only if the Client actually has some Request waiting to
be acknowledged in the Send Queue. Otherwise the Client would directly send the
Segment out without waiting for “MSS” amount of data to accumulate.

Where does this cause an issue?

Sending large data causes delays.
● A wants to send 5000 bytes on MSS of 1460.
● 3 full segments of size 1460 are sent, however 620 bytes are left which are not
sent immediately.
● The remaining bytes are only sent when the ACKs are received, as the previously
sent Segments are still waiting for an ACK which means Nagle’s algorithm is
active and the 4th segment would not be sent until either 1460 bytes are
accumulated or all pending ACKs are received.

This is why most clients today disable Nagle’s algorithm because you would rather get
performance than loss of bandwidth, done with the help of “TCP_NODELAY” option.
Feb 22, 2024
🥊Proxy vs Reverse Proxy
⚡Proxy

A server that makes requests on your behalf.

Client knows of the Server, but the Server only knows of the Proxy and not the Client.

So in action this looks like the following:

● I want to send a request to Google and I have a proxy configured.
● I will establish a Layer 4 TCP connection with the Proxy server.
● The Proxy will get the Layer 7 content of the Request and establish a TCP
connection with Google.
● It will then send my request data to Google with the source IP address set to its
own IP address.
● When Google receives my Request, it will have no idea of my existence and
instead would think that the Proxy server is the actual sender of the Request.

Use-Cases
1. Caching
● The proxy can be used as a cache for resources being used by multiple Clients,
so if a Client is asking for a resource that has been cached then another Client
can be given the cached version and save on network resources.
 ● Think of organizational level proxies as an implementation of this logic.
● Very easy to mix up this logic with that of a Reverse Proxy.

2. Anonymity
However it does require you to trust the Proxy that it will not give up your identity via
Request headers/data.

3. Logging
Because all the Requests hit the Proxy before being sent outside (especially in the
context of microservices), it can be used as a sidecar container for logging all Requests
and their statistics because it has that visibility into the entire service mesh’s Requests.

4. Block Sites
● The Proxy can apply a blacklist to the list of websites that its Clients can visit.
● Think hostel WiFi :)

⚡Reverse Proxy

The Server knows of the Client, but the Client only knows of the Reverse Proxy and not
the actual Server.

So it is basically the same process in reverse, the actual Client talks to the Server, but
this is a Reverse Proxy Server and it forwards the Client’s Request to an actual Server
that is basically hidden from the Client.

The Reverse Proxy Server is often called the Frontend Server or the Edge Server.
Use-Cases
1. Load Balancing

2. API Gateways / Ingress

Forward to a backend Server on the basis of what kind of hardware or logic is
embedded on those Servers.

3. Caching
This caching is much more straightforward to understand than the Proxy’s caching.

4. Canary Deployment
● Used for testing new features.
● Deploy the new features to one or a few Servers while the other ones will still
have the old version.
● Because of the reverse proxy, we can set a rule which directs 10% of the traffic to
the new feature Servers, while the rest goes to the old Servers and deployment
testing of this nature.

Proxy FAQ

1. Can a Proxy & Reverse Proxy be used at the same time?

Yes, but you won’t know any of it.

2. Can I use a Proxy instead of VPN for anonymity?

Not a good idea, as quite a few Proxies decrypt your traffic and actually look at your
data, which VPNs do not as they operate at Layer 3 (Network) whereas Proxies operate
at Layer 4 (Transport).

3. Is Proxy just for HTTP traffic?

No, but it is the most popular variant.
 🥊Layer 4 vs Layer 7 Load Balancers
Every Load Balancer is a Reverse Proxy but every Reverse Proxy may not be a Load
Balancer.

Load Balancer is just a Reverse Proxy with intelligent logic to balance the load.

Used to design fault tolerant systems.

⚡Layer 4 Load Balancer

● Start up.
● Establish multiple TCP connections with the backend Servers, for example, 10
per each backend Server.
● This process is called warming up so that we don’t waste time in TCP
handshaking when an actual Request comes in.
● When a Client makes a Request by establishing a TCP connection, the LB will
maintain state related to the Client (lookup table) and tag it to a single backend
TCP connection on some backend Server.
● Now all Requests from this Client will be sent to this TCP connection from that
backend Server only.
● The LB knows that anything received on this connection (source address + port)
has to be sent to this Client as well, that’s how Responses are handled.

➕ Pros
● Simple Load Balancing
● Efficient: No data lookup, just looks at IP and ports and rarely does buffering
depending on the MTU, but nothing like a Layer 7 LB would.
● More secure: Doesn’t need to decrypt your data, which L7 LB does because it
balances based on the packet content, so L4 LB is more secure.
● Works with any protocol.
● One TCP connection (NAT)

➖ Cons
● No smart load balancing
○ Because you’re not looking at the data.
 ○ Although this can be worked around by using multiple ports and based on
the port being used, the appropriate load requested can be determined.
○ If port 69 then high load, if port 42 then low load and so on as ports are
visible to L4.
● Not that effective with microservices architecture.
● Sticky per connection: Another way of saying no smart load balancing.
● No caching: Again because it doesn’t look at the content, it cannot cache
anything.
● Protocol unaware (can be dangerous) bypass rules.
○ This is a pro and a con.
○ As it is protocol unaware, logic can be used to bypass rules and access
random ports and other unsecure actions.

⚡Layer 7 Load Balancer

Same as L4 LB when it starts up, warms up TCP connections with the Backend Servers.
When a Client connects to the L7 Load Balancer via TCP, it becomes protocol specific.
Any logical Request will be forwarded to a new Backend Server.

As discussed before, the L7 Load Balancer will decrypt the packet and its data in order
to perform load balancing.
This means that the TLS certificate must reside in the L7 LB instead of the Backend
Servers. However, this is not really a desirable concept to many people.

How L7 LB understands that multiple Segments constitute a single unit

Also note that the same TCP connection can be used by multiple Clients as it is not tied
to a single Client like it was for L4 LB.

➕ Pros
● Smart Load Balancing.
● Caching is possible.
● Great for microservices.
● API Gateway logic can be implemented.
● API Gateway authentication can be implemented here.

➖ Cons
● Expensive: Because it does more work.
● Decrypts data: The LB serves as a TLS Terminator as the Client establishes TLS
with the LB and not the Backend Servers.
● Must share the TLS certificate.
● It needs to buffer in order to understand the Segments and implement logic.
● It needs to understand protocols in order to process Requests as well.
Feb 26, 2024

Each Stage in this Journey can be done by a separate thread/process or done by a

single thread/process and some combination of the above.
It is all use-case specific and there is no single right way to do things.

1. Accept
○ Received Request (raw bytes) is sent to the Accept queue of the Kernel.
○ Here it sits until the backend process calls the “accept” system call which
creates a file descriptor to represent the connection.
○ Separate thread is preferred to quickly accept Requests so that the
backend performance improves and the Accept stage doesn’t become a
bottleneck.
○ Once “accept” is complete, the handshake for TCP/QUIC is also completed
and ACK is sent to the Client.
2. Read
○ The Request data (raw bytes) is sent to the Receive queue of the Kernel.
○ It sits here till the backend process calls the “recv” or “read” system call
which moves the data from the Receive queue to the backend process.
3. Decrypt
○ It is known whether the Request data is encrypted or not.
○ If encrypted, then the backend process invokes the SSL library that the
code is linked to.
 ○ The library will decrypt the raw encrypted bytes to obtain the raw
decrypted bytes.
4. Parse
○ The protocol determined for communication is already known to the
backend.
○ Based on this information, the parsing library linked to the backend code
kicks in and parses the decrypted raw bytes to obtain the actual protocol
Requests.
○ Note that a single Request from the client can include any number of
protocol Requests or even contain a fragment of a complete protocol
Request. Those boundaries are what’s being determined here.
5. Decode
○ Actually decode the Request data in the appropriate format, like
deserialization of JSON or protobuf into a backend object, like JS Object.
○ For example, after stage 4, we will have JSON strings, but this is not
directly usable in JavaScript, we actually have to deserialize that into an
actual JS Object and that’s done in this stage.
○ Things like decoding from UTF-8 to ASCII, etc. depending on backend and
client configurations are also handled here.
○ One more task that could also be involved here is Request data
decompression.
6. Process
○ Finally the decrypted, parsed and decoded Request can be sent to the
backend code that actually processes the Request and performs
associated tasks like querying the database, reading from disk, compute,
etc.
○ It is highly recommended to have a dedicated worker for processing, the
worker pool pattern is very handy here.