Ecole Nationale Supériere d’Electricité et de Mécanique

Cyber Attack Simulation Project Reference

Prepared by: Laura Rodriguez

Instructor: Abdelkader Lahmadi
Date: May 12, 2022
Introduction
The purpose of this project is to create a tool that is able to scan a network, and
use reinforcement learning tools to generate the possible attack paths throughout the
network. This is to be done with two main tools: Cyber Battle Sim and Caldera. Caldera
generates the network scan, and cyber battle sim uses reinforcement learning to generate
the possible attacks.

Before Starting:
1. Make sure you have Python 3 installed on your machine. You can find it here:
https://www.python.org/downloads/

I
Contents
1 Definitions 1

2 Caldera 2
2.1 Getting Started with Caldera . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Pathfinder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Cyber Battle Sim 7

3.1 Getting Started with Cyber Battle Sim . . . . . . . . . . . . . . . . . . 7

4 The Program 9
4.1 Section 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2 Section 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3 Section 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.4 Section 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.5 Section 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 Common Errors 13
5.1 ”ModuleNotFoundError: No module named X" . . . . . . . . . . . . . 13
5.2 ”AttributeError: module ’lib’ has no attribute
’Cryptography HAS TLSEXT HOSTNAME’” . . . . . . . . . . . . . . . . . . 13

II
1 Definitions
The following section gives definitions for terms that I found useful during this
project.

Term Definition

Adversary A person or group of people that is trying to conduct a cyber attack

on other cyber resources.
DOS Denial of Service attack. An attacker makes a computer unavailable
to its intended users by limiting the computer’s functionality.
Lateral Movement Cyber attack technique where the attacker, after gaining
initial access, moves deeper into a network to look for sensitive data
and higher-value assets.
Local attack Cyber attack where attacker has an account on the system, used to
attempt unauthorized tasks.
Observation space the states that can be observed by the agent.(Reinforcement Learning
Probing The initial phase of any attack on a web applications.
Proxy A server application that is intermediary between a client requesting
a resource and the server that gives the resource.
Regular Expression DOS Attacker can exploit web browser, hang up web
application firewall, or attack vulnerable databases or web servers.
Remote Attack Cyber attack where attacker connects to the computer through
the network and takes advantage of bugs or weaknesses in system.
Web cache positioning attacker exploits web server behavior so that HTTP
response is served to others instead of intended recipients.

1
2 Caldera
Github: https://github.com/mitre/caldera

Caldera is basically a set of tools to assist cybersecurity professionals in reducing the

time and resources needed for routine cybersecurity testing.

Caldera has three main purposes:

1. Autonomous Adversary Emulation

Allows us to build a specific attacker profile and launch it into a network. It shows us
where the network is susceptible to an attack.
2. Autonomous incident reporting
Allows us to build an incident response and perform it on the host as necessary.
3. Manual red-team engagements
Allows us to augment existing offensive tools to perform security assessments on comput-
ers.

Although there are many tools that Caldera offers, the only tool that is currently relevant
to the project is the pathfinder plugin.

2.1 Getting Started with Caldera

Make sure you have Ubuntu installed on your machine. For windows, you can install
Ubuntu through the Windows store.

Installation Instructions:

1. Open Ubuntu and install the Caldera GitHub repository:

git clone https://github.com/mitre/caldera.git --recursive

2. Navigate into Caldera folder and install requirements:

cd caldera
pip3 install -r requirements.txt

3. Check to make sure it works by running the server. Run the following command, and
open localhost:8888/ on your browser.

python3 server.py --insecure

If all is working correctly, you should see a screen like this in your browser:

2
You can login using the username ’admin’ and password ’admin’. Once you log in,
you have access to most of Caldera’s plugins, except for pathfinder. We need to in-
stall pathfinder separately. Feel free to play around with the functions and follow some
tutorials to get acquainted with the environment. Some videos:

https : //www.youtube.com/watch?v = GukT j − i3U Dg

https : //www.youtube.com/watch?v = A7knprgoDB0
https : //www.youtube.com/watch?v = Y N IxwN LF 7dc
(1)

Common Errors:

1. AttributeError: module ’lib’ has no attribute ’Cryptography HAS TLSEXT

HOSTNAME’

Solution: add ’sudo’ to the beginning of the command. This solution works for other
errors too!
Example: sudo pip3 install -r requirements

3
2.2 Pathfinder
Pathfinder is a Caldera plugin that needs to be installed separately. It is a vulnerabil-
itiy scacnner. Basically, pathfinder scans a network, and shows the network vulnerabilities
are, and what they expose to adversaries. This can be useful in identifying potential de-
structive paths an adversary could take in a network.

Installation Instructions:
1. Navigate to the caldera/plugins directory.
2. Clone the pathfinder repository into the plugins directory.

git clone https://github.com/center-for-threat-informed-defense/

caldera pathfinder.git pathfinder --recursive

3. Navigate to the caldera/conf directory.

4. Edit the ’default.yml’ file, and add pathfinder to the list of plugins. You can edit
the file using vim. That section of the default.yml should look like this:

4. Start up the caldera server again (see step 3. of caldera installation instructions). In
your browser, you should now see that pathfinder has been added to the list of plugins
on the left, between manx and sandcat.

4
Using Pathfinder:

Tutorial: https://www.youtube.com/watch?v=gQRWkHFRG-s
1. Start the caldera server (see step 3 of installation instructions)

2. Click on the Pathfinder on the left side of the browser (see step 4 of installation
instructions)

3. Select the nmap scanner.

4. Input the IP of your target network. I used my own machine’s IP address. You
can find your machine’s IP with thise command in Ubuntu

ip addr

5. Input the ports you want to listen to. Recommedation: just input some common ports
(ex. 8888, 443, etc).

6. Click scan and wait until the button becomes clickable again. The scan should be
saved.

7. Navigate to the ”View” tab. You should see your new report in the ’Vulnerabil-
ity Report’ drop-down.

8. Click the ’Graph’ button. A graph of the network should appear that looks something
like this:

5
Obviously every network will look different, so it won’t look exactly like this one. How-
ever, this graph is what was used for the code.

9. Click on the ’Download Report’ button. This will download a .yml file that is eseential
to build the network for cyber battle sim. Essentially, this graph is a visual depiction of
the data in this report.

6
3 Cyber Battle Sim
Github: https://github.com/microsoft/CyberBattleSim

Cyber Battle Sim is a simulation arena used to automatically generate interactions be-
tween agents in a simulated network. It uses a network topology and pre-defined vul-
nerabilities to do this. It also uses the GymAI python library in order to successfully
implement reinforcement learning. There are many different simulations that can be
conducted with cyber battle sim. Some of them include an automated adversary, an
automated defender, or both! There are also different kinds of networks that can be
built. The platform is very customizable. For our purposes, we only used the automated
adversary on the network generated by Caldera pathfinder.
For our purposes, let’s define a node as representing a computer or machine, and an
edge as being a connection between two computers, or nodes. To use cyber battle sim
successfully, a very detailed network environment must be built from the pathfinder-
generated network. The following list shows the important components when creating a
cyber battle sim environment and simulation.

• Environment
The environment is defined by the network. This includes nodes, edges, vulnerabil-
ities, and much more.

• Action Space
The action space desccribes the set of actions that the agent (adversary) can take. In
cyber battle sim, these can be a local attack, remote attack, or authenticated
connection.

• Observation space
The observation space describes the set of facts that have already been observed.
This includes discovered nodes and credentials, privilege escalation levels, and avail-
able attacks.

• Reward
Basically the node value. The reward is given to the agent when a node is discovered
or owned.

3.1 Getting Started with Cyber Battle Sim

Installation Instructions:

1. Open Ubuntu and install the Cyber Battle Sim Github repository:

git clone https://github.com/microsoft/CyberBattleSim.git

2. Navigate into CyberBattleSim folder and install requirements:

7
 cd CyberBattleSim
pip3 install -r requirements.txt

3. Learn about cyber battle sim’s capabilities by exploring their notebooks, found at the
end of the home Github page. You can open the notebooks by typing this command into
your Ubuntu terminal, and opening http://localhost:8888 on your browser.

xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

You can open the notebooks by navigating to the ”notebooks” folder in the browser.
Look at toy-ctf.ipynb, and toy ctf.py(under cyberbattle/samples/toyctf) to see how an
environment is constructed and implemented.
NOTE: A great way to play around with cyber battle sim is to open a new notebook and
work through the existing notebooks line-by-line. This will give you a better feel of the
platform.

8
4 The Program
The main purpose of the program is to write a file that can be run to conduct a
simulation in cyber battle sim. The program is written in python. The program asks the
user to choose a file (the pathfinder report), parses it, and writes a new program with the
data gathered from the report, that can then be run to simulate an attack. The program
recreates the network in the same way that an environment is created in the toy ctf.py
file under cyberbattle/samples. For simplicity, the newly generated file/program will be
called cyber battle.py, and the report generated by caldera pathfinder, the vulnerability
report, will be called report.yml.

4.1 Section 1
1. The program starts with asking the user to choose a file from the local file directory.
This file should be the vulnerability report generated by pathfinder.

2. The pathfinder vulnerability report (report.yml) contains a lot of useful informa-

tion that is necessary to build the network model for cyber battle sim. To be able to
work with this data, the contents of the file need to be converted into a data structure
that is easier to navigate. full dict is the dictionary that encompasses all the data in
the report.

3. Since we are essentially recreating the network, we need to keep track of the nodes.
So, node names keeps track of the names, to make information about each node more
accessible from the dictionary, and to make it easier to loop across the nodes. 4. To create
the network, the vulnerabilities of each node must be listed in the node data structure
(as seen in toy ctf.py), along with information about each vulnerability.If you look at the
report, there is a list of CVEs associated with the first node. Each one of those is a vul-
nerability. However, there is no explicit information about them. Therefore, to be able to
include information about each vulnerability, I created a csv file listing all vulnerabilities
discovered so far (identified by their names), making it easier for the program to gather
information on each vulnerability. Essentially, the program calls a command to create a
dictionary from the csv file, making it easier to access this data.
NOTE: Most of the vulnerability information is speculated from the vulnerability name.
For example, the each vulnerability must be labeled as remote or local. I inferred this by
reading about each vulnerability and deciding whether it was susceptible to a remote or
local attack. For reference, the csv file looks like this and is called vulnerabilities.csv:

9
So in short, vulnerabilities dict stores all the vulnerability information.

4.2 Section 2
1. In this section we start writing the cyber battle sim file. So of course, a new file must
be opened.

2. The import statements are written to the new file (cyber battle.py)

3. In toy ctf.py, there is a method, register to register a newly created environment to

the AI Gym. So, this method is also written into the new file so that the environment
created can be registered when the program is run.

4.3 Section 3
1. At the beginning of toy ctf.py, there are several listening services labeled as default allow rules.
This allows us to configure the Firewalls and establish which listening services are permit-
ted for the attacker to use to gain access. These listening services are shown in report.ym.
Each port has a listening service associated with it. However, in cyber battle sim, we are
limited to using the four that are shown in toy ctf.py.

2. The program loops through all the nodes and their ports to collect the listening
services that can be used.

3. A checkIfListeningServiceValid method is called from methods.py. As shown

in methods.py, this only filters the non-usable listening services that are not supported
by cyber battle sim (see #1).

4. The next text to add to cyber battle.py is constructed using the new listening services
list, and written to the file.

10
4.4 Section 4
1. This section is where the topology of the network actually gets built. There is an
overarching loop that loops through each node in full dict. Each node has a set of
attributes that must be included (shown in toy ctf.py).

2. The first attribute shown is the listening services. The service for each port is in
report.yml. The listening services for all ports of one node are collected and then added
to cyber battle.py.
The firewall configuration is set to be the same for all the nodes, setting all outgoing
and incoming communication rules to the default allow rules that were previously
declared.

3. The value of the node is set to a constant. This can be changed if the we want
different nodes to have different values. The properties are also constant, and can be
changed however the user wants. Thee current properties are there as a placeholder.

4. The owned string property is the text that is printed when that node becomes
”owned”. This is set to a basic ”NODE node name OWNED”. Similar to other at-
tributes, this can be changed however the user wants.

5. This is the fun part. Next, the list of vulnerabilities must be added to the code.
The method getVulnerabilitiesText in methods.py is in charge of parsing the vulner-
abiltiies in report.yml and retrieving information about it from the vulnerabilities.csv file.
In this method, the outcome of each vulnerability is randomly picked from the ones listed
in vulnerabilities.csv (each vulnerability has a set of possible outcomes). The success
rate is set to the same for every vulnerability (this can be changed in the future). Each
outcome has a different set of attributes that must be defined as well. The vulnerability
text is then written to the file.

6. For each cyber battle sim network, there must be a starting point for the agent,
and the starting point must be a node that has vulnerabilities. The code sets the starting
point to be the first node encountered with vulnerabilities.

7. The above steps are repeated for each node in the network.

4.5 Section 5
1. This section writes the rest of cyber battle.py. The first writes the commands
needed to register our new environment. 2. The second part writes the steps to be taken
by the agent. The basic algorithm is, the agent tries random actions until it discovers
and owns the whole network. This is based off of the agent in toyctf-random.ipynb, in
the notebooks folder in cyber battle sim.
NOTE: The loop counters and stopping conditions should be modified. The current
values are placeholders.

So, if you run the program, a new file will be generated. You can run the new file,

11
and it will simulate an attack on the network that was generated from report.yml.

12
5 Common Errors
5.1 ”ModuleNotFoundError: No module named X"
This error means that you do not have the module X (could be any module) installed
on your machine, or for python3. To fix this error, simply install the library with this
command:

sudo pip3 install X

For example, if my error says, ”ModuleNotFoundError: No module named ’torch’”,

I will type

sudo pip3 install torch

5.2 ”AttributeError: module ’lib’ has no attribute

’Cryptography HAS TLSEXT HOSTNAME’”
Sometimes some files cannot be access due to the privilege level. To fix this error, simply
add sudo at the beginning of the command that is causing an error to run the command
with administrative privileges.

13