1

I. ADDITIONAL EVALUATION RESULTS

A. Environment
The tests have been performed in a virtualized environment with Mininet 1.0.0 [?]. Mininet is tool to create a
virtual network running actual kernel, switch and application code. It is an ideal way to test SDN systems, and is
quite exible about the possible topologies (the topology is dened with Python code).
B. Procedure
Here is the general procedure used to create and run the tests.
The rst step is to design the topology and write down the constraints. Lets consider that the le containing the
constraints is named test42
_
cstr.txt. Then we have to translate the topology into Mininets language. We have
to modify the le test
_
topo.py and add a class extending Mininets Topo class. Lets call this class Test42.
In the constructor we write the instructions necessary to describe the desired topology (switches, hosts, collector,
edges). This class will be used by Mininet to build the topology.
We also need a mapping between the host identiers and actual IP and MAC addresses. Information about how
to contact the switches OpenFlow port is required. This mapping may vary between different scenarios but we use
a generic one for the tests. It is written in the le mapping.txt.
The tools we wrote also require the topology and they derive it from the output of Mininets net command. This
means that we have to run Mininet in order to get this output. We usually launch it with the following command:
sudo mn -custom test
_
topo.py -topo test42 -mac -switch ovsk -controller remote. This tells
Mininet to launch a network described by the class mapped by the test42 identier in test
_
topo.py, to use
OpenVSwitch-based switches, to set the MAC address of the nodes equal to their identier and to use a remote
controller which defaults to localhost. We can then retrieve the output of the net command and write it into a
test42
_
topo.txt le.
The test network is now running but there is currently no rules dened in the OpenFlow switches. We wrote a
script that generates basic routing rules from the mapping and topology les. It can generate the following classes
of rules: the broadcast rule, which instructs the switches to forward to all active ports (except the input port)
the packets with destination MAC address ff:ff:ff:ff:ff:ff; rules to reach hosts directly connected to the
switches; rules to reach hosts connected to direct neighboring switch; and nally rules to reach remote hosts, i.e.
hosts with a network distance of at least two hops from a given switch. See Figure 1 for details. When more specic
rules are required, they are manually written.
Fig. 1. Three node distance types with respect to the green-squared switch
Once the rules have been inserted in the switches ow table, we can start running the test. First we have to
run the collector with a proper capture timeout, then run the packets generator. Once the collector has captured the
2
test packets, it outputs the relevant data in JSON format which can then be given to the constraints checker. The
checker then checks each constraint and outputs the results.
C. Binary connectivity
Here we present different tests to check binary connectivity. Let us consider the network shown in Figure 2. We
provide the following constraints:
1) Hosts h14 and h34 can reach any other host.
2) Any host can reach h14 and h34.
3) Groups isolated1 and isolated3 cannot communicate.
Fig. 2. Test network for binary connectivity
The following atoms are declared.
isol1(h11)
isol1(h12)
isol1(h13)
isol3(h31)
isol3(h32)
isol3(h33)
Fig. 3. Atoms for binary connectivity test network
3
Specic rules have been dened in order to verify that the binary connectivity constraints is working correctly in
different cases: packets from h11 to h31 are dropped at s1; packets from h12 to h32 are dropped at s2; packets
from h13 to h33 are dropped at s3. This allows us to verify the constraints with empty paths and partial paths.
See Table I for the different test constraints. The fth column (Checked) refers to the previously enumerated
constraints. For each test we provide also the opposite. The rst and second tests partially check the rst constraint
(i.e. hosts h14 and h34 can reach any other host) by checking the trafc between h14 and h34. We expect both
constraints to be statised.
The third, fourth and fth tests deal with the empty/partial path issue. Indeed, the 3
rd
test will not produce any
packet for the collector as the injected packets will be immediately dropped at switch s1. The 4
th
test will produce
a partial path of length 1, i.e. the path [s1]. The 5
th
test will produce a partial path of length 2, i.e. [s1 s2].
The last three tests use atom checks and combinations of equality and atom checks. They check the second and
third constraints.
Test Constraint Conditions Note Checked Veried
1
allow
H
s
= h
14
H
t
= h
34
Equality First
Yes
deny No
2
allow
H
s
= h
34
H
t
= h
14
Equality First
Yes
deny No
3
deny
H
s
= h
11
H
t
= h
31
Empty path Third
Yes
allow No
4
deny
H
s
= h
12
H
t
= h
32
Partial path Third
Yes
allow No
5
deny
H
s
= h
13
H
t
= h
33
Partial path Third
Yes
allow No
6
deny
isol1(H
s
) isol3(H
t
) Atoms Third
Yes
allow No
7
allow
isol1(H
s
) H
t
= h
34
Atom, equality Second
Yes
deny No
8
allow
isol3(H
s
) H
t
= h
14
Atom, equality Second
Yes
deny No
TABLE I
BINARY CONNECTIVITY TESTS
The implementation is thus working as expected regarding the binary connectivity constraints. See Table II for
data on the time spent to perform this test.
Generation 46.97
Injection 30.65
Collection 29.37
Checking 0.357
TABLE II
TIME SPENT BY THE DIFFERENT COMPONENTS. THE TIME IS GIVEN IN MILLISECONDS.
D. Path constraints
Let us consider the network shown in Figure 4. We dene the following constraints.
1) The path from s1 to s4 must go through s2.
2) The path from s4 to s1 must go through s3.
3) The path from s1 to s5 must go through s3.
4) The path from s5 to s1 must go through s2.
5) The path from s2 to s6 must go through s4 or s5.
6) The path from s3 to s6 must go through s4 or s5.
4
7) All the paths must contain the minimum number of hops.
Fig. 4. Test network for path constraints
The controller is written in the following way. It has the list of path constraints described above. If it receives a
packet whose destination is path-constrained, then it attempts to nd the shortest path to the required intermediary
switch, and retrieve the correct output port. Then it instructs the switch that sent the packet to use this port for
all future packets with the same destination MAC by adding an entry to the switchs ow table. If the controller
receives a packet whose destination is not path-constrained, it will simply attempt to nd the shortest path to the
destination, and modify the switchs ow table to use the computed path for future packets.
In order to precisely dene the paths in the network we have used the -arp argument when launching Mininet.
This argument allows the hosts to start with a full ARP table, thus avoiding the need for ARP broadcasts. Indeed,
without using a protocol such as STP, all the switches ports will remain active and an ARP broadcast will indenitely
loop in the network.
The following atoms are declared.
5
one(h11)
one(h12)
two(h21)
two(h22)
three(h31)
three(h32)
four(h41)
four(h42)
five(h51)
five(h52)
six(h61)
six(h62)
Fig. 5. Atoms for path constraints test network
See Table III for a summary of the tests. They consist in verifying each constraint by setting different kinds of
paths: strict paths (fully dened for each hop), partial paths (by using the Kleene operator), multiple paths (with
the union operator) and length constraints (with the dot symbol).
Test Path Conditions Note Checked Veried
1 s1,s2,s4 one(H
s
) four(H
t
) Strict path First Yes
2 s4,s3,s1 four(H
s
) one(H
t
) Strict path Second Yes
3 .
*
,s3,.
*
four(H
s
) one(H
t
) Partial path Second Yes
4 s1,s3,s5 one(H
s
) five(H
t
) Strict path Third Yes
5 s5,s2,s1 five(H
s
) one(H
t
) Strict path Fourth Yes
6 .
*
,s2,.
*
five(H
s
) one(H
t
) Partial path Fourth Yes
7 s2,s4|s5,s6 two(H
s
) six(H
t
) Union Fifth Yes
8 s2,.
*
,s6 two(H
s
) six(H
t
) Edges constrained Fifth Yes
9 s3,s4|s5,s6 three(H
s
) six(H
t
) Union Sixth Yes
10 s3,.
*
,s6 three(H
s
) six(H
t
) Edges constrained Sixth Yes
11 .... one(H
s
) six(H
t
) Length constraint Seventh Yes
12 .
*
one(H
s
) six(H
t
) No constraint None Yes
13 .
*
,s3,.
*
one(H
s
) two(H
t
) Partial path Seventh No
14 .. six(H
s
) one(H
t
) Impossible None No
15 s1,s3,s4 one(H
s
) four(H
t
) Strict path First No
TABLE III
PATH CONSTRAINTS TESTS
The implementation is behaving as expected. See Table IV for data on the time spent when performing this test.
Generation 69.27
Injection 53.37
Collection 38.72
Checking 4.73
TABLE IV
TIME SPENT BY EACH COMPONENT FOR THE PATH CONSTRAINTS TEST. THE TIME IS GIVEN IN MILLISECONDS
E. Load balancing
Let us consider the network shown in Figure 6. We dene the following constraints.
1) The path from s1 to s4 must be equally load balanced between s2 and s3.
2) The path from s4 to s1 must be load balanced by 70% through s2 and by 30% through s3.
6
Fig. 6. Test network for load balancing
The lack of support of multipath rules in the v1.0.0 of the OpenFlow protocol has forced us to make some
modications to the test procedure for this case: At switch s1, the packets whose destination is directly connected
to s4 are sent to the controller, and conversely for the switch s4. The controller handles the load balancing in a
ow-based manner. The controller randomly assigns an output port to each different ow (the random number is
of course weighted for each link).
The following atoms are declared.
one(h11)
one(h12)
four(h41)
four(h42)
Fig. 7. Atoms for load balancing test network
See Table V for a summary of the test. Note that each constraint is tested independently, i.e. the share of trafc
going through a given path is not derived from the trafc seen on another path. The rst three tests are the same
and ran with 10 samples per ow. The next three ran with 100 samples and the last test ran with 1000 samples.
These samples were UDP packets whose source port was randomized, ensuring the load balancer would treat them
as seperate ows. We can see that as the number of samples increase, the deviation decrease and the values become
more precise. The implementation is working as expected, but for future work, we could dene an acceptable
deviation related to the number of samples and reject the constraint if the result is outside the accepting range.
7
Test Path Conditions Samples Target Result
1
.
*
,s2,.
*
one(H
s
) four(H
t
) 10 0.5 0.55
.
*
,s3,.
*
one(H
s
) four(H
t
) 10 0.5 0.625
.
*
,s2,.
*
four(H
s
) one(H
t
) 10 0.7 0.65
.
*
,s3,.
*
four(H
s
) one(H
t
) 10 0.3 0.225
2
.
*
,s2,.
*
one(H
s
) four(H
t
) 10 0.5 0.425
.
*
,s3,.
*
one(H
s
) four(H
t
) 10 0.5 0.6
.
*
,s2,.
*
four(H
s
) one(H
t
) 10 0.7 0.675
.
*
,s3,.
*
four(H
s
) one(H
t
) 10 0.3 0.275
3
.
*
,s2,.
*
one(H
s
) four(H
t
) 10 0.5 0.35
.
*
,s3,.
*
one(H
s
) four(H
t
) 10 0.5 0.5
.
*
,s2,.
*
four(H
s
) one(H
t
) 10 0.7 0.725
.
*
,s3,.
*
four(H
s
) one(H
t
) 10 0.3 0.2
4
.
*
,s2,.
*
one(H
s
) four(H
t
) 100 0.5 0.542
.
*
,s3,.
*
one(H
s
) four(H
t
) 100 0.5 0.522
.
*
,s2,.
*
four(H
s
) one(H
t
) 100 0.7 0.715
.
*
,s3,.
*
four(H
s
) one(H
t
) 100 0.3 0.313
5
.
*
,s2,.
*
one(H
s
) four(H
t
) 100 0.5 0.564
.
*
,s3,.
*
one(H
s
) four(H
t
) 100 0.5 0.547
.
*
,s2,.
*
four(H
s
) one(H
t
) 100 0.7 0.724
.
*
,s3,.
*
four(H
s
) one(H
t
) 100 0.3 0.298
6
.
*
,s2,.
*
one(H
s
) four(H
t
) 100 0.5 0.534
.
*
,s3,.
*
one(H
s
) four(H
t
) 100 0.5 0.518
.
*
,s2,.
*
four(H
s
) one(H
t
) 100 0.7 0.708
.
*
,s3,.
*
four(H
s
) one(H
t
) 100 0.3 0.32
7
.
*
,s2,.
*
one(H
s
) four(H
t
) 1000 0.5 0.49
.
*
,s3,.
*
one(H
s
) four(H
t
) 1000 0.5 0.51
.
*
,s2,.
*
four(H
s
) one(H
t
) 1000 0.7 0.707
.
*
,s3,.
*
four(H
s
) one(H
t
) 1000 0.3 0.302
TABLE V
LOAD BALANCING TESTS
See Table VI for data on the time spent performing this test.
Samples Total packets Generation Injection Collection Checking
100 3200 1.815 1.686 0.624 0.022
1000 32000 19.169 32.165 6.581 0.186
TABLE VI
TIME SPENT BY EACH COMPONENT WHEN PERFORMING THE LOAD BALANCING TESTS. THE TIME IS GIVEN IN SECONDS.
F. Delay constraints
Let us consider the network shown in Figure 8. We would like to dene a delay constraint between s1 and s3.
The OpenFlow protocol as of v1.0.0 is not able to induce articial delays. To circumvent this we generated a sample
of 1000 test packets and we ran several delay checks on the collected traces. Over the sample, the mean latency
is 1.474306 ms with a standard deviation of 2.103613 ms and minimum, maximum latency of resp. 0.367880 ms
and 27.975082 ms.
Fig. 8. Test network for delay constraints
See Figure 9 for the details of the measurements. We clearly see that almost 90% of the measurements are under
1.47 ms. Not visible on the graph, we also dened two border checks, one at 0.366 ms and another at 27.976
8
ms, which is resp. 0.001 ms below and above the minima and maxima. As expected, the rst border check gave a
matching ratio of 0.0 (all measurements are above the value) and the second one gave a matching ratio of 1.0 (all
measurements are under the value). The implementation is thus working correctly.
As for the load balancing constraints, it can be useful to derive a deviation parameter in order to set an accepting
range for the delay constraints.
Fig. 9. Matching ratio for delay checks over increasing delay values