OpenStack

 and  OpenContrail  on  

FreeBSD  pla4orm  

Michał  Dubiel  
EuroBSDCon  2014,  Sofia,  Bulgaria  
 Outline  
•  OpenStack  
–  IntroducDon  
–  Nova  Compute  driver  
–  Nova  Network  driver  
•  OpenContrail  
–  Why?  
–  Overlay  networks  vs  vlans  
–  SoLware  architecture  
•  Status,  next  steps  
 Machines  in  a  datacenter  
VM   VM   VM   VM   VM   VM   VM   VM  

hypervisor   hypervisor  

MIGRATIONS  

VM   VM   VM   VM   VM   VM   VM   VM  

hypervisor   hypervisor  

Storage  appliance  
Cloud  operaDng  system  

source:  openstack.org  
 OpenStack  introducDon  
•  “Massively  scalable  cloud  operaDng  system”  
•  Aims  to  manage  
– Compute  
– Storage  
– Network  
•  Major  components  
–  Compute  (Nova)  
–  Storage  (SwiL,  Cinder)  
–  Networking  (Neutron)  
 
 OpenStack  components  
•  There  is  a  lot  of  soLware  components  
–  Nova  (compute  manager,  networking  manager,  
scheduler,  etc.)  
–  Neutron  (controller,  agents)  
–  Glance  (image  service)  
–  API  servers  
–  Message  queues  
–  Etc.  
•  FreeBSD  support  is  about  the  compute  node  
–  Depended  on  underlying  OS  pla4orm  
–  Another  hypervisor  (bhyve)  
Networking  service  -­‐  OpenContrail  

source:  openstack.org  
 Compute  node  
Nova  network   Nova  
server   Scheduler  

VM   VM   VM  

Nova   Nova  
network   compute  

Libvirt  

Bridge   bhyve  
Kernel  space  
 OpenStack  compute  node  
•  nova-compute:  manages  compuDng  
instances  on  host  machines  
–  Run/terminate/reboot  instances  
–  Aaach/detach  volumes  
–  Console  output  
•  nova-network:  manages  networking  
resources!
–  Responsible  for  sebng  up  networking  between  VMs  
–  Simple  soluDons  (bridges,  vlans,  etc.)  
 Spawning  a  VM  
•  Nova  scheduler  choses  a  compute  node  for  a  VM  
•  The  nova-­‐compute  fetches  the  VM  image  from  
glance  service  
•  The  nova-­‐compute  builds  a  libvirt  XML  definiDon  
for  the  VM  
•  The  nova-­‐network  configures  bridge  for  VM  
networking  
•  The  nova-­‐compute  invokes  libvirt  and  spawns  the  
VM  
–  Libvirt  adds  tap  device  to  the  bridge  connecDng  that  
way  the  VM  to  the  virtual  network  
 FreeBSD  Development  
•  Libvirt  support  for  bhyve    
–  Work  of  Roman  Bogorodskiy  
–  Few  new  features  and  fixes!
•  nova-compute  adjustments  for  new  
hypervisor  type  –  bhyve  
•  nova-­‐network  support  for  FreeBSD    
–  ifconfig  vs.  brctl,  ip  tool  
–  dnsmasq    
•  Devstack  support  for  FreeBSD  
Networking  service  -­‐  OpenContrail  

source:  openstack.org  
Rack,  servers,  VMs  

VM   VM   VM   VM  

hypervisor  

VM   VM   VM   VM   To  spine  switch  

hypervisor  

VM   VM   VM   VM  

hypervisor  

Server  rack  
 Datacenter  architecture  
Clos  network  
 ObservaDons  

•  Majority  of  network  endpoints  are  virtual  

•  Network  isolaDon  between  them  has  to  
be  available  
•  While  using  the  same  physical  network  
•  Endpoint  may  migrate  from  one  physical  
host  to  another  
 SoluDons  
•  Bridges  +  vlans  
–  What  nova-­‐network  provides  
–  Limited,  not  flexible  
•  Overlay  networking  (OpenContrail)  
–  Available  as  a  Neutron  plugin  
–  Flexible  
–  Scalable  
 VLANs  
•  VM’s  interfaces  placed  on  bridges  
–  Each  bridge  for  a  virtual  network  
•  4096  VLAN  tags  limit  
–  Can  be  extended  using  Shortest  Path  Bridging  
•  Difficult  to  manage  
•  Physical  switches  has  to  contain  the  VN  state  
 VM  migraDon  example  

VM1   VM2   VM4   VM5  

VM3   Physical  switch   VM6  

1   2  
Server  1   Server  2  
3  

VM7   VM8  

VM9  

Virtual  networks:  
Server  3  
 VM  migraDon  example  

VM1   VM2   VM4   VM5  

VM3   Physical  switch   VM6  

1   2  
Server  1   Server  2  
3  

VM7   VM8  
VM9   Payload  
VM9  

Eth  +  VLAN  tag  +  IP   Virtual  networks:  

Server  3  
 VM  migraDon  example  

VM1   VM2   VM4   VM5  

VM3   Physical  switch   VM6   VM9  

1   2  
Server  1   Server  2  
3  

VM7   VM8  
VM9   Payload  

Eth  +  VLAN  tag  +  IP   Virtual  networks:  

Server  3  
 Overlay  networking  
•  “Old”  technology,  relaDvely  new  for  data-­‐
centers  
•  Physical  underlay  network  
–  IP  fabric  
–  No  tenant  state  
•  Virtual  overlay  network  
–  Tenant  state  
–  Dynamic  tunnels  (MPLSoGRE,  VXLAN,  etc.)  
 VM  migraDon  example  

VM1   VM2   VM4   VM5  

VM3   Physical  switch   VM6  

1   2  
Server  1   Server  2  
3  

VM7   VM8  
S3   VM9   Payload  
Physical  network:  
VM9  

Eth  +  IP   Virtual  networks:  

Server  3  
 VM  migraDon  example  

VM1   VM2   VM4   VM5  

VM3   Physical  switch   VM6   VM9  

1   2  
Server  1   Server  2  
3  

VM7   VM8  
S2   VM9   Payload  
Physical  network:  

Eth  +  IP   Virtual  networks:  

Server  3  
 Advantages  
•  “Knowledge”  about  network  only  in  the  
soLware  (Controllers,  compute  nodes)  
•  Any  switch  works  for  IP  fabric  network  
–  No  configuraDon  
–  Only  speed  maaers  
–  Lower  price  
•  In  case  of  OpenContrail  standards-­‐based  
(MPLS,  BGP,  VXLAN,  etc.)  
SDN  in  cloud  orchestraDon  

Source:  www.opencontrail.org  
Architecture  overview  

Source:  www.opencontrail.org  
ConfiguraDon  node  

Source:  www.opencontrail.org  
Controller  node  

Source:  www.opencontrail.org  
 Compute  node  
Contrail  Control   Nova  
node   Scheduler  

VM   VM   VM  

Contrail   TCP   Nova  vif   Nova  

Agent   driver   compute  

NetLink  
/dev/flow  
Libvirt  
pkt  

Contrail  
bhyve  
vRouter  
Kernel  space  
vRouter  forwarding  plane  

Source:  www.opencontrail.org  
MPLSoGRE  example  

Source:  www.opencontrail.org  
 OpenContrail  summary  
•  High-­‐level  descripDon  of  networks  
–  allow    any    src-­‐vn  -­‐>  dst-­‐vn    svc-­‐1,  svc-­‐2  
•  Horizontally  scalable  
•  Fault  tolerant  
•  Works  with  exisDng  equipment  
•  Open  sourced  (FreeBSD  support  included  in  
official  repos)  
AnalyDcs  node  

Source:  www.opencontrail.org  
 FreeBSD  development  
•  vRouter  kernel  module  
–  New  module  
–  Common  parts  OS  agnosDc  (/dp-core)
–  FreeBSD  related  code  (/freebsd)  
•  Agent  support  for  FreeBSD  
–  Ioctls,  tap  devices  
–  Shared  memory  (/dev/flow)  
–  Listener  
–  Lots  of  refactoring  done  
 TODOs  
•  Libvirt  improvements  
•  OpenStack  improvements  
–  Support  limited  by  libvirt  capabiliDes  on  FreeBSD  
–  Firewal  (pf,  ipfw,  ipfilter)  
–  Currently  a  fork  of  nova  is  required  
•  Different  OpenContrail  operaDon  modes  
–  MPLSoUDP,  VXLAN,  etc.  
•  AutomaDc  provisioning  
–  Contrail-­‐installer  scripts    
–  Devstack  
  
 
 
Any  quesDons?