Running Condor in a Virtual

Environment with coLinux

Sumanth J.V
(sumanth@cse.unl.edu)
University of Nebraska -
Lincoln

Sumanth J.V, RCF/UNL, Condor W 1

eek 2006
 Motivation

• Most of the machines on

campus run Microsoft Windows.
• Most researchers jobs require
a UNIX environment.

Sumanth J.V, RCF/UNL, Condor We 2

ek 2006
 Initial Solution

• Many machines can dual boot

Linux and Windows.
• Reboot to Linux
• During after hours.
• Patched GRUB.
• Improvements
• Can we achieve better
utilization?
• Can we be less intrusive?
Sumanth J.V, RCF/UNL, Condor We 3
ek 2006
 Next Solution

• Virtualization
 Allows for better utilization
and is less intrusive.
• Picking a virtualization
solution
 VMWare
 Cooperative Linux

Sumanth J.V, RCF/UNL, Condor We 4

ek 2006
 VMWare

• Full system emulator.

• Guest OS runs unmodified.
• Emulation can be resource hungry
 intercepts most of the guest-kernel-to-
application communications.
 has to model a complete set of virtual
hardware devices.
 VMWare itself requires significant
memory.
• At the time of deployment, VMWare was
not free.
Sumanth J.V, RCF/UNL, Condor We 5
ek 2006
 coLinux

• Allows the Linux kernel to run

natively.
 As a Windows device driver.
• No bridge between the host kernel and
the guest kernel.
 Guest can run at near native speeds.
• Allocate memory only for kernel and
its forked processes.
• Windows service for guest OS startup.
• Requires a patched Linux kernel.
• Requires manual configuration.
Sumanth J.V, RCF/UNL, Condor We 6
ek 2006
coLinux Internals

Copyright - Dan Aloni

Sumanth J.V, RCF/UNL, Condor We 7

ek 2006
 Performance

POV-Ray execution time

Platform Time (min:sec)

Native Linux 39:23
coLinux 39:26
VMWare 40:53

Source: Build a heterogeneous cluster with coLinux and

openMosix - IBM developerWorks

Sumanth J.V, RCF/UNL, Condor We 8

ek 2006
 Condor on coLinux

• How to determine NonCondor

load average?
• How to determine
keyboard/mouse activity?
• What about networking?

Sumanth J.V, RCF/UNL, Condor We 9

ek 2006
 Windows Monitoring Service

• Monitors CPU load and coLinux load

 Difference approximates NonCondor
load.
• Monitors Keyboard/Mouse idle time.
• Makes this information available
to coLinux.
 The monitoring service binds to the
virtual networking device.

Sumanth J.V, RCF/UNL, Condor We 10

ek 2006
 coLinux Networking

• Bridged Networking
 Uses WinPcap.
 Advantage: Simple setup.
 Disadvantage: Requires a real IP.
• NAT
 Uses a TAP-Win32 networking device.
 Windows performs the NAT.
 coLinux does not have a real IP.
 Requires GCB or OpenVPN to handle inbound
connections.
 OpenVPN does not open up as many ports as GCB
and works with Windows and other OSs too.
 Other apps services like NFS can also work
behind a NAT with OpenVPN
Sumanth J.V, RCF/UNL, Condor We 11
ek 2006
 coLinux Networking

• All coLinux installs have the same

local IP e.g 192.168.0.40.
• How to identify nodes in a large
install?
 Change MAC address of TAP-Win32 device
to contain real host’s IP. E.g.
00:11:81:5D:44:04 on host 129.93.68.4.
 Set hostname on coLinux based on MAC
address. E.g. node-68-4 on
129.93.68.4.

Sumanth J.V, RCF/UNL, Condor We 12

ek 2006
 Configuring Condor

• host_stats.sh
 A simple script (on coLinux) to
connect to the monitoring
service.
 Prints out the Total Load, Condor
Load and keyboard/mouse idle
time.
• HostOsLoad = 0.01
• CoLinuxLoad = 0.02
• HostOsKeyboardIdle = 25
Sumanth J.V, RCF/UNL, Condor We 13
ek 2006
 Configuring Condor

• condor_config
 Setup STARTD_CRON to
periodically execute
host_stats.sh
 Suitably edit UWCS policy.
 GCB stuff.
• Why not install Condor on
Windows also?

Sumanth J.V, RCF/UNL, Condor We 14

ek 2006
 Automation of Installs

• gen_xml.exe
 Determines the name of the required real
network adapter.
 Generates MAC address for virtual network
device based on host’s IP.
 Writes coLinux’s XML file.
• Non-Interactive coLinux install.
• Setup Windows services.
• Unzip disk images.
• Can perform complete install by popping in
an install CD.
• Template of this CD will be made available.
Sumanth J.V, RCF/UNL, Condor We 15
ek 2006
 Acknowledgements

• PRISM (Priority Research In

Simulation & Modeling)
• Information Services
• Computer Science &
Engineering
• Research Computing Facility

Sumanth J.V, RCF/UNL, Condor We 16

ek 2006
 Thank You
Questions/Comments?

Contact for more

details/updates
sumanth@cse.unl.edu

Sumanth J.V, RCF/UNL, Condor We 17

ek 2006