The Collective: A Cache-Based

System Management Architecture

NSDI 2005

April 5, 2006
Jaesun Han (jshan@nclab.kaist.ac.kr)
NCLAB, Dept. of EECS, KAIST

Network Computing Laboratory

 Contents
Problem statements
Motivation
Idea : Virtual Appliance
The Collective Architecture
CVL Language
Repositories
The Collective Utility
Cache-Based System Management

Network Computing Laboratory | 2 Korea Advanced Institute of Science and Technology

 Handled Paper List
Virtual Appliances in the Collective:
A Road to Hassle-Free Computing (HotOS 2003)
Describe the vision of the Collective (Why it is needed)
Virtual Appliances for Deploying and Management S
oftware (Usenix LISA 2003)
Describe the bird’s view of the Collective
The Collective: A Cache-Based System Management
Architecture (NSDI 2005)
Focus on the cache-based mechanism for management

Network Computing Laboratory | 3 Korea Advanced Institute of Science and Technology

 Problem Statements
Two incidents to security patches
January 25, 2003 : SQL slammer worm attack (six months before, t
he vulnerability of MS SQL Server 2000 was patched)
June 5, 2003 : BugBear.B virus at Stanford Univ. (two years befor
e, the patch to the hole was issued)
Difficulties in managing distributed PCs
Purchasing the equipment and software, installing the software, tr
oubleshooting errors, performing upgrades and re-installing OS, pe
rforming backups, and finally recovering from problems caused by
mistakes, viruses, worm and spyware.
Solutions
Thin-client computing ?
Utility computing ?
 Centralized Management
Network Computing Laboratory | 4 Korea Advanced Institute of Science and Technology
 Motivation : Computing Appliances
Computing Appliances
Special-purpose computer hardware
Examples: firewalls, VPN gateways, game consoles, TiVos, file ser
vers, Google search appliances
Benefits
Easy to use and maintain
Strong isolation between two appliances
Maintained by the makers, not the users (for networked appliances)
Ref. http://en.wikipedia.org/wiki/Appliance

Network Computing Laboratory | 5 Korea Advanced Institute of Science and Technology

 Idea : Virtual Appliances
Hint : Computing environments as groups of appliances
Having an appliance for each application
Case1) AOL appliance, office suite appliance, video editing appliance
Case2) office suite appliance for work, office suite appliance for home
But, hardware appliances are expensive
Virtual Appliances
Providing a fixed function for the user without dedicated hardware

The state of a real appliance

Software VM
(the contents of
(OS & APP) (OS & APP)
the appliance’s disk)

Description of the hardware

Hardware
Computing Appliance
Network Computing Laboratory | 6
Virtualization
(Virtualized HW)
(e.g. two Ethernet,
512MB RAM)
Virtual Appliance
Korea Advanced Institute of Science and Technology
 Virtual Appliances
Virtual Appliances
A virtual machine on VMM (VMware GSX server)
Network  Using a virtual network
Storage  Using a network storage service
I/O devices (display, printer etc)  Talking over the network
Virtual Appliance Network (VAN)
Package of virtual appliances (VAP)
Examples : Groupware VAN, APM VAN, X Project VAN, SOHO VAN etc

Manager Office Browser

Virtual Virtual Virtual Dynamically installed
Appliance Appliance Appliance
on-demand
Virtualized Virtualized Virtualized
HW HW HW

Host OS & VMM

Initially installed
PC Hardware when delivered

Network Computing Laboratory | 7 Korea Advanced Institute of Science and Technology

 The Collective : Big Picture
CVL Publisher
Virtual Appliance Network

VAP1 VAP2 VAP3 VM

Virtual
Switch VM Download
& Update VM VM VM VM
Host Manager
VM VM VM VM
VM VM VM VM

Control
& Manage
Appliance Repository

VAN
Resourse CVL
Allocation

User
Compute Utility
Central Cluster Manager

Network Computing Laboratory | 8 Korea Advanced Institute of Science and Technology

 CVL Language
CVL : Collective Virtual appliance Language
Describe a VAP and a VAN
Interface {
Disks in VM Appliances var "required" mac, ip, subnet, netmask;
var defaultroute;
program disks & data disks }

datadisks specify all data disks Appliance {

var requires, provides;
Network Topology var "required" vanIF;
}
A virtual interface per VM appliance
VMAppliance extends Appliance {
Collective Utility assigns MAC, var "required" vm; I
var datadisks;
P address to each interface Interface ethernet0;
vanIF = "ethernet0";
Components in a VAN are connected }
to
the same Ethernet segment
VANAppliance extends Appliance {
Dependancies between appliances }
var defaultroute;

Determine bootup sequence

Pre-defined objects in CVL
provides & requires

Network Computing Laboratory | 9 Korea Advanced Institute of Science and Technology

 Groupware Component VAPs CVL
Groupware VAN CVL
Network Computing Laboratory | 10 Korea Advanced Institute of Science and Technology
 Repositories
Appliance Repository
Can be hosted anywhere in the file system where a user can creat
e subdirectory
Hold the versions of a single appliance (immutable)
CVL file
VMware virtual machine files (.vmx, .vmdk, and .vmss)
UNIX Collective User Interface command
cui create <repository> : create empty repository
cui publish <repository> <cvl>
: store the files as the latest version of the appliance in the repository
program disks  repository

Network Computing Laboratory | 11 Korea Advanced Institute of Science and Technology

 The Collective Utility
Roles
Resource registration & management
Resource allocation and configuration to VAPs
Blacklisting
Starting a VAP (cui start <cvl>)
1. Set up the program and data disks for all the component VM appl
iances in the directory containing the CVL file
2. Ensure that all the required services are available, required para
meters set, and required resources reserved
3. Brings up the VAN
Stopping a VAP (cui stop <cvl> [<comp>])
Updating a VAP (cui update <cvl> [<comp>])
Only the affected VM appliances are updated

Network Computing Laboratory | 12 Korea Advanced Institute of Science and Technology

 Example Appliances
Groupware Appliance

Software Development Environments

Include source codes, all required tools and libraries
Windows Appliances
Office 2000 Appliance and Internet Explore 6 Appliance
Not easy because system properties don’t reside in simple text file and
user data are tightly coupled with program data
To share program data and configuration settings between programs in
different appliances, Samba appliance is used
Network Computing Laboratory | 13 Korea Advanced Institute of Science and Technology
 Cache-Based System Management
User state System state
(user’s profile, (OS & all installed apps)
preferences, data files)

Virtual Appliance Transceiver

1. authenticates users
2. fetches and runs the latest copies
of appliances locally
3. backs up user state changes to the
data repository continuously
4. optimizes the system by managing
a cache to reduce delivered data

Using local cache

Reduce download time of data blocks from repositories
Adapt to various network conditions
Use prefetching to minimize cache miss
Network Computing Laboratory | 14 Korea Advanced Institute of Science and Technology
 System Overview
Data storage
system disks, user disks, ephemeral disks, memory image
User Interface
start, stop, suspend, reset, delete, user disk undo, publish
Management functions
System updates, Machine Lockdown (rolling back the installation
of undesirable software in shutdown), Backup (user disks)
Optimizations for different network connectivities
VAT includes a large on-disk cache
Keep local copies of the system and user disk blocks
Fetching data on demand + Prefetching data
LAN, WAN with Broadband, Disconnected Operation with Laptops,
Portable VATs, Remote Display

Network Computing Laboratory | 15 Korea Advanced Institute of Science and Technology

 Design of the VAT
KNOPPIX – VMware GSX server – VAT manager
VAT Maintenance
Automatically updated (with rsync)
VAT image is about 350 MB  160 MB compressed disk image
Storage Access
Repository is immutable
NFS over SSH : support demand paging
Caching
Small object cache for small data and meta-data CO
W cache for COW disk blocks
No invalidation is necessary
Prefetching
Minimize cache miss

Network Computing Laboratory | 16 Korea Advanced Institute of Science and Technology

 Bootup time evaluation

Network Computing Laboratory | 17 Korea Advanced Institute of Science and Technology

 VMware Virtual Appliance
http://www.vmware.com/appliances/
http://www.vmware.com/vmtn/appliances/

Network Computing Laboratory | 18 Korea Advanced Institute of Science and Technology