Why Assumptions about Cloud Performance Can Be Dangerous to Your Business

Ron Wilson, Director Cloud Strategy Ron.Wilson@compuware.com

Agenda
Why Does Performance Matter?
Brief Overview of Web and Cloud Performance Challenges Real-World Data: How Are Cloud Providers Performing? Cloud Optimization Opportunities Key Takeaways Q&A

Why Performance Matters: Revenue

found that a

second slowdown

4.3

% reduction in %

revenue/user*

stated that a

millisecond delay

400

fewer searches/users*

0.59

Noticed that users who experience the fastest page load times view % more pages/visits than users experiencing the slowest page load times*

50

reduced page load times from ~7 seconds to ~2 seconds, leading to a % increase in revenue and % reduction in hardware costs*

712

50

Source: Steve Souders @ Velocity Conference 2009

Why Web Performance Matters: Customer Satisfaction

40% of Consumers will abandon a site if made to wait
Consumer expectations for how quickly a web page should load
less than 1 second 1 second 2 seconds 3 seconds 5% 12% 30%
2 seconds

How long consumers will wait for a page to load before abandoning
less than 1 second 1 second

1% 2% 10% 27% 60%

0% 20% 40% 60% 80%

36%
3 seconds

More than 4 seconds

0% 10%

17%
More than 4 seconds
20% 30% 40%

Lost revenues Increased costs Reduced customer satisfaction LOB dissatisfaction with IT

eCommerce Web Site Performance Today white paper August 2009

Why Performance Matters: Cloud Adoption

Typical Web Application Delivery

The Web Application Delivery Chain
Load Balancers Web Servers Mobile Components App Servers DB Servers Storage

3rd Party/ Cloud Services

Local ISP

Browsers and devices

Users

Internet Major ISP

Mainframe
Network

Content Delivery Networks

Mobile Carrier

Traditional zone of control

The Challenge of Ensuring Quality Web Experiences

Systems management tools: OK user is NOT happy

The Web Application Delivery Chain 3rd Party/ Cloud Services

Inconsistent geo performance Bad performance under load Blocking content delivery Incorrect geo-targeted content Network peering Internet problems Outages

Load Balancers

Local ISP

Browsers and devices

Users

Web Servers
Mobile Components

App Servers
DB Servers Storage Mainframe Network

Configuration errors Application design issues Code defects Insufficient infrastructure

Network peering problems Bandwidth throttling Inconsistent connectivity

Poorly performing JavaScript Browser/device incompatibility Page size too big Too many objects Low cache hit rate

Major ISP

Configuration issues Oversubscribed POP Poor routing optimization Low cache hit rate

Content Networks

Network resource shortage Faulty content transcoding SMS routing / Deliverylatency issues Mobile

Carrier

Traditional zone Traditional zone of control of control

Zone of customer expectation of expectationexpectation Zone of customer customer Zone

The Business Impact of Poor Web Experiences

Systems management tools: OK
user is

NOT happy

The Web Application Delivery Chain

3rd Party/ Cloud Services
Ineffective SLAs and partner relationships

Load Balancers

Local ISP

Browsers and devices

Users

Web Servers
Mobile Components

App Servers
DB Servers Storage Mainframe Network

Over investment in infrastructure

Ineffective SLAs and partner relationships Internet

Major ISP
Over spending on CDNs

Lost revenue Brand damage Dissatisfied customers Increased call center volume Increased costs

Content Delivery Networks

Mobile Carrier

Traditional zone of control

Zone of customer expectation

Moving Web Applications to the Cloud: Benefits

Elastic and scalable

Focus on my business while someone manages infrastructure

Load Balancers

The Web Application Delivery Chain

3rd Party/ Cloud Services Local ISP Browsers and devices Users

Web Servers
Mobile Components

App Servers Cloud

DB Servers Storage Mainframe Network

Internet Major ISP

Content Delivery Networks

Mobile Carrier

Traditional zone of control

The Problem: The Cloud Creates Performance Concerns

The Web Application Delivery Chain Cloud is opaque Loss of visibility and control Load Balancers Traditional tools dont apply
Web Servers
Mobile Components

3rd Party/ Cloud Services

Local ISP

Browsers and devices

Users

App Servers Cloud

DB Servers Storage Mainframe Network

Internet Major ISP

Content Delivery Networks

Mobile Carrier

Traditional zone of control

The Answer: Adopt an Outside-In User Point of View

Full understanding of The Web Application Delivery Chain performance from user perspective
Load Balancers Test/monitor your site the Services customers use it: and devices Cloud way your Local ISP What they do (key pages and transactions) Where they do it (geographic locations) How they do it (browsers and mobile devices) When they do it (normal and peak usage) Determine the impact on their behavior and your business

3rd Party/

Browsers

Users

Web Servers
Mobile Components

App Servers
DB Servers Storage Mainframe Network

Internet Outside-in customer point of view Major ISP

Web CrossBrowser Testing

Gomez Platform
Web Web Web Load and Performance Performance Performance Management Business Content Delivery Mobile Testing Analysis Networks Carrier

Traditional zone of control

The Problem: The Cloud Creates Performance Concerns

The Web Application Delivery Chain

Cloud is shared Load Others can affect my performance

Balancers

3rd Party/ Cloud Services

Local ISP

Browsers and devices

My users

Web Servers My

app Mobile Components

App Servers

Other users Other app Internet Major ISP Other users

Other DB app Servers

Storage

Other Mainframe app

Network

Content Delivery Networks

Mobile Carrier

Other users

The Answer: Collective Intelligence

Multiple contributors help diagnose issues The Web Application Delivery Chain everyone for Cloud is shared
Load Balancers

3rd Party/ Cloud Services

Local ISP

Browsers and devices

My users

Web Servers My

app Mobile Components

App Servers

Other users Other app

Outside-in customer point of view Internet

Major ISP

Other DB app Servers

Storage

Other users

Other Mainframe app

Network

Content Delivery Networks

Mobile Carrier

Other users

Cloud Apps: You Must Be Able to Pinpoint Problems

The Web Application Delivery Chain
Load Balancers

3rd Party/ Cloud Services

Local ISP

Browsers and devices

My users

Web Servers

Is it my cloud App Servers provider DB Servers problem?

Mobile Components Storage Mainframe Network

Is it an ISP or Internet Major ISP problem?

Is it a service Internet provider problem?

Mobile Carrier

Is it a browser or Other device users problem?

Other users

Other users

Content Delivery Networks

A Year In The Cloud

An End-user Perspective on Cloud Performance

Introducing CloudSleuth.net

What We Measured

Cloned reference Web application deployed across various IaaS and PaaS providers
Added various services such as a Content Delivery Network, when available Structured application to highlight Cloud Performance issues

How We Measured

Various locations around the world

Backbone and Last Mile locations Every 15 minutes, 24/7, for over 1 year

Used a strict definition of availability

Not All Clouds Perform the Same Way

Average Response Time of Reference Application, as measured from US backbone locations

Taking the Long View Response Time

Average Response Time for reference transaction on Amazon S3, as measured from US locations
10 9
2010 Gomez All Rights Reserved

8 7 6 5 4

3
2 1 0 3/25/09 4/25/09 5/25/09 6/25/09 7/25/09 8/25/09 9/25/09 10/25/09 11/25/09 12/25/09 1/25/10 2/25/10

Taking the Long View - Availability

Weekly availability for reference transaction on Amazon S3, as measured from US locations
100.2

100

99.8

99.6

99.4

99.2

99 3/25/09

4/25/09

5/25/09

6/25/09 7/25/09

8/25/09

9/25/09 10/25/09 11/25/09 12/25/09 1/25/10

2/25/10

Geographic Latency GoGrid

Geographic Latency S3

Going International?

Amazon EC2 Europe West (Dublin) Performance

Response time for sample transaction of reference application hosted on Amazon EC2 Europe West (Dublin), as measured from major European cities

Its Not All About Network Latency

Enough Scary News

lets see something good for a change

Geographic Latency S3

Added Services Help A Lot!

Unique Opportunities for Optimization Better performance, for little work, at no extra cost?

Best Practice: Define Your Goals and Build a Plan Align goals across your organization Why are we moving to the cloud?
Common goals include:
1. Additional Capacity How much capacity do we need during normal and peak times?
If only there was a button to push!

2. Improved End-User Experience What performance goals are we trying to deliver against? 3. Greater Elasticity How quickly can the provider we select ramp up to meet our needs? 4. Flexible Bursting How fast do we need to be able to access additional capacity?

Best Practice: Keep Your End-users in Mind

Test your cloud applications the same way your customers use it:
What they do?
Customers care about completing tasks

Geographic disparities
4 secs 22 secs

NOT whether the homepage is available

Where they do it from?

Your customers dont live in the cloud, test from their perspective

across multiple devices and browsers

When they do it?

Test at peak and normal traffic levels, to find all the problems

What expectations do customers have?

Is 4 seconds fast enough or does it have to be quicker?

Best Practice: Performance Testing Cloud Capabilities

Evaluate vendors based on your goals
Capacity Test vendors to 15-20% past estimated capacity goals Elasticity Baseline end-user performance before & after testing Test during pre-deployment and in production Ramp elasticity testing to peak levels Burstability Isolate the cloud elements from other infrastructure to test Test the failover process

Best Practice: Set SLAs to Match Your Goals

Set SLAs based on your goals
End-user availability and response times Capacity & elasticity objectives

Burstability goals
Set SLAs based on how you are using the cloud

SaaS
Transactional Success Rate Fast end-user experience across key geographies

PaaS
Availability and performance of key components, services, and APIs

IaaS
Hardware & OS availability / performance Network connectivity

Capacity and elasticity requirements

Putting Together a Cloud Management Strategy Key Steps

Validate the burstability & elasticity of cloud solutions Identify application performance bottlenecks Measure performance under normal and peak conditions Baseline and monitor service level objectives Vendor Evaluation Deployment Production Service Level Testing Monitoring Management

Measure how end-user performance impacts behavior

Monitor performance & availability from the end-users perspective

THANK YOU
Questions?
Ron.Wilson@compuware.com