Professional Documents
Culture Documents
High Performance BI
High Performance BI
High Performance BI
Course Guide
Version: HPBI-921-Oct11-Color
20002011 MicroStrategy, Incorporated. All rights reserved.
This Course (course and course materials) and any Software are provided as is and without express or limited
warranty of any kind by either MicroStrategy, Inc. or anyone who has been involved in the creation, production, or
distribution of the Course or Software, including, but not limited to, the implied warranties of merchantability and
fitness for a particular purpose. The entire risk as to the quality and performance of the Course and Software is with
you. Should the Course or Software prove defective, you (and not MicroStrategy, Inc. or anyone else who has been
involved with the creation, production, or distribution of the Course or Software) assume the entire cost of all
necessary servicing, repair, or correction.
In no event will MicroStrategy, Inc. or any other person involved with the creation, production, or distribution of the
Course or Software be liable to you on account of any claim for damage, including any lost profits, lost savings, or other
special, incidental, consequential, or exemplary damages, including but not limited to any damages assessed against or
paid by you to any third party, arising from the use, inability to use, quality, or performance of such Course and
Software, even if MicroStrategy, Inc. or any such other person or entity has been advised of the possibility of such
damages, or for the claim by any other party. In addition, MicroStrategy, Inc. or any other person involved in the
creation, production, or distribution of the Course and Software shall not be liable for any claim by you or any other
party for damages arising from the use, inability to use, quality, or performance of such Course and Software, based
upon principles of contract warranty, negligence, strict liability for the negligence of indemnity or contribution, the
failure of any remedy to achieve its essential purpose, or otherwise.
The information contained in this Course and the Software are copyrighted and all rights are reserved by
MicroStrategy, Inc. MicroStrategy, Inc. reserves the right to make periodic modifications to the Course or the Software
without obligation to notify any person or entity of such revision. Copying, duplicating, selling, or otherwise
distributing any part of the Course or Software without prior written consent of an authorized representative of
MicroStrategy, Inc. are prohibited. U.S. Government Restricted Rights. It is acknowledged that the Course and
Software were developed at private expense, that no part is public domain, and that the Course and Software are
Commercial Computer Software provided with RESTRICTED RIGHTS under Federal Acquisition Regulations and
agency supplements to them. Use, duplication, or disclosure by the U.S. Government is subject to restrictions as set
forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFAR 252.227-7013
et. seq. or subparagraphs (c)(1) and (2) of the Commercial Computer SoftwareRestricted Rights at FAR 52.227-19, as
applicable. Contractor is MicroStrategy, Inc., 1850 Towers Crescent Plaza, Tysons Corner, Virginia 22182. Rights are
reserved under copyright laws of the United States with respect to unpublished portions of the Software.
Copyright Information
All Contents Copyright 2011 MicroStrategy Incorporated. All Rights Reserved.
Trademark Information
MicroStrategy, MicroStrategy 6, MicroStrategy 7, MicroStrategy 7i, MicroStrategy 7i Evaluation Edition,
MicroStrategy 7i Olap Services, MicroStrategy 8, MicroStrategy 9, MicroStrategy Distribution Services, MicroStrategy
MultiSource Option, MicroStrategy Command Manager, MicroStrategy Enterprise Manager, MicroStrategy Object
Manager, MicroStrategy Reporting Suite, MicroStrategy Power User, MicroStrategy Analyst, MicroStrategy Consumer,
MicroStrategy Email Delivery, MicroStrategy BI Author, MicroStrategy BI Modeler, MicroStrategy Evaluation Edition,
MicroStrategy Administrator, MicroStrategy Agent, MicroStrategy Architect, MicroStrategy BI Developer Kit,
MicroStrategy Broadcast Server, MicroStrategy Broadcaster, MicroStrategy Broadcaster Server, MicroStrategy
Business Intelligence Platform, MicroStrategy Consulting, MicroStrategy CRM Applications, MicroStrategy Customer
Analyzer, MicroStrategy Desktop, MicroStrategy Desktop Analyst, MicroStrategy Desktop Designer, MicroStrategy
eCRM 7, MicroStrategy Education, MicroStrategy eTrainer, MicroStrategy Executive, MicroStrategy Infocenter,
MicroStrategy Intelligence Server, MicroStrategy Intelligence Server Universal Edition, MicroStrategy MDX Adapter,
MicroStrategy Narrowcast Server, MicroStrategy Objects, MicroStrategy OLAP Provider, MicroStrategy SDK,
MicroStrategy Support, MicroStrategy Telecaster, MicroStrategy Transactor, MicroStrategy Web, MicroStrategy Web
Business Analyzer, MicroStrategy World, Application Development and Sophisticated Analysis, Best In Business
Intelligence, Centralized Application Management, Information Like Water, Intelligence Through Every Phone,
Intelligence To Every Decision Maker, Intelligent E-Business, Personalized Intelligence Portal, Query Tone, Rapid
Application Development, MicroStrategy Intelligent Cubes, The Foundation For Intelligent E-Business, The Integrated
Business Intelligence Platform Built For The Enterprise, The Platform For Intelligent E-Business, The Scalable
Business Intelligence Platform Built For The Internet, Industrial-Strength Business Intelligence, Office Intelligence,
MicroStrategy Office, MicroStrategy Report Services, MicroStrategy Web MMT, MicroStrategy Web Services, Pixel
Perfect, Pixel-Perfect, MicroStrategy Mobile, MicroStrategy Integrity Manager and MicroStrategy Data Mining
Services are all registered trademarks or trademarks of MicroStrategy Incorporated.
All other company and product names may be trademarks of the respective companies with which they are associated.
Specifications subject to change without notice. MicroStrategy is not responsible for errors or omissions.
MicroStrategy makes no warranties or commitments concerning the availability of future products or versions that
may be planned or under development.
Patent Information
This product is patented. One or more of the following patents may apply to the product sold herein: U.S. Patent Nos.
6,154,766, 6,173,310, 6,260,050, 6,263,051, 6,269,393, 6,279,033, 6,567,796, 6,587,547, 6,606,596, 6,658,093,
6,658,432, 6,662,195, 6,671,715, 6,691,100, 6,694,316, 6,697,808, 6,704,723, 6,741,980, 6,765,997, 6,768,788,
6,772,137, 6,788,768, 6,798,867, 6,801,910, 6,820,073, 6,829,334, 6,836,537, 6,850,603, 6,859,798, 6,873,693,
6,885,734, 6,940,953, 6,964,012, 6,977,992, 6,996,568, 6,996,569, 7,003,512, 7,010,518, 7,016,480, 7,020,251,
7,039,165, 7,082,422, 7,113,993, 7,127,403, 7,174,349, 7,181,417, 7,194,457, 7,197,461, 7,228,303, 7,260,577, 7,266,181,
7,272,212, 7,302,639, 7,324,942, 7,330,847, 7,340,040, 7,356,758, 7,356,840, 7,415,438, 7,428,302, 7,430,562,
7,440,898, 7,486,780, 7,509,671, 7,516,181, 7,559,048, 7,574,376, 7,617,201, 7,725,811, 7,801,967, 7,836,178, 7,861,161,
7,861,253, 7,881,443, 7,925,616, 7,945,584, 7,970,782 and 8,005,870. Other patent applications are pending..
How to Contact Us
MicroStrategy Education Services
1850 Towers Crescent Plaza
Tysons Corner, VA 22182
Phone: 877.232.7168
Fax: 703.848.8602
E-mail: education@microstrategy.com
http://www.microstrategy.com/education
MicroStrategy Incorporated
1850 Towers Crescent Plaza
Tysons Corner, VA 22182
Phone: 703.848.8600
Fax: 703.848.8610
E-mail: info@microstrategy.com
http://www.microstrategy.com
2011 MicroStrategy, Inc. 5
TABLE OF CONTENTS
Preface Course Description.................................................................... 11
Who Should Take this Course ............................................... 13
Course Prerequisites ............................................................. 13
Follow-Up Courses ................................................................ 13
Course Objectives ................................................................. 14
About the Course Materials ......................................................... 15
Content Descriptions ............................................................. 15
Learning Objectives ............................................................... 15
Lessons ................................................................................. 15
Opportunities for Practice ...................................................... 16
Typographical Standards....................................................... 16
Other MicroStrategy Courses ...................................................... 19
Core Courses......................................................................... 19
1. Introduction to High
Performance
Lesson Description ................................................................... 21
Lesson Objectives ................................................................. 22
The High Performance Initiative .................................................. 23
Steps Taken in the High Performance Direction.................... 24
Best Practices Lead to Optimal Performance........................ 26
MicroStrategy High Performance Highlights................................ 27
Components of Performance....................................................... 29
Course Structure.................................................................... 30
Lesson Summary......................................................................... 33
Table of Contents Deploying MicroStrategy High Performance BI
6 2011 MicroStrategy, Inc.
2. Caching and
Intelligent Cubes
Lesson Description ................................................................... 35
Lesson Objectives ................................................................. 36
Computational Distance............................................................... 38
The Importance of 64-Bit Systems .............................................. 40
Introduction to Caching................................................................ 41
Report Caches............................................................................. 42
Report Caching Overview...................................................... 42
Report Caching Best Practices.............................................. 44
Document Caching ...................................................................... 50
Document Caching Overview ................................................ 50
Document Caching Best Practices ........................................ 52
Object Caches ............................................................................. 53
Object Caching Overview ...................................................... 53
Object Caching Best Practices .............................................. 53
Element Caches .......................................................................... 54
Element Caching Overview.................................................... 54
Element Caching Best Practices............................................ 55
Cache Sizing Recommendations................................................. 57
Introduction to Intelligent Cubes .................................................. 59
The Intelligent Cube Publication Process.................................... 61
High Level Steps.................................................................... 61
Peak Memory Usage ............................................................. 62
Intelligent Cube Data Normalization Techniques................... 63
Intermediate Table Type VLDB Property............................... 67
When to Use Intelligent Cubes? .................................................. 68
Using Cubes for Highly Used Prompted Reports................... 69
Using Cubes for Highly Used Overlapping Reports............... 70
Using Cubes for Frequently Used Ad-hoc Analysis Reports . 72
Cube Sizing and Memory Usage Considerations........................ 73
Cube Loading and Swapping................................................. 73
Cube Size Constraints ........................................................... 73
Cube Size and System Scalability ......................................... 74
Other Cube and Report Design Best Practices ..................... 77
Incremental Cube Refresh........................................................... 78
Incremental Refresh Methods................................................ 78
Report Execution against Cubes ................................................. 85
View Report Execution .......................................................... 85
Dynamic Sourcing Execution................................................. 85
View Reports vs. Dynamic Sourcing Reports ........................ 88
Deploying MicroStrategy High Performance BI Table of Contents
2011 MicroStrategy, Inc. 7
Lesson Summary......................................................................... 89
3. Data Transfer Lesson Description ................................................................... 91
Lesson Objectives ....................................................................... 92
Introduction to Network Performance .......................................... 93
Network in a Typical Business Implementation ..................... 93
Case ExampleNetwork Impact on Cube Publication Times94
Key Network Concepts ................................................................ 96
Network Terminology............................................................. 96
Network Recommendations for High Performance ..................... 97
Network Recommendations................................................... 97
Place All Server Components in the Same Segment ............ 97
Consider Bandwidth and Latency Requirements................... 98
Use HTTP Compression........................................................ 99
Setting Up Web Proxy Server .............................................. 102
Distribution Services Performance ............................................ 103
Number of Recipients .......................................................... 103
Report or Document Size..................................................... 103
Delivery Method................................................................... 104
Delivery Format.................................................................... 105
Data Source......................................................................... 106
Concurrency ........................................................................ 108
Alerting................................................................................. 109
Data Personalization Method............................................... 109
Clustering............................................................................. 109
Lesson Summary....................................................................... 110
4. System Architecture
and Configuration
Lesson Description ................................................................. 113
Lesson Objectives ............................................................... 114
Server Specifications................................................................. 115
Processor............................................................................. 115
Memory................................................................................ 119
Disk Storage ........................................................................ 121
Operating Systems .............................................................. 123
Virtualization ........................................................................ 124
Intelligence Server Configuration............................................... 126
User Management ............................................................... 126
Resource Management ....................................................... 129
Workload Management........................................................ 132
Clustering............................................................................. 137
Table of Contents Deploying MicroStrategy High Performance BI
8 2011 MicroStrategy, Inc.
Web Server Configuration ......................................................... 143
JVM Settings........................................................................ 143
MicroStrategy Web Pool Sizes ............................................ 144
Using a Separate Web Server for Static Content ................ 146
Logging and Statistics.......................................................... 147
JavaScript ............................................................................ 147
Lesson Summary....................................................................... 148
5. Data Presentation Lesson Description ................................................................. 151
Lesson Objectives ............................................................... 152
High Performance Reports ........................................................ 153
Report Execution Flow......................................................... 153
Report Configuration Techniques to Optimize Performance 154
High Performance Dashboards ................................................. 159
The Dashboard Execution Flow........................................... 159
Dataset Techniques to Optimize Performance.......................... 162
Dashboard Data Preparation Steps..................................... 162
Reducing the Number of Datasets in a Dashboard ............. 163
Reducing the Amount of Data in a Dataset.......................... 167
Using Intelligent Cubes........................................................ 169
Design Techniques to Optimize Performance ........................... 172
General Performance Topics............................................... 172
DHTML Performance Topics ............................................... 184
Flash Performance Topics................................................... 187
Optimizing Performance for Mobile ........................................... 194
Execution Workflow for Mobile Devices............................... 194
Improving Execution Time for a Mobile Document or Dashboard
195
General Design Best Practices for Documents and Dashboards
200
MicroStrategy Mobile 9.2.1 Performance Optimizations...... 201
Lesson Summary....................................................................... 203
6. Data Warehouse
Access
Lesson Description ................................................................. 207
Lesson Objectives ............................................................... 208
Introduction to Data Warehouse Access ................................... 209
Database Query Performance ............................................. 209
SQL Generation Algorithm................................................... 210
Data Architecture Optimizations ................................................ 213
Report and Schema Design Optimizations................................ 219
Deploying MicroStrategy High Performance BI Table of Contents
2011 MicroStrategy, Inc. 9
Eliminating Unnecessary Table Keys .................................. 219
Including Filter Conditions In Fact Definitions...................... 220
Substituting Custom Groups with Consolidations................ 221
Designing Summary Metrics from Base Metrics.................. 222
SQL Generation Optimizations.................................................. 224
Logical Query Layer............................................................. 224
Database Optimization Layer............................................... 227
Query Optimization Layer .................................................... 238
Other Query Performance Optimizations................................... 243
Multi Source......................................................................... 243
ODBC .................................................................................. 246
Lesson Summary....................................................................... 250
7. Performance Testing
Methodology
Lesson Description ................................................................. 253
Lesson Objectives ............................................................... 254
Introduction to Performance Testing.......................................... 255
Why is Performance Testing Important?.............................. 255
What is Performance in a BI Environment?......................... 256
Why Is Having a Methodology Important?........................... 258
Performance Testing Methodology............................................ 260
Define System Goals ........................................................... 261
Quantify Performance.......................................................... 263
Profile the Action.................................................................. 264
Optimize the Action.............................................................. 275
Monitor the Environment...................................................... 277
Lesson Summary....................................................................... 278
Workshop Dashboard Overview ........................................................... 282
The Design Strategy ............................................................ 282
High-level Steps................................................................... 282
Detailed Steps ..................................................................... 283
Index ......................................................................................... 315
Table of Contents Deploying MicroStrategy High Performance BI
10 2011 MicroStrategy, Inc.
2011 MicroStrategy, Inc. 11
PREFACE
Course Description
After companies reach their business goals, the next area to focus on is
increasing productivity. MicroStrategy software is designed and implemented
with performance in mind and delivers top performance to scale.
MicroStrategy out-of-the-box software is optimized for a typical use case
reflecting the most frequently requested hardware and functionality. Some
tuning may be required to achieve optimal performance.
This course includes the best practices of implementing MicroStrategy BI and
provides the information necessary to build your applications with
performance in mind, tuning MicroStrategy for optimal performance.
In this course, you will first be introduced to the MicroStrategy High
Performance initiative, the main factors behind it, and the most important
outcomes of this initiative to date. Then, you will learn about the components
of performance and about recommendations on how to deploy a High
Performance MicroStrategy platform by using features and settings in the five
main areas of a BI system: In-memory caching and cubes, data transfer, system
configuration, reports, dashboards, mobile applications and data source
access.
Preface Deploying MicroStrategy High Performance BI
12 2011 MicroStrategy, Inc.
Finally, you will learn the basics about performance testing methodology,
gaining solid understanding of performance testing to choose the right type of
tests for their specific performance requirements.
The goal of this course is to provide you with recommendations on features,
settings, and parameters throughout the BI system, which will help you deploy
your MicroStrategy projects, achieving High Performance every step of the
way.
Deploying MicroStrategy High Performance BI Preface
2011 MicroStrategy, Inc. Who Should Take this Course 13
Who Should Take this Course
This course is designed for:
Administrators
Project Managers
Developers
Architects
Course Prerequisites
Before starting this course, you should know all topics covered in the following
courses:
MicroStrategy Administration: Configuration and Security
MicroStrategy Administration: Application Management
MicroStrategy Report Services: Dynamic Dashboards
Follow-Up Courses
After taking this course, you might consider taking the following course:
MicroStrategy Mobile for Apple iPad and iPhone
Preface Deploying MicroStrategy High Performance BI
14 Course Objectives 2011 MicroStrategy, Inc.
Course Objectives
After completing this course, you will be able to:
Describe the MicroStrategy High Performance initiative. Understand the
main impact to your environment provided by the MicroStrategy initiative.
Learn the high-level topics covered by the chapters of this course. (Page 22)
Describe different levels of caching in a MicroStrategy environment.
Understand best practices for leveraging caching for high performance.
(Page 36)
Understand the different instances of data transfer and their impact on the
BI system performance. Describe key network concepts and network
performance recommendations. Apply best practices techniques when
working with Distribution Services. (Page 92)
List the components of performance, understand the main performance
recommendations for server specification, system configuration, and the
Web environment. (Page 114)
Describe report and dashboard execution flow. Understand the
recommendations for designing high performance reports and dashboards.
(Page 152)
Apply the learned skills to optimize report queries to reduce the database
execution time. (Page 208)
Understand performance testing, so you can choose the right type of tests for
specific performance requirements. Design, implement, execute, and
analyze performance tests in a correct way. (Page 254)
Deploying MicroStrategy High Performance BI Preface
2011 MicroStrategy, Inc. About the Course Materials 15
About the Course Materials
This course is organized in to lessons and reference appendices. Each lesson
focuses on major concepts and skills that help you to better understand
MicroStrategy products and use them to implement MicroStrategy projects.
The appendices provide you with supplemental information to enhance your
knowledge of MicroStrategy products.
Content Descriptions
Each major section of this course begins with a Description heading. The
Description introduces you to the content contained in that section.
Learning Objectives
Learning objectives enable you to focus on the key knowledge and skills you
should obtain by successfully completing this course. Objectives are provided
for you at the following three levels:
CourseYou will achieve these overall objectives by successfully
completing all the lessons in this course. The Course Objectives heading in
this Preface contains the list of course objectives.
LessonYou will achieve these main objectives by successfully completing
all the topics in the lesson. You can find the primary lesson objectives
directly under the Lesson Objectives heading at the beginning of each
lesson.
Main TopicYou will achieve this secondary objective by successfully
completing the main topic. The topic objective is stated at the beginning of
the topic text. You can find a list of all the topic objectives in each lesson
under the Lesson Objectives heading at the beginning of each lesson.
Lessons
Each lesson sequentially presents concepts and guides you with step-by-step
procedures. Illustrations, screen examples, bulleted text, notes, and definition
tables help you to achieve the learning objectives.
Preface Deploying MicroStrategy High Performance BI
16 About the Course Materials 2011 MicroStrategy, Inc.
Opportunities for Practice
A Workshop is a reinforcement and assessment activity that follows two or
more lessons. Because a Workshop covers content and applied skills presented
in several lessons, it is a separate section on the level of a lesson.
The following sections within lessons provide you with opportunities to
reinforce important concepts, practice new product and project skills, and
monitor your own progress in achieving the lesson and course objectives:
Review
Case Study
Business Scenario
Exercises
Typographical Standards
The following sections explain the font style changes, icons, and different types
of notes that you see in this course.
Actions
References to screen elements and keys that are the focus of actions are in bold
Arial font style. The following example shows this style:
Click Select Warehouse.
Code
References to code, formulas, or calculations within paragraphs are formatted
in regular Courier.New font style. The following example shows this style:
Sum(sales)/number of months
Deploying MicroStrategy High Performance BI Preface
2011 MicroStrategy, Inc. About the Course Materials 17
Data Entry
References to literal data you must type in an exercise or procedure are in bold
Arial typeface. References to data you type in that could vary from user to user
or system to system is in bold italic Arial font style. The following example
shows this style:
Type copy c:\filename d:\foldername\filename.
Keyboard Keys
References to a keyboard key or shortcut keys are in uppercase letters in bold
Arial font style. The following example shows this style:
Press CTRL+B.
New Terms
New terms to note are in regular italic font style. These terms are defined when
they are first encountered in the course material. The following example shows
this style:
The aggregation level is the level of calculation for the metric.
Notes and Warnings
Precedes Exercises
Deploying MicroStrategy High Performance BI Preface
2011 MicroStrategy, Inc. Other MicroStrategy Courses 19
Other MicroStrategy Courses
Core Courses
Implementing MicroStrategy: Development and Deployment
MicroStrategy Architect: Project Design Essentials
MicroStrategy Desktop: Advanced Reporting
MicroStrategy Desktop: Reporting Essentials
MicroStrategy Mobile for Apple iPad and iPhone
MicroStrategy Report Services: Document Essentials
MicroStrategy Report Services: Dynamic Dashboards
MicroStrategy Web for Professionals
MicroStrategy Web for Reporters and Analysts
All courses are subject to change. Please visit the MicroStrategy website for the latest education
offerings.
Preface Deploying MicroStrategy High Performance BI
20 Other MicroStrategy Courses 2011 MicroStrategy, Inc.
2011 MicroStrategy, Inc. 21
1
INTRODUCTION TO HIGH
PERFORMANCE
Lesson Description
This lesson provides an introduction to the MicroStrategys High Performance
initiative. In this lesson, you will learn about the factors that drove
MicroStrategy to pursue the High Performance initiative and about the
benefits that the results of this initiative can bring to your environment. In
addition, you will learn about the MicroStrategy components of performance
and how this course is structured.
Introduction to High Performance Deploying MicroStrategy High Performance BI 1
22 Lesson Objectives 2011 MicroStrategy, Inc.
Lesson Objectives
After completing this lesson, you will be able to:
Describe the MicroStrategy High Performance initiative. Understand the main
impact to your environment provided by the MicroStrategy initiative. Learn
the high-level topics covered by the chapters of this course.
After completing the topics in this lesson, you will be able to:
Understand the factors behind MicroStrategys push for developing the
High Performance initiative. (Page 23)
List the most important achievements of the MicroStrategy High
Performance initiative. (Page 27)
Understand how this course is structured based on the MicroStrategy
components of performance. (Page 29)
Deploying MicroStrategy High Performance BI Introduction to High Performance 1
2011 MicroStrategy, Inc. The High Performance Initiative 23
The High Performance Initiative
After completing this topic, you will be able to:
Understand the factors behind MicroStrategys push for developing the High
Performance initiative.
Performance is a combination of speed and scale. Speed is important because it
affects the user experience. Speed is typically measured as user wait time or
response time. Scale is important because it determines how many people can
submit requests or reports to a business intelligence (BI) system, how many
reports can be supplied by the system, and how much data can be accessed.
Most BI technologies can do one or the other, but not both effectively.
MicroStrategy has always been focused on high performance, with the
combination of high speed at high scale.
Recent third-party reports have documented increased dissatisfaction with the
BI system performance
1
. While MicroStrategy is recognized as the leader in
high performance, the company continues to re-evaluate its performance to
ensure customers are achieving the highest performance possible with the
MicroStrategy platform.
MicroStrategy Positioning on Performance Matters
Introduction to High Performance Deploying MicroStrategy High Performance BI 1
24 The High Performance Initiative 2011 MicroStrategy, Inc.
Starting in 2007, MicroStrategy surveyed customer applications. It quickly
became clear that there is a continuing trend and inherent demand towards
improved BI performance. This insight led to the creation of a dedicated high
performance initiative. Based on the results of this survey, the goals of this
initiative were to:
Deliver up to 10x faster BI applicationsTodays BI applications must
efficiently access terabytes of data. Since most competing BI tools do not
include performance acceleration engines, average BI application query
response times often range from 10 seconds to one minute or more.
MicroStrategys high performance initiative will set a new performance
standard, aiming to deliver up to 10x faster query response time at any data
scale.
Provide faster than 3-second response time for most predictable
queries and analysesMicroStrategy research has found that most
business queries are predictable. Business people often run similar reports
on a daily, weekly, or monthly basis to understand operational
performance. Using its memory technology to cache computations and
place the results into server memory, MicroStrategy can dramatically
accelerate repetitive operational reports as well as most subsequent
analyses.
Provide faster than 5-second response time for the majority of ad hoc
queriesBy optimizing and accelerating all aspects of its BI platform, from
SQL generation to SQL execution to data rendering, MicroStrategy seeks to
enable 50% of all ad hoc queries to return in less than 5 seconds.
Steps Taken in the High Performance Direction
MicroStrategys high performance initiative includes the formation of its High
Performance and Scalability Lab, the creation of a dedicated Performance
Engineering team, and specific R&D efforts solely focused on providing
MicroStrategy customers with the highest levels of performance for BI
applications of all sizes.
Deploying MicroStrategy High Performance BI Introduction to High Performance 1
2011 MicroStrategy, Inc. The High Performance Initiative 25
MicroStrategy High Performance and Scalability Lab
MicroStrategy built a state-of-the-art research laboratory equipped with the
latest database hardware, software, and performance testing tools.
MicroStrategys multi-million dollar research laboratory has more than 100
servers running tests 24x7 on all supported platforms, including Linux,
Microsoft Win64, Oracle Solaris, IBM AIX, and HP-UX.
MicroStrategys standard benchmark test lines have two Intel
Nehalem-based Intelligence Servers with eight cores and 144 GB RAM each
for Intelligence Server, four Web servers with eight cores each, eight cores for
the MicroStrategy metadata, and eight cores to simulate user load. The entire
test station contains 112 cores in total.
In addition, MicroStrategy currently has at least two enterprise-class database
servers to support its testing efforts, including a four-node Oracle RAC
cluster with 96 cores in total, 256 GB RAM on each node, and 18 TB of usable
space; as well as a Teradata 13 5555H server, with eight cores, 48 GB RAM, and
18 TB of usable space.
MicroStrategy Performance Engineering Team
MicroStrategy has a dedicated team of performance engineers who work
closely with selected customers to understand and document the current
performance development of BI system configurations. This team runs
hundreds of performance tests each week to identify and eliminate system
bottlenecks and to build accurate system performance profiles. In addition, the
team is authoring technical notes, articles, and best practice documents to help
customers maximize performance of their BI applications.
Professional Services
MicroStrategy consulting services can help customers build high performance
BI applications as well as audit and fine-tune existing applications through:
Capacity PlanningMicroStrategys capacity planning-related services
involve a holistic approach to matching existing infrastructure to current
and growing BI requirements, including a quantitative assessment of
current system capacity and potential bottlenecks.
High Performance TuningMicroStrategys high performance-related
services involve a methodical approach to tuning MicroStrategy software
for maximum performance, including all components of the BI ecosystem
such as relational database management systems.
Introduction to High Performance Deploying MicroStrategy High Performance BI 1
26 The High Performance Initiative 2011 MicroStrategy, Inc.
Best Practices Lead to Optimal Performance
For most business intelligence customers, their initial requirement is to satisfy
their functional requirements and solve the problem at hand. MicroStrategys
BI platform is recognized as one of the most complete and technically
advanced in the industry.
The second requirement is stellar performance. After companies reach their
business goals, they want to reach them faster and do more in the same
amount of time for increased productivity. MicroStrategys software is
designed and implemented with performance in mind and delivers top
performance at scale. MicroStrategy customers implement a wide range of BI
functionality in their applicationsfrom the mainstream to the specialized,
from the conservative to the innovative use of BI. It is important to recognize
that depending on the application, different optimizations are needed for
optimal performance.
MicroStrategy software often allows multiple ways to implement a certain
reporting requirement providing our customers with the utmost flexibility.
Designing BI applications with performance in mind improves application
performance. MicroStrategy out-of-the-box software is optimized for a typical
use case reflecting the most frequently requested hardware and functionality.
Some additional tuning may be required to achieve optimal performance.
Deploying MicroStrategy High Performance BI Introduction to High Performance 1
2011 MicroStrategy, Inc. MicroStrategy High Performance Highlights 27
MicroStrategy High Performance Highlights
After completing this topic, you will be able to:
List the most important achievements of the MicroStrategy High Performance
initiative.
By optimizing and accelerating all aspects of its BI platform, from SQL
generation to data rendering, MicroStrategy seeks to enable 50% of all ad hoc
queries to return in less than 5 seconds. Specific areas of R&D include:
Faster Database QueriesMicroStrategys ROLAP technology leverages
database engines for complex calculations and data joins, employing the
latest techniques to reduce processing time and optimize overall query
performance, including: database-specific optimizations, workload
balancing across multiple databases using improved aggregate awareness,
and reduction on the number of SQL passes for sophisticated analyses to
improve database query time by 75%.
Larger Data CachesMicroStrategy is further enhancing its data loading
algorithms. The latest enhancements in MicroStrategy show performance
improvements of over 30% in cube data load time.
Dynamic Sourcing OptimizationsThese optimizations improve system
performance by increasing the number of reports that can be directed to
Intelligent Cubes and result in dynamic sourcing of reports from Intelligent
Cubes to be up to 80% faster.
Faster Metadata ResponsesOptimizations have been made to enable
faster project start-up and administration. Projects are loaded in parallel
rather than serially, reducing the Intelligence Server startup time.
Additional enhancements that have resulted in faster metadata operations
are as follows:
Configuration Wizard has been enhanced to support the process of
upgrading MicroStrategy projects and environments to the most recent
version of MicroStrategy.
Architect operations, such as dragging tables from the warehouse into
the Graphical Architect interface and schema manipulations, are up to
20% faster.
Object migration processes have been optimized to support migration of
objects across unrelated projects.
Introduction to High Performance Deploying MicroStrategy High Performance BI 1
28 MicroStrategy High Performance Highlights 2011 MicroStrategy, Inc.
A new algorithm optimizes the processing of security filters.
Project upgrade process is significantly faster.
Reduced threshold XML size results in less data to transfer.
New index optimizations on metadata tables reduce the metadata access
time.
Health Center Integration with High Performance InitiativeHealth
Center proactively collects customer data from customer systems. Based on
this data, Health Center identifies performance bottlenecks and
recommends steps to resolve those bottlenecks. This integration brings you
closer to MicroStrategy Technical Support, while providing you a better
understanding of your environment.
The components above are not listed in any specific order of access
during the execution of a query. The following image illustrates the five
components:
The Components of High Performance
Introduction to High Performance Deploying MicroStrategy High Performance BI 1
30 Components of Performance 2011 MicroStrategy, Inc.
Course Structure
The remainder of this course is divided into the following chapters:
Caching and Intelligent Cubes
Data transfer
System architecture and configuration
Data presentation
Data warehouse access
Performance testing methodology
Workshop
Caching and Intelligent Cubes
MicroStrategys memory technology is engineered to meet the increased
demand for higher BI performance, which is driven by the rapid expansion of
both data volumes and the number of BI users in organizations across
industries. MicroStrategy accelerates performance by pre-calculating
computations and placing the results into its memory acceleration engine to
dramatically improve real-time query performance. In this chapter, you will
learn about the main performance recommendations when using caches and
Intelligent Cubes.
Data Transfer
Data transfers over one or more networks are a very important component of a
BI implementation. A slow or poorly tuned network performance in any of
those transfers will translate into poor performance from a report or
dashboard execution perspective. In this chapter, you will learn about the main
recommendations to improve network performance and distribution services
executions.
Deploying MicroStrategy High Performance BI Introduction to High Performance 1
2011 MicroStrategy, Inc. Components of Performance 31
System Architecture and Configuration
Successful BI applications accelerate user adoption and enhance productivity,
resulting in demand for more users, data, and reports. MicroStrategy provides
the ability to adapt quickly to constant changes and evolve along with business
requirements. MicroStrategy Intelligence Server has been proven in real-world
scenarios to deliver the highest performance at scale with the fewest servers
and minimum IT overhead. This lesson introduces you to important concepts
regarding tuning your MicroStrategy platform, including recommendations for
the Intelligence Server, Web server, client machines, and configuring
clustering.
Data Presentation
Dashboards provide graphical, executive views into KPIs, enabling quick
business insights. MicroStrategy enables higher performing dashboards,
averaging 30-45% faster execution and interactivity. Using new compression
methods, MicroStrategy dashboards have a smaller footprint than ever
beforeup to 55% smallerresulting in faster delivery using less network
bandwidth. Dashboards deliver ever more analysis and data for end-users. In
this chapter, you will learn about the report and dashboard execution flows,
and about the key recommendations for data set and design optimization for
presenting data in Web and mobile devices. These series of design techniques
will help you develop reports and dashboards, while keeping performance in
mind.
Data Warehouse Access
High performance BI starts with optimizing SQL queries to retrieve results
from the database as quickly as possible. BI performance is dependent largely
on the time that the queries take to execute in the database. An average
reporting request usually takes 40 seconds to complete, out of which 34
seconds, or 85% of the query time, is spent executing in the database.
Therefore, it is critical to optimize report queries to reduce database execution
time. In this chapter, you will learn about the main recommendations to
retrieve source data for reports and documents in an efficient manner.
Introduction to High Performance Deploying MicroStrategy High Performance BI 1
32 Components of Performance 2011 MicroStrategy, Inc.
Performance Testing Methodology
Questions about the performance of the system need to be answered with
various performance testing. Unlike the straightforward feature testing that
verifies the functional correctness of the product, performance testing needs
special design and is more complex to execute and analyze. The goal of this
lesson is to provide you with a good understanding of performance testing, so
you can choose the right type of tests for specific performance requirements.
Workshop
The three-hour workshop will enable you to experience the performance gains
of redesigning a MicroStrategy dashboard, by following the methodology
presented in this course.
This course manual includes the best practices of implementing MicroStrategy
BI and provides the information necessary to build your applications with
performance in mind and to tune MicroStrategy for optimal performance.
Deploying MicroStrategy High Performance BI Introduction to High Performance 1
2011 MicroStrategy, Inc. Lesson Summary 33
Lesson Summary
In this lesson, you learned:
MicroStrategy continues to remain focused on delivering high
performance, with the combination of high speed at high scale.
Starting in 2007, MicroStrategy surveyed customers and found out there
was an inherent demand towards improved BI performance. This insight
led to the creation of a dedicated High Performance initiative.
The goals of the High Performance initiative are to deliver up to 10x faster
BI applications, provide faster than three-second response time for most
predictable queries and analyses, and provide faster than five-second
response time for the majority of ad hoc queries.
MicroStrategys High Performance initiative includes the formation of its
High Performance and Scalability Lab, the creation of a dedicated
Performance Engineering team, and specific R&D efforts solely focused on
providing MicroStrategy customers with the highest levels of performance
for BI applications of all sizes.
Introduction to High Performance Deploying MicroStrategy High Performance BI 1
34 Lesson Summary 2011 MicroStrategy, Inc.
2011 MicroStrategy, Inc. 35
2
CACHING AND INTELLIGENT
CUBES
Lesson Description
In this lesson, you will learn about the concept of computational distance and
the advantages of the 64-bit applications. Next, you will learn how to reduce
the computational distance and increase performance by applying best practice
techniques to the different levels of caching in a MicroStrategy
environmentreport, document, element, and object caching.
Additionally, you will learn about Intelligent Cubes, understand the process of
publishing cubes in memory, and identify the usage scenarios that represent
best candidates for taking advantage of Intelligent Cubes. You will aso learn
about the memory required for Intelligent Cubes in Intelligence Server, and the
tuning techniques that must be applied when designing cubes and reports.
Finally, you will be presented with the performance considerations when
designing reports that utilize Intelligent Cubes.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
36 Lesson Objectives 2011 MicroStrategy, Inc.
Lesson Objectives
After completing this lesson, you will be able to:
Describe different levels of caching in a MicroStrategy environment.
Understand best practices for leveraging caching for high performance.
After completing the topics in this lesson, you will be able to:
Understand the concept of computational distance and its impact on the
system response time. (Page 38)
Understand the importance of 64-bit systems in reducing the computational
distance. (Page 40)
Define caching, list and define cache types, and set cache properties. (Page
41)
Understand the best practices to leverage report caching. (Page 42)
Understand the best practices to leverage document caching. (Page 50)
Understand object caching and the best practices to leverage object caching.
(Page 53)
Understand element caching and the best practices to leverage element
caching. (Page 54)
Understand the rules of thumb to size caches in your system. (Page 57)
Understand the Intelligent Cubes functionality and how it complements the
MicroStrategy caching strategy. (Page 59)
Understand the process of publishing an Intelligent Cube. (Page 61)
Identify the usage scenarios that represent best candidates to take
advantage of Intelligent Cubes. (Page 68)
Understand the memory requirements for Intelligent Cubes and the tuning
techniques that must be applied when designing cubes and reports. (Page
73)
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Lesson Objectives 37
Refresh cube data using the different options available in the incremental
refresh feature. (Page 78)
Understand the performance considerations when designing reports to
utilize Intelligent Cubes. (Page 85)
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
38 Computational Distance 2011 MicroStrategy, Inc.
Computational Distance
After completing this topic, you will be able to:
Understand the concept of computational distance and its impact on the
system response time.
Any BI system consists of a series of processes and tools that take raw data at
the very bottomat the transactional level in a databaseand use varying
technologies to transform that data to the finished answer that the user needs.
At every step along the way, some kind of processing is done on the following
componentsthe database, network, BI application, or the browser.
The concept of computational distance refers to the length in terms of
systems, transformations, and other processes that the data must undergo
from its lowest level, all the way to being rendered on a browser as shown in
the image below:
Computational Distance
The longer the computational distance is for a given report, the longer it will
take to execute and render. The preceding image shows a hypothetical example
of a report that runs in 40 seconds. Each processing step on that report, such as
aggregation, formatting, and rendering, adds to the report's computational
distance, increasing the report overall execution time.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Computational Distance 39
Computational distance offers a useful framework from a performance
optimization perspective because it tells us that to improve the performance of
a report or dashboard, you must focus on reducing its overall computational
distance. The following image shows different techniques such as caching,
cubes, and aggregation that can be used to optimize performance for the 40
second hypothetical report:
Reducing the Computational Distance of a Report
This lesson focuses on the following two key computational distance reduction
techniques offered in the MicroStrategy platformcaching and Intelligent
Cubes.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
40 The Importance of 64-Bit Systems 2011 MicroStrategy, Inc.
The Importance of 64-Bit Systems
After completing this topic, you will be able to:
Understand the importance of 64-bit systems in reducing the computational
distance.
The server memory (RAM) is a critical resource for effective performance
optimization strategies involving caching and Intelligent Cubes. A system with
large memory resources enables architects to design more caches and
Intelligent Cubes that can source an increasingly larger set of reports and
dashboards. In turn, as more reports and dashboards are sourced by either
caches or Intelligent Cubes, the average user wait time is reduced. The
following image illustrates the increase in the % of reports sourced from
in-memory BI. Average response time drops with an increase in the amount of
server RAM:
Average Wait Time Versus In Memory BI Usage
By dramatically increasing the amount of addressable RAM in a server, the
64-bit technology is the ultimate high performance enabler. MicroStrategy
project implementations will have significantly more opportunities for
performance optimization using both caching and Intelligent Cubes when they
are deployed using 64-bit server technologies.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Introduction to Caching 41
Introduction to Caching
After completing this topic, you will be able to:
Define caching, list and define cache types, and set cache properties.
Caching is the retention of a recently used object in memory for the purpose of
improving query response time for future requests. Caching enables users to
retrieve results from in-memory stored files rather than executing queries
against a database. The MicroStrategy architecture offers several caching levels
within the different functional layers. This section covers the key caching
levelsreport, document, element, and object caching from the perspective of
performance optimization.
The following image displays the different levels of caching in the
MicroStrategy platform:
Cache Levels in the MicroStrategy Environment
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
42 Report Caches 2011 MicroStrategy, Inc.
Report Caches
After completing this topic, you will be able to:
Understand the best practices to leverage report caching.
Report Caching Overview
MicroStrategy report caches store all report results in Intelligence Server for
fast retrieval. Because they reduce computational distance to a bare minimum,
reports that can execute against caches will have better response times than
reports that execute against cubes, aggregate tables, or fact tables on the data
warehouse. Reports that are very frequently run, and have fixed and
predictable characteristics are perfect candidates for using MicroStrategy's
caching technology. User experience can be further enhanced by scheduling
the reports to run during batch window times. This way, by pre-caching reports
in batch, even the first report execution wait time can be minimized.
Report Cache Flow
The following image illustrates how report caching works:
How Report Caching Works?
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Report Caches 43
The basic workflow is as follows:
1 A user runs a report for the first time.
2 The report runs against the data warehouse database.
3 Intelligence Server caches the result set and also returns them to the user.
4 A second user subsequently runs the same report.
5 Intelligence Server searches its report caches.
6 Intelligence Server uses the report cache matching algorithm to determine
if it can reuse an existing cache.
7 If the report cache matching algorithm shows that the cache is not valid,
Intelligence Server queries the data warehouse. However, if the cache is
determined to be valid, Intelligence Server retrieves the result set from the
cache.
The greater the number of long-running reports in the project, the greater the
impact is on the performance. However, with report caching, you can reduce
the number of report requests that have to be processed against the data
warehouse and thereby improve processing time. The following image
illustrates how you can achieve better performance through report caching:
Report Caching Example
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
44 Report Caches 2011 MicroStrategy, Inc.
In this example, 20 users need to view the same data from the same report
each day. This report ranks product sales to display the best-selling and
worst-selling items, and it runs against a data warehouse table that contains
two million rows. Given the qualifications it is performing and the volume of
data in the table, this report takes 10 minutes to run. If each user queries the
data warehouse for the result set, collectively, their requests consume over two
hours of processing time. However, if you configure caching for this report,
only the first user who runs the report queries the data warehouse. The other
19 users can access the cache created by the first user. The processing time is
reduced to 10 minutes plus the time for cache retrieval, which in most cases
takes only a few seconds.
Being able to cache report results ensures that redundant queries are not sent
to the data warehouse and raises the level of performance as it reduces
processing time.
Report Caching Best Practices
In an ideal scenario, all reports that a user in a system could possibly need
within any given day should be pre-cached. This type of implementation would
practically guarantee very fast report response times for all users in the system.
However, more practically, there are three key factors that make pre-caching
every single report an impossibility:
the degree of personalization and security for the reports
the amount of RAM available on the Intelligence Server
the available batch window time
In general, MicroStrategy has found that because of the above three factors, in
a typical BI implementation about 10% of reports will benefit from caching.
The rest of this subsection explains in detail some important considerations
and the best practices to leverage report caching for performance optimization.
Enabling Caching for Frequently Used Reports Only
From a performance perspective, given that not every report can be cached, it
is important to prioritize and choose which reports are good candidates to be
cached. The frequency of use, especially for shared reports is a good
prioritization criteria. Report cache creation and usage consume server
resources such as RAM and disk space. As a result, MicroStrategy recommends
using true cache reuse as a criteria for choosing which reports to cache.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Report Caches 45
Typical MicroStrategy implementations involve shared reports that are
managed through a development, test, and production cycle; and personal
reports that are created by users directly in the production environment. Given
the typical configuration described above, the following report caching
recommendations should be considered:
1 Caching all reports at the project level is only practical for small projects
involving limited personal object creation.
2 Not all shared reports and documents are good caching candidates if the
diversity of content to be viewed by users is high. For example, a prompted
report that is heavily personalized by each user will not be a good candidate
for caching, but a prompted report in which 90% of job executions use the
same prompt answers could be a candidate for caching.
3 Caching should be disabled at the report object template level for report
wizard, report builder, and blank report templates. These templates are
used to create and save personal reports.
4 The Maximum number of caches setting in the Project Configuration Editor
enables administrators to control cache creation per project. It is
recommended to use this setting to limit the number of caches allowed.
You can use MicroStrategy Enterprise Manager to analyze system usage data
and identify the reports that fall within the frequently used shared reports
category and will therefore be good caching candidates.
Disabling Caches for Highly Prompted Reports
Caching for prompted reports is tied to specific prompt answers. This means
that for a prompted report cache to be reused, the exact same prompt answer
as the one with which the cache was created must be provided by the user. Any
other answer will cause Intelligence Server to reject the cache and issue a SQL
statement against the data warehouse. For a highly prompted report to be
effectively cached, every possible combination of prompt answers would need
to be pre-cached, making this a highly impractical, if not impossible,
proposition given RAM and batch window limitations.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
46 Report Caches 2011 MicroStrategy, Inc.
Avoiding Per-User Cache Generation
When caches are created for each user, by definition they cannot be shared
across the project user community. If those users tend to hit similar reports or
even identical reports but with slightly different prompt answers, caching on a
per-user basis means that most of the caching resources would be spent
creating and maintaining quasi-redundant sets of memory structures with very
little potential for reuse. Therefore, you should avoid per-user cache
generation. Where needed, this type of use case can be typically satisfied
through the use of security filters.
Enabling XML Caching for Reports
Report caches are stored in binary structures both in file and in memory. Given
that the Web server requires an XML representation of the report to generate
the necessary HTML code to ultimately render it on the client, a Binary to
XML conversion step is required as part of processing a cached report.
When XML caching is enabled, this extra conversion step can be completely
skipped, making the rendering of a cached report even faster. When cached,
the XML representation of the cached report takes up space both on disk and
in memory. As a result, turning XML caching on has an impact in terms of
increasing RAM usage on Intelligence Server.
To enable XML caching for reports:
1 In Desktop, launch the Project Configuration Editor.
2 In the Project Configuration Editor, in the Categories list, expand Caching,
expand Result Caches, and select Creation.
3 Under Project Default Behavior, select the Enable XML caching for
reports check box.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Report Caches 47
4 Click OK to save your settings.
XML Caching
You can view the sizes of both the binary and XML caches for a given
report in the Cache Monitor.
Allocating Enough Memory for Report Caches
Report caches are first created in memory within Intelligence Server. They are
later backed up to disk and reloaded into memory when accessed.
If Intelligence Server memory is insufficient to hold all new requested report
caches, the least recently used caches are unloaded automatically from
memory to make space for the new caches.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
48 Report Caches 2011 MicroStrategy, Inc.
The process of loading a cache from disk can add significant delays to the user
wait time. Large cache sizes take longer time to load and may displace other
caches that will need to be moved back from memory to disk to make space.
This process degrades performance both for the user who requested the cached
report and for users who would have been able to hit the now displaced caches.
Cache Load/Unload From Disk
Based on the above considerations, from a performance perspective it is
important to run MicroStrategy in a 64-bit environment whenever possible
given the significantly larger amounts of addressable memory it offers. If this is
not possible, you must at least ensure that all available Intelligence Server
memory is used optimally.
Memory capacity planning for report caches can be extremely challenging, but
at a high level, a good estimate of the amount of memory needed to effectively
cache a given set of reports can be obtained as follows:
1 Determine which reports should be cached on a given project. For example,
you can use Enterprise Manager to determine the most frequently used
shared reports as discussed earlier.
2 Generate caches for those reports and record cache sizes (both binary and
XML) using Cache Monitor. Note that in the case of prompted reports, you
will need to generate caches for each prompt answer you wish to cache.
3 If you plan to generate some user-specific report caches, you must add this
variable into your total cache memory utilization estimates.
After you have estimated the total amount of RAM that will be used for report
caching you can define it in Intelligence Server.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Report Caches 49
To define the memory available for report caching:
1 In Desktop, launch the Project Configuration Editor.
2 In the Project Configuration Editor, in the Categories list, expand Caching,
expand Result Caches, and select Storage.
3 Under Datasets, in the Maximum RAM usage (MBytes) text box, type the
amount of Intelligence Server RAM you want to allocate for report cache
storage.
For more details on the element browsing query flow, refer to the
MicroStrategy Administration: Configuration and Security course.
Element Caching Best Practices
Using Intelligent Cubes to Eliminate Element Database
Queries
The execution of a typical prompted report sourced directly against the data
warehouse generates two types of querieselement queries to populate the
prompt and report queries to calculate the report results. On the other hand, a
report sourced from an Intelligent Cube retrieves both the prompt elements
and the report dataset directly from the cube, alleviating the load on the data
warehouse and improving overall system performance.
Setting the Same Element Display Size for MicroStrategy Web
and MicroStrategy Desktop
The amount of memory that element caches use on Intelligence Server (set at
the project level) and on the MicroStrategy Desktop machine can be controlled.
To avoid the risk of memory swapping and performance degradation, it is
important to optimize the use of the memory assigned to element caches. If
different element display sizes are set on Intelligence Server and MicroStrategy
Desktop, the element caches cannot be shared between those two components,
and the risk of memory swapping increases.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
56 Element Caches 2011 MicroStrategy, Inc.
Instead, defining identical element display sizes on both Intelligence Server
and MicroStrategy Desktop allows element cache sharing and optimizes the
use of a limited resource, as illustrated in the following image:
Inefficient and Efficient Use of Element Caching
Assigning Enough Memory for Element Caching
You should allocate sufficient memory in the server for the storage of element
caches. Default values tend to be low. The table below shows the recommended
settings for element caching. You should consider allocating even more
memory for cases when some attributes contain large amounts of elements.
Element Caching Recommended Settings
Default Setting Recommended Setting
Web Desktop Web Desktop
Element Display Size 30 1000 50 50
Element Cache Size 120 4000 200 200
Avg. Amount of Data
(512 bytes/element)
60 KB 2,000 KB 100 KB
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Cache Sizing Recommendations 57
Cache Sizing Recommendations
After completing this topic, you will be able to:
Understand the rules of thumb to size caches in your system.
MicroStrategy recommends the following cache sizing to achieve high
performance on a MicroStrategy deployment:
Report CachesFor each application, counting 10 reports per user at an
average report size of 50 KB, the total memory for result caches per user per
project should be about 500 KB.
Document CachesDocument cache would require about 100 MB for 100
documents cached in a project with an average document size of 1 MB.
Element CachesAn average element cache memory requirement is
about 10 MB, for a display size of 50 elements. For 100 attributes, as seen in
a typical implementation, it is 512 bytes per element.
Object CachesObject cache calculation is based on the metadata size.
Object caches should range from 20-50% of the metadata size. A metadata
size of 1 GB should have an object cache of 500 MB, which corresponds to
50% of the metadata size.
Intelligent CubesCubes would take about 25 GB for an average of 50
cubes per project at 500 MB per cube. Intelligent Cubes have query
characteristics like databases, but because the cubes reside in the main
memory, they have the performance characteristics of caches. Thus, they
are not a part of the caching layer but are extremely important to provide
superior performance.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
58 Cache Sizing Recommendations 2011 MicroStrategy, Inc.
The following image summarizes the recommended cache sizes:
Recommended Cache Sizes
Use the following Caching Size Worksheet to estimate initial cache settings
based on your BI Implementation size:
Caching Size Worksheet
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Introduction to Intelligent Cubes 59
Introduction to Intelligent Cubes
After completing this topic, you will be able to:
Understand the Intelligent Cubes functionality and how it complements the
MicroStrategy caching strategy.
In the production environment with thousands of reports, it may not be
feasible to create caches for every single report. First of all, to create caches,
each report has to first run against data warehouse. In addition, many of the
reports may return overlapping data. Therefore, many caches may contain
redundant data. To overcome these types of challenges, you utilize Intelligent
Cubes.
For data warehouse databases which are not optimized for large data
inserts (such as Netezza) publishing a cube with SQL normalization may
take longer.
Normalize Intelligent Cube data in database using relationship
tablesThe data is normalized on the database side. This approach can
provide improved performance in scenarios where cube data includes a
large ratio of repeating data, dimensions include a large number of
attributes, and attribute lookup tables are much smaller than fact tables.
Direct loading of dimensional data and filtered fact dataThere is no
need to normalize the data. It generates optimized SQL that retrieves data
directly from a data warehouse and transforms the data as SQL passes in
the following manner:
Attribute passesOne pass for each attribute
Dimension passesOne pass for each dimension
Metric passesOne pass for all metrics in the Intelligent Cube
Peak memory usage must be considered rather than final cube size.
Available memory on the database to handle the temporary tables
created when database normalization is used
Total cube publication timeLimiting factor in terms of whether a cube
can be published during the system's batch window or not. Cube
publication involves SQL generation and execution against a database.
Making the cube definition as optimal as possible will pay off in terms of
shortening the total cube publication time.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
74 Cube Sizing and Memory Usage Considerations 2011 MicroStrategy, Inc.
Maximum number of result set rows allowedMicroStrategy only allows
2 billion rows to be published in an Intelligent Cube. This is a limitation
inherent to 64-bit platforms.
Cube Size and System Scalability
As the previous section explains, several system constraints exist that limit the
maximum cube size that could be generated in a given MicroStrategy
deployment. It is important to understand how cube size relates to overall
system scalability and throughput.
In MicroStrategys internal benchmarks, an 80 GB cube was published on a
server with 144 GB of total memory. This cube sustained response times of less
than two seconds for concurrency levels of up to 200 users. After the 200 user
mark, response time tended to degrade.
A second key observation was that smaller cube sizes sustain larger levels of
concurrency without significant performance degradation. For example, as per
the benchmarks, you can have up to 600 concurrent users for a 40 GB cube
while having up to 1800 concurrent users for a 10 GB cube.
The key learning is that while single-user response times tend to remain the
same as cube sizes increase, user and traffic scalability tends to diminish, as
shown in the following image:
Response Time Versus Throughput
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Cube Sizing and Memory Usage Considerations 75
The general recommendation is to design Intelligent Cubes with less than 5
GB. Staying within this threshold should provide optimal performance in
terms of report response times and overall user scalability. While larger cube
sizes are feasible, make sure that they can support large user concurrency
requirements. Alternatively, you should look for opportunities to reduce the
overall cube size. The following table displays throughput considerations for
Intelligent Cubes, depending on the cube size:
Factors Affecting Intelligent Cube Size
Attribute CardinalityAttributes that have a large cardinality (number of
unique attribute elements) can have a much larger impact on the memory
size of a cube than attributes that have only a few unique elements. In
addition, another important factor that impacts memory size is the number
of attribute forms that are returned for each attribute. Each new form
requires additional data to be stored in the cube.
MetricsEach metric added to a cube can also represent an additional cost
in terms of memory usage. Smart metrics tend to be especially costly in
terms of cube size because when a metric is defined as a smart metric, all
intermediate values for the metric calculation must be saved in the cube to
fulfill run-time aggregations. The Analytical Engine needs to store the
component metrics for dynamic aggregation and subtotals at different
levels, and each of the intermediate values requires the same space as a
metric value.
From a cube size perspective, the impact of a smart metric is roughly
equivalent to the impact of several simple metrics. The exact contribution
to cube size depends on how many intermediate values are required for the
formula of the smart metric. Given this impact, it is recommended to avoid
creating complex smart metrics when they will be used in combination with
Intelligent Cubes.
FiltersFilters can be used to restrict the amount of data returned for an
Intelligent Cube, and therefore, reduce its overall size.
Cube Size Recommendations
Cube Size Throughput Level
< 5 GB Optimum (Recommended)
5 - 15 GB Acceptable
> 15 GB Sub-optimal (Not Recommended)
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
76 Cube Sizing and Memory Usage Considerations 2011 MicroStrategy, Inc.
Cell Level FormattingCell level formatting is not available for Intelligent
Cubes. If a report with threshold formatting is converted to an Intelligent
Cube, the formatting is ignored during publication and the cube size
remains the same.
However, if MDX cube reports contain cell level formatting, this
information is stored when the report is converted into an Intelligent Cube
and leads to an increase in cube size.
Data TypesThe data types used to represent attributes and metrics affect
the memory requirements for these objects because they depend on how
the data types are represented in the database from where they are
retrieved. The following table provides the memory requirements based on
the data type present in the cube:
For each cell in the attribute lookup table and each metric cell in the
result table, one byte is used to keep track of whether the data is null or
not. The rest are used to save the data.
VARCHAR data type consumes more memory than other data types, so
it is recommended to be conservative when using VARCHAR.
InternationalizationFor each language supported by the Intelligent
Cube, each attribute includes an additional description field stored for the
language. As an example, an English-only cube of 338 MB in size can
increase to more that 1.5 GB by adding the necessary data to support an
additional nine languages.
Data Types and Cube Sizing
Data Type
Attribute Cost Metric Cost
Result Table
Byte Per Cell
Lookup Table
Byte Per Cell
Simple Metric
Bytes Per Cell
NUMBER 8 5 5
FLOAT (32) 8 9 9
DATE 8 45 45
TIMESTAMP 8 45 45
TIMESTAMP
WITH TIMEZONE
8 45 45
CHAR (N) 8 4N+1 4N+1
CHAR (N CHAR) 8 4N+1 4N+1
VARCHAR2 (N) 8 4N+1 4N+1
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Cube Sizing and Memory Usage Considerations 77
Other Cube and Report Design Best Practices
The following list discusses some other cube and report design best practices:
Match cubes and reports aggregation levelsWhenever possible, try to
match the level of aggregation of cubes and reports. This practice prevents
excessive aggregation work which greatly saves time and resources in the
data retrieval process.
Avoid Aggregating on Attributes with Big CardinalityAggregating a
report or a cube on attributes with big cardinalities may negatively impact
the report execution performance. When aggregating on attributes with big
cardinalities, the Analytical Engine needs to spend more time during the
data aggregation process.
Avoid Filtering on the Lowest Level AttributeWhen the lowest level
attribute of a report or cube contains a large number of elements, you
should avoid including it as a filtering criteria. You should also avoid
including lowest level attributes in the filter criteria when they are not part
of the template, while an attribute on the same hierarchy is on the template.
For example, consider a cube filtered on the Order attribute. The template
does not contain Order, but contains Customer Region, which is a higher
level attribute than Order, inside the Customer hierarchy. In this scenario,
performance is sub-optimal because the filter is included in the
aggregation.
Avoid Using Conditional MetricsConditional metrics add the filter
criteria to the metric aggregation. When the filter is added to the metric
aggregation, performance is not optimized.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
78 Incremental Cube Refresh 2011 MicroStrategy, Inc.
Incremental Cube Refresh
After completing this topic, you will be able to:
Refresh cube data using the different options available in the incremental
refresh feature.
You can set up incremental refresh settings to update the Intelligent Cube with
only new data, instead of having to republish the entire cube. This feature can
reduce the time and system resources necessary to update the Intelligent Cube
periodically, improving overall performance.
For example, you have an Intelligent Cube that contains weekly sales data. At
the end of every week, this Intelligent Cube must be updated with the sales
data for the current week. You can set up incremental refresh settings so that
only data for the current week is added to the Intelligent Cube.
The following requirements must be met for an Intelligent Cube to qualify for
an incremental refresh:
The Intelligent Cube must be updated based on attributes only. For
example:
an Intelligent Cube requiring update for Month or Region qualifies for
an incremental refresh.
an Intelligent Cube that contains data for the top 200 stores by Revenue
does not qualify for an incremental refresh.
Incremental Refresh Methods
Depending on your requirements, you can use the following methods to define
the incremental refresh options:
Intelligent Cube republish settingsRecommended if the Intelligent Cube
must be updated along one attribute, or if updates along multiple attributes
must be made simultaneously.
Incremental refresh filter is the default option for both ROLAP and
MDX Intelligent Cubes, and is unavailable for Intelligent Cubes created
using Freeform SQL queries or Query Builder.
To define an incremental refresh filter:
1 In Desktop, right-click the Intelligent Cube for which you want to define the
incremental refresh, and select Define Incremental Refresh Report.
2 In the Incremental Refresh Options window, under Refresh type, select one
of the following options:
UpdateThe incremental refresh filter is evaluated. If new data is
returned, it is added to the cube, and if the data returned is already in
the cube, it is updated where applicable.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
82 Incremental Cube Refresh 2011 MicroStrategy, Inc.
InsertThe incremental refresh filter is evaluated. If new data is
returned, it is added to the Intelligent Cube. Data that is already in the
Intelligent Cube is not changed.
DeleteThe incremental refresh filter or report is evaluated. The data
that is returned is deleted from the cube. For example, if the Intelligent
Cube contains data for 2008, 2009 and 2010, and the filter or report
returns data for 2009, all the data for 2009 is deleted from the cube.
Update onlyThe incremental refresh filter or report is evaluated. If
the data returned is already in the Intelligent Cube, it is updated where
applicable. No new data is added to the cube.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Incremental Cube Refresh 83
You can change these options at any time by opening the incremental
refresh in the Report Editor, and on the Data menu, selecting Configure
incremental refresh options.
3 Click OK.
The filter must only qualify on attributes that are present in the
Intelligent Cube.
5 To preview the data that will be updated in the Intelligent Cube, on the
View menu, select Preview Data. The data displays in grid view.
If you have security filter that prevents you from viewing some data,
the preview will only display the data that you are allowed to view.
However, when the incremental refresh is executed, all the data is
updated in the Intelligent Cube, regardless of security filters.
6 To execute the incremental refresh immediately, click Run Report.
7 Click Save and Close.
Incremental Refresh Report
You can define a report to update an Intelligent Cube. The results of the report
are compared to the data in the cube, and the cube is updated accordingly.
For metrics that are not included on the reports template, data is
not updated in the cube.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
84 Incremental Cube Refresh 2011 MicroStrategy, Inc.
To define an incremental refresh report:
1 In Desktop, right-click the Intelligent Cube for which you want to define the
incremental refresh, and select Define Incremental Refresh Report.
2 In the Incremental Refresh Options window, under Refresh type, select one
of the following options:
Update
Insert
Delete
Update only
If you have security filter that prevents you from viewing some data, the
preview will only display the data that you are allowed to view. However,
when the incremental refresh is executed, all the data is updated in the
Intelligent Cube, regardless of security filters.
8 To execute the incremental refresh immediately, click Run Report.
9 Click Save and Close to save and close the incremental refresh report.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Report Execution against Cubes 85
Report Execution against Cubes
After completing this topic, you will be able to:
Understand the performance considerations when designing reports to utilize
Intelligent Cubes.
The following sections provide architectural information and performance
considerations for the two types of reports that retrieve data from Intelligent
Cubes:
View ReportsView reports are reports that are directly linked to an
Intelligent Cube.
Dynamic Sourcing ReportsDynamic sourcing reports are regular
reports that automatically hit an available Intelligent Cube.
View Report Execution
The bullets below describe the factors that impact the execution performance
of a view report. It is important to understand them to prevent bottlenecks in
the execution process:
View Report DesignAvoid using filters on text and date attribute forms.
As explained above, before aggregating the data, the Analytical Engine
must evaluate each row in the cube and number comparisons are much
faster than date and text comparisons for this purpose.
Cube SizeThe bigger the size of the cube, the longer the view report
response time. This is mainly due to the fact that filtering and aggregation
must be performed against large numbers of rows of data.
Dynamic Sourcing Execution
Dynamic sourcing enables reports to automatically hit an available Intelligent
Cube if it contains all of the information required for the report.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
86 Report Execution against Cubes 2011 MicroStrategy, Inc.
When dynamic sourcing is enabled, the engine must ensure that the data of a
published cube can satisfy the report request and that by running against that
cube it can obtain exactly the same results it would obtain if it ran against the
data warehouse. Both the Intelligent Cube and the requested report are
analyzed according to a number of criteria.
The filter applied to conditional metrics has the same limitations as the
report filter in terms of dynamic sourcing.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
88 Report Execution against Cubes 2011 MicroStrategy, Inc.
View Reports vs. Dynamic Sourcing Reports
View Reports and Dynamic Sourcing Reports have some differences in terms of
execution. When designing reports, it is important to take into account these
differences, which are described below:
Metric Qualification Level for View Filters
Dynamic SourcingThe aggregation level of metrics in a view filter is
applied at the report level, which is the level defined by the report
template.
View ReportThe aggregation level of metrics in a view filter is defined
by the attributes in the cube
For example, suppose both the dynamic sourcing report and the view report
have the following attributes in the template: Customer City and Customer.
However, the cube that satisfies both of these reports data was created with
the following attributes in the template: Customer Region, Customer City,
and Customer.
If you create a view filter with a metric qualification in the view report, the
filter is calculated for all of the attributes in the cube which are Customer
Region, Customer City, and Customer.
If you create the same view filter in the dynamic sourcing report, the filter is
calculated for only the attributes of that report which are Customer City and
Customer
Report Filter
Dynamic SourcingThe criteria in the report filter display in the
WHERE clause of the SQL that executes against the cube.
View ReportView reports do not contain report filter
View Filter
Dynamic SourcingThe criteria in the view filter are calculated during
the Analytical Engine processing.
View ReportThe criteria in a view filter displays in the WHERE clause
that executes against the cube.
Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2
2011 MicroStrategy, Inc. Lesson Summary 89
Lesson Summary
In this lesson, you learned:
Computational distance offers a useful framework from a performance
optimization perspective because it tells you that to improve the
performance of a report or dashboard, you must focus on reducing its
overall computational distance.
By dramatically increasing the amounts of addressable RAM in a server,
64-bit technology is the ultimate high performance enabler.
Caching is the retention of a recently used element, object, or report results
for the purpose of improving query response time in future requests.
In MicroStrategy, there are four types of cacheselements, objects, reports,
and documents.
Report caches are results of previously executed reports that are stored in
memory or disk on the Intelligence Server machine, so that they can be
retrieved quickly.
From a performance perspective, it is important to prioritize which reports
are good candidates to be cached. Frequency of use, especially for shared
reports, is a good prioritization criteria.
The following recommendations to improve performance apply for report
cachesdisabling caches for highly prompted reports or with prompts on
attributes, avoiding per-user cache generation, enabling XML caching for
reports, and allocating enough memory for caches.
You can enable document caching in PDF, Microsoft Excel, and XML
formats at the project and document levels.
The cube publication process involves the following main stepsSQL
generation and query execution, warehouse data fetch, and data conversion
and serialization.
Cube designers must consider two critical performance factorsoverall
cube publication times and peak memory usage.
The peak memory used by Intelligence Server is expected to be two-five
times the cube size while publishing a cube.
Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI 2
90 Lesson Summary 2011 MicroStrategy, Inc.
MicroStrategy uses an optimization process referred to as normalization to
optimize the size of the Base Cube Table.
The following groups of frequently run reports with increasing levels of user
wait time are the good candidates for Intelligent Cubeshighly used
reports, multiple overlapping reports, and ad-hoc analysis reports.
It is important not to design more cubes than what can be fit into
Intelligence Server memory at any one time to avoid swapping.
The general recommendation is to design Intelligent Cubes with less than 5
GB. This threshold should provide optimal performance in terms of report
response times and overall user scalability.
The following tuning techniques apply to Intelligent Cubessegmenting
cubes along prompt answers, designing cubes that cover smaller groups of
reports, matching the level of aggregation of cubes and reports, avoiding
aggregating on attributes with big cardinality, avoiding filtering on the
lowest level attribute, and avoiding using conditional metrics.
You can set up incremental refresh settings to update the Intelligent Cube
with only new data, instead of having to republish the entire cube. This
feature can reduce the time and system resources necessary to update the
Intelligent Cube periodically, improving overall performance.
View reports and dynamic sourcing are two different methods used by
reports to utilize Intelligent Cubes. It is important to understand how each
method impacts performance when designing reports.
2011 MicroStrategy, Inc. 91
3
DATA TRANSFER
Lesson Description
In this lesson, you will learn about the instances of data transfer and their
impact on the BI system performance. Next, you will be introduced to a few key
computer networking-related concepts. Lastly, you will learn the
performance-related recommendations when working with Distribution
Services.
Data Transfer Deploying MicroStrategy High Performance BI 3
92 Lesson Objectives 2011 MicroStrategy, Inc.
Lesson Objectives
After completing this lesson, you will be able to:
Understand the different instances of data transfer and their impact on the BI
system performance. Describe key network concepts and network performance
recommendations. Apply best practices techniques when working with
Distribution Services.
After completing the topics in this lesson, you will be able to:
Understand the different instances of data transfer and their impact on BI
system performance. (Page 93)
Describe key computer network-related concepts. (Page 96)
Understand the main recommendations to improve network performance.
(Page 97)
Apply high performance best practice techniques when working with
Distribution Services. (Page 103)
Deploying MicroStrategy High Performance BI Data Transfer 3
2011 MicroStrategy, Inc. Introduction to Network Performance 93
Introduction to Network Performance
After completing this topic, you will be able to:
Understand the different instances of data transfer and their impact on BI
system performance.
Network in a Typical Business Implementation
Data transfers over the network are a very important component of a BI
system. Poorly tuned network performance in any one of those transfers will
translate into poor performance from a job execution perspective. The
following image shows the configuration of the Technology Performance
Benchmarking Labs and represents a typical BI infrastructure layout:
Typical BI Layout
In the image, the arrows between each one of the blocks represent data
transfers over the network.
Data Transfer Deploying MicroStrategy High Performance BI 3
94 Introduction to Network Performance 2011 MicroStrategy, Inc.
The data transfer occurs between the following components:
Case ExampleNetwork Impact on Cube Publication Times
Data fetching is essentially the transfer of results data from the data warehouse
to Intelligence Server for further processing by the Analytical Engine. One of
the cases handled by MicroStrategys testing labs illustrates well the kind of
impact that poor network performance can have on the performance of a
MicroStrategy rollout. When profiling cube publication times for cubes of
different sizes, the team noticed that the data fetch operation was taking an
unexpectedly long timemore than 70% of total cube publication time.
Data Transfer Components
Fact table, lookup table, and index data between the Oracle data warehouse and
the NetApp Storage Area Network (SAN)
Result datasets, cube data, and attribute element data between the Oracle data
warehouse and the Intelligence Server
Schema and report object definitions between the Metadata server and the
Intelligence Server
XML, binary report, and dashboard results data between the Intelligence Server
and the Web server
HTML, JavaScript, Flash, images, and other static content between the Web
server and the client
Deploying MicroStrategy High Performance BI Data Transfer 3
2011 MicroStrategy, Inc. Introduction to Network Performance 95
The following image illustrates the cube publication process.
Intelligent Cube Publication Process
The analysis concluded that the latency was due to network issues between the
Oracle data warehouse and Intelligence Server. The routing within the
benchmark labs network was accidentally configured in a way that caused
unnecessary network latency and was affecting cube publication performance
The following table displays results before and after the network tuning:
Cube Characteristics
Cube size (MB)20,335
Number of rows205 million
Number of objects 13 attributes and 9 metrics
Performance Results
Time (hh:mm:ss) Before the Network Fix After the Network Fix
Total Time 02:37:43 01:22:08
SQL Execution Time 00:24:00 00:20:02
Data Fetch Time 01:52:06 00:39:04
AE Processing Time 00:21:37 00:23:02
Data Transfer Deploying MicroStrategy High Performance BI 3
96 Key Network Concepts 2011 MicroStrategy, Inc.
Key Network Concepts
After completing this topic, you will be able to:
Describe key computer network-related concepts.
Network Terminology
Before discussing recommendations from a network performance perspective,
it is important to be familiar with the following key computer networking
concepts:
BandwidthRefers to the amount of data that passes through a network
connection over time. It is also known as throughput and is typically
expressed in terms of bits per second (bps). The greater the capacity, the
more likely it is that greater performance will follow.
LatencyIt is a synonym for delay. It refers to the time it takes for a packet
of data to get from one designated point to another. It is usually measured
by sending a packet that is returned to the sender.
Selecting this option compresses and caches files with the extensions
.htm, .html, and .txt.
9 In the Temporary folder text box, type the path to a local directory where
the compressed files will be kept, or click Browse to locate the directory.
For more information about the Tomcat HTTP Connector and available
options regarding compression, refer to your Apache Tomcat
Configuration documentation.
Data Transfer Deploying MicroStrategy High Performance BI 3
102 Network Recommendations for High Performance 2011 MicroStrategy, Inc.
Setting Up Web Proxy Server
When limited network bandwidth and high latency exist between the
MicroStrategy Web server and the client browser or when the browser cache is
not enabled in customer environments, it is recommended to set up a proxy
server to improve the performance of accessing resources. Proxy servers
generally have their own caching capabilities, including the ability to cache all
the static content (e.g. images, scripts, files, and so forth), enabling it to avoid
unnecessary and expensive round trips to the Web server.
MicroStrategy Labs conducted tests to measure the performance gains of
adding a proxy server to the architecture. Adding a proxy server to the
architecture improved the response times by more than 30%. The following
table presents the test results. It displays the time that it took to navigate
through the project objects and to execute multiple dashboards, with and
without a proxy server:
Web Proxy Test Results
Actions No Proxy With Proxy Difference
Navigation Actions (login,
browsing, logout)
3.27 sec 2.15 sec 34%
Multi Dashboards
Executions
50.71 sec 32.2 sec 36%
Deploying MicroStrategy High Performance BI Data Transfer 3
2011 MicroStrategy, Inc. Distribution Services Performance 103
Distribution Services Performance
After completing this topic, you will be able to:
Apply high performance best practice techniques when working with
Distribution Services.
Just like networks, Distribution Services is technically another means of data
transfer between the MicroStrategy server infrastructure and a client. In this
section, you will learn about the different factors that most influence the
delivery throughput and response times of Distribution Services.
Lesson Summary
In this lesson, you learned:
Slow or poorly tuned network performance in any of the data transfers
available in a MicroStrategy environment translates into poor performance
from a report or dashboard execution perspective.
Bandwidth, latency, and network segments are key computer networking
concepts that are important to understand before discussing
recommendations from a network performance perspective.
To avoid the latency introduced by router or firewall data packet
processing, it is recommended to keep all servers within the same network
segment.
Data transfers from Web server to the client are critical to performance so
from that perspective, it is important to ensure that good network
bandwidth and relatively low latency exists between the two.
HTTP compression can help in optimizing performance when there is
limited network bandwidth and latency between the Web server and the
Web client computers.
It is recommended to set up a proxy server to improve the performance of
accessing resources.
The number of recipients for a subscription tends to have more impact on
Distribution Services performance than all other factors, because it directly
correlates to the memory footprint on Intelligence Server.
The size of the report or document delivered in the message is also an
important factor to consider when looking to optimizing performance of
Distribution Services.
Different transmission methods have significantly different performance
levels, with file being the fastest and print being the slowest method.
Different formats have significantly different performance levels, with CSV
being the fastest and Flash being the slowest.
You should enable caching for reports and documents that are delivered by
subscriptions.
Deploying MicroStrategy High Performance BI Data Transfer 3
2011 MicroStrategy, Inc. Lesson Summary 111
When creating document subscriptions to update caches, select the Use
valid dataset caches check box to ensure available report caches are used
when the subscription triggers.
The number of concurrent scheduled jobs is a governing setting that
controls the amount of jobs that trigger at the same time on Intelligence
Server based on a schedule. Setting this governor correctly is an important
step in optimizing performance because this step defines the concurrency
rate in Intelligence Server.
Subscriptions triggered by alerting will have approximately 15% less
throughput than equivalent regularly scheduled subscriptions.
Subscriptions personalized with security filters will present approximately
15% faster response times than equivalent subscriptions personalized with
prompts.
For performance optimization purposes, it is recommended that you create
subscriptions with similar number of recipients, where feasible, to avoid
overloading a single node in a clustered environment.
Data Transfer Deploying MicroStrategy High Performance BI 3
112 Lesson Summary 2011 MicroStrategy, Inc.
2011 MicroStrategy, Inc. 113
4
SYSTEM ARCHITECTURE AND
CONFIGURATION
Lesson Description
This lesson introduces you to important concepts regarding tuning your
MicroStrategy platform. In this lesson, you will learn about server specification
options, and the MicroStrategy recommendations to achieve optimized
performance. Next, you will learn about the most important settings to tune
Intelligence Server and their recommended values. Finally, this lesson will
discuss the main parameters for tuning a clustering system and the Web
environment.
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
114 Lesson Objectives 2011 MicroStrategy, Inc.
Lesson Objectives
After completing this lesson, you will be able to:
List the components of performance, understand the main performance
recommendations for server specification, system configuration, and the Web
environment.
After completing the topics in this lesson, you will be able to:
Understand the MicroStrategy recommendations in regards to server
specifications to achieve optimized performance. (Page 115)
Describe the main settings and governors that mostly impact the system and
understand their recommended values when tuning the environment.
(Page 126)
Understand the main parameters for tuning the Web environment. (Page
143)
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Server Specifications 115
Server Specifications
After completing this topic, you will be able to:
Understand the MicroStrategy recommendations in regards to server
specifications to achieve optimized performance.
Every Business Intelligence deployment depends on server infrastructure, that
is, hardware and software to function properly. Similar to other factors such as
database schema and dashboard design, the characteristics of the server
infrastructure on which MicroStrategy is installed and configured will have a
significant influence in the overall performance of the system as experienced
by users running reports and dashboards. This section covers the different
characteristics of server infrastructure from a performance perspective.
Processor
In computing, a processor is the unit that reads and executes program
instructions, which are fixed-length chunks of data (typically 32- or 64-bit
depending on the processor architecture).
The core is the part of the processor that performs the reading and execution
operations for each instruction. A multi-core processor is a processing system
composed of two or more independent cores. A dual-core processor contains
two cores, a quad-core processor contains four cores, hexa-core processor
contains six cores, and so on.
Single-core processors can only process one instruction at a time while
multi-core processors can process multiple instructions simultaneously.
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
116 Server Specifications 2011 MicroStrategy, Inc.
In the case of MicroStrategy, the different engines and components on
Intelligence Server do take advantage of the additional processing power
provided by multi-core processors. The graph below displays internal test
results showing where Point A (i.e. the largest system throughput at which the
system response time is stable) is reached for different processor cores on
Intelligence Server.
Degradation Curve by Number of Cores
The graph shows that Intelligence Server performance in terms of throughput
achieved increases as you add more processor cores to the system. However,
the increase in throughput is not linear, that is, doubling up the number of
cores does not cause the system throughput to double.
The table below shows the ratio at which system throughput increases, based
on the same tests that produced the previous chart.
Throughput Versus Number of Cores
% Increase in Throughput (with stable response
time)
1 to 2 cores 100%
2 to 4 cores 84%
4 to 8 cores 63%
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Server Specifications 117
Architecture and Processing Speed
A processor executes a certain amount of instructions within a time frame or
cycle. The clock speed (or simply speed) of a processor is measured as the
number of cycles it can perform in a given second. A speed of one cycle per
second is called a hertz. A processor that has a frequency of 1 million cycles per
second has a speed of 1 megahertz.
As a general rule, faster clock speeds in a processor will translate into better
throughput (and thus better overall performance) because more instructions
per second will be executed. However, just as for number of cores, increase in
clock speed will not translate linearly into increase in system throughput.
Many other factors, including the processor architecture, the software itself
and the operating system on which it is running will have a direct impact on
scalability.
At MicroStrategy, the different engines and components on Intelligence Server
do take advantage of the additional processing speeds and modern processor
architectures. The table below demonstrates the difference in throughput for
Intelligence Server based on the same tests performed for the previous section,
but using different generations of processors:
Nehalem is the code name for the latest processor architecture from Intel. The
Nehalem processors used in the test were Intel Xeon and had a clock speed of
2.8GHz. The Pre-Nehalem processors are also Intel multi-core processors with
a clock speed of 2.66GHz. The significant increase in throughput cannot be
solely attributed to the additional processor speed, but also to changes in the
processor architecture, such as larger CPU caches and the new
Hyper-Threading technology that is specific to the Nehalem generation.
Throughput (User Queries/Min)
# of Cores
Pre-Nehalem
Processor
Nehalem
Processor
%Increase
1 211 394 87%
2 397 790 99%
4 794 1,451 83%
8 1,286 2,361 84%
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
118 Server Specifications 2011 MicroStrategy, Inc.
Utilization
Processor Utilization is another important parameter that must be taken into
account from a performance perspective. Regardless of the individual
processor characteristics, all processors reach a point of saturation, beyond
which the system performance degrades as more instructions are sent to the
processor by the system. This saturation point tends to be different based on
the application, so it is important to know what it is for MicroStrategy.
The graph below shows the resulting Processor (or CPU) Utilization Rates on
Intelligence Server as the report submission rate is increased.
CPU Utilization Rates
As the graph demonstrates, when Point A (i.e. the largest system throughput at
which the system response time is stable) is reached, the Processor Utilization
Rate is around 80%. Below that point, increases in submission rate translate in
equivalent increases in system throughput with a stable average response time.
Based on the above study, it is recommended to keep Intelligence Server
utilization rates at around 80% or less.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Server Specifications 119
Memory
Another very important performance factor in a MicroStrategy deployment is
the RAM available in the system. When a computer manipulates data, it places
the data in memory to be retrieved or manipulated as necessary. This
configuration results in optimum performance because computer memory is
optimized for fast access for further processing.
Disk Swapping
If the computer runs out of usable memory, it is forced to store all the
temporary data on its hard drive in what is called the page file or swap space.
When the processor is ready to use that information, the computer then has to
read it back from its hard drive and place it into memory where it can be
readily used. On average, accessing data from memory is about 10x faster that
accessing the same amount of information from disk.
MicroStrategy is an especially data-intensive application that requires
processing of potentially very large amounts of information while performing
tasks such as cube publication or large report/dashboard processing. Given the
above, it is very important from a performance perspective to have as much
memory as possible for a MicroStrategy implementation to avoid disk
swapping issues. The following are potential indicators of disk swapping:
The average amount of RAM required by Intelligence Server is around 80%
of total available RAM
The performance counter % Disk Time is greater than 80%
The performance counter Current Disk Queue Length shows a large
number of waiting requests
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
120 Server Specifications 2011 MicroStrategy, Inc.
The image below demonstrates the concept of Disk Swapping. In this case, the
memory demands of Intelligence Server and the other applications that also
reside within the server (e.g. Operating System, Database, so forth) have
exceeded the amount of available RAM in the server, resulting in disk
swapping and performance degradation.
Disk Swapping = Poor Performance
64-Bit
MicroStrategy recommends that all implementations be based on 64-bit
operating systems. The 64-bit operating systems do not have the same
restrictions on user address space that 32-bit operating systems have.
Although 64-bit operating systems do have user address space ceilings, this
limit is significantly higher than the one imposed by 32-bit operating system.
Intelligence Server Universal Edition is compiled as a 64-bit application and
runs on 64-bit operating systems. As a result, Intelligence Server Universal
Edition can take advantage of the larger user address space available on such
systems, significantly reducing the possibility of system performance
degradation due to disk swapping.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Server Specifications 121
Memory Utilization
From a performance perspective, to minimize disk swapping is recommended
to increase the amount of available RAM if server memory utilization regularly
reaches or exceeds the 80% threshold. Besides the obvious method of
physically adding additional memory chips to the server (assuming it has
remaining capacity on this front), there are a couple of alternative techniques
to consider:
Reducing the number of memory intensive applications that are running on
the Intelligence Server machine (for example databases or other
applications)
Shutting down processes, such as services and applications that are not
being used and still get allocated memory by the operating system
The image below shows an ideal Intelligence Server memory configuration. All
applications on the server, including MicroStrategy, are operating normally
and under the 80% threshold, hence, the need for disk swapping never arises.
Ideal Memory Utilization (<80%)
Disk Storage
Hard Drives (also simply called Disk) and Storage in general are an integral
component of any MicroStrategy deployment. Faster disk configurations will
yield faster input/output operations on the server, resulting in better
performance overall. Storage can be internal to the server (that is, the hard
drives are physically installed on the server hardware) or, more commonly for
applications that require large amounts of data such as MicroStrategy, external
in the form of SAN (Storage Area Network) or NAS (Network-Attached
Storage)
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
122 Server Specifications 2011 MicroStrategy, Inc.
RAID Configurations
RAID, an acronym for Redundant Array of Inexpensive Disks, is a technology
that provides increased storage reliability through redundancy, combining
multiple low-cost, less-reliable disk drive components into a logical unit where
all drives in the array are interdependent. The different schemes or
architectures are named by the word RAID followed by a number (for example,
RAID 0, RAID 1). Each scheme has a degree of reliability and performance.
Schemes with less reliability such as RAID 0 will generally cost much less than
highly reliable schemes such as RAID 10. RAID's various designs involve two
key design goals: increase data reliability and increase input/output (I/O)
performance. In general RAID 4 or RAID 5 present the best reliability-to-cost
ratio for a MicroStrategy deployment.
Disks Fragmentation in Windows Environment
When a disk is highly fragmented, the file system will have free segments in
many places, and some files may be spread over many extents. Access time for
those files will tend to become longer as the disk becomes more fragmented,
negatively impacting performance for all I/O intensive operations. From a
performance perspective, it is highly recommended to periodically defragment
the disk on Intelligence Servers running on Windows environment
(fragmentation is mainly a concern on Windows Operating Systems).
The two images below illustrate the concept of disk fragmentation:
Fragmented Disk = Performance Degradation
A Defragmented Disk
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Server Specifications 123
Disk Utilization
Similar to Processors and Memory, Disks and Storage in general also have a
utilization rate beyond which the system performance degrades or becomes
unreliable. Based on internal tests the threshold for Disk Utilization is
approximately 70% for MicroStrategy deployments.
Ideal Disk Utilization (<70%)
Operating Systems
The MicroStrategy platform must operate on top of an Operating System (OS).
Depending on how efficiently it uses the server's resources, the OS will have an
impact on overall system performance. The following chart shows the results of
internal tests comparing the performance, based on throughput, of Windows
vs. Linux-based MicroStrategy deployments.
Throughput Comparison - Linux vs. Windows
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
124 Server Specifications 2011 MicroStrategy, Inc.
The following chart shows the results of internal tests comparing the
performance, based on response time, of Windows vs. Linux-based
MicroStrategy deployments.
Response Time Degradation - Linux vs. Windows
While both operating systems yield relatively same levels of throughput and
similar average response times, the largest system throughput at which the
system response time is stable (A) is higher for Linux than for Windows. Based
on the above observations and for the same 64-bit hardware, Linux yields
higher levels of throughput than Windows (approx. 12%).
Virtualization
More MicroStrategy implementations are being deployed using Virtualization
technology such as VMWare, Microsoft Hyper-V, or LPAR on AIX. From an IT
management perspective, virtualization offers great benefits in terms of
improving server utilization rates and reducing maintenance overhead.
However, from a performance perspective, because it adds an extra layer of
indirection between the application and the actual hardware, virtualization
tends to have a negative impact. As implementation architects, it is important
to understand the tradeoffs between virtualization and performance.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Server Specifications 125
The following chart shows the results of internal tests conducted deploying
MicroStrategy on top of a VMWare vSphere Enterprise Plus environment.
Physical vs. Virtual Deployments
Thread borrowing is only allowed from high to low. That is, high priority
report requests can borrow medium and low priority threads, medium
priority report requests can borrow only from low priority threads, and
low priority requests can only use low priority threads.
Thread connection borrowing ensures that higher priority report requests get
satisfied as soon as possible without being queued up. The table below
describes the 5 different parameters you can use to prioritize report requests:
Prioritization Parameters
Parameter Description
User groups An executive management user group may be assigned HIGH priority so
that their reports may execute before anyone elses.
Application
type
Assign priorities for requests coming from Desktop, MicroStrategy Office,
and MicroStrategy Web. For example, by assigning medium priority on all
MicroStrategy Desktop based requests, you ensure that all developers
would be able to run reports on medium priority threads.
Projects Report requests from project A may run under high priority, while report
requests from project B run on lower priority.
Request
type
You can assign element requests to be of high priority and report requests
to be of low priority.
Cost An arbitrary value that a developer or an administrator can assign to a
report. By assigning a cost, the developer quantifies the resources required
to execute the report, such as report execution time or result set size.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Intelligence Server Configuration 135
Cost enables you to separate slower running reports that could monopolize
database connections from other faster running reports, as shown below:
Assigning Costs to Reports
Based on report costs, administrators can assign priorities ensuring that
critical jobs are executed first, leaving less critical jobs in the queue to be
executed at a later time. The table below shows a typical job prioritization
model, based on cost criteria:
MicroStrategy recommends that you allocate High/Medium/Low Priority
Threads in a 10/60/30 ratio. For example, 10% should be HIGH priority, 60%
or bulk should be MEDIUM priority, and 30% of all threads should be LOW
priority. Element requests should be assigned to high priority threads,
interactive reporting should be routed to medium priority threads, and batch
reports or high cost reports should be assigned to low priority threads.
Priority Matrix
Priority User Group BI Project
Request
Type
Application Report Cost Notes
High ALL ALL Element ALL Light Element browsing has the lightest
cost & higher priority
High Executive
Management
Sales
Analysis
ALL ALL Light Executive users lightest cost &
highest priority
Medium Business Analyst ALL ALL Web Medium
Medium Developer Sales
Analysis
ALL Desktop Medium Report development is a Medium
cost effort
Low ALL ALL Reports Scheduler Heavy
Low ALL ALL Reports Narrowcast Heavy Batch jobs are assigned lowest
priority threads
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
136 Intelligence Server Configuration 2011 MicroStrategy, Inc.
Server Load
The number of open jobs in the system is a variable that can have a high impact
on resource utilization. Execution of each job requires resources, such as
memory, CPU, Network, and disk space.
A job can be as simple as requesting elements or objects from the Metadata, or
as complex as executing highly formatted dashboards with multiple datasets.
Scheduled subscriptions, Narrowcast Server, and MicroStrategy Office can also
contribute to overloading the system with a large amount of concurrent jobs.
As more users are logged into the system, the chances of experiencing high
concurrency of executing jobs increases.
To ensure proper operation, the amount of jobs executing on Intelligence
Server can be limited by defining the following governing settings:
Jobs per user account
Jobs per user session
Executing jobs per user
Jobs per project
Intelligence Server Elapsed Time (for interactive and scheduled reports)
SQL timeout
In addition, the following methods are recommended to minimize the
generation of unnecessary jobs:
Controlling the report creation privilege
Analyzing usage patterns and creating mandatory prompt combinations
Defining efficient drill maps
Job Monitor and Performance Monitor are efficient tools to obtain a real-time
snapshot of the system state. Enterprise Manager helps monitor jobs and
observe patterns of resource utilization with a comprehensive historical view.
Logging and Statistics
Intelligence Server performance can degrade depending on the level and
amount of statistics that it is configured to collect.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Intelligence Server Configuration 137
For example, the collection of report job SQL statements is useful in certain
contexts for database administrators. However, because of the operational
overhead required, collecting SQL for every report job that executes in
Intelligence Server will likely degrade system performance.
As a general practice, it is recommended to only enable low level logging and
statistics collection in the context of troubleshooting or tuning scenarios, but
never in a production environment.
Clustering
Clustering Intelligence Servers can help the system to better handle the
workload of a project. Having multiple servers increases the number of users
that can connect to the system at the same time and the number of jobs that
can be processed simultaneously. Dividing a project workload can decrease the
time users spend in queue and the time it takes to process reports as more
resources are available.
Cache Sharing
In a clustered environment, the best configuration for cache sharing is to have
each node maintaining its own cache files. This approach has the following
benefits:
Optimized Network TrafficThere is a split in the amount of network
traffic among the nodes. In addition, because each node can access its local
cache, a great percentage of the workflow is accelerated.
Better Failover SupportIf one of the nodes shuts down, the backup
nodes can regenerate the caches that were stored by the failed node. In this
case, the workflow will not be significantly affected.
Backup Frequency
Backup frequency controls the frequency, in minutes, at which cache and
History List messages are backed up to disk. In clustered environments, it is
recommended to set this value to 0, which causes caches and History List
messages to be backed up to disk immediately as they are created in memory. If
caches and History List messages are available in disk as soon as they are
created, they are synchronized correctly across all clustered nodes.
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
138 Intelligence Server Configuration 2011 MicroStrategy, Inc.
History List Storage
History List messages can be stored in a database or in flat files. For
performance optimization purposes, it is recommended to store History List
messages in a database. The performance benefits of using database History
List are listed below:
Additional information about each History List message can be accessed,
such as message and job execution statistics. As a result, monitoring and
management of History List messages becomes more effective.
Database storage of the History List provides greater scalability and
improved performance. Instead of accessing several large files that reside
on the server machine, inbox information is retrieved from a database.
When the administrative task to delete History List messages is triggered, it
creates only one session on Intelligence Server rather than tens or hundreds
of separate sessions for each user.
The History List Messages monitor can be used to manage messages for
each MicroStrategy user.
Project Failover Latency
The Project Failover Latency setting is a server-level setting that defines the
wait time before a project is loaded on a backup server when a project failover
triggers. The minimum value for this setting is -1 and the maximum value is
999. Defining values such as 0 (zero) or -1 for this setting, disables the latency
period. This means that the project is not automatically loaded onto a
surrogate server. Consider the following information when defining the value
for this setting:
Setting a higher latency period prevents projects on the failed server from
being loaded onto other servers quickly. This behavior is positive from a
performance perspective if projects are large (they take a long time to load),
and the failed server can recover quickly. A high latency period provides the
failed server more time to recover.
Setting a lower latency period causes projects from the failed server to be
loaded relatively quickly in another server. This behavior is positive if it is
crucial that your projects are available to users at all times.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Intelligence Server Configuration 139
Configuration Recovery Latency
The Configuration Recovery Latency setting is a server-level setting that
defines the wait time before a node is set back to its original configuration,
after the node recovers from a project failover. The minimum value for this
setting is -1 and the maximum value is 999. Defining values such as 0 (zero) or
-1 disables the latency period, the project is not unloaded from the surrogate
server, and the project is not reloaded on the recovered server. Consider the
following information when defining the value for this setting:
Setting a higher latency period leaves projects on the surrogate server
longer, which will be beneficial if the projects are large and it is critical to
ensure the recovered server is stable for a specific period of time before the
project load process begins.
Setting a lower latency period causes projects on the surrogate machine to
be removed and loaded relatively quickly onto the recovered server, which
helps relief the extra load on the surrogate server.
Load Balancing
Load balancing is a property that determines the load distribution across
Intelligence Server nodes. The load balancing ratio should reflect any
asymmetries in the CPU power of the nodes. For example, Node 1 may have
twice the CPU power than Node 2. In this case, load balancing should be
configured so Node 1 handles twice the load than Node 2 does.
Cube Failover Support
In a multi-node clustered environment, it is possible that all cubes are
published on a single node. If the server for that node goes down, the cubes
would need to be republished before any further report or dashboard execution
that uses them could take place. This can have a significant impact on the
performance of the entire system.
From a system performance standpoint, it is better to pre-load the Intelligent
Cubes into each node before users execute any cube reports. In this case, users
will not experience any loading overhead when the cube is hit for the first time.
A complete solution is to create an Integrity Manager test script that runs cube
reports that access the cubes that need to be pre-loaded. The script can later be
executed using a batch file in Windows, which can be scheduled using the
Windows Scheduled Tasks utility.
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
140 Intelligence Server Configuration 2011 MicroStrategy, Inc.
If cubes are pre-loaded on other nodes and the primary node goes down,
users can still activate, de-activate, unload, and delete. If the cube is
unloaded, it cannot be loaded back.
For deploying the pre-loading of cubes approach, the following pre-requisites
must be met:
Each node needs to have enough memory to host all the cubes. Otherwise,
the pre-loaded cubes that have not been recently used are swapped out of
memory.
Only one report is needed for each cube that needs to be published. if
you define a view filter that returns no data on the report, it will help
to speed up the publishing process.
To pre-load cubes on all clustered nodes:
The procedure below only lists the relevant steps in the Integrity
Manager Wizard that you need to configure for cube pre-loading. If an
Integrity Manager Wizard step is not listed in this procedure, you can
keep the default values and click Next. For additional details on how to
create and save an Integrity Manager test script, refer to the
MicroStrategy Administration: Application Management course.
1 In MicroStrategy Integrity Manager, create a Single Project test.
2 On the Select Objects from the Base Project to be included in the Test page,
select the check boxes for the cube reports that are connected to the cubes
that need to be published.
3 On the Select Processing Options page, under Reports, select the SQL/MDX
check box.
4 Under Documents, select the Execution check box.
5 On the Summary page, click Save Test.
Ensure there is a sufficient gap in time between the running of the batch
file for loading the cubes into each node and the cube publication
schedule on the primary node.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Web Server Configuration 143
Web Server Configuration
After completing this topic, you will be able to:
Understand the main parameters for tuning the Web environment.
Web Server is the other key server component of the MicroStrategy
architecture. The following sections provide tuning and optimization
recommendations for the MicroStrategy Web Server.
JVM Settings
One of the key components that generally requires optimization in the context
of non-IIS Web servers is the Java Virtual Machine (JVM). Improper or
non-optimized JVM settings configuration can prevent the Web server from
running or cause it to be unstable and run very slowly.
The main JVM Settings to configure within the context of a MicroStrategy
implementation are:
Initial Heap Size(Xms)Specifies the initial Heap Size available to the
JVM applications, in megabytes. This represents the active part of the
storage heap.
Maximum Heap Size (Xmx)Specifies the maximum Heap Size available
to the JVM applications, in megabytes.
The Garbage Collection Process
The garbage collection process in the Web server is essential to understanding
the importance of tuning the Web server Heap Size. Garbage collection and
heap expansion are an essential part of a JVM operation.
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
144 Web Server Configuration 2011 MicroStrategy, Inc.
The amount of objects created by applications that run in the JVM defines the
frequency at which garbage collection occurs. When an application is running
and the objects created by the application take most of the space defined by the
Initial Heap Size, the JVM is unable to allocate any more objects and triggers
the garbage collection process. This process deletes the stored objects in the
heap that are no longer referenced by applications, making some free space in
the heap, so that new objects can be processed. Usually, during the garbage
collection process, all other processes running in the JVM stop. It is an
expensive process from a performance standpoint. If the garbage collection
fails to clean enough space in the heap, the Initial Heap Size is expanded.
The heap expansion can never go beyond the Maximum Heap Size.
Heap Size Configuration
A Maximum Heap Size that is very high can cause disk swapping, causing the
Web server performance to degrade drastically. It is very important from a
performance perspective to set the Maximum Heap Size to a value that allows
the heap to be contained within the available RAM.
In this sense, a 64-bit Web server environment provides much better
performance than a 32-bit one, given the amount of memory available as
potential heap space (especially for applications with a high number of
dashboards containing large datasets.)
Increasing the Initial Heap Size will generally improve startup speed because it
avoids triggering the garbage collection process early on. Increasing the
Maximum Heap Size improves throughput as long as the heap resides fully in
physical memory and does not generate swapping.
MicroStrategy Web Pool Sizes
TCP/IP connections are used between the MicroStrategy Web XML API and
Intelligence Server to pass requests to and retrieve the responses from
Intelligence Server. When MicroStrategy Web connects to an Intelligence
Server node, either manually or automatically, the MicroStrategy Web XML
API establishes a connection pool with each Intelligence Server node. The
connection pool consists of a number of TCP/IP connections equivalent to the
number specified in the Initial pool size property on the Administrator page.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Web Server Configuration 145
A connection pool is responsible for caching connections between the Web
server and Intelligence Server.
Web Connection Pool
When a user request comes in, the MicroStrategy Web XML API gets a free
connection in the connection pool to serve the request. All connections in the
connection pool are initially free. When in use for processing a user request,
they are set to busy. After a response for the request has been completed, the
connection is returned to the connection pool and displays as free.
When the MicroStrategy Web XML API detects that the number of free
connections in a connection pool is less than two and the connection pool has
not reached the maximum pool size, it dynamically expands the connection
pool. If the connection pool reaches the maximum pool size, as when a user
request comes in when all connections are busy, the request waits for a free
connection. The wait time is configurable through the Administrator page, by
using the Server busy timeout property. If this time is reached and no
connections are available, an error returns to the client browser.
There are two related settings in MicroStrategy Web that refer to the
connection pool:
Initial Pool SizeRepresents the initial number of connections in the
connection pool
Maximum Pool SizeRepresents the maximum number of connections in
the connection pool
A general recommendation is to set the Maximum Connection Pool parameter
between 1/3 and 1/6 of the Maximum user sessions allowed.
To define pool size settings in MicroStrategy Web:
1 Launch the MicroStrategy Web Administrator page.
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
146 Web Server Configuration 2011 MicroStrategy, Inc.
2 Under WEB SERVER, click Default properties.
3 Under Connection properties, define the values for Initial pool size and
Maximum pool size, as shown below:
Pool Size
4 Click Save.
Using a Separate Web Server for Static Content
In addition to dynamic content such as the actual report or dashboard data, the
requests from MicroStrategy Web comprise a series of static content, such as
css files, images and JavaScript files. While the default configuration is to have
all of the static and dynamic content in the main MicroStrategy Web Server, in
a production environment with high concurrency, this type of architecture
design is not recommended.
For performance optimization purposes, it is recommended to deploy a
separate Web server to handle the static content and to cache this information.
The cached static content and information does not need to be processed every
time it is requested. This implementation enables the MicroStrategy Web
Server to handle mostly the dynamic content, while the static content is stored
on the separate Web server.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Web Server Configuration 147
Logging and Statistics
A series of internal tests were performed to analyze the impact of statistics and
logging on MicroStrategy Web performance. The tests showed that enabling
logging and statistics does impact performance negatively and tends to
generate an overhead on JVM heap usage. In the worst case, enabling Web
statistics generated a 200% drop in performance in terms of Web server
response time and a 30% increase on JVM heap usage.
It is recommended to only enable logging and statistics in the context of
troubleshooting or tuning scenarios, but never in a production environment.
JavaScript
JavaScript is a scripting programming language most commonly used to add
interactive features to Web pages. JavaScript files are not generated
dynamically. When MicroStrategy is installed, a fixed amount of JavaScript
files will be stored in the Web server. Depending on the type of functionality
that is included in reports or documents at design time and the subsequent
actions and manipulations from users, requests for certain JavaScript files will
be made by the client. The more JavaScript files that need to be loaded in the
client, the longer will take for the client to render the report or dashboard.
In terms of performance optimization, it is important to understand what
actions can be taken in MicroStrategy Web to decrease the amount of
JavaScript files requested. The list below contains some recommendations to
improve client rendering based on reducing JavaScript file requests:
Disable Lock Row Header and Lock Column Header
Set Document Width Mode to be Fixed
Execute reports and documents in Full Screen Mode
Enable Browser Cache and Cookies
Enable HTTP Compression
Restrict the number of HTML Containers used
Lesson Summary
In this lesson, you learned:
Single-core processors can only process one instruction at a time while
multi-core processors can process multiple instructions simultaneously.
Intelligence Server performance in terms of throughput achieved goes up as
the number of processor cores goes up. The increase in throughput is not
linear.
As a general rule, faster clock speeds in a processor will translate into better
throughput, but not always linearly.
It is recommended to keep Intelligence Server utilization rates below 80%.
Have as much memory as possible for a MicroStrategy implementation to
avoid disk swapping issues.
In 32-bit versions of Microsoft Windows, applications are limited to 3 GB of
user address space. For this reason, MicroStrategy recommends that all
implementations be based on 64-bit operating systems.
It is recommended to increase the amount of available RAM if server
memory utilization regularly reaches or exceeds the 80% threshold.
In general RAID 4 or RAID 5 present the best reliability-to-cost ratio for a
MicroStrategy deployment.
It is highly recommended to periodically defragment the disk on
Intelligence Servers running on Windows environments.
The threshold for disk utilization rate beyond which the system
performance degrades or becomes unreliable is approximately 70% for
MicroStrategy deployments.
Linux yields higher levels of throughput than Windows (approx. 12%).
The overhead of virtualization causes some performance degradation,
reaching almost 25% for an eight-core system.
It is recommended to optimize Intelligence Server memory resources by
automatically disconnecting idle users, allowing more concurrency on
highly used applications, while reducing concurrency on less ones.
Deploying MicroStrategy High Performance BI System Architecture and Configuration 4
2011 MicroStrategy, Inc. Lesson Summary 149
Controlling the functionality that users are granted can be a useful
technique for optimizing system resource usage.
Intelligence Server enables system administrators to manage the number of
requests that can be submitted at any given time by using governor settings.
The default working set value is 200 MB, but it generally has to be
increased, based on estimations and specific environment characteristics.
It is recommended to optimize Intelligence Server memory by controlling
History List usage
Limit report data sizes to optimize Intelligence Server resource usage.
Limit the number of cells to export to text or Excel in MicroStrategy Web.
The optimal range of threads corresponds to the zone of maximum system
throughput.
Administrators can assign arbitrary costs to individual reports, separating
slower running reports that could potentially monopolize database
connections from other faster running reports.
To ensure proper operation, the amount of jobs executing on Intelligence
Server can be limited by defining governing settings
The best configuration for cache sharing in a clustered environment is each
node maintaining its own cache files.
It is recommended to store History List messages in a database.
It is better to pre-load the cubes into each node before users execute any
cube reports. In this case, users will not experience any loading overhead
when the cube is hit for the first time.
Improper or non-optimized JVM settings configuration can prevent the
Web server from running or cause it to be unstable and run very slowly.
The main JVM Settings to configure within the context of a MicroStrategy
implementation are Initial Heap Size and Maximum Heap Size.
Increasing the Initial Heap Size will generally improve startup speed
because it avoids triggering the garbage collection process early on.
Increasing the Maximum Heap Size improves throughput as long as the
heap resides fully in physical memory and does not generate swapping.
A general recommendation is to set the Maximum Connection Pool
parameter between 1/3 and 1/6 of the Maximum user sessions allowed.
System Architecture and Configuration Deploying MicroStrategy High Performance BI 4
150 Lesson Summary 2011 MicroStrategy, Inc.
It is recommended to deploy a separate Web server to handle the static
content and to cache this information. This implementation enables the
MicroStrategy Web Server to handle most of the dynamic content, while the
static content is stored on the separate Web server.
It is recommended to only enable low level logging and statistics collection
in the context of troubleshooting or tuning scenarios, but never in a
production environment.
The more JavaScript files that need to be loaded onto the client, the more
time it will take for the client to render the report or dashboard.
It is important to understand what actions can be taken in MicroStrategy
Web to decrease the amount of JavaScript files requested.
2011 MicroStrategy, Inc. 151
5
DATA PRESENTATION
Lesson Description
This lesson expands on how the BI ecosystem impacts report, dashboard, and
mobile performance. In this lesson, you will learn about the report and
dashboard execution flows, and the key recommendations for dataset and
design optimization. These series of design techniques will help you develop
fast executing reports and dashboards, presented in different devices.
Data Presentation Deploying MicroStrategy High Performance BI 5
152 Lesson Objectives 2011 MicroStrategy, Inc.
Lesson Objectives
After completing this lesson, you will be able to:
Describe report and dashboard execution flow. Understand the
recommendations for designing high performance reports and dashboards.
After completing the topics in this lesson, you will be able to:
Describe the report execution flow and understand the report configuration
techniques for optimizing performance. (Page 153)
Describe the dashboard execution flow. (Page 159)
Understand dataset techniques to optimize dashboard performance. (Page
162)
Understand design techniques for optimizing dashboard performance.
(Page 172)
Optimize your mobile device to experience performance gains when
rendering MicroStrategy dashboards. (Page 194)
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. High Performance Reports 153
High Performance Reports
After completing this topic, you will be able to:
Describe the report execution flow and understand the report configuration
techniques for optimizing performance.
Report Execution Flow
It is important to be familiar with the report execution flow to better
understand how some of the components affect the report performance in a
MicroStrategy environment. The following image depicts the different steps
involved in the report execution process:
Report Execution Flow
Data Presentation Deploying MicroStrategy High Performance BI 5
154 High Performance Reports 2011 MicroStrategy, Inc.
The following steps and components are involved in the report execution flow:
1 A user submits a report request from a Web browser.
2 The report request is sent to Intelligence Server via the Web server.
3 In the case of prompted reports, before proceeding to the next step,
Intelligence Server sends the request back to the user to obtain prompt
answers.
4 After the prompt is answered, Intelligence Server checks for a valid report
cache. If a valid cache exists, the result set is returned to Web server,
otherwise Intelligence Server proceeds to the next step.
5 Intelligence Server retrieves the report definition from the metadata.
6 The SQL Engine generates a SQL statement.
7 The Query Engine opens a connection to the data warehouse, submits the
SQL statement, and retrieves the result set.
8 The Analytical Engine performs any additional analytical processing, where
necessary, and formats and cross-tabs the result set.
9 Intelligence Server sends the complete report results in XML format to the
Web server. A report cache is also created during this time.
10 The Web server translates XML data into HTML and formats the report for
display in the Web client (browser).
11 The report results are displayed in the Web client.
Report Configuration Techniques to Optimize Performance
Large XML can require significant amount of Intelligence Server memory and
require a lot of processing cycles on both the Intelligence Server and Web
server. Therefore, several of the recommendations for optimizing the
rendering of reports are directly related with the strategies to minimize the size
of the XML. The following sections cover different report configuration
techniques that contribute to better performance.
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. High Performance Reports 155
Building Simple Reports
While it is important to ensure that reports deliver the required level of data
and analysis, from a performance perspective, simplicity generally translates to
faster report execution times. The simpler the reports, the smaller is the XML
and the faster is the overall response time.
Avoiding Working Set Reports
Working Set reports (also known as OLAP reports) provide great flexibility to
users, but it comes at a significant cost in terms of performance if the OLAP
features are not used. With Working Set reports, a user can have multiple
objects in the template, but only display a subset of these objects in the report
when it is executed. Subsequent report manipulations that use the objects in
the template do not require further SQL generation and tend to be faster.
However, the process of producing and rendering Working Set reports is more
expensive than the process for regular reports. For instance, Intelligence
Server must keep several versions of a Working Set report in memory, one
version for the base report (that is, the report with all the objects actually
available in the template) and one version for each subsetting report accessed
by the user (that is, each report version that the user has requested and has less
than all the template objects).
Data Presentation Deploying MicroStrategy High Performance BI 5
156 High Performance Reports 2011 MicroStrategy, Inc.
The following image illustrates this concept:
Working Set Report
For big Working Set reports, a considerable amount of XML is generated
requiring significant processing and memory resources in both the Intelligence
Server and Web server. When designing reports, ensure that there is a strong
need for the report to be a Working Set report. If the need is not there,
significant resources will be saved by just making it a regular report.
Using Only the Necessary Prompts
Prompts can be very beneficial to the report execution process because they
filter the data that returns to the client from the database and thus save
processing and memory resources at all levels of the MicroStrategy
architecture. The less data that needs to be fetched from the data warehouse,
the faster is the response time experienced by users.
However, it is important to understand that each prompt in a report requires
additional Intelligence Server resources. This is because Intelligence Server
has a process, called the Resolution step, that matches all prompt answers in a
report. Even though this process is fast and represents a very small percentage
of the overall report execution time, for high concurrency scenarios it can
become a costly bottleneck.
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. High Performance Reports 157
Refraining From Using Search Inside Prompts
While search objects can provide an alternative way to define the prompt
objects available for selection, they are not efficient in terms of report
execution and will have a negative impact on performance. The
behind-the-scenes process to search for objects is very expensive for both the
Intelligence Server and the data warehouse, which in turn impacts the report
response time.
Limiting the Use of Custom Groups
Custom groups based on multiple criteria generate SQL to retrieve the metric
results for each individual set. If more than one custom group is used on a
template, SQL must be generated to retrieve the results for each combination
of group of elements. This can quickly lead to a large number of SQL passes to
be generated even for an otherwise simple report. You should consider
reducing the number of custom groups on a report template wherever possible.
You can also consider reducing the number of elements in each custom group
to minimize the footprint for report execution.
Keeping Aggregations and Joins to a Minimum
When a report executes, the more tables in the data warehouse it has to access,
the longer the report takes to run. Therefore, when designing the report,
keeping the number of joins and aggregations that either the data warehouse
or the Analytical Engine will have to perform to a minimum will have a positive
impact on performance.
Using Incremental Fetch Whenever Possible
Large report result sets, when converted to XML, generally require significant
amounts of Intelligence Server memory. In addition, the wider the reports are
in terms of columns, the longer they take to be converted to HTML by
MicroStrategy Web, resulting in slow report execution times from a user
perspective. By rendering data in smaller slices, the incremental fetch setting
in MicroStrategy Web can help significantly reduce the user wait times, while
still handling the display of large result sets.
Data Presentation Deploying MicroStrategy High Performance BI 5
158 High Performance Reports 2011 MicroStrategy, Inc.
Limiting the Number of Drill Paths
The number of drill paths available for Web users affects the size of the XML
that must be generated by Intelligence Server for a given report. To optimize
report results rendering performance results in Web, it is recommended to use
the Maximum number of XML drill paths governor setting in Intelligence
Server. This setting limits the number of drill paths available to Web users and
is shown below:
Maximum Number of XML Drill Paths
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. High Performance Dashboards 159
High Performance Dashboards
After completing this topic, you will be able to:
Describe the dashboard execution flow.
The Dashboard Execution Flow
It is important to be familiar with the document execution flow to better
understand how some of the components may affect the performance of a
MicroStrategy environment. The following image depicts the different steps
involved in the dashboard execution process:
Dashboard Execution Flow
The steps and components involved in the dashboard execution flow are as
follows:
1 A user requests the execution of a Report Services dashboard.
Data Presentation Deploying MicroStrategy High Performance BI 5
160 High Performance Dashboards 2011 MicroStrategy, Inc.
2 Intelligence Server receives the request and processes it as follows:
It executes all datasets in the dashboard
After collecting the data, Intelligence Server creates a virtual dataset
and stores it into memory
You can define the incremental fetch options in both MicroStrategy Web
and in MicroStrategy Desktop, but incremental fetch is applied only
when the document is executed in MicroStrategy Web.
To enable incremental fetch of data for grids:
1 In MicroStrategy Web, open the document in Design Mode.
2 On the left-side pane, select Document Structure.
3 In the Document Structure pane, expand the section of the dashboard that
contains the grid for which you want to disable automatic resize option.
4 Right-click the grid object, and select Properties and Formatting.
Data Presentation Deploying MicroStrategy High Performance BI 5
186 Design Techniques to Optimize Performance 2011 MicroStrategy, Inc.
5 In the Properties and Formatting window, under Properties, click
Advanced.
6 Under Incremental Fetch, select the Enable incremental fetch in Grid
check box.
7 In the Maximum number of rows per page, define the fetch size. The
following image displays the Incremental Fetch setting:
8 Click OK to save the changes.
Enabling Incremental Fetch on Grouping and Text boxes
Incremental fetch divides large dashboards or layouts into pages, thereby
loading the data in batches rather than all at the same time. This feature
improves the usability and performance of a large dashboard or layout, by
reducing the load and overall memory usage on the Web server. If the
dashboard or layout is grouped, you can select any group as the level. If it is
not, then the block size is applied to the Detail section.
To enable incremental fetch on grouping and text boxes:
1 Edit the document.
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. Design Techniques to Optimize Performance 187
2 In the Document Editor, if the document contains multiple layouts, select
the layout to which you want to apply incremental fetch.
3 On the Tools menu, select Document Properties.
4 In the Document Properties window, under Layout Properties, select
Advanced.
5 Select the Enable incremental fetch check box.
6 In the Fetch Level drop-down list, select the grouping level to which you
want to enable incremental fetch.
7 In the Block Size box, type the number of objects to return.
8 Click OK.
Using Group By
Similar to the use of filtering selectors, using Group By in a DHTML dashboard
decreases the initial load size to only the slice of data selected on the Group By,
improving the dashboard performance.
This approach does not support the use of prompts and security
filters.
Use Flex programming best practices.
Use the AIR application as an alternative.
This approach limits the amounts of data that can be sent, but
requires the installation and maintenance of an additional
application.
Data Presentation Deploying MicroStrategy High Performance BI 5
194 Optimizing Performance for Mobile 2011 MicroStrategy, Inc.
Optimizing Performance for Mobile
After completing this topic, you will be able to:
Optimize your mobile device to experience performance gains when rendering
MicroStrategy dashboards.
Execution Workflow for Mobile Devices
The performance of a Mobile dashboard is dependent on several levels of
execution, as displayed in the diagram below:
Mobile execution Flow
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. Optimizing Performance for Mobile 195
If this check box is selected, caches are loaded for all subscribed reports
and documents when the application is launched.
To configure the project to cache documents without an expiration date:
1 In Desktop, right-click the desired project, and select Project
Configuration.
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. Optimizing Performance for Mobile 199
2 In the Caching: Result Caches: Maintenance subcategory, select the Never
expire caches check box.
3 Click OK.
If possible, run documents and dashboards against an Intelligent Cube to
increase data retrieval speed. Large warehouses can significantly slow down
data retrieval and have a large impact on overall performance. Warehouse data
should be split into multiple Intelligent Cubes for improved performance.
Combining and Removing Datasets
Because all datasets require time to execute and join together, remove any
datasets that are not used on your document or dashboard. Also, browse
through the objects included in each dataset to ensure that they are essential to
your document or dashboard. Combine datasets as much as possible to
minimize the total number.
Using Prompts and Selectors to Reduce the
Dashboard/Document Size
Implementing prompts can be a helpful method of reducing the amount of data
that is returned and reducing the footprint of your document or dashboard. To
control user input for a prompt, specify a minimum or maximum value, or set
limits on the number of possible answers. This enables you to create a more
predictable result set for each prompt.
To display only a subset of the available data and allow users to interact with
the result set after the document or dashboard has been displayed, use
selectors. Implementing selectors clears up screen space by minimizing the
amount of data that is displayed at one time. Use a slicing selector or filtering
selector based on the unique characteristics of each document.
Lesson Summary
In this lesson, you learned:
It is important to be familiar with the report execution flow to better
understand how some of the components affect the performance in a
MicroStrategy environment.
Several of the recommendations for optimizing the rendering of reports are
directly related with the strategies to minimize the size of the XML.
The simpler the reports are, the smaller the XML is and the faster the
overall response time is.
When designing reports, ensure that there is a strong need for the report to
be a Working Set report, otherwise save resources by making it a regular
report.
Search objects are not efficient in terms of report execution and should be
avoided.
Reduce the number of elements in custom groups to reduce the footprint
for report execution.
Keeping the number of joins and aggregations that either the database or
the Analytical Engine has to perform to a minimum has a positive impact
on report execution performance.
The incremental fetch setting in MicroStrategy Web and the Maximum
number of XML drill paths governor setting in Intelligence Serve can
optimize report results rendering in MicroStrategy Web.
The dashboard dataset execution and virtual dataset execution processes
can become bottlenecks for high performance dashboards. For this reason,
the techniques to optimize performance focus on these two processes that
are involved in the dashboard execution.
Reducing the number of datasets by eliminating unused datasets or by
consolidating them into fewer ones decreases the virtual dataset creation
time by the Analytical Engine.
Data Presentation Deploying MicroStrategy High Performance BI 5
204 Lesson Summary 2011 MicroStrategy, Inc.
Using a single instance of the dataset in the dashboard and making use of
multiple view filters on the template to extract different portions of the data
reduces the total number of dataset instances, and improves data
preparation time.
When you want to display the same dataset in multiple formats, add one
instance of the dataset and enable the Quick Switch functionality to avoid a
major impact in the dashboard performance.
Ensure that all datasets used in a dashboard only contain the amount of
data that will truly be used for analysis and display.
Use drilling and links to reduce dashboard data, while providing the option
to access other levels of data.
Dashboards with datasets containing similar attributes and metrics are
good candidates for using Intelligent Cubes to avoid query executions
against the data warehouse. Additionally, single-cube-as-dataset
dashboards perform 50% to 100% better than dashboards that hit multiple
cubes.
While DHTML tends to load and render dashboard elements such as
layouts and panels incrementally, Flash tends to load such elements
upfront. The dashboard designer needs to analyze all criteria to determine
which of the two technologies is more appropriate from a performance
standpoint.
To optimize the initial rendering of a dashboard, slice the data it displays
using filtering selectors.
A dashboard section implemented with text boxes can execute up to ten
times faster than the same section implemented with grids.
It is recommended to restrict the number of elements such as grids, graphs,
and visualizations in a dashboard. You can achieve this by consolidating
grids and graphs into a single visualization object.
Reducing grid sizes, limiting the number of layouts, disabling automatic
resize, limiting the use of thresholds and subtotals, and limiting the use of
smart metrics are all recommended to improve the dashboard
performance.
To improve performance on DHTML dashboards, the following
recommendations applyenabling on-demand fetching of panels; enabling
incremental fetch on grids, grouping, and text boxes; and using group-by.
A dashboard built with Flash has the following
componentsdashboardviewer.swf, data binary file, definition binary file,
and the Flash properties file.
Deploying MicroStrategy High Performance BI Data Presentation 5
2011 MicroStrategy, Inc. Lesson Summary 205
To improve performance on Flash dashboards, the following
recommendations apply: reducing the number of panels and panel stacks,
reducing formatting density, enabling Flash Caching on the client, reducing
use of Flash Widgets, and optimizing visualizations.
Mobile clients must compensate for slower Wi-Fi and 3G networks, and
memory constraints.
Tuning strategies can be implemented at each step of the execution process
to improve overall document or dashboard performance on the iPhone or
iPad.
To optimize your design and decrease execution time on your Mobile
device, the main techniques to follow are: implement a caching strategy,
combine or remove datasets, and use prompts and selectors to reduce the
amount of data.
Another method of improving the speed of data rendering is through the
use of Information Windows on iPad documents and dashboards.
The most common issues that contribute to performance lag for any
document or dashboard are overcrowding of screen space with superfluous
elements and data, redundant panels, and unnecessary selectors.
Certain limitations inherent in the iPhone and iPad may cause performance
issues in some of your documents or dashboards, such as the search as you
type feature for iPhone prompts and the binary size limit for iPad
dashboards.
Data Presentation Deploying MicroStrategy High Performance BI 5
206 Lesson Summary 2011 MicroStrategy, Inc.
2011 MicroStrategy, Inc. 207
6
DATA WAREHOUSE ACCESS
Lesson Description
This lesson provides solutions to query inefficiencies you may find when
running jobs in your MicroStrategy environment. In this lesson you will learn
about several query efficiency optimization techniques. First, you will be
introduced to query performance considerations. Then, this lesson provides
information about techniques to optimize performance in three distinct areas:
report design, SQL generation, and data architecting. Finally, you will learn
about query optimization techniques related to the Multi Source Option and
ODBC.
Data Warehouse Access Deploying MicroStrategy High Performance BI 6
208 Lesson Objectives 2011 MicroStrategy, Inc.
Lesson Objectives
After completing this lesson, you will be able to:
Apply the learned skills to optimize report queries to reduce the database
execution time.
After completing the topics in this lesson, you will be able to:
Understand the main considerations about query performance. (Page 209)
Understand the optimizations that can be done on the data warehouse side
to improve query performance. (Page 213)
Apply design techniques to optimize query performance when developing a
report. (Page 219)
Apply tuning techniques to optimize the SQL generated by MicroStrategy.
(Page 224)
Understand query optimization techniques related to the Multi Source
Option and ODBC. (Page 243)
Deploying MicroStrategy High Performance BI Data Warehouse Access 6
2011 MicroStrategy, Inc. Introduction to Data Warehouse Access 209
Introduction to Data Warehouse Access
After completing this topic, you will be able to:
Understand the main considerations about query performance.
Database Query Performance
When no caching or cube techniques are used, the performance of a typical
business intelligence query is mostly dominated by time in the database. In a
typical report request, the time spent on the database tends to average around
80% of the total report response time. Therefore, it is very important to
optimize the queries to reduce the time spent in the database.
Time Spent on the Database
To optimize query performance, there are five main actions that you can take:
Optimizing the number of SQL passes
Reducing full scans of large tables
Data Warehouse Access Deploying MicroStrategy High Performance BI 6
210 Introduction to Data Warehouse Access 2011 MicroStrategy, Inc.
Reducing the number of table joins
Selecting data from smallest tables
Leveraging database specific parameters
The image below shows a typical SQL query and highlights the five high-level
actions that can help to optimize the query performance:
High-Level Actions to Optimize Query Performance
SQL Generation Algorithm
The SQL Engine translates report definitions created by the report designer
into SQL Queries based on different parameters such as template and filter
definitions, schema object definitions (e.g. attributes, facts, tables, and so
forth) and VLDB settings.
Deploying MicroStrategy High Performance BI Data Warehouse Access 6
2011 MicroStrategy, Inc. Introduction to Data Warehouse Access 211
The image below summarizes the steps followed by the Engine to translate into
SQL a hypothetical report definition with Time dimension attributes - Year,
Quarter & Month, Product dimension - Category & Subcategory, and Metrics -
Units Sold and Units Received as well as a filter on Quarter:
SQL Generation Algorithm
1 The dimensionality (lowest level at which the report can be calculated) is
defined. In this example, the dimensionality is Month & Subcategory.
2 The SQL Engine merges the report filter with the metrics.
Data Warehouse Access Deploying MicroStrategy High Performance BI 6
212 Introduction to Data Warehouse Access 2011 MicroStrategy, Inc.
3 The SQL Engine categorizes metrics with same dimensionality and filtering
conditions into aggregate metric groups.
4 The SQL Engine then identifies the smallest common set of tables to source
metrics from. In this case, it finds ORDER_DETAIL for Units Sold and
INVENTORY for Units Received.
5 The SQL Engine builds the SELECT clause taking into account the
dimensionality of the metric group.
6 The FROM clause is built using source tables for metrics and tables
required to satisfy filtering criteria.
7 A join is constructed to apply the filter.
8 Finally, a group-by is applied, again using Dimensionality.
A thorough understanding of the SQL construction logic followed by the SQL
Engine is critical for designing systems that result in optimized queries. A
report developer or system architect can optimize a SQL query (and thus the
performance of the report supported by that query) by applying tuning
techniques in any of the following three phases of a data request:
Report and Schema DesignEnsuring that the objects in a report, and all
their supporting schema objects are setup to generate optimal SQL.
SQL GenerationMaking use of database specific parameters to influence
SQL generation.
Data ArchitectureMaking changes on the data warehouse side (that is,
outside of MicroStrategy) to improve database query performance.
While performance optimizations and tuning techniques exist at each one of
the layers and phases of the data request process, the impact of such
optimizations tends to be greater when they are done at the lowest level and
will tend to be more localized for higher layers. The order in which these
optimizations are done is important. When possible, they should ideally start
with lower layers such as data architecture and database parameters, move to
schema design, and finally report design.
Deploying MicroStrategy High Performance BI Data Warehouse Access 6
2011 MicroStrategy, Inc. Data Architecture Optimizations 213
Data Architecture Optimizations
After completing this topic, you will be able to:
Understand the optimizations that can be done on the data warehouse side to
improve query performance.
The discussions in this course manual so far have dealt with the data
warehouse access optimizations that are all controlled from within
MicroStrategy. This section covers changes and optimizations that can be done
on the data warehouse side to improve database query performance. While a
very large number of database optimization techniques exist and are quite well
documented, in this section, you will learn the five most relevant techniques
from a MicroStrategy perspective. These techniques are:
Denormalizing the physical warehouse schema
Building Indexes to Specific Data
Partitioning Fact Tables into Subset Tables
Using Views with Pre-calculated Data Aggregations
Building Separate Lookup Tables for Attributes
Denormalizing the Physical Warehouse Schema
A denormalized physical data warehouse schema provides better performance
when using MicroStrategy because it reduces the number of table joins needed
to retrieve relevant data. The image below illustrates this point. The left side
shows a highly normalized database that requires more table joins to resolve a
report. By contrast, the right side shows a re-designed database with some
degree of denormalization that results in a single join and an optimized
database query performance.
Data Warehouse Access Deploying MicroStrategy High Performance BI 6
214 Data Architecture Optimizations 2011 MicroStrategy, Inc.
Indexes have less impact when the number of distinct values in the
indexed column is low. When an index is not very selective, the
database optimizer may choose to not use it, performing a full table scan
instead, defeating the purpose of creating the index in the first place.
As a general rule, indexes should be defined on columns that have many
distinct values and tend to appear in filter conditions for the most used reports
or on those columns that make up the primary or foreign key in large fact
tables. Enterprise Manager can be a useful tool to identify the most used
columns in the WHERE clause of the report SQL.
Partitioning Fact Tables into Subset Tables
A partition is a division of a logical database or its elements into distinct
independent parts. Database partitioning is normally done for manageability,
performance, or availability reasons. The MicroStrategy SQL Engine is
partition-aware, that is, it can take advantage of a partitioned data warehouse
and select the smallest possible set of tables when sourcing reports.
Data Warehouse Access Deploying MicroStrategy High Performance BI 6
216 Data Architecture Optimizations 2011 MicroStrategy, Inc.
Partitioning is typically defined for the time dimension but can be used for
other dimensions as well, especially if attributes on that dimension are used
very frequently to filter data (by Country for example, where users in a
particular country will only look at that country's data).
In the example illustrated below, the ORDER_DETAIL fact table was
partitioned based on the month in which the orders were placed. For queries
that filter on Month, the SQL Engine will pick the correct month-level fact
table partition, resulting in smaller table scans and faster database query.
Partitioning Fact Tables
Using Views with Pre-calculated Data Aggregations
Unlike ordinary tables in a relational database, a view is not part of the physical
schema. Instead, it is a dynamic, virtual table computed from data in the
database. This means that any change in the underlying table data alters the
results shown in subsequent invocations of the view.
Views are useful from a performance perspective because they can pre-join
tables and simplify complex queries and act as aggregate tables. Additionally,
views can be a useful tool to isolate base tables from report queries.
Materialized Views in Oracle, Aggregate Join Indexes in Teradata, Materialized
Query Tables in IBM DB2 and Indexed Views in Microsoft SQL Server are all
features that were specifically implemented by database vendors to improve
query performance. The image below illustrates this concept.
Deploying MicroStrategy High Performance BI Data Warehouse Access 6
2011 MicroStrategy, Inc. Data Architecture Optimizations 217
The data warehouse contains two fact tables, one with 2 billion rows and
another one with 6 billion rows. One or more reports will require data from
both tables to be combined into a single query, which could potentially cause
full table scans on two very large fact tables. For this reason, the report
architects designed a view that returns 200 million rows of data and includes a
pre-aggregated column that is not available on either table:
AVG_CALL_TIME.
Creating Views to Improve Performance
For Oracle and Sybase, you also need to check the Enable
SQLDescribeParam check box at the DSN level, as shown below:
Parameterized insertion prepares the data to be inserted into many equal sized
packages. You can tune the size of each individual packet by setting the
MaximumRowsetSize parameter in the ODBCConfig.ini file, which determines
how many KB each individual package carries. The default value is 32 KB.
Deploying MicroStrategy High Performance BI Data Warehouse Access 6
2011 MicroStrategy, Inc. SQL Generation Optimizations 231
Varying the values for this setting can affect insertion performance as seen in
the graph below. The best value should be determined on a case by case basis
by testing directly on the specific database environment.
Insertion Times vs. Rowset Size
Lesson Summary
In this lesson, you learned:
In a typical report request, the time spent on the database tends to average
around 80% of the total report response time.
You can optimize a SQL query by applying techniques to the following
phases of data request: Report/Schema Design, SQL Generation, and
Architecture.
The order in which optimizations are done should ideally start with lower
layers, such as data architecture and database parameters, and
subsequently move to schema design, and report design.
Because it reduces the number of table joins needed to retrieve relevant
data, denormalized data warehouse schemas provide better performance
when using MicroStrategy.
Indexing is one of the most used data warehouse techniques to improve
query performance, because they eliminate the highly expensive process of
full table scans.
The MicroStrategy SQL Engine is partition-aware and can take advantage
of a partitioned data warehouse and select the smallest possible set of tables
when sourcing reports.
Views are useful from a performance perspective because they can pre-join
tables, simplify complex queries, and act as aggregate tables.
Building separate lookup tables for attributes which would otherwise be
sourced from a fact table eliminates potentially very large table scans.
By carefully designing the project schema and by taking into account
important optimization considerations, you can tune the SQL Engine to
generate optimal queries.
To optimize query performance, unnecessary joins can be eliminated by
using one of the following techniques: redefining parent child attribute
relationships, mapping attributes to specific tables, and deleting
unnecessary attributes.
Deploying MicroStrategy High Performance BI Data Warehouse Access 6
2011 MicroStrategy, Inc. Lesson Summary 251
Sometimes, a SQL query can be optimized by pushing filtering conditions
into fact expressions using CASE statements instead of defining them at the
metric level.
From a query optimization perspective, substituting Custom Groups with
Consolidations can be beneficial.
For cases when two dimensional metrics at different levels of the same
hierarchy are requested, defining the higher level metric using the lower
level metric as base will result in better database query performance.
The SQL generation process can be divided into three functional layers,
each of which can be optimized for query performance: Logical Query
Layer, Database Optimization Layer, and Query Optimization Layer.
As a general rule it is recommended to consolidate columns and column
expressions with the same functional meaning into a single Fact definition.
Reducing the number of fact tables needed to complete a data request is
another technique that will help optimize database query performance.
Aggregate tables can be a very useful performance improvement technique.
VLDB properties enable report designers and architects to alter the syntax
of a SQL statement and to take advantage of database-specific
optimizations.
Parameterized statements, when supported by the database, prepares the
data to be inserted into the database by splitting it into many equal sized
packages.
The SQL Hint VLDB property causes the SQL Engine to provide SQL hints
that guide the database optimizer to generate the most optimal query plan.
The SubQuery Type VLDB property enables report designers to select
between different types of subqueries. The most optimal option will
generally depend on each database platform's capabilities.
Some databases evaluate set operators more efficiently than the equivalent
logical operators in SQL.
It is important from a database performance perspective to ensure that the
best possible intermediate table type and the fastest report data population
method for a given database platform is selected.
The Transformation Formula Optimization VLDB property enables report
designers to improve the performance of expression-based
transformations.
Data Warehouse Access Deploying MicroStrategy High Performance BI 6
252 Lesson Summary 2011 MicroStrategy, Inc.
The main function of the query optimization layer is to optimize the report
query SQL by eliminating redundant SQL passes.
The amount of data required to resolve a query tends to be the major factor
influencing Multi Source performance.
When the number of result sets is large, SQLExtendedFetch may optimize
performance making faster retrieval of query results easier.
Based on internal testing, the recommendation is to set database drivers to
multi-process mode.
Setting the array size close to the size of the data to be transferred can
improve performance by 20%, at the expense of increasing memory usage.
2011 MicroStrategy, Inc. 253
7
PERFORMANCE TESTING
METHODOLOGY
Lesson Description
This lesson talks about performance testing in a BI environment. Questions
about the performance of the system need to be answered with various
performance testing. Unlike straightforward feature testing that verifies the
functional correctness of the product, performance testing needs special design
and is more complex to execute and analyze.
At the end of this session, you will be intimately familiar with the methods and
techniques we use internally to test, troubleshoot, and optimize
performance-related challenges. You will be able to apply these same methods
and techniques to improve performance within your own BI deployment.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
254 Lesson Objectives 2011 MicroStrategy, Inc.
Lesson Objectives
After completing this lesson, you will be able to:
Understand performance testing, so you can choose the right type of tests for
specific performance requirements. Design, implement, execute, and analyze
performance tests in a correct way.
After completing the topics in this lesson, you will be able to:
Understand the main considerations for performance testing. (Page 255)
Understand the performance testing methodology and apply it to your
environment needs. (Page 260)
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Introduction to Performance Testing 255
Introduction to Performance Testing
After completing this topic, you will be able to:
Understand the main considerations for performance testing.
In software engineering, performance testing is testing that is performed, from
one perspective, to determine how fast some aspect of a system performs under
a particular workload. It can also serve to validate and verify other quality
attributes of the system, such as scalability, reliability, and resource usage.
The Performance Testing Methodology along with much of the content in this
course, is part of the outputs of the High Performance Initiative. Before
describing the Performance Testing methodology, it is important to answer a
few questions:
Why is Performance Testing important?
What is Performance in a Business Intelligence Environment?
Why is having a Performance Testing Methodology important?
Why is Performance Testing Important?
Software performance is critical, especially for enterprise software
applications. Often times, it is the key reason for the success or failure of a
purchasing deal. A BI application that runs very slow or crashes from time to
time will never sell, no matter how rich its functionality are. Because of this,
the MicroStrategy BI product must pass strict performance testing before it is
released.
The MicroStrategy Enterprise Analysis QE team runs performance testing
in-house to certify that the product has good performance before it acquires
Generally Available (GA) status. The passing criteria is based on whether the
system successfully survived 24 hours of concurrency stress testing. Due to the
complexity of the BI system, even if the product passed performance criteria
during the in-house testing, it might present poor performance in a customer
environment.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
256 Introduction to Performance Testing 2011 MicroStrategy, Inc.
Many factors can affect the system performance: database server, Web
application server, network environment, system architecture, user usage
patterns, business logic, degree of code optimization, and so forth. Hence, it is
often difficult to achieve the performance goals in the customer environment
without any adjustments.
For engineers in the MicroStrategy technology department to be able to resolve
the issues either through tuning or code optimizations, it is crucial that you
understand your system performance goals, design the right tests to verify
system performance, and help identify the performance bottlenecks.
Depending on the characteristics of a BI implementation, different types of
performance testing might be needed to cover different areas. For example:
Benchmark TestingUsed to ensure the project meets SLAs
Load TestingUsed to ensure the system has enough spare capacity to
meet projected usage
Regression TestingUsed to ensure that everything is still performing
well after changes or upgrades have been done to the system
What is Performance in a BI Environment?
To be able to test and do something about performance, you must be able to
measure it. In other words, product performance must be specified in a
quantitative way, as a set of numeric values that describe how fast and how
efficiently a software system can complete specific computing tasks.
For example, instead of saying the Web application must respond to report
execution request quickly, a performance statement will say the Web
application must return results to the report execution request in 2 seconds.
For this reason, performance needs to be measured before you can do any
quantitative analysis. To support performance analysis to answer a specific
performance question, performance metrics need to be collected. The most
used performance metrics are described below.
Response Time
From the users perspective, software performance is translated to response
time. Response time is the time needed for the system to respond to a user
request. It is end-to-end time, from the instant the user submits a request to
the instant the user gets the results back.
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Introduction to Performance Testing 257
Response time consists of two parts:
System response timeTime from the instant the client receives the user
request to the instant the client receives the results data from the server.
Rendering timeTime the client spends to render the result. For instance,
when executing a Report Service document in MicroStrategy Web, after the
client (browser) gets all the results, the system response time is over.
However, it still needs to present the data by interpreting and executing
Java scripts, which corresponds to rendering time.
Whether a specific response time is acceptable or not depends on the request
type submitted by a user. For example, for report execution, 2 seconds of
response time is very attractive, while more than 10 seconds denotes poor
performance. Nevertheless, for a large exporting request that happens once a
day, 30 minutes may be acceptable.
Job Throughput
Job throughput refers to the average number of user requests that are
processed by the system in a unit of time. It directly shows the system
performance capacity. In general, throughput is measured by requests/second
or pages/second. Throughput is a key indicator in capacity planning projects,
because it describes the system performance capacity. It is also commonly used
in performance tuning. According to in-house testing data, 80% of the
performance issues are due to throughput bottlenecks.
CPU Usage
Any process running in an Operating System uses CPU time. For instance,
when Intelligence Server is idle, the server process uses zero CPU time. When
it receives the request to execute a report, the server process becomes busy,
until the report results return to the user. With proper tools, CPU time can be
calculated for any running process. For a specific user request, the smaller the
CPU usage, the better is the performance.
Memory Usage
Any application running in an Operating System uses system memory. The
memory usage of an application should reach stable state after running for a
long time.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
258 Introduction to Performance Testing 2011 MicroStrategy, Inc.
When the application process starts to handle a user request, it usually causes a
jump in memory. This is because the process needs additional memory to do
calculations and to hold temporary data. After the request is processed, the
additional memory is released and the memory consumption returns to its
original level. If the application keeps on running, the memory usage continues
to increase until it reaches a limit, which can cause the application to crash.
There are instances when the application consumes a greater than expected
amount of memory from the system. This usually indicates that there is
inefficient memory allocation in the program algorithms.
Data Size
Data size usually refers to output file size, such as the generated file of report
exporting.
Why Is Having a Methodology Important?
As the figure below illustrates, a MicroStrategy implementation is made up of
multiple layers, each one with its own characteristics and complexities that
may challenge even well-experienced performance testers.
MicroStrategy Platform
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Introduction to Performance Testing 259
Performance testing results are usually non-deterministic. The same test, when
executed in different environments, will probably produce different results.
Even in a consistent environment, running the same test twice can also
generate different results. This is because there are still some environment
factors, such as network traffic, that may vary and cannot be controlled.
User concurrency may also represent a challenge. It is more complex to do
reliable measurements of user requests, such as browsing folders or executing
reports, when there are many users logging in and performing other actions at
the same time. To come up with reliable results, it is crucial that you have a
methodology in place.
Without a proper methodology to address the inherent complexities of any
respectable BI deployment, most performance testing initiatives will at best
yield inconclusive results leading to ineffective solutions. At worst, they fail
and produce incorrect results, which is not a good position to be in when you
are under pressure to meet stringent SLAs or a tight delivery deadline.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
260 Performance Testing Methodology 2011 MicroStrategy, Inc.
Performance Testing Methodology
After completing this topic, you will be able to:
Understand the performance testing methodology and apply it to your
environment needs.
The key to make your performance testing fulfill your business and technical
goals is the adoption of a formal Performance Testing Methodology. The
MicroStrategy Performance Testing Methodology is used by MicroStrategy
Technology, Support, and Services teams to test, troubleshoot, and optimize
any individual performance challenge. It consists of five simple, but essential
steps that are illustrated below:
MicroStrategy Performance Testing Methodology
DefineGo beyond generalities such as performance is slow/bad and
define with precision the action that needs performance testing and what
your performance goals are for it.
QuantifyTake an initial high-level measurement of the action. Start with
single user testing, if single user performance is acceptable, then move to
concurrency testing.
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Performance Testing Methodology 261
ProfileGenerate a detailed profile of the action, understanding how each
component of the architecture contributes to the end-to-end response
times.
OptimizeOptimize each one of the layers, focusing on the biggest
contributors to overall response time first. Continue this process until you
reach acceptable performance levels as defined in Step 1.
MonitorEstablish a performance monitoring plan to ensure optimum
levels of performance in the system are moving forward.
It is very important to make sure that the actions, tests, decision points,
and changes made to the system are properly documented all along the
way. This helps in several ways:
It helps keep a record of the changes that were made to the system in
case a rollback becomes necessary
It enables different members of the organization to participate and
follow-up on the project at different times
It ensures that the performance testing process also becomes a learning
process for the organization
The following sections describe in detail each step of the MicroStrategy
performance testing methodology.
Define System Goals
The first step to accomplish successful results in the performance testing
process is to define the goals and expectations for the testing. Performance
goals are usually defined by the business side or the project stakeholders.
Having the right goals established will help you decide the right type of test to
run and the right tools to perform the test.
However, to accomplish successful performance testing and troubleshooting, it
is important to go beyond general terminology such as the system is slow or
performance is bad and define, with a high degree of precision, the exact
action that requires performance testing and the acceptable performance
target for it. A vague definition for a performance problem often leads to the
wrong type of performance testing. As a consequence, people's time is wasted
and incorrect results are generated.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
262 Performance Testing Methodology 2011 MicroStrategy, Inc.
A performance goal is usually described by the following parameters:
Response timeThere may be different response time goals for different
user profiles or different types of interactive requests.
ThroughputMaximum throughput for the system while still maintaining
desired response time
Concurrency or User LoadNumber of interactive users supported by
the system during the peak hour while maintaining desired response time
This detailed information can determine how best to optimize the action
to meet the performance targets.
The execution flow of a typical MicroStrategy dashboard, illustrated below, can
exemplify this technique:
Dashboard Execution Flow
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Performance Testing Methodology 267
Between the database and the dashboard, there is an underlying execution
workflow that has a direct impact on dashboard size and performance. For
information about the detailed steps involved with a dashboard execution, see
The Dashboard Execution Flow starting on page 159.
The image below depicts what the Performance Stack Diagram looks like for a
typical MicroStrategy Dashboard that executes against the Data Warehouse:
Performance Stack Diagram of a MicroStrategy Dashboard
The graph bar is now broken down into the following sections:
SQL Generation within Intelligence Server
Query Engine
Analytical Engine, Data Preparation Engine
XML Generation Engine
Web server processing time
Network Time
Client Time
This detailed Performance Stack Diagram is an important tool to help you
make decisions on where and what to optimize to improve performance. All of
the parameters and component execution times in this type of Performance
Stack Diagram can be obtained using the different out-of-the-box performance
counters and tools that ship with the MicroStrategy platform.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
268 Performance Testing Methodology 2011 MicroStrategy, Inc.
The tables below list the tools and the counters to measure for each component
of the MicroStrategy architecture.
The key to acquire useful results with these tools is ensuring that the
measurements are taken under Single User mode. In this mode, the systems
resources are 100% dedicated to completing the action you are profiling.
Performance Counters - Time
Performance Job flow information for MicroStrategy Dashboard Executions - Time
Component Counter Measure Time
Intelligence
Server
Diagnostics Configuration Document Execute Task
Find Report Cache
Resolution Step
SQL Generation
Query Engine
Analytical Engine
Data Preparation
XML Generation
Web Server Web Statistics Web Server Processing Time
Client Web Statistics Resource Loading
Java Script
Wait Time
Network - Any remaining difference
Stop Watch End to End time
Performance Counters - Size
Performance Job Flow Information for MicroStrategy Dashboard Execution - Size
Component Counter Measure Time
Size Document Cache Monitor XML
Web Statistics Data sent from Intelligence Server to
Web (bytes received)
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Performance Testing Methodology 269
Concurrency Performance Profiling
When the performance test in the Quantify stage is able to meet the target, the
next step is to profile the action in Concurrency mode. The Concurrency
Performance Profiling technique uses the Performance Degradation Curve.
The image below shows a Performance Degradation Curve with two measures
plotted against the same variable, Submission Rate:
System ThroughputRepresents the amount of reports that completed
successfully in one minute
Average Response TimeRepresents the time it took the reports to
complete against submission rate.
Performance Degradation Curve
The methodology diagram has a loop back arrow from the Optimize to
the Profile step to denote the importance of reprofiling after optimizing.
In the previous example, the Performance Stack Diagram showed that the test
for a Single User did not meet the performance target. Therefore, the goal is to
essentially shave off enough time from the overall execution to get rid of the
portion of time that exceeds the performance target. The true value of a
detailed performance profile can be demonstrated through this example,
because it allows you to focus your optimization efforts in those areas that will
truly make an impact.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
276 Performance Testing Methodology 2011 MicroStrategy, Inc.
For example, if you manage to reduce the execution time of the final
component by 50%, doubling its performance, you still do not meet the defined
performance target. The image below depicts this scenario:
Results when Optimizing the Final Component
However, If you focus on reducing the response time of the first component by
50%, you meet the performance target, as shown below:
Results when Optimizing the First Component
This targeted component-by-component optimization is not possible if you do
not spent the time establishing a full performance profile for the action in the
first place.
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Performance Testing Methodology 277
This course, from chapters 1 to 6, provided the knowledge and skills to
optimize the different components of a MicroStrategy BI architecture, after you
generate the full profile for actions on your environment.
Finally, after you profile, optimize, reprofile, and continue this cycle until the
tested action performs at acceptable levels, the next step is establishing a
monitoring plan to ensure that performance remains acceptable as the system
evolves down the road.
Monitor the Environment
The frequency and resources used for monitoring the environment will largely
depend on the characteristics of the BI environment and the action that you
will be monitoring. The key is to ensure that when changes are introduced into
the system, they do not degrade the performance of the actions that are
important to your business users.
The MicroStrategy platform contains tools such as Enterprise Manager, Health
Center, Command Manager, and Integrity Manager that enables you to
monitor your systems performance. In addition to these tools and the different
performance logging mechanisms available within the MicroStrategy platform,
you can also use commercially available software, such as Silk Performer or
Load Testing.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
278 Lesson Summary 2011 MicroStrategy, Inc.
Lesson Summary
In this lesson, you learned:
Performance testing can serve different purposes: it can demonstrate that
the system meets performance criteria, it can compare two systems to find
which one performs better, or it can measure what parts of the system or
workload cause the system to perform badly.
Product performance must be specified in a quantitative way, as a set of
numeric values that describe how fast and how efficiently a software system
can complete specific computing tasks.
The MicroStrategy Performance Testing Methodology is used by
MicroStrategy Technology, Support, and Services teams to test,
troubleshoot, and optimize any individual performance challenge.
It is very important to understand the different performance testing
available and what they accomplish to get the right results during your
testing.
To support performance analysis to answer a specific performance
question, performance metrics need to be collected.
The most used performance metrics are: response time, job throughput,
CPU usage, memory usage, and data size.
The key to make your performance testing fulfill your business and
technical goals is the adoption of a formal performance testing
methodology.
The main steps in the performance testing methodology are: define system
goals, quantify performance, profile the action, optimize the action, and
monitor the environment.
Define the action to be tested and its performance target with precision
because vague (or inexistent) performance definition statements lead to
time waste and incorrect solutions.
Take an initial single-user, high-level measurement. Single-user response
time represents the best possible response time because all of the systems
resources are dedicated to completing that action.
Deploying MicroStrategy High Performance BI Performance Testing Methodology 7
2011 MicroStrategy, Inc. Lesson Summary 279
Depending on the results of the single-user measure you need to do either
Single-User or Concurrency Performance Profiling.
For Single User Performance Profiling, the technique consists of generating
the Performance Stack Diagram, which shows how each component of the
architecture contributes to end-to-end response time.
For Concurrency Profiling, the technique consists of generating the
Performance Degradation Curve, which shows values obtained for User
Experience (Average Response Time) and System Throughput plotted
against increasing Submission Rates.
After a detailed Performance Stack Diagram is generated, methodically
optimize each one of the layers, focusing on the biggest contributors to
response time first.
Only one optimization should be introduced at a time in order to isolate the
effect of the change and determine its effectiveness.
It is very important to reprofile after each optimization and continue the
cycle until your performance targets are reached.
Performance Testing Methodology Deploying MicroStrategy High Performance BI 7
280 Lesson Summary 2011 MicroStrategy, Inc.
2011 MicroStrategy, Inc. 281
WORKSHOP
The object of this workshop is to enable you to experience the performance
gains of redesigning a MicroStrategy dashboard, by following the methodology
presented in this course.
Designing a High Performance Dashboard
ACME Corporation has been using MicroStrategy for several years, delivering
critical business data to key personal using well elaborated dynamic
dashboards. After the release of MicroStrategy 9.2.1, the IT managers at ACME
Corporations quickly decided to upgrade their MicroStrategy environment to
take advantage of the new code and the high performance features. One of the
main goals of the IT team is to reduce the execution time of a high profile
dashboard, which is frequently accessed by the companys stockholders, to less
than 10 seconds. The name of this dashboard is Pre-HP Analysis.
The Pre-HP Analysis dashboard currently takes more than 25 seconds to
execute. As the main dashboard developer at ACME Corporation, you have
been assigned the task to redesign the Pre-HP Analysis dashboard according to
the High Performance recommendations provided by MicroStrategy. The
stockholders expectation is to access the dashboard data in less than 8 seconds.
Workshop Deploying MicroStrategy High Performance BI
282 2011 MicroStrategy, Inc.
Dashboard Overview
The Pre-HP Analysis dashboard is an interactive interface that enables users to
analyze the main business metrics across time. It provides details on how the
business is doing based on geography and income level. The current version of
the dashboard contains 10 datasets, which are displayed across two panel
stacks, each providing different levels of analysis. These datasets are all View
Reports that are fed by a single cube (Cube 1).
This dashboard also contains several selectors that enable users to select the
data they want to see. Some of the selectors enable users to switch between
panels in a panel stack, while others enable users to display different metrics or
elements of an attribute, in grids and graphs.
The Pre-HP Analysis is located at: \MicroStrategy Tutorial\Public
Objects\Reports\High Performance\Workshop folder.
The Design Strategy
Based on the MicroStrategy methodology to achieve high performing
dashboards, the main steps you decide to take to redesign the dashboard are
the following:
Replace all the datasets with the Cube as datasetThis action will
eliminate the overhead caused by Intelligence Server extracting the data
from the cube and putting at the level of the dataset reports. It also
eliminates the need of Intelligence Server to create the virtual dataset and
to process the data to populate the different dashboard components.
Make all selectors Selectors as filtersThis action will significantly
reduce the initial load time of the dashboard because Selectors as filters
initially retrieve only one slice of data, instead of the full set.
High-level Steps
The high-level steps to redesign the Pre-HP Analysis dashboard to make it
execute in less than 10 seconds are listed below:
In Desktop, connect to the three-tier project source and browse to the
folder where the Pre-HP Analysis dashboard is located.
Deploying MicroStrategy High Performance BI Workshop
2011 MicroStrategy, Inc. 283
Make a copy of the Pre-HP Analysis dashboard and work on the copy
version (rename it High Performing).
In Web, run the High Performing dashboard before making any changes.
Time the execution in Flash.
Write down the time it takes for the dashboard to execute. At the end
of the workshop, you will compare this time with the time it will take
to execute the redesigned dashboard.
Remove all the datasets (all the view reports) from the dashboard.
Add Cube 1 as the only dataset.
Re-configure all the selectors:
After removing all the datasets, the selectors lose their source and
target. You need to reconfigure them.
Make all the appropriate selectors Selector as filter
Re-configure the Grid/Graphs
Map each of the cube's attributes and metrics to the appropriate grid or
graph.
Re-configure all the view filters for all grids and graphs.
After you delete the datasets, the view filters are also deleted.
Run the High Performing dashboard, timing its execution in Flash.
You can use the CTRL key to multi-select the datasets. Then
right-click the selection and select Delete from Document. In the
MicroStrategy Desktop pop up window, click Yes.
Deploying MicroStrategy High Performance BI Workshop
2011 MicroStrategy, Inc. 285
3 In the Datasets pane, click the Add Dataset link.
4 In the Select a report window, double-click the High Performance folder,
followed by the Workshop folder, and finally double-click the Cube and
VR folder.
5 Select Cube 1.
6 Click Open.
When you drag an object to the Graph Zone, a yellow bar displays
when you can drop it in the appropriate zone.
14 In the Dataset Objects pane, drag the Revenue Forecast metric and drop it
in the Graph Zone window, under METRICS, under the Revenue metric.
Deploying MicroStrategy High Performance BI Workshop
2011 MicroStrategy, Inc. 291
15 In the Graph Zone window, under SERIES, drag Metrics and drop it under
CATEGORIES.
Your Graph Zones window should look like the image below:
16 Close the Graph Zones window by clicking x.
17 On the Tools menu, select Document Structure.
18 In the Document Structure pane, under Month, select Graph111.
A window with the graph zones displays. You may need to resize the
Document Structure pane or use the scroll bars to allow the Graph Zone
window to fully display.
Workshop Deploying MicroStrategy High Performance BI
300 2011 MicroStrategy, Inc.
13 On the Tools menu, select Dataset Objects to display the attributes and
metrics to drag and drop in the appropriate graph zones.
14 In the Dataset Objects pane, drag the Customer State attribute and drop it
in the Graph Zone window, under SERIES.
15 In the Dataset Objects pane, drag the Order Count metric and drop it in the
Graph Zone window, under METRICS.
Your Graph Zones window should look like the image below:
16 Close the Graph Zones window by clicking x.
17 On the Tools menu, select Document Structure.
18 In the Document Structure pane, under Month, right-click Month
Selector, and select Properties and Formatting.
19 In the Properties and Formatting window, under Properties, select
Selector.
20 In the Source drop-down list, select Month.
21 Make sure that Graph20 displays under the Selected box.
22 Select the Apply selections as a filter check box.
23 Click Apply and click OK.
24 In the dashboard Toolbar, click Save As.
25 In the Save As window, click OK.
Deploying MicroStrategy High Performance BI Workshop
2011 MicroStrategy, Inc. 301
26 In the Confirm Overwrite window, click Yes.
27 In the Document Saved window, click Return to Design Mode.
28 In the Document Structure pane, under PanelStack3, expand Quarter.
29 Select Quarter.
30 Under Quarter, right-click CS Selector, and select Properties and
Formatting.
31 In the Properties and Formatting window, under Properties, select
Selector.
32 In the Source drop-down list, select Customer State.
33 Make sure that Graph151 displays under the Selected box.
34 Select the Apply selections as a filter check box.
35 Make sure the Show option for All check box is selected.
36 Click Apply and click OK.
37 In the Document Structure pane, under Quarter, select Graph151.
38 In the toolbar that displays by Graph151, click the Graph Zones icon.
A window with the graph zones displays. You may need to resize the
Document Structure pane or use the scroll bars to allow the Graph Zone
window to fully display.
39 On the Tools menu, select Dataset Objects to display the attributes and
metrics to drag and drop in the appropriate graph zones.
40 In the Dataset Objects pane, drag the Customer State attribute and drop it
in the Graph Zone window, under SERIES.
41 In the Dataset Objects pane, drag the Order Count metric and drop it in the
Graph Zone window, under METRICS.
Workshop Deploying MicroStrategy High Performance BI
302 2011 MicroStrategy, Inc.
42 In the Dataset Objects pane, drag the Month attribute and drop it in the
Graph Zone window, under CATEGORIES.
Your Graph Zones window should look like the image below:
43 Close the Graph Zones window by clicking x.
44 On the Tools menu, select Document Structure.
45 In the Document Structure pane, under Quarter, right-click Quarter
Selector, and select Properties and Formatting.
46 In the Properties and Formatting window, under Properties, select
Selector.
47 In the Source drop-down list, select Quarter.
48 Make sure that Graph151 displays under the Selected box.
49 Select the Apply selections as a filter check box.
50 Click Apply and click OK.
51 In the Document Structure pane, under Quarter, right-click Bar Selector,
and select Properties and Formatting.
52 In the Properties and Formatting window, under Properties, select
Selector.
53 In the Source drop-down list, select Month.
54 Make sure that Graph151 displays under the Selected box.
55 Select the Apply selections as a filter check box.
Deploying MicroStrategy High Performance BI Workshop
2011 MicroStrategy, Inc. 303
56 Make sure the Show option for All check box is selected.
57 Click Apply and click OK.
58 In the dashboard Toolbar, click Save As.
59 In the Save As window, click OK.
60 In the Confirm Overwrite window, click Yes.
61 In the Document Saved window, click Return to Design Mode.
62 In the Document Structure pane, under PanelStack3, expand Year.
63 Right-click Year Selector and select Properties and Formatting.
64 In the Properties and Formatting window, under Properties, select
Selector.
65 In the Source drop-down list, select Year.
66 Make sure that Graph163 displays under the Selected box.
67 Select the Apply selections as a filter check box.
68 Click Apply and click OK.
69 In the Document Structure pane, under Year, right-click CS Selector and
select Properties and Formatting.
70 In the Properties and Formatting window, under Properties, select
Selector.
71 In the Source drop-down list, select Customer State.
72 Make sure that Graph163 displays under the Selected box.
73 Select the Apply selections as a filter check box.
74 Make sure the Show option for All check box is selected.
75 Click Apply and click OK.
Workshop Deploying MicroStrategy High Performance BI
304 2011 MicroStrategy, Inc.
76 In the Document Structure pane, under Year, select Graph163.
77 In the toolbar that displays by Graph163, click the Graph Zones icon.
A window with the graph zones displays. You may need to resize the
Document Structure pane or use the scroll bars to allow the Graph Zone
window to fully display.
78 On the Tools menu, select Dataset Objects to display the attributes and
metrics to drag and drop in the appropriate graph zones.
79 In the Dataset Objects pane, drag the Customer State attribute and drop it
in the Graph Zone window, under SERIES.
80 In the Dataset Objects pane, drag the Order Count metric and drop it in the
Graph Zone window, under METRICS.
81 In the Dataset Objects pane, drag the Quarter attribute and drop it in the
Graph Zone window, under CATEGORIES.
Your Graph Zones window should look like the image below:
82 Close the Graph Zones window by clicking x.
83 On the Tools menu, select Document Structure.
84 In the Document Structure pane, under Year, right-click Bar Selector, and
select Properties and Formatting.
85 In the Properties and Formatting window, under Properties, select
Selector.
Deploying MicroStrategy High Performance BI Workshop
2011 MicroStrategy, Inc. 305
86 In the Source drop-down list, select Quarter.
87 Make sure that Graph163 displays under the Selected box.
88 Select the Apply selections as a filter check box.
89 Make sure the Show option for All check box is selected.
90 Click Apply and click OK.
91 In the dashboard Toolbar, click Save As.
92 In the Save As window, click OK.
93 In the Confirm Overwrite window, click Yes.
94 In the Document Saved window, click Return to Design Mode.
Phase 5: Redesign the Dashboard, Reconfiguring View Filters
Reconfiguring view filters in GridGraph objects
1 In the Document Structure pane, under PanelStack1, expand Month.
2 Right-click Graph3 and select Edit View Filter.
3 In the View Filter window, click Add Condition.
4 In the Filter On drop down list, select Category.
5 Click Select.
6 In the Available box, select Books and Electronics.
7 Click Add to selections.
Workshop Deploying MicroStrategy High Performance BI
306 2011 MicroStrategy, Inc.
You can find Graph111 under Month, Graph95 and Graph106 under
Quarter, and Graph97 and Graph2 under Year.
24 In the dashboard Toolbar, click Save As.
25 In the Save As window, click OK.
Workshop Deploying MicroStrategy High Performance BI
308 2011 MicroStrategy, Inc.
26 In the Confirm Overwrite window, click Yes.
27 In the Document Saved window, click Return to Design Mode.
28 In the Document Structure pane, under PanelStack1, expand Month.
29 Right-click Grid119 and select Edit View Filter.
30 In the View Filter window, click Add Condition.
31 In the Filter On drop down list, select Category.
32 Click Select.
33 In the Available box, select Books and Electronics.
34 Click Add to selections.