SSI Web Documentation PDF

SSI Web v8.
Software for Web Interviewing

and Conjoint Analysis
(Updated 2 June 2015)
Sawtooth Software, Inc.

Orem, Utah
http://www.sawtoothsoftware.com
In this manual, we refer to product names that are trademarked. Windows, Windows 95, Windows
98, Windows 2000, Windows XP, Windows NT, Windows Vista, Windows 7, Excel, PowerPoint, and
Word are either registered trademarks or trademarks of Microsoft Corporation in the United States
and/or other countries.
SPSS is a trademark of SPSS, Inc.
WS_FTP is a trademark of Ipswitch.
We express our gratitude to the Apache Software Foundation for the development of Apache HTTP
and mod_perl.
We express our gratitude to the Perl community.
Portions of lite.pm are copyrighted and used with permission by Shishir Gundavaram.
Other acknowledgements regarding copyrighted work are in the Readme.rtf file.
Bryan Orme, Editor

© Copyright 1999-2014 Sawtooth Software
About Technical Support
We’ve designed this manual to teach you how to use our software and to serve as a reference to answer
your questions. If you still have questions after consulting the manual, we offer telephone support.
When you call us, please be at your computer and have at hand any instructions or files associated with
your problem, or a description of the sequence of keystrokes or events that led to your problem. This way,
we can attempt to duplicate your problem and quickly arrive at a solution.
For customer support, contact our Orem, Utah office at 801/477-4700, email:
support@sawtoothsoftware.com.
Outside of the U.S., contact your Sawtooth Software representative for support.
Table of Contents
General Web Interviewing (CiW) Help
Getting Started...................................................................................................................................................... 1
Getting Started with Web Interviewing
...................................................................................................................................................... 1
Getting Started: 45-Minute Tour of SSI Web
...................................................................................................................................................... 4
Software Requirements: Users, Servers and Respondents
...................................................................................................................................................... 28
Creating/Opening a Study
...................................................................................................................................................... 29
What's New in......................................................................................................................................................
v8.4.0? 30
Composing Web......................................................................................................................................................
Surveys 39
Composing ......................................................................................................................................................
Questions 42
Adding
...................................................................................................................................................................
Questions to Your Survey 42
Setting
...................................................................................................................................................................
Page Breaks 45
The...................................................................................................................................................................
Start Question 46
Select
...................................................................................................................................................................
Questions 48
Numeric
...................................................................................................................................................................
Questions 55
Open-End
...................................................................................................................................................................
Questions 58
Text/HTML
...................................................................................................................................................................
Filler Questions 61
Grid
...................................................................................................................................................................
Questions 62
Constant
...................................................................................................................................................................
Sum Questions 65
Ranking
...................................................................................................................................................................
Questions 68
Semantic
...................................................................................................................................................................
Differential 72
Terminate/Link
...................................................................................................................................................................
Questions 75
Free
...................................................................................................................................................................
Format Questions 78
Introduction ............................................................................................................................................ 78
Radio ............................................................................................................................................ 80
Checkbox ............................................................................................................................................ 81
Select (Combo............................................................................................................................................
Box) 82
TextArea ............................................................................................................................................ 83
Text (Text Box) ............................................................................................................................................ 84
Text (Numeric) ............................................................................................................................................ 85
Hidden Variables ............................................................................................................................................ 86
JavaScript
...................................................................................................................................................................
in SSI Web 87
How CiW License Sizes are Counted
...................................................................................................................................................... 98
What Happens at the End of the Survey?
...................................................................................................................................................... 100
Formatting......................................................................................................................................................
Options 101
Survey
...................................................................................................................................................................
Settings (Style) 101
General Format ............................................................................................................................................ 109
Page Format ............................................................................................................................................ 111
Headers and ............................................................................................................................................
Footers 112
Progress Bar............................................................................................................................................ 113
Browser Settings ............................................................................................................................................ 115
Error M essages ............................................................................................................................................ 116
Advanced ............................................................................................................................................ 119
HTML
...................................................................................................................................................................
Elements to Enhance Your Survey 122
HTML
...................................................................................................................................................................
Cheat-Sheet 124
Graphics
...................................................................................................................................................................
Management 128
Survey Flow (Randomization, Skips, Loops)
...................................................................................................................................................... 129
Skip
...................................................................................................................................................................
Logic 131
Loops
................................................................................................................................................................... 136
Backing
...................................................................................................................................................................
Up and "Limbo" Data 138
Including Graphics
...................................................................................................................................................... 139
List Building ...................................................................................................................................................... 140
SSI Script ...................................................................................................................................................... 163
Displaying
...................................................................................................................................................................
Responses Given to Previous Questions 165
Functions
................................................................................................................................................................... 167
List Functions ............................................................................................................................................ 168
System Functions ............................................................................................................................................ 171
Quota Control ............................................................................................................................................
Functions 174
M ath Functions ............................................................................................................................................ 175
Loop Functions ............................................................................................................................................ 176
String Functions ............................................................................................................................................ 177
Error Functions ............................................................................................................................................ 178
CBC Functions ............................................................................................................................................ 179
ACA Functions ............................................................................................................................................ 181
CVA Functions ............................................................................................................................................ 183
M axDiff Functions ............................................................................................................................................ 184
ACBC Functions ............................................................................................................................................ 185
More
...................................................................................................................................................................
Detail on Boolean (True/False) Functions 187
Additional
...................................................................................................................................................................
Details on Certain Functions 190
Generating
...................................................................................................................................................................
Random Numbers 197
Unverified
...................................................................................................................................................................
Perl 198
Surveys in ......................................................................................................................................................
Multiple Languages 220
Print Survey ...................................................................................................................................................... 223
...................................................................................................................................................... 224
Respondent Passwords
Introduction to Passwords
...................................................................................................................................................... 224
Start Question Layout
...................................................................................................................................................... 225
Password Fields ...................................................................................................................................................... 226
Merged Fields ...................................................................................................................................................... 228
Adding/Importing Password Data
...................................................................................................................................................... 229
Pass-In Fields ...................................................................................................................................................... 233
Cookies and User-Defined Passwords
...................................................................................................................................................... 234
Quota Control ...................................................................................................................................................... 236
Introduction to Quota Control
...................................................................................................................................................... 236
Quota Dialog ...................................................................................................................................................... 240
Quota Control Settings
...................................................................................................................................................... 243
Advanced ......................................................................................................................................................
Quota Control Settings 244
Prequalification in Quota Control
...................................................................................................................................................... 246
Quota Changes Mid-Study
...................................................................................................................................................... 247
Testing/Running ......................................................................................................................................................
the Survey Locally 248
Testing/Running the Survey Locally
...................................................................................................................................................... 248
Survey Installation......................................................................................................................................................
on Web Server 250
Web Server Management
...................................................................................................................................................... 250
Manual Upload to Server
...................................................................................................................................................... 257
Deleting Practice Data Prior to Taking the Study "Live"
...................................................................................................................................................... 258
Troubleshooting Server-Side Problems
...................................................................................................................................................... 259
Making Changes after Fielding Begins
...................................................................................................................................................... 261
......................................................................................................................................................
Data Generator - Generating Test Data 263
......................................................................................................................................................
How Respondents Access the Web Survey 271
How Respondents Access the Web Survey
...................................................................................................................................................... 271
"One-Click" Respondent Access to Survey
...................................................................................................................................................... 272
Linking to ......................................................................................................................................................
Sawtooth Software's Web Survey from Another Survey Program 274
......................................................................................................................................................
CAPI Interviewing with "SSI Web CAPI" 276
CAPI Interviewing with "SSI Web CAPI"
...................................................................................................................................................... 276
Accumulating CAPI Files
...................................................................................................................................................... 279
Deleting Sensitive Data from CAPI Interviewing PCs
...................................................................................................................................................... 280
......................................................................................................................................................
Managing Survey Data 281
Database ...................................................................................................................................................... 281
Admin Module
...................................................................................................................................................... 284
Setting
...................................................................................................................................................................
Passwords for Administrative Access 286
Logging
...................................................................................................................................................................
On 287
Downloading
...................................................................................................................................................................
Survey Data 289
Viewing
...................................................................................................................................................................
Survey Data 290
Tabulating
...................................................................................................................................................................
Data On-Line 292
Downloading
...................................................................................................................................................................
Password Report 296
Incompletes
...................................................................................................................................................................
Report 297
Test
...................................................................................................................................................................
Survey Mode 298
Reset
...................................................................................................................................................................
Web Survey 300
Customizing
...................................................................................................................................................................
the Administrative Module 301
System Data Fields
...................................................................................................................................................... 303
......................................................................................................................................................
Accumulate/Export/Edit Data 304
Get Data ...................................................................................................................................................... 304
Export Data ...................................................................................................................................................... 305
Filters, Segments, and Weights
...................................................................................................................................................... 306
Export Data Dialog
...................................................................................................................................................... 309
Multiple
...................................................................................................................................................................
Response Question Format 310
Selecting
...................................................................................................................................................................
Data to Export 312
List
...................................................................................................................................................................
Recode 314
Exporting Open-End Data
...................................................................................................................................................... 315
Viewing/Editing Data
...................................................................................................................................................... 316
......................................................................................................................................................
Analyzing/Exporting Conjoint Data 317
Analyzing Data with Online Simulator
...................................................................................................................................................... 317
Moving Data from SSI Web to SMRT for Analysis
...................................................................................................................................................... 318
......................................................................................................................................................
Files created by SSI Web 322
Files created by SSI Web
...................................................................................................................................................... 322
CBC Help
......................................................................................................................................................
Getting Started with CBC 325
Overview of CBC Software
...................................................................................................................................................... 325
What is Choice-Based Conjoint?
...................................................................................................................................................... 327
CBC Tutorial and Example
...................................................................................................................................................... 330
......................................................................................................................................................
Designing CBC Studies 372
CBC Questionnaires and Designs
...................................................................................................................................................... 372
Defining Attributes and Levels
...................................................................................................................................................... 379
Specifying ......................................................................................................................................................
Prohibitions 382
Attribute Interactions
...................................................................................................................................................... 385
Response Type
...................................................................................................................................................... 386
Task Labels...................................................................................................................................................... 388
Number of......................................................................................................................................................
Attributes/Levels/Tasks in CBC 389
None Option/Dual-Response None
...................................................................................................................................................... 391
Specifying ......................................................................................................................................................
Fixed or Holdout Tasks 393
Specifying ......................................................................................................................................................
CBC Design Parameters 395
Testing the......................................................................................................................................................
CBC Design 399
Importing/Exporting CBC Designs
...................................................................................................................................................... 407
......................................................................................................................................................
Advanced CBC Designs 410
Conditional......................................................................................................................................................
Pricing for CBC 410
Conditional......................................................................................................................................................
Display 413
Alternative-Specific Designs
...................................................................................................................................................... 414
Partial-Profile Designs
...................................................................................................................................................... 418
Shelf-Facing Display
...................................................................................................................................................... 421
Customized......................................................................................................................................................
CBC Questions Using "Free Format" 424
Volumetric......................................................................................................................................................
CBC 429
......................................................................................................................................................
CBC Paper-and-Pencil Studies 431
......................................................................................................................................................
Analyzing the Data 433
Preparing Conjoint Data for Analysis
...................................................................................................................................................... 433
Counting Analysis
...................................................................................................................................................... 434
Estimating ......................................................................................................................................................
Utilities with Logit 438
Estimating ......................................................................................................................................................
Utilities with Latent Class 446
Estimating ......................................................................................................................................................
Utilities with HB 448
Moving Data from SSI Web to SMRT for Analysis
...................................................................................................................................................... 450
.CHO File ......................................................................................................................................................
Format 452
.CHS File Format
...................................................................................................................................................... 454
Adaptive CBC (ACBC) Help

......................................................................................................................................................
Getting Started with ACBC 457
Welcome to Adaptive CBC!
...................................................................................................................................................... 457
ACBC Specifications
...................................................................................................................................................... 458
Overview ...................................................................................................................................................... 459
Motivation......................................................................................................................................................
for Adaptive CBC 461
Sections and Flow for Adaptive CBC
...................................................................................................................................................... 463
Attributes ......................................................................................................................................................
and Levels for Adaptive CBC 473
Design Tab......................................................................................................................................................
(ACBC) 475
Design Tab--Test Design (ACBC)
...................................................................................................................................................... 479
Price in Adaptive CBC
...................................................................................................................................................... 482
...................................................................................................................................................... 484
Hands-On Tutorials
Tutorial #1......................................................................................................................................................
(Beaches) 485
Tutorial #2......................................................................................................................................................
(Pianos) 499
Tutorial #3......................................................................................................................................................
(Alternative-Specific Design) 506
......................................................................................................................................................
ACBC Design Algorithms 526
How Concepts Are Chosen
...................................................................................................................................................... 526
Deviation from Target Frequencies
...................................................................................................................................................... 531
......................................................................................................................................................
Advanced ACBC Surveys 532
Customized......................................................................................................................................................
(Constructed) Attribute and Level Lists 532
Price Adjustments (Within Summed Pricing)
...................................................................................................................................................... 536
Conditional......................................................................................................................................................
Display 538
Skipping Sections in ACBC
...................................................................................................................................................... 541
Alternative-Specific Designs
...................................................................................................................................................... 542
Unverified......................................................................................................................................................
Perl to Customize ACBC Questions 546
......................................................................................................................................................
Checklist Prior to Fielding 547
Checklist Prior to Fielding
...................................................................................................................................................... 547
......................................................................................................................................................
Analysis of ACBC Data 548
Counting Analysis for ACBC
...................................................................................................................................................... 548
Coding the......................................................................................................................................................
Information for Parameter Estimation 551
Modeling the Price Function
...................................................................................................................................................... 552
HB Estimation
...................................................................................................................................................... 556
Interaction......................................................................................................................................................
Effects 559
Interaction......................................................................................................................................................
Search Tool 561
Utility Constraints, Including Customized Constraints
...................................................................................................................................................... 565
Monotone ......................................................................................................................................................
Regression (Pure Individual Analysis) 568
Analysis with Tiny Sample Sizes
...................................................................................................................................................... 572
None Threshold & Calibration to Purchase Likelihood
...................................................................................................................................................... 574
Exporting ACBC Data to .CHO File
...................................................................................................................................................... 577
ACBC: How......................................................................................................................................................
Well Does It Work? 579
References...................................................................................................................................................... 583
MaxDiff Help
......................................................................................................................................................
Getting Started with MaxDiff 585
Introduction
...................................................................................................................................................... 585
Motivation......................................................................................................................................................
for MaxDiff 586
What is MaxDiff?
...................................................................................................................................................... 589
MaxDiff Tutorial and Example
...................................................................................................................................................... 590
......................................................................................................................................................
Creating a MaxDiff Experiment 616
Question and Label Text
...................................................................................................................................................... 617
Question Format
...................................................................................................................................................... 620
Developing......................................................................................................................................................
a List of Items 621
Designing ......................................................................................................................................................
the Study 624
......................................................................................................................................................
Analyzing MaxDiff Data 630
Counting Analysis
...................................................................................................................................................... 631
Individual-Level Score Estimation
...................................................................................................................................................... 632
Identifying......................................................................................................................................................
"Bad" Respondents 638
Aggregate......................................................................................................................................................
Score Estimation via Logit Analysis 640
Finding Segments via Latent Class Analysis
...................................................................................................................................................... 643
...................................................................................................................................................... 645
Anchored MaxDiff
......................................................................................................................................................
Customized MaxDiff Questions with "Free Format" 649
......................................................................................................................................................
Computing Scores On-The-Fly 654
......................................................................................................................................................
Paper-and-Pencil Studies 655
......................................................................................................................................................
MaxDiff Analysis Using CBC/HB v5.x 657
......................................................................................................................................................
MaxDiff Analysis Using Latent Class v4 659
......................................................................................................................................................
Robustness of MaxDiff Designs to Prohibitions 662
ACA Help
......................................................................................................................................................
Getting Started with ACA 669
Getting Started with ACA
...................................................................................................................................................... 669
What is ACA?
...................................................................................................................................................... 671
Basic Overview of ACA
...................................................................................................................................................... 673
ACA Tutorial and Example
...................................................................................................................................................... 674
Sections and Flow of ACA Questionnaires
...................................................................................................................................................... 714
Defining Attributes and Levels
...................................................................................................................................................... 720
Specifying ......................................................................................................................................................
Prohibitions in ACA 723
Setting ACA Control Parameters
...................................................................................................................................................... 724
How ACA......................................................................................................................................................
Data Are Stored 727
Computing......................................................................................................................................................
Utilities and Analyzing ACA Data 728
Is ACA the......................................................................................................................................................
Appropriate Technique? 730
Staying out......................................................................................................................................................
of Trouble with ACA 732
......................................................................................................................................................
Advanced ACA Questionnaires 734
Omitting the "Importance" Question
...................................................................................................................................................... 734
Customizing the "Importance" Question
...................................................................................................................................................... 736
......................................................................................................................................................
Algorithms and Estimation Techniques 738
Details of ACA/Web Utility Estimation
...................................................................................................................................................... 738
Choosing the Next Paired-Comparison Question
...................................................................................................................................................... 741
Updating Utilities During the Interview
...................................................................................................................................................... 743
ACA/HB Estimation
...................................................................................................................................................... 745
...................................................................................................................................................... 746
ACA File Formats
Format of .UTL and .ACD Files
...................................................................................................................................................... 746
......................................................................................................................................................
Real-Time Feedback to Respondents 751
......................................................................................................................................................
Moving Data from SSI Web to SMRT for Analysis 752
......................................................................................................................................................
Analyzing ACA Data with the Market Simulator 754
CVA Help
......................................................................................................................................................
Getting Started with CVA 756
Introduction to CVA
...................................................................................................................................................... 756
CVA Tutorial and Example
...................................................................................................................................................... 760
Is CVA the......................................................................................................................................................
Appropriate Technique? 801
......................................................................................................................................................
Designing/Creating a CVA Study 802
Attributes and Levels
...................................................................................................................................................... 802
Defining
...................................................................................................................................................................
Attributes and Levels 802
Level
...................................................................................................................................................................
Prohibitions and CVA Studies 806
Designing the CVA Tasks (Questions)
...................................................................................................................................................... 807
Introduction
...................................................................................................................................................................
to CVA Questionnaires 807
Advantages
...................................................................................................................................................................
and Challenges of CVA Questionnaires 809
Selecting
...................................................................................................................................................................
the Number of Tasks 810
Throw
...................................................................................................................................................................
out "Obvious" Tasks 811
CVA
...................................................................................................................................................................
Design Strategy 812
Testing
...................................................................................................................................................................
the CVA Design 817
Importing/Exporting
...................................................................................................................................................................
CVA Designs 819
Including
...................................................................................................................................................................
Holdout Choice Tasks in CVA Questionnaires 823
......................................................................................................................................................
CVA/Web Paper and Pencil Studies 824
CVA/Web......................................................................................................................................................
Paper and Pencil Studies 824
......................................................................................................................................................
CVA Advanced Designs 826
Conditional......................................................................................................................................................
Display for CVA 826
Conditional......................................................................................................................................................
Pricing for CVA 827
Modeling Interaction Effects with CVA
...................................................................................................................................................... 830
Sparse Designs and Large Samples
...................................................................................................................................................... 832
Data Analysis...................................................................................................................................................... 834
Estimating ......................................................................................................................................................
Part-Worth Utilities 834
CVA Ordinary Least Squares Settings
...................................................................................................................................................... 837
CVA Monotone Regression Settings
...................................................................................................................................................... 841
Technical Details ...................................................................................................................................................... 843
The CVA ......................................................................................................................................................
Designer 843
How CVA......................................................................................................................................................
Calculates Utilities 844
Avoiding Linear Dependency
...................................................................................................................................................... 849
The CVA/HB......................................................................................................................................................
Module for HB Estimation 851
What is CVA/HB (hierarchical Bayes for CVA)?
...................................................................................................................................................... 851
Running CVA/HB ...................................................................................................................................................... 853
Using Constraints in CVA/HB
...................................................................................................................................................... 856
Appendices for SSI Web

......................................................................................................................................................
A: Interpreting Conjoint Analysis Data 858
......................................................................................................................................................
B: Holdout Choice Tasks in Conjoint Studies 861
SMRT (Market Simulator) Help

...................................................................................................................................................... 864
What Is SMRT?
......................................................................................................................................................
Data and Utility Run Management 865
Importing Utility Runs
...................................................................................................................................................... 865
Merging Segmentation Data
...................................................................................................................................................... 868
Custom
...................................................................................................................................................................
Segmentation Variables 871
Weighting
...................................................................................................................................................................
Respondents 872
Data Management Tools
...................................................................................................................................................... 873
Tables Program ...................................................................................................................................................... 876
Using Simulators ......................................................................................................................................................
to Answer Strategic Questions 881
Overview ...................................................................................................................................................... 881
What Is a Market Simulation?
...................................................................................................................................................... 882
Why Conduct Market Simulations?
...................................................................................................................................................... 883
Typical Questions for Conjoint Simulators
...................................................................................................................................................... 885
Market Simulation ......................................................................................................................................................
Theory and Models 886
Base Case......................................................................................................................................................
Scenario 886
The Exponent (Scale Factor)
...................................................................................................................................................... 888
The Red-Bus/Blue-Bus Problem
...................................................................................................................................................... 889
Market Simulator Models
...................................................................................................................................................... 890
External Effects
...................................................................................................................................................... 893
Interpolating between Levels
...................................................................................................................................................... 895
The "None" Weight
...................................................................................................................................................... 897
......................................................................................................................................................
Using the Market Simulator 898
The Market Simulator Dialog
...................................................................................................................................................... 898
Adding a Simulation Scenario
...................................................................................................................................................... 901
Other Controls in Simulation Scenarios
...................................................................................................................................................... 903
Running the Simulation
...................................................................................................................................................... 906
......................................................................................................................................................
Practical Simulation Examples 907
Introduction
...................................................................................................................................................... 907
New Product Introductions
...................................................................................................................................................... 908
Estimating ......................................................................................................................................................
Demand Curves and Elasticities 910
Designing ......................................................................................................................................................
Products to Appeal to Unique Market Segments 913
......................................................................................................................................................
Technical Details for Simulations 916
First Choice Method
...................................................................................................................................................... 916
Share of Preference Options
...................................................................................................................................................... 917
Purchase Likelihood Option
...................................................................................................................................................... 920
Randomized First Choice
...................................................................................................................................................... 921
......................................................................................................................................................
Appendices for Simulation Topics 925
Importing Conjoint Data from Text-Only Files
...................................................................................................................................................... 925
How the Simulator Interpolates between Levels
...................................................................................................................................................... 929
How the Simulator Uses Latent Class Data
...................................................................................................................................................... 931
Index 932
General Web Interviewing (CiW) Help 1
1 General Web Interviewing (CiW) Help

1.1 Getting Started
1.1.1 Getting Started with Web Interviewing
Welcome to SSI Web!
SSI Web is a software system for writing questionnaires and conducting survey research studies over the
Internet or via CAPI-based (stand-alone PC) interviewing. You can press F1 at most any dialog/screen
within SSI Web to receive help about the options within that dialog. A complete copy of the help
documentation with a table of contents is available within the PDF file entitled SSI Web
Documentation.pdf that was installed with your SSI Web system. We suggest you start by viewing
Getting Started with SSI Web: A 45-Minute Hands-On Tour.
SSI Web is made up of various components that you can license from Sawtooth Software:
CiW is the core interviewing component, a general-purpose survey research tool suited for a variety
of simple to complex survey projects.
CBC is for Choice-Based Conjoint analysis, currently the most popular trade-off method.
ACBC is for Adaptive Choice-Based Conjoint, a trade-off method that is particularly well suited for
problems involving five or more attributes.
MaxDiff is for Maximum Difference (Best/Worst) scaling. This technique is for measuring the
importance or preference of a list of items. It shares much in common with conjoint analysis, but it
is applicable to a wider variety of situations and is easier than conjoint analysis.
CVA is for traditional, full-profile conjoint analysis. This is the original conjoint method and it is
particularly useful for small sample sizes and problems with few attributes.
ACA is for Adaptive Conjoint Analysis, an older adaptive trade-off technique for product design and
segmentation studies involving many attributes.
Sample CiW, ACBC, CBC, MaxDiff, CVA, and ACA studies and supporting files are included with your
installation in the /Samples directory. You can easily access these example studies by clicking Help |
Sample Studies. These questionnaires let you see examples and investigate how they are
constructed.
If you know a little HTML (which is not necessary to use this software), you can employ some "tricks"
that can help you customize the look of your survey beyond the standard options. This is especially the
case if using the "Free Format" question type. By opening the sample studies and looking at the
settings and text we've used, you can become familiar with some of the possibilities. There is also a
brief section available in the online help regarding HTML elements.
Using this web-based survey tool requires the following skills and knowledge base:
1. The analyst managing the survey should know the basics of survey writing. If using conjoint
analysis, the analyst should understand how to define attributes and levels properly, correctly
phrase the questionnaire text, interpret the conjoint utilities, and run simulations. It is not
necessary to be a statistician to use our software effectively, but knowledge of basic statistics
for market research is highly recommended.
2. The questionnaire is first composed locally on a PC running under Windows XP Service Pack 2
2 SSI Web v8
or later, using your web browser to preview/test the survey. The individual inputting/formatting
the survey should have a working knowledge of standard Windows applications such as word
processing and web browsers. Though it is not necessary, some knowledge of HTML is
valuable. To post your survey on the Web, you'll eventually upload your survey to a Web server
(Sawtooth Software also provides a popular hosting service).
3. To upload the survey to the Web, you must have access to an Internet/intranet server running
Perl 5.007 or later, with a database. This can be your company's own Web server, or your
ISP's (Internet Service Provider's) Web server. SSI Web manages the communication, file
transfer, and project setup on your web server.
Note: Sawtooth Software also provides web hosting services for licensed SSI Web customers. Please
call or email us for more information.
Steps to Creating a Web Survey
1. It is likely that you will first compose the script to your questionnaire using a word processing
package. This is fine as a starting point, as you can cut-and-paste text from your word processing
package into the questions you develop within SSI Web's survey building interface.
2. Create a new study in SSI Web.
3. Next, write your questionnaire, adding questions one-by-one within the Write Questionnaire
interface. You can preview these questions using the Preview function, that shows how your
Internet browser presents your questions.
4. Optionally, create respondent passwords, to provide a way to let qualified respondents in and
keep unqualified people out of the survey.
5. Preview the questions, questionnaire pages, or the entire questionnaire on your PC during
questionnaire development. You can also test or run your questionnaire locally on your PC using
the Local Test Server installed with SSI Web.
6. When you are satisfied with your questionnaire, upload your project to the Web server (or use
our hosting services). You then test your survey on the Web server, to make sure it functions
properly and saves the data correctly.
7. When you are satisfied that your survey looks and functions properly, invite respondents to take
the survey. The data are automatically stored on the server. You can view or perform tabulations of
the data in real time at any point during data collection using the Admin Module.
8. Download the data to your PC and use the data export capabilities within SSI Web to convert
the survey data to a generic text-only format, .csv format, Excel format, or SPSS format.
9. If running a conjoint analysis project (e.g. CBC, ACBC, ACA or CVA), use the menu system to
prepare the appropriate conjoint files for analysis with CBC/HB, the SMRT system, and/or with
Sawtooth Software Online Market Simulator.
Technical Support
Your license entitles you to free technical support via phone or email. We encourage you to contact us
at support@sawtoothsoftware.com or +1-801-477-4700 if you cannot find the answer to your problem or

question in this documentation. We are available Mon-Fri, 6:30 am to 5:30 pm Mountain Time. Outside
of regular business hours, you might try posting a question to our Online Forum at our website,
www.sawtoothsoftware.com/forum/.
We provide full technical support for composing and testing your Internet survey on your local PC and
provide instructions on how to upload your project to your server. We cannot take responsibility for or be
expected to provide technical support to debug any customized HTML, CSS, Perl, or JavaScript you may
add to your survey. You are also responsible to make sure that Perl is running, a database is available,
and that you can run Perl (CGI) scripts on your web server. We can provide some assistance related to
SSI Web's use with your server once the fundamental Perl functionality and a database is in place.
Privacy and Research
There is a great deal of concern regarding privacy rights and the Internet. Researchers must be
respectful of respondents and the valuable information they provide. One should never use information
collected during the course of research for purposes other than research. We suggest providing a
privacy policy near the front of your survey, to let respondents know how the data they provide will or
won't be used.
We condemn direct marketing, selling, list building, or PR campaigns that masquerade as research.
4 SSI Web v8
1.1.2 Getting Started: 45-Minute Tour of SSI Web

What is SSI Web?
SSI Web is a software system for creating Web-based questionnaires. These questionnaires can
include standard survey questions, such as "What is your primary occupation?" or "How many people
live in your household?". Other SSI Web studies may include specialized research techniques such as
Maximum Difference Scaling (MaxDiff), and conjoint (trade-off) analysis, including Choice-Based Conjoint
(CBC), adaptive Choice-Based Conjoint (ACBC), CVA (traditional full-profile conjoint), and Adaptive
Conjoint Analysis (ACA). The standard survey questions and conjoint questions are developed using the
same SSI Web system and are integrated within the same continuous survey. CBC, ACBC, MaxDiff,
CVA, and ACA are just specialized components within the SSI Web system.
SSI Web stands for "Sawtooth Software

Incorporated" and "Web" refers to the ability to
create Web-based surveys.
This 45-minute tour focuses on general surveys rather than conjoint analysis. Even if your primary intent
is to use SSI Web for its conjoint capabilities, you should find this tour useful, as we cover many topics
that are essential to all SSI Web surveys, whether including conjoint questions or not.
The Big Picture
The SSI Web system should be installed onto your own desktop computer. (You can install a demo
version of SSI Web from http://www.sawtoothsoftware.com/downloads ). SSI Web provides a simple
point-and-click interface for writing questionnaires and testing them—all on your own computer. For
example, you can make sure that any skip patterns, randomizations, and question verification (such as
whether respondents filled out all the required questions on a page) are working properly on your desktop
PC.
NOTE: You should NOT install the full SSI Web system on the Web Server.
After you finish writing your questionnaire on your PC, you just need to upload the project to the Web
Server (SSI Web handles the details of this once you give it the proper URL and login protocol for your
web server). This Web Server can be your own company's server, our servers (if you use our hosting
service), or a server that you "rent" from a third party provider, such as an ISP (Internet Service Provider).
By the way, we don't charge you any per-project or per-respondent interviewing fees if you use your own
or a "rented" server. You are free to interview as many people (who browse/connect to the Web Server)
and conduct as many projects as you like.
If you don't want to deal with hosting issues or

Web Server set-up, you can let us host your
survey. Fees start as low as $100 per survey.
For more information on how to upload your survey to a web server see: Web Server Management
Starting a New SSI Web Study
From this point on, we'll assume you have installed the SSI Web system on your computer. (You can
install a demo version of SSI Web at http://www.sawtoothsoftware.com/downloads).
Start the SSI Web system by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI
Web. If you are starting the system for the first time, SSI Web will ask you if you want to start a new
study (otherwise, click File | New Study.) Go ahead and start a new study, with a studyname no longer
than twenty-four characters long (e.g. "mydemo"). Note that this name appears within in the address
field and in files you use on the internet, so you probably shouldn't use words that reveal something
about your project that you wouldn't want revealed, such as your client's name.
Next, choose a folder on your hard drive to store your project. Browse to the folder (by clicking the
Browse… button) and (if needed) create a new folder using the Mak e New Folder button on the dialog.
Finally, select what type of study to create. For the purposes of this tour, select CiW Study.
6 SSI Web v8
When you start a new study (or open an existing study), the main menu is shown:
The first thing you should know is that while at any screen in SSI Web, you can press F1 to receive help
related to the items on that screen. You can also receive help by clicking Help from the main menu and
then by choosing SSI Web Help… or alternatively by clicking the ? button on the toolbar.
Writing a Simple Survey
From this point onward, we'll assume you've installed the SSI Web system and created a new study on
your hard drive. Let's consider a simple questionnaire:
Welcome to our survey! Today we are interested in ice cream preferences. Please type your
user name and password and then click the "Next" button to continue…
-------------------------------------<Page Break>---------------------------------------
Q1. Please tell me how often you typically eat ice cream…
Never
Rarely
Sometimes
Often
-------------------------------------<Page Break>---------------------------------------
Q2. In the past week, how many times have you eaten ice cream?
______ times
Q3. Which of these flavors do you often eat? (Select all that apply)
<Randomize list>
Chocolate Rocky-Road
Strawberry Peach
Vanilla Neapolitan
-------------------------------------<Page Break>---------------------------------------
Q4. Please describe the last occasion in which you ate ice cream…
-------------------------------------<Page Break>---------------------------------------
Thank you very much. That's all we need to know today.

8 SSI Web v8
To write this survey, click Compose | Write Questionnaire.... The Write Questionnaire dialog is
shown:
The panel displays the list of questions that are currently in your survey. SSI Web automatically adds
a Start "question" on the first page of your survey.
To modify the Start question, highlight it within the List of Questions using the mouse and click the
Edit… button (or double-click the question).
The Questionnaire Access and Passwords dialog is shown:
This dialog lets you specify what the respondent sees on the first page of the survey and also lets you
specify respondent passwords if you want to include passwords in your survey.
Each question in SSI Web has different text areas where you can directly type text to show on the
screen (and optional HTML). The three main areas for the introductory Start question are:
Header 1
Header 2
Footer
10 SSI Web v8
These text areas are arranged on the browser screen (if you were to preview the question) as:
"Header 1" "Header 2" and "Footer" provide an easy way for you to type information into SSI Web and
have these sections separated by a blank line. Let's add the introductory text for our ice cream
questionnaire. Within the "Header 1" field, type the text:
Welcome to our survey! Today we are interested in ice cream preferences. Please type your
user name and password and then click the "Next" button to continue…
Note that you can directly type the text into the "Header 1" entry field, or you can click the
"Pencil" icon to open a larger window for typing/editing text.
After you have done this, the dialog should look like:
Create User Names and Passwords
We want to establish usernames and passwords for respondents in this study (it is optional to set up
passwords for respondents). Click the Password Fields tab. The table on this tab is where you specify
any passwords respondents must specify prior to accessing the survey. You can add many password
fields, but it is typical to include just one or two password fields. In our survey, we'll be using a
username and a password. We'll name the variables (fields) myusername and mypassword.
Edit the first row of the table, inserting the name of the password variable (myusername) in the first
column. If you want the text to appear as dots when respondents type (to mask their password so that
others viewing the screen cannot read what is typed), click the Mask ed box. If you want the passwords
to be case sensitive (so that respondents have to use capital and lower case letters exactly as
specified), then click the Case Sensitive box.
Specify the text that you want shown next to the myusername entry box in the Row 1 Password Label
area. We'll use the text "User Name:". When you have finished supplying the settings for the
myusername password variable, the table looks like:
12 SSI Web v8
To add a new password variable, type mypassword in the second row of the table. Let's mask
mypassword when respondents type in their password, by clicking the Masked box in the second row of
the table. Let's also add the label: "Password:" in the Row 2 Password Label field. After you've specified
the mypassword row in the table, it should look like:
Next, click the Preview button at the bottom of the dialog.

A small browser window opens to give a representation of what this question would look like to your
respondents:
Perhaps we would prefer to bold the text in the opening paragraph. Click OK in the Preview window to
retire that window. Click the Start Question Text tab. To the right of the "Header 1" field, click the
"paintbrush" icon , use the drop-down box to change bold to "Yes" (to override the existing style
settings for this question) and click Preview. The question now appears as:
(Alternatively, for those that know some HTML, you could have enclosed the text you typed into the
Header 1 field within <b></b> tags to accomplish the same effect.)
Most surveys use a unique user name/password for each respondent. For simplicity sake, we'll use a
single User Name and Password combination for all respondents in this practice survey.
From the Questionnaire Access and Passwords, Password Fields tab, click the Password Data... icon:
First, you are asked if you'd like to save your work so far. Click Save and Continue. A table opens that
looks a bit like a spreadsheet. This table displays the contents of your passwords file. Edit this table to
add a User Name of "sawtooth" and a Password of "software" with 1000 total respondents allowed to use
that password. When finished, the screen should look like:
14 SSI Web v8
Earlier, when we first created the two password field variables, we had specified that the username and
password were not case sensitive, so respondents can access the survey by typing "sawtooth" and
"software" with either upper- or lower-case letters.
Each time you add a new row (a new myusername and mypassword combination), an empty row
appears at the bottom of the table. A million or more password combinations could be specified, if
needed. You can Auto-Generate... many random password combinations, or you could Import...
password information from an existing text file.
For simplicity sake, we have allowed 1000 respondents to use the same password combination. In most
studies, each respondent receives a unique password combination (Max Respondents=1). With unique
passwords for each respondent, respondents that terminate early can return and restart the survey where
they left off.
Adding Questions
Now let's add question 1. Click Close to close the passwords file and click OK to close the
Questionnaire Access and Passwords dialog to return to the Write Questionnaire screen. The
question to be added is:
Never
Rarely
Sometimes
Often
Whenever you add a question to the List of Questions, the new question is placed directly below
whatever question (or page break) is highlighted. If no question is highlighted, then the new question is
placed at the bottom of the list. If you want to rearrange the questions within the list, you can highlight
them and use the "Up" and "Down" arrows next to the list. Or, you can cut, copy and paste questions
by using the cut, copy and paste icons.
We want the first question to appear below the first page break. Highlight the page break on the list of
questions, then click the Add… button. The Add to Questionnaire dialog is shown:
First, choose the question type from the radio buttons shown. This is a "Select-type" question, so click
Select. Next, specify a name for the question in the Name field. Type "Q1" and click OK.
16 SSI Web v8
The Select Question dialog is displayed:
This question requires just a single response (a radio button), so you should click the Settings tab and
select Radio button (single response) in the upper-left hand corner of this dialog (it is already selected by
default).
Back on the Question Text tab, in the Header 1 Text field, type:
Next, let's specify the response options:
Never
Rarely
Sometimes
Often
Click the Response Options… tab. On the Response Options tab, click Add... and type the first
response "Never". Press the ENTER key twice and type the next item "Rarely." Repeat for
"Sometimes" and "Often."
(Time-Saving Hint: Many users already have their questionnaires prepared in Microsoft Word. You can
directly cut and paste a list of response options into SSI Web by highlighting the array of items in
Microsoft Word or Excel, pasting to the clipboard using Ctrl+C or the Copy button in Microsoft Word or
Excel (or another text editor), and then (within SSI Web) by clicking the Paste list member(s) from the
clipboard icon .)
Click OK to return to the Select Question dialog and click Preview: Your question should look like:
From SSI Web, Click OK to return to the Write Questionnaire dialog.
We hope by now you are starting to get a feel for adding questions to SSI Web. We won't give you click-
by-click details for the next three questions. Instead, we'll give you some bullet-pointed suggestions.
Q2. In the past week, how many times have you eaten ice cream?
______ times
This is a "Numeric" question

Type the "times" label in the Question text area
From the Settings tab, specify a Minimum value of 0 and Maximum of 24
From the Settings tab, specify to position the question text ("times") Right of input box.
When you are done and click Preview, the question should look like:
18 SSI Web v8
Next comes Q3:
<Randomize list>
Strawberry Peach
Vanilla Neapolitan
This is a "Select" type question

From the Settings tab, choose Check Box (multiple response)
Also from the Settings tab, for Number of Columns, specify 2
When at the Response Options tab, make sure to click Randomize Response Options
When you are done, click Preview and the question should look like (notice the randomized order):
Next comes Q4:

Q4. Please describe the last occasion in which you ate ice cream…
This is an "Open-end" type question
When you are done, click Preview and the question should look like:
The final part of the survey is a sign-off screen thanking respondents and letting them know they are
done. This really isn't a "question" in the sense that it doesn't require a response, but for the purposes of
adding it to the list, SSI Web considers it a Terminate/Link "question":
Thank you very much. That's all we need to know today.
This is a "Terminate/Link" type question

You can name the question "Finish"
From the Settings tab, make sure to check the Terminate Respondent box
When you are done and click Preview, the question should look like:
The "Powered by Sawtooth Software, Inc." is shown on the last page of the survey, but you can turn this
20 SSI Web v8
message off if you would like, under the Survey Settings | General Format area accessed by clicking
from the Write Questionnaire dialog (we'll discuss this dialog more at a later point).
Page Layout and Skip Patterns
While you were inserting questions into your question list, you may have also been adding page breaks
by clicking the Add Page Break button from the Write Questionnaire dialog. If you have not already
added page breaks between questions, we are ready to divide them into separate pages and add any
skip patterns.
Go to the Write Questionnaire dialog. Note that the Start question is on Page #1. If you haven't
already inserted page breaks, all other questions are on Page #2. The original questionnaire instructions
called for:
Q1 to be followed by a page break

Q2 and Q3 on the same page
Q4 on its own page
The termination screen at the very end, on its own page
This is actually quite easy to accomplish. With the mouse, highlight Q1 on the question list, then click
Add Page Break. Add a page break after Q3 and Q4. When you are done, your question list should
look like:
Now for the skip pattern. We know that if respondents answer item 1 (never eat ice cream) in Q1, they
should be terminated. We therefore want to set a skip pattern with the following logic:
From Q1, Skip To Finish If Q1=1
To do this, from the Write Questionnaire dialog, click Skip Logic…. The Page Skip Logic dialog is
shown. Click Add to add a skip pattern. Fill this next screen out as follows:
For simplicity sake, we only have created one terminating page both for people who disqualify (eat no ice
cream) and those who complete the survey fully. You could include multiple terminating points and mark
records that terminate as Qualified or Disqualified, or even assign "disposition numbers" to track
respondents who terminate at various points in the questionnaire.
Now that we have added our questions to the study, added page breaks, and defined skip logic, let's
preview our questionnaire.
Previewing the Questionnaire
We have already shown you how to preview each question, one at a time. We'll now show you how to
preview the entire survey. At the Write Questionnaire dialog (accessed from the main menu by
clicking Compose | Write Questionnaire or by clicking the "Pencil" icon), click the "Preview"
icon .
When you click Preview, you can choose what to preview. Choose Preview Questionnaire from Page 1.
A browser window automatically opens page 1 for your review. In the background, SSI Web has created
a Temp folder inside your study folder on your hard drive and written separate static HTML pages to that
folder named TempWeb1 through TempWeb5 (you have 5 pages in this questionnaire). These files are
linked one to another by the Next button, for your convenience, so you can browse from page to page by
just clicking the Next button. Another convenient way to preview the questionnaire (or share the
questionnaire with colleagues) is to save the survey to a single HTML file or PDF-format file. Click File |
Print Study to print the study in either of these formats (select Graphical View and click Print to File).
Note: The Preview Survey mode is just for preview purposes only—no data are stored and no skip
patterns are executed. These static pages are not uploaded to the Internet for data collection purposes.
Instead, your questionnaire file is uploaded to the Internet and the Perl scripts on your Web Server
understand the instructions within the questionnaire file and create each questionnaire page, as
requested by the respondent, "on the fly."
Test Running Your Survey Using "Local Test Server"
Although the Preview Survey function is nice, you generally will want to test your survey in runtime mode,
with any skip patterns, response verification, and randomizations in place. You could upload your files
and the Perl scripts to the Web Server, but SSI Web provides a more convenient way to test run your
survey locally on your PC or laptop.
22 SSI Web v8
When SSI Web was installed on your computer, web server software called "Apache" was also installed.
Apache lets your computer run the questionnaire just like a remote Web Server would.
Local Test Server
From the Write Questionnaire dialog, Click the Test Survey button and select to Test Survey
Locally.
Your browser opens up the first survey page. Enter your User Name (SAWTOOTH) and password
(SOFTWARE) and click the Next button. This is your survey, as it would appear to a respondent over
the Internet (there are slight visual differences from browser to browser).
To close the survey, simply close the browser window (using the X in the upper-right hand corner). To
run the survey again, click the Test Survey button .
Go ahead and take a survey or two, as we'll need the data for a later section of this tour.
Adding Polish and Style

You have probably noted that the survey we've created is pretty bland looking. We can add some polish
and style by clicking the Survey Settings icon from the Write Questionnaire dialog (or by clicking
Compose | Survey Settings from the main menu).
First, let's select a style from the Styles tab. Select a style you wish to use and click Use Selected at
the bottom of the dialog to implement the style. We'll select Panama as the style, but you might want
to experiment with other styles.
On the General Format tab, you can select to use graphical Submit, Previous, and Select/Checkbox
buttons. A library of buttons is provided, available in the C:\Program Files\Sawtooth Software\SSI
Web\graphics folder.
On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Ice Cream
Questionnaire as the header text.
The Progress Bar tab lets you add a progress bar to your survey. We'll click the check box to add a
progress bar to the page footer (it's on by default).
When you go back and test run the survey, your survey should have a bit more polish. Under Panama
style, our survey now looks like:
The Admin Module
When you eventually upload your project to your Web Server (or our servers if using our web hosting
service), an administrative area is available for you. You can access this administrative area from any
computer connected to the Web. You protect this area using a password, so that others cannot access
your survey settings or the survey data. You may change the password under the Field | Web Server
Management dialog.
With the Admin Module, you can…
Check how many completed records you have

View/Edit respondent data
Tabulate responses for questions
Download data to your hard drive
Reset survey (delete all data and start fresh)
Download passwords for respondents who have completed/not completed
In this tour, we'll just cover downloading data, but we encourage you to investigate the other options
within the Admin Module.
Running the Admin Module
From SSI Web's main menu, click Test | Admin Module. The Admin Module appears just as it would
over the Web, but we are only working with files and data locally on your hard drive.
24 SSI Web v8
Downloading Data
Downloading data copies the respondent records to a .csv file to a destination folder (such as your hard
drive). You can do this from the Admin Module by clicking Download. In that case, it will ask you to
browse to your study folder on your hard drive. You may download data that way, but it is easier to do it
from within SSI Web's interface (since you don't have to browse to your study folder). From within SSI
Web, click Test | Download Data. You are informed that the test data were successfully downloaded.
(This step also accumulates the data into your SSI Web project.)
The master file of respondent data is called studyname_data.sqlite, where studyname is the study name
you created (e.g. mydemo_data.sqlite). Before proceeding, you may want to make sure that this file is
indeed within your main project folder. A quick way to browse to your folder to check this is by clicking
the Explore Study icon on the tool menu, which automatically opens the study folder. The
studyname_data.sqlite file must be saved to that same folder.
Exporting (Converting) Data to Excel(TM) Format
After you have collected data and downloaded the respondent records to your hard drive, you typically
will want to convert the data to a common format for analysis. SSI Web can convert the data (and
labels) to SPSSTM, ExcelTM or text-only formats. For this tour, we'll just use the .CSV format which may
be opened using Excel.
If you haven't already done so, close the browser window for the Admin Module and return to the main
SSI Web menu. Click File | Data Management (or click the Export Data icon ) and the Data
Management dialog is shown:
You export data from SSI Web using Export Jobs. If you are exporting the data in multiple ways (such
as for multiple clients), you may wish to set up and save a different job for each client. For now, we just
have one job called ExportAllDefault. Click Edit Selected Job... to see the settings for this default job.
The Export Job dialog is displayed:
The Respondents to Include control lets you choose which respondents to include. Respondent
Numbering lets you specify the variable to use for the CaseID. The File Formats control lets you specify
the type of file to export (you may select from .csv, Excel, SPSS, and Tab/Lay). The Export Fields lets
you choose which variables to export. There are other settings that we won't cover in this short tutorial.
For this example, we'll just use the default settings. Note that the exported .csv data file will be placed
26 SSI Web v8
in the Output Folder (indicated at the bottom of this dialog), which is by default named as
ExportAllDefault. Click OK to return to the previous dialog and then click Export All Checked Jobs.
To see the result, using your Windows Operating System, browse to the folder (don't forget the handy
Explore Study icon on the tool menu that automatically explores your study folder). Within the
ExportAll/Export folder you will find a Default.csv file. Double-click the Default.csv file. If you have
Excel on your computer, it should be associated with a .csv file, so this action should immediately open
your data file within Excel. If you don't own Excel, the .csv file may be opened using Notepad or another
simple text editor.
Here are our results (yours will be different):
If we scroll to the right, we find the responses to the questions Q1 through Q4.
Responses for Q1 and Q2 are quite straightforward. Q1 was a single-select question and the first
complete respondent (listed on row 4 of the spreadsheet) answered with the second response option
(coded as a "2").
The second completed respondent (on row 5) clicked the third option for Q1 and typed a 3 for Q2 (how
many times eaten ice cream in last week). Q3 is a multiple-response (check box) question. This
respondent checked the first, third, fourth, and sixth items (a "1" indicates checked and a "0" indicates
not checked). SSI Web automatically unscrambles the random order and presents it here in the natural
order you specified when you created the survey.
Adding Graphics
As a last point, we'd like to show you how easy it is to add graphics to your survey. The typical kinds of
graphics files you use for web surveys are in .GIF or .JPG format. You can likely find a number of
graphics files with this format on your hard drive (use the File Find capability within Windows if you do
not immediately know of a .GIF or .JPG file on your computer).
You may recall that in SSI Web, most any place that you provide survey text, there is also a
"Pencil" icon, which opens up a larger Window for typing text. This larger window also includes a
toolbar, as shown below:
For example, this toolbar helps you bold (B), italicize (I), or Underline (U) text within your survey (using
the toolbar buttons automatically insert HTML instructions, which browsers know how to interpret to
create the desired effects).
You can also use the toolbar to add graphics. Place your cursor at the place in your text at which you
wish a graphic to show. Then, click the "camera" icon. A dialog appears that asks you to type a file
name or browse to find the graphic to add to the survey. Type the name, or browse to the file and click
the Open button to insert the graphic. Special HTML instructions are added to your text to tell the
browser what file to display and where to find it on the server.
When you preview or run the questionnaire using Local Test Server, the graphic is displayed. (Make
sure to keep your graphics small, as large graphics files can take a long time to show for respondents
with slow connections. Typical graphics sizes for surveys are about 20KB or smaller.)
Conclusion
This quick tour has only touched on the basics. We encourage you to search the online help, or to print
out the complete documentation. The documentation is provided in a .PDF format called "SSI Web
Documentation.PDF" within your SSI Web installation folder. And of course, if you have any questions,
you can contact us at +1-801-477-4700 or write us at support@sawtoothsoftware.com. We are
available Mon-Fri, during our business hours (Mountain Time Zone).
28 SSI Web v8
1.1.3 Software Requirements: Users, Servers and Respondents

For the User (Survey Developer):
PC running under Windows XP Service Pack 2, Windows Vista, Windows 7 or later.

Any browser with JavaScript enabled.
The Internet/Intranet Server:
Support Perl Version 5.007 or later (capable of running CGI scripts)

Permit "write" and "delete" capabilities to allow our programs to write and delete files on the
server. (If your ISP doesn't support these capabilities, we can recommend an ISP that does,
or we can host your surveys for a fee.)
Database (MySQL or Microsoft SQL)
For the Respondent:
Respondents can use any Internet-enabled device. Since the surveys use only HTML pages,
any browser that interprets standard HTML will work. By default, we require JavaScript be
enabled on the device. To modify this behavior, see the Advanced Survey Settings section.
1.1.4 Creating/Opening a Study

A "Study" is a collection of files pertaining to a particular research project. SSI Web requires that you
place your studies in separate directories (folders) on your local drive. If your study is named "abcd," a
folder called abcd is created and a study file named abcd.ssi is created within that folder:
/abcd/abcd.ssi
We suggest that you place your study directory on your local hard drive rather than a shared network
drive. If using a network drive, the performance can be slower and SSI Web does not support multiple
people accessing it at the same time.
To create a new study:
Click File | New Study, browse to the desired directory (or click the Create a New Folder icon)
and provide a studyname consisting of twenty-four or fewer characters. A folder called
STUDYNAME is also created, the STUDYNAME.ssi file is placed within that target directory,
and the study is opened.
To open an existing study:
Click File | Open Study and if necessary browse to the folder containing the study. Then
click the STUDYNAME.ssi file to open a study previously created. You may also select a
study name from the list of most recently opened studies, available under the File menu, or
when browsing folders double-click a STUDYNAME.ssi file.
To copy the current study to a new path or name:
Open an existing study. Click File | Save Study As.... Browse to the target folder and type a
study name that is no more than twenty-four characters. SSI Web automatically supplies a
.ssi extension to the filename and saves all associated study files (including passwords,
graphics directory, and data) using that naming convention to the target folder.
30 SSI Web v8
1.2 What's New in v8.4.0?

Updates for v8.4.0
The main improvements in this version make SSI Web faster and more efficient. Data accumulate,
export, and analysis perform faster, noticeably so with large data sets. Auto upload has been improved
to be compatible with more ftp server software.
Updates for v8.3.0
The bulk of the improvements are in the ACBC component. We added a few additional features that
improve the capabilities of CBC and MaxDiff.
CBC Improvements:
Interaction Search Tool In v8.3 a new aggregate logit utility estimation procedure is available to
help researchers investigate which interaction effects are potentially most significant. This routine
automatically estimates utilities for main effects only, then runs main effects plus each possible two-
way interaction effect (examined one at a time). It summarizes the results by listing the interaction
terms in order from most to least significant in terms of the p-value as computed from a 2-log
likelihood test (a more powerful statistical test than the 2-way counting procedure and Chi-Square
test). This automated routine should take away excuses for not investigating potential interaction
effects. The most significant interaction effects found by this automated tool under aggregate logit
are probably good candidates for inclusion in the final HB modeling.
Option to Constrain Groups to a Common Scale One complaint about latent class procedures is
that they can occasionally create a group of respondents who has essentially the same pattern of
preferences as another group of respondents, but whose main differentiation is just the magnitude of
their part-worth utilities. In other words, two groups could have the same preferences, but one group
could have much more response error (noise) than the other. We've added a new option in the latent
class estimation option for CBC so that in each iteration of the latent class procedure, the scale
factor for the part-worth utilities across groups is constrained to be equal to the average scale factor
for the groups. This constrains latent class from returning solutions where two groups differ only in
terms of scale factor (response error).
Not Applicable (Missing) Levels in Conditional Display If you have an Alternative-Specific or

Partial Profile design then it is possible for you to need to specify in a conditional display table what
is displayed when a missing level occurs. The 0 level in the table represents the not applicable level.
Previously, we defaulted to the text shown for level 1 of the not applicable attribute.
ACBC Improvements:
Alternative-Specific Designs Until this software version, ACBC users have been limited to
traditional experimental designs, where the attribute list is a common list applicable to all product
alternatives. In v8.3, users can specify that certain attributes (with all their levels) are only applicable
to certain level (e.g. brands, styles, etc.) of a primary attribute. This will open up new possibilities for
the application of ACBC. For example, if comparing desktops to laptop computers, the laptop has a
battery whereas desktops do not. There may be a conditional attribute upon the presence of battery,
such as three levels of battery life. With alternative-specific designs, the attribute Battery Life (with
its three levels) would only occur with laptop computers.
Merged Rows Especially with Alternative-Specific designs, it is possible that two or more attributes
actually refer to the same topic. For example, two separate attributes involving processor speed
could be associated with either desktops or laptops (the desktops tend to be faster). Rather than
show these as two separate rows in the choice task display (e.g. with blanks when the desktop
speed attribute doesn't apply to the laptop), you could merge these two attributes to be displayed on
just one row with a generic attribute label in first column, making the display appear more clean and
compact. For more information, please see Merged Rows.
Ability to Skip the Screener Section Starting in v8.3, it is possible to skip the Screener section,
meaning that the respondent can jump straight from the BYO question to the Choice Tournament, or
right into the Choice Tournament (skipping both the BYO and Screener sections). Note: no "None"
utility will be available if you skip the Screener section, unless you estimate the None utility using
the optional Calibration section.
Improved Design Efficiency V8.3 ACBC designs are significantly more efficient than before.
Previous versions of ACBC have done a reasonably good job of achieving independence of the
attributes (orthogonality), but v8.3 goes much further to increase the orthogonality of the experiments
at the individual-level. Orthogonality is directly related to D-efficiency. V8.3 features a level
relabeling/swapping routine that oversamples BYO levels at the same rate as before, but creates
near-neighbor cards that have significantly higher within-respondent D-efficiency. Our tests suggest
that v8.3 improves the D-efficiency (at the respondent level) by about 5%-40%, depending on the
attribute list and questionnaire setup.
Test Design Routine The previous ACBC software did not have a test design functionality, so it
was not easy to test the adequacy of designs for individuals or to assess the damage from
prohibitions prior to fielding a study, for example. V8.3 includes a test design routine that generates
a few or hundreds of "robotic" respondents (where the respondents answer randomly) and provides
key statistics regarding the quality of the designs: level frequency, balance, d-efficiency, and
standard errors (precision) of the utility parameters. It reports how often each level was shown to
each respondent, so it is easier to adjust the questionnaire settings to target the "show each level at
least twice and preferably three times per respondent" rule.
Avoiding Dominated Concepts A dominated concept is a product alternative that is logically

inferior to another concept (based on the Preference Order settings the researcher specifies).
Researchers have found that avoiding dominated concepts in conjoint designs adds only a very little
to the quality of the final utility results. However, it avoids the situation which sometimes bothers
clients and respondents, where an obvious question is asked that might make that person doubt the
logic behind the questionnaire. V8.3 allows the researcher to specify whether to avoid dominated
concepts. Some degree of level balance and orthogonality is typically lost when avoiding dominated
concepts (since it is just another form of prohibition that hinders the experimental design).
Depending on the study setup, it could be a relatively minor or a major effect upon the design quality.
The new Test Design routine can help you assess the impact of avoiding dominated concepts on
the precision of utility estimates.
Ability to Customize the "At Most" and "At Least" Text by Attribute In the "Must Have"
questions, for attributes with a researcher-specified order (best to worst or worst to best) we often
ask respondents if they must have "At Least" or "At Most" some level of an attribute. In previous
versions of the software, the same global text for these phrases needed to apply to all attribute
levels. Some researchers requested the ability to customize the text by different attributes and v8.3
supports this.
Interaction Search Tool For v8.3 a new aggregate logit utility estimation procedure is available to
help researchers investigate which interaction effects are potentially most significant. This routine
32 SSI Web v8
automatically estimates utilities for main effects only, then runs main effects plus each possible two-
way interaction effect (examined one at a time). It summarizes the results by listing the interaction
terms in order from most to least significant in terms of the p-value as computed from a 2-log
likelihood test. In our experience, most ACBC researchers ignore interaction effects and use main
effects only (since adaptive ACBC designs don't support a convenient Counting Analysis report for 2-
way interaction effects like CBC does). Yet, ACBC designs are well suited for estimating interaction
effects. This automated routine should take away excuses for not investigating potential interaction
effects. The most significant interaction effects found by this automated tool under aggregate logit
are probably good candidates for inclusion in the final HB modeling.
Option to Take BYO Concept Forward to Choice Tournament With v8.3, the researcher can
specify that the respondent's BYO-selected "ideal" product should be carried forward into the Choice
Tournament. For situations involving Summed Price or when not all attributes are shown in the BYO
question, this can make a lot of sense (and lead to a richer Choice Tournament). But, for cases in
which all attributes are shown in the BYO question and summed pricing is not involved, the BYO
concept should dominate any other concept that the experimental designer could generate, so it
may not be advisable to carry the BYO concept forward to the choice tournament.
Ability to Exclude Certain Attributes from "Must Have/Unacceptable" Probes Some

researchers have asked us to make it possible to exclude certain attributes from the Must Have/
Unacceptable probes. Prior to v8.3, Price was the only attribute that the researcher could exclude
from these probes. With v8.3, the researcher has full flexibility, by attribute, regarding the probes.
However, we caution the user that excluding an attribute from Must Have/Unacceptable probes may
potentially affect the utilities in undesirable ways. For example, an attribute involving levels allowed
to be marked as Must Have or Unacceptable can get larger extremes in utility than attributes
excluded from these probes.
Covariates in ACBC/HB Utility Estimation We've extended the same capabilities that CBC/HB
has regarding the use of covariates into the HB routine for ACBC. Covariates are typically profiling or
usage-based data on respondents. For example, respondent income could be used as a covariate in
HB estimation. If income were applied as a covariate, then when respondents are "smoothed" toward
the mean via Bayesian shrinkage, they are mainly smoothed toward the mean utilities of
respondents who are like them on the value of the covariates. So, for example, low income
respondents would mainly be influenced by other low income respondents with respect to the data
borrowing mechanism.
More Flexible Utility Constraints During utility estimation, the researcher can specify that certain
levels must be preferred to other levels within an attribute. This can make a lot of sense if the
attribute truly has a logical preference order that all respondents would agree upon. Previously, there
were limitations in the way that utility constraints could be specified. For example, you could not
specify that level 1 > level 2; and level 3 > level 4 (without also indicating how levels 1 and 2 related
to levels 3 and 4). With v8.3, the new dialog lets the researcher specify multiple independent chains
of ordinal constraints within an attribute.
Record and Export Tournament Concept Order In previous versions of ACBC, if you examined
the way the data were stored for the Choice Tournament, it would appear that the respondents
always picked the first concept from each set. That's just due to a shortcut we took in programming
where we rearranged the concept order that was actually shown to respondents and recorded the
chosen concept always in the first position. This bothered some ACBC users, because they wanted
to be able to examine the original order of the concepts as shown to respondents. So, starting in
v8.3, we are now saving the concept order as was seen by respondents.
Not Applicable (Missing) Levels in Conditional Display If you have an Alternative-Specific

design in ACBC then it is possible for you to need to specify in a conditional display table what is
displayed when a missing level occurs. The 0 level in the table represents the not applicable level.
MaxDiff Improvements:
Option to Constrain Groups to a Common Scale. One complaint about latent class procedures is
that they can occasionally create a group of respondents who has essentially the same pattern of
preferences as another group of respondents, but whose main differentiation is just the magnitude of
their part-worth utilities. In other words, two groups could have the same preferences, but one group
could have much more response error (noise) than the other. We've added a new option in the latent
class estimation option for MaxDiff so that in each iteration of the latent class procedure, the scale
factor for the part-worth utilities across groups is constrained to be equal to the average scale factor
for the groups. This constrains latent class from returning solutions where two groups differ only in
terms of scale factor (response error).
General Changes:
Better Handling of Extremely Large Surveys SSI Web previously could crash during certain
analysis steps when surveys became extremely large, such as 60K data fields or larger. This could
occur when accumulating or exporting data (but not during data collection). Such extremely large
data sets can now be handled in v8.3.
JavaScript Required In v8.3 by default all surveys now require the respondent to have JavaScript
turned on in their browsers. If they do not have JavaScript they will see an error message prompting
them to turn it on. You can edit this error message under Survey Settings | Error Messages. See
Survey Settings | Advanced to turn off this feature.
In case you did not see the features we implemented in the earlier v8.2.0 release, it included the
following new features.
Updates for v8.2.0
General Improvements:
Data Generator You can generate test records automatically using this new feature within the SSI
Web system. This is helpful for testing skip patterns and setting up crosstabulation reports prior to
collecting real data. Data can be generated under local testing on your computer or remote testing
on a web server. There are many advanced capabilities within the Data Generator, including the
ability to script specific responses to given questions (for example, to answer a series of screener
questions without screening out) and for other questions to answer randomly.
Streamlined Sawtooth Software Hosting Setup Those who use Sawtooth Software's hosting
service previously had to cut-and-paste server protocol information (database name, URL, FTP User
Name, passwords, etc.) from their hosting portal dialog into SSI Web's Field | Web Server
Management menu for each new study. This involved about seven separate cut-and-paste steps.
You no longer need to do this. The Field | Web Server Management menu automatically synchs
with our hosting platform (once you provide your email address and hosting password), so you just
need to select the desired hosting project as the target for uploading the study (one click, instead of
seven cut-and-pastes).
34 SSI Web v8
Smarter CAPI Data Merging If a new CAPI file is merged into an existing data set, SSI Web
resolves any duplicate respondent numbers by renumbering records with collisions. The problem
occurs if you try to accumulate the same CAPI file again (perhaps after new records have been
added). With previous versions of SSI Web, this would result in duplicate records (SSI Web would
renumber records again with collisions and add them to the data set). SSI Web v8.2 now
remembers the original ID numbers for CAPI respondents, so that it can distinguish respondents who
had been previously merged (but with renumbered IDs) versus those that are completely new and
need to be merged.
New Test Survey Option: Disable Terminate/Link Allows you to turn off Terminate/Link
functionality while testing your questionnaire. This allows you to test screener questions more
easily, because you can back up from the disqualification terminate page (rather than having to start
a new test respondent record each time you purposefully pick a path that disqualifies the test
record). This feature also lets you preview redirect links, such as those to a panel provider, before
the redirect executes.
Toggle Intelliprompt SSI Web's text editor color highlights recognized script functions,
autocompletes end-tags for HTML, and prompts you with syntax instructions regarding recognized
functions. When you click the Pencil icon you'll find two new icons in the right-hand corner of
the toolbar. The gear icon accesses a menu where you can turn off color highlighting and
prompting for certain scripting languages. Use the icon to turn off the autocomplete for HTML
endtags.
Anchored Scaling Standard MaxDiff scores are relative measures of utility. Anchored scaling
allows the researcher to establish a threshold anchor utility (often set to zero) that has more
concrete meaning. All item scores are then estimated with respect to that anchor. The anchor can
refer to the threshold between important vs. not important or a buy vs. no buy. Version 8.2 provides
capabilities to automatically insert dual-response indirect scaling questions (often referred to as
"Louviere" indirect anchor questions, after the inventor of MaxDiff, Jordan Louviere, who suggested
this as a possible anchoring mechanism). You may also use direct anchoring (often referred to as
"Lattery" direct anchor questions, after Kevin Lattery, who has suggested their use) by adding
ratings or rankings questions within your SSI Web questionnaire (using "CiW" question types), then
referring to these during utility estimation to establish anchor thresholds. (Note: we've also updated
our Online MaxDiff Analyzer service so that the TURF procedure can use the anchor utility as the
threshold for counting "reach".)
Logit and Latent Class Integration SSI Web v8.2 integrates aggregate (pooled) logit estimation
and latent class segmentation estimation within the MaxDiff Analysis menus. Aggregate logit can
be useful when the MaxDiff data are too sparse and not appropriate for HB estimation (the number of
times each item is shown to each respondent is too few). Latent Class finds needs-based classes
(segments or groups) using MaxDiff data, where respondents are quite similar within classes on the
item scores and quite different between classes. The latent class integration is much more
convenient than exporting the data to the standalone Latent Class system. The integrated Latent
Class capabilities are given to all MaxDiff v8 users, so you no longer need to buy a Latent Class
license to get this functionality.
Covariates within HB Analysis for MaxDiff You can now apply covariates in your HB runs for
MaxDiff within the SSI Web menus (previously this was only available through the standalone CBC/
HB system). Covariates are segmentation variables (such as user groups, company size) or
continuous variables that might be related to preferences (such as from Likert ratings, constant
sums, or numeric questions that capture income or company revenue). When covariates are used
within HB estimation, individual respondents "borrow" information principally from the means of their
covariate cohorts rather than from the overall population means. Using too many covariates can
cause estimation to slow considerably. The use of covariates does enhance differences among
respondents (especially along the dimensions of the covariates), but the predictive validity of the
scores does not generally improve.
(1 of 12) etc. Counting Functions for MaxDiff Questions Starting with v8.2, SSI Web by default
inserts script functions into the Header 2 of your MaxDiff questions, so that a running tally is shown
to respondents (1 of 12), (2 of 12), etc. This helps respondents see their progress and have more
clear expectations regarding how many repeated MaxDiff tasks they need to complete. You may
delete this notification if you wish.
Updates for v8.1.0
In case you did not see the features we implemented in the earlier v8.1.0 release, it included the
following new features.
General Improvements:
Mobile Improvements We’ve improved the rendering of SSI Web surveys and the Online
Administrative Module on mobile devices. If we detect that the screen resolution is less than 800
pixels wide, we include a mobile CSS style sheet that makes the text and buttons more readable
and clickable on mobile devices. The Online Administrative Module also has a new mobile interface
that allows mobile users to monitor the progress of their surveys and change quotas more easily on
a small screen.
CBC Improvements:
Best-Worst CBC Question Format You now have the option to ask respondents which product
concepts are the best and worst within the task, in addition to the standard best-choice only format.
Integration of CBC Counts and Logit The Counts and Logit analysis that was previously available
only with the SMRT software is now built into SSI Web, on the Analysis menu.
Integration of CBC/HB and Latent Class If you are licensed to use CBC/HB (hierarchical Bayes)
and Latent Class, then you can run HB and Latent Class estimation as integrated analysis
operations within the SSI Web Analysis menu.
Concept Labels, Corner Labels You may insert fixed labels above the product concepts, such as
"Option 1" or "Alternative A", etc. Two additional corner labels are also available, in the upper-left
and lower-left cells within a CBC task.
Countdown We've automatically added a countdown script "(1 of 12)" etc. to the CBC task layout.
This countdown required two new script functions: CBCCurrentTask() and CBCTotalTasks().
New SSI Script commands CBCAttributeValue, CBCDesignLevelValue

36 SSI Web v8
Improved Individual-Level Designs for MaxDiff Previous versions of the experimental designer
were nearly perfect in across-version level balance; but within-version (within-respondent) level
balance was often not as good as it could have been. The new designer obtains nearly perfect
across- and within-version level balance, leading to small improvements in utility estimation when
using HB.
ACBC Improvements:
ACBC Get Prices Routine This routine, which previously was a free tool from our website, is now
built into the ACBC Counts report. It tabulates how many times each unique summed price was
shown. This tabulation is useful when you are deciding on cut-points to use within a piecewise price
specification.
HB estimation when attributes are dropped using constructed lists. We've improved the way
that HB estimates utilities when some attributes are dropped by respondents (using constructed
lists and questions prior to the ACBC section). Prior to v8.1.0, if a large proportion of the
respondents dropped an attribute as unimportant, the minority of respondents who highly valued the
attribute could be unduly influenced by the rest of the population regarding the importance of that
attribute. Starting in v8.1.0, ACBC initially treats missing attributes as missing at random (no
information in the design matrix) and sets a respondent's utilities to zero for a dropped attribute at
the very end of the estimation process, when point estimates are summarized and the data are
written to the database.
Updates for v8.0.0
SSI web v8.0.0 makes some very big changes to the infrastructure of SSI Web, which carry over to all
aspects of using SSI Web (CiW and conjoint-related components). Many additional features were also
added. Here is a summary.
Side-by-Side Install Version 8 may be installed on the same machine as earlier versions. Previously,
only one version of SSI Web could exist on a machine at the same time. This makes it easier to adopt
the new version while still finishing up previous studies that were started under older versions of SSI
Web.
Looping We've added the ability to ask a question (or series of questions) multiple times, for each
element of a list. Loops within loops are also supported.
Data Management, Export Multiple Jobs We have made dramatic improvements to SSI Web in the
data management and export area. Multiple "export tables" (lists of questions to be exported) can be
saved as different "jobs," so you can manage multiple export formats and lists of questions to be
exported. Filters can be used when exporting. And, a new view/edit area allows you easily to see and
edit the data similar to how you might view or edit data in a standard spreadsheet table.
Database One of the biggest new features in v8 is the change to storing the data in a database file on
the server. This is more industry-standard and should hopefully resolve a few data corruption issues that
occasionally have occurred in previous versions of SSI Web. Using a database also allows power users
to write specialized data reporting tools (e.g. dashboards) that read the data base on the server.
However, if you have been using your own server to collect data for SSI Web, you'll now need to make
sure it supports a database (MySQL and Microsoft SQL are supported).
Crosstabs in Admin Module Previously, only one question could be tabulated at a time and no cross-
tabulation display or charting was available. Now, you can create pre-specified reports involving multiple
questions to be tabulated, that employ both cross-tabulation and charting.
Ability to have multiple instances of SSI Web open while copying-and-pasting questions from
one project to another (without having to open and close each instance of SSI Web, as previously).
This makes it even easier to copy questions from one study to another, or to maintain a large reference
questionnaire that serves as a questionnaire library.
Semantic Differential This is a new question type where respondents use a sliding scale (from left to
right) to make a choice between two words, such as "friendly" vs. "unfriendly". Both sliders and radio
button scale options are supported.
Open-end export enhancement Previously, open-end answers could be exported as grouped by

question. Now, you can also export open ends grouped by respondent.
Tooltip This is a new SSI Scripting Function that lets you more easily insert a Tooltip. A Tooltip is
associated with either text or graphics and displays a pop-up message when respondents mouseover the
text or object.
Don't Know/Refused code allowed in Numeric Sometimes, researchers want to allow respondents
to type a value outside of the normal range into a numeric question to indicate Don't Know/Refused. A
new feature in SSI Web v8 allows you to specify that value.
Progress Bar update We updated this area to include the option of tracking progress based on how
many questions have been seen out of the total number of questions. You additionally can specify what
% of the total number of questions you expect each respondent will see. This is helpful so that the
progress bar gives more consistent progress reports to respondents if you are doing randomized blocks
or looping. We also have created a new ProgressBarSet() command that lets you override and set the
progress bar for any page in the survey.
SetValue/GetValue Functions New SetValue and GetValue functions let you save new variables equal
to respondent answers or other scripting logic and retrieve (get) those values in later pages of the
questionnaire.
Recode lists During data export, you can re-assign the numeric values that are associated with list
items. This makes it easier to create data exports that satisfy the particular demands of your client.
Mootools to JQuery We have changed the Javascript library used by SSI Web from Mootools to the
more industry-standard JQuery.
Internal list labels When you create lists in SSI Web, you can now associate abbreviated labels with
each list element, for use in data labels and other data analysis areas. This is especially helpful if your
list elements include Javascript code or HTML syntax that you don't want to be used in the data labels
during the reporting phases.
Copy and paste lists out of SSI Web Previously, lists of items couldn't be copied to the clipboard and
pasted into other programs. Now, lists of items may be copied and pasted elsewhere.
Paste from Word button has been replaced with a Paste list member(s) from the
38 SSI Web v8
clipboard button . V8 supports copying and pasting lists from within SSI Web and from lists
outside SSI Web in the same way, so there is no need for a special button to "Paste from Word."
Number in front of lists in Word are stripped when copied into SSI Web There are times when the
lists of items we are pasting from Microsoft Word into SSI Web have index numbers in front of them.
This version of SSI Web strips those leading indices when pasting list elements into SSI Web.
Skip/quota logic export and import from .csv Sometimes the skip or quota logic is extensive and
you want to be able to export/import those instructions from a .csv file. SSI Web v8 now permits that.
CBC attributes increased to 250 for Advanced Design Module Some alternative-specific designs
could benefit from more than 30 attributes, so v8 now supports up to 250 attributes when using the
Advanced Design Module. There are also other interesting advanced applications (such as Menu-Based
Choice studies) that can benefit from having an experimental designer that can handle up to 250
attributes.
CBC conditional pricing and conditional display no longer limited to 4 attributes Previous
versions of SSI Web allowed you to create conditional pricing and conditional display elements that were
conditional on no more than 4 other attributes. This limitation has been lifted with v8.
Time per task now (optionally) included in CBC export of .cho file As in pre-SSI Web versions of
the CBC software, this information is for your info only and is not used in any utility calculations.
MaxDiff Export Option Includes ability to export the position numbers of selected items, which can be
useful for identifying straightliners and other patterns of responses.
Note: More detailed version information including bug fixes can be accessed on the Sawtooth Software
website at http://www.sawtoothsoftware.com/ssi-web-version-history
1.3 Composing Web Surveys

When you click Compose | Write Questionnaire, the Write Questionnaire dialog is displayed:
This dialog is the main workhorse within SSI Web. It provides a list of the current questions in your
study, shows question types for each and has many buttons that lead to other useful functions. You add
questions to your project using the Add... button, which leads to the Add to Questionnaire dialog that
lets you specify the question name and type. When you highlight a question on the list, the page
number this question appears on is shown in the footer of the dialog.
When you right-click a question name, a menu appears that include many of the options shown on the
dialog above, as well as the ability to rename the question.
40 SSI Web v8
Re-ordering Questions
You can re-order questions within the list by highlighting them and clicking the Up Arrow and Down Arrow
icons directly at the right of the questions list. You can also re-order questions by cutting and pasting
questions to a new location (see next section for further details).
Cutting/Copying Questions within and across Studies
You can cut or copy questions within the same study, or to another SSI Web study. To copy questions,
highlight a single or multiple questions (by holding down the Ctrl or Shift keys while selecting with the
mouse) and click the Copy button (or press Ctrl + C). The question(s) are stored in the buffer.
Then, place your curser within the current question list or in the question list for another SSI Web study
(you may open another SSI Web study within another instance of SSI Web). Click the Paste button
(or press Ctrl + V) and a copy of those question(s) are pasted into the target questionnaire. The
questions are automatically renamed to resolve duplicate labels, by incrementing a number at the end of
the question name.
Add...
Adds a new question directly following the question that is currently highlighted in the List of
Questions.
Add Page Break

Adds a page break directly following the question that is currently highlighted in the List of
Questions.
Edit...
Edits (lets you modify) the question that is currently highlighted in the List of Questions. You can
also double-click a question within the list to edit.
Delete
Deletes the question that is currently highlighted in the List of Questions.
Skip Logic...
Leads you to a dialog in which you can specify skip logic, to control the flow of the questionnaire.
Randomize...
Lets you add randomization of pages or questions on a page.
Looping...
Lets you ask a question or series of questions multiple times, for each member of a list.
Save
Saves the current changes to your study.
Close
Closes this dialog.
Print...
Prints a copy of the question list, along with the number of data fields used in each question.
Survey Settings...
Leads you to a dialog in which you can specify a number of global settings that control the look
and functionality of your survey, including fonts, global styles, colors, and global headers and
footers.
Lists...
Accesses the List Manager, for managing lists of items used in questions.
Questionnaire Access and Passwords...

Lets you add respondent passwords, or respondent data to be merged with the dataset once a
respondent enters the survey. Also lets you specify variables to be "passed in" (via the URL) when
the respondent clicks a link to access the survey.
Check for Errors...

Checks your questionnaire for logical and other errors.
Preview...
This option lets you preview how your questionnaire looks when it is posted on your web server.
When you click this button, you can select whether to view the entire questionnaire, the highlighted
question(s) on the list, or the entire page containing the highlighted question. The page(s) of your
survey are saved within the /temp subdirectory off of your study directory as TempWeb1.htm
through TempWebN.htm, where N is the number of pages in your study. TempWeb1.htm
automatically is opened in your default web browser. When you click the "Submit" button at the
bottom of each page, it links to the next page in sequence.
If you want to share a printed copy of survey with colleagues, you'll want to use the File + Print
Study functionality rather than Preview....
The TempWebX.htm files are only updated/modified when you click the Preview... button.
Changes you make to your questionnaire are not reflected in the TempWebX.htm files unless you
click the Preview... button.
Test the Survey Locally...

Uses the local test server to test your survey. Uses your default browser to display the questions.
All skip patterns, question verification, randomizations are executed.
42 SSI Web v8
1.3.1 Composing Questions

1.3.1.1 Adding Questions to Your Survey
When you click Add... from the Write Questionnaire dialog, the Add to Questionnaire dialog is
displayed:
The dialog lets you specify the question name and type to add to your survey.
Question Names and Types
Question names can include up to 24 letters or numbers. They cannot begin with a number and cannot
include spaces. They also cannot include a reserved name (the software will report whether you have
used a reserved name and ask you to make a new selection).
Available question types are:
Constant Sum
Adds a question in which respondents provide a series of numeric values that sum to a certain
value, or sum to the value provided in a previous question.
Free Format
Lets you compose the HTML to define most every aspect of a question or series of questions.
This is a powerful area for users who know HTML. One can compose special question
layouts not supported by the standard templates for CiW question types.
Grid
Lets you compose Select, Numeric, Constant Sum, Ranking, and Open-end questions in a
grid format (usually multiple rows and/or columns).
Numeric
Adds a numeric question. Numeric questions are text boxes in which respondents type
numeric values.
Open-end
Adds a text box question, in which respondents provide a written answer to an open-ended
question. Responses can be short (a few words), or long (many sentences).
Quota Questions
Lets you evaluate whether respondents have qualified for a quota bucket based on previous
question(s). This quota "question" is not really a question in the traditional sense (it doesn't
require any response), but is a qualification point at which you write logic to decide whether a
respondent has qualified to take the survey or not.
Ranking
Adds a question in which respondents can rank multiple items, from "1" meaning best to "n"
meaning worst.
Select
Lets you add a select-type question. Examples include radio button (select one from a list),
drop-down combo boxes, and select all that apply (check-box).
Semantic Differential
This question type allows respondents to rate items on a sliding scale, where a list of
descriptors is shown on the left and another list on the right. For example a given brand may
be shown and respondents are asked to rate the brand on a scale, where the left side of the
scale is labeled "unfriendly" and the right side "friendly." Radio buttons or sliders may be
used.
Terminate/Link
Adds a section of text or HTML instructions that requires no user input, that either ends the
survey or links respondents to another web site or survey.
Text/HTML Filler
Adds a section of text or HTML instructions that does not require any response.
Available exercise types are:
ACA, ACBC, CBC, CVA, and MaxDiff

If you are licensed to use these analytical tools, you can add these exercise(s) to your study.
The control parameters and attribute list associated with each analytical exercise determines
the number of questions and question types that are added for that exercise. Multiple
exercises (even of the same type) may be added to the same SSI Web study.
44 SSI Web v8
Copy from Question of Similar Type
If you want to add a new question to the questionnaire that is very similar or identical to another question
within this same survey or within another SSI Web project, you can save a lot of time by copying that
question and giving it a new name.
You can create copies of questions by highlighting a question or questions from the Write
Questionnaire dialog, clicking the Copy Question icon (or pressing Ctrl + C), placing your cursor
in the list of questions of the target survey (either the current or a different SSI Web survey) and clicking
the Paste icon (or pressing Ctrl + V). If copying files to a different SSI Web survey, just open
another instance of SSI Web. After copying the question(s) to the clipboard, navigate to the second
survey and paste the questions within the question list.
If you copy a question named ABC within the same questionnaire, SSI Web automatically names the
copy ABC1 and a second copy (produced by clicking the Paste icon a second time) is called ABC2, etc.
You can rename the copies of questions by editing the question and clicking the Rename button.
1.3.1.2 Setting Page Breaks

There are a number of options for organizing your questions in your survey. You can display each
question on a separate page in your survey. Or, you can organize questions into groups displayed on
separate pages.
You could also put all questions on just one page, but there are a number of reasons to compose your
questions across multiple pages:
1. Too many questions on a single page can make for a very long and clumsy form.
2. Response verification for numeric ranges or combo boxes would only occur at the very end
of the survey. Respondents might have a difficult time reviewing such a long list of questions to
discover their errors.
3. Respondents might look ahead to sections of the questionnaire that you'd rather they didn't
see until the proper time.
4. Respondents who wanted to quit a survey early and resume at a later time would lose all
their work. If you organize your survey into separate pages, respondents who quit early only
lose the questions they've answered since clicking the last Submit button (assuming each
respondent has a unique password with a quota of 1, or that you ask SSI Web to place a
cookie on the respondent's machine).
We suggest that you organize your survey questions in logical groups displayed on separate pages.
You might discover that some questions (such as conjoint questions, or questions requiring verification)
are best shown one at a time, even though this means more back-and-forth between the server and the
respondent's PC.
Setting Page Breaks and Previewing Pages
1. Highlight the question that you want the page break to follow.
2. Click Add Page Break and a page break is inserted after the highlighted question.
Multiple consecutive page breaks (without any questions defined on the separate pages) are not
permitted. You are asked to delete the redundant page breaks when you compile your questionnaire or
prepare it for running on the web server or local server.
Clearing Page Breaks
1. Highlight the page break you wish to delete.
2. Click the Delete button.

46 SSI Web v8
1.3.1.3 The Start Question

Every SSI Web questionnaire includes a Start "question" that may not be deleted. The Start question
can serve as an introductory screen where you provide text and/or graphics (additional questions can
also be added to the same page). The Start question can also include password fields, where
respondents type passwords to access the survey. Many SSI Web users assign respondents
passwords but bypass the Start screen using “one-click” access to the survey. This technique uses a
hyperlink (usually in an email message) that passes the necessary login information directly to SSI
Web's main processing script, ciwweb.pl. If you are letting respondents access the survey without a
user name or password, the Start screen serves only as a welcome/instructional page.
The Start question is always on the first page in the List of Questions on the Write Questionnaire
dialog.
You use the Questionnaire Access and Passwords dialog to compose either introductory text and/or
password login fields. You access this screen by highlighting the Start question on the list of questions
and clicking Edit.... Changes to the layout of the Start question are reflected in the login.html file that is
eventually placed on the server for data collection.
Start Question Layout
The Start question has four main sections:
1. Header 1
2. Header 2
3. Password Fields (optional)
4. Footer
If you use passwords, they are indented according to the Question Indent Width you specify in the
Survey Settings | Page Format dialog.
The Questionnaire Access and Passwords dialog has four buttons: Preview, OK, Cancel, and
Advanced.
Advanced... This button accesses advanced settings for this question.
Preview Lets you preview how the Internet browser displays the current question.
OK Saves changes to the current question and returns to the Write

Questionnaire dialog.
Cancel Returns to the Write Questionnaire dialog without saving changes to the
current question.
You specify header and footer text by typing directly into the Text fields within the dialog or by Clicking
the "pencil" icon to edit the text within a larger window. You can include HTML instructions in any
of the Text fields to enhance your surveys with additional styles or graphics. The toolbar within the text
editor offers a number of buttons to automatically add HTML instructions within your survey.
You can modify the Size, Color, Justification, and also control Bolding, Italics, and Underlining by
clicking the "paintbrush" icon .
Example: Composing a Start Page
In this example, we'll assume you are not using respondent passwords. The use of respondent
passwords is described later in the Respondent Passwords section of this documentation.
1. While at the Write Questionnaire dialog, highlight the <Start> question in the List of
Questions by clicking it with the mouse.
2. Click Edit....
(Skip Header 1)
3. Header 2:
Text: Welcome to our survey. Please click next to continue.
4. Click Preview:
Notes: You can include other questions on a Start page besides password fields. However, if respondent
passwords are being required and a respondent restarts the survey, the respondent must completely fill
out the Start screen (including the additional variables) prior to continuing. If the respondent enters
different values on the password screen for these additional questions than he/she answered before, the
respondent cannot jump forward in the survey to where he/she left off. The respondent must complete
the entire survey again. Questions involving randomization cannot appear on the first page.
48 SSI Web v8
1.3.1.4 Select Questions

Select questions are very common in survey research. Three general types of Select questions are
offered:
1. Radio Button (single response)

2. Check Box (multiple response)
3. Combo Box (single response)
To add a Select question to your survey, click the Add... button from the Write Questionnaire dialog.
The new question is inserted directly after any highlighted question in the List of Questions. If no
question is highlighted, the new question is inserted at the end of the list.
The Add to Questionnaire dialog is displayed. Choose the Select question type. Type a question
name in the Question Name field. Question names can include up to 24 letters or numbers (no spaces
or underscores permitted, and cannot start with numbers). Click OK. The Select Question dialog is
displayed.
Select Question Layout
The Select question has four main sections:
1. Header 1
2. Header 2
3. Question
4. Footer
The Question text line appears directly above the response options. The Question text line is indented
according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The
response options are indented a bit further relative to the Question text line.
The Select Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel.
Rename Lets you rename the current question.
OK Saves changes to the current question and returns to the Write Questionnaire
dialog.
current question.
Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text
using the paintbrush icon.

Choosing the Type of Select Question
You choose the type of Select question by selecting (under the Settings tab) from a Radio Button (single
response), Check Box (multiple response) or Combo Box (single response).
Radio Button (single response):
When using a Radio Button select question, you also specify on the Settings tab how many columns to
use in presenting the response options (the example above formats the response options in a single
column) and whether to require a response.
Check Box (multiple response):
When using a Check Box select question, you also specify on the Settings tab how many columns to
use in presenting the response options (the example above formats the response options in a single
column) and the minimum and maximum number of boxes that respondents are allowed to check.
Combo Box (single response):
When using a Combo Box select question, you also specify on the Settings tab whether a response is
required and what text to display as the default (the example above uses "(Click Here)" as the default
text).
Specifying Response Options
Next, you specify response options by clicking the Response Options tab. When you specify response
options for a select-type question, you are creating/using a "predefined" list of response options that can
be used in this question and in other questions. This predefined list has a name (up to 24 characters,
starting with a letter). By default, the list name is QuestionNameList, but you can change this name if
you wish.
1. From the Response Options tab, select an existing list of items, or add a new list of items by
50 SSI Web v8
clicking the Add... button.
2. If adding new items, type the list item in the space provided.
3. Click OK when finished and the list item is added.
Tip: If you are adding many response options at the same time, you can save time by pressing ENTER
twice after typing each list item. Pressing ENTER twice is equivalent to clicking OK and then clicking
Add....
Time-Saving Hint: You can cut-and-paste lists of response options from Microsoft Word or Excel by
highlighting the array of items in Word or Excel, copying to the clipboard using Ctrl+C or the Copy button
in Word or Excel (or another text editor), and then (within SSI Web) by clicking the Paste list member(s)
from the clipboard icon.
A numeric index is displayed at the left of your list items for reference, but is not displayed as part of the
question.
Click OK to save your changes and to return to the Select Question dialog.
Formatting Response Options
On the Settings tab, the number of response options you specified for this question is displayed in the
Number of Response Options field (which you cannot edit). Response options can be formatted in a
single column, or over multiple columns by specifying a value greater than 1 in the Number of Columns
field. For example, if you have 11 response options (say, colors) and specify four columns, the response
options are distributed as follows:
Which of these colors is your favorite?
1) Red 4) Blue 7) Violet 10) Magenta

2) Purple 5) Yellow 8) Brown 11) Gray
3) Pink 6) Green 9) Orange
Other Specify Responses
Often, researchers need to provide an "other specify" option, that includes a open-end text field for typing
a response.
SSI Web automatically inserts an open-end text field next to a response option when you indicate that a
response item is a Respondent Specify ("Other Specify"). When adding list items under the Response
Options tab, highlight the item that is to include an other specify entry field. Check Respondent Specify
("Other Specify") and use the Settings... button to indicate if the respondent is required to specify text if
they click this item and the minimum and maximum characters permitted.
The open-end data for the other specify response are saved as a new text variable in the database,
called QuestionName_Item#_other, where QuestionName is the name of the question and Item# is the
item's number on the list (natural order). More than one list item may be an "other specify" for a
question.
Please note that "Other Specify" open-end response boxes are not supported with combo box select
questions. It would seem much more appropriate to use a radio button layout if including "other
specifies."
Typed other specify responses can be carried forward as response options to later questions if using
constructed lists.
Exclusive ("None of the Above") Responses
When check-box (multiple response) questions are used, researchers often need to specify that an item
is "exclusive," meaning that if it is chosen, no other items are valid. The classic example is the "None of
the above" response item. When adding list items under the Response Options tab, highlight the item
that is exclusive. Check Exclusive ("None of the Above"). More than one list item may be "exclusive"
for a question.
Randomizing Response Options
To randomize response items for a question, from the Response Options tab, click Randomize
Response Options. This randomizes all items except for any Other Specify or "None of the Above"
options at the bottom of the list. When using this method to randomize options, information regarding
the specific order shown to each respondent is not saved in the data.
If you need additional flexibility for randomizing a subset of the range of items beyond that provided with
this simple dialog, you have great power and flexibility when constructing lists and using the Randomize
command. When you randomize list items using constructed lists, information is saved in the database
regarding the specific order shown to each respondent.
Re-ordering, Copying and Pasting Response Options
The "up" and "down" arrow icons can be used for moving the selected response option up or down in the
list.
You can also highlight a single or multiple items and cut or copy-and-paste the items to a new position
on this list, or to a response list for another select-type, ranking, or constant-sum question. Highlight the
item(s) to be moved or copied, then click the Copy or Cut icons. Highlight a new place on the list (or
within a response list for a different question). Click the Paste icon.
52 SSI Web v8
Referencing Select Questions in SSI Script

Radio and Syntax:
Combo boxes QuestionName
Examples:
Q1
[% Q1 %]
Description:
You refer to single-response (radio) button using the question name. For example,
[% Q1 %] returns the numeric response associated with a radio button select
question named Q1.
Check Boxes Syntax:

QuestionName_Response#
Examples:
Q1_4
[% Q1_4 %]
Description:
For multiple-response questions, each response option is either checked (value=1)
or un-checked (value=0). You refer to each response option within a multiple-
response question using underscore "_" notation. For example, [% Q1_4 %]
returns a "1" or a "0" depending on whether the fourth response option in multiple-
response question Q1 was checked.
Example: Composing a Radio Button (Single Response) Question
1. Click Add from the Write Questionnaire dialog.
2. Select the Select question type.
3. Type a valid name in the Question Name field, such as Education, then click OK.
4. From the Settings tab, click Radio Button (single response).
5. Click the Response Options tab. Add the following six response categories by clicking
Add... and pressing ENTER twice after typing each response option:
Some high school

Completed high school
Some college
Completed college
Some graduate studies
Completed advanced degree
Click the Question Text tab.
6. Header 1:
Text: Please indicate the highest degree of education you attained:
7. Click Preview:
Typing a 2 into the Number of Columns field of the Settings tab spreads the response options
across two columns.
Tip: Try adding optional HTML instructions in Header 1 to italicize "highest" and add a horizontal line
beneath the question:
Header 1 Text: Please check the <i>highest</i> degree of education you attained:
Footer Text: <hr>

Notes: You can italicize the word "highest" by using the toolbar accessible when you click the "pencil"
edit icon . Highlight the word "highest" with the mouse and then click the italics button (looks like
an italics I ). You can also add a horizontal line by editing the Footer text (again using the "pencil" edit
icon) and clicking the "horizontal rule" button (looks like a dash "—")
Example: Composing a Check Box (Multiple Response) Question
3. Type a valid name in the Question Name field, such as Music, then click OK.
4. From the Settings tab, click Check Box (multiple response).
5. Click the Response Options tab. Add the following seven response categories by clicking
Add... and pressing ENTER twice after typing each response option:
Country
Classical
New Age
Jazz
Soft Rock
Hard Rock
Klingon Opera
6. Header 1:
Text: Which types of music do you enjoy listening to?
7. Header 2:
Text: (Select all that apply)
8. Click Preview:
Note: When using the multiple-select check box, you can require respondents to check a minimum and
maximum number of items (from the Settings tab). (These can be specified using SSI Script)
54 SSI Web v8
Example: Composing a Combo Box (Single Response) Question
3. Type a valid name in the Question Name field, such as Party, then click OK.
4. From the Settings tab, click Combo Box (single response). Specify the Default Text to be used in
the combo box is "(No Response)"
5. Click the Response Options tab. Add the following response categories by clicking Add... and
pressing ENTER twice after typing each response option:
Democrat
Independent
Republican
None of the above
When the combo box is displayed, the default text you specified on the Settings tab is always shown
and selected by default. Respondents choose a category by clicking the arrow to reveal the drop-down
list of response options. If you check Require Response from the Select Question dialog, respondents
must select a category other than the default option before proceeding to the next page in the survey.
6. Header 1:
Text: Which political party do you most closely align with?
7. Question:
Text: (Click the down arrow and choose an option)
8. Click Preview:
Tip: You can use combo boxes for collecting numeric responses. Simply type numbers for the
response categories. Combo boxes can make sure that answers conform to certain values when
respondents answer the question, rather than requiring the verification that occurs for the numeric
question type when respondents click the Submit button.
For example, a question might ask respondents how many people are in their family. The seven
response options (not including the default option) might be: 1, 2, 3, 4, 5, 6, 7 or more.
1.3.1.5 Numeric Questions

Numeric questions let respondents type a numeric answer from -99999999 to 999999999 into an answer
field. First we'll describe how to compose Numeric questions and then we'll provide a simple example for
you to try.
When you add a numeric question to your survey, the new question is inserted directly after any
highlighted question in the List of Questions on the Write Questionnaire dialog. If no question is
highlighted, the new question is inserted at the end of the list.
The Numeric Question dialog is used for formatting numeric questions.
Numeric Question Layout
The numeric question has four main sections:
1. Header 1
2. Header 2
3. Question
4. Footer
The Question text line with its answer field is indented according to the Question Indent Width you
specify in the Survey Settings | Page Format dialog.
The Numeric Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel.
Advanced... This button accesses advanced settings for this question, including the width of the
question's input box.
Numeric Question Settings
From the Settings tab, you can control the range of values that can be entered by specifying a Minimum
Value and a Maximum Value (SSI Script may also be used). You can also specify a legitimate value
outside the Min and Max range to allow respondents to indicate they Don't Know/Refuse. The value
constraints are not enforced until the survey runs on the server.
There are three main controls for the numeric question:
Require Response
Indicates whether respondents are required to enter a valid response before continuing the
survey.
Allow Decimal Answers
Controls whether answers with a decimal point are accepted as valid answers. You can
specify whether commas or periods are to be used as decimal delimiters under the Survey
Settings | Advanced dialog.
56 SSI Web v8
(Note: Up to 15 decimal places of precision will be saved to the data record. Also, the
Tabulation program within the Admin Module truncates all answers with decimal points,
treating them as integers, prior to computing frequencies, min, max, and means.)
Don't Know/Refused Value
Let's you specify a value outside the Minimum and Maximum allowed range, so respondents
can indicate that they answer the question as Don't Know/Refused. For example, you might
ask respondents to indicate their age is "999" if they do not want to type their age.
Question Width (pixels)
Specifies how wide the entry box should appear to respondents. A narrow box still allows
respondents to type as many digits as they want. However, not all the digits may be able to
be shown at the same time if the box is too narrow.
Position Question <Left/Right> of Input Box
Determines whether the text in the Question text area appears at the left or right of the
numeric entry box.
Referencing Numeric Questions in SSI Script
Numeric Syntax:
Questions: QuestionName
Examples:
Q1
[% Q1 %]
Description:
You refer to numeric questions using the question name. For example, [% Q1 %]
returns the numeric response associated with Q1.
Example: Composing a Numeric Question
2. Select the Numeric question type.
3. Type a valid name in the Question Name field, such as AGE, then click OK.
4. Click the Settings tab. Specify a Minimum Value of 18 and a Maximum Value of 99 by
clicking and editing those fields. Specify a Don't Know/Refuse value of 999.
5. Header 1:
Text: What is your age?
(Skip Header 2)
6. Question:
Text: Age:
7. Footer:
Text: (If you really don't want to tell us your age, just type 999)
8. Click Preview:
Tip: Try adding optional HTML instructions in the Footer to bold the "999" and add a horizontal
line beneath the question:
Footer Text: (If you really don't want to tell us your age, just type <b>999</b>)<hr>
Notes: You can bold the "999" by using the toolbar accessible when you click the "pencil" edit icon
. Highlight the "999" with the mouse and then click the bold button (looks like a bold B). You can
also add a horizontal line by editing the Footer text (again using the "pencil" edit icon) and clicking the
"horizontal rule" button (looks like a dash "—")
58 SSI Web v8
1.3.1.6 Open-End Questions

Open-end questions let respondents type any characters (including letters and numbers) into an answer
field. You can create either fixed entry fields (single line) or free entry fields (multiple lines) that give text
scrolling capabilities when typing long text strings.
First we'll describe how to compose Open-end questions and then we'll provide a simple example for you
to try.
When you add an open-end question to your survey, the new question is inserted directly after any
highlighted question in the List of Questions of the Write Questionnaire dialog. If no question is
The Open-end Question dialog is used for constructing open-end questions.
Open-end Question Layout
The open-end question has four main sections:
1. Header 1
2. Header 2
3. Question
4. Footer
The Question text line with its answer field is indented according to the Question Indent Width you
specify in the Survey Settings | Page Format dialog.
The Open-end Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel.
Open-end Question Settings
From the Settings tab, you can specify options that control how the open-end box looks and operates.
You can specify whether to Require Response, whether the entry box has a single or Multiple Lines.
You can specify the Maximum and Minimum Number of Characters required, as well as the Width and
Height of the box (in pixels).
Referencing Open-End Questions in SSI Script
Open-End Syntax:
Questions: QuestionName
Examples:
Q1
[% Q1 %]
Description:
You refer to open-end question using the question name. For example, [% Q1 %]
returns the typed response for Q1.
Example: Composing a Single Line Open-end Question
2. Select the Open-end question type.
3. Type a valid name in the Question Name field, such as FirstName, then click OK..
Question names can include up to 24 letters or numbers (no spaces or underscores permitted,
and cannot start with numbers).
4. From the Settings tab, uncheck Multiple Lines. Specify a Maximum Number of Characters
of 15.
5. Header 1:
Text: What is your first name?
(Skip Header 2)
6. Question:
Text: Name:
(Skip Footer)
7. Click Preview:
Example: Composing a Multiple Line Open-end Question
2. Select the Open-end question type.
3. Type a valid name in the Question Name field, such as VanStyle, then click OK..
4. From the Settings tab, specify to use a width of 400 (pixels) and a height of 100 (pixels).
5. Header 1:
Text: What do you like or dislike about the styling of the Summit minivan?
6. Header 2:
Text: (Type your response below)
(Skip Question)
(Skip Footer)
60 SSI Web v8
7. Click Preview:
1.3.1.7 Text/HTML Filler Questions

Sometimes you may want to add transitional text that does not require any response. You can specify
text "questions" that stand alone on a survey page, or that separate other survey questions. Note that
Text/HTML filler questions don't limit you to just text. You can include graphics or tables using HTML
instructions.
Text/HTML Filler questions do not count toward the data field limit specified by your software license.
No data are saved for Text/HTML Filler questions.
When you add a Text/HTML Filler question to your survey, the new question is inserted directly after any
highlighted question in the List of Questions on the Write Questionnaire dialog. If no question is
Formatting Options
You use the Text Question dialog to format text questions.
Text Question Layout
The text question has four main sections:
1. Header 1
2. Header 2
3. Body
4. Footer
The Body text line with its answer field is indented according to the Question Indent Width you specify in
the Survey Settings | Page Format dialog.
The Text Question dialog has four buttons: Rename, Preview, OK and Cancel .

62 SSI Web v8
1.3.1.8 Grid Questions

Introduction to Grid
General
Headers and Footers
Rows and Columns
Grid questions can include any of the standard question types used in SSI Web surveys (numeric, radio,
check-box, combo-box, constant sum, ranking, or open-end), but Grid questions give you much greater
power to format them in new ways and within a unified table.
The most common use of the grid question is to provide multiple rows and columns for labels and inputs,
such as this array of radio buttons:
The main response area for this grid question has 4 rows and 5 columns (not counting the rows and
columns for labels). There are four separate radio button variables, one variable for each row. This grid
question uses four data fields toward your license capacity. The "Question Direction" is by rows. The
alternating colors by rows help respondents orient themselves to the question direction.
Grid questions can sometimes be challenging for new users to author. When you add a Grid question to
your survey, an intuitive Grid Wizard is available to help you set up the essentials of the Grid. The key to
using the main Grid interface efficiently is to always keep in mind whether the question direction is by
rows (as in the previous example) or columns (as illustrated below).
This grid question requires eight data fields to store a respondent's answers ("checked" vs. "not
checked" for each box).
Grid questions also can include numeric or text entry boxes and rankings or constant-sums. They also
can include combinations of different question types within the same grid.
There are many options within the Grid area. We suggest you review the help text for each area, but
especially that you experiment with the various settings and click Preview to see the results.
As you learn to use Grid questions, you may find that the increased formatting options lead you to use
Grid questions to compose standard questions that don't involve multiple rows or columns.
The Grid question area has six main tabs (Question Text, Grid Text, Format, Rows, Columns and Sk ip
Logic). The Rows and Columns tabs have near-identical functionality, depending on whether the
Question Direction is by rows or columns. Therefore, we describe their functionality in one common
documentation area.
If the Question Direction is by Rows, then additional controls for adding new variables to the Grid are on
the Rows tab. If the Question Direction is by Columns, then you add and edit variables from the
Columns tab.
Introduction to Grid
General
Headers and Footers
Rows and Columns
Referencing Grid Questions in SSI Script
Numeric, Open- Syntax:

end, Combo, GridName_r#_c#
and Check
boxes Examples:
Q1_r2_c4
[% Q1_r2_c4 %]
Description:
You refer to single-response field in a grid using the grid name plus row and column
reference. For example, [% Q1_r2_c4 %] returns the value associated with row 2
column 4 in grid Q1.
For check boxes, the value is "1" (checked) or "0" (not checked).
Radio button Syntax:

questions GridName_r#
GridName_c#
Examples:
Q1_r2
Q1_c4
[% Q1_r2 %]
[% Q1_c4 %]
Description:
64 SSI Web v8
Radio buttons in grids can use either row-based or column-based question

directions. The most common layout is the row-based question direction. With row-
based question direction, the responses are stored as Q1_r1, Q1_r2, etc. With
column-based questions, the responses are stored as Q1_c1, Q1_c2, etc.
1.3.1.9 Constant Sum Questions

Constant sums are used to let the respondent allocate, say, 100 points across multiple items. Common
uses include amount of time spent in various activities, or amount of money spent on various items. You
can also ask respondents to allocate numbers that sum to a particular value that the respondent provided
in an earlier question.
Constant sum questions require a data field to be stored for each response option. Therefore, if
respondents are to allocate points across 10 response items, this uses 10 data fields toward your
license capacity.
Constant Sum Question Layout
The Constant Sum question has four main sections:
1. Header 1
2. Header 2
3. Question
4. Footer
styles you apply (color, bold, size, etc.) in the Question row also apply to the response options.
The Constant Sum Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel.
dialog.
current question.
Specify response options by clicking the Response Options tab. When you specify response options for
a constant sum question, you are creating/using a "predefined" list of response items that can be used
in this question and in other questions. This predefined list has a name (up to 24 characters, starting
with a letter). By default, the list name is QuestionNameList, but you can change this name if you wish.
66 SSI Web v8
Add....
highlighting the array of items in Word or Excel, pasting to the clipboard using Ctrl+C or the Copy button
in Word or Excel, and then (within SSI Web) by clicking the Paste list member(s) from the clipboard
icon .
question.
Click OK to save your changes and to return to the Constant Sum Question dialog.
list.
item(s) to be moved or copied, then click the Copy or Cut icons. Move the cursor to a new place on the
list (or to a response list for a different question). Click the Paste icon.
Constant Sum Settings
Total
Type a value such as "100" into this field. This is the amount that the numeric entry items
should sum to. (You can provide a permissible range around the total, such as plus or minus
5, by clicking the Advanced... button on this dialog.) You can also specify [%
GetValue(questionname)%], where questionname is a name of a previous numeric question
in your survey. In that case, the value the respondent typed earlier in the questionnaire is
used as the required sum.
Allow Decimals
Specify if respondents are allowed to use decimals.
Number of Columns
Lets you arrange the response options across multiple columns.
Require Response to Question

Specifies whether respondents are required to answer the question.
Require Response to each individual input box

Use this field to specify whether respondents must supply a value for every response option of
the multi-part constant sum question.
Include a "Total" at the bottom

If you want a running total to be displayed within the question, check this box. Also specify a
label to be placed next to that field. The totals field appears as a grayed-out text box that
updates as respondents enter values in the other fields in the constant-sum question. The
totals value is not stored in the database.
(In the Error Messages tab under Survey Settings..., if respondents do not fill out the question
properly, you can display messages to the user regarding the current sum and the required
total. ErrTotal( ) returns the required total, and ErrCurSum( ) returns the current sum.)
Referencing Constant Sum Questions in SSI Script
Constant Sum Syntax:

Questions: QuestionName_Response#
Examples:
Q1_4
[% Q1_4 %]
Description:
For constant-sum questions, each response option can contain a numeric value.
You refer to each response option within a constant-sum question using underscore
"_" notation. For example, [% Q1_4 %] returns the numeric value typed into the
fourth response option in Q1.
68 SSI Web v8
1.3.1.10 Ranking Questions

Ranking questions are useful if you need respondents to rank multiple items from best to worst. Two
response formats are possible: combo box or numeric entry. In either case, respondents mark the best
item as "1" and the worst item as "K", where K is the total number of items to be ranked. Each value
must only be used once. An error message prompts respondents if the ranking question is not
completed properly.
Ranking Question Layout
The Ranking question has four main sections:
1. Header 1
2. Header 2
3. Question
4. Footer
styles you apply (color, bold, size, etc.) in the Question row also apply to the response options.
The Ranking Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel.
Advanced... This button accesses advanced settings for this question, including customized
error text for the respondent.
dialog.
Cancel Returns to the Write Questionnaire dialog without saving changes to the current
question.
Specify response options by clicking the Response Options tab. When you specify response options for
a ranking question, you are creating/using a "predefined" list of response items that can be used in this
question and in other questions. This predefined list has a name (up to 24 characters, starting with a
letter). By default, the list name is QuestionNameList, but you can change this name if you wish.
Add....
in Word or Excel and then (within SSI Web) by clicking the Paste list member(s) from the clipboard icon
question.
Click OK to save your changes and to return to the Ranking Question dialog.
list.
item(s) to be moved or copied, then click the Copy or Cut icons. Move the cursor to a new place on the
list (or to a response list for a different question). Click the Paste icon.
Ranking Settings
Ranking Format
Select whether to use Numeric input Boxes or Combo Boxes (drop-down control). If you use
combo boxes, integers 1 through n are automatically provided. You can only use these preset
integers within combo boxes in a ranking question. If you want to use other characters or
words, such as "best, middle, worst", you have that flexibility if specifying a ranking question
within a grid.
Default Text (combo boxes only)

70 SSI Web v8
Text to display when the respondent has not ranked an item yet.
Require Response to Question

Specifies whether respondents are required to answer the question.
Number of Items to Rank

Indicate whether respondents are required to rank all items, or a range of items (such as
ranking only the top three, or ranking from three to five items). A question is counted toward
your license capacity for each response option that is ranked.
Note: If you allow respondents to rank a range of items (e.g. from three to five items), the
default error text ("Rank Count") for Rank ing questions on the Survey Settings + Error
Messages tab will not be appropriate. You may update that error message and customize
message for the current rank ing question using the Advanced button on the edit Rank ing
Question dialog. You may use ErrMin() and ErrMax() SSI Script instructions to retrieve the
minimum and maximum range values.
Combo Box Settings (only applicable if Ranking Format is Combo Box)

Specify the default text such as "click here" to display in the combo box for ranking.
Formatting Response Options
On the Settings tab, the number of response options you specified for this question is displayed in the #
Response Options field (which you cannot edit). Response options can be formatted in a single column,
or over multiple columns by specifying a value greater than 1 in the Number of Columns field. For
example, if you have 11 response options (say, colors) and specify four columns, the response options
are distributed as follows:
Which of these colors is your favorite?
1) Red 4) Blue 7) Violet 10) Magenta

2) Purple 5) Yellow 8) Brown 11) Gray
3) Pink 6) Green 9) Orange
Referencing Ranking Questions in SSI Script
Ranking Syntax:
Questions: QuestionName_Response#
Examples:
Q1_4
[% Q1_4 %]
Description:
For ranking questions, each response option contains a numeric value. You refer to
each response option within a ranking question using underscore "_" notation. For
example, [% Q1_4 %] returns the numeric rank value for the fourth response option
in Q1.
1.3.1.11 Semantic Differential
This question type asks respondents to evaluate an object (such as "Anderson Grocery Stores" in the
example below) on a sliding scale, where the scale is framed using pairs of opposing adjectives, such as
"Clean" and "Dirty."
The scale may use sliders (as shown in the example above) or radio buttons.
Along the top of the question are scale anchors ("Strongly Describes" and "No Opinion") indicating the
strength of preference toward the left or right. These scale anchors are optional and may be specified
using a list (pre-defined or constructed). The scale anchors may be assigned a one-to-one
correspondence to the scale positions of the slider/radio buttons. Alternatively, the scale anchors do not
need to correspond one-to-one with the slider/button positions. In the example above, even though it
appears that there are three main positions for the scale (from "Strongly Describes" left to "Strongly
Describes" right), the slider/radio buttons could be made to allow the respondent to select more or fewer
than three values along the scale.
Semantic Differential Question Layout
The Semantic Differential question has three main sections:
1. Header 1
2. Header 2
3. Footer
The Semantic Differential Question dialog has five buttons: Rename, Advanced..., Preview, OK and
Cancel.
Advanced... This button accesses advanced settings for this question, including customized
error text for the respondent.
dialog.
72 SSI Web v8
Cancel Returns to the Write Questionnaire dialog without saving changes to the current
question.
Response Options Tab
Left and Right Lists
Semantic Differential questions require two lists: a list to show in the left side of the scale and a list
for the right side of the scale. Both lists must have the same number of list members (or parent list
members). A constructed list may be used as either the left- or right-side lists. Both cannot be
constructed lists. The non-constructed list maintains a one-to-one association with the constructed
list. For example, if the left-side list is constructed and it contains items 1, 3, and 5, then the right-
side list also displays items 1, 3, and 5 from its list.
Scale Anchor Text
Here you (optionally) provide text that appears above the sliding scale. A pre-defined or constructed
list containing the text (list items) may be specified. The number of scale anchors you supply need
not have a one-to-one correspondence with the number of scale points the respondents use to answer
the question.
Saved Values
Use Scale Anchor text List Values as Stored Values If this option is checked, a one-to-one
correspondence between scale anchor items is established with the response scale. The values
stored in the database correspond to the parent list item numbers for the specified Anchor Text list.
Custom Saved Values If you want to specify a degree of granularity to the scale that may be
different from the number of Anchor Text points and to store the respondent's answer using a
custom range of values, check this option. You can specify any values (or use SSI Script). For
example: Left Side Value = -10 and Right Side Value= 10 leads to 21 scale positions. The furthest
left position is stored as -10 and the furthest right position as 10.
Notes: These Custom Saved Values may be positive or negative. The number for the left side
may be greater than or less than the number on the right side; but they may not be the same
number. Decimals are not allowed.
If a list is chosen for saved values, the values are simply the list item numbers. This means a
constructed list that is brought in with items <1, 7, 5> would record a 7 in the database if the
middle point on the scale is chosen.
Row Randomization
You can randomize the order of the rows in the Semantic Differential question using this button. But,
if using this option on the Response Options tab, no information will be stored in the data record
regarding the exact order shown to each respondent. To randomize the order of rows and save
information about the order of items shown to each respondent, use a constructed lists instead as
either the Left-Side or Right-Side lists.
Column Randomization
Randomly Switch Columns Click here to randomly interchange all the left-hand items with the right-
hand items. This involves a whole scale interchanging of two columns of items. The data are
automatically stored as if no randomization occurred. No information is stored in the data record
regarding the order of presentation to each respondent.
Randomly Switch Individual Items Click here to randomly interchange the left-hand and right-hand
items, on a row-by-row basis. The data are automatically stored as if no randomization occurred. No
information is stored in the data record regarding the order of presentation to each respondent.
Require Response To Each Row
If this is checked, a user must either click on or move the thumb in the case of slider. With radio
button display, a button must be clicked for each row in the question.
When no response is required, if the thumb has not been clicked (or a radio button not checked) in a
row, no information is written to the data record. This is a missing response.
Format Tab
Question Format
This area lets you change aspects of how the question is displayed. The Question Width, Width of
Left and Right Columns (%), Cell Border, Cell Padding, and Alternating Colors are all pre-set by the
style you have selected (and no values appear in the boxes next to all these settings). To override the
style settings, you may enter numeric values in these boxes, or select colors, in the case of
Alternating Colors.
Response Format
Rather than sliders, you can choose radio buttons for the question.
Slider or Radio Buttons. Slider requires Javascript be enabled on the respondent's browser (which is
the default requirement). If Javascript is not enabled, then radio buttons will be displayed. Thus, you
should check that the Semantic Differential question looks good with both Slider or Radio Button
options.
Slider Settings
Handle Start Position (slider only)
This is the position where the slider handle (thumb) control is initially displayed. Options are Left,
Center, Right, Invisible and Custom. Invisible means that no slider control is initially displayed.
When the respondent clicks on the slider bar area, the slider control appears where clicked.
Custom allows you to specify a value on the scale for the starting position. Or, you can use SSI
Script to set a position based on a previous question, or even a random starting point.
Handle Graphic
74 SSI Web v8
The "thumb" icon for the slider may be customized with your own graphic (use a 22 x 22 pixel
graphic).
Notes:
If Javascript is turned off, an alternate entry method will be provided. If the scale values are tied to a list,
radio buttons will display. If it is tied to user-defined values, a text box will be displayed.
Referencing Semantic Differential Questions in SSI Script
Semantic Syntax:
Differential QuestionName_Response# , etc.
Questions:
Examples:
Q1_4
[% Q1_4 %]
Description:
For Semantic Differential questions, the variable is saved multiple times for each list
element used. You refer to each response associated with each item using
underscore "_" notation. For example, [% Q1_4 %] returns the response to the
fourth item in the parent list. The underscore index always refers to the original
parent list number rather than the actual order that was displayed to respondents.
1.3.1.12 Terminate/Link Questions

This question type is used to indicate that a respondent should stop at this point in the survey
(terminating question), or that the respondent should immediately *link to another survey or web site,
either produced by SSI Web or another software system. No other questions can appear on the same
page as a terminate/link question type.
*Note: It is a violation of the SSI Web license agreement to link multiple SSI Web surveys together such
that the total resulting survey exceeds the data fields supported by your license.
Terminating Question
When a respondent reaches a Terminate/Link question marked as "terminate," the survey is over.
Reaching a terminating question prohibits respondents from backing up to change answers to previous
questions. A terminating question must appear on a survey page by itself. Most survey authors include
at least one, and sometimes multiple terminating questions in their surveys, where each is formatted on
a separate page. A common use of the terminating question is to terminate a respondent who is not
eligible to complete the survey, given his/her answers to previous questions. You can have more than
one terminating question in your survey.
To make the current Terminate/Link question a terminating question, click the Settings tab and check
the Terminate Respondent box. Specify whether respondents reaching this question are Qualified/
Complete (5) or Disqualified (4). You can additionally assign a disposition code for each terminating
question (a value from 0 to 999) that is saved in the respondent's data set. This is useful for tabulating
how respondents terminated the questionnaire, if there are multiple termination points.
While terminating respondents, you could choose to immediately link them to a web site or another
survey (rather than show the text for the current question on the screen). In that case, you would also
check the Link to a Questionnaire or Website box. This functionality is described further below.
Linking Question
SSI Web can link to other surveys or another web page using the linking question type*. The linking
question must appear on its own page and must have no other questions on that page (and it cannot
display text, as this question is not actually "shown" to respondents). When the page including the
linking question is encountered, SSI Web immediately links to the Web Address (URL) specified on the
Settings tab. That URL can be a web site, another program, or a login script to start up another survey.
(The page containing the link question is not shown, but signals to SSI Web to execute the link.)
Typically, you will want to terminate respondents and mark them with Qualified/Complete (5) status when
linking to another survey or website. To do so, check the Terminate Respondent box and use the drop
down control to specify the Termination Status. Once a respondent is marked as Qualified/Complete,
they cannot restart the current survey. If you are linking to an outside survey or program and plan to
have respondents re-enter the current survey to complete it, you should not check the Terminate
Respondent box.
When you link to other SSI Web surveys, the URL you use in the Settings tab specifies the ciwweb.pl
file (a Perl script) for the next survey to start up. That survey will most likely be located in a separate
folder on your server, or perhaps on a new server altogether (see below).
The typical approach is to bypass the login (password) screen for the linked survey and automatically
76 SSI Web v8
pass through the user name, password, or other information from the first survey. We'll describe how to
do this below.
URL link with Passwords
If you want to link to a new SSI Web survey and automatically "pass through" the user name and/or
password, you should link respondents directly to the Perl script located in the cgi-bin directory. This
lets qualified respondents start the survey without having to type in passwords or see the login page.
Let's imagine that we wanted to link to a new SSI Web survey called "tep1" installed on the
sawtoothsoftware.com server, located in the "gonzo" study root directory, with valid passwords named
myusername of "michael" and mypassword of "A6Z99". In the Web Address (URL) field on the
Settings tab, we would specify:
http://www.sawtoothsoftware.com/gonzo/cgi-bin/ciwweb.pl?
studyname=tep1&myusername=michael&mypassword=A6Z99
When linking from an SSI Web survey to another SSI Web survey, to dynamically insert the respondent's
passwords (named, as an example, myusername and mypassword), use [%myusername%] and [%
mypassword%], for example:
http://www.sawtoothsoftware.com/gonzo/cgi-bin/ciwweb.pl?studyname=tep1&myusername=[%
myusername%]&mypassword=[%mypassword%]
You can also pass additional variables into the data set, as long as those are defined as pass-in fields:
studyname=tep1&mypassword=michael&zip=98370
In this example tep1 is the study name, a password called mypassword is used and the zip code for
each respondent is already known and passed into the database. You can pass in as many additional
variables as you want, as long as they are initially defined as pass-in fields in the passwords area and
the length of the URL doesn't exceed that supported by browsers.
When respondents are linked to the new survey, if the passwords are valid (and the quota is not
complete), the respondent is sent to the page following the password screen. If the respondent is
restarting an interview, the page where he/she left off is displayed.
Dealing with Special Characters in URLs
Data that include special characters should use the SSI Script function EncodeForUrl( ). This function
will encode non-alphanumeric characters in the ASCII table between 32 and 126.
For example, email addresses contain the @ symbol. If you need to pass an email address as the
username, it should be passed in the following manner:
http://www.sawtoothsoftware.com/gonzo/cgi-bin/ciwweb.pl?studyname=tep1&myusername=[%
EncodeForUrl(emailaddress)%]
To be safe, you could always use the EncodeForUrl( ) function when using SSI Script to include a
variable within your URL.
*Note: It is a violation of the SSI Web license agreement to link multiple SSI Web surveys together
such that the total resulting survey exceeds the data fields supported by your license.
78 SSI Web v8
1.3.1.13 Free Format Questions

1.3.1.13.1 Introduction
Warning:
Because of the complexities of the Free Format Question type, you must completely test your
survey to ensure that it functions properly and that the data for your entire survey are being
saved correctly. We do not provide support to help you troubleshoot your own HTML and
JavaScript.
Response Types
Currently the Free Format Question allows you to use any of the following types of variables:
RADIO
CHECKBOX
SELECT (Combo Box)
TEXT (Numeric)
TEXT (Text Box)
TEXTAREA
HIDDEN
Introduction
The SSI Web system provides pre-set templates for you to construct commonly used question types.
Most users find these adequate to compose their surveys. However, some projects (or clients) may
require that you create a certain look/layout with a question type that our templates don't allow. For
such instances, the Free Format question type can provide those who know HTML near total control of
the look of a question or entire survey pages containing many questions. For example, you can create
complex "grid" questions, with combinations of numeric, open-end, check-box or radio button responses.
To see an example of the possibilities, please see the "Free Format" study that was installed in your /
Samples directory.
You must be familiar with HTML to use the Free Format question type. See http://www.w3schools.com/
html for a simple HTML tutorial.
As the name of the Free Format question suggests, you supply your own HTML code to display the
question and input controls. Like all other question types, you can specify a Header 1, Header 2, and
Footer text (if you are applying Styles to your survey, you may wish to utilize these question sections for
better uniformity of look across both your standard and your free format questions.)
SSI Web still provides the <form> tag and its parameters. But, most everything that you would put within
(but not including) the <body></body> tags is up to you (with the exception of the "submit" button
control). Free Format question HTML code cannot have any <html>, <body>, or <form> tags (like all
CiW questions, SSI Web generates these tags).
If you are applying Styles in your questionnaire (most surveys do), your HTML must include proper CSS
"hooks" for the question to look like the others within your survey.
Adding a Free Format Question
A Free Format question can contain a single, or multiple questions (variables). If using multiple
variables, they do not necessarily have to be all of the same response type. You can include
combinations of numerics, combo boxes, radio buttons, check boxes and open-end text fields within the
same Free Format question. To add a Free Format question to your survey, click the Add... button from
the Write Questionnaire dialog. The new Free Format question is inserted directly after any highlighted
question in the List of Questions. If no question is highlighted, the new question is inserted at the end of
the list.
The Add to Questionnaire dialog is displayed. Choose the Free Format option. Type a question name
in the Question Name field. Question names can include up to 24 letters or numbers (no spaces or
underscores permitted, and cannot start with numbers).
Please note that the name given to the Free Format question plus an underscore character that SSI Web
supplies acts as a prefix to all of the variables you define within this particular Free Format question.
For example, you may wish to add a multi-part numeric question that has three response blanks. If you
name your Free Format question "FF1", you later assign three variable labels with FF1_ as a prefix (e.g.
"FF1_A", "FF1_B"," FF1_C"). By requiring you to provide a unique prefix name for Free Format
questions, SSI Web can help you make sure you do not accidentally give two different variables within
your survey the same name, resulting in potential data loss.
Click OK. The Free Format Question dialog is displayed, with three tabs: Question Text, Variables/
Question HTML, and Sk ip Logic.
Question Text Tab
Like all other question types, you can specify a Header 1, Header 2, and Footer text. Even though free
format questions let you specify all the HTML for your question (and you could specify all the HTML
within the Variables/Question HTML tab), you may wish to type text/HTML into these three different
sections so that if you apply Styles, your free format questions will carry many of the same style
formatting as your other standard SSI Web questions.
Variables/Question HTML Tab
The Variables/Question HTML tab is divided into two parts. The lower section contains a text box where
you compose your HTML code. (HTML can also be written in a separate editor and then copied and
pasted into this section.) The upper section contains a list box of Free Format variables. Remember, a
Free Format question actually can contain many sub-questions (Free Format Variables). The Free
Format Variables you add to the Variable Names list must correspond exactly to the input tags and
variable names you write within your HTML section (including distinction between upper and lower case).
(If you use the Free Format Controls icon this helps ensure that your variable names match.) Each
Free Format Variable counts as a CiW question against your license limitation.
Free Format Variables
Free Format Variables must have unique names that begin with the name of the Free Format question.
When specifying the name of a variable, the Free Format question name is made the first part of the
variable name automatically. It is very important that the full variable name be exactly the same as the
name parameter for the corresponding input tags in your HTML code section (including case).
80 SSI Web v8
1.3.1.13.2 Radio
Warning:
saved correctly.
General Information about Free Format Questions
The RADIO variable is used for asking the respondent to select a single option from a set of response
options. Within the Free Format Variable dialog, specify a Free Format Variable with RADIO as the
Variable Type. SSI Web relies heavily on styles and default behavior is to use graphical radio buttons.
This increases the polish of the SSI Web questionnaire, but it raises the bar for the sophistication of
HTML that should be supplied when writing free format questions.
You may optionally change the type of data that are stored by using the drop-down box on the Free
Format Variable dialog to change from the default data type (Whole Number) to Decimal or Text.
Radio Buttons in Free Format
By default, SSI Web questionnaires use graphical radio buttons (specified under Survey Settings,
General Format tab). The HTML used by SSI Web to display radio buttons is more complex than the
standard HTML used to display radio buttons. You can use View Source within your browser to examine
(and copy) the syntax SSI Web uses when displaying standard Select-type questions. But, a simpler
solution is to use the Free Format Tools button that you access when editing the HTML section of
Free Format questions using the icon. It inserts template HTML, including the use of the SSI
Web function called RadioSelect.
When you click to add Radio buttons, a dialog appears that lets you specify vertical, horizontal, or no
table. The table provides a structure for displaying the radio buttons. Depending on the layout of your
radio button question, you'll select one of these options. We recommend you add the template structure
and click the Preview button to see which layout best fits what you want to accomplish.
The name attribute for each <input> tag must be the name of the variable (all radio <input> tags should
have the same name attribute). The VALUE attribute for each <input> tag should be set to a unique
value, so you can determine which option the respondent selected (we recommend sequential integers
starting with one).
1.3.1.13.3 Checkbox
Warning:
saved correctly.
The CHECKBOX variable is used for presenting a set of check boxes in your survey. Within the Free
Format Variable dialog, specify a Free Format Variable with CHECKBOX as the Variable Type. Even
though the response to each check box is stored independently as a unique variable name, you do not
specify at this point each variable name associated with each check box. Specify the Number of
Response Options, or the number of check boxes you have in the set. You may also specify the Min
Checks and the Max Checks to force the respondent to check a certain number of check boxes (within
the range you specified).
Within the HTML editing area, specify an <input> HTML tag with the type set to "checkbox" for each
check box in the set. The easiest way to do this is by using the Free Format Tools button . When
using the Free Format Tools button, SSI Web automatically provides default HTML with appropriate
variable naming conventions.
Checkbox Select in Free Format
By default, SSI Web questionnaires use graphical radio buttons/checkboxes (specified under Survey
Settings, General Format tab). The HTML used by SSI Web to display checkboxes is more complex
than the standard HTML used to display checkboxes. You can use View Source within your browser to
examine (and copy) the syntax SSI Web uses when displaying standard Select-type questions with
checkboxes. But, a simpler solution is to use the Free Format Tools button that you access when
editing the HTML section of Free Format questions using the icon. It inserts template HTML,
including the use of the SSI Web function called CheckSelect.
When you click to add checkboxes, a dialog appears that lets you specify vertical, horizontal, or no
table. The table provides a structure for displaying the checkboxes. Depending on the layout of your
checkbox question, you'll select one of these options. We recommend you add the template structure
and click the Preview button to see which layout best fits what you want to accomplish.
The name attribute for each <input> tag must be the name of the variable (all radio <input> tags should
have the same name attribute). The VALUE attribute for each <input> tag should be set to a unique
value, so you can determine which option the respondent selected (we recommend sequential integers
starting with one).
82 SSI Web v8
1.3.1.13.4 Select (Combo Box)

Warning:
saved correctly.
To display a combo box type input within your Free Format Question, you define a Free Format Variable
with SELECT (Combo Box) as the Variable Type.
Within the HTML editing area, specify a <select> HTML tag in your HTML code. The easiest way to do
this is by using the Free Format Tools button . When using the Free Format Tools button, SSI Web
automatically provides default HTML with appropriate variable naming conventions.
The name of the Free Format Variable must be the same as the name attribute in your HTML code for
the <select> tag (including case).
Checking Required Response requires a respondent to choose an option other than the first visible
default option. Be sure to change the Default_option text between the <option> tags. Not checking the
Required Response allows the respondent to skip this question.
The MULTIPLE (multiple input list box) attribute is not yet supported in SSI Web.
SELECT (Combo Box) Example
(The following parameters represent a Free Format Question with one variable defined.)
Free Format Question Name: FF1

Variable Name: FF1_COLOR
Variable Type: SELECT (Combo Box)
Error Message String: Favorite Color
Reject Default Response: (checked)
Example HTML Code:
Please select your favorite color: <BR>

<SELECT NAME="FF1_COLOR">
<OPTION SELECTED VALUE="">Choose One</OPTION>
<OPTION VALUE="1">Red</OPTION>
<OPTION VALUE="2">Green</OPTION>
<OPTION VALUE="3">Blue</OPTION>
</SELECT>
Note that we assigned numeric values to the response options. This permits you to tabulate responses
within the Data Reports in the online administrative module. If "Require Response" is selected, SSI Web
verification will not let the "Choose One" option to be selected.
1.3.1.13.5 TextArea
Warning:
saved correctly.
The TEXTAREA variable is used for asking longer, open-ended text questions. Within the Free Format
Variable dialog, specify a Free Format Variable with TEXTAREA as the Variable Type.
Within the HTML editing area, use a <TEXTAREA> HTML tag in your HTML code. The easiest way to
do this is by using the Free Format Tools button . When using the Free Format Tools button, SSI
Web automatically provides default HTML with appropriate variable naming conventions.
the <TEXTAREA> tag (including case). If desired you can check the Require Response option and then
specify the minimum number of characters you require the respondent to enter. SSI Web forces the
respondent to enter at least the minimum and no more than the maximum number of characters before
continuing to the next question.
TEXTAREA Example

Variable Name: FF2_WORKEXP
Variable Type: TEXTAREA
Require Response: (checked)
Error Message String: Work Experience
Minimum Characters: 30
Example HTML Code:
Please tell us about your past work experience: <BR>

<TEXTAREA NAME="FF2_WORKEXP" ROWS="3" COLS="35">
</TEXTAREA>
(Regardless of the length permitted in the textarea box, responses are trimmed to a maximum of 10,000
characters to maintain database integrity.)
84 SSI Web v8
1.3.1.13.6 Text (Text Box)

Warning:
saved correctly.
The TEXT (Text Box) variable is used for asking shorter text questions. Within the Free Format Variable
dialog, specify a Free Format Variable with TEXT (Text Box) as the Variable Type.
Within the HTML editing area, use an <input> HTML tag with the type attribute set to "text." The
easiest way to do this is by using the Free Format Tools button . When using the Free Format Tools
button, SSI Web automatically provides default HTML with appropriate variable naming conventions.
the <input> tag (including case). If desired, you can check the Require Response option and specify the
minimum and maximum number of characters you wish the respondent to enter (similar to the
TEXTAREA variable) before continuing.
TEXT (Text Box) Example

Variable Name: FF3_LASTNAME
Variable Type: TEXT (Text Box)
Error Message String: Last Name
Minimum Characters: 2
Example HTML Code:
Last Name: <BR>

<INPUT TYPE="text" NAME="FF3_LASTNAME" SIZE="100">
*Note: The INPUT tag has a SIZE attribute that determines the size of the text box. It also has a
MAXLENGTH attribute that limits the maximum number of characters that can be typed into the box.
Regardless of the number of characters specified, responses are trimmed to a maximum of 10,000
characters to maintain database integrity.
1.3.1.13.7 Text (Numeric)

Warning:
saved correctly.
The TEXT (Numeric) variable is used for asking numeric only type questions. Within the Free Format
Variable dialog, specify a Free Format Variable with TEXT (Numeric) as the Variable Type.
Within the HTML editing area, use an <input> HTML tag with the type attribute set to "text." The
easiest way to do this is by using the Free Format Tools button . When using the Free Format Tools
button, SSI Web automatically provides default HTML with appropriate variable naming conventions.
the <input> tag. If desired, you can check the Require Response option, that forces respondents to
enter a numeric value. You must specify a Minimum Value and a Maximum Value for the variable. SSI
Web requires the respondent's answer to be within that specified range (or the value may take on the
Don't Know/Refused value, if you specify one). If you would like the respondent to be able to enter
decimal type values, then check the Allow Decimal Answers check box.
TEXT (Numeric) Example

Variable Name: FF4_CARS
Variable Type: TEXT (Numeric)
Error Message String: Number of Cars
Minimum Value: 0
Maximum Value: 10
Allow Decimal Answers: (not checked)
Example HTML Code:
How many cars do you currently own?

<INPUT TYPE="text" NAME="FF4_CARS" SIZE="2" MAXLENGTH="2">
(Note: Up to 15 decimal places of precision will be saved to the data record.)

86 SSI Web v8
1.3.1.13.8 Hidden Variables

Example #1: Setting the hidden variable equal to a random number used in the survey.
Suppose you declared a random value earlier in the survey (such as "RandNum(53)") and wanted to
store the value of that random number in your data set.
1. Add a Free Format question to your survey and a variable (e.g. myvar1_rand) of type hidden.
2. Edit the HTML (using the "pencil" icon and using the Free Format Tools button select
myvar1_1 from the drop-down control list, to insert the following text in your HTML:
<input name="myvar1_rand" type="hidden">
Then, edit that text to assign the value equal to the function RandNum(53):
<input name="myvar1_rand" type="hidden" value="[%RandNum(53)%]">
Example #2: Setting the hidden variable equal to the sum of two numeric variables on a previous screen.
Assume the two previous numeric variables were Value1 and Value2.
1. Add a Free Format question to your survey and a variable (myvar1_sum) of type hidden.
2. Edit the HTML (using the "pencil" icon and select the Free Format Tools button to
insert the following text in your HTML:
<input name="myvar1_sum" type="hidden" value="[%Q1+Q2%]">

1.3.1.14 JavaScript in SSI Web

JavaScript in SSI Web
JavaScript Introduction
JavaScript Basics
Custom JavaScript Verification
SSI Web JavaScript Library
JavaScript Debugging
JQuery Tips
JavaScript Introduction
JavaScript is a scripting language that adds enhanced functionality and programming power to a web
page. JavaScript code can verify that the input on a web page is correct before submitting it to the server.
It can be used to do calculations on user input. Together with CSS, it can be used to change the
appearance of the web page in real time. It is important to understand that JavaScript runs in the web
browser, not on the server.
SSI Web makes extensive use of JavaScript. It is used to help validate responses to questions, fill in
Constant Sum total boxes, and to un-check select options when a "None of these apply" is chosen, etc.
Most of SSI Web's system JavaScript is contained in ssi_javascriptX_X_X.js and
ssi_admin_javascriptX_X_X.js located in the graphics/system directory.
SSI Web includes the jQuery JavaScript library. More information about this library can be found at
http://jquery.com.
For a complete tutorial on JavaScript see http://www.w3schools.com/js.
JavaScript Basics
How to Include Custom JavaScript in SSI Web

Variables
Flow Control
Functions
Form Objects
How to Include Custom JavaScript in SSI Web
JavaScript can be included in an SSI Web survey almost anywhere text can be entered. For example,
you could place some JavaScript code in the header of a question or as part of a response option. To
include JavaScript on a page in SSI Web, you enclose JavaScript code in <script> </script> tags. For
example, suppose you wanted to display a message in a pop-up box on the first page of your survey.
You could place this text in Header 1 of the first question:
<script type="text/javascript">

88 SSI Web v8
</script>
The JavaScript code between the <script> tags is executed when the page is loaded. In this case, as
soon as the web page is loaded in the browser, a popup box appears with the message "Welcome to the
survey". Any JavaScript that needs to be available to the whole survey can be included in the HTML
<head> Tag (Survey Settings, Headers and Footers).
JavaScript can be included as attributes of HTML tags. For example:
<a href="#" onclick="alert('hello');">Click Here</a>
When the link above is clicked, a message box with the message "hello" appears.
JavaScript can also be included in a separate file so that it can be accessed by multiple pages in your
survey. This file can then be placed in your graphics folder and included with the following syntax:
<script type="text/javascript" src="[%GraphicsPath()%]CustomJavaScriptFile.js"></script>
The text above should be placed in the HTML <head> Tag (Survey Settings, Headers and Footers).
Variables
Variables in JavaScript can be declared with the "var" statement. For example:
var NumDaysInWeek = 7;
var MonthName = "March";
After this code is executed NumDaysInWeek will contain 7 and MonthName will contain "March". Notice
that quotes are used when assigning text to a variable.
To retrieve the value of a question on the current page you can use SSI Web's built-in
SSI_GetValue(QuestionName) JavaScript function. Given any question name (or variable name) this
function will return the value entered for that question on the current page. For example:
var Q1Value = SSI_GetValue("Q1");

var BlueChosen = SSI_GetValue("FavColors_1");
In the above example, the function SSI_GetValue retrieves the value for Q1 as entered on the page. It is
then assigned to the Q1Value variable. The next line retrieves the value for the first check box in a
Select question. BlueChosen is assigned a 1 if it is checked and a 0 otherwise.
To retrieve the response to a question from a prior page and use that value in your JavaScript, you should
use SSI Script in conjunction with JavaScript. For example, if NumWeeks was asked on a prior page of
the survey and you wanted to use its value to do a simple calculation and display the result on the
current screen, the following JavaScript could be used:
var NumDaysInWeek = 7;
var Weeks = [%NumWeeks%];
document.write(NumDaysInWeek * Weeks);
Before the web page comes down to the browser the SSI Script [%NumWeeks%] is changed to the
value that was entered on a prior page by the respondent. Lets assume that value was 4. The JavaScript
loaded by the browser is:
var Weeks = 4;
To load text from a prior question, use:
var Name = "[%LastName%]";
which when it arrives at the browser would be:
var Name = "Smith";
Arrays, which are variables that can hold a collection of items, are also available:
var ItemList = [1,2,3,4,5,6,7];

document.write(ItemList[0]);
In the example above, an array of 7 items is declared. The first item in the array (with value 1) is written
to the screen. Individual array items can be accessed with 0-based index numbers between [ and ]. So
[0] returns the 1st item, [1] the 2nd, and so on.
Flow Control
JavaScript "if" statements can be used to conditionally execute code:
var Age = SSI_GetValue("Q1");
if(Age < 18)

{
alert("You do not qualify for this survey.");
}
In the code above, the value entered for Q1 is assigned to Age. If Age is less than 18 a message box
appears.
If there are multiple paths that can be taken the "else if" statement can be used. For example:
var Age = SSI_GetValue("Q1");
if(Age < 18)

{
alert("You do not qualify for this survey.");
}
else if(Age < 65)
{
alert("You qualify for group 1.");
}
else if(Age >= 65 && Age <= 125)
{
alert("You qualify for group 2.");
}
90 SSI Web v8
else
{
alert("Are you sure you are older than 125?");
}
If the first "if" statement is false, the next "else if" logic is tested, and so on. If a section evaluates to
true, its code is executed. Execution flow then continues after the final "else" or end of the block.
The following symbols are used in conditional logic:

"&&" for AND
"||" for OR
"==" for equal
<, >, <=, >=, etc.
The "for loop" allows your code to loop for a given amount of iterations. For example:
var myArray = [1,2,3,4,5];

var i = 0;
var sum = 0;
for(i = 0; i < myArray.length; i++)

{
sum += myArray[i];
}
alert(sum);
The variable "i" starts at 0 and counts up to the length of the array (in this case 5). The code inside the
"for" loop block is executed 5 times. The code accesses each item in the array, storing the sum of all
the items in the "sum" variable.
Functions
Functions are sections of code that can be defined and used when needed by "calling" them. For
example:
function AddTwo (Num1, Num2)

{
var Result = Num1 + Num2;
return Result;
}
alert(AddTwo(4, 6)); //The value 10 will appear in a pop-up box.

alert(AddTwo(56, 33); //The value 89 will appear in a pop-up box.
In the code above, a function named "AddTwo" is defined. This function takes two numbers for input
named "Num1" and "Num2". Any two numbers that are passed into this function are added and the
result is returned.
Functions are best defined in the HTML <head> Tag (Survey Settings, Headers and Footers) or in a
separate file that is included in the HTML <head> Tag. This makes it so that the functions are available
throughout the whole study.
Form Objects
Each survey page created by SSI Web contains an HTML form tag named "mainform". All of the HTML
input elements for the questions on this page are inside this form. Using JavaScript, you can get access
to JavaScript objects that represent these form elements. You can use JavaScript to manipulate these
input elements by changing their properties and functions. Having access to these form elements gives
you access to their properties and functions. The following JavaScript code assigns a form object to a
variable:
var Q1Obj = document.mainform["Q1"];
This can also be done by calling an SSI Web JavaScript system function like this:
var Q1Obj = SSI_GetFormObject("Q1");
Now Q1Obj can be used to get and set values and to set up events etc. For example:
Q1Obj.value = 5;
Q1Obj.addEvent("keyup", SetToFive);
function SetToFive()
{
this.value = 5;
}
Here, we are assigning a 5 to the "Q1" numeric question. This causes a 5 to appear in the numeric box
on the screen. The next line of JavaScript then assigns an event to the question. If the respondent types
into the Q1 box every time a key is pressed and released ("keyup") then the function "SetToFive" is
called. In this function "this" represents "Q1Obj". This code essentially makes it so that the respondent
cannot alter the value for "Q1". The events "keyup", "click", "mouseover", "mouseout", etc. can be used.
To get a value from a question on the page, use SSI_GetValue("QuestionName"). This function handles
the complexities of different question types (radio, checkbox, numeric, etc.).
Custom JavaScript Verification
Warning:
This is an advanced area, intended for those who know JavaScript. You must completely test your
survey and custom JavaScript code to ensure that it functions properly and that the data for your
entire survey are being saved correctly. Sawtooth Software does not support your custom
JavaScript.
There might be times when our default question verification will not accomplish the functionality you
want. Most questions (with the exception of the password, Terminate/Link, and conjoint questions) have
a Custom JavaScript Verification section. This is available by editing the question, clicking on the
"Advanced" button, and then selecting the "Custom JavaScript Verification" tab.
Custom JavaScript Verification allows you to specify custom verification for your questions. Your custom
verification, written in JavaScript, is invoked when the respondent clicks the "Submit" button. You can
92 SSI Web v8
configure your custom JavaScript to be called "before" or "after" the built in question JavaScript
verification.
Your respondents must have JavaScript enabled (most do) in order for your custom JavaScript to work.
In the example above, an error message is displayed and respondents cannot continue the survey if they
have specified that they are older than their father.
The age of the respondent ("respondentAge") and of the father ("fatherAge") are asked on the current
page. These values have not been submitted to the server yet or recorded in the database. They are on
the same page where this custom JavaScript is executed. The values of questions on the same page as
the JavaScript that you are writing are available through the following JavaScript:
SSI_GetValue("QuestionName")
The JavaScript SSI_GetValue is defined in SSI Web's system JavaScript file ssi_javascriptX_X_X.js
located in the graphics/system directory.
In the example above if the value of "respondentAge" is greater than or equal to "fatherAge" then
"strErrorMessage" is set to a custom error message. Setting strErrorMessage equal to something other
than empty will cause the system JavaScript to display an error message on the screen and highlight the
current question. The current page will also not submit to the server as long as strErrorMessage has a
value assigned to it.
If multiple questions on a single page all have custom JavaScript verification, then each section of
custom JavaScript is run in order from top to bottom.
If you would like to retrieve information from previous pages you need to use SSI Script. If the
respondent's age was submitted on a previous page, the JavaScript would then employ SSI Script and
look like this:
if([%respondentAge%] >= SSI_GetValue("fatherAge"))
SSI Script executes on the server and the respondent's age (lets assume it is 37) is inserted. The
resulting JavaScript is sent down to the browser and looks like this:
if(37 >= SSI_GetValue("fatherAge"))
SSI Web JavaScript Library
SSI Web comes with a system JavaScript file named ssi_javascriptX_X_X.js. It is located in the
graphics/system directory and is included and available on every page of the survey. The system
JavaScript file is used by SSI Web to help validate responses to questions, fill in Constant Sum total
boxes in real time, and to un-check select options when an "Other Specify" is chosen etc. There are a
few functions from this library that you might find useful.
SSI_GetValue("QuestionName")
Returns the current value for the specified QuestionName, for any question type (radio, checkbox,
numeric, etc.). A zero ("0") is returned for questions that have been left empty.
Example:
var Num1 = SSI_GetValue("Q1"); // returns the value of question Q1
var FirstStateChecked = SSI_GetValue("State_1"); // If the first check box of question "State" is
checked 1 will be returned, otherwise 0.
SSI_GetFormObject("QuestionName")
Returns the object associated with the form element on the page for QuestionName.
Example:
var Num1Obj = SSI_GetFormObject("Q1"); // Assigns the form object for question "Q1" to
"Num1Obj".
Num1Obj.value = 46; // Assign the value of 46 to the Q1 question.
SSI_RoundNumber(Number, NumDecimalPlaces)
Round "Number" to "NumDecimalPlaces".
Example:
var Value1 = SSI_RoundNumber(SSI_GetValue("Q1"), 2); // SSI_GetValue returns the value for Q1
it is then rounded to 2 decimal places.
alert(Value1); // Assuming Q1 was 45.236 the Value1 would yield 45.24
SSI_PriceFormat(Number, "ThousandSymbol", "DecimalSymbol")

Formats a number with the thousands symbol and decimal symbol.
94 SSI Web v8
Example:
var FormatValue = SSI_PriceFormat(5123693.25, ",", "."); // 5,123,693.25 is returned.
SSI_CustomGraphicalCheckbox(GraphicalCheckboxObj, InputObj, blnCheck)
SSI_CustomGraphicalCheckbox is a function that has been provided so that you can write custom
code that gets called when a graphical checkbox is clicked. This function must be defined by you in
your custom JavaScript. The function will be called by SSI Web's system JavaScript each time a
check box is clicked.
GraphicalCheck boxObj represents the graphical item on the page that is being clicked. InputObj is
the actual hidden checkbox on the page that the graphical checkbox represents. blnCheck is true if
the checkbox has been checked, otherwise it is false.
Example:
function SSI_CustomGraphicalCheckbox(GraphicalCheckboxObj, InputObj, blnCheck)
{
if(InputObj.name == "mySelect_1" && blnCheck == true)
{
alert("The first box was checked");
}
}
SSI_CustomGraphicalRadiobox(GraphicalRadioObj, InputObj)
SSI_CustomGraphicalRadiobox is a function that has been provided so that you can write custom
code that gets called when a graphical radio button is clicked. This function must be defined by you in
your custom JavaScript. The function will be called by SSI Web's system JavaScript each time a radio
box is clicked.
GraphicalRadioboxObj represents the graphical item on the page that is being clicked. InputObj is the
actual hidden radio button on the page that the graphical radio button represents. When ever this
function gets called the radio button represented by InputObj will have been checked.
Example:
function SSI_CustomGraphicalRadiobox(GraphicalRadioboxObj, InputObj)
{
if(InputObj.name == "mySelect")
{
alert("A radio button with the value " + InputObj.value + " has been selected.");
}
}
SSI_SetSelect(strInputName, blnSelect)
SSI_SetSelect allows you to select (or check) a radio button or checkbox through JavaScript. This is
especially helpful with graphical radio buttons and checkboxes.
strInputName is the name of the input id for the particular HTML input item. For example to select the
radio button with list value 2 in question "Q1" strInputName would be "Q1_2". blnSelect should be
"true" to select the radio button or checkbox and "false" to de-select it.
Example:
SSI_SetSelect("colors_3", true); //Checks list item with value 3 in the colors question.
SSI_SetSelect("colors_5", false); //Un-checks list item with value 5 in the colors question.
SSI_RadioReset(strInputName)
SSI_RadioReset will de-select all of the radio buttons in the question defined by strInputName.
strInputName is the name of the select radio question or row or column if it is in a grid.
Example:
SSI_RadioReset("colors"); //De-selects all of the radio buttons for the "colors" question.
SSI_RadioReset("choices_r1"); //De-selects all of the radio buttons in the first row of the grid
question.
JavaScript Debugging
You are responsible to ensure your custom JavaScript is working. The following tips are highly
recommended when trying to debug your custom JavaScript:
Turn on script error reporting. This is done in Internet Explorer by clicking Tools | Internet Options |
Advanced. Under the Advanced tab go to the Browsing section and check Display a
notification about every script error. This is extremely useful in alerting you to mistakes in your
code.
Use the "alert( )" function. If you are unsure what certain values are at certain points in your code,
you can print them out to the screen by using the "alert( )" function. For example:
alert("My value at this point is: " + SSI_GetValue("Q1"));
Firefox has a plug-in named "Firebug" that provides a powerful JavaScript debugger.
Test your custom JavaScript in multiple browsers (e.g. Microsoft Internet Explorer, Firefox, etc.).
JQuery Tips
SSI Web uses the jQuery JavaScript library. This JavaScript library makes it simpler to interact with
HTML and to setup JavaScript events on the page etc. jQuery is included on every page that SSI Web
creates. This makes it possible to write custom JavaScript code that uses jQuery. For documentation on
jQuery see http://jquery.com
Creating a jQuery Object

One of the most powerful things about jQuery is the simplicity in which you can create a JavaScript
object from an HTML element on the page. For example lets say you have the HTML for a numeric
question on the page:
<input id="q1" class="text_input" name="q1" type="text" size="3">
If you wanted to interact with this HTML using jQuery you could use the following JavaScript syntax:
$("#q1")
96 SSI Web v8
The "$" sign (or alternatively "jQuery") is a function that returns a JavaScript object based on the selector
text that is passed to it. The selector text is basically CSS. The "#q1" above is the CSS code that you
would use to select the HTML that has an "id" equal to "q1". To select q1's Header 1 you could do this:
$("#q1_div .header1")
With this jQuery object you can now access the jQuery library. For example:
$("#q1_div .header1").html("Header 1 text");
will place the text "Header 1 text" into the header of "q1".
Running Code After the Page Loads

Often it is useful to call JavaScript code once the web page has loaded. To do this use the following
JavaScript syntax:
$(document).ready(function(){
alert("hello");
});
</script>
In the above example as soon as the page is loaded the JavaScript code alert("hello") will run.
Getting/Setting Values from Input Fields

To get or set a value to an input field use the "val" function. For example:
$("#age").val(55);
sets the value of age to 55 and
$("#age").val();
returns the value that is stored in "age".
If you need to get the actual JavaScript object (not the jQuery object) then you can do this:
$("#age")[0];
This is the same as:
document.mainform["age"]
Setting Up Events
Events are actions that happen on a web page such as a mouse click etc.. jQuery makes it easy to
write JavaScript code that can respond to those events. For example lets say that you wanted to change
the background color of a question when a respondent moves their mouse over the question. This would
give the question a highlight effect. Here is some sample code:
<style type="text/css">
.highlight_question
{
background-color: #EEEEEE;
border: 1px solid #c0c0c0;
}
</style>
$(document).ready(function(){
$(".question").mouseover(function(){
$(this).addClass("highlight_question");
});
$(".question").mouseout(function(){
$(this).removeClass("highlight_question");
});
});
</script>
First a CSS class of "highlight_question" is defined. This class has the settings of a light grey
background color and a grey border.
Next the jQuery JavaScript code is defined. The $(".question") grabs all of the HTML for the questions on
the page. For each of the questions on the page it applies the "mouseover" event. When the mouse is
moved over the question the "highlight_question" CSS class will be applied to the question. This is done
with the "addClass" command. The "this" keyword contains the current "question" object.
The code then defines the behavior for what should happen on the "mouseout" event. It removes the
"highlight_question" class when the mouse moves off the question.
98 SSI Web v8
1.3.2 How CiW License Sizes are Counted

CiW is licensed with a specific capacity, counted in terms of stored data fields (not questions). The
sizes are:
CiW/100
CiW/500
CiW/1000
CiW Unlimited
(If you purchase CBC, ACA, CVA, ACBC, or MaxDiff without an accompanying CiW license, you receive
a CiW/50 license capacity.)
Many question types use just one data field (variable) per question:
Numeric
Select (Radio button or Combo box)
Open-end
Some question types don't use any stored data fields:
Text/HTML Filler
Terminate/Link
Select-type questions with multiple responses use more than one data field (variable) per question, but
we count these as just one stored data field toward your license capacity. The example below stores
three data fields, but we count only one against your license capacity:
When grid questions are used, these can involve multiple data fields counted against your license. In the
grid question below, there are four separate radio button variables, one variable for each row. This grid
question below uses four data fields toward your license capacity.
This grid question below actually requires eight data fields to store a respondent's answers ("checked"
vs. "not checked" for each box). But, we count only two data fields toward your license capacity in this
case.
Looping has a huge effect on data field usage. If you loop over a question for each element of a list, then
the number of variables used is multiplied by the list elements in the list.
The use of constructed lists can also greatly impact your data field usage. Consider a list of colors,
such as:
1. Red
2. Green
3. Blue
4. Yellow
5. Purple
6. Peach
7. Pink
8. Orange
Let's also assume that we'll first ask respondents which colors they like (one data field counted toward
your license capacity for this multiple-response question). Next, we construct a new list that includes
only the colors each individual prefers. If we show only the colors preferred as part of a grid question and
ask respondents to rate each preferred color, SSI Web counts eight data fields toward your license, even
though each individual would likely see just a subset of the eight colors in the follow-up question.
"Other specify" responses (in select-type questions) add an additional field per other specify and are
counted as a question toward your license capacity.
Ranking questions count one question against the license for each rank that is cast. Therefore, if
ranking ten options, ten questions are counted. If ranking just the top three of ten options, then three
questions are counted.
When you click the icon from the Write Questionnaire dialog, SSI Web prints the number of data
fields used by each question in your survey.
100 SSI Web v8
1.3.3 What Happens at the End of the Survey?

You should plan what the respondent does at the end of your survey. Your survey can "dead
end" (default), you can send respondents to a specific site, or you can link respondents to another
survey. To terminate a survey, you should place a Terminate/Link Question question within your survey.
Creating Your Own Hyperlink to Send Respondents to a Specific Site
SSI Web's Terminate/Link question can automatically link a respondent to a specific site. However, you
can also create your own hyperlink (clickable area) that sends respondents to a specific web page or
site. If using this method, you should still include a Terminate/Link Question and provide proper settings
regarding the respondent's status (complete or incomplete) when reaching this question. You can then
embed a hyperlink (following HTML conventions) or button in the header, body or footer section of the
Terminate/Link question.
As an example, the following text creates a hyperlink to send a respondent back to Sawtooth Software's
home page:
Click <a href= "http://www.sawtoothsoftware.com"> HERE </a> to finish the survey and go to
Sawtooth Software's home page.
(You can add hyperlinks when using SSI Web's text editor by clicking the "anchor" icon and customizing
the address for the HTML that is automatically inserted.)
You can also embed a clickable graphic that sends respondents to a specific web site. Assume that
you created an attractive graphic named "finish.gif" that says "Finish Survey" with your logo in the
background. If you wanted respondents to click the graphic to go to the Sawtooth Software web site,
you would first add a hyperlink using the "anchor" icon. Then, within the anchor HTML, you would add
the graphic by clicking the Add Image icon and selecting the graphic to add. The resulting HTML
might look like:
<a href = "http://www.sawtoothsoftware.com.com"> <img src= "[%GraphicsPath( )%]finish.gif"

BORDER="0"> </a>
If you are using this software to conduct repeated central-site or mall-intercept surveys, you may want
the survey to return to the Password (login) screen. In that case, specify "login.html" as the destination
in your hyperlink. You are not limited to one hyperlink--multiple "exit" options are possible.
Do not embed a hyperlink in the middle of your survey as this would lead to incomplete interviews.
1.3.4 Formatting Options

1.3.4.1 Survey Settings (Style)
The Survey Settings dialog (accessed by clicking or by clicking Compose | Survey Settings...)
lets you specify settings that apply to all questions within your questionnaire. It has eight tabs:
Style
General Format
Page Format
Headers and Footers
Progress Bar
Browser Settings
Error Messages
Advanced
The Style tab on the Survey Settings dialog lets you select different Styles (implemented using
Cascading Style Sheets, or CSS) to apply to your entire survey. This dramatically changes the
appearance of your survey, creating some unique and professional displays. Select from the styles
currently shipping with the software (as shown in the Styles Library window), or develop and import your
own styles.
Please note that the style you select controls many aspects of the survey, including text formatting,
fonts, and bolding. This has the benefit of controlling the entire look of your survey, without the need to
modify the formatting for each question individually. If you want to modify an aspect of the style that
applies to all questions in the survey, you can do so on the Page Format tab via the Paintbrush
icons. If you want to modify an aspect of the style just for a specific question, you can do that by editing
the question and using the Paintbrush icons.
To use a style, highlight the style you wish to use within the Style tab of the Survey Settings dialog. A
sample graphic is shown to give you some idea regarding the look of the style. To apply the style to the
study, select the desired style and then click the Use Selected button. To discontinue using styles,
click No Style.
Using Styles
From the Compose | Survey Settings dialog, select the Style tab. A list of System Styles and any User
Styles that you might have created are shown. User Styles are shown toward the bottom of the list.
Duplicate style names are not allowed.
When you choose a style from the library and click Use Selected the appropriate style folder is copied to
your study's graphics folder. Your study then uses this style to alter its formatting.
102 SSI Web v8
How Styles Work
Overview
CSS Basics
SSI Web HTML "Hooks" for CSS
Modify an Existing Style
Create Your Own Style
Overview
By default, the style settings throughout SSI Web are empty. For instance if you create a new question
and click the Paintbrush icon for Header 1 there is no font size specified. If no font size is
specified at the question level, the font size specified at a higher level is used.
SSI Web uses CSS (Cascading Style Sheets) to control the style in surveys. CSS features cascading
levels of priority. A font size can be specified for all text in a survey or for a particular question. A font
size defined for a particular question overrides the font size defined at the study level.
The diagram below helps illustrate how CSS is used in SSI Web, demonstrating the different levels of
Style priority. Styles defined at the question level in SSI Web override styles defined in the survey style.
Styles defined in the survey style override styles defined in the base style sheet ssi_style.css.
CSS Basics
CSS stands for Cascading Style Sheets. CSS allows you to modify the style (background color, font
style, spacing, etc.) of the HTML for web pages. CSS allows you to separate the style from the structure
of a web page.
CSS can be defined within the HTML elements. For example:

<p style="color: green;">

Please answer the questions below.
</p>
The CSS "color: green;" changes the text color within the <p> tag to green. This style can be separated
from the <p> tag. The following instructions:
<style type="text/css">
p
{
color: green;
}
</style>
<p>
Please answer the questions below.
</p>
change all of the text within <p> tags on page to green.
Suppose that you have multiple web pages and you want the text inside all <p> tags for all pages to be
green. You can create a separate text file named "mystyle.css" that includes the following CSS:
p
{
color: green;
}
At the top of each web page inside of the <head> tag you would then include:
<link rel="stylesheet" type="text/css" href="mystyle.css">
This controls the style for all web pages from one file. To change the color from green to blue you would
simply edit mystyle.css. Using CSS in this way makes changing the style across your web pages
much easier than defining the style on each individual page. Styles defined in an external file have lower
priority than styles defined within a <style> tag on the web page. Styles defined within the HTML inside
of a "style" attribute have a higher priority than styles defined within the <style> tag.
CSS consists of various style "rules" that define how to modify the appearance of the HTML. In the CSS
rule below the "p" is the selector and the "color: green" is the declaration.
p
{
color: green;
}
The selector identifies which part of the HTML will be affected by the declaration. The declaration
consists of a property and a value separated by a colon. Multiple declarations are separated by
semicolons. For example:
104 SSI Web v8
p
{
color: green;
font-size: 18px;
padding: 10px;
}
The selector can be the name of any HTML element. For example: p, div, span, table, tr, td, etc. Multiple
elements can be added to the rule separated by commas. For example:
div, p, td
{
color: green;
}
The CSS above changes text within <div>, <p>, and <td> tags to green. You can also specify that only
<p> tags within <div> tags should be selected:
div p
{
color: green; /*Turn the text green*/
}
Notice that there is a space separating "div" from "p". This indicates that only <p> tags within <div> tags
are affected. Also notice the comment "Turn the text green." CSS comments are contained between /*
and */. Any text between /* and */ will not affect the CSS.
A Class Selector is a way that you can identify specific HTML elements. Suppose you have the following
HTML:
<div class="header1">
How old are you?
</div>
The <div> above has an attribute named "class" with a value of "header1". You identify a class in CSS by
placing a "." in front of its name. For example
.header1
{
color: green;
}
The CSS above makes the text within tags with the "header1" class be green. SSI Web uses class
selectors heavily. Classes are designed to identify one or more elements on the web page. If there is a
unique element identified with an "id" then a "#" sign is used instead of a ".". For example:
<div id="Q1_div">
How old are you?
</div>
The following CSS could be used to select this <div>:
#Q1_div
{
color: green;
}
SSI Web includes many classes and IDs within the HTML that it creates to help you identify specific
areas of the page. You can then use these "class" and "id" names within your CSS.
More information on CSS can be found at http://www.w3schools.com/css.
SSI Web HTML "Hooks" for CSS
SSI Web generates the HTML for your survey based on the different question types and settings you
configure in the software. The HTML contains "class" and "id" attributes in many of its tags. For example
the HTML for Header 1 looks like this:
<div class="header1"> Header 1 Text </div>
The "class" attribute in the <div> tag above identifies Header 1. The following CSS can then be used to
change the Header 1 text color throughout the whole survey:
.header1
{
color: green;
}
The diagram below shows some of the main classes present in SSI Web's HTML.
106 SSI Web v8
Additional classes and ids can be examined by viewing SSI Web's HTML. This can be done using "View
Source" in your web browser.
Some HTML elements in SSI Web contain multiple classes and an id. Various ways have been provided
to allow the CSS to manipulate the presentation or style of the survey. The HTML for a select question
named "Q1" looks like this:
<div id="Q1_div" class="question select">

...
</div>
Notice that two classes "question" and "select" have been assigned to this <div> tag.
The following CSS changes the Header 1 style for Q1. No other questions are affected.
#Q1_div .header1
{
color: blue;
}
The # sign above identifies the HTML tag with the "Q1_div" id. After "#Q1_div" there is a space and then
".header1". This CSS rule changes the text for Header 1 to blue inside question Q1. The following
draws a black border around all questions:
.question
{
border: 1px solid black;

}
The following changes the border only for Select questions to green:
.select
{
border: 1px solid green;
}
Modify the Look of an Existing Style
To modify an existing System Style, from the Style tab click the icon. Browse to the system style
folder (C:\Program Files\Sawtooth Software\SSI Web\Styles) and select the style you wish to
modify. When you select the style folder and click OK, SSI Web copies this style folder into your User
Styles folder and asks you to rename it (so that there aren't duplicate names for styles).
As an example, let's say that you want to change the background color of the charcoal style to green.
First, from the Style tab, click the icon and browse to the System Styles folder (C:\Program Files
\Sawtooth Software\SSI Web\Styles) and select charcoal from the list. Click OK and you are
prompted to change the name so that there aren't duplicate style names. Specify greencharcoal as the
new style and SSI Web renames the style and copies it into your User Styles folder:
C:\Users\<user>\Documents\Sawtooth Software\SSI Web\Styles
Next, you modify the contents of the greencharcoal.css file to modify the style. To change the
background color to green, open the greencharcoal.css file with a text editor (e.g. Notepad). At the top
of the CSS file you will see:
/* Background setting for whole page, currently grey */

body
{
background-color: #DDDDCC;
}
You should change background-color: #DDDDCC; to background-color: green; (or you can directly
specify the hex code for green rather than use the key word "green") and save the file. You can also
change the contents of the comment contained between /* and */. You might also delete the
preview.jpg since it no longer reflects the modified style. You could capture a screen shot of the
modified style and save it (size the graphic to a maximum of 500 x 375 pixels) in the greencharcoal
folder as preview.jpg.
Next, highlight the greencharcoal style in your style list and click Use Selected. The greencharcoal
style folder with its .css file are copied into your study folder, within the graphics directory. Now, when
you preview your study you will see the green background.
If you need to continue to make modifications to your customized .css file, you should recognize that
when you click Use Selected, the .css file being used is now within your study folder. You should
continue to make changes there if you want to see the changes reflected in the current study. If you
want to use your modified style in the future for other studies, you should copy the .css file from your
study directory back into your User Styles folder.
108 SSI Web v8
Create Your Own Style (User Styles)
The best way to learn how to create your own style is by examining one of the existing system styles in
the System Style Library that ships with SSI Web. These styles are located by default in:
C:\Program Files\Sawtooth Software\SSI Web\Styles
To create your own style, create a new folder within your User Styles area (C:\Documents and Settings
\<user>\My Documents\Sawtooth Software\SSI Web\Styles Library) and give it the name of your
new style. You will then need to create a CSS file with the same name as your style and place it inside
the folder. For example, to create a style named red, create a folder also named red and place a file
named red.css inside it. Place this folder in the User Styles path on your hard drive or import it into the
Style Library by going to the Style tab and Style Library, clicking the icon then by browsing to the
red folder. Highlight the new style and click Use Selected to apply it to your study.
A graphics file named preview.jpg can optionally be placed in your newly created style folder. This file
should contain a screenshot of the style and is used in SSI Web to give a small preview of what the style
looks like. We recommend the graphic be sized to 500 x 375 pixels.
1.3.4.1.1 General Format

Display
Browser Title
This field lets you specify the text that appears in the title bar of your browser (very top of browser
window). This lets you further customize the browser appearance for your survey. For example,
you may want the browser title bar to say: "Customer Satisfaction Study 2007".
Font
You can specify the font face used for text in the survey. Type a font face, such as Times or Arial.
Do not put quotes around the font face. The font face doesn't apply to text in response fields for
the open-end, numeric or combo box question types. It is best to use a standard font, since
respondents may not have the particular unique font you might want to use installed on their
machine. If the font you specify is not available on the respondents machine, the browser will
choose a default font. If you specify multiple fonts separated by commas, if the first font is not
found on the respondent's machine, the second font will be attempted, and so on.
Font Format
You may override the Font Face specified in the Style along with other characteristics of font
display (bolding, italics, etc.) using this control.
Sawtooth Software Logo

If you don't wish to display the Sawtooth Software notification on the last page of your survey, un-
check the Display "Powered by Sawtooth Software" on the last page box.
Navigation Buttons
Text
Respondents click the Submit and Previous buttons at the bottom of the pages of the survey. You
customize the text that appears on this button by typing text directly into the Submit Button or
Previous Button fields.
Graphics
Styles by default have graphics buttons (for Submit and Previous buttons) that are applied if you
choose to use graphical navigation buttons. You can select other graphics by selecting Settings...
and browsing for the graphic using the Camera icon. When you select the graphic in this
manner, it is copied into your Graphics folder when you prepare for web upload. If you browse to
the graphic, the correct HTML and an appropriate path is configured for your graphic using the
GraphicsPath() SSI Scripting syntax.
We have provided some sample "Next" and "Previous" button graphics in the Graphics Library
folder.
Using graphics requires respondents to have JavaScript enabled in their browsers. If the respondent
does not have JavaScript, the respondent will see a generic next button with the following label
">>".
Button Justification
Specify whether the Submit button is left justified, center justified, or right justified on the page, by
selecting a Submit Button Justification option.
110 SSI Web v8
Respondent Input
You can use graphical radio and checkboxes, which lets you polish the look of your survey, as well as
make the buttons any size you like. If you decide to create your own graphical buttons, you'll need to
create a graphic for the check ed and also the uncheck ed state of the button.
1.3.4.1.2 Page Format

Page Format
The Page is the broader area within the browser on which the questions are positioned.
By default, the settings are blank, meaning that these settings are being controlled by the Style that is
selected. You can override the Style settings for Page Width, Page Border, Page Justification,
Space Between Questions, Page Background Color and Screen Background Color.
Screen Background Color

This sets the color for the area surrounding the main body of the questionnaire, extending to the edges of
the browser window.
Question Format
Here, you can override the settings in the Styles for all questions in your survey.
The Question area is subset of the total Page area. By default, the settings are blank, meaning that
these settings are being controlled by the Style that is selected. Click the Paintbrush icons to
override the Style settings for Header 1, Header 2, Question, and Footer sections of the question.
You may also override the Style settings for all questions for the Question Width, Question Indent,
Question Justification, Question Background Color, and Alternating Colors.
112 SSI Web v8
1.3.4.1.3 Headers and Footers

Page Header and Footer
This section allows you to specify a general header and footer that appears on every page of your survey.
Headers and footers can contain text, HTML formatting commands, and graphics. A Preview button lets
you see how your Web browser displays the header and footer you provide.
A common use of headers and footers is to print some company information, logos, or a phone number
to call for help if respondents have questions about the survey.
<Head> Tag
This area lets you specify additional information to include between the HTML <HEAD> and </HEAD>
tags.
Here are some examples of things you might include:
CSS Tags
JavaScript
Meta tags
1.3.4.1.4 Progress Bar

Sometimes it is useful to display a progress bar at either the top or bottom of each page to let
respondents know how far they have come in a survey and approximately how far they have to go.
Show Progress Bar
You can automatically include a progress bar in the Page Footer or Header by checking this box and
selecting the position. You can exert additional control over the positioning of the progress bar by
placing it manually within the global header or footer (see below).
Manually Including a Progress Bar
To place a progress bar manually in the questionnaire, simply type [%ProgressBar( )%] within the text
boxes of the Survey Settings dialog box under Headers and Footers tab. The progress bar won't appear
on the login page.
Progress Bar Settings
Beginning Value (%): Specifies the beginning value for the progress bar. This is useful if you
are linking multiple SSI Web surveys together and need to specify a certain beginning value.
Ending Value (%): Specifies the ending value for the progress bar. This is useful if you are
linking multiple SSI Web surveys together and need to specify a certain ending value.
Progress Based Upon
Question position in the questionnaire method (default method): This method of measuring
progress is based on (current page)/(total # pages). Works best for questionnaires not
involving loops or randomized blocks.
Question seen method: This method shows progress based on (#questions seen)/(total
number of questions based on Estimated % of questions each will see). This method
potentially works better for questionnaires involving loops and randomized blocks.
Estimated % of the questions each respondent will see: Typically, you would specify 100
here. For example, a questionnaire with no skip patterns (where respondents see all
questions) shows each respondent 100% of the questions. But, if you have planned a
questionnaire in which respondents see only half the questions (such as randomly selecting
respondents to see either the first half or last half of the questions), then you should specify a
50 for this field. Otherwise, the progress bar would suddenly jump forward for respondents
who are randomly skipped over the first half of the questions. For the other half of the
respondents, the progress bar would slowly advance to 50%, until it suddenly jumped to 100%
.
Progress Bar Format
Alignment: Choose from left, center, or right.
Width: Specifies the width of the progress bar, in pixels.
Background Color, Bar Color, Bar Background Color and Text Color
114 SSI Web v8
These controls let you customize the look of your progress bar. Click the drop-down button to access
the "color picker" to select a new color.
1.3.4.1.5 Browser Settings

General Settings
Disable Back Button

By checking this option, the ability to back up in a survey is removed. This feature requires
JavaScript to be enabled in the browser. This feature may not work in all browsers (including
most mobile browsers).
Simple Browser Settings
This area lets you specify that SSI Web should remove the regular toolbar and buttons used by the
respondent's browser and show the survey within a "simple browser." This is useful for maximizing the
window real estate for use in presenting your survey and for removing buttons such as the "back" button
from many common browsers.
To see how the simple browser operates, you need to test your survey in Local Test Server or on your
remote server. Preview mode doesn't support simple browser.
Notes: The respondent's browser must support JavaScript to enable simple browser. If the respondent
has disabled JavaScript, the survey will appear in normal mode. Warning: Simple browser may not
always work correctly on every configuration. We recommend you test simple browser using different
browsers, as the presentation can vary. If you use simple browser, and the respondent has enabled
JavaScript, even if the browser still displays the toolbar including the "back" button, SSI Web prevents
the users from backing up.
Specify Window Size:

If you choose not to use full screen window, this lets you specify the width and height of the simple
browser window.
Use Full Screen Window:

This setting maximizes the browser window for most browsers and prohibits the respondent from
resizing the window. Because the simple browser can remove the toolbar and icons the
respondent is accustomed to seeing, it can be helpful to provide an easy way for them to close the
simple browser window once they've finished the survey. If you provide text in the Close Text field,
it becomes a clickable hyperlink at the upper right-hand corner of the survey that closes the
window. You may include a graphic instead of text. If the Close Text field is empty, no link will
display for closing the window.
Specify Transition Page Text:

If you use the simple browser, SSI Web inserts a "transition" page to let respondents know that
they are about to launch a new browser window. It also gives them instructions if the window does
not appear (holding the CTRL key while clicking next, or disabling their pop-up blocking software).
For internationalization, you will want to translate the Transition Page Text into the target language.
Also note that there is no Preview button to view this text. You must test the survey on a web
server or on Local Test Server to see the implementation.
116 SSI Web v8
1.3.4.1.6 Error Messages

The Error Messages tab on the Survey Settings dialog lets you customize the error messages shown to
respondents during the course of an interview. For example, when respondents specify an invalid
response, or skip a required question, error messages prompt them to remedy the error. This area is
also useful for customizing the messages for foreign languages.
Customizing Error Messages
When you click the Error Messages tab, the Error Message list is displayed.
You can choose the color, styles, size and justification for the error messages. Default text is provided,
that you can customize for the purposes of your survey. The default text provided refers respondents to
a certain question number on the current page. However, you can also customize your text to refer to
specific question names (Note: if you refer respondents to specific question names, you should include
the question name as part of the header text of that question).
These are the following error type classes:
Page Error
The general message that appears at the top of the page when there is an error of any sort.
Missing Answers
Reminds respondents to complete a question before continuing.
Minimum Checks
Tells respondents that the minimum number of checks required in a multiple-response select
question (check box) has not been satisfied.
Maximum Checks
Tells respondents that the maximum number of checks permitted in a multiple-response
select question (check box) has been exceeded.
Out of Range (Numeric)

Lets respondents know that the value they specified for a numeric question is outside the
specified range. You specify the minimum and maximum range when you compose numeric
questions using the Numeric Question dialog.
Invalid Input (Numeric)

This error message appears if respondents type an invalid character into a numeric entry box.
Invalid characters are values other than numbers or the period and/or comma permitted for
decimal answers. You may want to expand the error message text to explain these other
conditions to respondents.
Minimum Characters (Openend Questions)

Informs respondents that they haven't typed enough characters in an open-end question. You
can specify a minimum number of characters when you compose open-end questions using
the Open-end Question dialog.
Maximum Characters (Openend Questions)

Informs respondents that they have typed too many characters in an open-end question. You
can specify a maximum number of characters when you compose open-end questions using
the Open-end Question dialog.
Password Closed
Appears when a valid password combination is used, but the password quota cell is full (e.g.
the username/password combination has already been used.)
Finished
Informs respondents that their survey already has been submitted.
Invalid Login
Lets respondents know that the password combination they've specified is invalid.
Rank
Informs respondents that the answers to a ranking question are not valid.
Rank Count
Informs respondents that they have not yet fully ranked enough items in a ranking style
question to continue.
None of the Above

Informs respondents that they cannot select an "exclusive" answer (such as "None of the
above") together with other responses for the same question. This message is only shown
with Perl verification.
Constant Sum
Informs respondents that the responses to a constant sum question do not total the valid
amount.
Submit Only Once

If respondents click the submit button multiple times, this message asks them to wait for a
moment while their responses are being processed.
MaxDiff Unique Answers

If respondents click the same response for the "best" and "worst" items in a MaxDiff question,
this message is displayed.
ACBC Prohibited
Warns respondents that they cannot select levels in the BYO question of an ACBC survey
that are prohibited.
ACBC Price Adjustments Warning

If additional price adjustments are specified and the respondent changes a BYO selection that
would affect the price of other selected options within the BYO, this warning is displayed.
Survey Paused
If a survey administrator updates the questionnaire (e.g. questions added, text modified) while
it is in field, respondent access to the survey is temporarily blocked while updates are being
made. During this brief period respondents see this message.
CBC Best-Worst Unique Answers

If the respondent tries to indicate that the same product concept is both best and worst within
a Best-Worst CBC question, this message is displayed.
118 SSI Web v8
Require JavaScript
If JavaScript is not enabled on the respondent's browser, this message is displayed. You can
turn off the JavaScript requirement on the Advanced tab.
Error Message Arguments
The text of the error messages includes Functions that are used to display numeric values or text strings
relevant to each error in the error message. The available Functions are:
ErrField ( )
Specifies the field (question) number on the most recent interview page, where the first
question on the page is question number 1.
ErrMin ( )
Specifies the minimum value for a numeric question, or the minimum number of characters for
an open-end question.
ErrMax ( )
Specifies the maximum value for a numeric question, or the maximum number of characters
for an open-end question.
ErrQName ( )
Returns the question name for the question on the current page causing the error. Use this
argument if you prefer to refer respondents to specific question names rather than a question
sequence number on the current page. (Note: if you refer respondents to specific question
names, you should include the question name as part of the header text of that question).
ErrText ( )
Returns the error text provided under the Advanced... button for questions, or in the error text
field for Free Format questions. If no text is provided in the error text field, the ErrText variable
will be replaced by nothing (be missing) when run over a server.
ErrTotal ( )
Returns the total number of items to be ranked in a ranking-style question, or the total number
of points to be allocated in a constant-sum question.
ErrCurSum ( )
Returns the current number of points that have been allocated in a constant sum question.
The actual error messages that a respondent sees when taking the questionnaire (with numeric values
substituted for the string arguments above) can only be viewed when your survey is running on the Local
Test Server or on a remote web server.
1.3.4.1.7 Advanced
Save Respondent Information
This section lets you disable SSI Web's automatic storage of information about the respondent's
browser, operating system, and IP address. There may be some instances in which you are
prohibited from collecting such information and you can un-check the information that should not be
saved.
The information collected on respondent's browser, operating system, and IP address may be
exported with your other survey data.
Even if you decide not to collect the identifying information described here, a variable indicating
whether Javascript is enabled within the respondent's browser is always collected.
Save Respondent Time Per Page
Check this box to store the time (in seconds) spent per page. When you enable this, variables are
stored into the database that store the time. You can export these times under the File | Export
Data option.
If the respondent backs up and re-answers a page (using SSI Web's Previous button), new time spent
answering that page will be added to the previous time recorded. If respondents use the Back button
on the browser to re-answer a page, the total time spent on the re-answered page may not be
accounted for correctly.
Additional Settings
Use JavaScript
By default, SSI Web requires that all respondents taking the survey have JavaScript enabled in their
browsers. There are many features within SSI Web that require JavaScript to function properly.
Respondents who do not have JavaScript enabled will see an error message (specified in the Error
Messages tab) that asks them to enable JavaScript prior to taking the survey. If you want to allow
respondents without JavaScript enabled to take the survey, you can uncheck this box. However,
please be aware that depending on the functionality you are using within your survey, this may make
it impossible to answer certain questions or impossible to see certain dynamic features or graphics
on the screen. If you uncheck this box, we urge you to turn off JavaScript in your browser and test
the survey to ensure that it functions and collects the data properly.
Note: JavaScript is required for certain question types such as sliders in Semantic Differential
questions, Constant Sum totals box, conditional graphics in ACBC, Constant Sum, BYO questions
within ACBC, custom JavaScript, etc. This is not necessarily an exhaustive list of the affected
components.
Use Custom HTML DOCTYPE
The DOCTYPE declares the HTML version used on the web page. Many web browsers will read this
line and make a decision how to display the HTML. The DOCTYPE declares that the HTML is
following standards. You can check the HTML of the page for errors by going to http://
validator.w3.org/.
120 SSI Web v8
Using a DOCTYPE can alter the way that the web page is displayed in some browsers. Some of your
custom HTML might not be formatted quite the same in some browsers with the DOCTYPE at the top
as it is without it. You might consider adding a custom DOCTYPE to accommodate custom HTML
that you might have.
Remove System Meta Tags from Header Section
Removes the following tags located inside of the <head></head> section:
1. <meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7">

If the respondent is using Internet Explorer, this forces whatever version they are using to act
like IE7.
2. <meta http-equiv="content-type" content="text/html; charset=UTF-8">

This tells the browser to interpret the page as HTML and using the UTF-8 character set.
3. <meta name="robots" content="noindex, nofollow">

This tells search engines like Google to not “crawl”, or make things on your site show up in
search results.
4. <meta name="viewport" content="width=device-width, initial-scale=1">

This setting helps content to be scaled better for mobile devices.
Decimal Point Character
If you are including numeric questions in your survey and have specified that decimal answers are
permitted for those questions, you need to specify the character to be reserved for the decimal point.
In the US, a period (default) is used. Other countries may use a comma. If a period is selected, only
numbers and a single period are acceptable answers. For example, the number "one thousand one
and one half" must be specified as: "1001.5", not "1,001.5". If other characters are used, a warning
message appears.
If you specify that a comma or period is to be used as the decimal point character, both periods AND
commas are acceptable as decimal points. In other words, the number "one thousand one and one
half" can be specified as: "1001.5" or "1001,5", but not "1,001.5", or "1.001,5". (Note: the Data
Reports program within the Admin Module truncates all decimal answers, treating them as integers,
prior to computing frequencies, min, max, and means.)
Advanced Server Settings
The default for each setting below is empty. Please test the survey well if you change any of these
settings.
Respondent Number Start
This setting allows you to specify the first Internal Respondent Number that will be used for this study.
For example if 500 is entered then the first respondent will be assigned 500 as their Internal
Respondent Number. The next respondent will be assigned 501 and so on.
Max Fields per Database Table
This setting specifies the maximum number of columns allowed in any of the database data tables.
Max Database Tables to Join
This setting specifies the maximum number of database tables that can be joined together.
122 SSI Web v8
1.3.4.2 HTML Elements to Enhance Your Survey

(This section gives a brief overview of using HTML elements within SSI Web. More HTML help is
included in the HTML Cheat-Sheet .)
HTML (HyperText Mark up Language) is the language of the Web. You don't need to know any HTML
code to use this software. But users who know (or are willing to learn) a few simple HTML tricks can
gain some nice benefits.
You can include HTML instructions to:
Add styles to specific words or letters (like bold, italics, or underlining)

Change font sizes of specific words or letters
Display graphic images
Nearly any HTML instruction can be included in your survey. You simply type HTML elements into the
text entry fields in the SSI Web interface.
Notes: Be careful with the HTML elements you add within your surveys! You should only include HTML
instructions that belong between the <BODY> </BODY> tags of the HTML script. It would be a mistake
to use <HTML> or <FORM> tags in the HTML code you add to your questionnaires, as it will likely
interfere with the <HTML> and <FORM> section instructions that our software automatically generates
when it runs on the server. Also, you probably wouldn't want to add a hyperlink that would send
respondents out of the survey, turn off the forms mode using </FORM>, create a submit button or
additional input field that would interfere with the survey progress.
When using the Free Format question type, you will probably need to know more than just the "basics"
of HTML. The Free Format question type turns all of the question-writing responsibility over to you,
including specifying the <input> tags. We have some very simple examples documented in the Free
Format section.
Any number of books and websites can instruct the interested user on HTML basics. Here is a partial
list of HTML elements that can be useful (and safe to use) in your Internet surveys:
Adding Styles
Italics
to turn on: <i>
to turn off: </i>
Type: Please be <i>very</i> careful!
to create: Please be very careful!
Bold
to turn on: <b>
to turn off: </b>
Underline
to turn on: <u>
to turn off: </u>
Adding Hard Returns
Internet browsers typically wrap your text if it doesn't fit into the allotted space. Sometimes you may
want to control where your phrases break. You can insert hard returns using the <br> element.
Type: Please be <br>very careful!
to create: Please be
very careful!
Type: <p>
to create: hard return plus blank line
Inserting Horizontal Lines
Type: <hr>
to create: Horizontal line spanning the current text cell, or spanning the current line.
124 SSI Web v8
1.3.4.3 HTML Cheat-Sheet

Text Tags
<pre></pre>
Creates preformatted text
<h1></h1>
Creates the largest headline
<h6></h6>
Creates the smallest headline
<b></b>
Creates bold text
<u></u>
Creates underline text
<i></i>
Creates italic text
<font size=?></font>
Sets size of font, from 1 to 7
<font color=?></font>
Sets font color, using name or hex value
Links
<a href="URL"></a>
Creates a hyperlink
<a href="mailto:EMAIL"></a>
Creates a mailto link
<a name="NAME"></a>
Creates a target location within a document
<a href="#NAME"></a>
Links to that target location from elsewhere in the document
Formatting
<p></p>
Creates a new paragraph
<p align=?>
Aligns a paragraph to the left, right, or center
<br>
Inserts a line break
<blockquote>
</blockquote>
Indents text from both sides
<div></div>
Tag used to format a block of HTML. Helpful in use with CSS.
<span></span>Used to format inline HTML, such as just a single word in a sentence. Useful with CSS.
<dl></dl>
Creates a definition list
<dt>
Precedes each definition term
<dd>
Precedes each definition
<ol></ol>
Creates a numbered (ordered) list
<li></li>
Precedes each list item, and adds a number
<ul></ul>
Creates a bulleted (unordered) list
Graphical Elements
<img src="name">
Adds an image
<img src="name" align=?>

Aligns an image: left, right, center; bottom, top, middle
<img src="name" border=?>

Sets size of border around an image
<hr>
Inserts a horizontal rule
<hr size=?>
Sets size (height) of rule
<hr width=?>
Sets width of rule, in percentage or absolute value
126 SSI Web v8
<hr noshade>
Creates a rule without a shadow
Tables
<table></table>
Creates a table
<tr></tr>
Sets off each row in a table
<td></td>
Sets off each cell in a row
Table Attributes
<table border=#>
Sets width of border around table cells
<table cellspacing=#>
Sets amount of space between table cells
<table cellpadding=#>
Sets amount of space between a cell's border and its contents
<table width=# or %>

Sets width of table -in pixels or as a percentage of document width
<tr align=?> or <td align=?>

Sets alignment for cell(s) (left, center, or right)
<tr valign=?> or <td valign=?>

Sets vertical alignment for cell(s) (top, middle, or bottom)
<td colspan=#>
Sets number of columns a cell should span
<td rowspan=#>
Sets number of rows a cell should span (default=1)
<td nowrap>
Prevents the lines within a cell from being broken to fit
Forms
<select multiple name="NAME" size=?></select>

Creates a scrolling menu. Size sets the number of menu items visible before you need to scroll.
<option>
Sets off each menu item
<select name="NAME"></select>
Creates a pulldown menu
<textarea name="NAME" cols=40 rows=8></textarea>

Creates a text box area. Columns set the width; rows set the height.
<input type="checkbox" name="NAME">

Creates a check box. Text follows tag.
<input type="radio" name="NAME" value="x">

Creates a radio button. Text follows tag
<input type="text" name="foo" size=20>

Creates a one-line text area. Size sets length, in characters.
<input type="submit" value="NAME">

Creates a Submit button
<input type="image" border=0 name="NAME" src="name.gif">

Creates a Submit button using an image
<input type="reset">
Creates a Reset button
128 SSI Web v8
1.3.4.4 Graphics Management

SSI Web assumes a server installation that contains a separate graphics directory. We use
the function [%GraphicsPath( )%] to specify the path to that graphics directory. When the
survey is run on a server, the text "[%GraphicsPath( )%]" is replaced by the paths specified
under Field | Web Server Management. The function [%GraphicsPath( )%] is replaced with
the text supplied in the Relative Path From CGI Scripts to Graphics field.
If you use the [%GraphicsPath( )%] function, Local Test Server always looks to the graphics
subdirectory off of your study directory regardless of what you have specified under Field | Web Server
Management. But, when running on the server, SSI Web uses the paths to graphics specified under
Field | Web Server Management.
One benefit to using the [%GraphicsPath( )%] function is that if you need to change the path definitions
for graphics, you only need to change them in one place (under Field | Web Server Management).
If you want to, you can choose not to use the system variable when defining the paths to graphics. You
can provide an <img> tag with a relative or absolute path of your choice (where the path is the path from
the Perl scripts to the directory containing the graphic, or the login.html file to the graphic). However,
customizing the paths will most likely mean that the graphics will not be able to be found when testing
with Preview or the Local Server.
1.3.5 Survey Flow (Randomization, Skips, Loops)

Randomizing Question/Pages/Blocks
There are times where the researcher wants to randomize a series of pages, questions on a page, or
blocks of questions. This is most often done to reduce response bias due to question order. Question
order bias occurs when respondents answer differently to later questions because of the influence of
previously answered questions. If questions are displayed in random order, question order bias still
exists at the individual level. But, because individuals each receive a different question ordering, the bias
is nearly eliminated when viewing aggregate results.
You can randomize the order that questions are asked within the same page. You can also randomize
the order of pages. You cannot randomize the order of questions that span from one page to another,
when there are multiple questions per page.
Randomizing Questions on a Page
To randomize the order that questions are asked on a page:
1) Click Compose | Write Questionnaire and click the Randomize... | Questions on a Page
button. From the Randomize Questions on a Page dialog, click the Add... button.
2) Select the beginning and ending question. All questions from the beginning to the ending
question (including the beginning and ending questions) are displayed in random order when
the questionnaire runs on a Web server.
Randomizing the Order of Pages
1) Click Compose | Write Questionnaire and click the Randomize... | Pages button. From
the Randomize Pages dialog, click the Add... button.
2) Select any question from the first and last pages in the randomized sequence. All pages
including and between the anchor questions are displayed in random order when the
questionnaire runs on a Web server.
Randomizing Blocks of Questions
Example: imagine a questionnaire with three blocks of questions Q1-Q3 (block 1), Q4-Q10 (block 2), and
Q11-Q20 (block 3). We may want to randomize the order that those three blocks of questions occur.
For example, one possible ordering is Q4-Q10, Q1-Q3, Q11-Q20.
1) Click Compose | Write Questionnaire and click the Randomize... | Blocks button. From
the Randomize Blocks dialog, click the Add Set button.
2) Add each block of questions by clicking Add Block... and then choosing a beginning and
ending question for each block.
3) Select to either randomize the order that the blocks are asked, or to ask the blocks in the
order as indicated in a constructed list. For example, if the blocks were to be asked in the
130 SSI Web v8
order as represented within a constructed list and that list contained elements ordered 3, 2, 1
then the blocks would be asked in order: block 3, block 2, block1.
Notes: Each block must be separated by at least one page break. Questions can be additionally
randomized within blocks.
Showing a Random Subset of Questions/Blocks
Example: imagine a questionnaire with 10 questions (or blocks of questions). You wish to randomize
their presentation and have respondents complete a randomly selected 5 of the 10 questions.
1) Within Compose | Lists..., create a list called list10 with 10 members (you may just use
integers 1 through 10 as the list member text). Add a constructed list called randlist5, that
uses parent list list10. Use the following constructed list logic to randomize the list10
members and trim the list to just 5 members:
ADD (list10)
RANDOMIZE ()
SETLISTLENGTH (5)
2) Click Compose | Write Questionnaire and click the Randomize... | Blocks button. From
the Randomize Blocks dialog, click the Add Set button.
3) Add each question, or block of questions, by clicking Add Block... and then choosing a
beginning and ending question for each block. (If you are selecting random subset of single
questions, the beginning and ending question for each block will refer to the same one
question.)
4) Select to radio button (Use Constructed List) to randomize the order that the blocks are
asked according to the constructed list, randlist5. When the questionnaire runs, if the
constructed list contained elements 4, 2, 8, 7, and 1 the question blocks would be asked in
order: block 4, block 2, block 8, block 7, and block 1.
Notes: Each block must be separated by at least one page break.

1.3.5.1 Skip Logic

Skip patterns (also known as branching) let respondents skip over sections of the questionnaire that
don't apply, typically based on previous answers. SSI Web provides flexible and powerful skip logic.
Skip instructions can either be executed before showing a question, or after the question is answered.
We refer to these classifications as pre-sk ips and post-sk ips:
Pre-Skips - The skip instruction is evaluated and, if true, executed just prior to loading a
question. The typical use is to conditionally skip a question if a condition is true. Otherwise,
the question is displayed.
Post-Skips - The skip instruction is evaluated and, if true, executed after a respondent has
answered the question and submitted the data on the page containing this question.
You have the option of providing a target question to skip to, or to skip to the "next question" (in the case
of pre-skips). A comment field is associated with each skip logic entry you provide, so you can make
any notations to help you as you construct and test the questionnaire.
Skip patterns only operate when the survey runs on a web server (not during preview mode).
Skip Settings
Skip from
Specify the question to skip from using this drop-down box.
Skip Type
This control is for specifying whether this is a Pre-Sk ip (skip is executed Before the question is
shown) or Post-Sk ip (skip is executed After the question is answered).
Always skip
Indicates an unconditional skip. The skip is always executed and no skip logic is necessary.
Skip if the following logic evaluates to true

Check this button if the skip is only to be executed if certain skip logic is true (provided in the text
field directly below).
Skip to
Specify the question to skip to using this drop-down box.
Comment
You can optionally provide notes within this field to assist you in composing and testing your
survey.
Skip Logic Syntax
SSI Web provides powerful skip logic syntax for executing skip patterns. You type this logic into the
text field directly below the Sk ip if the following logic evaluates to true radio button.
Supported Logical Operators:
and
or
132 SSI Web v8
not
Supported Mathematical Operators:
=, <, >, <=, >=, <> "not equal to"

+, -
/ "division"
* "multiplication"
^ "raise to the power of"
mod "modulo"
and all other SSI Web Functions.
You can check whether the instructions (syntax) you provide are valid by clicking . Syntax is also
checked when you prepare your survey for local or remote web server.
(Note: skip logic has been implemented using SSI Script.)
Example Syntax
Below are some examples of valid skip syntax. If these statements evaluate to true, then the skip is
executed.
Q1 = 2 3 "Skip if Q1 is equal to 23"

Q1 _ 3 = 1 "Skip if the third response option is checked in multi-response question Q1"
Q1 < = 1 0 "Skip if Q1 is less than or equal to 10"
Q1 < > 9 9 "Skip if Q1 is not equal to 99"
Ag e > 1 7
( Ag e > = 1 8 ) a n d ( Ge n d e r = 1 )
( Ag e > = 1 8 ) o r ( Q1 = 2 3 ) o r ( Q2 > 3 . 5 3 4 )
( ( a g e Qu e s t i "Shown" is a function that
o n > 1 7 ) Or ( No t Sh o wn ( a g e Qu e s t i o n ) ) )
returns true if the question specified in the parenthesis has been seen
Q2 3 / ( Q2 4 + Q2 5 ) > = 2 3 . 5
( Q2 3 * Q1 ) ^ 0 . 5 < 1 0 "^" means "raised to the power of"
Q1 Op e n En d = " h e l l o wo r l d "
Sh o wn ( Q1 ) "Skip if the respondent saw Q1, even if the respondent left Q1 blank"
L i s t Ha s Pa r e n t Me mb e r ( Q1 L i s t , 4 ) "Skip if Q1List contains list member #4 from its
parent list"
If you change the name of a question or list in your questionnaire, the question or list name is NOT
automatically updated in your skip logic. You must do this manually.
Skip Logic Flow of Execution
Pre-skips are evaluated before a page loads. If the questions on the page are randomized, any
skip patterns associated with these questions are evaluated in their unrandomized order.
If a question is skipped, any post skip logic associated with the skipped question is ignored. (Note
that questions can be skipped due to skip logic, or also due to containing an empty constructed
list.)
Post-skips are evaluated after the page is submitted, according to the unrandomized order of
questions on the page. The first post-skip logic that evaluates to true is executed and the rest are
ignored.
Warning: a pre-skip that has a destination to a different page may cause other questions on the same
page not to be displayed. For example, consider a page containing questions Q10, Q11 and Q12. If a
pre-skip on Q12 skips to a later page, if the pre-skip logic associated with Q12 is true, it causes the
entire page to be skipped. In this case, one would want to place Q12 on its own page to ensure that
Q10 and Q11 are displayed.
Hint: You can create HTML/Filler questions that contain skip instructions (and perhaps no actual text to
be shown on the screen). This can help in some instances when you want to place skip instructions on
a page, but do not want the skip instructions to be directly associated with a specific question (in case
that question was conditionally skipped).
Performing Skips Based on "Select All That Apply" (Multi-Select) Questions
You can perform skip patterns based on "select all that apply" questions. Assume a multiple-response
question named Q1 that has 5 options. Respondents can register up to 5 answers. If item number three
on the list was selected, then that item is referenced using "underscore notation": Q1_3 = 1. If item
three was not selected, then Q1_3 = 0. If item number 1 was selected, then Q1_1 = 1, etc. (Note for
Ci3 users: the "underscore" notation here acts quite differently from "dot notation" for multiple selects as
is used in Ci3. With our web-based interviewing system, you cannot detect which item in a multiple-
select was selected first, second, etc. as you can with Ci3.)
Constant Sum and Ranking questions also employ underscore notation. For example, in a ranking
question named Q10, if the third item is ranked 2nd, then Q10_3 = 2. Grid-based questions can involve
"double-underscore" notation, to indicate both row and column (for example Q10_r1_c2). (Hint: if you are
uncertain regarding how a variable involving underscore notation is named, you can preview the
associated question and view its source HTML to confirm the variable's name.)
Randomizing items within a multiple-response list does not affect the operation of the system variable
Question_#. The item number after the underscore always refers to that item's position within the
original, unrandomized list.
Unconditional Skips
Sometimes you want to perform a skip that has no conditions (it is always executed). The section below
"Randomly Displaying a Subset of a Block of Questions" is an example that uses an unconditional skip.
Unconditional skips can also be useful, for example, if displaying a follow-up message when the
response is somehow unacceptable. Consider a numeric question (Age) that is used to indicate the
respondent's age. You can create an Explanatory Text question (no response required) on the next page
directly following that numeric question. That question might be named LessThan18 and the text might
state:
"You must be 18 to qualify for this survey, please have the individual in your household who is
18 or older and whose birthday is closest to today continue this survey."
If the numeric value for Age is greater than 17, we want to skip to the next question in the survey (Q1).
But, if the value is 17 or less, LessThan18 is displayed with the text asking for an individual 18 or older.
134 SSI Web v8
An unconditional skip is created from LessThan18 back to Age, as follows:
Skip from: LessThan18

Skip Type: Post-Skip (After the question is answered)
Always skip (checked)
Skip to: Age
Comment: Always skip back to Age from this question.
The survey cannot continue until a valid age is entered. (We aren't advocating this approach from a
survey research methodology standpoint, but have used this example as an illustration of unconditional
skips.)
Skip Example:
Randomly Displaying One out of n Pages
Let's assume you have ten pages of questions, but you want respondents to randomly see just one
page. You could place an unconditional skip after the last question on each page, that skips to the
question directly following the ten pages of questions. Then, simply randomize the pages. When the
questionnaire runs, a random page appears and the skip is executed (skipping out of the series of 10
questions) when the page is submitted.
Skip Example:
Displaying a Random Subset of Questions on the Same Page
This is a somewhat advanced example that illustrates additional capabilities of SSI Web. This example
assumes you have a working knowledge of constructed lists.
Consider a situation in which there were 20 questions, but you wanted respondents to see just five of
these questions, selected randomly. This could be done as follows:
1. Create a predefined list called (for illustration) Integers using Compose | Lists... (the list has
20 items, integers 1 through 20, or letters a through t, etc.).
2. Under Compose | Lists, create a constructed list called NewIntegers with parent list
Integers. Add the Integers list to NewIntegers and then randomize the elements using the
Randomize instruction. Drop all but the first five list elements using the ListMax instruction.
3. Place a pre-skip on each of the 20 questions, which skips the question if:
L i s t Va l u e ( Ne wI n t e g e r s , 1 ) < > X An d L i s t Va l u e ( Ne wI n t e g e r s , 2 ) < > X An d

L i s t Va l u e ( Ne wI n t e g e r s , 3 ) < > X An d L i s t Va l u e ( Ne wI n t e g e r s , 4 ) < > X An d
L i s t Va l u e ( Ne wI n t e g e r s , 5 ) < > X
Where "X" is 1 for the first question, 2 for the second question, etc.
Additionally, if you place all 20 questions on separate pages, you can randomize the order of the pages
by clicking Randomize... from the Write Questionnaire dialog.
Prohibited "Skip Backs"

You cannot skip to any question on the first page.
Notes:
Questions not answered by a respondent evaluate to zero or "" (empty string). Therefore, a skip based
on Q5A < 5 will be true if Q5A was not seen by this respondent (Not Shown( )) or was seen but not
answered (Not Answered( )).
If no questions are shown on a page due to pre-skips, the entire page is skipped as well.
You can conduct skip patterns based on JavaScript status. If JavaScript( ) evaluates to true, then the
respondent has JavaScript enabled. If JavaScript( ) evaluates to false, then JavaScript is disabled.
Note: SSI Web by default requires JavaScript and has an automatic error message (within the Error
Messages tab) that asks respondents who have JavaScript disabled to enable JavaScript before
proceeding.
136 SSI Web v8
1.3.5.2 Loops
Looping lets you specify a list and a group of questions to be asked for each member of that list.
Examples of its use might be:
to ask identical questions for each member of a family or each employee in a company
to ask identical questions about the "first time, second time, and so on" of a purchase, illness,
vacation, or any other repeated occurrence
to obtain ratings for a list of attributes for a set of brands (loop within a loop).
For looping sequences, you'll typically want to show respondents the list element associated with that
loop. You show the text of the list element for the current loop by using the SSI Script instruction [%
LOOPLABEL( ) %]. For loops within loops, you'll need to specify a loop name within the argument (see
details further below) to distinguish which list element you are referring to.
Loops require 2 anchor questions, a list to enumerate (either pre-defined or constructed) and a name.
Loop names must meet the same criteria as SSI Web question names: 24 characters max, no symbol
characters, and must start with a letter. If the constructed list is empty, the loop sequence is skipped.
Looping across a question leads to dot notation question names in the data record. For example, if Q1
is looped over 4 times (for each of 4 elements in a list), the data are stored as Q1.1, Q1.2, Q1.3, and
Q1.4. Using a loop within a loop leads to double dot notation: Q1.1.1, etc., where the outer loop(s) are
represented by the earlier dot(s) in the sequence. Triple dot notation, etc. is possible for a loop within a
loop within a loop.
You can invoke skip patterns within loops, where the target of the skip is the "Next Loop Iteration" or the
"End of Loop". If you are within a question called Q1, you can refer (in skip patterns) to Q1 and it will
take the value from the current loop for Q1 (you do not have to refer to a specific "dot notation" to mean
the current loop iteration).
Restrictions:
It isn't possible to show all looped questions (for all loops) on the same page. (If you have a loop
associated with a four item list for Q1, you always get four pages; you cannot display all four Q1's on a
single page).
Start, Terminate/Link, and Quota questions are not allowed in loops. Conjoint and MaxDiff exercises are
not allowed in loops.
Script Functions within Loops:
Various SSI Script functions are available within loops:
LOOPLABEL(), LOOPLABEL(Loop Name) Returns the label for the list item for the current
loop. This function only works in questions established as part of a loop. The Loop Name is an
optional argument that specifies, in the case that more than one loop applies (e.g. nested
loops), which loop label to return.
LOOPVALUE(), LOOPVALUE(Loop Name) Returns the parent list item number for the
current item in the loop. This function only works in questions established as part of a loop.
The Loop Name is an optional argument that specifies, in the case that more than one loop
applies (e.g. nested loops), which loop value to return.
LOOPITERATION(), LOOPITERATION(Loop Name) Returns the current iteration count of

the current loop. This corresponds to the position number of the current item in the looped list.
For example the first time through the loop this function returns 1, the 2nd time it returns 2 and
so on. This function only works in questions established as part of a loop. The Loop Name is
an optional argument that specifies, in the case that more than one loop applies (e.g. nested
loops), which loop iteration value to return.
AIELoopList(Loop Name, Question Name, Value) Similar to the AIE list function, except
that it adds (to a constructed list) the list item from the loop list that matches instead of the
list item from the question.
AIGLoopList(Loop Name, Question Name, Value) Like the AIG list function, except that it
adds (to a constructed list) the list item from the loop list that matches instead of the list item
from the question.
AILLoopList(Loop Name, Question Name, Value) Like the AIL list function, except that it
adds (to a constructed list) the list item from the loop list that matches instead of the list item
from the question.
ANELoopList(Loop Name, Question Name, Value) Like the ANE list function, except that
it adds (to a constructed list) the list item from the loop list that matches instead of the list
item from the question.
138 SSI Web v8
1.3.5.3 Backing Up and "Limbo" Data
If respondents back up in a survey (or are skipped backward) to re-answer a previous question, answers
to the later questions (which are now only conditionally valid) are thrown into "limbo" status and are not
saved in the final data record (though they are available in the history table). Respondents must re-
answer those questions.
Respondents can back up in a survey by either clicking the browser's back button (or the Previous
button) or by being skipping back to a previous question.
Example:
As respondents answer questions to an SSI Web survey, the answers are recorded in the database like
this:
Q1, 5
Q2, 3
Q3, 2
Q4, 13
Here is an example of backing up and re-answering a previous question:
Q1, 5
Q2, 3
Q3, 2
Q4, 13
Q2, 7 (This respondent has backed up and re-answers Q2)
The data are then cleaned and old data (limbo data) are preserved in the history table. Now the data
record would contain:
Q1, 5
Q2, 7
and the respondent needs to answer Q3 and Q4 again.

1.3.6 Including Graphics

You can display graphics most any place within an SSI Web study. Those who know some HTML
recognize that graphics can be added using the IMG tag. SSI Web automates the way that SSI Web
manages graphics for testing (under Local Test Server) and for running over servers. We recommend you
use SSI Web's automatic ability to manage the "path" to your graphics and that you follow the
instructions below for inserting graphics within your questionnaire.
(Note: Please see the section on Graphics Management for more information regarding how SSI Web
manages graphics and how you can customize how graphics are used and stored on the server.)
Adding Graphics
To add a graphic, click the Add Image icon on the toolbar accessed by first clicking the "Pencil" edit
icon . Then, browse to the folder containing your graphic and double-click the graphics file. When
you do this, the following happens:
1. An HTML <img> tag including SSI Web scripting is inserted within your questionnaire:
<img src="[%GraphicsPath( )%]yourgraphic.jpg" border="0">
where "yourgraphic.jpg" is the name of the graphic file you added and [%GraphicsPath( )%] is
the SSI Web script with tags "[% %]" and system function GraphicsPath( ) that returns the
path from the cgi-bin directory to your graphics directory, as specified under Field | Web
Server Management...
2. The file "yourgraphic.jpg" is copied into a graphics directory directly off of your study
directory.
(Note: .jpg, .gif, .png, etc. can be used.)
You may be tempted to type the <img> tag contents yourself for a new graphic, or cut-and-paste the
<img> tag contents to another part of your survey and alter the referenced graphic. If you do this, make
sure to copy the new graphic into the graphics directory, or the graphic will not appear when testing your
survey. For this reason, you may discover that it is easier to just use the Add Image icon.
We suggest you use the system function GraphicsPath( ) for managing the paths to graphics for your
survey. However, you can edit other aspects of the <img> tag, such as providing size parameters or
changing the border width.
Different browsers support different graphics formats. For best results across different browsers, your
image should probably be saved in .GIF,.JPG., or .PNG format.
Internet browsers display images in their original size; they do not automatically size them to fit within
your interview or in relation to your question area. Plan ahead and size your images to look good under
different screen resolutions.
140 SSI Web v8
1.3.7 List Building

Introduction
List Functions Reference
Market research questionnaires almost always include lists of brands, statements, response options
(scale points), etc. Sometimes, lists of brands or response options need to be used over and over again
within the same questionnaire. It is useful if the list is established and simply reused when writing
subsequent questions. SSI Web saves all lists of response options for your questionnaire and lets you
select from this list when creating new questions.
Some questionnaires require more sophisticated use of lists. For example, one might display a list of
brands and ask respondents which brands they have used recently. Then, only those brands selected
are used in a subsequent question, such as for indicating which of those brands being used is the
"favorite." (Some researchers refer to this as piping.) Such lists need to be customized ("constructed")
for each respondent. SSI Web offers these capabilities.
There are two general classifications of lists in SSI Web:
1) Predefined List - a list that always remains the same for all respondents
2) Constructed List - a list built individually for each respondent (according to instructions
you specify), depending on answers to previous questions or randomization.
Whenever you create a list of response options for a select-type, ranking, constant sum, or grid-style
question, you are creating a list that you can reuse in a later question. For example, if you create a
select-type question Q1:
Q1: What is your gender?
Male
Female
Refused
the responses are automatically stored as a predefined list named Q1List that includes the following list
members:
Male
Female
Refused
This list, once established, can be used in other questions. Note that if you change the list, all
questions using that list are affected.
For straightforward questionnaires, thinking about lists may be unnecessary. But, for complex
questions, the ability to build constructed lists is indispensable.
We've described how predefined lists are automatically stored whenever you create a select-type
question. There is also a general Lists dialog where you can add or edit predefined and constructed
lists. You access this list manager by clicking Compose | Lists from the main menu. You must use
the Lists dialog to create constructed lists.
in Word or Excel, and then (within SSI Web) by clicking the Paste list member(s) from the clipboard
icon .
In our documentation, we'll use the following terminology to refer to aspects of lists:
How Constructed Lists Work
Constructed lists are customized lists of brands, statements, response options, etc. that are built "on-
the-fly" based on the answers that a respondent gives (and that were submitted on a previous page) or
that are built based on randomization or other user-defined instructions. They often include a subset of
the original members in a list. For example, consider a question that asks respondents which colors
they like. The researcher might first create a predefined (parent) list that contains the colors:
Red
Green
Blue
Purple
Yellow
Next, the researcher might create a question called Q1 using that predefined list of colors. A respondent
might answer that question in this manner:
Q1. Which colors do you like?

Red
Green
Blue
Purple
Yellow
This is a multiple response select question and the data are stored internally within SSI Web as follows:
Q1_1, 0
Q1_2, 1
Q1_3, 0
Q1_4, 1
Q1_5, 0
It is useful to review this so you can understand how the system stores data and how to refer to answers
142 SSI Web v8
from previous questions when using skip patterns or displaying the responses to previous questions on
the screen. With multiple-select questions, any checked options are indicated with a "1" and non-
checked options with a "0". "Underscore" notation is used to refer to the list members. For example,
Q1_4 refers to the fourth member in the list (Purple). Even if you decided to randomize the list, "_4"
always refers to the fourth member on the original predefined list. SSI Web always uses indices that
refer to the original list order so that no matter what randomization or sub-selection is employed, you
can easily k eep track of which members have been seen or answered.
With SSI Web's list-building capabilities, you can create a new constructed list containing the colors
chosen in Q1. (Examples and instructions for this are given later in this section.) The constructed list
for this respondent, would contain the members this respondent chose in Q1:
Green
Purple
In Q2 (placed on a page following Q1), we might use this list when asking this respondent which of the
colors previously selected is her favorite:
Q2. Which of the colors you previously said you liked is your
favorite?
Green
Purple
If this respondent answers that Purple is her favorite color, the data are stored internally within SSI Web
as follows:
Q2, 4
Note that the response to Q2 is coded as a "4," because Purple is member #4 on the original parent list
of colors.
Displaying the Text for List Members in Questions
Most of the time, you will just show an entire predefined or constructed list in a question. To display the
list as response options to a question, you'll simply select it from the drop-down control of available lists
while editing the question.
Occasionally, you may want to display the label for a particular list member on the screen.
The following SSI Web script returns the label for the list member:
[%ListLabel (ListName, member#)%]
When the survey runs, SSI Web replaces the entire expressions above with the requested label.
Skip Patterns for Questions That Use Constructed Lists
Constructed lists must be built using list members from only a single predefined ("parent") list. When
referring to responses to a question involving a constructed list, we just refer to the list values on the
original parent list.
For example, consider the following question called Vehicles:
<Question: Vehicles>
Which of the following vehicles would you consider buying?
…that uses a predefined list called VehiclesList:
Honda Accord
Mazda 626
Saturn SL2
Toyota Corolla
Ford Focus
We might build a new constructed list called ConsideredList that contained only those models that the
respondent chose in question Vehicles. Then, we might ask a new question called MostLikely:
<Question: MostLikely>
Of those vehicles you are considering buying, which do you think you are most likely to
purchase?
<Display ConsideredList, containing only those vehicles chosen in question Vehicles.>
Let's assume we needed to execute a skip if the respondent chose Saturn SL2 as the most likely vehicle
in question MostLikely. You do not know how many vehicles are in ConsideredList for each
respondent or what position Saturn SL2 might occupy on that list. But this is no matter, as you simply
need to refer to the original value from the parent predefined list Vehicles. Saturn SL2 is associated with
a value of 3 on the parent list. Therefore, we would skip if MostLikely = 3.
With multi-response select questions that use constructed lists, you may want to skip based on whether
a particular list member was chosen. Again, you refer to values on the original predefined parent list. If
we wanted to skip if a member with a value of 12 from the original parent list was chosen in multi-select
question Q23, we would skip if Q23_12 = 1. Since constructed lists draw members from a single
predefined parent list, there is no ambiguity.
Summary of List-Building Instructions
Below, we first provide a summary of commonly-used list-building instructions, followed by a more

detailed description of these commands. Following that, we provide some questionnaire-programming
examples.
Constructed lists must be built using list members from only a single predefined ("parent") list.
ADD
Adds members from a specified list to the end of the constructed list.
AIC ("Add If Chosen"), ANC ("Add if Not Chosen")

Adds members chosen (AIC) or not chosen (ANC) in a select-type question onto a new
constructed list.
AIE ("Add If Equal"), ANE ("Add if Not Equal"), AIL ("Add If Less"), AIG ("Add if Greater")
144 SSI Web v8
Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons,
constant sum, or ranking question that were given answers equal to (AIE), not equal to (ANE),
less than (AIL) or greater than (AIG) a specific value.
INSERT
Inserts members from a specified list to an exact position within a constructed list.
LISTMAX , LISTMIN, SETLISTLENGTH

Controls the maximum or minimum length of a list.
RANDOMIZE
Randomizes the members of a constructed list.
REMOVE
Removes the members in the specified list from the constructed list.
REVERSE
Reverses the order of the members in the specified list.
SORTBYVALUE
Sorts members of a constructed list into their natural order (as in the pre-defined list)
Rules Governing List Operations
1. All constructed lists are initially empty.

2. All members added to a constructed list must come from a common parent list.
3. You may add members from another constructed list to a constructed list, as long as both
constructed lists share the same parent predefined list.
4. A member can only appear once in the constructed list (if you try to add a member after it
already exists on a constructed list, it will not be added again).
5. List building commands are executed sequentially.
6. A question using an empty constructed list (as response options) is skipped.
When you view questions involving constructed lists in Preview mode, all members from the parent list
are displayed (since we don't know how the respondent has answered previous questions).
List Instructions
In this section, we describe each list building instruction in more detail. These instructions build or
modify constructed lists. Remember, a key rule for building lists is that the members must originate
from the same predefined list.
ADD
ADD is used to add a specific member or members from a list (either predefined or constructed) to the
constructed list.
The formats are:
ADD (ListName)
ADD (ListName, Member#)
ADD (ListName, StartMember#, EndMember#)
The referenced list can be either a predefined or constructed list. The member numbers refer to the
current member positions on the referenced list. (If you are adding members from a constructed list, the
member numbers refer to the current member positions on that constructed list, not the members'
original positions in the predefined list.)
StartMember# and EndMember# are optional and are used to designate a range of list members to add
to the constructed list. If these two arguments are omitted, all list members are added to the
constructed list. If only one argument is supplied, then only that one list member is added to the
constructed list.
If the current constructed list already contains the list member to be added, then the duplicate list
member is not added to the list. The original member remains in its position.
See Examples B and E at the end of this section for illustration.
AIC and ANC
Adds members chosen (AIC) or not chosen (ANC) in a select-type or grid question onto the constructed
list. AIC stands for "Add If Chosen" and ANC stands for "Add if Not Chosen."
The formats are:
AIC (QuestionName)
AIC (QuestionName, Member#)
AIC (QuestionName, StartMember#, EndMember#)
ANC (QuestionName)
ANC (QuestionName, Member#)
ANC (QuestionName, StartMember#, EndMember#)
The question named must be a select-type question. If from a grid-style question, appropriate column
numbers or row numbers must be included in the question label, such as Q1_c1 (for column-based grid
question Q1) or Q1_r1 (for row-based grid question Q1). If from a MaxDiff question, the question label
follows these conventions: ExerciseName_TaskNumber_b (for "best" response) or
ExerciseName_TaskNumber_w (for "worst" response).
The last two arguments are optional and are used to designate a range of list members to add to the
constructed list. If these two arguments are omitted, all qualifying list members are added to the
constructed list. If only one argument is supplied, then only that one list member (if chosen or not
chosen in questionname) is added to the constructed list.
The member numbers in the arguments always refer to the original predefined parent list (the parent list
indices), not the particular position number that member happens to occupy in the list used in the
referenced questionname.
Members are added to the constructed list in the order they were displayed to the respondent (unless
using question-level randomization with pre-defined lists).
146 SSI Web v8
If the member to be added to a constructed list is an Other Specify response, the text supplied by the
respondent is added. This is an automatic and helpful feature. Technical Note: if there is just one Other
Specify response in a predefined list, but two or more questions (using the same predefined list) collect
multiple answers for this same Other Specify member, there is a potential conflict if you try to add the
Other Specify members across multiple questions onto a constructed list. As functions are executed
sequentially, the other response referred to by the last function is used. Using ADD or INSERT always
forces the open-end box to appear with Other Specify responses.
See Examples A, D, and E at the end of this section for illustration.
AIE, ANE, AIL, AIG
Adds members from a question that were given answers equal to (AIE), not equal to (ANE), less than
(AIL) or greater than (AIG) a specific value.
The formats are:
AIE (QuestionName, Value)

ANE (QuestionName, Value)
AIL (QuestionName, Value)
AIG (QuestionName, Value)
Where QuestionName can be:
constant-sum (including within grids)

ranking (including within grids)
grid-style radio (where all rows or columns are radio)
grid-style numeric
If using a grid-style question that only uses radio buttons, the questionname is referred to as the original
grid name, without any "underscore" notation to indicate row or column numbers. If it is a row-based
question, all rows members meeting the qualification are added to the constructed list. If column-based,
members in columns are added to the constructed list.
Sometimes, a grid question might use numeric entry fields. When numeric entries are part of a grid-
style question (including ranking and constant-sum questions within grid questions), the row or column
must be referenced in the AIE, ANE, AIL, or AIG instruction using underscore notation. For example:
Q1. How many of these types of fruits do you typically eat at work and at home during an
entire week?
Home Work
Apples _____ _____
Bananas _____ _____
Peaches _____ _____
Let's assume we needed to create a new list containing the fruits that an individual eats at least 3 times
or more at work. We'd use the instruction:
AIG (Q1_c2, 2)
Since this is a column-based grid question, row members are added to the constructed list.
Members are added to the constructed list in the order they were displayed to the respondent (unless
using question-level randomization with pre-defined lists).
See Example C at the end of this section for illustration.
INSERT
Inserts members from a specified list (either predefined or constructed) to an exact position within a
constructed list.
If a member to be inserted already exists on the constructed list, it is first deleted from the constructed
list. Then, it is added at the indicated position.
The formats are:
INSERT (TargetPosition#, ListName)
Adds a list (either predefined or constructed) to the constructed list starting in the target position number
indicated.
INSERT (TargetPosition#, ListName, Member#)
The member from the referenced list (either predefined or constructed) is inserted into a specific position
on the constructed list. For example, consider inserting the color "Red" into the constructed list [Purple,
Green] at the second position. The result is [Purple, Red, Green].
INSERT (TargetPosition#, ListName, StartMember#, EndMember#)
Here, a range of elements are moved from the referenced list (either predefined or constructed) to a target
position in the constructed list.
LISTMAX , LISTMIN, SETLISTLENGTH
Controls the maximum or minimum length of a list.
The format is:
LISTMAX (Value)
LISTMIN (Value)
SETLISTLENGTH (Value)
The LISTMAX instruction is useful if you want a list to contain no more than a certain number of
members. If a list has more members than is specified in the LISTMAX instruction, the extra members
are removed from the bottom of the list.
148 SSI Web v8
For example, assume a predefined list with 50 European cities, of which we want respondents to rate
exactly 10 of the cities chosen randomly. We first would ADD all fifty cities onto the new constructed
list. Next, we would RANDOMIZE( ) the list of cities. Finally, we would use the LISTMAX instruction:
LISTMAX (10)
to remove the bottom 40 members from the constructed list, leaving the first 10 for respondents to rate.
The LISTMIN instruction ensures that a minimum number of elements exist on a constructed list by
adding members (if needed) chosen randomly from the parent list.
For example, assume that we'll ask respondents to rate 10 cities from a total list of 50 cities. Further
assume that we'll want to ask respondents to rate cities with which they are familiar. We might first ask
respondents to check such familiar cities. If they check fewer than 10, we want to randomly add
additional cities to the list to be rated.
To accomplish this, we use the AIC instruction to add any cities to the list that were chosen in a select-
type familiarity question. Finally, we would use the LISTMIN instruction:
LISTMIN (10)
to add new members (if needed) onto the list so that there were at least 10 members on the constructed
list. If any members are added, they are chosen randomly from the parent list among members not
already existing on the constructed list that are not "None of the above" or "Other specify" items (SSI
Web never permits duplicate members on a constructed list).
You can set a list length to a certain value by using the LISTMAX and LISTMIN commands together (with
the same fixed integer as the argument). However, a simpler way to accomplish this is to use the
SETLISTLENGTH instruction. Elements that need to be deleted from or added to the list are treated in
the same way as with LISTMAX and LISTMIN instructions (if list members need to be randomly added
they will not be chosen if they are "None of the above" or "Other Specify").
See Example D at the end of this section for illustration.
MIRROR
Adds list members from the parent list that correspond to the list values in ListName.
The format is:
MIRROR (ListName)
MIRROR (ListName, Offset)
Mirror is useful to map or mirror one list onto another when those lists have different parents. For
example suppose that you have a list of colors:
ColorNamesList
1 - Red
2 - Blue
3 - Green
4 - Purple
5 - White
and you also have a list of images
ColorImagesList
1 - <img src="[%GraphicsPath()%]red.jpg">
2 - <img src="[%GraphicsPath()%]blue.jpg">
3 - <img src="[%GraphicsPath()%]green.jpg">
4 - <img src="[%GraphicsPath()%]purple.jpg">
5 - <img src="[%GraphicsPath()%]white.jpg">
In addition there is a constructed list named ColorsChosenList. When the survey is run imagine that it
had items 1,3, and 4. If you wanted to create a constructed list from ColorImagesList that corresponded
to the same items in ColorsChosenList you could do this:
MIRROR (ColorsChosenList)
This command would add these items:
1 - <img src="[%GraphicsPath()%]red.jpg">
3 - <img src="[%GraphicsPath()%]green.jpg">
The optional Offset parameter allows you to alter the mapping adding the value specified in Offset to
each value in ListName. For example:
MIRROR (ColorsChosenList, 1)
would return:
2 - <img src="[%GraphicsPath()%]blue.jpg">
5 - <img src="[%GraphicsPath()%]white.jpg">
RANDOMIZE
Randomizes the members of a constructed list.
The formats are:
RANDOMIZE ( )
RANDOMIZE (SeedValue#)
RANDOMIZE (StartMember#, EndMember#)
RANDOMIZE (SeedValue#, StartMember#, EndMember#)
Many times, you'll want list members to be presented randomly to avoid order bias.
Specifying RANDOMIZE ( ) randomizes all elements on the constructed list. Specifying a starting and
ending member randomizes a subset of the list. (Remember, a constructed list initially is empty. Prior
to randomizing a list of members, they must first be added to the constructed list.)
If you do not provide a seed value, then the list name is used as a seed. The seed value and the internal
150 SSI Web v8
respondent number are used to ensure that the randomization will be different across lists and
respondents. The seed value can range from (1 to 99999). Using a particular seed value ensures that
you can obtain the same randomization across different lists (containing the same number of members)
for the same respondent. (Warning: if you use multiple RANDOMIZE instructions when building a
constructed list and if you are explicitly specifying seed values, take care to use different seed values.
Using the same seed value in this case will likely result in poor randomization of the constructed list.)
There is a more simple way to randomize members of a list: by clicking the Randomize Response
Options box on the Response Options tab when editing a select-type question. This method does not
involve constructed lists, but it also doesn't let you store and later analyze the randomized order of the
members (as is done when you randomize constructed lists). Also, when you re-use a predefined list of
members and click the Randomize Response Options box, you obtain a different random sequence.
Sometimes, you may want to re-use a list in a survey with the same randomized order for each
respondent. Using constructed lists and the RANDOMIZE instruction can accomplish that.
Once RANDOMIZE has been used to randomize elements of a constructed list, other list-building
instructions can be used to add or delete members from this new list.
See Examples B, C, D, and E at the end of this section for illustration.
REMOVE
Removes the members in the specified list from the constructed list.
The formats are:
REMOVE (ListName)
REMOVE (ListName, Member#)
REMOVE (ListName, StartMember#, EndMember#)
The REMOVE instruction removes or deletes a member or members in the specified list (either
predefined or constructed) from the current constructed list. If the members to be removed are not in the
list, then nothing happens.
A common use for REMOVE involves removing a single member (such as a "None of the above"
response from a constructed list. Consider a predefined list named hotels, whose 20th member is "None
of the above." Assume you wanted to remove the "None of the above" response from the new
constructed list we are building (whose parent list is hotels). You would use the following instruction:
REMOVE (hotels, 20)
You can also use the REMOVE command to "subtract" a series of members of one list from another.
The list to be subtracted can be a predefined list or a constructed list. For example, if you are
constructing a list and want to subtract elements 5 through 10 of another constructed list named Q17Sel
from your constructed list, you would specify:
REMOVE (Q17Sel, 5, 10)
If the REMOVE instruction refers to a constructed list, the parent list for both that referenced list and the
constructed list you are building must be the same.
See Examples A and E at the end of this section for illustration.

Example A:
Asking a follow-up question using brands selected previously
Learning Objectives:
a) Illustrates the following functions:
- AIC "Add If Chosen"

- REMOVE
b) Demonstrates how to take brands previously selected forward to a follow-up select-type

question.
c) Show how respondent-supplied text for Other Specifies can be used in follow-up questions
d) Illustrates how follow-up questions using an "empty" constructed list are automatically
skipped.
The Questionnaire:
<Question: LastSix>
Which of the following fast-food restaurants have you visited in the last 6 months? <Randomize order for
options 1 through 9. "None of the above" is exclusive.>
1. A&W
2. Arby's
3. Burger King
4. Dairy Queen
5. Hardee's
6. McDonald's
7. Subway
8. Taco Bell
9. Wendy's
10. Other Specify (1) ______________________
11. Other Specify (2) ______________________
12. None of the above
<Question: MostOften>
Of those you have visited in the last 6 months, which one do you visit most often?
<Show restaurants chosen in previous question. Include Other Specify responses.>
Solution:
For this illustration, we assume you already know the basics of building questions in CiW. Therefore, we
only provide details regarding the more advanced functions of list building.
1. LastSix uses a list of twelve response members (fast-food restaurants) that you must specify as a
predefined list. Specify this predefined list by adding LastSix to your questionnaire as a select-type
question (check-box select format) and while editing LastSix using the Response Options tab, or you
can directly add the predefined list under Compose | Lists (the List Manager dialog). In either case,
you are establishing a predefined list of members that we can use for LastSix and elsewhere in the
152 SSI Web v8
questionnaire. We'll use this predefined list as a source "parent list" for building the constructed
(customized) list of fast-food restaurants we'll use for question MostOften. Call the predefined list
LastSixList (which is the default name for the list if adding it from the Response Options tab for
LastSix). CiW is not case sensitive with respect to list names.
2. After specifying the 12 list members in LastSixList, highlight member 10 ("Other Specify (1)") and
check the available box to indicate that it is an "other specify" response. Repeat for the additional
Other Specify list member 11. For list member 12, highlight it and check the box to indicate that it is
"Exclusive" (meaning that if respondents check this category, it is the only check allowed). We also
need to randomize members 1 through 9, and we can do this by clicking Randomize Response
Options from the Response Options tab. (Note: the "None of the above" and "Other specify"
members automatically will not be randomized if they are the last members on the list.)
Add a page break after LastSix. Remember, MostOften must be displayed on a page after LastSix,
since the respondent needs to submit the answers from LastSix before we can know how to
customize the list of members for MostOften.
3. The list of members to be displayed in MostOften must be customized (constructed) for each
respondent, depending on the answers given in question LastSix. To accomplish this, we create a
new constructed list. From the main menu, click Compose | Lists to access the Lists dialog. Click
Add and then you should select Constructed List. Name the new list ChosenRestaurantsList. The
key rule to remember when constructing new lists is that all the response options (list members)
moved into this new list must originate from the same predefined (parent) list. For this questionnaire,
all the response options you'll use to build ChosenRestaurantsList come from LastSixList. Using
the drop-down Parent List control, select the parent list LastSixList.
To begin with, the constructed list is empty even though you have specified that you'll be using
members from LastSixList. Type the list building instructions that create the new
ChosenRestaurantsList:
AIC (LastSix)
REMOVE (LastSixList, 12)
Any instructions you use are processed one-at-a-time, in a linear fashion, started from the first
instruction and working through the last instruction. You can check your syntax by clicking the
"Check" icon.
The first instruction tells SSI Web to "Add If Chosen" any members chosen in the question LastSix
onto the new constructed list.
The second instruction tells SSI Web to remove the 12th member (None of the above) of the predefined
list LastSixList from the new constructed list. This remove step is necessary because we don't want
to display the MostOften question with the member "None of the above" to respondents who didn't
visit any restaurants in the last six months. SSI Web automatically skips a question that uses an
empty constructed list, so respondents who did not visit any restaurant are not asked the follow-up
question MostOften. No additional skip logic is required.
(Alternatively, we could have used the single instruction AIC (LastSix, 1, 11) to indicate that only
members 1 through 11 should be moved onto the new constructed list.)
4. Add the question MostOften as a select-type (radio) question. From the Response Options tab,
select that this question should use as response options the existing list ChosenRestaurantsList.
Example B:
Asking a question for each member of a randomized list, using a grid-style
question
- ADD
- RANDOMIZE
b) Demonstrates how to randomize a list of members to be shown in a question.
c) Demonstrates how to use a constructed list in a grid-style question.
The Questionnaire:
<Show grid, where rating scale is in the columns and statements are in the rows. Randomize order of
statements.>
Solution:
This example does not need to employ constructed lists at all, since you can simply request that the
members be randomized within grid questions. However, to illustrate constructed lists, we provide this
as a simple example. Using constructed lists in this case does provide an advantage over simply
requesting that the rows be randomized in a grid-type question. If you create a constructed list with the
randomized members, information describing the order of the members for respondents is saved and can
be analyzed. Also, if you wanted to show this same randomized list in another part of the questionnaire
(preserving the randomized order within respondents), you could do so when using constructed lists.
154 SSI Web v8
1. The question in this example uses a list of six statements to describe a commercial respondents
have just seen. You'll need to specify these six members as a predefined (parent) list. You can
either specify the list while writing the question, or from the main menu by clicking Compose | Lists
(accessing the List Manager dialog). In either case, you are establishing a predefined list of members
that you can use in this question and elsewhere in the questionnaire. For this example, we suggest
you use the list manager (Compose | Lists, then select Predefined List) to build the predefined list
named StatementsList containing the six statements. (CiW is not case sensitive with respect to list
names.)
2. Next, we need to create a new constructed list that includes all the elements from our predefined
list StatementsList, but in randomized order. From the main menu, click Compose | Lists to access
the Lists dialog. Click Add, and then select Constructed List. Name the new list
RandStatementsList. The key rule to remember when constructing new lists is that all the response
options (list members) moved into this new list must originate from the same predefined (parent) list.
For this questionnaire, all the response options you'll use to build RandStatementsList come from
StatementsList. Using the drop-down Parent List control, select the parent list StatementsList.
members from StatementsList. Type the list building instructions that create the new constructed
list:
ADD (StatementsList)
RANDOMIZE ()
These instructions add all members from the predefined list StatementsList onto the current
constructed list (RandStatementsList). Then, all members are randomized.
3. Finally, you need to add the grid-style question to the questionnaire, which uses row-based
question direction, with all radio buttons. While editing the question, on the Rows tab, select that you
want to use the constructed list RandStatementsList.
Example C:
Adding brands to a list that were rated highly in a grid question
- AIG ("Add If Greater")

- RANDOMIZE
The Questionnaire:
<Question: FamRx>
<Question: Effective>
Solution:
1. The question uses a list of five drugs. Specify this predefined list by adding this question (call it
FamRx) to your questionnaire as a grid-type question (row-based question direction) and while
editing FamRx and within the Rows tab, adding the list members. Alternatively, you can directly add
the predefined list under Compose | Lists (the List Manager dialog). In either case, you are
establishing a predefined list of members that we can use for FamRx and elsewhere in the
(customized) list of familiar drugs. Call the predefined list FamRxRowList (which is the default name
for the list if adding members from the Rows tab for FamRx). CiW is not case sensitive with respect
to list names.
Add a page break after question FamRx.
2. The list of members to be displayed in Effective must be customized (constructed) for each
respondent, depending on the answers given in question FamRx. To accomplish this, we create a
new constructed list. From the main menu, click Compose | Lists to access the Lists dialog. Click
Add, and then select Constructed List. Name the new list FamiliarDrugsList. The key rule to
remember when constructing new lists is that all the response options (list members) moved into this
new list must originate from the same predefined (parent) list. For this questionnaire, all the response
options you'll use to build FamiliarDrugsList come from FamRxRowList. Using the drop-down
Parent List control, select the parent list FamRxRowList.
156 SSI Web v8
members from FamRxRowList. Type the list building instructions that create the new
FamiliarDrugsList:
AIG (FamRx, 2)
RANDOMIZE ()
These instructions add any members from question FamRx rated greater than a 2 to the constructed
list and then randomize their order.
(Note: Remember that SSI Web codes the values for grid questions sequentially from 1 on the left-
most column to 4 in the fourth column. If you had reversed your labels to display a 4 on the left and a
1 on the far right, SSI Web would not pay attention to your labeling and would still consider the ratings
as running from 1 to 4 from left to right.)
3. Finally, add a new grid question after the page break named Effective (row-based question
direction) with radio buttons and specify that it uses the new constructed list FamiliarDrugsList from
the Rows tab.
Example D:
Asking follow-up grid questions about brands chosen in a previous question
- AIC ("Add If Chosen")

- LISTMAX
- RANDOMIZE
b) Demonstrates how to include a subset of brands selected in follow-up grid-type questions.
c) Demonstrates how to limit the number of brands used in subsequent questions to no more
than a maximum number. Shows how to force the client's brand onto the list, if respondents
have visited the client's restaurant.
The Questionnaire:
Q10. Which of the following fast-food restaurants have you visited in the last 6 months? <Randomize
order for options 1 through 9. "None of the above" is exclusive.>
A&W
Arby's
Burger King
Dairy Queen
Hardee's
McDonald's
Subway
Taco Bell
Wendy's
Other Specify (1) ______________________
Other Specify (2) ______________________
None of the above
Next, we'd like to ask you to evaluate the restaurants you have visited in the last six months.
Q11a. How would you rate the restaurants you have visited in terms of Cleanliness?
1= Poor
2= Fair
3= Good
4= Excellent
Q11b. How would you rate the restaurants you have visited in terms of High Quality Food?
Q11c. How would you rate the restaurants you have visited in terms of Good Value for Money?
Q11d. How would you rate the restaurants you have visited in terms of Enjoyable Atmosphere?
<For Q11a through Q11d, use grid layout, with restaurants chosen in Q10 in the rows and the rating
scale in the columns. Randomize order of presentation of restaurants, with a maximum of 5 restaurants
to be rated. Always include Subway if respondent chooses Subway in Q10. Do not ask respondents to
rate any "Other Specify" restaurants from Q10. Randomize the order of presentation of questions Q11a
through Q11d and place each on a separate page.>
Solution:
1. Q10 uses a list of twelve response members (fast-food restaurants) that you must specify as a
predefined list. Specify this predefined list by adding Q10 to your questionnaire as a select-type
question (check box select format) and while editing Q10 using the Response Options tab, or you can
directly add the predefined list under Compose | Lists (the List Manager dialog). In either case, you
are establishing a predefined list of members that we can use for Q10 and elsewhere in the
(customized) list of fast-food restaurants we'll use for Q11. Call the predefined list RestaurantsList.
CiW is not case sensitive with respect to list names.
2. After specifying the 12 list members in RestaurantsList, highlight member 10 ("Other Specify (1)")
and check the available box to indicate that it is an "other specify" response. Repeat for the
additional Other Specify list member 11. For list member 12, highlight it and check the box to
indicate that it is "Exclusive/None of the above" (meaning that if respondents check this category, it is
the only check allowed). We also need to randomize members 1 through 9 and we can do this by
clicking Randomize Response Options from the Response Options tab. (Note that the "None of the
Above" and the "Other Specify" are automatically not randomized if they are at the bottom of the list.)
Add a page break after Q10.
3. For Q11, we need to show the text "Next, we'd like to ask you to evaluate the restaurants you
have visited in the last six months." at the top of the question, but we'll be randomizing the
presentation of the remaining sections (subsections a through d). Specify a Text/HTML Filler
question called Q11intro and include the introductory sentence "Next, we'd like to ask you to
evaluate the restaurants you have visited in the last six months."
158 SSI Web v8
4. The most challenging aspect of this example is to create the customized list of restaurants to
include in Q11a through Q11d. We need to:
Include restaurants chosen in Q10 (up to a maximum of 5)

Randomize order of presentation of restaurants
Always include Subway if respondent chooses Subway in Q10
Do not ask respondents to rate any Other Specify restaurants from Q10.
To accomplish this, we create a new constructed list to use in Q11a. From the main menu, click
Compose | Lists to access the Lists dialog. Click Add, and then you should select Constructed List.
Name the new list VisitedRestaurantsList. The key rule to remember when constructing new lists
is that all the response options (list members) moved into this new list must originate from the same
predefined (parent) list. For this questionnaire, all the response options you'll use to build
VisitedRestaurantsList come from RestaurantsList. Using the drop-down Parent List control, select
the parent list RestaurantsList.
members from RestaurantsList. Type the list building instructions that create the new
VisitedRestaurantsList:
AIC (Q10, 7)
AIC (Q10, 1, 9)
RANDOMIZE (2, 9)
LISTMAX (5)
RANDOMIZE ()
We describe these instructions one at a time below.
AIC (Q10, 7)
First, we force Subway (the 7th member on the parent list) onto the constructed list if it is chosen
(you'll see further below why it was important to force it initially into the first position).
AIC (Q10, 1, 9)
Next, we add any other members (but not including the "Other Specifies" or "None of the above")
chosen in Q10. These new members are placed on the constructed following Subway, if Subway was
chosen. Note that if a list member (Subway) already exists on the constructed list, if a new
instruction attempts to add the same member, it will not be added again (no duplicates are permitted).
RANDOMIZE (2, 9)
We always want to include Subway on the final list (if chosen). Subway, if chosen, is in the first
position on the constructed list. Other brands (following in positions 2 up to 9) are randomized.
ListMax (5)
We only want to ask about a maximum of 5 restaurants in Q11, so we trim the list using the ListMax
instruction (which always trims from the bottom).
Randomize ( )
Finally, we randomize the remaining elements on the list, so that Subway (if chosen) isn't always in
the first position.
4. Add Q11a through Q11d as grid-style questions to the questionnaire. Each of these uses the
same setup and constructed list. Remember for each question to use the constructed list
VisitedRestaurantsList as the row members. The only aspect that changes across the questions is
the performance attribute referred to in the heading text (such as Cleanliness, High Quality Food,
Good Value for Money, Enjoyable Atmosphere). The easiest way to accomplish this is by copying
and pasting Q11a within the list of questions. Highlight Q11a and click the Copy icon (or
press Ctrl-C), then click the Paste icon (or press Ctrl-V). A copy of the question is placed in
the question list, but with a numeric index after the name. You can change the default name while
editing the question, by clicking the Rename button. Create three more copies of Q11a and change
the wording in the header of each to insert "High Quality Food," "Good Value for Money," and
"Enjoyable Atmosphere."
Insert a page break between the Q11a to Q11d questions and randomize the order of these questions
by clicking the Randomize…Pages button on the Write Questionnaire dialog and selecting Q11a
and Q11d as the anchor questions for the randomization sequence.
Example E:
Gas station questionnaire
- ADD
- AIC ("Add If Chosen")
- ANC ("Add if Not Chosen")
- RANDOMIZE
- REMOVE
b) Uses skip patterns to skip a question and to terminate disqualified respondents
c) Demonstrates how respondent-supplied "other specify" answers can be used as response

members in subsequent questions.
The Questionnaire:
Q1. Which of the following gas stations have you visited in the last year to purchase fuel for your
vehicle?
<Select all that apply. Randomize options 1 through 5. Response option 9 is exclusive. Terminate if
don't ever purchase fuel.>
1 Chevron
2 Shell
3 Sinclair
4 Texaco
5 Union 76
6 Other Specify (1) _________________________
160 SSI Web v8
7 Other Specify (2) _________________________

8 Other Specify (3) _________________________
9 I don't ever purchase fuel (none of the above)
Q2. Of those that you have visited, which gas station do you use most often?
<Select one. Show gas stations chosen in Q1, including any "other specifies". >
Q3. Among those gas stations you have not visited in the last year, which of these are available in your
community? <Show all gas stations not selected in Q1. Select all that apply.>
Q4. What are the main reasons you haven't visited Texaco, even though it is available in your
community?
<Open-end. Ask if not visited Texaco in last year and is available within respondent's community>
Q5. Thank you, those are all the questions we have for you today.
Solution:
only provide details regarding the more advanced functions of list building and skip patterns.
1. Q1 uses a list of nine response members (gas stations) that you must specify as a predefined list.
Specify this predefined list by adding Q1 to your questionnaire as a select-type question (check box
select format) and while editing Q1 using the Response Options tab, or you can directly add the
predefined list under Compose | Lists (the List Manager dialog). In either case, you are establishing a
predefined list of members that we can use for Q1 and elsewhere in the questionnaire. We'll use this
predefined list as a source "parent list" for building subsequent constructed (customized) lists we'll
need for later questions in this example. Call the predefined list StationsList. CiW is not case
sensitive with respect to list names.
2. After specifying the 9 list members in StationsList, highlight member 6 ("Other Specify (1)") and
check the available box to indicate that it is an "other specify" response. Repeat for the additional
Other Specify list members 7 and 8. For list member 9, highlight it and check the box to indicate that
it is "Exclusive/None of the Above" (meaning that if respondents check this category, it is the only
check allowed).
Also note that you need to make Q1 a multi-select (Check Box) question and this is specified on the
Settings tab.
3. Prior to showing the response members in Q1, you need to randomize the order of the gas
stations (members 1 to 5). There are two ways to do this. You can either randomize the members
for this question only (question-level randomization, from the Response Options tab while editing
question Q1), or you can create a new list (a "constructed" list) to use in Q1 using the response
members from your predefined list StationsList and then randomize the subset of members on that
new list. The advantages of using the list-building approach to randomizing the list of gas stations for
Q1 are that you can use this same randomized list later in the questionnaire, should you want to
present this list again using the same randomized order for that respondent. Also, should you want to
know the order of the randomized members for each respondent, that information would be available in
the database. Although it is easier to use question-based randomization, let's use list building
operations with constructed lists for illustration.
From the main menu, click Compose | Lists… to open the Lists dialog. Click Add, and then you
should select Constructed List. Name the new list RandStationsList. The key rule to remember
when constructing new lists is that all the response options (list members) moved into this new list
must originate from the same predefined (parent) list. For this questionnaire, all the response options
you'll use to build RandStationsList come from StationsList. Using the drop-down Parent List
control, select the parent list StationsList. Next, type the list building instructions that create the
new RandStationsList. To begin with, the constructed list is empty even though you have specified
that you'll be using members from StationsList. For this example, you first need to move the
members from the parent list onto the new list, and following that need to randomize members 1
through 5. The following instructions typed into the Constructed List Instructions edit box provided
accomplish this:
ADD (StationsList)
RANDOMIZE (1, 5)
Any instructions you use are processed one-at-a-time, in a linear fashion, started from the first
instruction and working through the last instruction. You can check your syntax by clicking the
"Check" icon.
"ADD (StationsList)" takes all list members from the predefined StationsList and adds them to
the constructed list you are building "RandStationsList".
"RANDOMIZE (1, 5)" randomizes elements 1 through 5 of the current constructed list.
Note that switching the order of these instructions would matter. The randomize instruction would be
meaningless in the first position, as there would be no members yet in the constructed list to
randomize. (ADD adds the list members.)
4. Now that you have created the list containing the response options to use in Q1 (the constructed
list, RandStationsList), you need to tell CiW to use that list within Q1. If you haven't already done
so, create a new select-type question called Q1. While editing Q1, click the Response Options tab.
Click the Existing List button and using the drop-down control select to use RandStationsList as
response options for Q1. The list members from the parent list are displayed as a reference.
5. In select-type question Q2, you need to show gas stations that were chosen in Q1. This list of
response options will obviously vary per person, so we also must build a constructed list for Q2.
Click Compose | Lists… to access the Lists dialog. Add a new constructed list named
VisitedStationsList. The parent (predefined) list that this new constructed list will draw from is
StationsList. Use the AIC ("Add If Chosen") command to add any members chosen in Q1 to this
new constructed list:
AIC (Q1)
Then, when editing Q2 in the Write Questionnaire area, select the VisitedStationsList constructed
list as the response options.
Note that when the AIC operates on an Other Specify response, SSI Web automatically adds the
respondent's typed response as a member on the constructed list rather than the original list text.
The original list member value is also recorded in the database.
As you add new questions to this questionnaire, insert a page break between each question by
clicking the Add Page Break button.
6. In select-type question Q3, you need to show gas stations that were not chosen in Q1. This list of
162 SSI Web v8
response options obviously varies per person and so we must build another constructed list. Click
Compose | Lists to access the Lists dialog. Add a new constructed list named NotVisitedList. The
parent (predefined) list that this new constructed list will draw from is StationsList. Use the ANC
command "Add if Not Chosen" to add any members not chosen in Q1. We would not want to display
any not-selected Other Specify or "none of the above" responses in Q3, so we'll use the REMOVE
instruction to remove members 6-9 (the "other specifies" and "none of the above") from the list:
ANC (Q1)
REMOVE (StationsList, 6, 9)
"ANC (Q1)" takes any members not selected in Q1 and adds them to this new constructed
list.
All members not selected in Q1 have been added to the constructed list with the previous
instruction. Since you don't want any of the "other specify" or "none of the above" response
members to appear on this new list, use the instruction "REMOVE (StationsList, 6, 9)" to
remove members 6 through 9 of StationsList from this new constructed list.
7. Q4 is an open-end question and Q5 is a terminating question type. They are involved in skip
patterns and it makes sense to add these questions to the questionnaire before specifying the related
skip logic.
We only want to ask Q4 if the respondent did not visit Texaco in the last year and it was available
within their community. We could place a skip instruction on Q4 to determine whether Q4 needs to
be skipped. This is called a "pre-skip." Before the question loads on a page, SSI Web determines
whether this question should be displayed or skipped. For this questionnaire, a "pre-skip" could be
associated with Q4 that skips Q4 if: Q1_4=1 OR Q3_4=0. In other words, skip Q4 if Texaco was
chosen in multiple-select question Q1 (Texaco was visited) or if Texaco was not chosen in Q3
(Texaco was not available in the community). Make sure to use the drop-down box on the skip logic
tab to indicate that the skip type is Pre-Sk ip (executed before the question is shown) and to skip to
the Next Question.
8. Finally, we need to create a skip from Q1 to the final question if respondents don't ever purchase
fuel. While editing Q1, click the Sk ip tab and specify to skip from Q1 to Q5 if the following is true:
Q1_9=1. (The ninth item in Q1 is "checked".)
1.3.8 SSI Script

SSI Script is a scripting language that you can use within SSI Web. You can use SSI Script to write
small sections of customized code that will enhance your web survey.
Where can SSI Script be Used?

Displaying Responses to Previous Questions
Functions
Unverified Perl
Where can SSI Script be Used?
SSI Script can be used in the following areas:
Skip Logic - In Skip Logic skips are executed if the SSI Script that you provide evaluates to true.
If, for example, the expression:
((Q1 + 3) > Q2) And (Q5 < 8)
evaluates to true then the skip is taken. Note that no beginning and ending [% %] SSI Script tags
should be used when specifying SSI Script within Skip Logic.
List Building - Constructed lists are created by using SSI Script List Functions. For example:
AIC(Q1)
INSERT(5, ColorList)
Note that no beginning and ending [% %] SSI Script tags should be used when specifying SSI
Script within List Building.
Inline – Inline scripting involves inserting sections of SSI Script into your text or HTML. To enter a
section of SSI Script into your text it must be wrapped by [% and %]. For example the following
text could be entered into the header of a question:
Why do you spend [% (PhoneHours / TotalHours) * 100 %] percent of

your time at work on the phone?
SSI Script is processed/executed on the server before it is sent down to the respondent's browser.
The script between [% and %] is executed and the result is inserted into its place. If the
respondent had answered 18 for the question PhoneHours and 40 for the question TotalHours, then
the above text would be displayed to the respondent as:
Why do you spend 45 percent of your time at work on the phone?
Question Settings – You can enter SSI Script in some question settings . For example, the Total
setting in a Constant Sum question accepts the answer to a previous question. So, if [% Q1 %]
were entered into the Totals field, then the respondent's answer to Q1 would be used as the total
required for the Constant Sum question. Note that beginning and ending [% %] SSI Script tags are
needed when specifying SSI Script within question setting fields.
Quota Control – You can enter SSI Script in the qualification logic that indicates whether a
respondent qualifies to count toward a Cell Limit. Note that beginning and ending [% %] SSI Script
164 SSI Web v8
tags should not be used when specifying SSI Script within quota control qualification logic.
We admit that it might be confusing to remember when [% %] tags are needed or not when adding SSI
Scripting to your questionnaire--or whether SSI Scripting is permitted at all in specific areas.
Fortunately, SSI Web will give you an error message if your syntax is incorrect. When in doubt, just try
using SSI Script and then click the "Check" icon to have SSI Web check your syntax. For
example, it may tell you that the "[" character is not valid (which indicates that the tags are not needed).
If you see the "Pencil" icon next to a field in SSI Web, that is an indicator that SSI Scripting is
permitted for that field.
Comments
You might find it useful to add comments to your SSI Script. To add a comment, use a single quote
character prior to the comment text. For example:
'Here I'm adding the items that were chosen

AIC(Q1)
or
[% 4 + Q1 'Here we are adding 4 to the answer to Q1 %]
Everything after the single quote on the same line (except for the closing "%]" ) is a comment and is
ignored when the SSI Script is executed.
1.3.8.1 Displaying Responses Given to Previous Questions

You can display responses given to previous questions by referring to the question names (in skip logic,
list building logic, or inline SSI Scripting). (Some researchers refer to this as piping.) If you are
displaying text to the screen (for example, returning the answer for a previous question within a Header1
field for a question), you must enclose the question name within [% ... %] tags, such as:
[% Q1 %] returns the response recorded in the database for Q1
If you are referring to the answers in other areas of SSI Web, skip logic, and quota control, you do not
include the [% ... %] tags.
If the response is not in the database, a blank is returned. In some cases, you might not want to return
the answer in the database, but the label that is associated with it. With single select type questions
such as select, radio, combo, or grid, you can use LABEL(Q1) to return the label. Example:
[% LABEL(Q1) %] returns the label associated with the response recorded in the database for
Q1
Example: Showing Response Given to a Previous Numeric Question
Let's assume that you wanted to display the numeric value that the respondent provided earlier in the
questionnaire in a question called "housesize." For example, in the Header1 text section of a select-
type question called "numminors", you might type:
You previously said that you have [% housesize %] people in your household. How
many of those individuals are minors (17 or younger)?
When the questionnaire runs over the server, the numeric value that was provided in the question named
"housesize" is displayed instead of "[% housesize %]".
Example: Showing the Label for a Single-Select Question
Let's assume you asked respondents to indicate their favorite color, using a select-type (radio) button
question called "FavoriteColor." For example, in the Header1 text question of a follow-up question, you
might type:
You previously said that your favorite color was [%Label(FavoriteColor)%] .
Example: Showing the Label for a Single-Select Question from a Grid
Let's assume you asked respondents to indicate their favorite color, using a select-type (radio) button
question within the third column in a grid with column-based questions called "Colors."
166 SSI Web v8
To show the respondent's favorite color on the screen in a later question, you might specify:
You previously said that your favorite color was [%Label(Colors_c3)%] .
Because Colors is a column-based grid question, the underscore notation referring to column "_c3"
returns the row label associated with the answer to column 3. If the grid question were row-based,
underscore notation returns the column label associated with the response to the referenced row.
1.3.8.2 Functions
SSI Web includes many functions that can be used in various areas of the software:
List Functions
System Functions
Quota Control Functions
Math Functions
Loop Functions
String Functions
Error Functions
CBC Functions
ACA Functions
MaxDiff Functions
CVA Functions
ACBC Functions
Functions can be used in skip logic and list-building syntax. They can also be used in the middle of
questionnaire text (such as in a Header1 section of a question), in error messages sent to respondents,
or in Unverified Perl.
168 SSI Web v8
1.3.8.2.1 List Functions
SSI Script: Description:

ADD (ListName) Adds members from a specified list to the
ADD (ListName, Member#) end of the constructed list.
ADD (ListName, StartMember#, EndMember#) More info.
AddSorted (QuestionName) Adds members rated/ranked within a multi-
AddSorted (QuestionName, intDirection) part radio/rank/numeric question to a
constructed list, in ascending or descending
order. Specify a 0 for intDirection to indicate
descending order; specify 1 for ascending
order.
AIC (QuestionName) Add If Chosen: Adds members chosen in a
AIC (QuestionName, Member#) select-type question onto a new constructed
AIC (QuestionName, StartMember#, list. More info.
EndMember#)
AIE (QuestionName, Value) Add If Equal: Adds members in a grid
numeric, grid constant sum, grid ranking,
grid with all radio buttons, constant sum, or
ranking question that were given answers
equal to a specific value.
More info. (Also see Loop functions).
AIG (QuestionName, Value) Add If Greater: Adds members in a grid
greater than a specific value.
AIL (QuestionName, Value) Add If Less: Adds members in a grid
less than a specific value.
ANC (QuestionName) Add if Not Chosen: Adds members not
ANC (QuestionName, Member#) chosen in a select-type question onto a new
ANC (QuestionName, StartMember#, constructed list.
EndMember#) More info.
ANE (QuestionName, Value) Add if Not Equal: Adds members in a grid
ranking question that were given answers not
DisplayListLabels (ListName, Returns a string with every list item
"SeparatorText", separated by "SeparatorText" with
"FinalSeparatorText") "FinalSeparatorText" prior to the last list
item. For example: DisplayListLabels
(fruitlist, "," , "and") returns: grape, peach,

and cherry.
FormatListLabels (ListName, Returns a string with all list items joined
"BeforeText", "AfterText") together with "BeforeText" before each item
and "AfterText" after each item. For
example "<li>" and "</li>" as BeforeText and
AfterText returns a bulleted list.
INSERT (TargetPosition#, ListName) Inserts members from a specified list to an
INSERT (TargetPosition#, ListName, exact position within a constructed list.
Member#) More info.
INSERT (TargetPosition#, ListName,
StartMember#, EndMember#)
ListHasParentMember (ListName, Is true if the specified parent member exists
ParentMember#) in the specified list.
More info
ListLabel (ListName, Member#) Returns the label of a specified member
number in a list.
ListLabelsArray (ListName) Returns a comma-separated list of member
labels enclosed in quotes.
ListLength (ListName) Returns the number of members in the list.
ListMax (Value) Controls the maximum length of a list.
More info.
ListMin (Value) Controls the minimum length of a list.
More info.
ListValue (ListName, Member#) Returns the list member position from the
original parent list for the specified member#
on the specified constructed list.
ListValuesArray (ListName) Returns an array of list positions from the
original parent list for the specified
constructed list (example: [1, 4, 6]).
Loop Functions for Lists Click this link for available functions.
Mirror (ListName) Adds list members from the parent list that
Mirror (ListName, Offset) correspond to the list values in ListName.
More info.
Randomize () Randomizes the members of a constructed
Randomize (SeedValue) list.
Randomize (StartMember#, EndMember#) More info.
Randomize (SeedValue, StartMember#,
EndMember#)
Remove (ListName) Removes the members in the specified list
Remove (ListName, Member#) from the constructed list.
Remove (ListName, StartMember#, More info.
EndMember#)
Reverse () Reverses the members in a list. For

example, list members A, B, C, D when
170 SSI Web v8
reversed become D, C, B, A.
RIC (QuestionName) Remove If Chosen. Removes items from the
RIC (QuestionName, Member#) list that were chosen in QuestionName.
RIC (QuestionName, StartMember#,
EndMember#)
SetListLength () Controls the length of a list.
More info.
SortByLabel () Sorts a list based on list labels. Numbers
are sorted below letters.
SortByValue () Sorts the members in a constructed list with
respect to their natural order (lowest to
highest value) from their parent predefined
list. For example, list members with values
5, 3, 2, 7 would be sorted as 2, 3, 5, 7.
1.3.8.2.2 System Functions

SSI Web includes a number of functions that can be used to display information to the screen, within
skip logic, within constructed list logic, and in other fields within the SSI Web menu system.
The "SSI Script" column below gives the syntax for System Functions.

Answered(QuestionName) Is true if the question was answered by the
respondent, false otherwise.
More info
BlockPosition() Returns the current block number when used
within a Randomized Block sequence.
BooleanToNumber(Boolean) Converts a variable of type "Boolean" to type
"Number" for use in SSI Scripting.
Browser() Returns information on respondent's browser.
CheckSelect(QuestionName, Value) For use within Free Format questions.
Returns the HTML for a checkbox, including
the specific HTML compatible with Graphical
Select buttons. More info.
Debug() Returns debug information that assists you
when testing the questionnaire.
DisplayTotal (FirstQuestionName, Returns a grayed out box displaying the total
LastQuestionName, Width) of all questions beginning with
FirstQuestionName and ending with
LastQuestionName.
More info.
EncodeForURL(QuestionName) Converts characters that are not valid for use
in URLs to URL-supported strings.
GetValue(QuestionName) Returns value from data record for the given
variable name More info.
GraphicsPath() Returns the system graphics path.
IPAddress() Returns respondent's IP Address.
JavaScript() Is true if they have JavaScript, false
otherwise.
More info
Label(QuestionName) Returns the label that is associated with the
answer to a single select response type
question.
LinkButton("URL") Returns an HTML "submit" button (this helps
in linking surveys).
NumberToBoolean(Number) Converts a variable of type "Number" to type
"Boolean" for use in SSI Scripting.
NumberToString(Number) Converts a variable of type "Number" to type
"String" for use in SSI Scripting.
172 SSI Web v8
NumChecked(QuestionName) Returns number of response options

checked. QuestionName can be a select
checkbox (i.e. Q1), a row or column of
checkboxes (i.e. Q1_r1 or Q1_c1), or a Free
Format question (i.e. Q1_checked).
OperatingSystem() Returns information on Respondent's
operating system.
PageNumber() Returns current page number.
PageTime(StartPage) Returns the time (in seconds) between
PageTime(StartPage, EndPage) StartPage and EndPage (integer values).
More info
PopUp("Link Text", "File.htm", Width, Returns the HTML for a JavaScript pop-up.
Height) File.htm is assumed to be in the graphics
folder. Or, File.htm can be elsewhere if
referenced with full http:// path. More info
ProgressBar() Returns the progress bar to the screen.
Typically placed in the Global Header or
Global Footer.
ProgressBarOff() Turns off the progress bar on the page where
it is placed.
ProgressBarSet(Value) Sets the progress bar to the Value.
QuestionName() Returns the name of the current question.
May be used in Header 1, etc. sections, and
also in skip patterns.
RadioSelect(QuestionName, Value) For use within Free Format questions.
Returns the HTML for a radio button,
including the specific HTML compatible with
Graphical Select buttons. More info.
RandNum(Seed) Returns a random number. If one parameter
RandNum(Seed, Min#, Max#) is given, returns a uniform random decimal
value between 0 and 1. If three parameters
are given, returns a random integer value
from Min# to Max#. Seed must be an
integer.
More info.
RemovePrevious() Removes the Previous button from the page
where it is placed.
RespNum() Returns the internal respondent number, a
unique number that identifies the respondent.
SetValue(QuestionName, Value) Stores to the data record the value for the
given QuestionName. Warning: will overwrite
existing data in the target QuestionName.
More info.
Shown(QuestionName) Is true if the question has been seen by the
respondent, or false if the question was not
seen by the respondent.
More info
StringToNumber(String) Converts a variable of type "String" to type

"Number" for use in SSI Scripting.
Studyname() Returns the name of a study.
SysRand() Returns a random number seeded by the
SysRand(Min#, Max#) system clock (different seed every time
instruction is encountered).
SystemTime() Returns a number representing the current
time (number of non-leap seconds since
January 1, 1970).
ToolTip("LinkText", "HelpText", Used for creating a "tooltip" during
Width, Height) mouseover. More Info.
TotalPages() Returns total number of pages in study.
UserAgent() Returns respondent's User Agent.
For additional flexibility in SSI Scripting, see the Unverified Perl documentation.
Notes on use of GetValue (QuestionName) function:
The GetValue (QuestionName) function returns the value of the question as recorded in the database.
It can be used with various questions:
Numeric, Open-End or Select (radio) question types:

[%GetValue(G3)%] where G3 is the name of a question
Select (check box) question types:

[%GetValue(G3_1)%] where G3 is the name of a question, the number after the underscore
indicates the response item#, and the value returned is "1" (checked) or "0" (not checked)
Grid question type

[%GetValue(G3_r2_c3)%] where G3 is the name of a grid question and the response is in a
particular row (_r#) and column (_c#)
Note: [%QuestionName%] is the same as [%GetValue(QuestionName)%]. So [%G3%] will return the

value of question G3.
See also SetValue().

174 SSI Web v8
1.3.8.2.3 Quota Control Functions

SSI Web includes a number of functions that can be used when employing Quota Control. The standard
quota control options available in SSI Web's interface give a great deal of control. However, advanced
users may wish to use additional functions having to do with quota control.
The "SSI Script" column below gives the syntax for Quota Control Functions.

GetValue(QuestionName) Returns the quota cell value to which the
respondent has been assigned for the quota
question referred to by QuestionName;
returns 0 if unassigned; returns -1 if
respondents did not qualify due to logic, and
-2 if respondents qualified but the quota was
already filled.
QuotaCellName(QuestionName) Returns the quota cell name to which the
respondent has been assigned; returns
empty string (nothing) if unassigned
IsQuotaCellOpen Returns true (1) if the cell is open, false (0) if
(QuestionName, QUOTA_CELL_NAME) the cell is closed
IsQuotaCellOpen
(QuestionName, QUOTA_CELL_VALUE)
QuotaCellRemaining Returns the number of completes left to fill
(QuestionName, QUOTA_CELL_NAME) the quota cell. Returns 0 if quota cell is full.
QuotaCellRemaining
QuotaCellLimit Returns the quota cell limit
(QuestionName, QUOTA_CELL_NAME)
QuotaCellLimit
QuotaCellCompletes Returns number of completes in the quota
(QuestionName, QUOTA_CELL_NAME) cell
QuotaCellCompletes
AreAllQuotasClosed() Returns true if all cells have been closed for
all quotas. This might be useful if you don't
want to show a "survey closed" message but
would like to skip the respondent to a
different survey.
AreAllQuotaCellsClosed Returns true if all cells have been closed for
(QuestionName) the specified quota.
AreAnyQuotasClosed() Returns true if all cells have been closed in

any quota.
1.3.8.2.4 Math Functions

SSI Web includes a number of math functions that can be used to display information to the screen,
within skip logic, within constructed list logic, for use with Hidden Variables, and in other fields within the
SSI Web menu system.
The "SSI Script" column below gives the syntax for Math Functions.

Abs (*NUMBER) Returns the absolute value.
Ceiling (NUMBER) Rounds the NUMBER up to the closest integer.
Exp (NUMBER) Returns e (2.71828183) raised to the power of
NUMBER
Floor (NUMBER) Rounds the NUMBER down to the closest integer.
FORMATNUMBER (NUMBER, Formats a number with commas and decimal
"ThousandsSeparator", places. More Information.
"DecimalSeparator",
NumDecimalPlaces)
Log10 (NUMBER) Returns the log (base 10) of NUMBER.
Ln (NUMBER) Returns the natural log of NUMBER.
POWER (NUMBER1, NUMBER2) Returns NUMBER1 raised to the NUMBER2 power.
ROUND (NUMBER) Rounds NUMBER to the #DECIMALS places. If no
ROUND (NUMBER, #DECIMALS) decimal places are specified, rounds to nearest
integer.
ROUNDTONUMBER (NUMBER, ROUNDTO) Rounds NUMBER to the nearest ROUNDTO value,
ROUNDTONUMBER (NUMBER, ROUNDTO, such as to the nearest 10, 25, 50, 100, 250, 500,
#DECIMALS) 1000, 5000, 10000, etc.
Sqrt (NUMBER) Returns the square root of NUMBER.

*For all math functions, NUMBER can be any QuestionName that contains a numeric value or any
numeric expression (including functions and mathematical operators) that returns a numeric value.
176 SSI Web v8
1.3.8.2.5 Loop Functions

SSI Web includes a number of loop functions for use within Looping (asking a question or series of
questions multiple times for each element of a list).
The "SSI Script" column below gives the syntax for Loop Functions.

AIELoopList (Loop_Name, Similar to the AIE list function. Adds the list item
Question_Name, Value) from the loop list to the constructed list if the
answer to the question is equal to Value. (See
additional information.)
AIGLoopList (Loop_Name, Similar to the AIG list function. Adds the list item
answer to the question is greater than the Value.
(See additional information.)
AILLoopList (Loop_Name, Similar to the AIL list function. Adds the list item
answer to the question is less than the Value.
ANELoopList (Loop_Name, Similar to the ANE list function. Adds the list item
answer to the question is not equal to the Value.
LOOPITERATION ( ) Returns the current iteration count for the list item
LOOPITERATION (Loop_Name) for the current loop. Loop Name is used in case
more than one loop applies (e.g. nested loops).
LOOPLABEL ( ) Returns the label for the list item for the current
LOOPLABEL (Loop_Name) loop. Loop Name is used in case more than one
loop applies (e.g. nested loops).
LOOPVALUE ( ) Returns the parent list item number for the list item
LOOPVALUE (Loop_Name) for the current loop. Loop Name is used in case
more than one loop applies (e.g. nested loops).
1.3.8.2.6 String Functions

SSI Web includes a number of string functions for operations involving text variables (such as openends,
etc.).
The "SSI Script" column below gives the syntax for String Functions.

IsNumber (QuestionName) Is true if the entire string is all digits, false
otherwise.
More info
Length (QuestionName) Returns the number of characters contained in
the response.
Mid (QuestionName, BeginPosition, Extracts a substring of a string given by
Length) QuestionName. The substring is extracted
starting at the character number specified in
BeginPosition (1-based counting) with length
specified in Length.
TextExists (QuestionName, Searches for "TextToLookFor" in the answer
"TextToLookFor") of QuestionName. If found, a 1 is returned; if
not found, a 0 is returned.
ToLower (QuestionName) Returns a lowercased version of the response.
ToUpper (QuestionName) Returns an uppercased version of the

response.
To concatenate strings, use the & operator. For example:
[% "Hello " & Name %]
If Name is an open-end question and the respondent answered "Bob" for Name, then the SSI Script
would return:
Hello Bob
178 SSI Web v8
1.3.8.2.7 Error Functions

Error Functions are used in displaying a message to the respondent that a question has not been
completed properly and that they must remedy the problem before continuing. The Error Functions only
work when specified within the Compose | Survey Settings | Error Messages tab.

ErrCursum () Returns the current sum of the constant sum question.
ErrField () Returns the question number on the page.
ErrMax () Returns the maximum value allowed for the question.
ErrMin () Returns the minimum value allowed for the question.
ErrQName () Returns the question name.
ErrText () Returns the question-specific error message that is optionally
specified at the question level, under the Advanced... button.
ErrTotal () Returns the total amount that the responses to the constant sum
question should add to.
1.3.8.2.8 CBC Functions

CBC Functions are only applicable in CBC (Choice-Based Conjoint) questionnaires. The CBCDesign
function is useful when advanced researchers want to customize the format for their CBC questions.

CBCAttributeLabel Returns the attribute label for the given
(ExerciseName_CBCQuestionName, attribute row position. Useful if the
AttributePosition#) attribute position is randomized in the
design.
CBCAttributeValue Returns the attribute value for the given

(ExerciseName_CBCQuestionName, attribute concept position and level
ConceptPosition#, LevelTextPosition#) position. Useful if the attribute position is
randomized in the design.
CBCDesignLevelText Returns the level text for the specified

(ExerciseName_CBCQuestionName, CBC question in the specified concept
ConceptPosition#, LevelTextPosition#) position and level position within the
task. This function is used when
researchers want to customize the
format of their CBC questions.
CBCDesignLevelValue Returns the level value for the specified
(ExerciseName_CBCQuestionName, CBC question in the specified concept
ConceptPosition#, LevelTextPosition#) position and level position within the
task. Useful if the attribute position is
randomized in the design.
CBCDesignConceptValue Returns the actual concept number from
(ExerciseName_CBCQuestionName, the design that is being displayed to the
ConceptPosition#) screen in ConceptPosition#. Position
numbers will only differ if concept sorting
is in place. This function is used when
format of their CBC questions.
CBCNone () Returns the "None" text for the CBC
question (traditional "None" option only)
and is only used in Shelf Facing Display
CBC questions.
CBCVersion () Returns the design version number and
question name. This function is useful
when creating paper-and-pencil
interviews. (Example: D-1, T-
CBCRAN1). Only available within a CBC
task.
CBCCurrentTask() Returns the index number for the ith
CBC task in the exercise. For the first
task it will return a 1, the second one a
2, etc. Must be used inside of the CBC
question.
CBCTotalTasks() Returns the total number of CBC tasks
180 SSI Web v8
in the current exercise. Must be used

inside of a CBC question.
1.3.8.2.9 ACA Functions

ACA Functions are only applicable in ACA (Adaptive Conjoint Analysis) questionnaires.

ACAAttribute () Returns the text for the attribute label within
the ACARating or ACAImportance questions.
ACABest () Within the ACAImportance question, returns
the level text for the "best" level rated from the
previous ACARating question.
ACABestLevelLabel Returns label for best attribute level of Attribute
(ExerciseName, Attribute#, Rank#) #. Rank# optional. Rank# of 2 returns
second-best attribute level.
ACABestLevelValue Returns level number for best attribute level of
(ExerciseName, Attribute#, Rank#) Attribute #. Rank# optional. Rank# of 2
returns second-best attribute level number.
ACACalMin () Used in ACACalibration to display minimum
ACACalibration value used.
ACACalMax () Used in ACACalibration to display maximum
ACACalibration value used.
ACAImportance Returns the importance score for the attribute,
(ExerciseName, Attribute#) as specified by Attribute#. May be used after
the ACAImportance question section. (See
also ACAStrictImportance.)
ACALeastImpAttLabel Returns label for the least important attribute.
(ExerciseName, Rank#) Rank# optional. Rank# of 2 returns second-
least important attribute.
ACALeastImpAttValue Returns attribute number of the least important
(ExerciseName, Rank#) attribute. Rank# optional. Rank# of 2 returns
second-least important attribute number.
ACAMostImpAttLabel Returns label for the most important attribute.
(ExerciseName, Rank#) Rank# optional. Rank# of 2 returns second-
most important attribute.
ACAMostImpAttValue Returns attribute number of the most important
(ExerciseName, Rank#) attribute. Rank# optional. Rank# of 2 returns
second-most important attribute number.
ACAPriorsBestLevelLabel Returns the best level text for attribute#, based
(ExerciseName, Attribute#) only on Priors information.
ACAPriorsBestLevelValue Returns the best level number for attribute#,
(ExerciseName, Attribute#) based only on Priors information.
ACAPriorsWorstLevelLabel Returns the worst level text for attribute#,
ACAPriorsWorstLevelValue Returns the worst level number for attribute#,
ACAStrictImportance Returns the strict importance score for the
(ExerciseName, Attribute#) attribute, as specified by Attribute#. May be
182 SSI Web v8
used after the ACAImportance question

section.
ACAUtility Returns the part-worth utility for the attribute
(ExerciseName, Attribute#, and level, as specified by Attribute# and
Level#) Level#. May be used after the ACAImportance
question section.
More info.
ACAWorst () Within the ACAImportance question, returns
the level text for the "worst" level rated from the
previous ACARating question.
ACAWorstLevelLabel Returns label for worst attribute level of

returns second-worst attribute level.
ACAWorstLevelValue Returns level number for worst attribute level of
returns second-worst attribute level number.
1.3.8.2.10 CVA Functions

CVA Functions are only applicable in CVA (Conjoint Value Analysis) questionnaires.

CVAVersion () Returns the design version number and
interviews involving multiple versions.
(Example: D-1, T-CVA_1). Only
available within a CVA task.
184 SSI Web v8
1.3.8.2.11 MaxDiff Functions

MaxDiff Functions are only applicable in studies containing MaxDiff exercises. The functions below are
useful when advanced researchers want to customize the format for their MaxDiff questions.

MaxDiffDesignLabel Returns the item text for the specified
(ExerciseName_MaxDiffQuestionName, MaxDiff question for the specified
Position#) Position#. This function is used when
format of their MaxDiff questions.
MaxDiffDesignValue Returns the item value for the specified
(ExerciseName_MaxDiffQuestionName, MaxDiff question for the specified
Position#) Position#. This function is used when
format of their MaxDiff questions.
MaxDiffRankAttLabel(ExerciseName, Rank#) Returns the list item label for the item
with the rank specified. So a rank of 1
would return the list item label with the
highest maxdiff logit score. More info.
MaxDiffRankAttValue(ExerciseName, Rank#) Returns the list item value for the item
with the rank specified. So a rank of 1
would return the list item value with the
highest maxdiff logit score. More info.
MaxDiffScore(ExerciseName, ItemIndex) Returns the maxdiff logit score for the
ItemIndex of the list associated with
ExerciseName. More info.
MaxDiffCurrentSet() Returns the index number for the ith
MaxDiff set (question) in the exercise.
For the first set it will return a 1, the
second one a 2, etc. Must be used
inside of the MaxDiff question.
MaxDiffTotalSets() Returns the total number of MaxDiff sets

(questions) in the current exercise.
Must be used inside of a MaxDiff
question.
MaxDiffVersion () Returns the design version number and

interviews involving multiple versions.
(Example: D-1, T-MXDIFF_1). Only
available within a MaxDiff task.
1.3.8.2.12 ACBC Functions

ACBC Functions are only applicable in studies containing Adaptive CBC exercises.

ACBCBYOLabel(ExerciseName, Attribute#) Returns the level label selected in the BYO
question for the indicated attribute#.
ACBCBYOValue(ExerciseName, Attribute#) Returns the level value selected in the BYO
question for the indicated attribute#.
ACBCCalibrationText() Displays ACBC Calibration question text
from the Concept Text settings area. Must
be used within an ACBC Calibration
question.
ACBCCurrentCalibration() Returns the index of the current Calibration
Concept. Needs to be used inside of an
ACBC Calibration.
ACBCCurrentChoiceTask() Returns the index of the current Choice Task.
Needs to be used inside of an ACBC
ChoiceTask.
ACBCCurrentScreener() Returns the index of the current Screening
Task. Needs to be used inside of an ACBC
Screener.
ACBCIsMustHave Returns True if the attribute and level
(ExerciseName, Attribute#, Level#) specified has been declared a "Must Have."
ACBCIsUnacceptable Returns True if the attribute and level
(ExerciseName, Attribute#, Level#) specified has been declared "Unacceptable."
If referencing a Summed Price attribute,
specify a price value rather than the Level#.
ACBCMustHaveRules() Displays all previous "Must Have" rules.
Must be used within an ACBC Must Have
question.
ACBCMustHaveText() Displays "Must Have" question text from "No
Rules Exist" and "Rules Exist" labels. Must
be used within an ACBC Must Have
question.
ACBCNumCalibrations() Returns the total number of Calibration
Concepts. Can use this inside of a
ACBCCalibration. If used outside of an
ACBCCalibration needs to have the
ACBCExerciseName passed as a
parameter.
ACBCNumChoiceTasks() Returns the total number of Choice Tasks.
Can use this inside of a ACBCChoiceTask. If
used outside of an ACBCChoiceTask needs
to have the ACBCExerciseName passed as a
parameter.
ACBCNumScreenedInConcepts(ExerciseName) Returns the total number of Product
186 SSI Web v8
Concepts marked "a possibility" by the

respondent.
NOTE: Does not include the BYO if it is
included in the tournament.
ACBCNumScreeners() Returns the total number of Screening Tasks.
Can use this inside of a ACBCScreener. If
used outside of an ACBCScreener needs to
have the ACBCExerciseName passed as a
parameter.
ACBCUnacceptableRules() Displays all previous "Unacceptable" rules.
Must be used within an ACBC Unacceptable
question.
ACBCUnacceptableText() Displays "Unacceptable" question text from
"No Rules Exist" and "Rules Exist" labels.
Must be used within an ACBC Unacceptable
question.
ACBCWinnerLabel(ExerciseName, Returns the level label for the choice
Attribute#) tournament winner for the indicated attribute#
(if no tournament exists, returns nothing).
Hint: use ACBCNumScreenedInConcepts to
determine if you should skip a question in
which you might show the tournament
winner.
ACBCWinnerValue(ExerciseName, Returns the level value for the choice
Attribute#) tournament winner for the indicated
attribute# (if no tournament exists, returns
nothing). Hint: use
ACBCNumScreenedInConcepts to determine
if you should skip a question in which you
might show the tournament winner.
BYOCondText(ExerciseName, CondText#) Within a BYO question, returns the
conditional text/graphic element for the
BYOCondText(ExerciseName, CondText#, CondText# relationship from the conditional
XPos,YPos) relationship table. If XPos and YPos (values
expressed in Pixels) are given, then the
conditional element is shown at that fixed
position on the screen (hovering graphic).
1.3.8.3 More Detail on Boolean (True/False) Functions

Some functions in SSI Web are Boolean (true/false) functions that do not return a specific "value". They
simply evaluate to true or false. You can use the syntax "not" prior to the Boolean function (as shown in
the examples below).
ANSWERED
ANSWERED is true if the question was answered by the respondent. Note that ANSWERED evaluates
to false if the respondent did not ever see the question (use SHOWN if you are interested in whether the
question was seen).
The format is:
Answered(QuestionName)
For example, you may specify a skip in SSI Web that executes if the following is true:
Answered(Q1)
In other words, skip if Q1 was answered by the respondent. You could also execute a skip if Not
Answered(Q1), which would execute if Q1 was not answered.
For questions with multiple responses such as Free Format or Grid the full variable name must be
specified. For example Answered(G1_r1) will test whether or not the first row of radio buttons was
answered in a Grid question. Answered(G1) will not work.
Note: Multi-Select and Constant Sum questions are exceptions to these rules. Multi-Select (Check-box)
questions, if ever displayed to the respondent are always considered "answered," with a value of 1 or 0.
Similar with Constant Sum. If the Constant Sum is answered all entries left blank are filled with zeros in
the data record and thus are considered answered.
ISNUMBER
ISNUMBER is true if the entire string is made up of numeric values (0 to 9), false otherwise. For
example, the string "01605" contains only numeric characters.
The format is:
IsNumber(QuestionName)
IsNumber(Q1)
In other words, skip if the respondent's answer to Q1 contains only numbers. You could also execute a
skip if not IsNumber(Q1), which would execute if Q1 contained characters other than numbers 0 to 9.
JAVASCRIPT
JAVASCRIPT is true if the respondent's browser has JavaScript enabled, false otherwise.
188 SSI Web v8
The format is:
JavaScript( )
JavaScript( )
In other words, skip if the respondent's system has JavaScript enabled. You could also execute a skip
if not JavaScript( ), which would execute if the respondent's browser did not have JavaScript enabled.
LISTHASPARENTMEMBER
LISTHASPARENTMEMBER is true if the specified parent member exists in the specified list, false
otherwise.
The format is:
ListHasParentMember(ListName, ParentMember#)
ListHasParentMember(BrandAwareList, 12)
In other words, skip if the constructed list named BrandAwareList contains item #12 (from the original
predefined parent list for BrandAwareList). You could also execute a skip if not
ListHasParentMember(BrandAwareList, 12), which would execute if the BrandAwareList did not contain
item #12.
SHOWN
SHOWN is true if the question was seen by the respondent, or false if the question was not seen by the
respondent.
The format is:
Shown(QuestionName)
Shown(Q1)
In other words, skip if Q1 was seen by the respondent (did not need to be answered). You could also
execute a skip if Not Shown(Q1), which would execute if Q1 was not seen by the respondent.
For questions with multiple responses such as Free Format or Grid the QuestionName parameter above
can be either the root name or the specific variable name. For example if you have a grid question
named G1 you can call shown with G1 or with a specific variable such as G1_r3. Shown(G1) will return
true if they ever saw any part of G1 (G1_r1, G1_r2, etc.). Shown(G1_r3) will return true only if they saw
row 3 of G1 (could have been removed by a constructed list).

190 SSI Web v8
1.3.8.4 Additional Details on Certain Functions

POPUP
Returns the HTML for a JavaScript pop-up. The HTML file that contains the text to be shown in the popup
(File.htm) is assumed to be in the graphics folder. Or, the HTML file can be elsewhere if referenced with
the full http:// path.
The format is:
PopUp("Link Text", "File.htm", Width, Height)
Example 1:
Assume that you have saved an HTML file containing the help in the study graphics folder. The text
below is placed within the question text, such as within Header 1 or Header 2.
[% PopUp("Click Here for Survey Help", "SurveyHelp.htm", 500, 500) %]
When the survey is run, the above function displays a link that the respondent may click to launch a
popup window containing the information in SurveyHelp.htm.
The respondent can retire the popup window and return to the survey by clicking the "X" in the upper-right
corner of the popup window.
Example 2:
In this example, we reference an HTML page not in the graphics folder, but somewhere else on your web
site. To do this, we include the full URL path to the HTML file.
[% PopUp("Click Here for Survey Help", "http://www.mysurvey.com/mystudy/help/

SurveyHelp.htm", 500, 500) %]
Example 3:
In this example, we replace the "Click Here for Survey Help" text with a graphic.
[% PopUp("<img src=' " & GraphicsPath() & "help.jpg' alt=' ' border='0'>", "Help.htm", 500, 500)
%]
FormatNumber
This function formats numbers with thousands separators, decimal separators, and decimal places. The
format is:
FormatNumber (Number, ThousandsSeparator, DecimalSeparator, #DecimalPlaces)
The ThousandsSeparator and DecimalSeparator are enclosed in double-quotes. For example, to make
the ThousandsSeparator a comma, you should specify "," as the argument.
Example 1: In the US, the value 7123456 would be formatted as 7,123,456.00. The following script
formats this value to US standards:
FormatNumber(7123456, ",", ".", 2)
Example 2: In Germany, the value 7123456 would be formatted as 7.123.456,00. The following script
formats this value to German standards:
FormatNumber(7123456, ".", ",", 2)
Example 3: In most cases, you'll be formatting a value contained in a variable (such as a question
asked earlier in the survey):
FormatNumber(Q28, ",", ".", 0)
Technical notes: DecimalPlaces can be replaced by a variable that returns a value. Thousands
separators and decimal separators can be replaced by a variable that returns text.
Strict Importance
If you use ACA's "Strict" importance function, we do not assume that importance is defined simply by
computing the range from the most and least preferred levels within an attribute. We refer to the spread
in utility between this respondent's prior best and worst levels (as specified in the attribute a priori order,
or by the respondent in the ACARAT questions). If the best minus worst utility is negative, we set the
range equal to zero. Strict importance keeps virtually unimportant attributes that potentially have
reversals (out-of-order utilities) from getting inflated importance weights. It reduces the effect of random
error on importance calculations.
Consider the following part-worth utilities:
$10 1.45
$15 1.50
$20 0.75
$25 0.50
Under standard importance calculations, we take the full spread in utilities for this attribute (1.50 - 0.50 =
1). With Strict importance, we take the spread in importance between the prior best and worst levels
(1.45 - 0.50 = 0.95).
As another example, consider the following part-worth utilities:
Has widget "A" 0.20

No widget "A" 0.30
If having widget "A" is expected to be preferred to not having widget "A," based on prior rank-order (or
from the respondent's ACARAT question), these utilities suggest a "reversal." The standard importance
calculation takes the full range of 0.10 into the computation. Strict importance recognizes this as a
reversal and sets the importance equal to 0 for this attribute. We assume that the reversal is due to this
attribute having virtually no importance, with random error overcoming any correct signal from the
respondent regarding his/her true preference for levels within this attribute.
Note: this function only applies within the ACA questionnaire. The market simulator (SMRT software)
uses the standard definition of attribute importance.
192 SSI Web v8
PageTime
PageTime(StartPage, EndPage) returns a value (in seconds) between StartPage and EndPage.
EndPage is optional. If no EndPage is supplied, PageTime returns the total time spent on StartPage.
You can specify integers as arguments (the page number for the current question is shown at the bottom
of the Write Questionnaire dialog when you highlight any question on the list). For example, you may
also use: [%PageTime(PageNumber()-3, PageNumber())%] to return the time spent on the previous
three pages, without worrying about hardcoding integer values. You might apply that script within a skip
pattern to display a warning message if a respondent answered the previous pages too quickly.
If you check the Save Respondent Time per Page setting in the Survey Settings + Advanced tab
dialog, the time for each page (in seconds) is stored in the database with variable naming convention
sys_pagetime_#, where # is the page number. When you export the data, you can export these
variables and they appear in the export list below any constructed list items.
If the respondent backs up and re-answers a page (using SSI Web's Previous button), new time spent
answering that page will be added to the previous time recorded. If respondents use the Back button on
the browser to re-answer a page, the total time spent on the re-answered page may not be accounted for
correctly.
RadioSelect
This function returns HTML for a radio button, for use within Free Format questions. When Graphical
Select buttons are enabled under the Survey Settings, General Format tab, the HTML required for
displaying graphical radio buttons is relatively complex. The RadioSelect function returns the HTML
necessary for displaying regular or graphical select questions.
The syntax for this instruction is:
RadioSelect(QuestionName, Value)
Where QuestionName is the full QuestionName + Variable extension for a Free Format variable; and
Value is the numeric (or text) value that is stored when the respondent clicks this particular radio button.
Example:
<table>
<tr class="clickable">
<td class="input_cell">
[%RadioSelect(freeformat_radio, 1)%]
</td>
<td>
ResponseOption_1
</td>
</tr>
</td>
<td>
ResponseOption_2
</td>
</tr>
</td>
<td>
ResponseOption_3
</td>
</tr>
</table>
In the example above, the Free Format variable is named freeformat_radio. The RadioSelect
instruction (highlighted in blue above) is called for each of the three response options (radio buttons) and
the appropriate value is saved into the database for each option. The HTML needed for showing
Graphical Radio buttons is supplied when the survey runs on the server.
Notice that we've declared classes "clickable" and "input_cell" within the <tr> and <td> tags. This
makes the graphical select feature and the highlight text feature (when respondents mouse-over the
response item text) work.
CheckSelect
This function returns HTML for a checkbox, for use within Free Format questions. When Graphical
Select buttons are enabled under the Survey Settings, General Format tab, the HTML required for
displaying graphical radio buttons and checkboxes is relatively complex. The CheckSelect function
makes the Free Format question appear and function like the other Select-type questions in your
questionnaire.
CheckSelect(QuestionName, Value)
Where QuestionName is the full QuestionName + Variable extension for a Free Format variable; and
Value is the numeric value associated with each item in the checkbox question. Example:
<table>
[%CheckSelect(freeformat_check, 1)%]
</td>
<td>
ResponseOption_1
</td>
</tr>
</td>
<td>
ResponseOption_2
194 SSI Web v8
</td>
</tr>
</td>
<td>
ResponseOption_3
</td>
</tr>
</table>
In the example above, the Free Format variable is named freeformat_check. The CheckSelect
instruction (highlighted in blue above) is called for each of the three response options (check boxes).
The CheckSelect instruction will return HTML that declares the variables freeformat_check_1,
freeformat_check_2, and freeformat_check_3, and sets all variables equal to 1 if selected, and 0
otherwise. The HTML needed for showing Graphical Checkboxes is supplied (if needed) when the survey
runs on the server.
Notice that we've declared classes "clickable" and "input_cell" within the <tr> and <td> tags. This
makes the graphical select feature and the highlight text feature (when respondents mouse-over the
response item text) work.
DisplayTotal
Returns a grayed out box displaying the total of all questions beginning with FirstQuestionName and
ending with LastQuestionName.
DisplayTotal (FirstQuestionName , LastQuestionName, Width)
The FirstQuestionName and LastQuestionName question names need to have the same base question
names with a starting number at the end of FirstQuestionName and a last question number at the end of
LastQuestionName. DisplayTotal will sum the values for all question names in between.
For example:
[%DisplayTotal (Num1, Num5, 5) %]
will display the total of Num1 + Num2 + Num3 + Num4 + Num5. Notice that all question names must
contain the same base name, in this case "Num".
DisplayTotal is useful when using Free Format questions to implement custom constant sum CBC
questions. May also be used for non-CBC questions.
ToolTip ("LinkText", "HelpText", Width, Height)
Creates a tool tip for "LinkText". When the respondent moves their mouse over the top of this text a box
will be displayed with "HelpText" in it.
ToolTip(LinkText, HelpText) or ToolTip(LinkText, HelpText, Width, Height) - Width and Height are optional
- Example: [%ToolTip("SSI Web", "Great product!", 100, 50)%]
The return type on the ToolTip function is a string (text).
If you want to use your own custom HTML (to include images etc.) to create a tool tip then you don't
even need to use the ToolTip function. Simply place HTML similar to the example below in your study
and it will create a tool tip. Make sure you include the "tool_tip_link" and "tool_tip_text" classes. The
HTML containing the "tool_tip_text" class must immediately follow the "tool_tip_link" HTML.
<span class="tool_tip_link">Car XYZ</span>

<div class="tool_tip_text">
<img src="[%GraphicsPath()%]car_pic.jpg" alt="" border="0">
<br>This is an "awesome" car!
</div>
Warning: Do not place your Tooltip code within a <p> </p> tag! The tooltip help text will not
display.
SetValue (QuestionName, Value)
SetValue() allows you to save values to existing variables in the database (it does not create new
variables on the fly within the database). For example:
[%SetValue(Q1, 5) %]
sets the value for Q1 to 5 in the database.
Currently we do not have a place in the SSI Web interface where you can specify variable names. You
could use an existing question in the question list (such as a Free Format hidden variable). But, we
suggest you use Pass-In Fields (located in the Start question) to set up variables that can be used by
SetValue(). For example you could create a Pass-In field named "age". You could then set the value for
"age" to 45 midway through the survey using the following command:
[%SetValue(age, 45) %]
To retrieve the value for age at a point later in the questionnaire you can use:
[%GetValue(age) %]
which returns the current value for age stored in the database.
LoopList Functions
(AIELoopList, ANELoopList, AIGLoopList, AILLoopList)
Loops ask a question or series of questions for each element of a given list (either predefined or
196 SSI Web v8
constructed). LoopList functions operate on this list.
Example:
Consider a numeric question (TIMESBOUGHT) involved in a loop named BRANDLOOP, where we ask
how many times each brand has been purchased in the last month.. The loop is based on list
BRANDLIST, containing the full list of brands in the study.
Here is the question text for the TIMESBOUGHT question (for example, in Header 1):
How many times have you purchased [%LoopLabel()%] in the last month?
For each brand, the brand name appears in place of [%LoopLabel()%]. The question is asked multiple
times, for each member in BRANDLIST.
Suppose we want to add any brand to a new list that has been bought at least twice (those with
TIMESBOUGHT>1). Within the List Manager, we can create a new constructed list, adding elements to
that list based on the TIMESBOUGHT question that is within the loop. The parent list for this
constructed list (BRANDSBOUGHTLIST) needs to be the same parent list that is specified for the Loop
List. The following :
AIGLoopList(BRANDLIST, TIMESBOUGHT, 1)
Suppose a particular respondent sees the following brands and answers as follows for each:
Brands TimesBought
Snickers 3
Mounds 0
Reese's 2
M&M's 1
The final constructed list (BRANDSBOUGHTLIST) for this respondent will contain elements "Snickers,
Reese's" (because the respondent has indicated that each of those is purchased at least twice).
1.3.8.5 Generating Random Numbers

Often, you may wish to generate a random number during a questionnaire. For example, you may wish
to randomly branch to one question or another. Or, you may wish to use a random number in a
mathematical function. SSI Web uses the RandNum and SysRand functions to generate random
numbers.
Randnum Function
The random number generator is seeded first by the internal respondent number (plus any seed you
specify). That keeps different respondents from receiving the same random value. If you provide a seed
value in the RandNum function, you can call the RandNum function again for that same respondent and
receive the same value each time.
The formats are:
RandNum(Seed)
This returns a random decimal value greater than or equal to 0 and less than 1. The random values
are distributed rectangularly, meaning that there is an equal probability of receiving a value between
0 and 0.1, 0.1 and 0.2, 0.3 and 0.4, etc. The Seed must be an integer zero or greater and is used
to seed the random number generator. The seed value used is actually the internal respondent
number plus the specified seed. Therefore, if you call the RandNum function again for this
respondent with the same given seed you will receive the same random value. But, different
random values are generated for different respondents.
RandNum(Seed, Min#, Max#)

This returns a random integer in the range of Min# to Max#. Using a specific seed integer is useful
if you want to be able to generate the same random value later in the questionnaire. The Seed must
be an integer zero or greater and is used to seed the random number generator. The seed value
used is actually the internal respondent number plus the specified seed. Therefore, if you call the
RandNum function again for this respondent with the same given seed you will receive the same
random value. But, different random values are generated for different respondents.
SysRand Function
The SysRand function is used for generating truly random values (not repeatable), where the random
number generator is always seeded by the system clock. Caution: when respondents back up in the
survey, a new random number will be chosen the next time the SysRand function is encountered.
The formats are:
SysRand( )
This returns a random decimal value greater than or equal to 0 and less than 1. The random values
are distributed rectangularly, meaning that there is an equal probability of receiving a value between
0 and 0.1, 0.1 and 0.2, 0.3 and 0.4, etc. The seed value used is the system clock at the moment
the instruction is encountered.
SysRand(Min#, Max#)
This returns a random integer in the range of Min# to Max#. The seed value used is the system
clock at the moment the instruction is encountered.
198 SSI Web v8
1.3.8.6 Unverified Perl

Unverified Perl can be used for additional flexibility in SSI Scripting.
SSI Script has been implemented using the Perl Programming Language. When [% Q1 + 3 %] is
specified in SSI Web, the server side Perl scripts actually receive that as [% GETVALUE("Q1") + 3;
%] which is valid Perl source code. The Perl scripts evaluate this code and return the result. If
additional power and flexibility are needed in your SSI Script, you can enter straight Perl code to
accomplish the task. This is definitely an advanced area. SSI Web does not check your Unverified Perl
for errors. You must thoroughly test your Perl logic.
Unverified Perl Rules

Unverified Perl Functions
Perl Programming Language Basics
Examples
Unverified Perl Rules
Unverified Perl can be entered by using the key words "Begin Unverified Perl" and "End
Unverified". For example:
[%
Begin Unverified Perl
GETVALUE("Q1") + 3;
End Unverified
%]
Your Unverified Perl code is not checked for errors.
Unverified Perl is executed on the server before the web page is sent down to the browser. So
in order to execute this:
[%
GETVALUE("Q1") + 3;
End Unverified
%](
Q1 must have been answered on a previous page. You cannot refer to the answer to a
question if the question is being asked on the same page.
Each statement must end with a semicolon. For example:
[%
my $intVar = GETVALUE("Q1") + 3;
$intVar = $intVar + (GETVALUE("Q3") / 3);
return $intVar;
End Unverified
%]
Notice the semicolons at the end of each line.
The last statement of the Unverified Perl is evaluated and the result returned. For example:
[%
End Unverified
%]
returns $intVar after all the math is completed. You can also return explicitly using the
"return" keyword. For example:
[%
return $intVar;
End Unverified
%]
To compare two values, use == for numbers and eq for text. (!= means not equal for numbers
and ne means not equal for text.) For example:
if(GETVALUE("Q1") == 5)
{
AIC("Q3");
}
if(GETVALUE("QName") eq "Sally")
{
ADD("ColorList", 4);
}
End Unverified
Functions must be in upper case. (There are a few exceptions to this rule. Please see
Unverified Perl Functions for details). For example, while creating a constructed list you can
specify:
if(GETVALUE("Q1") > 5)
{
AIC("Q3");
}
200 SSI Web v8
End Unverified
Notice the functions GETVALUE and AIC are upper case.
Question names and list names are case-sensitive and must be enclosed in double quotes
when used as function parameters. For example:
ADD("ColorsList");
AIC("q1");
End Unverified
The list ColorsList is enclosed in double quotes. "colorslist" is not valid, because the case
does not match the case of the name of the defined list. Question names are similar.
Avoid using single quotes in unverified Perl. If you need to use a single quote, you can do so
in a separate Perl script file that you include.
Unverified Perl Functions
List Functions:
SSI Script: "Unverified" Perl: Description:
ADD (ListName) ADD ("ListName") Adds members from a specified list to the
ADD (ListName, Member#) ADD ("ListName", end of the constructed list. More info.
ADD (ListName, Member#)
StartMember#, ADD ("ListName",
EndMember#) StartMember#,
EndMember#)
AddSorted (QuestionName) ADDSORTED ("QuestionName") Adds members rated/ranked w ithin a multi-
AddSorted ADDSORTED part radio/rank/numeric question to a
(QuestionName, ("QuestionName", constructed list, in ascending or
intDirection) intDirection) descending order. Specify a 0 for
intDirection to indicate descending order;
specify 1 for ascending order.
AIC (QuestionName) AIC ("QuestionName") Add If Chosen: Adds members chosen in a
AIC (QuestionName, AIC ("QuestionName", select-type question onto a new
Member#) Member#) constructed list. More info.
AIC (QuestionName, AIC ("QuestionName",
StartMember#, StartMember#,
EndMember#) EndMember#)
AIE (QuestionName, Value) AIE ("QuestionName", Add If Equal: Adds members in a grid
Value) numeric, grid constant sum, grid ranking,
grid w ith all radio buttons, constant sum, or
ranking question that w ere given answ ers
More info. (See also Loop functions.)
AIG (QuestionName, Value) AIG ("QuestionName", Add If Greater: Adds members in a grid
greater than a specific value. More info.
(See also Loop functions.)

AIL (QuestionName, Value) AIL ("QuestionName", Add If Less: Adds members in a grid
less than a specific value.
More info. (See also Loop functions.)
ANC (QuestionName) ANC ("QuestionName") Add if Not Chosen: Adds members not
ANC (QuestionName, chosen in a select-type question onto a
Member#) new constructed list. More info.
ANC (QuestionName,
StartMember#,
EndMember#)
ANE (QuestionName, Value) ANE ("QuestionName", Add if Not Equal: Adds members in a grid
not equal to a specific value. More info.
(See also Loop functions.)
DisplayListLabels DISPLAYLISTLABELS Returns a string w ith every list item
(ListName, ("ListName", separated by "SeparatorText" w ith
SeparatorText", "SeparatorText", "FinalSeparatorText" prior to the last list
"FinalSeparatorText") "FinalSeparatorText") item. For example: DisplayListLabels
(fruitlist, "," , "and") returns: grape, peach,
and cherry.
INSERT (TargetPosition#, INSERT (TargetPosition#, Inserts members from a specified list to an
ListName) "ListName") exact position w ithin a constructed list.
INSERT (TargetPosition#, INSERT (TargetPosition#, More info.
ListName, Member#) "ListName", Member#)
INSERT (TargetPosition#, INSERT (TargetPosition#,
ListName, StartMember#, "ListName",
EndMember#) StartMember#,
EndMember#)
FormatListLabels FORMATLISTLABELS Returns a string w ith all list items joined
(ListName, ("ListName", together w ith "BeforeText" before each
"BeforeText", AfterText") "BeforeText", AfterText") item and "AfterText" after each item. For
example "<li>" and "</li>" as BeforeText
and AfterText returns a bulleted list.
ListHasParentMember LISTHASPARENTMEMBER Is true if the specified parent member
(ListName, ParentMember#) ("ListName", exists in the specified list. More info
ParentMember#)
ListLabel (ListName, LISTLABEL ("ListName", Returns the label of a specified member

Member#) Member#) number in a list.
ListLabelsArray LISTLABELSARRAY Returns a comma-separated list of member
(ListName) ("ListName") labels enclosed in quotes.
ListLength (ListName) LISTLENGTH ("ListName") Returns the number of members in the list.
ListMax (Value) LISTMAX (Value) Controls the maximum length of a list.
More info.
ListMin (Value) LISTMIN (Value) Controls the minimum length of a list.
More info.
ListValue (ListName, LISTVALUE ("ListName", Returns the value of the specified member
202 SSI Web v8
Member#) Member#) number in a list.

ListValuesArray (ListName) LISTVALUESARRAY Returns a list of list values (example: [1, 4,
("ListName") 6]).
Mirror (ListName) MIRROR ("ListName") Adds list members from the parent list that
Mirror (ListName, Offset) MIRROR ("ListName", correspond to the list values in ListName.
Offset) More info.
Randomize () RANDOMIZE () Randomizes the members of a constructed
Randomize (SeedValue) RANDOMIZE (SeedValue) list.
Randomize (StartMember#, RANDOMIZE (StartMember#, More info.
Randomize (SeedValue, RANDOMIZE (SeedValue,
Remove (ListName) REMOVE ("ListName") Removes the members in the specified list
Remove (ListName, REMOVE ("ListName", from the constructed list.
Member#) Member#) More info.
Remove (ListName, REMOVE ("ListName",
Reverse () REVERSE () Reverses the members in a list. For

example, list members A, B, C, D w hen
reversed become D, C, B, A.
RIC (QuestionName) RIC ("QuestionName") Remove If Chosen. Removes all items
RIC (QuestionName, RIC ("QuestionName", from the list chosen in QuestionName.
Member#) Member#)
RIC (QuestionName, RIC ("QuestionName",
StartMember#, EndMember#) StartMember#, EndMember#)
SetListLength () SETLISTLENGTH () Controls the length of a list.
More info.
SortByLabel () SORTBYLABEL () Sorts a list based on list labels. Numbers
are sorted below letters.
SortByValue () SORTBYVALUE () Sorts the members in a constructed list
w ith respect to their natural order (low est
to highest value) from their parent
predefined list. For example, list members
w ith values 5, 3, 2, 7 w ould be sorted as
2, 3, 5, 7.
System Functions:

Answered(QuestionName) ANSWERED("QuestionName") Is true if the question w as answ ered by
the respondent, false otherw ise.
More info
BlockPosition() BLOCKPOSITION() Returns the current block number w hen
used w ithin a Randomized Block
sequence.
Browser() BROWSER() Returns information on respondent's
brow ser.
CheckSelect CHECKSELECT For use w ithin Free Format questions.
(QuestionName, Value) ("QuestionName", Value) Returns the HTML for a checkbox,
including the specific HTML compatible
w ith Graphical Select buttons. More info.

Debug() DEBUG() Returns debug information that assists you
w hen testing the questionnaire.
DisplayTotal DISPLAYTOTAL Returns a grayed out box displaying the
(FirstQuestionName, ("FirstQuestionName", total of all questions on the same page
LastQuestionName, Width) "LastQuestionName", beginning w ith FirstQuestionName and
Width) ending w ith LastQuestionName.
More info.
EncodeForURL(QuestionName) ENCODEFORURL("QuestionName Converts characters that are not valid for
") use in URLs to URL-supported strings.
GetValue(QuestionName) GETVALUE(QuestionName) Returns value from data record for the
given variable name. See also SetValue().
More info.
IPAddress() IPADDRESS() Returns respondent's IP Address.
GraphicsPath() GRAPHICSPATH() Returns the system graphics path.
JavaScript() JAVASCRIPT() Is true if they have JavaScript, false
otherw ise.
More info
Label(QuestionName) LABEL("QuestionName") Returns the label that is associated w ith
the answ er to a single select response
type question.
LinkButton("URL") LINKBUTTON("URL") Returns an HTML "submit" button (this
helps in linking surveys).
NumChecked(QuestionName) NUMCHECKED("QuestionName") Returns number of response options
checked. QuestionName can be a select
checkbox (i.e. Q1), a row or column of
checkboxes (i.e. Q1_r1 or Q1_c1), or a
Free Format question (i.e. Q1_checked).
OperatingSystem() OPERATINGSYSTEM() Returns information on respondent's
operating system.
PageNumber() PAGENUMBER() Returns current page number.
PageTime(StartPage) PAGETIME(StartPage) Returns the time (in seconds) betw een
PageTime(StartPage, PAGETIME(StartPage, StartPage and EndPage (integer values).
EndPage) EndPage)
PopUp("LinkText", POPUP("LinkText", Returns the HTML for a JavaScript pop-up.
"File.htm", Width, Height) "File.htm", File.htm is assumed to be in the graphics
Width, Height) folder. Or, File.htm can be elsew here if
referenced w ith full http:// path. More info
ProgressBar() PROGRESSBAR() Returns the progress bar to the screen.
Typically placed in the Global Header or
Global Footer.
ProgressBarOff() PROGRESSBAROFF() Turns off the progress bar on the page
w here it is placed.
ProgressBarSet(Value) PROGRESSBARSET(Value) Sets the Progress Bar to the Value.
RemovePrevious() REMOVEPREVIOUS() Removes the Previous button from the
page w here it is placed.
QuestionName() QUESTIONNAME() Returns the name of the current question.
May be used in Header 1, etc. sections,
and also in skip patterns.
RadioSelect RADIOSELECT For use w ithin Free Format questions.
204 SSI Web v8
(QuestionName, Value) ("QuestionName", Value) Returns the HTML for a radio button,
including the specific HTML compatible
w ith Graphical Select buttons. More info.
RandNum(Seed) RANDNUM(Seed) Returns a random number. If one
RandNum(Seed, Min#, Max#) RANDNUM(Seed, Min#, Max#) parameter is given, returns a uniform
random decimal value betw een 0 and 1. If
three parameters are given, returns a
random integer value from Min# to Max#.
Seed must be an integer. If Seed is 0, the
system clock is used to seed the random
number generator.
More info.
RespNum() RESPNUM() Returns the internal respondent number, a
unique number that identifies the
respondent.
SetValue(QuestionName, SETVALUE Stores to the data record the value for the
Value) ("QuestionName", Value) given QuestionName. Warning: w ill
overw rite existing data in the target
QuestionName. More info.
Shown(QuestionName) SHOWN("QuestionName") Is true if the question has been seen by
the respondent, or false if the question
w as not seen by the respondent
More info
Studyname() STUDYNAME() Returns the name of the study.
SysRand() SYSRAND() Returns a random number seeded by the
SysRand(Min#, Max#) SYSRAND(Min#, Max#) system clock (different seed every time
instruction is encountered).
SystemTime() time() Returns a number representing the current

time (number of non-leap seconds since
January 1, 1970).
ToolTip("LinkText", TOOLTIP("LinkText", Used for creating a tooltip help upon
"HelpText", Width, "HelpText", Width, mouseover. More info.
Height) Height)
TotalPages() TOTALPAGES() Returns the total number of pages in the
study.
UserAgent() USERAGENT() Returns respondent's User Agent.
Quota Control Functions:

GetValue(QuestionName) GETVALUE("QuestionName") Returns the quota cell value to w hich the
respondent has been assigned for the
quota question referred to by
QuestionName; returns 0 if unassigned;
returns -1 if respondents did not qualify
due to logic, and -2 if respondents qualified
but the quota w as already filled.
QuotaCellName QUOTACELLNAME Returns the quota cell name to w hich the
(QuestionName) ("QuestionName") respondent has been assigned; returns
empty string (nothing) if unassigned
IsQuotaCellOpen ISQUOTACELLOPEN Returns true (1) if the cell is open, false (0)
(QuestionName, ("QuestionName", if the cell is closed

QUOTA_CELL_NAME) QUOTA_CELL_VALUE)
IsQuotaCellOpen
(QuestionName,
QUOTA_CELL_VALUE)
QuotaCellRemaining QUOTACELLREMAINING Returns number of completes left to fill the
(QuestionName, ("QuestionName", quota cell. Returns 0 if quota cell is full.
QuotaCellRemaining
(QuestionName,
QUOTA_CELL_VALUE)
QuotaCellLimit QUOTACELLLIMIT Return the quota cell limit
(QuestionName, ("QuestionName",
QuotaCellLimit
(QuestionName,
QUOTA_CELL_VALUE)
QuotaCellCompletes QUOTACELLCOMPLETES Returns number of completes in the quota
(QuestionName, ("QuestionName", cell
QuotaCellCompletes
(QuestionName,
QUOTA_CELL_VALUE)
AreAllQuotasClosed() AREALLQUOTASCLOSED() Returns true if all cells have been closed.
This might be useful if you don't w ant to
show a "survey closed" message but
w ould like to skip the respondent to a
different survey.
AreAllQuotaCellsClosed AREALLQUOTACELLSCLOSED Returns true if all cells have been closed
(QuestionName) ("QuestionName") for the specified quota.
AreAnyQuotasClosed() AREANYQUOTASCLOSED() Returns true if all cells have been closed
for any quota.
Math Functions:

Abs (*NUMBER) abs (NUMBER) Returns the absolute value of NUMBER.
Ceiling (NUMBER) CEILING (NUMBER) Rounds the NUMBER up to the closest
integer.
Exp (NUMBER) exp (NUMBER) Returns e (2.71828183) raised to the
pow er of NUMBER
Floor (NUMBER) int (NUMBER) Rounds the NUMBER dow n to the closest
integer.
FORMATNUMBER (NUMBER, FORMATNUMBER (NUMBER, Formats a number w ith commas and
"ThousandsSeparator", "ThousandsSeparator", decimal places. More Information.
"DecimalSeparator", "DecimalSeparator",
NumDecimalPlaces) NumDecimalPlaces)
Log10 (NUMBER) LOG10 (NUMBER) Returns the log (base 10) of NUMBER.
Ln (NUMBER) log (NUMBER) Returns the natural log of NUMBER.
POWER (NUMBER1, NUMBER2) POWER (NUMBER1, NUMBER2) Returns NUMBER1 raised to the NUMBER2
pow er.
206 SSI Web v8
ROUND (NUMBER) ROUND (NUMBER, 0) Rounds NUMBER to the #DECIMALS

ROUND (NUMBER, #DECIMALS) ROUND (NUMBER, #DECIMALS) places. If no decimal places are specified,
rounds to nearest integer.
ROUNDTONUMBER (NUMBER, ROUNDTONUMBER (NUMBER, Rounds NUMBER to the nearest ROUNDTO
ROUNDTO) ROUNDTO) value, such as to the nearest 10, 25, 50,
ROUNDTONUMBER (NUMBER, ROUNDTONUMBER (NUMBER, 100, 250, 500, 1000, 5000, 10000, etc.
ROUNDTO, #DECIMALS) ROUNDTO, #DECIMALS)
Sqrt (NUMBER) sqrt (NUMBER) Returns the square root of NUMBER.
*For all math functions, NUMBER can be any QuestionName that contains a numeric value or any numeric expression
(including functions and mathematical operators) that returns a numeric value.
Loop Functions:

AIELoopList (Loop_Name, AIELOOPLIST ("Loop_Name", Similar to the AIE list function. Adds the list
Question_Name, Value) "Question_Name", Value) item from the loop list to the constructed
list if the answ er to the question is equal to
Value.
AIGLoopList (Loop_Name, AIGLOOPLIST ("Loop_Name", Similar to the AIG list function. Adds the
Question_Name, Value) "Question_Name", Value) list item from the loop list to the
constructed list if the answ er to the
question is greater than the Value.
AILLoopList (Loop_Name, AILLOOPLIST ("Loop_Name", Similar to the AIL list function. Adds the list
Question_Name, Value) "Question_Name", Value) item from the loop list to the constructed
list if the answ er to the question is less
than the Value.
ANELoopList (Loop_Name, ANELOOPLIST ("Loop_Name", Similar to the ANE list function. Adds the
Question_Name, Value) "Question_Name", Value) list item from the loop list to the
constructed list if the answ er to the
question is not equal to the Value.
LOOPITERATION ( ) LOOPITERATION ( ) Returns the current iteration count for the
LOOPITERATION (Loop_Name) LOOPITERATION list item for the current loop. Loop Name is
("Loop_Name") used in case more than one loop applies
(e.g. nested loops).
LOOPLABEL ( ) LOOPLABEL ( ) Returns the label for the list item for the
LOOPLABEL (Loop_Name) LOOPLABEL ("Loop_Name") current loop. Loop Name is used in case
more than one loop applies (e.g. nested
loops).
LOOPVALUE ( ) LOOPVALUE ( ) Returns the parent list item number for the
LOOPVALUE (Loop_Name) LOOPVALUE ("Loop_Name") list item for the current loop. Loop Name is
used in case more than one loop applies
(e.g. nested loops).
String Functions:

IsNumber (QuestionName) ISNUMBER ( Is true if the entire string is all digits, false
GETVALUE("QuestionName")) otherw ise.
More info
Length (QuestionName) length ( Returns the number of characters
GETVALUE("QuestionName")) contained in the response.
Mid (QuestionName, MID Extracts a substring of a string given by
BeginPosition, (GETVALUE("QuestionName"), QuestionName. The substring is extracted
Length) BeginPosition, starting at the character number specified
Length) in BeginPosition (1-based counting) w ith
length specified in Length.
TextExists (QuestionName, TEXTEXISTS (GETVALUE( Searches for "TextToLookFor" in the

"TextToLookFor") "QuestionName"), answ er of QuestionName. If found, a 1 is
"TextToLookFor") returned; if not found, a 0 is returned.
ToLower (QuestionName) lc Returns a low ercased version of the
(GETVALUE("QuestionName")) response.
ToUpper (QuestionName) uc Returns an uppercased version of the
(GETVALUE("QuestionName")) response.
Error Functions:

ErrCursum () ERRCURSUM () Returns the current sum of the constant
sum question.
ErrField () ERRFIELD () Returns the question number on the page.
ErrMax () ERRMAX () Returns the maximum value allow ed for the
question.
ErrMin () ERRMIN () Returns the minimum value allow ed for the
question.
ErrQName () ERRQNAME () Returns the question name.

ErrText () ERRTEXT () Returns the question-specific error
message that is optionally specified at the
question level, under the Advanced...
button.
ErrTotal () ERRTOTAL () Returns the total amount that the

responses to the constant sum question
should add to.
CBC Functions:

CBCAttributeLabel CBCATTRIBUTELABEL Returns the attribute label for the given
(ExerciseName_CBCQuestion ("ExerciseName_CBCQuestio attribute row position. Useful if the
Name, nName", attribute position is randomized in the
AttributePosition#) AttributePosition#) design.
CBCAttributeValue CBCATTRIBUTEVALUE Returns the attribute value for the given
208 SSI Web v8
(ExerciseName_CBCQuestionN ("ExerciseName_CBCQuestion attribute concept position and level

ame, Name", position. Useful if the attribute position is
ConceptPosition#, ConceptPosition#, randomized in the design.
LevelTextPosition#) LevelTextPosition#)
CBCDesignLevelText CBCDESIGNLEVELTEXT Returns the level text for the specified CBC
(ExerciseName_CBCQuestio ("ExerciseName_CBCQuestio question for the specified concept position
nName, nName", and level text position. This function is
ConceptPosition#, ConceptPosition#, used w hen researchers w ant to
LevelTextPosition#) LevelTextPosition#) customize the format of their CBC
questions.
CBCDesignLevelValue CBCDESIGNLEVELVALUE Returns the level value for the specified
(ExerciseName_CBCQuestio ("ExerciseName_CBCQuestio CBC question in the specified concept
nName, nName", position and level position w ithin the task.
ConceptPosition#, ConceptPosition#, Useful if the attribute position is
LevelTextPosition#) LevelTextPosition#) randomized in the design.
CBCDesignConceptValue CBCDESIGNCONCEPTVALUE Returns the actual concept number from
(CBCQuestionName, ("CBCQuestionName", the design that is being displayed to the
ConceptPosition#) ConceptPosition#) screen in ConceptPosition#. Position
numbers w ill only differ if concept sorting
is in place. This function is used w hen
researchers w ant to customize the format
of their CBC questions.
CBCNone () CBCNONE () Returns the "None" text for the CBC
question (traditional "None" option only)
and is only used in Shelf Facing Display
CBC questions.
CBCVersion () CBCVERSION () Returns the design version number and
w hen creating paper-and-pencil
interview s. (Example: D-1, T-CBCRAN1)
CBCCurrentTask() CBCCURRENTTASK() Returns the index number for the ith CBC
task in the exercise. For the first task it
w ill return a 1, the second one a 2, etc.
Must be used inside of the CBC question.
CBCTotalTasks() CBCTOTALTASKS() Returns the total number of CBC tasks in
the current exercise. Must be used inside
of a CBC question.
ACA Functions:

ACAAttribute () ACAATTRIBUTE () Returns the text for the attribute label
w ithin the ACARating or ACAImportance
questions.
ACABest () ACABEST () Within the ACAImportance question,
returns the level text for the "best" level
rated from the previous ACARating
question.
ACABestLevelLabel ACABESTLEVELLABEL Returns label for best attribute level of
(Attribute#, Rank#) (Attribute#, Rank#) Attribute #. Rank# optional. Rank# of 2
returns second-best attribute level.
ACABestLevelValue ACABESTLEVELVALUE Returns level number for best attribute
(Attribute#, Rank#) (Attribute#, Rank#) level of Attribute #. Rank# optional. Rank#
of 2 returns second-best attribute level
number.
ACAImportance ACAIMPORTANCE Returns the importance score for the
(Attribute#) (Attribute#) attribute, as specified by Attribute#. May
be used after the ACAImportance question
section. (See also ACAStrictImportance.)
ACALeastImpAttLabel ACALEASTIMPATTLABEL Returns label for the least important
(Rank#) (Rank#) attribute. Rank# optional. Rank# of 2
returns second-least important attribute.
ACALeastImpAttValue ACALEASTIMPATTVALUE Returns attribute number of the least
(Rank#) (Rank#) important attribute. Rank# optional. Rank#
of 2 returns second-least important
attribute number.
ACAMostImpAttLabel ACAMOSTIMPATTLABEL Returns label for the most important
(Rank#) (Rank#) attribute. Rank# optional. Rank# of 2
returns second-most important attribute.
ACAMostImpAttValue ACAMOSTIMPATTVALUE Returns attribute number of the most
(Rank#) (Rank#) important attribute. Rank# optional. Rank#
of 2 returns second-most important
attribute number.
ACAPriorsBestLevelLabel ACAPRIORSBESTLEVELLABEL Returns the best level text for attribute#,
(Attribute#) (Attribute#) based only on Priors information.
ACAPriorsBestLevelValue ACAPRIORSBESTLEVELVALUE Returns the best level number for
(Attribute#) (Attribute#) attribute#, based only on Priors
information.
ACAPriorsWorstLevelLabel ACAPRIORSWORSTLEVELLABEL Returns the w orst level text for attribute#,
(Attribute#) (Attribute#) based only on Priors information.
ACAPriorsWorstLevelValue ACAPRIORSWORSTLEVELVALUE Returns the w orst level number for
(Attribute#) (Attribute#) attribute#, based only on Priors
information.
ACAStrictImportance ACASTRICTIMPORTANCE Returns the strict importance score for the
(Attribute#) (Attribute#) attribute, as specified by Attribute#. May
be used after the ACAImportance question
section.
ACAUtility ACAUTILITY Returns the part-w orth utility for the
(Attribute#, Level#) (Attribute#, Level#) attribute and level, as specified by
Attribute# and Level#. May be used after
the ACAImportance question section.
More info.
ACAWorst () ACAWORST () Within the ACAImportance question,
returns the level text for the "w orst" level
rated from the previous ACARating
question.
ACAWorstLevelLabel ACAWORSTLEVELLABEL Returns label for w orst attribute level of
(Attribute#, Rank#) (Attribute#, Rank#) Attribute #. Rank# optional. Rank# of 2
returns second-w orst attribute level.
ACAWorstLevelValue ACAWORSTLEVELVALUE Returns level number for w orst attribute
(Attribute#, Rank#) (Attribute#, Rank#) level of Attribute #. Rank# optional. Rank#
of 2 returns second-w orst attribute level
number.
CVA Functions:
210 SSI Web v8

CVAVersion () CVAVERSION () Returns the design version number and
interview s w ith multiple versions.
(Example: D-1, T-CVA_1)
MaxDiff Functions:

MaxDiffDesignLabel MAXDIFFDESIGNLABEL Returns the item text for the specified
(ExerciseName_ ("ExerciseName_ MaxDiff question for the specified
MaxDiffQuestionName, MaxDiffQuestionName", Position#. This function is used w hen
Position#) Position#) researchers w ant to customize the format
MaxDiffDesignValue MAXDIFFDESIGNVALUE Returns the item value for the specified
(ExerciseName_ ("ExerciseName_ MaxDiff question for the specified
MaxDiffQuestionName, MaxDiffQuestionName", Position#. This function is used w hen
Position#) Position#) researchers w ant to customize the format
MaxDiffRankAttLabel MAXDIFFRANKATTLABEL Returns the list item label for the item w ith
(ExerciseName, Rank#) ("ExerciseName", Rank#) the rank specified. So a rank of 1 w ould
return the list item label w ith the highest
maxdiff logit score. More info.
MaxDiffRankAttValue MAXDIFFRANKATTVALUE Returns the list item value for the item w ith
(ExerciseName, Rank#) ("ExerciseName", Rank#) the rank specified. So a rank of 1 w ould
return the list item value w ith the highest
maxdiff logit score. More info.
MaxDiffScore MAXDIFFSCORE Returns the maxdiff logit score for the
(ExerciseName, ("ExerciseName", ItemIndex of the list associated w ith
ItemIndex) ItemIndex) ExerciseName. More info.
MaxDiffCurrentSet() MAXDIFFCURRENTSET() Returns the index number for the ith
MaxDiff set (question) in the exercise. For
the first set it w ill return a 1, the second
one a 2, etc. Must be used inside of the
MaxDiff question.
MaxDiffTotalSets() MAXDIFFTOTALSETS() Returns the total number of MaxDiff sets

(questions) in the current exercise. Must
be used inside of a MaxDiff question.
MaxDiffVersion() MAXDIFFVERSION() Returns the design version number and

interview s involving multiple versions.
(Example: D-1, T-MXDIFF_1). Only
available w ithin a MaxDiff task.
212 SSI Web v8
ACBC Functions:

ACBCBYOLabel ACBCBYOLABEL Returns the level label selected in the BYO
(ExerciseName, ("ExerciseName", question for the indicated attribute#.
Attribute#) Attribute#)
ACBCBYOValue ACBCBYOVALUE Returns the level value selected in the BYO
(ExerciseName, ("ExerciseName", question for the indicated attribute#.
Attribute#) Attribute#)
ACBCCurrentCalibration() ACBCCURRENTCALIBRATION() Returns the index of the current Calibration
Concept. Needs to be used inside of an ACBC
Calibration.
ACBCCurrentChoiceTask() ACBCCURRENTCHOICETASK() Returns the index of the current Choice Task.
Needs to be used inside of an ACBC
ChoiceTask.
ACBCCurrentScreener() ACBCCURRENTSCREENER() Returns the index of the current Screening
Task. Needs to be used inside of an ACBC
Screener.
ACBCIsMustHave ACBCISMUSTHAVE Returns True if the attribute and level specified
(ExerciseName, ("ExerciseName", has been declared a "Must Have".
Attribute#, Level#) Attribute#, Level#)
ACBCIsUnacceptable ACBCISUNACCEPTABLE Returns True if the attribute and level specified

(ExerciseName, ("ExerciseName", has been declared "Unacceptable". If
Attribute#, Level#) Attribute#, Level#) referencing a Summed Price attribute, specify
a price value rather than the Level#.
ACBCNumCalibrations() ACBCNUMCALIBRATIONS() Returns the total number of Calibration

Concepts. Can use this inside of a
ACBCCalibration. If used outside of an
ACBCCalibration needs to have the
ACBCExerciseName passed as a parameter.
ACBCNumChoiceTasks() ACBCNUMCHOICETASKS() Returns the total number of Choice Tasks. Can
use this inside of a ACBCChoiceTask. If used
outside of an ACBCChoiceTask needs to have
the ACBCExerciseName passed as a
parameter.
ACBCNumScreenedInConcepts ACBCNUMSCREENEDINCONCEPTS Returns the total number of Product Concepts
(ExerciseName) ("ExerciseName") marked "a possibility" by the respondent.
NOTE: Does not include the BYO if it is
included in the tournament.
ACBCNumScreeners() ACBCNUMSCREENERS() Returns the total number of Screening Tasks.
Can use this inside of a ACBCScreener. If
used outside of an ACBCScreener needs to
have the ACBCExerciseName passed as a
parameter.
ACBCWinnerLabel ACBCWINNERLABEL Returns the level label for the choice
(ExerciseName, ("ExerciseName", tournament w inner for the indicated attribute#
Attribute#) Attribute#) (if no tournament exists, returns nothing).
w hich you might show the tournament w inner.
ACBCWinnerValue ACBCWINNERVALUE Returns the level value for the choice
(ExerciseName, ("ExerciseName", tournament w inner for the indicated attribute#
Attribute#) Attribute#) (if no tournament exists, returns nothing).

w hich you might show the tournament w inner.
BYOCondText(ExerciseName, BYOCONDTEXT Within a BYO question, returns the conditional
CondText#) ("ExerciseName", text/graphic element for the CondText#
CondText#) relationship from the conditional relationship
BYOCondText(ExerciseName, table. If XPos and YPos (values expressed in
CondText#, XPos, YPos) BYOCONDTEXT Pixels) are given, then the conditional element
("ExerciseName", is show n at that fixed position on the screen
CondText#, XPos, YPos) (hovering graphic).
Perl Programming Language Basics
There are many books and web sites devoted to teaching Perl. An introductory Perl tutorial can be found
at: http://perldoc.perl.org/perlintro.html.
Variables
Flow Control
Custom Functions
Variables
Scalars – Scalars can hold single values such as an integer, real number, or text. Scalar
names must always begin with a $ sign. All variables in Unverified Perl must be declared
using the word "my". For example:
my $numDays = 12;
my $numTotalSeconds = 0;
my $txtMessage = "The total number of seconds is:";
$numTotalSeconds = $numDays * 24 * 60 * 60;
return $txtMessage . $numTotalSeconds;
Notice that before you can use $numDays, it needs to be declared using my $numDays. The
first line declares $numDays as a scalar variable and then assigns it to contain the value 12.
When run, the above script returns:
The total number of seconds is: 1036800
(The "." in $txtMessage . $numTotalSeconds concatenates two values.)
Arrays – Arrays are variables that hold a collection of items. Array names must begin with an
@ sign. For example:
my @Scores = (55, 22, 4, 17);

my $FirstScore = 0;
$FirstScore = $Scores[0];
The first line above creates an array named Scores. It then initializes @Scores to a list of
214 SSI Web v8
scores: 55, 22, 4, and 17. To extract the elements of the array, you must replace the @ with
a $ and reference the element using [ NUMBER ] where NUMBER is the ith list item, zero
based. So [0] returns the 1st item, [1] the 2nd and so on.
Hashes – Hashes are variables that hold a collection of items that consist of key value pairs.
The values are accessed by referring to their key. Hash names begin with a % sign. For
example:
my %ScoresLookup = ();
%ScoresLookup = (
'Bob' => 55,

'Sally' => 22,
'Frank' => 4,
'Pam' => 17
);
return "Sally has a score of: " . $ScoresLookup{'Sally'};
The above code declares a hash named ScoresLookup. It assigns a set of key value pairs to
ScoresLookup. Finally it returns the sentence:
Sally has a score of: 22
Notice that to access an item's value, we replace the % in the hash name with $ and use
{'KEY'} where KEY is the key that is associated with a certain value.
Flow Control
The "if" statement can be used to conditionally execute code. For example:
{
AIC("Q3");
}
The AIC("Q3") function is only executed if the answer to Q1 is greater than 5. The following
statement is also valid:
{
AIC("Q3");
}
elsif(GETVALUE("Q1") < 5)
{
AIC("Q2");
}
else
{
AIC("Q5");
}
If the answer to Q1 is greater than 5, then AIC("Q3") is executed. If it is less than 5, then
AIC("Q2") is executed. If it is equal to 5, the first two tests fail and AIC("Q5") is executed.
In conditional logic, use the following symbols:
"&&" for "And"

"||" for "Or"
"==" for "="
For example:
if (GETVALUE("Q1") == 3 && (GETVALUE("Q2") > 4 || GETVALUE("Q2") < 100))

{
AIC("Q5");
}
The "for" loop allows you to execute a section of code over and over again. For example:
my @Scores = (55, 22, 4, 17);

my $TotalOfScores = 0;
my $i = 0;
my $FirstScore=0;
for($i = 1; $i <= 4; $i++)

{
$TotalOfScores = $TotalOfScores + $Scores[$i – 1];
}
$FirstScore = $Scores[0];
Another way that you could do this is with a "foreach." For example:
my @Scores = (55, 22, 4, 17);

my $TotalOfScores = 0;
my $SingleScore = 0;
foreach $SingleScore (@Scores)

{
$TotalOfScores = $TotalOfScores + $SingleScore;
}
The "foreach" fills $SingleScore with each value inside @Scores one at a time until it reaches
the end of @Scores.
Custom Functions
You can create your own Perl functions by using the "sub" keyword. For example:
sub AddTwoNumbers
{
my($NumOne, $NumTwo) = @_;
my $TheSum = $NumOne + $NumTwo;
return $TheSum;
}
The above code creates a new function named "AddTwoNumbers". The line:
my($NumOne, $NumTwo) = @_;

216 SSI Web v8
is the syntax required to pass two parameters to the function. To use this new function, use:
my $Result = 0;
$Result = AddTwoNumbers(5, 6);
Result would then be assigned the number 11. When "AddTwoNumbers" is called, 5 is
assigned to $NumOne and 6 to $NumTwo. The "return" keyword passes the result from the
function back to the caller.
Examples
Conditional List Logic – In constructed list logic, you might conditionally call a function. For
example:
AIC("q1");
if(LISTHASPARENTMEMBER("ListA", 5))
{
ADD("ParentList", 7);
}
End Unverified
This example begins by adding all of the items that were chosen from "q1". An Unverified Perl
"if" statement is then used to add list item #7 if list item #5 is in the list.
Dynamic Text - This example shows how to use Unverified Perl to show dynamic text to the
screen. It also shows how to concatenate .
[%
if(GETVALUE("q1") > 1)
{
return "Tell us about each of your " . GETVALUE("q1") . " cars";
}
else
{
return "Tell us about your car";
}
End Unverified
%]
If the answer to "q1" is greater than 1 (say 5), the question header is in the plural form:
Tell us about each of your 5 cars
If "q1" is 1, the singular form is used:
Tell us about your car

Notice the line:
return "Tell us about each of your " . GETVALUE("q1") . " cars";
The period concatenates values together.
Working with Variables and Saving them to the Database – First, create a Free Format
question with a variable of type "hidden." Then, include the following text in the HTML section
of the Free Format question:
<input type="hidden" name="ff_income" value="

[%
my $TakeHome = 0;
$TakeHome = GETVALUE("QIncomeVar") * .85;
return $TakeHome;
End Unverified
%]
">
The text above defines an HTML hidden input field. The name of the input field is "ff_income"
and the value is computed by SSI Script. In the SSI Script, a variable named "$TakeHome" is
defined. "$TakeHome" is then assigned the value of the question "QIncomeVar" (answered on
a previous page) multiplied by .85 and the result is returned. When the SSI Script runs on the
server, it might return:
<input type="hidden" name="ff_income" value="42500">
When the form is submitted, the data field "ff_income" now contains 42500.
You can then use this value throughout your survey and it is saved in your database.
Calling Other Perl Scripts – You can use Unverified Perl to call other Perl scripts on your
server. For example:
[%
require "CustomPerl.pl";
my $Num = 5;
my $strResult = "";
$strResult = CustomPerl::MyFunction($Num);
End Unverified
%]
The "require" statement above makes all of the Perl code in "CustomPerl.pl" available for you
218 SSI Web v8
to call in your SSI Web survey. The function "MyFunction" which is declared in
"CustomPerl.pl" is now available.
Connecting to a Database – The following code includes the code contained in "dbrun.pl"
with the keyword "require". It then calls a function defined in "dbrun.pl" called
"RunDBCommand" and returns the result from a database query.
[%
require "dbrun.pl";
my $intRecordNum = 1;
my $strCommand = "SELECT * FROM User_Directory WHERE id ='" . $intRecordNum
. "';";
my $strResult = "";
eval
{
my $sth = 0;
my $dbh = 0;
($sth, $dbh) = dbrun::RunDBCommand($strCommand, 0);
if ($sth != 0 && $dbh != 0)

{
my $row = $sth->fetchrow_arrayref;
if ($row)
{
$strResult .= "<font color=\"blue\"><b>Record #</b></font>"
. $row->[0];
$strResult .= "<h3>" . $row->[1] . "</h3>";
}
else
{
$strResult .= "<br><b>No users were found. Please try
again.</b>";
}
}
$dbh->disconnect;
};
if ($@)
{
$strResult .= "<br><br>" . $@ . "<br><br>";
exit();
}
$strResult .= "<br><br>\n\nSQL:" . $strCommand;
End Unverified
%]
220 SSI Web v8
1.3.9 Surveys in Multiple Languages
Using the XML .XLF/XLIFF Format Import/Export
The most efficient way to develop surveys in multiple languages is to first program the entire survey
(including all skip patterns, etc.) within SSI Web in the source language (for this explanation, we'll
assume English). Then, using File | Translation | Export Text for Translation... export all the English
text of the survey to an XML .xlf/XLIFF file format. Translation houses commonly use software tools that
support this format. The XLIFF file includes each text string twice: once in the source language and a
second time for the translated version of that text. For example, the translation house initially sees the
following information within the .xlf file:
<source language> Don't eat at Pepe's house

<target language> "Needs-Translation"
Using their software tools, the translation house translates the target language text. The .xlf file they
return to you might include:
<source language> Don't eat at Pepe's house

<target language> No comas en la casa de Pepe
Use File | Save As to create a copy of your existing English (source language) project, giving the new
SSI Web project a new name such as "mystudy_spanish." You would open the mystudy_spanish
study and using File | Translation | Import Translated Text..., bring all the translated text into your new
Spanish study. All of the <target language> phrases found in the .xlf file overwrite the existing English
phrases. A backup of your survey (prior to importing and overwriting the text) is saved in your project's
Recovery folder.
This results in a separate, parallel study for each language. But, the variable names are identical across
different versions of the questionnaire. So, you could accumulate multiple datafiles (.csv) into SSI Web if
you wanted to append the data from the various languages.
Create the XLIFF File
When you select File | Translation | Export Text for Translation... the following is displayed:
A separate .XLF file must be created for each Target Language you wish to translate the questionnaire
into. In the example above, we are translating from English (Source Language) into Spanish (Target
Language).
Each language is identified by either a two-character or three-character code ("en" for English, etc.).
These codes are called the ISO 639-1 or ISO 639-2 codes. These standardized codes are required by
many programs that operate on the .XLF file, so they can perform some automated processes.
If the language you are using is not identified in the list above, you should visit: http://www.loc.gov/
standards/iso639-2/php/code_list.php to find the appropriate code. Type the codes directly into the text
box interface above.
Directly Typing or Copy-Pasting Text into SSI Web
We assume that you have the ability to enter the survey text (using a keyboard or other input device) or
that you have the translated text in a Word document, or some other program that displays the text
correctly (the characters display as they are intended to be shown in the survey).
When authoring surveys in SSI Web, you should ensure that the characters appearing in the input boxes
in SSI Web's interface are the characters that respondents should see. (They may not be formatted with
222 SSI Web v8
the correct size, color, or font, but they should be the correct character. If you see a "?", a box
character, or just the wrong character(s), then the respondents are most likely not going to see the
correct character either.) SSI Web uses a Unicode representation of the characters and the
STUDYNAMEqst.cgi file uses UTF-8 Unicode character encoding. You do not need to do anything
additional to make SSI Web display your text correctly. Do NOT add a <META> tag specifying the
character set. (SSI Web automatically includes a <META> tag indicating that the UTF-8 character set
is being used.)
SSI Web supports double-byte characters in file names for graphics and also in files you name during
export.
If you use Excel to open a .txt or .csv file that you have exported that contains double-byte characters,
you should first open Excel and then open the file using File | Open. This brings up a import wizard
where you can specify to use Unicode encoding. You should have Excel 2003 or later to open files
properly containing UTF-8 Unicode encoded characters.
Many languages work without any special considerations (such as Spanish, German, French, etc.).
Just enter the characters or copy and paste them into SSI Web and they will appear correctly. If the
characters do not appear correctly, then try the procedures outlined below.
Test your survey carefully. Be careful to check the survey in Preview mode against your original Word
document to ensure that the displayed text is correct. Be sure to test your survey on multiple browser
types and using older versions of browsers since they all display HTML slightly different.
Before You Begin
First Steps
A. Add Windows support for the language in which you'll be working.
a. Windows XP Users:
1. Click Start + Control Panel + Regional and Language Options.
2. Under the Languages tab, check the two boxes for Install files for complex script and
right-to-left languages (including Thai) and Install files for East Asian languages.
3. Restart your computer when prompted. You may also be asked to provide your
Windows installation disks. You may need to have administrative rights to make these
changes.
b. Windows Vista Users (Ultimate or Enterprise Only):

1. Click Start + Settings + Control Panel + Clock, Language and Region + Regional
and Language Options
2. Go to the Keyboard and Languages tab. Under Display Language, click Install/uninstall
languages and then follow the steps. If you are prompted for an administrator password
or confirmation, type the password or provide confirmation.
B. (This step is for users who will be copying from Microsoft Office into SSI Web.) If it is not already
installed, add language support in Microsoft Office for the languages you'll be using. This may require
you to download additional components from the Microsoft website. Click Start + Control Panel + Add
or Remove Programs. Click Microsoft Office and click Change. When the setup dialog appears,
select Add or Remove Features and then Next >. Select Office Shared Features – International
Support and make sure that all font support is installed.
1.3.10 Print Survey

Many researchers want to print a hard-copy representation of their survey. Typical reasons for doing this
are:
1. To share a copy of the questionnaire with a client,

2. For internal review (quality assurance check).
SSI Web's Print Survey (File | Print Study) capability can print/save a copy of your questionnaire in two
formats:
a) Text and Settings View: a text-only format intended for internal review that includes many
details about each question: question text, response options, answer settings (such as min,
max, number of responses, randomization, etc.), skip logic, lists, constructed list logic, attribute
and level lists, and conjoint settings.
b) Graphical View: a graphical PDF or HTML format that shows what the questionnaire looks like
to the respondent as displayed by a web browser, with question name labels alongside each
question for easy reference.
Treeview Navigation
You can navigate the "Treeview" menu to select which items to include in your printout. Click the "+"
icons to expand and view the items under each node. The selected questions on the Question List are
shared between the Text and Graphical views. Thus, if you select a subset of questions to print, if you
switch from one view to the other, the selection applies to both views.
Selecting a subset of questions from the question list. Often, you may want to print questionnaire
for just a subset of the questions. First, de-select all questions in your list. Expand the Question List
node by clicking its "+" icon. Right-click the label Question List and select uncheck all. Then, you can
individually select the questions by checking the box next to each question label.
224 SSI Web v8
1.4 Respondent Passwords

1.4.1 Introduction to Passwords
Posting your questionnaire online gives anyone connected to the Internet potential access to your
survey. The potential imposter/repeater problem is well-known. By default, SSI Web's password
functionality is turned off, meaning that respondents may access the survey as many times as they
want. You can limit or control this problem by assigning passwords within the Questionnaire Access
and Passwords dialog.
Password Fields
You can require the respondent to type a single or multiple passwords (SSI Web supports up to
255 characters per password). You can also specify how many times each password combination
may be used to access the survey. Most of the time, each password combination is specified for
single use (Max Respondents=1). This has two benefits: each respondent may only take the
survey once, and if respondents return to an incomplete survey, SSI Web can restart them at the
point they left off. Restarting is not allowed for a password where Max Respondents permitted is
greater than 1 (unless you use Cookies). Providing respondents with passwords permits them to
access just their survey; it does not give them access to other aspects of your Web server,
including the stored data.
Merged Fields: Sometimes you already have some respondent data prior to fielding a survey
(such as descriptive data for online panelists) and you want this information to be available during
the questionnaire to display certain prior-known information or to execute skip patterns. SSI Web
lets you supply up to 250 Merged Fields (variables) in addition to passwords that will be
automatically merged into each respondent's record upon accessing the survey. Both the
Password Fields and the Merged Fields are stored within the same database file.
Pass-In Fields: It is common to "pass-through" data into the survey on the URL link that
respondents click to start the survey. For example, you might be launching an SSI Web survey
from a third-party application and want to pass-through the respondent ID# to be stored as well in
the respondent's record in SSI Web. You define these Pass-In Fields within the Passwords area.
The number of pass-in fields is only limited by the constraints on length of URL strings imposed by
browsers.
Cookies: By default, we do not write cookies to respondents' hard drives. However, you can turn
cookies on and if respondents do not block them (through their browser configuration settings),
they can be used to restart incomplete questionnaires. The cookie will also keep a respondent
from completing more than one survey (unless, of course, the respondent disables the cookie). If
you ask SSI Web to write cookies, they will be used when the respondent could not otherwise be
uniquely identified by the password specified to start the survey (e.g. passwords with Max
Respondent>1, or no passwords in place at all).
User-Defined Passwords: When this option is enabled, if the password that the respondent uses
is not found, the respondent is still permitted to take the survey and the password(s) the
respondent used are stored (so that the respondent could restart the survey at a later point if
needed). Case sensitivity is ignored with User-Defined Passwords. You can employ User-Defined
Passwords in addition to pre-defined passwords within the same study.
You can use the Questionnaire Access and Passwords dialog to generate thousands or even millions of
random passwords, or you can import passwords (as well as Merged and Pass-In fields) from text files.
1.4.2 Start Question Layout

Every SSI Web study includes a Start question, with associated Questionnaire Access and
Passwords. The Start question is commonly used to welcome respondents and ask them to type
passwords prior to accessing the rest of the survey. If you aren't requiring passwords, this question may
just be used to display introductory text (without a required response). Moreover, if you are bypassing
the first page of the survey using One-Click Respondent Access to the Survey, respondents will never
see the page including the Start question.
The Start question has four main sections:
1. Header 1
2. Header 2
3. Password Fields (optional)
4. Footer
If you use passwords, they are indented according to the Question Indent Width you specify in the
Survey Settings | Page Format dialog.
The Questionnaire Access and Passwords dialog has four buttons: Advanced, Preview, OK and
Cancel.
OK Saves changes to the current question and returns to the Write

current question.
Text entry fields and format controls (Color, Size, Bold, Italic, Underline, Justification) are provided for
formatting question text.
Editing Text Fields

You specify header and footer text by typing directly into the Text fields within the dialog or by
Clicking the "pencil" icon to edit the text within a larger window. You can include HTML
instructions in any of the Text fields to enhance your surveys with additional styles or graphics.
The toolbar within the text editor offers a number of buttons to automatically add HTML instructions
within your survey.

226 SSI Web v8
1.4.3 Password Fields

The Password Fields tab lets you specify any password fields (variables) that respondents are required to
provide prior to accessing the survey. Up to 125 Password Fields (variables) may be specified. If
multiple passwords are in place, all information must match prior to accessing the survey (for example,
the respondent cannot access the survey by matching two out of three password fields correctly).
Typically, if passwords are used, only one or two passwords are defined in this table (for example, a
username and password).
The table is displayed by default starting with a single blank row, where you may type a password Field
Name and supply additional settings. As you type information into this table, additional blank rows open
beneath your last entry:
To specify a password field, type a label in the first row of this table, such as myusername. (The label
must conform to question naming conventions in SSI Web.)
Choose the data type (Text, if both letters and numbers are permitted; or Number if only numbers
are permitted). Numbers are limited to the range 1 - 999,999,999.
Click Mask ed if you want the text to appear as dots when the respondent types within the text box
(so that others cannot read what actual letters are being typed).
Click Case Sensitive if you want to distinguish between upper- and lower-case letters. For
example, if Case Sensitive is checked, the password "John" is not equal to "john". In general, we
wouldn't recommend using case sensitive passwords.
Row <#> Password Label: This text field lets you specify the words that will appear next to the
text entry box, for each password specified using a row in this table. For example, highlight the
first row and type "Please type your Username:". If you have multiple passwords in this table,
highlight each password to access the associated Password Label field.
Justify: This setting controls whether the text you supply in the Row Label field is left-justified,
centered, or right-justified.
You can add another password field by typing in the next blank row, or by clicking the Insert a new
password row icon. To delete a row, click the Delete the selected password row(s) icon. To reorder the
password fields, click the up and down arrow icons.
When you click Preview, you are shown an example of how the Start question will appear to
respondents:
Click the Password Data icon to add the actual password data to your study.
228 SSI Web v8
1.4.4 Merged Fields

Sometimes you already have data for respondents prior to their completing the survey and you want this
information to be available during the questionnaire (for example, to display certain prior-known
information or to execute skip patterns). SSI Web lets you add up to 248 data fields (variables) in
addition to the passwords that will be automatically merged into each respondent's record when the
respondent accesses the survey. Merged data may be used just as regular survey data for skip
patterns, displaying prior answers to the screen, etc. Both the passwords and the Merged Fields are
stored within the same database file.
Use the table to define the Merged Fields included in your password database. As an example, suppose
we already have information on our respondents (prior to interviewing them) regarding their zip code and
names. In this survey, we'll be asking them to evaluate a list of grocery stores in their area. Since we
already know each respondent's zip code, we can merge each respondent's zip code into each
respondent's record as soon as the respondent provides his or her unique password(s) and starts the
survey. We'll use the zipcode variable to build a constructed list of grocery stores available within the
zipcode. Furthermore, we'll use the respondent's first name to customize the welcome screen, such as
"Thank you for taking this survey, Mary."
The Firstname variable may be referenced with SSI Scripting as would any other question variable within
the study. For example:
Thank-you for taking this survey, [% Firstname %].
You can add another Merged Fields variable by typing in the next blank row, or by clicking the Insert a
new merged row icon. To delete a row, click the Delete the selected merged row(s) icon. To reorder the
merged fields, click the up and down arrow icons.
Click the Password Data icon to add the actual merged fields data into the password database.
1.4.5 Adding/Importing Password Data

After you have specified the names for Password(s) or (optional) Merged Fields that you will be using in
the study, you need to provide the password data. SSI Web stores the password data (including any
Merged Fields, as shown in the example below) within a .csv file named studyname_pwds.csv). There is
virtually no limit to the number of rows of information in this table (we have had success importing or
autogenerating a few million password rows). The number of columns in the password table along with
available memory for your PC will affect your ability to import or autogenerate huge password tables.
Click the Password Data icon to access this database file. The file is opened and the display is similar
to the look of a spreadsheet.
In the example directly below, we have opened the Password Data dialog for a study where we've already
defined one password field myusername as well as two additional variables we already know prior to
interviewing the respondent (Merged Fields) called, zipcode and Firstname. We will not be asking the
respondent to specify a zipcode and Firstname after logging into the survey, since we'll already be able
to match and merge that information from merged data once the respondent provides a correct (and
unique) myusername.
Max Respondents indicates how many respondents will be allowed to access the survey using the
password combination for that row. The default is 1 and this has the benefit of allowing respondents to
restart an incomplete survey where they left off.
Now that we've defined the three variables to be included in the password data, we just need to add the
rows of data. There are multiple ways to add the password and merged fields information.
Directly typing: Highlight the field you wish to edit with the mouse and begin typing.
Copy and Paste from a Windows application: While in a separate Windows application such
as Microsoft Excel, select an area by clicking and dragging the mouse over the desired cells.
230 SSI Web v8
Press Ctrl+C (or Edit + Copy) to copy the contents into the clipboard. Switch windows back to
SSI Web. Position the cursor where you want the data to begin within the Password Data table.
Press Ctrl+V (or Edit + Paste) to paste the information into the Passwords Data table.
Importing from a .CSV file: Click the Import... button to merge information from a text-only .CSV
(comma-separated value) file. The file must include a header row, with labels matching the labels
you defined within SSI Web for Passwords and Merged Fields. If you want to import a Max
Respondents field, you must also include that column within your .CSV file, with a column header
label Max Respondents. If you do not include a Max Respondents field in your .CSV file, SSI Web
automatically inserts Max Respondents=1 for all rows.
When you attempt to import data into a Passwords file that already contains rows of information,
you are asked whether you would like to append to or overwrite the existing data.
Note: if you are importing a data file that has lots of columns, there is a copy-and-paste trick to
avoid having to first type each field name into SSI Web (see below).
Auto Generate: Click Auto Generate... to generate passwords with random values (text or
numeric). You are asked to supply the following:
Number of Rows to Generate: Specify how many rows you wish to generate.
Number of Characters: Specify how many characters (letters and/or numbers) should
be used for each password.
Use Letters/Numbers: Check whether you wish to generate passwords that include
only letters or only numbers.
Exclude: Indicates which characters should never be used. By default, the following are
excluded, because they can lead to confusion when respondents are asked to re-type
their passwords:
I (Capital I, as in India)
O (Capital O, as in Oregon)
0 Zero
1 One
You can add any additional characters you'd like to exclude.
Of course, if you are automatically including the password information in the URL string
(One-Click Access), then respondents will not be asked to re-type their passwords and
it is not necessary to exclude any confusing characters.
Auto Generate doesn't check for duplicate records as it generates the records.
Therefore, if you wish to generate a large number of passwords, you may need to generate
an even larger number to yield the target number of passwords once duplicates have been
purged. To purge duplicates, click the Verify Password Data icon (looks like a red check
mark). If you happen to forget this step, you will be warned regarding any duplicate
passwords when testing your survey under Local Test Server or when you Prepare for Web
Upload prior to fielding your study. At that point, you must return to delete any duplicates
here on the Password Data dialog before testing or fielding your study.
Auto Generate doesn't check for words that may be considered offensive. If you are
auto-generating text-only passwords, you may want to check for offensive words.
Supported Characters for Passwords:
Only ASCII codes 32 to 126 may be used for passwords. The following table displays these valid
characters:
Merged Field data are not limited to ASCII codes 32 to 126.
Trick for Importing a Large Table of Password Fields (without having to retype the field names):
The general process for importing Password (and Merged Fields) information from a .CSV file requires
that you first specify one-at-a-time the column labels for all variables you are importing. If you have a
.CSV file with variables whose header labels already conform to SSI Web question naming conventions,
there is a copy-and-paste trick for bringing those labels into SSI Web that can save you a great deal of
time and hassle.
1. Create a .CSV (comma-separated value) file that contains your password data (including any
additional merged variable fields and a Max Respondents field if you need anything other than Max
Respondent=1).
2. Within a spreadsheet program like Excel, highlight the first row containing the header labels.
Copy this selection to the clipboard by pressing Ctrl+C or Edit+Copy. In a separate worksheet,
click Edit+Paste Special+Transpose. (This copies the header labels to the second Excel
worksheet and transposes them into a single column with multiple rows.)
3. Within your second Excel spreadsheet where the labels are now arranged as a single column
with multiple rows, highlight just the labels for the password variables. Press Ctrl+C or Edit+Copy
to copy this selection to the clipboard. Within SSI Web, open the Questionnaire Access and
Passwords dialog. Open the Password Fields tab. Put your cursor in the upper-left cell in that
table and press Ctrl+V or Edit+Paste to paste in just the fields to be used as passwords.
4. Repeat the steps outlined directly above to copy-and-paste the Merged Field labels into the
Merged Fields tab.
232 SSI Web v8
5. Now that all labels have been defined within SSI Web, click the Password Data... icon to open
the password file. Click Import... and follow the prompts to import the password data from your
.CSV file (or cut and paste the data from your Excel spreadsheet directly into the grid).
1.4.6 Pass-In Fields
Some users need to pass in variable(s) to their SSI Web survey, but don't have those data ahead of time
so as to include them as Merged Variables in the password file. For example, some users choose to
link respondents from a third-party interviewing software into an SSI Web study and need to pass through
and store the respondent ID so that the two datasets can later be merged. However, the respondent ID
might be dynamically assigned in the third-party application and not known ahead of time.
The number of variables that can be passed into an SSI Web study is limited only by the restriction
imposed by browsers. Each can contain up to 255 characters of text.
Use the Pass-In Fields tab on the Questionnaire Access and Passwords dialog to specify any pass-in
variables.
To pass in a variable, it must be included on the URL link that respondents click to access the survey.
For example, the following link passes in two variables called ID and brandpref:
http:://www.sawtoothsoftware.com/brd3/cgi-bin/ciwweb.pl?
studyname=brd3&username=john&password=DGHQ&ID=42836290&brandpref=14
When the respondent clicks this link, it automatically starts the SSI Web survey called brd3, provides a
username and password, and saves the ID and brandpref values to the data for this respondent. The
Start question (page 1) is bypassed and the first page the respondent sees is page 2. For more
information about bypassing the starting page, please see One-Click Respondent Access to Survey.
(Note: be careful about specifying whether the pass-in field must be numeric only. If this restriction is in
place and text is passed into the variable, an error is written to the log file.)
234 SSI Web v8
1.4.7 Cookies and User-Defined Passwords

Cookies
Cookies are pieces of text sent between the server and respondent's computer. The text is stored in a
file on the respondent's PC in a folder reserved by the browser for storing cookie information. By default,
SSI Web does not write cookies to respondents' computers. However, you can turn cookies on using
this dialog, and if respondents do not block them (through their browser configuration settings), they can
be used to restart incomplete questionnaires. The cookie will also keep a respondent from completing
more than one survey (unless, of course, the respondent disables cookies or deletes the cookie).
If cookies are enabled within your SSI Web study, when the respondent accesses the survey, a cookie
is written to the respondent's hard drive that records the respondent's internal respondent ID and starting
time. The cookie never expires. If a respondent logs into a survey and SSI Web cannot uniquely identify
this respondent (because of passwords with Max Respondents>1, or because the study doesn't use
passwords), the cookie information is read by SSI Web to identify and restart the respondent (or to
determine that this respondent has already completed the survey). If all your passwords are unique per
respondent (Max Respondents=1), then there really is no sense in turning cookies on, as they would
never be needed to identify a respondent.
To enable cookies for your study, from the Questionnaire Access and Passwords dialog, Settings tab,
check Use cook ies to allow respondents to restart the survey.
Remember, cookies cannot guarantee that respondents can restart an abandoned survey or that
respondents cannot complete multiple surveys from the same PC. Respondents can disable and block
cookies. Some respondents may have configured their computers (or anti-virus software) to warn them
when there is an attempt to write a cookie to their hard drives. Such respondents may be negatively
inclined or even suspicious of your survey if they receive such a warning.
User-Defined Passwords
This option lets respondents create their own password(s) to access the survey. All respondents are
allowed into the survey, and returning respondents will be able to restart their survey where they left off if
they provide the same password.
You can employ User-Defined Passwords in addition to pre-defined passwords within the same study.
To enable User-Defined Passwords, from the Questionnaire Access and Passwords dialog,
Settings tab, check Allow respondents to define their own passwords.
Min Size (characters) lets you control the minimum number of characters that may be specified for
User-Defined Passwords.
If you specify a minimum number of characters, you should warn respondents in the login
instruction text regarding the minimum characters required. Also, you should modify the
default Invalid Login error message that currently says: "Invalid Password/User Name.
Access denied." to include a tip that the passwords must have at least a certain number of
characters (edit the error message from the Survey Settings + Error Messages tab).
Otherwise, respondents trying to access the survey may not know why they are receiving an
error message.
Additional Rules:
Numeric user-defined passwords may not be larger than 999,999,999.

Numeric user-defined passwords may not begin with a 0.
Numeric user-defined passwords may include a single decimal point.
236 SSI Web v8
1.5 Quota Control

1.5.1 Introduction to Quota Control
Survey researchers often want to close the survey to new respondents after a specific number of
completed records, or after a certain number of records are completed by specific types of individuals
(e.g. males, females). This section discusses Quota Control—functionality available in SSI Web for
obtaining user-specified numbers of completed records within given classifications.
The desire for Quota Control can be economic: respondents completing the survey are paid a cash
incentive and the researcher does not want to go over-budget. For example, we might want to achieve
1000 total completes. Quota Control is also used to achieve a sample reflecting known characteristics
of a population: representative proportions of male, female, low income, high income, etc. Using Quota
Control can also help us conserve our relationship with respondents: we limit the amount of time we ask
of respondents to a few screener questions if we do not need their completed record.
Quota Based on a Single Question
Consider a B-to-B study, wherein the researcher wants to interview 1000 total respondents, with a certain
number of completes within different company sizes (large, small):
Quota
Quota Cell Definitions: Cell Limits:
Large Company N=700
Small Company N=300
Total: N=1000
A respondent starts the survey and is first asked what size company she works for. This is called the
screener question. After the screener question is asked, we determine whether another respondent with
this characteristic is needed. If her Quota Cell is already filled with enough completed records for her
company size, then we can skip her to the end and mark her as disqualified. Otherwise, she is allowed
to continue the survey. This process is represented by the diagram below:
The Quota Qualification Point is inserted in the questionnaire from the Write Questionnaire dialog, by
inserting a quota "question" (in the example above, directly following Q1). The quota question appears in
the question list alongside other questions. The quota question really isn't a question in the traditional
sense (because we don't actually show anything on the screen for this question or ask the respondent to
provide an answer to a Quota question). The Quota question type represents a Quota Qualification Point
where we define the quota qualification logic and determine if the respondent belongs to a quota cell that
is already full (this determination is made when the page containing the quota question is submitted).
Based on whether a quota is full, we decide at the Quota Qualification Point whether to allow the
respondent to continue or send the respondent to another point in the survey (such as a terminate
screen).
Multiple Independent Quotas
number of completes within both different company sizes (large, small) and types (manufacturing,
services):
Quota
Company Size:
Large Company N=700
Small Company N=300
Company Type:
Manufacturing N=500
Services N=500
Total: N=1000
In this case we have two independent quotas: company size and company type. Within company size,
there are two quota cells (large and small). Within company type, there are also two quota cells
(manufacturing and services). Note that we aren't concerned with (for example) how many people from
large manufacturing firms are interviewed (we don't care about the joint occurrences of the two quota
variables).
A respondent starts the survey and is first asked what size and type company she works for. These are
called screener questions. After each screener question is asked, we can determine whether this
respondent is needed. If for either company size or company type her Quota Cell is already filled with
enough completed records, we can skip her to the end and mark her as disqualified. Otherwise, she is
allowed to continue the survey. This process is represented by the diagram below:
In the example above, once we determine that the respondent belongs to a quota cell that is full, we
238 SSI Web v8
immediately terminate the respondent.
Quotas Representing the Joint Occurrence of Multiple Variables
number of completes within different company sizes (large, small) and types (manufacturing, services):
Quota
Large Company, Manufacturing N=400
Large Company, Services N=300
Small Company, Manufacturing N=100
Small Company, Services N=200
Total: N=1000
As with the previous example, we plan to obtain 700 large company interviews, 300 small company
interviews, 500 manufacturing interviews, and 500 services interviews. But in this example, we are
concerned with how many respondents represent each company size AND type (joint occurrence). To
do this, we establish a single quota variable (at a single Quota Qualification Point) with four possible
cells (by inserting a Quota question from the Write Questionnaire dialog). For each cell, we use
qualification logic that references both the company size and company type variables using AND logic.
The questionnaire flow is shown below:
Controlling Total Number of Respondents
The simplest application of quota control is to limit the number of total completed records. From the
Write Questionnaire screen, insert a Quota question (preferably on page 1, so that respondents starting
a closed survey will immediately be skipped to your termination screen that tells them that the survey is
closed). Add a single quota cell, specify the cell limit (such as 1000, for 1000 total completed records)
and check Always qualify. This tells SSI Web that all respondents qualify for a single quota cell,
irrespective of previous answers, and the maximum number of completed records is 1000.
Additional Notes and Capabilities
SSI Web's quota control is flexible and powerful. Quota membership may be based on many variables,
resulting in a virtually unlimited number of quota cells. Screener variables can be "passed in" through a
URL link or asked at the beginning of the questionnaire. Quota control cannot guarantee that the
quotas you establish for groups are not exceeded. There may be some overage. For example, you
may wish to interview just 100 males. A simple rule would be to let males continue the survey if you
haven't yet achieved 100 completed male records. Imagine that 99 males have already completed the
survey and five males are currently in progress (have passed the Quota Qualification Point and are in the
middle of the survey), but these five haven't yet finished the survey and been counted as complete. A
new male respondent reaches the qualification point and is allowed to take the survey (since only 99
complete records have been recorded). We can see how this could result in 105 completed records, if
all the respondents in progress eventually complete the survey.
SSI Web has more sophisticated rules for determining if the Quota Cell Limit has been or is expected to
be reached. You can specify that SSI Web should assume that, say, 80% of the respondents who have
passed the Qualification Point and are actively taking the survey will eventually finish (i.e. a 20% drop-out
rate). You can also specify that respondents who haven't completed the survey within a certain amount
of time should be marked as Inactive. If these options are in place, when a new respondent reaches the
Qualification Point, SSI Web considers the number of respondents in this respondent's quota cell
already completed and actively in progress (discounted by 20%) to decide if the new respondent is
needed. We refer to this ability to determine more intelligently if more respondents are needed within a
Quota Cell as Predictive Quota Control. Predictive quota control can certainly limit the amount of
overage, but it still cannot ensure that quota cell limits are not exceeded.
Restarts: When a respondent abandons a survey and returns to take the survey (restarts), if the
respondent belongs to a quota that is now filled, the respondent will be immediately skipped to the
Disqualify Question associated with that quota (the question specified on the Settings tab).
240 SSI Web v8
1.5.2 Quota Dialog

Quota Control lets you specify how many interviews you want to complete overall, or for certain types of
respondents. Once the quota is reached, new respondents who qualify for quota cells that have already
filled can be skipped to a page that tells them that the survey is full. For a review of quota control, click
here.
Let's begin with the simple case where the researcher wants to limit the number of total surveys to 500
completed respondents. From the Write Questionnaire dialog, add a Quota question at a point very
early on in the survey (we suggest page 1 so that respondents don't have to spend time answering
questions if the quota is filled). The Quota question appears in the list of questions, but it really isn't a
question in the traditional sense, since nothing is displayed on the screen and no respondent answer is
required. Rather, it may be considered a Quota Qualification Point, at which SSI Web evaluates the
quota logic defined in the Quota question, determines if that quota cell has been filled and assigns the
respondent to a quota cell.
When you add a Quota question, the Quota dialog is displayed:
Click the Add button to add a quota cell to the table. In this first example, we've added just one quota
cell (named "Completes"), where all respondents qualify for this cell (we've clicked Always qualify rather
than giving specific qualification logic). The relevant information specified in the Cells tab dialog is as
follows:
Value Cell Name Cell Limit Qualification Logic

1 Completes 500 Always qualify
The Cell Value is a numeric value that SSI Web assigns to represent a quota cell. This is the cell's
numeric code. Cell Name is a short label for describing the quota cell which can contain no spaces.
As another example, the researcher may wish to limit the survey to 500 males and 550 females, based
on a previous question named Gender:
Value Cell Name Cell Limit Qualification Logic

1 Males 500 Gender=1
2 Females 550 Gender=2
This is an example of mutually-exclusive qualification logic: respondents are either male or female. With
SSI Web's Quota Control, it is possible to set up quota cells that are not mutually exclusive. Even if
quota cell definitions are not mutually exclusive, respondents are assigned to one (and only one) quota
cell. The order you enter the Quota Cells matters if the Qualification Logic is not mutually exclusive (by
default, Qualification Logic is evaluated from top to bottom of the list). By default, respondents are
assigned to the first quota cell for which they qualify, from top to bottom.
Quota cells can become more complex, such as a combination of Gender and Age. Also, if you do not
like the default Cell Values, you can change them:
Cell Value (Code) Cell Name Cell Limit Qualification Logic

101 YoungMales 250 Gender=1 AND Age=1
102 OlderMales 200 Gender=1 AND Age=2
201 YoungFemales 300 Gender=2 AND Age=1
202 OlderFemales 240 Gender=2 AND Age=2
Click Add to add new quota cells, or (after highlighting a quota cell) click Delete to delete a quota cell.
The Qualification Logic rules and syntax are described in the section that follows.
Even though you have specified specific Cell Limits, it is very possible for you to exceed those limits by
some (usually small) amount. For example, respondents may be entering the survey quickly, and since
those who are In Progress are not counted toward the total completed records, quota control will let
extra respondents into the survey before the Cell Limit has been reached. There are more sophisticated
rules (Predictive Quota Control) for accounting for respondents who have passed the Qualification Point
and are In Progress and these are described in the documentation for Predictive Quota Control
(Advanced tab).
Note: Changes you mak e to the quota control settings must be uploaded to the web server for these
modifications to affect data collection. Please read the section on Changing Quota Cell Settings During
Data Collection for more information.
242 SSI Web v8
Qualification Logic Syntax
SSI Web uses SSI Script for its qualification logic syntax (the syntax rules are the same as for skip
patterns). You type this logic into the Qualification Logic text field.
Supported Logical Operators:
and
or
not
Supported Mathematical Operators:
=, <, >, <=, >=, <> "not equal to"

+, -
/ "division"
* "multiplication"
^ "raise to the power of"
mod "modulo"
and all other SSI Web Functions.
You can check whether the instructions (syntax) you provide are valid by clicking . Syntax is also
checked when you prepare your survey for local or remote web server.
Example Syntax
Below are some examples of valid Qualification Logic syntax. If these statements evaluate to true, then
the respondent is qualified for that particular cell.
Q1 = 23 "Qualify for this cell if Q1 is equal to 23"

Q1 >= 10 " Qualify for this cell if Q1 is less than or equal to 10"
Q1 <> 99 " Qualify for this cell if Q1 is not equal to 99"
Age > 18
(Age > 18) and (Gender = 1)
(Age > 18) or (Q1 = 23) or (Q2 > 3.534)
((ageQuestion > 18) Or (Not Shown(ageQuestion))) "Shown" is a function that returns true if the
question specified in the parenthesis has been seen
Q23/(Q24+Q25) >= 23.5
(Q23*Q1)^0.5 < 10 "^" means "raised to the power of"
Q1OpenEnd="hello world"
1.5.3 Quota Control Settings

Disqualified respondent sk ip to question <select question> (Disqualified Question)
On the Settings tab, you specify the question that respondents should be skipped to who do
not qualify for an open Quota Cell.
Notes:
Skip patterns (post-skips) are permitted on the same page as the Quota question. Quota
Cell Membership is evaluated before the post-skips.
The quota disqualified skip is taken before any user-defined post skips.
Quota Membership will not be assigned if respondents pre-skip off the page containing
the Quota question.
Quota Cell Membership is evaluated even if the Quota question is skipped due to a
constructed list length of zero used on the page.
Quota Cell Membership is not evaluated if the page containing the Quota question is
skipped over due to skip logic.
Multiple quotas on a page are evaluated in natural order, even if questions on the page
have been randomized.
Quota membership is not evaluated if the quota question is pre-skipped.
Distinguishing Over-Quota vs. Disqualified
Some researchers need to track whether a respondent was disqualified due to not logically qualifying for
the quota versus the respondent qualified for the quota but the quota was already filled. SSI Web
records a quota cell value of -1 if respondents did not qualify due to logic and -2 if respondents qualified
but the quota was already filled. You can refer to these values within Skip logic if you need to redirect
people to a different terminate screen based on one condition or the other. For example, all respondents
are first sent to the same disqualification question as defined on the Settings tab of the Quota dialog.
However, you can create "preskips" associated with that page which skip the respondent to the
appropriate terminate screen depending on whether the quota cell value is -1 or -2.
244 SSI Web v8
1.5.4 Advanced Quota Control Settings

In the previous sections describing Quota Control, we've emphasized that it is probable that you will
actually get more completed records within each Quota Cell than specified in Cell Limits. This is
especially the case if you simultaneously invite a large number of individuals to complete the survey. For
example, if you wait to close a Quota Cell until the last respondent qualifying for this cell has completed
the survey, there may be a number of respondents In Progress who eventually complete the survey. You
will exceed your Cell Limit by these respondents still In Progress.
The settings on this dialog let you fine-tune the decision rules (Predictive Quota Control) for determining
if the Cell Limits have been, or are predicted to be, reached. The best settings for your project depend
on a number of factors, the most significant being the rate at which respondents enter the survey.
Do not use Predictive Quota Control

Allow respondents qualifying for a quota cell to continue If the quota limit has not been completely filled
The simplest rule is to allow respondents qualifying for a Quota Cell to enter and complete the
survey until the number of completed respondents within that Quota Cell reaches the Cell
Limit. Of course, this leads to the greatest likelihood of overage. But, it also fills the quota
cells the fastest.
Use Predictive Quota Control

Allow respondents qualifying for a quota cell to continue if the number of completes plus "in progress"
respondents that qualify for the quota cell is less than the quota limit
When you check this option, SSI Web takes into account In Progress respondents when
deciding whether new respondents entering the survey are needed to fill the Cell Limits. This
approach reflects Predictive Quota Control. By counting all or a percentage of In Progress
respondents toward the Cell Limit and not counting Inactive respondents, you can significantly
reduce the overage on Cell Limits. This leads to the distinction between a quota cell that is
full versus one that is closed. A quota cell is full when the number of completed respondents
reaches the Cell Limit. A quota cell is closed when the Cell Limit has been reached or when it
is projected to be reached by the number of completes plus In Progress respondents.
With Predictive Quota Control, the Cell Limits may take longer to fill than when letting all new
respondents enter the survey until enough completes are achieved. But if overage is costly,
the tradeoff is usually preferable. As an illustration, if some In Progress respondents projected
to complete the survey don't actually finish, you may have turned new respondents away that
you later realize were actually needed. The settings on this Advanced tab help you reduce the
overage while filling Cell Limits relatively quickly.
In general, we suggest you use settings that reduce the likelihood of turning new respondents
away, fill quota cells quicker, and result in only modest overage of Cell Limits.
Use time limit
By default, SSI Web counts all respondents who have passed the Quota question but have not
yet completed the survey as In Progress. But, by checking the Use time limit box, you can
specify a specific number of minutes to transpire (since the respondent has qualified for a
quota cell) after which the respondent will be marked as Inactive rather than In Progress. By
identifying which respondents are Inactive and highly unlikely to complete the survey, you can
distinguish these from the In Progress respondents and allow new fresh respondents to begin
the survey. (Respondents who have quit the survey can later restart if their quota cell is not
filled. Note: respondents can restart even if their quota cell is closed, but not filled.)
Larger Time Limits fill quota cells slower, but result in less overage. For example, if you
specify a very long Time Limit of 120 minutes, it could take up to two hours (depending on the
Projected Completion Rate) until a replacement is sought for an In Progress respondent who
quits the survey. If you are uncertain regarding a Time Limit, we suggest trying about 30
minutes (for a survey that takes on average 10 to 15 minutes).
Use projected completion rate
Assigning a Projected Completion Rate to In Progress respondents allows SSI Web to make
more intelligent decisions regarding whether to close or open a Quota Cell. Typically, about
60-90% of respondents that answer the first few questions eventually complete the survey.
This also means that around 10-40% drop out. By only counting a fraction of the respondents
In Progress toward the Cell Limits, you are less likely to turn away new respondents that you
actually need.
Larger Projected Completion Rates fill quota cells slower, but result in less overage. Imagine
that you project that 90% of In Progress respondents will complete the survey and only 60%
actually do. Until the Time Limit expires for respondents who quit so they are marked as
Inactive, new respondents will have been turned away that could have been used to fill the
quota cell. If you are uncertain regarding a Projected Completion Rate, we suggest trying
about 60%.
Check for cell membership
If your Quota Cells aren't mutually exclusive (if respondents can qualify for more than one Quota Cell),
you'll want to consider the order that SSI Web evaluates the Quota Cells and Qualification Logic. By
default, SSI Web checks Sequentially for Quota Membership. SSI Web starts at the top of the Quota
Cell table (on the Cells tab) and works down the list. The respondent is assigned to the first Quota Cell
for which she qualifies.
Sometimes, (if respondents can qualify for multiple cells) you may want to fill quota cells randomly.
Sequentially
Evaluate and assign quota cell membership sequentially, from top to bottom of the table
specified in the Cells tab.
Randomize all
Evaluate and assign quota cell membership randomly, with probability of assignment
proportional to quota cell limits.
Randomize subset
Evaluate and assign quota cell membership randomly (with probability of assignment
proportional to quota cell limits) within a subset of the range of quota cells from the table
specified in the Cells tab. Those quota cells outside the range are evaluated sequentially.
246 SSI Web v8
1.5.5 Prequalification in Quota Control

This section describes a technique called Pre-Qualification for checking if all the Quota Cells for which a
respondent could potentially qualify are full. If all potential quotas are full, we wouldn't want to waste the
respondents time by asking additional questions.
The simplest approach is to check at the very start of the survey (on page 1) if all quotas are full. To do
this, add a post-skip (a skip that is executed after the respondent submits the question) to the Start
question (or another question on page 1) with the following logic:
Sk ip if the following logic evaluates to true:
AreAllQuotasClosed()
Click Here to view other SSI Script functions that involve Quota Control.
1.5.6 Quota Changes Mid-Study

SSI Web allows you to change some of your quota control settings even after you have begun data
collection. Common changes include:
Increasing or decreasing Quota Cell Limits
Modifying the Quota Cell Name
Adjusting Predictive Quota Control settings, including Time Limit and Projected Completion
Rate
Adding additional quota cells
You make these modifications within the SSI Web interface on your local computer (make sure to upload
the changes, stored in the STUDYNAMEqst.cgi file, to the web server so that the settings take effect
within the server installation.)
If you use the Field | Web Server Management option to automatically upload your files to the server,
this will be automatically handled.
There are certain settings you should not change after beginning data collection:
Quota Cell Values
Quota Cell Qualification Logic (you can change the logic to correct errors, but respondents
already assigned to cells will not be reassigned)
Quota question names
248 SSI Web v8
1.6 Testing/Running the Survey Locally

1.6.1 Testing/Running the Survey Locally
Once you have composed some questions and specified passwords, you can test/run your questionnaire
locally on your PC using the Local Test Server that is installed along with the SSI Web system. Click
Test | Test Survey Locally....
First, your project is checked for errors.
Note: When testing your study, the first "test" respondent will always receive internal respondent
number "1". Randomizations are seeded by respondent number. To see different randomizations and
conjoint designs, you must take a few surveys.
Caution: We do not recommend using Local Test Server for collecting real data. It raises the
possibility of accidental data loss. Rather, we recommend you use the SSI Web CAPI
interviewing procedure if collecting data locally on your PC.
If you are using the demo version of the software, you are limited to a total of 5 completed records (either
qualified or disqualified). The demo software is not to be used for professional research studies.
Test Survey Locally

Click this to test/run your survey using the Local Test Server.
Test Survey Locally with Options...

Click this to test/run your survey, but with the ability to Skip to Question, Show Question
Names, Show Variable Names, Use a Specific Internal Respondent Number, Remove
all Skip Logic, or Remove Randomization*.
Admin Module...
Click this to run your Admin Module, to view, accumulate or tabulate results for your test
study running on your Local Test Server.
Download Data
Click this to download data for your test study running on your Local Test Server.
Reset
Click this to Reset your test study. Reset means deleting all data and resetting all passwords
and quotas for your test survey!
Settings...
Accesses the Local Test Server Settings dialog, that permits you to define whether the
Apache server should start up (and run in the background) when SSI Web is opened and
closed when it is closed, or whether the user can control manually when the Local Test Server
is started and stopped.
* Note: Remove Randomization does not remove all instances of randomization, such as attribute
randomization in a conjoint exercise or constructed list logic you have built on your own, etc.
Acknowledgment
The Local Test Server included in SSI Web includes software developed by the Apache Software
Foundation for use in the Apache HTTP server project (http://www.apache.org/).
Troubleshooting
If the survey is not working (Login page cannot be found) you may need to adjust the browser settings.
In Explorer, select Tools | Internet Options | Connections | LAN Settings. Make sure if you are using
a proxy server that you bypass proxy server for local addresses.
Some corporate LANs have network configurations that do not allow a Local Test Server to work.
Contact your IT department for help.
250 SSI Web v8
1.7 Survey Installation on Web Server

1.7.1 Web Server Management
SSI Web can upload your survey to a web server (Field | Web Server Management) to publish your
questionnaire to the Web. You can use Sawtooth Software's hosting service (see http://
www.sawtoothsoftware.com/services/hosting), or do it yourself using your company's server or a rented
third party server. The server must have access to a database (MySQL or Microsoft SQL Server are
supported databases) and support Perl.
(Use Field | Prepare for Manual Upload if you want to manually upload the project using your own FTP
software.)
Using Sawtooth Software Hosting Service
Sawtooth Software provides a convenient service for hosting your surveys, which charges a per
respondent fee and a server rental fee per month.
Within the SSI Web system, when you click, Field | Web Server Management, a dialog appears
prompting you for your hosting account information. Click on "Create New Account" to create a new
hosting account. Once you are approved and have an account, enter your email and password to link
SSI Web with your hosting account.
Once you click Save, the dialog changes:

To create a new hosting project click New Project. This will take you to the online hosting service where
you specify the details of your project, including a project name, and set up a new server. Once the
server is ready (often takes a minute or two and you are notified via email when your project is ready) you
will see the project name in the Hosting Project drop down list. Select your project and then click
Upload Survey to Server to upload the survey. To delete the survey from the server click "Delete
Survey from Server." (This step will delete all your data for the selected hosting project! A warning
message is displayed before data are deleted so you can confirm your intentions).
Optional: You may upload multiple surveys to the same project, as long as they reside in separate
folders on the server (e.g. English and German versions of the same survey). The Remote study path
allows you to specify the path on your server where this study will be uploaded. If the path does not
exist SSI Web creates it. Let's say for example that your server root URL is: http://
mystudy.cloudssi.com. By default the study is uploaded to the root of the web server and the
respondent login link is: http://mystudy.cloudssi.com/login.html. If you'd like to upload multiple
studies to the server and you'd like to put them in separate folders, then you would enter the folder name
into the Remote study path setting. For example if you entered study1 as the remote path then your final
link would be: http://mystudy.cloudssi.com/study1/login.html.
Self Hosting
When you click Field | Web Server Management, you are shown the Web Server Management
dialog. Click the Self Hosting tab if you plan to host the survey on your own web server.
You first must specify the Database Name, Database User Name, Database Password, and Database
Server, Database Name. Your ISP or your IT department will provide you that information.
252 SSI Web v8
You may have multiple servers that you use and each server's FTP (File Transfer Protocol) configuration
can be stored within this dialog. When you open a different project in SSI Web, any server configuration
information you have entered for other projects (on your same computer) is shared and made available.
Click Add or Delete buttons to add/delete an existing configuration.
Each project must be uploaded to its own folder on the server. If you try to upload a project to a folder
already containing an SSI Web project (at the root level), SSI Web will warn you of this and not proceed.
Your ISP or your IT department will provide you with the Web Root URL, Host Name, User Name, and
Password needed to connect with and save files to your server. The Web Root URL is the URL of your
server with no project specific info such as http://www.sawtoothsoftware.com. The Host Name provides
the ftp location of your server; the User Name and Password log you into that server. We encourage you
to use strong passwords that include a combination of alphabetic and numeric characters and are at
least eight characters long.
Advanced Settings:
From the Web Server Management dialog, when you click Advanced, the Advanced Server
Configuration dialog is displayed:
Database Settings: If you must specify the Database Server Host Name and the Database Server Port
Number (if the server manager specifies that you need to do so), you can do so here.
Survey Paths
Sometimes you may find that your ISP doesn't let you set up directories the way we've outlined. Or, you
may simply have other preferences regarding your study's architecture on your server. You can change
the names and locations of the directories on this dialog.
This dialog lets you specify relative paths for the Administrative, CGI Scripts and Data directories.
The three paths are as follows:
Relative Path from

Study to CGI Scripts: Path from login.html and admin.html to the directory that
contains ciwweb.pl and admin.pl
Relative Path from

CGI Scripts to Admin: Path from Perl scripts to the /admin directory (may contain SSI
Script)
Relative Path from

CGI Scripts to Graphics: Path from Perl scripts to the /graphics directory (may contain
SSI Script)
254 SSI Web v8
(The ability to specify SSI Script in the relative paths provides more flexibility, for example with common
cgi-bin folders, so that a generic path statement can apply to multiple studies.)
When you use the Add Image icon to add graphics to your study, your <img> tags then contain the
function [%GraphicsPath( )%] in their paths. [%GraphicsPath( )%] will be replaced on the web server
with the path that you provided here in the Server Directory Paths dialog. For example if you set your
graphics path to "../My Graphics/" and you included <img src="[%GraphicsPath( )%]picture.jpg"
BORDER="0"> in your questionnaire, then on the web server that path would dynamically change to
<img src="../MyGraphics/picture.jpg" BORDER="0">.
If you rename the directories that the files are stored in on your server or use absolute file references, put
the new MyGraphics file in your CGI Scripts directory so that the Perl scripts can find your study.
If you can only run Perl scripts from one directory and you want to have multiple studies running at the
same time. The paths in the Survey Paths dialog must be set correctly for each study and then each
study's STUDYNAME_path.cgi file must be placed in the cgi-bin directory. For example, if two studies
"T1" and "T2" share the same Perl files, then the cgi-bin directory would need to contain the T1_path.cgi
and T2_path.cgi files.
(Auto upload will detect if you are using the common cgi-bin setup and put the STUDYNAME_path.cgi
into the cgi-bin folder. If the cgi-bin folder does not already exist, auto upload will create it and upload
the perl files there as well.)
FTP
FTP Connection determines how auto upload will copy your files to the server. By default, SSI Web
attempts to detect the connection type with (Autodetect). If you already know the connection, you may
change it here.
Normal is the standard FTP network protocol for uploading files with no security. Over SSL is an
extension to FTP that will securely upload your files using Transport Layer Security (TLS) or Secure
Sockets Layer (SSL) if your server supports FTPS or FTP-SSL or FTP-Secure. Over SSH uses the
Secure Shell protocol 2.0 (SSH-2) to securely upload your files if your server supports SFTP or Secure
FTP or tunneling FTP through an SSH Connection. We encourage you to use SSL or SSH which encrypt
all data traffic.
Transfer Mode determines how the connection is established. Passive Mode is the default settings on
most servers, and should also be used if your computer is behind a firewall. Sometimes, you will be
asked to use a different Port number from the default. If you have been given a Port number, specify that
here. Active Mode tells the server to initiate the connection to your computer using an optionally
specified range of ports.
If the default folder permissions that SSI Web uses don't work for your particular server configuration, you
may change the permissions for specific folders here. You can always click Reset to Default to revert to
the original settings.
Use the Remote Study Folder on the Server Directory Paths tab to create a new folder for the current
SSI Web study on your web server. For example, specify study1 to place your study within the study1
folder on your web server.
After clicking Upload Survey to Server, another dialog with the Command Status window shows you
the current operation the auto upload function is performing. The Connection Status window shows the
FTP commands that SSI Web has used. This window acts like a progress log.
Upload Survey to Server attempts to upload your project to the given server in the specified remote
study folder. It will initially check to see if a project with the same name exists in given remote folder. If
the names match, it checks the specified database to see if the same project name exists. If the
database already contains the same project name in a different remote folder, the current upload is
aborted. Otherwise the project is uploaded to the given folder and upon completion; the project settings
are entered into the database.
After a project is completed, you may wish to have the project files removed from the remote server by
running Delete Survey from Server. This removes all project related information from the remote server
like database tables and files. If you attempt to remove a project with a different name or the files have
already been removed, SSI Web will warn you of this and not proceed.
Admin Module
Survey administrators log into the Online Administrative Module using a browser (the login page is
admin.html on the web server) to manage studies in the field and download data. A User Name and
Password are required to access the Administrative Module.
This area is used to specify the passwords you use to access the Admin Module. Default user names
and random passwords are provided for your convenience (new random passwords are created for each
new study), but you can change these if you like.
User Name
Lets you provide a name for the Admin Module User.
Read-Only Password
Lets you provide a password (passwords are case-sensitive and can only use numbers or
letters). Read-only passwords convey limited administrative rights (e.g. you cannot download
the data, delete or accumulate disqualified/incomplete data, edit the passwords file, or reset
the survey.)
Read-Modify Password
Lets you provide a password (passwords are case-sensitive and can only use numbers or
letters). Read-modify passwords convey full administrative rights.
We encourage you to use strong passwords that include a combination of alphabetic and numeric
characters and are at least eight characters long.
Customizing the Admin Module
You may customize the look of the Admin Module by creating your own CSS file. In the Admin Module
CSS File field on the Admin Module tab, enter the name of your custom CSS file. You can place this
CSS file in the graphics folder or you can refer to a remote file by listing the full URL (i.e. http:// …). If the
CSS file is placed in the graphics folder, you will need to include the graphics path. For example:
[%GraphicsPath()%]your_style.css
By including a custom CSS file we will remove all "Sawtooth Software" branding from the Admin Module.
256 SSI Web v8
To remove our title and include your company's logo include the following CSS:
#title_text
{
display: none;
}
#title
{
background:url('your_logo.gif') no-repeat center center;
height: 50px;
}
1.7.2 Manual Upload to Server

Overview
SSI Web can automatically upload your project to a server using Field | Web Server Management. If
you want to manage the process yourself, you can manually upload the survey files using FTP software.
Surveys are created using the SSI Web authoring interface on a local PC. The SSI Web authoring
program saves various files that store the study setup, passwords, and questionnaire settings. These
study-related files, together with SSI Web's Perl script files (programs that run the survey), are uploaded
to a remote web server. The data stored on the web server can be managed and retrieved using SSI
Web's Online Administrative Module.
To manually upload your files, use Field | Prepare for Manual Upload.... You will need to specify the
survey database information and click Prepare Survey Files. This will create a the Web Upload folder
with all the files necessary to run the survey on the web server. FTP software can then be used to
upload these files to the web server. After the files have been uploaded, it is critical that you log into the
Admin Module to apply any changes to the database (you will be prompted to "Apply Changes" within
the Admin Module).
Please see our online documentation at http://www.sawtoothsoftware.com/server-setup for more

information about this.
258 SSI Web v8
1.7.3 Deleting Practice Data Prior to Taking the Study "Live"

You should collect some practice data and check that everything is turning out as you expect prior to
taking a study "live." If you use Test Survey Mode, you do not need to worry about deleting practice
data, as test data are kept separate from actual study data. However, if you still want to test the actual
study (in real data collection mode), then we provide the following information.
After collecting the practice data, you will probably want to delete these records prior to starting actual
data collection. There are two methods for deleting the practice respondent data and resetting the
survey:
1) Using the Admin Module
Access the Admin Module by connecting to the Web and browsing to you admin.html page. Use the
Administrative Read-Modify username and password specified under Field | Web Server Management.
Once at the main menu, choose Reset Web Survey and click Continue. (Warning: this option deletes
all survey data. While that may be fine for practice data, you must be very careful with this option not to
delete real data in the future.)
2) Using the View Data function in the Administrative Module.
To delete selected practice data, go to View/Edit in the Administrative Module and delete the desired
records. You might, for example, assign a username of "test" for test records. You could then search
for only those records that had "test" as a username and delete them.
1.7.4 Troubleshooting Server-Side Problems

Below is a list of suggestions you might find helpful in troubleshooting web problems:
1. For server setup problems, see our Server Setup Documentation on the web.
2. Make sure Perl is working. Use our Test1.pl. If it doesn't work, contact your system administrator.
3. Make sure that the path on the first line of the script (ciwweb.pl and admin.pl) is the correct path to
your Perl executable. We have set the path at the most common directory for the Perl interpreter
[sometimes called the Perl executable] which is: /usr/bin/perl. Another common location is /usr/local/
bin/perl. Consult the Web server administrator if you don't know the correct path.)
4. When uploading your files, make sure you upload them in "binary" mode so that the number of bytes
in the files do not differ between your machine and the server.
5. Unix servers are case sensitive when it comes to filenames. Make sure that the case on your
filenames matches. We'd suggest using all lower case names. It is a good idea to make your study
names all lower case.
6. Make sure you set the permissions correctly on the files (see our documentation for details). The
admin directory needs "read," "write," and "delete" permissions. When troubleshooting, you can enable
full permissions. Once it is working, you can tighten your permissions.
7. Some ISPs or web servers require a special subdirectory where Perl scripts can run. We refer to this
as the common cgi-bin. Find out from your ISP what their rules are for running Perl scripts. If you use a
directory other than cgi-bin, make sure you update this information in your server directory paths under
the Field | Web Server Management dialog.
8. Make sure your server is configured to treat .cgi files the same as .pl files. We name most of our
server side files with *.cgi. If your server is configured correctly this will ensure that these files are not
visible to others.
9. Some servers require alternate setups that are different from our default procedure. In these cases,
you might need to change the paths in Server Directory Paths. Under the Field | Web Server
Management dialog, you can modify the system paths.
When changing the paths under Server Directory Paths, the three paths are as follows:
Administrative Directory: Path from where the Perl files are running to the /admin
directory
CGI Scripts Directory: Path from login.html and admin.html to the directory that
contains ciwweb.pl and admin.pl
Relative Path from

CGI Scripts to Graphics: Path from where Perl scripts are running to the /graphics
directory
10. If you have changed the default directory structure, you may need to move the file
STUDYNAMEpath.cgi to the same directory as the Perl files so the scripts can find it. Here are the
search paths for STUDYNAMEpath.cgi from the directory from which the Perl scripts are running:
260 SSI Web v8
../admin/
If not found in ../admin/, then it looks for the STUDYNAMEpath.cgi in:
current directory (where Perl scripts are running)
If not found there, it looks for:
current directory/studyname/admin
If not found there, it looks for the STUDYNAMEpath.cgi in:
../studyname/admin/
11. If you want to delete all the respondent data up on the web that is associated with a study, go to the
admin.html and choose Reset Survey.
SSI Web Perl test scripts
If you are experiencing difficulty in getting your questionnaire to run on the web, it may be helpful to
troubleshoot the problem by using very simple Perl scripts. You can find these scripts in your \User
Resources directory in your SSI Web installation. Once these 2 test scripts run successfully, your SSI
Web survey will also have a very good chance of running successfully.
Upload the test script to your cgi-bin and test it by opening up a browser and going to http://
www.yoursite.com/cgi-bin/test1.pl.
TEST1.PL - simply tests Perl linkage. A debug test message is written ("Debug Test #1
Successful") if Perl is installed correctly and permissions are set correctly.
Failure here would mean that the Perl interpreter is not being called and executed. You can check for
the correct path to the interpreter on the first line of the script and that Perl has been installed on the
machine. You can also check the permissions on the script to make sure it has "executable"
permission. If you are still having problems, call your ISP and explain that you need help running a
simple "hello world" Perl script.
Test2.pl - Helps you determine where your current working directory for Perl is.
1.7.5 Making Changes after Fielding Begins

Ideally, you will not have to make changes to your survey once you have begun interviewing (see
warnings below). However, the reality is that you will often be asked to make changes. Most every
change to your project involves modifying the project locally on your hard drive and uploading updated
files to the Web server. If you use the automatic upload capability (Field | Web Server Management)
SSI Web will only upload the files that have been affected by your local changes.
Even though SSI Web makes it easy to upload changes, don't get lulled into thinking that any
change you make to your project after fielding has begun is OK. The warnings listed below still
apply, since it is possible for respondents currently taking the survey to be temporarily disrupted by your
accessing and modifying files on the server. Also, more importantly, some changes you can make to
your survey pose no problems for later analysis; but, some changes you might make can cause a great
deal of trouble later. Please see the section below entitled Common Survey Changes.
If you are using the manual upload method with your own FTP software, then you first prepare the project
files for web upload (Field | Prepare for Manual Upload). In that case, we recommend uploading just a
few selected files that have modified (but not typically the entire project).
Warnings
Making any change to the project files on your internet server will potentially disrupt respondents as they
are taking the survey. For a very brief period respondents might see the "survey paused" error message.
Once changes have been applied, respondents may continue.
Apply Changes
If you are manually uploading your study via FTP, most study changes will not be made until you
log into the Admin Module to click "Apply Changes." By doing this, you update the database (for
example, the list of variables to be saved). The database is not changed just by uploading files.
Common Survey Changes
1. Changing Text, Skip Patterns, Adding Questions: Sometimes, you need to make changes to
your questionnaire. You can use SSI Web to modify the survey locally on your hard drive. Then, click
Field | Web Server Management to make the change on the server. The automatic web upload feature
will transfer the necessary changes to your web server. Use the "Test Mode" functionality within the
Admin Module to ensure that the changes were actually implemented.
If using the manual method to upload files to the server, click Field | Prepare for Manual Upload to
prepare new files for uploading to the server. These files are saved to a folder on your hard drive within
your study folder called /Web Upload. These changes will modify the STUDYNAME_questionnaire.cgi
and the STUDYNAME_layout.cgi files located in the /Web Upload/admin/dbadmin/db_setup folder.
Upload these files to the db_setup folder on your web server, overwriting the existing files. Make sure to
log into the Admin Module on your web server and click "Apply Changes". Otherwise, no changes will
actually be made to your survey. Use the "Test Mode" functionality within the Admin Module to ensure
that the changes were actually implemented.
262 SSI Web v8
2. Changing a Question Name: This is problematic, as the new survey layout will not contain the
variable name for the early respondents who completed the questionnaire prior to the change. If you
must make this change, you can put the original question on a new page with an unconditional skip.
That way, you will be able to export the data from the original question name and the new question name
separately. To upload the changes, follow the same instructions as item 1 above.
3. Changing a Question Type: Sometimes, users may need to change the question type from Text to
Numeric. This poses a problem for data export and you may not be able to export the data prior to
making the change if you do this. Therefore, we suggest the same remedy as item 1 above.
4. Changing Respondent Passwords: To change respondent passwords, the STUDYNAME_pwds.cgi

located in the db_setup folder will need to be updated on the server. Follow similar instructions as item
1 above.
5. Changing Administrative Passwords (the passwords you use to access the Online Administrative
Module): Open your project using SSI Web, locally on your hard drive. Click Field | Web Server
Management dialog to access and modify your Administrative Access passwords. Click Field | Web
Server Management to make the change on the server.
If using the manual method to upload files to the server, click Field | Prepare for Manual Upload to
prepare new files for uploading to the server. These files are saved to a folder on your hard drive within
your study folder called /Web Upload. For this change, the only file that needs to be uploaded to the
server is the STUDYNAME_config.cgi file (within the /Admin directory). Copy the
STUDYNAME_config.cgi file from the /Admin folder on your hard drive to the /Admin directory within
your study's server installation (overwriting the existing file).
Testing Your Survey after the Change
The Admin Module includes a Test Survey capability that lets you enter a mirror image of your survey as
a respondent (but your test data are not mixed with the real data). You should do this to ensure that the
change you made has taken properly on the server installation and that you haven't somehow disabled
your study.
1.8 Data Generator - Generating Test Data

The Data Generator allows users to create synthetic data to test a survey prior to fielding. This can help
with testing skip patterns and quotas, for example.
The Data Generator automatically answers the questionnaire, producing random answers to the survey
questions (unless you override this behavior and specify that certain questions should be answered a
specific way). SSI Web can generate data locally or on a remote study at a designated URL. All skips,
randomization, quotas, terminate links, passwords, pass-in fields, and merged fields function identically
to how they function with real respondents. This means the Data Generator can be used for creating test
data quickly, for testing study logic or serving as a study tester.
If you want to generate test data remotely for a survey already posted to a remote server, click Field |
Generate Data.... The Generate Data dialog is shown:
If you are testing a study posted to a remote server, Study URL should direct the Data Generator to the
remote study URL's login.html page. Study bypass URLs are also acceptable (see help).
If Test Locally | Generate Data... is clicked, Study URL is not visible. This lets you generate data locally
on your machine.
264 SSI Web v8
Start at Question sets the first question SSI Web will use to begin generating data.
Starting Respondent Number makes any generated respondent data begin at this number. If
respondents already exist with those numbers, they are overwritten.
Remove All Sk ip Logic ignores all skips for this session.
Remove Randomization ignores page randomization, randomized blocks, and questions that exist in
the survey for this session.
Use Browser shows a user how SSI Web is generating data instead of simple progress messages.
A web browser is opened and the user can see questions being automatically answered. Generating
data without Use Browser checked will run as many simultaneous respondents at a time as it can.
It will not use any custom JavaScript for verification or updating hidden values. Use Browser will
execute custom JavaScript. If Use Browser is checked, Show Question Names and Show Variable
Names becomes enabled. These options display additional help text in the questionnaire while SSI
Web is generating data.
Clicking Generate causes SSI Web to verify the study, make the local test folder, and generate data on
the machine. After the study is verified and the local test folder is made, the Generating Data dialog is
shown:
When data generation is finished, the Generate Data dialog is shown:

Clicking View Data in Admin Module opens a browser and logs the user into the Admin Module.
Clicking Get Data downloads the generated data into SSI Web. If any problems occurred during data
generation (like bad list building logic, bad skip logic, no available passwords), then the button View
Errors is available.
This can help to identify where and why errors occurred.
Advanced Settings:
From the Generate Data dialog, when the Advanced button is clicked, the Generate Data Advanced
Settings dialog is displayed:
266 SSI Web v8
Answer rate for questions that are not required: This tells SSI Web how often to ignore questions
that are not Required (such as a numeric or select question where respondents are allowed to
proceed without providing a response). Required questions (including Conjoint and MaxDiff) are
always answered.
Completion Rate: This indicates the percent of generated respondents that will be expected to
complete the entire survey. A proportion of the generated respondents will drop out at randomly
selected questions when set to a value less than 100.
Simultaneous Respondents: If Use Browser is unchecked, this setting controls how many
respondent records are generated at the same time. The maximum value is equal to the number of
available cores on your machine. SSI Web ignores this value when using the browser. (This setting
is automatically adjusted to the number of cores if you try to set it to a value higher than the number
of cores on your machine.)
Browser page load wait: This setting is to help the SSI Web Data Generator Use Browser work
properly on your computer. Faster machines allow this setting to be Shorter. Slower machines may
require this setting to be Longer. If you find the SSI Web Data Generator browser encountering page
load errors when Use Browser is checked, adjust the setting to be Longer.
Defined Values:
The Defined Values tab provides a way for users to supply specific answers instead of allowing the
program to answer randomly. Clicking Defined Values shows the tab:
All questions in the study are listed here except for page breaks, start question, terminate/link, quotas,
and text/filler. Questions in loops are organized according to the loop suffix. Text at the bottom of the
dialog box provides help on acceptable input for a given question. For example, in the screen above, S1
is a Select Radio button question with four acceptable values and is a required question. In the field
column, you may specify any value that will result in an answer of 1, 2, 3 or 4. Defined values may be
used to force data generation to take specific paths. If a study contains restrictive screener questions, it
may be virtually impossible for randomly generated answers to avoid screening out correctly. Therefore,
defined values help the Data Generator to make it through the screener question section.
Defined values may contain any of the following for producing acceptable answers:
A fixed value – an answer that will be the same for each generated respondent record. In the
example above, this could be a 3.
Numeric range – a random answer that is constrained between two numbers. Numeric ranges
are indicated by two numbers separated by a dash, like 1-3. Decimal numbers are allowed. The
number of decimal places will be determined by the largest number of trailing digits after a
period. (1.24-3.1415 will use 4 decimal places of precision.)
Scripting functions – Integer, Decimal, Mask, Regex, Stop, Ignore, Value. The syntax for these
functions is as follows:
- Integer(num1, num2) – Create a random whole number between num1 and num2 inclusively.
Num1 may be bigger or smaller than num2.
268 SSI Web v8
- Decimal(num1, num2, decimals) – Create a random fractional number between num1 and
num2 inclusively with the specified number of decimal places. “decimals” must be a number
greater than or equal to 0, less than or equal to 16.
- Mask(pattern) – Create a random input based on a specified mask. The result is formatted
by the pattern that specifies the valid characters that can be contained in a particular
location in the pattern. The following characters have special significance, all other
characters are considered normally
Characters Description
# Any Number
\ Escape character, used to escape any of the special
formatting characters
U Any upper case letter
L Any lower case letter
A Any letter
? Any letter or number
* Anything
Example: Mask(#####) will generate a 5 digit number like a zip code. Mask(###-##-####)
will generate a 9 digit number in a Social Security number format.
- Regex(pattern) - Create a random input that would match the specified regular expression.
See http://www.regular-expressions.info for information on regular expressions.
Characters Description
. Any character
\d Any digit 0-9
\s White space (spaces, tabs, etc.)
\w Alphanumeric and _
* Repeat preceding character 10 times
{n,m} Repeat preceding character minimum n times, maximum
m times
Examples:
o Regex("\d*") - Generates a 10 digit number.

o Regex(".{5,10}") - Generate 5 to 10 random characters.
o Regex("\w*@\w*\.com") - Generates random email addresses.
o Regex("\d\d\d-\d\d-\d\d\d\d") - Generates random numbers that are formatted like Social
Security numbers.
- Stop() – Data Generator will halt when this question is reached. When Use Browser is
checked, data generation will pause and the process can be resumed by clicking the
button. If Use Browser is not checked, the data respondent will drop out of the study and a
new respondent will be started.
- Ignore() - Data Generator will not answer this field unless it is required. If the question is
required, a random answer is generated.
- Value(variablename) – Copy the data generated answer from another field. For example, if a
study has email address and confirm email address fields or password and confirm
password fields, Value can duplicate the random value given for password in the confirm
password field.
Unverified Perl – the scripting functions used by the Data Generator have been implemented
using the Perl Programming Language. The Perl code is evaluated locally in SSI Web, not the
server, and the result is returned to the Data Generator. See the Unverified Perl documentation.
The same scripting functions as described earlier can be called but must be all capitalized, and
all string arguments must be surrounded by single or double quotes. SSI Script functions used
in authoring surveys will not work in this area. Sometimes in Unverified Perl, it might be nice to
have information about the current question like the current displayed design for a MaxDiff
question, attribute and level information for ACBC BYO. To retrieve this information, the Perl
variable $DB_QUESTION_INFO is available and is accessed just like a Perl Hash reference
object.
Defined values can also contain multiple and different answer methods separated by commas. For
example, three different fixed answers and a numeric range can be supplied for given question and one of
them will be randomly selected.
Use Browser:
If Use Browser is checked when Generate is clicked, the Data Generator Browser dialog is shown:
270 SSI Web v8
The speed that the questions are answered can be changed by moving the slider handle of the left track
bar from Slow to Fast. The Data Generator can be paused by clicking the button and resumed by
clicking the button. The buttons and are enabled when data generation is paused.
answers one item on the current page (like a checkbox, radio button, combo box, or text input). If
the page has been entirely answered, will submit the page. answers all questions and
submits the current page. It then pauses on the next page. Users can pause the process, answer
some questions manually, and click to resume. The Data Generator will begin answering the
current page.
1.9 How Respondents Access the Web Survey

1.9.1 How Respondents Access the Web Survey
Once you've uploaded your survey to your Internet server and verified that it is working properly, you need
to decide how to invite respondents to complete it. The most common way to invite respondents is via
email, with an embedded hyperlink. Other methods include linking from another survey or website, or
directly typing an address within the browser.
Respondents can open the Start page by browsing to (or clicking a link to): http://
www.yourwebserver.com/study1/login.html
where www.yourwebserver.com is the URL for your server

study1 is the folder containing your project (your "Remote Path")
mystudy is the name of your study
Getting to your survey:
1. Email URL links. Most every Email package is "Web-enabled," meaning that it recognizes URL
addresses such as http://www.sawtoothsoftware.com as Internet sites. Whenever the Email package
encounters an Internet address in the course of a message, it changes the text string to a hyperlink
(underlines the text, changes its color, and makes it a clickable area). When Email recipients click
hyperlinks, their computer automatically connects to the Web and their browser points them to your
survey site referenced in the hyperlink. You can also include the password information (or other "pass
through information") in the hyperlink for “one-click” access to the survey with password verification.
2. Link from Website or Other Survey. Respondents browsing a site on the Internet or taking
another survey click a submit button or hyperlink (either text-based or clickable graphic) that connects
them to your survey. You can also include the password information or other "pass through
information" in the hyperlink. This provides “one-click” access to the survey with password
verification.
3. Typed URL. Respondents connect to the Internet and type the URL into their browser (or cut and
paste it from an Email message) to access your survey (the login.html page).
272 SSI Web v8
1.9.2 "One-Click" Respondent Access to Survey

Your survey is just "one click" away for respondents if you email them a hyperlink. The hyperlink
includes the address for your survey and optionally "passes along" information about passwords or other
identifying (passed-in) information. Most email programs automatically interpret http:// as a hyperlink.
We suggest you use one of the many bulk* email packages available on the market. These permit you
to automatically process hundreds or thousands of email messages at a time. When the respondent
receives the email, they do not have access to the email addresses of other respondents; they only have
the return path email address to your account that sent the message. An automatic bulk email
package can also import customized text strings from a database into the email message. Those text
strings might include the respondent's user name and/or password generated by SSI Web's Password
Program/Generator.
(*Check applicable laws and regulations dealing with bulk email.)
Simple URL Hyperlink
If you type the following into an email message, it becomes a clickable "hyperlink" (underlined clickable
text string, that sends respondents automatically to a web page):
http://www.sawtoothsoftware.com/gonzo/login.html
Note that the "study root directory" on the server is called "gonzo" and the link sends the respondent to
the opening login (password) page, called "login.html". However, the respondent is required to see the
login page (page 1) and type a password to continue the survey.
URL Hyperlink with Passwords
If you want respondents to simply click a link that automatically logs them into a survey and supplies
their passwords and/or additional "passed-in" data (that you've defined in the passwords table), you
should create a URL that links respondents directly to the Perl script located in the cgi-bin directory.
When respondents click such a link to start the survey, they skip the Start page (page 1). The following
text included in an email message starts a survey with a studyname "tep1" located in the "gonzo" study
root directory, with password fields named myusername and mypassword, with valid values of
"michael" and "A6Z99":
You can also pass additional variables into the data set, as long as those are defined as Pass-in Fields
within the Questionnaire Access and Passwords dialog. For example, if we define a Pass-in Field
called panelid, we can specify this directly following any password information, as below:
studyname=tep1&myusername=michael&mypassword=A6Z99&panelid=7362836
When respondents start the survey by clicking the hyperlink, if the passwords are valid (and the quota is
not complete), the respondent is sent to the page following the password screen. If the respondent is
restarting an interview, the page where he/she left off is displayed. Any passed in information is stored in
the database, as if it were any other question in the survey.
sys_skipto
If you want to restart an SSI Web questionnaire and skip to ("link to") a specific question name, you can
do so by including a "sys_skipto" variable within the URL hyperlink (note: only works for restarts). For
example:
studyname=tep1&panelid=7362836&sys_skipto=Q11
The hyperlink above restarts the tep1 survey and skips respondents directly to Q11.
The most common use for this functionality is for linking multiple surveys (such as multiple conjoint
modules) while still maintaining quota control. For example, consider the following study design:
Study1 (CiW-type survey)

Study2 (CBC survey #1)
Study3 (CBC survey #2)
All respondents start in Study1 and complete Q1-Q10. After Q10, respondents receive either Study2 or
Study3, depending on answers within Study1. After completing the CBC section, respondents return to
Study1 to complete the survey (Q11-Q20) and be counted toward the quotas that are maintained in
Study1.
To do this, a skip pattern is inserted within Study1 that skips respondents from Q10 to one of two
Terminate/Link questions placed in Study1. The Terminate/Link questions mark respondents as
incomplete and link them to either Study2 or Study3. Then, at the end of Study2 and Study3, Terminate/
Link questions link respondents back to Study1, with sys_skipto=Q11 placed within the URL.
Respondents finish Q11-Q20 and reach a Terminate/Link question in Study1 that marks respondents as
complete.
Note: if restarting a survey and sk ipping forward within the questionnaire, mak e sure not to set
passthrough variables to new values. If a passthrough variable has already been stored and is later
resubmitted as a different value, the original value is maintained.
Important Note:
If the text string for the URL and passwords becomes very long, your email package may automatically
"wrap" the text in the middle of the hyperlink by inserting a hard return. This disables the hyperlink and it
will no longer work. Some email packages by default wrap after 80 characters. Some email packages
let you control how many characters are displayed before automatic wrapping occurs. For example, if
using Microsoft's Outlook Express, select Tools | Options | Send and under the Mail Sending Format
click the Settings... button. There is a field to specify "Automatically wrap text at ___? characters,
when sending".
Make sure to test that the hyperlink works properly by sending test messages to yourself and other
individuals (preferably outside your company's own internal email system) prior to emailing invitations to
respondents.
Sending email in HTML format will decrease the chance of links wrapping incorrectly and becoming
broken.
274 SSI Web v8
1.9.3 Linking to Sawtooth Software's Web Survey from Another

Survey Program
Some users may find the need to link to Sawtooth Software's web survey software from another HTML
form or Web-based survey. One of the most critical aspects is that you pass through an identifier that
can be saved as the user name or password with the Sawtooth Software data set. That way,
respondents from the two separate data sets can be matched during data processing.
Note: Linking multiple SSI Web surveys that result in more CiW questions than your license
conveys is a violation of your license agreement.
URL Hyperlink to Pass Respondents Directly in SSI Web Survey
If you want to automatically pass respondents from another Web survey page to an SSI Web survey and
automatically "pass through" the user name and/or password, you should link respondents directly to the
SSI Web Perl script located in the cgi-bin directory. The following text included in an email message
starts a survey with a studyname "tep1" located in the "gonzo" study root directory, with a valid
username of "michael" and a password of "A6Z99":
In this next example, only a username is used:
http://www.sawtoothsoftware.com/gonzo/cgi-bin/ciwweb.pl?studyname=tep1&myusername=michael
You can also pass additional variables into the data set, as long as those are defined as Pass-in Fields
in the Questionnaire Access and Passwords dialog:
studyname=tep1&mypassword=A6Z99&Q1=1&Q2=5
In this example tep1 is the study name and there is just one password named mypassword, with A6Z99
as a valid value for mypassword. Q1 and Q2 are additional variables to be passed into the data set.
These variables were previously defined in the Pass-in Fields tab of the Questionnaire Access and
Passwords dialog.
When respondents start the survey by clicking the hyperlink, if the passwords are valid (and the quota is
not complete), the respondent is sent to the page following the Start page (page 1). If the respondent is
restarting an interview, the page where he/she left off is displayed.
Warning: If you are "passing in" a variable to an ACA, CVA, or CBC study that you plan to use as
a case identifier for later merging the conjoint data with other survey variables, you should only
use numeric values. SSI Web will not permit the creation of conjoint data files for summarizing
or computing utilities that use non-numeric case identifiers.
Form Submit Button
You can create a "submit button" that sends respondents to the address of a Sawtooth Software survey
and "passes through" a variable to be saved as the user name and/or password. Note that your survey
program must be able to customize the HTML text on the fly with respect to the values of the variables
passed through.
Here's some sample HTML:
<HTML>
<BODY>
Click the button below to continue survey.
<FORM METHOD="GET" ACTION = "http://www.website.com/survey/cgi-bin/ciwweb.pl">
<INPUT TYPE="HIDDEN" NAME="myusername" value="michael">
<INPUT TYPE="HIDDEN" NAME="mypassword" value="A6Z99">
<INPUT TYPE="HIDDEN" NAME="studyname" value="tep1">
<INPUT TYPE="SUBMIT" VALUE="Continue">
</FORM>
</BODY>
</HTML>
Linking from SSI Web to Another System
You can link from SSI Web to another system by specifying the Terminate/Link question type. You can
dynamically insert respondent identifiers (such as user name and password) or even responses to
questions within the SSI Web survey within the HTML of that hyperlink.
For more information about dynamically inserting user name, password, or answers to previous
questions within the hyperlink, please see the section on Displaying Responses to Previous Questions.
276 SSI Web v8
1.10 CAPI Interviewing with "SSI Web CAPI"

1.10.1 CAPI Interviewing with "SSI Web CAPI"
SSI Web generally is used for interviewing over the Internet. But, SSI Web studies can also be used for
interviewing in "CAPI" mode (Computer Aided Personal Interview), which generally refers to data
collection using a laptop or a PC not connected to the internet. We recommend Windows 2000 or later
for the PC or laptop used for collecting data.
You do not need to buy an additional module or software package other than the general SSI Web
system to conduct CAPI interviewing (the CAPI module can be downloaded at no charge from our web
site). The main steps are as follows:
1. Develop your study as you would for Web interviewing, including setting up passwords, etc.
2. Test the study to ensure it works properly, including checking skip patterns, examining the
resulting data, conducting a pre-test, etc.
3. Use Field | Create CAPI Packages to create a "CAPI package," which is a single file (.CAPI
extension) that contains all the information necessary to run the study. If multiple PCs (not
networked) are used in interviewing, you may want to create multiple .CAPI files, one for each
interviewing PC, for better field data management.
4. Install the "SSI Web CAPI" software on laptops or PCs used for interviewing. This is freely
downloaded software from our website. Or, you can burn this installation program onto a CD-
ROM to deliver to the data collection facility. (The CAPI installation program is also included in
the "CAPI" subdirectory of the SSI Web program.)
5. Email or otherwise deliver the .CAPI file(s) to the data collection facility. They load the study
into SSI Web CAPI and begin interviewing. They can give you periodic reports regarding the
number of complete, incomplete and disqualified records.
6. The data collection facility sends you a data file for each .CAPI file that was loaded and used
for interviewing, which represents all data records collected to that point.
Create CAPI Package Files Dialog
Each .CAPI file you create contains all the files and information needed to run the study (the
questionnaire, passwords, graphics, Perl scripts, etc.) By including the Perl scripts in each .CAPI file,
this ensures that each project is fully self-contained and the data collection site can manage multiple
studies without regard to which version of SSI Web was used to create the studies.
Beginning CAPI Package Index Number

Creating multiple .CAPI packages and indexing them is a useful aid for field data
management, however, there isn't any requirement that you create multiple packages. In
reality, multiple .CAPI files from the same study are identical in every way (assuming you
didn't change anything else about your study between .CAPI file creation sessions, such as to
add new passwords or modify the questionnaire). Select a beginning number for the .CAPI
package(s) to be created in the current session. If you select "1" as the starting point (with
multiple packages to create), the .CAPI packages are named starting with
STUDYNAME_1.CAPI. The next .CAPI packages in the series follow as
STUDYNAME_2.CAPI, STUDYNAME_3.CAPI, etc.
You can create .CAPI packages in multiple sessions. For example, you may discover after a
few days of interviewing that you want to create a few additional .CAPI packages. If you earlier
had created .CAPI packages 1-10, you might now create .CAPI packages 11-20. (If for some
reason you wanted to start naming the files in the second session starting with #83, it wouldn't
matter. Again, the numbers attached to files are merely to help you organize and manage
files coming from potentially multiple sources in the field.)
Some SSI Web users that are familiar with our other interviewing systems (such as Ci3 or
SMRT) may wonder how the beginning respondent numbers are specified for each .CAPI file,
or how many respondents are allowed per .CAPI installation. Internally, we use a random
seed based on the system clock to assign each record an internal respondent number (this is
used as a seed for question/item randomizations and assignment to CBC designs). If you
interview many respondents, it is possible to get duplicates (there are 475,254 possible unique
internal respondent IDs). However, we generally recommend that you use a different variable
(such as User Name or Passwords) as respondent IDs, or that you simply export the data with
the cases numbered sequentially from 1 through n. We don't suggest you use SSI Web's
internal respondent number in analysis. Each .CAPI installation can collect up to 475,254
possible respondents.
Number of CAPI Package Files to Create

Select the number of .CAPI files to create within the current session. If the PCs or laptops
used for interviewing operate independently, then you may want to create one file per
interviewing PC, for ease of field data management. (If you and your field agency would not be
bothered by different batches of respondents having the same file name, then you are welcome
to create a single .CAPI package and distribute this same file to all data collection sites and
PCs.)
The name of the .CAPI files follows directly from your study name. If the study is named
ABCD and you create 10 CAPI packages, starting with an index number of 1, the files are
named ABCD_1.CAPI, ABCD_2.CAPI . . . ABCD_10.CAPI.
Warning: Only one .CAPI file with the same study name should be used by each
computer. If (using the SSI Web CAPI program) the interviewer loads a second .CAPI
file (say XXXX_2.CAPI) onto a PC that has already been interviewing using a
XXXX_1.CAPI installation, all the data files for the XXXX_1.CAPI installation will be
overwritten (though we always store vital data in a sub-directory of the /Archive folder
within a SSI Web CAPI installation prior to deleting any data). We provide a warning
dialog within the CAPI module prior to any step that would delete the data.
Directory to Create CAPI Package Files:

This indicates the folder in which to save the .CAPI files and is located inside your study
folder. You cannot change this folder.
Include Administrative Module

If you want the individual(s) conducting data collection to have access to SSI Web's Online
Administrative Module, check this. We generally suggest not including the module, unless
the individual(s) performing data collection have a good understanding of the functionality of the
Administrative Module.
Even without including the Administrative Module, the SSI Web CAPI program lets the data
collection facility accumulate data for each .CAPI file into a single data file and gives them a
tally of the number of completes, incompletes, and disqualified records in each data file.
278 SSI Web v8
Interview "Recycle"
When running in CAPI mode, you may want the questionnaire to automatically recycle from the last
page of the survey back to the first, so that it doesn't drop to the main SSI Web CAPI menu each time.
The easiest way to do this is to add a Terminate/Link question at the end of your survey (on its own
page), that (on the Settings tab) specifies a Web Address (URL) of "../login.html".
1.10.2 Accumulating CAPI Files

To accumulate CAPI data files, you should use the standard Data Management area, File | Data
Management | Get Data, then select Local data and select all the .csv files.
280 SSI Web v8
1.10.3 Deleting Sensitive Data from CAPI Interviewing PCs

We have designed SSI Web CAPI to make it very difficult to lose important data. Prior to any step in
which data might be deleted, we warn the interviewer that important data are about to be deleted. Even if
the interviewer chooses to perform an operation within the SSI Web CAPI interface that deletes data, we
still save a copy of vital data files within a sub-directory of the /Archive folder within the SSI Web CAPI
installation. This keeps interviewers from accidentally deleting critical data. However, this also makes it
more difficult to clean sensitive data completely from a PC at the end of a project (e.g. prior to returning
rental PCs).
Even if you uninstall the SSI Web CAPI program, data files created during project administration are still
maintained on the hard drive.
To completely delete study data from an interviewing PC (running under SSI Web CAPI):
1. From the SSI Web CAPI menu, highlight the study to be deleted on the list of Available Studies.
Click Delete Study button. This deletes the data for the project from the project folder. However, for
safety precautions, we place critical data files in a sub-folder of the /Archive directory as a result of this
deletion step.
2. Browse to the ..\Documents and Settings\YourUserName\Application Data\Sawtooth Software,

Inc\SSI Web CAPI Module\Archive directory. You may find sub-folders within that directory with the
study name, plus a time-stamp reflecting the exact time and date the data were archived. Delete any of
these folders containing sensitive data.
3. Empty the Recycle Bin. (Even though you deleted the folders in Step 2, Windows retains these in
the Recycle Bin until you empty it.)
1.11 Managing Survey Data

1.11.1 Database
SSI Web saves all respondent data to a database. Both MySQL and Microsoft's SQL databases are
supported.
Your database administrator will need to set up a database that SSI Web can use. You will need a
database name and a database user name and password to access it. Take these settings and enter
them into SSI Web under Field | Web Server Management.
When uploading a study to the web server, SSI Web uploads various files and folders, one of which is the
"db_setup" folder (located inside of the admin folder). This folder contains three main files:
STUDYNAME_layout.cgi
STUDYNAME_questionnaire.cgi
STUDYNAME_pwds.cgi (only present if you have passwords associated with your study)
These files contain the information necessary to set up the database. The layout.cgi file contains
information about every data field in the study. The pwds.cgi file contains the passwords that need to be
loaded in the study. The questionnaire.cgi file contains all of the survey text and logic. After the
database is set up this file becomes the qst.cgi file in the admin folder.
The Auto Upload feature of SSI Web uploads these files and creates the database tables necessary to
run the survey. If you need to make changes Auto Upload applies the changes to the survey (including
changing the database if necessary) automatically.
If you are manually uploading a survey, after you have uploaded the contents of the Web Upload folder,
make sure to log into the Admin Module. The first time you log into the Admin Module the survey
database tables are created. Each time you upload changes make sure to click "Apply Changes" in
the Admin Module. No questionnaire or password changes are made until you click "Apply
Changes".
During the period that your survey is being updated there is a small chance that active respondents will
see the Survey Paused error message. This message by default says:
"The survey is currently being updated. Please try again in a few seconds."
They are able to click the Next button at the bottom of the page to continue where they left off.
Appropriate Perl modules must be installed on your web server to allow SSI Web's Perl scripts to
communicate with the database.
See http://www.sawtoothsoftware.com/server-setup for details.
Database Tables Created by SSI Web
Each study creates various tables in the database that you have specified in the Server Database
Settings. Each table name has the study name as a prefix.
STUDYNAME_info
This table contains miscellaneous information about the study.
282 SSI Web v8
STUDYNAME_data1
This table contains most of the respondent data. Each row in the table contains the data for an
individual respondent. The first field of this row is the Internal Respondent Number. The Respondent
Number is a unique number that identifies each respondent.
Since databases have limits on how many fields (or columns) each row can have, SSI Web might
store additional respondent data in additional data tables (data2, data3, etc). Rows in multiple data
tables are linked together by the Internal Respondent Number. MySQL can hold about 1,600
columns per table. Microsoft SQL can hold about 1,000.
STUDYNAME_map
The map table provides a way to look up which data field is in which data table. Each row in the
map table contains a data table number and all of the data fields that can be found in that data
table.
STUDYNAME_history
The history table stores the respondent's history, enabling the respondent to back up in the survey.
STUDYNAME_clists
Constructed lists are stored here.
STUDYNAME_passwords
If you have defined passwords in your study this table is created. Contains all of the respondent
passwords defined in the current study.
STUDYNAME_quotas
This table is created and used if you have quotas defined in your study. The quotas table keeps
track of quota status including all of current quota cell limit counts.
STUDYNAME_saved_reports
This table is created and used if you have saved any data reports in the Admin Module. Contains
saved report settings.
STUDYNAME_admin_log
This table keeps track of Admin Module activity. Actions such as login, edit data, and delete data
are recorded.
STUDYNAME_aca_avg_imp
This table is only created if using ACA with "Use Group Means for Prior Importances" selected.
Contains ACA Group Means information.
STUDYNAME_url_restore
From time to time the survey URL is used to recover a respondent's session and to figure out the
Internal Respondent Number etc. This table stores information necessary to restore a respondent
session if necessary.
STUDYNAME_design_log
Various types of surveys (i.e. ACBC surveys) require designs to be generated as respondents take
the survey. This table stores any messages that might be generated while creating the design.
Because each database table has the studyname as a prefix, a single database can store multiple SSI
Web studies. If you want to have two studies with the same name on your server, then you'll need to set
up a separate database for each study (to avoid conflicts).
NOTE:
If mid-survey you remove a question, or change its name, SSI Web renames the old question to have a
prefix of "_OLD" in front of it. The prior data collected under the old name are still in the database but will
not be available for download or export etc. The information could be extracted manually. A survey Reset
will remove all _OLD data columns.
Utilities such as phpMyAdmin (http://www.phpmyadmin.net) can be very useful to administer MySQL

databases. Tools such as these allow you to manually interact with the survey data.
284 SSI Web v8
1.11.2 Admin Module

SSI Web includes an Admin Module that makes it easy to manage your survey in the field, including:
Tracking completes and incompletes

Downloading data at any point during fielding
Viewing real-time tabulations of the data
Viewing/editing individual records
Testing your survey prior to fielding
Downloading lists of passwords that are incomplete/complete
Shutting down the survey
Resetting your survey
You can access the Admin Module from any computer connected to the Internet and it is protected by
passwords that you set when you develop your project in SSI Web.
Accessing Admin Module from Any Web Connection
Survey authors or clients should be pointed to the admin.html file, which is the login page for the Admin
Module. User name and passwords for the online administrative module are set in the Field | Web
Server Management menu.
Launching Admin Module from SSI Web
You can avoid having to manually browse to the Admin Module and log in by directly launching the
Admin Module using Field | Admin Module.... SSI Web automatically logs you in using your
administrative passwords that are stored in your project settings.
When you log into the Admin Module using either two methods above, the main dialog is shown:
Note: Using the Admin Module requires a browser with Javascript enabled.
Audit Trail
Advanced: There is a database table STUDYNAME_admin_log that keeps a record of activity in the
Admin Module: when users log in, if data were deleted, etc.
286 SSI Web v8
1.11.2.1 Setting Passwords for Administrative Access

You should assign both a User Name and Password to limit access to the Admin Module. Two levels of
access are available within the Admin Module: Read Only and Read-Modify.
You may want to give only Read Only access to your client or other individuals who need to monitor the
progress of the study, while retaining Read-Modify access for yourself.
To Specify Administrative User Name and Passwords
1. Click Field | Web Server Management. The Web Server Management dialog is
displayed.
2. Type the User Name and Passwords into the spaces provided.
Default random user names and passwords are provided for your convenience (new random names and
passwords are created for each new study), but you can change these if you like. Up to 12 letters or
numbers may be used for each field. The user name and passwords are case sensitive. For example, if
the User Name is AdminJudy, the administrator cannot type adminjudy or ADMINJUDY to access the
data online.
Read Only vs. Read/Write Administrative Rights
Read Only administrative rights let the user...
View the Study Summary

View Survey Data
Run Data Reports, including Cross-tabs
View the Passwords Reports
View the Incompletes Report
Test Survey
Read/Write administrative rights permit all the above operations, plus...
Download Data
Edit Survey Data
Delete Selected Records
Close a Survey
Reset a Survey
(There is an advanced capability for further customizing the menu items that are displayed for Read Only
or Read/Write rights, documented later in this chapter.)
1.11.2.2 Logging On
You or your client can access the Admin Module by directing your browser to the admin.html file located
in your study's root directory (you can rename the admin.html file if you'd like before uploading it to the
server).
If your study has many concurrent respondents, the more computationally intensive aspects of the
Admin Module (such as Data Reports) can place significant load on the server, which could delay
performance for respondents currently taking surveys. These operations are more safely done when you
expect relatively "light" traffic.
When you access the admin.html page with your browser, you are asked to supply a Name and
Password. When you initially created your study, random passwords were created for administrative
access. You may have kept those randomly-generated passwords or changed those passwords for your
needs under Field | Web Server Management. If you cannot remember your password, you can go to
Field | Web Server Management to review the passwords.
You must type the correct name and password to access the administration module. The User Name
and Password are case-sensitive. (If the User Name is AdminJudy, then adminjudy and ADMINJUDY
are not valid.)
Type the Name and Password and click the Login button.
Main Menu (Read/Modify Access)
After the system verifies a correct Name and Password (for Read/Modify access), the main menu is
displayed, with the following options:
By default, the number of complete, incomplete, and disqualified records are shown. The options listed
on the left are clickable links leading to additional menus.
288 SSI Web v8
Main Menu (Read Only Access)
After the system verifies a correct Name and Password (for Read Only access), the main menu is
displayed, with the following options:
The options listed on the left are clickable links leading to additional menus.
1.11.2.3 Downloading Survey Data

We recommend you download your data often while a survey is fielding (at low traffic times if you have a
particularly large study, as these operations can take up computing power and respondents taking the
survey could experience a decrease in performance). This provides an additional safeguard to avoid the
possibility of lost data.
Downloading data makes a copy of the data file in the destination directory you specify; it doesn't move/
delete the data from your server. In other words, whenever you download data, you are downloading a
copy of all records every time (a cumulative process). For example, if on day one, 100 respondents
complete the survey and you download the data, then on day two 75 more respondents complete the
survey and you again download the data, 175 records will be downloaded on day two.
Click the Download Data link from the main menu and the following is displayed:
When you choose Download Finished Surveys Only, your browser asks you to supply a destination drive
and directory. In most cases, you will probably specify the study directory on your hard drive. The
accumulated data file, named studyname_data.csv, is copied to your hard drive.
290 SSI Web v8
1.11.2.4 Viewing Survey Data

When you click View Data from the main menu, you are next given the opportunity to View All Data
Records or to Search for Data Records.
View All Data Records
If you choose to View All Data Records, the View Data dialog is displayed. Select the record(s) on the
left to view and click View Selected Records. To select multiple records, hold down the Shift or Ctrl
keys while clicking on record numbers.
You can also delete selected records (if you have logged on with Read/Modify access). If you delete a
record, you can indicate whether to allow the deleted respondent to restart or not.
Search for Data Records
If you choose to Search, the View Data - Search dialog is displayed:

This dialog allows you to search for specific records by different variables, or combinations of variables.
For example, you could search for all records where Q1=5 (Variable Name "Q1", Operator "=", Value
"5"). After the search is complete, the records that qualify on the search criteria are displayed in the
View Data dialog described above.
292 SSI Web v8
1.11.2.5 Tabulating Data On-Line

The Data Reports area lets you run frequency tabulations (including cross-tabulations and charts) of the
data in real-time by browsing to and logging into the Online Administrative Module. You can analyze
complete and/or incomplete records.
Rows (stubs) These are the questions you are tabulating.
Columns (banners) If you want the results for your Row variables to be split out by respondent
subgroups (as separate columns), specify the filtering variables here.
Filters Here you specify whether you want to tabulate results for All
Respondents, Qualified/Completes Only, Incomplete, or Disqualified.
Add additional filters You can specify additional filters, so results are tabulated only for a
subset of respondents.
Click Create Report to run the tabulation.
Tabulations for Numeric/Single Select Questions
Tabulation reports for Numeric and Single select questions are quite similar, since each question
contains a single response. Here is an example for the question age, for just nine respondents:
The Value column shows the values that were recorded in the data. The Label column displays the label
used for each response category (if applicable). Count shows how many respondents used each unique
value. Percent shows the percentage of respondents (who answered the question) that used each value.
Finally, the N (total number of respondents that answered the question), Min (minimum value used), Max
(maximum value used), and Mean (average of values used).
Note: You can download the data for any table in this report to a .csv file by clicking the Download link.
Data Reports for Multiple Select Questions
Consider the following question:

294 SSI Web v8
The Data Report for this question may look like:
With select-type questions, a Label column shows the text associated with each value for this question.
The Count column shows that a total of 5 + 3 + 4 + 2 = 14 responses were provided by the 6
respondents who received this question. The percents are percents of respondents (83% of respondents
said they use their laptops at home) rather than percents based on total responses given. As a result,
the percents for multi-select questions will not usually sum to 100%.
Please note that all respondents who saw this question and submitted the corresponding page are
counted in the base for multiple response questions, even if they did not check any of the boxes.
Saving Report Specifications
You can save report settings so that you can run them again at a later point. Please note that only the
settings are saved. Each time the saved report runs, it will be applied to any data currently in the study.
When you save a report, the next time you open the Data Reports area, you are shown a list of existing
reports to select from:
Notes:
Grid Questions: Multiple-response check-box questions within grids are available for
tabulation. The variable for a row-based question is listed using the convention grid_r1_c*
where the asterisk means that check boxes across all columns for the question in row1 will be
tabulated. The convention for column-based questions is grid_r*_c1 where the asterisk
means that check boxes across all rows for the question in column1 will be tabulated.
296 SSI Web v8
1.11.2.6 Downloading Password Report

You can download a list of passwords for respondents with different states of completion for the survey.
This can be helpful if you want to send reminder notices to respondents who have not yet completed the
survey, or if you want to identify those who have already completed the survey, etc.
While in the Admin Module, from the main menu, click Reports | Password. The following is displayed:
When you click any of these links, you are given a download link that saves a .csv file.
1.11.2.7 Incompletes Report

This is sometimes referred to as a Drop-outs Report, though for a live survey we don't know for sure if all
individuals reported have truly dropped out or are just incomplete and still in the process of completing
the survey. For that reason, we call it the Incompletes Report.
In the example below, 9 total respondents have started the survey (we learn this from the Study
Summary report). 5 respondents completed the survey and 4 are incompleted records. These
incomplete respondents could have dropped out, or perhaps they are still working on the survey.
When we click the Incompletes Report link, the following report is shown:
This shows us the last question that the 4 incompleted respondents saw. 2 respondents last saw the
Q1 question, 1 respondent last saw the sel1 question, and 1 respondent last saw the Q6 question.
298 SSI Web v8
1.11.2.8 Test Survey Mode

SSI Web includes the ability to test the survey using test survey mode. When in test survey mode, any
test data are kept separate from real respondent data. You can run test surveys even during data
collection and it will not interfere with actual respondents.
When in test survey mode, a bright orange banner indicating "TEST Mode" appears across the top of the
survey to confirm that you are not collecting real data:
When you start a survey in Test Mode, a helpful dialog appears:
Check any functions that you want to enable during the test survey, and click the Next button.
Notes: Quotas are not enforced in test mode. Also, you cannot restart incomplete test surveys. The
error log file is shared between regular and test modes.
Admin Module (Test Mode)
When you have collected data in test survey mode, you are also given the full capabilities of the Admin
Module again in test mode (with bright orange coloration in the dialogs so that you can distinguish
working with test data from working with real survey data).
When you click Test Survey from the main menu, a link is provided to Enter Test Mode. The following
is displayed:
When in test survey mode, any operations you do within the test mode Admin Module are confined to
the test data. The real data are unaffected. This includes all steps that delete data or reset the survey.
Note: resetting real data will not reset your test data.
300 SSI Web v8
1.11.2.9 Reset Web Survey

This option resets your survey data. It deletes all respondent data and resets password status to reflect
no completed surveys. Resetting your survey doesn't delete the questionnaire, just the respondent data.
This is a convenient way to delete practice data prior to taking a study live. (Note: Test Data generated
in Test Survey Mode is not deleted. You must select Reset Web Survey within Test Survey Mode.)
The following information is displayed when you click Reset Web Survey:
WARNING:
Clicking "Reset Survey" below will cause this web survey ranq to be reset. Resetting a survey includes:
1) Deleting ALL respondent data. This includes incomplete and complete data records from both the data
and admin directories. The internal respondent counter will also be reset.
2) Resetting all the password information. All passwords will be reset and ready to use again. Changes
made to Maximum Respondents in the On-line Data Management Module will remain.
Only reset a survey if you have been testing and want to clean out test data and restart or if you are
finished with your survey and no longer have use for the data.
Clicking Reset Survey will DELETE ALL OF YOUR DATA for this study!
Click the Reset Survey link to proceed.

1.11.2.10 Customizing the Administrative Module

(This is an advanced area.)
The Admin Module uses a default format and appearance, but you can modify the settings. You can
also change the default behavior regarding which menu items are available based on the level of
password (read-write vs. read only).
Customizing the Menu Items
Under the Field | Web Server Management tab dialog, you specify Administrative Module Access
passwords: Read-Only and Read-Modify administrative rights. These two permission levels control which
menu items are available within the Admin Module.
Read Only administrative rights let the user...
View the Study Summary

View Survey Data
Run Data Reports, including Cross-tabs
View the Passwords Reports
Test Survey
Read/Write administrative rights permit all the above operations, plus...
Download Data
Edit Data
Delete Selected Records
Close a Survey
Reset a Survey
You can change these default settings by creating a file named STUDYNAME_access.cgi and by
placing it in the admin folder in your server setup. If this file is present, SSI Web ignores the user name
and password for the Admin Module as specified in the Survey Settings dialog.
Using STUDYNAME_access.cgi allows you to specify multiple passwords with customized levels of
access to the Admin Module.
Here is an example demonstrating the STUDYNAME_access.cgi file format:
STUDYNAME_access.cgi
{
"admin_access" =>
{
"Bob|3829" => {"read_access" => 1},
"Joe|8743" => {"study_summary" => 1, "marginals" => 1},
},
}
In the example above, the user "Bob" with password "3829" is able to access all of the settings that
come with Read Access. "Joe" on the other hand with password "8743" will only see the Study
Summary and Data Reports menu options.
302 SSI Web v8
The available settings are:
full_access
read_access
study_summary
download_data
view_data
data_reports
password_reports
test_survey
close_survey
reset_survey
incompletes_report
Customizing the Look of the Admin Module
You can customize the look of the Admin Module by creating your own CSS file. In the Web Server
Management dialog, under the Server Directory Paths tab, enter the name of your custom CSS file.
You can place this CSS file in the graphics folder or you can refer to a remote file by listing the full URL
(i.e. http://...). If the CSS file is placed in the graphics folder then you will need to include the graphics
path. For example: [%GraphicsPath()%]your_style.css
By including a custom CSS file, you can remove all "Sawtooth Software" branding from the Admin
Module.
To remove our title and include your company's logo, include the following CSS:
#title_text
{
display: none;
}
#title
{
background:url('your_logo.gif') no-repeat center center;
height: 50px;
}
1.11.3 System Data Fields

SSI Web not only stores the responses people give to the surveys, but it also stores additional fields
(system variables).
System Variable Name Description

sys_SequentialRespNum Sequential number given to data as it is imported into SSI Web.
sys_RespNum Unique identifier assigned to each data record as it is created on the
server.
sys_DataSource How the data were generated:
1=Web data collection; 2=CAPI; 3=Paper-and-Pencil; 4=Local Test;
5=Admin Module Test Mode; 6=Data Generator.
sys_StartTime Time stamp created when the survey was created.
sys_EndTime Time stamp representing the last time survey data were submitted.
sys_RespStatus Respondent Status:
2=Incomplete; 4=Disqualified; 5=Qualified/Complete
sys_DispositionCode Optional custom code assigned to data record when the respondent
terminates.
sys_LastQuestion Last question seen by the respondent.
sys_UserJavaScript Whether or not the respondent had JavaScript enabled.
1=User had JavaScript enabled; 0=User had JavaScript disabled.
sys_OperatingSystem Operating System used by the respondent.
sys_Browser Web browser used by the respondent.
sys_IPAddress The IP Address that was used by the respondent to connect to the
server.
sys_pagetime_* Amount of seconds respondent spent on the particular page.
sys_ElapsedTime Total amount of seconds respondent spent in the survey (sys_EndTime
- sys_StartTime).
Please see additional information about turning off the automatic collection of certain personal
information,such as the respondent's browser, operating system, and IP address.
304 SSI Web v8
1.12 Accumulate/Export/Edit Data

1.12.1 Get Data
Click File | Data Management and the Data Management dialog is displayed. This dialog offers three
tabs:
Get Data
Export Data
View/Edit Data
Select the Get Data tab.
Before you can manage/export/edit data, you need to move any data from a web server or CAPI files into
a master data file stored locally on your desktop/laptop. That process is called Accumulating the data.
Data Source
Web Server If you are accumulating data from a web server, specify the URL path to the admin.html
file. If you are hosting with Sawtooth Software, you can obtain this URL through your hosting
management portal. This will download the data directly from the admin module and accumulate it
into SSI Web.
Import Study Data Files If you have a data file stored locally (or CAPI files) that you would like to
accumulate to your study, specify the path or browse for that file (or files).
Merge External Data Allows you to merge additional variables and data that were not collected in SSI
Web surveys into your data file. You are prompted to specify the path or browse to a .csv file. The
first row must contain labels. Respondent number must be named sys_RespNum. Variable names
must not begin with numbers or contain spaces.
Paper-and-Pencil Data MaxDiff and some conjoint methods (CBC, CVA) allow you to specify paper-
and-pencil data in a .csv file. If accumulating paper-and-pencil data, specify the path or browse for
that file (or files).
All accumulated data are stored within the studyname_data.sqlite database file.
1.12.2 Export Data

Before you can export data, you must have downloaded the data from the server (or from CAPI files) and
accumulated the data.
tabs:
Get Data
Export Data
View/Edit Data
Select the Export Data tab.
Within the Export Data tab, you create Jobs to export. Each export job may contain a single or multiple
files to export. For example, you may want to export your data to both a .csv and an SPSS data file.
As another example, you may need to export the data in a different manner for different clients. You can
create and save a different job for each client. The settings for each job are saved in your project, so you
can always go back to a job and repeat the export.
You can export the data from your Internet questionnaire to one of the following files:
Comma Delimited (.csv) (Supported by Microsoft Excel)

This file can be directly opened with Microsoft Excel(TM). It is a comma-delimited text file,
where the first line includes variable labels.
Export Labels File. If you are using Sawtooth Software's online analysis tool, you should
export a labels file. Note that this will slow down the export process.
Microsoft Excel 2002- 2003 (.xls)

We suggest this format when using Excel 2002 or 2003 (limited to no more than 256
columns), especially if you are exporting open-end responses that have been typed using
double-byte character sets. The file is saved with a .xls extension.
Microsoft Excel 2007 (.xml)

This file format can support up to 16,384 columns. We suggest this format when using Excel
2007 or later, especially if you are exporting open-end responses that have been typed using
double-byte character sets. The file is saved with an .xml extension.
SPSS (.sav)
This file can be directly opened with SPSS. It is a binary file.
Fixed-Width Text (.tab and .lay)

The .tab file is a fixed column text file. This file can be read by most any statistical
processing or cross-tab package. The layout for the .tab file is included in the .lay file, which
is also a text file. The .lay file specifies the variable names and in which column in the .tab file
each variable begins and ends.
A default name and path is provided in the Output Filename: field. You can directly edit the name. By
clicking the browse button, you can specify the folder in which you wish to save the file.
306 SSI Web v8
1.12.3 Filters, Segments, and Weights

When you export or analyze data in SSI Web, you often want to capture or otherwise identify subsets of
the respondents (via Filters and Segments). Sometimes (especially for CBC Counts, Logit, or Latent
Class) you may want to Weight respondents so that some respondents are counted more in the
analysis than others.
Filters
A Filter is a rule (logical expression) you specify that identifies a subset of respondents.
Click Analysis | Filters... to access and specify filters. Each Filter for your project is listed within the
Filters grid, one Filter (rule) per row. Filters may be directly typed into the grid, or using the +Add Filter
menu tool. The +Add Filter menu tool is easier to use than directly typing into the grid, since it assists
you in creating appropriate syntax and in remembering the variable names available to you within your
project.
A simple example of a filter is the logical expression Age > 34. When you create a filter, you give the
filter a name such as OlderAged.
Once you have established this filter, if you export the data and select the filter OlderAged, then only
respondents with Age > 34 are exported.
Filters may involve complex logic including AND, OR, and the symbols =, <> (not equal to), <, >, <=,
>=, Contains, Does Not Contain, Is Blank, Is Not Blank.
Contains and Does Not Contain are for use with open-ended text (strings). (Although it would be rare to
use open-ended text to define segments, we'll show an example here as an illustration.) For example, if
you want to identify any respondents who typed the word "economy" (without any spelling errors!) in an
open-ended variable named opinion, you could create (Using the Logic Builder) a filter named
SaidEconomy and specify the logic:
Notice that with the string logic Contains and Does Not Contain, the Value must be enclosed in double-
quotes. The Value is not case sensitive. Respondents who type economy, Economy, ECONOMY, or
EcONoMy will all evaluate to TRUE.
The Custom Logic builder may be used if, for example, complex expressions with parentheses need to
be specified.
Filters may also be used for identifying subsets of respondents for assignment into new Segments you
create.
Segments
When you create a new Segment, a new variable (field) is created within the database. You can see that
a new field is created by invoking File | Data Management... and clicking on the View/Edit Data tab.
The data for all records are shown and a new field (column variable) for your Segment is created.
Defining new Segment variables is useful when, for example, you need to assign respondents into
buckets (segments) based on a continuous variable. For example, imagine you had asked people to
type their age within a numeric question named Age. You might decide to re-classify respondents into
four groups and store that group information in a new Segment variable called NewAge:
Category Description
1 18-34
2 34-45
3 45-65
4 66+
To do this, you first need to create four Filters (rows) within the Filters table. The four filters each define
the four outcomes in the above table. For example, the first category is created using the Filter logic:
Age > 17 AND Age < 35.
After you have specified filters to identify respondents in each of the four groups, click Analysis |
Segments and Weights.... Click Add Segment, and name the segment NewAge. Then, select the
four filters you previously defined as the qualification logic for categories 1-4.
You can also use filters and segments to identify groups of people based on combinations of more than
one variable, such as a) young and male, b) young and female, c) old and male, d) old and female.
Weights
In some analysis procedures within SSI Web (such as CBC's Counts, Logit, and Latent Class), you may
want to assign weights to respondents so that some respondents count more than others within the
analysis.
Of course, any variable that contains a numeric value may be used as a weight. For example, you may
have asked a numeric-type question named FamilySize where respondents have typed the number of
members in their family. The variable FamilySize may be used as a weight during analysis.
Also, the Segments and Weights dialog may be used when you need to specify specific weights to
apply to respondents who qualify based on existing Filters.
Using an example from the previous section, imagine you had created four new filters (bucket definitions)
based on a combination of Gender and Age variables:
308 SSI Web v8
a) young and male

b) young and female
c) old and male
d) old and female
You could assign weights to each of these four categories by using the Segments and Weights dialog.
Click Add Weight and specify a name such as AgeGenderWT. For each row in the table, select the
Filter name (that you've previously defined) and specify a weight (the Value) to apply to respondents who
qualify on each filter:
Filter Value
YoungAndMale 1.5
YoungAndFemale 0.5
OldAndMale 1.4
OldAndFemale 0.8
1.12.4 Export Data Dialog
When you click File | Data Management and Add Job... the Export Job dialog is displayed.
Respondents to Include
You may filter which respondents to include in your export using a Filter Builder. The filter can be
based on existing variables, or advanced filters may be built based on complex logic involving a single
or multiple variables.
Respondent Numbering
Each record to export contains a Respondent Numbering field. This dialog lets you specify the field to
use as the Respondent Number. Note: when exporting conjoint data for analysis in SMRT, your
Respondent Number must be numeric and unique.
Export Settings
Export Name: Here you specify the name for the Export Job (a text label)
Type: Lets you specify what kind of file to export (.csv, Excel, SPSS, Tab/Lay, Loop, or conjoint/
MaxDiff exercise, if one is available in your study).
File Formats: You can specify multiple file formats to be exported simultaneously from the same
Export Job (e.g. .csv, Excel, SPSS, Tab/Lay)
Export Fields: Click this to specify which variables to include in this Export Job.
Output Filename: Here you specify the name of the file (the prefix only). The extension name is
automatically provided depending on the type(s) of data file(s) selected to export.
310 SSI Web v8
1.12.4.1 Multiple Response Question Format

From the Export Job dialog, you may click the File Formats button. The Raw Data Export File Formats
dialog is displayed.
This dialog lets you specify how multi-select responses should be formatted:
Checked/Unchecked (one per column)

Item Numbers (one per column)
Item Numbers (single column of chosen items)
Multiple-Response Questions
Multi-select questions are ones such as the following:
Strawberry Peach
Vanilla Neapolitan
When the data are exported, this question becomes six fields: Q3_1 through Q3_6. Consider a
respondent that checks Vanilla (item #3) and Peach (item #5). SSI Web can export the data formatted
in two ways (you should select which method makes most sense given your tabulation software or data
processing needs):
Item Numbers:
Q3_1 Q3_2 Q3_3 Q3_4 Q3_5 Q3_6

3 5
Checked / Unchecked:
Q3_1 Q3_2 Q3_3 Q3_4 Q3_5 Q3_6

0 0 1 0 1 0
Exporting Open-Ends
You can export openend ends to a text file by clicking File | Export Data | Open-Ends Only.... Or, you
can export open-end responses as a data field within a standard data export under File | Export Data |
All Data.... From the Data Formatting tab of the Export All Data dialog, check Export text from "Other /
Specify" Responses.
When you export an other/specify field, it is named questionname_item#_other. For example, if in

Q1, the seventh item was an other specify response, the variable name is Q1_7_other.
A Note on Column Widths
When you export data to SPSS or Fixed-Width Text using the .tab/.lay format, numerics and open-end
responses can take on different widths, depending on how long of an answer the respondents in a data
set have provided. By default, SSI Web selects a minimum width required to export the data for the
current respondents. But, if you later add respondents, your variable widths might change in a
subsequent export. If you want to fix the export widths and the resulting export table, so that new data
will not alter the variable widths, you should select Export columns using fixed widths. The drawback is
that the resulting data table is not as compact.
Exporting to SPSS
When you export data to SPSS, you can specify which portion(s) of your question should be used as
variable labels. You can also specify whether to strip HTML from value and variable labels.
The final option allows you to include Additional Question-Specific text in variable labels (grid header, list
labels, etc.).
SPSS Variable Labels cannot be more than 120 characters in length, so anything beyond this will be
truncated.
312 SSI Web v8
1.12.4.2 Selecting Data to Export

This dialog lets you specify the data you wish to export to the output file. All items in the Export List on
the right are exported when you click the Export button. You can move any items from the left window
(list of available data) to the Export list using the arrow buttons.
The > button moves just the highlighted items. The >> button moves all items. You can highlight a
range of items by holding down the Shift key and clicking with the mouse. You can highlight multiple
items selectively by holding down the Ctrl key and clicking each one with the mouse.
Sequential Respondent # are integers 1 through n assigned to each respondent record as it

was accumulated.
Internal Interview Numbers are those that the system assigns when respondents complete the
survey over the Web. There can be gaps in the sequence if some respondents started a
survey but didn't finish. The internal respondent numbers are alphanumeric and are converted
to numeric.
Start Date/Time is the time (as recorded on the server, often GMT-Greenwich Mean Time) that
respondents entered the survey.
End Date/Time is the time (as recorded on the server, often GMT-Greenwich Mean Time) that
respondents last submitted data.
Respondent Status (2 = incomplete; 4 = disqualified; 5 = complete)
Pagetime is the time that respondents took to answer each page.
Notes Regarding Exporting Multiple-Response Data:
With multiple-response select questions, more than one variable is stored for the question. For example:
Q1. Which of the following colors do you like?

(Select all that apply)
Red Green
Blue Violet
Yellow None of the above
The data for this question are saved as six separate variables: Q1_1, Q1_2, Q1_3, Q1_4, Q1_5 and
Q1_6. There are two ways that you can export the responses:
Checked/Unchecked: If respondents chose the item, the value is 1, if not, it is set to 0.

Item Numbers: Values corresponding to the response options are listed, for as many items as
were selected.
Grid questions can also save multiple responses. Grid data use the convention gridname_r#_c#, where
the r# refers to the row number and c# refers to the column number. For example, q11_r3_c5 stores
information for the grid named q11 and the response contained in row 3 column 5.
Some Notes Regarding Missing and Blank Data
There are different ways to achieve blanks in data positions and it is important to understand the
distinction when analyzing the data.
1) Respondent never saw the question. The data are blank for questions on the pages the respondent
never saw. Or, the respondent failed to click the submit button on a page with questions. Skip patterns
can create blank data.
2) Respondent saw the question, but did not respond. For each CiW question-type, you can specify
whether the question can be skipped by respondents. If you need to distinguish between whether none
of the categories of a multiple-response question truly apply for a respondent versus whether the
respondent simply skipped the question, you should probably include a "None Apply" response category
and then require a response for that question.
314 SSI Web v8
1.12.4.3 List Recode
During the export process, you can recode responses to questions involving lists. For example, you can
set the value of all "other specify" responses to "99" and all "None of the above" responses to "98". You
can also recode any other list items you wish, such as changing responses to a radio button scale
recorded in the data file as 1 to 11 to be the new range 0 to 10.
When you add recoding, you create a Recode Map name, which you can use across export jobs (so you
only have to create the recoding scheme once).
Global List Recode: This allows you to specify that all questions using a certain list should reflect the
specified recoding.
Question Specific Override: Allows you to override the Global List Recode for a given question or
questions.
Notes:
Only integer values (either negative or positive) may be specified for recodes. Only responses based on
lists may be recoded.
Recoding has no effect on the way the questionnaire operates: data are stored using their original list
item numbers, all logic uses original list items, etc.
1.12.5 Exporting Open-End Data

The Export All Data option creates a data file (STUDYNAME.tab) intended for reading by a statistical or
cross-tabulation package. If you want to export the open-ends in their entirety in a format that is
intended for use with word processors or coding programs, you should choose File | Export Data |
Open-ends Only. The Export Open-end Data dialog is displayed.
Respondent Numbering
Any datafield may be used for respondent numbering.
By default the open-ends are saved in the STUDYNAME.txt file, but you can change this name if you
want. The data are grouped by question. Within each question, responses are displayed by respondent
number.
Here is a sample open-end file for two respondents and three open-end responses:
Wh y _ l i k e
196 I hav e t r i ed ot her s of t dr i nk s , but I l i k e Sa wt o o t h Sp r i t z b e s t .
234 Do n ' t k n o w, I j u s t l i k e i t s z i ng.
Fi r s t _ Na me
196 J ohn
234 Ma r y
L a s t _ Na me
196 J ones
234 Bu r n s
Any carriage returns respondents type are converted to spaces in the STUDYNAME.txt file.
316 SSI Web v8
1.12.6 Viewing/Editing Data
tabs:
Get Data
Export Data
View/Edit Data
Select the View/Edit Data tab.
The data are displayed in a grid. Most fields are editable, but certain fields (such as
sys_SequentialRespNum, sys_RespNum and sys_ElapsedTime) may not be modified and are greyed
out.
Respondent records may be deleted using the Delete Row icon. Variables may be deleted using the
Delete Column icon.
Additional variables that were not collected may be merged to the data using the Add Merged Variable
button. When this is clicked, a dialog appears asking for the variable name and data type. The
variable name must be a valid SSI Web name. Regarding data types:
Categorical
Usually discrete integers starting with 1, such as 1=male, 2=female, 3=refused. Usually used for a
nominal scale, where the researcher specifies labels for each category. The size of the numbers
typically has little or no meaning for the analysis. During analysis, may be used for filters/segments/
covariates.
Continuous
Numeric values associated with quantities such as income, age, etc. where the size of the numbers
has meaning. Can contain decimal places of precision. During analysis, may be used for filters/
segments/covariates and weighting.
Text
The data may contain letters, numbers, and special characters (such as !, $, ^, etc.). During
analysis, may be used for filters/segments, but may not be used for weighting.
See additional information about System Variables.

1.13 Analyzing/Exporting Conjoint Data

1.13.1 Analyzing Data with Online Simulator
Sawtooth Software provides online market simulators for analyzing conjoint data and also MaxDiff. After
you compute your part-worth utilities (in the case of conjoint methods) or scores (in the case of MaxDiff)
you import the .HBU, .UTL, or .CSV files directly into the online simulator for analysis. The online
simulator is a service that you purchase on a monthly or quarterly basis, depending on the needs of your
project. Many researchers find the online simulator to provide a more attractive and simple interface for
clients (both internal and external).
Sawtooth Software's SMRT system provides a more powerful, but less attractive interface for conducting
market simulations.
318 SSI Web v8
1.13.2 Moving Data from SSI Web to SMRT for Analysis
If conducting conjoint analysis studies (CBC, ACBC, ACA and CVA) with SSI Web, you'll often want to
move the data from SSI Web into the SMRT software package for analysis (market simulations, counting
analysis, logit, etc.). Sometimes data are moved directly into SMRT, and other times data are moved
from SSI Web into a preliminary analysis program such as hierarchical Bayes and then afterward to
SMRT.
CBC (Choice-Based Conjoint) Data
When you download SSI Web data from the web server, your data reside in your SSI Web project folder
in a database format studyname_data.sqlite file. The steps for moving data into SMRT are:
1. Compute utilities using logit, latent class, or HB within the SSI Web system. When utility
estimation is run, a generic utility file with an extension .hbu is saved to a subfolder within your
SSI Web project.
2. Start the SMRT software (a different program from SSI Web, also downloadable from our
website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software
SMRT.
3. Within SMRT, create a new study for analyzing the results (select File | New and choose a
folder and a studyname). You can choose any folder or studyname that you want, as this new
study functions independently of your original SSI Web study.
4. Import the generic utility file into SMRT, by clicking Analysis | Run Manager | Import....
Select files of type as "Generic Conjoint File (*.hbu)" and browse to the subfolder of your SSI
Web project directory that contains the logit, latent class, or HB run you wish to import into
SMRT. Double-click the file. It is imported into SMRT and available as a utility run for
conducting market simulations.
Note: If you use standalone versions of CBC/HB or Latent Class software (rather than the built-in
applications within SSI Web) to estimate part-worth utilities, use the .CHO (or .CHS, in the case of
constant sum responses to be analyzed with CBC/HB) and .ATT files with those systems (see the CBC/
HB or Latent Class manuals for details) to produce a data file containing respondent part-worths (a .HBU
file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting utility file
into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import…
and following the prompts to browse to and select your .HBU or Latent Class probability file (.Pxx).
Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.
ACBC (Adaptive Choice-Based Conjoint) Data
1. Estimate part-worth utilities within SSI Web by clicking Analysis | Calculate ACBC
Utilities Using HB.... This process creates a text-only format file called STUDYNAME.hbu.
SMRT.
4. Import the STUDYNAME.hbu file (file of respondent part-worth utilities) into your SMRT
study by clicking (from the SMRT software menu) Analysis | Run Manager | Import… and
following the prompts for importing CBC utilities (selecting .HBU file type). Once you import
your utilities, you can select Analysis | Market Simulator and begin simulations.
Note: If you use CBC/HB or Latent Class software to estimate part-worth utilities, use the .CHO (or
.CHS, in the case of constant sum responses to be analyzed with CBC/HB) and .ATT files with those
systems (see the CBC/HB or Latent Class manuals for details) to produce a data file containing
respondent part-worths (a .HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis).
Then, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu)
Analysis | Run Manager | Import… and following the prompts to browse to and select your .HBU or
Latent Class probability file (.Pxx). Once you import your utilities, you can select Analysis | Market
Simulator and begin simulations.
ACA (Adaptive Conjoint Analysis) Data
1. If using the default OLS (Ordinary Least Squares) part-worth utility estimation method, from
the SSI Web menu, click Analysis | Calculate ACA Utilities using Ordinary Least
Squares…. This process creates a text-only format file called STUDYNAME.utl.
2. Start the SMRT software (a different program from SSI Web, downloadable from our
SMRT.
4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT
study by clicking (from the SMRT software menu) Analysis | Run Manager | Import…,
following the prompts for importing ACA utilities (selecting ACA v4.x Utilities *.utl as the file
type, and browsing to your .utl file). Once you import your utilities, you can select Analysis |
Market Simulator and begin simulations.
Note: If you use ACA/HB software to estimate part-worth utilities, you can either run ACA/HB as a
separate program, or if you own ACA/HB v3 or later, you can run the program as an integrated
component from within the SSI Web interface. If running the estimation outside of SSI Web, from the
SSI Web menu, select File | Export Data | Prepare ACA Data Files (*.acd and *.aca)... to export the
ACA information to a STUDYNAME.acd format into the same folder containing your ACA/HB system.
See the ACA/HB manual for details of producing a data file containing respondent part-worths (a .HBU
file).
After estimating part-worth utilities using either ACA/HB approach (within SSI Web menu or outside
320 SSI Web v8
program), import the resulting utility file into your SMRT study by clicking (from the SMRT software
menu) Analysis | Run Manager | Import… and following the prompts to browse to and select your .hbu
file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.
CVA (Conjoint Value Analysis) Data
1. If using the OLS (Ordinary Least Squares) part-worth utility estimation method, from the
SSI Web menu, click Analysis | Calculate CVA Utilities using Ordinary Least Squares….
If using the Monotone Regression part-worth utility estimation method, from the SSI Web
menu, click Analysis | Calculate CVA Utilities using Monotone Regression…. Either
process creates a text-only format file called STUDYNAME.utl.
2. Start the SMRT software (a different program from SSI Web, downloadable from our
SMRT.
4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT
study by clicking (from the SMRT software menu) Analysis | Run Manager | Import…,
following the prompts for importing CVA utilities (selecting CVA Traditional Conjoint Utilities
*.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can
select Analysis | Market Simulator and begin simulations.
Note: If you use CVA/HB software to estimate part-worth utilities, run the program as an integrated
component from within the SSI Web interface. After estimating part-worth utilities using the CVA/HB
approach, import the resulting utility file into your SMRT study by clicking (from the SMRT software
menu) Analysis | Run Manager | Import… and following the prompts to browse to and select your .hbu
file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.
Moving Segmentation (Generic Survey) Data from SSI Web to SMRT
We assume you have first followed the procedures above for estimating part-worth utilities and
establishing those as a utility run in a new project within SMRT.
1. From the SSI Web system, export the data you wish to merge into SMRT. From the SSI Web
menu, click File | Data Management then select the Export Data tab. Click Add Job... (provide a
name and select a respondent numbering option for your export job). Select the respondent
numbering method (the numbering method must match the method you chose when exporting the
conjoint data or estimating utilities) and export the data you wish to merge into SMRT (only
numeric data may be merged). Select either a .CSV file (comma-delimited text-only file) or a fixed
text-only file (.TAB).
2. Within the SMRT software, open the project previously established that contains the conjoint
data/utility run. Use File | Merge Variables | Add… to merge the data into your SMRT project
(see the SMRT manual or online help within SMRT for details). After merging the data, if using
SMRT v4.11 or earlier, you'll additionally need to specify Custom Segmentation Variables (Under
Analysis | Custom Segments) if you wish to use the new variables as banner points or respondent
filters during simulations.
322 SSI Web v8
1.14 Files created by SSI Web

1.14.1 Files created by SSI Web
Files created when creating surveys on the local PC
Name Description
STUDYNAME.ssi Main authoring file. Contains all questionnaire information.
STUDYNAME.ssi.bak Backup of .ssi file.
STUDYNAME_pwds.csv Passwords database, contains merged data.
Files created by CBC studies only

STUDYNAME_CBC1_<DesignID>.cgi Contains design information
Files created by CVA studies only

STUDYNAME_CVA1_<DesignID>.cgi Contains design information
Files created by studies with MaxDiff exercises

STUDYNAME_MXD<ExerciseID>_<DesignID>.cgi Design file for a particular MaxDiff exercise.
Files created in the Web Upload folder to upload to the web server
Name Description
STUDYNAME_questionnaire.cgi Main web questionnaire file before database is loaded.
STUDYNAME_path.cgi Contains server path information.
STUDYNAME_pwds.cgi Contains password information
STUDYNAME_config.cgi Contains database and admin module information
login.html Login page. Entry point for respondents.
admin.html Admin Module login page. Entry point for study administrators.
Files created by CBC studies only

STUDYNAME_CBC1_<DesignID>.cgi Contains design information
Files created by CVA studies only

STUDYNAME_CVA1_<DesignID>.cgi Contains design information
Files created by studies with MaxDiff exercises

STUDYNAME_MXD<ExerciseID>_<DesignID>.cgi Design file for a particular MaxDiff exercise.
Perl Files
ciwweb.pl
admin.pl
maxdifflibX_X_X.pl
ciwlibX_X_X.pl
authlibX_X_X.pl
pverlibX_X_X.pl
grdlibX_X_X.pl
enterlibX_X_X.pl
acalibX_X_X.pl
cbclibX_X_X.pl
acbclibX_X_X.pl
(where X_X_X is the version number of the software, e.g. 8_0_0)

Files created during data collection on the Web server (You should not delete these files manually
during fielding. If you click the Reset Survey option in the Online Administrative Module, these files and
folders are deleted.)
Name Description
STUDYNAME_qst.cgi Main web questionnaire file after database is loaded.
error_log.cgi Error log file reporting errors in the survey. Please
review contents and contact Sawtooth Software if
this file exists in your /admin directory.
STUDYNAME_ACBC_log.cgi ACBC log file created by ACBC studies. Contains
warnings and error messages.
Files located in the graphics/system and their corresponding image files

jquery-X.X.X.min.js Located in graphics/system. JavaScript library.
jquery-ui-X.X.X.min.js Located in graphics/system. JavaScript library.
ssi_admin_styleX_X_X.css CSS Stylesheet for Admin Module
ssi_admin_javascriptX_X_X.css Javascript library for Admin Module
ssi_styleX_X_X.css CXX Stylesheet for surveys
ssi_javascriptX_X_X.js Javascript library for surveys
(where X_X_X is the version number of the software, e.g. 8_0_0)
Files created during analysis of data on the local PC
Name Description
STUDYNAME.csv Exported data file (Comma-Separated Values)
STUDYNAME.lay Data layout file for .tab file
STUDYNAME.xml Used in the online simulator
STUDYNAME.sav Exported SPSS(TM) data file
STUDYNAME.tab Exported text-only data file
STUDYNAME.txt Text file of open-end responses (created by export)
STUDYNAME.xls Exported Excel data file
CBC Files
CBCEXERCISENAME.att CBC Attribute label file (created by export)
CBCEXERCISENAME.cho CBC choice data file, discrete choice only (created
by export)
CBCEXERCISENAME.chs CBC choice data file, allocation data only (created
by export)
ACBC Files
ACBCEXERCISENAME.acbccal Calibration concept design and responses, by
respondent
ACBCEXERCISENAME.alp Estimated population means for part-worths from
HB, by iteration
ACBCEXERCISENAME_Counts.csv Counts results by individual (created by Analysis
+ACBC Counts)
ACBCEXERCISENAME.cov Covariance matrix from HB
ACBCEXERCISENAME.csv Contains part-worth utilities computed via Analysis
+HB (or monotone)
ACBCEXERCISENAME.gamma Contains history of estimates of task-specific scale
factors from HB (Otter's method)
324 SSI Web v8
ACBCEXERCISENAME.hbu Contains part-worth utilities computed via Analysis

+HB (or monotone)
ACBCEXERCISENAME.log Contains summary report of fit statistics by
iterations for HB runs
ACBCEXERCISENAME.prc Prior covariance matrix used for HB runs
ACBCEXERCISENAME.restart Restart file for HB runs
ACBCEXERCISENAME.std Standard deviations by part-worth parameter by
respondent from HB
ACBCEXERCISENAME.sum Contains final estimate of population means and
covariances for HB runs
ACBCEXERCISENAME.val Values file describing any linear coding of utilities for
use in SMRT
MaxDiff Files
MAXDIFFEXERCISENAME.att MaxDiff attribute label file (created by export)
MAXDIFFEXERCISENAME.cho MaxDiff choice data file (created by export)
MAXDIFFEXERCISENAME.eff Effects file defining coding method and interaction
terms (created by export)
MAXDIFFEXERCISENAME.position Respondent answer positions within each task
(created by export)
MAXDIFFEXERCISENAME.mtrx Prior covariance matrix (created by export)
MAXDIFFEXERCISENAME.val Values file specifying values for linear or user defined
variables (created by export)
ACA Files
ACAEXERCISENAME.aca Contains attribute level text to accompany .utl for
ACA
ACAEXERCISENAME.acd ACA audit trail file (export to HB)
ACAEXERCISENAME.hbu ACA/HB conjoint utility file (created by computing
utilities via HB)
ACAEXERCISENAME.utl ACA conjoint utility file (created by computing
utilities)
CVA Files
CVAEXERCISENAME.att Contains attribute level text to accompany .utl for
CVA
CVAEXERCISENAME.hbu CVA/HB conjoint utility file (created by computing
utilities via HB)
CVAEXERCISENAME.utl CVA conjoint utility file (created by computing
utilities)
CBC Help 325
2 CBC Help
2.1 Getting Started with CBC
2.1.1 Overview of CBC Software
CBC is a component within the SSI Web suite of Internet interviewing products. CBC is used for
conducting Choice-Based Conjoint (CBC) studies over the web, in non-web "CAPI" interviewing mode, or
via paper-and-pencil questionnaires. CBC questions are usually integrated within a longer SSI Web
questionnaire that includes other standard survey questions. The number of additional questions that
can be added depends on your license. A CBC study can include additional standard survey questions
that collect up to 50 data fields. An unlimited number of additional questions can be used if the largest
CiW interviewing license is acquired.
CBC studies are used for learning about respondents' preferences for the combinations of features that
make up products or services. CBC analysis can help with (among other things) product design, line
extensions, pricing research, and market segmentation.
question in this documentation. We are available during our business hours (Mountain Time Zone).
(Note: we provide full technical support for composing and testing your Internet survey on your local PC
and provide written instructions on how to install your questionnaire on your server. We cannot take
responsibility for or be expected to provide technical support to debug any customized HTML or
JavaScript you may add to your survey. You are also responsible to make sure that Perl is running, a
database is available, and that you can run Perl (CGI) scripts on your web server. We can provide some
assistance related to SSI Web's use with your server once the fundamental Perl functionality and a
database is in place.)
A sample CBC questionnaire (cbc.ssi) and supporting files are available in the \Samples folder within
your SSI Web installation. You can easily access this example study by clicking Help | Sample
Studies. This questionnaire lets you see an example of a CBC survey and investigate how it is set up.
If you know a little HTML (which is not necessary to use CBC), you can employ some "tricks" to help
you customize the look of your survey beyond the standard options provided by the software. By opening
the sample study and looking at the settings and text we've used, you can become familiar with some of
the possibilities.
Using CBC requires the following skills and knowledge base:
The analyst managing the survey should know the basics of conjoint analysis, including defining
attributes and levels properly, correctly phrasing the questionnaire text and response scales,
interpreting the part-worth utilities, and running simulations. It is not necessary to be a
statistician to use our software effectively, but knowledge of basic statistics for market research
is highly recommended.
The questionnaire is first composed locally on a PC running under Windows XP, Vista, or
Windows 7. The individual inputting/formatting the survey should have working knowledge of
standard Windows applications such as word processing, Web browsers and/or spreadsheets.
Though it is not necessary, some knowledge of HTML is valuable.
For web-based data collection, you may use our survey hosting service. You can also host
surveys on your own (one must have access to an Internet/intranet server running Perl 5.007 or
326 SSI Web v8
later, with a database). This can be on your company's own Web server, or your ISP's (Internet
Service Provider's) Web server. SSI Web is able to automatically upload the files needed to
perform data collection on your server. But, in some cases the user must know how to use FTP
software to create directories, upload files to those directories, and set permissions on the files
and directories. Click here to receive detailed instructions on the directories, files and
permissions.
Relationship between CBC and SMRT
CBC is used for designing, creating, and fielding CBC studies (either web-based, non-web "CAPI"
interviewing, or paper-and-pencil modes). But you conduct market simulations (what-if analysis) using
the Online Simulator (or the older SMRT software). SSI Web and SMRT are installed and run
independently--but they may communicate through a common file format (*.CHO), enabling the SMRT
software to easily read the data exported from a CBC study.
If you don't already own SMRT software, you receive a special copy (along with a manual) to analyze
your CBC data.
Steps in Conducting a CBC Study
1) Formulate attributes and levels and type them into the CBC interface.
2) Make decisions regarding your study design: decide how many product concepts to display per
choice question (task), whether to include a "None" option, and how many choice tasks to field in your
study. Using the point-and-click CBC interface, add those CBC questions to your SSI Web study.
3) Test your questionnaire in preview mode, by using the Local Test Server included in SSI Web, or by
uploading the files to your Web server. Make modifications and re-test.
4) Decide how many unique versions of the CBC questionnaire to use in your overall design (default is
300). If using paper-and-pencil data collection, it is common to use a few to a dozen design versions.
Generate the design. Using the Test Design module, test the CBC design to make sure the design is
efficient to estimate the part-worth utilities (main effects).
5) If using web-based data collection, upload your study to an Internet server and continue testing--by
yourself, with colleagues, and later preferably with a small number of pilot respondents. Receive
feedback from colleagues and pilot respondents. Examine the data to make sure all data are being
stored properly using the Admin Module and by downloading the data to your hard drive. Export the data
to a text-only *.CHO file. Estimate part-worth utilities using HB (hierarchical Bayes) or Latent Class.
Import the results into the SMRT software package or the online market simulator for analysis.
6) After ensuring that your CBC questionnaire works properly, that the data are being stored correctly,
and that your design is not deficient, invite respondents to take the survey.
7) Monitor the data collection using the Admin Module. Download the data, conduct analysis (options
include Counts, Logit, Latent Class, and HB) and then Prepare files for market simulations within the
SMRT system (or the online market simulator).
CBC Help 327
2.1.2 What is Choice-Based Conjoint?

CBC is used for conducting "Choice-Based Conjoint Analysis" studies over the web, in non-web
"CAPI" (Computer-Aided Personal Interview) interviewing mode, or via paper-and-pencil questionnaires.
CBC studies are used for learning about respondents' preferences for the combinations of features that
make up products or services. CBC analysis can help with (among other things) product design, line
extensions, pricing research, and market segmentation. The market simulators that result from CBC
analysis enable managers to test numerous product formulations and competitive scenarios. Market
acceptance/competitiveness, price sensitivity, and cannibalization are just some of the issues that
market simulators based on CBC data can probe.
The main characteristic distinguishing choice-based conjoint analysis from earlier types of conjoint
analysis is that the respondent expresses preferences by choosing concepts (products) from sets of
concepts, rather than by rating or ranking them. Over the last decade, choice-based conjoint has
become the most widely used conjoint-related technique.
The Choice Question
A CBC question is often referred to as a "task." A set of products (concepts) is displayed on the screen
and the respondent chooses among the concepts. For example:
This is a basic display for illustration. The user has a great deal of control over the fonts, colors, and
layout. Graphics may also be used. Realistic-looking "store shelf displays" can be developed with
scores of product graphics displayed if using the Advanced Design Module for CBC.
The attributes that make up each concept are carefully chosen so that the independent effect of each
attribute level upon a product concepts' likelihood of choice can be estimated. CBC software
automatically designs the concept combinations based on a few user-defined inputs.
Two important decisions to make when constructing choice tasks are 1) how many concepts to present
per task, and 2) how many total tasks to ask. These are sometimes difficult issues to answer and
ongoing research is helping to define guidelines and understand the consequences. A typical study
might include about ten choice tasks, with about four to six product concepts per task. The composition
of the choice tasks usually will vary between respondents. Each respondent receives one version of a
larger pool of efficient designs. Because respondents are randomly selected to receive a different version
328 SSI Web v8
of the overall design, we refer to these designs as "random" designs. However, the term random can be
misleading, as the designs themselves are carefully selected using a computer algorithm that ensures
that each separate version has level balance and near orthogonality.
Often some "holdout" choice tasks are added to the CBC questionnaire. These are not initially used for
estimating the preferences (part-worth utilities) for the respondents, but are used to check the internal
validity of the estimated utilities. We refer to these as "holdout" tasks or "fixed" choice tasks. They are
"fixed" in the sense that the product combinations within these tasks are shown in exactly the same way
to all respondents. (After holdout tasks have been used for validation purposes or selection of alternate
models, some researchers choose to employ them in a final model for part-worth estimation.)
The Role of CBC
Choice-based conjoint analysis has attracted much interest over the last two decades in the marketing
research field. There are many reasons for its rise to dominance among the conjoint-related methods:
Researchers tend to favor it because the task of choosing a preferred concept is

similar to what buyers actually do in the marketplace. Choosing a preferred
product from a group of products is a simple and natural task that anyone can
understand. An option is also available for asking repondents to choose the least
preferred concept within that task, in addition to the most preferred.
Choice-based conjoint analysis lets the researcher include a "None" option for respondents,
such as "I wouldn't choose any of these." By selecting that option, respondents who do not
like any of the options can express their lack of interest. Comparing "None" usage across
groups of respondents can reveal segments that are relatively more or less likely to purchase
product concepts.
Most conjoint analysis studies use "main effects only" assumptions. Choice-
based conjoint analysis typically involves leveraging the data across respondents,
making it feasible to quantify interactions. This capability is enhanced by the
(controlled) random designs used by the CBC System, which, given a large
enough sample, permit study of all interactions, rather than just those expected to
be of interest when the study was designed. It should be noted, however, that
using HB estimation for estimating individual-level part-worths often reduces the
need to model additional interaction terms (in other words, the interactions
observed in aggregate models are often due to unrecognized heterogeneity).
It is possible in choice-based conjoint analysis to have "product-specific" (alternative-

specific) attributes. For example, in studying transportation we might consider walking
shoes and bicycles. The attributes describing shoes are different from those describing
bicycles, and yet one might want to learn how much improvement in walking shoes would be
required to switch a respondent from cycling to walking. CBC's Advanced Design Module
permits alternative-specific designs.
Choice-based conjoint analysis does have a disadvantage, however: it is an inefficient way to elicit
preferences. Each concept is described using many attributes and each choice set contains several
concepts. Therefore, the respondent has to do a lot of reading and process a lot of information before
giving each answer. Moreover, the response data are sparse. With standard discrete choice, the choice
does not indicate how much more preferred that option is relative to the remaining options, or the relative
preferences for the not-chosen alternatives.
For those reasons, in the early days of CBC research, choice-based conjoint studies were usually not
CBC Help 329
used to estimate the values that individual respondents attached to attribute levels, as had been done
with traditional ratings-based conjoint methods. Instead, data from groups of respondents were
aggregated for analysis. This has been done either by combining all respondents or by studying subsets
defined by specific market segments. "Part-worth utility values" could be produced for each group of
respondents that summarize the choices made by those individuals. And, as in other conjoint methods,
the part-worth values could be used to simulate and predict respondent reactions to product concepts
that may not have actually appeared in the choice tasks (questions).
Aggregating respondents in CBC analysis assumed respondent homogeneity, which was not always
appropriate or desirable. Developments since the mid-1990s have recognized segment-based or even
respondent-by-respondent differences for CBC analysis. Latent Class analysis (as offered in the CBC
Latent Class Segmentation Module) can simultaneously delineate relatively homogeneous segments and
estimate their unique part-worth functions.
Computationally-intensive Bayesian estimation (built into the SSI Web interface, and also available in
Sawtooth Software's standalone CBC/HB System) has permitted estimating individual-level part-worths
from choice data. HB generally leads to more accurate predictions and greater flexibility during analysis.
It is generally considered a gold standard today, though other sophisticated part-worth estimation
methods often produce very similar quality results.
If you are relatively new to CBC research, we highly recommend that you visit our Technical Papers
library at http://www.sawtoothsoftware.com/support/technical-papers. We recommend you download
and read:
The CBC Technical Paper

An Overview and Comparison of Design Strategies for Choice-Based Conjoint Analysis
The Benefits of Accounting for Respondent Heterogeneity in Choice Modeling
Special Features of CBC Software for Packaged Goods and Beverage Research
330 SSI Web v8
2.1.3 CBC Tutorial and Example

Introduction
This tutorial leads you step-by-step through a simple CBC study, from thinking about a marketing
problem, planning a questionnaire and creating a new study in SSI Web, to entering a list of the
attributes and levels, generating the questionnaire and analyzing the results.
SSI Web is a powerful program and this tutorial only touches on the basics. You'll become aware of
different or more sophisticated approaches by reading other sections in this manual, by viewing on-line
help and by experimenting with SSI Web.
We also suggest the following two articles, available within our Technical Papers Library at http://
www.sawtoothsoftware.com/support/technical-papers: "CBC Technical Paper" and "Getting the Most
from CBC."
(Note: this example and the data presented are purely fictional. The suggested steps represent one
approach to this type of mark eting problem, but may not necessarily reflect the single "best" way to
conduct such a research study using the SSI Web system.)
The Marketing Problem
You have been hired as a consultant to a little-known startup company called Performance Plus. The
brilliant design engineers at Performance Plus have developed a golf ball that flies farther than average
balls. The proposed name for the ball is "Long Shot." Because Performance Plus is not well known
among golfers, your client has considered approaching a well-known maker of golf clubs and balls
(Golfers, Inc.) to market the ball using their existing brand name for balls: "Eclipse." Sales for Eclipse
have been declining and they welcome the opportunity to benefit from the new technology. A line
extension brand called "Eclipse+" has been proposed. However, the royalty Golfers, Inc. is proposing
seems high.
Two other manufacturers already market "high performance" balls: High-Flyer Pro (by Smith and
Forester), and Magnum Force (by Durango). High performance balls command a price premium of
between 20% to 50% over traditional balls and the market is growing rapidly.
One day between rounds of golf with the president of Performance Plus (you gracefully lost), you jointly
draft up the following questions on the back of your score card.
1) How well could Long Shot by Performance Plus hope to compete with existing competitors
Smith and Forester and Durango in the high performance ball market?
2) Should Performance Plus form an alliance with Golfers, Inc.? How much incremental value
does the Golfers, Inc. brand name "Eclipse" add?
3) How do golfers trade off performance vs. price for high performance balls?
4) How should "Long Shot" be manufactured and positioned in terms of performance and price
to maximize profitability?
5) Do novice or experienced golfers differ in their preferences for high performance golf balls?
Could this lead to a target market strategy?
CBC Help 331
Defining a List of Attributes and Levels
One of the first steps in using any Conjoint technique is to define the problem in terms of attributes and
levels. Back at the office, you review the points written on the back of the score card, and you write
down the following:
Brand
High-Flyer Pro, by Smith and Forester
Magnum Force, by Durango
Eclipse+, by Golfers, Inc.
Long Shot, by Performance Plus
Performance
Drives 5 yards farther than the average ball
Price
$4.99 for package of 3 balls
Each of these levels is mutually exclusive; golf balls are described using a single level from each
attribute. The range of the attributes is based on technical specifications that Performance Plus has
given you. As for prices, your research suggests that the two competitors are currently selling packages
of three balls for between $7.99 to $10.99. You select a wider range to cover the price levels you are
interested in, but don't consider any prices below $4.99, having determined that such a low price cannot
cover the manufacturing and marketing costs and result in a profit.
A Study Plan and Sample Scripted Questionnaire
Now that you have formulated some attributes and levels, you begin to think about the rest of the
questionnaire, and plan a design and field strategy.
You decide to invite golfing enthusiasts (by email) to visit a site on the web and take a CBC survey.
You've purchased an opt-in list of 1500 email addresses from a widely-read golfing magazine. Each of
the participants who complete the survey will be entered into a drawing for a free set of golf clubs from a
leading manufacturer. You expect a completion rate of between 15% to 25%, resulting in a final sample
size of between 225 to 375.
You have scripted the following introductory text screens and questions in your word processing
package:
Start:
Thank you for your willingness to participate in this study.
You are part of a special group of golfers we've selected to ask for feedback regarding golf
equipment. By completing this survey, you'll be eligible for a drawing to receive a free set of
PrimoDriver clubs.
332 SSI Web v8
(Click the Next button to continue)
Frequency:
First, we'd like to ask you about how often you golf. Would you say that you golf...
5 times or fewer per year

from 6 to 25 times per year
more than 50 times per year
Skill:
We'd like to get a feel for your skill level. Over the last year, what was your best score on a par-
72, 18-hole course?
120 strokes or more

Between 100 and 120 strokes
Between 85 to 100 strokes
Above par, but lower than 85
At or below par (72 or lower)
Venue:
Do you usually golf at...
Public courses
Private courses
IntroCBC:
In the next section, we'll be asking you specifically about high performance golf balls. These balls
are premium balls specially engineered to travel farther than the typical ball with good accuracy.
We'd like you to imagine that you are considering purchasing golf balls for your next golf outing.
We'll show you some different high performance golf balls and ask which one you would
purchase.
Some of the golf balls you are going to see are not currently available on the market, but we'd like
you to imagine that they were available today. It is important that you answer in the way you
would if you were actually buying golf balls.
If you wouldn't purchase any of the balls we'll show you, you can indicate that by choosing
"None". By choosing none, you indicate that you'd buy another brand, or would continue using
existing balls in your golf bag.
At this point in the questionnaire, you plan to ask the Choice-Based Conjoint questions (tasks). You
want to measure preference for the different levels of the attributes and their impact on choice for high
performance golf balls. To accomplish this, you decide to specify 17 CBC tasks (below is an example of
one such task). The features will combine freely to form up to 48 (4 brands x 3 performances x 4 prices)
CBC Help 333
different product offerings. Even though there are only 48 total possible product concepts in this
relatively small design, there are thousands of different ways to display multiple product concepts at a
time in choice sets.
Choice Task (one of 15 constructed tasks using a controlled randomization design strategy):
If you were considering buying golf balls for your next outing and these were the only alternatives, which
would you choose?
High-Flyer Pro, by Magnum Force, by Eclipse+, by Golfers,

Smith and Forester Durango Inc.
Drives 15 yards farther Drives 15 yards farther Drives 5 yards farther None: I Wouldn't
than the average ball than the average ball than the average ball Purchase Any of These
$10.99 for package of 3 $6.99 for package of 3 $6.99 for package of 3

balls balls balls
Even though these CBC tasks are called "random tasks," that term is easily misunderstood. Some
people take it to mean that the composition of the questions is randomly (haphazardly) selected.
However, the questions are carefully chosen according to design principles of balance and independence
(orthogonality). We call the tasks "random" because respondents are randomly selected to receive one
of many carefully constructed, unique versions of the CBC questionnaire.
In addition to the random choice tasks, you also plan to ask two fixed "holdout" tasks that will be
constant across all respondents. You plan to do this for three main reasons:
1) By doing some research on the Internet and at local golf stores, you have learned the
performance specifications and average prices for the two existing balls on the market (High-
Flyer Pro and Magnum Force). Your client has also given you a good feel for the product
specifications they think could match up well with these two brands. Therefore, you and your
client can imagine a specific potential product scenario you'd like to directly assess.
2) Your client has no experience with conjoint methods and seems a bit uncertain regarding
the reliability and accuracy of the market simulator. Demonstrating that the market simulator
can accurately predict responses to some holdout fixed scenarios should boost his confidence
in the method.
3) You may want to test how well different part-worth utility analysis methods work for the
study. You'd like to have a couple of "holdout observations" by which to compare the
predictive accuracy of alternative solutions you might develop.
These fixed tasks will not be constructed randomly, but are to be displayed exactly with the specific
levels you assign. These two tasks will be asked as the 7th and 9th tasks. In total, you'll ask 15 random
+ 2 fixed = 17 tasks. In the first fixed holdout, you plan to show the relevant competition versus your
client's offering with your client's brand name. In the second fixed task, you plan to show that same
relevant competition versus the Eclipse+, by Golfers, Inc.
Here are those two fixed holdout concepts:
Fixed Holdout Task #1 (7th overall choice task):

334 SSI Web v8
If you were considering buying golf balls for your next outing and these were the only alternatives,
which would you choose?
(7 of 17)
High-Flyer Pro, by Long Shot, by Magnum Force, by None: I Wouldn't

Smith and Forester Performance Plus Durango Purchase Any of These
Drives 15 yards farther Drives 10 yards farther Drives 5 yards farther

than the average ball than the average ball than the average ball

balls balls balls
Fixed Holdout Task #2 (9th overall choice task):
If you were considering buying golf balls for your next outing and these were the only alternatives,
(9 of 17)
Magnum Force, by Eclipse+, by Golfers, High-Flyer Pro, by None: I Wouldn't

Durango Inc. Smith and Forester Purchase Any of These


balls balls balls
Notice that your client's offering is in the center position for both tasks, but the position of its competitors
is rotated. The two fixed holdout tasks are separated by a random choice task so that respondents will
generally not recognize that it is a repeated task with just one small difference (the brand name attached
to your client's ball).
Lastly, you decide to ask about gender and income. These might prove useful as a respondent filter in
analysis, for bench-marking versus future waves of research, or for future target/database marketing
efforts.
Gender:
Are you..
Male
Female
Refused
Income:
This is the last question in the survey. What is your total household income for the last calendar
year, before taxes?
Less than $30,000

CBC Help 335
Between $30,000 and $60,000

Between $60,000 and $100,000
More than $100,000
Refused
Signoff:
That completes our survey. Thank you very much for your input.
Now that you have developed a list of attributes and levels and scripted a rough-draft of your
questionnaire and the design of the choice tasks, you are ready to create the study and compose the
questionnaire within the SSI Web system.
Using SSI Web to Write the Questionnaire
Start by opening the Sawtooth Software SSI Web program. If you have a standard installation, you start
it by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI Web. "SSI" stands for
"Sawtooth Software Inc." and "Web" conveys the idea that this software is for developing Web-based
surveys (although SSI Web can also be used for standalone computer interviewing with its CAPI
interviewing module.) CBC is one component within that software program. The next step is to open a
new study and define a study name.
After starting SSI Web, choose File | New Study…. The New SSI Web Study dialog appears, with your
cursor active in the Name field.
The Location field lists the folder (directory) that SSI Web currently is using to store studies. You can
use any folder you like for this tutorial project. You can browse to and create new folders by clicking the
Browse... button to the right of the Location field and then (after browsing to the folder in which you want
to create a new sub-folder to contain this study) by clicking the "new folder" icon. Each studyname in
SSI Web has a maximum number of twenty-four characters (either letters or numbers) and SSI Web
requires that the study be located within its own folder. The folder that SSI Web automatically creates
for your study also carries the same name as the study. For this tutorial, you might create a name such
as golfexample. From the New SSI Web Study dialog, specify golfexample as the studyname.
336 SSI Web v8
Click the OK button. You are returned to the main menu and a new "Study Navigator" window is
displayed along the left-hand side of the screen.
As you work with this study, items appear in the Study Navigator window, listing the functional areas you
have been using or that are now available to you. This list of items provides a Shortcut link to access
those parts of SSI Web. Alternatively, you can access those same areas by clicking icon buttons on
the toolbar or by using the pull-down menus.
Now that we have created an SSI Web study called golfexample, we can add our CBC exercise to that
study. Click Compose | Write Questionnaire... to access the Write Questionnaire dialog. Place your
cursor at the place in the questionnaire when you'd like to add the CBC exercise. Right now, there are
no questions in your survey, so you can just click the last page break in the questionnaire list, and click
the Add... button. Specify to add a new CBC exercise named CBCgolfexercise to the study (you can
have multiple CBC exercises within the same study).
Click OK and the CBC Exercise Settings dialog is shown. Click the Question Text tab and you see the
following:
CBC Help 337
Default text is supplied within the Header 1 field. Paste the text we'll be using for this task into the
Header 1 field:
If you were considering buying golf balls for your next outing and these were the only
alternatives, which would you choose?
Header 2 contains special "SSI Script" (a command language that works within SSI Web) for showing
respondents which task number they are on, such as "(3 of 12)" etc. You do not need to modify this
script for it to work automatically.
Entering the List of Attributes and Levels
When composing conjoint analysis studies, one typically begins by specifying the list of attributes and
levels in the software.
To enter the list of attributes and levels you developed, go to the Attributes tab on the CBC Exercise
Settings dialog.
338 SSI Web v8
To add the first attribute, click the Add button (at the bottom left of the Attributes panel).
Type in the first attribute name: Brand. The attribute name is a label that the respondent may see in the
interview (if you select the option to show the attribute labels at the left of each choice task). If you want
to use a shorter label to display to you as the questionnaire author for program settings and data export,
specify that label in the Internal Label field. If you do not specify an internal label, the label in the
Attribute Display Text is used. Click OK to accept this information and close the dialog.
CBC Help 339
Now that at least one attribute name is in the list, the Add button under the Levels panel becomes
active. Also note that the Brand attribute is highlighted in the Attributes panel. With the Brand attribute
highlighted in the Attributes panel, click the Add button under the Levels panel to add levels within the
Brand attribute. The Level Text dialog is displayed. Type High-Flyer Pro, by Smith and Forester in
the Level Display Text field. To add the next level of Brand, press the ENTER key twice. Type in the
next level: Magnum Force, by Durango. Repeat the process for the remaining two levels of brand.
When you are ready to add the next attribute (Performance), click the Add button under the left
Attributes panel, type the attribute label, and click OK to place that new attribute on the attribute list.
With that new attribute highlighted on the attributes list, click the Add button under the Levels panel to
add the three levels of that attribute.
Follow the same pattern for the last attribute, Price. For your convenience, we repeat the full list of
attributes below. Note that you can copy-and-paste attribute level text from this document (and other
text documents) into the text fields within SSI Web. After highlighting the words to copy with your
mouse, use the shortcuts Ctrl-C to copy and Ctrl-V to paste into the desired field. (Hint: you can select
a list of attributes or levels from Word and paste into SSI Web using the Paste list member(s) from the
clipboard icon. This can save a great deal of time.)
Brand
High-Flyer Pro, by Smith and Forester
Magnum Force, by Durango
Eclipse+, by Golfers, Inc.
Long Shot, by Performance Plus
Performance
Price
Now that you have finished entering the list of attributes and levels, we'll use the other tabs to continue
developing our CBC exercise.
Additional Study Parameters
After you have specified your list of attributes and levels, you can specify other study parameters to
govern your CBC exercise. From the CBC Exercise Settings dialog, click the Design tab. The upper
left-hand corner of that dialog shows the following:
340 SSI Web v8
Here, you can specify how many Random (experimentally designed) choice tasks and how many fixed
(user specified) tasks you want. For this golf ball study, we want 15 random and 2 fixed tasks. Change
the Random Choice Task s field to 15.
As planned, the questionnaire shows three products on the screen, plus a None. Change the Number of
Concepts per Task to be 3 (excluding the None option).
Recall that we want to include a "None" option in this CBC questionnaire. This gives respondents the
option to indicate that they would choose none of the products displayed on the screen.
The Traditional None Option is selected by default. You can specify the text to use for your None option
by clicking the Settings... button. Click Settings... and specify the following text within the None
Option dialog:
None: I Wouldn't Purchase Any of These
Next, there are some Advanced Settings that we should review, so click the Show Advanced Settings
button to show more features on the dialog.
CBC Help 341
Recent research has suggested that the Balanced Overlap design approach (for selecting the
combinations of features to show respondents) is a robust approach for studies like this, so select
Balanced Overlap as the Random Task Generation Method.
It is useful to have many versions of the questionnaire (each showing a different combination of golf
balls). This leads to efficient designs for estimating part-worth utilities and also controls for order bias.
The default in the software is to generate 300 unique versions of the "Randomized" CBC tasks. Such a
quantity may be overkill, but the software can handle this quite easily, so it seems prudent to go with a
large number such as 300.
Click OK to return to the Format tab. The upper part of that tab shows the following settings:
342 SSI Web v8
The Format Options control how wide the choice tasks might appear on the respondent's monitor, how
much white space is included between concepts and attribute levels, and how many columns to use to
display the product concepts.
Number of Columns describes how many columns will be used to arrange the product concepts on the
screen. By default, we assume you want one column displayed per product (and per the None concept).
You can change that if you wish (for example, to show product concepts on multiple rows), but it isn't
necessary for this study.
The Alternating Concept Colors are used in alternating concepts in the choice task to help make the
choice task more readable and provide a visual indicator from one choice task to another that the
question has changed (sometimes choice tasks can look so similar that respondents might not notice
from, say, the first task to the next that anything is different.) It appears that no alternative colors have
been selected, but indeed alternating colors have already been selected as part of the Style that is
currently being used for your SSI Web study. If you would like to override the default colors used in your
style, you can select new colors here.
You can override the default font color and size specified in the Style that is currently being applied to
your SSI Web study (Styles are selected by going to Compose | Survey Settings, Style tab) by clicking
the Paintbrush icon.
To preview how the CBC task will look when it runs on the respondent's computer using Windows
Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved and
no special "error checking" or "validation" is performed for the question).
CBC Help 343
This may or may not be the look you want for the questionnaire. You can modify the font and sizes of
the font using the Paintbrush icon. Or, you can select a different Style (selecting styles was
described in the 45-minute tutorial for SSI Web).
Go ahead and close the browser window. At this point, you might try modifying some of the settings on
the Format tab. After each change you make, you can click Preview to see the effect of the changes.
Generating the Experimental Design
In CBC studies, we use a carefully chosen experimental design plan. The "experiment" involves
observing how respondents react to different golf ball specifications. The "design" reflects the attribute
combinations that make up the golf balls and how these combinations are placed into choice sets.
Ideally, each respondent would receive a unique questionnaire version, with different combinations of golf
balls arranged within sets in unique ways. There are only a finite number of possible combinations, so
there potentially can be some identical choice tasks across respondents, but the idea is to improve
measurement of the effects of the attribute levels (including reducing order and learning effects) by
ensuring a high degree of variability in the choice tasks across individuals.
CBC lets you generate up to 999 unique versions of the questionnaire to be uploaded to the web server.
By pre-specifying the design plans in a file residing on the server, we can simply assign each new
respondent to the next design version within the file, which places minimal demands on the web server.
Even if you have more than 999 respondents, once a reasonably large number of designs are allocated
across respondents, the statistical gains of using even more questionnaire versions are very minimal.
The default in the software is to use 300 design versions. Even 300 designs is probably overkill, but the
software manages hundreds of designs for typical studies quite easily, and it seems beneficial to reduce
order bias and improve design efficiency by using numerous questionnaire versions.
If you aren't already at the CBC Exercise Settings dialog, go there by navigating to the CBC exercise
from the Study Navigator panel, or by clicking Compose | Write Questionnaire and editing a question
within the exercise in the question list. Click the Design tab, and the following is displayed:
344 SSI Web v8
Many of the features of this dialog are beyond the scope of this tutorial. You can read additional details
by pressing F1. We'll cover some of the basics here.
Under the Design Settings area, we specify the Random Task Generation Method. This indicates the
strategy that CBC uses to generate the "random" choice tasks in our study. We'd recommend using
the Balanced Overlap method for this study.
Number of Questionnaire Versions refers to how many unique versions of the CBC questions that SSI
Web will save into a file. When respondents enter the questionnaire, they are assigned the next
questionnaire version. Once a respondent is assigned the 300th version, the next respondent begins
again with version #1, and so forth.
Attribute Randomization lets you randomize the order in which the attributes appear in the choice task.
We specified the attribute list in the order Brand, Performance and Price. If we use the default (which is
not to randomize the order), Brand will always appear in the top and Price always at the bottom. This
seems like a natural presentation order for this study, so we'll retain the No Randomization setting.
Concept Sorting controls how the concepts are arranged within the choice task. By default, the order of
presentation is randomized. But, if we wanted the first brand always to appear in the first concept
position, etc., we could specify to sort concepts based on the Natural Order for Brand. You can
investigate details regarding Concept Sorting by pressing F1. For this tutorial, we'll retain the defaults.
Once we are comfortable with our settings, we click Generate Design.

CBC Help 345
You are given two warnings, notifying you that you have yet to specify the composition of the two fixed
tasks in your study. We'll do that later, so you can ignore the warnings.
The following report is displayed:
CBC Design Efficiency Test

Copyright Sawtooth Software
8/23/2012 3:31:51 PM
Task generation method is 'Balanced Overlap' using a seed of 1.

Based on 300 version(s).
Includes 4500 total choice tasks (15 per version).
Each choice task includes 3 concepts and 3 attributes.
A Priori Estimates of Standard Errors for Attribute Levels

-------------------------------------------------------------
Att/Lev Freq. Actual Ideal Effic.
1 1 3375 (this level has been deleted) High-Flyer Pro, by Smith and Forester
1 2 3376 0.0275 0.0275 0.9970 Magnum Force, by Durango
1 3 3375 0.0275 0.0275 1.0023 Eclipse+, by Golfers, Inc.
1 4 3374 0.0275 0.0275 0.9999 Long Shot, by Performance Plus
2 1 4500 (this level has been deleted) Drives 5 yards farther than the
average ball
2 2 4500 0.0231 0.0231 0.9964 Drives 10 yards farther than the
average ball
2 3 4500 0.0230 0.0231 1.0052 Drives 15 yards farther than the
average ball
3 1 3375 (this level has been deleted) $4.99 for package of 3 balls
3 2 3374 0.0276 0.0275 0.9954 $6.99 for package of 3 balls
Note: The efficiencies reported above for this design assume an equal number of
respondents complete each version.
The report indicates that 300 versions of the CBC questionnaire were created. We know that each
version of the questionnaire had 15 "random" choice tasks, so a total of 300 x 15 = 4500 choice tasks
were generated. Some of the details of the report are beyond the scope of the unit, but you can obtain
more information within the Help text. The "Freq." column indicates how many times each level is
represented in the design. Within each attribute, you can see that CBC has nearly perfectly balanced
the presentation of levels. The "Effic." Column indicates the relative efficiency of the design. Design
efficiency runs from the worst (0.0) to the best (1.0) and we can see that this particular design is quite
efficient with respect to main effects.
If you include prohibitions in your design, sometimes designs can be quite inefficient. If the Effic. values
are relatively low, this is a warning that your estimates may lack precision and you should reconsider
your setup. In some cases, you will receive a warning stating that the design is deficient or will notice
asterisks rather than data in the test design report. You should re-evaluate your design should this
occur. Failure to correct the problem can result in unusable data.
Click Close to close the test design report, and click OK to close the CBC Exercise Settings dialog.
346 SSI Web v8
Let's now examine the questionnaire we've built to this point:
You'll note that SSI Web has added 15 Random (CBCgolfexercise_Random1 through
CBCgolfexercise_Random15) and 2 Fixed CBC tasks (CBCgolfexercise_Fixed1,
CBCgolfexercise_Fixed2) to the survey. These questions are based on your settings and the default
template within SSI Web. The Fixed tasks aren't yet in the proper positions (we wanted them in the 7th
and 9th task positions in the survey). You also haven't yet specified the attribute combinations used for
your user-defined Fixed tasks, so SSI Web has inserted the questions with default levels that we'll
CBC Help 347
modify shortly.
Before we fine tune those CBC questions, let's turn our attention to formatting other aspects of the
survey.
Passwords and the Start Screen
Placing a survey on the web makes it convenient for a geographically dispersed population to take
surveys. However, the danger is that the survey may become available to people that have not been
invited to take the survey. Also, some respondents might try to take the survey multiple times.
Assigning respondent passwords is a way to deal with both of these issues. Password assignment is
beyond the scope of this tutorial, so to make things simple we'll assume that no passwords are to be
used.
The Start screen is placed as the first page in your questionnaire, whether you are using passwords or
not. If you are using passwords, it is the page in which respondents type passwords to access the
survey. If not using passwords, you'll use this screen to specify any introductory/welcome text.
Let's enter the introductory text into the Start screen using the Write Questionnaire dialog. You can
access the Write Questionnaire dialog by selecting Compose | Write Questionnaire… or by clicking
the "pencil" icon on the toolbar.
The introductory text and other survey questions would most likely be initially developed within a word
processing document. Assuming you really had such a document, you might use the Ctrl-C to copy
and the Ctrl-V shortcuts to paste the information into SSI Web. We suggest you simply copy-and-paste
the text within this document into your SSI Web questions rather than re-type the text (if viewing this
document with Acrobat Reader, you can use the "text select" icon from the Acrobat Reader toolbar.)
The introductory screen is as follows:
Start:
You are part of a special group of golfers we've selected to ask for feedback regarding golf
equipment. By completing this survey, you'll be eligible for a drawing to receive a free set of
PrimoDriver clubs.
We'll insert this introductory screen in the Start question that is automatically the first question of any
SSI Web survey. From the Write Questionnaire dialog open the Start question by highlighting it in the
list of questions and clicking Edit.... Alternatively, you can double-click the Start question, and it
automatically opens. The Questionnaire Access and Passwords dialog is displayed:
348 SSI Web v8
Most question types in SSI Web have "Header 1", "Header 2," "Body" and "Footer" sections. (The Start
question has all but the "Body" section.) These are "text" areas in which you can insert any text
(including HTML). When the question is viewed with a web browser, the sections are organized roughly
as follows:
For this introductory screen (that includes three paragraphs of information), it seems to make sense to
place the first paragraph in the "Header 1" area, the second paragraph in the "Header 2" area, and the
third paragraph in the "Footer" area.
CBC Help 349
Type (or cut and paste) the following text for the Start question into the text areas in the Start question.
Put the first paragraph in Header 1, the second paragraph in Header 2, and the third paragraph in Footer.
You are part of a special group of golfers we've selected to ask for feedback
regarding golf equipment. By completing this survey, you'll be eligible for a drawing
to receive a free set of PrimoDriver clubs.
To preview how the question will look when it runs on the respondent's computer using Windows
Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved and
no special "error checking" or "validation" is performed for the question).
SSI Web automatically places paragraph breaks (extra blank lines) between the Header 1, Header 2, and
Footer sections. If you put all the text in a single section, you may see that when the web browser
interprets the text, it all runs together without any blank lines between the paragraphs (unless you insert
some HTML instructions to force blank lines between paragraphs). We'll talk about using HTML within
your text to take greater control over the layout, font, and style later in this tutorial.
Click the OK button in the Preview window to close that window and return to the previous SSI Web
dialog.
After viewing the text in preview mode, you might decide that it is really too small, or that you want to
make the text bold. You can change the size and styles (bold, italic, underline) or the text justification
for the three major text sections by clicking the Paintbrush icon on the Start Question Text tab.
After changing any setting, click Preview again.
After you are happy with the layout of the Start screen, click OK to return to the Write Questionnaire
dialog. If you need to move a question once it has been added to the List of Questions, simply highlight
the question to be moved and click the or buttons to move the questions within the list (you
can also highlight a question and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within
the current questionnaire, or even to another SSI Web study.)
Now you are ready to specify the first Select-Type question:

350 SSI Web v8
Frequency:
First, we'd like to ask you about how often you golf. Would you say that you golf...
5 times or fewer per year

more than 50 times per year
First, make sure that you highlight the page break directly beneath the Start question on the List of
Questions (when you add a new question, it is placed directly below the highlighted question/page break
on the list). At the Write Questionnaire dialog, click Add..., choose Select as the CiW question type,
and type Frequency for the question name. Click OK and the Select Question dialog is shown.
Place the heading text for the question in the Header 1 field. To specify the response options, click the
Response Options tab. From the Response Options tab, use the Add... button to add the four response
options for this question. When you are finished, the question should look something like:
Now that you have seen how to specify Select question types in SSI Web, you have the tools you need
to specify the remaining four select-type questions for the golf ball questionnaire (Skill, Venue, Gender
and Income—please refer to the questionnaire text for these questions as presented near the front of
this tutorial).
Remember that when you add a new question to the List of Questions, it is inserted directly below the
question that was highlighted when you clicked the Add... button.
Formatting Text in SSI Web Questions
In addition to the Select questions we added to the questionnaire previously, there are two text-only
instructional screens to add. Referring to the questionnaire we outlined earlier, these "questions" are
named IntroCBC and Signoff. Even though the respondents aren't asked to provide specific inputs to
these, we refer to these as "questions" in the sense that we add them to the questionnaire as if they
were standard questions and they are listed separately by their "question" name in the List of Questions.
After the Venue question follows the text-only instructions:

CBC Help 351
IntroCBC:
In the next section, we'll be asking you specifically about high performance golf balls. These balls
are premium balls specially engineered to travel farther than the typical ball with good accuracy.
We'd like you to imagine that you are considering purchasing golf balls for your next golf outing.
We'll show you some different high performance golf balls and ask which one you would
purchase.
Some of the golf balls you are going to see are not currently available on the market, but we'd like
you to imagine that they were available today. It is important that you answer in the way you
would if you were actually buying golf balls.
If you wouldn't purchase any of the balls we'll show you, you can indicate that by choosing
"None". By choosing none, you indicate that you'd buy another brand, or would continue using
existing balls in your golf bag.
This text layout is a bit more challenging than we dealt with before (when we put three paragraphs of
introductory text into the Start question). There are more than three separate paragraphs here and we'll
need to deal with the additional element of bolding selected text.
Add a Text/HTML Filler question directly following the Venue question in the questionnaire. To do so,
highlight the Venue question, click Add… and specify the Question Name as IntroCBC and the
question type as Text/HTML Filler.
The Text/HTML Filler dialog is displayed.
Earlier in this tutorial, when we used the Start question to format the opening page of the survey, we
placed each paragraph of text in a separate Header 1, Header 2, or Footer section. We saw that SSI
Web automatically places blank lines between text in these sections. However, with the text in the
IntroCBC question, there are many more paragraph breaks. We'll take the opportunity here to introduce
the concept of using a few simple HTML instructions within our survey text.
HTML stands for "HyperText Markup Language" and provides simple ways for you to enhance the look of
your surveys, such as by bolding or underlining words, or making paragraph breaks within text.
Browsers know how to interpret HTML instructions when displaying the page on the screen. If you know
HTML, you can use it whenever you wish within SSI Web surveys to accomplish your aims. If you don't
know HTML, it isn't very difficult to learn a few HTML tricks, or you can use the toolbar available within
the editor that appears when you click "Pencil" icons in SSI Web:
Either type or cut-and-paste the text for the IntroCBC question into the Header 1 text field (Any of the
other three fields could be used also, though the Body field is indented). Click the "Pencil" icon to
display the larger text editing window. Initially, the text appears something like this:
352 SSI Web v8
Even though it appears that there are extra blank lines between the paragraphs, web browsers ignore
these hard returns (as well as more than one consecutive space characters), so you'll need to provide
HTML instructions to insert these paragraph breaks (hard return plus blank line). HTML instructions are
placed within <> brackets, called "tags." For example, the HTML instruction to create a paragraph break
begins with an "open" paragraph tag written as <p> and optionally ends with a "close" paragraph tag
written as </p>. The text to be formatted as a separate paragraph is enclosed within these tags. You
can either directly type HTML within your document, or you can highlight text to be modified and click
the icons on the HTML toolbar in the editor. If you highlight the first paragraph with your mouse:
In the next section, we'll be asking you specifically about high performance golf balls. These
balls are premium balls specially engineered to travel farther than the typical ball with good
accuracy.
and then (with the text in the first paragraph highlighted) click the "Paragraph" icon on the toolbar,
this inserts a <p> prior to the text and a </p> after the text:
<p> In the next section, we'll be asking you specifically about high performance golf balls.
These balls are premium balls specially engineered to travel farther than the typical ball with
good accuracy.</p>
When the browser interprets this text, it doesn't display the "tags" but instead separates the text
CBC Help 353
enclosed within the tags as a separate paragraph. Repeat the same for each of the paragraphs in the
IntroCBC question.
Next, we need to bold certain words in the text. The HTML tags for bolding text are <b></b> (with the
text to be bolded placed between the open and close bold tags). You can either directly type these tags
within the document, or highlight the text to be bolded with the mouse and click the "Bold" icon on
the toolbar. After you finish separating the paragraphs with <p></p> tags and bolding the appropriate
text with <b></b> tags, it should look something like:
Click OK to return to the Text/HTML Filler dialog and then Preview to see how the web browser
displays this question. It should look like:
354 SSI Web v8
When you add the Signoff at the end of the survey, make sure to add it at the end of the survey as a
Terminate/Link question type. On the Settings tab of the Terminate/Link question, click Terminate
Respondent to indicate that respondents that reach this question are finished. SSI Web may warn you
at this point that a terminating question must stand alone on its own "page" in the survey. This leads us
into our next discussion on page layout.
Page Layout and Study Settings
It may be helpful at this point to review how SSI Web breaks the survey into separate pages and some
basic global settings that affect the look and functionality of your SSI Web questionnaire. The Write
Questionnaire dialog also displays the page breaks that may be currently set for your questionnaire.
CBC Help 355
This dialog shows how the various questions we've specified (or that SSI Web has automatically added
to the list) are arranged across different pages. Page breaks are indicated by "---------------<Page>".
Notice that the preliminary questions (Frequency, Skill and Venue and IntroCBC) we've added to the
questionnaire are all currently arranged on the same page. Let's assume we wanted to break these up,
one question per page. You add a page break after the Frequency question by highlighting Frequency
on the list and clicking the Add Page Break button. Frequency is now placed on its own page.
Repeat this operation to place Skill, Venue, IntroCBC and CBCgolfexercise_Random1 on separate
pages. Make sure to place the final Terminate/Link screen Signoff on its own (last) page.
While we had been adding page breaks for our questionnaire, you may have noted that the two fixed
CBC tasks (CBCgolfexercise_Fixed1 and CBCgolfexercise_Fixed2) are not yet in the 7th and 9th
CBC task positions. You can move a question on the list by highlighting the question in the List of
Questions and clicking the or buttons to move the question to another point in the
356 SSI Web v8
questionnaire. Move CBCgolfexercise_Fixed1 directly after CBCgolfexercise_Random6 and move

CBCgolfexercise_Fixed1 directly after CBCgolfexercise_Random7. You may need to re-arrange the
page breaks so that each CBC question remains on its own page.
Specifying Fixed Tasks
A fixed choice task looks just like typical "Random" (experimentally designed) tasks. However, rather
than let the design algorithm determine the combination of attribute levels to be shown for each
respondent, you specify the codes for the levels to be displayed in each fixed product concept. Recall
that the purpose of specifying fixed holdout tasks for this example is to achieve a controlled and direct
measure of the existing competitors' offerings versus our client's proposed offerings. You'll also be able
to use the results to check the ability of the market simulator you develop using the 15 random choice
tasks to predict the responses to the 2 fixed holdout tasks.
Recall that our list of questions already includes two fixed tasks named CBCgolfexercise_Fixed1 and
CBCgolfexercise_Fixed2. These tasks will look exactly like the experimentally designed (random)
tasks within the survey, except that you'll specify precisely which levels to show in those two tasks.
Edit any of your CBC questions (or use the Study Navigator panel to directly go to the CBC exercise).
Go to the Design tab on the the CBC Exercise Settings dialog.
To modify the levels displayed in the fixed choice task, click the Fixed Task Designs… button.
From this one dialog, you can specify the attribute combinations to be used in each of your fixed tasks.
To change from one fixed task to another, use the Fixed Choice Task drop-down box at the upper-left
hand corner of the dialog. We'll first modify CBCgolfexercise_Fixed1.
Recall that the first fixed holdout was to display the following three products:
High-Flyer Pro, by Smith Long Shot, by Magnum Force, by None: I Wouldn't

and Forester Performance Plus Durango Purchase Any of These


balls balls balls
Use the drop-down controls provided to specify the three product concepts as shown.
Click the OK button when finished. Preview the fixed holdout task to make sure it looks as you expect.
Make any necessary changes.
CBC Help 357
Next, you need to modify the second fixed holdout task. From the CBC Fixed Choice Task Settings,
used the Fixed Choice Task drop-down control to select CBCgolfexercise_Fixed2. Modify it to have
the appropriate specifications for our second fixed choice task:
Magnum Force, by Eclipse+, by Golfers, High-Flyer Pro, by Smith None: I Wouldn't

Durango Inc. and Forester Purchase Any of These


balls balls balls
Again, preview this task to make sure it looks as expected.

You have probably noted that the survey we've created is pretty bland-looking. We can add some polish
First, let's select a style from the Styles tab. Select a style you wish to use and click Use Selected at
On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Golf Ball
progress bar to the page footer.
358 SSI Web v8
Local Test Server
Locally.
Your browser opens up the first survey page. This is your survey, as it would appear to a respondent
over the Internet (there are slight visual differences from browser to browser).
After you have edited the survey to the point that you are pleased with its look, content, and functionality,
you should examine the test data you've collected using Local Test Server (this process is described in a
separate tutorial called "Getting Started with SSI Web: A 45-minute Hands-On Tour" available on the
Help menu), then at least examine the CBC data using Counting analysis (described later). The results
should generally reflect your preferences (assuming you answered in a rational manner). If they don't,
this suggests something may be wrong with your study setup.
After you are convinced that the survey is functioning properly, you should also pretest your survey
among your colleagues. They can give you feedback regarding the usability of the survey, and you can
examine the resulting Counts data or perhaps even the part-worths (assuming you have enough data) to
make sure the data at least have face validity.
CBC Help 359
Pretesting and Fielding
Let's now return to the marketing problem and story we began at the beginning of this document.
After you have tested the survey using Local Test Server, you post the survey to the Web on your web
server, on a server provided by an ISP (Internet Service Provider), or through Sawtooth Software's web
hosting service. (Setting up your survey on the web is beyond the scope of this unit, but is described in
detail in the SSI Web help documentation. Fielding options also include hosting on your own company's
server, or using Sawtooth Software's hosting service).
The next day, you send an email to your client, with a link to take the survey. The president of
Performance Plus first takes a survey and suggests some minor wording changes. After you make
those changes, you invite six other individuals at Performance Plus take the survey. You download the
results and analyze the resulting data. Due to the small sample size, the results are a bit noisy (and
there is an obvious bias toward your client's balls), but the data seem to feel right.
Next, you recruit six golf enthusiasts within your city to come to a central site to take the survey. You
watch silently as they take the survey. At the end of each session, you ask each respondent follow-up
questions to ensure that there weren't any parts that were difficult to understand or just didn't make
sense. You ask them specifically about the choice tasks, making sure that there wasn't too much
information on the screen at once, and that they didn't feel overwhelmed with the task. After debriefing
the test respondents and analyzing their data to ensure that the results looked reasonable, you make a
few small adjustments to the questionnaire and attribute descriptions, and proceed to field.
Using a bulk email program, you send an email to the list of 1500 golfing enthusiasts from the opt-in list
mentioned at the beginning of this unit. After a week, and a reminder email sent to those who had not
yet completed the survey, you achieve 250 total completes.
The completed data reside on the server where you uploaded the SSI Web survey. The details for
viewing and downloading the data from the server are provided in other areas of the SSI Web
documentation and are beyond the scope of this CBC unit. Even so, it is useful to at least describe the
steps you would perform to access and download the data from the Web, estimate the part-worth
utilities, and begin analysis using the market simulator.
Data Management, Utility Estimation, and Market Simulations
SSI Web includes an Admin Module so that you can monitor or access your project from any device that
is connected to the Web. The Admin Module is password protected and your passwords for access are
specified on the Field | Web Server Management dialog. SSI Web generates random administrative
access passwords whenever you create a new study, but you can modify them to suit your needs.
To download the data from the 250 respondents to the golf ball study, you would browse to the
administrative module for your study on the web site (again, we are speaking hypothetically, as for this
tutorial study there is no such site set up). Once at the administrative module, you would download your
data (by clicking Data Management | Download from the main menu), making sure to save your data
(named studyname_data.sqlite, where STUDYNAME is your study name) to the same folder on your
hard drive in which you developed the SSI Web project. Alternatively, from the SSI Web interface, you
can simply click Field | Download Data... to download your data automatically.
CBC researchers employ a variety of techniques for analyzing their data. One common approach for
initially understanding the basic, summary preferences for the market is called "Counting" analysis. This
360 SSI Web v8
produces proportions from 0.0 to 1.0 for each level in your study, reflecting how often this level was
chosen, when available in a choice set. The higher the proportion, the higher preference for the level.
While this method of analysis has good intuitive appeal, there are more powerful ways to analyze the
data based on estimation of part-worth utilities and subsequent market simulations.
There are three part-worth estimation routines that Sawtooth Software provides. The first technique is
called multinomial logit, which "pools" respondent data in a single aggregate model. This technique was
the first part-worth estimation technique that Sawtooth Software used for analyzing CBC data in the early
1990s. It is a good technique for quickly summarizing the results for the sample, but it is quite
susceptible to the IIA (red-bus/blue-bus) problem, which is discussed in many technical papers and in
the CBC help documentation. Two other estimation techniques are commonly used for developing the
final simulator models delivered to clients: Latent Class and Hierarchical Bayes (HB). These techniques
model respondent heterogeneity (recognize differences between respondents in terms of preferences)
and in most cases provide more useful part-worths (than aggregate logit) for the purpose of market
simulations.
It is beyond the scope of this tutorial to go into much depth on these part-worth estimation methods.
The three part-worth estimation programs (logit, latent class, and HB) all work seamlessly within the SSI
Web interface. The final step of market simulations (what-if analysis) is not integrated within SSI Web.
You need to take the part-worth utility results of the utility estimation programs and import them into the
market simulator of your choice. Sawtooth Software offers two such simulators: 1) an older market
simulator from the late 1990s called SMRT, or 2) a recent web-based market simulator called the Online
Simulator. SMRT is available for your use for no additional licensing fee, though your clients would need
to purchase a license to use the SMRT simulator. The Online Simulator carries a fee for both your use
and client use.
Analyzing the CBC Data Using Counts
Hypothetical data for 250 respondents for this golf ball study are provided within the CBC tutorial study
installed with SSI Web. To open that project, click Help | Sample Studies | CBC Sample.
If you used a randomized design (which you did for this project), usually the first step in analyzing the
choice results is to conduct a "Counting" analysis. CBC's Counts program reports the percent of times
each attribute level was chosen when it was available on the screen. Counts provides an intuitive
measure of the impact of each attribute level on overall choice for golf balls. Counts are proportions
ranging from 0 to 1. For example, a Count of 0.31 for an attribute level would mean that when a golf ball
was displayed including that particular level, respondents chose it 31% of the time.
To access the Counts program, select Analysis | Calculate CBC Counts....
By default, the Counts program analyzes all one-way and two-way Count proportions. Notice also by
default that the Counts program uses only responses to the 15 randomized choice tasks you fielded
(under CBC Counts Settings, Task s it indicates that 15 random tasks are included). The two fixed
holdout tasks are not included. That is because results from Counts assume random designs where
each attribute level appears an equal number of times with each level of the other attributes.
Randomized designs (with no prohibited level combinations) make it possible to analyze the effect of
each attribute level independent of all other levels. That characteristic does not usually hold if fixed
holdout tasks are included in the analysis.
From the CBC Counts dialog, click Compute Counts... and the following report is displayed in the report
window:
CBC Help 361
Brand:
Total
Total Respondents 250
High-Flyer Pro, by Smith and Forester 0.369
Magnum Force, by Durango 0.329
Eclipse+, by Golfers, Inc. 0.188
Long Shot, by Performance Plus 0.156
Within Att. Chi-Square 352.705
D.F. 3
Significance p < .01
Performance
Total
Drives 5 yards farther than the average ball 0.169

D.F. 2
Price:
Total
$4.99 for package of 3 balls 0.390

D.F. 3
Brand: x Performance:
Total
High-Flyer Pro... Drives 5 yards farther... 0.265
High-Flyer Pro... Drives 10 yards farthe... 0.408
High-Flyer Pro... Drives 15 yards farthe... 0.432
Magnum Force, ... Drives 5 yards farther... 0.234
Magnum Force, ... Drives 10 yards farthe... 0.331
Magnum Force, ... Drives 15 yards farthe... 0.418
Eclipse+, by G... Drives 5 yards farther... 0.094
Eclipse+, by G... Drives 10 yards farthe... 0.200
Eclipse+, by G... Drives 15 yards farthe... 0.265
Long Shot, by ... Drives 5 yards farther... 0.083
Long Shot, by ... Drives 10 yards farthe... 0.182
Long Shot, by ... Drives 15 yards farthe... 0.204
Interaction Chi-Square 22.065

D.F. 6
362 SSI Web v8
Brand: x Price:
Total
High-Flyer Pro... $4.99 for package of 3... 0.556
High-Flyer Pro... $10.99 for package of ... 0.201
Magnum Force, ... $4.99 for package of 3... 0.454
Magnum Force, ... $10.99 for package of ... 0.215
Eclipse+, by G... $4.99 for package of 3... 0.287
Eclipse+, by G... $10.99 for package of ... 0.081
Long Shot, by ... $4.99 for package of 3... 0.282
Long Shot, by ... $10.99 for package of ... 0.039

D.F. 9
Performance: x Price:
Total
Drives 5 yards... $4.99 for package of 3... 0.277
Drives 5 yards... $10.99 for package of ... 0.078
Drives 10 yard... $4.99 for package of 3... 0.398
Drives 10 yard... $10.99 for package of ... 0.121
Drives 15 yard... $10.99 for package of ... 0.192

D.F. 6
None
Total
None chosen: 0.218
At first, this report may seem overwhelming, so we'll break it up and discuss it in pieces. First, Counts
CBC Help 363
reports that 250 respondents were used. Then, the count proportions for brand name are displayed:

Eclipse+, by Golfers, Inc. 0.188
High-Flyer Pro was the most preferred ball name and brand on average, being chosen 36.9% of the times
that is was presented and available for choice. Your client's ball name and brand, Long Shot, by
Performance Plus, was the least preferred at 15.6%. If your client markets their ball under the Eclipse+
name with the Golfers, Inc. brand, choice probability improves from 15.6% to 18.8%. These are ratio
quality data, so one might infer from the counts that using the Golfers, Inc. name and brand increases
the probability of choice by 21% (18.8/15.6 - 1).
Next comes the counts for Performance:

Respondents on average chose balls that flew 15 yards farther more than twice as often as those that
flew 5 yards farther. There appears to be a non-linear effect of performance on choice probability.
Likelihood of choice nearly doubles as performance increases from +5 yards farther to +10 yards farther
than the average ball. Much less is gained in terms of choice probability by increasing flight from +10
yards to +15 yards. A preliminary conclusion might be that your client should make sure their ball is
rated to travel at least 10 yards farther than the average ball. But improving the performance to travel 15
yards farther than the average ball might not be worth the extra manufacturing cost, if that cost is
significant.
Last comes price:

As expected, respondents prefer lower prices over higher ones. Probability of choice decreases
monotonically for each step increase in price.
To this point, you've only analyzed one-way (main effect) effects of attribute levels on choice. You can
further consider the probabilities of choice when a combination of two attribute levels are available for
choice. There are three tables of two-way probabilities in our study. Rather than look at all three, let's
examine the table (brand x price) that appears (by the Chi-Square statistic) to show the most promise of
being interesting (and reflect a potentially significant interaction effect) for our study:

364 SSI Web v8

Eclipse+, by G... $10.99 for package of ... 0.081
Long Shot, by ... $10.99 for package of ... 0.039
This table shows the probability of choice for each brand when it was shown at each price. This table is
more involved than the simpler one-way tables you examined. Therefore, it may be helpful to plot the
results with a graphics or spreadsheet package.
If you plot price on the x-axis and probability of choice on the y-axis, it would appear like the familiar
demand curves one learns about in economics. We should note, however, that counting analysis has
some drawbacks and inaccuracies associated with it. There are more accurate ways to generate
demand curves with CBC using the market simulator. Also, demand curves from CBC assume perfect
information, equal distribution, and other assumptions mentioned in the online Help.
The "pseudo demand curves" seem to suggest that the Eclipse+ name is preferred over all levels of price
to the Long Shot name. Also, the gap appears to widen slightly as prices increase—though we cannot
tell from this chart whether that is a significant or a chance occurrence.
After you have spent some time looking at the choice results for the randomized tasks, you might
consider also looking at the results for the fixed holdout tasks. Recall that we asked these fixed
scenarios to gauge preference for what may play out as actual future market scenarios. We also asked
these fixed questions so that we could see how well the market simulator predicts the preferences for
those two questions.
Recall that the first holdout task was asked as the seventh task, and the second holdout as the ninth.
To analyze responses for the first holdout, you need to use the Analysis | CBC Counts Report program
again, this time isolating only the seventh choice task. To do that, while in the CBC Counts dialog, in
the Settings + Task s Filter, only check the _Fixed1 task. Click Close to close the dialog. You will
only be interested in one-way (Main Effect) counts, so in the Settings... Advanced area, un-check the 2-
Way Interactions selection (leaving only Main Effects checked). Click OK and Compute Counts.... The
Counts for brand are displayed:

Eclipse+, by Golfers, Inc. *
Recall that we did not display Eclipse+ in the first holdout task, so it is marked with an asterisk. For
review, here is the configuration for the first fixed holdout task, with the choice probabilities listed:
31.2% 22.8% 20.4% 25.6%
High-Flyer Pro, by Smith Long Shot, by Magnum Force, by None: I Wouldn't

and Forester Performance Plus Durango Purchase Any of These

CBC Help 365

balls balls balls
If you isolate only the second fixed holdout task (by selecting only _Fixed_T2 in the Choice Task s to
Include dialog) and click Compute Counts..., you'll get the following probabilities for the second holdout
task:
18.0% 26.4% 30.4% 25.2%
Magnum Force, by Eclipse+, by Golfers, High-Flyer Pro, by Smith None: I Wouldn't

Durango Inc. and Forester Purchase Any of These


balls balls balls
Each of these choice probabilities are based only on 1 task x 250 respondents = 250 total choice tasks,
whereas the previous counting data was based on a total of 15 tasks x 250 respondents = 3,750 total
tasks.
In accordance with the previous conclusion we made based on the random choice tasks, the holdout
data suggest that there is marginal benefit from using the Eclipse+, by Golfers, Inc. name.
As a final note on counting analysis, one should not put too much emphasis in the None percentage.
We shouldn't conclude that since the None percentage is running at about 25% that roughly three-
quarters of these respondents would be expected to purchase a premium ball for their next golf outing. It
is our experience that respondents tend to exaggerate their likelihood to purchase or choose products in
survey research. The actual proportion of buyers that purchase premium balls would probably be
significantly less. (The Dual-Response None option available within CBC can help reduce the
overstatement of purchase intent.)
Now that you have spent some time learning about the Choice results using the Counts program, you are
ready to run logit analysis and construct a market simulation.
Analyzing the Choice Data Using Logit
Aggregate logit is an older and less advanced technique for analyzing choice results. But, it is a quick
way to analyze results, and a recommended starting point for learning about the essential characteristics
of your data. There are great advantages to Latent Class and Hierarchical Bayes analysis programs, but
for simplicity we'll focus on logit for this tutorial. (If you are licensed to use the Latent Class and
Hierarchical Bayes analysis systems, then these may also be run from within the SSI Web interface.)
Logit analysis estimates an effect, or logit "utility" for each level of each attribute. It also can be used to
estimate interaction effects. A utility refers to a degree of worth or preference for a product feature. As
with any complex analytical tool, we suggest you learn as much about the technique as you can, to
ensure that you interpret the results wisely. Much more information about logit analysis and logit utilities
can be found in the online help.
To compute logit, select Analysis | Estimate CBC Utilities - Other Methods | Logit.... First click the
366 SSI Web v8
Settings... button. You should provide the following settings (it should be there by default):
Respondent Filter (Include All Respondents)

Weights (Equal)
Tasks (15 Random, 0 Fixed)
After you have checked these settings and closed the CBC Logit Analysis Settings dialog, click
Estimate Utilities... The following report is displayed (some details in the report are omitted below for
clarity):
Number of Respondents 250
Log-likelihood for this model = -4600.39426

Log-likelihood for null model = -5198.60385
------------
Difference = 598.60385
Effect Std Err t Ratio Attribute Level

1 0.54407 0.03526 15.42997 1 1 High-Flyer Pro...
2 0.36260 0.03523 10.29098 1 2 Magnum Force, ...
3 -0.37368 0.04088 -9.14155 1 3 Eclipse+, by G...
4 -0.53299 0.04274 -12.46984 1 4 Long Shot, by ...
5 -0.47256 0.03278 -14.41561 2 1 Drives 5 yards...

6 0.12703 0.02914 4.35978 2 2 Drives 10 yard...
7 0.34553 0.02845 12.14726 2 3 Drives 15 yard...
8 0.65849 0.03540 18.59904 3 1 $4.99 for pack...

9 0.17237 0.03668 4.69901 3 2 $6.99 for pack...
10 -0.09275 0.03867 -2.39866 3 3 $8.99 for pack...
11 -0.73811 0.04475 -16.49244 3 4 $10.99 for pac...
12 0.00751 0.04141 0.18131 NONE
The column labeled "Effect" contains the utilities for each level of each attribute. The larger the utility,
the more preferred the level. The utilities sum to 0 within each attribute (they are zero-centered). These
utilities are used within CBC's market simulator to compute interest (share of choice) among products in
competitive scenarios.
The next column displays the standard errors for each logit effect. Earlier under Counting analysis, we
saw that Eclipse+ was preferred to Long Shot, but we could not tell at that point if the result was
statistically significant. Here, we note that the utility for Eclipse+ is 0.15931 utility points higher than
Long Shot (-0.37368 - -0.53299). We also note the standard errors for each of these utilities. The pooled
standard error for the difference between these two utilities is equal to the square root of the sum of the
squared standard errors, or sqrt (0.040882 + 0.042742) = 0.059143. To get the t-value, we divide the
difference in the two utilities by the pooled standard error 0.15931 / 0.059143 = 2.693. If we look this up
on a standard t-table, we see that this difference is significant at roughly the 99% confidence interval.
Therefore, we conclude that the Golfers, Inc. name is more preferred than the Performance Plus name.
The logit run above reflects main effects only (no interactions). The results from Counts suggested that
interaction effects might be significant. Adding interaction terms to the logit model (and the resulting
simulator) might significantly improve our ability to predict respondent choices. We will not go into detail
in this tutorial regarding how to test for significance of interaction terms in logit. To briefly summarize the
CBC Help 367
process, since we believe that changes in price may act differently on different brands, we decide to
rerun the logit model specifying the brand x price interaction. We compare the overall fit (log likelihood)
of the model before and after the inclusion of the interaction terms. With this data set, we find that the
interaction term improves the fit.
To include the interaction between brand and price in your logit run, click Analysis | Estimate CBC
Utilities - Other Methods... | Logit.... Click the Settings... button and then click Attribute Coding. At
the bottom of the dialog, click the Add... button to add interactions.
Two columns are displayed, with the attributes in both column. To specify the interaction between brand
and price, click brand on the left side of the dialog and price on the right side. Click OK. You are
returned to the CBC Logit Analysis Settings dialog. Click OK again to close the dialog. Then, click
Estimate Utilities.... A new solution is computed including main effects and interactions.
Running Market Simulations
Once you have computed logit utilities, you can use those within a market simulator to predict choices
for any combination of products defined using the attributes and levels you measured. You can test
what-if scenarios, conduct sensitivity analysis, and predict interest in different product concepts in
competitive scenarios. Sawtooth Software offers two market simulators: an web-based simulator
(designed for streamlined client use) and a more technically capable desktop-installed simulator (SMRT).
For the purposes of this tutorial, we will demonstrate the use of the SMRT simulator.
Launch the SMRT application (a different application from SSI Web). Click File | New and select a folder
and project name, and if prompted specify that the study type is CBC (Choice-Based Conjoint).
While still within SMRT, click Analysis | Run Manager and click the Import... button. At the bottom of
the dialog, under Files of type, change the setting to Generic Conjoint File (*.hbu).
Next, browse to your SSI Web project directory to the subfolder named CBCgolfexercise_logit where
SSI Web has written the logit utilities you computed earlier (saved as a CBCgolfexercise_logit.hbu
file). Double-click the CBCgolfexercise_logit.hbu file and the utilities are imported into SMRT. This
one file gives the SMRT market simulator all the attribute and level labels, as well as utilities for your
study.
To open the Market Simulator within SMRT, click Analysis | Market Simulator or double-click Mark et
Simulator from the study Navigator window.
The logit run that you imported is displayed in the Utility Runs list.
The first time you enter the Market Simulator for a study, the list of Simulation Scenarios is empty.
Create a simulation scenario by clicking Add....
When you click the Add... button, the Scenario Specification dialog is displayed.
Recall that one of the reasons for including fixed holdout tasks in your survey was to check the accuracy
of the market simulator. We use the simulator to predict the outcome of a choice task that wasn't
included when estimating the attribute utilities. To check this, you decide to specify the first fixed
product scenario.
The first step is to type a scenario name into the Name field, in the upper left-hand corner of the dialog.
Type Fixed Holdout Task 1.
368 SSI Web v8
Next, you'll specify the three products that were included in that choice task. The area you type the
products into looks like the grid of a spreadsheet. The first column is where you type the product label.
As you click the various columns associated with the attribute levels, the level codes will appear in the
window below to remind you what codes are associated with each attribute's levels.
The attribute level codes for the three products that were shown in the first holdout fixed task were:
Product Name Brand Performance Price

High-Flyer Pro 1 3 4
Long Shot 4 2 2
Magnum Force 2 1 3
Specify these three products in the product entry grid. To add new rows to the grid (for additional
products) click the Insert Product button.
After you have specified these three products, there are a few other things we should do before running
the simulation. First, we need to specify a simulation method. For the purposes of this tutorial, the
method we'll select is called Share of Preference. This method works well when the products that are
specified are basically unique, with little or no overlap in attribute levels used to define those products.
The three products we entered are unique in terms of brand, performance and price, so our situation
seems to fit this well. If some products shared a level specification (e.g. having the same performance),
we would probably favor using the default Randomized First Choice method.
Next, we need to tell the simulator to estimate a share of preference associated with the None product.
(We generally suggest not estimating share of preference for the None product; but in this case, where
we want to predict responses to a holdout task that included a None choice, it makes good sense). To
do so, click the Advanced Settings... button. (The settings on this dialog are described in more detail
in online Help).
Specify a "None" Weight of 1 and click OK to close the dialog.
Click OK again to close the Scenario Specification dialog and return to the main Market Simulator
dialog.
Choose a utility run to use for your simulation. You should have saved a run that computed main effects
plus the interaction between brand and price. Select that run by highlighting it in the list of Utility Runs.
To simulate shares for a scenario you have specified, place a check mark in the box next to the scenario
name in the Simulation Scenarios list, and then click Compute!.
The following report is displayed (we've inserted commentary within brackets):
Scenario: Fixed Holdout Task 1

Utility Run: CBCgolfexercise_logit.hbu Import
Average Utility Values Zero-centered

Rescaling Method: Diffs
Total
CBC Help 369

Eclipse+, by Golfers, Inc. -31.69
Long Shot, by Performance Plus -52.98
Drives 5 yards farther than the average ball -39.56


$8.99 for package of 3 balls -6.57
$10.99 for package of 3 balls -70.69
High-Flyer Pro... $4.99 for package of 3... -3.24

High-Flyer Pro... $6.99 for package of 3... -4.28
Magnum Force, ... $4.99 for package of 3... -14.57
Magnum Force, ... $6.99 for package of 3... -17.64
Eclipse+, by G... $4.99 for package of 3... -3.10
Eclipse+, by G... $8.99 for package of 3... -3.06
Eclipse+, by G... $10.99 for package of ... -5.03
Long Shot, by ... $10.99 for package of ... -34.38
None 2.64
<<The above average utilities are rescaled logit utilities using the zero-centered "diffs" method. The
diffs method rescales utilities so that the total sum of the utility differences between the worst and best
levels of each attribute across attributes (main effects) is equal to the number of attributes times 100.
Note: the attribute utilities are influenced by the number of respondents in the simulation and respondent
weighting, but are not affected by the product specifications you enter. After you have seen these once
for a particular group of respondents, you may choose to omit them in subsequent simulations by un-
check ing the Display Utilities box in the Scenario Specification dialog.>>
Product Simulation Settings

Simulation Mode: Simulation
Model: Share of Preference
None Weight: 1
Exponent: 1
Product Specifications
Brand Perf... Price
High-Flyer Pro 1 3 4
Long Shot 4 2 2
Magnum Force 2 1 3
<<Above are the product level codes you specified for the three products in this simulation. Below are
the simulated shares of preference (choice) for these products.>>
370 SSI Web v8
Shares of Preference for Products
High-Flyer Pro 30.05

Long Shot 21.55
Magnum Force 22.41
None 25.99
In the table below, we've summarized the actual and predicted choices (rounded to the nearest tenth)
shares for the first holdout task. Notice that the simulated shares of preference are very similar to the
actual share of choices respondents gave to fixed holdout task number one (which we analyzed earlier
using Counts):
Product Name Actual Choice Simulated Choice Absolute Error

Shares Shares
High-Flyer Pro 31.2 30.1 1.1
Long Shot 22.8 21.6 1.2
Magnum Force 20.4 22.4 2.0
None 25.6 26.0 0.4
To quantify how closely the simulated shares match the actual choice shares, we've computed the
absolute value of the difference between the actual and simulated shares in the last column. If we
average the errors in the last column, we find the Mean Average Error (MAE) is 1.2. (We should note
that the MAE for this example is much lower than is generally observed when comparing simulation
predictions versus actual holdout shares).
We won't take the space here to show the MAE computation for holdout task #2, though you may decide
to specify another simulation scenario and compute predicted shares for that task.
You can use MAE (or Mean Squared Error) to compare the results of different simulation models (i.e.
Logit versus Latent Class or HB). You can also use it to tune the scaling parameter (exponent) or other
advanced settings covered in more depth in the online help.
The Market Simulator is a powerful tool for testing nearly an unlimited number of possible market
scenarios. You can use the simulator to answer the strategic questions related to the fictitious golf ball
situation posed at the beginning of this unit.
For example, the main question facing Performance Plus was whether they could hope to compete with
the two existing performance balls in this market. The simulation results suggest that if they were able
to provide the specified performance at the simulated price and could achieve the level of distribution and
awareness of the existing brands (whose specifications and prices didn't change) they could expect a
market share roughly equal to Magnum Force and lower than the leader, High-Flyer Pro.
A secondary question was whether they should market their new golf ball under the "Eclipse+, by
Golfers Inc." name. If you specify the second holdout task as a new simulation scenario, the following
shares result (displayed next to the previous simulation for fixed holdout task #1):
CBC Help 371
Simulated Shares of Choice
Fixed Task 1 Fixed Task 2

High-Flyer Pro 31.2 High-Flyer Pro 28.3
Long Shot 22.8 Eclipse+ 26.2
Magnum Force 20.4 Magnum Force 21.1
None 25.6 None 24.5
By marketing their ball under the "Eclipse+, by Golfers, Inc." name instead of "Long Shot, by
Performance Plus", share of choice increases from 22.8 to 26.2, representing a 15% relative increase. If
you have a good memory, you'll note that this conclusion is similar to what we inferred from Counts data
and by comparing the actual choices of the two fixed holdouts. The simulated shares, however, should
be both more reliable and more accurate. They are not subject to some inherent weaknesses of
counting analysis and they leverage much more data (15 randomized tasks) than isolating only the
choices given to the two fixed holdout tasks.
It is important to remember that the shares of preference resulting from conjoint predictions are not
equivalent to actual market shares, and often look quite different. Many other factors in the real world
factor into market shares and cannot be measured and reflected solely by conjoint data. Conjoint
assumes perfect information, equal distribution and availability, and that each respondent is in the
market and able to purchase. Conjoint results reflect the potential market acceptance, given proper
promotion, distribution and time.
372 SSI Web v8
2.2 Designing CBC Studies

2.2.1 CBC Questionnaires and Designs
CBC creates Web-based, CAPI, or paper-and-pencil interviews. CBC comes with a 50-question CiW
system, that provides the capability of asking up to 50 other standard survey questions (e.g. numerics,
open-ends, check-box). One can purchase additional capacity for standard survey questions by
purchasing a larger CiW license.
The Choice Question
A CBC question is often referred to as a "task." A set of products (concepts) are displayed on the
screen, and in the standard formatting the respondent chooses among the concepts. For example:
Choosing the best concept among a set of concepts is the standard approach that is advocated by most
experts in the industry. Other options supported by the CBC software include:
Constant-sum (allocation), where respondents type a number beneath concepts based on relative
preference, percent of volume, or volume
Best-Worst CBC (respondent selects best and worst concepts within each task--requires
Advanced Design Module license)
Another format that is gaining popularity is to ask about the "none" concept in a second-stage
question, called Dual-Response None.
Two important decisions to make when constructing choice tasks are 1) how many concepts to present
per task, and 2) how many total tasks to ask. These are sometimes difficult issues to answer and
ongoing research is helping to define guidelines and understand the consequences.
From a statistical viewpoint, choice tasks are not a very efficient way to learn about preferences.
Respondents evaluate multiple concepts, but only tell us about the one they prefer (in traditional discrete
choice). We don't learn how strong that preference is relative to the other product concepts. Showing
more product concepts per screen increases the information content of each task, though it increases
task difficulty. Recent research has shown that respondents are quite efficient at processing information
about many concepts. It takes respondents considerably less than twice as long to answer choice
CBC Help 373
tasks with four concepts than with two concepts. Recent research has also pointed out the value of
forcing at least some levels within attributes to repeat within each choice task (level overlap). This can
be accomplished by showing more products on the screen than levels available per attribute. In general,
for marketing applications, we recommend showing around four to six concepts per task. However, there
may be instances (e.g. a beverage study with 25 brands plus price), where showing more product
concepts per screen is appropriate and more realistically portrays the actual buying situation. With the
Advanced Design Module for CBC, you can display up to 100 product graphics on the screen, presented
as if they were resting on "store shelves." It is worth noting that some choice contexts (such as
healthcare outcomes) involve challenging and often unfamiliar choice scenarios and researchers in this
area have sometimes found that asking respondents to evaluate more than two concepts within a choice
task is inordinately difficult.
With randomized choice designs, given a large enough sample size, we could model preferences at the
aggregate level by asking each respondent just one choice task. In practice, researchers recognize that
individuals are expensive to interview and that it makes sense to collect more information from each
person in CBC studies. With multiple observations per respondent, one can model across-respondent
heterogeneity in preferences, which leads to more accurate choice simulators.
In a 1996 meta-analysis of 21 CBC data sets, we found that multiple observations per respondent are
quite valuable and that respondents could reliably answer up to at least 20 questions, and perhaps even
more. However, we discovered that respondents process earlier tasks differently from later questions.
Respondents paid more attention to brand in the first tasks and increased their focus on price in later
questions. (See Johnson and Orme's article entitled "How Many Questions Should You Ask in Choice-
Based Conjoint?" available for downloading from the Technical Papers section of our home page: http://
www.sawtoothsoftware.com/support/technical-papers).
Over the last five years, other researchers have examined the same issues and have suggested that
respondents (especially internet and panel respondents) are probably less patient and conscientious with
long CBC surveys involving many attributes than respondents of 20 years ago. They are perhaps quicker
to resort to simplification heuristics to navigate complex CBC tasks and responses beyond about the
tenth task don't seem to be revealing much more about each respondent's choice process. These
authors have recommended using fewer than a dozen tasks, even if the attribute list is relatively long.
They have recommended meeting the information requirements involved with long attribute lists by
increasing the sample size.
With CAPI or paper-based CBC research, we recommend asking somewhere in the range of 8 to 15
choice tasks. With Web interviewing, fewer tasks might be appropriate if there is an opportunity for
increased sample sizes (which often is the case). If estimating individual-level utilities using CBC/HB,
we'd recommend at least six choice tasks to achieve good results based on simulated shares, but about
10 choice tasks or more for developing robust predictions at the individual level (again assuming a typical
design). Before finalizing the number of concepts or tasks to be asked, we urge you to pretest the
questionnaire with real respondents to make sure the questionnaire is not too long or overly complex.
With CBC, all of the tasks are tied to the same layout, including header text and footer text. You cannot
change the number of product concepts or other settings within your choice tasks midstream (though
you can include multiple CBC exercises within the same study, which can be interwoven). In addition to
the regular choice tasks, you can add tasks that have fixed designs ("holdout" tasks) described later.
You can also insert other text screens or generic survey questions between choice tasks.
The "None" Option
Choice-based conjoint questions can be designed to have a "None" option, sometimes referred to as the
374 SSI Web v8
constant alternative. It is argued that the None option in CBC tasks better mimics the real world, since
buyers are not required to choose products that don't satisfy them. The None option can also be used to
reflect a status quo choice, such as "I'd continue to use my current long-distance service provider."
If the None option is present, a separate utility weight is computed for the None parameter (under CBC/
HB, latent class, or logit analysis). The more likely respondents are to choose None relative to product
concepts, the higher the utility of the None option. The None parameter can be used in market
simulations to estimate the proportion of respondents that would not choose any of the simulated
product concepts. You can include more than one constant alternative in the questionnaire if using the
CBC Advanced Design Module.
In general, we recommend including None in CBC questionnaires, but paying less attention to (or
completely ignoring) it in market simulations. The propensity to choose None can be a telling measure
when comparing groups or individuals. We suggest that percent of Nones in Counts or simulations be
viewed in a relative rather than an absolute sense.
The use of the "None" concept in partial-profile CBC studies (only supported by the Advanced Design
Module for CBC) is problematic. The None weight varies significantly depending on how many attributes
are displayed in the partial-profile task. Patterson and Chrzan (2003) showed that as the number of
attributes increases, the propensity to choose None also increases.
Randomized Versus Fixed Designs
Two schools of thought have developed about how to design and carry out choice-based conjoint studies.
1) Some researchers prefer fixed orthogonal designs. Such designs often employ a single version
of the questionnaire that is seen by all respondents, although sometimes respondents are divided
randomly into groups, with different groups receiving different questionnaire versions (blocks, or
subsets of the larger fixed design). Fixed orthogonal designs have the advantage of high efficiency
at measuring main effects and the particular interactions for which they are designed. It is
interesting to note that for imbalanced asymmetric designs (where there are large differences in the
numbers of levels among attributes) fixed orthogonal designs often can be less efficient than
random designs.
2) Other researchers, particularly those accustomed to computer-administered interviewing, prefer

controlled random designs. The CBC software, for example, can design interviews with nearly-
orthogonal and leve-balanced designs, in which each respondent receives a unique set of questions.
Attribute and concept order can be randomized across respondents. Such designs are often
slightly less efficient than truly orthogonal designs (but can be more efficient with asymmetric
designs), but they have the offsetting advantage that all interactions can be measured, whether or
not they are recognized as important at the time the study is designed. Randomized plans also
reduce biases due to order and learning effects, relative to fixed plans.
CBC can administer either fixed, random designs, or designs generated from an outside source (via the
ability to import a design from a .csv file). If a fixed design is chosen, then the researcher must specify
that design (usually by importing it). If a randomized design is chosen, then it will be produced
automatically and saved to a design file that is later uploaded to your server if using Web-based data
collection, or to another PC if using CAPI-based interviewing. If using paper-based interviewing, a limited
set of questionnaire versions (a subset of the potential random plan) is often used. Most CBC users
have favored the ease of implementation and robust characteristics of the randomized approach.
We have tried to make CBC as easy to use and automatic as possible. The researcher must decide on
CBC Help 375
the appropriate attributes and their levels and compose whatever explanatory text and/or generic survey
questions are desired during the interview. Absolutely no "programming" is involved. Every aspect of
designing the interviews and conducting the analysis is managed through the point-and-click Windows
interface. Thus, we hope that CBC will make choice-based conjoint analysis accessible to individuals
and organizations who may not have the statistical or programming expertise that would otherwise be
required to design and carry out such studies.
Random Design Strategies
Fixed experimental designs may be specified in CBC, but most users will rely on one of the four
randomized design options. When CBC constructs tasks using its controlled randomization strategies,
some efficiency is often sacrificed compared to strictly orthogonal designs of fixed tasks. But, any loss
of efficiency is quite small, usually in the range of 5 to 10%. However, there are important compensating
benefits: over a large sample of respondents, so many different combinations occur that random designs
can be robust in the estimation of all effects, rather than just those anticipated to be of interest when the
study is undertaken. Also, potential biases from learning and order effects can be reduced.
The earliest versions of CBC offered two randomized design options: complete enumeration and the
shortcut method. Though we refer to these as "randomized designs," these designs are chosen very
carefully, as will be explained.
The complete enumeration and shortcut methods generate designs conforming to the following
principles:
Minimal Overlap: Each attribute level is shown as few times as possible in a single task. If an
attribute's number of levels is equal to the number of product concepts in a task, each level is
shown exactly once.
Level Balance: Each level of an attribute is shown approximately an equal number of times.
Orthogonality: Attribute levels are chosen independently of other attribute levels, so that each
attribute level's effect (utility) may be measured independently of all other effects.
Minimal overlap has benefits from a statistical standpoint in terms of increasing precision for main
effects; but it is suboptimal for measurement of interactions between attributes. Also, we've recently
recognized that including level overlap can lead to more thoughtful responses that reveal deeper
preference information than minimal overlap. To help illustrate that point, consider the extreme case of a
respondent who requires Brand A. If only one Brand A is available per task, the respondent's only option
is to choose Brand A each time, and such questions are completed in a matter of a few seconds. But, if
two Brand A products sometimes appear in the design, then the respondent is encouraged to ponder and
express what additional aspects (other than brand) affect the decision.
Level overlap can be added to designs simply by increasing the number of concepts per task. For
example, if each attribute has four levels, then showing six concepts per task means that two levels
must be repeated in each choice task. Six concepts per task also provides more raw information for
estimating part-worth utilities than four concepts per task. (Yet another reason to advocate showing
more rather than fewer concepts per task.)
Another way to add level overlap within your designs is to use the Balanced Overlap or purely Random
methods.
Below, we describe the design methods in more detail and conclude with more advice on when to use
376 SSI Web v8
the different design approaches.
Complete Enumeration:
The complete enumeration strategy considers all possible concepts (except those indicated as
prohibited) and chooses each one so as to produce the most nearly orthogonal design for each
respondent, in terms of main effects. The concepts within each task are also kept as different
as possible (minimal overlap); if an attribute has at least as many levels as the number of
concepts in a task, then it is unlikely that any of its levels will appear more than once in any
task.
Complete enumeration may require that a very large number of concepts be evaluated to
construct each task, and this can pose a daunting processing job for the computer. The base
CBC software permits up to 10 attributes, with up to 15 levels each. Suppose there were 4
concepts per task. At those limits the number of possible concepts to be evaluated before
displaying each task would be 4 x 15^10 = 2,306,601,562,500!
This is far too great a burden for even the fastest computers today and would take a great deal
of time for the design generator to produce a design file. Generally, CBC design specifications
in practice are more manageable and speed of computation is usually not an issue.
Furthermore, with CBC, the researcher generates the experimental plan prior to conducting data
collection. Thus, respondents do not need to wait for a new design to be generated--they are
simply randomly assigned to receive one of the many questionnaire versions previously
generated.
The time required to compute designs under Complete Enumeration is more sensitive to the
number of attributes in your study than to the number of levels for attributes. If you have just a
few attributes, you should experience little delay in design computation, even if one of your
attributes has scores of levels.
Shortcut Method:
The faster "shortcut" strategy makes a much simpler computation. It attempts to build each
concept by choosing attribute levels used least frequently in previous concepts for that
respondent. Unlike complete enumeration that keeps track of co-occurrences of all pairs of
attribute levels, the shortcut strategy considers attributes one-at-a-time. If two or more levels of
an attribute are tied for the smallest number of previous occurrences, a selection is made at
random. With the shortcut method, as well as with complete enumeration, an attempt is made
to keep the concepts in any task as different from one another as possible (minimal overlap).
When there is more than one less-frequently-used level for any attribute, an attempt is made to
choose one that has been used least in the same task.
Designs composed using complete enumeration are of high quality, and those composed by the
shortcut method are also quite acceptable.
Balanced Overlap Method:
This method is a middling position between the random and the complete enumeration
strategies. It permits roughly half as much overlap as the random method. It keeps track of the
co-occurrences of all pairs of attribute levels, but with a relaxed standard relative to the complete
enumeration strategy in order to permit level overlap within the same task. No duplicate
concepts are permitted within the same task.
CBC Help 377
The balanced overlap method takes about the same time to compute as complete enumeration
and is slower than the random or shortcut methods.
Random Method:
The random method employs random sampling with replacement for choosing concepts.
Sampling with replacement permits level overlap within tasks. The random method permits an
attribute to have an identical level across all concepts, but it does not permit two identical
concepts (on all attributes) to appear within the same task.
The random method computes about as quickly as the shortcut method. Unless the primary
goal of the research is the study of interaction effects, we generally do not recommend using the
purely Random method.
Recent research suggests that including level overlap in CBC designs is valuable. One can force
some degree of level overlap even though using Complete Enumeration or Shortcut methods
simply by increasing the number of concepts per task (to exceed the number of levels per
attribute). If you cannot increase the number of concepts per task, balanced overlap directly adds
a modest degree of level overlap and may generally perform a bit better than complete
enumeration. We strongly encourage CBC users to use Balanced Overlap and/or to achieve more
level overlap in their CBC questionnaires by displaying more concepts per task than the largest
number of levels in any one attribute.
CBC's Design File
CBC automatically creates a design file that is
a) uploaded to the server if using web-based data collection,

b) is copied to other PCs if using CAPI-based data collection, or
c) is used to generate a questionnaire file if using paper-based data collection.
During the process, you are asked to specify how many versions of the questionnaire should be written
to that design file. For computer-based interviewing, we generally recommend that you include at least
100 versions, as there are benefits to fielding many versions of the questionnaire. Optimally, each
respondent would receive his/her own version of the questionnaire. However, there are diminishing
returns and including more than about a dozen versions often is of very little to no additional practical
benefit. For that reason, we limit the number of versions that can be placed in the design file to 999. If
you have more than 999 respondents, this means that some respondents will by necessity receive the
same designs. For all practical purposes, once there are so many unique versions of the questionnaire
in the total design pool, this poses no harm.
Suggestions for Randomized Designs
Generally, we recommend asking about 8 to 15 choice tasks, though this recommendation may vary
depending on the attribute list and sample size. We prefer random task generation (especially Balanced
Overlap). We caution against questionnaires with no level overlap. If a respondent has a critical "must-
have" level, then there is only one product per choice task that could satisfy him/her. One can
accomplish some level overlap by simply displaying more product concepts than levels exist per
attribute. But, the default "Balanced Overlap" design methodology ensures at least a modest amount of
level overlap.
378 SSI Web v8
Another reason to include at least some degree of overlap in the CBC designs is when interaction terms
are of interest. Overlap for an attribute can be added to a design simply by using more concepts than
attribute levels in tasks.
In summary, we suggest using balanced overlap (or the complete enumeration/shortcut methods, if some
level overlap can be added to the tasks by virtue of showing more concepts on the screen than there are
levels per attribute) for main-effects only designs. If detecting and measuring interactions is the primary
goal (and sample size is relatively large), then the purely random approach is favored. If the goal is to
estimate both main effects and interactions efficiently, then overlap should be built into the design, at
least for the attributes involved in the interaction. Using more concepts than attribute levels with
complete enumeration, or utilizing the compromise balanced overlap approach would seem to be good
alternatives.
Importing Other Designs
Some researchers will receive a specific design generated by a design specialist. You can import such
designs into CBC.
Mixed Overlap Designs
Some researchers may want to force some attributes to have minimal overlap and other attributes to
have some overlap (i.e. using Balanced Overlap). Imagine a situation in which the first attribute is to have
minimal overlap (each level represented once per task) and the remaining attributes are to be designed
using Balanced Overlap. The researcher could generate a design using all but the first attribute under
Balanced Overlap, and export the design to a .csv file. Then, the researcher could modify the .csv file
using Excel to insert an attribute in the first position where each level appears once per task. The last
step is to import that new design into a CBC study that reflects all attributes.
One could use a similar approach to combine designs where multiple attributes were generated using
Balanced Overlap and multiple attributes were generated using Complete Enumeration. Of course, care
should be taken to test the design efficiency of the final design.
CBC Help 379
2.2.2 Defining Attributes and Levels

The base CBC software is limited to 10 attributes with at most 15 levels of an attribute, though we
encourage you to develop attributes that have about 5 or fewer levels whenever possible. (The advanced
design module extension of CBC allows up to 100 attribute, each with 254 levels).
(Hint: you can select a list of attributes or levels from Word or Excel and paste into SSI Web using the
Paste list member(s) from the clipboard icon. This can save a great deal of time.)
An attribute is a characteristic of a product (e.g. color) which can take on various levels (e.g., red, yellow,
blue). Every attribute must have at least two levels. The underlying theory of conjoint analysis holds that
a buyer places a certain part-worth (or utility value) on each attribute level, and that the overall utility of
any product is obtained by summing up the part-worth values of its specific attribute levels.
In conjoint experiments, we show respondents product concepts described using different combinations
of attribute levels and ask them to somehow express their preferences for those concepts. One way of
thinking about conjoint analysis is that we are conducting a designed experiment for each respondent to
see how preferences vary as we change attribute levels.
After observing how respondents evaluate products in response to changes in the underlying attribute
levels, we can estimate the impact (part-worth) each attribute level has upon overall product preference.
Once we learn respondents' preferences for the various attribute levels, we can predict how buyers might
respond to any potential combination of levels in our study, whether or not that actual product was ever
displayed during the interview.
Conditional Relationships…
(Not available within ACA studies.)
The conditional relationships button lets you associate conditional prices or graphics with one or
more attributes in your study.
Conditional Graphics: Sometimes, you may want to associate a graphic with an attribute or
multiple attributes. For example, if a combination of brand and form factor make up a graphic,
one can specify the two attributes separately and associating a conditional graphic with their
combined levels. This permits estimation of separate effects for brand and form factor.
Guidelines
In generating attributes and levels, consider the following guidelines:
1. Attributes should be independent. It is therefore important to economize; including attributes with

overlapping meanings is wasteful and can lead to biased part-worth utilities.
Furthermore, levels for related attributes often cannot combine naturally and freely with one another.
Though it can lead to more realistic interviews, it is often detrimental (and sometimes fatal) to prohibit
levels from occurring with others.
2. Levels within each attribute should be mutually exclusive. This point becomes clear when you
specify products using the market simulator (during the analysis phase) and are forced to choose only a
380 SSI Web v8
single level from each attribute.
Consider the following attributes for a study on optional automobile features:
Optional Features:
Sunroof
GPS (Global Positioning System)
Extended warranty
This formulation doesn't permit simulating preference for a car that has both a Sunroof and a GPS.
Similarly, we could not simulate preference for an automobile that had none of these features. There are
two ways to resolve this quandary:
a) Create an attribute with levels representing all potential combinations of these features.
This results in an eight-level attribute, if you include the option that none of these features is
available.
b) Formulate three separate attributes each with two levels: (No Sunroof, Sunroof), (No GPS,
GPS), (No Warranty, Warranty).
Option (a) adds seven parameters to the model (after dummy-coding) and forces the measurement of an
explicit three-way interaction. With the more complex model definition, we can investigate whether there
are diminishing returns by bundling the features. By splitting the options into three distinct binary
attributes (Option b), only three parameters are added to the model; however, interactions are not
automatically measured.
3. Attribute levels should cover the full range of possibilities for existing products as well as products
that may not yet exist, but that you want to investigate. Although the market simulator allows you to
extrapolate and interpolate, only linear interpolation and extrapolation are possible. Although
interpolation is likely to produce acceptable results, extrapolation is prone to error and should be
avoided. One way to ensure that you are including the appropriate levels and ranges is to ask your client
to specify ahead of time the market simulations to be run during the analysis phase of your study. That
exercise can often reveal weaknesses in your attribute specifications.
4. Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions

is one of the most common mistakes. The problem usually begins when either the analyst (or the
analyst's client) notices that some product combinations displayed during the interview are not realistic,
given what currently exists in the market. Sometimes a product is shown with all the best features at
the lowest price; or two attribute levels that would not naturally occur in the real world are paired
together. The inclination is simply to prohibit such combinations. We urge you to exercise restraint
when considering prohibiting levels from occurring with one another.
Too many prohibitions can lead to confounded effects and the complete inability to calculate stable
utilities. It is better to prompt respondents that they will see combinations during the interview that are
not yet available in the market or that seem unlikely. You can urge respondents to answer as if these
products were actually available today.
There are other strategies for dealing with prohibitions. Consider the example below with brands of soda
and package types:
Brand:
Sawtooth Spritz
Kong Kola
CBC Help 381
Splut
Package Type:
2-liter bottle
6-pack of 12-oz cans
Suppose that Splut was only available in 6-packs of cans. Furthermore, you are displaying actual
pictures of the products, and thus can only display actual (not potential) products. Rather than define a
prohibition between Splut and the 2-liter bottle, it would make more sense to combine these two
attributes as a single attribute with five levels:
Sawtooth Spritz in a 2-liter bottle

Sawtooth Spritz in a 6-pack of 12-oz cans
Kong Kola in a 2-liter bottle
Kong Kola in a 6-pack of 12-oz cans
Splut in a 6-pack of 12-oz cans
Under this strategy, no prohibitions are required.
5. The number of levels you choose to define an attribute can have a significant bearing on the results.
The first concern has been called the "Number-of-Levels Effect." All else equal, attributes defined on
more levels tend to get more importance.
The second concern is that you limit the number of levels on which quantitative attributes are described.
We suggest not including more than about five levels to describe attributes such as price or speed. It's
usually better to have more data at each price point than to have thinner measurements at more price
points. Measuring too many points along a quantitative function can result in troublesome reversals. If
you cover the entire range of interest with fewer levels, you can interpolate between levels within the
market simulator to get finer granularity if needed.
6. Attributes that cannot be adequately described in words should be represented in multimedia. But if
attributes do not require multimedia to adequately communicate their properties, it would probably be a
mistake to make them multimedia. Though the interview might appear more attractive, it might bias the
results in favor of multimedia attributes.
382 SSI Web v8
2.2.3 Specifying Prohibitions

This dialog lets you specify which combinations of attribute levels (or attributes altogether) may not be
shown together within the same product concept in conjoint questions.
For standard (within-concept) prohibitions, select an attribute in the upper pane and a second attribute in
the lower pane. A grid appears, in which you can check which level combinations should be prohibited.
For advanced prohibitions for CBC studies (described below), select either the Advanced Prohibitions or
Null Level Prohibitions tabs (requires license to the Advanced Design Module).
A Warning
Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions is

one of the most common mistakes. The problem usually begins when either the analyst (or the analyst's
client) notices that some product combinations displayed during the interview are not realistic, given what
currently exists in the market. Sometimes a product is shown with all the best features at the lowest
price; or two attribute levels that would not naturally occur in the real world are paired together. The
inclination is simply to prohibit such combinations. We urge you to exercise restraint when considering
prohibiting pairs.
Full-profile CBC studies are particularly sensitive to attribute prohibitions. Too many prohibitions can
lead to imprecise part-worth estimation. It is better to prompt respondents that they will see
combinations during the interview that are not yet available in the market or that seem unlikely. You can
urge respondents to answer as if these products were actually available today.
If you use any prohibitions, you must make sure to test your design and check the efficiency of the
estimates for the parameters of interest.
How Many Prohibitions Can I Safely Add?
A common question that users ask is "how many prohibitions can I specify without seriously damaging
the results of my study?" This cannot be answered without more information. For it is not the sheer
number of prohibitions but the specific pattern of prohibitions that more directly affects the degree of
correlation among the attribute levels in the design matrix and thus the design efficiency.
Assume that the researcher wants to specify 3 prohibitions between a 2-level attribute and a 3-level
attribute. There are just 6 possible combinations that can occur when combining those two attributes. If
the researcher prohibits 3 of those combinations from occurring, this eliminates half of the possible
combinations between those attributes. There would be less damage to the efficiency of the design if
three total prohibitions were allocated across three separate attribute combinations (one per pair of
attributes).
As a side note, the researcher in this instance may decide to specify the three non-prohibited
combinations of those two attributes as a single attribute, rather than as two separate (and not
independent) attributes. The main drawback of this approach is that after combining these two
attributes, the researcher will not be able to compute the attribute importance or the relative part-worths
of each attribute independently.
CBC Help 383
Advanced Prohibitions (CBC Advanced Design Module Only)
For many years, our CBC software only supported within-concept prohibitions between the levels of pairs
of attributes. In other words, within the same product concept, one level of one attribute would be
prohibited from appearing with one level of another attribute. For example, we would prohibit Brand A
from appearing with the color Green within the same product concept. Starting in v6 of CBC, we began
to support the following advanced prohibitions:
n-way prohibitions
between-concept prohibitions
null-level prohibitions
N-Way Prohibitions
You can define prohibitions that simultaneously involve up to n attributes, where n is the number of
attributes in your study. For example, a 3-way prohibition involves prohibiting the combination [Brand A,
Style C, Green] from appearing (where brand, style, and color are three separate attributes).
Between-Concept Prohibitions
You can define prohibitions restricting what attribute combinations may occur in competition with another
within the same choice task. For example, we may wish to prohibit a [Brand A, Style C] product from
being shown within the same choice task in competition with a [Brand A, Style B] product. These may
be defined as 1-way to n-way prohibitions.
Null-Level Prohibitions
Some researchers include "blank" levels within attributes (often by using the HTML syntax for
the attribute level text, which displays a blank space on the screen). The null (blank) level reflects the
absence of the attribute. However, if many attributes have null levels, the analyst may wish to limit how
many of them appear together in their null state. CBC software lets you specify a minimum and
maximum number of null levels that are allowed to appear together within a concept (though specifying
that the number of null levels permitted should be constant leads to confounded designs and is not
permitted). You must inform the software of which attribute levels are null levels.
As you place constraints on the design to enforce a narrow range of permitted null levels per concept,
the design efficiency is reduced. Furthermore, the greater the null level constraints, the worse that
Shortcut, Complete Enumeration, and Balanced Overlap methods will perform relative to the purely
Random method. With quite constrained solutions, the Random design method performs best. For this
reason, it is very important that you test different design methods and compare the overall efficiency of
the designs when using null-level prohibitions.
Effect of Prohibitions on Design Efficiency
Within-concept prohibitions are generally more damaging to design efficiency (the ability of the
questionnaire to estimate the part-worth utility parameters with good precision) than between-concept
prohibitions. A higher-order prohibition (for example, a single 3-way, within-concept prohibition) is less
damaging to the precision of utility level estimates than a 2-way within-concept prohibition. This follows
logically, since a single 3-way prohibition leaves many more attribute level combinations available to
display than a single 2-way prohibition, which is more restrictive.
Please note that to estimate interactions between two attributes, all possible level combinations of those
two attributes must have been displayed within concepts to respondents (across all respondents).
384 SSI Web v8
Therefore, specifying a 2-way, within-concept prohibition would preclude the ability to estimate an
interaction effect between the two attributes involved in that prohibition. However, specifying a single 3-
way prohibition would not prohibit estimating 2-way interaction effects (since all 2-way combinations of
attribute levels would still be represented).
CBC Help 385
2.2.4 Attribute Interactions

Most conjoint methods assume "main effects only" estimation. These are simple additive models in
which the value of a product concept is equal to the simple additive sum of its parts (the attribute levels).
The part-worths for each attribute are measured independently of all others. Main effects (the
independent effect of each attribute) tend to capture the vast majority of variance to be explained in
choice data. However, there are instances in which the presence of interactions between attributes
makes the simple model not as accurate as it could be. Adding interaction terms to the main effects
model, in these instances, could add important explanatory power and improve predictive accuracy.
In a Sawtooth Software conference, the following explanation of an interaction was provided, "Interactions
occur when the combined effect of two attributes is different from the sum of their two main effect utilities.
For example, being stranded on a deserted island is pretty bad, say it has a utility of -40. Attending a
party hosted by cannibals is also a bad thing, say with a utility of -50. But attending a party hosted by
cannibals on a deserted island could be altogether worse, in grisly sorts of ways (utility -250)." ("An
Overview and Comparison of Design Strategies for Choice-Based Conjoint Analysis," Keith Chrzan and
Bryan Orme, 2000 Sawtooth Software Conference.)
One of the strengths of CBC is its ability to estimate the effects of interactions. We think interactions
would be found more often using conjoint-related methods if the sample sizes, experimental designs, and
estimation methods allowed us to measure them with more precision. We think that CBC provides an
excellent way to produce relatively precise results when attribute interactions are of concern.
It can be demonstrated that interaction effects can be revealed through choice simulations using main-
effect models that do not directly model interaction terms, if the source of the interactions between
attributes is principally due to differences in preference among groups or individuals. If Latent Class or
HB are used to model main-effects, the need to additionally model interactions should be lessened.
Either way, the randomized choice designs offered in CBC are appropriate for either aggregate or
disaggregate analysis; for main-effects or models that involve interactions. We recommend you be on
the lookout for significant interactions--even if using disaggregate analysis.
To obtain strongest results for interactions, some level overlap should be incorporated into your CBC
designs. See the section entitled CBC Questionnaires and Designs for more information.
Be careful about adding interaction terms to the model, as they come at a cost and can easily lead to
overfitting. In other words, the extra parameters within the model may not be worth the amount of extra
fit to the data and worse predictions may result for market simulations involving new choice scenarios. It
would be a mistake to simply add all interaction terms between attributes to the model simply because
the analyst wanted to ensure that if interaction effects existed that they were captured.
386 SSI Web v8
2.2.5 Response Type

Discrete Choice
The most common way of asking respondents to answer a CBC task is to ask them to select their one
highest-utility product concept (Discrete Choice). This is a robust approach and has been the standard
for three decades of CBC research.
Best-Worst Choice
Best-Worst Choice asks respondents to select both the highest and the lowest utility product concepts
within each task. Some researchers have argued in favor of this format, for at least two reasons:
1) It is efficient to ask respondents to give us more information per choice task. It takes only a little
bit more time to indicate both best and worst concepts rather than just the one best or the one worst
concept.
2) In some kinds of research contexts (such as healthcare outcomes for the treatment of cancer),
the respondent is perhaps even more interested in avoiding exceptionally bad outcomes than
achieving exceptionally good ones. Thus, asking the respondent to select the worst concept in
addition to the best would seem useful.
However, there are some drawbacks to the Best-Worst format.
1) It isn't well known whether the additional time spent asking respondents to indicate worst
concepts in each task wouldn't be better spent asking respondents to answer a few more best-only
choice tasks. For example, which would be better: 10 Best-Worst tasks or 13 Best-Only tasks?
Both approaches would likely take about the same amount of time to complete.
2) The magnitude of the response error made in Best choices is usually different from Worst
choices. Also, academics (Allenby et al.) have recently raised the issue that some respondents
orient themselves by choosing the worst concept first and the best concept last. And, depending on
the order of choice (best first or worst first) the errors involved in making the judgments differ, as well
as the context in terms of number of alternatives per set. Allenby et al. have raised concerns
regarding whether it is appropriate to treat the choices equally (in terms of equal error magnitude, or
scale factor, and task context) when estimating part-worth utilities.
For now, we don't have a strong opinion on the matter, but offer the best-worst choice option because
many researchers are interested in using this approach for CBC studies. We hope to see more research
on the suitability of the approach vs. standard discrete choice.
For utility estimation, the best and worst tasks are formatted as independent choice tasks. The design
matrix for best tasks is coded exactly as in standard Discrete Choice (best-only format). The design
matrix is simply multiplied by -1 for worst tasks. (This is the same procedure as is used for our MaxDiff
utility estimation.)
Constant Sum/Chip Allocation
Constant Sum/Chip Allocation allows respondents to indicate how many of each concept they would
choose. Often, researchers ask respondents to allocate points (chips) across the concepts that sum to
a fixed amount, such as 10. But, it is possible to relax that requirement, so the points do not sum to
CBC Help 387
some constant. (In either case, the HB and Latent Class estimation routines by default percentage the
chips to 100% within each choice task, so there isn't a formal recognition that a respondent may have
allocated 20 chips in task 1, but 13 chips in task 2.)
Constant Sum has the benefit of collecting a lot more statistical information for each choice task. But, it
is well known that respondents often struggle with providing reliable chip-allocation data. So, we
recommend caution and pretesting before using this option.
388 SSI Web v8
2.2.6 Task Labels

You may place concept labels and top/bottom corner labels within your CBC questionnaire. The graphic
below shows the position of these elements within a task.
You can repeat the concept labels directly above the response options (radio buttons or numeric entry
boxes) by clicking the Repeat Concept Labels Above Input box.
Warning: The concept labels are for putting generic labels to help the respondent navigate the
task. They are not for putting attribute labels that could have any specific meaning (such as
brands) that should affect preference. No Alternative-Specific Constants (ASCs) are estimated for
the concept labels, so it would be a big mistake to type anything into these levels that would lead
to a preference for one concept over another.
CBC Help 389
2.2.7 Number of Attributes/Levels/Tasks in CBC

With full-profile presentation, respondents see concepts that are described on all the attributes. With
full-profile presentation, there is a limit to how many attributes one should include in choice-based
conjoint analysis. It is our opinion that concepts described by more than about eight attributes may
confuse respondents and may cause them to respond superficially. But, it is difficult to establish a
general limit, since the motivation and knowledge of respondents, the length of the attribute text, and the
use of graphics affect how many attributes will work well.
If you must study more attributes than respondents can comfortably deal with in a CBC interview, you
might consider some other method of conjoint analysis. Adaptive CBC may be a good alternative, since
it is possible to drop entirely unimportant attributes from the product profiles. The Advanced Design
Module for CBC can create partial-profile designs, where only a subset of the attributes is ever presented
within a single choice task. Another conjoint technique, ACA (Adaptive Conjoint Analysis), can also
handle larger numbers of attributes, though it has weaknesses especially in pricing research studies.
Number of Attribute Levels in CBC
The base CBC system lets you measure up to 15 levels for any one attribute (though the Advanced
Design Module expands that number to 254 levels per attribute). Most projects will probably involve five
or fewer levels per attribute, although attributes such as Brand (or SKUs reflecting brand x package
combinations) may easily require many more levels for the purposes of a project. For typical CBC
studies, it is usually better to have fewer levels on which attributes are described, along with approximate
balance in the number of levels across attributes. With packaged-goods and beverage research, it may
be reflective of real-world conditions to include dozens of levels to reflect brand or brand x package size
combinations, but considerably fewer levels for variations in price.
We caution against using more than about five levels to define quantitative functions such as Price or
Speed if utilities will be estimated for each discrete level (CBC's standard approach). CBC's market
simulator permits interpolation between levels, so many additional points along a function may be
estimated. Defining a quantitative attribute on too many levels spreads the data thin and increases the
likelihood of getting reversals (out-of-order) utilities that are counterintuitive and problematic in
simulations. Constrained estimation (monotonicity constraints) can help out in these situations.
Another approach is to fit linear terms to quantitative attributes (also supported by our utility estimation
programs). In that case, and if you believe the appropriate function is approximately linear, it may be
justified to include more than five levels for quantitative attributes.
Number of Tasks in CBC
Because each respondent can receive a unique set of CBC tasks (questions) under randomized designs,
as few as just one task may be asked per respondent while still allowing estimation of part-worths (given
a large enough sample size). Not surprisingly, most CBC questionnaires include multiple tasks, since it
seems a waste not to collect more information from each respondent. With multiple observations per
respondent, one can model heterogeneity in preferences, which leads to more accurate choice
simulators.
In a 1996 meta-analysis of 21 CBC data sets, we found that multiple observations per respondent are
quite valuable and that respondents could reliably answer up to at least 20 questions, and perhaps even
more. However, we discovered that respondents process earlier tasks differently from later questions.
Respondents paid more attention to brand in the first tasks and increased their focus on price in later
questions. (See Johnson and Orme's article entitled "How Many Questions Should You Ask in Choice-
390 SSI Web v8
Based Conjoint?" available for downloading from the Technical Papers section of our home page: http://
www.sawtoothsoftware.com/support/technical-papers).
Over the last five years, other researchers have examined the same issues and have suggested that
respondents (especially internet and panel respondents) are probably less patient and conscientious with
long CBC surveys involving many attributes than respondents of 20 years ago. They are perhaps quicker
to resort to simplification heuristics to navigate complex CBC tasks and responses beyond about the
tenth task don't seem to be revealing much more about each respondent's choice process. These
authors have recommended using fewer than a dozen tasks, even if the attribute list is relatively long.
They have recommended meeting the information requirements involved with long attribute lists by
increasing the sample size.
With CAPI or paper-based CBC research, we recommend asking somewhere in the range of 8 to 15
choice tasks. With Web interviewing, fewer tasks might be appropriate if there is an opportunity for
increased sample sizes (which often is the case). If estimating individual-level utilities using CBC/HB,
we'd recommend at least six choice tasks to achieve good results based on simulated shares, but about
10 choice tasks or more for developing robust predictions at the individual level (again assuming a typical
design). Before finalizing the number of concepts or tasks to be asked, we urge you to pretest the
questionnaire with real respondents to make sure the questionnaire is not too long or overly complex.
CBC Help 391
2.2.8 None Option/Dual-Response None

Choice-based conjoint questions can be designed to have a "None" option, sometimes referred to as the
constant alternative. It is argued that including a None option in CBC tasks mimics the real world, since
buyers are not required to choose products that don't satisfy them. The None option can also be used to
reflect a status quo choice, such as "I'd continue to use my current long-distance service provider."
CBC gives you the option of including a None option in the questionnaire. If the None option is present, a
separate logit utility weight is computed for the None parameter (if using logit, CBC/HB, or latent class
analysis). The more likely respondents are to choose None relative to product concepts, the higher the
utility of the None option. The None parameter can be used in market simulations to estimate the
proportion of respondents that would not choose any of the simulated product concepts.
In general, we recommend including None in CBC questionnaires, but paying less attention to (or
completely ignoring) it in market simulations. The propensity to choose None can be a telling measure
when comparing groups or individuals. We suggest that percent of Nones in Counts or simulations be
viewed in a relative rather than an absolute sense.
The use of the "None" concept in partial-profile CBC studies (only supported by the Advanced Design
Module for CBC) is problematic. The None weight varies significantly depending on how many attributes
are displayed in the partial-profile task.
Justification for the None concept refers to vertical justification of the None text within its concept area.
Dual-Response None
Some researchers have advocated asking the "None" choice as a second-stage question in discrete
choice questionnaires (see "Beyond the Basics: Choice Modelling Methods and Implementation
Issues" (Brazell, Diener, Severin, and Uldry) in a 2003 ART/Forum Tutorial, American Marketing
Association). You can specify the dual-response None layout by selecting it as the None Option from
the Design tab. Using this option does not complicate analysis, as the software handles this more
sophisticated application of the None automatically.
The "Dual-Response None" approach is as follows. Rather than ask respondents to choose among, say,
four alternatives {A, B, C and None}; respondents are first asked to choose among alternatives {A, B, and
C}, and then next asked if they really would buy the alternative they selected in the first stage (yes/no).
392 SSI Web v8
The dual-response None dramatically increases the propensity of respondents to say "None," which
many researchers would argue better reflects actual purchase intentions than when the "None" is asked
in the standard way. But, no information is lost due to the selection of the "None," since respondents
are first asked to discriminate among available products. Thus, the "Dual-Response" none can provide a
"safety net": we can estimate a "None" parameter without worrying about the potential decrease in
precision of the other parameters if the incidence of "None" usage is quite high.
The dual-response None has its drawbacks. It adds a little bit more time to each choice task, since
respondents must provide two answers rather than one. But, the additional time requirement is minimal,
since respondents have already invested the time to become familiar with the alternatives in the choice
task. It is also possible that asking the "None" as a separate question may bias the parameters of
interest, though this has not been shown in practice, to the best of our knowledge.
Brazell et al. have suggested that the benefits of the dual response seem to outweigh any negatives.
They have conducted split-sample experiments with respondents that demonstrate that the parameters
(other than the "None") are not significantly different (after adjusting for scale) when using the standard
"None" vs. "Dual-Response None" formats.
For details regarding estimation of part-worth utility parameters with dual-response None in place, see
the CBH/HB manual.
CBC Help 393
2.2.9 Specifying Fixed or Holdout Tasks

CBC lets you specify a single or multiple "fixed" tasks. Fixed refers to the fact that every respondent is
shown the same choice task, with the product concepts defined in exactly the same way. You must
define your own fixed tasks; CBC does not design them for you. (By default, all fixed tasks are
initialized to level "1" for each attribute.)
Most CBC users will opt for a randomized design, since they are quite efficient, automatic, and permit
great flexibility in analysis. Some CBC users with design expertise may choose to implement a fixed
design, which is most easily done by importing the design from a .csv file. A fixed design can be slightly
more efficient than a randomized design in measuring the particular effects for which it was designed.
For most CBC users we recommend using randomized tasks for part-worth estimation and specifying
one or more fixed holdout tasks that are not used for utility estimation. We think it is wise to include
holdout choice tasks in conjoint interviews, even though they may not appear to be needed for the main
purpose of the study. They almost always turn out to be useful, for these reasons:
They provide a proximal indication of validity, measured by the utilities' ability to predict
choices not used in their estimation.
They permit identification and removal of inconsistent respondents (if using HB).
They can be used for testing specific product configurations under consideration. Much value
can be added by direct measurement of these concepts.
They can be used for testing the accuracy of market simulators. They aid considerably in
comparing alternative models (logit, Latent Class, or HB) and choice simulation strategies.
(Note: if comparing the ability of different models to predict holdout choices, it is important to
adjust the scale parameter to maximize the fit of each model prior to making comparisons.)
If holdout concepts have been defined with differing degrees of product similarity, they can be
used for tuning the appropriate correction for product similarity in Randomized First Choice
modeling.
It's hard to design good holdout concepts without some prior idea of respondent preferences. There's no
point in asking people to choose among concepts where one dominates in the sense that most everyone
agrees it is best. And, similarly, it's good to avoid presenting concepts that are equally attractive, since
equal shares of preference would be predicted by a completely random simulator. If you present triples
of concepts, it's probably best if their shares of choices are somewhere in the neighborhood of 50/30/20.
When conducting CBC studies, if you plan to do segmentation with latent class analysis, it's wise to
consider the kinds of groups you expect to get and to design products in holdout choice sets so that one
alternative will be much more preferred by each group.
If you plan to use the Randomized First Choice simulation model, it is helpful to include holdout tasks
that reflect severe differences in product similarity. For example, in a holdout choice set featuring four
product alternatives, two products might be identically defined on all except one or two attributes. By
including products with differing similarities, the appropriate adjustment for product similarity can be
tuned in the Randomized First Choice Model.
It isn't necessary to have many holdout sets to check the fundamental validity of your utilities. However,
if you want to use those choices to identify and eliminate inconsistent respondents, you need several
choice sets.
394 SSI Web v8
Finally, if you do have several choice sets, it's useful to repeat at least one of them so you can obtain a
measure of the reliability of the holdout choices. Suppose your conjoint utilities are able to predict only
50% of the respondents' holdout choices. Lacking data about reliability, you might conclude that the
conjoint exercise had been a failure. But if you were to learn that repeat holdout tasks had reliability of
only 50%, you might conclude that the conjoint utilities were doing about as well as they possibly could
and that the problem lies in the reliability of the holdout judgments themselves.
Analyzing Holdout Concepts
If you have specified fixed holdout choice tasks within the CBC questionnaire, you can analyze the
results by exporting the responses under the File | Data Management area.
Some researchers repeat choice tasks to achieve a measure of test-retest reliability. This type of
analysis often is done at the individual level. If you plan to analyze holdout choice tasks at the individual
level, you should export the data for analysis using another software program.
CBC Help 395
2.2.10 Specifying CBC Design Parameters

Experimental Design Creation Settings
Random Tasks
The Random Choice Task s are the questions/scenarios generated by CBC's design algorithms for
the purposes of estimating part-worth utilities. The attribute levels making up the product concepts
are formulated to have good statistical properties (such as level balance and independence of
attributes). It is typical to include about 8 to 15 of these types of choice tasks in your study.
Fixed Tasks
If you want to specify specific choice scenarios for CBC to ask of every respondent, these are
called Fixed Choice Task s. If you add these to your questionnaire, you must specify the attribute
level combinations that define the tasks (by clicking the Fixed Task Designs... button). Fixed
Tasks are often used to include scenarios of particular interest to the client and for collecting
additional choice tasks that may be used to assess internal validity. If you are not inclined to
include any fixed tasks in your study, you may set the number of fixed tasks to 0.
Concepts per Task
Here you specify how many product concepts (alternatives) to include in each choice task.
Although it is tempting to want to make it easier on respondents by displaying fewer concepts per
task, we encourage you to ask respondents to consider many product concepts per task. For
example, if each of your attributes had 3 levels, it would be a good idea to show 4 or 5 product
concepts on the screen, so that some levels repeat within the choice task. This not only means
each choice task provides more information, but including level overlap encourages respondents to
provide more thoughtful responses and leads to stronger estimation of potential interaction effects.
None Option
You may include an option for respondents to specify that they would choose None of the products.
This option is sometimes referred to as the "constant alternative" and may also be used to specify
a status quo state, such as "I would continue with my current provider rather than switch to one of
these plans."
A "Dual Response None" option is also available, wherein respondents first must select a product
concept and are next asked whether they would actually purchase that product or not.
Click here to read more information about the None response options.
Advanced Settings
Random Task Generation Method
This control lets you select which design method CBC should employ in determining the
combinations of attribute levels to be shown in the Random Choice Task s for your survey. We
encourage you to read the section on CBC Questionnaires and Design Strategy to learn more about
CBC's design strategies. For standard CBC studies, we generally suggest using either the
Balanced Overlap or Complete Enumeration design strategies. (Note: these methods are
396 SSI Web v8
computationally intensive, and for very large numbers of attributes and levels may require a long
time to generate a design file.)
Questionnaire Versions
CBC automatically creates a design file that is a) uploaded to the server if using web-based data
collection, b) is copied to other laptops/PCs if using CAPI-based data collection, or c) is used to
generate a questionnaire file if using paper-based data collection. During the process, you are
asked to specify how many versions of the questionnaire should be written to that design file. For
computer-based interviewing, we generally recommend that you include at least 100 versions, as
there are benefits to fielding many versions of the questionnaire. Optimally, each respondent would
receive his/her own version of the questionnaire. However, there are diminishing returns and
including more than about a dozen versions often is of very little to no additional practical benefit.
For that reason, we limit the number of versions that can be placed in the design file to 999. If you
have more than 999 respondents, this means that some respondents will by necessity receive the
same designs. For all practical purposes, once there are so many unique versions of the
questionnaire in the total design pool, this poses no harm.
Design Seed
This is the seed value used in the random number generator for the design algorithm. By using a
different starting seed, a different design will result. With very many versions of the questionnaire,
the overall design efficiency should be nearly identical no matter the random seed. With only a few
versions of the questionnaire, using a different starting seed can have a meaningful effect on the
overall efficiency.
Randomize Attribute Position within Concepts
Attribute Randomization
Attribute Randomization specifies whether to present the attributes in random presentation order
within a concept. If randomization is enabled, the attribute list is randomized once per
respondent, and all tasks within a respondent's interview will display the attributes in that given
order. This can be useful to control order effects across respondents.
If Randomize All is selected, then all attributes are randomized. If Randomize a Subset is
selected, then the Settings... button allows you to specify the range of attributes to randomize (the
attributes outside the range are not randomized).
First Randomized Attribute
The first attribute in the range of attributes to be shown in random presentation order. Specify a
"1" if you wish all attributes to be randomized. If, for example, you want the first and second
attributes always to appear as the first two attributes in a product concept, specify a "2."
Last Randomized Attribute
The last attribute in the range of attributes to be shown in random presentation order. Specify
the last attribute number if you wish all attributes to be randomized. If, for example, you had a
total of five attributes in your study and you wanted the fourth and fifth attributes always to
appear as the last two attributes in a product concept, specify a "3."
CBC Help 397
Concept Sorting within Task
Concept sorting refers to the order in which the product concepts (profiles) are displayed in a task.
For example, consider a choice task with three product alternatives. The default behavior in CBC
software is to randomize the order in which these product concepts are displayed. Thus, a given
brand may appear in the first concept in one task, but in the last concept position in the next task.
However, researchers do not always want the levels appearing in random concept positions within
the task. There are two options for researchers to control concept ordering:
Randomized Level Order. With this option, the researcher selects (using the Settings...
button) a specific attribute that is used to control the concept presentation order. For
example, the researcher may choose the "brand" attribute. The levels for brand are
randomized once per respondent, and then the product concepts are displayed in that new
order for all choice tasks for that respondent. For example, respondent #1 may see brands in
the order 2, 3, 1 for the three concepts in the task (left, middle, right), for all tasks. Then,
respondent #2 may see the brands in the order 3, 2, 1 for the concepts in his questionnaire.
This strategy helps control order bias, without making respondents reorient themselves to new
positions for, say, brands in each new task.
Natural Level Order. With this option, the concepts are shown in the natural level order for a
selected attribute, such as "brand." Brand 1 will always appear in the first concept, brand 2 in
the second concept, etc.--assuming that the researcher uses a Complete Enumeration or
Shortcut design strategy (which enforces minimal overlap), has no prohibitions and shows as
many concepts on the screen as there are brand levels in the study.
This strategy is often used in "shelf facing" displays (Advanced Design Module only), or in
cases in which the researcher doesn't want to make respondents reorient themselves to new
positions for, say, brands in each new task. However, using a fixed concept order may
introduce order biases.
Additional Settings
Show Levels of First Attribute no more than Once per Task
Sometimes with attributes like brand, the researcher wants each brand to appear exactly once per
choice task. If you have 4 brands and are showing 4 concepts per task, this happens naturally
with Complete Enumeration or Shortcut strategies--unless you have included prohibitions in your
study. When prohibitions are in place, it is possible for a brand to be repeated within the same
choice task. To avoid this possibility, you can check this box and the designer will ensure that
levels of the first attribute appear just once per task. (Of course, this isn't feasible if you are trying
to show more concepts per task than there are levels in the first attribute.)
Advanced Design Module Settings
Those with the base CBC license alone are limited to Traditional Full-Profile CBC Designs. Users with
the Advanced Design Module may also choose Alternative-Specific Designs and Partial-Profile Designs.
398 SSI Web v8
These approaches are described later in this documentation.
Test Design Report Settings
Include Two-Way Frequencies in Test Design Report
Check this to include an additional analysis within the Test Design Report of joint frequencies of
levels. These joint frequencies reflect the number of times each attribute level appeared with each
other attribute level within the same product concept across all concepts and tasks of your design.
Generate Design
When you click the Generate Design button, CBC automatically creates a design file using your
current CBC settings that you upload to the server (for web-based data collection), or install on
laptops/PCs (for CAPI-based collection), or that is used in generating questionnaire files for paper-
and-pencil collection. It also produces a test design report, letting you know whether the design can
efficiently estimate the main effect utilities pertinent to your study. You should pay close attention
to the design report to make sure the design efficiencies for the attribute levels in your study are
acceptable and that the report does not state "Design Deficient" or have any asterisks in place of
the estimated standard errors.
Please note that in some cases it may take a long time to generate a design if using the Complete
Enumeration or Balanced Overlap methods. A typical study with few attributes and levels will face
little delay (a few seconds) to generate hundreds of designs. CBC gives an estimate of the time
remaining to generate the design. For testing purposes you may wish to generate few designs for
the sake of speed, but once you are ready to post the survey to the Web (or use CAPI data
collection) you may request that hundreds of designs be generated and saved to the design file.
Import/Export Design
Some users may choose to alter CBC's designs, or use their own design instead. Use the Export/Import
Design capability when performing such advanced options.
CBC Help 399
2.2.11 Testing the CBC Design

Introduction
In CBC, a design refers to the sum total of the task descriptions across all respondents. The design
contains information about the combinations of attribute levels that make up the product concepts within
the tasks. The design is saved to a design file that you upload to your web server. Optimally efficient
CBC designs can estimate all part-worths with optimal precision; meaning that the standard errors of the
estimates are as small as possible, given the total observations (respondents x tasks), the number of
product concepts displayed per task, and respondent preferences.
CBC's random design strategies generally result in very efficient designs. These designs are not
optimally efficient, but are nearly so. In the case of large sample sizes, a large number of questionnaire
versions in the design file, and no prohibitions, one can confidently field a questionnaire without testing
the design.
However, there are conditions that can result in inefficient designs. Sometimes, a design can be so
inefficient as to defy all attempts to compute reasonable part-worth utilities. We have heard of entire
data sets with hundreds of respondents going to waste because the user neglected to test the design.
Therefore, it is imperative to test your design whenever any of the following conditions exist:
any prohibitions are included

sample size (respondents x tasks) is abnormally small
the number of versions you plan to use is few
CBC's default Test Design capability ("Quick Test"/OLS Efficiency) only tests the efficiency of main
effects (the separate utility estimate for each attribute level). It provides a good approximation of the
relative efficiency of the CBC design with respect to each attribute level. Many researchers using
standard CBC plans find this default test adequate for their purposes. For more sophisticated design
testing, an Advanced Test option is available for simulating "dummy" respondent answers and reporting
the standard errors (from a logit run) along with D-efficiency.
Our design testing methods assume aggregate analysis, though most CBC users eventually employ
individual-level estimation via CBC/HB. That said, CBC's design strategies can produce designs that are
efficient at both the aggregate and individual levels (though we don't specifically provide valid measures of
the standard errors for individual respondent estimates within our test design procedures).
Prohibitions are often the culprit when it comes to unacceptable design efficiency. If your prohibitions
result in unacceptably low design efficiency under the Complete Enumeration or Balanced Overlap
Methods, you should try the Shortcut or Random design strategies. These latter two methods are less
constrained than the more rigorous former ones, and will sometimes result in higher design efficiencies in
the case of many prohibitions.
Testing the Efficiency of Your Design
When you choose Test Design from the Specify CBC Interview Parameters dialog, CBC automatically
tests the design and displays the results within the results window (the results are also saved to a file
named STUDYNAME_EXERCISENAMEtest.txt). CBC automatically generates a data file of simulated
(dummy) respondent answers appropriate for advanced design testing.
CBC includes two test procedures:

400 SSI Web v8
Test Design ("Quick Test", Frequencies and OLS Efficiency)

(the default test routine used in previous versions of CBC, based on OLS theory)
Advanced Test (Simulated Data, Logit Report, and D-Efficiency)
(a more rigorous test, based on conditional logit theory)
Test Design (Frequencies and OLS Efficiency)
Following is a sample report, as it appears in the results window:
CBC Design Efficiency Test

Task Generation Method is 'Complete Enumeration' using a seed of 1

Includes 1000 total choice tasks (10 per version).
Each choice task includes 3 concepts and 6 attributes.
A Priori Estimates of Standard Errors for Attribute Levels

---------------------------------------------------------------
Att/Lev Freq. Actual Ideal Effic
1 1 75 (this level has been deleted) Brand A
1 2 75 0.2890 0.2887 0.9981 Brand B
1 3 75 0.2841 0.2835 0.9958 Brand C
1 4 75 0.2936 0.3062 1.0873 Brand D
2 1 100 (this level deleted) 1.5 GHz

2 2 100 0.2207 0.2182 0.9776 2.0 GHz
2 3 100 0.2275 0.2182 0.9200 2.5 GHz
3 1 100 (this level deleted) 3 lbs

3 2 100 0.2297 0.2182 0.9022 5 lbs
3 3 100 0.2235 0.2182 0.9533 8 lbs
4 1 100 (this level deleted) 60 GB Hard Drive

4 2 100 0.2234 0.2182 0.9543 80 GB Hard Drive
4 3 100 0.2204 0.2182 0.9806 120 GB Hard Drive
5 1 100 (this level deleted) 512 MB RAM

5 2 100 0.2199 0.2182 0.9850 1 GB RAM
5 3 100 0.2203 0.2182 0.9809 2 GB RAM
6 1 100 (this level deleted) $500

6 2 100 0.2237 0.2182 0.9516 $750
6 3 100 0.2222 0.2182 0.9648 $1,000
For each level, the number of times it occurs within the design is counted and provided under the column
titled "Freq." Optimally efficient designs show levels within each attribute an equal number of times.
For each attribute and level, an approximation is made of the relative standard error of each main effect
under aggregate analysis and assuming that each version is seen just once across the total
observations. Test Design uses ordinary least squares (OLS) rather than multinomial logit for this
CBC Help 401
purpose, and it uses only the information about the design of the choice tasks, rather than respondents'
answers. (A multinomial logit model is used in CBC's analysis modules.) This test design method gives
relative standard error estimates similar to (but not identical to) those of multinomial logit. With this test,
the emphasis is not a precise estimate of each standard error for a given number of respondents, but
rather the pattern of their relative magnitudes with respect to one another.
The Sample Output
We'll describe the output, using fragments of the file and showing the parts described in bold.
Each line is labeled with the attribute and level in the first columns:

1 2 75 0.2890 0.2887 0.9981 Brand B
The number of times each level occurs in the design is displayed under the column labeled "Freq."

1 2 75 0.2890 0.2887 0.9981 Brand B
For estimation, it is necessary to omit one level from each attribute. The first level of each attribute is
automatically deleted from this analysis:

1 2 75 0.2890 0.2887 0.9981 Brand B
The column labeled "Actual" gives estimated standard errors for the data file analyzed:

1 2 75 0.2890 0.2887 0.9981 Brand B
1 3 75 0.2841 0.2835 0.9958 Brand C
1 4 75 0.2936 0.3062 1.0873 Brand D
The column labeled "Ideal" gives an estimate of what those standard errors would be if the design were
precisely orthogonal and had the same number of observations:

1 2 75 0.2890 0.2887 0.9981 Brand B
1 3 75 0.2841 0.2835 0.9958 Brand C
1 4 75 0.2936 0.3062 1.0873 Brand D
The column labeled "Effic" gives the relative efficiency of this design in terms of estimating each
parameter, compared to the hypothetical orthogonal design (it is the square of their ratio):

1 2 75 0.2890 0.2887 0.9981 Brand B
1 3 75 0.2841 0.2835 0.9958 Brand C
402 SSI Web v8
1 4 75 0.2936 0.3062 1.0873 Brand D
When we consider the entire "Effic" column, we see that the randomized design had a median efficiency
of about 97 percent, relative to a hypothetical orthogonal design. The estimates of standard errors for
orthogonal designs are only approximate, and with a very small data file such as this there can be quite
a lot of variability in estimation.
Notice that the standard error estimated for attribute 1, level 4 is actually smaller than the value
estimated for a hypothetical orthogonal design:

1 2 75 0.2890 0.2887 0.9981 Brand B
1 3 75 0.2841 0.2835 0.9958 Brand C
1 4 75 0.2936 0.3062 1.0873 Brand D
Anomalies such as this are likely to occur when using small samples of test respondents, and shouldn't
be of concern.
It is important to test the design, since if too many prohibitions are included it is possible to develop
designs that do not permit estimation of desired effects. When that occurs, the estimated standard
errors for those effects will be infinite and their estimated efficiencies will be zero. Your attention is
called to such occurrences by the presence of asterisks instead of numbers, and/or by error messages,
that signal that additional thought is needed about what can be prohibited from occurring. If you see a
warning stating that your design is deficient or if you see asterisks listed for the standard errors,
your design is deficient.
Especially if using few questionnaire versions, you will find that the quality of the design is affected by
the design seed. You may want to try different design seeds to obtain slightly better designs.
Advanced Test
Rather than just offering a relative measure of efficiency, the Advanced Test design estimates the
absolute precision of the parameter estimates under aggregate estimation, based on the combined
elements of design efficiency and sample size (respondents x tasks). The estimated standard errors are
only absolutely correct if the assumptions regarding the underlying part-worths and the error in
responses are correct. The Advanced Test is useful for both standard and complex designs that include
interaction or alternative-specific effects. It also reports a widely accepted measure of design efficiency
called D-efficiency, which summarizes the overall relative precision of the design.
Unlike the simpler default test, the Advanced Test takes into account that design efficiency for CBC
studies depends on how the concepts are grouped in sets. The level contrasts within sets determine
how much information that set contains with respect to the levels of interest. Technically, the utility
balance among the concepts within those sets also affects overall design efficiency, and thus
respondents' preferences need to be known to assess the efficiency of a design. However, most
researchers are more comfortable planning designs that are efficient with respect to uninformative (zero)
part-worth utility values, and that is the approach we take.
The Advanced Test simulates random (dummy) respondent answers for your questionnaire, for as many
respondents as you plan to interview. The test is run with respect to a given model specification (main
effects plus optional first-order interactions that you specify).
CBC Help 403
To perform the Advanced Test, you need to supply some information:
Number of Respondents
% None (if applicable to your questionnaire)
Included Interaction Effects (if any)
With this information, CBC simulates random respondent answers to your questionnaire. Using random
respondent answers is considered a robust approach, because it estimates the efficiency of the design
for respondents with heterogeneous and unknown preferences.
Simulated respondents are systematically assigned to the versions of your questionnaire (the first
respondent receives the first version, the second respondent the second version, etc.). If you are
simulating more respondents than versions of the questionnaire, once all versions have been assigned,
the next respondent starts again with the first version. If your study includes a None alternative, then the
None is selected with expected probability equal to the value you previously specified.
Once the data set has been simulated, the Advanced Test performs an aggregate logit (MNL) run,
estimating the effects you selected (by default, only main effects are considered). Sample results are
shown below:
Logit Report with Simulated Data

------------------------------------------------------------
Main Effects: 1, 2, 3, 4, 5, 6, 7
Interactions: 1x6
Build includes 300 respondents.
Total number of choices in each response category:
Category Number Percent

----------------------------------------------------
1 787 21.86%
2 753 20.92%
3 778 21.61%
4 792 22.00%
5 490 13.61%
There are 3600 expanded tasks in total, or an average of 12.0 tasks per
respondent.
Aggregate
1 0.01171 0.03186 0.36757 1 1 Brand A
2 0.01427 0.03182 0.44850 1 2 Brand B
3 -0.00202 0.03195 -0.06333 1 3 Brand C
4 -0.02396 0.03215 -0.74517 1 4 Brand D
5 0.01118 0.02638 0.42372 2 1 1.5 GHz

6 0.00600 0.02639 0.22715 2 2 2.0 GHz
7 -0.01717 0.02654 -0.64704 2 3 2.5 GHz
8 -0.00534 0.02638 -0.20235 3 1 3 lbs

9 0.01096 0.02631 0.41654 3 2 5 lbs
10 -0.00562 0.02645 -0.21251 3 3 8 lbs
404 SSI Web v8
11 -0.02986 0.02655 -1.12501 4 1 60 GB Hard Drive

12 0.04165 0.02620 1.58962 4 2 80 GB Hard Drive
13 -0.01179 0.02644 -0.44580 4 3 120 GB Hard Drive
14 -0.00618 0.02644 -0.23387 5 1 512 MB RAM

15 -0.03820 0.02661 -1.43548 5 2 1 GB RAM
16 0.04438 0.02615 1.69683 5 3 2 GB RAM
17 0.05398 0.02610 2.06837 6 1 $500

18 -0.01099 0.02647 -0.41497 6 2 $750
19 -0.04300 0.02669 -1.61105 6 3 $1,000
20 0.09273 0.04975 1.86389 Brand A by $500

21 -0.01982 0.05101 -0.38851 Brand A by $750
22 -0.07291 0.05124 -1.42294 Brand A by $1,000
23 -0.11173 0.05083 -2.19797 Brand B by $500
24 0.03271 0.05040 0.64897 Brand B by $750
25 0.07902 0.05060 1.56170 Brand B by $1,000
26 -0.02233 0.05050 -0.44214 Brand C by $500
27 -0.00902 0.05104 -0.17680 Brand C by $750
28 0.03135 0.05094 0.61548 Brand C by $1,000
29 0.04132 0.05038 0.82021 Brand D by $500
30 -0.00386 0.05102 -0.07574 Brand D by $750
31 -0.03746 0.05164 -0.72529 Brand D by $1,000
32 -0.45916 0.04862 -9.44349 NONE
The strength of design for this model is: 3,256.006

(The ratio of strengths of design for two designs reflects the D-Efficiency
of one design relative to the other.)
Details regarding the logit report may be found in the section entitled Estimating Utilities with Logit.
The beginning of the report lists the effects we are estimating (main effects for attributes 1 through 6,
plus the interaction effect between attributes 1 and 6). All random tasks are included in estimation.
Next, we see that 300 respondents each with 12 tasks were simulated using random responses, taking
into account the expected probability for None. We specified that the None would be chosen with 15%
likelihood, and indeed the None percentage is very close to that (13.61%). If we had used more
respondents, the probability would have been even closer to 15%. The remaining choices are spread
approximately evenly across the four other alternatives in the questionnaire.
Next, the logit report based on the random responses is shown. You shouldn't pay any attention to the
effects (utilities) in the first column, as we are using random data. The T-ratios are also not useful, for
the same reason. The important column to study is the Aggregate Std Err (Standard Error) column.
The standard errors reflect the precision we obtain for each parameter. Lower error means greater
precision. This design included no prohibitions, so the standard errors are quite uniform within each
attribute. If we had included prohibitions, some levels might have been estimated with much lower
precision than others within the same attribute.
For our simulated data above, the levels within three-level attributes all have standard errors around
0.026. The one four-level attribute has standard errors for its levels around 0.032. We have obtained less
precision for the four-level attributes, since each level appears fewer times in the design than for the
three-level attributes. The interaction effects have standard errors around 0.051.
CBC Help 405
Suggested guidelines are:
Standard errors within each attribute should be roughly equivalent

Standard errors for main effects should be no larger than about 0.05
Standard errors for interaction effects should be no larger than about 0.10
The second two criteria are rules of thumb based on our experience with many different data sets and our
opinions regarding minimum sample sizes and minimum acceptable precision. Ideally, we prefer
standard errors from this test of less than 0.025 and 0.05 for main effects and interaction effects,
respectively. These simulated data (300 respondents with 12 tasks each) almost meet that higher
standard for this particular attribute list and set of effects.
D-Efficiency
D-efficiency summarizes how precisely this design can estimate all the parameters of interest with
respect to another design, rather than how well the design can estimate the utility of each level of each
attribute (as with the simpler default test). D-efficiency is described in an article by Kuhfeld, Tobias, and
Garratt (1994), "Efficient Experimental Design with Marketing Research Applications," Journal of
Marketing Research, 31 (November), 545-557.
To arrive at D-efficiency, we should define a few terms:
Xt = design matrix for task t with a row for each alternative

xi = ith row of Xt
pi = probability of choice of alternative i
v = probability-weighted means of rows: v = i pi x i
Zt = matrix with ith row z i = pi1/2 ( x i - v)
Z = matrix made by appending all Zt matrices
Z'Z is known as the "Information Matrix"

The determinant of Z'Z measures the strength of the design.
Because the magnitude of the determinant of Z'Z depends on the number of parameters estimated, to
provide a measure of strength independent of p we consider the pth root of the determinant:
|Z'Z|1/p
Where Z is the probability-centered design matrix, Z'Z is the "Information Matrix," and p is the number of
parameters estimated.
The pth root of the determinant doesn't result in a single value bounded by 0 and 1.0 (as with the simpler
test efficiency report), and this value is meaningless without reference to the same computed for
comparison design. This value also depends on the number of respondents x tasks, so when comparing
two designs, it is important to hold the number of respondents x tasks constant. We use the term
"efficiency" to compare the relative strengths of two designs. The relative D-efficiency of one design with
respect to the other is given by the ratio of the pth root of the determinants of their information matrices.
The design with a larger value is the more efficient design. (Note: we only consider the precision of
parameters other than the "None" parameter when computing the strength of the design.)
406 SSI Web v8
Consider design A with no prohibitions and design A' with prohibitions. The pth root of the determinant of
the information matrix is computed for both (holding the number of respondents, tasks, concepts per
task, and None % constant). If design A' has a value of 2,500 and design A has a value of 3,000, design
A' is 2,500/3,000 = 83.3% as efficient as design A. The inclusion of prohibitions resulted in a 1 - 0.833 =
16.7% loss in efficiency.
Efficiency for Specific Parameters (Attribute Levels)
Sometimes, you may be more concerned about the efficiency of the design for estimating a specific
parameter (such as the utility for your client's brand) rather than an overall efficiency of the design across
all parameters. Let's assume that your client asked you to implement prohibitions between the client's
brand name and other levels. Further assume that the overall relative strength of the design with
prohibitions relative to the design without prohibitions is 97%. On the surface, this seems like little
overall loss in efficiency. However, you note that the standard error (from the logit report using simulated
data) for your client's brand was 0.026 prior to implementing the prohibition, but 0.036 afterward. The
relative efficiency of the design with prohibitions relative to the non-prohibited design with respect to this
particular attribute level is:
a2/b2
Where b is the standard error of the estimate for the client's brand name after the prohibition and a is the
standard error prior to the prohibition. In this case, the relative design efficiency of the prohibited
compared to the non-prohibited design with respect to this particular level is:
0.0262/0.0362 = 0.52
And the impact of these prohibitions on estimating your client's brand utility is more fully appreciated.
Additional Note: The pattern of random answers (respondent preferences) for random respondent data
will have a small, yet perceptible, effect on the reported Strength of Design (relative D-Efficiency). For
the purpose of estimating the absolute standard errors of the parameters, we suggest using the same
number of dummy respondents as you plan to achieve with your study. But, for comparing the merits of
one design to another, you can reduce the effect of the random number seed by increasing the dummy
respondent sample size significantly, such as to 5000 or 10000 respondents. Increasing the sample
size will greatly reduce the variation of the Strength of Design measure due to the random seed used for
respondent answers.
CBC Help 407
2.2.12 Importing/Exporting CBC Designs

One of the strengths of the CBC system is its ability to generate experimental designs (the combinations
of attribute levels to show respondent) in an efficient and easy manner. Also, multiple versions of the
questionnaire are generated, so different respondents see different combinations of attributes within the
choice tasks.
Some researchers wish to export the design, to examine ahead of time the combinations that will be
asked of respondents (and perhaps to perform additional analysis regarding the design), or to make
minor modifcations to the design.
Other researchers may have developed questionnaire designs using other software and wish to import
those designs for fielding within the CBC system. They can do this through CBC's Import Design
capability.
For either purpose, the design file uses the same comma-separated (.CSV) layout. A .CSV file can be
directly opened and saved with Excel, and the examples below assume you are working with the
comma-separated file in Excel. (If using a text editor, the values in the file are separated by commas.)
If you Import a design, any existing design will be overwritten.
Import/Export File Format
The file format is as follows.
Column 1
Indicates the questionnaire version. Any fixed tasks in your study are listed as version 0 (all the
examples below include a single fixed task). Fixed tasks are shown to every respondent and are
only defined once in the design file (as version 0).
Column 2
Indicates the task (choice set) number.
Column 3
Indicates the concept (alternative) number.
Columns 4 through n
Shows the attribute levels associated with the current concept, task, and version number.
Traditional CBC Layout:
In the example below, each respondent receives a single fixed choice task (version 0, task 1). There are
also two random tasks in the questionnaire (i.e. calibration tasks used for part-worth utility estimation).
Only two versions of the questionnaire are shown below, though additional versions would continue,
following the same pattern. There are three attributes in this study, and three concepts are shown per
task.
408 SSI Web v8
Alternative-Specific Layout:
In the example below, Attributes 2 and 3 are alternative-specific. When an attribute is not shown in a
product concept, it receives a level "0". Attribute 2 only appears when level 2 of attribute 1 is present.
Attribute 3 only appears when level 3 of attribute 1 is present.
Partial-Profile Layout:
When an attribute is not shown in a choice task, it receives level "0". In the example below, two of the
three attributes are shown per task.
CBC Help 409
Additional Rules:
For Random tasks (tasks used in estimation), the version must start with 1 and proceed in that order.
If random tasks are present in the file to import, the number of versions and number of tasks must match
the number of versions and tasks specified in the software.
The number of concepts must be constant across all tasks. The number of concepts must match the
number of concepts specified in the software.
If Partial Profile is specified in the software, the software setting for Number of Attributes Shown in Each
Concept must match the number specified in each concept in the file (random tasks only).
Any prohibitions specified in CBC will not be checked against the imported design.
410 SSI Web v8
2.3 Advanced CBC Designs

2.3.1 Conditional Pricing for CBC
In pricing research, it is sometimes very useful if prices for product concepts are made to depend on
other attribute levels (such as brands). The first versions of CBC could handle this in a limited way by
using prohibitions. For example, one could prohibit high prices from being shown for some products, and
prohibit low prices from being shown with others. Such prohibitions, however, can lead to very inefficient
designs.
There are some work-arounds for dealing with situations where prohibitions with price seem to be
needed. For example, if package size is an attribute, prices for the 64 oz package would be much
higher than prices for the 16 oz size. One approach is to express price as a percentage above or below
the "normal price" for that package size, such as "10% above the average price." Another is to show
unit prices, rather than prices per package. For example, with detergent we might study 16, 32, and 64
ounce packages, but present prices in terms of "cents per ounce." These are ways of making a single
set of price levels work for all package sizes without specifying any prohibitions, but it would be
preferable to display actual prices appropriate for each package size.
The conditional pricing option lets you create a look-up table to determine the prices to show for specific
combinations of attributes. Drawing upon the example above, assume we have three attributes as
follows to describe detergents:
BRAND
Brand A
Brand B
Brand C
PACKAGE
16 oz. package
32 oz. package
64 oz. package
PRICE
"Low price"
"Medium price"
"High price"
The text for the price levels above is not shown during the interview, and serves as a place-holder only.
The price levels displayed during the interview vary according to the package sizes as follows:
Conditional Pricing Lookup Table
Low Price Medium Price High Price

16 oz. package $1.49 $2.09 $2.69
32 oz. package $2.79 $3.99 $5.19
64 oz. package $4.89 $6.99 $9.09
To construct this lookup table, we started with average prices for the three package sizes (middle
column). To determine the prices in the "Low Price" column, we decreased the average price by 30%
(and rounded to the nearest 9 cents). The "High Price" was calculated by increasing the average price
CBC Help 411
by 30% (and again rounding to the nearest 9 cents).
The example above illustrates how to make price dependent on a single variable (package size). CBC
lets you make price conditional on as many attributes as you wish. However, making price conditional
on more than one additional attribute can lead to addditional wrinkles and complications for analysis.
You will need to deal with an additional level of complexity when analyzing the results of studies that
involved conditional pricing. In the example above, even though nine unique prices were shown to
respondents, CBC's analysis programs by default still regard these as just three levels of price: Low,
Medium and High. However, we interpret the results keeping in mind that larger package sizes were
displayed with higher prices on average. We interpret the part-worth of the 32 oz. package taking into
account that it was shown at an average price roughly $1.90 cents higher than the 16 oz. package. The
main effect contains information not only about how desirable one package size is versus another, but
also about the relative price levels at which they are offered. Therefore, it is perfectly legitimate if the
count proportion or logit effect for the 32 oz. package is lower than the 16 oz. package. This would
indicate that respondents on average did not feel the larger size was worth the extra $1.90.
The conditional pricing table must also be taken into account when using the market simulator. Unless
the conditional pricing table is explicitly used in the market simulator, to simulate a 32 oz. package at
$3.99, level 2 for price is specified, etc. However, with the most recent releases of the Online Simulator
and for SMRT, when you export the data for analysis in SMRT (the .CHO and .ATT files), the conditional
pricing table is also exported for use in the SMRT market simulator. The conditional prices are
automatically read from a special footer section in the .ATT file into the Assign Level Values area of the
SMRT market simulator (please double-check them to verify that the information was interpreted
correctly, as we parse text strings to extract the price values). Once the conditional prices are within the
SMRT market simulator, you simply specify the actual conditional price during simulations. For
example, to simulate a 32 oz. package at $3.99, instead of specifying level 2, you specify a level value of
3.99.
Users should note that the use of Randomized First Choice (or the historical Share of Preference with
Correction for Product Similarity) can be problematic with conditional pricing. These methods assume
that products sharing the same attribute levels are identical on those levels and require some downward
correction in share. But, with conditional pricing tables, even though two brands may share the same
"level" for price, the conditional pricing table may have actually resulted in unique prices for brands within
the questionnaire. Therefore, RFC may implement a correction for product similarity that isn't actually
warranted, assuming that the brands' prices were identical when in fact they were not. To avoid this
outcome, we suggest turning off the correction for product similarity with respect to price within the RFC
advanced dialog.
In our previous example, we used (near) constant deviations from average prices to build a proportional
conditional table. Some researchers choose to use constant absolute price differences between levels
of the conditional attribute(s) instead of proportional changes. In either case, the tables have symmetric
properties, which are desirable from an analysis standpoint. Proportional or symmetric conditional
pricing tables sometimes make it possible to adequately model the data using main effects only (should
the interaction effects turn out to be not significant.) Otherwise, specification of interactions may be
necessary to fit the data properly, and the additional parameters estimated are done so with relatively
less precision than main effects.
If your conditional pricing table reflects significant deviations from symmetry, you must specify
appropriate interaction effects to correctly fit the data, even if the data don't justify the additional
parameters added to the model. A two-way interaction can correctly fit data collected with an
asymmetric conditional pricing table where price is dependent on one other attribute. If price was
dependent on more than one attribute and the table is not proportional, a three-way interaction would be
412 SSI Web v8
required to correctly model the effects. Our logit, latent class and CBC/HB programs are limited to
modeling only main effects and two-way interactions. Therefore, to analyze three-way interactions, you
would need to reformat the CBC data file (either the .csv or .CHO file) and collapse the two conditional
attributes (using your own data processing techniques) prior to specifying the interactions with price. Up
to 254 levels per attribute are permitted in CBC with the advanced design module, so this affords some
flexibility in this area.
Specifying Conditional Pricing
Before you can specify conditional prices, you first must have defined your list of attributes and levels.
Instead of specifying the text for actual prices for your price levels, you should have used placeholder
text such as "low price," "medium price," and "high price." When the questionnaire is administered, the
placeholder text you assigned is overwritten by the text string you specify in the conditional pricing table.
When you create a conditional price lookup table in CBC, you must specify the number of attributes that
participate in the relationship (there should be at least two). If price is dependent on one other attribute
(such as package size), two attributes participate in the relationship: package size and price. Click
Compose | Write Questionnaire... and then edit any CBC question within the CBC Exercise, then click
the Attributes tab and select the Conditional Relationships… button. A dialog appears in which you
specify attributes involved in the relationship. First, select your price attribute. The level text for this
attribute is overwritten by the prices that you specify in the conditional pricing table. Then, select the
attribute(s) upon which price is dependent.
The conditional pricing table is displayed, with as many rows in the table as required to completely
define the prices for all combinations of price and the dependent attribute(s). You can cut and paste
conditional pricing information directly from a spreadsheet program into the conditional pricing grid.
There can be a maximum of one conditional pricing table per exercise. However, some users have
discovered that they can use conditional display tables just as they would use conditional pricing tables.
In that case, more than one conditional pricing variable may be included. The only difference is that the
conditional prices specified as conditional display variables are not automatically carried forward to the
SMRT market simulator. In that case, you may need to use relative indices (such as 0.8, 1.0, 1.2, etc.)
when defining products in the market simulator, rather than having SMRT automatically use the
conditional pricing table within the simulator.
Missing levels are indicated with a 0 in the table (for use with Alternative-Specific designs or partial-
profile).
CBC Help 413
2.3.2 Conditional Display

Conditional display is often used for showing a graphic when levels from one or more attributes appear
together within a concept in a CBC task. For example, you may have two separate attributes for brand
and package size, but a single picture reflects the joint brand/package size concept. Where the text for
a particular brand and package size might appear in a CBC concept, you may wish to replace that with a
graphic.
But conditional display is not limited to just graphics, as it simply adds (or replaces, depending on your
Attribute Visibility setting) the existing level text. Other possibilities include:
Multi-media files
Text
JavaScript
Attribute Label (Optional)
If you are using Conditional Display and have chosen to show the graphic (or other conditional display
"element") at the top or bottom of the choice task, you can optionally specify an attribute label to be
shown in the choice task (on the left-hand side of the task). This attribute label is only used (and is only
applicable, otherwise it is "grayed out") if you have specified that attribute labels are to appear in the
choice task. You specify that attribute labels should appear within the choice task by clicking Compose
| Write Questionnaire… and editing one of your choice tasks from the List of Questions. Then, from
the Format tab check the Show Attribute Labels box.
Note: if in Step 3 of the Conditional Relationships Wizard, you specified that the conditional display
element is to replace the text of a certain attribute within the task, then the attribute label used for the
conditional element is the original attribute label for the replaced attribute text (that you originally typed
within the Specify Attributes and Levels dialog).
414 SSI Web v8
2.3.3 Alternative-Specific Designs

(The capabilities described in this section are only available if you own the CBC Advanced Design
Module.)
Some CBC projects require more flexible designs than the standard approach used with our CBC
system. With standard CBC, if there are six attributes in the study, all six appear within each product
concept.
Researchers have employed a specialized type of CBC design wherein some or all product alternatives
have their own unique sets of attributes. Conditional Pricing is one example of such a design, where
each brand might have its own set of prices. But, designs can be much more complex than that. For
example, consider modes of transportation: cars versus buses. Each option has its own set of features
(beyond just price) uniquely associated with that mode of transportation.
Consider the following attributes for cars and buses:
Car Bus
Parking fee $5.00/day Picks up every 20 min.
Picks up every 5 min.
25 cents per one-way trip

$1.00 per one-way trip
Assume we wanted to measure three other (constant) alternatives: walking, biking, or "I'd choose another
way to get to work."
The CBC question might look like the following:
If you lived 1 mile away from your workplace in downtown New York and these were your
choices for getting there, which would you choose?
Ride the bus Bike Drive my car Walk I'd choose another way to
Picks up every 10 min. Parking fee get to work
$1.00 per one-way trip $8.00/day
To implement this design using CBC's Advanced Design Module, you define the attributes as follows:
Attribute 1:
Walk
Bike
Ride the bus
Drive my car
CBC Help 415
Attribute 2:
Attribute 3:
$1.00 per one-way trip
Attribute 4:
Parking fee $5.00/day
Before proceeding, it is useful to introduce some terminology to describe the different roles attributes can
play in alternative-specific designs.
Primary attribute: an attribute whose levels appear in every product concept and upon which
conditional attributes (described below) depend. In the example above, attribute 1 is the primary
attribute.
Conditional attribute: attributes only displayed with a particular level or levels of the primary
attribute. Attributes 2 through 4 are conditional attributes.
Common attribute: attributes displayed with all levels of the primary attribute. (The example above
doesn't include a common attribute.) In traditional CBC designs, all attributes are common.
Specifying an Alternative-Specific Design
Choose Compose | Write Questionnaire... and the select a CBC question within the desired CBC
Exercise to edit. This will bring up the CBC Exercise Settings dialog. From that dialog, click the
Design Tab and then check the Alternative-Specific CBC Design radio button.
To create the "None" alternative, select Traditional None Option under the Format tab and specify the
text: "I'd choose another way to get to work." Additional constant alternatives are defined as levels within
the primary attribute.
To complete the design, set the following attribute level prohibitions (make sure to check "alternative-
specific design" prior to setting these prohibitions):
Attribute 1, Level 1 with Attributes 2 through 4 (all levels)

Attribute 1, Level 2 with Attributes 2 through 4 (all levels)
Attribute 1, Level 3 with Attribute 4 (all levels)
Additional Guidelines for Alternative-Specific Designs

416 SSI Web v8
For alternative-specific designs, we generally suggest the Complete Enumeration strategy (unless you
have so many attributes that the design generation process is prohibitively slow). If the time to generate
the design is too long, then you might consider 1) generating fewer total designs (e.g. 20 instead of 300)
or 2) the Shortcut design strategy. In either case, use the advanced test design procedure (estimated
standard errors via logit and relative D-efficiency test) to determine the effect of one strategy versus the
other.
Recall that within our paradigm for alternative-specific designs, main effects for conditional attributes are
measured specific to particular levels of the primary attribute. If the measurement of higher-order effects
is the primary concern, the Balanced Overlap or purely Random design method can work well given a
large sample size.
The "None" alternative is always displayed in the last concept position (or in the question's header or
footer). Other constant alternatives you define (with exhaustive prohibitions) within the primary attribute
are positioned randomly within the task by default, but you can control their order of presentation on the
Design tab of the CBC Exercise Settings dialog. Depending on how many concepts are displayed per
task and the design method, it is possible that a constant alternative may not appear in some tasks. If
you want the constant alternatives to appear in every task, you should display at least as many
concepts as levels in the primary attribute and then choose the Shortcut design strategy.
It is possible to have multiple primary attributes and multiple conditional "branches" in an alternative-
specific design. Conditional attributes can be shared (common) between two or more levels of the
primary attribute. You can choose to have those common attribute aligned in the same row within your
choice task by checking Align Common Attributes on Same Row from the Design tab.
Fixed alternative-specific tasks (or entire designs) may be specified.
Primary attributes must appear in the attribute list before conditional attributes.
Design Efficiency and Alternative-Specific Designs
Alternative-specific designs usually require a complex set of prohibitions. Therefore, we strongly

encourage you to test the design to ensure acceptably stable estimation of the part-worths. For such
designs, we suggest you use the advanced test design procedure (estimated standard errors via logit
and relative D-efficiency test).
When testing the design, you should check is that it doesn't return the message: "WARNING - THE
DESIGN IS DEFICIENT." You should pay particular attention to the standard errors of the part-worths.
Lack of convergence or very large standard errors are an indication that your design is deficient.
Hint: One rule of thumb is to look for standard errors from logit for main effects of no greater than about
0.05, and standard errors for interaction effects to be no greater than about 0.10 (alternative-specific
attribute effects have standard errors roughly the same magnitude as first-order interaction effects).
If your design is like those described in our transportation example where prohibitions are between all
levels of the conditional attribute(s) with a particular level (or levels) of the primary attribute, you will likely
be on safe ground. If you define additional prohibitions, you may encounter problems during analysis.
You can specify two-way interactions for alternative-specific designs, though in our example it doesn't
make sense to specify two-way interactions between primary and conditional attributes, since the effects
measured are already "specific" to the reference levels of the primary attribute. However, for this
example design, it is possible to study the interaction between the frequency of bus pick-up and the cost
CBC Help 417
per bus trip, since there were no prohibitions between those two attributes. If a common attribute were
defined (applying to all levels of the primary attribute), it is possible to specify other interactions, such as
between the primary and common attribute.
Analysis proceeds as with any other CBC study. The only difference is that when specifying product
concepts in the market simulator, you type "N/A" for conditional attributes that aren't applicable.
Logit, Latent Class, or HB estimation are possible with alternative-specific designs.

418 SSI Web v8
2.3.4 Partial-Profile Designs

(The capabilities described in this section are only available if you own the CBC Advanced Design
Module.)
In spite of their advantages, CBC questions can overwhelm some respondents, particularly if there are
many attributes. The base CBC system can measure up to ten attributes (shown in full profile), but more
than about eight attributes may overwhelm respondents in some situations.
Some researchers have proposed "partial-profile" choice experiments as a way to estimate preferences
for a large set of attributes. With partial-profile designs, each choice task includes a subset of the
attributes (typically around five). Across all tasks and respondents, a much larger list of attributes is
evaluated. The CBC Advanced Design Module permits up to 100 attributes.
Partial-profile choice designs have now been regularly used in CBC studies for over a decade. A growing
body of research suggests that they can be valuable for some situations. We haven't yet formed a
definite opinion about their performance versus other methods like Adaptive CBC (ACBC) and ACA for
dealing with large numbers of attributes. We hope that including partial-profile in the Advanced Design
Module encourages further research and experimentation.
With partial-profile designs, we assume respondents can evaluate the product concepts holding all
attributes not represented constant. If respondents cannot maintain this ceteris paribus mind set, the
resulting data may be incorrect. Therefore, when asking the choice question, we suggest including
language such as, "Please assume that all features not shown are alike in all other ways," or "Please
assume that these toasters differ only on the features shown." This may help respondents answer
appropriately, but it is still no guarantee.
The use of the "None" concept in partial-profile CBC studies is problematic. The None weight varies
significantly depending on how many attributes are displayed in the partial-profile task.
Analysis methods for partial-profile include logit, Latent Class, and HB. The success of the methods
(particularly for Latent Class and HB) hinge upon how much information is available from each
respondent on each attribute. We caution that individual-level estimation may not be stable for partial-
profile designs if the information available from each respondent relative to the number of parameters to
be estimated is low. Given enough information per respondent relative to the number of parameters to be
estimated, Hierarchical Bayes estimation may provide useful individual-level utilities for use in market
simulations where the results are summarized across respondents. Still, choice data are not as rich in
terms of statistical information content as ratings-based data. The individual-level estimates for large
partial-profile designs may contain a significant amount of noise and counter-intuitive relationships
(reversals). If the goal of the research is to estimate stable utilities for individual-level profiling and
predictions, the partial-profile approach alone may be insufficient. For sparse partial-profile designs, it
may be helpful in CBC/HB software to use a lower prior variance assumption (such as 0.5) and a higher
degrees of freedom for prior covariance matrix (such as a value equal to 1/4 or 1/2 of your sample size)
to avoid potential overfitting.
Many split-sample partial profile / full profile methodological studies have shown that the parameters
derived from either experiment are quite similar, including Price. Some recent methodological
experiments presented at the Sawtooth Software conference suggested that the parameters were quite
similar, but that the importance (slope) of Price was understated using partial-profile. Therefore, we
suggest users should be aware of this potential outcome and proceed with caution when using large
partial-profile designs in CBC for pricing research.
CBC Help 419
Specifying Partial-Profile Designs
Choose Compose | Write Questionnaire... and edit a CBC question within the CBC exercise to bring up
the Conjoint Settings | CBC Settings... to bring up the CBC Exercise Settings dialog. From that
dialog, click the Design Tab and then click the Partial-Profile CBC Design option (revealed by clicking
the Show Advanced Settings button).
There are a number of control parameters governing partial-profile designs. For purposes of illustration,
let's assume there are 12 total attributes in the study. The researcher wants to display 5 attributes per
task with attributes 1 and 12 to appear in every choice task.
Number of Attributes to Show: In this example, 5 attributes are displayed in each choice task,
so we specify a 5. Recent research suggests that between 2 to 4 attributes may be optimal to use
in partial-profile studies. The length of the level text has a bearing on the appropriate number, along
with the familiarity and interest respondents have for the product category.
Rotate Attributes into Concepts Starting with: In this example, attribute 1 appears in every
choice task. Therefore, attributes rotate in and out of the choice tasks starting with attribute 2. (If
you want all attributes to rotate into the tasks, always specify 1). Note that, all else equal,
attributes displayed in every task are measured with greater precision than those rotated into the
tasks.
and Ending with Attribute: In this example, attribute 12 appears in every choice task. Therefore,
attributes rotate into choice tasks ending with attribute 11. (If you want all attributes to rotate into
the tasks, specify the last attribute number.)
If the Attribute Randomization drop-down is enabled, the attributes appear in random order (held
constant within respondent) within the concept. If No Randomization is selected (default), the
attributes appear in their natural order.
Design Strategies for Partial-Profile Designs
With partial-profile designs, the design selection has two stages. The first stage involves choosing the
subset of attributes displayed in the choice task. To formulate the first task, a subset of attributes is
randomly chosen. For all subsequent tasks, the two-way joint frequency table of attribute presentation
within choice tasks is examined. Attributes are chosen so that the off-diagonal cells in this table remain
approximately balanced. This strategy is identical for the two methods.
The second stage of the design selection involves deciding which levels of the selected attributes are
displayed for each concept.
For partial-profile designs, we generally suggest either the Balanced Overlap or the Complete
Enumeration strategy (unless you have so many attributes that the design generation process is
prohibitively slow). If the time to generate the design is too long, then you might consider 1) generating
fewer total designs (e.g. 20 instead of 300) or 2) the Shortcut design strategy. In either case, use the
advanced test design procedure (estimated standard errors via logit and relative D-efficiency test) to
determine the effect of one strategy versus the other.
The purely Random design method should only be used if the measurement of interactions (through
aggregate analysis) is the primary goal, the attribute list is relatively small and the sample size is quite
large. However, partial-profile designs make it much more difficult to measure interactions between
attributes, when these attributes may only occur together within a product concept a minority of the time.
420 SSI Web v8
Notes for Partial-Profile Designs
Estimation of main effects and especially interaction terms for attributes rotated into choice tasks is
significantly less precise than with manageable full-profile CBC designs.
We strongly suggest you test your design prior to fielding to ensure you can estimate reasonably stable
part-worths. The Test Design module provided with CBC may be used. When using the default Test
Design routine (Frequencies and OLS Efficiency), you should recognize that the design efficiency is
estimated with respect to a hypothetical full-profile orthogonal design. The efficiency of part-worths for
attributes rotated into the design is naturally lower than with a full-profile design (where the attribute is
always present), since comparatively less information about that attribute is available.
We suggest using the advanced test design procedure (simulated respondent data, logit estimation
report, and relative D-efficiency) to assess the quality of different partial-profile designs. (But, remember
that the most D-efficient design isn't necessarily the design that will lead to the most precise utility
estimates. As more attributes are shown to respondents, the D-efficiency increases, but the difficulty of
the task and the resulting respondent error increases.)
Fixed partial-profile tasks (or entire designs) may be specified.
Partial-profile designs are more robust in the face of level prohibitions than full profile designs. For
example, consider a study with 10 attributes, where 4 attributes are shown at a time in product
concepts. Further consider that there are prohibitions between the levels of attributes 1 and 2. When
attributes 1 and 2 appear together within the same choice task, the prohibitions introduce correlations
(and resulting lower precision) within the design matrix. However, for the many tasks in which attributes
1 and 2 appear separate from one another, the prohibitions have no negative effect on the design
efficiency. These tasks in which attributes 1 and 2 appear separately provide uncompromised
information (assuming no bias due to partial-profile displays) for stabilizing the parameters for attribute 1
and 2.
CBC Help 421
2.3.5 Shelf-Facing Display

The capabilities described in this section are only available if you own the CBC Advanced Design
Module.)
If you own the Advanced Design Module for CBC, you can use the "Shelf Facing" Display option. The
shelf display option is particularly useful for beverage and packaged goods research. Here is an example
of a shelf facing display:
To achieve this look, the user supplies graphics in which each product is situated on a shelf "segment,"
such as:
When the graphics are placed side-by-side (with no space between them) and CBC by default
422 SSI Web v8
automatically places a border (usually black for most browsers) underneath the graphic, the resulting
display looks like a continuous shelf.
Note: when using the "shelf display" option, you will probably not want the brands to "shuffle" around
randomly from task to task, but will want them always located in the same position. On the Design tab,
make sure to change the Concept Sorting to Natural Level Order, and specify to sort by the natural order
of the Brand attribute using the Settings... button.
Shelf Facing Settings
To create a shelf display look, you must tell CBC how many total rows (shelves) are to be used, and how
many product concepts to display per row. For example, if you have 45 total products to display, you
might choose to display them on 5 rows (shelves). The number of products shown per row might be
something like:
Row # Number of Concepts on Row

1 7
2 8
3 10
4 9
5 11
Note that the widths of the shelves are entirely controlled by the width of the graphics used in the product
concepts, together with the number of graphics you have chosen to display per row. CBC doesn't re-size
your graphics--they are always displayed in their natural size.
Shelf Facing Format Settings
CBC controls the way the HTML tables are specified for shelf facing display. (We have provided CSS
"hooks" within the HTML for shelf facing display.)
There are a number of settings that control how the "shelf display" functions work. The following picture
illustrates the area each setting affects:
For Top Attribute Back ground Color, we suggest you pick the same color as is used in the background
of your graphics (Note: this color will only show on the screen if the graphics you develop have different
heights, or if you chose a transparent background color in your graphics). It is usually a good idea to
CBC Help 423
pick the same background color as used for the Task Back ground Color, which is selected on the
Format tab.
Separation between Shelves adds a blank line between the shelves.
None Option Position in Shelf Facing Tasks
In typical CBC task displays, the "None" option is always shown as the last concept in each task. You
can also use that option with shelf facing. However, shelf facing display permits more flexibility. You
can move the "none" text and the radio button associated with the text into the CBC tasks' header or
footer. This is not the "global" HTML header, but is the regular text header or footer that you edit under
Write Questionnaire | Edit.
To place the "None" concept in the header or footer of the task, type the function [%CBCNone( )%] in
either the header or footer question text boxes of the CBC choice task, accessed through Write
Questionnaire | Edit. You may want to supply some HTML to format this "None" concept exactly as
you want it.
424 SSI Web v8
2.3.6 Customized CBC Questions Using "Free Format"

Warning: This is an advanced feature that if not executed properly can result in errors in design,
data collection, and part-worth estimation. It requires careful, manual checking of Free Format
HTML syntax, and double-checking the validity of your .CHO file prior to fielding the study.
Introduction
This section describes how to create custom CBC surveys that go beyond the standard layout formatting
provided in SSI Web. To customize your CBC survey, you specify HTML in Free Format questions that
build the custom look and "tap into" CBC's experimental design plans (to display the appropriate
attribute text or graphics on the screen) using SSI Script system commands.
Designing/Programming the Questionnaire
1. Create a CBC Web exercise.

Create a CBC Web exercise the standard way by entering attributes and levels, generating a design,
and adding the CBC questions within the SSI Web questionnaire. (You can include the standard CBC
layout, Best-Worst layout, Constant-Sum layout, or Dual-Response None.)
2. Create a Free Format question.

To create a custom CBC question add a new Free Format question above your first CBC task (on the
same page). Name the question something similar to "FFTaskRandom1". The question name needs
to end with "Random1". This will tie this question to your first CBC question
(cbcExerciseName_Random1).
NOTE: We suggest creating just one Free Format question now. Once you get it working you can
then copy and paste the Free Format question multiple times so that you eventually will have a Free
Format question for each CBC task.
To create the custom HTML we suggest you start from a standard CBC questionnaire and copy the
relevant source HTML (using the View Source function in your browser) for that layout as produced by
SSI Web (this is a good starting point, but you will need to remove some things, such as hidden fields
calculating pagetimes, etc. in order for it to work properly).
If you are applying Styles to your survey, you may wish to refer to the styles documentation to see
how to make your Free Format CBC questions use the style settings that are applied to standard
CBC questions.
3. Insert SSI Script for level text.

In the HTML that you create for the Free Format question, insert SSI Scripting that "taps into" the
attribute text and the CBC experimental design. Use the following function to return the text for levels
in the CBC design:
[%CBCDesignLevelText (ExerciseName_CBCQuestionName, ConceptPosition#, LevelTextPosition#)

%]
See the example below. After you are finished test your work to ensure that the levels in your Free
Format question are coming up correctly. Test the survey locally (Test | Test Survey Locally). You
CBC Help 425
should be able to compare the levels being shown with the regular CBC question on the same page.
The text and graphics for the standard CBC concepts and the custom CBC concepts should match.
<table>
<tr>
<td>[%CBCDesignLevelText (cbc_Random1, 1, 1) %]</td>
</tr>
<tr>
</tr>
<tr>
</tr>
</table>
4. Add input row to the bottom of the task.

You now need to add a way for your custom Free Format CBC task to collect respondent input. In the
Free Format question go to the "Variable Names" section and add the appropriate variables. See the
examples below.
Standard CBC Tasks

Standard CBC Tasks only need one input variable. Name it "response". The whole variable name
should be named "FFTaskRandom1_response" and be of type "radio". Next include the input HTML in
the HTML section. We recommend clicking the pencil icon and then using the hammer and wrench
icon in the tool bar of the text editor. This will automatically insert the correct HTML. The value of the
input then needs to be adjusted to account for any possible CBC Design Concept Sorting. To do this
we use the CBCDesignConceptValue SSI Script function.
[%CBCDesignConceptValue (ExerciseName_CBCQuestionName, ConceptPosition#)%]
<tr>
<td class="input_cell clickable">
[%RadioSelect(FFTaskRandom1_response,
CBCDesignConceptValue(cbc_Random1, 1))%]
</td>
</td>
426 SSI Web v8
</td>
</td>
</tr>
Create as many radio buttons as there are concepts in your CBC Task. For the None Option do not
use CBCDesignConceptValue.
Here is an example of how you would include a radio button if the None Option was your 5th concept:

[%RadioSelect(FFTaskRandom1_response, 5)%]
</td>
Best-Worst CBC Tasks

Follow the instructions for the Standard CBC Tasks above except you will need to create two rows or
groups of radio buttons. Create two variables of type "radio". One variable for the best row named
"responsebest" and one variable for the worst row named "responseworst". Here is an example of the
HTML to include:
<tr class="cbc_best_row">
<td class="input_cell clickable best_worst_graphics">
[%RadioSelect(FFTaskRandom1_responsebest,
</td>
</td>
</td>
</td>
</tr>
<tr class="cbc_worst_row">
[%RadioSelect(FFTaskRandom1_responseworst,
</td>
</td>
CBC Help 427

</td>
</td>
</tr>
NOTE: The CSS classes "cbc_best_row", "cbc_worst_row", "best_worst_graphics" included in the

above HTML will turn on the default Best-Worst select graphics (green check, red x).
Constant Sum CBC Tasks

Follow the instructions for the Standard CBC Tasks above except you will need to create an input box
for each concept. For example if you have 4 concepts per CBC Task then you will need to create the
following variables of type text (Numeric): "response1", "response2", "response3", "response4", etc.
Because of possible Concept Sorting in the CBC design you will need to dynamically create the
names for the input in the HTML section using the SSI Script function CBCDesignConceptValue.
Here is an example:
<tr>
<td><input name="FFTaskRandom1_response[%
CBCDesignConceptValue(cbc_Random1, 1)%] type="text" size="5"></td>
</tr>
NOTE: SSI Web's syntax checker will warn you that it cannot find a "_response1" variable in the
HTML syntax etc. corresponding to the variable names you specified in the Free Format "Variable
Names" section. That is because you are dynamically inserting the integers at runtime using the
CBCDesignConceptValue command syntax. Therefore, you can ignore this warning.
For CBC Constant Sum you might also want to display a totals box. To do so, use this function:
[%DisplayTotal(FirstQuestionName, LastQuestionName, Width)%]
For example:
[%DisplayTotal(FFTaskRandom1_response1, FFTaskRandom1_response4, 3)%]
Dual-Response None CBC Tasks:

First, make sure you have specified that you wish to use the Dual-Response None Option on the
Format tab of the CBC Exercise Settings dialog. Add another variable within your FFTaskRandom1
Free Format question called "none" (FFTaskRandom1_none). Here is the HTML you would need to
include:
<input name="FFTASKRandom1_none" type="radio" value="1">Yes<br>

428 SSI Web v8
<input name="FFTASKRandom1_none" type="radio" value="2">No<br>
5. Suppress the regular CBC questions.

You don't actually delete them from the questionnaire. We have created a fast way to toggle them to
a suppressed mode. To suppress the regular CBC questions, add the following HTML comment to
the Header 1 field of your CBC random choice tasks:

Including the above text in Header 1 of a random CBC task will remove the regular CBC task from
view. It will also cause your Free Format CBC question to be verified the same way as the regular
CBC task (require answers etc.). Finally it will copy the respondent answers given to your Free
Format question the regular CBC database variables. This allows you to perform analysis as if the
regular CBC questions were seen.
6. Test saving data.

Test run the survey again, this time making sure that your answers are being recorded properly both in
the Free Format question and in the regular CBC question. If you have specified a concept sort order
(on the Design tab of the CBC Exercise Settings dialog), CBC sorts the concept order prior to
displaying the concepts, so you need to verify the final data file to ensure that the answers correspond
to the actual product concepts you chose.
7. Copy and Paste.

Once you are confident that your first Free Format custom CBC question is displaying correctly and
saving correctly, copy and paste the question multiple times so that there is one per regular CBC
task. Make sure to change the HTML references from "Random1" to "Random2", "Random3", etc.
Finally test again verifying all of the questions. You might choose to turn the CBC questions back on
to verify that all of your custom tasks are displaying correctly as compared to the actual tasks (see
step #5 above).
We cannot stress enough how important it is for you to test this well. We suggest the following: while
taking the questionnaire, print each question (with your answer). Download your data, export the data
to a .CHO or .CHS file (.CHS is used for constant-sum layout), and compare the attribute levels
recorded in the .CHO or .CHS file along with your answers to the printed questionnaire.
You might consider using the SSI Script functions CBCAttributeValue and CBCDesignLevelValue.
These functions return an integer that corresponds to the attribute or level being shown. This can help
with advanced conditional logic (Unverified Perl etc.).
We strongly encourage you to test your custom CBC questionnaire thoroughly, including check ing the
final data, to ensure that the attribute levels are presented correctly in the custom choice task s, that the
corresponding design information is being correctly saved to the .CHO or .CHS file, and that respondent
answers to the custom choice task s are also being stored correctly in the .CHO or .CHS file.
CBC Help 429
2.3.7 Volumetric CBC

(This is an advanced topic, and requires customized data processing of the .CHS file. See the
section in this documentation on .CHS File Format for file format information.)
In some situations, it might make sense to use allocation-based choice tasks, but to not force the
answers to sum to any particular value. For example, for FMCG (fast-moving-consumer-goods) we might
ask respondents how many of each product within a richly defined task they would purchase. There is
some debate in the industry whether respondents can complete such a task reliably; but, if they could
provide accurate answers to these questions, it would allow the researcher to examine whether the
overall category volume would increase or decrease given a certain competitive scenario. Furthermore,
we would be able to report absolute shares of unit volume using the market simulator rather than just
relative shares of preference.
We do not necessarily know the best way to model such data, but we offer the following suggestion
(which has been applied by other experienced users as well):
1. Use a script-based *program to modify the .CHS file. First, change the header for each
respondent so that it reflects that a "None" concept was asked in each task (though you didn't
actually display a "None" concept in the questionnaire).
2. Add a new row of zeros to each choice task, reflecting the "None" concept.
3. For each respondent, scan the choice tasks and determine the largest volume of product ever
"purchased" (the sum of allocations across concepts within a choice task). Call that volume
max_volume (each respondent has a different value for max_volume). For each respondent, set
the "None" alternative to receive zero allocation for that one choice task reflecting the highest
max_volume.
4. For each respondent, set the allocation for the "None" concept equal to max_volume minus
the volume "purchased" in each other task.
5. Estimate individual-level part-worth utilities using CBC/HB.
6. Import the part-worth utilities into the SMRT market simulator. Merge a weighting variable
(through SMRT's File | Merge Variables) for each respondent, equal to max_volume. Use
max_volume as the weight in weighted simulations (select Weights... from the Scenario
Specification dialog). Include the "None" concept in market simulations (set the "None" weight to
1.0 under the Advanced Settings within the Scenario Specification dialog.) The resulting shares
of preference are shares of volume. The share allocated to the "None" concept reflects potential
sales volume not captured by the products in this particular simulation scenario.
Notes and Cautions:
One should take care that the use of max_volume as a weight does not result in a few respondents
exerting too much weight on the final population estimates. Furthermore, relying on max_volume as
the definitive potential volume "purchased" for each respondent and the relative weight against which the
"None" parameter is scaled places a great deal of reliance on this one data point from a single choice
task (or a single typographical error). If a respondent accidentally records an erroneous and
unrealistically high volume for a single task, it can greatly affect the scaling of this respondent's data with
respect to the "None" parameter and this respondent's influence upon the simulated shares of
preference. Another approach might be to solicit max volume directly from respondents in a question
that precedes the CBC questions. For example, we could ask respondents to think back and identify
430 SSI Web v8
the maximum number of units of product they bought in any one shopping trip in the past 12 months. If
a given respondent reports “7,” then we might word the CBC questions like this: In the following
questions you can choose as many or as few of each widget as you like, but no more than 7.” We still
do the volumetric analysis but now our respondent-specific max_volume measure is less prone to
unfortunate typos.
CBC/HB normalizes the allocations within each task to sum to a constant value. For this reason, one
should reapply the max_volume weight in weighted simulations.
* We have written such a program to perform steps 1-4 above, and it is available for free download from
http://www.sawtoothsoftware.com/support/downloads/tools-scripts.
CBC Help 431
2.4 CBC Paper-and-Pencil Studies

SSI Web lets you run your CBC studies via Web, over PCs (CAPI installation), or as paper-and-pencil
questionnaires (sometimes a "paper-based questionnaire" is one that you are implementing via a 3rd
party web-interviewing platform). The process of running paper-based CBC studies is very similar to
computer-based studies. You should set up your attributes, levels, and compose your choice tasks in
the same manner as if you were going to run a computerized study. However, you should probably
choose plain black (text) on white (background) for questionnaire colors, unless you plan to print the
questionnaires in color.
For CBC, even though you will not have a typical "randomized" design with paper-based questionnaires
(with, say, 500 questionnaire versions), you should still use CBC's random choice tasks. Rather than
having so many possible versions of the questionnaire (perhaps a unique version for each respondent),
you will create just a few versions (design blocks) of the questionnaire and assign respondents
(randomly) to take one of the different questionnaire versions. It is not necessary that an equal number
of respondents complete each block of the questionnaire to obtain satisfactory utility estimates.
We generally recommend that you include enough versions of the questionnaire so that the number of
random choice tasks times the number of questionnaire versions is greater than or equal to 80
(assuming no prohibitions, and typical attribute level specifications). However, we urge you to spend
extra time looking at the design efficiency report that was created when you generated your paper-and-
pencil design plan. With very few questionnaire versions, the design seed can have an effect on design
efficiency. Also, you may want to take further steps to generate "dummy" response data (holding the
number of respondents constant) under different design plans (e.g. three versions vs. four versions, etc.).
Please see more information on testing your design in this documentation.
Saving the Questionnaire
When you click the Field | Create Paper & Pencil Interviews… button, each version of your
questionnaire is saved to a separate file within the Paper-and-Pencil folder within your study directory.
The files are named STUDYNAME_EXERCISENAME_V#.htm, where STUDYNAME is the four-character
or less study name for your project, and # is the version number of the questionnaire (always beginning
with version #1). These files are in HTML format and may be opened with most any recent word
processing software.
You should do any additional formatting you like to make the questionnaire appear as you wish prior to
printing.
Note: It is helpful to print the version# and task# with each task in a paper-and-pencil study. This gives
an additional reference while performing data entry, helping ensure that you do not introduce errors when
specifying the version numbers seen by respondents. An easy way to automatically label your choice
tasks is to add a function [%CBCVersion( )%] in the header or footer of the choice task. When you do
this, each task includes a label such as D - 1, T - CBCRAN1, meaning "Design #1, Task CBCRAN1."
Fielding the Questionnaire
Make sure when fielding the questionnaire that you are able to match the different questionnaire versions
correctly with respondent answers. You should also try to randomly distribute the questionnaire versions
among respondents, so that roughly equal numbers of respondents complete each version. It is not
necessary that an equal number of respondents complete each block of the questionnaire to obtain
satisfactory utility estimates. Even somewhat large deviations from equal distribution will not affect the
432 SSI Web v8
quality of your results very much.
Data Entry
After you have collected the data, you prepare a .csv file containing the respondent answers. This
file follows a required .csv (comma separate values) format, with specific labels in the first row of
the file. To see that format, prepare a template for yourself by clicking Field | Create
Accumulated Data Template File.... A file named Accumulated Data.csv is saved to your study
folder. Use this template to arrange your data, and make sure to use the prescribed header row
within your data file.
The layout is:
Column 1: Respondent#
Column 2: Version#
Column 3: CBC answer #1, etc.
Any missing data are coded as blank (meaning, if you viewed the data file using Excel, the cell
would be empty; if you viewed the file with a text editor, the missing value would look like two
consecutive commas with no text in between). If the respondent skipped all CBC questions, then
you still must provide a Version# for the missing respondent (if you want the respondent to be a
case within the data set).
Constant-Sum Data: multiple responses are provided for each task, one per concept. During
analysis, responses are normalized to 100% within task.
Dual-Response None: the Dual-response None answer (coded as 1 or 2) is indicated after each
choice task response.
Accumulate Respondent Answers:
Click Field | Accumulate Paper & Pencil Data... and browse to the .csv file containing your
respondent data.
CBC Help 433
2.5 Analyzing the Data

2.5.1 Preparing Conjoint Data for Analysis
CBC data are typically analyzed using Counts and HB (hierarchical Bayes estimation). Some
researchers also use Latent Class for developing market segments. All these tools (Counts, Latent
Class, and HB) are available right withing the SSI Web interface. (You could also export the CBC data
to a .cho/.chs file and use the standalone versions of Latent Class and CBC/HB if you wish.)
Once utilities are estimated, the results may be used within the Online Market Simulator, or within the
simulator provided within the older SMRT platform. You received a copy of SMRT for analyzing CBC data
with your purchase of CBC.
Prior to conducting analysis, you should have downloaded your data from the Web server to your study
directory on your hard drive. We recommend you use the Data Management capabilities within SSI Web
to examine the records for any respondents that should be deleted (speeders, straightliners, etc.). You
may delete any suspect records within the Data Management area.
434 SSI Web v8
2.5.2 Counting Analysis

Counts provides quick and automatic calculation of the main effects and joint effects for collected CBC
data. It calculates a proportion of "wins" for each level, based on how many times a concept including
that level is chosen, divided by the number of times a concept including that level appeared in the choice
task.
When you run Counts (Analysis | Calculate CBC Counts...), the results are displayed in the report
window. You can weight the data, select subsets of respondents or tasks to process, or include banner
points (segment the respondents by columns).
Following are data for a sample of 100 respondents, each of whom answered 8 randomized choice tasks,
each including 4 concepts and a "None" option. The data are real, but the attribute levels have been
disguised. Main effects and two-way interactions are included in the report.
Brand
Total
Brand A 0.387
Brand B 0.207
Brand C 0.173
Brand D 0.165

D.F. 3
Shape
Shape 1 0.259
Shape 2 0.245
Shape 3 0.195

D.F. 2
Significance: p < .01
Size
Large Size 0.263
Medium Size 0.240
Small Size 0.196

D.F. 2
Price
Price 1 0.132
Price 2 0.175
Price 3 0.254
Price 4 0.372

CBC Help 435
D.F. 3
None
None chosen: 0.068
Brand x Shape
Brand A Shape 1 0.415

Brand B Shape 1 0.232
Brand CShape 1 0.201
Brand DShape 1 0.188

D.F. 6
Significance: not sig
(Due to space considerations, we have not shown the remainder of the report. Five additional
tables would follow specifying the remaining two-way joint effects: Brand x Size; Brand x Price;
Shape x Size; Shape x Price; Size x Price.)
Each of the main effects is the proportion of times when a concept containing that attribute level occurs
that the concept is selected by respondents.
Brand A was the most popular, having been selected 38.7 percent of the times it occurred. Brand D was
least popular, having been selected 16.5 percent of the time it occurred. Since there are four brands,
and also four concepts per task, each brand appeared once in every task. The sum of proportions for the
four brands (not shown) is 0.932. The balance, 0.068, is the proportion of tasks in which respondents
selected "None."
Price level 1 was the most expensive, and concepts at that price level were only selected about a third
as often as concepts at the lowest price, level 4. Since Price also has four levels, each level appeared
exactly once in each task, and the sum of their proportions is identical to the sum for Brand.
The Size and Shape attributes only had three levels, so their levels sometimes appeared twice in a task.
That produces proportions with smaller sums. If a level appears twice in the same task, and if one of
the concepts including it is selected, then the other concept is rejected. When an attribute has fewer
levels than the number of concepts in a task, the sum of its proportions will be lowered. When making
comparisons across attributes it is useful first to adjust each set of proportions to remove this artifact.
One way to do so is to divide all the proportions for each attribute by their sum, giving them all unit
sums.
Counts reports a Chi Square statistic for each main effect and joint effect indicating whether the
436 SSI Web v8
proportions in that table differ significantly from one another. We refer to that Chi Square test as the
"Within Att. Chi-Square." In the case of a main effect count, the Chi Square indicates whether levels of
that attribute differ significantly in their frequency of choice. Beware of interpreting the Chi Square from
aggregate counts as a measure of "Importance" for an attribute or assuming that the main-effect Chi
Square test that is not significant indicates that the attribute had little impact on choice. Disagreement
between individuals on what level is preferred can mask the impact of an attribute when respondent
choices are aggregated. For example, if half of the respondents strongly prefer Brand A over Brand B,
and the other half feels exactly the opposite, the aggregate count proportions will be equal, and the Chi
Square will also be zero. In that case, we would be in error to infer that brand had no impact on choice
for individuals.
The tests for joint effects measure differences among the proportions in the table beyond those due to
the main effects. For example, suppose the proportions in the first row of a joint-effect table were all just
half the size of corresponding proportions in the second row. Such differences would be due to the main
effect for the row attribute, and would not show up as a large Chi Square for the joint effect. A large Chi
Square value suggests a significant interaction effect between the two attributes. (No Chi Square is
reported for 3-way joint effects).
Each effect is classified "not significant," "significant with p< .05," "significant with p< .01." Actual
values of Chi Square and the appropriate number of degrees of freedom are also reported, so you can
consult a table of Chi Square to determine the precise level of significance for each effect.
The Chi Square tests reported by Counts are computed differently than those reported by Logit, and will
not agree precisely, though effects with highly significant Chi Square values for either module should also
have highly significant values for the other.
Note: The Chi-Square interaction tests are not as sensitive as the 2-Log Lik elihood tests. CBC software
provides an automated Interaction Search Tool that automatically investigates all potential 2-way
interaction effects under aggregate logit.
As with other conjoint methods, it is often useful to summarize choice data with numbers representing
the relative importance of each attribute. With utility values we base importance measures on
differences between maximum and minimum utilities within each attribute. However, with proportions,
corresponding measures are based on ratios within attributes. To summarize relative importance of each
attribute we might first compute the ratio of the maximum to the minimum proportion for each attribute,
and then percentage those ratios to sum to 100. We generally do not recommend computing attribute
importances using aggregate (summary) data. Attributes on which respondents disagree will appear to
have less importance in the aggregate, even though respondents feel very strongly about their differences
in opinions. We recommend computing attribute importances using Latent Class or HB utilities.
The joint effects tables provide the same kind of information as main effects, but for pairs of attributes
rather than attributes considered one-at-a-time. For example, consider the following table for Brand and
Price:
Price1 Price2 Price3 Price4 Avg
Brand A 0.262 0.320 0.398 0.570 0.387

Brand B 0.083 0.146 0.254 0.347 0.207
Brand C 0.104 0.100 0.163 0.321 0.172
Brand D 0.078 0.129 0.206 0.249 0.165
Average 0.132 0.174 0.255 0.372

CBC Help 437
We have labeled the rows and columns for clarity, and also added a row and column containing averages
for each brand and price. Comparing those averages to the main effects for Brand and Price, we see that
they are identical to within .001. The similarity between main effects and averages of joint effects
depends on having a balanced design with equal numbers of observations in all cells. That will only be
true with large sample sizes, when either the Complete Enumeration or Shortcut methods are used, and
when there are no prohibitions.
Finally, counts data are ratio-scaled. A count proportion of 0.30 versus 0.15 means that respondents on
average chose (preferred) the first level twice as much as the second. Since preference for an attribute
level depends upon the desirability of the other alternatives within that same attribute, it is not appropriate
to directly compare a count proportion from one attribute level to a level from a different attribute.
Some Notes on Counting Analysis
As mentioned at the beginning of this section, Counting analysis is a quick way to summarize the
results of choice data. However, for getting the most from your CBC data, we recommend more
sophisticated means of analysis, such as Logit, Latent Class, or HB. Counts analysis reflects some
known biases that can prove problematic in some situations.
Given a large enough sample size, the number of times each level was displayed should be nearly
balanced, in a 1-way, 2-way and even 3-way sense (though CBC's design methods only pay attention to
1- and 2-way representation). But with smaller sample sizes, random imbalances in the design can
distort counts proportions. For example, if a particular brand level happened to be shown at a low price
more often than other brands, the count proportion for that brand could be distorted upward. Other
methods of analysis (Logit, Latent Class, or HB) are not subject to this difficulty.
The counting analysis produces different results from the logit approach. CBC's counting results are just
the proportions of times when levels (or combinations of levels) are offered that they are in fact chosen.
Since all attributes are constantly varying, the counting result for a specific attribute level does not
depend on that level alone. Even an extremely undesirable level may be chosen because it is paired with
desirable levels. The result is that the main-effects counts proportions are biased in the direction of
being flatter than the results of logit simulations, in which all omitted attributes are assumed to be
constant.
Other effects may work in an opposite direction. Under the Complete Enumeration, Shortcut, and to a
lesser degree the Balanced Overlap method, CBC tries not to show the same attribute level twice in the
same task. This means that a strong level has weaker levels as competition, which tends to accentuate
differences between levels, partly overcoming the effect described above. For example, consider the
case of the brand x price joint effect counts table, often used to reflect demand curves.
If attributes have different numbers of levels, the maximum possible choice proportion differs between
them, which can make it even more difficult to compare results between attributes.
438 SSI Web v8
2.5.3 Estimating Utilities with Logit

Note: Aggregate Logit has been used for more than three decades in the analysis of CBC data. It
is useful as a top-line diagnostic tool (both to assess the quality of the experimental design and to
estimate the average preferences for the sample), but we recommend using Latent Class or HB for
developing final and more accurate results.
When you run Logit (Analysis | Estimate CBC Utilities - Logit...), the results are displayed in the report
window and utilities are saved into a subfolder within your project directory. You can weight the data,
select subsets of respondents or tasks to process.
What are Utilities?
A utility is a measure of relative desirability or worth. When computing utilities using logit, latent class,
or HB every attribute level in a conjoint project is assigned a utility (also referred to as a part worth). The
higher the utility, the more desirable the attribute level. Levels that have high utilities have a large
positive impact on influencing respondents to choose products.
When using logit, Latent Class, or HB, the raw utilities are zero-centered within each attribute. For
example:
Level Utility
$300 -0.6
$200 0.1
$100 0.5
This example shows respondents preferring lower price levels to higher ones. For information on
interpreting conjoint utilities, please see the following section entitled Interpreting Conjoint Utilities.
Choosing Effects for Logit
By default, CBC estimates utilities for all main-effects. Main-effects reflect the impact of each attribute
on product choice measured independently of the other attributes. Main-effect models are the simplest
models, resulting in a single utility value associated with each attribute level in the study.
You can add additional terms to the model to account for two-way attribute interactions. For example, if
the combination of "Red" with "Mazda Miata" results in greater preference than the main-effect utilities
would suggest, an interaction term can capture and reflect that synergy. Interaction terms can also
reflect differences in price sensitivity for different brands.
We suggest only adding interaction terms that result in a significant improvement in the overall fit of the
model. Including too many terms in a model can lead to overfitting, which has the potential of modeling a
good deal of noise along with true effects.
When using aggregate logit, it is important to investigate whether including interaction terms can
significantly improve the fit of the model. With methods that recognize respondent differences (Latent
Class, and HB), many complex effects (including, but not limited to, interactions) can be reflected in
market simulations using only main-effects. It is our experience that including interaction terms in logit
models can often significantly improve predictability of the model, but that those same terms added to
Latent Class or HB are not as valuable and can even, in some cases, be detrimental. Thus, with little
effort, one can achieve excellent results with Latent Class or HB using CBC's default main-effects. Even
CBC Help 439
so, one should be on the look out for opportunities in Latent Class and HB to improve the models by
including interaction effects.
Running Logit Analysis
Logit analysis is an iterative procedure to find the maximum likelihood solution for fitting a multinomial
logit model to the data. When only main-effects are estimated, a part worth is produced for each attribute
level, which can be interpreted as an average utility value for the respondents analyzed. After logit
converges on a solution, the output is displayed in the results window, and the results are saved into a
subfolder within your project directory. You can run logit multiple times, and the results will overwrite the
previous run. (You can make copies of your logit output folder to archive any runs you wish to keep
around for further reference.)
The computation starts with estimates of zero for all effects (utilities), and determines a gradient vector
indicating how those estimates should be modified for greatest improvement. A step is made in the
indicated direction, with a step size of 1.0. The user can modify the step size; a smaller step size will
probably produce a slower computation, but perhaps more precise estimates. Further steps are taken
until the solution stops improving.
For each iteration the log-likelihood is reported, together with a value of "RLH." RLH is short for "root
likelihood" and is an intuitive measure of how well the solution fits the data. The best possible value is
1.0, and the worst possible is the reciprocal of the number of choices available in the average task. For
these data, where each task presented four concepts plus a "None" option, the minimum possible value
of RLH is .2.
Iterations continue until the maximum number of iterations is reached (default 20) or the log-likelihood
increases by too little (less than 1 in the 5th decimal place); the gradient is too small (every element less
than 1 in the fifth decimal place).
The user also has the option of saving variances and covariances of the estimates. The square roots of
the variances are equal to the standard errors, and are always displayed. The default is not to display
variances and covariances.
Here is the output for a computation, using the same data as were used in the second example
described in the previous chapter on Counting analysis (about 100 respondents, each of whom answered
8 randomized choice tasks, each including 4 concepts and a "None" option).
CBC System Multinomial Logit Estimation

Copyright 1993-2012 Sawtooth Software
Main Effects
Specifications for this run:
Max iterations 20
Variances and covariances not saved
Step size 1.00000
Max change in loglike 8e-007
Iter 1 log-likelihood = -1471.20588 rlh = 0.24828

440 SSI Web v8
Converged.
Log-likelihood for this model = -1462.73822

Log-likelihood for null model = -1699.56644
Difference = 236.82822Chi Square =473.656

1 0.62150 0.05126 12.12462 1 1 Brand A
2 -0.05740 0.06021 -0.95331 1 2 Brand B
3 -0.26472 0.06411 -4.12943 1 3 Brand C
4 -0.29938 0.06509 -4.59957 1 4 Brand D
5 0.13859 0.04899 2.82895 2 1 Shape 1
6 0.07652 0.04962 1.54217 2 2 Shape 2
7 -0.21510 0.05199 -4.13734 2 3 Shape 3
8 0.15207 0.04895 3.10636 3 1 Large Size
9 0.04925 0.04939 0.99716 3 2 Medium Size
10 -0.20132 0.05201 -3.87086 3 3 Small Size
11 -0.52970 0.07101 -7.45947 4 1 Price 1
12 -0.22737 0.06409 -3.54773 4 2 Price 2
13 0.17347 0.05708 3.03928 4 3 Price 3
14 0.58361 0.05185 11.25616 4 4 Price 4
15 -1.07590 0.12434 -8.65299 NONE
The output starts with a listing of values that govern the computation. The values shown here are default
values, and the user will rarely need to modify them. The default value for maximum number of iterations
is 100, and this computation required only 6.
Logit analyses are often evaluated by Chi Square statistics. The procedure is to determine the log
likelihood that would be obtained, given the sample size and the nature of the data, if the estimated
effects were all zero. That log likelihood is compared to the log likelihood for the estimates obtained.
Twice the difference between those two log likelihood values is distributed as Chi Square, with degrees of
freedom equal to the number of parameters estimated.
The number of parameters estimated here is 11, obtained by adding the total number of levels and
subtracting the number of attributes. With 11 degrees of freedom, a Chi Square of about 25.0 would be
significant at the .01 level. The obtained value of 473.6 is safely larger than this, so we would conclude
that respondent choices are significantly affected by the attribute composition of the concepts.
Within each attribute, the effects sum to zero. That is because we actually omit one level for each
attribute in doing the estimation, and then supply a value afterward for the missing level that is equal to
the negative of the sum of the others. Our use of "effects coding" makes this possible. In the logit
analysis output, "effect" is synonymous with utility.
To the right of each estimate is a standard error, and to the right of that is a t ratio. The t ratio is a
measure of the significance of the difference between that level's effect and the average of zero for all
levels within the attribute.
(Additional statistics such as CAIC, Percent Certainty, and Relative Chi Square are for comparison to
runs produced by the Latent Class Module add-on. See the Latent Class manual for further details.)
CBC Help 441
When there are no interaction effects, as in this example, the relative attractiveness of a concept can be
assessed by adding up the effects for its component attribute levels. For example, consider these two
hypothetical concepts:
Concept 1 Concept 2
Effect Effect
Brand A 0.62150 Brand B -0.05740

Shape 3 -0.21510 Shape 1 0.13859
Small -0.20132 Large 0.15207
Price 3 0.17347 Price 1 -0.52970
Total 0.37855 -0.29644
These two hypothetical concepts are each "scored" by adding up the effects of their component attribute
levels. Concept 1 should be strongly preferred to Concept 2 by these respondents. In fact, we can go
one step further and estimate how strong that preference would be. If we exponentiate (take the antilog
of) each of the total values we can then express them as percentages to predict the proportion of
respondents who would choose each concept if they had to choose one or the other:
Total exp(total) Percent

Concept 1 0.37855 1.460 66.3%
Concept 2 -0.29644 0.743 33.7%
Total 2.203
If forced to choose between Concept 1 and Concept 2, about two thirds of these respondents should
choose Concept 1 and one third should choose Concept 2.
To show how the "None" parameter is used, we add a third item with total equal to -1.07590, the
estimate for choosing "None."
Total exp(total) Percent

Concept 1 0.37855 1.460 57.4%
Concept 2 -0.29644 0.743 29.2%
"None" -1.07590 0.341 13.4%
2.544
In a three-way contest between Concept 1, Concept 2, and the option of choosing neither, about 57%
should choose Concept 1, 29% should choose Concept 2, and 13% should choose "None."
To see how logit output and analysis by counting choices output are similar, we use the same procedure
to estimate the distribution of choices for four concepts differing only in brand, plus a "None" option. The
resulting numbers in the logit output should be similar to the main-effects for Brand as reported in the
output for analysis by counting choices.
Effect exp(effect) Prop- From Diff-

ortion COUNT erence
Brand A 0.62150 1.862 0.400 0.387 0.013
Brand B -0.05740 0.944 0.203 0.207 -0.004
Brand C -0.26472 0.767 0.165 0.173 -0.008
Brand D -0.29938 0.741 0.159 0.165 -0.006
None -1.07590 0.341 0.073 0.068 0.005
Total 4.655 1.000 1.000 0.000

442 SSI Web v8
Although the estimates derived from logit analysis are not exactly the same as those observed by
counting choices, they are very similar. The differences are due to slight imbalance in the randomized
design (and due to other factors mentioned at the end of the previous chapter). The estimates produced
by the logit calculation are slightly more accurate. With a larger sample size we might expect
differences between the two kinds of analysis to be even smaller.
After logit has finished running, results are displayed to the results window, including summary
information about the computation is displayed (the total number of tasks and time for the computation).
You are given the option to save the run to the utilities file. You can also print, or cut-and-paste the logit
report to a file.
We emphasize that logit analysis is an advanced statistical technique that requires understanding to be
used properly. We have made it easy to conduct logit analyses with CBC. The burden is on you to use
the results wisely. We urge you to consult appropriate reference materials. We also urge you to
consider the limitations of logit analysis relative to the Latent Class and HB (hierarchical Bayes)
estimation approaches.
Notes on Determining Significant Interaction Effects
By default, CBC's logit program calculates all main-effect utilities. Sometimes main-effects models can
be improved upon by adding interaction terms. We'll define an "improved" model as one that:
1. Significantly improves the model fit in terms of log-likelihood,
2. Improves the accuracy of the market simulator in terms of aggregate shares vs. fixed holdout
choice tasks or market shares.
We urge you to only include interaction terms in the model that significantly improve results. To
determine whether to include interaction terms, we suggest you first study the two-way tables provided
by Counts analysis. The Chi-square statistic is one indication of possible significant interaction, but we
suggest you look further. We also suggest plotting the Counts proportions with a scatter plot display to
verify that deviations from proportionality in the table seem plausible given what you know about buyer
preferences.
Most importantly, we suggest you conduct a "2 log-likelihood" test. We alluded to this test earlier in the
section with respect to determining whether the computed logit effects significantly improve our ability to
predict respondent choices relative to the null model (effects of zero). A 2 log-likelihood test can also be
used to determine whether the addition of an interaction term to the logit model significantly improves the
fit. We will illustrate with a simple example.
Assume a simple CBC design with two attributes, each with four levels. Under main-effects, a total of
eight part-worths result. Recall, however, that each four-level attribute is coded as just three parameters
in the logit model under effects-coding. Assume we run this main-effects logit model and achieve a log-
likelihood of -1500.00.
The interaction between two four-level attributes, each effects-coded as three parameters, results in an
additional 3 x 3 = 9 parameters added to the model. Suppose we include this interaction term and the
log-likelihood improves to -1490.00. Adding the interaction term has improved the log-likelihood by
10.00. Two times the log-likelihood is distributed as Chi-square. We refer to a Chi-square table
(available in any statistics textbook), and look up the p-value for a Chi-square of 20.00 with 9 degrees of
freedom (the number of additional parameters added to the model). We find that the p-value is roughly
CBC Help 443
0.02, suggesting a significant improvement in the model by adding the interaction terms with a
confidence level of 98%.
CBC software provides an automated Interaction Search Tool that automatically investigates all potential
2-way interaction effects under aggregate logit.
Technical Notes on Logit
CBC uses multinomial logit analysis (MNL) to estimate effects for attribute levels. Additional information
is provided by Hosmer and Lemeshow in Applied Logistic Regression (Wiley, 1989).
MNL is similar in many ways to multiple regression and discriminant analysis. Like those methods, it
seeks "weights" for attribute levels (or for combinations of them, if interactions are included in addition to
main effects). Those weights are analogous to "utilities" in conjoint analysis, and are computed so that
when the weights corresponding to the attribute levels in each concept are added up, the sums for each
concept are related to respondents' choices among concepts.
MNL assumes that the relative probabilities of respondents' choosing each concept within any task can
be estimated from the weights in the following way:
1. Sum the weights for the attributes appearing in each concept to get a value analogous to that
concept's "total utility."
2. Convert total utilities to positive values by exponentiating them. The resulting values may
be considered analogous to relative probabilities, except that they do not lie within the unit
interval.
3. Normalize the resulting values so that within each task they sum to unity, by dividing the
values for concepts within each task by their sum.
Suppose we have such probabilities for the concepts in one choice task. Then we can say that,
according to our model, the likelihood of a respondent's choosing each concept is equal to the probability
for that concept. If we assume choices for different tasks to be independent, then we can compute the
likelihood (under our model) of seeing any particular pattern of choices for any number of respondents
and tasks. That likelihood is just the product of a number of computed probabilities. If there are N
respondents, each having responded to k choice tasks, then the likelihood of that pattern of responses is
just the product of Nk such probabilities.
MNL chooses weights that maximize the likelihood of the observed pattern of respondent choices, using
probabilities derived from the weights as just described.
Since likelihoods are obtained by multiplying many probabilities together and each is less than unity,
likelihoods are usually very small positive numbers. It is more convenient to think about their logarithms,
and users of logit estimation usually speak of "log-likelihoods" rather than likelihoods themselves. This
is just a matter of convenience, because the set of weights that maximize the likelihood must also
maximize the log likelihood.
In CBC we also report another somewhat more intuitive measure, "root likelihood," which we abbreviate
"RLH." This is just the geometric mean of the probabilities corresponding to the choices made by
respondents, obtained by taking the Nk'th root of the product of the Nk probabilities. The best possible
value of RLH is unity, achieved only if the computed solution correctly accounts for all the choices made
in all tasks by all respondents.
444 SSI Web v8
The computation method used by CBC is iterative. It starts with a "null model" consisting of weights all
equal to zero, and computes the likelihood of the respondents' choices, given probabilities corresponding
to that initial solution. During that computation information is also accumulated about a "gradient
vector," indicating how the initial solution should be changed for greatest improvement. Those changes
are made, and a second iteration evaluates an improved solution, while also accumulating information
about how a third iteration might improve on the second.
The iterative process continues until one or more of the following occurs:
The number of iterations exceeds a limit. The default is a limit of 100.
The change in log-likelihood from one iteration to the next is less than a limit (convergence limit
1). The default is 1 in the fifth decimal place.
We do not anticipate that users will need to change these defaults, but it can be done by clicking the
Estimation Settings button. The dialog includes four parameters:
1. Step size (default = 1.0). This number governs the sizes of changes made in the
iterative computation. If set to a smaller number, such as .5 or .1, the computation will
be slower but may be somewhat more precise.
2. Maximum number of iterations (default 100)
3. Log-likelihood Convergence Limit (default is 1 in the fifth decimal place).
4. Report details. If set to Full, more information about variances and covariances of
estimates is provided in the output.
Effects Coding
Logit analysis, like least squares, suffers from the dependencies among attribute levels when all the
levels of an attribute are entered into the computation. For this reason one level of each attribute is
deleted automatically, although results for that level may be obtained by inference from the others, and
those results are provided in the output.
Logit deletes the last level (rather than the first, as done by Test Design). Remaining levels are
represented using "effects coding." This is a way of coding for levels so that the average effect within
each attribute will be zero. After the logit computation, we provide the estimate for each deleted level
just by computing the negative of the sum of the included levels. Logit analysis permits estimates not
just of main effects, but also of two-way interactions.
Tests of Significance
CBC offers two kinds of significance tests: a t ratio and an overall Chi Square test. Probably the more
useful is the Chi Square for the difference between two "nested" models, where the second includes all
effects of the first plus one or more others. Twice the difference between the log-likelihoods of those two
models is distributed as Chi Square, with degrees of freedom equal to the number of additional effects in
the second model. This test is performed automatically in each logit analysis, comparing the results of
that analysis to what would be obtained with no effects at all.
CBC Help 445
The test can also be computed manually to compare any other nested runs, such as one run with all
main effects and another with just a subset, or one with just main effects and another with main effects
and one or more interactions. (The Chi Square of the difference is the difference of the Chi Squares.)
The other test is a t ratio provided for each attribute level. This tests the difference between that level and
the average of all levels for that attribute (which is zero). These t ratios can provide useful guidance, but
for measuring whether an effect is significant the overall Chi Square test is preferable.
The standard errors used to compute t tests are taken from the matrix of covariances among estimates
that is obtained by inverting a sum of squares and cross-products matrix. If many parameters are being
estimated or there are relatively few observations, this matrix may be ill-conditioned (having determinant
less than 1.0E-10). in this case a "ridge adjustment" is made by adding unity to each diagonal position
of the sum of squares and cross-products matrix, reinverting, and displaying an error message. We do
this so the computation will provide useful information; however, the presence of a "ridge adjustment"
message indicates the solution to be unsatisfactory, and you should probably remove some interaction
terms and re-estimate.
446 SSI Web v8
2.5.4 Estimating Utilities with Latent Class

Note: Latent Class is often used to discover market segments (for use as banner points) from CBC
data. But, Latent Class is not used nearly as often as HB among our customers for producing the
final utilities that are used when developing market simulators.
SSI Web includes a built-in Latent Class estimation routine. You may also decide to analyze data
separately using our standalone Latent Class software.
When you run Latent Class (Analysis | Estimate CBC Utilities - Other Methods | Latent Class...), the
results are displayed in the report window and utilities are saved into a subfolder within your project
directory. You can weight the data, select subsets of respondents or tasks to process.
Latent Class is a utility estimation method for use with CBC (Choice-Based Conjoint) or MaxDiff data. It
divides respondents into segments having similar preferences based on their choices in CBC
questionnaires. It uses latent class analysis for this purpose, which simultaneously estimates part worth
utilities for each segment and the probability that each respondent belongs to each segment. Latent
Class is an integrated analytical component within the SSI Web platform (and we also provide a
standalone Latent Class software system). Results from latent class runs (part worths in a generic .hbu
file format, plus segment membership as filters/banner points) can be taken forward into market
simulations.
Latent class has a role analogous to that of CBC's logit program, but rather than finding average part
worth utilities for all respondents together, it detects subgroups with differing preferences and estimates
part worths for each segment. The subgroups have the characteristic that the respondents within each
group are relatively similar but the preferences are quite different from group to group. You may specify
how many groups are to be considered, such as the range of 2 through 6. A report of the analysis is
shown on the screen and appended to a log file (stored in a subfolder of your project directory), and the
part worths for subgroups along with each respondent's probabilities of membership in the groups are
stored in a generic .hbu utility file (also in a subfolder of your project directory) for subsequent analysis or
later use by the simulator.
The latent class estimation process works like this:
1. Initially, select random estimates of each group's utility values.

2. Use each group's estimated utilities to fit each respondent's data, and estimate the relative
probability of each respondent belonging to each group.
3. Using those probabilities as weights, re-estimate the logit weights for each group. Accumulate
the log-likelihood over all groups.
4. Continue repeating steps 2 and 3 until the log-likelihood fails to improve by more than some
small amount (the convergence limit). Each iteration consists of a repetition of steps 2 and 3.
Latent class reports the part worth utilities for each subgroup or "segment." Latent class analysis does
not assume that each respondent is "in" one group or another. Rather, each respondent is considered to
have some non-zero probability of belonging to each group. If the solution fits the data very well, then
those probabilities approach zero or one.
Use of latent class analysis as a segmentation method has been examined in many articles in the
marketing literature, and it has been found to be effective for that purpose. Thus, it holds promise of
solving the problems occurring with aggregate conjoint analysis:
There is usually sparse information at the individual level for defining segments.
If there truly are segments with different preferences, an aggregate analysis may give incorrect
answers.
CBC Help 447
Latent class became popular in about the mid-1990s as a tool for analyzing CBC data sets. The model
typically provided more insight about the structure of respondent preferences than aggregate logit, and
the resulting market simulations were usually more accurate than similarly defined aggregate models.
The latent class approach was effective in reducing the negative effects of the IIA assumption in logit
analysis.
For more information on our latent class analysis approach, please see the full standalone Latent Class
manual.
448 SSI Web v8
2.5.5 Estimating Utilities with HB

Note: HB is the most commonly used utility estimation approach among our users for developing
the utilities used in market simulators. We strongly recommend you use HB estimation for your
final models. Counts and Logit are useful for obtaining quick top-line summary results. Latent
Class is especially useful for segmenting respondents.
SSI Web includes a built-in HB estimation routine. As an alternative, you may choose to analyze data
separately using our standalone CBC/HB software.
When you run HB (Analysis | Estimate CBC Utilities - HB...), the results are displayed in the report
window and utilities are saved into a subfolder within your project directory. You can select subsets of
respondents or tasks to process.
Introduction
The earliest methods for analyzing choice-based conjoint data (e.g. the 70s and 80s) usually did so by
combining data across individuals (e.g. counting and aggregate logit). Although many researchers
realized that aggregate analyses could obscure important aspects of the data, methods for estimating
robust individual-level part-worth utilities using a reasonable number of choice sets didn't become
available until the 90s.
The Latent Class Segmentation Module was offered as the first add-on to CBC in the mid-90s, permitting
the discovery of groups of individuals who respond similarly to choice questions.
Landmark articles by Allenby and Ginter (1995) and Lenk, DeSarbo, Green, and Young (1996) described
the estimation of individual part worths using Hierarchical Bayes (HB) models. This approach seemed
extremely promising, since it could estimate reasonable individual part worths even with relatively little
data from each respondent. However, it was very intensive computationally. The first applications required
as much as two weeks of computational effort using the most powerful computers available to early
academics!
In 1997 Sawtooth Software introduced the ICE Module for Individual Choice Estimation (now since
retired), which also permitted the estimation of part worths for individuals, and was much faster than HB.
In a 1997 paper describing ICE, we compared ICE solutions to those of HB, observing:
"In the next few years computers may become fast enough that Hierarchical Bayes becomes
the method of choice; but until that time, ICE may be the best method available for other than
very small data sets."
Over the next few years, computers indeed became faster, and our CBC/HB software soon could handle
even relatively large-sized problems in an hour or less. Today, most datasets will take about 15 minutes
or less for HB estimation.
HB has been described favorably in many journal articles. Its strongest point of differentiation is its ability
to provide estimates of individual part worths given only a few choices by each individual. It does this by
"borrowing" information from population information (means and covariances) describing the preferences
of other respondents in the same dataset. Although ICE also made use of information from other
individuals, HB did so more effectively and required fewer choices from each individual.
Latent Class analysis is also a valuable method for analyzing choice data. Because Latent Class can
identify segments of respondents with similar preferences, it is an additional valuable method. Recent
research suggests that default HB is actually faster for researchers to use than LC, when one considers
CBC Help 449
the decisions that should be made to fine-tune Latent Class models and select an appropriate number of
classes to use (McCullough 2009)
Our software estimates an HB model using a Monte Carlo Markov Chain algorithm.
We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have
been helped by several sources listed in the References. We have benefited particularly from the
materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing
Association's Advanced Research Technique Forum, and from correspondences with Professor Peter
Lenk.
For more information about HB estimation, please see the CBC/HB Manual.
450 SSI Web v8
2.5.6 Moving Data from SSI Web to SMRT for Analysis

from SSI Web into a preliminary analysis program such as hierarchical Bayes, and then afterward to
SMRT.
SSI Web project.
SMRT.
HB and Latent Class manuals for details) to produce a data file containing respondent part-worths (a
.HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting
utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager |
Import… and following the prompts to browse to and select your .HBU or Latent Class probability file
(.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin
simulations.
menu, select File | Export Data | All Data…, select the respondent numbering method (the
numbering method must match the method you chose when exporting the conjoint data or
estimating utilities), and export the data you wish to merge into SMRT (only numeric data may be
merged). Select either a .CSV file (comma-delimited text-only file) or a fixed text-only file (.TAB).
CBC Help 451
452 SSI Web v8
2.5.7 .CHO File Format

You can export CBC data (discrete choice) from SSI Web to a .CHO format. The .CHO format files are
used in many other Sawtooth Software programs for analyzing CBC data, including Latent Class, and
Hierarchical Bayes (HB) applications. Some power-users modify the .CHO file (usually with an
automated script outside of SSI Web) for certain advanced applications. The format is as follows:
8960 2 6 12 1
6 1
3 1
2 1 2 3 2 3
3 3 3 1 3 1
4 2 1 2 1 2
2 32
.
.
.
(lines 3-7 repeated, for 11 more tasks for this respondent)
Line 1: Respondent record information

8960 2 6 12 1
Respondent "Extra" Number of Number of None option
Number Variables Attributes Choice Tasks 0=N, 1=Y,
2=Dual Response None
Line 2: "Extra" Variables (may include as many numbers as specified on Line 1)

6 1
Interview Segmentation
Duration Variable
Lines 3-7 are repeated for each choice task
Line 3: Task 1 format

3 1
Number of concepts in Depth of preference in first
first task task (always 1)
Line 4: Levels displayed in first concept

2 1 2 3 2 3
Level 2 of Level 1 of Level 2 of Level 3 of Level 2 of Level 3 of
Attribute 1 Attribute 2 Attribute 3 Attribute 4 Attribute 5 Attribute 6
Note: Missing levels (partial profile and alternative specific designs) are indicated with "0"'s.
CBC Help 453
Line 5: Levels displayed in second concept

3 3 3 1 3 1
Line 6: Levels displayed in third concept

4 2 1 2 1 2
Line 7: Choice information

2 32
Concept Task duration
Chosen
* In this example, the none option, if chosen, would be represented as concept 4.

** Missing answers are indicated by Concept Chosen set to "0."
Coding Dual Response None Answers
In the standard .CHO layout, the coding of each task is completed by a line with two values:
"Choice" and "Task Duration." With the "Dual-Response None" format, each choice task is completed
by a line with four values, such as:
Line 7: Dual-Response None Choice information

3 27 1 4
1st stage Task duration Buy=1 Task duration
choice No Buy=2
454 SSI Web v8
2.5.8 .CHS File Format

The .CHS file format is very similar to the .CHO file format, and is used for constant sum (allocation)
style CBC studies. The .CHS format files are used in Sawtooth Software's CBC/HB program. Some
power-users modify the .CHS file (usually with an automated script outside of SSI Web) for certain
advanced applications. The format is as follows:
8960 2 6 12 0
6 1
3
2 1 2 3 2 3 7
3 3 3 1 3 1 3
4 2 1 2 1 2 0
.
.
.
(lines 3-6 repeated, for 11 more tasks for this respondent)
Line 1: Respondent record information

8960 2 6 12 0
Respondent "Extra" Number of Number of None option
Number Variables Attributes Choice Tasks 0=N, 1=Y
Line 2: "Extra" Variables (may include as many numbers as specified on Line 1)

6 1
Interview Segmentation
Duration Variable
Lines 3-6 are repeated for each choice task
Line 3: Task 1 format

3
Number of concepts in
first task
Line 4: First concept information

2 1 2 3 2 3 7
Level 2 of Level 1 of Level 2 of Level 3 of Level 2 of Level 3 of Point
Attribute 1 Attribute 2 Attribute 3 Attribute 4 Attribute 5 Attribute 6 allocation
Note: Missing levels (partial profile and alternative specific designs) are indicated with "0"'s.
Line 5: Second concept information

3 3 3 1 3 1 3
Line 6: Third concept information

CBC Help 455
4 2 1 2 1 2 0
Note: if a "None" concept is present, it is included as the last alternative in the task, with all attribute
level codes as "0."
456 SSI Web v8
Adaptive CBC (ACBC) Help 457
3 Adaptive CBC (ACBC) Help

3.1 Getting Started with ACBC
3.1.1 Welcome to Adaptive CBC!
We're excited about Adaptive CBC (ACBC). It is changing the way we think about and conduct stated
choice research for complex problems involving about five or more attributes.
Sawtooth Software's products have benefited immensely from the interaction between practitioners and
the academic community. We are not secretive about the algorithms and methodologies so that others
may scrutinize and test our work. At our conferences and in the academic literature, the methods have
been examined and tested. Some investigations have been favorable toward our work, while others have
been critical. When the criticism has merit, we strive to improve the techniques.
ACBC continues this tradition. Over the last six years, we have presented the methodology and results
at both academic and practitioner research conferences. We have submitted it for review among leading
academics in discrete choice modeling. It underwent a thorough beta test process with over 50 users
wherein over 50 studies were fielded and analyzed during an eight-month beta test program. Since the
release of ACBC in March 2009, more than one-thousand projects have been completed. Two papers
were delivered at the Sawtooth Software conference in March of 2009 that highlighted successful
applications. Two additional case studies were delivered at the May 2009 joint SKIM/Sawtooth Software
event in Prague, Czech Republic on ACBC. Then, four additional papers involving ACBC were delivered
at the Sawtooth Software 2010, 2012, and 2013 conferences.
According to our customer tracking survey conducted in 2011, 2012, and 2013, ACBC projects
represented 11% (in 2011), 13% (in 2012), and 12% (in 2013) of conjoint analysis studies conducted over
the previous 12 months (by Sawtooth Software customers if using CBC, ACBC, ACA, or CVA). ACBC's
usage was greater than ACA and CVA's use combined. This shows that in the four years that ACBC
has been available, it has established itself as an important new tool in our users' conjoint repertoire.
New features in this current release of the ACBC software will likely fuel the popularity of ACBC software
for choice studies.
458 SSI Web v8
3.1.2 ACBC Specifications

ACBC can be used to study up to:
100 attributes
250 levels per attribute
Unlimited number of respondents
Typical studies in practice will cover about 5 to 12 attributes, with about 2 to 6 levels per attribute.
However, because the attribute list can be "constructed" (customized) for each respondent, it is possible
to study many more attributes and many more levels per attribute (for example, levels of "brand") than
would be prudent using traditional full-profile conjoint methods. But whenever omitting attributes and
levels from a respondent's design, some assumptions must be made, and we recommend the user
carefully review the section entitled: Customized (Constructed) Attribute and Level Lists.
3.1.3 Overview
Adaptive CBC (ACBC) is a component within the SSI Web suite of Internet interviewing products. ACBC
is used for conducting adaptive Choice-Based Conjoint (CBC) studies over the web or in non-web "CAPI"
interviewing mode (but not paper-and-pencil questionnaires). ACBC questions are usually integrated
within a longer SSI Web questionnaire that includes other standard survey questions. The number of
additional questions that can be added depends on your license. An ACBC study can include additional
standard survey questions that collect up to 50 data fields. An unlimited number of additional questions
can be used if the largest CiW interviewing license is acquired.
ACBC studies are used for learning about respondents' preferences for the combinations of features that
make up products or services. We recommend that ACBC be used in instances where there are about
five or more attributes. Studies with few attributes (such as brand+package size+price) commonly done
with standard CBC are generally less appropriate for ACBC. ACBC analysis can help with (among other
things) product design, line extensions, pricing research, and market segmentation.
Your license entitles you to free technical support via phone or email (support@sawtoothsoftware.com).
We encourage you to contact us at support@sawtoothsoftware.com or +1-801-477-4700 if you cannot
find the answer to your problem or question in this documentation. We are available Mon-Fri during our
business hours (Mountain Time Zone). (Note: we provide full technical support for composing and testing
your Internet survey on your local PC, and provide written instructions on how to install your
questionnaire on your server. We cannot take responsibility for or be expected to provide technical
support to debug any customized HTML or JavaScript you may add to your survey. You are also
responsible to make sure that Perl is running, a database is available, and that you can run Perl (CGI)
scripts on your web server. We can provide some assistance related to SSI Web's use with your server
once the fundamental Perl functionality and a database is in place.)
A sample ACBC questionnaire (lapt.ssi) that includes 250 respondent records is available in the
\Samples folder within your SSI Web installation. You can easily access this example study by
clicking Help | Sample Studies | ACBC Sample. This questionnaire lets you see an example of an
ACBC survey, investigate how it is set up, and perform analysis.
If you know a little HTML (which is not necessary to use ACBC), you can employ some "tricks" to help
you customize the look of your survey beyond the standard options provided by the software. By opening
the sample study and looking at the settings and text we've used, you can become familiar with some of
the possibilities.
Please note that ACBC is the most advanced conjoint analysis system that we've designed, and the
user should have the appropriate experience and training to ensure good results. Using ACBC requires
the following skills and knowledge base:
The analyst managing the survey should be proficient in conjoint analysis, including defining
attributes and levels properly, correctly phrasing the questionnaire text and response scales,
interpreting the part-worth utilities, and running simulations. It is not necessary to be a
statistician to use our software effectively, but knowledge of basic statistics for market research
is highly recommended.
The questionnaire is first composed locally on a PC running under Windows XP, Vista, Windows
7 or 8. The individual inputting/formatting the survey should have working knowledge of standard
Windows applications such as word processing, Web browsers and/or spreadsheets. Though it
is not necessary, some knowledge of HTML is valuable.
For web-based data collection, you may use our survey hosting service for a fee. You can also
host surveys on your own (one must have access to an Internet/intranet server running Perl
460 SSI Web v8
5.007 or later, with a database). This can be on your company's own Web server, or your ISP's
(Internet Service Provider's) Web server. SSI Web is able to automatically upload the files
needed to perform data collection on your server. But, in some cases the user must know how
to use FTP software to create directories, upload files to those directories, and set permissions
on the files and directories. Click here to receive detailed instructions on the directories, files
and permissions.
Relationship between ACBC and SMRT
ACBC is used for designing, creating, and fielding ACBC studies (either web-based or non-web "CAPI"
interviewing). But you actually perform market simulations using either the Online Simulator or the
SMRT software. ACBC and SMRT are installed and run independently--but SMRT imports the .HBU file
containing part-worth utilities.
If you don't already own SMRT software, you receive a special copy with your ACBC license to analyze
your ACBC data.
Steps in Conducting an ACBC Study
1) Formulate attributes and levels and type them into the ACBC interface.
2) Make decisions regarding your study design: decide how to organize and lay out your BYO, Screener,
Choice Tasks and (optionally) Calibration sections. Using the point-and-click ACBC interface, add those
sections with their associated questions to your SSI Web study.
3) Test your questionnaire in preview mode, by using the Local Test Server included in SSI Web, or by
uploading the files to your Web server. Compute utilities for very small sample sizes (even n=1) using
the method of Monotone Regression. Examine the utilities for your test questionnaires. Make
modifications and re-test.
4) If using web-based data collection, upload your study to an Internet server, and continue testing--by
yourself, with colleagues, and later preferably with a small number of pilot respondents. Receive
feedback from colleagues and pilot respondents. Examine the data to make sure all data are being
stored properly using the Admin Module, and by downloading the data to your hard drive. Compute
utilities using monotone regression (or using HB if your sample size is large enough). Import the results
into the Online Simulator or SMRT software package for market simulation analysis.
5) After ensuring that your ACBC questionnaire works properly, that the data are being stored correctly,
and that your questionnaire produces reasonable results, invite respondents to take the survey.
6) Monitor the data collection using the Admin Module, including downloading the data often as an
additional safeguard against data loss. Download the data, and then conduct analysis: tabulate the BYO
choices, as well as the indications of "Unacceptables" and the "Must-Haves" for levels. Estimate part-
worth utilities in preparation for running market simulations. See the Online Simulator, SMRT manual or
online help within the SMRT software for documentation regarding market simulations.
3.1.4 Motivation for Adaptive CBC

Background
Choice-Based Conjoint (CBC) has been the most widely used conjoint technique among our user base
since about the year 2000. The marketing research community has adopted CBC enthusiastically, for
several reasons. Choice tasks seem to mimic what actual buyers do more closely than ranking or rating
product concepts as in conventional conjoint analysis. Choice tasks seem easy for respondents, and
everyone can make choices. And equally important, multinomial logit analysis provides a well-developed
statistical model for estimating respondent partworths from choice data.
However, choice tasks are less informative than tasks involving ranking or rating of product concepts.
For this reason, CBC studies have typically required larger sample sizes than ratings-based conjoint
methods. In CBC studies, the respondent must examine the characteristics of several product concepts
in a choice set, each described on several attributes, before making a choice. Yet, that choice reveals
only which product was preferred, and nothing about strength of preference, or the relative ordering of the
non-preferred concepts. Initially, CBC questionnaires of reasonable length offered too little information to
support multinomial logit analysis at the individual level, unless the number of attributes and levels was
kept at a minimum. More recently, hierarchical Bayes methods have been developed which do permit
individual-level analysis, but interest has remained in ways to design choice tasks so as to provide more
information.
Problems with Traditional CBC
In recent years marketing researchers have become aware of potential problems with CBC
questionnaires and the way respondents answer CBC questions.
The concepts presented to respondents are often not very close to the respondent's ideal.
This can create the perception that the interview is not very focused or relevant to the
respondent.
Respondents (especially in Internet panels) do choice tasks very quickly. According to
Sawtooth Software's experience with many CBC datasets, once respondents warm up to the
CBC tasks, they typically spend about 12 to 15 seconds per choice task. It's hard to imagine
how they could evaluate so much information on the screen in such short order. It seems
overwhelmingly likely that respondents accomplish this by simplifying their procedures for
making choices, possibly in a way that is not typical of how they would behave if buying a real
product.
To estimate partworths at the individual level, it is necessary for each individual to answer
several choice tasks. But when a dozen or more similar choice tasks are presented to the
respondent, the experience is often seen to be repetitive and boring, and it seems possible
that respondents are less engaged in the process than the researcher might wish.
If the respondent is keenly intent on a particular level of a critical attribute (a "must have"
feature), there is often only one such product available per choice task. Such a respondent is
left with selecting this product or "None." And, respondents tend to avoid the "None"
constant, perhaps due to "helping behavior." Thus, for respondents intent on just a few key
levels, standard minimal overlap choice tasks don't encourage them to reveal their preferences
much more deeply than the few "must have" features.
Gilbride et al. (2004) and Hauser et al. (2006) used sophisticated algorithms to examine patterns of
respondent answers, attempting to discover simple rules that can account for respondent choices. Both
groups of authors found that respondent choices could be fit by non-compensatory models in which only
462 SSI Web v8
a few attribute levels are taken into account. We find that when choice sets are composed so as to have
minimal overlap, most respondents make choices consistent with the hypothesis that they pay attention
to only a few attribute levels, even when many more are included in product concepts. In a recent study
with 9 attributes, 85 percent of respondents' choices could be explained entirely by assuming each
respondent paid attention to the presence or absence of at most four attribute levels.
Most CBC respondents answer more quickly than would seem possible if they were giving thoughtful
responses with a compensatory model (a model that assumes that respondents add the value of each
feature and choose the product concept with the highest overall utility). Most of their answers can be
accounted for by very simple screening rules involving few attribute levels. Combine those facts with the
realization by anyone who has answered a CBC questionnaire that the experience seems repetitive and
boring, and one is led to conclude there is a need for a different way of asking choice questions, with the
aim of obtaining better data.
There has been a lot of effort dedicated to designing efficient CBC experiments featuring orthogonality
and high D-efficiency. These efforts have assumed that respondents answer using an additive process
consistent with the logit rule. We have become increasingly convinced that most respondents to
complex conjoint studies employ non-compensatory heuristics at odds with the logit rule, and that efforts
to improve design efficiency assuming compensatory processing may be misdirected. ACBC's design
strategy is effective for respondents that employ various degrees of non-compensatory and
compensatory processes. In terms of the traditional design criteria, ACBC's designs would be judged
inferior. But this is an inappropriate standard given that so many respondents apply cutoff rules, such as
screening based on unacceptable and must-have levels.
The Quest for Better Data
We believe CBC is an effective method that has been of genuine value to marketing researchers, but that
it can be improved. And we believe the greater need at this point is not for better models, but rather for
better data. Adaptive CBC is a promising method that responds well to the problems above. It uses the
trusted full-profile method of presenting product concepts. Its surveys are more engaging for
respondents. Rigorous comparison studies to traditional CBC suggest that our adaptive form leads to
data that are more accurate and predictive of choice behavior. It captures more information at the
individual level than traditional CBC surveys and may be used even with small (for example, business-to-
business) samples.
Adaptive CBC is best applied for conjoint-type problems in which there are about five attributes or more.
Studies involving few attributes (such as the brand+package+price studies commonly done in FMCG
research) wouldn't seem to benefit from ACBC. The store-shelf approach in our standard CBC software
would seem more appropriate for such studies.
3.1.5 Sections and Flow for Adaptive CBC

The objectives of the Adaptive CBC interview are as follows:
Provide a stimulating experience that will encourage more engagement in the interview than
conventional CBC questionnaires.
Mimic actual shopping experiences, which may involve non-compensatory as well as
compensatory behavior.
Screen a wide variety of product concepts, but focus on a subset of most interest to the
respondent.
Provide more information with which to estimate individual partworths than is obtainable from
conventional CBC analysis.
The interview has several sections, with each section quite different from the previous.
Typically, an ACBC interview includes the following three core sections:
o BYO (Configurator)
o Screening Section
o Choice Tasks
But, ACBC offers the flexibility to skip sections, for advanced researchers who decide (for certain
research situations) that some sections do not fit the buyer's (chooser's) decision making process.
BYO (Configurator) Section:
In the first section of the interview the respondent answers a "Build Your Own" (BYO) question to
introduce the attributes and levels, as well as to let the respondent indicate the preferred level for each
attribute, taking into account any corresponding feature-dependent prices. A typical screen for this
section of the interview is shown below:
464 SSI Web v8
An alternate display incorporates combo boxes rather than radio buttons:
Past research has shown that respondents enjoy BYO questionnaires and answer them rapidly, and that
the resulting choices have lower error levels than repetitive choices from CBC questionnaires.
Based on answers to the BYO questionnaire, we create a pool of product concepts that includes every
attribute level, but for which attribute levels are relatively concentrated around the respondent's preferred
attribute levels.
Advanced: Users sometimes entirely skip the BYO section (by indicating that all attributes should not be
included in the BYO section). This has an effect on the later product concepts shown to respondents. If
no BYO section is shown, then we equally sample across all levels within the experimental design,
much like a standard CBC experiment (rather than oversampling the BYO-selected levels).
Notes:
The BYO-selected product concept can be shown in later sections of the SSI Web survey using SSI
Script Functions.
Some attributes with obvious a priori preference order would seem to be out of place within a BYO
question. It may not make sense in some situations to assign price premiums to the preferred levels,
and it would be "obvious" to ask respondents to indicate their preferred level. In those cases, you can
drop such attributes from the BYO question type (but these attributes appear within the remaining
sections of the ACBC survey).
For attributes without obvious a priori preference order, it would generally make sense to include them
within the BYO question to learn respondents' preferences. Then, ACBC could focus on product
concepts similar to those preferences. But, researchers may choose to exclude such attributes from the
BYO section, if they desire.
Screening Section:
In the second section of the interview the respondent answers "screening" questions, where product
concepts are shown a few at a time (we recommend showing 3 to 5 at a time per screen, for about 7 to 9
total screens of concepts). In the Screening Section, the respondent is not asked to make final choices,
but rather just indicates whether he/she would consider each one "a possibility" or "not a possibility." A
typical screen from this section of the interview is shown below:
466 SSI Web v8
The Screening Section is also used to estimate the "None" parameter threshold.
Advanced: Users may choose to skip the Screening Section and in that case all generated product
concepts are taken forward into the Choice Tasks Section, as if the respondent had indicated that each
one was "a possibility." If you skip the Screening Section, a "None" parameter threshold is not available,
unless you include the final Calibration Section and also perform the additional step of calibrating the
estimated utilities.
Unacceptables:
After a few screens of concepts have been evaluated, we scan previous answers to see if there is any
evidence that the respondent is using non-compensatory screening rules (meaning, that there are cut-off
rules that are absolutes and cannot be compensated for by the presence of enough other good features).
For example, we might notice that he/she has avoided some levels of an attribute, in which case we ask
whether any of the consistently avoided levels is an "Unacceptable". Here is a typical screen for this
question:
Past research with ACA has suggested that respondents are quick to mark many levels as unacceptable
that are probably just undesirable. We considered that the same tendency might apply here. To reduce
this possibility, we only offer cutoff rules consistent with the respondent's previous choices and we allow
the respondent to select only one cutoff rule on this screen.
After each screen of typically three to five products has been screened (as a "possibility" or not), another
"unacceptable" screen is shown and the respondent has another opportunity to add a subsequent cutoff
rule. If the respondent identifies any "unacceptable" levels, then all further concepts shown will avoid
those levels.
Unacceptable questions provide a less aggressive way for respondents to indicate non-compensatory,
cut-off rules than "Must-Haves" (described below), and for that reason we recommend giving it
precedence in the ACBC questionnaire. A recent split-sample research project we conducted suggested
that the Unaccepable question be asked first. We suggest waiting to ask Must-Have questions until at
least two Unacceptable questions have been asked. By default, we do not ask respondents if a certain
price level is unacceptable (for Summed Pricing), but you can change that if you'd like (on the Pricing
tab). If you include price in Unacceptables questions, ACBC scans the previous answers to determine
the highest price ever selected as "a possibility" in the Screener or BYO questions. That highest price is
offered as an unacceptable threshold.
The fact that a level has been marked unacceptable can be used in skip patterns (such as to ask a
follow-up question regarding why a level was marked unacceptable) using SSI Script Functions.
Must Haves:
If the only products that a respondent has marked "a possibility" contain certain attribute levels (or
ranges of levels for ordered attributes), we ask whether that level is a Must-Have. For example:
468 SSI Web v8
After each screen of typically three to five products has been screened (as a "possibility" or not), another
"must have" screen is shown and the respondent has another opportunity to add a subsequent cutoff
rule. If the respondent identifies any "must have" levels, then all further concepts shown will satisfy
those requirements.
The fact that a level has been marked a must-have can be used in skip patterns (such as to ask a follow-
up question regarding why a level was marked a must-have) using SSI Script Functions.
Choice Tasks Section:
Once the respondent has completed the planned number of screens of Screening questions (typically 7
to 9 screens, where each screen includes 3 to 5 concepts), we transition to the Choice Tasks Section
(tournament). The respondent is shown a series of choice tasks presenting the surviving product
concepts (those marked as "possibilities") typically in groups of three, as in the screen below:
At this point, respondents are evaluating concepts that are close to their BYO-specified product, that
they consider "possibilities," and that strictly conform to any cutoff (unacceptable/must-have) rules. To
facilitate information processing, we gray out any attributes that are tied across the concepts, leaving
respondents to focus on the remaining differences. Any tied attributes are typically the most key factors
(based on already established cutoff rules), and thus the respondent is encouraged to further
discriminate among the products on the features of secondary importance.
The winning concepts from each triple then compete in subsequent rounds of the tournament until the
preferred concept is identified. If displaying concepts in triples, it takes t/2 choice tasks to identify the
overall winner, where t is the number of concepts marked as "possibilities" from the previous section (in
the case that t is odd and t/2 is not an integer, one rounds down to determine the required number of
triples).
Although it may seem to some that the goal of the tournament section is to identify an overall winning
concept, the actual goal is to engage respondents in a CBC-looking exercise that leads to good tradeoff
data for estimating part-worth utilities. The winning product concept can be shown in later sections of
the SSI Web survey using SSI Script Functions.
Advanced: Users may skip the Choice Tasks section by indicating that no concepts should be carried
forward into the Choice Tasks.
Calibration Section (Optional):
The fourth section of the interview is optional, and may be used to estimate a different "None" parameter
from that provided by the Screening Section.
The respondent is re-shown the concept identified in the BYO section, the concept winning the Choice
Tasks tournament, and (typically) four others chosen from among both previously accepted and rejected
concepts. For each of those concepts we ask how likely he/she would be to buy it if it were available in
the market, using a standard five-point Likert scale, with a screen similar to the one below:
470 SSI Web v8
This section of the interview is used only for estimation of a partworth threshold for "None." Partworths
from other sections of the interview are used to estimate the respondent's utility for each concept, and
then a regression equation is used to produce an estimate of the utility corresponding to a scale position
chosen by the researcher, such as, for example, somewhere between "Might or Might Not" and
"Probably Would." Within the market simulator, if the utility of a product concept exceeds the None
utility threshold, it is chosen.
The order of Calibration Concepts is as follows:
BYO
"Not a possibility" concept
A "winner" from the tournament
A "loser" from the tournament
(Repeat pattern of last three, if needed)
Winning concept from the tournament
Summary:
Based on our experience, we recommend the following flow for an Adaptive CBC questionnaire:
Page # Description
1. Introductory Text
2. BYO (or Most Likelies)
3. Transition Text
4. Screener #1 (showing about 4 concepts)
7. Unacceptables #1
9. Unacceptables #2
10. Must Have #1
12. Unacceptables #3
13. Must Have #2
16. Must Have #3
19. Must Have #4
21. Transition Text
22. Choice Tasks Set #1
23. Choice Tasks Set #2
24. Choice Tasks Set #3, etc. until winning product is identified
25. Transition Text
26. Calibration Concept #1 (optional)
While the schematic above gives a general recommendation regarding an appropriate ACBC interview,
472 SSI Web v8
the recommended number of questions in each section depends on the number of attributes in the study
and how accurate results are desired at the individual level. For more detailed recommendations, please
see the next section entitled Design Tab (ACBC).
In our experience, ACBC questionnaires take about 7 to 15 minutes (median survey length) when
respondents are asked to trade off 9 to 10 attributes each having from 2 to 6 levels. While this is longer
than traditional CBC questionnaires, respondents find the process more engaging and we capture more
information to model each respondent accurately.
The interview as a whole attempts to mimic the actual in-store buying experience that might be provided
by an exceptionally patient and interested salesperson. For example, after the BYO section we might
explain that this exact product is not available but many similar ones are, which we will bring out in
groups to see whether each is worthy of further interest. The Choice Tasks section is presented to the
respondent as an attempt to isolate the specific product which will best meet the respondent's
requirements (though in reality the main purpose for the researcher is to further refine the utility
estimates across all non-rejected attribute levels).
If the respondent has answered conscientiously, he/she will find that the final product identified by ACBC
as best is actually more preferred than the original BYO product. This occurs because the overall prices
of the products generated in the product pool are varied from the fixed BYO prices (assuming "Summed
Pricing" has been used). Therefore, typically at least one of the product concepts will have better
features than the BYO product at the same price, the same features at a lower price, or a combination of
these benefits. This makes it seem that the ACBC interview has actually done a good job finding a
product that exceeds the quality of the BYO product and fits the needs of the respondent.
3.1.6 Attributes and Levels for Adaptive CBC

For the most part, designing attributes and levels for ACBC follows the same guidelines as for other
conjoint methods. We'll assume the reader already is acquainted with this practice (refer to the relevant
sections in the ACA, CVA, and CBC documentation). But, there are some key differences related to the
treatment of price and the possibility of using constructed lists to customize the attribute list each
respondent sees.
Most projects will probably present each respondent the full list of attributes and levels in the study.
However, ACBC gives you the option to drop attributes and/or levels from consideration (you need to do
this prior to the respondent starting the ACBC questions). This is facilitated by SSI Web's constructed
list building logic. As you enter your attributes and levels for your ACBC study, you'll be specifying them
within Predefined Lists. The attribute list is a Predefined List with as many elements as attributes in
your study. Each element in the attribute list is in turn associated with a Predefined List that
enumerates the levels.
Generally, we recommend that no more than about 12 attributes with no more than about seven levels
each be taken forward to the ACBC questionnaire. If your study involves more attributes and levels than
this, you may want to use preliminary questions within SSI Web (CiW) to let respondents eliminate any
attributes that are completely unimportant and any levels that are completely irrelevant.
Although it is possible to include a Price attribute as an attribute with discrete levels (such as $10, $15,
$20), we generally recommend using ACBC's Summed Price approach and to treat price as a
continuous variable. This is described in the next section of this documentation.
Sequence Order and Preference Order
In the "Unacceptables" and "Must Have" questions, we need to understand something about the nature
of the levels (sequential order and/or preference order) to phrase our questions correctly and to make
correct logical deductions regarding which levels to eliminate from future consideration. For example,
assume we are conducting a study on automobiles and one of the attributes is fuel efficiency (in Miles
Per Gallon) with levels:
20 MPG
25 MPG
30 MPG
35 MPG
If after the first few screens of product concepts, the respondent only indicates that automobiles with at
least 30 MPG are "possibilities," then the correct way to phrase the "Must Have" question would be
something like this:
The automobile must have... "Fuel Efficiency: At least 30 MPG"
For ACBC to know whether to use text such as "at least" or "at most" requires knowledge of whether the
attribute has sequential (such as numeric) order, and which direction the order goes. With the fuel
efficiency attribute, the sequential progression is from low to high values. For ACBC to know whether to
eliminate more extreme levels from consideration if a less extreme level is marked "unacceptable" or
"must have" requires understanding if the attribute involves a rational preference order. In this case (all
else equal) there is rational preference order from worst to best. So, if the respondent confirms that "at
least 30 MPG" is a cutoff rule, then we know that both 20 MPG and 25 MPG are unacceptable.
474 SSI Web v8
When defining attributes, you must specify whether each attribute has:
Sequential (e.g. numeric) order (low to high or high to low)

Preference order (worst to best or best to worst)
The preference order is only used in the questionnaire for determining how to offer and enforce cutoff
rules. It is not automatically applied as a utility constraint within estimation. You have the opportunity to
enforce monotonicity (utility) constraints separately within the estimation modules.
Here are some examples of attributes and their classifications:
Non-ordered:
Brand
Coke
Pepsi
Sprite
Sequential Order (high to low), Preference Order (Best to Worst):

Printer Speed
40 pages per minute
25 pages per minute
15 pages per minute
10 pages per minute
Sequential Order (low to high), No Preference Order:

Square feet in home
1250 square feet
1750 square feet
2250 square feet
3000 square feet
4000 square feet
5000 square feet
(Note that a person may not prefer a 5,000 square foot home to a 2,250 square foot home,
because it involves extra work to clean, and is more expensive to furnish and maintain.)
3.1.7 Design Tab (ACBC)

For general information about ACBC interview setup, please refer to the section entitled: Sections and
Flow for Adaptive CBC. To learn about how to skip different sections in the ACBC questionnaire, visit:
Skipping Sections in ACBC.
The key decisions to make when designing your ACBC study are:
How many total product concepts (cards) will each respondent evaluate? This is equal to
the Number of Screening Task s x Number of Concepts per Screening Task . We generally
recommend asking each respondent to evaluate between 18 to 40 product concepts, depending on
the attribute list length and complexity (see recommendations in the table below). A good rule of
thumb is to include enough cards so that each level taken forward to the ACBC survey appears at
least twice, and preferably three times per respondent (you can check this quickly for different
questionnaire settings using the Test Design... feature that generates dummy respondent data and
tabulates the number of times each level appears across the product concepts).
How many attributes to vary from the BYO selections when generating product concepts
(cards)? At the beginning of the interview, respondents indicate which levels are "best" in the
product they'd buy. We call this configured product the "BYO Concept." When generating new
product concepts for respondents to evaluate, we choose near-neighbors to the BYO Concept, by
varying a subset of the attributes that have been included in the BYO Concept. For example,
assume that there are 8 total (non-price) attributes in your study, and all were included in the BYO
question. If you choose to vary from 2 to 4 of the attributes (Minimum Attributes to Vary = 2;
Maximum Attributes to Vary = 4), a new near-neighbor product concept could be generated by
varying 3 of the attributes from their BYO-selected levels and retaining the other 5 attributes at their
BYO-selected levels. This idea is illustrated below. To generate a near-neighbor concept,
attributes 3, 4, and 7 have been selected to be varied from their BYO-selected levels. The changed
levels are indicated in green, italics:
Attribute # BYO Concept Near-Neighbor Concept

1 Level 3 Level 3
2 Level 2 Level 2
3 Level 4 **Level 1**
4 Level 1 *Level 3**
5 Level 2 Level 2
6 Level 3 Level 3
7 Level 5 **Level 4**
8 Level 1 Level 1
The software will not allow you to vary more than half (+1) of the attributes included/chosen in the
BYO exercise. (This is an important feature for advanced questionnaires that use constructed lists
or alternative-specific designs. Even if the number of attributes used in the BYO-selected concept
is different, the ACBC software will never vary more than half (+1) of the attributes from the BYO-
selected levels when generating new product concepts.)
How many Unacceptables and Must-Haves questions to ask? The choice of how many
Unacceptables and Must Haves questions largely depends on how many Screening Task s you
decide to display. Please refer to Sections and Flow for Adaptive CBC for illustrations of the
typical layout, and how the Unacceptables and Must Haves are positioned within an ACBC study.
Generally, we suggest asking the first Unacceptables question after three Screening Task s have
476 SSI Web v8
been shown. Then, we recommend showing the first Must Haves question after the second
Unacceptables question. This leads to the easy rule of thumb:
# Unacceptables = (# Screening Tasks - 3)

# Must-Haves = (# Screening Tasks - 4)
The more Unacceptable and Must-Haves questions that are asked, the larger the percent of levels
respondents mark as Unacceptables will result. One can increase the number of Unacceptables
and Must-Have questions by building a larger pool of product concepts (cards), asking more
Screener questions, and by showing fewer concepts at a time in each Screener question.
How many concepts should be brought into the Choice Tournament? The choice
tournament shows either two or three product concepts at a time (pairs or triples), and not-selected
concepts are eliminated until an overall winner remains. It takes t/2 such choice tasks to identify
an overall winner if showing triples (in the case that t is odd and t/2 is not an integer, one rounds
down to determine the required number of triples), and t-1 tasks if showing pairs. Thus, if a
respondent takes all 40 product concepts into the choice tournament (all 40 are marked "a
possibility" in the Screening section), then 20 Choice Tournament tasks will be asked if shown in
triples, or 39 if shown in pairs. We generally recommend that you limit the number of concepts
taken into the choice tournament to a little over half of your total product concepts (cards) and that
you show the tasks in triples. Thus, if you plan to show respondents 32 total product concepts,
we'd recommend bringing up to about 20 products into the tournament (leading to a maximum of
20/2 = 10 choice tasks in that section). Of course, if respondents mark fewer product concepts as
"possibilities" in the Screener section, then fewer choice tasks will be needed to identify the overall
winner.
Avoiding Dominated Concepts A dominated concept is a product alternative that is logically

inferior to another concept (based on the Preference Order settings the researcher specifies).
Researchers have found that avoiding dominated concepts in conjoint designs adds only a very
little to the quality of the final utility results. However, it avoids the situation which sometimes
bothers clients and respondents, where an obvious question is asked that might make that person
doubt the logic behind the questionnaire. V8.3 allows the researcher to specify whether to avoid
dominated concepts. Some degree of level balance and orthogonality is typically lost when
avoiding dominated concepts (since it is just another form of prohibition that hinders the
experimental design). Depending on the study setup, it could be a relatively minor or a major effect
upon the design quality. The new Test Design routine can help you assess the impact of avoiding
dominated concepts on the precision of utility estimates. Note: if the experimental designer cannot
find a design within the allotted time that satisfies the constraint of avoiding dominated concepts, it
may need to show a dominated concept to produce a valid questionnaire for the respondent. If this
occurs, a message is stored in the design log table of the database. You can download this from
the Admin Module, by click ing Survey Administration + Advanced + Design Log Table. This
downloads a .csv file that you can view in a program such as Excel, showing per respondent
messages related to the designer process.
Including the BYO-Selected Concept in the Choice Tournament Here, you specify whether
the respondent's BYO-selected "ideal" product should be carried forward into the Choice
Tournament. For situations involving Summed Price or when not all attributes are shown in the
BYO question, this can make a lot of sense (and lead to a richer Choice Tournament). But, for
cases in which all attributes are shown in the BYO question and summed pricing is not involved,
the BYO concept should dominate any other concept that the experimental designer could
generate, so it may not be advisable to carry the BYO concept forward to the choice tournament.
Recommendations
Based on our experience with ACBC, we can offer some recommendations regarding questionnaire
design settings. Most studies will probably bring all attributes and levels into the ACBC section, so the
table below reflects the idea that each person has an identically-sized design.
In all cases, we recommend that no more than about 7 total levels per attribute be brought into the ACBC
part of the survey (other levels could be dropped as "undesirable" or "not available" using prior CiW
questions and constructed list logic). We also recommend that no more than about 10 attributes be
brought into the ACBC part of the survey (irrelevant attributes with virtually no importance could be
dropped using prior CiW questions and constructed list logic).
Depending on the number of attributes in the ACBC section of the survey, we recommend you consider
the following Design settings (as approximate guidelines):
Attributes brought into ACBC: 3 5 7 10 12

(Not counting a "summed price" attribute)
Screening Tasks:
Number of Screening Tasks 6 6 7 8 8
Number of Concepts per Screening Task 3 4 4 4 5
Minimum Attributes to Vary from BYO Selections 1 1 2 2 2
Maximum Attributes to Vary from BYO Selections 2 2 3 4 4
Unacceptables:
Number of Unacceptables 3 3 4 5 5
Must Haves:
Number of Must-Haves 2 2 3 4 4
Choice Task Tournament:

Maximum Number of Concepts in Choice Tournament 12 14 16 20 24
Calibration Concepts:
Number of Calibration Concepts (optional) *6 *6 *6 *6 *6
Screening Task Generation:

BYO-Product Modification Strategy Mixed Mixed Mixed Mixed Mixed
*Only if estimation of a new "None" parameter using Purchase Likelihood intent scale is required. A
"None" parameter automatically results from the Screening Tasks.
Note: if purely individual-level analysis is required (monotone regression) and/or sample sizes are quite
small (n<50), we recommend increasing the Number of Screening Task s by about 2 in each case (and
also increasing the Maximum Number of Concepts in Choice Tournament by 4 in each case).
478 SSI Web v8
Test Design
This functionality lets you generate multiple test (dummy) respondents who answer the ACBC questions
randomly. See the Design Tab--Test Design topic for more information.
3.1.8 Design Tab--Test Design (ACBC)

This functionality lets you generate multiple test (dummy) respondents who answer the ACBC questions
randomly. This is valuable, for multiple reasons:
1. The rule of thumb for ACBC (to achieve a high degree of precision at the individual level) is to
generate enough near-neighbor product concepts in each respondent's design such that each level is
seen (in the core set of near-neighbor concepts, not considering replacement concepts) at least two
times and preferably three times per respondent (assuming you are using the recommended approach
of BYO + Screeners + Choice Tasks). The Test Design tool summarizes the number of times each
level appears across the product concepts so you can easily check this. As few as five test
respondents will give you the needed information.
Any levels displayed to a respondent fewer than 2 times are coded in red to warn you that the design
is sparse. Any levels displayed 2 times are coded in yellow, as a moderate caution. If you are
interviewing 100s of respondents and are willing to sacrifice some precision at the individual level in
favor of shorter questionnaires, you may decide that fielding a sparse design is perfectly suitable for
your situation.
2. Sometimes you'll need to add prohibitions between attributes (so that certain levels of one attribute
cannot display with certain levels from a different attribute). When this happens, you'll experience at
least some reduction in the precision of utility estimates. The Test Design tool reports the standard
errors (from aggregate logit) both with and without the inclusion of the prohibition(s) so you can
understand the percent change in efficiency due to prohibitions. You'll need to use typically 500 or
more test respondents to get a reasonably stable read on design efficiency with and without
prohibitions, which can take more than 20 minutes to generate, depending on the speed of your
machine. Please note, that due to the nature of having random responders answer adaptive ACBC
questionnaires, where the adaptive designs oversample BYO-selected levels, the stability of the
estimated standard errors using these robotic respondents is not as robust as with similar tests for
non-adaptive CBC designs.
Note: prohibiting just one level from one attribute from appearing with one level from a different
attribute leads to so little loss in design efficiency that you shouldn't preoccupy yourself with trying to
test how much loss in precision is associated with that one prohibited combination.
3. Sometimes you'll want to test the precision of the utility estimates under different questionnaire
settings (especially changing the number of near-neighbor concepts in a respondent's design). You
can compare the estimated standard errors for different questionnaire settings, holding the number of
test respondents constant. You'll need to use typically 500 or more test respondents to get an
accurate read in this case, which can take around 30 to 45 minutes to generate.
Notes: ACBC's designer works as long as it can with the available computing resources (1 second
or less per respondent) to generate designs with high relative D-efficiency. This means that if you
use the same questionnaire setup and repeat a Test Design run (on the same or a different
machine), you will often see different results.
If you use constructed lists, these are ignored for Test Design and all attributes and levels are
taken into the ACBC survey. If you want to test the design with constructed lists implemented,
then you should use the Data Generator capability to generate the test records and then analyze
the data using other means.)
480 SSI Web v8
Test Design Reports
The Test Design report has four additional tabs that you may investigate.
Level Counts tab The Test Design tool summarizes the number of times each level appears across
the product concepts (in the core set of near-neighbor concepts, not considering replacement
concepts). As few as five test respondents will give you the needed information. Any levels
displayed to a respondent fewer than 2 times are coded in red to warn you that the design is sparse.
Any levels displayed 2 times are coded in yellow, as a moderate caution. If you are interviewing
100s of respondents and are willing to sacrifice some precision at the individual level in favor of
shorter questionnaires, you may decide that fielding a sparse design is perfectly suitable for your
situation.
Note: when Avoid Dominated Concepts is enabled (as it is by default), it can often mak e it very
difficult for the algorithm to find designs with target level balance. For some situations, if you are
finding it difficult to obtain designs with a reasonable degree of level balance (for non-BYO selected
levels), you might decide to turn off the Avoid Dominated Concepts capability. There is a tradeoff
between level balance, D-efficiency, and avoiding dominated concepts. Avoiding dominated
concepts usually leads to designs with lower D-efficiency and worse level balance. You can use the
Test Design capability to compare the precision of utility estimates (standard errors) if avoiding
dominated concepts vs. not.
Standard Errors tab This report shows the standard errors from pooled logit (across all robotic
respondents) for each of the attribute levels. All sections of your ACBC questionnaire, including
replacement concepts are used in this analysis. If you have specified any prohibitions, then
standard errors are shown for your current questionnaire as well as for a questionnaire that does not
include any prohibitions. This allows you to see how much relative efficiency is lost per attribute
level due to the prohibitions you've specified. Note that it requires about 500 "robotic" respondents
or more to obtain decent stability for the standard errors report! Running 500 respondents might
take 20 minutes or more to generate, depending on the speed of your machine. For even more
precision in this report, use 1000+ respondents.
Note that if you are using Summed Price, we assume linear price estimation.
Warnings tab For each robotic respondent and each design generation "pass" (we typically
generate multiple designs for each respondent, then select the best one in terms of D-efficiency), we
report whether there were any issues that prohibited the process from accomplishing all the specified
goals. For example, the prohibitions you may have specified or the request to avoid dominated
concepts may make it impossible for the designer to satisfy the goals of level balance and
orthogonality. In such cases, certain goals may need to be relaxed to allow the software to generate
a valid questionnaire for the respondent. Most commonly, the warning may be reported as "Design
optimize timed out. Proceeding with current concepts." That message usually means that for a
"pass" in the designer, it wasn't able to do all the swapping and relabeling that could possibly be
investigated, because it ran out of time (we don't want to make the respondent wait too long to see
the next question in the questionnaire). Even if the swapping and relabeling steps are terminated
early, the quality of the design should still be very good. Breaking out of a "pass" early typically
leads to designs that are 98% as good as designs where the swapping and relabeling iterations were
completed. The first few swaps and relabelings usually lead to the largest gains; later ones typically
yield tiny gains.
D-Efficiency tab For each robotic respondent, we compute the D-efficiency of the experimental
design as if it were a traditional full-profile card-sort conjoint analyzed via OLS (where respondents
saw each product concept just once and provided a rating for each). This approach to computing D-
efficiency is the same as employed in our CVA software for traditional full-profile ratings-based
conjoint. The D-efficiency if including the BYO tasks as additional rows in the design matrix is
shown, followed by the D-efficiency if only using the experimental design for the core product
concepts (as with traditional full-profile conjoint), not including any replacement cards. BYO
concepts involve one concept per attribute level, coded as an extreme partial-profile concept where
the attribute level in question is coded in the design matrix (and the other attributes are coded as
zeros). An exception is when level prices are associated with an attribute for use within summed
price, where the BYO concepts are coded to reflect any price tradeoffs among its levels.
Note: because of the way ACBC generates "near-neighbor" concepts to each respondent's BYO-
specified ideal (oversampling BYO-selected levels), the D-efficiency of ACBC's array of product
concepts is lower than for traditional full-profile arrays that are level-balanced and orthogonal. This is
expected and many research studies comparing ACBC to more traditional D-optimal designs have
shown that the ACBC process leads to part-worth utilities with typically greater precision than the D-
optimal approach (e.g. non-adaptive CBC). The benefits of the adaptive ACBC process seem to
outweigh the losses due to using less statistically efficient designs.
Individual Data tab Each robotic respondent's answers to the questionnaire are reported here.
482 SSI Web v8
3.1.9 Price in Adaptive CBC

Price is often included in choice studies, and evidence to date suggests that ACBC is a strong approach
for studying the impact of price on choice. Although studies comparing ACBC and CBC have found very
strong similarities in the respective part-worth utilities, the ACBC utilities tend to have more precision and
price tends to get a bit more importance (relative to the other attributes) in ACBC relative to CBC.
Summed Prices
In traditional conjoint studies, the researcher applies price as another attribute (factor) in the study
design, and specifies typically 3 to 5 levels of price (e.g. $100, $150, $200, $250). (One certainly could
take this approach with ACBC.) The problem with such an approach is that $100 is sometimes shown
with a collection of high-end features and $250 is sometimes shown with low-end features. With our
standard CBC software, conditional pricing allowed the researcher to specify that certain attributes
should be conditional on price, and always include a premium or discount. With ACBC, we have taken
the idea of conditional pricing further, by allowing the researcher to specify incremental prices for up to all
attributes in the study. When displaying the total product price, we sum the prices associated with the
levels across all attributes in the product concept, and then we vary that summed price by a randomly
drawn price variation (such as anywhere from -30% to +30%), as specified by the researcher. The
researcher can also indicate that prices (after being disturbed randomly) should be rounded to, say, the
nearest $100, or $5, $1, or $0.10 (for whatever currency symbol is appropriate).
To specify Summed Pricing (prices per attribute), do the following:
Add a level to your attributes list for the Price attribute

From the Attributes tab, use the drop-down dialog to indicate that the attribute is a "<Summed
Pricing Attribute>"
Use the Pricing tab to specify any base price, plus prices per attribute levels in your study
Using the Summed Pricing approach leads to product concepts that show realistic prices (and therefore
reflect higher utility balance across product concepts relative to traditional CBC). Products with high-end
features will generally carry higher prices, and products with low-end features will generally carry lower
prices. Under Summed Pricing, thousands of potential unique prices will have been shown to
respondents, and the utility function is estimated by fitting a linear (or non-linear function). Under
Summed Pricing, we may estimate price as a continuous function (described further in the section
entitled "Modeling the Price Function"). When treating price in this manner, we are able to separate
(partial-out) the effect of price from the effects of other attributes' levels. Therefore, one can interpret the
utilities for the other levels independent of the price increments that were associated with them (which
one cannot do when using conditional price in our CBC software).
Choosing Price Increments
A challenge with using summed price is selecting appropriate price levels to associate with different
attribute levels. For example, for the attribute RAM, incremental prices might be:
1 GB RAM +$0
2 GB RAM +$200
4 GB RAM +$500
This indicates that 2GB laptops should be shown at a price $200 higher (on average) than 1GB products.
This imbedded price for RAM will be transparent to respondents within the Screener, Choice Tasks, and
Calibration Sections, as only the total price will be shown for the product concept. However, within the
BYO task, respondents will see that selecting 2GB RAM adds $200 to the base price of the product, and
4GB RAM adds $500 to the base price. In essence, in the BYO (configurator) task respondents are
trading off the value of improving RAM vs. increasing the price. This provides one tradeoff of RAM vs.
price among the many choice tasks trading off product features (including RAM) with price across the
total ACBC survey.
Some attributes may not require associated levels of price, as the inclusion of one level or another really
doesn't affect the price of the product in the real world. Examples include colors, styles, and often
brands. Such attributes could be given incremental prices of zero, and choosing levels for these
attributes within the BYO task would not change the overall price of the selected concept.
Researchers will naturally be concerned that the price levels they choose to associate with levels in the
study might have a significant effect on the outcome. If we could instruct a computer to answer the
ACBC questions according to known cutoff rules and known utilities (with some level of respondent
error), the choice of price levels (within reason) attached to attributes should not bias the estimates of
utility for attribute levels and the price function. It would be ridiculous to use outlandish prices attached
to RAM for a laptop such as it costing an extra $1,000 for each level of RAM. Such a mistake would
certainly affect the final utilities.
In one of our methodological (split-sample) experiments, we varied the price levels attached to some of
the attributes in our study to see if the resulting part-worth utilities and predictive ability (in terms of hit
rates and share prediction accuracy) resulting from the two questionnaire versions would be affected.
Our experiment showed that after estimating price as a separate linear function, the remaining part-
worths associated with the attributes that we manipulated were essentially the same. The hit rates for
one variant of the experiment vs. the other were not significantly different. But, we need to see additional
experiments on this point before we declare that it doesn't matter what prices (within reason) you
associate with attribute levels when using summed pricing.
So, our recommendation is to select reasonable incremental prices for price levels, but not to be overly
concerned if your incremental prices deviate from average willingness to pay for the different attribute
levels. If you find it very difficult to decide on incremental levels for some attributes, preliminary
qualitative research could be conducted to help determine appropriate prices.
Specifying a Single Continuous Price Variable
For some projects you face, the idea of assigning price increments to specific attribute levels and
showing these to respondents just doesn't seem to work out. Perhaps it's impossible to think about
price premiums by feature, or specific prices cannot be agreed upon with the client. If that is the case,
you can specify a single base price for the product concept (along with the desired range of price
variation to test, such as from -30% to +30%). This gives you the ability to fit a multi-segmented
piecewise price function to the continuous price variable, discovering the specific points of non-linearity
for each respondent rather than having to specify specific price increments prior to fielding the study.
When the BYO question is displayed, ACBC recognizes that no level-based prices are attached to
feature attributes, so the BYO question deletes the pricing column and simply shows a series of select
questions asking respondents which levels are preferred (for each attribute except total price). However,
this might seem unusual for ordered attributes, where one level is clearly superior. If that occurs, another
option in the software is to omit the attribute from the BYO question. In that case, the software assumes
no BYO level has been selected and will sample equally across all levels of that attribute in the
generation of near-neighbor concepts (see the section entitled How Concepts Are Chosen for more
details).
484 SSI Web v8
3.2 Hands-On Tutorials

ACBC has flexible options for conducting a variety of studies. We assume the user already has
significant experience using the SSI Web system, including CBC. In these tutorials, we give outline
instructions and avoid click-by-click detail.
Tutorial #1 (Beaches)
Tutorial #2 (Pianos)
3.2.1 Tutorial #1 (Beaches)

Tutorial Specifications:
Subject Matter: Choosing which beach to visit on vacation

Design Summary: 8 attributes (no price attribute involved)
Note: We assume you are already familiar with SSI Web programming and standard CBC (Choice-Based
Conjoint) research. If not, we recommend you complete the Tutorials for CiW and standard CBC.
Setting: You have been asked to work with Dr. Stephen Leatherman, "Dr. Beach," to determine which
characteristics people most want in a beach. Dr. Leatherman is a professor at Florida International
University that specializes in the study of beaches, and he is interested in refining his annual "Best
Beaches" report.
Part 1: Attribute List
After discussions with Dr. Leatherman, you come up with a list of attributes that you both feel are the
primary drivers of what makes a beach attractive.
Attributes:
Sand softness:
Water temperature:
Water safety:
Beach cleanliness:
Public Facilities:
Bottom structure:
Typical demographic:
Crowds:
Levels:
Sand softness:
Sand and rocks
Coarse sand
Medium sand
Fine sand
Water temperature:
60°F/15°C average (wetsuit required)
70°F/21°C average
80°F/27°C average
90°F/32°C average (bathwater)
Water safety:
No rip currents, few dangerous animals
Occasional rip currents, occasional dangerous animals
Frequent rip currents, occasional dangerous animals
Beach cleanliness:
Trash, glass and manmade rubble commonly found
Trash, glass and manmade rubble occasionally found
Trash, glass and manmade rubble rarely found
486 SSI Web v8
Public Facilities:
No public restrooms, showers or changing facilities
Public restrooms available, but no showers or changing facilities
Public restrooms, showers & changing facilities available
Bottom structure:
Sandy bottom underwater past shoreline
Rocky bottom underwater past shoreline
Coral bottom underwater past shoreline
Mud bottom underwater past shoreline
Popular with young partygoers
Popular with young families
Popular with mature adults
Crowds:
Uncrowded - ample open space
Somewhat crowded - some open space
Crowded - very little open space
Step 1: Create a new study

To begin programming this survey, open SSI Web and click File | New Study… Name the survey
"beaches".
Step 2: Add attributes and levels as Lists (List Manager)

Once you've created your new survey, we need to add the attributes and levels we've just defined. Use
the List Manager (Compose | Lists…) to specify 9 total (predefined) lists:
1. attributes
2. sandsoft
3. watertemp
4. watersafety
5. cleanliness
6. facilities
7. bottom
8. demographic
9. crowds
For list: attributes, add the 8 attribute labels (as List Members):
Sand softness:
Water temperature:
Water safety:
Beach cleanliness:
Public Facilities:
Bottom structure:
Crowds:
For list: sandsoft, add the following List Members:
Sand and rocks

Coarse sand
Medium sand
Fine sand
Etc. Until you have fully specified all 9 lists.
Step 3: Attribute Settings

Click Compose + Write Questionnaire to open the Write Questionnaire dialog, then after the first page
break, add (using Add…) an ACBC Exercise named "beach."
The ACBC Exercise Settings dialog appears, with eight tabs: Attributes, Pricing, Alternative-Specific,
Prohibitions, Design, Shared Question Settings, Conditional Display, and Merged Rows. Select the
Attributes tab.
For the List of Attributes, select attributes.
A dialog opens with one row per attribute (corresponding to each list element of attributes). For each
attribute, use the drop-down to specify the associated list. For example, for attribute Sand softness:
select sandsoft.
When you are done, the dialog should look like this:
Next, we need to inform ACBC if each attribute has sequence order (low to high or high to low) and if so,
the preference order (best to worst or worst to best). We also need to specify if each attribute should be
488 SSI Web v8
included in the BYO and Must-Haves/Unacceptables sections. Two of the attributes (Water safety and
Beach cleanliness) would seem to be obvious questions for most every respondent (respondents would
generally prefer safer, cleaner beaches). It wouldn't seem to make much sense to ask respondents
about these two attributes in the BYO question. If for some reason you felt that an attribute should not
be included in the Must-Haves or Unacceptables questions, then you would also indicate it here (this
could have a significant effect on estimated utilities, so we generally recommend caution with this
functionality).
Specify the following settings on the Attributes tab:
Attribute Sequence Order Preference Include In BYO Include in Must-

Order Haves and
Unacceptables
Sand Softness None None Yes Yes
Water temperature Low to high None Yes Yes
Water safety Low to high Best to worst No Yes
Beach cleanliness High to low Worst to best No Yes
Public facilities None None Yes Yes
Bottom structure None None Yes Yes
Typical demographic None None Yes Yes
Crowds None None Yes Yes
Step 4: Establish Necessary Prohibited Pairs

We generally discourage the use of prohibitions in conjoint exercises. However, we also recognize that,
occasionally, a few prohibitions might be necessary to avoid illogical combinations. And, very modest
prohibitions will not affect design efficiency much. (Warning: too many prohibitions can invalidate your
study, so be careful!)
In our study, it might be unreasonable to believe that an uncrowded beach would be popular with partying
college students. We'll therefore add a prohibition between "Popular with young partygoers" and
"Uncrowded - ample open space".
On the Prohibitions tab, Click Add Prohibition… and specify a prohibition between Attribute 7, Level 1
and Attribute 8, Level 1.
This looks like:

Back on the Prohibitions tab, the prohibitions grid should now look like:
(Of course, you could have just directly typed these values into this grid, rather than use the Add
Prohibition… button.)
Step 5: Design Settings

Go to the Design tab. This is where the bulk of your settings that control the flow of the questionnaire
are found. The defaults that are shown are:
490 SSI Web v8
Let's reduce a few settings (to shorten the questionnaire, for the purposes of this tutorial):
Screening Tasks: change from 8 to 6

Minimum Attributes to Vary from BYO Selections: change from 2 to 1
Maximum Attributes to Vary from BYO Selections: change from 4 to 3
Number of Unacceptables: change from 5 to 3
Number of Must Haves: change from 4 to 2
Maximum Number of Product Concepts in Choice Tournament: change from 20 to 16
When you've made those edits, the dialog should look like:
This will lead to 6 Screening Tasks x 4 Concepts per Screening Task = 24 total "cards/concepts" in the
design.
The respondents will get 3 Screening Task s prior to seeing their first Unacceptables question.
The number of choice tasks in the Choice Tournament will be limited to 16/2 = 8.
Note that in the BYO question the respondent is only asked to specify their preferred beach on six of the
eight attributes (we omitted Water Safety and Beach Cleanliness from the BYO question). The general
rule is to vary no more than half of the BYO attributes from their BYO selections when generating the
pool of near-neighbor concepts. Thus, we have changed the Min and Max attributes to vary to a more
modest 1 and 3 attributes (instead of the default 2 and 4 attributes). For each new near-neighbor
concept, between 1 to 3 of the six attributes included in the BYO question will be changed from their
BYO selections. The other two attributes (Water Safety and Beach Cleanliness) will have levels chosen
at random (controlled for balance) to complete each full concept.
Step 6: View Questionnaire Layout and Edit Questions

From the ACBC Exercise – beach dialog, click OK to return to the Write Questionnaire dialog.
You should see the following question order for the ACBC section:
Question Name Question Type

beach_BYO Build-Your-Own
beach_Screener1 Screening Task
492 SSI Web v8
beach_Unacceptable1 Unacceptables Question

beach_MustHave1 Must-Have Question
beach_MustHave2 Must-Have Question
beach_ChoiceTask1…8 Choice Task Tournament
Edit BYO Question
Double-click the BYO question to edit it.
First, modify the text in Header 1 to say:
Please describe the beach you would most want to visit during summer vacation.
Click your preferred choice for each feature below.
Now, let's click Preview to see what we've got to this point.
Go to the Question Settings dialog to examine the various formatting options you have for this question.
For example, you can change from a radio-button layout to a combo-box layout by selecting: BYO
Layout = Combo Box. Try that, then click Preview to see the result.
You can modify other settings if you would like to investigate further formatting options.
Insert HTML Filler Question (optional)
Next, it would be helpful to inform respondents about the two attributes that we decided to drop from the
BYO question (because they would have obvious answers).
We'd recommend that you insert a new HTML Filler question directly following the BYO question (on a
separate page) that informs respondents about the two attributes that we omitted from the BYO
question. Place the following into the Header 1 of that HTML Filler question.
For your information, in the next stages of this interview, we'll also be asking you about
beaches that differ in the following ways:<p>
<b>Water Safety:</b><br>
<ul>
<li>No rip currents, few dangerous animals
<li>Occasional rip currents, occasional dangerous animals
<li>Frequent rip currents, occasional dangerous animals
</ul><p>
<b>Beach Cleanliness:</b><br>
<ul>
<li>Trash, glass and manmade rubble commonly found
<li>Trash, glass and manmade rubble occasionally found
<li>Trash, glass and manmade rubble rarely found
</ul>
Edit Screening Tasks
Double-click any of the Screening task s to edit the layout of these questions.
Edit Header 1 to say:
Here are a few beaches you might like. Do any of these look like possibilities? For
each, indicate whether it is a possibility or not.
Click Preview, to see what we've got with the defaults.
You will note that the questionnaire might not fit well on your screen. It might be good to make the
question a bit wider, so there isn't so much text wrapping across multiple lines. Another approach might
be to make the attribute level text smaller. Both of these operations may be done from the Question
Settings tab.
The Style that is selected for the study controls the layout and look of each question. You could try
changing styles to see if a different style provides a better layout for ACBC questions. Or, you can
directly override the style to control the width of the tasks and the font sizes.
To override the font size used for the level text, click the icon within the Level Display area of the
Question Settings tab. On that dialog, you can specify a font size (in pixels), which overrides the font
size controlled by the study Style. Specify a "12" for text with a 12 pixel point size.
Edit Unacceptable Questions
Double-click any of the Unacceptable questions to edit the layout of these questions.
The text header for this question varies depending on whether it is the first time this type of question has
been seen by a respondent (a longer description) or whether the respondent has already seen this
question type before (requiring a shorter description). Thus, we dynamically bring the text in using the
function: [%ACBCUnacceptableText()%]
Leave the function as-is. You can edit the text supplied in this question using the Question Settings tab.
From the Question Settings tab, edit the No Rules Exist Label to specify the text to display the first time
the respondent encounters the question. Click the icon edit the text. Write:
Would any beach having the features below be <b>totally unacceptable</b>? If so,
mark the <b>one feature</b> that is most unacceptable to you, so I can focus better
on beaches that meet your needs.
Edit the Rules Exist Label to specify the text to display on subsequent Unacceptable questions (after
the respondent has become familiar with this type of question). Write:
In addition to these unacceptable characteristics:<br><br> [%

494 SSI Web v8
ACBCUnacceptableRules() %] <br><br>Is there another unacceptable feature?
Next, edit the None of the Above Label to say:
None of these is entirely unacceptable
Let's insert a graphic of an interviewer to the left of this question (change Side Panel Position from Right
to Left). Click the icon to edit the Side Panel Text, click the "camera" icon, and browse to a
graphic (use "Kylie, business, small format" the graphic's name is "kylie_300_bus_4.jpg")
Next, click Preview to see what we've got to this point.
Edit Must Have Questions
Double-click any of the Must Have questions to edit the layout of these questions.
The text header for this question varies depending on whether it is the first time this type of question has
been seen by a respondent (a longer description) or whether the respondent has already seen this
question type before (requiring a shorter description). Thus, we dynamically bring the text in using the
function: [%ACBCMustHaveText()%]
You edit the text supplied in this question using the Question Settings tab.
Edit the No Rules Exist Label to specify the text to display the first time the respondent encounters the
question. Click the icon edit the text. Write:
I don't want to jump to conclusions, but I've noticed you've chosen beaches with
certain characteristics shown below. Are any of these <b>absolute musts</b> for the
beach you would like to visit? If so, mark the <b>one most important feature</b>, so I
can focus better on beaches that meet your needs.
Edit the Rules Exist Label to specify the text to display on subsequent Must-Have questions (after the
respondent has become familiar with this type of question). Write:
In addition to:<br><br> [% ACBCMustHaveRules() %] <br><br>Are there any other

absolute musts?
Next, edit the None of the Above Label to say:
None of these is an absolute must
Let's insert a graphic of an interviewer at the right of this question. Change the Side Panel Position from
Left to Right. Click the button for Side Panel Text, click the "camera" icon, and browse to a
graphic (use "Kylie, business, small format" the graphic's name is "kylie_300_bus_3.jpg"). You can get
these graphics from our website, at: http://www.sawtoothsoftware.com/survey-graphics
Next, click Preview to see what we've got to this point.

Edit Choice Task Tournament Questions
Double-click any of the Choice Task Tournament questions to edit the layout of these questions.
Edit Header 1 to say:
Among these three, which beach would you most want to visit for summer vacation?
(I've grayed out any identical features, so you can just focus on the differences.)
You can also change Level Display Format so that the text is 12 point text always, irrespective of what
Style is applied to the study.
Step 7: Randomize the Attribute List
To control for order bias, let's randomize the order of presentation of attributes (randomized once per
respondent, and the order remains constant through the interview).
Create a new "Constructed" list called Rattributes (with "Parent" list = attributes). Use the following
instructions to build Rattributes:
ADD (attributes)
RANDOMIZE()
Then, on the Attribute tab within the ACBC exercise, choose Rattributes as the attribute list instead of
attributes.
You could do a similar trick to randomize levels within any attribute you wanted as well.
Step 8: Test Design

Before you take the questionnaire to field, you should use the Test Design functionality within ACBC
software to send several "robotic respondents" through the survey, as if they were real respondents. This
is a good practice, because Test Design will help you evaluate whether your ACBC questionnaire
conforms to recommended best practices and is likely to lead to good results.
From the Exercise Settings + Design tab, you'll see an area called Test Design:
By default, five robotic respondents are set to take your ACBC questionnaire (answering randomly). For
the key issue you should check--level frequencies--five respondents are sufficient. For high quality
individual-level estimates, we recommend that each level appear at least twice and preferably three times
for each respondent. If you plan to have relatively large sample size and are mainly interested in
496 SSI Web v8
population estimates and predictions, then you could ignore that advice and use a shorter questionnaire
(where each level isn't necessarily shown at least 2 or 3 times).
For this questionnaire, if the respondent doesn't establish any Unacceptable or Must Have rules, 24
cards (product concepts) are generated (6 Screening pages that include 4 concepts per page). We are
concerned about how many times each attribute level appears across these 24 core product concepts.
Let's click Test Design and examine the report (due to space limitations, we've pasted the report into
three sections, so you can see the frequency report for all eight attributes below):
Five robotic respondents have completed the questionnaire. The report sorts the respondents in order
from the respondent with the lowest count on any one level to the respondent with the highest count. For
example, respondent #3 has a red cell for 90-degree Water Temperature (that level only occurs one time
across all 24 product concepts included in respondent #3's questionnaire). This could be a point of
concern, however the other 4 respondents have each level represented at least 2 times (those are color
coded in yellow).
The general rule of thumb for ACBC is for each level to appear at least two times and preferably three
times in each respondent's core design. We've nearly accomplished that here. If your sample size is
relatively large and you don't require especially precise estimates at the individual level, then a result as
we've displayed above would be satisfactory. But, if we required more precision at the individual level, we
could change the settings for the ACBC questionnaire (on the Design tab) to include 7 screening tasks
rather than 6 screening tasks (leading to 4 additional product concepts included in each respondent's
design). The questionnaire will be a bit longer for each respondent, but you will achieve more information
in the experimental design. If you do this, you should re-run the Test Design program to review the new
frequencies for levels across attributes given the slightly longer questionnaire.
The Test Design report has four additional tabs that you may investigate. We recommend you press F1
to get more information regarding the report on each tab. We'll summarize some key points here:
Standard Errors tab This report shows the standard errors from pooled logit (across all robotic
respondents) for each of the attribute levels. If you have specified any prohibitions, then standard
errors are shown for your current questionnaire as well as for a questionnaire that does not include
any prohibitions. This allows you to see how much relative efficiency is lost per attribute level due to
the prohibitions you've specified. Note that it requires about 500 "robotic" respondents or more to
obtain decent stability for the standard errors report! Running 500 respondents might take 20
minutes or more to generate, depending on the speed of your machine. For even more precision in
this report, use 1000+ respondents.
Warnings tab For each robotic respondent and each design generation "pass" (we typically
generate multiple designs for each respondent, then select the best one in terms of D-efficiency), we
report whether there were any issues that prohibited the process from accomplishing all the specified
goals. For example, the prohibitions you may have specified (or the request to avoid dominated
concepts) may make it impossible for the designer to satisfy the goals of level balance and
orthogonality. In such cases, certain goals may need to be relaxed to allow the software to generate
a valid questionnaire for the respondent. More commonly, the warning may be reported as "Design
optimize timed out. Proceeding with current concepts." That message means that for that "pass" in
the designer, it wasn't able to do all the swapping and relabeling that could possibly be investigated,
because it was running out of time (we don't want to make the respondent wait too long to see the
next question in the questionnaire).
D-Efficiency tab For each robotic respondent, we compute the D-efficiency of the experimental
design as if it were a traditional full-profile card-sort conjoint analyzed via OLS (where respondents
saw each product concept just once and provided a rating for each). This approach to computing D-
efficiency is the same as employed in our CVA software for traditional full-profile ratings-based
conjoint. The D-efficiency if including the BYO tasks as additional rows in the design matrix is
shown, followed by the D-efficiency if only using the experimental design for the product concepts
(as with traditional full-profile conjoint). Note: because of the way ACBC generates "near-neighbor"
concepts to each respondent's BYO-specified ideal, the D-efficiency of ACBC's array of product
concepts is lower than for traditional full-profile arrays that are level-balanced and orthogonal. This is
expected and many research studies comparing ACBC to more traditional D-optimal designs have
shown that the ACBC process leads to part-worth utilities that have typically more precision at the
individual level than the D-optimal approach, non-adaptive CBC. We expect ACBC's designs to have
lower D-efficiency than traditional conjoint designs. Because of the benefits of the adaptive ACBC
process, the practical end results are usually better than for traditional conjoint studies.
Individual Data tab Each robotic respondent's answers to the questionnaire are reported here.
Step 9: Test Questionnaire on Local Test Server

After you've seen the results of robotic respondents taking your ACBC questionnaire randomly, you
should test the survey using human respondents who are answering reasonably. You can learn quite a
lot by observing real respondents answering your survey. This pre-testing of the questionnaire often
leads the researcher to address problems with questionnaire wording or questionnaire length.
From the Write Questionnaire dialog, click the + Test Survey... to take the survey yourself (and/
or ask other test respondents to take the survey). Answer the questions realistically and reasonably.
After you have completed your test survey(s), download the data by clicking + download data. A
datafile is saved to your study folder (this is the main respondent data file). Please recognize that these
test respondents are treated as real respondent data within the Analysis interface (you should delete
498 SSI Web v8
them prior to conducting analysis with your final, real respondents).
We recommend you analyze your test data using Analysis + Estimate ACBC Utilities - HB. ACBC
usually obtains a great deal of accuracy at the individual level, so the utilities reported for your test
respondents should have a great deal of face validity. You should review that the utilities seem to match
quite well the preferences and strategies employed by the test respondents.
You can also take the test respondent utilities forward to a market simulator (such as Sawtooth
Software's Online Simulator or the SMRT market simulator) to conduct further analysis. By testing the
design first with robotic respondents and later with real test respondents, you should gain a great deal of
confidence that the ACBC survey will go well. We also recommend doing a "soft launch" to small
sample of respondents (a dozen to thirty is often sufficient), downloading the data, and estimating the
utilities using HB. Look at the interview time per respondent, the estimated utilities, and any open-end
comments the respondents have given you regarding their evaluation of the questionnaire.
3.2.2 Tutorial #2 (Pianos)

Subject Matter: Grand pianos for home use

Design Summary:
5 attributes
Summed Pricing, with additional price adjustments
Screening questions to develop list of considered brands (constructed list)
Note: This is an advanced ACBC tutorial. If you are beginning with ACBC, we suggest you first
complete the "Beaches" Tutorial #1.
Setting: This project involves pricing research for grand piano purchases for home use, in the 6-foot
(parlor grand) to 7.5-foot (small concert grand) range. There are very large price differences between
brands, with the Steinway brand in particular charging a very high premium due to the hand-crafted
nature of its pianos. We would like you to consider that you would be interviewing only sophisticated
buyers in this niche market (who are both in the market and are highly knowledgeable about this product
category).
Attribute List:
Attributes:
Brand:
Length:
Finish:
Bench:
Price:
Levels:
Brand:
Baldwin
Bechstein
Bosendorfer
Boston
Chickering
Estonia
Kawai
Kimball
Knabe
Petrof
Samick
Steinway
Wurlitzer
Yamaha
Young-Chang
Length:
6-foot
6.5-foot
7-foot
7.5-foot
500 SSI Web v8
Finish:
High-gloss black finish
Flat black finish
Wood grain finish
Bench:
Standard bench
Concert bench
Standard + concert bench
Price:
"Summed Price" per specifications below
Summed Pricing Grid Specifications:
Level
Level: Prices: Additional Price Adjustments:
Base Price $7,000
Baldwin $6,000
Bechstein $10,000
Bosendorfer $20,000
Boston $12,000
Chickering $4,000
Estonia $5,000
Kawai $8,000
Kimball $4,000
Knabe $2,000
Petrof $6,000
Samick $2,000
Steinway $30,000
Wurlitzer $4,000
Yamaha $8,000
Young-Chang $0
6-foot $0
6.5-foot $5,000 Steinway (+$15,000); Bosendorfer (+$10,000)
7-foot $10,000 Steinway (+$25,000); Bosendorfer (+$18,000)
7.5-foot $18,000 Steinway (+$40,000); Bosendorfer (+$25,000)
High-gloss black finish $0

Flat black finish $0
Wood grain finish $0
Standard bench $0
Concert bench $0
Standard + concert bench $250
The table above provides the data to compute the total price for each grand piano (prior to varying its
price by a random variation in the range -30% to +30%). For example, a 7-foot Yamaha with a standard
bench would cost:
Base Price ($7,000) + Yamaha ($8,000) + 7-foot ($10,000) + Standard bench ($0) = $25,000
But, Steinway and Bosendorfer are elite brands whose prices escalate (beyond the other brands) for the
larger instruments in the 6.5 to 7.5-foot range. The additional premiums are given as Price Adjustments.
For example, the price of a 7-foot Steinway with a standard bench is:
Base Price ($7,000) + Steinway ($30,000) + 7-foot ($10,000) + Adjustment for Steinway @ 7-foot
(+$25,000) + Standard bench ($0) = $72,000
Dynamic Brand List (Constructed Lists):
We recommend that not more than about seven levels of any attribute be brought into the ACBC
exercise. Because this project involves 15 separate brands, we should drop some brands for each
respondent and focus just on relevant brands. It is rare for an informed grand piano buyer to consider
more than a handful of brands. Therefore, we will ask some preliminary screening questions to elicit the
brands under consideration (and cap the maximum number of brands at seven).
ACBC estimates a full set of utilities for the respondent, even though most brands are excluded from
each respondent's ACBC exercise. For each respondent, we inform utility estimation that the not-
considered brands are inferior to the considered brands. Synthetic choice sets are added to the BYO
portion of the design to indicate that the not-considered brands were rejected in favor of considered
brands at their respective prices (if price premiums per brand are specified).
It is very important to consider that the data for the BYO section is coded as if respondents are
making choices for each attribute separately traded off versus any price premiums for that same
attribute (from the "Summed Pricing" grid). Therefore, if preliminary screening questions are
used to establish which brands are chosen for inclusion in the ACBC exercise, these screening
questions should refer to any price premiums (or discounts) that are associated with the levels.
For example, the screening question for brand could be represented as follows:
Q1. Please consider your upcoming purchase of a grand piano (6-foot to 7.5-foot). For
your reference, the typical price for a 6-foot piano for different brands is as follows:
Typical Price for 6-foot Piano:
Baldwin $13,000
Bechstein $17,000
Bosendorfer $27,000
Boston $19,000
Chickering $11,000
Estonia $13,000
Kawai $15,000
Kimball $11,000
Knabe $9,000
Petrof $13,000
Samick $9,000
Steinway $37,000
Wurlitzer $11,000
Yamaha $15,000
502 SSI Web v8
Young-Chang $7,000
If these were your only options, which of the following brands would you consider? <show
list of brands, multi-select format>
Baldwin
Bechstein
Bosendorfer
Boston
Chickering
Estonia
Kawai
Kimball
Knabe
Petrof
Samick
Steinway
Wurlitzer
Yamaha
Young-Chang
Programming the Project in SSI Web
As with the previous Beaches exercise (Tutorial #1), use the List Manager to establish pre-defined lists
for the attributes and levels. (For the summed Price attribute, it should be included as a list element of
the Attribute list.) The lists should be:
List: Attributes
Levels:
Brand:
Length:
Finish:
Bench:
Price:
List: Brands
Levels:
Baldwin
Bechstein
Bosendorfer
Boston
Chickering
Estonia
Kawai
Kimball
Knabe
Petrof
Samick
Steinway
Wurlitzer
Yamaha
Young-Chang
List: Length
Levels:
6-foot
6.5-foot
7-foot
7.5-foot
List: Finish
Levels:
High-gloss black finish
Flat black finish
Wood grain finish
List: Bench
Levels:
Standard bench
Concert bench
Standard + concert bench
Add an ACBC exercise to your study, and use the Attributes tab to specify your list of attributes and the
level lists associated with each attribute. When you are finished, it should look like:
(Note that for now, we are specifying the "Brand" list, which contains all 15 brands. Soon, you'll write
screener questions and build a constructed list that includes up to 7 considered brands for each
respondent. You'll select that constructed list as the List of Levels for brand rather than the pre-defined
Brand list that is currently shown above.
Next, specify your Summed Pricing Grid using the Pricing tab:
504 SSI Web v8
Using the drop-down control, select each attribute and cut-and-paste or directly type the price values into
the Component Prices column. Note that we inserted the base price of $7,000 into the Base Price field.
We have specified to vary summed prices by +/-30%, rounded to the nearest $250.
Next, you need to establish the Price Adjustments for Steinway and Bosendorfer. Click the Price
Adjustments... button.
The grid above gives you great flexibility for inserting additional price adjustments. In our situation, the
price adjustments are quite simple, because they are based on just two attributes (Attribute #1: Brand &
Attribute #2: Length).
For example (row 1 above), we add $15,000 to the cost of Attribute 2, Level 2 (Length: 6.5-foot) if
Attribute 1, Level 12 (Steinway) occurs.
The last major step to programming this exercise is to create a screener question (Q1) that asks
respondents to provide a list of brands they consider. It could also be done in two steps: a question that
asks for the one most likely brand, and then a follow-up question that asks for other considered brands.
In either case, you develop a new constructed list of brands with no more than 7 brands on the list. On
the Attributes tab, you'll select the constructed list of brands for Attribute 1 rather than the pre-defined
brands list.
(We show an example using constructed lists for a Brand attribute in the section entitled Customized
(Constructed) Attribute and Level Lists.)
506 SSI Web v8
3.2.3 Tutorial #3 (Alternative-Specific Design)
Subject Matter: Desktop vs. Laptop for work use

Design Summary:
7 attributes associated with laptops
5 attributes associated with desktops
Summed Pricing or alternative-specific pricing
Conditional display to merge attribute rows
Note: This is an advanced ACBC tutorial. If you are beginning with ACBC, we suggest you first
complete the "Beaches" Tutorial #1.
Introduction
With the release of ACBC v8.3, Adaptive CBC (ACBC) provides greater power and flexibility through
alternative-specific designs. This advance makes it possible to field much more flexible and customized
conjoint designs in ACBC than before.
With traditional conjoint attribute lists, every attribute needs to apply to every product concept. This
works fine for many studies, but researchers commonly face situations in which some attributes only
apply to certain brands or levels of technology, etc. Such is the case with alternative-specific designs.
The classic example cited in the literature involves transportation choices to get to work. Commuters
can take the bus, the train, a personal car, or a bicycle. The attributes for buses and trains (frequency,
round-trip fare, wifi availability) are different from the attributes for cars or bicycles. And yet, we might
want to study how much improvement in buses and trains would lead a person to switch from driving a
car or riding a bicycle.
The example we highlight in this tutorial involves the choice of laptops versus desktops, Windows OS
(operating system) versus Mac OS. There are attributes of laptops (e.g. battery life) that aren’t
applicable to desktops. And, there are brands for Windows OS machines that don’t apply to Mac OS
machines (Apple is the only brand for Mac OS).
The Client Situation
Imagine a client brings you a tradeoff study that leads to an attribute list as portrayed in the schematic
shown on the next page (as might have been drawn up in a white boarding session by you and your
client). The thought is that people have a choice between laptops and desktops when buying a machine
for work purposes.
In this example, we have also attached incremental prices to certain levels, indicative of ACBC’s
“Summed Pricing” approach. However, we will also show you how to execute this study if you do not
wish to do level prices and summed pricing.
What you and your client are envisioning is a choice layout such as this, for tradeoffs among laptops and
desktops:
508 SSI Web v8
You also realize that the decision could involve some non-compensatory (must-have) aspects. People
are often sharply divided between Windows and Mac OS. Also, some people absolutely must have a
laptop as their work computer. ACBC does good job accounting for non-compensatory decision-making
and customizing the choice scenarios for different respondent preferences.
Specifying the Attribute List:

The first step is to start a new SSI Web project (v8.3.2 or later) and specify the attribute lists.
Start SSI Web, by clicking Start + All Programs + Sawtooth Software + Sawtooth Software SSI
Web.
Click File + New Study and specify a new study such as computers.
Click Compose + Lists… to access the list manager.
Click Add Predefined List to add the first list, named: attributes.
List Name: Attributes

Type of Computer:
Operating System:
Brands (Windows OS):
Brands (Mac OS):
Laptop Screen Size:
Laptop Hybrid Mode:
Laptop Battery Life:
Laptop Hard Drive:
Desktop Screen Size:
Desktop Hard Drive:
Summed Price
To save time, you can copy-and-paste the list elements into the interface by copying them from this
document (highlighting and then pressing CTRL-C) and then within the SSI Web interface clicking the
Paste list members from clipboard icon .
Using the same pattern, add the other pre-defined lists to the list manager:
List Name: ComputerType

Desktop
Laptop
List Name: OS
Windows OS
Mac OS
List Name: BrandsWindows

Dell
Lenovo
Gateway
HP
List Name: BrandsMac (this attribute for display purposes only; not utility estimation)
Apple
Apple
List Name: LaptopScreenSize

13-inch
15-inch
17-inch
List Name: LaptopHybridMode

Screen can flip or fold into tablet shape for tablet-like experience

List Name: LaptopBatteryLife

4 hour
6 hour
12 hour
List Name: LaptopHardDrive

128 GB traditional (spinning platter)
128 GB solid state (flash memory)
List Name: DesktopScreenSize

17-inch
21-inch
24-inch
510 SSI Web v8
List Name: DesktopHardDrive

1 TB traditional (spinning platter)
Adding the ACBC Exercise

Now that you have specified the lists involved in this ACBC study, you are ready to add the ACBC
exercise itself to your SSI Web questionnaire. If you are still in the List Manager, click OK to close the
dialog.
Click Compose + Write Questionnaire. Click Add… and then add an ACBC Exercise to your study
called myacbc.
When you click OK, the ACBC Exercise Settings dialog is displayed.
For List of Attributes, select Attributes. Under the List of Levels column, select the list associated with
each of the attributes in your study. For Summed Price, select <Summed Pricing Attribute>.
On this same dialog,
1. Specify that the Sequence Order for Laptop and Desktop screen size is Low to High.
2. Specify that the Sequence Order for Laptop Battery Life should be Low to High. Specify that
the Preference Order for this attribute should be Worst to Best.
3. Under Include in BYO (build-your-own question), specify that the Brands (Mac OS) attribute
should not be shown in the BYO question.
4. Also, specify that the Brands (Mac OS) attribute should not appear in the Must Haves or
Unacceptables questions.
When you’re done, the grid should look like:

Specifying the Level Pricing for Summed Pricing

From the ACBC Exercise Settings dialog, click the Pricing tab.
For Type of Computer, specify the following level prices:
Level Component Prices

Desktop 600
Laptop 700
Using the drop-down control near the top of this dialog, select the other attributes with component-based
prices and specify their prices:
Operating System:

Windows OS
Mac OS 300
Laptop Screen Size:

13-inch
15-inch 100
17-inch 150
Laptop Hybrid Mode:

Screen can flip... 200

Laptop Battery Life:

4 hour
6 hour 50
12 hour 200
Laptop Hard Drive:

128 GB solid state (flash memory) 50
256 GB traditional (spinning platter) 50
Desktop Hard Drive:

512 SSI Web v8

1 TB traditional (spinning platter) 200
On the same dialog, specify that the prices should randomly vary +/-30% from the summed prices (that’s
already the default). Let’s also round the prices to the nearest $25.
Specifying Alternative-Specific Attributes

Next, we need to specify that certain attributes are conditional (dependent) upon the Type of Computer:
Laptop Screen Size

Laptop Hybrid Mode
Laptop Battery Life
Laptop Hard Drive
Desktop Screen Size
Desktop Hard Drive
Also, certain attributes are conditional (dependent) upon Operating System:
Brands (Windows OS)

Brands (Mac OS)
From the ACBC Exercise Settings dialog, click the Alternative-Specific tab.
Click Add… and choose Type of Computer as the Primary Attribute. The Primary Attribute is the
triggering attribute which conditional attributes depend upon.
Specify which attributes are conditional upon Type of Computer:

The dialog above allows you to select which attributes (with all their levels) are conditional upon a level or
levels of the Primary Attribute.
Click Next and specify the following series of checks:
The dialog above has the two levels of the primary attribute (Type of Computer) as rows and the
conditional attributes as columns. We’ve indicated above that laptop attributes should only display for
laptop computers (bottom row) and desktop attributes with desktop computers (top row).
Click OK to finish adding this alternative-specific relationship.
Click Add… to specify the second alternative-specific relationship. Choose Operating System as the
Primary Attribute. Then, select Brands (Windows OS) and Brands (Mac OS) as the two attributes that
are conditional on Operating System.
514 SSI Web v8
Click Next and then specify the following checks:
Click OK to finish adding this alternative-specific relationship.
Specifying the Design

Click the Design tab to specify the settings that govern the design generation and questionnaire flow.
The settings you choose depend on how much information you want to ask from each respondent (how
much time you want them to spend making choices). If you have a relatively large sample size, you may
decide to have each respondent work less hard (and achieve less precision at the individual level). If you
have relatively small sample size, you’ll want to ask each respondent to exert more effort so you can
obtain higher precision with each respondent.
For the purposes of this example, we’ll have ACBC generate 28 near-neighbor concepts for each
respondent to evaluate. We’ll arrange these into 7 Screening Tasks screens, where each task displays
4 concepts.
We also need to decide how many attributes we want ACBC to change from the respondent’s BYO-
selected ideal product when generating near-neighbor concepts. To decide this, it’s helpful to review
what each product concept looks like (in terms of numbers of attributes involved with each alternative).
Here’s a sample display of three product concepts that possibly could be generated:
Note that a Laptop concept is made up of 8 different attributes (7 not including Summed Price). A
Desktop concept is made up of 6 different attributes (5 not including Summed Price). As a general rule,
we don’t want to change more than half of the attributes from the BYO-selected concept when generating
a new near-neighbor concept. So, it would seem to make sense to vary from minimum = 2 to maximum
= 3 of the BYO-selected attributes when generating a new near-neighbor concept.
Regarding the number of Unacceptables and Must-Haves, we have found for alternative-specific designs
that the Unacceptables questions can often present an overwhelming number of potential unacceptable
levels for the respondent to review. Thus, let’s just ask Must Have questions, which don’t show so many
potential levels to the respondent to overly tax their attention.
A recommended series of settings for our study is given below:
These settings lead to the following questionnaire flow for this study:
BYO
Screener 1 (showing 4 concepts)
Must Have 1
Must Have 2
Must Have 3
Choice Tournament Task 1 (showing 3 concepts)
516 SSI Web v8
(Not every respondent will see all 10 Choice Tournament Tasks. Only respondents who indicate that 20
or more of the 24 concepts are a possibility in the Screener section will see all 10 tournament tasks.
For example, if a respondent indicates that 12 of the 28 near-neighbor concepts are a possibility in the
Screener section, then 12/2=6 Choice Tournament Tasks are displayed.)
Testing the Design

ACBC can automatically generate robotic respondent answers, where the answers are random. This lets
you see how many times each level appears for each respondent and the standard errors (the precision,
as estimated using aggregate logit) per level and attribute.
From the Design tab, by default the test design suggests using just 5 robotic respondents. This is fine
for your first test (to see how many times each level appears for each respondent).
Click Test Design… and the following report is displayed:
(Note that we’ve divided the report above into three sections so we can represent it here in this
document. The frequencies for the levels shown for 5 respondents is given.)
Any cells with a count of 1 or less are marked in red. Any cells with 2 counts are marked in yellow.
Other cells with counts >2 are not color-coded. With standard (non alternative-specific designs) in
ACBC, we generally recommend each level appear a minimum of 2 times and preferably 3 times per
respondent. With alternative-specific designs, that isn’t always possible or necessary. For example,
respondent #1 selected a laptop with a Mac OS (we can tell because of the oversampling of the BYO-
selected levels in the design). Therefore, the near-neighbor designs will show fewer Desktops and
Windows OS machines for this respondent to evaluate. For Windows OS brands and the Desktop Hard
Drive size levels, there are two instances for this respondent (who prefers a laptop with a Mac OS) in
which levels appear just once. This, in our opinion, is to be expected and should still lead to reasonable
utility estimation (given the benefits of HB estimation for borrowing information across respondents in the
population).
Next, let’s get a feel for the precision across the sample that we could expect if we were to field such a
study. Let’s imagine that we had planned to interview 250 respondents. We change the 5 respondents
to 250 respondents and run Test Design again. After about 30 minutes or more of computation, the
report is shown.
Standard
Attribute Level Errors
Desktop 0.024
Laptop 0.024
Windows OS 0.022
Mac OS 0.022
Dell 0.043
Lenovo 0.044
Gateway 0.044
HP 0.043
13-inch 0.034
15-inch 0.033
17-inch 0.034
Screen can flip or fold into tablet shape… 0.025

 0.025
4 hour 0.035
6 hour 0.033
12 hour 0.035
128 GB traditional (spinning platter) 0.052

128 GB solid state (flash memory) 0.052
518 SSI Web v8
17-inch 0.041
21-inch 0.039
24-inch 0.041

1 TB traditional (spinning platter) 0.050
Summed Price 0.097
NONE 0.046
(Note: we have excluded the Apple brand attribute from the analysis, since it is just a place-holder for
display purposes only. The full utility associated with the Apple brand is thus captured in the Mac OS
partworth.)
With non-adaptive CBC studies, we have generally recommended that researchers can use aggregate
logit to test the precision of their CBC studies, given a certain number of expected respondents. Our
rule of thumb for the aggregate logit test and random responders is to strive for standard errors of main
effects of 0.05 or less (for generic attributes) and standard errors of 0.10 or less for alternative-specific
effects and interaction terms. Similar rules-of-thumb apply as well to ACBC. But, you will often find that
ACBC studies can achieve better precision than CBC with fewer respondents due to the interactive
nature of the questionnaire that keeps respondents engaged longer in making tradeoff decisions and
choices.
This report for n=250 shows that the alternative-specific attributes have standard errors no larger than
about 0.05. Generic attributes (Type of Computer and Operating System) that apply to every product
concept have standard errors around 0.02. So, these standard errors are well within the guidelines
suggested by the rule of thumb.
An interesting anomaly is the standard error for the Summed Price attribute. Summed Price does lead
to some correlation in the design matrix between the Summed Price variable and the other attributes
(since the summed price is somewhat related to the levels for the other attributes); thus the standard
error is larger than we would see for attributes within typical orthogonal designs. Other research we have
conducted suggests that varying price at least +/-30% from the summed prices leads to reasonable
precision of the price coefficient—provided you help the solution converge properly by constraining price
to be negative sign (lower prices preferred to higher prices).
Fine-Tuning the Questionnaire from a Human Perspective

The previous section dealt only with using robotic respondents to take the ACBC survey and examining
the precision of the utility estimates via aggregate logit. However, our purpose is to interview real
respondents and we need to know that the interview reads well and flows well for them.
We haven’t taken the time in this tutorial to coach you regarding editing the text for each ACBC question
(BYO, Screening Tasks, Must Haves, and Choice Tournament tasks), fine-tuning the layout of each
question, or for providing transitional HTML/Filler questions between the sections. Such coaching is
given in the standard ACBC tutorial that ships within the Help of the SSI Web system.
To test the questionnaire, click Test + Test Survey.
When you test the survey, the first thing you will notice in the BYO is that it is strange to have the 2nd
level of Laptop Hybrid Mode be blank. It seems to make a lot of sense to have that level be blank
within the Screening and Choice Tasks section, but we need to see some text showing for that blank
level in the BYO question and also in the MustHave questions. There is straightforward way to
accomplish this with Conditional Display. However, we’ll show you an Unverified Perl trick for doing this
here. Rather than putting  as the level text for the second (null) level of Laptop Hybrid
Mode, you can use the following unverified Perl code in its place (edit the list associated with Laptop
Hybrid Mode and put the following text as its second element):
[%
begin unverified perl
if(QUESTIONNAME() eq "myacbc_BYO"||
QUESTIONNAME() eq "myacbc_MustHave1"||
QUESTIONNAME() eq "myacbc_MustHave2"||
QUESTIONNAME() eq "myacbc_MustHave3")
{
return "No hybrid capability";
}
else
{
return "";
}
end unverified
%]
This script tells the browser to display “No hybrid capability” on all screens except the BYO and Must
Have screens, where a blank will be displayed.
Merged Rows:
If you view a Screening Task for the study, as specified to this point, it looks something like:
520 SSI Web v8
While the display above indeed shows the necessary information for each Laptop or Desktop, the
information is laid out inefficiently and in some cases out of order. For example, rather than showing
brands on two separate rows (those for Windows on a separate row from those for Mac OC), it would be
nice to merge the brand labels onto the same row. Same issue applies to Screen Size and Hard Drive.
ACBC’s Merged Rows capability lets you accomplish this.
Click Compose + Write Questionnaire. Double-click any of the ACBC questions to open the ACBC
Exercise Settings dialog. Click the Exercise Settings tab then click the Merged Rows tab.
Click Add… to add a merged row specification. We’ll first add a merged row to collapse the brand
information to the same row. Click the two brand attributes in the left panel, then type Brands: in the
Merged Row Label field:
Click OK to save this merged row specification.

Next, let’s collapse the Laptop and Desktop screen size attributes into a common row. Click Add… to
add another merged row specification. Specify:
Click OK to save this merged row specification.

Last, let’s collapse the Desktop and Laptop hard drive attributes. Click Add… to add another merged
row specification. Specify:
522 SSI Web v8
Now, when you preview a Screener Task, it looks like:
The attribute information is now much more compactly and logically laid out on the screen. Even though
it looks to the respondent like there is just one generic attribute for screen size and one generic attribute
for hard drive, the experimental design and utility calculation still recognize that there are alternative-
specific attributes to be managed in the experimental design and estimated in terms of utility scores.
At this point, we have roughed in the ACBC survey. You can test run the survey and fine-tune the layout
and wording of the questions. You may also wish to insert some transitional screen text between the
major sections of the ACBC survey, using Text/HTML Filler questions.
Non-Summed Price Option:
At the beginning of this tutorial, we mentioned that we’d describe how to program this survey without the
Summed Pricing option. A more traditional approach to this kind of conjoint analysis problem is to use
two alternative-specific price attributes:
Desktop Prices: Laptop Prices:

$500 $600
$600 $750
$700 $900
$900 $1,200
$1,200 $1,500
If following this approach, then you would also typically skip the BYO question (since respondents would
just pick the best levels of each attribute, since there is no price penalty exacted for, say, upgrading the
hard drive).
If you would like to program this version of the ACBC questionnaire, take the following steps:
1. With the computers study still open in SSI Web, create a copy of your study (so you don’t lose
your Summed Pricing version of the study), using File + Save Study As and call the new study
computers2.
2. Click Compose + Lists and delete the Summed Price element from the Attributes list. (You
will receive some warnings that you are about to change a list being used in the questionnaire
and in the ACBC study. Confirm that you want to do this.)
Then, while still editing the Attributes list, add two new list elements at the bottom of the list (the
11th and 12th elements of that list):
Desktop Prices:
Laptop Prices:
3. Add two new predefined lists within the List Manager:
List: DesktopPrices
$500
$600
$700
$900
$1,200
List: LaptopPrices
$600
$750
$900
$1,200
$1,500
4. Edit the ACBC Exercise. On the Attributes tab, select the lists associated with Desktop prices
and Laptop prices. Specify Sequence Order of Low to High and Preference Order of Best to
Worst for both of these attributes.
Specify that all attributes should not be shown in the BYO question (we will skip the BYO
question).
We’d recommend you not display either price attribute in the Must Haves and Unacceptables.
524 SSI Web v8
When you are done with step 4, the dialog should look like:
5. On the Alternative-Specific tab, highlight the Type of Computer Alternative-Specific

specification so that we can modify it to also include information about the two new pricing
attributes. Click Edit… to edit that specification. Click Edit Conditional Attributes… to add
the two price attributes to the Alternative-Specific specification:
Click Save to commit the changes. Then, edit the checks on the Edit Alternative-Specific
Relationship dialog to include the two new attributes:
Click OK (twice) to commit the changes.
6. You next may receive a warning that the BYO concept cannot be included in the tournament if it
hasn’t been asked (remember, we are skipping the BYO question). On the Design tab, un-
check the box that says Include BYO in Tournament.
7. Add a new Merged Rows specification from the Merged Rows tab. Click Add… and then make
that dialog look like the following:
This merges the two separate price attributes onto the same row in the display.
526 SSI Web v8
3.3 ACBC Design Algorithms

3.3.1 How Concepts Are Chosen
ACBC creates an array of product concepts for the respondent to evaluate within the Screener and
Choice Tasks sections of the questionnaire. These concepts are chosen as "near-neighbors" to the
product concept the respondent chooses in the BYO task, but still include the full range of levels taken
into each respondent's ACBC survey. Because the BYO-specified product concept differs across
respondents and the number of attributes and levels taken into each respondent's ACBC exercise can be
dynamic (though in most projects the list will be static), it isn't possible to create an experimental design
prior to fielding the study (as is done in non-adaptive CBC studies). Customized designs must be
generated on-the-fly for each respondent. Because dozens or even hundreds of respondents might
simultaneously be completing surveys over the web, we are concerned about the amount of computing
resources demanded of the server to generate designs. Therefore, we've developed a relatively quick
algorithm. The algorithm cannot be said to produce optimal designs, but its designs are near-orthogonal,
and have proven to work exceptionally well in many methodological studies to date comparing ACBC to
standard CBC.
Inputs
The respondent provides the following input to the customized design:

C0, a vector with as many elements as attributes included in this respondent's BYO question,
describing which levels were included in the BYO concept.
The analyst provides some inputs that control the design:

T, the number of total product concepts to generate,
Amin, the minimum number of attributes to vary from the BYO concept,
Amax , the maximum number of attributes to vary from the BYO concept (restricted to be no
more than 1/2 the number of attributes in the BYO question +1, not including Summed Price),
If a "summed price" attribute is used, a range is provided specifying how much price should
vary (randomly) from the summed components' price (e.g. 30% below to 20% above summed
price).
The Design Algorithm
Near-orthogonal designs are generated using a controlled, randomized process. The steps involved in
selecting each of T concepts in the design are as follows:
1. Randomly select an integer (Ai) from Amin to Amax that specifies how many attributes
within C0 will be modified to create new (near-neighbor) concept Ci.
2. Randomly select Ai elements within C0 to modify.
3. Randomly select new (non-BYO selected) levels for the attributes chosen in step 2 (all other
attributes remain at the BYO-selected levels).
4. Check to ensure that the concept chosen doesn't violate any prohibited pairs and is not a
duplicate to another concept previously selected for this respondent. If prohibited or duplicate,
discard the concept and return to step 1.
5. For non-BYO selected levels, examine whether relabeling levels to another non-BYO selected
level within the same attribute improves the relative D-efficiency of the design for this
respondent. Examine whether swapping non-BYO selected levels between two concepts
improves the relative D-efficiency. Any relabeling or swapping that increases the efficiency
while not making the target level count balance worse is accepted.
Steps 1-5 are repeated as many times as can be done within about one second per respondent. This
means that faster servers can lead to slightly more efficient designs. Faster server response happens
when there is less load on the server, faster hardware, or both. However, do not become too preoccupied
with these issues, as even one pass through the design algorithm leads to highly efficient designs.
Once a design is quite good (such as from a first pass through the algorithm), comparing the results of
such a design to a different one that has an even higher degree of D-efficiency will usually show little
practical difference in terms of the quality of the final utilities.
Issues Related to Level Balance and Efficiency
"Counts" arrays (at the individual level) are maintained to ensure that the designs have much greater
balance than would be achieved if the above strategy involved purely random selections. A counts array
keeps track of how many times each element has been selected or modified. For example, we maintain
a counts array for each attribute that records how many times each level within that attribute (other than
the BYO-selected level) has been included in the design. When a relative deficit occurs in the target
frequency of selection, we increase the likelihood that the element with the deficit will be selected in the
next concept (the mechanism for doing this is described in the section Deviation from Target
Frequencies). This allows for "controlled" randomized designs, and leads to a relatively high degree of
level balance (but not perfect level balance). Our approach leads to a high degree of balance for: a) how
many times Amin to Amax attributes were varied when generating T concepts, b) how many times each
attribute was varied, and c) how many times each level (other than the BYO-specified level) was included
across the T concepts.
For attributes without logical preference order (such as color, style, and brand), we try to achieve level
balance across the non-BYO chosen levels. For attributes featuring levels with numeric/logical order
(such as speed, size, weight, etc.), we overweight by a factor of two the selection of levels directly
adjacent to the BYO-chosen level. Thus, the counts array describing the frequency of level occurrence
across T=36 concepts for four levels of speed might be as follows (assuming the respondent chose level
three in the BYO concept): 3, 6, 21, 6. Levels 2 and 4 are oversampled by a factor of 2x relative to level
1.
When a "summed price" attribute is in place, we first sum the total price for the concept (according to
the level-based prices specified in the questionnaire setup). We then draw a random price variation
(continuous random variable, rectangular distribution) within the range of price variation specified by the
questionnaire author (for example a -30% to +30% price variation involves drawing a price multiplier from
0.7 to 1.3). Our randomization strategy ensures a high degree of balance across the price variations.
We divide the range of price variation into quartiles, and randomly select (without replacement) which
quartile will be used for the next random price draw (a random continuous value within that quartile). For
example, the range for the price multiplier of 0.7 to 1.3 is broken into quartiles 0.7 to 0.85, 0.85 to 1.0,
1.0 to 1.15, and 1.15 to 1.3 (for tracking purposes). If we need to draw a price variation from the fourth
quartile in creating concept Ci, we draw a random price multiplier anywhere within the range 1.15 to 1.3.
After multiplying the summed price by that random price multiplier, the resulting price can then be
rounded to the nearest x dollars (or monetary units of choice), where x is controlled by the survey author.
As examples, one can round to the nearest 100 dollars, the nearest 1 dollar, or the nearest 0.1 dollars.
One can also choose to add a constant after the rounding, such as -0.01, such that rounding to the
nearest 0.1 dollars and subtracting 0.01 leads to prices ending in 9s. The actual price shown is recorded
528 SSI Web v8
in the data file for use in analysis.
Certain constraints (such as attribute prohibitions or avoiding dominated concepts) can make it difficult or
impossible to maintain a desired degree of balance. If the design algorithm is unable to find a new
concept Ci with the counts balancing goals in place, it will relax the balancing criteria in order to
generate a valid concept. In that case, a message will be written to a special ACBC log file name
studyname_ACBC_log.cgi stored within the Admin folder for your information and review.
From the standpoint of achieving maximum statistical efficiency, the optimal approach would vary all
attributes independently, as in the traditional orthogonal array. By choosing only a narrow subset of the
attributes to vary (from the BYO-specified concept) when creating each new concept, lower statistical
efficiency results. However, ACBC's approach has three important benefits to counteract the loss in
statistical efficiency: 1) Respondents can answer with less noise when fewer attributes are varying within
each concept, 2) The concepts seem more relevant and plausible to the respondent, since they are near-
neighbors to the BYO-specified product, and 3) The design concentrates on learning about preferences
for the levels directly surrounding the respondent's highest levels of preference. In our previous
methodological tests with about eight or nine total attributes, we have found that using 2 and 4 for Amin
and Amax , respectively, works quite well. We tried varying from 2 to 5 attributes when generating each
concept in one of our experiments, and didn't see an increase in performance. At a recent Sawtooth
Software (2013), two additional tests were done that experimented with how many attributes to vary from
the BYO selections for the near-neighbor designs. Whether fewer or more attributes were varied, ACBC
results were quite robust and of high quality.
Because the designs are adaptive, one cannot estimate the exact design efficiency prior to fielding the
study. However, the Test Design capability simulates respondents who answer ACBC questionnaires
randomly, followed by aggregate logit estimation to investigate the size of standard errors (reflecting the
precision of estimates) for pooled parameters.
When Attributes Are Dropped from the BYO Section
There are instances in which an attribute with a priori preference order seems out-of-place in a BYO
question. If price premiums aren't assigned to such an attribute, it seems painfully obvious to ask
respondents which level they would prefer in their BYO product. In such cases, you can drop attributes
from the BYO question. The software will also let you drop attributes from the BYO question even if you
don't establish a priori preference order (though this is an exception rather than typical practice, and you
lose an opportunity to learn which level is preferred via the BYO question).
When attributes with known a priori order are skipped in the BYO section, for software developer's
convenience, the software will automatically fill the missing BYO answers as if the respondent had
picked the best level. For attributes dropped in BYO without known a priori order, the software will
automatically fill the missing BYO answers with randomly selected levels for those attributes. However,
in either case, the design algorithm doesn't treat these automatically filled responses as legitimately
chosen BYO levels for purposes of utility estimation or design generation. All levels of the skipped
attributes are sampled roughly evenly in the near-neighbor design. "Dropped" attributes from the BYO
section are not included in the vector C0, and are thus not counted as an attribute to be modified from its
BYO level.
Strategies for Selecting Attributes to Vary (Attribute Balance vs. Mixed

Approach)
The first three methodological tests we ran comparing ACBC to standard CBC used the strategy
described above, where for the generation of each new concept we randomly selected a subset of
attributes to vary from the BYO-selected levels such that each attribute had been varied from its BYO-
selected level about an equal number of times. However, it has occurred to us that such a strategy
misses an opportunity to improve the amount of information we gain from each respondent in the typical
cases in which we have asymmetric designs (where the number of levels per attribute varies). For
studies with asymmetric designs, attributes with fewer levels have their levels probed more often than
attributes with more levels.
Consider an attribute with 6 levels versus another attribute with 2 levels. If the number of times we
decide to vary the attributes from the BYO-selected levels is equal for these two attributes, it is clear that
the number of times that the 1 non-BYO selected level from the 2-level attribute will be probed (included
in the design) is greater than for the 5 non-BYO levels from the 6-level attribute. However, if our strategy
is to reduce uncertainty over the entire attribute level space, it would follow that we should select the 6-
level attribute more often for varying from its BYO-selected level than the 2-level attribute. This can be
accomplished simply by changing the manner in which we increment the counter array that keeps track
of how many times each attribute has been varied from its BYO-selected level. For the first three studies
we ran, every time we altered an attribute from its BYO-selected level, we added a "1" to the counts
array with respect to that attribute (where the counts array has as many elements as attributes in the
study). The goal was to select subsets of attributes to modify from BYO-selected levels such that the
counts array was balanced, for example, [18, 18, 18, 18, 18, 18] for a study involving six attributes, to
indicate that each attribute had been varied 18 times from its BYO-specified level.
If we want to select attributes to vary such that there is greater balance in the number of times each non-
BYO level has been probed, we could add 1/k to the counts array each time we select an attribute for
variation from its BYO-selected level, where k is equal to the number of levels in that attribute. This
would reduce the uncertainty across the entire attribute level space, but at the cost of adding a potential
psychological bias. If one attribute is shown to vary from its BYO-selected level much more often than
another across the product concepts, then undue attention may be called to that attribute. For that
reason, we have implemented a mixed strategy that is a middling position between these two
approaches (attribute-selection balance versus level-selection balance). It puts more attention toward
testing non-BYO levels for attributes with more levels than for attributes with fewer levels; but it doesn't
enforce strict level balance. We believe that the mixed approach should be better than the attribute
balance approach, but look to future empirical research regarding this issue. Such research must involve
human respondents rather than computer-generated respondents, since it involves a potential
psychological bias.
Generating Replacement Cards
A key aim of ACBC questionnaires is to identify any levels that are "unacceptable" or "must haves."
Rather than ask the respondent upfront regarding cutoff rules, we formulate hypotheses regarding what
levels might be unacceptable or must-haves based on the respondent's observed choices within the
Screening section. For example, if we notice that the respondent only selects as "a possibility"
concepts featuring Brand A, we might suspect Brand A is a "must-have" level. After we have observed a
pattern of such choices, we present a list of "unacceptable" or "must have" rules that we suspect the
respondent might be employing. If the respondent confirms a non-compensatory rule on the list, we then
mark as "not a possibility" any concepts not yet evaluated that would fail to satisfy that rule. This leads
530 SSI Web v8
to more efficient questionnaires and the opportunity to create "replacement cards" that probe deeper
within the respondent's relevant preference space. For example, if after a respondent has evaluated the
first 10 concepts (marking each as a "possibility" or "not a possibility"), the respondent verifies that
Brand A is a "must-have" rule, this might eliminate from consideration 8 of the upcoming concepts (all
featuring a brand other than Brand A) that we had planned to ask the respondent to evaluate. We
generate 8 replacement concepts (all featuring Brand A) according to the design algorithm specified
above, the exception being that the brand attribute will be removed from consideration as a potential
selection in step 2 (above).
Concept Presentation in the First Few Screening Tasks
We rely heavily upon respondents' evaluations of the first few screens of concepts within the Screening
section to identify cutoff rules. It is typical to show four or five product concepts per screen within the
Screening section. It seems useful to us that the first two screens of concepts should reflect the full
variety of levels included in each respondent's attribute list. Therefore, after generating T concepts as
described above, we attempt to select sets of concepts to show in the first two screens that contain the
full range of levels included in the study. We employ a simple algorithm that involves counting how
many levels are represented on each screen, and selects sets from our T concepts that lead to a
relatively high score on that count.
Concept Presentation in the Choice Tasks Tournament
The concepts the respondent indicates are "possibilities" in the Screening section are taken forward to
the Choice Tasks section where they are compared in triples until an overall winner is identified. It takes
t/2 triples to identify the overall winner, where t is the number of concepts to be evaluated (in the case
that t is odd and t/2 is not an integer, one rounds down to determine the required number of triples). It is
easier to compare the concepts if some of the attributes are tied (share the same level) across the triple.
When this happens, we "gray-out" the row so that respondents can focus on the differences when
choosing the best concept. We can reduce the cognitive difficulty of the task if we assemble the
concepts so that a healthy number of rows are "grayed-out." To that end, we take modest steps to
increase the likelihood of tied attributes, but stop well short of maximizing the "amount of gray." This
sacrifices some design efficiency (in the traditional sense, assuming compensatory processing), but the
tasks are more easily completed and the responses will contain less error.
3.3.2 Deviation from Target Frequencies

In the previous section, we described the method that ACBC uses to account for how many times each
non-BYO level has been used in the design and how many times each attribute has been varied from its
BYO-specified level. We described that the algorithm increases the likelihood that a level or attribute will
be chosen if a deficit in the "counts array" is noted. We also said that the process tends to lead to
good, but not perfect balance and target frequencies. With ACBC, we are not approaching design
generation in a traditional sense (we are using an adaptive procedure). Strict attribute/level balance is
contrary to our aims and to the goal of reducing uncertainty in the utility estimates.
There are a few reasons why the target level frequencies or number of times each attribute has been
varied from its BYO-specified level may not perfectly match the goals of the algorithm:
1. If prohibitions are in place (including the setting to avoid dominated concepts), it may be
impossible to achieve target frequencies.
2. When respondents specify certain unacceptable or must-have rules, certain levels are forced in/
out of the design for replacement cards, which by necessity leads to imbalance.
3. Even under the "Attribute Balance" design generation option, attributes with very few levels tend
to be chosen less often (for modification from their BYO levels). That is because there are fewer
levels to choose from to generate new, unique concepts. If duplicate concepts are generated, the
algorithm chooses new attributes to try. This outcome is probably favorable, because better overall
level balance occurs if we spend more effort investigating (relative to BYO choices) attributes with
more levels than attributes with fewer levels.
4. When subsets of attributes are chosen to be modified from their BYO-specified levels and levels
are chosen within the attribute that is being varied, we scan to see which one of those chosen
attributes or levels has the highest representation in the current respondent's design. We then
discard that choice of attribute or level in favor of the attribute or level that has the least
representation in this respondent's design. This process nudges us in the direction of achieving
targeted frequencies. But because only one attribute or level per draw may be discarded in favor of
one under-represented attribute or level, it may not be able to "fill the holes" fast enough to achieve
the desired attribute/level frequencies. But, if we were to allow the algorithm to catch up as quickly
as needed in favor of perfectly matching target frequencies, patterns of attribute/level inclusion
could emerge to the detriment of randomization and design efficiency.
5. If for some reason ACBC is unable to generate enough concepts following its design algorithm
to satisfy the demands of the questionnaire, the attribute/level frequency targets are ignored and
new concepts are generated following a randomization strategy. Note: if the experimental designer
cannot find a design within the allotted time that satisfies the requested goals and constraints, it
may need to relax the goals and constraints to produce a valid questionnaire for the respondent. If
this occurs, a message is stored in the design log table of the database. You can download this
from the Admin Module, by click ing Survey Administration + Advanced + Design Log Table.
This downloads a .csv file that you can view in a program such as Excel, showing per respondent
messages related to the designer process.
532 SSI Web v8
3.4 Advanced ACBC Surveys

3.4.1 Customized (Constructed) Attribute and Level Lists
Warning: this is an advanced area. We caution you to proceed carefully if using constructed lists.
ACBC includes the ability to customize the list of attributes and levels taken into the ACBC survey for
each respondent (using SSI Web's List Building). We have had success using as many as 10 attributes
with 33 total levels within ACBC studies. We don't know whether respondents can effectively manage
even more attributes than this, and future research may answer that question.
ACBC is designed to handle as many as 100 attributes, with up to 250 levels per attribute. This capacity
is provided because computers can handle the computational and memory requirements, not because
we'd ever recommend doing such a large study!
For most studies in practice, each respondent will evaluate the entire list of attributes and levels within
the ACBC sections of the survey. However, because the list of attributes and the levels within each
attribute are stored as lists within SSI Web, it is possible to create constructed (customized) lists of
attributes and levels for each respondent. It is possible to exclude entire attributes from consideration in
ACBC (because we believe these attributes to be entirely irrelevant to a respondent and carry no
importance) or to exclude certain levels from attributes (such as excluding irrelevant brands that are
unacceptable or not available). Any of these exclusions must be done prior to beginning the ACBC
section of the SSI Web survey.
We must stress that we don't know the best way to ask a respondent to indicate which attributes are
irrelevant to him or her so that we could exclude them from the attribute list. This is left to your
judgment (though we recommend that the range of levels associated with an attribute should be shown to
respondents when asking them to mark irrelevant attributes).
We also don't know the best way to ask regarding which levels (such as levels of brand) are
unacceptable. However, if levels have associated level-based component prices in your design, you
should display the level-based prices when eliciting which levels to take forward to the ACBC
questionnaire. The utility estimation procedure assumes that levels were accepted/rejected considering
their component-based prices.
Please consider the following assumptions that we make when dealing with customized (constructed)
lists of attributes and levels.
1. If an attribute is omitted from a respondent's customized attribute list, we assume it has zero
importance (all levels will carry a part-worth of zero for that attribute).
2. If a level is omitted from a respondent's customized attribute list, you must indicate whether the
omitted level should be assumed to be: a) Generally inferior to all included levels (but not
absolutely unacceptable), b) Absolutely unacceptable, c) Missing randomly, and preferences
should be imputed from the population distribution.
Notes and Warnings:
ACBC can handle the case of just one level being brought into a constructed list for an attribute. But,
zero levels within a constructed list for an attribute will produce a fatal error in the interview; therefore,
you'll want to program sk ip logic to ensure that respondents either reconsider their answers (leading to at
least one level in the level list) or that they sk ip over the ACBC questionnaire.
Be careful if dropping selected levels for a sequentially ordered attribute with a preference direction. If
you do this, you must bring a contiguous set of attribute levels into a respondent's design, and you
should impose utility constraints upon this attribute when estimating part-worths. In the case of
sequentially ordered attributes with preference direction, we do not add additional choice information
indicating that dropped levels are inferior to included ones; we ask you to inform utility estimation of this
via global utility constraints.
Building Constructed Lists Using Unverified Perl
If you are conditionally dropping attributes or levels from consideration, you will sometimes need to use
Unverified Perl when building the constructed lists (a constructed attribute list, or a constructed level list
for an attribute).
For example, consider the case in which you are studying too many brands of grand pianos to include in
each respondent's study design (recall that we recommend bringing no more than about 7 levels for each
attribute into each respondent's ACBC study design). In this example, further assume that we are using
summed pricing, and these 15 brands have associated level-based relative prices:
Baldwin +$6,000
Bechstein +$10,000
Bosendorfer +$18,000
Boston +$12,000
Chickering +$4,000
Estonia +$5,000
Kawai +$8,000
Kimball +$4,000
Knabe +$2,000
Petrof +$6,000
Samick +$2,000
Steinway +$25,000
Wurlitzer +$4,000
Yamaha +$8,000
Young-Chang +$0
The actual parent list of brand levels that will feed into the constructed list of brands associated with
Attribute 1 in ACBC might be called PianoBrands:
Predefined List: PianoBrands

Baldwin
Bechstein
Bosendorfer
Boston
Chickering
Estonia
Kawai
Kimball
Knabe
Petrof
Samick
Steinway
Wurlitzer
Yamaha
534 SSI Web v8
Young-Chang
If we use screener questions to ask respondents which brands they would consider and which they
would never consider, we should show them with their associated relevant prices. We should do this
because the way we encode the information for the BYO for utility estimation assumes that respondents
considered every brand in the parent list at their relative prices: that they chose their considered brands
and rejected the non-considered brands.
So, we might ask a select-type screener question named Q1 prior to beginning the actual ACBC
questions in which the following list is presented (with relevant reference prices):
Predefined List: Q1PianoBrands

Baldwin (typical price $13,000 for 6-foot piano)
Bechstein (typical price $17,000 for 6-foot piano)
Bosendorfer (typical price $25,000 for 6-foot piano)
Boston (typical price $19,000 for 6-foot piano)
Chickering (typical price $11,000 for 6-foot piano)
Estonia (typical price $12,000 for 6-foot piano)
Kawai (typical price $15,000 for 6-foot piano)
Kimball (typical price $11,000 for 6-foot piano)
Knabe (typical price $9,000 for 6-foot piano)
Petrof (typical price $13,000 for 6-foot piano)
Samick (typical price $9,000 for 6-foot piano)
Steinway (typical price $32,000 for 6-foot piano)
Wurlitzer (typical price $11,000 for 6-foot piano)
Yamaha (typical price $15,000 for 6-foot piano)
Young-Chang (typical price $7,000 for 6-foot piano)
None/No More of the Above
However, we'll want to use answers to Q1 (or multiple similar questions) to build a constructed list of the
brand names to be used as the level list for Attribute 1 within the ACBC survey. In other words, if
respondents pick:
Kimball (typical price $11,000 for 6-foot piano)

Samick (typical price $9,000 for 6-foot piano)
from the list (Q1PianoBrands) we show in Q1, we want to add the following brand items from the original
parent list (PianoBrands) to the constructed list to be used in the ACBC study for Attribute 1:
Kimball
Samick
One can use a series of IF-Then statements in unverified perl to accomplish this, building a new list
called PianoBrandsForACBC using PianoBrands as the parent list, such as:

if(GETVALUE("Q1_1") == 1)
{
ADD("PianoBrands", 1);
}
{
}
{
}
end unverified
Etc., for as many items as need to be added to the list.
But, we'll show you a more generic and elegant way to do this in unverified perl. For this example, we
create two new constructed lists:
ScreenedInPianoBrands
PianoBrandsForACBC
To build ScreenedInPianoBrands, you select Q1PianoBrands as the parent list, and you use the
following list-building logic:
AIC (Q1, 1, 15)
Create a constructed list named PianoBrandsForACBC (select PianoBrands as the parent list). Insert
the following constructed list logic:
MIRROR (ScreenedInPianoBrands)
This command will include all of the members from PianoBrands that correspond to the list values in
ScreenedInPianoBrands.
536 SSI Web v8
3.4.2 Price Adjustments (Within Summed Pricing)

ACBC provides the ability to assign price premiums or discounts to levels within your study. You do this
via "Summed Pricing" by assigning Level Prices on the Pricing tab. However, there may be instances
where you want to implement price adjustments to go beyond the simple additive prices offered through
Summed Pricing.
Consider an example with brands and styles, as might be set up with standard Summed Pricing:
Levels: Level Prices:
Brand A $10
Brand B $15
Brand C $12
Size 1: $0
Size 2: $5
Size 3: $8
Brand B is a premium brand, and it is possible that in reality the pricing for sizes is quite different for
Brand B. Perhaps for Brand B, Size 1 adds $0; Size 2 adds $10, and Size 3 adds $20. To implement
this exception for Brand B, we use Price Adjustments.
Price adjustments are seen explicitly by respondents within the BYO task (where respondents see the
prices per level), but are not explicitly evident to the respondent in the subsequent tasks, where only the
total price is shown.
Because ACBC estimates continuous functions for Summed Price (and due to the random variation,
typically in the range of about +/-30%, given to the summed price amount), Price Adjustments should
have little to no practical effect on the final partworth utilities and slope of the price function. But, price
adjustments should improve the realism of the design, in terms of displaying concepts that have more
realistic prices.
When you click Price Adjustments... (from the Pricing tab), a dialog is displayed that contains a grid.
Using the grid, price adjustments can be based on a single attribute, or the combination of multiple
attributes. In the example below, we do a simple price adjustment just based on the occurrence of
attribute 1 level 2 (Brand B).
In the example, above, if attribute 1 level 2 appears (Brand B) extra amounts are added to the prices for
attribute 2. So, if Brand B appears, an additional $5 is added to "Size 2" and an extra $12 is added to
"Size 3." Those prices are added above and beyond the generic level-based prices specified on the
Pricing tab.
One can make price adjustments based on the combination of multiple levels from different attributes, by
using the cells further right within the grid.
538 SSI Web v8
3.4.3 Conditional Display

Conditional display/graphics lets you display a graphic (or text, or video clip) on the screen, based on a
level (or levels) included in a product concept. For example, a graphic is shown based on the first four
attributes in this study on polyethylene utility tabes (size, leg style, color, and fold-in-half feature):
Conditional display can be shown above or below the concept. Or, conditional display elements can
even replace text within concept.
One of the challenges of creating conditional graphics based on the simultaneous combination of
multiple attributes is that the total number of unique graphics to develop can be very large. In the
example above with polyethylene utility tables, the conditional graphic is based on four attributes, each
with just 3, 2, 2, and 2 levels, for a total of 3x2x2x2=24 total graphics. This is fairly manageable. But,
with a larger number of levels and/or attributes, the total number of graphics could quickly get out of
hand.
Adding a Conditional Relationship
To specify conditional display elements, go to the Exercise Settings tab within your ACBC exercise.
Select the Conditional Display tab.
Add a conditional display element by clicking Add... Then, specify which attributes participate in the
relationship (up to as many attributes as are in your study).
A Conditional Display Table is opened for that relationship, with one row per possible attribute
combination. You can directly type text into the fields, or you can cut-and-paste from programs like
Word or Excel. To add graphics to the table, use the Pencil icon and then browse for the graphic
using the Camera icon .
Attribute Visibility... Although each attribute needs to be displayed in the BYO section, some
attributes may seem entirely redundant if they are already included in the graphical representation of the
product concept. Therefore, you may wish to make certain attributes invisible after the BYO question has
been completed. This button accesses a grid where you can select any attributes to remove from
subsequent sections.
Display Controls
For each conditional display relationship, you can specify whether the element to display should occur in
BYO and/or in questions after BYO.
BYO
Display in BYO: Check this to display the conditional element (text or graphics) within the BYO
question.
Position: Choose whether the conditional text/graphic should appear above, below, to the left, or to
the right of the product specification grid. You can also choose to customize the position (see
below), which allows you to specify a fixed position on the screen where the graphic is displayed
(so it appears to "hover" over the area even though the respondent may scroll up and down within
the window).
Placeholder Width and Height: Before the levels have been picked in BYO, SSI Web doesn't know
what conditional element (such as a graphic) to display. But, SSI Web can reserve an area for the
graphic to display so that other text on the screen doesn't suddenly reposition on the screen when
the conditional display text/graphic is available to be shown. Specify in pixels the width and height
of the placeholder to reserve.
Default Text/Graphic: Here, you can specify the text or graphic to display by default, prior to SSI
Web learning enough information from the BYO selections to show a conditional element.
Questions After BYO
Display in Questions After BYO: Check this to display the conditional element (text or graphics) in
questions after BYO.
Position: Choose whether the conditional text/graphic should appear above or below the product
concept, or whether it should replace the text of an attribute within the concept.
Attribute Label (Optional): If you are showing attribute labels in the left of the grid, you can specify
a label to appear to the left of the conditional text/graphics.
540 SSI Web v8
Custom Graphic Display in BYO
If you select Position: Custom for the position of the conditional text/graphic in BYO, then you need to
do a little extra work to tell SSI Web how to position the conditional text/graphic within the BYO
question.
An SSI Script function is provided to help with this:
[ % BYOCo n d T e x t ( Ex e r c i s e Na me , Co n d T e x t # ) %]
[ % BYOCo n d T e x t ( Ex e r c i s e Na me , Co n d T e x t # , XPo s , YPo s ) %]
Where "ExerciseName" is replaced with the name of the Exercise, and "CondText#" is replaced with the
number of the row for the conditional text you entered, as seen in ACBC Excercise Settings +
Conditional Display. If you had named your ACBC exercise "pianos," and if you wanted to display the
first row listed in the ACBC Exercise Settings + Conditional Display, then the Perl would read:
[% BYOCondText(pianos, 1) %]
If you place this SSI Script function within the Header 1, Header 2, Footer, Side Panel, Top Panel, or
Bottom Panel sections, the conditional display element (such as a graphic) is shown in that position. If
you use the additional XPosition and YPosition elements, then the graphic is shown at a fixed coordinate
on the screen (and the graphic seems to hover over that coordinate if the respondent scrolls).
3.4.4 Skipping Sections in ACBC

Warning: Sk ipping sections in ACBC is an advanced capability and should only be undertak en after a
careful review of the potential consequences for analysis.
For certain research situations, you may find that some question types within ACBC just don't seem to
make sense in terms of representing the buyer's/chooser's decision-making process. ACBC is very
flexible, supporting the following questionnaire section flow patterns:
1. BYO
2. Screening
3. Choice Tasks
4. BYO + Screening
5. BYO + Choice Tasks
6. Screening + Choice Tasks
7. BYO + Screening + Choice Tasks (this is the suggested, typical ACBC approach)
Moreover, all 7 patterns above support the option of including a Calibration Section.
To obtain an estimate of the None parameter, either the Screening or Calibration sections must be used.
To skip each of the sections, do the following:
Skip BYO Section: From the ACBC Exercise Settings + Attributes dialog, specify that each
attribute should not appear in the BYO.
Skip Screening Section: From the ACBC Exercise Settings + Design dialog, specify 0 screening
tasks, 0 Unacceptables, and 0 Must Haves.
Skip Choice Tasks: From the ACBC Exercise Settings + Design dialog, specify 0 product
concepts brought into the choice tournament.
Research presented at the 2013 Sawtooth Software Conference by Fotenos et al. found that skipping the
Screening section (pattern 5 above) led to equally good results for predicting complicated holdout choice
tasks as the standard pattern 7 (though it did not lead to an estimate of the None parameter). More
research is needed to determine which sections are most valuable under which situations.
We provide the flexibility in ACBC to customize the experiment to fit the researcher's view of what is
most appropriate for each research study. Perhaps more research will be presented at future
conferences that compares different ACBC questionnaire approaches.
542 SSI Web v8
3.4.5 Alternative-Specific Designs

With standard conjoint designs, if there are six attributes in the study, a level from all six attributes
appears within each product alternative. But, there are some types of conjoint designs wherein some or
all product alternatives have their unique sets of attributes. With alternative-specific designs, some
product alternatives can have a different number of attributes from others. For example, consider
desktop and laptop computers, where laptop computers have batteries but desktop computers do not.
To implement such a design in ACBC, we create an attribute list:
AttributeList:
Brand:
Type of machine:
Battery life (laptop):
Hard drive capacity:
Processor:
BrandList:
Dell
Acer
Gateway
TypeList:
Desktop
Laptop
BatteryList:
2 hour battery life
4 hour battery life
6 hour battery life
HardDriveList:
600 GB
800 GB
1250 GB
ProcessorList:
2 GHz
3 GHz
3.5 GHz
Before proceeding, it is useful to introduce some terminology to describe the different roles attributes can
play in alternative-specific designs:
Primary attribute: an attribute whose levels appear in every product concept and upon which
conditional attributes (described below) depend. In the example above, attribute 2, type of machine,
is the primary attribute.
Conditional attribute: attributes only displayed with a particular level or levels of the primary
attribute. In the example above, attribute 3, battery life, is a conditional attribute. It will only be
shown for laptop machines. This attribute will be "blank" for desktop computers.
Common attribute: attributes displayed with all levels of the primary attribute. In the example
above, brand, hard drive capacity, and processor are common attributes (these attributes appear
with all possible product concepts).
Specifying Alternative-Specific Design in the Software
Assuming you have already specified your attribute and level lists and have added an ACBC exercise to
your SSI Web survey, you first should open that exercise by double-clicking the exercise name from the
study navigation window, by double-clicking any ACBC question within the exercise, or by highlighting
any ACBC question and clicking Edit....
From the ACBC Exercise Settings dialog, select the Alternative-Specific tab. Continuing with the
example of laptop and desktop computers that we introduced above, we need to specify that the battery
life attribute is conditional upon the primary attribute, type of machine.
Click Add... to add an alternative-specific relationship.
Choose the primary attribute (type of machine).
Select all attribute(s) that are conditional upon type of machine. In this case, just battery life. Then,
click Next.
Next, click which conditional attribute(s) apply to which levels of the primary attribute. In this case,
battery life is conditional upon type of machine = laptop.
That's all that is needed to make the Battery Life attribute only appear with Laptop computers.
There is nothing unique to be done during analysis, other than to recognize two issues: 1) You shouldn't
try to specify an interaction effect between a conditional attribute and the primary attribute (the
conditional attribute has already been estimated with respect to the level it applies to for the primary
attribute), 2) In the market simulator (Sawtooth Software's Online Simulator or SMRT), when an attribute
doesn't apply to a product option, you specify that the attribute has a level "NA" (Not Applicable).
Missing Level Text
When an attribute doesn't apply to the current concept, by default nothing is displayed in that position.
You can change what is displayed (by attribute) using Missing Level Text.... The text you use here
must be something neutral that indicates the lack of any delivered value for that attribute (since the utility
estimation assumes nothing is present for that attribute to which we could attach any utility value).
544 SSI Web v8
Unacceptables Questions
You may find with alternative-specific designs that the first time an Unacceptable question appears, it
can be overwhelming to the respondent--lots of levels being displayed that are suspected as potentially
unacceptable to the respondent. This occurs because alternative-specific designs typically have more
attributes and levels than traditional ACBC studies and alternative-specific attribute levels don't occur as
often across product concepts as do generic attributes. Thus we have less information than traditional
ACBC studies (given the same number of Screeners) for confirming which levels are actually acceptable
to the respondent. Therefore, we recommend you either 1) collect more Screener choices prior to
displaying the first Unacceptables question than with traditional ACBC studies, or 2) only use Must Have
questions (show 0 Unacceptables questions).
Minimum and Maximum Attributes to Vary from BYO Selections
Alternative-specific designs typically involve more thought when specifying the design settings,
especially the minimum and maximum number of attributes to vary from BYO selections when
generating near-neighbor concepts.
To review, if you use ACBC in the standard way, you first ask respondents to select their preferred
product concept within the BYO (Build-Your-Own) question. From the Design tab within the software,
you specify how many of the attributes selected in the BYO question should be changed from the BYO-
selected levels when generating new near-neighbor product concepts.
Let's first consider a standard ACBC design where all the attributes are common to all product concepts.
Let's further assume a total of 8 attributes in the study design, where all 8 attributes are shown in the
BYO question. It would be very typical with such a design to indicate that a minimum of 2 to a
maximum of 3 attributes should be changed from the BYO selections when generating new product
concepts. This would mean that either 5 or 6 of the 8 attributes included in the BYO question would be
retained at the respondent-selected levels, and the other 2 or 3 attributes would be changed.
However, let's consider a more challenging alternative-specific design featuring a choice of transportation
to get to work of Bus, Bicycle, Train, or Car. Furthermore, let's assume that Bus, Bicycle, and Train
each have 3 other attributes that are specific to those options. But, Car only has one other attribute that
applies to it (color of car). Consider what would happen if the respondent selected a red car in the BYO
question and we asked the software to vary from a minimum of 2 attributes to a maximum of 3 attributes
from the BYO selections. The software would need to change at least 2 of the attributes from the levels
selected in the BYO question. Since the respondent selected only "car" + "red" in the BYO question,
each of those levels would need to be changed to another level within the same attribute. Once "car"
was changed to some other level (Bus, Bicycle, or Train), then color would no longer apply. The result is
that no other car concepts could be shown in the remainder of the survey, even though the respondent
preferred a car in the BYO question!
Fortunately, the software will warn you if this problem is about to occur. You should, instead, specify
that a minimum of 1 attribute should be varied from the BYO selected concept, to ensure multiple cars
(each at different colors) can be shown in this respondent's ACBC survey.
Another Example:
For example, consider a design where Bus has 1 other attribute that applies to it, Car has 3 other
attributes that apply to it, and Train and 5 other attributes that apply to it. One important rule to
remember when dealing with this is to remember that even if you ask it to do it, ACBC will never vary
more than one-half +1 of the number of attributes involved in a BYO-selected concept. To review:
Bus has 1 other alternative-specific attribute

Car has 3 other alternative-specific attributes
Train has 5 other alternative-specific attributes
Appropriate settings for this design are as follows:
Minimum number of attributes to vary from BYO-selected concept = 1

Maximum number of attributes to vary from BYO-selected concept = 3
Given these settings, let's imagine what happens if a Train is selected in the BYO question. Up to 3 of
the attributes in that Train concept could be varied when generating near-neighbor concepts. But, if Bus
is selected in the BYO question, we want to have an opportunity to show other Bus concepts in the near-
neighbor set of concepts. That can only happen if the minimum number of attributes to vary is equal to
1.
Again, if you accidentally specified that the minimum number of attributes to vary from BYO selections
was 2 in this case, then a respondent who selected a Bus in the BYO question would not have an
opportunity to see another bus within the ACBC survey. The software would warn you if this was about
to happen, but you'd need to heed the warning and change the minimum number of attributes to vary
from 2 down to 1 to resolve the problem.
Display Issues and Alternative-Specific Designs (Merged Rows)
When an attribute doesn't apply to a product concept, it is shown as "blank". This may be just fine for
certain kinds of alternative-specific design studies, such as the one mentioned at the top of this entry
involving laptop and desktop computers. However, there are other instances in which too many blanks
would appear in the choice task, leading to poor readability of the product concepts. In those cases, the
ability to Merge Rows is provided.
Click here for more information.

546 SSI Web v8
3.4.6 Unverified Perl to Customize ACBC Questions

Sometimes, ACBC users would like to customize their interviews beyond what the standard interface
offers. Examples include:
Showing a different version of the attribute level text in the BYO question than in the Screeners
and Choice Tasks sections. For example, for binary on/off features, displaying text to indicate
"Not Included" in the BYO section, but leaving that level "blank" for the remaining questions. Or,
showing a slightly longer version of the attribute level text in the BYO question than in
subsequent sections.
Changing the Header 1 text for subsequent Screener questions, after the first Screener has
been seen.
Example 1: Changing the Attribute Level text for the BYO question
Put the following into an element of the attribute list for Attribute 3 Level 1 (where "widget" represents the
name of your ACBC exercise):
[%
if(QUESTIONNAME() eq "widgit_BYO")
{
return "Here's the level text to show in the BYO question for
Attribute 3 Level 1";
}
else
{
return "Text for Attribute 3 Level 1";
}
end unverified
%]
Example 2: Changing the Header 1 text for subsequent Screener questions, after the first
Screener has been seen
Place the following in the Header 1 section of the Screener question (where "widget" represents the
name of your ACBC exercise):
[%
if(QUESTIONNAME() eq "widgit_Screener1")
{
return "Here are a few widgets you might like.";
}
else
{
return "Here are a few more widgets you might like.";
}
end unverified
%]
3.5 Checklist Prior to Fielding

3.5.1 Checklist Prior to Fielding
1. Is this an appropriate study for ACBC? Appropriate studies typically involve about five or more
attributes. Studies with very few attributes (such as Brand-Package-Price studies) are often more
appropriately studied using standard CBC.
2. Have you used the Test Design capability on the Design tab to have the software generate dummy
(robotic) respondents that answer randomly? Have you examined the attribute level frequency report for
these dummy test records? Does each non-BYO level occur at a minimum of 2 times (3 times,
preferably)?
3. Does each respondent evaluate no more than about 7 levels per attribute? If more than about 7 levels
per attribute are being studied, then a constructed list should be used and respondents should evaluate
the most preferred/relevant levels.
4. If you are studying a large number of attributes or levels per attribute (such that using constructed
lists to discard levels and attributes from consideration within the ACBC survey is necessary), is your
sample size sufficient to stabilize the parameters across the full list of attributes and levels? Are you
willing to assume that discarded attributes are entirely unimportant to the respondent?
5. When you take the questionnaire, do the Unacceptables and Must-Have questions properly identify
(using correct "at least" or "at most" phrasing) the levels you have consistently selected or rejected?
(This confirms you set the correct worst to best or best to worst attribute direction).
6. Take a practice questionnaires yourself, making sure that the computed part-worths reflect the
preferences you expressed in the questionnaire.
7. Have you asked colleagues or a small convenience sample of target respondents to take the survey?
Have you debriefed them regarding their experiences and analyzed their data? Were any sections/
instructions confusing? Do the computed part-worth utilities reasonably reflect their individual
preferences?
8. Have you fielded the study (a "soft launch") among a few target respondents, examining the same
issues as directly above?
548 SSI Web v8
3.6 Analysis of ACBC Data

3.6.1 Counting Analysis for ACBC
Counting different outcomes in ACBC data can provide useful insights. ACBC software produces
multiple reports, each given on different tabs:
Summary Tab:
BYO: How often levels were included
Unacceptables: How often levels were unacceptable
Must-Haves: How often levels were must-haves
Screeners: How many products were screened into the consideration set
Choice Tournament: How often levels were included in the "winning" concept
Individual Results Tab:

BYO: Selections by respondent
Unacceptables: Selections by respondent
Must-Haves: Selections by respondent
Choice Tournament Winner: Selections by respondent
Level Frequencies Tab:

How often each level of each attribute occurred across the concepts viewed by the respondent,
including any replacement cards. Note that we only tabulate the levels across the unique concepts
generated by the near-neighbor design algorithm (or replaced due to Must Haves or Unacceptables).
If a concept is shown multiple times to a respondent (such as once in the Screeners section and
again in the Tournament section), we only tabulate that unique concept once.
Summed Price Frequencies Tab:

We tabulate how many times each unique price was shown across respondents. This report can be
helpful for determining how many product concepts were shown at each price point across
respondents for the purpose of selecting cut-points for Piecewise Price function specification.
To perform counting analysis, click Analysis + Calculate ACBC Counts...
A summary tabulation is shown on the screen. This counts report can be interesting information to
supplement the findings of the more traditional utility analysis and market simulations.
Also, a .csv file (with labels) named ACBCEXERCISENAME_Counts.csv is written to a sub folder in your
study folder that contains the counting information listed above for each respondent. Within that file:
For the BYO and Tournament Winner, a column is written to the .csv file for every level in the
study. A "1" indicates it was included in the concept; a "0" means it is not included.
For the number of Screener questions marked "a possibility" a single value is written to the file.
For the Must-Haves information, a column is written to the .csv file for every level in the study. A
"1" indicates it is a "Must-Have" level; a "0" means it is not a "Must-Have" level.
For the Unacceptables information, a column is written to the .csv file for every level in the
study. A "1" indicates it is an "Unacceptable" level; a "0" means it is not an "Unacceptable"
level.
How to Treat Missing Levels or Attributes from Constructed Lists
If you used constructed (dynamic) lists to customize the list of attributes and levels brought into the
ACBC questions for each respondent, you'll need to specify how to treat missing levels. We'll refer to as
levels/attributes as being partially missing or entirely missing attribute.
Partially Missing levels result when, say, only a subset of brands are brought forward to the ACBC
questions for a respondent.
An Entirely Missing Attribute results in all its levels not appearing in the ACBC questions for a
respondent (for example, if that attribute was previously determined as entirely unimportant).
Click the Settings... button to control how the missing levels should be treated in the report:
We have established defaults to specify how to treat levels that are either partially missing or from an
entirely missing attribute.
When tallying the BYO product specifications, we assume you will want any partially-missing
levels marked as 0 and any entirely missing attribute levels marked as missing.
When tallying the Unacceptables and Must Haves, we assume you will want any partially-missing
levels marked as unacceptable and any entirely missing attribute levels marked as missing (not
known).
When tallying the Tournament Winners, we assume you will want any partially-missing levels
marked as 0 and any entirely missing attribute levels marked as missing.
You can change these defaults if you wish.

550 SSI Web v8
How Unacceptables and Must-Haves Are Determined
Unacceptables:
1. VIA CONSTRUCTED LISTS: If the researcher has specified to treat missing attributes or levels as
unacceptable, those are marked as unacceptable for the respondent.
2. INFERRED FROM MUST-HAVE QUESTION: For Attributes with Sequence Order set to "None", if
the respondent indicates that one level is must-have, all other levels are marked unacceptable.
3. INFERRED FROM MUST-HAVE QUESTION: For Attributes with Sequence Order, the respondent
indicates that a certain level is a "must-have" threshold (e.g. must have "at least" or "at most" this
level). Any levels lower (if at least) or higher (if at most) than this level are marked unacceptable.
For example, suppose MPG (miles per gallon) is the attribute in question with levels 20MPG, 25MPG,
30MPG, and 35MPG. If the respondent marks that "at least" 30MPG is a must-have, 20MPG and
25MPG are marked unacceptable.
As a second example, if Seating Capacity is the attribute in question with levels 4-person, 5-person,
6-person, 7-person, and 8-person. If the respondent marks that "at most" 7-person seating is a must-
have, 8-person is marked unacceptable.
4. DIRECTLY FROM UNACCEPTABLE QUESTION: For Attributes with Preference Order set to
"None", if a respondent indicates a level is unacceptable, it is marked as unacceptable.
5. DIRECTLY FROM UNACCEPTABLE QUESTION: For Attributes with Preference Order, if the
respondent indicates that a certain level is unacceptable, that level and any levels less preferred than
it are marked unacceptable.
Exception in the case of constructed lists: If an attribute is marked as having preference order, if an
"unacceptable" level (omitted due to constructed list) is "more preferred" than the first "acceptable"
level, it is changed to "acceptable".
Must Haves:
1. If an attribute has only one level not marked as unacceptable, it is a "must-have."
2. If an attribute is marked as having preference order, the first "most preferred" acceptable level is
marked as "must-have."
3.6.2 Coding the Information for Parameter Estimation

The information in the core three sections of the ACBC questionnaires can be coded as a sequence of
choice tasks and may be estimated using maximum likelihood estimation under the MNL model. By
default, non-price attributes are coded as part-worth functions using effects coding (a type of dummy
coding procedure).
BYO and Most Likelies
The choice of level for each attribute may be coded as a choice task, where respondents chose 1 of K
levels (traded off with price if level prices are being used within the Summed price feature). Thus, if 10
total attributes (other than price) are included in the study, this section will contribute 10 tasks.
If level(s) are omitted from a respondent's design (using constructed lists), the appropriate coding
depends on whether the attribute involved has rational preference order. For omitted levels for attributes
without rational preference order (such as brand, color, etc.), we append additional information to the
design matrix in the form of binary tasks and an additional "threshold" parameter. The threshold "level"
is "compared" to each of the included levels (at their component prices, if applicable) and "rejected"; then
compared to all excluded levels (at their component prices, if applicable) and "chosen." For levels
missing from a respondent's design for attributes that have rational order, we do not add any new
information to the respondent's data. Rather, we suggest you apply monotonicity constraints for that
attribute across the sample.
Screener Section
Each respondent has marked T concepts as "possibility" or "not a possibility." We treat these each as
binary choices, where the respondent is assumed to compare the utility of the concept to the utility of a
constant threshold prior to making a choice. The constant threshold is included in the design matrix as
an additional dummy coded column.
Choice Tasks Tournament
These are coded as per standard CBC practice.

552 SSI Web v8
3.6.3 Modeling the Price Function

With ACBC, a popular option is to treat the price attribute as a "summed," continuous variable. With
summed price, the survey author assigns component-based (level-based) prices (and optionally, a base
price for the product). When we display the total price of the product (in the Screener, Choice Tasks,
and Calibration sections), we add the prices for its levels (plus any optional base price), and then we vary
the price randomly by a user-specified amount of random variation (e.g. -30% to +30%). This has the
benefit of displaying overall prices that are more in line with the general quality of the products (for
example, it avoids showing a poor product at relatively high prices). But, the drawback is that if price is
not varied randomly (and independently) enough, strong enough correlations between price and product
features can result and will lead to relatively imprecise estimates of both. In general, we recommend that
overall price be varied by at least -30% to +30% to reduce such multicollinearity and ensure reasonably
precise estimates. More guidance related to this can be found in the white paper: "Three Ways to Treat
Overall Price in Conjoint Analysis" available in our Technical Papers library at
www.sawtoothsoftware.com.
Because there are myriad combinations of attribute levels that can make up a product (where each level
can have assigned component prices) and prices are varied randomly within a specified range of
variation, thousands of unique "displayed" prices can be found across a sample of respondents. To
model the effect of price on product choice, we have chosen three effective coding methods: Linear, Log-
Linear, and Piecewise.
Linear Function: The total price shown to respondents is coded as a single column in the design
matrix. This assumes that the impact of price on utility is a linear effect. (However, note that when
projecting respondents' utilities to choices in the simulation module, one typically exponentiates
the total utility for products, so the final impact of price on choice probabilities is modestly non-
linear.) The benefit of linear coding is that only one parameter is estimated, conserving degrees of
freedom. But, if the actual impact of price on utility is non-linear, the linear assumption for this
model is simplistic and suboptimal.
(Advanced Note: We temporarily transform whatever price range you provide to the range -1 to +1
for coding in the design matrix so that HB converges quickly. Then, we re-expand the parameters
back to the original scale when reporting the final utilities.)
Log-Linear Function: The total price shown to respondents is transformed by the natural log and
coded as a single column in the design matrix. This assumes that the impact of price on utility is
log-linear, or somewhat curved, as shown below:
As with the Linear function, only one parameter is estimated, which conserves degrees of freedom.
But, if the actual impact of price on utility doesn't conform well to the smooth log-linear curve, this
model is suboptimal. Because of the method we use to estimate discrete utilities for price points
for communication with the SMRT market simulator, you must specify a number of interior price
points in addition to the endpoints for the curved function to be reflected within market simulations.
Piecewise Function: This is a more advanced and flexible function--and the approach we
recommend for most datasets. The user provides specific breakpoints and a different slope (linear
function) is estimated to characterize the relationship of price to utility between each breakpoint.
For example, consider a conjoint study where the total price (after adding the price components
and randomly varying the prices by, say, -30% to +30%) results in a range of prices actually shown
to respondents from $725 to $5,250. The user supplies "breakpoints" to use in developing the
piecewise price function. Let's assume the user hypothesizes that the price function has elbows at
$1,500 and $3,000, but is linear between adjacent breakpoints and end points. In other words, the
user might hypothesize a relationship between price and utility as is depicted in the chart below:
554 SSI Web v8
Rather than estimate a single slope (parameter) that characterizes a smooth relationship from $725
to $5,250, we are in this case fitting three slopes (ß 1, ß 2 and ß 3). The particular function above
suggests that this respondent (remember that these betas are computed at the individual level)
reacts more strongly to price changes between $725 and $1,500, and then less strongly to prices
between $1,500 and $3,000. Then, the reaction is stronger again for prices higher than $3,000.
One could specify as many breakpoints as desired along the price continuum, but each additional
breakpoint leads to another parameter to be estimated, and overfitting (including reversals) will
eventually result. A key decision is where to place the breakpoints. It would be a mistake to
place breakpoints such that very few products seen by respondents actually fell within a
range bounded by two breakpoints. In the example above, if very few products were evaluated
that fell in the range $750 to $1,500, the model would have very little information by which to obtain
a stable estimate of ß 1. We recommend you run Analysis + Calculate ACBC Counts and look at
the results on the Summed Price Frequencies tab of the resulting report. This allows you to view
the distribution of prices of the product concepts shown to respondents to help you determine
appropriate breakpoints.
The benefit of piecewise functions is that they will provide much better fit to the data than a linear
function if the price function is truly non-linear. The drawbacks are that they lead to more
parameters to estimate, and importantly that the analyst must determine the appropriate number
and position of the breakpoints. We also restrict you from estimating interaction terms involving
piecewise Price, as such designs are often deficient. Why is this so? Consider an interaction
between brand and price, where some brands never occur in the high tier range. We'll be
attempting to estimate betas for interaction effects where there is no information in the design
matrix.
If the piecewise function is used, we simply carry the utility forward to the market simulator just for
the prices corresponding to the end points and breakpoints for each respondent. When simulating
prices between adjacent breakpoints, the market simulator applies linear interpolation. Thus, using
the piecewise model results in utilities associated with discrete price levels (determined by the
analysis) for use in the market simulator--the same process as is used in Sawtooth Software's
ACA, CBC, and CVA systems. However, the choice of specific price levels is done post hoc rather
than prior to fielding the study.
(Advanced Note: We temporarily transform whatever price range you provide to the range -x to +x,
where x is the number of price parameters to estimate in the model so that HB converges more
quickly. Then, we re-expand the parameters back to the original scale when reporting the final
utilities.)
Recommendations:
When using piecewise functions for summed price, it may take significantly longer for HB to converge.
You may wish to significantly increase the number of initial iterations (prior to assuming convergence).
In general, we recommend the piecewise function, using between 2 to 6 breakpoints (aside from the
endpoints). Recent research at the 2013 Sawtooth Software Conference presented by the SKIM Group
suggests that even more breakpoints (potentially a dozen or more) is potentially useful for ACBC studies,
as long as you constrain price to be negative. This is based on only one study, so we would want to see
additional evidence before recommending this generally.
We strongly recommend constraining price functions (typically negative slope).
Even though we restrict you from estimating interactions involving piecewise price, note that the very
nature of creating a piecewise coding should inherently capture some interaction effects with price. For
example, if premium brands are associated with a relatively large level-based price, then the beta
associated with the upper price range will to a great extent be estimated specific to the occurrence of
premium brands.
556 SSI Web v8
3.6.4 HB Estimation
In a previous section entitled Coding the Information for Parameter Estimation we described the strategy
for coding respondent answers into choices from sets of available options. We have employed HB
effectively to estimate part-worth utilities for ACBC experiments. Details regarding our implementation of
HB may be found in the CBC/HB product manual and in the CBC/HB Technical Paper.
Output Files
We display a summary report on the screen after estimation is completed, where the part-worth utilities
are summarized as normalized zero-centered diffs. Importance scores are also displayed, along with the
standard deviations of these measures.
The diffs method rescales utilities so that the total sum of the utility differences between the worst and
best levels of each attribute across attributes (main effects) is equal to the number of attributes times
100. It removes the often strong differences in the magnitude of "scale" that make it hard to compare
raw utilities across respondents or between groups of respondents.
The utility run report displayed to the screen has multiple tabs. The first tab includes the summary
report as just described. The second tab contains the raw part-worth utilities per respondent (same
information as saved to the .csv and .hbu files, described below). The third tab includes the normalized
(zero-centered diffs) part-worth utilities, appropriate for subsequent segmentation/cluster analysis.
Various output files are written to the study directory:
Most Commonly Used Files:
.csv
Comma-separated values file containing the part-worth utilities (point estimates, one row per
respondent). May be opened directly with Excel, and contains a header row containing labels.
These are raw utilities, appropriate for market simulations. If using utilities for segmentation
analysis (such as cluster analysis), one should use normalized utilities (e.g. zero-centered diffs)
rather than raw utilities.
.hbu
Contains the respondent part-worth utilities (same as written to the .csv file). This file is formatted
as text-only, and may be imported into SMRT for market simulations (note: only the utility run from
an ACBC study is imported into SMRT. ACBC's .CHO file is incompatible with SMRT and should
not be imported into SMRT). See also .hbu file format.
.sum
Contains final estimates of the population means and covariances.
Files Containing Diagnostics to Assess Convergence
.log
Contains a history of RLH, Avg. Variance, and Parameter RMS, which is useful information to
assess convergence.
.alp
Contains a history of the estimated population mean for part-worths. One row is written per draw
saved (file is only written if Save Draws is enabled).
.gamma
If using Otter's Method for estimating task-specific scale factors, this file contains the estimated
scale factors for the three sections in ACBC used for part-worth estimation (BYO, Screeners,
Choice Tasks). One row is written per draw saved (file is only written if Save Draws is enabled).
Files Containing Advanced Information
.cov
Contains the estimated variance-covariance matrix for the distribution of part-worths across
respondents. Only the elements on-or-above the diagonal are saved. One row is written per draw
saved (file is only written if Save Draws is enabled).
.dra
Contains estimated part-worths for each individual saved from each nth iteration, where n is the
Skip Factor for Using Random Draws. The file is formatted like the .hbu file, without the header.
.std
Contains the within-respondent standard deviations among random draws. Only created if you
elect not to save random draws.
Otter's Method
One consideration when estimating part-worth utilities from ACBC is the large difference in scale
(response error) for the three main sections of the questionnaire. If using generic HB estimation, a key
assumption is that the three types of tasks can be combined within the same dataset, even though we
have observed that each section has different scale factor. Even though the BYO section has larger
scale than the other two sections, we ignore this fact when using generic HB to estimate the parameters
across the tasks.
We have wondered about the practical effect of the differences in scale for the three sections when using
a generic HB model. We have been fortunate to benefit from the expertise of Dr. Thomas Otter, a
recognized expert regarding HB application to choice experiments. Otter has built more sophisticated
HB models that separately model scale for the three sections in ACBC questionnaires. He has found
that the generic HB model performs very nearly as well as the more sophisticated models. You can turn
on "Otter's Method" for accounting for differences in scale during HB estimation through the HB interface
provided in ACBC.
When Otter's method is selected, an additional diagnostic file is written to disk when Save Random
Draws is enabled that reports the relative gammas (scale factors). By default, the scale factor for the
Screener section is constrained via very strong priors to remain around 1.0. Gammas for BYO,
Screeners, and the Choice section are written to the .gamma file (in that order). A gamma of less than
one indicates that this section of the survey has smaller scale than the Screener section. The final utility
estimates are placed on the scale commensurate with the Screener section.
Note: Otter's Method requires substantial sample size to stabilize the estimates of scale for the
different sections of the survey. We recommend it be used when you have about 300 respondents
or more. Also, we recommend a greater number of used iterations than default, as the parameters
seem to "wander" in larger amplitude cycles (than standard HB) due to the additional estimation
of the scale parameters.
558 SSI Web v8
For more information, please see:
Otter, Thomas (2007), "HB Analysis for Multi-Format Adaptive CBC," Sawtooth Software Conference
Proceedings, Sequim, WA.
3.6.5 Interaction Effects

An interaction effect is when two or more levels combining have a different utility for people than the
simple sum of their separate parts. For example, most people like cheese; most people like chocolate--
but very few people like chocolate-covered cheese!
By default, interaction effects are not estimated in ACBC's HB routines. But, you can include selected
first-order interactions (interactions between two attributes) if you wish. You may also test the precision
of interaction effects prior to data collection using the Test Design capability from the Design tab. You
specify how many dummy respondents to generate and these robotic respondents answer the
questionnaire randomly.
After collecting the data, you can ask ACBC to automatically test all potential first-order interaction
effects (those considering attributes taken two-at-a-time) using the Analysis + Interaction Search Tool
dialog.
Most conjoint models assume a simple additive model. This implies that the effect (utility) of each
attribute can be estimated and considered independent of all other attributes. Indeed, main effects tend
to account for the majority of the explainable variance in the data. However, there are instances in which
significant interaction effects exist and market simulators can be improved if those effects are included in
the model.
Including interaction effects in CBC or ACBC models is especially critical if pooled (aggregate) analysis
is employed. However, it has been demonstrated that the need to model interaction effects is
significantly reduced when using individual-level estimation (as is the norm for ACBC). But, that doesn't
mean that one can simply ignore interaction effects if using HB. We recommend you be on the lookout
for interaction effects and that you investigate including them in ACBC models. ACBC's designs appear
to be as effective as standard CBC designs with respect to accommodating interaction effects.
Standard CBC gives two useful tools for identifying interaction effects that may provide improved fit to the
model. CBC's Counts program reports a interaction Chi-Square test for all attributes taken two-at-a-time.
This can help you identify which effects (at least when considered in a pooled model) may be significant
and useful. In the CBC documentation, we have encouraged users also to use aggregate logit and the "2
Log-Likelihood" test to test the strength and significance of first-order interactions (and the Analysis +
Interaction Search Tool available within our CBC and ACBC software now automates that).
With Adaptive CBC designs, standard counting analysis is not appropriate (since the designs are not
level balanced). But, one can perform a series of pooled logit models to investigate potential interaction
effects using the "2 Log-Likelihood" test (the Analysis + Interaction Search Tool available within CBC
and ACBC software now automates that). It is interesting to note, however, that a strong lift in likelihood
due to the inclusion of an interaction effect for a pooled model may not necessarily translate into a strong
lift for an individual-level model.
Many of the interactions detected with aggregate modeling in CBC over the years have been due to
respondent heterogeneity. For example, if the same people who prefer convertibles also prefer the color
red for automobiles, then the "convertible + red" interaction effect may be mostly accounted for simply by
estimating individual-level models (with main effects). But, interaction effects may more pervasive within
each individual's psyche, and in those cases interaction effects can improve individual-level model
performance.
Useful interactions can be confirmed by examining the model fit to holdouts, or by seeing if there
appears to be a positive lift in RLH as reported by HB.
560 SSI Web v8
Unless there is compelling evidence for including an interaction in individual-level ACBC models, we
recommend omitting it to avoid overfitting.
3.6.6 Interaction Search Tool

(This functionality is available for both CBC and ACBC analysis.)
Introduction to Interaction Effects

Most conjoint analysis studies over the last 30 years have assumed a simple additive model, where the
total utility of a product concept was simply equal to the sum of its separate part-worth utilities. This
approach has tended to work quite well in practice, especially for conjoint studies modeled using
individual-level estimation. However, there are many cases when two or more levels (from different
attributes) combining have a different utility for people than the sum of their separate parts. For example,
most people like cheese; most people like chocolate--but very few people like chocolate-covered cheese!
HB modeling to estimate individual-level part-worths for CBC and ACBC has improved the predictive
ability of market simulators. Many of the interaction effects that appeared significant under aggregate
(pooled) logit estimation were largely due to unrecognized heterogeneity. As an illustration, an
interaction effect between “premium brand” and “price” may owe to the fact that the same respondents
who strongly prefer premium brands are also less price sensitive. Once the main effects for brands and
prices are captured within individual (e.g. via HB estimation), the interaction effect that appeared so
significant under pooled estimation can nearly disappear once we’ve captured heterogeneity. This benefit
of individual-level modeling has been a good thing, but it has also bred a lot of complacency among
Sawtooth Software users employing HB, many of whom no longer tak e the time to investigate interaction
effects.
CBC provides a counting report that quickly summarizes (via the Chi-Square statistic) which interaction
effects are significant. ACBC, however, uses an adaptive design strategy that doesn't make it possible
to use counting analysis and Chi-Square statistics to test for interactions. Although the adaptive ACBC
designs do a fine job of supporting the precise estimation of interaction effects, most ACBC users don't
take the time to investigate their inclusion in utility estimation models. This may be due to a lack of
knowledge, but it is most likely due to a lack of time. We hope to solve both of these problems by
including this automated Interaction Search Tool within the SSI Web interface.
The automated Interaction Search tool makes it very easy to test all potential 2-way interaction effects in
your CBC or ACBC data. It does this using the 2-log likelihood (2LL) test, which is a more sensitive
statistical test than the Chi-Square statistic from counting analysis.
We should note that the 2LL tests computed using pooled logit do not involve the same model that most
users end up estimating (HB). However, the findings of the 2LL tests from aggregate logit should be a
useful guide for considering which interaction effects may perform well within HB estimation.
Warning: including too many interaction effects can lead to overfitting, convergence problems,
and extremely long run times. Be prudent and highly selective regarding the inclusion of
interaction terms within your CBC and ACBC models.
The 2-Log Likelihood Test for Interaction Effects

The null hypothesis is that there is no interaction effect. We test each interaction effect, one at a time,
to see if there is enough evidence to reject the null hypothesis with a given degree of confidence.
562 SSI Web v8
The test involves these steps (the interaction search tool automates these steps):
1. Using aggregate (pooled) logit, estimate the model using main effects only. Record the log-
likelihood for this model.
2. Using aggregate logit, estimate the model using main effects plus a 2-way interaction effect.
Record the log-likelihood for this second, larger model.
3. Compute the difference in log-likelihood between the main effects only model and the main effects
model that includes a selected interaction effect. That value times 2 is distributed Chi-Square, with
degrees of freedom equal to the difference in the number of parameters in the two models. Use the
Chi-Square distribution to compute the p-value, which is the likelihood that we would observe a
difference this large in fit just by chance. If the p-value is <0.05, then we are at least 95% confident
that the interaction effect is significant.
Repeat steps 2 and 3 for all potential 2-way interaction effects. For an 8-attribute study, there are (8)
(7)/2 = 28 unique interaction effects to test. Then, sort the results in terms of lowest p-value to highest
p-value.
Please note that since we are doing a series of independent t-tests, 5% of them will pass the threshold
of 95% confidence just due to chance. To reduce the lik elihood of false positives, you should probably
consider a much higher threshold than 95% confidence (p<.05).
Modified 2-Log Likelihood Test Using Individual-Level Utilities File

Many readers will note that the aggregate logit 2LL tests don't reflect the eventual analysis (HB) they will
be running. The significant interaction effects discovered using the aggregate logit 2LL tests may be due
to unrecognized heterogeneity. Indeed, it would be best to run separate HB models to account for
heterogeneity while testing each interaction effect, but doing so would probably take too long for most
practitioners and studies in practice.
A second available test within this automated Interaction Search Tool accounts for heterogeneity within
the main effects, but estimates the strength of interaction effects using pooled aggregate logit. If you
already have a file containing individual-level main effects (such as from HB estimation), you can specify
that the test should leverage this file. (You'll typically find this file in a folder within your SSI Web project
folder called myexercise_hb, where myexercise is the name of your CBC or ACBC exercise). If you
supply this additional file, then the series of 2LL tests runs even faster than basic method described
above (though the small time savings is not the real reason to use this option). We'll now describe the
differences between the two approaches.
In the basic 2LL test described in the previous section, we estimate main effects by first coding the
attribute levels in a design matrix using effects-coding. For example, with 6 attributes each with 3 levels,
there are (6)(3-1)=12 parameters to be effects-coded in the design matrix and estimated via aggregate
logit. Aggregate logit finds the single set of population utility values that best fit the individual choice
tasks across the population. Because respondents often differ significantly in their preferences, the fit
that a single set of average utilities can provide to each respondent's choices is typically somewhat poor.
After estimating main effects in this way, a second model is estimated that uses not only the 12
columns in the design matrix representing the main effects, but additional effects-coded columns for the
interaction effects.
In the modified 2LL test leveraging individual-level utilities, rather than coding the main effects as an
effects-coded design matrix, we code the design matrix with a single column containing the total
predicted main-effect utility for each product concept as computed from the individual-level utility file. For
instance, consider the example we previously described with 6 attributes each with 3 levels. The
standard 2LL test involves encoding each product concept in the design matrix as a series of 12 effects-
coded columns (containing +1s, 0s, and -1s). However, the modified test encodes each product concept
in the design with a single value: the predicted utility of that concept for this given respondent, based on
the supplemental individual-level utility file (typically coming out of HB estimation). Thus, if we looked at
the design matrix we might find an X value associated with a product concept such as 3.641, where
3.641 is the total utility across the six attributes for the product shown to this given respondent within
this given task. If that same product concept was shown to a different respondent, the X value in the
design matrix for that respondent would almost certainly be something different, such as 2.713.
In the modified 2LL test, the additional columns in the design matrix accounting for the interaction effects
are effects-coded as with the standard test.
Aggregate logit is used as before to estimate the log-likelihood fit of the model to the choices. You will
note that the log-likelihood fit is much better for these models (closer to zero), since they leverage the
heterogeneity as reflected in the supplemental HB utilities file. In essence, we are explaining much more
of the variance using the individual-level main-effect utilities, then investigating (via pooled analysis) how
much additional fit is gained by adding certain interaction effects.
Theoretically, this modified test should do a better job (than the standard LL test) of prioritizing which
interaction effects would be the most useful within HB estimation.
Settings
The Interaction Search Tool provides some additional settings for you to choose.
Confidence Interval This controls the threshold for bolding the significant results and writing out
data for charting, within the Charts tab. Interaction effects that meet this threshold of significance
are bolded and written to the Charts tab.
Individual-level utility file This lets you specify a file containing main-effect utility estimates for
the respondents for running the modified 2LL test that accounts for heterogeneity in the main effects.
Attribute Coding Allows you to set the coding (part-worth, linear, log-linear, or piecewise) used for
the different attributes. (Note: we don't permit interaction effects between attributes and piecewise
coded price.)
Task Filter Allows you to select which tasks to include in the analysis.
Output
Click Run Interaction Search Tool... to perform the analysis. The software summarizes (on the Search
Summary tab) the largest interaction effects from smallest p-value (least likely to occur by chance) to
564 SSI Web v8
largest p-value. The 2LL test is a very sensitive test and it is our experience that you will often find
multiple interaction effects with p values less than 0.001. We recommend you also look at the difference
in the Percent Certainty (directly related to the RLH, pseudo R-squared) values between models. Due to
major differences between aggregate logit and HB models, there are no well-defined statistics for how
much improvement in RLH from an aggregate logit model constitutes a clear opportunity to improve your
HB models. In our experience so far, an interaction effect that can increase the Percent Certainty for the
aggregate models by 1% or more would seem like a good candidate to potentially improve an HB model.
If you have holdout choice tasks in your study, you should also examine whether the addition of
interaction terms improves the holdout predictability.
Interactions that lead to ill-conditioned models (deficient designs) are listed with dashes (---) in the
output (and are sorted to the bottom of the report). Such ill-conditioned models result when the
attributes involved in the interaction also include prohibitions between them.
The net effect of the main effects plus interactions is summarized for you within tables (on the Charts
tab) that can be cut-and-pasted into Excel for further investigation. Plotting these results using line
graphs can help you visualize the interaction effect in question. Only the interactions that exceed the
chosen significance threshold are written to the Charts tab.
You can review the detailed results of any of the logit runs by selecting the run from the Runs tab.
Notes:
ACBC doesn't support interactions involving piecewise price functions, so such interactions are sk ipped
by the Interaction Search Tool.
Interactions between attributes and the None thresholds within ACBC are not estimable. For expediency
in programming, the interaction search tool attempts to estimate these interactions, but break s out when
it finds they are inestimable. These are sorted to the bottom of the list as least valuable and you should
ignore them.
3.6.7 Utility Constraints, Including Customized Constraints

The HB and Monotone Regression estimation routines included in ACBC software allow you to impose
monotonicity constraints (utility constraints). For example, if you know that the relationship between
price and preference should be negative for all respondents, you can constrain all respondents'
partworths to reflect a negative slope for price.
Specify utility constraints by clicking the Use Constraints box from the Attribute Information tab (within
the HB or Monotone estimation dialogs). Then, click the Edit... button that is enabled. A drop-down
control appears that lets you specify constraints for attributes in your study.
If you use the Summed Price option in ACBC, we generally recommend constraining its slope to be
negative.
Constraints have the benefit of eliminating "reversals" in partworths (out-of-order relationships among
levels with known rational preference order). Constrained utilities generally will show improved individual-
level hit rates of holdout tasks. But, constraints introduce some bias in the parameters and sometimes
result in models that have slightly lower aggregate predictive accuracy (such as for shares of preference).
As another potential problem, if constraints are applied to some attributes, but not to others, this can
change the sensitivity of the simulation models in favor of the constrained attributes.
If the main goal of the research is individual-level predictive accuracy, we recommend applying
constraints for all attributes: global constraints for attributes with known a priori order, and customized
constraints for other attributes based on responses to additional questions in the survey. If the main goal
of the research is share prediction accuracy for segments or for the population, we generally recommend
avoiding utility constraints (except for Summed Price, which generally should be constrained). However,
if using ACBC with very small sample sizes (especially n<30), constraints across all attributes may be a
good idea both for individual-level prediction and aggregate share prediction accuracy.
Global (Universal) Constraints
Global constraints may be applied when one can safely assume that all respondents agree regarding the
relative preference order of levels within an attribute. When imposing constraints in ACBC, a grid
appears for each attribute, such as:
Speed (Pages Per Minute):

Relative
Level Preference
5 PPM
7 PPM
10 PPM
12 PPM
To impose global utility constraints, you can specify relative preference values in the right-hand column of
the grid. Larger values indicate higher preference. For example:
Speed:
Relative
Level Preference
5 PPM 1
7 PPM 2
10 PPM 3
566 SSI Web v8
12 PPM 4
We could easily have used values of [10, 20, 30, 40] or [1, 3, 10, 1000] instead of [1, 2, 3, 4], and the
results would be the same. The values found in the grid for Relative Preference are only referenced to
constrain the relative preference order of the final utilities. The utility for 12 PPM will be constrained to
be higher than the utility for 10 PPM (because a "4" is greater than a "3"), etc.
When fields within the Relative Preference column are all either missing (blank) or equal, the attribute's
utilities are not constrained.
Partially-Constrained Attributes
If only some levels of an attribute should be constrained, then the value for the non-constrained levels
should be left "blank". For example:
Style:
Relative
Level Preference
Style A--version 1 2
Style B
Style C
In the example above, level 1 is constrained to be preferred to level 2. But, levels 3 and 4 are
unconstrained.
Multiple Independent Chains of Constraints within Attributes
Consider a situation where Style A--Version 1 is preferred to StyleA--version 2 AND Style C is preferred
to Style B. But, we don't know how the two pairs of levels interrelate. In that case, you can add another
column to the grid:
Style:
Relative Relative
Level Preference Preference
Style B 1
Style C 2
In the example above, level 1 is constrained to be preferred to level 2. Levels 4 is preferred to level 3.
But, the relationship of levels 1 & 2 relative to 3 & 4 is not constrained.
Customized Constraints
ACBC can apply customized (idiosyncratic) constraints in addition to global constraints. Attributes like
speed and price may have rational preference order across all respondents (global). But, attributes such
as brand, color, and style do not have the same preference order for every respondent. ACBC allows you
to specify customized constraints for such attributes, using information collected in another part of the
survey. Customized constraints only involve within-attribute utility constraints, not between-attribute
(importance) constraints, such as have been implemented in Sawtooth Software's ACA system.
For example, you might ask respondents to rate different brands (in a question named RateBrand) on a
scale from 1 to 5. The answers to this question can be used as customized ordinal constraints (you
must ensure that higher values correspond with higher preference for any questions you reference as
constraints). For example:
Brand:
Relative
Level Preference
Brand A RateBrand_1
Brand B RateBrand_2
Brand C RateBrand_3
Brand D RateBrand_4
The values stored within the RateBrand question are used to determine the applicable constraints for
each respondent. For example, if a respondent rates the brands as follows:
Brand Rating
Brand A 5
Brand B 3
Brand C 3
Brand D 2
Then, utilities for this respondent are constrained according to the following ordinal relationships:
Brand A > Brand B

Brand A > Brand C
Brand A > Brand D
Brand B > Brand D
Brand C > Brand D
Notice that brands B and C are not constrained with respect to one another (neither are they tied); their
utility values with respect to one another are left to be determined by the data.
Partially-constrained attributes (as with global constraints) can be specified by using a missing/blank
value for a non-constrained level. A blank relative preference indicates that this level should not be
constrained.
Note: If using summed pricing and associating price premiums with levels, any ratings applied as
constraints should refer to the relative utility of the level independent of any level-based pricing.
Respondents should be ask ed to rate the levels assuming all else equal, including price.
568 SSI Web v8
3.6.8 Monotone Regression (Pure Individual Analysis)

Recommendations:
Monotone Regression can estimate partworth utilities uniquely for each respondent, without the benefit of
borrowing information across a sample of respondents. With extremely tiny samples of a few
respondents (such as fewer than 5), monotone may be preferred to HB. In our experience, even with as
few as 10 or 20 respondents, HB may be preferred. With hundreds of respondents, we've seen
consistent evidence that HB works better than monotone regression.
If using monotone regression, we recommend that your ACBC questionnaires be of substantial length,
such that all non-BYO levels occur in each respondent's design at least 4x. If using monotone
regression, we expect you are working with tiny sample sizes, and we encourage you to impose utility
constraints on all attributes. You can specify global constraints for attributes with known logical order,
especially price, and idiosyncratic constraints for attributes like brand, style, and color.
As a word of caution, partworth utilities are given arbitrary scale under monotone regression,
unlike HB where the scaling of the partworths directly maps to probabilities of choice via the logit
rule. Thus, you should adjust the Exponent (Scale Factor) appropriately in market simulators that
employ partworths developed with monotone regression. Otherwise, the overall sensitivity of the
model may be significantly lower or higher than appropriate for predicting buyer choices.
Methodology:
Monotone regression uses a method similar to that described by Richard M. Johnson in "A Simple
Method of Pairwise Monotone Regression", Psychometrika, 1975, pp 163-168. Monotone regression
may be appropriate for estimation problems where the model is intended to fit the data only in a rank
order sense. An example of such a situation would be full profile conjoint analysis, with data consisting
only of rank orders of preference for a collection of product concepts. More recently, monotone
regression has been adapted to choice modeling, where the data consist of only ones and zeros, with
ones indicating chosen concepts and zeros indicating those not chosen. In this context, monotone
regression seeks a set of partworths which imply higher utility for each chosen concept than for non-
chosen concepts in the same choice set.
The method is iterative, starting with random values and finding successive approximations which fit the
data increasingly well. Two measures of goodness of fit are reported: Theta and Tau.
Tau
Suppose a choice task presented four concepts, P, Q, R, and S, and the respondent chose concept P.
Also, suppose that at some intermediate stage in the computation, utilities for these concepts are
estimated as follows:
Concept Utility Chosen

P 4.5 1
Q 5.6 0
R 1.2 0
S -2.3 0
We want to measure "how close" the utilities are to the rank orders of preference. One way we could
measure would be to consider all of the possible pairs of concepts, and to ask for each pair whether the
member with the more favorable rank also has the higher utility. But since our observed data consist
only of a one and three zeros, we can focus on the differences in utility between the chosen concept and
the three non-chosen concepts.
In this example the preferred concept (P) has a higher utility than either R or S, but it does not have a
higher utility than Q.
Kendall's Tau is a way of expressing the amount of agreement between the preferences and the
estimated utilities. It is obtained by subtracting the number of "wrong" pairs (1) from the number of "right"
pairs (2), and then dividing this difference by the total number of pairs. In this case,
Tau = (2 - 1) /3 = .333
A Tau value of 1.000 would indicate perfect agreement in a rank order sense. A Tau of 0 would indicate
complete lack of correspondence, and a Tau of -1.000 would indicate a perfect reverse relationship.
(Although this example involves only one choice set, in a real application the number of correct and
incorrect pairs would be summed over all choice sets in the questionnaire.)
Tau is a convenient way to express the amount of agreement between rank orders of two sets of
numbers, such as choices and utilities for concepts. However, it is not useful as a measure on which to
base an optimization algorithm, because of its discontinuous nature. As a solution is modified to fit
increasingly well, the Tau value will remain constant and then suddenly jump to a higher value. Some
other measure is required which, while similar to Tau, is a continuous function of the utility values.
Theta
For this purpose Johnson described a statistic "Theta." In his original article, Johnson measured
differences in utility by squaring each difference. For this example we have:
Preference Utility Squared

Difference Difference
P minus Q -1.1 1.21
P minus R 3.3 10.89
P minus S 6.8 46.24
----
Total 58.34
Theta is obtained from the squared utility differences in the last column of the table above. We sum the
squares of those utility differences that are in the "wrong order," divide by the total of all the squared
utility differences, and then take the square root of the quotient. Since there is only one difference in the
wrong direction,
Theta = square root(1.21/ 58.34) = .144
Theta can be regarded as the percentage of information in the utility differences that is incorrect, given
the data. The best possible value of Theta is zero, and the worst possible value is 1.000.
Now that we have defined Theta, we can describe the nature of the computation.
The process is iterative. It starts with random values as estimates of the partworths. In each iteration a
direction of change (a gradient vector) is found, for which a small change in partworths will produce the
greatest improvement in Theta. Small changes are made in that direction, which continue as long as
Theta improves. Each iteration has these steps:
1. Obtain the value of Theta for the current estimates of partworths and a direction
570 SSI Web v8
(gradient) in which the solution should be modified to decrease Theta most rapidly.
2. Try a small change of the partworths in the indicated direction, which is done by
subtracting the gradient vector from the partworth vector and renormalizing the
partworth estimates so as to have a sum of zero within each attribute and a total
sum of squares equal to unity. Each successive estimate of partworths is
constrained as indicated by the a priori settings or additional utility constraints.
3. Re-evaluate Theta. If Theta is smaller than before, the step was successful, so
we accept the improved estimates and try to obtain further improvement using the
same procedure again, by returning to step (2). If Theta is larger than before, we
have gone too far, so we revert to the previous estimate of partworths and begin a
new iteration by returning to step (1).
If any iteration fails to improve Theta from the previous iteration, or if Theta becomes as small as 1e-10,
the algorithm terminates. In theory, the iterations could continue almost indefinitely with a long series of
very small improvements in Theta. For this reason it is useful to place a limit on the number of iterations.
A maximum of 50 iterations are permitted, and within any iteration a maximum of 50 attempts at
improvement are permitted.
To avoid the possibility of stumbling into a bad solution due to a poor starting point, the process is
repeated 5 separate times from different starting points. For each respondent, the weighted average of
the five resulting vectors of partworths is computed (weighted by Tau, where any negative Tau is set to an
arbitrarily small positive number). A weighted Tau is also reported with this final estimate of partworths.
Although Theta was originally defined as a ratio of sums of squared differences, more recently we have
experimented with a modified measure which uses absolute values of differences. The current software
permits the user to choose which of the two measures to use.
We would really like to find a way to optimize Tau, which is a "pure" measure of fit to observations which
consist solely of zeros and ones. However, because Tau does not vary continuously with small changes
in partworths, it was necessary to find some other statistic which was "like" Tau but which varied
continuously with small changes in the partworths, and that is the role played by Theta.
The rationale for the quadratic definition of Theta is one of historical precedent. The principle of least
squares has been used widely and for a long time in statistics. However, when using squared
differences, the value of Theta is affected much more by larger differences than by smaller ones, so the
algorithm may tolerate many small errors to reduce a few large errors.
The rationale for the linear definition of Theta is that when based on absolute values of errors, Theta is
more similar to Tau, in the sense that the algorithm will tolerate a few large errors in order to avoid many
smaller errors.
Our experience with the linear definition of Theta has been limited to a few recent data sets. However,
the linear version seems to result not only in better values of Tau (as one would predict) but also in
slightly better hit rates for holdout choice sets, and better (more consistently negative) estimates of
partworth coefficients for price. We leave it to our users to experiment with the two versions of Theta,
and we will be interested to learn whether the linear version continues to outperform the quadratic
version.
Limitations
No interaction effects: With purely individual-level analysis, there generally isn't enough information to
estimate interaction effects.
No piecewise regression: This restriction avoids the common occurrence of having little (or no)
information to estimate betas for regions of the price demand curve that are less relevant to the
respondent. For example, a respondent who chooses a BYO product in the very expensive range may
see very few or no products in the very inexpensive region for which we are attempting to estimate a beta
coefficient.
Note: The restriction of using only linear/log-linear price functions might seem limiting. We can imagine
advanced researchers building their own mark et simulators that include individual-level threshold price
cutoffs. For example, one could use the linear price coefficient estimated via monotone regression, but
provide a strong utility penalty should the price exceed the respondent's expressed "unacceptable" price
threshold (this information can be exported per respondent using the Counts function).
572 SSI Web v8
3.6.9 Analysis with Tiny Sample Sizes

Generally, sample sizes in the multiple hundreds are used for conjoint-analysis surveys. But, there are
instances in which the market consists of just a few customers, and the researcher wants to gain
insights regarding these few people. We have heard of entire conjoint studies designed for the purposes
of understanding the tradeoffs of a single customer.
One of the advantages of ACBC is the ability to stabilize part-worth utility estimates with smaller sample
sizes than typically are used with traditional CBC. With ACBC, more information is collected at the
individual-level, and respondents seem willing to spend even greater time in the adaptive CBC survey than
the non-adaptive CBC. Respondents have reported equal or greater engagement/interest/satisfaction
after taking ACBC interviews than with much shorter CBC interviews.
Our research suggests that purely individual-level analysis (via monotone regression) is feasible for
ACBC, though when hundreds of respondents are available in the dataset, HB estimation consistently
provides more accurate results. (Purely individual-level analysis means that no information is "borrowed"
from other respondents in the sample, so results could be computed even for n=1.) Despite the general
superiority of HB analysis over monotone regression with ACBC data, we've wondered at what sample
size one might start to favor monotone regression over HB. Is a sample size of n=30 sufficient for HB,
such that the population estimates of means and covariances can be leveraged to produce more
accurate part-worth utilities than monotone regression? What about n=20?
Certainly, answers to these questions hinge upon how many attributes and levels are in the study, and
whether the questionnaire has been made relatively short or long (as the researcher has flexibility in
defining how many total concepts are generated in the near-neighbor pool for the Screening section).
Elsewhere in this documentation, we have suggested that you review the designs generated for a few
test respondents to ensure that each (non-BYO selected) level taken forward to the ACBC questionnaire
appears at least two or three times if using HB, and potentially 4 or more times if purely individual-level
estimation is required due to very small sample sizes.
We re-analyzed an existing dataset (the "laptop" dataset that is described in our published ACBC
papers) to investigate the use of HB with even tiny sample sizes. The dataset contained 286
respondents. In addition to the ACBC interview, each respondent completed four holdout choice tasks
that were formatted like CBC tasks. We can use those to compute hit rates (how well the part-worth
utilities for each person can predict which product concept the respondent would choose in the holdout
sets).
Using purely individual-level analysis (monotone regression), we obtained a 52.8% hit rate. Pooling all
respondents, we achieved 56.5% hit rate under HB. This confirms other findings, where HB has proven
more successful than monotone regression when analyzing hundreds of respondents in ACBC. Default
settings were used for both HB and monotone regression, with the exception that price was constrained
negative.
Next, we split the sample into random partitions, estimating HB separately within each subset of
respondents. We re-computed the total hit rate (across all 286 respondents) under each condition. For
example, 286 respondents can be divided equally into two groups of 143. But, dividing the sample into
four groups results in four subsamples of 72, 72, 71, and 71 respondents. In the table below, when the
number of respondents in the sub-sample didn't divide evenly into 286, we report the largest sample size
represented (for example, 72, when sub-samples of 71 and 72 were used).
Sample Sizes of Hit Rate

Random Partitions (HB)
n=286 56.5%
n=143 55.4%
n=72 55.0%
n=36 55.3%
n=18 53.9%
n=9 53.5%
As expected, the hit rate for HB estimation declines as less information is available to "borrow" from
respondents within the same dataset to estimate part-worth utilities. Even with sample sizes as small
as n=9, HB's hit rate is 53.5% compared to 52.8% for monotone regression. So, it appears that HB can
be effective with ACBC data for even very tiny sample sizes as low as n=9.
We should caution that the absolute scale of the parameters increases as smaller sample sizes are
used with HB. The Parameter Root Mean Squared (typical size of the point estimates of the part-worth
utilities) was 2.13 when all 286 respondents were included in a single HB run, but was 2.95 when the
respondents were estimated in sub-samples of n=9. This leads us to reemphasize that one should tune
the market simulator via the Exponent to achieve proper scaling of simulated shares, as the scaling
resulting from an HB estimation of ACBC data is affected by sample size and often needs tuning down
(below 1.0) to best predict market behavior.
574 SSI Web v8
3.6.10 None Threshold & Calibration to Purchase Likelihood

One of the benefits of traditional CBC modeling is the ability to include a "None" option. However,
interpretation of the strength of the None parameter as well as appropriate application within market
simulations have been difficult issues in CBC research. Adaptive CBC can estimate a None threshold
using two approaches:
None parameter estimated from Screening Tasks

None parameter inferred from purchase intent scales via Calibration Concepts
None Parameter Estimated From Screening Tasks
The Screening Tasks present product concepts to the respondent and ask if each is a possibility or not.
These tasks are coded for utility estimation as binary choices of the product concept versus a threshold
of acceptability. The parameter estimate for that threshold of acceptability may be used as a surrogate
for the None parameter within choice modeling. However, recognize that this None parameter has some
characteristics that depart from the traditional accepted practice of computing a None threshold for CBC
projects:
It is scaled based on binary choices (associated with labels such as "a possibility" and "not a
possibility") rather than in the context of a purchase/choice among multiple available options in
a marketplace.
The None parameter estimated from ACBC's binary tasks will generally be stronger (higher) than
from traditional CBC tasks. The product concepts presented in ACBC surveys are generally of
higher utility and closer to the respondent's ideal than typical CBC questionnaires. When
respondents become conditioned to having the opportunity to choose among generally more
relevant and desirable concepts, they react more negatively to concepts of moderate utility that
might have seemed acceptable to them within the context of a standard CBC questionnaire.
None Parameter Inferred from Calibration Concepts
Some users may want to calibrate the None parameter (threshold) using 5-point purchase likelihood
questions. This was suggested by Jon Pinnell in a paper entitled, "Alternative Methods to Measure
Changes in Market Size in Choice Based Conjoint" presented at the 2006 Sawtooth Software/GMI
Sydney Conference Event. In justifying the 5-point purchase likelihood intent scale, Pinnell reports:
"While the 2-point buy/no-buy is consistent with the historical use of a none alternative in choice
modeling, we prefer a 5 point follow-up question in that it provides more information and is also
consistent with most methods outside discrete choice modeling to gauge market size and product
appeal. In addition, many marketers have developed internal normative databases reflecting the five point
purchase intent scale and in-market performance and have used those to develop action standards
based on the five point purchase intent scale."
Our use of the rating scale is in the tradition of concept testing with a five-point Likert scale and departs
from choice theory and multinomial logit assumptions. Given the problems in obtaining proper scale for
the None parameter under sound choice theory, many will consider this departure to be reasonable and
defensible--especially if the firm has prior data regarding how Likert scales map to eventual product
acceptance.
Unlike ACA, our calibration procedure does not rescale part-worth utilities by multiplying by a slope and
adding an intercept. The only purpose of the calibration concept data within ACBC is to estimate a new
threshold None utility value (replacing the None threshold computed from the Screening concepts). It is
entirely optional to include these calibration concept questions in the ACBC survey (as one can simply
retain the original None parameter).
To estimate a new None utility using the Calibration Concept data, click Analysis + Estimate
ACBC None Threshold... Select an existing .hbu file as the Source File (this file contains your
part-worth utilities). Specify a new file with .hbu extension as the Destination File (this file will be
created using the original utilities plus the newly estimated None utility).
A key assumption when estimating a new value for the None utility is the point on the Likert scale that
should represent the buy/no buy threshold for respondents. Obviously, this could differ by respondent.
However, we generalize that the buy/no buy threshold on the scale is constant across respondents and
further assume that you as the forecaster can identify that appropriate scale point (easy, right?).
The scale points on the Likert scale are typically labeled as follows:
5 Definitely Will Buy

4 Probably Will Buy
3 Might or Might not Buy
2 Probably Will not Buy
1 Definitely Will not Buy
If, for example, you indicate that the None threshold is the product utility corresponding to a 3.2 on the
scale, we estimate that utility (for each respondent) by referring to the already-estimated part-worth
utilities and the calibration concept data. The calibration ratings are regressed (at the individual level) on
the final utilities that resulted from HB or Monotone Regression. We'd recommend that respondents be
shown five or six calibration concepts, to get reasonably stable estimates from the regression. Using the
respondent-specific estimated slope (Beta1) and intercept (C), we can compute the utility corresponding
to a 3.2 on the Likert scale using the equation (3.2 - C) / Beta1.
The estimation of the None threshold from the purchase likelihood scale is a post hoc procedure that
does not involve re-estimating the part-worths. As it uses OLS, the computation is nearly instantaneous,
and the analyst can quickly compute new None threshold parameters using different assumptions about
the point on the rating scale that corresponds to the threshold point for buy/no buy.
When you calibrate the None parameter, you create a new .hbu file. Only two fields are modified in that
file: the None parameter value and the fit statistic. The calculation of the None parameter was described
directly above. The fit statistic is the correlation resulting from the calibration regression step (the square
root of the R-squared). When this None utility threshold is used within the market simulator, a new
"product" with utility equal to the None threshold value is introduced in competition with the other product
concepts within that market simulation. The share captured by the "None" product reflects the percent
of the market projected not to buy a product within this scenario.
The default setting for the point on the rating scale corresponding to the buy/no buy threshold is 3.0. We
have no evidence that this is proper for your project needs, and recommend you decide for yourself the
appropriate setting. Using a recent dataset shared with us, we found that using a 2.8 on the 5-point
purchase likelihood scale led to a None parameter from the OLS calibration that on average was about
the same magnitude as estimated via HB using the binary judgments of the Screener section (where the
binary choices were labeled "A possibility" or "Won't work for me"). Thus, using the default 3.0 as the
buy/no buy threshold resulted in a slightly stronger None parameter utility than was initially estimated
using the Screener section information.
576 SSI Web v8
Exception Handling:
Given respondent error, regressions at the individual level based on so few data points may sometimes
lead to irrational outcomes, such as negative slopes (suggesting a negative relationship between
estimated utilities and scores on the Likert scale). Also, if the respondent answers the calibration
concepts using the same scale point, there is no variation to model. Therefore, we must handle these
exceptions.
If the estimated slope is negative, the None threshold is set equal to the average utility for products
exceeding the threshold point on the Likert scale minus 1. If the slope is negative and no products are
above the threshold, we set the none utility equal to the largest utility of any product plus 1. If the slope
is negative and all products are above the threshold, we set the none utility equal to smallest utility of
any product minus 1. If there is no variation in the Likert scale responses, we do the following: if all
products are rated equally at a point below the threshold point on the scale, we set the none utility equal
to the largest utility of any product plus 1; if all products are rated equally on or above the threshold, we
set the None utility equal to the smallest utility of any product minus 1. With all these exceptions, a fit
statistic of 0 is written to the new utility file.
Because the None utility is estimated using the equation (UserSpecifiedScalePoint -

ConstantFromRegression)/Beta, a small positive Beta near zero can cause the None utility to become
extremely large in absolute magnitude. To guard against the most extreme cases, if the estimated beta
is between 0 and 0.05, we set it to 0.05.
For a recent dataset shared with us, 88% of the sample had calibration data that were well behaved and
did not require exception handling (there was variation in the dependent variable and the estimated
slopes were positive and larger than 0.05). Of the 12% that required exception handling, 5% had tied
ratings (across four calibration concepts), 6% had negative slopes, and 1% had slopes between 0 and
0.05 that we trimmed to 0.05.
3.6.11 Exporting ACBC Data to .CHO File

Some researchers may wish to export the data to a .CHO file for analysis using other software programs,
such as our CBC/HB system or Latent Class software. (Notes: SMRT cannot import a .CHO file from an
ACBC study correctly, as the new .CHO file includes additional incompatible information. If your design
uses constructed lists that drop entire attributes from consideration, please see the warning further
below.)
1. Select File + Data Management
2. Select the Export Data tab
3. Add a new export job by clicking Add Job... (name your job something like ".CHO export")
4. Click Add to add a new Export action. Select the ACBC (.cho, .att) type.
5. If you have more than one ACBC exercise within your SSI Web project, select which ACBC exercise
to export.
6. If you used constructed lists to drop certain levels of certain attributes for some respondents, select
Settings and specify how the dropped levels should be treated.
7. Click OK to finish adding the new job to the export list.
8. From the Data Management dialog, Export Data tab, click Export Jobs (make sure the jobs you
want exported are checked within the list).
When you export a .cho file, ACBC reports how many respondents have been exported to the file. It also
informs you regarding which attribute numbers in the .cho file are coded as "user-specified." This simply
means that these columns in the .cho file should be used in the estimation software "as-is": they require
no further effects-coding or dummy-coding. When you open the .cho file within our CBC/HB or Latent
Class software, you should change the coding method for these attributes to "user-specified." The "user-
specified" columns within ACBC studies reflect the Summed Price attribute, the None parameter, and
any threshold parameters for attributes using constructed lists (where selected levels were dropped from
consideration).
Under the Settings dialog, you can select which respondents to export as well as the method to use for
respondent numbering.
You can also select which tasks to include in the .CHO file: BYO (by attribute), Screener Section,
Choice Task Tournament.
Scaling of Summed Price
The default prior settings in CBC/HB software work well as long as the data are scaled similarly within
the design matrix. Generally, attributes are effects-coded, which transforms the level indices to values of
1, 0 or -1. But, if your summed prices are in the hundreds, thousands or millions, it can pose serious
problems for proper convergence in CBC/HB. Thus, the software allows you to rescale your price units
by supplying a price multiplier on the Settings dialog. We recommend a price multiplier that brings your
summed prices into units scaled in the 1s or 10s. For example, a price range of 1 to 5 or 10 to 50
seems to work well. But, prices scaled from 100 to 500 or especially 100,000 to 500,000 would not work
well.
578 SSI Web v8
Please note that if using summed prices, your results when using CBC/HB to estimate utilities will not
exactly match the results from running HB within the ACBC interface. That is due to the way we have
chosen to rescale price in ACBC to ensure proper convergence under unpredictable price scaling
conditions. Within the ACBC interface, HB automatically transforms prices to a -1 to +1 range within the
design matrix. Then, before writing the results out to the utility file, it scales the utilities back to the
original price scaling. Using a .cho export, however, will result in prices that are not zero-centered. This
will especially affect the constant (the None parameter) in the model.
How to Treat Missing Levels
If you used constructed lists for some attributes, we recognize that it is possible that respondents have
some levels deleted prior to the ACBC interview. For example, you may have deleted certain brands that
were unacceptable or not available to the respondent. You must specify how to treat these levels in the
coding of the .cho file. Missing levels can be treated as:
Inferior to included levels

Unavailable
Missing at random
This parallels the options that are available within the built-in estimation interface for HB within the ACBC
system. If you specify that missing levels are to be treated as inferior to included levels, additional
choice tasks are written to the file for each respondent, informing utility estimation that included levels
are superior to (chosen over) inferior levels. If you specify that missing levels are either Unavailable or
Missing at random, these additional tasks are not written. The result is the same for the .cho export
whether you select these latter two options.
Warning Regarding Using .CHO File and Missing Attributes
If you used constructed lists to wholly drop certain attributes from the ACBC questionnaire that were
deemed entirely unimportant (using questions prior to the ACBC sections), then you should be careful
about using CBC/HB, Latent Class, Aggregate Logit or another routine outside ACBC software for
estimation. Unless you take special steps to inform utility estimation that certain attributes have
been dropped and should have no importance for certain respondents, your utility estimation will
overstate the importance of such attributes that were dropped from consideration. ACBC's built-in
CBC/HB and Monotone Regression routines inform utility estimation regarding missing (and entirely
unimportant attributes) by inserting utilities of zero for the levels of such attributes for respondents who
have dropped the attribute from consideration in ACBC. Therefore, if you use another routine, you should
take steps such as post-processing the utilities to set the part-worths (at the individual level) to zero for
any attributes that have been dropped from consideration by those respondents.
3.7 ACBC: How Well Does It Work?

To this point, we have conducted three fairly ambitious methodological studies comparing ACBC to
standard CBC. Because we have documented the results of those experiments in two white papers
published on our website, we refer the reader to those articles and just cover the highlights here.
Chris Chapman of Microsoft conducted a series of comparisons of CBC and ACBC as it relates to real
product selection, and reported the results at the 2009 Sawtooth Software Conference (Chapman et al.
2009).
The SKIM group also conducted a recent study to compare ACBC and CBC (Fotenos et al. 2013).
To read more about the details of these studies, we recommend you download and read the following
white papers, available from our Technical Papers Library at www.sawtoothsoftware.com:
"A New Approach to Adaptive CBC"

"Testing Adaptive CBC: Shorter Questionnaires and BYO vs. 'Most Likelies' "
"CBC vs. ACBC: Comparing Results with Real Product Selection"
"ACBC Revisited" (Fotenos et al. Sawtooth Software 2013 Proceedings)
ACBC passed through an eight-month beta test program. Over fifty beta testers conducted over 40
studies during that period. We received very positive feedback from our beta testers. Some of their
experiences are described on our website, in the ACBC area.
Judging Success
The critical design issue when comparing one conjoint method to another is the criteria for success.
Ideally, the researcher would have access to sales data (or subsequent actual choices of respondents)
and could compare predicted choices with actual choices.
In the absence of data on real-world choices, many researchers turn to holdout choice tasks included
within stated preference surveys. Traditionally, these have looked exactly like standard CBC questions:
typically 3 to 5 product concepts. One of the key things we and other researchers have learned about
standard CBC tasks is that respondents often key on just a few levels to make decisions. They often
establish a few cutoff rules for excluding or including concepts within their consideration sets. And, if
one uses standard CBC tasks that employ "minimal overlap" (where each level is typically available no
more than once per task), often only one product concept can satisfy respondents. Choices to such
tasks often reflect simplified (superficial) behavior, and other choice tasks designed in the same way not
surprisingly are quite successful in predicting answers to those holdouts. We have found that ACBC
does about as well as CBC (sometimes better, sometimes worse) in predicting these kinds of holdout
CBC tasks. And, that parity doesn't concern us, as we'll further explain.
We have wondered whether traditional holdout tasks really do a good job in mimicking actual purchase
behavior, or whether they reflect simplified processing heuristics that are a byproduct of respondents
(especially internet sample) completing marketing research surveys with less motivation than they would
apply to real-world purchases. When people make real decisions, they often narrow down the choices to
an acceptable consideration set (with respect to must-have and must-avoid features) and then make a
final choice within the consideration set. To better mimic that process in the holdouts, our first three
methodological studies used a customized type of holdout CBC task that involved comparing winning
concepts chosen from previous CBC tasks. For example, the respondent might be shown four standard
(fixed) CBC holdout tasks, but the fifth holdout included the four winning concepts from the first four
tasks. Such customized tasks lead to questions that can probe beyond just the first few critical levels of
580 SSI Web v8
inclusion and reflect more challenging tradeoffs. We have found that ACBC consistently predicts these
choices more accurately than traditional CBC.
Some researchers (such as Elrod 2001) have stressed that if we validate to holdout tasks within survey
questionnaires, not only should the holdout tasks be excluded from the calibration tasks used for
estimating the utilities, but a sample of holdout respondents should be held out as well. For example,
we should be using calibration tasks from respondent group A to predict holdout tasks from respondent
group B (and the actual tasks used in group B should not have been used in the experimental design for
group A). When this is done, one does not compute individual-level hit rates (since it is no longer a
within-respondent analysis), but one uses market simulations and measures share prediction accuracy.
Simulations of market choices for group A are used to predict choices to fixed holdouts for group B.
We were fortunate enough to have a robust enough sample in our first ACBC methodological study to
follow Elrod's recommendations. In addition to the six hundred respondents in the sample used to
estimate part-worth utilities, we were able to collect a separate sample of 900 respondents who all
received a single-version (fixed) set 12 CBC tasks. We found that ACBC predicted shares of choice for
these scenarios just as well as CBC. But, when we split the 900 respondents into three equal-sized
groups based on interview time, we found that ACBC performed worse than CBC in predicting the
choices for respondents who were the quickest but better than CBC for respondents that took the most
time (and presumably were providing more in-depth and careful answers).
We mention these conclusions and findings to provide direction to researchers who may want to test the
merits of ACBC relative to more traditional methods like CBC. If standard holdout CBC tasks are used
as the criterion for success, there is methods bias that strongly favors CBC in predicting those tasks. If
respondents are using the same (probably unrealistic) simplification heuristics to answer calibration CBC
tasks, they will employ the same heuristics in answering holdouts. One needs to consider carefully the
appropriate criteria for success. You may find that ACBC does not consistently beat CBC in predicting
static minimal-overlap holdout choice tasks, for the reasons outlined above. It occurs to us, however,
that if static choice tasks include a greater number of concepts than are traditionally used, more overlap
will be introduced, increasing the likelihood that respondents will need to make a more careful decision
regarding multiple acceptable concepts per task prior to making a decision. In those cases (which are
probably more representative of real choice situations), ACBC may very well consistently beat CBC.
We suspect that ACBC will be more predictive of real-world buying behavior than CBC, but we need
practitioners and firms with access to the appropriate sales data to verify this. This is an exciting new
area of research, and we are optimistic that ACBC will indeed prove better than traditional CBC for
complex conjoint problems involving about five or more attributes.
Previous Results and Conclusions
To date, we have completed three substantial methodological studies comparing ACBC to CBC. The
study names, sample sizes, and numbers of attributes are as follows:
Laptop PC Study (n~600 calibration respondents; n~900 holdout respondents; 10 attributes)

Recreational Equipment Study (n~900; 8 attributes)
Home Purchases study (n~1200; 10 attributes)
The experiments all used web-based interviewing. In the case of the Laptop and Home Purchase
studies, we used Western Wats panel. For the Recreational Equipment study, customer sample was
provided by the client. For the Laptop and Home Purchase studies, we implemented a carefully
controlled random split-sample design, where respondents were randomly assigned to receive ACBC or
CBC questionnaires, or assigned to be holdout respondents. With the Recreational Equipment study,
because of conditions related to working within the client's time line, we weren't able to conduct a
perfectly controlled random assignment of respondents to different conditions, and the sample pull
receiving the ACBC survey had potentially different characteristics from those receiving CBC. This
weakens our ability to draw firm conclusions from that study.
Interview Time
The ACBC questionnaires took longer than the CBC questionnaires to complete (50% longer in the
Housing survey, 100% longer in the Laptop study, and 200% longer in the Recreational Equipment
study). We suspect that the Recreational Equipment study was relatively longer because it interviewed
respondents from a customer list, who may have been even more enthusiastic and engaged in the
process. The Housing survey was relatively shorter, because we investigated whether we could shorten
the survey length without sacrificing much in terms of predictive validity (see results below, which show
that even the shortened ACBC surveys were successful).
A key finding is that even though ACBC respondents were asked to do a longer task, they rated the
survey as more interesting and engaging than respondents taking CBC surveys rated the CBC surveys.
Our thought is that when collecting conjoint data, speed alone shouldn't be the goal, as we cannot ignore
quality. And, if we can improve the survey experience for the respondent even though we are asking
them to spend more time, then the additional investment to acquire better data is well worth the effort.
Interest/Satisfaction with the Survey
For the Laptop and Home Purchases studies (which provide the strongest results due to the controlled
experiments), the ACBC respondents reported higher satisfaction and interest in the survey (than CBC
respondents). They reported that the survey was more realistic, less monotonous/boring, and that it
made them more likely to want to slow down and make careful choices (than the CBC respondents). All
these differences were significant at p<0.05.
Hit Rates
In the two studies involving random assignment to either ACBC or CBC cells, the ACBC respondents had
higher hit rates (for the customized CBC holdout task) than CBC respondents, with differences significant
at p<0.05:
Hit Rates, Custom Holdout Task
ACBC CBC
Laptop Study: 61 50
Houses Study: 44 37
For the Recreational Equipment study (which lacked a completely random split-sample control), ACBC
had a 2-point edge in hit rate prediction over CBC, but the result was not statistically significant. All
these figures reflect the generic HB estimation method.
Some researchers have wondered whether the edge in predictive ability demonstrated by ACBC over
CBC might be accounted for by the longer questionnaire time for ACBC. Maybe ACBC's success is due
to more precise estimates because we've asked more questions of each respondent. While this seems
plausible, we doubt that this can entirely explain the results, because it has been demonstrated that hit
rates for CBC respondents gain very little beyond about 10 to 15 choice tasks (and the CBC surveys for
the Laptop and Houses studies both included 18 tasks). We come to this conclusion based on a meta-
582 SSI Web v8
study of 35 commercial data sets by Hoogerbrugge and van der Wagt of SKIM Analytical as presented at
the 2006 Sawtooth Software Conference. They randomly threw out one task at a time and estimated the
loss in hit rate precision depending on questionnaire length. Based on their research, in the range of 10
to 15 choice tasks, the gains in hit rate rapidly decreased and in some cases leveled off through up to 19
tasks. The net pattern and absolute magnitude of the gains (observed across 35 data sets) by extending
the questionnaire would not seem to be able to eventually make up the gaps of 11 and 7 absolute hit rate
points in favor of ACBC as seen in our two experiments. In the future, it would be good to see research
that time-equalized the CBC and ACBC questionnaires (though respondents receiving the CBC
questionnaire will face a monotonous challenge!).
Share Predictions
In the Laptop study, we were able to estimate shares for holdout CBC tasks completed by a new sample
of 900 respondents. The 900 new respondents completed 12 fixed CBC tasks. The part-worths
generated from ACBC performed just as well as those from CBC in predicting these holdout shares,
when including all respondents. But, we split the 900 respondents into three equal samples, based on
time taken in the interview. After segmenting based on time, we found that CBC predicted the fastest
respondents better than ACBC, but ACBC predicted the middle and slower respondent groups better
than CBC (with the best prediction occurring for the holdouts completed by the slowest respondents). In
each case, we tuned the simulations for scale, so the differences cannot be explained by the random
error components (which were significantly different for the three segments of respondents). Also, a
Swait-Louviere test (which also controls for scale) found differences in the parameters across the groups
of holdout respondents. This finding suggests that ACBC can do a better job at predicting choices made
by respondents who take the most time and presumably are being more careful and considerate in their
choices.
For the Houses Study, we did not collect holdout respondents. However, we were able to use the part-
worths from the ACBC and CBC respondents to predict choices made by the two groups of respondents
combined (a total of 1200 respondents in the validation sample). Thus, ACBC part-worths were being
used to predict holdout responses made by both ACBC and CBC respondents; and CBC respondents
the same. Based on predictions of four fixed holdout tasks (each with four product alternatives), the
ACBC predictions were better than those from CBC (the mean absolute errors in prediction were 3.00
and 5.47 share points for ACBC and CBC, respectively).
Price Sensitivity
In the three methodological studies we conducted, the price sensitivity derived from ACBC estimates
was quite similar to that from CBC. Our beta testers have reported some instances where ACBC has
produced significantly greater price sensitivity than CBC. In those instances, the ACBC result was
viewed as potentially more realistic. We hope to see further research conducted in this area.
3.7.1 References
Chapman, Chris, James L. Alford, Chad Johnson, Ron Weidemann, and Michal Lahav. (2009), "CBC vs.
ACBC: Comparing Results with Real Product Selection," Sawtooth Software Conference Proceedings,
199-206.
Elrod, Terry. (2001), "Recommendations for Validation of Choice Models," Sawtooth Software
Conference Proceedings, Sequim, WA.
Fotenos, Christopher, Jeroen Hardon, and Marco Hoogerbrugge, "ACBC Revisited," Sawtooth Software
Conference Proceedings, Orem, UT.
Gilbride, Timothy and Greg M. Allenby (2004), "A Choice Model with Conjunctive, Disjunctive, and
Compensatory Screening Rules," Mark eting Science, 23, 3, (Summer) 391-406.
Hauser, John R., Ely Dahan, Michael Yee, and James Orlin (2006) "'Must Have' Aspects vs. Tradeoff
Aspects in Models of Customer Decisions," Sawtooth Software Conference Proceedings, 169-181.
Hoogerbrugge, M. and K. van der Wagt (2006), "How Many Choice Tasks Should We Ask?" Sawtooth
Software Conference Proceedings, Sequim, WA.
Johnson, Rich and Orme, Bryan (2007), "A New Approach to Adaptive CBC," Sawtooth Software
Conference Proceedings, Sequim, WA.
Orme, Bryan and Rich Johnson (2008), "Testing Adaptive CBC: Shorter Questionnaires and BYO vs.
'Most Likelies'," Technical Paper Available at www.sawtoothsoftware.com.
584 SSI Web v8
MaxDiff Help 585
4 MaxDiff Help
4.1 Getting Started with MaxDiff
4.1.1 Introduction
Have you ever been asked to measure respondents' preferences for things such as brands, product
features, job-related benefits, or product packaging? Have you ever been asked to prioritize a list of
performance attributes or gauge the potential impact of different advertising claims? If so, you should
consider the class of trade-off related techniques available within MaxDiff, a component within the SSI
Web system.
MaxDiff is an approach for obtaining preference/importance scores for multiple items (brand preferences,
brand images, product features, advertising claims, etc.) using marketing or social survey research.
MaxDiff is a component within Sawtooth Software's SSI Web platform for Web- and CAPI-based
interviewing. It may be used for designing, fielding, and analyzing:
MaxDiff (best-worst scaling) experiments

Method of Paired Comparisons (MPC) experiments (choices from pairs)
Choices from subsets of three items, four items, etc.
Projects may be conducted over the Internet, using computers not connected to the internet (CAPI
interviewing), or via paper-and-pencil questionnaires (using the included MaxDiff paper-and-pencil utility
and the separately purchased CBC/HB system).
MaxDiff makes it easy for researchers with only minimal exposure to statistics to conduct sophisticated
research for the scaling of multiple items. The trade-off techniques used in MaxDiff are robust and easy
to apply (for example, much easier than the related conjoint analysis). The resulting item scores are
also easy to interpret, as they may be placed on a 0 to 100 point scale.
586 SSI Web v8
4.1.2 Motivation for MaxDiff

Researchers across many disciplines face the common task of measuring the preference or importance
of multiple items, such as brands, product features, employee benefits, advertising claims, etc. The
most common (and easiest) scaling approaches have used rating, ranking, or chip allocation (i.e.
constant sum tasks).
Example Rating, Ranking, and Allocation questions are shown below (this software does not employ any
of these approaches, but we repeat them here for exposition):
Example Rating Task:
Please rate the following in terms of importance to you when eating at a fast food restaurant.
Use a 10-point scale, where "0" means "not important at all" and "10" means "extremely
important"
___ Clean bathrooms

___ Healthy food choices
___ Good taste
___ Reasonable prices
___ Has a play area
___ Restaurant is socially responsible
___ Courteous staff
Example Ranking Task:
Please rank (from most important to least important) the following in terms of importance to
you when eating at a fast food restaurant. Put a "1" next to the most important item, a "2"
next to the next most important item, etc.
___ Clean bathrooms

___ Good taste
___ Has a play area
___ Courteous staff
Example Allocation Task:
Please tell us how important the following are you when eating at a fast food restaurant. Show
the importance by assigning points to each attribute. The more important the attribute, the
more points you should give it. You can use up to 100 total points. Your answers must sum
to 100.
___ Clean bathrooms

___ Good taste
___ Has a play area
___ Courteous staff
MaxDiff Help 587
Total: ______
There are a variety of ways these questions may be asked, including the use of grid-style layout with
radio buttons and drag-and-drop for ranking. And, it is commonplace to measure many more items than
the seven shown here. (These examples are not intended to represent the best possible wording, but are
meant to be representative of what is typically used in practice.)
The common approaches (rating, ranking, and allocation tasks) have weaknesses.
Rating tasks assume that respondents can communicate their true affinity for an item using a
numeric rating scale. Rating data often are negatively affected by lack of discrimination among
items and scale use bias (the tendency for respondents to use the scale in different ways, such as
mainly using the top or bottom of the scale, or tending to use more or fewer available scale points.)
Ranking tasks become difficult to manage when there are more than about seven items, and the
resulting data are on an ordinal scale only.
Allocation tasks are also challenging for respondents when there are many items. Even with a
manageable number of items, some respondents may have difficulty distributing values that sum to
a target value. The mechanical task of making the allocated points sum to the target amount may
interfere with respondents revealing their true preferences.
Researchers seek scaling approaches that feature:
Ease of use for respondents from a variety of educational and cultural backgrounds
Strong discrimination among the items
Robust scaling properties (ratio-scaled data preferred)
Reduction or elimination of scale use bias
A very old approach, the Method of Paired Comparisons (MPC), seems to perform well on all these
requirements. With MPC, respondents are shown items, two at a time, and are asked which of these
two they prefer most (or which is most important, etc.).
Example Paired Comparison Task:
The respondent is not permitted to state that all items are equally preferred or important. Each
respondent is typically asked to evaluate multiple pairs, where the pairs are selected using an
experimental plan, so that all items have been evaluated by each respondent across the pairs (though
typically not all possible pairings will have been seen) and that each item appears about an equal
number of times.
It should be noted that MPC can be extended to choices from triples (three items at a time), quads (four
588 SSI Web v8
items at a time) or choices from even larger sets. There would seem to be benefits from asking
respondents to evaluate three or more items at a time (up to some reasonable set size). One research
paper we are aware of (Rounds et al. 1978) suggests that asking respondents to complete sets of from 3
to 5 items produces similar results as pairs (in terms of parameter estimates), and respondents may
prefer completing fewer sets with more items rather than more sets with just two items.
A much newer technique, MaxDiff, may perform even better than MPC, especially in terms of overall
question efficiency. MaxDiff questionnaires ask respondents to indicate both the most and least
preferred/important items within each set.
Example MaxDiff Task:
Interest in MaxDiff has blossomed recently, and papers on MaxDiff have won "best presentation" awards
at recent ESOMAR and Sawtooth Software conferences.
MaxDiff Help 589
4.1.3 What is MaxDiff?

MaxDiff is a technique invented by Jordan Louviere in 1987 while on the faculty at the University of
Alberta (Flynn and Marley, 2012, "Best Worst Scaling: Theory and Methods"). With MaxDiff,
respondents are shown a set (subset) of the possible items in the study, and are asked to indicate
(among this subset with a minimum of three items) the best and worst items (or most and least
important, etc.).
Below is an example, involving a set of four items.
Example MaxDiff Task:
MaxDiff may be thought of as a more sophisticated extension of the Method of Paired Comparisons
(MPC). Consider a set in which a respondent evaluates four items, A, B, C and D. If the respondent
says that A is best and D is worst, these two "clicks" (responses) inform us on five of six possible
implied paired comparisons:
A>B, A>C, A>D, B>D, C>D
where ">" means "is more important/preferred than."
The only paired comparison that we cannot infer is B vs. C. In a choice among five items, MaxDiff
questioning informs on seven of ten implied paired comparisons.
MaxDiff questionnaires are relatively easy for most respondents to understand. Furthermore, humans
are better at judging items at extremes rather than in discriminating among items of middling importance
or preference. And since respondents make choices rather than expressing strength of preference, there
is no opportunity for scale use bias. This is an extremely valuable property for cross-cultural research
studies.
Technical Note: Although we use the terms MaxDiff and Best-Worst interchangeably, academics
(including the inventor, Jordan Louviere) are recently mak ing a distinction between the terms. The
distinction is drawn depending on the route used to code the data for estimating the scores. The method
we use to code the data is the Best-Worst approach. Thus, this software would more formally be
described as providing Best-Worst rather than MaxDiff analysis. The benefits of modeling as sequential
best-worst (as two separate choices) allows the researcher to more easily investigate the differences in
preference scores and response error between best and worst judgments.
590 SSI Web v8
4.1.4 MaxDiff Tutorial and Example

Introduction
This tutorial leads you step-by-step through a simple MaxDiff study, from thinking about a research
problem and planning a questionnaire to entering a list of MaxDiff items in SSI Web, generating and
testing the questionnaire, and analyzing the results.
SSI Web is a powerful program, and this tutorial only touches on the basics for running a MaxDiff
experiment. You'll become aware of different or more sophisticated approaches by reading other
sections in the on-line help and by experimenting with SSI Web.
(Note: this example is based on a presentation delivered at the 2003 Sawtooth Software Conference by
Amanda Kraus, Diana Lien, and Bryan Orme. Though many of the items we use in this example are
identical to the ones used in the conference presentation, the data we have developed for this example
are purely fictional. The suggested steps represent one approach to this type of research problem, but
may not necessarily reflect the single "best" way to conduct such a research study using the SSI Web
system.)
The Research Problem
As the economy has improved (2003 perspective), the Navy has faced declining reenlistment rates
among its sailors. The Navy is looking for ways to improve the reenlistment rate. Previous research has
shown that reenlistment terms, quality of life issues, and the work environment are all significant factors
when sailors face the reenlistment decision. Pure economic aspects matter as well, such as the
amount of the reenlistment bonus. You have been brought on as a consultant to help the military design
and execute a research study to prioritize a list of possible job and quality of life enhancements as well
as reenlistment terms. The Navy is keenly interested in how reenlistment bonuses may affect sailors'
desire to enlist relative to other non-monetary issues. Those improvements which are most likely to
increase the reenlistment rates at the lowest expense to the military should be implemented first. They
would like to implement this study over the Web, and expect that between 1,000 and 2,000 sailors might
complete the study.
At first, you consider whether a conjoint method such as CBC would be appropriate. You ask your
military client, Jane, if the resulting model needs to be able to project the likely reenlistment rate when
combinations of job enhancements are taken together. Jane assures you that although that type of
analysis would be of interest to military planners, the primary consideration is the ability to prioritize the
items from most to least effective in terms of improving reenlistment rates. The planners need to be able
to directly compare the effect of each job enhancement or monetary bonus.
After pondering the problem, you recognize that a strength of MaxDiff is its ability to place multiple items
on the same scale using a trade-off methodology. With conjoint analysis (such as CBC), the client
would not be able to directly compare the preference for levels of one job-related item to another. MaxDiff
doesn't formally support adding the preference scores across items and predicting the overall benefit of
the job package or likelihood of reenlistment given a specific job package (as would CBC), but that is not
a requirement for the study. Therefore, you decide to use MaxDiff.
Developing a List of Items
You meet with Jane to discuss the elements (items) to be studied in the MaxDiff exercise. Jane
explains that a series of focus groups had been conducted over the last four months wherein the reasons
MaxDiff Help 591
for reenlistment versus leaving the Navy were discussed. Hundreds of ideas had been presented, but
there seemed to be some common themes. Furthermore, Jane had discussed the myriad ideas with
military planners. Many of the ideas were clearly off-the-table (such as "king-sized beds on
submarines," "more parties," and "supermodels as supervisors"). However, there were a number of
areas of overlap: job conditions that the military was in the position to change and those that also were
on the minds of the sailors.
The list included the following:
Length of reenlistment obligation

Location or duty assignment guarantees for next assignment
Faster promotion
Living conditions when in port (2-person barracks vs. 4-person barracks)
Increase in sea pay
Percent of on-job time using your skills and training
Time off each week allowed for on-line college coursework
Online career counseling services
Personal counseling services
Reenlistment bonus
This looked like a good start, but it seemed obvious to both you and Jane that some further refinement
and quantification of benefits was needed. The items needed to carry specific meaning to the sailors and
the results needed to be actionable for the military planners. Most of the items seemed to require
multiple levels, whereas some seemed to stand on their own. After thinking about the reasonable items
(and some back-and-forth with Jane's internal clients, the military planners) a final list was drawn up:
Factors Affecting Reenlistment Decision:

3-year reenlistment obligation
Location guarantee for next assignment
Duty guarantee for next assignment
Both location and duty guarantees for next assignment
Get promoted 6-months sooner than expected
Live in 2-person barracks when in port
Choice between living in 2-person barracks or 4-person barracks when in port
$50 per month pay increase when out at sea
Half of on-job time using your skills and training
Three-quarters of on-job time using your skills and training
Receive online career counseling services
Receive in-person personal counseling services
Receive phone-based personal counseling services
Current reenlistment bonus ($2,500)
Increased reenlistment bonus ($5,000)
Next, you worked with Jane to decide which items would need to be prohibited from displaying with each
other versus which could freely combine with all other items within the same set. For example, it would
not make sense to ask sailors to choose between a $50 pay increase versus a $100 pay increase when
out at sea. Every rational sailor would choose higher pay over lower, so this would be a wasted
comparison. Other items such as living in a 2-person barracks versus a 4-person barracks didn't seem
592 SSI Web v8
to have a rational preference order (though having a choice between 2-person barracks or 4-person
barracks would seem to be preferred to no choice). Perhaps some sailors would prefer to live with more
people, whereas others would prefer just one roommate.
You organized the list into groupings of items (listed as groups A through F), where the items within
groups A through E would require at least some prohibitions (restrictions that they could not be
displayed together within the same choice set).
A) Prohibit these two items:

B) Prohibit third item with appearing with either item 1 or item 2:

C) Prohibit third item with appearing with either item 1 or item 2:

D) Prohibit these two items:

E) Prohibit these items from appearing together:

F) No prohibitions with other items:

Scripting the Questionnaire
A real research project would involve other questions about each sailor, including gender, current rank or
job specification, current location, inclination to reenlist, etc. Standard survey questions (such as select
questions with radio buttons or check-boxes) can be developed within the SSI Web platform (of which
MaxDiff is a subcomponent). Rather than take time within this tutorial to add such classification
questions, we'll focus on the MaxDiff section of the survey. Interested users can look to the CiW Tutorial
within this same online help and manual for further help with standard survey questions.
You decide to include the following introductory text leading into the MaxDiff questions:
MaxDiff Help 593
Start:
Thank you for taking time to help the Navy consider ways to improve your working environment
and career development. Our goal is to make the Navy a satisfying career for you and to improve
your desire to reenlist when the decision arises.
We would like you to imagine that you are facing the decision whether to reenlist or not--today.
Please think about the issues involved, including future career planning, family/relationship
considerations, job location, and compensation.
Click the Next button below to continue...
IntroMaxDiff:
If you were making the decision to reenlist or not today, there would certainly be many factors
that affect your decision to reenlist or not. Real life decisions are not always easy.
We want to gain some insight into how you make this decision by asking you to evaluate a series
of tradeoff scenarios. In each scenario, we'll show you four possible factors affecting your career
in the Navy and the reenlistment decision.
We'll ask which factor (among this set of four) would make you most want to reenlist, and which
would make you least want to reenlist.
We need to ask you repeated tradeoff scenarios (involving different factors each time) so that we
can learn what is truly important to you when deciding whether to reenlist.
Layout for MaxDiff Questions
Next, you need to decide the layout of the MaxDiff questions. Research has suggested that showing
more than about five items at a time in MaxDiff questions may not be very beneficial. Some of the items
in your study are a bit on the wordy side, so you decide to show just four items at a time.
The MaxDiff documentation suggests showing each item between 3 to 5 times on average per
respondent across the sets. With 20 items in the study, the number of questions to display so that
each item is shown on average 3 times per respondent is equal to:
3(K/k)
where K is the total number items in the exercise, and k is the number of items per set
Therefore, you would need to ask each respondent a minimum of 3(20/4) = 15 sets. The "nice to have"
number of sets would be 5(20/4) = 25. After discussing with the client, you decide that respondents
should be able to complete 20 sets with little problem (each item displayed on average 4 times to each
respondent). You estimate that each set will take about 20 seconds to complete (prior research
presented at the Sawtooth Software Conference reports that figure), for a total average interviewing time
of between 6 to 7 minutes for the MaxDiff section.
To try to keep respondents engaged and working productively through the MaxDiff exercise, you plan to
show respondents a progress bar and also provide a count-down message after every five MaxDiff
questions.
594 SSI Web v8
Here's an example of the MaxDiff question layout you plan to use:
After every five MaxDiff questions, you plan to show some text that provides encouragement and a
countdown, such as:
ProgressCounter:
Good work so far. We know that some of these tradeoffs are challenging.
We need to ask you to evaluate 15 more sets so that we can better determine how you make
your reenlistment decision.
And at the end, you want to display a screen that thanks respondents, and then immediately sends
them to a Navy site www.navy.mil.
Finish:
Thank you very much for your help. Your data will be used to help Navy planners as they work to
make the Navy an even better career choice.
Click the Next button to finish submitting your data and to return to the Navy's official website.
MaxDiff Help 595
interviewing module.) MaxDiff is one component within that software program. (Although not emphasized
here, MaxDiff may be used for conducting paper-based MaxDiff projects.)
Browse... button to the right of the Location field, and then (after browsing to the folder in which you want
SSI Web has a maximum number of twenty-four characters (either letters or numbers), and SSI Web
for your study also carries the same name as the study. For this tutorial, you might create a name such
as navyproject. From the New SSI Web Study dialog, specify navyproject as the studyname.
Click the OK button. You are returned to the main menu, and a new "Study Navigator" window is
displayed.
Adding the Introduction Screens to the Survey
(If you are already familiar with the Start (passwords) question, Text/HTML Filler and Select questions
within SSI Web surveys, you may choose to skip forward to the section entitled "Adding MaxDiff
Questions to the Survey").
596 SSI Web v8
Let's enter the introductory text using the Write Questionnaire dialog. You can access the Write
Questionnaire dialog by selecting Compose | Write Questionnaire… or by clicking the "pencil" icon
on the toolbar.
The introductory text and survey questions would most likely be initially developed within a word
processing document. Assuming you really had such a document, you might press Ctrl-C to copy, and
Ctrl-V to paste the information into SSI Web. We suggest you simply copy-and-paste the text within
this document into your SSI Web questions rather than re-type the text (if viewing this document with
Acrobat Reader, you can use the "text select" icon from the Acrobat Reader toolbar.)
The first question to compose is the introductory screen:
Start:
Thank you for taking time to help the Navy consider ways to improve your working environment
and career development. Our goal is to make the Navy a satisfying career for you and to improve
your desire to reenlist when the decision arises.
We would like you to imagine that you are facing the decision whether to reenlist or not--today.
Please think about the issues involved, including future career planning, family/relationship
considerations, job location, and compensation.
SSI Web survey. The Start question is also the place where you would ask respondents to type their
passwords to access the survey. We'll not use passwords for this sample survey.
From the Write Questionnaire dialog open the Start question by highlighting it in the list of questions
and clicking Edit.... Alternatively, you can double-click the Start question, and it automatically opens.
The Questionnaire Access and Passwords dialog is displayed::
MaxDiff Help 597
as follows:
598 SSI Web v8
Thank you for taking time to help the Navy consider ways to improve your working
environment and career development. Our goal is to make the Navy a satisfying
career for you and to improve your desire to reenlist when the decision arises.
We would like you to imagine that you are facing the decision whether to reenlist or
not--today. Please think about the issues involved, including future career planning,
family/relationship considerations, job location, and compensation.
The main dialog only shows a portion of the text, but you can click the "pencil" icon adjacent to
each text area to expand the text editing window.
To preview how the question will look when it runs on the respondent's computer using Microsoft Internet
Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved,
and no special "error checking" or "validation" is performed for the question).
your text to take greater control over the layout, font, and style later in this tutorial.
Click the OK button on the Preview window to close that window and return to the previous SSI Web
dialog.
make the text bold. You can modify the font and sizes of the font using the Paintbrush icon. Or,
you can select a different Style (selecting styles was described in the 45-minute tutorial for SSI Web).
After changing any setting, click Preview again.
After you are happy with the layout of the introductory screen, click OK to return to the Write
MaxDiff Help 599
Remember that when you add a new question to the List of Questions, it is inserted directly below the
question (or page break) that was highlighted when you clicked the Add... button. If you need to move a
question once it has been added to the List of Questions, simply highlight the question to be moved and
click the or buttons to move the questions within the list (you can also highlight a question
and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within the current questionnaire, or
even to another SSI Web study.)
Now, add a the second introductory screen on a separate page. The text for that screen is as follows:
If you were making the decision to reenlist or not today, there would certainly be
many factors that affect your decision to reenlist or not. Real life decisions are not
always easy.
We want to gain some insight into how you make this decision by asking you to
evaluate a series of tradeoff scenarios. In each scenario, we'll show you five
possible factors affecting your career in the Navy and the reenlistment decision.
We'll ask which factor (among this set of five) would make you most want to reenlist,
and which would make you least want to reenlist.
We need to ask you repeated tradeoff scenarios (involving different factors each
time) so that we can learn what is truly important to you when deciding whether to
reenlist.
Note that there already is a page break following the Start question. Highlight the page break and click
Add.... Select Text/HTML Filler as the question type with IntroMaxDiff as the question name. Click
OK to add and then edit the question by double-clicking it from the list of questions.
This introductory screen has four paragraphs and it also includes two words that need to be underlined
(most and least). Therefore, we'll show you how to how to format text in HTML. While we're showing
you how to add underlining to text, we will show you how to insert paragraph breaks as well.
Copy and paste the full text for the IntroMaxDiff question from this document into the Header 1 field.
Next, click the "pencil" icon adjacent to the Header 1 text area to expand the text editing window.
Highlight the first full paragraph with the mouse and click the "paragraph break" icon which inserts
the HTML text <p> at the beginning of the paragraph and </p> at the end of the paragraph. Repeat the
same steps for all four paragraphs.
Next, we need to underline the words most and least. Highlight "most" and click the "underline" icon
. Repeat the same for the word "least". When you are finished, the dialog should look like:
600 SSI Web v8
Click OK to close the text editor dialog and Preview to see how it is displayed in the browser window.
At this point, we are ready to add the MaxDiff questions to the questionnaire.
Adding MaxDiff Questions to the Survey
In this section, we'll add the MaxDiff questions to the survey, directly following the two introductory
screens we previously added. From the Write Questionnaire dialog, insert a page break beneath the
IntroMaxDiff question. While the page break beneath IntroMaxDiff is highlighted, click Add... and
select MaxDiff as the Exercise type. Specify a name for this exercise, such as Reenlist. (Note: you
can have more than one MaxDiff exercise within the same SSI Web questionnaire. In this tutorial, we'll
only insert a single exercise.) Click OK and the MaxDiff Exercise dialog is shown.
The MaxDiff Exercise dialog has six tabs:
Question Text (here you type the text to display directly above and below the question)
Label Text (here you specify the text to display directly next to the item text)
Items (here you specify the list of items in the exercise)
Format (you may format your questionnaire, including widths and colors from this tab)
Design (the number of items to display per set and number of sets is specified here)
Sk ip Logic (this dialog is inactive and only present for consistency with other dialogs in the SSI
Web system)
We'll begin by specifying the 20 items used in this exercise within the Items tab. Click the Items tab. A
list box is displayed which allows you to specify a new list of items to be used in this SSI Web study.
Or, you can select from an existing list of items. Since we have not yet specified a list within this study,
we'll add a new one. By default, the list name is provided as ReenlistList. Let's go ahead and use this
default name.
MaxDiff Help 601
(Note that other options such as making items on the list "Other Specify" items or "None of the Above"
options are available. These are for consistency with other dialogs in the SSI Web System, but are of no
use in MaxDiff exercises.)
From this dialog, we could click the Add... button and type the items individually, but it is easier to
copy-and-paste the list of items from an existing document. Copy-and-Paste the list of items by
highlighting the items below...

...and pressing Ctrl-C to copy them into the clipboard, and then switching to the Items tab within the SSI
Web System and clicking the Paste list member(s) from the clipboard icon. The 20 items are all
automatically inserted within the list.
Next, we'll specify the text to display above the MaxDiff sets into the Question Text tab. Click the
Question Text tab. Some default text is available within this question, but we'll replace it with wording
more appropriate for our study. Paste the following text into the Header 1 field:
Imagine you were making the decision to reenlist in the Navy today.
Which of the following four factors would make you most and least likely to want to
reenlist?
Insert paragraph breaks and underlining as we showed you earlier in this tutorial. Delete the wording in
Header 2 and the Footer.
We are almost finished formatting the basics for our MaxDiff exercise. Click the Label Text tab, and
type the labels to display over the radio buttons respondents click to indicate best and worst items. The
text to use is as follows:
Best label: Makes me <u>Most</u> want to reenlist

Worst label: Makes me <u>Least</u> want to reenlist
602 SSI Web v8
Next, we need to tell SSI Web how many MaxDiff questions (sets) to display in the questionnaire. Click
the Design tab. Basic design settings for your MaxDiff exercise are displayed:
You may recall that we previously decided that each set would present 4 items at a time and that the
respondent would receive 20 questions (so each item was shown four times on average). Specify 4 in
the Number of Items per Set (Question) field. Specify 20 in the Number of Sets (Questions) per
Respondent field.
At this point, we might preview the MaxDiff question to see how we are doing so far. Click the Preview
button, and you may see something like this:
Note that by default the preview displays the first four items in our list therefore some of the items that
we wanted to prohibit are being displayed together. We can add the prohibited combinations under this
same Design tab.
Inserting prohibitions is a capability provided within the Advanced Settings on this dialog. (After retiring
the Preview dialog) click Show Advanced Settings to reveal additional controls, including the
Prohibitions... button.
MaxDiff Help 603
Click the Prohibitions... button. We need to prohibit the following pairs of items for our exercise:
Prohibit this item... ...from appearing with this item.

3-year reenlistment obligation 5-year reenlistment obligation
Location guarantee for next assignment Both location and duty guarantees for next
assignment
Duty guarantee for next assignment Both location and duty guarantees for next
assignment
Live in 2-person barracks when in port Choice between living in 2-person barracks or 4-
person barracks when in port
Live in 4-person barracks when in port Choice between living in 2-person barracks or 4-
person barracks when in port
$50 per month pay increase when out at sea $75 per month pay increase when out at sea
Current reenlistment bonus ($2,500) Increased reenlistment bonus ($5,000)
Current reenlistment bonus ($2,500) Increased reenlistment bonus ($10,000)
Increased reenlistment bonus ($5,000) Increased reenlistment bonus ($10,000)
Click an item to prohibit from the list on the left, then click the item to prohibit from the list on the right.
Continue picking items until all prohibitions have been specified. Click the OK button when finished.
So that the designer runs quickly for the purposes of this tutorial, specify 30 iterations.
(Note: specifying prohibitions can cause the design algorithm to process much more slowly than if no
prohibitions are specified. With the 9 prohibitions in this example, it tak es more than 10 minutes to
attempt 1000 iterations of the design. Yet, the designs found with even 30 iterations are extremely good
quality, so we recommend reducing the number of iterations attempted to 30 for the purposes of this
tutorial.)
Now, we are ready to "generate the design." The design is all the combinations of items that
respondents will see across their 20 choice sets (questions).
The sets need to be chosen carefully, so each item appears about an equal number of times, and that
each item is shown about an equal number of times with every other item (non-prohibited item, that is).
Click Generate Design, and the software tries 30 separate attempts (recall we reduced the attempts
from 1000 to 30 to save time in this tutorial) to create a balanced experiment (using a different random
number start each time). The best attempt is selected, and the results are shown to you in a design
report.
When the computation finishes, you are shown a report (which you have the opportunity to save to a text
file if you wish).
We'll explain the report, one step at a time. First, design specifications are reported:
604 SSI Web v8
Number of Items (Attributes): 20

Number of Items per Set: 4
Number of Sets per Respondent: 20
Number of Versions: 300
Number of Iterations: 30
Random Number Seed: 1
Number of Prohibitions: 9
Iteration 20 was chosen
Above, we see that the software made 30 total attempts (iterations) to generate a design plan, and the
20th attempt was chosen as "best." In total, the software created 300 versions of the questionnaire
(where each version features different item combinations within the sets) to display to respondents.
Multiple versions of the questionnaire are generated to improve the stability of the item scores and also
to reduce psychological order and context effects that might occur if every respondent received the same
combinations of items. Although you plan to interview 1000 or more respondents, it would probably be
overkill to generate even more unique questionnaire versions. (Some might argue that 300 versions is
already overkill!) Additional versions would provide imperceptibly little measurable benefit, though the
software lets you specify up to 999 versions. Each respondent will receive one of these versions of the
questionnaire, and many of the questionnaires will therefore be repeated.
Below, the one-way balance is reported. This shows how many times each item is displayed across all
300 versions of the questionnaire plan, where each version had 20 sets displaying 4 items at a time. If
the design was perfectly balanced, each item would be displayed 1200 times. As can be seen, the
design is indeed perfect with respect to one-way frequencies:
One Way Frequencies:

Item Times Used
----------------
1 1200
2 1200
3 1200
4 1200
5 1200
6 1200
7 1200
8 1200
9 1200
10 1200
11 1200
12 1200
13 1200
14 1200
15 1200
16 1200
17 1200
18 1200
19 1200
20 1200
Mean = 1200
Std Dev. = 0
The quality of the design is summarized as the standard deviation across the 20 reported frequencies, or
0 is this case (no deviation from perfection). This number is used to classify the quality of the design for
multiple attempts (iterations) the software makes. Attempts that demonstrate a lower standard deviation
across the frequency counts are deemed better.
MaxDiff Help 605
Next, the report summarizes how many times each item appeared together with every other item within
sets. Ideally, each item would appear an equal number of times with every other item. You may recall
that we prohibited nine combinations from occurring, so there cannot be perfect balance. These
prohibited combinations are shown with 0 frequency in the table below (for example, item 1 shown with
item 2), highlighted in red.
Although the balance in the non-prohibited cells is not nearly as good as the one-way counts (the nine
prohibitions certainly inhibited the MaxDiff designer from finding a balanced design), the results are still
quite acceptable in practice. How would you know if you were in deep trouble due to prohibitions? If you
have too many prohibitions, you would receive an error message reporting that it is impossible for the
designer to ensure connectivity across items in the study. If you have to prohibit a single item from
appearing with more than half of the other items in the study, then this may be excessive. Again, we'd
like to stress that it is much more important that the one-way frequencies be nearly balanced than to
achieve approximate balance in the two-way frequency table.
(Note for advanced users: We were interested in the potential loss in design efficiency resulting from
these nine prohibitions, so we performed a simple test. We assumed 250 respondents receiving this
study as just designed versus the same study in which there were no prohibitions. The design with no
prohibitions had perfect one-way balance, and nearly perfect two-way balance (low of 189 and high of 190
counts per two way cells). Upon estimating MNL parameters and studying the standard errors, we
determined that the difference in overall design efficiency between these two designs was only 1%. For
those familiar with conjoint analysis, the important point is that prohibitions are much less detrimental to
MaxDiff designs than conjoint designs. That is because prohibitions in conjoint are within-concept
whereas prohibitions in MaxDiff are between-concept. And, the nine prohibitions in MaxDiff eliminate a
much smaller proportion of the full-factorial combinations than would nine prohibitions in a typical conjoint
plan where each attribute has three to five levels.)
Finally, the report shows how often each item appears in the four positions within sets (top, bottom,
etc.). This type of balance has no bearing on statistical design efficiency. But, balance here helps
reduce psychological order effects (e.g. we wouldn't want a given item to always appear in the top
position each time). The positional balance for our design is nearly perfect:
606 SSI Web v8
Positional Frequencies:
| Pos. 1 2 3 4
----------+-------------------------------------
Item 1| 300 300 300 300
2| 300 301 299 300
3| 300 300 300 300
4| 300 299 299 302
5| 301 299 300 300
6| 300 300 301 299
7| 300 300 300 300
8| 300 300 300 300
9| 300 301 299 300
10| 299 300 301 300
11| 300 299 301 300
12| 300 301 300 299
13| 301 300 300 299
14| 300 300 300 300
15| 300 300 301 299
16| 300 300 300 300
17| 300 300 300 300
18| 299 300 300 301
19| 300 300 300 300
20| 300 300 299 301
Mean = 300
Std Dev. = 0.59161
At this point, we have the basics programmed for this MaxDiff study. You can click OK to return to the
Write Questionnaire dialog, where you'll see the twenty MaxDiff questions automatically inserted within
your project:
MaxDiff Help 607
Inserting the "Progress" and "Finish" Questions
When respondents have completed five MaxDiff questions, you had wanted to display a screen that
says:
ProgressCounter:
Good work so far. We know that some of these tradeoffs are challenging.
We need to ask you to evaluate 15 more sets so that we can better determine how you make
your reenlistment decision.
Insert a new Text/HTML Filler question by highlighting the Page Break directly beneath the Reenlist_5
question, and then clicking Add.... Select Text/HTML Filler as the question type and name the
question ProgressCounter. Copy-and-paste the text for this question from this document and format
608 SSI Web v8
the text into paragraphs. Preview the question to make sure it looks as expected. Click OK to return to
the Write Questionnaire dialog, and insert another page break directly following this ProgressCounter
question.
If you would like, you can insert similar "progress screen" reminders after the 10th and 15th MaxDiff
questions.
Last, we need to insert the final screen in which we tell respondents that they have completed the survey
and link them to the Navy's official site.
Finish:
Thank you very much for your help. Your data will be used to help Navy planners as they work to
make the Navy an even better career choice.
Click the Next button to finish submitting your data and to return to the Navy's official website.
To do this, scroll to the bottom of the list of questions on the Write Questionnaire dialog. Highlight the
final question in the list (Reenlist_20) and insert a page break. Then, click the Add... button and specify
a Text/HTML Filler question named Finish.
Copy-and-paste the text for this question from this document, implement any formatting you desire.
Click OK to return to the Write Questionnaire dialog.
Next, insert a page break directly following the Finish question. Then, insert a new Terminate/Link
question directly following that entitled FinishLink. No text should be shown on this page. This page
simply redirects respondents immediately to the Navy's website. While editing the FinishLink question,
click the Settings tab. Check the Terminate Respondent box indicating that respondents seeing this
page should be terminated and marked as completed records. Then, check the second Link to a
Questionnaire or Website box and type within the provided field so that the respondent is sent to http://
www.navy.mil. When you are finished, that dialog should look like:
MaxDiff Help 609
It may be helpful at this point to review how SSI Web breaks the survey into separate pages, and some
basic global settings that affect the look and functionality of your SSI Web questionnaire. We also want
to add a progress bar to our survey. The Write Questionnaire dialog also displays the page breaks that
are currently set for your questionnaire.
Return to the Write Questionnaire dialog. This dialog shows how the various questions we've specified
(or that SSI Web has automatically added to the list) are arranged across different pages. Page breaks
are indicated by "---------------<Page>".
From the Write Questionnaire dialog, click the Survey Settings icon (alternatively, click
Compose | Survey Settings...). This dialog controls global aspects of the questionnaire (meaning that
the settings here control the general behavior of the questionnaire rather than page- or question-specific
aspects.) Examples include the fonts and colors used in the questionnaire, the wording for the "Next"
button, and the global header and footer that can appear at the top or bottom of each page of the survey.
The details of this dialog are beyond the scope of this unit, but you can learn more about the items on
the dialog by pressing the F1 key. However, we do wish to insert a progress bar. This is easily done by
clicking the Progress Bar tab, and checking the Automatically show the progress bar box with the
position selected as Page Footer.
Save your changes and exit the Write Questionnaire dialog.

First, let's select a style from the Styles tab. Select a style you wish to use, and click Use Selected at
On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Reenlistment
610 SSI Web v8
Local Test Server
Locally.
MaxDiff Help 611
examine the resulting Counts data or perhaps even compute individual-level scores using the Analysis |
MaxDiff Scores dialog (assuming you have enough data) to make sure the data at least have face
validity.
At this point, you would set up the study on your server (or use the hosting service provided by Sawtooth
Software). Server setup is beyond the scope of this tutorial, but is covered in this SSI Web manual. You
would invite the respondents to complete the survey (probably using an email message with a clickable
link).
Data Management
SSI Web includes an Admin Module so that you can monitor or access your project from any computer
connected to the Web. The Admin Module is password protected, and your passwords for access are
To download the data for our study, you would browse to the administrative module for your study on the
web site (again, we are speaking hypothetically, as for this tutorial study there is no such site set up).
Once at the administrative module, you would download your data (by clicking Download Data from the
main menu), making sure to save your data (named studyname_data.sqlite, where STUDYNAME is your
study name) to the same folder on your hard drive in which you developed the SSI Web project.
Alternatively, from the SSI Web interface, you can simply click Field | Download Data... to download
your data without logging into the Admin Module.
Once you have downloaded the data, you are ready to analyze the data by Counting analysis (Analysis |
MaxDiff Counts) or through a more sophisticated methods called hierarchical Bayes analysis through
clicking Analysis | MaxDiff Scores.
Analysis: Counts and Computed Scores
At this point, we would like you to open the Tutorial study which contains hypothetical MaxDiff data for
you to analyze. Click Help | Sample Studies | Sample MaxDiff.
We'll assume that you have collected data over the web and have downloaded the data to the folder on
your hard drive containing the study files (as described in the previous section).
A quick summary method called "counts" is often used for summarizing respondents' preferences.
Counting analysis simply looks at how many times an item was available within sets, and counts how
many times it was chosen either best or worst. The percent of times an item was chosen as best or
worst (when it was available) is termed the "count proportion."
Run counting analysis for the tutorial data set by clicking Analysis | Calculate MaxDiff Counts.... The
MaxDiff Counts dialog is displayed. It has various settings, including the ability to select which MaxDiff
exercise to use (we only have one in our study) and to select which sets to include in the analysis (we'll
use all sets). Click Compute Counts..., and after a few moments of computation, a counts report is
displayed:
MaxDiff Counts Analysis
MaxDiff Exercise Reenlist

Sets Included All
Respondents Included Completes Only
612 SSI Web v8
Total Number of Respondents 250

Total Number of Sets 5000
Number Times Best Times Worst
Item of Times Selected Count Selected Count

Number Shown Best Proportion Worst
Proportion
3-year reenlistment obligation 1 999 310 0.310 177 0.177

5-year reenlistment obligation 2 1000 72 0.072 561 0.561
Location guarantee for next assi 3 999 191 0.191 286 0.286
Duty guarantee for next assignme 4 1000 202 0.202 283 0.283
Both location and duty guarantees 5 1000 361 0.361 135 0.135
Get promoted 6-months sooner than 6 1000 183 0.183 307 0.307
Live in 2-person barracks when in 7 1000 252 0.252 231 0.231
Live in 4-person barracks when in 8 1000 208 0.208 276 0.276
Choice between living in 2-person 9 1000 294 0.294 186 0.186
$50 per month pay increase when o 10 1000 230 0.230 271 0.271
$75 per month pay increase when o 11 1000 246 0.246 225 0.225
Half of on-job time using your sk 12 1001 185 0.185 311 0.311
Three-quarters of on-job time usi 13 1001 199 0.199 251 0.251
Time off each week allowed for on 14 1000 190 0.190 309 0.309
Receive online career counseling 15 1000 234 0.234 268 0.268
Receive in-person personal counse 16 1001 231 0.231 238 0.238
Receive phone-based personal coun 17 1000 195 0.195 279 0.279
Current reenlistment bonus ($2,50 18 1000 276 0.276 189 0.189
Increased reenlistment bonus ($5, 19 999 386 0.386 110 0.110
Increased reenlistment bonus ($10 20 1000 555 0.555 107 0.107
The report shows results for 250 respondents and 5000 sets (remember, each respondent received 20
choice sets; 250 x 20 = 5000). You may recall that we had originally generated 300 questionnaire
versions for respondents to potentially complete. We only interviewed 250 respondents, so not all
questionnaire versions were used. However, we see from the "Number of Times Shown" column that
each item was shown almost exactly the same number of times (subsets of questionnaire versions are
also carefully balanced). In the "Times Selected Best" column, we see how many times each item was
selected best (across all sets and respondents). Dividing the "Times Selected Best" by the "Number of
Times Shown" results in the "Best Count Proportion." The higher the proportion, the more impact this
item has on reenlistment.
Counts proportions are a good way to get a quick (top-line) look at the data. This method of analysis is
very intuitive. But, we don't suggest you use counts results in final reporting. Because of prohibitions in
your design or slight imbalances, the data are not as accurate as when using the more time-consuming
HB estimation routine, automatically provided under Analysis | Estimate MaxDiff Scores - HB
(recommended).... Also, the counts probabilities tend to be artificially "smoothed" relative to the final
estimated scores where we see larger differences between items.
The item selected best most often was "Increased reenlistment bonus ($10,000)." Thus, we can see
that these sailors (at least our hypothetical ones) pay most attention to monetary rewards. Even the
$5,000 reenlistment bonus tends to trump every other issue. If you are interested, you can also examine
the proportion of respondents choosing each item as worst. This data set was generated with artificial
symmetric properties, so there is no difference between bests and worsts other than random noise.
However, this isn't always the case with real data sets, where respondents may not use a consistent
scoring approach when judging bests and worsts.
Let's turn our attention to the more powerful and accurate method of analysis offered in the MaxDiff
MaxDiff Help 613
System: hierarchical Bayes (HB). A typical data set may take about 10 to 15 minutes to run using HB
(large data sets may take hours!). HB estimates a set of scores for each respondent. These scores
may be saved to an Excel-compatible file. How HB actually works is beyond the scope of this tutorial,
and we refer you to technical papers on HB on our website, www.sawtoothsoftware.com.
Click Analysis | Estimate MaxDiff Scores - HB (recommended) to run the HB estimation routine and
estimate scores for each respondent. The MaxDiff Scores dialog is shown. As with the Calculate
MaxDiff Counts dialog, you can select which MaxDiff exercise to use and which sets to include.
Click Prepare for Scores Computation.... First, the software reads the data set and prepares it in a
more compact way for more efficient processing. As part of this step, the software reports how many
respondents it has found and how many times items were chosen in each position of the question.
** Data Summary **
Total respondents = 250
Total 'Best' choices = 5000
Total 'Worst' choices = 5000
Average sets per respondent = 20.00
'Best' responses by position:

(1) 24.38%
(2) 26.34%
(3) 25.08%
(4) 24.20%
'Worst' responses by position:

(1) 24.76%
(2) 25.28%
(3) 24.78%
(4) 25.18%
Click Compute Scores, and the estimation of scores using hierarchical Bayes (HB) begins. You do not
need to have any special training in statistics to consistently achieve good results with this analysis. HB
uses an iterative method that requires many thousands of iterations to stabilize and complete the run.
We have chosen defaults to govern the HB run that should work well for most every situation. Advanced
users can modify the settings if they desire.
Partway through the estimation, the MaxDiff Scores dialog may look something like this:
614 SSI Web v8
This display shows how well the estimated scores are stabilizing over the planned thousands of
iterations. The graphic plots the estimated scores at each iteration of the process. The estimates first
all start at 0, and then trend toward their final values (which may be either negative or positive). Once the
process has "converged" the estimates will tend to randomly "wobble" up and down, but there should be
no noticeable trend remaining.
The graphic is divided into two regions: a gray region (iterations prior to assumed convergence) and a
white region (iterations after assumed convergence). Once the iterations cross into the white region, you
should look for no noticeable trend in the data. The plotted scores should just continue to wobble
randomly up and down, with no strong and sustained move in any specific direction. If there is
noticeable trend after the plotted values are in the white region of the graphic, then you should repeat the
estimation using more iterations (under the Advanced button).
(Note that for sake of brevity we didn't include "constraints" in this estimation to force the scores for
some items to be higher than others, such as $10,000 bonus higher than $5,000 bonus. You do not
need to enforce such constraints during estimation, though some researchers choose to do so.)
When the estimation finishes, the graphical plot of the estimation remains on the screen. Click the
button stating: The estimation is complete. Click here to close.
When you click the "close" button, a report is given which resembles a spreadsheet, with multiple
"sheets" that can be accessed by clicking the tabs at the bottom of the dialog labeled "Summary,"
"Rescaled," and "Raw." The first (Summary) sheet contains a summary of the final scores, 95%
confidence intervals,and other information regarding the settings for your run:
MaxDiff Help 615
The second (Rescaled Scores) and third (Raw Scores) sheets contain the scores for each respondent,
including a fit statistic. Most researchers will pay most attention to the Rescaled scores as they are
easiest to interpret and present to others. These individual-level item scores are positive values ranging
from 0 to 100 that sum to 100. This approach has the valuable property of ratio-scaling. That is to say,
an item with a score of 20 is twice as important (or preferred) as an item with a score of 10.
Respondents who answer randomly will generally have low fit statistics. We have provided guidelines (fit
statistic thresholds) for identifying "bad" respondents elsewhere in this documentation. You can use the
suggested cutoff rules to discard respondents that are most likely answering randomly.
Click the Save As... button to save this report. Two formats are offered:
Microsoft Excel format --an Excel Workbook .XLS file is saved with multiple sheets (one for
each of the three sheets displayed in this dialog)
.CSV (Comma Delimited) --only the current displayed sheet is saved to the file. To save all
information from the three sheets to .CSV format, select each sheet separately and save each
to a unique .CSV file.
616 SSI Web v8
4.2 Creating a MaxDiff Experiment

Each SSI Web questionnaire may include one or more MaxDiff experiments (a series of questions
designed in a careful manner to lead to the efficient estimation of item scores). (Remember, even if you
are not asking the "worst" half of the MaxDiff question or if you are doing a method of paired
comparisons study, we still refer to the experiment in the software as "MaxDiff.")
Each MaxDiff experiment uses a list of items. This list may be a pre-defined (fixed) list, or it can be built
dynamically for each respondent (a constructed list). If a constructed list, it must have a pre-specified
number of items.
To add a MaxDiff experiment with a list of items to measure, from the Write Questionnaire dialog, click
Add | MaxDiff Exercise. Specify a name for the exercise (starting with a letter and having no more than
25 total numbers or letters). Click OK and the MaxDiff Exercise dialog is shown.
The MaxDiff Exercise dialog has six tabs:
Question Text: Here, you specify the text that appears above and below the MaxDiff questions.
Label Text: Here, you specify the labels that appear above the columns associated with "best"
and "worst" (or "best" only, if you are not asking respondents to identify the "worst" item in the
set).
Items: This tab lets you add or delete items from the list used in the MaxDiff exercise. Please
note that you can use any predefined (fixed) list, or you can select a constructed (dynamic) list
for use in MaxDiff experiments.
Format: The layout style and colors are specified on this dialog.
Design: Here, you specify how many items to show per question, how many questions to ask,
any level by level prohibitions, and other controls that govern the experimental plan.
Sk ip Logic: This tab is not active, and is available only for consistency with other parts of the
SSI Web software.
MaxDiff Help 617
4.2.1 Question and Label Text

Preparing Respondents for the Repetitive Task
Prior to asking respondents to complete a MaxDiff (or Method of Paired Comparisons exercise), it is
helpful to prepare them for what they are about to do, and why. For many respondents, the repeated
sets may seem annoying or redundant. If they don't grasp the reason for completing a dozen or more
sets and especially if they perceive no end to the questions, they may abandon the questionnaire or
provide data of questionable quality.
Thus, we recommend preparing respondents for the repetitive nature of the task. You might use text
such as:
"We think you'll find this next section both interesting and perhaps a bit challenging. We are going
to display a number of tradeoff scenarios. In each scenario, we'll show you five features of
restaurants. We'll ask which feature (among this set of five) is the most important and which is
least important to you when selecting a restaurant. We need to ask you repeated tradeoff
scenarios (involving different features) so that we can learn what is truly important to you in a
restaurant."
After the fifth set, you might insert a Text/HTML Filler question that thanks the respondent for the work
they've done so far, and encourages them to continue. You might use text such as:
"Thank you for your effort so far. We know some of these tradeoff questions don't have easy
answers. We need to ask you some more questions like these so we can further learn how you
evaluate restaurants."
After the tenth set, you might state that there are 10 more trade-off questions to go, and then the
respondent will be finished with this section. After 15 sets, you might again state that there are just 5
more questions to go.
Again, the overall principle is to prepare the respondents for the task, explain why you need to ask what
may appear to be repetitive questions, and give them some indication of progress as they move through
the questionnaire.
MaxDiff Question Text
MaxDiff lets you control the text that appears within the question. The text positions within a MaxDiff
question are as follows:
618 SSI Web v8
Default text for the MaxDiff style question is provided. However, you may wish to ask only the first half of
the MaxDiff question (the "best" selection) without asking respondents to identify the "worst" item. Also,
MaxDiff supports Method of Paired Comparison (MPC) experiments, where respondents choose between
items shown two at a time.
Edit the text so that it is appropriate for your situation. This same question text is used for all MaxDiff
questions in this exercise.
Here are some suggested texts, where you substitute appropriate wording for the red "Xs":
Questionnaires Focused on "Importance"
MaxDiff Questionnaire focused on "Importance":
Header 1: Please consider how important different features are when selecting a XXXXXXX to
purchase.
Header 2: Considering only these X features, which is the <u>Most Important</u> and which is
the <u>Least Important</u>?
Footer: Click the "Next" button to continue...
Best Label: Most<br>Important
Worst Label: Least<br>Important
MaxDiff or MPC Questionnaire focused on "Importance" (without asking respondent to indicate the
"worst"):
Header 1: Please consider how important different features are when selecting a XXXXXXX to
purchase.
Header 2: Considering only these X features, which is the <u>Most Important</u>?
Best Label: <Empty>
Worst Label: <Empty>
Questionnaires Focused on "Preference"
MaxDiff Questionnaire focused on "Preference":

MaxDiff Help 619
Header 1: Please consider different features of XXXXXXX that you might purchase.
Header 2: Considering only these X features, which is the <u>Most Desirable</u> and which is
the <u>Least Desirable</u>?
Best Label: Most<br>Desirable
Worst Label: Least<br>Desirable
MaxDiff or MPC Questionnaire focused on "Preference" (without asking respondent to indicate the
"worst"):
Header 1: Please consider different features of XXXXXXX that you might purchase.
Header 2: Considering only these X features, which is the <u>Most Desirable</u>?
Best Label: <Empty>
Worst Label: <Empty>
To Repeat or Not Repeat the Header Text
Some researchers may find that repeating the Header text each time seems overly redundant. A more
concise layout for the MaxDiff tasks can be presented if the Header text doesn't appear each time. If
you want to present the Header text initially, but delete it from subsequent questions, you might consider
leaving the Header text blank, but then add Text/HTML Filler questions (containing the text you wish to
display directly above the MaxDiff questions) directly above the first few MaxDiff tasks.
620 SSI Web v8
4.2.2 Question Format

The Format tab on the MaxDiff Exercise dialog provides many options for formatting your MaxDiff or
Method of Paired Comparison (MPC) questions. We suggest you experiment with the settings and click
the Preview button to view the effects.
There has not been much research conducted on the best layout for MaxDiff questions. We provide
different layouts here for your experimentation and use. If these options do not give you the exact look
you need for your question, you can customize your own MaxDiff layout using Free Format and your own
scripted HTML.
One significant methodological question is whether to ask for both "best" and "worst" (when displaying
three items or more per set) or whether to omit the "worst" question.
Should We Ask for "Worsts"?
MaxDiff allows researchers to ask for "best" and "worst" choices within subsets of items (set size >=3),
or to ask only for "bests." Collecting both bests and worsts contributes more information. However, it
has been shown that the scores resulting from best choices may differ (statistically significant
differences) from those developed only using worst choices. However, the results tend to be quite similar
between bests and worsts. There is some debate among leading academics regarding the statistical
properties of "worsts" and whether including both bests and worsts is appropriate. We at Sawtooth
Software do not know which approach is best (to ask for worsts or not). It would seem to depend on the
purpose and context of the research. Asking only for "bests" is theoretically more sound, but asking for
"worsts" seems to offer practical value. We hope that offering flexibility in this software will lead to more
experimentation in this area.
Anchored Scaling: Direct and Indirect Methods
Respondents indicate which items are relatively better (or worse) than others within standard MaxDiff
questionnaires. Thus, the scores are estimated on a relative scale, without any indication that the items
are good or bad, important or unimportant, in an absolute sense. So, a respondent who dislikes all the
items could have final HB scores that look essentially identical to another respondent who likes all the
items a lot. Including a broad range of items in your study (from generally low to high importance/
preference) helps reduce these problems. Inserting an item that has concrete monetary meaning or
conveys a status quo can also be used to anchor the scores. MaxDiff's limitations related to the relative
scaling of items have been debated for years now, and two additional approaches to anchored scaling
have emerged: the Indirect and Direct methods. Read more about Anchored Scaling.
MaxDiff Help 621
4.2.3 Developing a List of Items

Choice studies such as those supported by the MaxDiff software, which include,
MaxDiff
Method of Paired Comparisons (MPC)
Choices from sets of three (triples), sets of four (quads), etc.
usually involve at least eight items (i.e. features, or attributes). With seven items or fewer, a designed
choice experiment may risk overkill. When dealing with so few items and with reasonably sophisticated
respondents, perhaps an allocation-based task would be sufficient. Even so, we could imagine
conducting a paired comparison exercise with as few as four or five items, if the demands of the project
required a simple respondent task (appropriate for people of all educational and cultural backgrounds)
and avoidance of the response-style bias typically seen with traditional ratings scales.
On the high end, we at Sawtooth Software have been involved in a paired comparison experiment that
involved 160 employment-related benefits/conditions. Sample size was in the thousands of respondents,
and we pooled information across respondents for estimation. We're certain other researchers have
conducted even larger studies than that.
Specifying a List of Items
On the Items tab of the MaxDiff Exercise dialog, you can select an existing list to use in the MaxDiff
exercise, type a new list by clicking the Add... button, or copy-and-paste a list of items from another
program by clicking the Paste list member(s) from the clipboard button . Depending on your
license, you can specify up to 500 items. Items can involve either text, graphics, or a combination of
text and graphics.
Recommendations for Item Descriptions:
Item text should be clear and succinct. Anything you can do to help respondents process
information more quickly and accurately is helpful.
When possible, items should be as specific and actionable as possible. For example, it would
probably be more useful to describe a level as "Fuel efficiency increased from 5 liters/hour to 4.5
liters/hour" instead of "Improved fuel efficiency." That said, we realize that such quantitative
certainty doesn't make sense for many studies and items. For example, in an image study, we
may very well ask respondents to select which picture of an automobile most "Makes me feel
successful when I drive it." How one defines "successful" in a concrete or quantitative way is
probably not useful to the goals of the study.
Items can be made to be multi-level and mutually exclusive. For example, in a study of fast-
food restaurant features, rather than ask about "Fast service" generically, you might create three
separate items that probe specific levels of fast service:
Order and receive food within 3 minutes

622 SSI Web v8
When considering a fast-food restaurant, it would seem rational that all respondents would prefer
faster service to slower service. Therefore, it makes no sense to include more than one of these
levels within the same set. You should specify prohibitions between these levels (from the Design
tab) so that they never appear compared with one another. It is also possible during estimation to
require that faster levels of service receive a higher score than slower levels of service (by
specifying monotonicity constraints).
Another example of multi-level, mutually-exclusive levels was shown in the award-winning paper at
the 2004 Sawtooth Software Conference by researcher Keith Chrzan. Using a MaxDiff
methodology, he studied price sensitivity for options for automobiles. Each option was available at
multiple prices, so he could plot the resulting scores as relative demand curves. (Of course, the
same option never appeared within the same set at different prices.)
You may mix multi-level items with single-level items within the same study. However, we
should note that the increased frequency of multi-level attributes in the design might bias
respondents to pay more attention to that dimension. But, we are not aware of any research yet in
MaxDiff or MPC that substantiates that concern.
Graphics are possible! Remember, SSI Web lets you use graphics as list items.
It is sometimes helpful to create a "reference" level with monetary meaning. Some

researchers have found it helpful to associate one of the levels with a specific monetary equivalent.
For example, in studying job-related conditions and benefits, it might be useful to include a level
that says: "Receive an immediate $500 bonus." Or, if studying improvements to a product, we
might include a level that says: "Receive a $20 off coupon." In both cases, we can associate a
specific item score with a specific and immediate monetary gain. The scores for other levels may
then be compared to this monetary-based reference level (the reference item's scores could also be
subtracted from all other scores, to anchor the scores around the 0-based reference point). Also
note that it is possible to make the monetary reference point multi-leveled with mutually exclusive
levels, such as "$5 off coupon," "$10 off coupon," "$15 off coupon," "$20 off coupon." This provides
multiple monetary-grounded reference points for comparing the scores of other items in the study.
Status Quo level to establish reference point (anchor). In other studies, such as those
studying potential modifications to existing products, services, or (to be more specific) employment
conditions, it might make sense to include a reference level reflecting no change; for example, "No
change to current work environment." That way, item scores that might have a negative affect
(relative to "no change") can be identified. The score for the status quo item could be subtracted
from the other items' scores, to anchor the scores around the 0-based reference point.
Using Constructed Lists in MaxDiff Experiments
Warning: This is an advanced feature that if not used carefully can result in errors in data
collection and incorrect model results. Please test your survey thoroughly prior to fielding
the study. After collecting data, analyze the data via Counts to ensure that all items from the
parent list have been shown a reasonable number of times to the sample of respondents. If
some items were never shown to any respondents, they will be marked as having been
shown zero times (with an asterisk) and estimation using HB will be incorrect unless these
items are omitted from the analysis.
SSI Web software can build constructed (dynamic) lists based on answers to previous questions or
some other rule you define (such as randomization or always forcing a client's brand onto a list). You
may use the constructed list in later questions or even in MaxDiff exercises. When at the items tab on
MaxDiff Help 623
the MaxDiff Exercise dialog, you may select a constructed list rather than the typical pre-defined list
used in MaxDiff experiments. This powerful option even allows for the inclusion of respondent-supplied
other-specify text as an item in the MaxDiff exercise.
The key limitation for using constructed lists is that all constructed lists for respondents must have
the same length. SSI Web requires you to use SETLISTLENGTH as the last instruction in the
constructed list logic used in a MaxDiff list. The Number of Items per Set specified on the Design tab
must match the number of items specified in the SETLISTLENGTH instruction.
Warning: the SETLISTLENGTH instruction ensures that the constructed list has a fixed size
or length. If the SETLISTLENGTH instruction needs to add members from the parent list it
will not add "Other Specify" or "None of the above" type members. This can cause a
problem if there are no other members to add. Please thoroughly test the constructed list in
the MaxDiff exercise prior to fielding the study.
SSI Web uses the design generated from the Design tab, but draws items from the constructed list
according to the design by position on the constructed list. The values stored for analysis (both for the
design shown to respondents and for the answer given) correspond to the original pre-defined list.
Warning: the design report displays frequencies for a design having the same number of
items as specified in the SETLISTLENGTH instruction and in a fixed order. However, each
respondent's actual items may vary, and therefore the true frequency of occurrences for each
parent item and co-occurrences of items may differ significantly from that shown in the
design report.
When estimating item scores, MaxDiff estimates (via HB) a full set of item scores for all items in the pre-
defined list, even if each respondent received only a customized subset of the items. These scores for
items not seen take on values near population averages. Depending on your reason for using a
constructed list that is a subset of the pre-defined list, this may or may not be a desired outcome.
624 SSI Web v8
4.2.4 Designing the Study

The Design Tab presents a number of options to control your study design:
Number of Items (Attributes)
In this field, you see the number of items (i.e. levels or features) to include in the exercise. This
field is not editable, but is computed by examining the current list you have specified on the Items
tab Depending on your license, you can specify up to 500 items on the Items tab.
Number of Items per Set (Question)
Generally, we recommend displaying either four or five items at a time (per set or question) in
MaxDiff questionnaires. However, we do not recommend displaying more than half as many items
as there are items in your study. Therefore, if your study has just eight total items, we would not
recommend displaying more than four items per set.
If you would like to use the Method of Paired Comparisons approach, then specify just two items
per set.
Research using synthetic data suggests that asking respondents to evaluate more than about five
items at a time within each set may not be very useful in MaxDiff studies. The gains in precision of
the estimates are minimal when using more than five items at a time per set. The small statistical
gains from showing even more items may be offset by respondent fatigue or confusion.
Number of Sets (Questions) per Respondent
We generally recommend asking as many sets (questions) per respondent such that each item
has the opportunity to appear from three to five times per respondent. For example, consider a
study with 20 items where we are displaying four items per set. With 20 total sets, we know that
each item will be displayed 4 times (assuming the design is perfectly balanced). This leads to the
following decision rule and formula:
For best results (under the default HB estimation used in MaxDiff), the suggested number of sets is
at least:
3K/k
where K is the total number of items in the study, and k is the number of items displayed per
set.
The software will warn you if the number of sets you have requested leads to each item being
displayed fewer than 2 times per respondent.
These recommendations assume HB estimation (the default method in MaxDiff). Advanced

researchers may elect to export the data (File | Export Data | Prepare MaxDiff Data Files (*.cho,
etc.)...) for analysis using Sawtooth Software's Latent Class system, or another analytical program
of their choice. When pooling data across respondents using Latent Class or aggregate logit,
you may be able to achieve quite reasonable population-level scores using even fewer sets
per respondent than these recommendations. If you use relatively few sets per respondent
(relative to the number of parameters to estimate), aggregate modeling may be the better option to
stabilize and summarize parameters across subgroups or the population.
MaxDiff Help 625
Generate Design
When you are pleased with your settings, click Generate Design to produce a design (which is stored
internally). (You can export the design to a text file if you wish using the Export Design... button
described at the end of this section.)
The design algorithm follows these guidelines:
1. Create a design that features item frequency balance. A balanced design is one in which
the one-way frequencies are nearly equivalent (how many times each level appears across
the entire design) and two-way frequencies are also nearly equivalent (how many times each
pair of items appears within the same set across the entire design). When one-way and two-
way frequencies are balanced, this is termed orthogonality.
2. Ensure that each version of the design (respondent questionnaire) features connectivity
(meaning that all items are linked either directly or indirectly), unless the Allow Individual
Designs Lack ing Connectivity option is checked. Without connectivity it becomes very
difficult for the default estimation procedure in MaxDiff (HB) to scale the items properly
relative to one another for each respondent.
3. After the designs have been generated based on the guidelines in 1 and 2, swap the order of
the items within each set so that each item appears approximately an equal number of times
in each position. Positional balance reduces or virtually eliminates order bias. Finally,
randomize the order of the tasks within each version.
The design process is repeated 1000 separate times, and the replication that demonstrates the best
one-way balance is selected. If multiple designs have the same degree of one-way balance, then we
select among those designs based on the best two-way balance. If multiple designs have the same
degree of one-way and two-way balance, then we select among those designs based on the best
positional balance. With small to moderate sized designs and no prohibitions, this usually happens
within a few seconds.
Test Design
If you have already generated a design, you may at a later point want to review the properties of the
design in terms of:
Number of versions used

Design seed used
One-way and two-way frequencies
Positional balance
Click Test Design to produce a report showing the characteristics of the design currently being used for
your MaxDiff exercise.
Advanced Settings
Note: it is not necessary to modify the default settings in this section to achieve robust results. The
default recommendations and guidelines for MaxDiff should work well for most situations. However,
626 SSI Web v8
researchers wanting to fine-tune their models or employ a different method for estimating item scores
may find the following advanced settings useful.
Number of Versions
Though it is possible to estimate scores relatively efficiently using just a single questionnaire
version for all respondents, there is practical benefit to using multiple versions (sometimes called
"blocks") of the questionnaire. With multiple versions of the questionnaire, different respondents
see different series of questions. Across respondents, this dramatically increases the variation in
the way items are combined within sets, which can reduce potential context biases (which are
usually modest).
The default is 300 questionnaire versions, which ensures a very large overall design and a great
deal of combinatorial variation in the sets. Using 300 versions is more than adequate, since
beyond a handful of questionnaire versions the further practical benefits of increased variation are
barely detectable. Since using SSI Web makes it nearly automatic to produce and use multiple
versions of the questionnaire, it makes sense to use a large number. The software allows you to
request up to 999 questionnaire versions. The two main take-aways are, 1) it makes sense to use
many questionnaire versions, 2) it is not necessary for each respondent to receive a unique
questionnaire version.
If using the separate MaxDiff designer for conducting paper-and-pencil studies, we recommend
using at least four different versions of the questionnaire. You need to find a good balance between
difficulty of study administration and design quality.
Number of Iterations
By default, the MaxDiff designer repeats its algorithm 1000 times and returns the best design found
across those passes. The quality of the design is based on the one-way, two-way, and positional
balance (in that order of precedence). For many designs, 1000 iterations can be done within a few
seconds. We encourage you to increase the number of passes, as it may enable you to find a
slightly better design. Up to 999,999 iterations may be specified.
Design Seed
Specify a value from 1 to 999,999,999. This integer (seed) is used for determining the starting point
for the design algorithm. Different seeds will yield different designs, all having approximately the
same overall design efficiency. You can try different starting seeds to see if you obtain slightly
better designs.
Favor Two-Way Balance
The MaxDiff designer by default puts primary emphasis on balancing the number of times each
item is included (one-way balance), with secondary emphasis on two-way balance (how often each
paired combination of levels appears together). If you have specified prohibitions, these two goals
are at odds with one another. The software puts more emphasis toward balancing the two-way
frequencies if you check this box. Recognize, however, that with prohibitions it may be impossible
to achieve near-perfect balance in either the one-way or two-way frequencies.
We strongly suggest checking this box when displaying two items at a time within sets (pairwise
presentation). The design algorithm will almost always produce one-way frequencies that differ very
little from when one-way balance is favored. But, the two-way frequency balance will usually be far
superior.
MaxDiff Help 627
Allow Individual Designs Lacking Connectivity
Connectivity is the property that the items cannot be divided into two sets wherein no comparison
is made between any item in one set and another item from the other. Even if creating choice sets
randomly, if respondents receive enough sets, it would be difficult for the items to lack connectivity
for any given respondent. However, if you select a design with relatively many items and relatively
few tasks per respondent, many individual versions of the questionnaire could lack connectivity.
The default for the software is to reject any version (questionnaire for one respondent) that lacks
connectivity. But in the case of many items and few sets per respondent, it may become
impossible to satisfy connectivity. If you would like to permit versions lacking connectivity (which
may be the case if you ask few questions of any one respondent, probably leading to aggregate
analysis), then you can check this box. Make sure to specify many questionnaire versions. To
ensure stable estimation, advanced users may wish to generate dummy (random) response data,
export the data for estimation under aggregate logit, and examine the size of the standard errors to
ensure reasonable precision.
Prohibitions
Occasionally, your study might include multi-level items characterizing degrees or amounts of
some feature. For example, in studying features of fast food restaurants, you may wish to include
the following three items in your study list:

When considering a fast-food restaurant, it would seem rational that all respondents would prefer
faster service to slower service. Therefore, it makes no sense to include more than one of these
levels within the same set. You should specify prohibitions between these levels by clicking the
Prohibitions... button so that they never appear compared with one another.
For those familiar with conjoint analysis, it may be heartening to learn that prohibitions are not as
damaging in MaxDiff experiments as with conjoint analysis. For example, in a conjoint study
where there are 4 prohibitions specified between two 4-level attributes, 4 of the possible 16
combinations (or 25%) are prohibited. Depending on the pattern of prohibited combinations, this
could be very detrimental. However, in a typical 20-item study, there are 1/2(20*19) = 190 possible
combinations of items (taken two at a time). Prohibiting 4 of these represents only about 2% of the
possible combinations. This will have very little impact on the overall results. Please see
Robustness of MaxDiff Designs to Prohibitions for more details.
When you compute the Generate Design button, if the software returns warnings that the
questionnaire versions lack connectivity, this is clear sign that your prohibitions are excessive (after
ruling out the possibility that you are not displaying enough sets per respondent).
Importing a Design from an External File
Some advanced researchers may wish to import their own design(s) from an external text file using the
Import Design... button. To demonstrate the required file format, consider a MaxDiff exercise with the
following specifications:
20 items in the exercise

628 SSI Web v8
4 items per set

20 sets per respondent
2 versions of the questionnaire
The design must have the following comma-delimited format (the header row with labels is optional):
Version,Set,Item1,Item2,Item3,Item4
1,1,10,1,19,7
1,2,19,17,15,3
1,3,17,10,16,6
1,4,6,20,12,13
1,5,13,2,9,5
1,6,2,9,13,11
1,7,15,18,1,14
1,8,7,3,20,8
1,9,5,6,18,9
1,10,3,12,2,16
1,11,8,3,10,4
1,12,15,7,5,2
1,13,11,16,14,19
1,14,4,19,11,12
1,15,14,11,17,8
1,16,20,15,13,4
1,17,9,1,17,18
1,18,18,8,16,6
1,19,1,5,12,20
1,20,14,4,7,10
2,1,4,11,6,15
2,2,10,13,14,18
2,3,20,10,9,1
2,4,11,20,8,5
2,5,16,13,5,7
2,6,12,19,7,17
2,7,12,14,8,17
2,8,13,18,3,2
2,9,18,7,4,11
2,10,17,4,15,16
2,11,5,14,10,15
2,12,19,2,3,8
2,13,2,15,17,20
2,14,16,12,5,9
2,15,4,2,19,10
2,16,6,11,14,3
2,17,8,7,4,6
2,18,3,1,11,19
2,19,1,9,16,20
2,20,12,9,2,14
When you import a design from an external file, the designs are stored within the system as if the
MaxDiff designer had generated them. You can click Test Design from the Design tab to compute one-
way, two-way, and positional frequencies for the design you supplied through the import process.
During data collection, each new respondent entering the survey is assigned the next questionnaire
MaxDiff Help 629
version in the design. Once all questionnaire versions have been assigned to respondents, the next
respondent receives the first design, etc.
Exporting the Current Design
Some users may wish to examine their designs or use the design details to create paper-and-pencil
questionnaires. You can export the current design to a comma-delimited format with the layout as
described in the previous section by clicking Export Design....
630 SSI Web v8
4.3 Analyzing MaxDiff Data

MaxDiff data involve choices: respondents are shown items and asked to choose among them. This kind
of data is very useful in marketing and social research applications. The analysis involves observing the
probabilities of choice for the items. More detailed explanation than given here is available within the
"MaxDiff Technical Paper," available on our website, www.sawtoothsoftware.com.
Four types of analysis are offered in the MaxDiff System:
Counting Analysis (for basic, topline results)

Individual-Level Score Estimation (recommended)
Aggregate Score Estimation via Logit (advanced)
Latent Class Estimation (advanced)
MaxDiff Help 631
4.3.1 Counting Analysis

Counting Analysis
Counting analysis takes into account how often each item was available for choice, and how many times
it was selected as best or worst. We divide the number of times each item was chosen by the number
of times it was displayed to the respondents (and available for choice). Intuitively, we are asking: "When
an item is available within a set, what is the probability that respondents select it as best or worst?" To
obtain this analysis, you first must have downloaded the data from your web server. Then, from the SSI
Web system, click Analysis | MaxDiff Counts....
Here is an example of the output:
These "counts proportions" do not add to 100% (since all items were not displayed in each set). But if
you would like, you may copy-and-paste the values into Excel and re-scale them to sum to 100. The
interpretation is the same irrespective of the scale.
If your experiment is perfectly balanced (each item displayed an equal number of times, and each pair of
items shown together an equal number of times, etc.) then the results should closely resemble (but with
"flatter" scaling) what is produced by the most sophisticated analysis method provided by MaxDiff,
described in the next section.
632 SSI Web v8
4.3.2 Individual-Level Score Estimation

MaxDiff uses a sophisticated estimation technique to produce scores for each respondent on each item.
You do not need to have any special training in statistics to consistently achieve good results. Item
scores for each respondent may be saved to an Excel-compatible file. These scores are very easy to
interpret and present to others.
The recommended statistical technique used to estimate scores for each respondent is called HB
(hierarchical Bayes estimation). This is a well-documented and trusted advanced statistical technique in
the market research industry. It is beyond the scope of this documentation to explain HB, but
information is available in technical papers on Sawtooth Software's website at
MaxDiff questionnaires provide typically too little information to stabilize scores for any one individual
tak en by itself. The HB estimation routine is able to stabilize the estimates for each individual by
"borrowing" information from the body of respondents in the same data set. Therefore, by adding new
respondents to the data (and re-computing results), the resulting scores for existing respondents will
change a little. And, you cannot obtain stable results unless a large enough pool of respondents is
available for HB estimation. Typical sample sizes start at one hundred or more respondents, though
most projects will probably involve from 300 to 1000 respondents.
If using HB analysis, it is also important that each respondent have provided enough data (relative to the
number of items in your study) to stabilize the score estimates and to avoid overfitting. For
recommendations regarding proper questionnaire length, see the section entitled Designing the Study.
Warning: Because the information from MaxDiff questionnaires is relatively "sparse," it requires a
reasonable number of respondents to obtain stable results at the individual level. If you run the
MaxDiff Scores procedure with tiny sample sizes (such as a twenty or fewer respondents), you
might see unstable (and extreme) scores for individuals.
Note: Individual-level score estimation in Max/Diff Web does not perform all the types of analysis
that some analysts may desire. As an example, some analysts may take the rescaled or raw scores
into another statistical package and perform additional analysis such as histograms to view the
distribution of scores across individuals.
Settings and Options
When you click Analysis | Estimate MaxDiff Scores - HB (recommended)..., a simple dialog is
displayed. The default settings in MaxDiff Scores - HB estimation should work well for almost every
application.
Respondent ID
If estimating utilities using Latent Class or HB, you should select the Respondent ID to use for
identifying the respondents in the utility files and segment membership files.
Respondent Filter
Filters allow you to perform analysis on a subset of the data. Categories of the sys_RespStatus
(Incomplete, Disqualified, or Qualified/Complete) variable may be selected for inclusion in the
analysis. Using the Filter Builder... additional filters may be selected from variables in your
study, or new variables that you build based on existing variables or combinations of existing
variables.
MaxDiff Help 633
Sets to Include
Clicking Sets to Include area accesses a dialog in which you can specify which sets to include.
Most users will utilize all sets within analysis. However, there may be instances in which
advanced users wish to use only a subset of the full data.
Constraints
Sometimes you know beforehand that some levels rationally should be preferred to others. For
example, perhaps you included three items in your list related to end-of-year bonus:
Receive a $500 end-of-year bonus

Receive a $750 end-of-year bonus
Receive a $1,000 end-of-year bonus
Every rational respondent should prefer higher bonus to lower bonus, and you can inform the
estimation routine of this preference. Click the Constraints section if you would like to specify
known preference orders and constrain the estimated scores to conform to these expectations.
The method that MaxDiff uses to constrain the scores is called "simultaneous tying" and it is
described in the CBC/HB Technical Paper on our website.
Anchored Scaling
If you included Indirect Dual-Response Anchored Scaling questions within your MaxDiff
questionnaire, you can use the anchor information within score estimation by clicking the Include
Dual-Response Indirect Answers in Analysis checkbox. Unchecking this box means that scores
are estimated in the standard way, without considering the answers to the Indirect Anchored
Scaling questions and without estimating the scores relative to an anchor.
To use the Direct Binary Approach, click the Use Direct Binary Approach checkbox. You must
have collected additional information outside the MaxDiff questions regarding how each item (or
each of a subset of items) relates to the anchor (importance threshold). A dialog opens up with a
row for each item in your MaxDiff study. Specify the logic for each item that was rated or ranked
relative to the anchor threshold. For example, if the question RatItem1 contains a rating for item 1
and if a value greater than 3 for RatItem1 indicates that item 1 exceeds the anchor threshold, the
logic to specify for the the item 1 row is RatItem1 > 3.
Covariates
Covariates are variables outside the MaxDiff responses such as respondent classifications (small
company, large company, high income, low income, etc.) or continuous variables (household
income, preference score on variable x, etc.) that might be useful to leverage within HB estimation.
Without covariates, population mean item scores (for respondents included in analysis) are
referenced when estimating the scores for each respondent. When using covariates, cohort-
specific mean scores are referenced. This holds the potential benefit that respondents "borrow"
information primarily within their cohort rather than generally across the sample. Using covariates
tends to slow the computation speed, and too many covariates may cause your system to grind to
a near halt. Although the use of covariates seems to offer promise for better utility scores, many
research papers have reported that covariates do little to nothing to improve the predictive validity of
the scores. Even so, discrimination between respondent groups (especially on the dimensions of
the covariates) does increase with the application of covariates.
If you click Settings... and then click the Settings category at the left of the subsequent dialog you can
review (or modify) the settings that will be used in your score computation. We'd like to emphasize that
most users will not need to investigate this area. (For more information about the controls used in this
area, please refer to the CBC/HB Technical Paper, available for downloading from our website in the
technical papers library).
634 SSI Web v8
Number of Iterations before Using Results

The estimation routine that MaxDiff employs (HB) uses many thousands of iterations to find stable
scores prior to actually "using" (saving) the results for each respondent. 20,000 iterations is the
default, and should give you a healthy margin of safety, in the sense that the scores should have
every opportunity to stabilize (assuming appropriate design and sample size). You may change
the number of iterations if you wish.
Number of Draws to Be Used for Each Respondent

Each iteration that HB completes results in an estimate of each respondent's scores called a
"draw." From one draw to the next, the scores can vary by a random perturbation. One could plot
the distribution over many draws and we'd generally find that the draws are distributed normally.
MaxDiff averages these draws for each respondent across as many draws as indicated in this field,
and reports that average as the final raw score for each item. Theoretically, the more draws used
per respondent, the more precise the estimates. However, you will find that 10,000 draws already
provides a high degree of precision.
Prior Degrees of Freedom

This value is the additional degrees of freedom for the prior covariance matrix (not including the #
parameters to be estimated), and can be set from 2 to 100000. The higher the value, the greater
the influence of the prior variance and more data are needed to change that prior. The scaling for
degrees of freedom is relative to the sample size. If you use 50 and you only have 100 subjects,
then the prior will have a big impact on the results. If you have 1000 subjects, you will get about the
same result if you use a prior of 5 or 50. As an example of an extreme case, with 100 respondents
and a prior variance of 0.1 with prior degrees of freedom set to the number of parameters estimated
plus 50, each respondent's resulting scores will vary relatively little from the population means. We
urge users to be careful when setting the prior degrees of freedom, as large values (relative to
sample size) can make the prior exert considerable influence on the results.
Prior Variance
The default is 1 for the prior variance for each raw score, but users can modify this value. You can
specify any value from 0.1 to 100. Increasing the prior variance tends to place more weight on fitting
each individual's data, and places less emphasis on "borrowing" information from the population
parameters. The resulting posterior estimates are relatively insensitive to the prior variance, except
MaxDiff Help 635
1) when there is very little information available within the unit of analysis relative to the number of
estimated parameters, and 2) the prior degrees of freedom for the covariance matrix (described
above) is relatively large.
Random Starting Seed

Any computer algorithm that uses random numbers needs a starting "seed" value. You may set
this as an integer from 0 to 32000. When using different random seeds, the final scores will vary,
but insignificantly, assuming convergence has been reached and many draws have been used. If
you specify a "0," this uses the system's clock as the starting seed. If you want to be able to
repeat results and achieve the same answer, then you should use a specific integer greater than 0.
Responses to Include
If you asked MaxDiff questions that included both a selection of "best" and "worst," you can select
which response types to use in analysis. Researchers have noted that there can be statistically
significant differences in scores developed from bests versus worsts. This control lets you
investigate these issues if you'd like, including the ability to use answers from bests only in
computing item scores.
The Computation Run
When you click Analysis | Estimate MaxDiff Scores - HB (recommended)..., a simple dialog is
displayed. You can click the Prepare for Scores Computation button to prepare the data file and
compute the scores using your current settings. The data are prepared and you are shown a summary
report of the number of respondents included in the run:
** Data Summary **
Total respondents = 300
Total 'Best' choices = 4500
Total 'Worst' choices = 4500
Average sets per respondent = 15.0
' Be s t ' r es pons es by pos i t i on:

( 1) 25. 64%
( 2) 25. 35%
( 3) 24. 89%
( 4) 24. 12%
' Wo r s t ' r es pons es by pos i t i on:

( 1) 24. 34%
( 2) 24. 68%
( 3) 25. 25%
( 4) 25. 73%
Click the Compute Scores button to proceed with HB estimation. Depending on the size of your data
set, it can take between fifteen minutes to a few hours or more to run! HB uses an iterative method that
requires many thousands of iterations to stabilize and complete the run. We have chosen defaults to
govern the HB run that should work well for most every situation. Advanced users can modify the
settings if they desire.
While HB runs, a graphic is shown that characterizes how well the estimated scores are stabilizing over
the planned thousands of iterations. The graphic plots the estimated scores at each iteration of the
process. The estimates first all start at 0, and then trend toward their final values. Once the process
has "converged" the estimates will tend to randomly "wobble" up and down, but there should be no
636 SSI Web v8
noticeable trend remaining. Below is an example:
The graphic is divided into two regions: a gray region (iterations prior to assumed convergence) and a
white region (iterations after assumed convergence). Once the iterations cross into the white region, you
should look for no noticeable trend in the data. The plotted scores should just continue to wobble
randomly up and down, with no strong and sustained move in any specific direction. The smaller the
sample size or the more items included, the larger the wobble. If there is noticeable trend after the
plotted values are in the white region of the graphic, then you can repeat the analysis using more
iterations.
When the estimation finishes, the graphical plot of the estimation remains on the screen. Click the
button stating: The estimation is complete. Click here to close.
When you click the "close" button, a report is given which resembles a spreadsheet, with multiple
"sheets" that can be accessed by clicking the tabs at the bottom of the dialog labeled "Summary,"
"Rescaled Scores," and "Raw Scores." The first (Summary) sheet contains a summary of the final
scores, upper and lower 95% confidence bounds, as well as other information regarding the settings for
your run. The second (Rescaled Scores) and third (Raw Scores) sheets contain the scores for each
respondent (presented in two different formats), including a fit statistic.
Rescaled (Probability) Scores These individual-level item scores are positive values summing
to 100 that reflect the likelihood of items being chosen within the questionnaire (unless you are
using Anchored MaxDiff, where the probability scores are scaled differently). Most researchers
will probably use this scaling procedure, as it is easiest to interpret and present to others. This
approach has the valuable property of ratio-scaling. That is to say, an item with a score of 20 is
twice as important (or preferred) as an item with a score of 10. Click here for more details
regarding the rescaling procedure.
Raw Scores These are weights that directly follow from the MNL (multinomial logit) procedure
employed within the HB engine. The items can have positive or negative weights and are zero-
centered (the "average" item has a weight of 0). These weights are on an interval scale, which
does not support ratio operations. In other words, you cannot state that an item with a score of
2.0 is twice as important (or preferred) as an item with a score of 1.0. Advanced analysts may
choose to use these raw scores.
MaxDiff Help 637
Zero-Anchored Interval Scale If you have used Anchored MaxDiff, another tab shows you
scores for each respondent where the anchor threshold is set to 0 for each respondent. The
scores are normalized to have a range of 100 utility points for each respondent. Positive scores
are associated with items judged above the anchor threshold; negative scores are associated
with items judged below the anchor threshold.
Click the Save As... button to save this report. Two formats are offered:
Microsoft Excel format --an Excel Workbook .XLS file is saved with multiple sheets (one for
each of the three sheets displayed in this dialog)
.CSV (Comma Delimited) --only the current displayed sheet is saved to the file. To save all
information from the three sheets to .CSV format, select each sheet separately and save each
to a unique .CSV file.
(Note: the rank-order position of items based on average scores can differ between the raw and rescaled
reports. However, the changes in rank-order position are seen generally when there is little absolute
difference in scores between adjacent items. The overall interpretation of the results and main
conclusions should not differ whether using the raw or rescaled reports.)
638 SSI Web v8
4.3.3 Identifying "Bad" Respondents

Fit Statistic and Identifying Random Responders
MaxDiff displays an average "fit statistic" to the screen during HB estimation and also writes to file the
individualized fit statistic along with each respondent's scores on the items. This fit statistic ranges from
a low of 0 to a high of 1. It characterizes the internal consistency for each respondent.
It is natural to ask what minimum fit statistic respondents should have to be able to recognize more
thoughtful responses from purely random data. The fit statistic can help you determine if a respondent
has provided purely random answers and should be discarded.
The fit statistic largely depends on how many items were shown in each set. If four items are shown in
each set, then any random set of scores should be able to predict respondents' answers correctly with
25% likelihood (fit statistic=.25). If two items are shown per set (paired comparison approach), then the
likelihood given random data is 50% (fit statistic=.5). Thus we can generalize that the fit statistic should
be at a minimum 1/c, where c is the number of items displayed per set. We should hope that
respondents don't provide random answers and that the estimated scores should perform considerably
better than chance. However, the score estimation algorithm used in MaxDiff (HB) attempts to fit the
respondent's choices, even if they are just random. Thus, the actual fit we observe even from random
data is mostly above the chance rate.
The table below displays a suggested minimum fit statistic assuming we want to achieve 95% correct
classification of random responders. In developing this table, we assumed a 20-item study wherein
respondents are shown each item either two or four times across all sets.
Suggested Minimum Fit Statistic to Identify Random Responders with 95%

Correct Classification
Suggested Suggested
Items per Set Minimum Fit. Each Item Minimum Fit. Each Item
Shown Four Times to Each Shown Twice to Each
Respondent Respondent
2 .574 .634
3 .371 .402
4 .282 .305
5 .227 .247
6 .191 .208
Technical Notes: We simulated 1000 respondents answering randomly to each questionnaire, then
estimated the scores using HB: prior variance=2, d.f.=5. Both "best" and "worst" answers were
simulated for each respondent (with the exception of the 2 items case). For each simulated data set, we
sorted the respondents' fit statistics from highest to lowest and recorded the 95% percentile fit. If asking
only "bests," because the amount of information for each respondent is reduced, the Suggested
Minimum Fit would be higher.
Additional Notes: The table above was created using standard MaxDiff, not anchored MaxDiff. Anchored
MaxDiff would lead to different norms regarding fit for identifying "bad" respondents. If using anchored
MaxDiff, you can estimate the results using HB without the anchor, so that you may use the
MaxDiff Help 639
recommendations above.
640 SSI Web v8
4.3.4 Aggregate Score Estimation via Logit Analysis

Note: Aggregate Logit has been used for more than three decades in the analysis of choice data.
It is useful as a top-line diagnostic tool (both to assess the quality of the experimental design and
to estimate the average preferences for the sample). Logit can be quite useful for studies in which
you are studying very many items and where respondents cannot see each item enough times to
support individual-level analysis of scores via HB. You can compute aggregate logit scores for
the entire sample, or for different respondent groups (e.g. large companies vs. small companies).
When you run Logit (Analysis | Estimate MaxDiff Scores-Other Methods... | Logit...), the results are
displayed in the report window and utility scores are saved into a subfolder within your project directory.
You can weight the data, or select subsets of respondents or tasks to process. A single set of utility
scores (representing an average across the included respondents) is estimated that provides a maximum
likelihood fit to the respondents' choices. Aggregate logit pools all respondents within the analysis and
treats them as if a single respondent had completed a very long interview.
The utility score results are reported using three different rescaling transformations:
Raw Utilities
Rescaled Scores (0 to 100 scaling)
Zero-Centered Interval Scale (where the scores have a range of 100 and a mean of zero)
The choice of rescaling method for reporting depends on the kinds of comparisons you intend to make
(e.g. among items vs. between respondent groups) and the sophistication of the audience.
What Are Raw Logit Utility Scores?
A utility score is a measure of relative desirability or worth. When computing utility scores using logit,
latent class, or HB every item in the MaxDiff project receives a utility (preference score). The higher the
utility, the more likely the item is to be chosen as best (and not chosen as worst). The score for the last
item in your study is always fixed at 0. The other items' scores are estimated in relation to this final item
having a fixed zero score. Because only the relative utility values have meaning, it doesn't matter which
item is chosen as the fixed 0 point.
The utility scores coming directly out of the logit, latent class, or HB algorithm are called raw scores.
These scores have logit scaling properties, usually involve both positive and negative values, and are on
an interval scale. An item with a score of 2.0 is higher (more important or more preferred) than an item
with a score of 1.0. But, when interpreting raw logit utility scores, we cannot say that the item with a
score of 2.0 is twice as preferred as an item with a score of 1.0. To do that, we must transform the utility
scores to a positive probability scale that supports ratio operations.
It's easier for most people to interpret the logit scores if we transform them to be all positive values on a
ratio scale, where the scores sum to 100. Rescaled Scores (0 to 100 scaling) does that. Because it is
on a ratio scale, a score of 4 is twice as preferred as a score of 2. Click here for more details regarding
this rescaling procedure.
MaxDiff Help 641
Zero-Centered Interval Scale
The more consistent respondents are in their answers (and the more they agree with one another), the
larger the magnitude of the Raw Logit Scores from aggregate analysis. The magnitude of the scaling can
also affect the sensitivity of the Rescaled Scores (0 to 100 scaling). Researchers sometimes want to
zero-center and normalize the raw utility scores to have a constant range (such as 100), allowing them
better ability to compare results across different groups of respondents (who might have different degrees
of consistency and who may have quite different preferences for the final reference item) . The Zero-
Centered Interval Scaling does this. The scores are centered around zero (given a mean of zero) and the
range is set to 100 points.
As with Raw Logit scores, an item with a score of 2.0 is higher (more important or more preferred) than
an item with a score of 1.0. But, we cannot say that the item with a score of 2.0 is twice as preferred as
an item with a score of 1.0.
More Information about Running Logit Analysis
Logit analysis is an iterative procedure to find the maximum likelihood solution for fitting a multinomial
logit model to the data. A raw utility score is produced for each attribute level (where the last item is fixed
at a utility of zero for identification of the model) which can be interpreted as an average utility value for
the respondents analyzed. After logit converges on a solution, the output is displayed in the results
window, and the results are saved into a subfolder within your project directory. You can run logit
multiple times, and the results will overwrite the previous run. (You can make copies of your logit output
folder to archive any runs you wish to keep around for further reference.)
The computation starts with estimates of zero for all items' scores (utilities), and determines a gradient
vector indicating how those estimates should be modified for greatest improvement. A step is made in
the indicated direction, with a step size of 1.0. The user can modify the step size; a smaller step size
will probably produce a slower computation, but perhaps more precise estimates. Further steps are
taken until the solution stops improving.
For each iteration the Chi-Square (directly related to the log-likelihood) is reported, together with a value
of "RLH." RLH is short for "root likelihood" and is an intuitive measure of how well the solution fits the
data. The best possible value is 1.0, and the worst possible is the reciprocal of the number of choices
available in the average task. For a study in which four items were shown in each MaxDiff set, the
minimum possible value of RLH is .25.
Iterations continue until the maximum number of iterations is reached (default 20), or the log-likelihood
increases by too little (less than 1 in the 5th decimal place), or the gradient is too small (every element
less than 1 in the fifth decimal place).
The user also has the option of saving variances and covariances of the estimates (when you request a
Full report). The square roots of the variances are equal to the standard errors, and are always
displayed. The default is not to display variances and covariances.
The Logit Rule and MaxDiff
The raw logit utility scores are estimated such that their antilog is proportional to choice likelihood. We'll
642 SSI Web v8
demonstrate this property below.
Consider a MaxDiff task in which 4 items were shown to a sample of respondents. Under aggregate
logit, a single vector of utility scores is estimated that best fits the respondents' answers (across all
respondents and all tasks). Let's imagine that items D, A, G, and J are shown within a MaxDiff set. The
utility scores for those items are:
Item Utility
Item D -0.58
Item A 0.29
Item G 0.43
Item J 0.00
Given these utility scores, we can use the logit rule to predict the likelihood that the sample would pick
each item as "Best" within this set of four items. To do this, we take the antilog of each of the item's
scores (we "exponentiate" the scores) and then normalize the results to sum to 1.0.
Likelihood of Selected "Best"
Item Utility Exp(Utility) Likelihood (Best)

Item D -0.58 0.56 0.13 (0.56/4.43)
Item A 0.29 1.34 0.30 (1.34/4.43)
Item G 0.43 1.54 0.35 (1.54/4.43)
Item J 0.00 1.00 0.23 (1.00/4.43)
------ ------
Sum: 4.43 1.00
Item G is most likely to be chosen best among these four items, with a 35% chance of being selected.
Item D is 13% likely to be selected best.
Given these utility scores, we can also predict the likelihood that the sample would pick each item as
"Worst" within this set of four items. To do this, we take the antilog of the negative of each of the item's
scores (we "exponentiate" the negative of the scores) and then normalize the results to sum to 1.0.
Likelihood of Selected "Worst"
Item Utility -Utility Exp(-Utility) Likelihood (Worst)

Item D -0.58 0.58 1.79 0.43 (1.79/4.18)
Item A 0.29 -0.29 0.75 0.18 (0.75/4.18)
Item G 0.43 -0.43 0.65 0.16 (0.65/4.18)
Item J 0.00 0.00 1.00 0.24 (1.00/4.18)
------ ------
Sum: 4.18 1.00
Item D is most likely to be chosen worst among these four items, with a 43% chance of being selected.
Item G is 16% likely to be selected worst.
MaxDiff Help 643
4.3.5 Finding Segments via Latent Class Analysis

Note: Latent Class is often used to discover market segments (for use as banner points) from
MaxDiff data. Segment membership is reported on the Segment Membership tab of the output.
SSI Web includes a built-in Latent Class estimation routine for MaxDiff. You may decide instead to
analyze data separately using our standalone Latent Class software, but the two routines use the same
procedure and it is more convenient to perform the analysis within the SSI Web system.
When you run Latent Class (Analysis | Estimate MaxDiff Scores - Other Methods | Latent Class...),
the results are displayed in the report window and utility scores are saved into a subfolder within your
project directory. You can weight the data, or select subsets of respondents or tasks to process. You
can constrain certain utility scores to be higher or lower than other utility scores.
Interpreting Latent Class Scores
SSI Web reports the preference scores on three different scales:
Zero-Centered Interval Scores

Raw Scores
When comparing differences across groups, we recommend you use either the Zero-Centered Interval
Scores or the Rescaled Scores (0 to 100 scaling) rather than the Raw Scores. Not every group might
think the reference item (the final item) has equal utility, and yet Raw Scores scales all items with
respect to that final item set to 0. Moreover, with Raw Scores, the more consistent the answers are
within a group (the better the within-group model fit), the larger the magnitude of the scores. Just like
you shouldn't compare the weight of a person standing on a planet of unknown size in another galaxy to
another person standing here on the earth, you shouldn't compare latent class groups directly on the
Raw Scores. The scores for the two groups may be on vastly different scales.
Zero-Centered Interval Scores zero-centers and normalizes the raw utility scores to have a constant
range (such as 100), allowing you better ability to compare results across different groups of respondents
(who might have different degrees of consistency and who may have quite different preferences for the
final reference item).
Rescaled Scores (0 to 100 scaling) uses the exponential transform to place the items on a ratio scale,
where the sum of scores for each group is 100. The ratio scale gives you the ability to say that an item
with a score of 2.0 is twice as preferred as an item with a score of 1.0. The Raw Scores and Zero-
Centered Interval Scores do not support such ratio comparisons among items. We should note that the
consistency of different respondent groups can affect the relative sensitivities of their ratio scales. Click
here for more details regarding the Rescaled Scores (0 to 100 scaling) procedure.
Suggestions Regarding the Use of Latent Class with MaxDiff
Latent Class is not guaranteed to get the globally optimal (maximum likelihood) answer each time it
runs. There is a random component to the process and it is possible for Latent Class to find a
suboptimal solution, so you should ask the software to run multiple replicates from different starting
points to make sure you achieve a high quality solution. By default, 5 replications are performed. You
can increase this for even greater precision and confidence in your final solution.
Latent Class includes fit statistics that help indicate which number of groups best fits the structure of the
644 SSI Web v8
data (e.g. 2-group, 3-group, etc.). The standalone Latent Class manual. provides details regarding how
to use these statistics to judge the quality of different solutions.
Selecting the right number of groups to use in your final solution involves a balancing act of judging the
quality of the fit statistics for different solutions as well as considering their managerial usefulness. The
segments should be reasonably sized and have useful interpretation.
Some Details Regarding Latent Class Analysis
Latent Class divides respondents into segments having similar importances/preferences based on their
choices in MaxDiff questionnaires. It uses latent class analysis for this purpose, which simultaneously
estimates utility scores for each segment and the probability that each respondent belongs to each
segment. Latent Class is an integrated analytical component within the SSI Web platform (and we also
provide a standalone Latent Class software system).
Latent class has a role analogous to that of CBC's logit program, but rather than finding average utility
scores for all respondents together, it detects subgroups with differing preferences and estimates utility
scores (with logit scaling) for each segment. The subgroups have the characteristic that the respondents
within each group are relatively similar but the preferences are quite different from group to group. You
may specify how many groups are to be considered, such as the range of 2 through 6. A report of the
analysis is shown on the screen and appended to a log file (stored in a subfolder of your project
directory), and the utility scores for subgroups along with each respondent's probabilities of membership
in the groups are stored in a .csv file (also in a subfolder of your project directory) for subsequent
analysis.
The latent class estimation process works like this:
1. Initially, select random estimates of each group's utility values.

2. Use each group's estimated utilities to fit each respondent's data, and estimate the relative
probability of each respondent belonging to each group.
3. Using those probabilities as weights, re-estimate the logit weights for each group. Accumulate
the log-likelihood over all groups.
4. Continue repeating steps 2 and 3 until the log-likelihood fails to improve by more than some
small amount (the convergence limit). Each iteration consists of a repetition of steps 2 and 3.
Latent class reports the utility scores for each subgroup or "segment." Latent class analysis does not
assume that each respondent is "in" one group or another. Rather, each respondent is considered to
have some non-zero probability of belonging to each group. If the solution fits the data very well, then
those probabilities approach zero or one.
For more information on our latent class analysis approach, please see the full standalone Latent Class
manual.
MaxDiff Help 645
4.4 Anchored MaxDiff

Respondents indicate which items are relatively better (or worse) than others within standard MaxDiff
questionnaires. Thus, the scores are estimated on a relative scale, without any indication that the items
are good or bad, important or unimportant, in an absolute sense. Some researchers have viewed that as
a limitation for certain studies and certain clients.
Anchored MaxDiff lets the researcher draw a line (a utility boundary) between important and unimportant
items (positive vs. negative impact, buy or no-buy, etc.). That utility boundary, for example, could be set
at 0. Thus, any items that are positive are considered important and those that are negative are not
important (in an absolute sense). Anchored MaxDiff score estimation is available for aggregate logit,
latent class, and HB score estimation routines within the SSI Web platform.
Anchored MaxDiff provides potentially more information than standard MaxDiff, but it comes at a cost.
One of the key benefits of standard MaxDiff is that it is free from scale use bias, making it a very nice
technique for studying relative preferences across countries or across respondents who have different
scale use bias tendencies.
With anchored MaxDiff, although a rating scale is not being used, the tendency for one group of
respondents to generally be more positive/agreeable than another group of respondents can lead
to similar problems as scale use bias, which was one of the main problems researchers have
wanted to eliminate by using MaxDiff!
Furthermore, using anchored MaxDiff scores within cluster segmentation (or latent class
segmentation) might lead to respondent groups that are being delineated as much by their
tendency to be positive/agreeable regarding the position of items vs. the anchor as by their relative
scores for the items of interest within your study. (An obvious solution to this second issue is to
develop the segmentation using un-anchored scores, but then to profile the segments using the
anchored scores.)
Dual-Response Indirect Method
Jordan Louviere, the inventor of MaxDiff scaling, proposed a dual-response, indirect method for scaling
the items relative to a threshold anchor of importance or desirability. You may add indirect scaling
questions to your MaxDiff survey by clicking the Add Dual-Response Question box from the Format tab
within your MaxDiff exercise. Below each MaxDiff question, a second (hence, "dual") question is
inserted, such as the following:
If the respondent clicks "None of these are important to me" then we inform utility estimation that
all four items shown in this MaxDiff set should have lower utility than the anchor threshold. The
answer is coded in the data file as "3".
646 SSI Web v8
If the respondent clicks "Some of these are important to me" then we know that the "best" item
selected within the MaxDiff set should have higher utility than the anchor and the "worst" item
selected should have lower utility than the anchor. The answer is coded in the data file as "2".
If the respondent clicks "All of these are important to me" then we inform utility estimation that all
four items should have higher utility than the anchor threshold. the answer is coded in the data
file as "1".
When "Some of these are important to me" is selected, we do not have any information about how the
two non-selected items (from this set of four) relate to the anchor. Thus, the dual-response indirect
method provides incomplete information about how each of the items shown in a MaxDiff set relates to
the anchor.
Note: if the dual-response indirect method is applied to questions that show just two items per set
(paired comparisons questions), then we achieve complete information regarding how the two items
shown relate to the anchor.
The indirect method should probably not be used with more than 4 items shown per set. Increasing the
number of items per set increases the likelihood that respondents will select "Some of these are
important to me," which provides incomplete information. Because of this issue, the software issues a
warning if you try to use the indirect method with more than 4 items displayed per MaxDiff set.
The indirect method works well in practice. Evidence has been presented at the Sawtooth Software
Conference that the indirect method tends to lead to more items being scaled above the anchor threshold
than the direct method. In other words, the indirect method leads to more items being judged as
"Important" or a "Buy" compared to the direct method (described directly below).
Direct Binary Approach Method
Both Bryan Orme (in 2009 at the SKIM/Sawtooth Software European Conference) and Kevin Lattery (at
the 2010 Sawtooth Software Conference) have demonstrated how standard ratings questions (or sorting
tasks) may be used to anchor MaxDiff items. Lattery's work is more generally referenced today, as his
paper is more complete in comparing direct and indirect anchoring methods. Using questions outside
the MaxDiff section, we may ask respondents directly whether each item (or each of a subset of the
items) is important or not. A 2-point scale could be used, or respondents could be asked to sort items
into two buckets: important and unimportant buckets. A 5-point scale could also be used, where items
rated either top box or top-two box could signal that these exceed the importance threshold boundary,
etc.
The direct method is quite flexible. To use it, you simply add the type of rating or sorting task you wish
anywhere within your SSI Web survey, referencing the same list of items you are using in your MaxDiff
exercise. Later, during score estimation (under the Analysis menu), you indicate that you wish to use
the Direct Binary Anchoring method and then specify the logic that references the scaling questions and
communicates which value(s) on the scale indicate that each item exceeds the anchor threshold. For
example, if you used a 2-point ratings question within a grid question called ImpItems to rate each of the
items, then item 1 would exceed the anchor threshold if ImpItems_r1 > 1. ( _r1 stores the rating for the
first row within a ratings grid.)
The direct method works well in practice. Evidence has been presented at the Sawtooth Software
Conference that the direct method tends to lead to more items being scaled below the anchor threshold
than the indirect method. In other words, the direct method leads to more items being judged as "Not
MaxDiff Help 647
Important" or a "No buy" compared to the indirect method (described directly above). Context bias can
become a potential problem with the direct method. If the number of items grows to the point that the
researcher wishes to show the items across multiple screens, then the context of the other items on the
screen can affect the absolute judgments given.
Note: if you want to use the direct method and have each respondent rate only a subset of the items
within the direct scaling questions, you should select a variety of items ranging from low utility to high
utility for each respondent. One way to do that is by asking SSI Web to compute MaxDiff scores on-the-
fly and by using the available SSI Script functions for MaxDiff to build a constructed list of a subset of
items based on the scores. For example, if 30 total items are in your study, you might use this
procedure to have respondents rate half the items: those with on-the-fly rankings of 1, 3, 5, 7, through 29.
Or, a subset of 10 items might be rated: those with on-the-fly rankings of 1, 3, 6, 9, 12, 15, 18, 21, 24,
27, and 30.
Probability Rescaling Method
The probability-scaled scores when using Anchored scaling are positive values where the Anchor is set
to 100 and the largest possible value is the number of items shown to respondents in each set * 100.
The probability-scaled scores are developed as follows. Perform the following transformation on the raw
results (where the anchor is zero for each person or class) for each item score for each respondent:
Where:
Ui = raw logit weight for item i.

eÛi is equivalent to taking the antilog of Ui. In Excel, use the formula =EXP(Ui).
a = Number of items shown per set
The interpretation of this transformation is that we're computing the likelihood that each item would be
selected as best from a set including that item and other items of equal utility to the Anchor threshold
(where the set size is equal to that shown to respondents in the questionnaire).
At this point, the threshold value probability value is constant and equal to 1/a. Out of convenience, we
set the threshold value equal to 100 by multiplying all scores by 100/(1/a).
One disadvantage of this scale is that each respondent does not receive equal weighting when
computing population means. Respondents who believe all items fall below the threshold have a
maximum score of 100, whereas respondents who believe some items exceed the Anchor utility have a
maximum score of the number of items shown in each set * 100.
Zero-Anchored Interval Scores
If you have used Anchored MaxDiff, scores are displayed where the anchor threshold is set to 0. The
zero-anchored interval scores are normalized to have a range of 100 utility points. Positive scores are
associated with items judged above the anchor threshold; negative scores are associated with items
judged below the anchor threshold.
648 SSI Web v8
Raw Scores
If you have used Anchored MaxDiff, scores are displayed where the anchor threshold is set to 0. The
raw scores have logit scaling (the raw coefficients that result from using logit analysis and maximum
likelihood estimation). Positive scores are associated with items judged above the anchor threshold;
negative scores are associated with items judged below the anchor threshold.
Validation Research-on-Research for Anchored Scaling MaxDiff
Sawtooth Software and Kevin Lattery (from Maritz) have separately conducted a few research-on-
research projects to test and validate different methods for anchored scaling in MaxDiff. To read more
about different approaches to direct and indirect anchored scaling approaches, please see:
"Using Calibration Questions to Obtain Absolute Scaling in MaxDiff" (Orme 2009)

"Anchored Scaling in MaxDiff Using Dual-Response" (Orme 2009)
"Anchoring MaxDiff Scaling Against a Threshold - Dual Response and Direct Binary
Responses" (Lattery 2010)
MaxDiff Help 649
4.5 Customized MaxDiff Questions with "Free Format"

Warning: This is an advanced feature that if not executed properly can result in errors in design,
data collection, and analysis. It requires careful, manual checking of Free Format HTML syntax.
Introduction
Some researchers may encounter situations in which they need to customize the look and layout of their
MaxDiff or MPC exercises beyond that which is provided by the MaxDiff templates within the SSI Web
System. The instructions provided here will help you customize the look of your MaxDiff exercise.
To implement your own question format, you'll use the Free Format question type. You'll insert a Free
Format question directly above each standard MaxDiff question. Within these Free Format questions,
you'll be able to display the items to be asked in each MaxDiff question (according to the experimental
plan) in most any way you desire. Then, you'll use a special programming instruction to suppress the
standard MaxDiff questions so they do not display.
Designing/Programming the Questionnaire
1. Create a MaxDiff exercise.

Create a MaxDiff exercise the standard way by entering items, generating a design, and adding the
MaxDiff questions within the SSI Web questionnaire.
2. Create a Free Format question.

To create a custom MaxDiff question add a new Free Format question above your first MaxDiff
question (on the same page). Name the question something similar to "FFMD1". The "1" at the end
of the question ties this Free Format question to the first MaxDiff question, for example
"MaxDiffExerciseName_1".
NOTE: We suggest creating just one Free Format question now. Once you get it working you can
then copy and paste the Free Format question multiple times so that you eventually will have a Free
Format question for each MaxDiff question.
To create the custom HTML we suggest you start from a standard MaxDiff question and copy the
relevant source HTML (using the View Source function in your browser) for that layout as produced by
SSI Web (this is a good starting point, but you will need to remove some things, such as hidden fields
calculating pagetimes, etc. in order for it to work properly).
If you are applying Styles to your survey, you may wish to refer to the styles documentation to see
how to make your Free Format MaxDiff questions use the style settings that are applied to standard
MaxDiff questions.
Here is some example HTML that you can use as a starting point:
<table class="inner_table" width="100%">

<tr>
<td class="best_header" width="15%">
Best
</td>
650 SSI Web v8
<td class="items_header" width="70%">

</td>
<td class="worst_header" width="15%">
Worst
</td>
</tr>
<tr>
<td class="best_input_cell alt_color1 input_cell clickable">
Best radio button here
</td>
<td class="item_text_cell alt_color1" width="70%">
<div class="item_text">Item 1 here</div>
</td>
<td class="worst_input_cell alt_color1 input_cell clickable">
Worst radio button here
</td>
</tr>
<tr>
Best radio button here
</td>
<div class="item_text">Item 2 here</div>
</td>
Worst radio button here
</td>
</tr>
</table>
3. Define Free Format Variables

Create Free Format variables for the "best" and "worst" responses. Variables for "best" must end with
"_b". Variables for "worst" must end with "_w". For example add the following "radio" variables for the
first Free Format question "FFMD1":
FFMD1_b
FFMD1_w
4. Insert SSI Script.

To display the item text (the text or graphic you specified) for each item in the MaxDiff exercise you
will need to use SSI Script. Use the following function to return the text for each item in the MaxDiff
design:
[%MaxDiffDesignLabel(ExerciseName_MaxDiffQuestionName, Position#) %]
Next to each item you will need to display a radio button for best and one for worst. The values for
MaxDiff Help 651
these radio buttons must match the item displayed. To do that use the following SSI Script:
[%MaxDiffDesignValue(ExerciseName_MaxDiffQuestionName, Position#)%]
See the example below.
<tr>
[%RadioSelect(FFMD1_b, MaxDiffDesignValue(MaxDiff_1, 1))%]
</td>
<div class="item_text">
[%MaxDiffDesignLabel(MaxDiff_1, 1) %]
</div>
</td>
[%RadioSelect(FFMD1_w, MaxDiffDesignValue(MaxDiff_1, 1))%]
</td>
</tr>
<tr>
[%RadioSelect(FFMD1_b, MaxDiffDesignValue(MaxDiff_1, 2))%]
</td>
<div class="item_text">
[%MaxDiffDesignLabel(MaxDiff_1, 2) %]
</div>
</td>
[%RadioSelect(FFMD1_w, MaxDiffDesignValue(MaxDiff_1, 2))%]
</td>
</tr>
The SSI Script RadioSelect is used in the example above to create a radio button. If graphical radio
buttons are turned on then graphical radio buttons will be displayed. "FFMD1" above refers to the
current Free Format question. "MaxDiff_1" above refers to the first regular MaxDiff question.
After you are finished test your work to ensure that the items in your Free Format question are
coming up correctly. Test the survey locally (Test | Test Survey Locally). You should be able to
compare the items being shown with the regular MaxDiff question on the same page. The items for
the standard MaxDiff question and the items for the custom MaxDiff question should match.
5. Anchored Scaling Dual-Response (Optional)

First, make sure you have specified that you wish to use Anchored Scaling in the regular MaxDiff
question. Add another "radio" variable within your Free Format question that ends with
"_anchor" (FFMD1_anchor). Make sure to use the value 1 for "all", 2 for "some", and 3 for "none".
Here is an example of the HTML you would include:
<table>
652 SSI Web v8
[%RadioSelect(FFMD1_anchor, 1)%]
</td>
<td class="option_cell">
<div class="options">
<label for="FFMD1_anchor_1">All</label>
</div>
</td>
</tr>
</td>
<label for="FFMD1_anchor_2">Some</label>
</div>
</td>
</tr>
</td>
<label for="FFMD1_anchor_3">None</label>
</div>
</td>
</tr>
</table>
6. Suppress the regular MaxDiff questions.

Next you need to hide the regular MaxDiff questions. You don't actually delete them from the
questionnaire. We have created a fast way to toggle them to a suppressed mode. To suppress the
regular MaxDiff questions, add the following HTML comment to the Header 1 field of your MaxDiff
questions:

Including the above text in Header 1 of a regular MaxDiff question will cause it not to display on the
page. It will also cause your Free Format MaxDiff question to be verified the same way as the regular
MaxDiff questions (require answers, item cannot be selected as best and worst etc.). Finally it will
copy the respondent answers given to your Free Format question to the regular MaxDiff database
variables. This allows you to perform analysis as if the regular MaxDiff questions were seen.
MaxDiff Help 653
7. Test saving data.

Test run the survey again, this time making sure that your answers are being recorded properly both in
the Free Format question and in the regular MaxDiff question. You need to verify the final data record
to ensure that the answers correspond to what you selected.
8. Copy and Paste.

Once you are confident that your first Free Format custom MaxDiff question is displaying correctly,
verifying correctly, and saving correctly, copy and paste the question multiple times so that there is
one per regular MaxDiff question. Make sure to change the HTML references from "FFMD1" to
"FFMD2" and from "MaxDiff_1" to "MaxDiff_2", etc. Finally test again verifying that all of the questions
are working properly. You might choose to turn the MaxDiff questions back on to verify that all of your
custom questions are displaying correctly as compared to the actual tasks (see step #4 above).
We strongly encourage you to test your custom MaxDiff questionnaire thoroughly all the way through to
analysis.
654 SSI Web v8
4.6 Computing Scores On-The-Fly

If you want to compute individual-level MaxDiff scores on-the-fly (realtime during the interview), you can
use the following functions:
MaxDiffRankAttLabel(ExerciseName, Rank#)
MaxDiffRankAttValue(ExerciseName, Rank#)
MaxDiffScore(ExerciseName, ItemIndex)
These functions allow you to return (e.g. show to the screen, or use in calculations or skip patterns) the
label and score for each of the items in your MaxDiff exercise. You can also return the label and/or
score for the top, second, third, etc. item in terms of importance/preference. You can use the functions
above together with list-building instructions (e.g. ADD) to force the top n items onto another list. If the
list used in ExerciseName is a constructed list, then the ItemIndex refers to items on the original parent
list. If you want ItemIndex to refer to the item position in the constructed list, then you can substitute
ListValue(listname, 1) in place of ItemIndex. For example, if you want to reference the score for the first
item on your constructed list (named q1list), you would specify MaxDiffScore(ExerciseName,
ListValue(q1list, 1)).
Note: If you wish to store the rank order of items, labels, or scores of items presented to
respondents (as computed by these functions) within the data file, then you need to do additional
steps, such as add Free Format questions to your survey with Hidden Variables, and set the value
of these Hidden Variables equal to rank order or scores of MaxDiff Items. Respondents don't see
these Free Format questions with hidden variables; they are just used to store information to your
data file.
These functions use the individual-level logit method to compute positive scores summing to 100. The
scores developed in this way only rely on data from the current respondent (the scores cannot benefit
from information of other respondents, as final score computation using HB does). As a result, the
scores will have less stability and precision than if computed across a sample of respondents under HB
(hierarchical Bayes).
We recommend that each item be shown at least 4 times per respondent for reasonable individual-level
score estimation using these functions. For more detail on the individual-logit method and comparisons
to HB estimation for MaxDiff, see the white paper entitled "MaxDiff Analysis: Simple Counting, Individual-
Level Logit, and HB" available for downloading from our Technical Papers library at
Technical Details:
Because individual-level logit includes sparse data that may not converge under logit, we break out if the
log-likelihood fails to improve by 0.03 in an iteration. We also break out if the range in the logit scores is
greater than 15. For all practical purposes, these break-out rules lead to ample precision in the
estimated scores, and keep the scores reasonably scaled. The computation is nearly instantaneous,
and should not burden servers nor result in a wait time for respondents.
The scores returned using the functions above are normalized, positive scores that sum to 100. The
normalization is done as follows:
1. The raw logit scores for all items are first zero-centered by subtracting off their mean.
2. The scores from step 1 are transformed by eû/(1+eû); where e is the exponential constant.
3. The scores from step 2 are normalized to sum to 100.
MaxDiff Help 655
4.7 Paper-and-Pencil Studies

Overview
SSI Web lets you run your MaxDiff studies via Web, over PCs (CAPI installation), or as paper-and-pencil
questionnaires. The process of running paper-based MaxDiff studies is very similar to computer-based
studies. You should set up your levels and compose your MaxDiff tasks in the same manner as if you
were going to run a computerized study. However, you should probably choose plain black (text) on
white (background) for questionnaire colors, unless you plan to print the questionnaires in color.
For MaxDiff, you will not have so many different versions of the questionnaire as with computerized
interviewing, because that becomes difficult to manage on paper. Rather than having so many possible
versions of the questionnaire, you will create just a few versions (design blocks) of the questionnaire and
assign respondents (randomly) to take one of the different questionnaire versions.
With paper-based experiments, one typically uses just a few versions of the questionnaire since each
additional questionnaire version adds more administrative hassle. Researchers often employ between
three to eight unique versions of the questionnaire when conducting paper-and-pencil experiments. One
sacrifices a little in terms of design efficiency and control for context and order biases by using just a few
versions (relative to the 300 or more versions often employed in computerized interviewing), but the
practical impact on quality of results is usually relatively small.
It is not necessary that the same number of respondents complete each version of the questionnaire,
though it is helpful to achieve approximate balance.
Steps for Paper-Based Studies
Generate the Questionnaire:
After you have created the questionnaire and generated the design, save the results to a file using
Field | Create Paper & Pencil Interviews. Your questionnaire is saved in HTML format to a
STUDYNAME_EXERCISENAME_V#.htm file, where V# is the version number. That file is placed
in a /Paper-and-Pencil/ExerciseName folder within your study folder.
If you would prefer to deal with the item #s in a .csv data file rather than formatted text within an
HTML document, you can export the design by clicking the Export Design... button from the
Design tab of our MaxDiff Exercise dialog. A file named
STUDYNAME_EXERCISENAME_Design.csv is saved to your study folder.
Collect the Data:
Make sure when fielding the questionnaire that you are able to match the different questionnaire
versions correctly with respondent answers. You should also try to randomly distribute the
questionnaire versions among respondents, so that roughly equal numbers of respondents
complete each version.
Prepare the Answers:
After you have collected the data, you prepare a .csv file containing the respondent answers. This
file follows a required .csv (comma separate values) format, with specific labels in the first row of
the file. To see that format, prepare a template for yourself by clicking Field | Create
Accumulated Data Template File.... A file named Accumulated Data.csv is saved to your study
656 SSI Web v8
folder. Use this template to arrange your data, and make sure to use the prescribed header row
within your data file.
The layout is:
Column 2: Version#
Column 3: MaxDiff answer #1, etc.
The MaxDiff answers can be recorded either as position numbers in the questionnaire or item
numbers selected. When you accumulate data, you are asked to specify which type of coding you
are using.
If using the indirect anchored scaling method, "None are important" is coded as "3", "Some" as 2,
and "All" as 1.
Any missing data are coded as blank (meaning, if you viewed the data file using Excel, the cell
would be empty; if you viewed the file with a text editor, the missing value would look like two
consecutive commas with no text in between).
Accumulate Respondent Answers:
Click Field | Accumulate Paper & Pencil Data... and browse to the .csv file containing your
respondent data.
MaxDiff Help 657
4.8 MaxDiff Analysis Using CBC/HB v5.x

In this section, we assume you own the CBC/HB v5.x software and have prepared a .CHO file containing
respondent data. You obtained the .CHO file by clicking File | Data Management | Export Data and by
adding an export job that has an Export Settings Type of MaxDiff.
When you prepare the .CHO file, a number of other files are created that (although not required) are
useful for setting up your CBC/HB project:
ExerciseName.ATT (Attribute file)

ExerciseName.EFF (Effects file)
ExerciseName.VAL (Values file)
ExerciseName.MTRX (Default Prior Covariance Matrix)
Open the CBC/HB v5.x software. Browse to and open the .CHO file. CBC/HB v5 will recognize the
presence of the .ATT, EFF, .VAL, and .MTRX files when placed in the same folder as the .CHO file. It
will prompt you to import these files as well.
Once you have imported the data (along with the optional files), note that there will be k-1 total attributes
in the study (to avoid linear dependency, meaning that one of the columns can be perfectly predicted by
a linear combination of other columns), representing your k total items in the MaxDiff design. The final
item will be constrained to have a utility of zero, and the remaining items are estimated with respect to
the final item's score.
We should note that when using HB to estimate parameters for many items under dummy coding, the
estimates of the parameters (relative to the reference "0" level) can sometimes be distorted downward,
often quite severely when there are many items in the study and when the number of questions asked of
any one respondent is relatively few. (This distortion of course makes it appear as if the "omitted" item's
estimate is "too high" relative to the others.) To avoid this difficulty, we automatically export the
ExerciseName.MTRX file containing an appropriate prior covariance matrix. By default, we assume a
prior variance of 2. The proper prior covariance matrix for use with MaxDiff exercises is a (k-1) x (k-1)
matrix of values, where k is equal to the total items in your MaxDiff study. Make sure the custom prior
covariances file (the information from .mtrx) is actually being used by checking that the Use a Custom
Prior Covariance Matrix box is check on the Advanced tab.
If you examine the .MTRX file (with a text editor), you will find a matrix composed of "2"s across the main
diagonal, and "1"s in the off-diagonal positions (assuming you are using the default prior variance
specification of 1.0), such as:
2 1 1 1 . . .
1 2 1 1 . . .
1 1 2 1 . . .
1 1 1 2 . . .
. . . . . . .
. . . . . . .
. . . . . . .
If you had requested a different prior variance, the values in this matrix are multiplied by the desired
variance constant.
When you have finished these steps, click Estimate Parameters Now... from the Home tab. Weights for
the scores are written to the .HBU and .CSV files. Remember, the weight of the reference value
(the last item in your design that was omitted) for each respondent is 0 (you'll need to fill this in
when done). The other items are estimated with respect to that reference item.
658 SSI Web v8
We strongly encourage you to consider using covariates in your HB run, employing questions asked
elsewhere in the questionnaire that are predictive of differences in item scores between respondent
segments.
Transforming Weights to 0-100 Scale
To convert the raw weights to the 0-100 point scale, first zero-center the weights by subtracting the mean
weight for each respondent from each respondent's weights. Then, perform the following transformation
for each zero-centered item score for each respondent:
Where:
Ui = zero-centered raw logit weight for item i

eUi is equivalent to taking the antilog of Ui. In Excel, use the formula =EXP(Ui)
Finally, as a convenience, we rescale the transformed item scores by a constant multiplier so that they
sum to 100.
The logic behind this transformation is as follows: We are interested in transforming raw scores
(developed under the logit rule) to probabilities true to the original data generation process (the counts).
If respondents saw 4 items at a time in each MaxDiff set, then the raw logit weights are developed
consistent with the logit rule and the data generation process. Stated another way, the scaling of the
weights will be consistent within the context (and assumed error level) of choices from quads. Therefore,
if an item has a raw weight of 2.0, then we expect that the likelihood of it being picked within the context
of a representative choice set involving 4 items is (according to the logit rule):
Since we are using zero-centered raw utilities, the expected utility for the competing three items within
the set would each be 0. Since e0 = 1, the appropriate constant to add to the denominator of the
rescaling equation above is the number of alternatives minus 1.
Note: Prior to SSI Web v6.2, we used a different rescaling procedure that assumed that the scaling of
the logit weights was consistent with a data generation process of respondents picking one item from t
items, where t was the total number of items in the experiment. However, upon further review, this
assumption provided scaling that was too extreme when considering the actual data generation process
(where items are being chosen from subsets of items much smaller than t). The benefit of the new
scaling procedure is that you will note fewer inconsistencies between the summary raw scores and
probability-scaled scores for the population. The new probability-scaled estimates for the population will
be more stable.
MaxDiff Help 659
4.9 MaxDiff Analysis Using Latent Class v4

In this section, we assume you own the standalone Latent Class v4.0 (or v2.5) software and have
prepared a .CHO file containing respondent data. You obtained the .CHO file by clicking File | Data
Management | Export Data and by adding an export job that has an Export Settings Type of MaxDiff.
Or, if using the paper-and-pencil data collection procedure, you would have generated a .CHO file using
the MaxDiff Experiment Designer software.
When you export a .CHO file from the SSI Web system, the following additional files are created for use
in Latent Class v4:
ExerciseName.ATT (Attribute file)

ExerciseName.EFF (Effects file)
ExerciseName.VAL (Values file)
These files are all described in the Latent Class software documentation—but they have been formatted
in special ways to accommodate MaxDiff analysis. (Note, Latent Class v4 is required for MaxDiff
estimation. If you are licensed with Latent Class v3.x, you can obtain a free copy of v4 from Sawtooth
Software)
For your information, the formats for the files automatically created by SSI Web are described below:
ExerciseName.ATT is also a text-only file, containing labels for the items (attributes) in your MaxDiff
study (remember, if you have 8 items in total, you'll be using n-1, or 7 labels in the .ATT file). To conform
to the rule in standard Latent Class that each attribute have at least two levels, we represent each
MaxDiff item as a two-level attribute. For example, your ExerciseName.ATT file should contain:
Red
(use a single blank line here to separate the levels)
Red
(use two blank lines here to separate attributes, etc.)
Green
Green
Blue
Blue
ExerciseName.EFF is a text-only file, containing the attribute numbers to be used in analysis. If you
have 8 items in total in your MaxDiff study, you need to follow a dummy-coding procedure in which you
estimate n-1 or 7 parameters (the utility of the omitted parameter is set to zero, and all other utilities are
scaled relative to the omitted item). Assuming 8 items, specify the following on the first line of your
ExerciseName.EFF file:
#1 #2 #3 #4 #5 #6 #7
The "pound notation" is specialized for use when the analyst wishes to override the typical effects-coding
procedure that Latent Class v2 uses for conjoint data.
ExerciseName.VAL is also required whenever using "pound notation," and is a text-only file with two
660 SSI Web v8
lines per item in your study. Again, assuming an 8 item study, you are only estimating 7 parameters,
and your ExerciseName.VAL file should have 14 total lines (2 for each of the 7 parameters). The first six
lines look like:
"Attribute 1 Label"
1 2
"Attribute 2 Label"
1 2
"Attribute 3 Label"
1 2 (etc.)
Although we are only estimating a single parameter for each attribute, we list the level values as "1" and
"2." This is just to satisfy the Latent Class software (which was designed for conjoint experiments,
where every attribute has at least two levels), and to ensure that the scaling of data in the text report that
we produce is proper.
Running Latent Class
Start Latent Class v4 by clicking Start + Programs + Sawtooth Software + Sawtooth Software Latent
Class. Click File + Open. Browse to the .CHO file created by the MaxDiff Designer software.
On the Attribute Information tab, modify all attributes to have "User-specified coding." (Note that there
will be k-1 total attributes in the study, representing your k total items in the MaxDiff design.)
Click Estimate Parameters Now... from the Home tab. Utility values are written to the .LCU and .CSV
files. Remember, the utility of the omitted value for each group is 0, and the other items are measured
with respect to that omitted item.
Transforming Weights to 0-100 Scale
To convert the raw weights to the 0-100 point scale, first zero-center the weights by subtracting the mean
weight for each segment from each segment's weights. Then, perform the following transformation for
each zero-centered item score for each segment:
Where:
Ui = zero-centered raw logit weight for item i

eUi is equivalent to taking the antilog of Ui. In Excel, use the formula =EXP(Ui)
Finally, as a convenience, we rescale the transformed item scores by a constant multiplier so that they
sum to 100.
The logic behind this transformation is as follows: We are interested in transforming raw scores
(developed under the logit rule) to probabilities true to the original data generation process (the counts).
If respondents saw 4 items at a time in each MaxDiff set, then the raw logit weights are developed
consistent with the logit rule and the data generation process. Stated another way, the scaling of the
MaxDiff Help 661
weights will be consistent within the context (and assumed error level) of choices from quads. Therefore,
if an item has a raw weight of 2.0, then we expect that the likelihood of it being picked within the context
of a representative choice set involving 4 items is (according to the logit rule):
Since we are using zero-centered raw utilities, the expected utility for the competing three items within
the set would each be 0.0. Since e0 = 1, the appropriate constant to add to the denominator of the
rescaling equation above is the number of alternatives minus 1.
Note: Prior to SSI Web v6.2, we used a different rescaling procedure that assumed that the scaling of
the logit weights was consistent with a data generation process of respondents picking one item from t
items, where t was the total number of items in the experiment. However, upon further review, this
assumption provided scaling that was too extreme when considering the actual data generation process
(where items are being chosen from subsets of items much smaller than t). The benefit of the new
scaling procedure is that you will note fewer inconsistencies between the summary raw scores and
probability-scaled scores for the population. The new probability-scaled estimates for the population will
be more stable.
662 SSI Web v8
4.10 Robustness of MaxDiff Designs to Prohibitions

This section investigates the robustness of MaxDiff designs to prohibitions using Sawtooth Software's
MaxDiff System and simulated (synthetic) respondent data.
The Case for Prohibitions
MaxDiff questionnaires present respondents with typically four to six items at a time and ask which item
is "best" and which is "worst." In practice, enough sets are shown to the respondent such that each
item occurs about two to five times. (In some applications, the "worst" question may be skipped
altogether.) Sawtooth Software's MaxDiff component within the SSI Web system helps researchers
design, field, and analyze MaxDiff studies very rapidly.
One advanced option in the software is the ability to specify that certain levels should never be paired
within the same set. For example, in a fast-food restaurant study, we might wish to study the
importance of the length of time to order and receive food. We might use three items in our MaxDiff
design related to this:

When people go to fast-food restaurants, we presume they want to receive their food as quickly as
possible. Therefore, it wouldn't make sense to show respondents a MaxDiff task in which any two of
these items are paired. Respondents would always prefer the faster speed to slower, so this would be a
wasted comparison.
MaxDiff Help 663
Prohibitions and Design Efficiency
Those researchers familiar with conjoint analysis know that prohibiting certain level combinations from
occurring in the conjoint questionnaire can often be very damaging to design efficiency (the ability of the
questionnaire to estimate preference weights with precision). However, it turns out that prohibitions are
generally not very harmful in MaxDiff questionnaires. We demonstrate that point below using simulated
data sets with differing numbers of prohibitions.
Why are prohibitions more damaging in conjoint analysis than for MaxDiff? Prohibitions in conjoint
analysis are "within-concept" and lead to correlations between attributes in the independent variable
matrix. For MaxDiff, the prohibitions are "between-concept" and are less damaging to orthogonality. The
reduction in design efficiency in MaxDiff studies from prohibitions is due in part from deviations from
orthogonality. Other contributing factors are the imbalance that is sometimes introduced in the
frequency of items and the reduction in the degree of interconnectivity among items. In optimal MaxDiff
studies, each item is shown an equal number of times and is also compared an equal number of times
with every other item. And, for any one respondent, we want to minimize the total number of steps that it
takes to directly or indirectly compare all items. For example, if A is compared to B in question one and
B is compared to C in question two, then A is "linked" to C through two steps.
"Gold Standard" Design with No Prohibitions
In a MaxDiff study involving 20 items, there are ½t(t-1) possible two-way combinations of items that can
be prohibited. The ½(20)(19)=190 possible paired levels are represented by the white region of the table
below:
We begin by simulating data for a typical MaxDiff study that involves 20 items shown in sets of 4. Three-
664 SSI Web v8
hundred computer-generated respondents each received a 20-set questionnaire and "answered"

randomly ("bests" only). Each respondent received a unique version of the questionnaire, as generated
by the MaxDiff designer.
To compute design efficiency, we estimate an aggregate logit model and observe the precision of the
estimates (the standard errors). We also compute D-efficiency. The efficiency of one design relative to
another is found by taking the ratios of the D-efficiencies for the two designs. The base case example
(with no prohibitions) is our "gold standard" for comparison to designs with prohibitions.
MaxDiff Help 665
Modest Prohibitions Case
First we implement a modest number of prohibitions. Let's consider that the last five items in the study
are mutually exclusive, leading to the following prohibitions (prohibited levels marked with an "x"):
These ten prohibitions are grouped, with many occurring within the same row or column. This is
potentially more damaging than if ten prohibitions were scattered randomly across the table. But, even
these prohibitions have very little effect on design efficiency for MaxDiff. We generated a new series of
questionnaires involving these prohibitions and again had 300 respondents answer randomly. The
standard errors of the estimates were only slightly larger than in the non-prohibited case. This design is
99.99% as efficient as the "gold standard" design with no prohibitions. For all practical purposes, this
design is as efficient as the "gold standard."
666 SSI Web v8
Extreme Case
Let's consider a much more extreme case. We'll retain the previous prohibitions, but add other
prohibitions which at face value might seem to be quite damaging:
With these challenging (insane?) prohibitions, it becomes impossible for the MaxDiff designer to find a
very balanced design. It returns an imbalanced solution wherein the most represented item (item #2)
occurs over three times as often as the least represented (and most prohibited) item (item #18). The
largest standard error occurs for item #18 with a standard error almost double that of the most precise
estimates for items in the design with no prohibitions. While these results would be concerning, these
prohibitions wouldn't ruin the MaxDiff project (especially if we had reasonable sample sizes). The overall
relative design efficiency (considering all items) for this highly prohibited case is 92% as efficient as "gold
standard" design.
MaxDiff Help 667
"Conjoint-Style" Prohibitions
As a final example, MaxDiff may be used for "conjoint-style" brand x price experiments, such as was
presented by Keith Chrzan in the 2003 Sawtooth Software Proceedings. Keith demonstrated the use of
MaxDiff for pricing options on automobiles. Each option was tested at multiple prices (i.e. each item text
included a description of the option plus its price). Thus, his project was a discrete choice study
implemented as a MaxDiff exercise. An exercise like this is specified by developing a list of items (such
as options for automobiles) where each option is displayed at different prices. Then, prohibitions are
specified such that an item isn't shown twice (at different prices) within the same set.
Let's consider a study in which we have four options each at five prices. The prohibitions table looks like
this:
Repeating the simulation process with synthetic data, we find that this design is 99.5% as efficient as
the "gold standard" plan with no prohibitions.
In this section, we've only considered statistical efficiency to assess the issue of prohibitions. We find
that MaxDiff designs are remarkably robust to prohibitions. However, there may be additional
considerations, such as context bias, that researchers may wish to consider when using prohibitions in
MaxDiff questionnaires.
668 SSI Web v8
ACA Help 669
5 ACA Help
5.1 Getting Started with ACA
5.1.1 Getting Started with ACA
ACA is software for conducting Adaptive Conjoint Analysis studies over the Internet, over an intranet, or
using PC (CAPI-based) data collection. ACA is part of the SSI Web software suite. For more general
information about ACA, we suggest that you read the "ACA Technical Paper" available for downloading
from our home page at www.sawtoothsoftware.com.
question in this documentation. We are available Mon-Fri during our business hours (Mountain Time
Zone). (Note: we provide full technical support for composing and testing your Internet survey on your
local PC, and provide written instructions on how to install your questionnaire on your server. We cannot
take responsibility for or be expected to provide technical support to debug any customized HTML or
JavaScript you may add to your survey. You are also responsible to make sure that Perl is running, a
database is available, and that you can run Perl (CGI) scripts on your web server. We can provide some
assistance related to SSI Web's use with your server once the fundamental Perl functionality and a
database is in place.)
A sample ACA questionnaire (tut2.ssi) and supporting files are available within your installation of SSI
Web. You can easily access this example study by clicking Help | Sample Studies. This questionnaire
lets you see an example of an ACA survey and investigate how it is set up.
If you know a little HTML (which is not necessary to use ACA), you can employ some "tricks" that will
help you customize the look of your survey beyond the standard options provided by the software. By
opening the sample study and looking at the settings and text we've used, you can become familiar with
some of the possibilities.
Using ACA requires the following skills and knowledge base:
The analyst managing the survey should know the basics of conjoint analysis, including
defining attributes and levels properly, correctly phrasing the questionnaire text and response
scales, interpreting the conjoint part-worth utilities, and running simulations. It is not
necessary to be a statistician to use our software effectively, but knowledge of basic statistics
for market research is highly recommended.
The questionnaire is first composed locally on the PC under Windows XP, Vista, or Windows
7. The individual inputting/formatting the survey should have working knowledge of standard
Windows applications such as word processing, Web browsers and/or spreadsheets. Though
it is not necessary, some knowledge of HTML is valuable.
For web-based data collection, you may use our survey . You can also host surveys on your
own (one must have access to an Internet/intranet server running Perl 5.007 or later, with a
database). This can be on your company's own Web server, or your ISP's (Internet Service
Provider's) Web server. SSI Web is able to automatically upload the files needed to perform
data collection on your server. But, in some cases the user must know how to use FTP
software to create directories, upload files to those directories, and set permissions on the
files and directories.
670 SSI Web v8
To get started, we suggest you view other areas within this online help library, especially:
What is Adaptive Conjoint Analysis?

Basic Overview of ACA
Sections and Flow of an ACA Questionnaire
ACA Help 671
5.1.2 What is ACA?

Adaptive Conjoint Analysis (ACA) was first introduced by Sawtooth Software in 1985 and went on to
become the most widely used conjoint software and conjoint analysis technique in the world during the
1990s (Wittink, Huber, and Vriens 1997), (Green 2000). Most recently, Choice-Based Conjoint
techniques (including CBC and Adaptive CBC) have become more popular than ACA, but ACA still finds
use and has unique benefits for certain situations.
Conjoint (tradeoff) analysis is a technique frequently used for market research applications. Such
applications are usually concerned with consumer preference and they attempt to assess the impact of
specific product features on overall preference. Products or services are thought of as possessing
specific levels of defined attributes, and a respondent's "liking" for a product is modeled as the sum of
the respondent's "part-worths" (sometimes called utilities) for each of its attribute levels. For example,
attributes for a car could include: Price, Type, Gas Mileage, and Color. Levels of the Gas Mileage
attribute might include 15 MPG, 20 MPG, and 25 MPG; and levels for the Type attribute might include:
Convertible, Minivan, Compact, and Full Size.
The strength of conjoint analysis is its ability to ask realistic questions that mimic the tradeoffs that
respondents make in the real world. Respondents evaluate product alternatives (concepts) described by
various attributes and indicate which products they prefer. By analyzing the answers, conjoint analysis
can estimate the weights and preferences respondents may have placed on the various features in order
to result in the observed product preferences. In contrast to direct questioning methods that simply ask
how import each attribute is or the desirability of each level, conjoint analysis forces respondents to
make difficult tradeoffs like the ones they encounter in the real world. As in real purchase decisions,
buyers cannot get all the best features at the lowest prices.
It would be time consuming and difficult for respondents to evaluate all possible product combinations in
order to provide information on their values for the various product features. Conjoint analysis offers the
researcher a more efficient way to obtain such information: only a carefully chosen set of hypothetical
product concepts is presented to respondents for evaluation. For example, a respondent might be asked
to choose between the following two concepts:
The answers to this and successive questions are used to determine the respondent's part-worths for
each of the attribute levels. Once part-worths have been determined, the respondent's overall utility
(preference) for a given product can be estimated by summing the part-worths for each attribute level that
describes that product.
Perhaps the most serious practical problem for users of conjoint analysis is that of dealing with large
numbers of attributes. Clients often have a detailed understanding of their product category, and previous
research has usually identified many issues of interest. The respondent, on the other hand, usually has
less interest and enthusiasm, and is rarely willing to submit to a long interview. The researcher is often
in conflict. Should there be a long interview, risking unhappy respondents and data of questionable
672 SSI Web v8
quality? Or should the researcher insist on a narrower focus in the interview, providing the client with
less breadth than desired? ACA resolves this conflict by eliminating the tradeoff altogether.
For example, a traditional (non-ACA) full-profile cardsort conjoint study with 12 attributes and four levels
per attribute would require estimation of 37 part-worth utilities (the number of attributes x (the number of
levels per attribute -1) +1). If the questions presented to the respondent have been well chosen, they will
be difficult, perhaps requiring much thought. In addition, under traditional full-profile conjoint analysis,
each product concept is described using all 12 attributes, requiring much reading on the part of the
respondent. At the very minimum, the respondent would have to provide 37 answers; if there is any
random component to the responses we would need more observations. A rule of thumb is to require
three times as many observations as parameters being estimated. This would require the respondent to
answer 111 questions, each presenting a concept described on 12 attributes. Adaptive Conjoint
Analysis solves this problem by customizing the interview for each respondent: the respondent is asked
in detail only about those attributes and levels of greatest relevance, and the task is made more
manageable due to partial rather than full profiles (only showing a subset (usually 2 to 5) of the attributes
in any one question). The term "adaptive" refers to the fact that the computer-administered interview is
customized for each respondent; at each step, previous answers are used to decide which question to
ask next, to obtain the most information about the respondent's preferences.
Once the data are collected, ACA lets the researcher simulate respondent preferences for new or
modified products. The Market Simulator can be used to explore "what if" scenarios, such as changes in
product formulation. The researcher describes a group of hypothetical products by specifying each
product's level on each attribute. Respondent part-worth utilities are used to estimate strengths of
preference or buying likelihoods for each product, and results are accumulated over respondents to
provide shares of preference among competing products or average estimated buying likelihoods for each
product.
ACA Help 673
5.1.3 Basic Overview of ACA

ACA for the Web lets the researcher design a computer-interactive conjoint interview for data collection
over the Web, an intranet, or using CAPI-based (stand-alone PC) interviewing. The interview can
consider many attributes and levels, paying special attention to those the respondent considers most
important. Questioning is done in an "intelligent" way; the respondent's part-worths are continually re-
estimated as the interview progresses, and each question is chosen to provide the most additional
information, given what is already known about the respondent's values. In addition to the conjoint-
related ACA questions, ACA comes with a CiW/50 system that can include up to 50 "standard" survey
questions for capturing additional information for use in market segmentation. These 50 questions can
include randomizations and skip patterns. (More than 50 questions can be fielded in conjunction with
ACA through larger CiW licenses. Both CiW and ACA are products within the SSI Web suite of Web
interviewing products.) Respondent data are saved to a database on the Web server (or on separate
PCs, if using CAPI-based data collection), and are available in real-time upon completion of the interview.
ACA was developed using established industry-standard technology (Perl and HTML) available on most
Web servers. The interviews can be completed on nearly any Web browser. The steps for designing and
conducting an ACA study are typically as follows:
1. Using a Windows-based tool running locally on a PC (without being connected to the Web),
the researcher designs the ACA study: types in the attributes and levels, adds additional
survey text, and chooses from a menu of options that control the overall flow of the interview.
The researcher formats questions using the point-and-click interface and views the questions
one page at a time using Internet Explorer. Additionally, the researcher creates a table of
passwords to assign to respondents and quotas for each password combination.
2. Once the researcher is satisfied with the look, flow and content of the interview, the interview is
uploaded onto a Web (or intranet) server (or, optionally, to multiple PCs for CAPI-based
interviewing). The online help includes detailed instructions regarding the directories, files and
permissions that must be included in the installation.
3. The researcher tests the ACA survey online by accessing the interview over the Web (or
intranet) using a Web browser, making sure the interview is administered as expected, and
checking the resulting data using the Admin Module to ensure that the data are being stored
correctly. A pretest is often conducted during this stage to evaluate the suitability of the
survey and to make sure the resulting utility data have face validity.
4. Respondents are invited to participate in the survey (usually via email) and are given the URL
address and password. While the interview is in progress, the researcher and/or client can
view or download the data at any time without disturbing the data collection or interviews that
may be in progress.
5. When interviewing is complete, the data are downloaded using SSI Web's Admin Module, and
the respondent utilities are estimated using ACA's utility calculator locally on the researcher's
PC (or using ACA/HB software for hierarchical Bayes estimation). The researcher then uses
either the Online Market Simulator or the Market Simulator software included with the ACA
software to analyze the results and conduct market simulations.
674 SSI Web v8
5.1.4 ACA Tutorial and Example

Introduction
This tutorial leads you step-by-step through a simple Adaptive Conjoint Analysis (ACA) study, from
thinking about a marketing problem, planning a questionnaire and creating a new study in SSI Web, to
entering a list of the attributes and levels, generating and testing the questionnaire, and analyzing the
results.
SSI Web is a powerful program, and this tutorial only touches on the basics for running an ACA survey.
You'll become aware of different or more sophisticated approaches by reading other sections in the on-
line help and by experimenting with SSI Web.
We also suggest the following two articles, available within our Technical Papers Library at http://
www.sawtoothsoftware.com/support/technical-papers: "ACA Technical Paper" and "History of ACA."
(Note: this example and the data presented are purely fictional. The suggested steps represent one
approach to this type of mark eting problem, but may not necessarily reflect the single "best" way to
conduct such a research study using the SSI Web system. This tutorial only uses 5 attributes, to mak e
sure that it can run within student lab systems that are limited to 5 attributes. Most ACA studies have
between 8 to 15 attributes.)
An old college pal of yours named Gary calls you one day and invites you to dinner. You don't have
other plans, so you readily agree. After all, it's been years since you've seen him, you'll be eating at the
finest steak house in the city, and--most importantly--he's buying.
Gary was a very bright double major in chemistry and business. You once talked him into taking a
graduate-level marketing research course with you, which he reluctantly did, but his heart was clearly in
chemistry. After graduation, you both went your separate ways--you to a successful consulting career,
and Gary to a manufacturing company specializing in rubber and plastics.
At dinner, it doesn't take long to learn why Gary was so willing to pay. Gary mentions that his company,
Electra, Inc., has been developing next-generation foam rubber compounds, and has been hunting for
new markets to apply the technology. Gary, an avid mountain biker, tells you that the new materials
might be used for puncture-proof bicycle tires.
As you dig into your medium-rare porterhouse, Gary explains that mountain bikers often ride into remote
areas. If they puncture a tire, it could mean a long walk home. Consequently, many carry extra inner
tubes and tools to repair a tire if needed.
"The main problem," Gary admits," is weight. Solid core tires are heavier than traditional inflatable inner
tube tires, which might turn off many riders. And the more dense and durable the foam compound, the
heavier the tire." Gary rattles off facts related to typical tire weights compared to weights of solid foam
core tire prototypes Electra had already developed. By this time, Gary's steak is only half-eaten and
getting cold, but he doesn't seem to care.
He further explains why the firmness or softness of a tire matters. When biking on paved roads, a very
firm tire with a smooth tread is most efficient. "Bikers who do a lot of aggressive off-road dirt riding will
often take some air out of the tires to increase the amount of tread area that grips the trail, and to absorb
some of the bumps," he explains. He points out that with regular inner tube tires, a person can adjust
ACA Help 675
the air pressure of the tire to fit the conditions. "With a solid core tire," he notes, "you'd be stuck with
just one firmness."
"The tread pattern is also important," he continues. "Off-road riders need an aggressive tread, but people
who ride mainly on paved roads need a relatively smooth tread." By now his steak is chilled and nearly
inedible, so the waiter takes it away (but not before asking Gary if the steak hadn't met his
expectations). You both order up desserts. Gary rambles another ten minutes about marketing
strategy--specifically whether his company can market the product on its own or if it might need to
pursue an alliance with an established mountain bike tire name.
"Can you help us do a research study to figure out what kind of interest bikers might have in a solid core
tire, and the proper combinations of weight, tread pattern, durability, and firmness?" he suddenly asks.
The abruptness of his question catches you by surprise. You confidently smile, however, as the thought
occurs to you that Gary should have paid more attention in that graduate marketing research course.
Only five minutes into Gary's presentation you had already been thinking about how a trade-off analysis
technique like ACA might help answer these issues. You remind him about the one-week unit on
conjoint analysis that you both completed in that graduate-level marketing research class. After some
refresher, his eyes light up at the idea.
Together, you develop a list of key questions on a napkin:
1) Would mountain bikers accept a solid core tire? What types of bikers would it most appeal
to?
2) How might bikers trade off tire weight and tire firmness for the solid core tires? At what point
would the heavier weight of solid core tires significantly limit demand?
3) What tire tread patterns might be preferred for a solid core tire?
4) Does it make sense to align with an established player in tire manufacturing, or can Gary's
company market the tire under the Electra name effectively?
As you part ways for the evening, you invite Gary to your office to further discuss the problem and
possible solutions.
Study Scope and Budget
The next day, you receive an E-mail from Gary. He reports that there is good support at his company to
conduct some market research. However, the budget he has been given to work with is limited. He
suggests a small investigational study with about thirty respondents. (Most ACA studies in practice
include much larger sample sizes.) He floats the idea that mountain bike enthusiasts might be brought
to a central facility, shown the prototype tires, and take an ACA survey.
In a return E-mail, you agree that Gary's approach might work well, but caution him that using just 30
people to draw conclusions regarding a market of hundreds of thousands would be quite a stretch. "The
data must only be used to form initial hypotheses, " you warn. "Before drawing firm conclusions, those
hypotheses should be tested by another study with a larger sample size."
The following week at your office, you and Gary review the objectives written during your dinner
676 SSI Web v8
conversation on the napkin. You propose that you take the hour together to start thinking about some
conjoint attributes and levels.
Gary joins you in the small conference room. It has a large white board and markers, which would prove
useful, and a dish full of peanut M&Ms, which you have a particular weakness for. Before diving in, you
review some ground-rules regarding conjoint attributes and levels. You summarize the following rules at
the top of the white board:
1. Attributes should be as close to independent of one another as reasonably possible. ACA

(like most conjoint techniques) assumes a simple additive model, and violations of simple
additivity within ACA are nearly impossible to detect.
2. Because attributes are independent of one another, the levels across all attributes should
be able to combine freely.
3. Levels within each attribute must be mutually exclusive. Tires are described using a single
level from each attribute.
4. Because just the level text appears on ACA's screens when defining different tire choices,
the text for each level must stand on its own.
Because of the small sample size in the study, you recommend that each attribute include few levels--
perhaps three or four at most. With larger sample sizes, you might have decided to measure four or
even five levels of "quantitative" (ordered) attributes like weight and tread durability.
There is rarely a need to measure more than five levels of an ordered attribute, you explain, as the
market simulator lets you interpolate between measured levels during analysis. Including too many
levels for an attribute significantly lowers the precision of the utility estimates for each level. It also can
lead to a problem in conjoint analysis called the "Number of Levels" effect. Attributes defined on
relatively more levels tend to get an artificial boost upward in importance. ACA is less prone to this bias
than other full-profile conjoint methods, you assure him, but it is still a concern. By this time, you notice
Gary's eyes beginning to glass over, so you cut to the chase.
"OK then," you announce, "the primary objective seems to be to compare the standard inner tube tire to
the new foam core tire. Let's begin by writing down those two levels." On the white board, you write:
Tire Type
Standard inner tube tire
Puncture-proof solid foam core tire
"And we need to ask respondents to consider the different degrees of firmness of the solid core tires,"
Gary interjects. "Some people might want a firm tire, and others less firm." You agree, and write
another candidate attribute with its three levels:
Firmness of Solid Foam Core Tire

Very firm (best for pavement)
Somewhat firm (for pavement or dirt)
Less firm (for best traction in dirt)
After specifying these first two attributes, you immediately notice a problem. "It seems that these
attributes already violate ground rules 1 and 2," you declare. Gary furrows his brow in thought. You
continue: "It wouldn't make sense to combine any level of the second attribute, Firmness of Solid Core
Tire, with the first level of the first attribute, Standard inner tube tire."
ACA Help 677
Gary nods in understanding, and suggests a possible solution: "Don't conjoint analysis methods let you
prohibit certain levels from combining with others? Let's just prohibit the firmness levels for solid core
tires from ever appearing with the standard inner tube tire." By this time you are beginning to remember
why Gary hadn't enjoyed the market research course as much as you had.
"Prohibitions are usually detrimental to conjoint studies," you begin. "And if it seems that you absolutely
need to prohibit certain level combinations, it might just be pointing to poor attribute design." You pause
a moment for effect. "I think this is one of those cases," you say. "In ACA, each product concept is
defined by exactly one level from each attribute--one cannot just skip an attribute for a given product
alternative. When you have attributes that only apply to some alternatives or brands, those relationships
require an 'alternative-specific' design. A different kind of trade-off technique called Choice-Based
Conjoint can deal nicely with alternative-specific designs." (Gary again gets a glassy-eyed look.) You
explain to Gary that the objectives of his study (small sample size, product design research, possibility
of computerized interviewing) seem to favor ACA, so you will need to think of a way around the problem.
After a few moments, the solution occurs to you.
Rather than treat those two attributes ("Tire Type" and "Firmness of Solid Core Tire") separately, you
recognize that there are really only four unique ways the levels of those attributes can logically combine.
You decide to collapse the two separate attributes into a single one with the four possible conditions:
Tire Type/Firmness
Puncture-proof solid foam core. Very firm (best for pavement)
Puncture-proof solid foam core. Somewhat firm (for pavement or dirt)
Puncture-proof solid foam core. Less firm (for best traction in dirt)
Standard inner tube tire. Adjustable firmness (for pavement or dirt)
With that hurdle behind you, the next series of attributes flow more easily. As for tire weight, the folks at
Electra determined that a solid core tire might weigh twice as much as a standard tire. A more
expensive process and compound could reduce the weight to about 50% heavier than a standard tire.
You write:
Weight
Tire weight: 2 lbs. (about same as regular tire)
Tire weight: 3 lbs. (50% more than regular tire)
Tire weight: 4 lbs. (double the weight of regular tire)
Gary takes the marker from you and quickly lists three common tread types on the board:
Tread Type
Off-road tread
On-road tread
On-road / Off-road tread
He then lists his company along with two key tire makers that his firm might either have to compete
against or form an alliance with:
Brand Name
Michelin
Tioga
Electra
Finally comes the issue of tread wear durability. He writes these levels on the board:
678 SSI Web v8
Tread Wear Durability

Tread lasts for 1000 miles
You nod approvingly, while grabbing another handful of M&Ms. Seems that he is catching on--and quite
enjoying the process. The meeting has been a success. You shake hands, and walk Gary to the door.
A Sample Scripted Questionnaire
A few days later, you receive another E-mail from Gary. He has scripted some expanded attribute
definitions and a few additional questions to ask study participants. You put that together with some
text you usually use for ACA studies, creating a unified document:
Start:
Thank you for agreeing to participate in this research study!
We are asking mountain biking enthusiasts like yourself to help us plan and develop new high
performance mountain bike tires. This interactive survey should take about 10 to 15 minutes of
your time.
Miles:
How many miles per week on average would you say you ride your mountain bike?
Fewer than 10 miles
About 10 to 25 miles
More than 75 miles
Triptype:
Which of the following best describes you?
I always ride in areas where I could call a friend or "hitch a ride" if I had mechanical
difficulties.
I sometimes ride to remote areas where I would have to either repair my bike or walk
my bike out if I had mechanical difficulties.
I frequently ride to remote areas where I would have to either repair my bike or walk
my bike out if I had mechanical difficulties.
Terrain:
Which of the following best describes your mountain bike riding habits...?
100% pavement
75% pavement / 25% dirt
ACA Help 679

100% dirt
Notes to Gary: At this point in the questionnaire, we need to ask respondents to rate levels within the
brands, tire types, and the tread patterns. (We can't know ahead of time which levels they prefer.) This
is the first section of what is called the "Priors," also known as the "self-explicated" section.
IntroACA:
During the rest of this survey, we're going to ask you to consider features of mountain bike tires
that you might purchase. The tires might differ in terms of:
Brand Name: well known versus less well known brands.
Tire Type: standard versus a foam core tire. The foam core tires are not inflated with air and are
therefore puncture-proof. Foam core tires come in different degrees of firmness. More firm is
better for riding on pavement. Less firm is best for shock absorption and traction on off-road trails.
Tire Tread Pattern: On-road, Off-road tread, or On-road/Off-road tread. On-road tread is
relatively smooth on the entire surface of the tire. Off-road tread has an aggressive pattern on the
entire surface. On-road/Off-road tread has a flat narrow channel on the very bottom of the tire for a
smoother ride on pavement, but aggressive tread elsewhere.
Tire Weight: from a standard weight (about 2 lbs. per tire) to double that weight (about 4 lbs. per
tire).
Tread Durability: tread lasting from 1000 to 3000 miles on average.
Introrate:
In this section, we'll ask you to rate different features based on your preference.
You'll be shown a list of features on the left side of the screen. Use your mouse to indicate how
desirable each feature is.
When making your decisions, please assume that the mountain bike tire is exactly the same in
every way except for the features shown.
680 SSI Web v8
ACA Rating Task for Preference within Attributes:
<Repeat the same question for Tire Type/Firmness, and Tread Type>
Introimp:
In this next section, we want to find out how important different features are to you.
We'll ask you to indicate how important it would be for you to get the features you'd most want to
have in a mountain bike tire.
We'll ask you how important the following features are:
Brand Name, Tire Type, Tire Tread Pattern, Tire Weight, and Tread Durability.
We'll be using a 7-point importance scale, where the top category is "Extremely Important".
Before you start rating the importance of each attribute, you should decide ahead of time which
attribute you'll give the highest score to.
Notes to Gary: The second stage of the "Priors" section asks respondents to state how important each
attribute is. We display the best and worst levels of each attribute, and ask the respondent to tell us
how important it is to get the best level of that attribute instead of the worst one. We use the answers to
these questions to decide which attributes to focus on in the conjoint pairs section that follows. We
should put these Importance questions on the same page, so that respondents can adjust their answers
to consistently use the scale. If we placed each importance question on a separate page, then
respondents might record ratings early on that they'd like to modify, given the features that were asked
later.
ACA Help 681
ACA Importance Task:
< Repeat for all attributes. Show all importances on same page.>
Intropair:
By now, we have a rough idea of your opinions about the features being studied today. We still
need a little more information to learn how you would decide which mountain bike tire to buy.
Based on your previous responses, we'll make up some different alternatives for you to consider.
In each question we present two tires, each described by combinations of features. One is
shown on the left side of the screen, and the other on the right. We'll ask you which tire you'd
prefer, and to indicate your strength of preference.
We will only be asking you to compare a few features at a time. Please assume that the
alternatives are identical in every other way except for the differences shown.
Notes to Gary: The "Pairs" section is the conjoint (trade-off) part of the ACA survey. ACA first focuses
on the attributes that the respondent already stated were most important. It uses the information from
the two earlier stages to construct two product alternatives (described by various combinations of
attribute levels) that are nearly balanced in preference. Respondents are asked to choose between
them. By varying the product profiles and observing the respondent's preferences, we are able to further
refine the rough set of utilities already established in the initial Priors section. Respondents usually
answer from a dozen to thirty of these conjoint questions. After each Pairs question, ACA updates its
estimate of the respondent's utilities. It uses that information to generate a new challenging trade-off for
the respondent to consider.
682 SSI Web v8
ACA Conjoint Pairs Task:
<Use 9-point scale, where 1 = "Strongly Prefer Left," 3 = "Somewhat Prefer Left," 5 = "Indifferent," 7 =
"Somewhat Prefer Right," and 9 = "Strongly Prefer Right". Repeat multiple times, rotating different
attribute levels to describe the tires.>
Introcalib:
This is the last section. Based on all your responses, we're making up some mountain bike tires
for you to consider.
You should like the first tire least, the second one best, and any others should fall in between.
We'll ask you how likely you would be to buy each tire if it were available right now. Type a
number from 0 to 100 where a "0" means that you "Definitely would not buy" and "100" means
that you "Definitely would buy."
Notes to Gary: The last section of the ACA survey is the "Calibration Concepts." Respondents are
presented usually five or six product concepts (one at a time). Respondents rate each product concept
using a likelihood to buy scale. This is an optional section, and is only needed if we want to run
"purchase likelihood" simulations during analysis. People often exaggerate stated purchase likelihoods,
and we may conclude that competitive market simulations versus the standard inner tube tire is a better
way to judge relative interest in the solid foam tires. The part-worth utilities have already been estimated
before this stage. The Calibration Concepts section scales the utilities to best fit the purchase likelihood
ratings.
ACA Help 683
ACA Calibration Concepts Task:
Finish:
interviewing module.) ACA is one component within that software program. The next step is to open a
new study, and define a study name.
After starting up SSI Web, choose File | New Study…. The New SSI Web Study dialog appears, with
your cursor active in the Name field.
for your study also carries the same twenty-four character name as the study. For this tutorial, you
might create a name such as academo1. From the New SSI Web Study dialog, specify academo1 as
the studyname.
684 SSI Web v8
Click Compose | Write Questionnaire... to add an ACA exercise to your study. Each SSI Web study
can contain one or multiple ACA exercises. When you insert a question or a conjoint exercise within
your list of questions, it is inserted directly below the question or page break you initially highlight.
Highlight the page break following the Start question, and click Add.... Specify a name for the ACA
exercise, such as tires.
You use the Attributes tab of the ACA Exercise Settings dialog to enter the list of attributes and levels:
ACA Help 685
Type the first attribute name: Brand in the Attribute Display Text field. The attribute name is a label that
the respondent will see in the interview. If you want to use a shorter label to display to you as the
questionnaire author for program settings and data export, specify that label in the Internal Label field. If
you do not specify an internal label, the label in the Attribute Display Text is used. Click OK to accept
this information and close the dialog. Now that at least one attribute name is in the list, the Add button
686 SSI Web v8
under the Levels panel becomes active. Also note that the Brand attribute is highlighted in the
Attributes panel. With the Brand attribute highlighted in the Attributes panel, click the Add button under
the Levels panel to add levels within the Brand attribute. The Level Text dialog is displayed. Type
Michelin in the Level Display Text field. To add the next level of Brand, press the ENTER key twice.
Type in the next level: Tioga. Repeat the process for the remaining level, Electra.
Back on the Specify Attributes and Levels dialog, you will notice that there is a drop-down box labeled
Level Rank ing. This lets you inform SSI Web about any a priori ordered attributes. Some attributes
have a rational preference order (e.g. lower prices always preferred to higher prices, faster speeds
preferred to slower speeds, etc.). You can avoid asking respondents to rate the levels for attributes that
carry an obvious preference order by choosing Worst to Best or Best to Worst level order. This is a
benefit, but be forewarned that if you misinform SSI Web about the correct order of preference, the
resulting part-worth utilities will be terribly wrong. The current attribute, Brand, does not have an obvious
preference order. You don't know ahead of time which level(s) the respondent prefers. Therefore, you
should leave this attribute as No a priori rank ing.
When you are ready to add the next attribute (Tire Type/Firmness), click the Add button under the left
Attributes panel, type the attribute label, and click OK to place that new attribute on the attribute list.
With that new attribute highlighted on the attributes list, click the Add button under the Levels panel to
add the four levels of that attribute.
Follow the same pattern for the remaining attributes. For your convenience, we repeat the full list of
mouse, use the shortcuts Ctrl-C to copy, and Ctrl-V to paste into the desired field. (Hint: you can select
clipboard icon . This can save a great deal of time.)
Please remember to specify the Level Rank ing for each attribute by highlighting each attribute in the
Attributes panel and using the drop down control for each to set the Level Rank ing. You should pre-
specify the ranking for any attributes with rational order of preference. Make sure to specify Best to
Worst ranking for Weight, and Worst to Best ranking for Tread Wear Durability. Leave the Level Ranking
as No a priori rank ing for the other attributes.
Brand Name
Michelin
Tioga
Electra
Tire Type/Firmness
Puncture-proof solid foam core. Very firm (best for pavement)
Puncture-proof solid foam core. Somewhat firm (for pavement or dirt)
Puncture-proof solid foam core. Less firm (for best traction in dirt)
Standard inner tube tire. Adjustable firmness (for pavement or dirt)
Tread Type
Off-road tread
On-road tread
On-road / Off-road tread
Weight
Tire weight: 2 lbs. (about same as regular tire)
ACA Help 687
Tire weight: 3 lbs. (50% more than regular tire)

Tire weight: 4 lbs. (double the weight of regular tire)
Tread Wear Durability

After you have specified the list of attributes and levels, you are ready to set up your ACA questions. To
do so, select the Design tab.
Four main areas appear on this dialog: Priors Randomization, Importance Settings, Pairs Settings, and
Calibration Settings. It may help to review some terms that describe the different sections of the ACA
survey:
Priors (the self-explicated level ratings and attribute importances.)
Pairs (the conjoint section of the interview. Pairs of products defined on multiple levels are
displayed, and respondents indicate the one they prefer, using a rating scale running from left
to right.)
Calibration (an optional section called Calibration Concepts used when you need to run
purchase likelihood market simulations. Respondents rate a few product concepts individually
on a likelihood of buying scale.)
688 SSI Web v8
Let's begin thinking about the Priors. Before showing any conjoint questions, we need to ask
respondents to rate the levels within any attributes that we don't know ahead of time the rational
preference order. For example, brand name, tire type, and tread type. We also specify control settings
regarding the attribute importance question that follows the level ratings. The Priors Randomization area
(with the suggested settings for this questionnaire) is shown below.
We suggest you randomize the order of the levels displayed on the screen and the order of the attribute
presentation for the importance questions. Check the Randomize Level Order in Rating Questions and
Randomize Attribute Order in Rating and Importance Questions boxes. These settings will help control
for any psychological order bias.
Next, let's move to the Pairs Settings area of this dialog. The Pairs represent the conjoint section of the
interview. Pairs of products defined on multiple levels are displayed, and respondents indicate the one
they prefer, using a rating scale running from left to right. Rather than take the space within this tutorial
to explain the settings, we refer you to the context-sensitive help provided by typing F1. For this
Mountain Bike Tires tutorial, we suggest the following settings:
Maximum attributes for Pairs Section 5
Number of Pairs questions 20
Number of Pairs questions at

each stage 10
Number of attributes in each

pairs question in the first stage 2
Number of attributes in each

pairs question in the last stage 3
We suggest including all 5 attributes in the Pairs section (Maximum attributes for Pairs Section = 5).
Even though the number of pairs suggested by ACA's formula applied to this particular list of attributes
and levels (explained when you press F1 for help) is just 14, you may decide that your respondents can
reliably answer a few more. More Pairs information is good to have (particularly with studies that have
small sample sizes), as long as respondents don't become so fatigued that they give poor quality data or
drop out of the survey. You decide for the mountain bike tire study that respondents can reasonably
complete 20 pairs. Larger studies with more attributes may lead you to require respondents to answer
more questions, though most researchers don't ask respondent many more than about 30 Pairs
questions. (See an article entitled "Accuracy of Utility Estimation in ACA" in the Technical Papers
Library at http://www.sawtoothsoftware.com/support/technical-papers for more information about how the
number of pairs questions affects the precision of part-worth utility estimates.)
Let's turn to the Calibration Settings area of this dialog. This is an optional section that you might
ACA Help 689
decide to drop. With the Calibration Concepts section gone, this is another justification for increasing
the number of Pairs questions from the "suggested" 14 for this particular set of attributes and levels to
20. If you need to use Purchase Likelihood simulations during analysis, then you must include a
Calibration Concepts section. We'll include it in this tutorial so that you can have practice setting up a
Calibration Concept section. Set these two fields to:
Number of Calibration Concepts = 5

Number of attributes in each concept = 5.
Close the ACA Exercise Settings by clicking OK. SSI Web automatically places the ACA questions
into your survey:
SSI Web has added a number of ACA related questions to the survey. These questions are based on
your settings and the default template within SSI Web.
690 SSI Web v8
Before we examine or fine tune those ACA questions, let's turn our attention to the issue of assigning
passwords for respondents to enter at the opening screen of an SSI Web survey.
used.
processing document. Assuming you really had such a document, you might use the Ctrl-C to copy,
Start:
We are asking mountain biking enthusiasts like yourself to help us plan and develop new high
performance mountain bike tires. This interactive survey should take about 10 to 15 minutes of
your time.
automatically opens. The Questionnaire Access and Passwords dialog is displayed::
ACA Help 691
as follows:
692 SSI Web v8
We are asking mountain biking enthusiasts like yourself to help us plan and develop
new high performance mountain bike tires. This interactive survey should take about
10 to 15 minutes of your time.
your text to take greater control over the layout, font, and style later in this unit.
Click the OK button of the Preview window to close that window and return to the previous SSI Web
dialog.
make the text bold. By default, the font appearance and color schemes are based on the Style selected
for your project (Compose | Survey Settings, Style tab). You can try different styles to see how it
affects the appearance of the questionnaire. You can also override the text and color settings in the
Style by clicking the Paintbrush icon and specifying new settings to override the appearance of
this question
After you are happy with the layout of the introductory screen, click OK to return to the Write
Questionnaire dialog. Because we aren't using respondent passwords, we'll ignore the additional tabs
on the Questionnaire Access and Passwords dialog.
ACA Help 693
Specifying Select Questions
(If you are already familiar with adding Select questions within SSI Web surveys, you may choose to
skip forward to the section entitled "Fine-Tuning the ACA Questions").
When you add a new question to the List of Questions, it is inserted directly below the question that was
highlighted when you clicked the Add... button. If you need to move a question once it has been added
to the List of Questions, simply highlight the question to be moved and click the or buttons
to move the questions within the list (you can also highlight a question and click Ctrl-X or Ctrl-C to cut or
copy and Ctrl-V to paste questions within the current questionnaire, or even to another SSI Web study.)
Miles:
How many miles per week on average would you say you ride your mountain bike?
Fewer than 10 miles
More than 75 miles
on the list). At the Write Questionnaire dialog, click Add..., choose Select as the question type, and
type Miles for the question name. Click OK and the Select Question dialog is shown.
Response Options tab. From the Response Options tab, use the Add... button to add the four response
Now that you have seen how to specify Single Select question types, you have the tools you need to
specify the remaining two select-type questions for the mountain bike tires questionnaire (Triptype and
Terrain—please refer to the questionnaire text for these questions as presented near the front of this
unit).
694 SSI Web v8
In addition to the Select questions we just added to the questionnaire, there are some text-only,
named IntroACA, Introrate, Introimp, Intropair, Introcalib, and Finish. Even though the respondents
aren't asked to provide specific inputs to these, we refer to these as "questions" in the sense that we add
them to the questionnaire as if they were standard questions, and they are listed separately by their
"question" name in the List of Questions. Such questions are called Text/HTML Filler.
After the Terrain select-type question, follows the text-only instructions:
IntroACA:
During the rest of this survey, we're going to ask you to consider features of mountain bike tires
that you might purchase. The tires might differ in terms of:
Brand Name: well known versus less well known brands.
Tire Type: standard versus a foam core tire. The foam core tires are not inflated with air and are
therefore puncture-proof. Foam core tires come in different degrees of firmness. More firm is
better for riding on pavement. Less firm is best for shock absorption and traction on off-road trails.
Tire Tread Pattern: On-road, Off-road tread, or On-road/Off-road tread. On-road tread is
relatively smooth on the entire surface of the tire. Off-road tread has an aggressive pattern on the
entire surface. On-road/Off-road tread has a flat narrow channel on the very bottom of the tire for a
smoother ride on pavement, but aggressive tread elsewhere.
Tire Weight: from a standard weight (about 2 lbs. per tire) to double that weight (about 4 lbs. per
tire).
Tread Durability: tread lasting from 1000 to 3000 miles on average.
introductory text into the Start question). There are more than three separate paragraphs here, and we'll
need to deal with the additional element of bolding selected text.
Add a Text/HTML Filler question directly following the Terrain question in the questionnaire. To do so,
highlight the Terrain question, click Add…, and specify the Question Name as IntroACA and the
question type as Text/HTML Filler. The Text/HTML Filler dialog is displayed.
Earlier in this tutorial, when we created the Start, we placed each paragraph of text in a separate Header
1, Header 2, or Footer section. We saw that SSI Web automatically places blank lines between text in
these sections. However, with the text in the IntroACA question, there are many more paragraph
breaks. We'll take the opportunity here to introduce the concept of using a few simple HTML instructions
within our survey text.
Browsers know how to interpret HTML instructions when displaying the results to the screen. If you
ACA Help 695
know HTML, you can use it whenever you wish within SSI Web surveys to accomplish your aims. If you
don't know HTML, it isn't very difficult to learn a few HTML tricks, or you can use the toolbar available
within the editor that appears when you click "Pencil" icons in SSI Web:
Either type or cut-and-paste the text for the IntroACA question into the Header 1 text field (Any of the
696 SSI Web v8
During the rest of this survey, we're going to ask you to consider features of mountain bike
tires that you might purchase. The tires might differ in terms of:
And then (with the text in the first paragraph highlighted), click the "Paragraph" icon on the toolbar.
This inserts a <p> prior to the text and a </p> after the text:
<p>During the rest of this survey, we're going to ask you to consider features of mountain bike
tires that you might purchase. The tires might differ in terms of:</p>
IntroACA question.
the toolbar. After you finish separating the paragraphs with <p></p> tags and bolding the appropriate
text with <b></b> tags, it should look something like:
ACA Help 697
698 SSI Web v8
You should now have the tools necessary to add the other Text/Only HTML Filler questions in their
appropriate positions in the survey: Introrate, Introimp, Intropair, and Introcalib (Finish is a
Terminate/Link question, explained in the next paragraph). Remember to place Introrate directly prior to
the tires_Rating1 question, Introimp directly prior to the tires_Importance1 question, Intropair
directly prior to the tires_Pair1 question, and Introcalib directly prior to the tires_Calibration1
question.
Make sure when adding the Finish question at the end of the survey to add it as a Terminate/Link
question type. On the Settings tab of the Terminate/Link question, click Terminate Respondent to
indicate that respondents that reach this question are finished. SSI Web may warn you that a
terminating question must stand alone on its own "page" in the survey. This leads us into our next
discussion on page layout.
ACA Help 699
Notice that the Select questions we've added to the questionnaire are all currently arranged on the same
page. Let's assume we wanted to break these up, one question per page. You set a new page break
after the Miles question by highlighting Miles on the list and clicking the Add Page Break button.
Miles is now placed on its own page. Repeat this operation to place Terrain, IntroACA, and Introrate
on separate pages. Delete the page break between tires_Importance3 and tires_Importance4 to place all
the Importance questions on the same page. Make sure to place the final text screen Finish on its own
(last) page.
From the Write Questionnaire dialog, click the Survey Settings icon (alternatively, click
Compose | Survey Settings...). This dialog controls global aspects of the questionnaire (meaning that
the settings here control the general behavior of the questionnaire rather than page- or question-specific
aspects.) Examples include the fonts and colors used in the questionnaire, the wording for the "Next"
700 SSI Web v8
button, and the global header and footer that can appear at the top or bottom of each page of the survey.
The details of this dialog are beyond the scope of this unit, but you can learn more about the items on
the dialog by pressing the F1 key.
Fine-Tuning the ACA Questions
After you have finished these steps, your questionnaire should be complete (though it may lack polish).
You will note that SSI Web used a set of default names to represent the many ACA questions in the
questions list. SSI Web built those questions drawing from a default template. Some of the text for the
questions you will certainly want to edit to fit the appropriate wording for your study. Depending on how
picky you are about the look of your survey, you could spend a good deal of time adjusting each
question and adding HTML elements to achieve just the right look.
Let's view one question from each ACA section, as an introduction to fine-tuning ACA questionnaires.
The first section in ACA is the "Ratings" section, where respondents are asked to rate any levels within
attributes for which we don't know ahead of time the order of preference (levels for attributes such as
brand or color). From the List of Questions in the Write Questionnaire dialog, highlight and edit the
tires_Rating1 question (note that there are only three Ratings questions: one for each attribute for which
we haven't assumed a Level Rank ing (a priori) order.
Click Preview and one of the level rating questions is displayed (due to randomization, the attribute
shown and also the order of the levels may be different):
Note that SSI Web automatically "fills in" the attribute name ("Weight", in this example) in the text, and
places the levels at the left-side of the grid. If you close this window and view the text in the Header 2
text field (using the "Pencil" icon is easier), you'll see that the source text for the question header is:
Pl eas e r at e t he f ol l owi ng [ %ACAAt t r i but e( ) %] s <br > i n t er ms of how

des i r abl e t hey ar e.
Notice that SSI Web uses a "system function" called ACAAttribute( ) (delimited by [%...%] ) to fill in the
text you supplied for the attribute name for attribute 1. This makes it easier to compose ACA surveys,
as the text you supply for the attribute names is automatically carried forward in this section of the
questionnaire. Of course, you may find that the text you used doesn't lead to very proper grammar when
ACA Help 701
inserted this way within the template sentence. In that case, you can modify the wording as you like.
However, recognize that you only supply the text once for each major section in ACA, and this same
text is used across all questions in that section. So, the text must be generic enough to be meaningful
across all questions in the section.
Also note the use of the <br> tag, which instructs the browser to insert a line break at that point.
Hint: If you want to customize the text directly above or below any ACA question, you can insert a new
Text/HTML Filler question intermingled within the questions in the ACA section (you can also insert other
question types mid-stream within an ACA segment should you be inclined). Make sure to format the
page breaks from the Write Questionnaire dialog so that the new customized text you've inserted
appears properly within the separate pages of the ACA questionnaire.
From the Write Questionnaire dialog, highlight the tires_Importance1 question and click Edit…. This
is the Importance question in ACA, where respondents are shown the best and worst levels from each
attribute and are asked to rate the contrast in those two levels in terms of importance. Click Preview,
and the following is displayed (depending on your attribute randomization, you may see a different
attribute):
Note that you'll need to edit the "XXXXXXXX" characters to be appropriate for the subject of your survey
("mountain bike tires" would be appropriate here). To do this, return to the ACA Importance Questions
dialog and edit the text directly within the Header 2 text field. Also note the use of two System
Functions (%ACABest( ) and %ACAWorst( )) within this question that dynamically return each
respondent's most preferred and least preferred levels (based on the answers provided in the earlier
ACARAT questions or based on your previous Level Rank ing setting for attributes.
When you edit the tires_Pair1 question and preview it, the following is displayed:
702 SSI Web v8
Again, you need to edit the "XXXXXXXX" characters to be appropriate for your survey.
Finally, when you edit and display the tires_Calibration1 question, its default presentation is:
Again, you'll need to edit the "XXXXXXXX" to be appropriate.

ACA Help 703
On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Mountain Bike
Tire Questionnaire as the header text.
style (and by selecting graphical Previous and Next buttons), our survey now looks like:
Local Test Server
Locally.

704 SSI Web v8
After you have edited the survey to the point that you are pleased with the look, content, and functionality
of your survey, you should do some careful testing to ensure you haven't made a fundamental error in
your ACA setup. We suggest you take the survey yourself under Local Test Server. You should answer
each question legitimately and carefully. Pay special attention to the ACA questions to make sure that
the attribute levels appear correctly: the most preferred levels should always appear above the least
preferred levels in the Importance questions. The Pairs questions should generally not pose "obvious"
trade-offs (all the levels on the left better than the ones on the right, or vice-versa). All the worst levels
should appear together within the first Calibration Concept, and the best levels in the second Calibration
Concept. If the importance questions, Pairs, or Calibration Concepts are not appearing correctly, this
suggests that the Level Rank ing (a priori) order settings are incorrect for some attributes. You must be
vigilant to avoid this error. Even experienced ACA researchers have been known to set a Level Rank ing
order incorrectly when rushed.
After you have taken the questionnaire carefully as suggested above, you should download the data
(downloading data is described in a separate tutorial called "Getting Started with SSI Web: A 45-minute
Hands-On Tour"), then compute and examine your part-worth utilities using the market simulator
(described later). The part-worths should very closely reflect your preferences. If they don't, this
suggests something may be wrong with your study setup. Generally the most common error is that you
have not correctly set an ordered attribute as Best to Worst or Worst to Best. Such errors are
disastrous, so double or even triple-check your attribute settings.
examine the resulting part-worths to make sure the data at least have face validity.
After you have tested the survey using the Local Test Server, you post the survey to your web server,
using Sawtooth Software's hosting service, or using an web server set up for you by an ISP (Internet
Service Provider). (Setting up your survey on the web is beyond the scope of this unit, but is described
in detail elsewhere. Fielding options also include hosting on your own company's server, or using
Sawtooth Software's hosting service). You send an email to Gary, with a link to your survey, and he
browses to and takes the survey. He suggests some minor wording changes. After you make those
changes, Gary invites six other individuals at Electra take the survey. You download the data and
analyze. Due to the small sample size, the results are a bit noisy (and there is an obvious bias toward
the Electra brand name), but the data seem to feel right.
Next, you recruit six mountain biking enthusiasts within your city to come to a central site to see a
demonstration of the new foam tire technology and take the survey. After a short hands-on
demonstration of the new foam core tires, they sit down to computers connected to the Web and take
the survey. You and Gary watch silently as they take the survey. At the end of each session, you ask
each respondent follow-up questions to ensure that there weren't any parts that were difficult to
understand or just didn't make sense. You ask them specifically about the ACA sections, making sure
that they understood what to do at each stage of the survey, and that they didn't feel overwhelmed by the
task. After debriefing the test respondents and analyzing their data to ensure that the results look
reasonable, you make a few small adjustments and proceed to field.
Thirty respondents are brought to the field facility, shown the solid foam tire prototype, and complete the
survey. The completed data reside on the ISP's server where you uploaded the SSI Web survey. The
details for viewing and downloading the data from the server are provided in other areas of the SSI Web
ACA Help 705
documentation and are beyond the scope of this ACA unit. Even so, it is useful to at least describe the
steps you would perform to access and download the data from the Web, estimate the part-worth
utilities, and begin analysis using the market simulator.
Data Management, Utility Estimation, and Moving the Data into SMRT
To download the data from the 30 respondents to the mountain bike tires study, you can browse to the
tutorial study there is no such site set up). Once at the administrative module, you would download your
data (by clicking Download from the main menu), making sure to save your data (named
studyname_data.sqlite, where STUDYNAME is your study name) to the same folder on your hard drive in
which you developed the SSI Web project. Alternatively, from the SSI Web interface, you can simply
click Field | Download Data... to download your data without logging into the Admin Module.
Once you have downloaded the data, you are ready to estimate part-worths. Assuming you had
downloaded the data as described above, you would click Analysis | Calculate ACA Utilities Using
Ordinary Least Squares…. Computation takes just a few seconds. If you try this with this tutorial
study, you will receive an error stating that you have no data within your project folder to analyze.
However, hypothetical data for this project are stored in a tutorial folder within the accompanying SMRT
software that you received together with your SSI Web system.
SMRT stands for "Sawtooth Software Market Research Tools" and contains the market simulation tool
used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. SMRT is the
platform that (given the proper license with its user identification codes) runs ACA for Windows studies
and includes the tutorial data for this Mountain Bike study. We'll discuss how you would move data
from your ACA project into SMRT (hypothetically, since the data aren't actually in your ACA project) but
then open the SMRT software system and continue using the Mountain Bike tutorial data that were
installed there.
When you calculate part-worth utilities using Analysis | Calculate ACA Utilities Using Ordinary Least
Squares… this process creates a text-only format file containing case IDs and part-worth estimates
called STUDYNAME.utl.
To move the ACA part-worth utility data into SMRT for analysis, you would (again, this is for discussion
purposes only—you should not actually perform these steps for this Mountain Bike tutorial):
1. Start the SMRT software by clicking Start | Program Files | Sawtooth Software | Sawtooth
Software SMRT.
3. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by
clicking (from the SMRT software menu) Analysis | Run Manager | Import… and following the
prompts for importing ACA utilities (selecting ACA v4.x Utilities *.utl as the file type, and
browsing to your .utl file). Once you import your utilities, you can select Analysis | Market
706 SSI Web v8
Simulator and begin simulations.
Analyzing the ACA Data Using SMRT's Market Simulator
SMRT is a companion software system to SSI Web that can be used for analyzing the results of ACA
and CBC studies. To start SMRT, click Start | Programs | Sawtooth Software | Sawtooth Software
SMRT. Open the Mountain Bike Tutorial study by clicking File | Open and then browsing to find the
Tires1.smt file located in the ...Program Files\Sawtooth Software\SMRT\Tutorial folder.
Once the study is open, from the main menu, click Analysis | Market Simulator. You will note that
there is a list of available Utility Runs (your recently computed run is available), and a list of Simulation
Scenarios (currently empty). Click Compute!, and a report of average utilities and importances for your
data set is displayed in the report window:
ACA Help 707
Scenario:
Utility Run: ACA Utility Run
Average Utility Values
Rescaling Method: Zero-Centered Diffs

Michelin 4.73
Tioga 18.14
Electra -22.88
Puncture-proof... Very firm... -6.62

Puncture-proof... Somewhat firm... 4.60
Puncture-proof... Less firm... -21.91
Standard inner tube tire... 23.94
Off-road tread -20.16

On-road tread 1.14
On-road / Off-road tread 19.02
Tire weight: 2 lbs... 62.51

Tire weight: 3 lbs... 8.94
Tire weight: 4 lbs... -71.45
Tread lasts for 1000 miles -37.17

Tread lasts for 2000 miles -2.14
Tread lasts for 3000 miles 39.31
Average Importances
Brand Name 13.99

Tire Type/Firmness 19.98
Tread Type 22.44
Weight 27.22
Tread Wear Durability 16.37
<<The above average utilities are rescaled utilities using the zero-centered "diffs" method. The diffs
method rescales utilities so that for each individual the total sum of the utility differences between the
worst and best levels of each attribute across attributes is equal to the number of attributes times 100.
It is very important that you learn how to interpret part-worth utilities properly. Please see a document
entitled "Interpreting Conjoint Analysis Data" available in our Technical Papers library at http://
www.sawtoothsoftware.com/support/technical-papers.
It is difficult to make firm conclusions from just these thirty respondents, but we might try to form some
tentative conclusions:
708 SSI Web v8
First, looking just at the report of average importances (the second section above), it appears that the
most important thing to respondents is the weight of the tire. This confirms some of Gary's concerns
and implies that Electra should pay attention to controlling the weight of the solid core tires. The least
important attribute is brand, which is good news if Electra wants to break into this market without an
established name.
Paying attention to the Average Utility Values report, Electra seems less preferred than the other two
brands (no surprise there). On average, most respondents seem to prefer standard tires to the new
technology. It looks like there might be a slight inclination toward a hybrid (On-road/Off-road) tread.
There appears to be significant resistance in increasing tire weight, particularly from 3 to 4 lbs.
Although you don't have much data to work with for segmentation purposes in the Tires1 study (just 30
respondents), you should note that you can run market simulations by "banner" point. Banner points are
the segmentation variables that run across the columns of a cross-tabulation report. If you select a
Banner variable using the drop-down menu at the upper-left of the Market Simulator dialog, and then
click Compute!, you will note that the utility/simulation report is broken out by different groups of
respondents, depending on how they answered to the Banner variable.
A lot of important effects due to the competitive context, respondent heterogeneity and correlations of
preferences among attribute levels can occur within a data set. These may be completely obscured by
viewing just average utilities and importances. Also, the concept of utility scores seems difficult to grasp
for many managers. For these reasons, running market simulations offers a more powerful and revealing
way to look at conjoint data.
Market Simulations
The first time you open the Market Simulator for a study, the list of Simulation Scenarios is empty. A
Simulation Scenario is a particular "what-if" scenario, wherein certain products are placed in competition
with one another in a simulated marketplace.
Prior to fielding your study, you had confirmed with Gary which specific competitive scenarios he wanted
to test. You checked those against the list of attributes you and he had developed to ensure that the
attributes covered the proper ranges and were specified adequately.
Gary was most interested in two specific scenarios. The first one assumed that Electra was offering the
only solid core tire in the market. Each brand was assumed to offer tires to appeal to different riding
styles/terrain.
Simulation Scenario 1:
Product Brand Tire Type/Firmness Tread Weight Durability

Electra1 Electra Solid/Very firm On-road 3.75 lbs. 2500 miles
Electra2 Electra Solid/Somewhat first On/Off-road 3.5 lbs. 2000 miles
Electra3 Electra Solid/Less firm Off-road 3 lbs. 1000 miles
Michelin1 Michelin Standard On-road 2 lbs. 3000 miles
Michelin2 Michelin Standard On/Off-road 2 lbs. 2500 miles
Michelin3 Michelin Standard Off-road 2 lbs. 1500 miles
Tioga1 Tioga Standard On-road 2 lbs. 3000 miles
ACA Help 709
Tioga2 Tioga Standard On/Off-road 2 lbs. 2500 miles

Tioga3 Tioga Standard Off-road 2 lbs. 1500 miles
Notice that some of the levels of Weight and Durability are in between the levels we measured in the
study. The market simulator lets us interpolate between levels, so this is not a problem. However, it will
be easier to specify these levels if we have assigned "Level Values" to each level.
To assign Level Values, from the main menu click Analysis | Market Simulator and then click the
Assign Level Values button at the very lower left-side of the dialog. The Assign Level Values dialog is
displayed. Using the drop-down box, select Weight. Click the first row in the Value column
(corresponding to 2 lbs.). Type "2." Click the next two cells and enter "3" and "4."
Next, select Tread Wear Durability using the drop-down box, and type "1000," "2000," and "3000" for the
values. Click OK to return to the Market Simulation menu.
Create a new simulation scenario by clicking Add....
When you click the Add... button, the Scenario Specification dialog is displayed.
The first step is to type a scenario name into the Name field, in the upper left-hand corner of the dialog.
Type Base Case.
Next, you'll specify the nine products for the first scenario. The area you type the products into looks
like the grid of a spreadsheet. The first column is where you type the product label. As you click the
various columns associated with the attribute levels, the level codes will appear in the window below to
remind you of the codes associated with each attribute's levels.
The attribute level codes for the nine products for the first scenario were:

Electra1 3 1 2 3.75 2500
Electra2 3 2 3 3.5 2000
Electra3 3 3 1 3 1000
Michelin1 1 4 2 2 3000
Michelin2 1 4 3 2 2500
Michelin3 1 4 1 2 1500
Tioga1 2 4 2 2 3000
Tioga2 2 4 3 2 2500
Tioga3 2 4 1 2 1500
Specify these nine products in the product entry grid. To add new rows to the grid (for additional
Hint: While working within the product specification grid, you can press the Enter key to advance to the
next cell. You can use the arrow keys to move the cursor within the grid. You can also cut-and-paste
information within the grid, or from a spreadsheet.
After you have specified these nine products, next choose a simulation method. The default method is
710 SSI Web v8
Randomized First Choice. We generally recommend this method for competitive market scenarios like
this one, so we'll use the default. Because we have already examined the utilities and importances, un-
check the Display Utilities box.
dialog.
Choose a utility run to use for your simulation. At this point, you should only have one run in the list, so
this is your only choice. If there were multiple utility runs (for example, another run from the ACA/HB
system), you could select that run by highlighting it in the list of Utility Runs.
name Base Case in the Simulation Scenarios list, and then click Compute!. SMRT examines each
respondent's record and predicts which product(s) each respondent would likely choose based on the
product specifications and each respondent's utilities.
Scenario: Base Case

Utility Run: ACA Utility Run

Model: Randomized First Choice
Total Sampling Iterations: 100000
Attribute Variability Multiplier: Autocalibrated Value (0.509807)
Product Variability Multiplier: 0
Exponent: 1
<<Above are the settings that govern this run. It is not necessary at this point for you to k now what
these settings mean (#iterations, Multipliers, Exponent). These are explained in other areas of this on-
line help and also in the manual.>>
Brand Tire Type Tread Type Weight Durability

Electra1 3 1 2 3.75 2500
Electra2 3 2 3 3.5 2000
Electra3 3 3 1 3 1000
Michelin1 1 4 2 2 3000
Michelin2 1 4 3 2 2500
Michelin3 1 4 1 2 1500
Tioga1 2 4 2 2 3000
Tioga2 2 4 3 2 2500
Tioga3 2 4 1 2 1500
<<Above are the product level codes you specified for the nine products in this simulation. Below are
the simulated shares of preference (choice) for these products, along with their standard errors.>>
Product Shares of Preference
Shares Std Err

Electra1 7.55 1.22
ACA Help 711
Electra2 9.23 1.41

Electra3 8.89 1.48
Michelin1 14.51 1.49
Michelin3 7.43 1.12
Tioga1 19.07 2.37
Tioga2 13.68 1.48
Tioga3 7.99 1.04
Though a total is not shown, the shares of preference sum to 100%. Standard errors are presented next
to each product's shares, which are useful for expressing confidence intervals around the share
estimates.
Again, it is difficult to form anything but tentative conclusions from just 30 respondents. Even so, we
might conclude from this simulation run that if Electra were to market a solid core tire it would generally
achieve a lower share of the market than established brands with standard tires. These simulated
shares assume that Electra could achieve equal distribution and brand awareness relative to Michelin
and Tioga.
The base case simulation suggests that there is relatively balanced demand for the three types of solid
core tires.
scenarios. It is important to remember that the shares of preference resulting from conjoint predictions
are not equivalent to actual market shares, and often look quite different. Many other factors in the real
world influence market shares and cannot be measured and reflected solely by conjoint data. Conjoint
The second scenario Gary was interested in is nearly the same as the first, but assumes that Michelin
also comes out with solid core tires (products 10-12). It also assumes that the weight and durability of
the new Michelin tires are slightly inferior to Electra's tires.
Simulation Scenario 2:

Electra1 Electra Solid/Very firm On-road 3.75 lbs. 2500 miles
Electra2 Electra Solid/Somewhat firm On/Off-road 3.5 lbs. 2000 miles
Electra3 Electra Solid/Less firm Off-road 3 lbs. 1000 miles
Michelin1 Michelin Standard On-road 2 lbs. 3000 miles
Michelin2 Michelin Standard On/Off-road 2 lbs. 2500 miles
Michelin3 Michelin Standard Off-road 2 lbs. 1500 miles
Tioga1 Tioga Standard On-road 2 lbs. 3000 miles
Tioga2 Tioga Standard On/Off-road 2 lbs. 2500 miles
Tioga3 Tioga Standard Off-road 2 lbs. 1500 miles
MIchelin4 Michelin Solid/Very firm On-road 4 lbs. 2000 miles
712 SSI Web v8
Michelin5 Michelin Solid/Somewhat firm On/Off-road 3.75 lbs. 1500 miles

MIchelin6 Michelin Solid/Less firm Off-road 3.5 lbs. 1000 miles
You can save a lot of time in entering this scenario if you retain the settings for the first nine products
when you create the new scenario. SMRT lets you do this. Click Add..., and from the Add Scenario
dialog use the drop-down control to choose Base Case as the starting template for the new scenario.
You can name this second simulation scenario Competitive Reaction. After placing your cursor in the
final row of the product specification grid, click Insert Product three times to open three additional rows
at the bottom to enter Michelin's three new tire products.
The level codes/values for the additional three products are:

MIchelin4 1 1 2 4 2000
Michelin5 1 2 3 3.75 1500
MIchelin6 1 3 1 3.5 1000
After entering the new product specifications, click OK to return to the Market Simulator dialog.
Before running the new Competitive Reaction scenario, you may wish to clear the previous results from
the report window by clicking Clear and un-checking the box for the Base Case scenario. Then, click
Compute!, and the following is produced as part of the market simulation output:
Shares Std Err

Electra1 5.44 0.88
Electra2 7.02 1.21
Electra3 7.18 1.31
Michelin2 9.88 1.23
Michelin3 5.76 0.84
Tioga1 18.39 2.26
Tioga2 13.13 1.46
Tioga3 7.58 0.95
Michelin4 4.66 0.81
Michelin5 4.34 0.63
Michelin6 3.63 0.66
It is important to note that the simulation model we are using (Randomized First Choice) does a good job
of accounting for differential substitution for similar (near substitute) products. This Competitive Reaction
reflects a line extension for Michelin, and new products that should compete very closely with Electra.
For the purposes of illustration, let's place the results from the two simulation runs within the same data
table:
Base Competitive
ACA Help 713
Case Reaction
Shares Shares
Electra1 7.55 5.44
Electra2 9.23 7.02
Electra3 8.89 7.18
Michelin1 14.51 12.98
Michelin3 7.43 5.76
Tioga1 19.07 18.39
Tioga2 13.68 13.13
Tioga3 7.99 7.58
Michelin4 N/A 4.66
Michelin5 N/A 4.34
Michelin6 N/A 3.63
----- -----
Totals 100.00 100.00
(Note that you can copy-and-paste results directly from the Market Simulator report window directly into
a spreadsheet. A few simple copy-and-paste steps can build the above table. You can even copy-and-
paste results from within the Market Simulator report window to other places within the same window.)
Examining the above table, note that the introduction of the solid core Michelin tires took significant
share from both Electra's solid core and Michelin's standard tires. The new entries compete strongly
with existing Michelin tires due to the same brand name and with Electra's tires due to an imitation of the
new technology. In contrast, Tioga's shares are not much changed from the Base Case to the
Competitive Reaction scenario. Though, with the additional competition offered by Michelin's line
extension, the shares had to decrease at least some.
Note that Michelin's solid core tires achieve a lower share than Electra's solid core tires. One tentative
conclusion from this is that the edge in performance specifications for Electra's solid core tires can
overcome the negative of having a less preferred brand name.
How might the market change if Michelin and/or Tioga can produce solid core tires that match Electra's
specifications? You may wish to run additional scenarios to test those possibilities.
714 SSI Web v8
5.1.5 Sections and Flow of ACA Questionnaires

Suppose the attributes being studied describe automobiles, with these levels:
BODY TYPE
Two-Door Sedan
Four-Door Sedan
Hatchback
Minivan
Convertible
DRIVE CONFIGURATION
Rear-Wheel Drive
Front-Wheel Drive
All-Wheel Drive
ORIGIN
Made in USA
Made in Japan
Made in Europe
COLOR
Red
Blue
Green
Yellow
White
Black
PRICE
$18,000
$20,000
$22,000
$24,000
$26,000
The ACA interview has several sections, each with a specific purpose. We've provided examples of the
questions in each section, together with brief explanations of their purposes.
1) Preference for Levels (Required Section)
(The "ACA Rating" question type.)
First, the respondent rates the levels for preference. This question is usually omitted for attributes (such
as price or quality) for which the respondent's preferences should be obvious. (When you input the
attributes and levels, you can specify that the order of preference for levels is "best to worst" or "worst to
best" and the ratings question is skipped for such attributes.) The screen may look like the following:
ACA Help 715
(The rating scale can be defined from 2 to 9 points. We suggest using at least 5 scale points. In any
case, it is probably not wise to use fewer scale points than the number of levels in any one attribute for
which the Rating question is asked.)
The respondent is required to check one radio button per attribute level. If a level is skipped, ACA will
prompt the respondent to complete the question prior to moving to the next page in the interview.
2) Attribute Importance (Optional Section)
(The "ACA Importance" question type.)
Having learned preferences for the levels within each attribute, we next determine the relative importance
of each attribute to this respondent. This information is useful in two ways. First, it may allow us to
eliminate some attributes from further evaluation if the interview would otherwise be too long. Second, it
provides information upon which to base initial estimates of this respondent's utilities.
(Notes: if using ACA/HB for part-worth utility estimation, you may elect to omit the Importance section.
If the standard format of the Importance question as described below doesn't fit your needs, you can
also customize your own importance questions. Getting the Importance section right is critical. For
example, we strongly recommend educating respondents regarding the full array of attribute options prior
to asking them importance questions, so they use the scale consistently and know ahead of time which
attribute(s) should be reserved to receive the highest scores.)
As a matter of fundamental philosophy, we do not ask about attribute importance with questions such as
"How important is price?" The importance of an attribute is clearly dependent on magnitudes of
differences among the levels being considered. For example, if all airline tickets from City A to City B
were to cost between $100 and $101, then price couldn't be important in deciding which airline to select.
However, if cost varied from $10 to $1000, then price would probably be seen as very important.
Our questioning is based on differences between those levels the respondent would like best and least,
as illustrated:
716 SSI Web v8
(The rating scale can be defined from 2 to 9 points. We suggest using at least 4 scale points.)
It is worth mentioning that there are different ways to phrase the "importance" question, and indeed
referring to attribute "importance" is sometimes difficult for some respondents to comprehend. How the
wording of this question should be phrased depends on the audience. Other possibilities with the ACA
software include:
The "Regrets" Format:
The "Unexpected Windfall" Format:
At this point we have learned which attributes are most important for this respondent and which levels are
preferred. From now on, the interview is focused on those most important attributes and combinations of
the levels that imply the most difficult trade-offs.
ACA Help 717
3) Paired-Comparison Trade-Off Questions
(The "ACA Pairs" question.)
Next, a series of customized paired-comparison trade-off questions is presented. Up to this point in the
interview, we have collected "prior" information; no conjoint analysis has been involved. The Pairs
section elicits the conjoint tradeoffs. In each case the respondent is shown two product concepts. The
respondent is asked which is preferred, and also to indicate strength of preference. The example below
presents concepts differing on only two attributes. Although concepts may be specified on up to five
attributes, simple concepts like these present an easy task for the respondent, and are a useful way to
begin this section of the interview.
(The rating scale can be defined from 2 to 9 points. We suggest using at least 7 scale points.)
The number of attributes appearing in each concept is specified by the author, and can be varied during
the interview. Concepts described on more attributes have the advantage of seeming more realistic. It is
also true that statistical estimation is somewhat more efficient with more attributes.
However, with more attributes, the respondent must process more information and the task is more
difficult. Experimental evidence indicates that a tradeoff occurs: as the number of attributes in the
concepts is increased, respondents are more likely to become confused. It appears best to start with
only two attributes. Most respondents can handle three attributes after they've become familiar with the
task. Preliminary evidence suggests that beyond three attributes, gains in efficiency are usually offset
by respondent confusion due to task difficulty.
The computer starts with a crude set of estimates for the respondent's part-worths, and updates them
following each submitted page (you can specify how many pairs questions will be completed per page).
The crude estimates are constructed from the respondent's preference ranking or rating for levels, and
ratings of importance of attributes. Each pairs question is chosen by the computer to provide the most
incremental information, taking into account what is already known about this respondent's part-worths.
The interview continues in this manner until a termination criterion (specified by the author) is satisfied.
Every time the respondent completes a page of pairs question(s), the estimate of the respondent's part-
worths is updated. Updating the part-worths improves the quality of subsequent pairs questions. We
strongly encourage you to place page breaks often throughout the pairs questions. If you can count on
fast load times (respondents will be using fast web connections), we suggest page breaks after each
pairs question.
718 SSI Web v8
4) Calibration Concepts (Optional Section)
(The "ACA Calibration" question type).
Finally, the computer composes a series of "calibration concepts" using those attributes determined to
be most important. These concepts are chosen to occupy the entire range from very unattractive to very
attractive for the respondent. The respondent is asked a "likelihood of buying" question about each.
We first present the concept we expect the respondent to like least among all possible concepts, and
the second is the one we expect to be liked best. Those two concepts establish a frame of reference.
The remaining concepts are selected to have intermediate levels of attractiveness.
This information can be used to calibrate the part-worth utilities obtained in the earlier part of the interview
for use in Purchase Likelihood simulations during analysis. Conjoint part-worths are normally determined
only to within an arbitrary linear transformation; one can add any constant to all the values for any
attribute and multiply all part-worths by any positive constant. The purpose of this section is to scale the
part-worths non-arbitrarily, so that sums of part-worths for these concepts are approximately equal to
logit transforms of the respondent's likelihood percentages.
The screen format is:
If you plan only to conduct share of preference simulations, and particularly if you are using the ACA/HB
system for hierarchical Bayes estimation, there is no need to include calibration concepts in your ACA
survey. See Estimating ACA Utilities for more information on the use of calibration concepts.
Setting Page Breaks in ACA Surveys
ACA lets you include multiple questions per page. If the page cannot fit on the respondent's screen, a
scroll bar is provided for respondents to scroll down the page. At the bottom of each page is a "Next"
button that respondents click when they have completed the current page. Page breaks must occur
between each of the major ACA sections:
Rating
Importance
ACA Help 719
Pairs
Calibration Concepts
You may include additional page breaks within each section if you think it is more convenient for your
respondents and provides a more desirable overall layout. We suggest you include page breaks within
the Pairs section so that the utilities can be updated and the subsequent pairs have an opportunity to
become more customized and better balanced. The number of pairs questions to include per page
represents a tradeoff between the benefits of updating versus the time required to process and receive a
new page of pairs questions over the respondent's Web connection.
720 SSI Web v8
5.1.6 Defining Attributes and Levels

The base CBC software is limited to 10 attributes with at most 15 levels of an attribute, though we
encourage you to develop attributes that have about 5 or fewer levels whenever possible. (The advanced
design module extension of CBC allows up to 100 attribute, each with 254 levels).
(Hint: you can select a list of attributes or levels from Word or Excel and paste into SSI Web using the
Paste list member(s) from the clipboard icon. This can save a great deal of time.)
In conjoint experiments, we show respondents product concepts described using different combinations
of attribute levels and ask them to somehow express their preferences for those concepts. One way of
thinking about conjoint analysis is that we are conducting a designed experiment for each respondent to
see how preferences vary as we change attribute levels.
After observing how respondents evaluate products in response to changes in the underlying attribute
Conditional Relationships…
(Not available within ACA studies.)
The conditional relationships button lets you associate conditional prices or graphics with one or
more attributes in your study.
Conditional Graphics: Sometimes, you may want to associate a graphic with an attribute or
multiple attributes. For example, if a combination of brand and form factor make up a graphic,
one can specify the two attributes separately and associating a conditional graphic with their
combined levels. This permits estimation of separate effects for brand and form factor.
Guidelines

overlapping meanings is wasteful and can lead to biased part-worth utilities.
Furthermore, levels for related attributes often cannot combine naturally and freely with one another.
Though it can lead to more realistic interviews, it is often detrimental (and sometimes fatal) to prohibit
levels from occurring with others.
ACA Help 721
Optional Features:
Sunroof
Extended warranty
a) Create an attribute with levels representing all potential combinations of these features.
This results in an eight-level attribute, if you include the option that none of these features is
available.
Option (a) adds seven parameters to the model (after dummy-coding) and forces the measurement of an
explicit three-way interaction. With the more complex model definition, we can investigate whether there
are diminishing returns by bundling the features. By splitting the options into three distinct binary
attributes (Option b), only three parameters are added to the model; however, interactions are not
automatically measured.

when considering prohibiting levels from occurring with one another.
Too many prohibitions can lead to confounded effects and the complete inability to calculate stable
utilities. It is better to prompt respondents that they will see combinations during the interview that are
not yet available in the market or that seem unlikely. You can urge respondents to answer as if these
products were actually available today.
and package types:
Brand:
Sawtooth Spritz
Kong Kola
722 SSI Web v8
Splut
Package Type:
2-liter bottle

more levels tend to get more importance.
ACA Help 723
5.1.7 Specifying Prohibitions in ACA

This dialog lets you specify which combinations of attribute levels (or attributes altogether) may not be
shown together in pairs (and for ACA questionnaires in calibration concept questions).
A Warning
Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions is

currently exists in the market. Sometimes a product is shown with all the best features at the lowest
price; or two attribute levels that would not naturally occur in the real world are paired together. The
inclination is simply to prohibit such combinations. We urge you to exercise restraint when considering
prohibiting pairs.
Too many prohibitions can lead to imprecise part-worth estimation. It is better to prompt respondents
that they will see combinations during the interview that are not yet available in the market or that seem
unlikely. You can urge respondents to answer as if these products were actually available today.
How Many Prohibitions Can I Safely Add?
A common question that users ask is "how many prohibitions can I specify without seriously damaging
the results of my study?" This cannot be answered without more information. For it is not the sheer
number of prohibitions but the specific pattern of prohibitions that more directly affects the degree of
correlation among the attribute levels in the design matrix, and thus the design efficiency.
Assume that the researcher wants to specify 3 prohibitions between a 2-level attribute and a 3-level
attribute. There are just 6 possible combinations that can occur when combining those two attributes. If
the researcher prohibits 3 of those combinations from occurring, this eliminates half of the possible
combinations between those attributes. There would probably be less damage to the efficiency of the
design if three total prohibitions were allocated across three separate attribute combinations (one per pair
of attributes).
As a side note, the researcher in this instance may decide to specify the three non-prohibited
combinations of those two attributes as a single attribute, rather than as two separate (and not
independent) attributes. The main drawback of this approach is that after combining these two
attributes, the researcher will not be able to compute the attribute importance or the relative part-worths
of each attribute independently.
724 SSI Web v8
5.1.8 Setting ACA Control Parameters

You control the way the ACA questionnaire is implemented using the Design tab on the ACA Exercise
Settings dialog.
This dialog provides the following areas:
Priors Randomization
Pairs Settings
Calibration Settings
Importance Settings
The controls on each dialog are described below.
Priors Randomization
Randomize Level Order in Rating Questions

You can randomize the order of attribute levels (such as levels of brand, color, style, etc.) that are
displayed in ACARAT (level ratings) questions. This can control for order bias.
Randomize Attribute Order in Rating and Importance Questions

If you select this option, the order that attributes are presented in the ACARating (level ratings) and
ACAImportance (importance) questions is randomized. (The same randomized order is used for
both sections, for a given respondent.)
Pairs Settings
Maximum Attributes for Pairs Section

This controls how many attributes will be carried forward to the pairs questions. If you specify as
many attributes as are in your study, all attributes will be used within the pairs questions. If you
specify a number n which is less than your total number of attributes, only the n most important
attributes (as rated by the respondent in the Importance question) will be used in the Pairs section.
You can take as many as 20 attributes forward to the Pairs section. We generally recommend
taking all attributes (up to 20) forward to pairs questions if you are using ACA/HB estimation. If
using the standard OLS estimation, we suggest you bring around 8 to 12 attributes into Pairs.
However, we have not conducted any research to confirm that about 8 to 12 is optimal, and it
depends on the number of Pairs questions you plan to ask. The more Pairs questions asked, the
more attributes you might consider carrying into the Pairs section.
Number of Pairs Questions (0-50)

This number controls how many total conjoint pairs are to be asked. We suggest that you ask a
number of pairs equal to 3 (N - n - 1) - N, where:
N = total number of levels taken into the Pairs section

n = total number of attributes taken into the Pairs section
There are many times when the number of pairs suggested by the above formula are more than
respondents can manage. (Most ACA surveys will have between 15 to 30 pairs questions.) We
suggest you determine how many pairs questions respondents can reliably answer, and not try to
ask more pairs than is prudent.
ACA Help 725
There are also instances with very small ACA designs in which the above formula leads to very few
suggested pairs (i.e. sometimes six or fewer). In that case, we recommend that you increase the
number of pairs questions to a number that your respondents can reasonably answer.
Number of Pairs Questions at Each Stage

This value controls how many pairs questions are asked before the next degree of complexity is
implemented. For example, if you want the first 10 pairs questions to show two attributes at a
time, and the next 10 pairs to show three at a time, you should specify "10."
Number of Attributes in Each Pairs Question in the First Stage (2-5)

This value specifies how many attributes will be presented at a time throughout the first stage. We
recommend showing two attributes at a time in the first few pairs.
Number of Attributes in Each Pairs Question in the Last Stage (2-5)

Controls the maximum complexity (number of attributes shown) in conjoint pairs questions. We
generally recommend that three be the maximum number of attributes in the last pairs. Although,
if attribute text is particularly short and you feel that respondents would not become overly
confused or fatigued, you may wish to display four or five attributes in the last pairs.
Calibration Settings
Number of Calibration Concepts (0,3-8)

Controls how many calibration concept questions will be shown. If you want to use the Purchase
Likelihood model during analysis, you should include at least three Calibration Concepts. We
recommend using at least five if you plan to include this section. If you plan to use ACA/HB to
compute part-worths (hierarchical Bayes) and if you do not need to run the purchase likelihood
model, you should probably not include a Calibration Concepts section.
The total number of calibration concepts cannot exceed the number of attributes (or the number of
attributes in each concept--see below) plus one.
Number of Attributes in Each Concept (2-8)

Specifies how many attributes are to appear within each Calibration Concept question. Only the
most important attributes are included (as rated in the Importances section). We recommend
including no more than about six attributes.
If you have specified prohibited pairs, there are situations in which fewer attributes than the number
you specify are actually displayed within the Calibration Concept section.
Importance Settings
Use Default ACA Importance Questions

By default, an importance question is asked for each attribute.
Skip Importance Questions (Advanced)

Advanced users may wish to skip the importance questions (ACA/HB estimation required). We
strongly suggest you read the "Omitting the 'Importance' Question" documentation within this
online manual and also the article in the Technical Papers section of our website entitled "The
'Importance' Question in ACA: Can It Be Omitted?" If you select this option, you then choose
among the following advanced options:
726 SSI Web v8
Use Group Means for Prior Importances

You can use importances based on previous respondents (collected on the same server or
CAPI installation). These importances are used for selecting the attribute level combinations
in the conjoint pairs. ACA/HB estimation is required.
Number of Preliminary Respondents before Using Group Means

This defines how many respondents must complete the survey prior to using group
means. Before group means are used, we assign prior equal importances.
Set Prior Importances Equal for All Attributes

If you select this option, all attributes will be assumed to have equal prior importances. The
order of attributes used in the pairs section will be randomized. Utility balance for conjoint
pairs will be based on the level ratings/rankings already established in the questionnaire or by
the author when setting a priori attribute level order, and then updated by the respondent's
subsequent pairs questions. ACA/HB estimation is required.
Set Prior Importances Based on other Questions

Some users may wish to customize the importance questions. You can use any series of
questions or variables that reflects positive values (decimal values allowed). Click the
Specify Questions... button to associate a question/variable with each attribute importance.
ACA Help 727
5.1.9 How ACA Data Are Stored

ACA stores information in its data file detailing the attribute level combinations shown at each stage of
the interview along with the respondents' answers. You can download the data file from the Web server
using SSI Web's Admin Module, or accumulate the files from different PCs if using CAPI-based data
collection. Then, using SSI Web you can export the responses to the "standard" (non-conjoint related)
questions to a text-only fixed/delimited format using File | Export Data | All Data .
You can also export the ACA data to an "audit trail" .ACD file by clicking File | Data Management and
the by clicking Add Job... and under Export Settings selecting Type ACA. When you export the job, a
studyname.ACD file is written that records information about the combinations of levels displayed at
each stage of the interview along with respondent answers. Our ACA/HB software reads the .ACD file
when estimating part-worth utilities under HB.
To obtain OLS estimates for the attribute levels, you should run ACA's utility calculator. The utility
calculator reads the information from the respondent data file (studyname_data.sqlite), computes the
part-worths, and saves the results to a STUDYNAME.utl file. This format is identical to the .UTL files
used in ACA version 4 for the PC. Click HERE to view the formats for the .UTL and .ACD files.
728 SSI Web v8
5.1.10 Computing Utilities and Analyzing ACA Data

Before computing part-worth utilities, you must first have downloaded a data file (studyname_data.sqlite)
from your Web server (or accumulated the data from PCs), and placed it within your study directory,
where all other study-related files are resident. We use Ordinary Least Squares to compute part-worth
utilities. The resulting part-worths are saved to a STUDYNAME.utl file, which is a text-only file with a
specific format. (Please see Appendix A for more information on interpreting conjoint analysis results.)
Select Analysis | Calculate ACA Utilities using Ordinary Least Squares.... Choose Settings.... You
are able to specify a number of options on this dialog:
Respondent Numbering Method

You can choose from five options for assigning respondent numbering. This lets you specify how
the respondent numbers (case IDs) are to be generated. User Name is one of the passwords from
the passwords module. You should only choose this if these values were purely numeric (no
letters). Password is also one of the passwords from the passwords module. You should also
only use this if the passwords were purely numeric (no letters). Internal Interview Numbers are
those that the system assigns when respondents complete the survey over the Web. There can be
gaps in the sequence if some respondents started a survey but didn't finish. Sequentially Assigned
Numbers (default) are integers 1 through n. Other lets you choose another numeric variable to use
for respondent numbering. This last option is useful if you are linking surveys from another source
and need to merge the data sets by a "passed-in" variable.
Respondents to Include
All Respondents: Choose this option to export all respondents: both complete and
incomplete. Data for questions that were not completed are left blank.
Qualified/Completes Only: Choose this option to export only qualified complete data records.
A survey is considered qualified complete when respondents click the Submit (Next) button
on a terminating question that is marked with a qualified status.
Disqualified and Incompletes Only: Choose this option to export only disqualified and
incomplete data records. A survey is considered disqualified when respondents click the
Submit (Next) button on a terminating question that is marked with a disqualified status. A
survey is considered incomplete if respondents have not reached a terminating question.
Prepare for Utility Computation...

Click Prepare for Utility Computation... to compute utilities and save the STUDYNAME.utl file to
the current folder. A report appears summarizing the results of the computation, and (on a
separate tab) listing the respondent utilities one respondent record per row. You can save the
results to an Excel file by clicking Save As....
Output
When OLS estimation is completed, the output is displayed in a report window. The first (Summary) tab
displays the average part-worth utilities across respondents (normalized by the zero-centered diffs
method), and the average attribute importances. On the second (Individual Utilities) tab are the raw utility
data for each respondent, organized as one row per respondent. Click the Save As... button to save the
data as an Excel (.XLS) file.
Zero-Centered Diffs Method: The Summary tab displays the average rescaled utilities (as well as the
standard deviations) under the zero-centered "diffs" method. The diffs method rescales utilities so that
ACA Help 729
for each individual the total sum of the utility differences between the worst and best levels of each
attribute across attributes is equal to the number of attributes times 100. This normalizes the data so
that each respondent has equal impact when computing the population average.
A studyname ACA.utl file is also automatically generated, which is a text-only file with a specific
format. This file may be directly imported into the SMRT program for market simulations.
730 SSI Web v8
5.1.11 Is ACA the Appropriate Technique?

Many methods are available for collecting and analyzing conjoint data, and the researcher contemplating
a conjoint study must choose among them. We at Sawtooth Software have had many years of direct
experience with these methods, as well as the benefit of many conversations with users of our own and
other software. Based on that experience, we offer the following suggestions:
The Full Profile Method (such as used in CVA) was the first conjoint method introduced to the marketing
research community, and it is considered the original standard. Green and Srinivasan (1990)
recommended use of the full profile method when the number of attributes was six or fewer. Our opinion
is that this is a useful guideline, but there isn't a hard limit, and the number of attributes should depend
on the complexity of the attribute descriptions and respondent interest/knowledge of the product
category.
During the 1980s and 1990s, we tended to favor ACA for projects involving about six or more attributes.
As choice-based approaches with HB estimation have captivated the industry and shown their worth, we
have shifted our attention toward CBC and Adaptive CBC. Adaptive CBC combines some of the best
aspects of ACA of CBC, and the early evidence suggests that it is a valuable new technique. We have
not conducted nor seen research that directly compares ACA to Adaptive CBC, but we expect that
ACBC should have important advantages when the number of attributes is between 5 and 12. When the
number of attributes exceeds about 15, and pricing research isn't a goal, we don't know which method
would prevail.
Over the last fifteen years, it has become clear to us that the self-explicated importance question is the
aspect of ACA that either "makes or breaks" it. If the importance questions are asked poorly, they can
misinform utility estimation and be harmful. An example of poor execution is to not educate respondents
ahead of time regarding the array of attributes, and to ask the importance questions one-by-one on
separate screens. Some ideas for improving the quality of the importance questions are given in the
section of this documentation entitled Customizing the Importance Question.
ACA is a "main effects only" model, and assumes there are no interactions among attributes. Many
conjoint practitioners agree that one must remain alert for the possibility of interactions, but that it is
usually possible to choose attributes so that interactions will not present severe problems. Like other
conjoint methods, ACA can deal with interactions in a limited way by collapsing two attributes into one.
For example, we could deal with an interaction between car color and body style by cross-classifying the
levels:
Red Convertible
Black Convertible
Red Sedan
Black Sedan
However, if the attributes in question have many levels, or if an attribute (such as price, for example) is
suspected of having interactions with many others, then collapsing attributes will not be enough. In that
case too many parameters must be estimated to permit analysis at the individual level, and the most
common solution is to evaluate interactions by pooling data from many respondents.
ACA has been shown to have weaknesses in pricing research, where it often underestimates the
importance of price. Since about 1995, some ACA researchers have included price in their ACA studies,
but have adjusted its importance based on other information included in the survey. This was called the
"dual conjoint" approach, and there are articles on this in our Technical Papers Library on our website.
When ACA is executed well, with a good importance section implementation, it provides solid results for
ACA Help 731
applications such as product design and segmentation work. It can work with the smallest of sample
sizes, even in the case of revealing the preferences for a few or even one individual. Even though the bulk
of interest lately is in favor of CBC and ACBC, those who have become expert in ACA methodology and
ACA study execution will continue to use the method confidently in appropriate contexts for many years
to come.
For more information about selecting the appropriate conjoint method, see the article in the technical
papers library on our website entitled: "Which Conjoint Method Should I Use?" or use the Interactive
Advisor at http://www.sawtoothsoftware.com/method-advisor.
732 SSI Web v8
5.1.12 Staying out of Trouble with ACA

Here are some hints for staying out of trouble with ACA:
Proper execution of the Importance questions: Over the last 10 years, it has become clear to us
that the self-explicated importance question is the aspect of ACA that either "makes or breaks" it. If the
importance questions are asked poorly, they can misinform utility estimation and be harmful. An
example of poor execution is to not educate respondents ahead of time regarding the array of attributes,
and to ask the importance questions one-by-one on separate screens. Some ideas for improving the
quality of the importance questions are given in the section of this documentation entitled Customizing
the Importance Question.
Using too many prohibitions: ACA lets you specify that certain combinations of attribute levels
shouldn't occur together in the questionnaire. But if you prohibit too many combinations, ACA won't be
able to produce a good design, and may fail altogether. You can present combinations of levels that do
not exist in the market today, and including unusual combinations can often improve estimation of
utilities. During market simulations, you can avoid those combinations that seem unusual. In short,
prohibitions should be used sparingly.
Reversing signs of ordered attribute levels: If you already know the order of preference of attribute
levels, such as for quality or price, you can inform ACA about which direction is preferred and avoid
asking respondents those questions. You inform ACA by means of the a priori settings: Worst to Best,
or Best to Worst. A common mistake is to accidentally specify the wrong order, which can lead to
nonsensical data that can be difficult to salvage. To avoid this situation, take the interview yourself,
making sure that the questions are reasonable (neither member of a pair dominates the other on all
included attributes). Also, answer the pairs section with mid-scale values and then check to make sure
the utilities are as you expect them. If you do happen to misspecify the a priori order of levels, the ACA/
HB module can be used quite effectively to recompute the utilities and help salvage the situation.
Using ACA for pricing research when not appropriate: There are three aspects to this point.
All "main effects" conjoint methods, including ACA, assume that every product has the same
sensitivity to price. This is a bad assumption for many product categories, and CBC or ACBC
may be a better choice for pricing research, since they can measure unique price sensitivity for
each brand.
When price is just one of many attributes, ACA may assign too little importance to it. In a few
previously published articles, researchers have reported that it may sometimes be appropriate to
increase the weight that ACA attaches to price. This is particularly likely if the author includes
several attributes that are similar in the minds of respondents, such as Quality, Durability, and
Longevity. If redundant attributes like these are included, they may appear more important in
total than they should be, and other attributes, such as price, may appear less important than
they really are. This problem is exacerbated if a wide range for price is specified.
It is not a good idea to use the "share of preference with correction for product similarity" with
quantitative variables such as price. Suppose there are five price levels, and all products are
initially at the middle level. As one product's price is raised, it can receive a "bonus" for being
less like other products which more than compensates for its declining utility due to its higher
price. The result is that the correction for product similarity can lead to nonsensical price
sensitivity curves. This problem also can occur (but typically to a lesser degree) when using the
improved method for dealing with corrections for product similarity: Randomized First Choice
ACA Help 733
(RFC). We suggest conducting sensitivity analysis with the Share of Preference method when
modeling demand curves for quantitative attributes like price.
Using unequal intervals for continuous variables: If you use the ranking rather than the rating option,
ACA's prior estimates of utility for the levels of each attribute have equal increments. That works well if
you have chosen your attribute levels to be spaced regularly, for example with constant increments such
as prices of .10, .20, .30, or proportional increments such as 1 meg, 4 megs, or 16 megs. But if you use
oddly structured intervals, such as prices of $1.00, $1.90, and $2.00, ACA's utilities are likely to be
biased in the direction of equal utility intervals. This problem can be avoided if you use the ACA/HB
module to compute ACA utilities.
Including too many attributes: ACA lets you study as many as 30 attributes, each with up to 15
levels. But that doesn't mean anyone should ever have a questionnaire that long! Many of the problems
with conjoint analysis occur because we ask too much of respondents. Don't include n attributes when n-
1 would do!
Including too many levels for an attribute: Some researchers mistakenly use many levels in the
hope of achieving more precision. With quantitative variables such as price or speed, you will have more
precision if you measure only 5 levels and use interpolation for intermediate values. If you must measure
more than 5 levels, we strongly encourage you to use the ACA/HB module for estimating utility values. It
can do a much better job at measuring those levels that are not studied in detail by any one respondent.
Interpreting simulation results as "market share": Conjoint simulation results often look so much like
market shares that people sometimes forget they are not. Conjoint simulation results seldom include the
effects of distribution, out-of-stock, or point-of-sale marketing activities. Also, they presume every buyer
has complete information about every product. Researchers who represent conjoint results as forecasts
of market shares are asking for trouble.
Not including adequate attribute ranges: It's usually all right to interpolate, but usually risky to
extrapolate. With quantitative attributes, include enough range to describe all the products you will want
to simulate. It is a good idea prior to data collection to ask the client to list the product scenarios that
should be investigated in market simulations. This exercise can often reveal limitations or oversights in
your attribute level definitions.
Imprecise attribute levels: We assume that attribute levels are interpreted similarly by all respondents.
That's not possible with "loose" descriptions like "10 to 14 pounds," or "good looking."
Attribute levels not mutually exclusive: Every product must have exactly one level of each attribute.
Researchers new to conjoint analysis sometimes fail to realize this, and use attributes for which many
levels could describe each product. For example, with magazine subscription services, one might
imagine an attribute listing magazines respondents could read, in which a respondent might want to read
more than one. An attribute like that should be divided into several, each with levels of "yes" and "no."
734 SSI Web v8
5.2 Advanced ACA Questionnaires

5.2.1 Omitting the "Importance" Question
Background
Although ACA has proven to be a useful and popular technique over more than two decades, many
researchers have argued that the self-explicated importance section in the ACA "Priors" may be a weak
link. The self-explicated importances can be confusing to some respondents, and the tendency for
respondents to state that most attributes are important may flatten the final derived importances.
Flattened importances mean that the effect of attributes in market simulations is biased toward greater
similarity across attributes. As a result, ACA users sometimes report that critically important attributes
tend to carry too little weight, and attributes of very little consequence tend to carry too much weight.
This problem is exacerbated when there are very many attributes in the study.
Field Experiments
At the 2004 Sawtooth Software conference, we reported on an experiment in which we dropped the
importance questions altogether. The experiment was conducted within an actual commercial study and
included 20 total attributes and around 1500 respondents. (See "The 'Importance' Question in ACA: Can
It Be Omitted?" available within our technical papers library at http://www.sawtoothsoftware.com/support/
technical-papers). We tested two methods of obtaining prior attribute importance information for the
purposes of selecting attribute combinations to display in the conjoint pairs:
Assume equal prior attribute importances

Assign prior importances based on the average derived importances from previous respondents
We should note that in either case updating of part-worth utilities occurred per the standard practice for
ACA: once conjoint pairs had been answered by the respondent, this respondent's answers were added
to his/her prior information, and the updated part-worth utility estimates were used for selecting the next
conjoint pair(s).
We used ACA/HB to estimate final part-worth utilities, where the utilities were constrained by the within-
attribute level ranking/ratings but the importances were not constrained. Therefore, the only information
used for deriving the relative strength of attributes came from the conjoint pairs section. We found that
dropping the importance questions...
improved the prediction of shares for holdout choice sets.

resulted in degraded predictions of individual choices (hit rates),
resulted in more differentiated and different derived importances,
reduced the interview time.
And, we found that both methods of dropping the importances worked about equally well.
The margin of victory in predicting holdout shares would have been even greater had we used the time
savings from dropping the self-explicated importance section to ask even more pairs. Most researchers
and managers are more interested in the accuracy of aggregate share predictions rather than individual-
level classification rates. And, if a key deliverable is a summary of respondents' preferences using part-
worth utility or importance charts, dropping the importance questions should result in more valid
displays. Therefore, there are advantages to dropping importances. The advantages come, however, at
the expense of a bit larger sample size to stabilize the results compared to standard ACA. Also, we
recommend using the saved time to ask additional pairs questions.
ACA Help 735
We repeated the study in a different context (international study, with 10 attributes) and confirmed the
findings of the first study. Based on these two favorable outcomes, we are offering the ability to drop the
importance question in the commercial version of ACA. We hope that providing this added functionality
will encourage further research in this area.
Using Importances from Prior Respondents
We should note that in the second experiment referenced above we improved the method for using prior
group importances for selecting the design in the conjoint pairs (relative to that used in the first
experiment). Rather than use the previous respondents' group importances as a definitive rank order for
arranging the attributes in the cyclical plan used for the first-stage 2x2 pairs, we assigned the rank order
according to random draws from a rectangular distribution, such that the probability of an item achieving
a high rank-order position was proportional to its prior importance score. This leads to greater variety in
the conjoint pairs design (especially in the first pairs showing two attributes at a time, where the
attributes are arranged in a circle, from best to worst), while still using information regarding the relative
strength of the attributes for selecting paired product concepts with a reasonable degree of utility
balance. Especially in cases where the researcher is taking a subset of the attributes into the pairs
section, selecting the subset of most important attributes with likelihood proportional to prior importances
ensures that even least important attributes (based on the prior group average) will still have a chance to
be evaluated in the pairs section. Since part-worth estimation is performed using HB, the respondents
that evaluate least important attributes in the pairs will contribute useful information for refining the
estimates of these part-worths across the population.
Once respondents have completed the interview, their final part-worth utilities (OLS estimation) are used
to derive importance scores, and those importance scores are used to update the previous population
importances (each respondent equally weighted). SSI Web writes a file containing updated group
importances named STUDYNAMEavgimp.cgi on the server during data collection. This file is reset (to
equal average importances) if the Reset Web Survey is selected in the admin module.
Recommendations
Generally, when you can share information across respondents on the same server (or CAPI installation)
we'd recommend using the method of assuming prior importances based on previous respondents. We
think that based on the improvement to the algorithm, this method will prove slightly better than
assuming equal importances for priors. We again stress that if dropping the importance section, much
of the time savings should be devoted to asking additional pairs questions.
When you omit the importance questions and use ACA/HB for estimation, the appropriate setting is to fit
Pairs only (fitting Priors is inappropriate), and to not use prior importances as constraints. However,
even if you did try to use the importances as constraints, we write out importances that are equal for all
respondents such that no between-attribute importance constraints would apply.
736 SSI Web v8
5.2.2 Customizing the "Importance" Question
The stated (self-explicated) importance questions in ACA have been the target of some criticism over the
years (see "The 'Importance' Question in ACA: Can It Be Omitted?" available within our technical papers
library at http://www.sawtoothsoftware.com/support/technical-papers). Some of the main arguments
against the current implementation of an importance rating scale in ACA are:
1. Respondents tend to say that many attributes are very important, which can result in "flatter"
final importances (lower discrimination) than reality.
2. Respondents may have a hard time understanding the framing of the question (assigning a
weight to the difference between extreme levels).
3. There is evidence that respondents may not answer the questions as reliably as the within-
attribute ratings (ACARAT questions).
Starting with version 6 of SSI Web, the Importance questions can be customized or omitted altogether,
drawing upon aggregate importances from previous respondents as "priors" to inform the pair generation
algorithm and subsequent calibration concepts. With this approach, ACA/HB is required for proper part-
worth utility estimation (since after dropping importances there is not enough information available at the
individual level to estimate part-worths under standard OLS estimation).
Some users, however, may not have the luxury of relatively large sample sizes needed to stabilize part-
worth estimates after dropping the Importance questions and furthermore may not be interviewing from a
common server (or SSI CAPI installation) such that prior importance scores could be shared and updated
across respondents. To help overcome some of the issues noted above, some researchers may wish to
substitute customized self-explicated importance questions and have ACA use that information for
informing the subsequent pair generation algorithm, calibration concepts, and final part-worth utility
estimation under OLS.
Custom Importance Question Ideas
One example that may work well is to first educate respondents about the full list of attributes (including
the range of levels for each attribute), and then ask the respondent to identify the one most important
attribute. Then, the researcher asks the respondent to assign a "10" to that most important attribute,
and to rate the other attributes with respect to that most important attribute. Respondents could be
instructed that if an attribute is half as important as the most important one, they should give it a "5."
As another example, assume that ACA is being used to interview just a few key business managers
within an organization regarding the desirability of different projects or capital expenditures. Further
suppose that these managers are analytically sophisticated and are quite comfortable with the notion of
assigning weights to attributes using a constant-sum (chip-allocation) question. Under these conditions,
the researcher might believe that these particular respondents will provide better information if using a
constant-sum allocation of importance across attributes rather than the standard ACA approach for
asking about attribute importance.
Implementing Custom Importance Questions
To implement a customized importance questions, click the Set Prior Importances Based on other
Questions/Variables option, and then click the Specify Questions... button.
For each attribute in the table, specify a question name (or variable value). For example, CS_1 returns
the value contained in variable CS_1. This might refer to a constant sum question named CS where the
ACA Help 737
first response option (_1) refers to the first attribute.
The customized importance scores must contain only positive numeric values. ACA assumes these
positive values have ratio scaling (i.e. that a score of 10 is twice as important as a score of 5). For
internal utility calculation purposes, the importance data are rescaled to the range 0 and 7. The rescaled
importance values (range of 0 to 7) are stored in the data file in place of the "skipped" variables
Exercise_Importance1 through Exercise_Importancen.
Variables containing customized importance information can come from most any question type within
SSI Web, asked at any point in the survey, including Free Format and hidden variables. They may even
be passed into the questionnaire through the "one-click access to survey."
738 SSI Web v8
5.3 Algorithms and Estimation Techniques

5.3.1 Details of ACA/Web Utility Estimation
The following describes ACA's method of part-worth utility computation using OLS (Ordinary Least
Squares regression).
ACA includes two major sections: self-explicated priors and conjoint pairs. Part-worth utility estimates
for these two sections are determined as follows:
Prior Utilities:
If rank orders of preference are asked (not currently offered in ACA) we convert them to relative
desirabilities by reversing them. For example, ranks of 1, 2, and 3 would be converted to values 3, 2,
and 1, respectively. If desirability ratings are asked (only method offered in ACA), those are retained "as
is."
The average for each attribute is subtracted to center its values at zero. For example, desirability values
3, 2, and 1 would be converted to 1, 0, and -1, respectively.
The values for each attribute are scaled to have a range of unity. For example, desirability values of 1, 0,
and -1 would be converted to .5, 0, and -.5.
The importance ratings for each attribute are scaled to range from 1 to 4, and then used as multipliers for
the unit-range desirability values. Thus, if an attribute has desirabilities of .5, 0, and -.5 and an
importance of 3, we get -1.5, 0, and 1.5.
The resulting values are initial estimates of part-worths, with these characteristics:
For each attribute the range of utility values is proportional to stated importance, and attribute
importances differ by at most a factor of 9 (the maximum number of scale points that can be specified for
importance questions).
Within each attribute the values have a mean of zero, and differences between values are proportional to
differences in desirability ratings or rank orders of preference.
Pairs Utilities:
An independent variable matrix is constructed with as many columns as levels taken forward to the pairs
questions. If a level is displayed within the left concept, it is coded as -1; levels displayed within the
right-hand concept are coded as +1. All other values in the independent variable matrix are set to 0.
A column vector is created for the dependent variable as follows: the respondents' answers are zero-
centered, where the most extreme value for the left concept is given a -4, and the most extreme value on
the right +4. Interior ratings are fit proportionally within that range.
Each pairs question contributes a row to both the independent variable matrix and dependent variable
column vector. Additionally an n x n identity matrix is appended to the independent variable matrix,
where n is the total number of levels taken forward to the pairs questions. An additional n values of 0 are
also appended to the dependent variable matrix. The resulting independent variable matrix and
dependent variable column vector each have t + n rows, where t is the number of pairs questions and n is
ACA Help 739
the total number of levels taken forward to the pairs questions. OLS estimates of the n attribute levels
are computed by regressing the dependent variable column vector on the matrix of independent variables.
Combining the Priors and Pairs Utilities:
The priors and pairs part-worths are normalized to have equal sums of differences between the best and
worst levels of each attribute across all attributes. (Note that the procedures described above
automatically result in zero-centered part-worths within attribute for both sets of part-worths.) The prior
part-worth utilities for levels also included in the pairs questions are multiplied by n/(n+t), where n is the
total number of levels used in the Pairs section, and t is the number of pairs questions answered by the
respondent. Any element in the priors that was not included in the Pairs section is not modified. The
pairs utilities are multiplied by t/(n+t). The two vectors of part-worths (after multiplication by the weights
specified above) are added together. These are the final part-worths, prior to calibration.
As a final step, the part-worth utilities are calibrated. It is widely recognized that the part-worths arising
from most conjoint methods are scaled arbitrarily, and that the only real information is contained in the
relative magnitudes of differences among them. So far, that is true of ACA as well.
However, the calibration concepts permit scaling of part-worths in a non-arbitrary way. In any product
category, some respondents will be more interested and involved than others. We attempt to measure
each respondent's degree of involvement by asking "likelihood of buying" questions for several concepts
that differ widely in attractiveness. The data obtained from those concepts is useful in three ways:
Correlations between part-worths and likelihood responses may be used to identify

unmotivated or confused respondents. Respondents whose likelihood responses are not
related to their part-worths should probably not be included in subsequent preference
simulations.
The level of likelihood responses may identify respondents who are more or less involved in
the product category. Respondents who give low likelihood responses even to concepts
custom-designed for them should probably be treated as poor prospects in simulations of
purchase behavior.
Variation in likelihood responses may also identify respondents who are "tuned in" to the
product category. A respondent who gives a low likelihood rating to the least attractive
concept and a high rating to the most attractive should be made to respond sensitively in
preference simulations, whereas someone who gives every concept similar likelihood values
should be made insensitive in simulations.
Each respondent is first shown what should be the least attractive possible concept, followed by the
most attractive possible concept, as determined from his or her own answers. Those two concepts
establish a frame of reference. The remaining concepts are of middling attractiveness. We determine an
intercept and one regression coefficient to apply to utilities to best predict logits of likelihood responses.
Those parameters are then used in a final scaling of utilities, which are therefore no longer arbitrarily
scaled. The procedure is as follows:
Let: p = the predicted likelihood of buying a concept

x 1 = the concept's utility based on the final "uncalibrated" utilities
b1 = the coefficient used to weight the utilities
a = an intercept parameter
The actual likelihood response is a single digit on a scale with n points. Responses are trimmed to the
740 SSI Web v8
range of 5 to 95. Then, using the logit transformation, we model buying likelihood as a function of the
respondent's utilities as:
ln [ p / (100 - p) ] ~ a + b1x 1
If the regression coefficient is less than 0.00001, we assume the estimation is faulty and use a
conservative positive value (0.00001). The r-squared (measure of fit) reported in the .UTL file is set to 0 in
such cases. If the calibration concepts section is not included in the interview, the respondent is
assumed to have answered 0 and 100 to the worst and best concepts, respectively, and 50 to the other
concepts.
To calibrate the part-worths, each is multiplied by b1. The intercept a is divided by the number of
attributes, and the quotient added to the part-worth for every attribute level. The part-worths can be
added up and antilogs of their sums are predictions of odds ratios for claimed likelihood of purchase of
any concept, just as though that concept had been included in the calibration section of the
questionnaire.
A Note about Hierarchical Bayes Estimation
OLS has been successfully used in ACA calculations for well over a decade. However, a new technique
called hierarchical Bayes estimation provides a more theoretically satisfying way of combining
information from priors and pairs. The results are also usually better from the practical standpoint of
improved predictions of holdout questions. We recommend that the interested reader investigate ACA/
HB by downloading the technical paper from our Web site (http://www.sawtoothsoftware.com/download/
techpap/acatech.pdf).
ACA Help 741
5.3.2 Choosing the Next Paired-Comparison Question

The first part of an ACA interview is concerned with screening the attribute levels and learning enough
about the respondent's preferences to construct initial part-worth estimates. After that is done we begin
the paired-comparison section, in which pairs of concepts are shown and preference questions are
asked. Following each response (submitted page of pairs questions) we update our estimates of part-
worths and then decide what pair of concepts to present next.
The number of possible concepts is very large, and we need some reasonably efficient procedure to
choose a pair of them at each stage that will be most beneficial in some way. There are several
principles to keep in mind when thinking about how to choose concepts.
Concepts should be chosen by a method that gives the author as much control as possible
over the interview, in terms of the complexity of the concepts and the number of questions
asked.
The design should be as "balanced" as possible. Observations should be spread as evenly

as possible over all attribute levels, and the columns of the design matrix should be as
nearly orthogonal as possible.
We should ask the respondent questions that require careful consideration. There is no
point in asking questions for which we already know the answer, such as "High quality at a
low price" versus "low quality at a high price." We learn more if we choose concepts nearly
equal in attractiveness.
Our procedure addresses these points. The author may specify the number of attributes to appear in
each concept. The range is from two to five. It is possible to start with only two attributes per concept
and, after the respondent has gained experience, to increase their complexity.
The concepts in a pair always have different levels of the same attributes. Our procedure for choosing
those concepts is:
Count the number of times each pair of attributes has appeared together in any concept.
Pick a set of attributes at random from among those whose members have previously
appeared together the fewest times.
For each of the chosen attributes, repeat similar logic to find levels that have been paired
least frequently.
Examine all possible ways of combining these levels into concepts (with just two attributes
there are only two possible ways; with 5 attributes there are 16 ways. Find the pair of
concepts most nearly equal in attractiveness, using the current estimates of the
respondent's part-worth utilities.
Randomly determine which concept will appear on each side of the screen.
Accordingly, ACA presents pairs of concepts that are as nearly equal as possible in estimated utility. At
the same time, constraints are imposed to ensure that the overall design is nearly orthogonal. Within
concepts, each pair of attributes is presented with equal frequency, and within each attribute, each pair
of levels is presented with equal frequency. In addition, if the paired-comparison questions show only
two attributes at a time, further steps are taken to insure that the overall design is "connected."
Such an approach has these benefits:
1) It gives the respondent the impression that the system is paying attention to his or her
742 SSI Web v8
answers, and it seems to be asking increasingly insightful questions.
2) It helps keep the respondent operating within the defined range of a response scale rather
than at its ends.
3) It provides data on "tight" inequalities if estimation is later to be done by nonmetric

methods.
ACA lets the author specify certain pairs of attribute levels that must not appear together in the same
concept. The procedure described above is modified slightly to take account of such prohibitions. When
concepts are described on only two attributes, ACA chooses the first few questions in a slightly different
way. (When the concepts are described on only two attributes, it would be possible to blunder into a
design in which the attributes would be divided into subsets in such a way that those in one subset
would never be paired with those in another subset. Such designs would provide no information about
the relative importance of attributes in different subsets, and ACA automatically corrects the design in
such a situation.)
ACA's designs usually have good statistical efficiency, although they are not strictly orthogonal.
Statistical efficiency is increased as more attributes are used in each concept, and it is also possible to
produce concepts more nearly equal in attractiveness when there are more attributes with which to work.
However, using larger numbers of attributes has the unfortunate consequence of making the questions
more complicated, and respondents are more easily confused.
Both anecdotal and experimental evidence has shown that it is usually best to start with only two
attributes per concept and, after a few pairs, to increase the number of attributes to three. Beyond three
attributes, gains in efficiency are usually offset by respondent confusion due to task difficulty.
ACA Help 743
5.3.3 Updating Utilities During the Interview

Estimates of the respondent's part-worth utilities are updated after each page of paired-comparison
responses is submitted.
First consider the general case of how least squares regression coefficients can be updated to include
the effect of an additional observation.
Let X be a matrix of predictor variables with a row for each of n observations and a column for
each variable.
Let y be a vector of responses for the first n observations.
Let z' be a row vector of predictor values for a new observation, appended as a row to X.
Let r be a response for the new observation.
Considering only the first n observations, we have the regression equation:
Xb ~ y
n
where
-1
b = (X'X) (X'y) (1)
n
is the vector of coefficients that would be obtained by least squares estimation based on the first n
observations.
Now consider adding one observation. The expanded layout is:
| X | | y |
| | b ~ | |' (2)
| z'| n+1 | r |
where
-1
b ~ = (X'X + z'z) (X'y + zr)
n+1
is the least squares estimate based on n+l observations. Suppose we already have bn , X, y, z, and r,
and we want to obtain bn+1. First consider an identity. Let
-1
744 SSI Web v8
v = (X'X) Z (3)
Then it can be shown that
-1 -1 v'v
(X'X + z'z) = (X'X) - ----- (4)
1 + v'x
Substituting into equation (2), we get
r - z'b
n (5)
b = b + v --------
n+1 n 1 + v'z
Equation (5) gives a formula for updating the estimate of part-worths following each response, a relatively
easy computation since the numerator and denominator on the right are both scalars. We must also
update the inverse as in equation (4). That is also fairly easy since the vector v is already available. If
we are dealing with k attribute levels, then an updating cycle requires about 2k (k + 1) multiply and add
operations. This is a significant savings when compared to the cost of re-estimating "from scratch" after
each response, and the final results are identical.
Now consider how this scheme is applied to the specific situation in ACA:
Before the first updating we set X equal to the identity matrix and both bn and y equal to
the initial utility estimates.
The vector z consists of plus and minus 1's and 0's. An element equals 1 if the
corresponding attribute level appeared in the concept on the right of the screen, -1 if in the
concept on the left of the screen, and 0 if that level did not appear in either concept.
The response r is coded so that +4 means "strongly prefer right," -4 means "strongly prefer
left," and 0 means indifference.
ACA Help 745
5.3.4 ACA/HB Estimation
HB estimation is considered the "gold standard" for ACA part-worth utility estimation, exceeding the
quality of the default OLS estimation. The benefits of HB estimation for ACA include:
1. Greater precision of estimates for each individual,

2. It relaxes the assumption of equidistant part-worths in the Priors for a priori ordered attributes,
3. Improved accuracy of part-worths for predicting holdout concepts,
4. It provides a theoretically more defensible approach for combining self-explicated and conjoint data.
We recommend that the interested user refer to the ACA/HB documentation (or the ACA/HB Technical
Paper, available in our Technical Papers library at www.sawtoothsoftware.com) for more details about
ACA/HB and its settings dialog.
Almost all of the capabilities of the standalone ACA/HB product are available within the SSI Web
interface, except:
1. Ability to save draws

2. Unacceptable extrapolation
3. Ability to restart interrupted HB runs
4. Advanced output files are not written (.dra, .cov, .hbv, .log)
Output Files
When HB estimation is completed, the output is also displayed in a report window. The first (Summary)
tab displays the average part-worth utilities across respondents (normalized by the zero-centered diffs
method), and the average attribute importances. On the second (Individual Utilities) tab are the raw utility
data for each respondent, organized as one row per respondent. Click the Save As... button to save the
data as an Excel (.XLS) file.
Zero-Centered Diffs Method: The Summary tab displays the average rescaled utilities (as well as the
standard deviations) under the zero-centered "diffs" method. The diffs method rescales utilities so that
for each individual the total sum of the utility differences between the worst and best levels of each
attribute across attributes is equal to the number of attributes times 100. This normalizes the data so
that each respondent has equal impact when computing the population average.
When you run ACA/HB through the SSI Web interface, a studyname ACA.hbu file is also automatically
generated in a subfolder of your study. The layout of this file is supplied in the ACA/HB manual. This file
may be directly imported into the SMRT program for market simulations.
746 SSI Web v8
5.4 ACA File Formats

5.4.1 Format of .UTL and .ACD Files
The .UTL File
The .UTL file contains the estimated part-worths for each respondent. Each respondent's data are
formatted across multiple lines. An example is as follows:
104 935 0 23 7
9.465 9.746 -9.999 -9.999 -9.999 -2.108 -1.305 -1.275 -0.547 -0.635
-9.999 -9.999 -9.999 -9.999 -1.665 -1.764 -4.172 -2.473 -4.131 -3.254
-2.473 -6.029 -4.550
Following is a description of each line:
Line Field Description Notes

Line 1 1 Respondent Number Number identifies record
2 Correlation A "-1" indicates Calibration Concepts were
not shown and as a result a correlation was
not calculated; consequently the Purchase
Likelihood model cannot be used for
simulations
3 Merged Variables Always zero when generated by ACA

4 Number of Utility Levels Number of levels in record
5 Interview Duration Always a zero in ACA
Line 2+
1 First Utility Level Value of level
2 Second Utility Level Value of level
... Remainder of Utility Levels And so on, for all existing utility levels.
There can be up to 18 lines of part-worth utility levels; each line has 8 columns per level, up to 10 levels
per line; up to a maximum of 180 levels. Each value has 3 places after the decimal point.
The STUDYNAME.ACD File (Audit Trail)
The STUDYNAME.ACD file contains an "audit trail" of the questionnaire for each respondent. The file
contains information about which questions were presented and about the responses. In earlier versions
of ACA, such as ACA v4 for DOS, this file also contained three sets of part-worths, computed at various
stages of the interview. We expect that most ACA users will use the .ACD file generated by ACA with
ACA/HB (Hierarchical Bayes estimation). ACA/HB does not use the three sets of part-worths available
in previous versions of the .ACD file, so we have inserted "0.000" values to maintain compatibility with the
prior .ACD file format.
There is one record for each respondent, in the order respondents appear in the STUDYNAME.UTL file.
Following is an example of one respondent's record in the audit trail file:
104 12 5 21
5 1 4 1 4 3 2 5 0 0 0 0
ACA Help 747
7 6 4 1 3 2 0 0 0 0 0 0
14 9 2 1 2 5 4 3 6 0 0 0
16 17 3 2 3 1 0 0 0 0 0 0
19 23 1 6 5 4 3 2 1 0 0 0
-1 -7 2 8 0 0 0 0 0 0 7
-6 -10 7 9 0 0 0 0 0 0 6
-9 -16 10 17 0 0 0 0 0 0 3
-17 -20 15 18 0 0 0 0 0 0 4
-1 -20 2 21 0 0 0 0 0 0 8
-6 -10 -19 8 9 18 0 0 0 0 4
-1 -16 -19 2 15 17 0 0 0 0 7
-1 -10 -16 2 9 15 0 0 0 0 7
-6 -10 -20 7 9 21 0 0 0 0 7
-8 -17 -21 6 16 17 0 0 0 0 7
-1 -7 -10 -15 2 8 9 17 0 0 7
-1 -9 -16 -19 2 10 15 21 0 0 7
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000
4 5
1 6 9 15 21 0.000 12
2 7 10 16 17 0.000 99
1 7 10 16 17 0.000 24
2 6 9 14 20 0.000 55
0.000 0.000 0.000 0.000
First line:
The first line contains the respondent number (104); the number of pairs that were shown in the Pairs
section of the interview (12), the number of attributes in the study (5); and the number of attribute levels
in the study (21).
Attribute levels are assigned consecutive numbers. Numbers are assigned beginning with the levels of
the first attribute, then the levels of the second attribute, and so on. For example, the chart below shows
how numbers are assigned to the levels of the attributes in the questionnaire that generated this data file:
Body Type Configuration Color Manufacturer Price

Two-Door Sedan (1) Rear-Wheel Drive (6) Red (9) US Manufacturer (14)
$16,000(17)
Four-Door Sedan (2) Front-Wheel Drive (7) Blue (10) Japanese Manufacturer (15)
$18,000(18)
Hatchback (3) All-Wheel Drive (8) Green (11) European Manufacturer (16)
$20,000(19)
Minivan (4) Yellow (12)
$22,000(20)
Convertible (5) White (13)
$24,000(21)
Second and next lines, the Importance Ratings:

748 SSI Web v8
Next follows one line for each attribute. Each line has 12 columns (unless there are more than 9 levels
for any attribute, in which case the layout is expanded to 18 columns). These 12 columns allow for the
maximum number of levels to be displayed (9), the two attribute levels that were shown, and the
respondent's answer to the Importance question.
The numbers in the first and second columns in each line represent the attribute levels that were shown
in the Importance Rating section. The number in the third column is the respondent's answer to the
Importance question. Then, the remaining 9 columns have numbers for each level that show the
preference rating for each level, in order, for as many levels as exist for the attribute. These numbers are
ratings, where the "1" indicated the least-preferred level. Any remaining columns contain zeros.
Using our example record that has 5 attributes, the lines just described are lines 2-6:
5 1 4 1 4 3 2 5 0 0 0 0
7 6 4 1 3 2 0 0 0 0 0 0
14 9 2 1 2 5 4 3 6 0 0 0
16 17 3 2 3 1 0 0 0 0 0 0
19 23 1 6 5 4 3 2 1 0 0 0
The second line represents the second attribute (Drive Configuration); the first number, 7 (Front-Wheel
Drive), and the next number, 6 (Rear-Wheel Drive) represent the attribute levels that were shown in the
Importance Rating Section. The third number, 4, is the respondent's answer to the Importance question.
The next numbers, 1, 3, and 2, indicate that the first level (Rear-Wheel Drive) was least preferred (1), the
second level (Front-Wheel Drive) was most preferred (3), and the preference for the third level (All-Wheel
Drive) was in between (2). The second attribute has only three levels, so the remaining columns on this
line contain zeros.
Next lines, the Paired-Comparisons:
Next follows one line for each of the pairs that were shown in the Pairs section of the interview. Each
line has 11 columns. These 11 columns allow for the maximum number of levels to be displayed in the
Pairs (5 in each of 2 concepts) and one column for the respondent's answer. The numbers represent the
levels that were shown. Numbers with negative signs indicate that the level appeared on the left side of
the screen, and those without negative signs indicate that the level appeared on the right side of the
screen. The final number on the line is the respondent's answer.
Using our example record that has 12 pairs, the lines just described are 7-18:
-1 -7 2 8 0 0 0 0 0 0 7
-6 -10 7 9 0 0 0 0 0 0 6
-9 -16 10 17 0 0 0 0 0 0 3
-17 -20 15 18 0 0 0 0 0 0 4
-1 -20 2 21 0 0 0 0 0 0 8
-6 -10 -19 8 9 18 0 0 0 0 4
-1 -16 -19 2 15 17 0 0 0 0 7
-1 -10 -16 2 9 15 0 0 0 0 7
-6 -10 -20 7 9 21 0 0 0 0 7
-8 -17 -21 6 16 17 0 0 0 0 7
-1 -7 -10 -15 2 8 9 17 0 0 7
-1 -9 -16 -19 2 10 15 21 0 0 7
The first of those lines represents the first pair; the first number, -1, indicates that level number 1, Two-
Door Sedan, appeared on the left of the screen. The second number, -7, indicates that level number 7,
Front-Wheel Drive, appeared on the left of the screen. Those levels were paired with the next two
ACA Help 749
numbers, which appeared on the right of the screen: 2 (Four-Door Sedan) and 8 (All-Wheel Drive).
The final number in the line, 7, indicates the response to this Pairs question.
Next lines, the Utilities Before Calibrations:
In ACA v4, the next lines represent the part-worths for each level of each attribute, in order, for this
respondent, prior to calibration. In ACA, we insert "0" values as placeholders. Each line has up to 10
part-worths.
Using our example record that has 23 attribute levels, the lines just described are 19-21:
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000
The values are displayed for each level.
Next lines, the Prior Utilities:
In ACA v4 the next lines represent approximate part-worths for each level of each attribute, in order, for
this respondent. These part-worths are based only on the information from the interview prior to the Pairs
section. In ACA, we insert 0 values as placeholders. Each line has up to 10 part-worths.
Using our example record, the lines just described are 22-24:
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000
Next lines, the Contribution to the Utilities from the Pairs Section:
In ACA v4 the next lines represent part-worths for each level of each attribute, in order, for this
respondent. These are the components of the part-worths from the Pairs section. In ACA, we insert 0
values as placeholders. Each line has up to 10 utilities.
Using our example record, the lines just described are 25-27:
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000
Next line, Number of Calibration Concepts and Number of Attributes:
The next line contains two numbers that report the number of Calibration Concepts shown to the
respondent and the number of attributes in each concept.
In our example record, this is line 28:
4 5
750 SSI Web v8
The first number, 4, indicates that 4 concepts were shown. The second number, 5, indicates that there
were 5 attributes in each concept.
Next lines, Calibration Concepts:
Next, there is one line for each calibration concept. For as many columns as there were attributes in
each concept, numbers indicate the attribute levels included in the concept. Then, the next column
shows the calculated utilities for the concept (in ACA we set those to 0.000). The last column contains
the respondent's purchase likelihood value for the concept.
In our example record, the lines just described are lines 29-32:
1 6 9 15 21 0.000 12
2 7 10 16 17 0.000 99
1 7 10 16 17 0.000 24
2 6 9 14 20 0.000 55
The previous line, 28, indicated there were 4 concepts, so there are 4 lines (29-32 ), one for each
concept. Line 28 also indicated there were 5 attributes in each concept, so each of the 4 lines uses the
first 5 columns to show which attribute levels were included in that concept. For example, the first five
numbers in line 29 indicate these levels were shown: 1 (Two-Door Sedan), 6 (Rear-Wheel Drive), 9 (Red),
15 (Japanese Manufacturer), and 21 ($24,000). In ACA v4, the next number indicates the total utility for
the concept. In ACA we put a value of 0.000 there as a placeholder. The last number, 12, represents
the respondent's purchase likelihood value for the concept.
Last line, Calibrating Coefficients:
In ACA v4 the last line contains the calibrating coefficients and measure of fit from the logit regression,
which is used to predict likelihoods of purchase and to weight the priors and pairs components when
estimating "optimal" utilities. In ACA we insert "0.000" as placeholders for those values.
In our example record, the line just described is line 33:
0.000 0.000 0.000 0.000

ACA Help 751
5.5 Real-Time Feedback to Respondents

There are some situations in which you might want to show respondents their attribute importances and/
or part-worth utilities at the completion of an ACA survey. ACA enables this by making the part-worths
and attribute importances available as system functions that you can display on the screen.
We suggest the following steps:
1. Generate a Text question after the ACA questions. This question can be in any position
following the last ACA Pairs question. This question will be used to display the ACA utilities
report.
2. There is a template Text/HTML question at the end of the sample ACA study file (located in
the Help menu) you can use as a starting point. Follow the patterns in that Text/HTML
question (it is helpful to know something about HTML coding when working with this file.)
Copy the coding patterns as many times as needed to match your attribute list. You will note
that the Text/HTML question refers to system functions that return part-worth utility and
importance values. The attribute importances are returned with the function
ACAImportance(Attribute#) (or you may use ACAStrictImportance(Attribute#) which controls
for reversals). The level part-worths are returned using the function ACAUtility(Attribute#,
Level#). In both cases, you will probably want to round them to the nearest integer for display
purposes (using the Round function). You can use these functions at any time during an ACA
interview (following the priors questions). If you are using the function within a text field (such
as Header1, Header2, etc.) that uses HTML instructions, you should enclose the function
within [% ... %].
[% ACAImportance(ExerciseName, 4) %] (Returns the value for the importance of

attribute 4)
[% ACAUtility(ExerciseName, 6, 3) %] (Returns the value for the utility of the third
level of attribute 6)
[% Round (ACAImportance(ExerciseName, 4)) %] (Returns the value for the

importance of attribute 4, rounded)
[% Round (ACAUtility(ExerciseName, 6, 3)) %] (Returns the value for the utility of
the third level of attribute 6, rounded)
3. When the survey runs on the Web, values will be inserted in place of the functions, and the
bars in the bar chart for importances will extend the appropriate lengths to reflect the relative
importance of each attribute.
Notes: Utilities are reported as "zero-centered diffs." Importances are reported as values that sum to
100. The utilities are generated by the efficient updating routine used when the ACA Survey runs on the
Web. These utilities are very similar to, but not exactly the same, as the final utilities generated by the
ACA OLS Utility Calculator.
For each page that includes a report of ACA utilities or importances, the utilities are re-computed "from
scratch". This can be computationally intensive for the server. If you show these values over many
pages, the computations must be repeated for each page.
If you change the number of attributes, levels in your study, or the text for attributes/levels, make sure to
manually change the HTML report to reflect those changes. There are so many aspects about ACA that
are automatic that it is easy to forget to do this.
752 SSI Web v8
5.6 Moving Data from SSI Web to SMRT for Analysis

from SSI Web into a preliminary analysis program such as hierarchical Bayes, and then afterward to
SMRT.
SSI Web project.
SMRT.
HB and Latent Class manuals for details) to produce a data file containing respondent part-worths (a
.HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting
utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager |
Import… and following the prompts to browse to and select your .HBU or Latent Class probability file
(.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin
simulations.
menu, select File | Export Data | All Data…, select the respondent numbering method (the
numbering method must match the method you chose when exporting the conjoint data or
estimating utilities), and export the data you wish to merge into SMRT (only numeric data may be
merged). Select either a .CSV file (comma-delimited text-only file) or a fixed text-only file (.TAB).
ACA Help 753
754 SSI Web v8
5.7 Analyzing ACA Data with the Market Simulator

You were given a companion piece of software called the Sawtooth Software Mark et Simulator, which is
also referred to as SMRT. You use this system to analyze your ACA part-worth utilities and to run
market simulations (what-if scenarios). Follow the instructions in the Sawtooth Software Market
Simulator manual to import and analyze your ACA data.
To get started:
1. Open the SMRT application. If you have a default installation, you start the application by
choosing Start | Programs | Sawtooth Software | Sawtooth Software SMRT.
2. Choose File | New to define a new study. If asked, specify ACA (Adaptive Conjoint
Analysis) as the type of study.
3. Select Analysis | Run Manager. Click the Import button. Under Files of Type, specify
ACA Utilities (.UTL). Browse for the .UTL file that the utility calculator generated, highlight it,
and click the Open button to import the data.
4. Once the utility data have been imported, choose Analysis | Market Simulator to start the
Market Simulator.
ACA Help 755
756 SSI Web v8
6 CVA Help
6.1 Getting Started with CVA
6.1.1 Introduction to CVA
Overview
The CVA System is Sawtooth Software's package for traditional full-profile conjoint analysis. The
traditional full-profile conjoint method was the first "flavor" of conjoint used by marketing academics in the
early 1970s (Paul Green and colleagues). It continues to be useful today, 40 years after its first
introduction.
CVA is a component within Sawtooth Software's SSI Web platform. CVA ("Conjoint Value Analysis")
includes:
A designer for creating the stimuli (product profiles) displayed in conjoint interviews
A questionnaire writer for formatting the conjoint questions for computer-based interviews or
paper-and-pencil surveys
Two utility calculators for estimating each respondent's part-worth utilities (strengths of
preference) for product features
Access to the standard SMRT mark et simulator that is also used in ACA (Adaptive
Conjoint Analysis), CBC (Choice-Based Conjoint), or ACBC (Adaptive CBC).
Optionally, users may purchase the CVA/HB add-on module for hierarchical Bayesian estimation of part-
worth utilities. This add-on module functions seamlessly within the SSI Web interface.
CVA is one of four conjoint software packages from Sawtooth Software. The other three are CBC,
Adaptive CBC (ACBC), and ACA. We have designed each package to bring unique advantages to
different research situations.
About Conjoint Analysis
Conjoint analysis is useful for learning how potential buyers of a product or service value its various
aspects or features. Conjoint analysis is equally useful for studying products and services (for
convenience, we use the word "product" in both cases).
Examples of questions that lead market researchers to use conjoint analysis are:
Which would be more valuable to the customer: longer warranty or more product
performance?
What would be the effect of raising our price 10%?
What would happen if we added another product to our line that had 20% less performance
but a 30% lower price?
When using conjoint analysis, a product is thought of as being made up of various attributes (sometimes
called "factors"). For example, automobile attributes may be: manufacturer, color, type of engine,
number of doors, and price.
Each attribute has several possible levels. For example, the levels of "number of doors" may be: two,
four, and five (to include station wagons). The levels of "engine type" may be: 4 cylinder, 6 cylinder, and
CVA Help 757
8 cylinder. (The word "feature" is often used in place of "attribute level.")
Conjoint data are collected from survey respondents by asking about their preferences for hypothetical
product concepts, described in terms of specific attributes and levels. In analyzing the data, one infers
the value to each respondent of having each possible attribute level. These estimated values are called
part-worths or utilities. With CVA, we can estimate each respondent's part-worth utility values for each
level of each attribute from data collected by the conjoint questionnaire. These values are then used to
make predictions about how respondents would choose among new or modified products. The goal is to
conclude what product changes would have the most beneficial (or least detrimental) effect on share of
preference, or which would maximize the likelihood that buyers would choose specific products.
In ratings-based conjoint analysis, product concepts can be shown to respondents one at a time, or they
can be presented in pairs. Pairwise presentation can be harder for the respondent, because each
question requires understanding two concepts rather than just one. However, the comparative nature of
the pairwise task may let the respondent make finer distinctions and contribute more information than
single concept (popularly called card sort) presentation.
The term full profile refers to product concepts that are described with respect to all the attributes being
studied. For example, if the products in a particular category had six important attributes, a full profile
product concept would describe the product on all six attributes.
CVA produces either single concept or pairwise full profile designs. It is questionable that respondents
can consider product concepts carefully that are described using more than about six attributes in CVA.
However, over the years we occasionally have heard of people wanting to use CVA's experimental
design program for applications that we had not envisioned. To give users this flexibility, and because
modern PCs offer processing speeds and memory not available before, over the last few years we have
decided to allow CVA to support up to 30 attributes, each with up to 15 levels. Despite this capacity, we
recommend that the number of attributes and levels for studies in practice be much smaller than these
limits. With more than about six attributes, the questions may become too complex for most
respondents to handle, and the number of questions that must be asked of each respondent may
become excessive. If more attributes and levels are necessary, it can be advantageous to show
respondents pairs of concepts described on subsets of the attributes (partial profile), as in our ACA
System for Adaptive Conjoint Analysis. But, there are other more advanced approaches for tackling
larger attribute lists that show promise as well, including partial-profile CBC and Adaptive CBC (ACBC).
About CVA
CVA features a variety of options and functionalities. Not all of these tools need to be used in a
particular project.
The authoring and design tools can be used to produce computer-administered or paper-and-pencil
questionnaires. We do not recommend the questionnaire design tool be used for studies with many
attributes. With too many attributes, the questions are likely to provide too much information to the
respondent, who may become overloaded and confused. For studies with more than about six to eight
attributes, we often recommend using the Adaptive CBC or the ACA System for Adaptive Conjoint
Analysis.
The calculation routines can be used to analyze questionnaire data and calculate respondent utilities.
The two standard methods are "Ordinary Least Squares" (OLS) regression and "monotone" (nonmetric)
regression. The OLS routine is appropriate for ratings-based data, and the monotone program for
rankings-based data. An optional advanced module called CVA/HB is available for hierarchical Bayes
(HB) estimation, which has consistently been shown to obtain slightly better results.
758 SSI Web v8
CVA may also be used to analyze traditional full-profile conjoint data not designed or fielded using
Sawtooth Software's tools. You may import designs and respondent data into the CVA system for
subsequent analysis using the utility calculation and market simulation tools.
Sawtooth Software's market simulator (SMRT platform) is shared across our four conjoint systems. The
market simulator lets the researcher specify a market consisting of several products defined in terms of
attribute levels, and then estimate the shares of preference that each product would receive. If you have
used single-concept presentation (described later) and purchase likelihood ratings, then the market
simulator can be used to estimate purchase likelihood for product profiles.
Question Formats
Pairwise Presentation
The default questionnaire layout in CVA is Pairwise Comparison. With pairwise presentation,
respondents are asked to compare two products, as shown below:
When respondents compare two products side-by-side, it can help them draw finer distinctions between
products offering different features. However, if two products are shown per question, respondents must
evaluate twice as much information as single concept designs (described below). We suggest the
pairwise design for most CVA conjoint projects.
Single Concept Presentation
Single concept presentation is commonly known as "card-sort." Respondents can be shown one
product at a time and asked to rate how likely they would be to purchase each. An example is shown
below:
CVA Help 759
This method is particularly useful if the "purchase likelihood model" is to be used in the simulator.
CVA's ordinary least squares (OLS) routine is appropriate for analyzing single concept ratings.
Hierarchical Bayesian (HB) estimation may also be used with single concept ratings.
Alternatively, each of a set of product profiles can be printed on individual cards, and respondents can be
asked to sort the cards from most to least preferred. In this case, the responses are rank orders of
preference, and CVA's monotone regression routine is suggested for developing utilities.
Evidence to date suggests that either questionnaire technique (pairwise or single-concept) produces
quite similar relative part-worths and importances.
760 SSI Web v8
6.1.2 CVA Tutorial and Example

Overview
This tutorial leads you step-by-step through a simple CVA (Conjoint Value Analysis) study, from thinking
about a marketing problem, planning a questionnaire and creating a new CVA study within SSI Web, to
entering a list of the attributes and levels, generating the questionnaire and analyzing the results.
CVA is a powerful program, and this tutorial only touches on the basics. You'll become aware of different
or more sophisticated approaches by studying other sections within this on-line help and by
experimenting with the CVA System.
(Note: this example and the data presented are purely fictional. The suggested steps represent
one approach to this type of marketing problem, but may not necessarily reflect the single "best"
way to conduct such a research study using the CVA system. This tutorial only uses 3 attributes,
to keep things simple and to make sure that it can run within student lab systems.)
Over the last five years, the Wendell State University football program has risen to prominence on the
national scene. As a graduate of this prestigious university and a rabid football fan, you have reveled in
the success. Wendell State's football coach, Biff Brierton, has developed arguably the best defensive
line in the nation, three years running. It is nearly impossible any more to buy tickets after the season
starts, as the somewhat undersized stadium sells out early.
While you are perusing the media guide for this year's team, the phone rings. It is your former marketing
professor, Dr. Coldridge. You've kept in contact with her over the last few years as you've built your
small strategic consulting practice. It seems Dr. Coldridge also has your favorite subject (Wendell State
football) on her mind.
The administration at Wendell State has been looking for ways to capitalize on the football team's
newfound success. One plan is to add a second tier to the East side of the stadium with additional
seating and between 25 to 35 luxury skyboxes. Each skybox would be an enclosed suite, seating from
12 to 24 people, depending on the size. The glass front of each climate-controlled skybox would provide
an excellent view of the playing field.
Dr. Coldridge explains that an initial study has already been completed that showed a high level of
interest among key alumni and corporate customers for the skybox concept at Wendell State stadium.
But further questions remain regarding how to design the skyboxes and the price premium to charge for
boxes nearer the center of the field. Knowing how often you apply conjoint analysis within your practice,
Dr. Coldridge asks if you are interested in assisting her with a follow-up study. You, of course, agree.
The following week, Dr. Coldridge visits your office. Always well organized, she has brought a list of
objectives for the research:
1) What is the relative demand for 12-seat, 18-seat, and 24-seat skyboxes? (The planners
need to know how many to build of each size)
2) How much price premium should be charged for skyboxes near the center of the stadium
(on the 40- or 50-yard lines) versus those near the sides (near the 20- to 30-yard lines)?
You have a limited sample frame to work with: just 180 names of key supporters who have the economic
CVA Help 761
means to buy skyboxes. Based on your past experience, you think that you may achieve a response
rate of around 25 to 50%. This would yield between 45 and 90 respondents.
Given the relatively small sample size, the limited number of attributes to study, and the objectives of
pricing research, you decide to use traditional full-profile conjoint analysis. The sponsors of the study
understand that the relatively small sample size will lead to good directional information, but the findings
will be accompanied by a relatively large margin of error. Because you are dealing with rating scales,
you cannot know the standard errors ahead of time, because you cannot predict the standard deviation of
the ratings. This is different from common opinion polls that measure proportions (i.e. the percent
projected to vote for candidate A or B) where there are well-known formulas for estimating the standard
errors of proportions given different sample sizes.
The next step for you and Dr. Coldridge is to formulate some attributes and levels. You suggest that she
ask the planners to specify some preliminary scenarios (an array of competitive skybox configurations/
features) to measure. These scenarios in the language of the CVA software will form a base case, or
variations from the base case, to be used in the market simulator.
The next week, Dr. Coldridge returns with three principal configurations the planners are considering.
She stresses that there can be some variations on the three themes (mixing and matching different
costs and field locations) but that these specifications definitely cover the full range of options that the
planners want to measure.
Satisfied with the progress so far, you both decide to formulate a list of attributes and levels. Based on
the information in the CVA documentation, you review the following list of ground rules with Dr.
Coldridge.
1. Attributes should be as close to independent of one another as reasonably possible. CVA

(like most conjoint techniques) assumes a simple additive model, and violations of simple
additivity within CVA are nearly impossible to detect.
2. Because attributes are independent of one another, the levels across all attributes should
be able to combine freely.
3. Levels within each attribute must be mutually exclusive. Skyboxes are described using a
Most conjoint analysis projects involve no more than about five levels per attribute. There is rarely a need
to measure more than about five levels of an ordered attribute, you explain, as SMRT's market simulator
lets you interpolate between measured levels during analysis. Including too many levels for an attribute
762 SSI Web v8
significantly lowers the precision of the utility estimates for each level. It also can lead to a problem in
conjoint analysis called the "Number of Levels" effect. Attributes defined on relatively more levels tend to
get an artificial boost upward in importance.
Using the white board in your office, you begin working on a preliminary list. First you write down the
three seating capacities. Because each level must communicate fully the meaning of the attribute, you
add some additional wording.
Seating
12-person seating capacity
Next, you turn to the location of the skybox. You note that there are only two variations available in the
base case scenario provided by the planners. Because you want to roughly balance the number of levels
across attributes, you decide to treat this also as a three-level attribute.
Location
On about the 20-yard line
Between the 40- and 50-yard lines
Finally, the issue of pricing: You don't like the fact that the prices are described in terms of ranges (e.g.
$15,000 to $20,000 for the least expensive options). This leaves too much interpretation up to the
respondent. Do we mean $15,000 or do we mean $20,000? You suggest that each level be defined at a
specific price, but that you cover the full range of possible prices to be studied:
Price
$15,000 per year
$22,500 per year
$32,500 per year
You next figure how many conjoint questions you would need to ask to achieve reasonably stable
estimates for each individual. The total number of parameters to estimate is equal to:
Total Number of Levels - Total Number of Attributes + 1
For this list (9 total levels and 3 attributes), the number of parameters to estimate is 9-3+1=7. CVA
software suggests asking three times as many questions as parameters to estimate. This would imply
21 questions. You both decide this is a reasonable task.
But then Dr. Coldridge gets a concerned look on her face. She suggests that some of the combinations
don't seem to make much sense. It seems hard to believe that a skybox on the 50-yard line would be
available for just $15,000 per year, or that skyboxes on the 20 yard line would go for $32,500.
You both agree that ideally you'd like to avoid all prohibitions in this design. Given the small sample
size, you need to have as efficient plan as possible for estimating stable part-worths. You decide to
investigate the damage these prohibitions might do to the efficiency of the design using the CVA
experimental design program. If the reduction in design efficiency is only slight, then a modest
prohibition or two may be well worth achieving more realistic product profiles and some added utility
balance in the design (by avoiding dominated alternatives).
CVA Help 763
A Sample Scripted Questionnaire
A few days later, you receive an E-mail from Dr. Coldridge She has scripted some expanded attribute
definitions and a few additional questions to ask study participants. You put that together with some
text you usually use for CVA studies, creating a unified document:
Start:
Thank you for agreeing to participate in this study! We are asking prominent supporters of the Wendell
State football program to give us their opinions regarding a new expansion project to the existing football
stadium.
The expansion will involve additional seating, including about 25 to 30 luxury skybox suites. These
luxury suites offer premier views of the playing field, catering, and air-conditioned comfort. Skyboxes
range in size from 12-seat to 24-seat suites and cost from $15,000 to $32,500 per year to lease (usually
6 or 7 home football games).
You should use your mouse to answer all questions. Click the Next button below to continue...
Attendance:
Thinking about last year's season, how often did you attend home football games at Wendell State
stadium?
Every home game

Most every home game
Some home games
No home games
IntroCVA1:
During this survey, we're going to ask you to consider features of luxury skyboxes.
These luxury boxes offer plenty of space for you and your guests to relax and enjoy the game. These
are fully enclosed suites, with floor-to-ceiling glass fronts for complete visibility of the playing field.
Features include:
Air conditioning/heating
Fully carpeted, padded swivel chairs
Video monitors
Buffet table in the back
Valet parking and catering are also available.
IntroCVA2:
The skyboxes might differ in terms of:
Seating Capacity: 12, 18, or 24-person seating capacity
Location: On about the 20-yard line, on about the 30-yard line, or between the 40- and 50-yard lines
Cost per Year: between $15,000 to $32,500 per year. Minimum lease 2 years.
764 SSI Web v8
IntroCVA3:
Based on these features, we'd like you to compare different options for skyboxes. In each comparison,
one skybox will be on the left, and another on the right.
Each skybox option will have slightly different features. In each case, we'd like to know which one you
prefer. We'll ask you a number of these tradeoff questions so that we can learn what features of
skyboxes affect your opinions and preferences.
CVA Conjoint Pairs Task (21 of these questions)

Which of these two luxury skyboxes would you prefer?
<Show skybox on left> or <Show skybox on right>
Respondents are ask ed to choose between different sk yboxes, using a 9-point sliding scale. By varying
the sk ybox profiles and observing the respondent's preferences, we are able to determine the
preferences respondents have for the different features of the sk ybox. Respondents will answer 21 of
these questions.
Interest:
During this survey, you have had a chance to trade off different options for luxury skyboxes. We now
would like you to consider just this one luxury skybox:

$22,500 per year
How likely would you actually be to purchase this luxury skybox (2-year lease) if it was available for the
next season?
Extremely likely
Very likely
Somewhat likely
Not very likely
Not at all likely
Holdout:
If these were the only three skybox alternatives offered, which would you prefer?
Option A: Option B: Option C:
12-person seating capacity, 18-person seating capacity, 24-person seating capacity,

on about the 20-yard line, on about the 30-yard line, between the 40- and 50-yard
$17,500 per year $22,500 per year lines,
$30,000 per year
For the sak e of brevity, we have only included one holdout choice task . In an actual CVA study, you
would probably want to include at least three or four different holdout choice task s.
Finish:
CVA Help 765
interviewing module.) CVA is one component within that software program. The next step is to open a
new study, and define a study name.
for your study also carries the same twenty-four character name as the study. For this tutorial, you
might create a name such as cvademo1. From the New SSI Web Study dialog, specify cvademo1 as
the studyname.
Now that we have created an SSI Web study called cvademo1, we can add our CVA exercise to that
study. Click Compose | Write Questionnaire... to access the Write Questionnaire dialog. Place your
cursor at the place in the questionnaire when you'd like to add the CVA exercise. Right now, there are
no questions in your survey, so you can just click the last page break in the questionnaire list, and click
the Add... button. Specify to add a new CVA exercise named CVAskybox to the study (you can have
766 SSI Web v8
multiple CVA exercises within the same study).
Click OK, and the CVA Exercise Settings dialog is shown.
To enter the list of attributes and levels you developed, go to the Attributes tab on the CVA Exercise
Settings dialog.
CVA Help 767
Type in the first attribute name: Seating. The attribute name is a label that the respondent may see in
the interview (if you select the option to show the attribute labels at the left of each choice task). If you
want to use a shorter label to display to you as the questionnaire author for program settings and data
export, specify that label in the Internal Label field. If you do not specify an internal label, the label in
the Attribute Display Text is used. Click OK to accept this information and close the dialog. Now that at
768 SSI Web v8
least one attribute name is in the list, the Add button under the Levels panel becomes active. Also note
that the Seating attribute is highlighted in the Attributes panel. With the Seating attribute highlighted in
the Attributes panel, click the Add button under the Levels panel to add levels within the Seating
attribute. The Level Text dialog is displayed. Type 12-person seating capacity in the Level Display
Text field. To add the next level of Seating, press the ENTER key twice. Type in the next level: 18-
person seating capacity. Repeat the process for the last level of seating (24-person seating capacity).
When you are ready to add the next attribute (Location), click the Add button under the left Attributes
panel, type the attribute label, and click OK to place that new attribute on the attribute list. With that
new attribute highlighted on the attributes list, click the Add button under the Levels panel to add the
three levels of that attribute.
Follow the same pattern for the last attribute, Price. For your convenience, we repeat the full list of
mouse, use the shortcuts Ctrl-C to copy, and Ctrl-V to paste into the desired field. (Hint: you can select
clipboard icon . This can save a great deal of time.)
Seating
Location
Between the 40- and 50-yard lines
Price
$15,000 per year
$22,500 per year
$32,500 per year
Make sure to specify a Worst to Best ranking for Location, and a Best to Worst ranking for Price. On
the Attribute Specification dialog, there is a drop-down box at the bottom of the dialog for specifying
the Level Rank ing. You should pre-specify the ranking for any attributes with rational order of preference.
Generating the CVA Questions
After you have specified the list of attributes and levels, you are ready to generate the CVA questions.
You use CVA's designer to create the combinations of products to show in the questionnaire. The
combination of features that make up the conjoint questions across all conjoint questions in your study
is collectively called the experimental design.
To generate the CVA questions, go to the Design tab of the CVA Exercise Settings dialog:
CVA Help 769
If you are uncertain regarding the different settings available and decisions to make about your CVA
study, if in doubt, the default settings are ones that seem to work well in practice. We recommend you
press F1 (help) if you have questions about any of the controls on any of these tabs. Context-sensitive
help will appear when you press F1 to help you understand what each setting does. In many cases,
advice regarding different settings is given.
Let's first turn to the Design tab. For the CVAskybox study, we have decided to use the default number
of suggested questions (21) and Pairwise Comparison Tasks. We'll use the default settings to generate
an experimental design. We'll assume that we will be interviewing online so that the software can easily
manage multiple versions of the questionnaire. By default, CVA will create 10 different versions of the
questionnaire (if you were interviewing via paper-and-pencil, you probably wouldn't want to manage so
many different versions of the questionnaire). Each questionnaire version will have strong enough
characteristics to stand on its own to provide enough information to lead to precise utility estimates for
each respondent. But, having more than one version of the questionnaire will additionally control
psychological order and context effects.
The process of creating the conjoint questions for each version of the questionnaire is as follows:
1. CVA generates a candidate pool of questions much larger than the number of questions you
intend to use.
2. CVA searches within the candidate set for the most nearly optimal subset to use (according to
the number of questions you've requested) to estimate part-worths with a high degree of precision.
The search uses the criterion of D-optimality.
3. The search process is repeated multiple times, and the best result retained.
4. Steps 1 through 3 are repeated for each additional questionnaire version.
By default, CVA uses a random seed to begin the search process. By default, we will use a "1" as the
Random seed (you can review this by clicking the Show Advanced Settings button to view the
advanced settings).
We are now ready to generate the conjoint questions for each of 10 versions of the questionnaire (the
experimental design). From the CVA Design tab, click Generate Design.
770 SSI Web v8
After a short period of computation, a report is shown. Your cursor is automatically positioned at the top
of the report, which displays the summary of the results for the different questionnaire versions:
The top 10 version D-Efficiencies:

Pass D-Efficiency
9 0.99740
40 0.99740
74 0.99740
98 0.99740
67 0.99740
29 0.99739
30 0.99707
68 0.99674
7 0.99674
88 0.99674
To generate ten questionnaire versions, CVA repeated the search process 100 separate times (passes).
The D-Efficiencies for the best 10 passes are displayed (and these best 10 passes are used as the 10
versions of the questionnaire). The 9th, 40th, 74th, 98th and 67th passes all produced D-Efficiencies of
0.99740. At first glance, it might appear to the casual observer that these five questionnaires are
identical. However, CVA's design algorithm ensures that each of these questionnaires, despite identical
D-Efficiencies, involves a unique series of questions. (To illustrate how this can occur, consider
replicating a conjoint design to create a second version, and for that second version interchanging the
labels for levels 1 and 2 of the first attribute. The design efficiency is unaffected by interchanging the
labels (it doesn't change the coding of the X matrix), but the questions are now different across the two
versions.)
D-efficiency ranges from a low of 0 to a high of 1.0. Design efficiencies near 1.0 suggest that this design
is about as good as can be achieved to estimate main effects with a high degree of precision, given our
particular settings and the number of questions we requested (21). With most design specifications, it is
impossible to achieve D-efficiency of exactly 100%.
Should we be concerned that designs 6 through 10 have slightly lower D-Efficiency than the best 5
designs? For all practical purposes, the difference in a D-Efficiency of 0.99740 versus 0.99674 is
negligible. However, you can always repeat the design procedure and ask CVA to work harder (by
altering the Advanced Settings) so that the top 10 designs all have a better chance of being the optimally
efficient design.
Close the CVA Design Report window by clicking Close.
Investigating the Effect of Prohibitions
(Note: the example below shows how to use prohibitions to achieve more realistic pricing for different
sk yboxes. Please note that one could use the advanced "conditional pricing" approach as described
elsewhere in this documentation. To k eep things simpler, we demonstrate the "prohibitions" approach
within this introductory tutorial.)
You may recall that Dr. Coldridge had suggested that some combinations of Location and Price might be
prohibited. A 50-yard line skybox at the lowest price may be entirely unreasonable.
Prohibitions will always lower the D-efficiency of the design as reported by CVA. However, there are
other considerations when designing good conjoint studies. Conjoint tradeoffs that result in an obvious
CVA Help 771
answer (e.g. an alternative on the right clearly superior on all aspects over the one on the left) are less
efficient than those questions implying more difficult tradeoffs. However, too much balance in terms of
expected utility among competing alternatives can make for very challenging questionnaires and
increased response error. The researcher should strive for a good balance between efficiency and utility
balance. Sacrificing a small degree of D-efficiency for the sake of better utility balance and avoiding
nonsensical combinations is probably a wise compromise.
The two prohibitions suggested by Dr. Coldridge might be represented using the following grid of joint
occurrences:
The "Xs" represent joint occurrences of Location and Price that should be prohibited. We refer to these
as extreme "corner" prohibitions, meaning that the expected utility of these cells is expected to be both
highest and lowest in the table. Prohibiting these corner conditions will lead to pairwise conjoint
questions that imply more difficult tradeoffs, and the unlikely combinations that were concerning Dr.
Coldridge will be eliminated.
Extreme corner prohibitions are less damaging to design efficiency than prohibitions on the same row or
column of the joint frequency table. Let's specify the prohibitions represented in the grid above. To
specify prohibitions, go to the Attributes tab of the CVA Exercise Settings dialog. Click the
Prohibitions... button.
The Prohibitions dialog is displayed. Click Location in the upper panel, and then click Price in the
second lower panel that appears. Then, check the two cells in the grid representing the prohibitions
between Attribute 2 Level 3 and Attribute 3 Level 1; and Attribute 2 Level 1 and Attribute 3 Level 3. As
you hover your mouse over each cell, tool tips appear to remind you what level text is associated with
each prohibition.
772 SSI Web v8
Click OK, and a message appears warning you that the design you previously created is now invalid.
Click Yes to delete the previous design file.
Return to the CVA Exercise Settings on the Design tab, and click Generate Design.
When you recompute the design, you will discover that the design efficiencies for the top 10 versions of
the questionnaire range from 0.95451 to 0.95630. This represents about a 4% decline in efficiency
relative to the design with no prohibitions (recall that the D-efficiencies for the top 10 versions previous
ranged from 0.99674 to 0.99740). Together with Dr. Coldridge, you decide that the small loss in overall
D-Efficiency is worth the added realism that the prohibitions bring to the conjoint tradeoffs.
To see the effect of the prohibitions on each attribute in the design, you should run a more in-depth
diagnosis of the design as provided by CVA's Test Design facility. Close the CVA Design Report dialog
by clicking Close. Then, click the Test Design button from the Design tab.
CVA Design Efficiency Test

Tasks are 'Pairwise' using a seed of 1.

Includes 210 total tasks (21 per version).
Each task includes 3 attributes.
Design Version Efficiencies

-------------------------------------------------------------
Version Efficiency
1 0.95630
2 0.95601
3 0.95546
4 0.95516
5 0.95516
CVA Help 773
6 0.95511
7 0.95508
8 0.95477
9 0.95477
10 0.95451
Attribute Efficiency Coefficients (including all versions)

-------------------------------------------------------------
Attribute Efficiency
Seating 0.996
Location 0.839
Price 0.849
The D-Efficiencies for the top 10 versions are repeated. Then, efficiencies for each attribute are reported
(considering all 10 versions have been appended). The prohibitions we specified were between attributes
2 and 3. We see evidence in a loss in efficiency particularly concentrated on these two attributes
(relative efficiencies of 0.839 and 0.849). Note that the efficiency estimation for each attribute is based
on a different method than D-Efficiency, the details of which are provided in the CVA Technical Paper.
We suggest you pay most attention to D-Efficiency, as it is a stronger measure of design efficiency and
is more accepted in the industry. The relative efficiency coefficients by attribute are useful for pointing
out problem areas in a design.
The next section of the Test Design report shows the one-way frequency of levels across all versions of
the questionnaire (the Total column), and for each of the 10 questionnaire versions:
One-Way Frequencies (by version)

-------------------------------------------------------------
Level Total Ver1 Ver2 Ver3 Ver4 Ver5 Ver6 Ver7 Ver8 Ver9 Ver10
12-person seating ca 139 14 14 14 14 14 14 14 13 14 14
On about the 20-yard 122 12 12 12 12 12 13 12 12 12 13

On about the 30-yard 176 18 17 17 18 18 17 18 18 18 17
Between the 40- and 122 12 13 13 12 12 12 12 12 12 12
$15,000 per year 124 12 12 12 13 13 13 13 12 12 12

$22,500 per year 173 18 17 17 17 17 17 17 18 17 18
$32,500 per year 123 12 13 13 12 12 12 12 12 13 12
Note that the ten versions of the design are nearly perfectly balanced with respect to the first attribute
(only version 8 shows a slight imbalance). This could be remedied by repeating the design procedure
and using the Advanced Settings to ask CVA to work longer on the problem. Attributes 2 and 3 are not
nearly as balanced, due to the prohibitions we imposed.
Finally, the two-way frequencies for the aggregate design (across all ten versions) are displayed:
Two-Way Frequencies (all versions)

-------------------------------------------------------------
Att/Lev 1/1 1/2 1/3 2/1 2/2 2/3 3/1 3/2 3/3
1/1 139 0 0 42 58 39 43 56 40
1/2 0 141 0 40 60 41 41 60 40
1/3 0 0 140 40 58 42 40 57 43
2/1 42 40 40 122 0 0 61 61 0
2/2 58 60 58 0 176 0 63 52 61
2/3 39 41 42 0 0 122 0 60 62
774 SSI Web v8
3/1 43 41 40 61 63 0 124 0 0
3/2 56 60 57 61 52 60 0 173 0
3/3 40 40 43 0 61 62 0 0 123
Note that Attribute 2 level 1 was prohibited from ever showing with Attribute 3 Level 3; and Attribute 2
Level 3 was prohibited from being shown with Attribute 3 Level 1. We haven't taken the space to show
the two-way frequency tables for each version of the questionnaire, but they are also available for your
reference.
Click Close to exit the CVA Test Design Report, then click OK to exit the CVA Exercise Settings
dialog.
Questionnaire Layout
Let's now examine the questionnaire we've built to this point. Click Compose | Write Questionnaire or
click the "pencil" icon on the toolbar, and the following dialog is displayed:
CVA Help 775
You'll note that SSI Web has added 21 conjoint tasks (CVAskybox_1 through CVAskybox_21) to the
survey. These questions are based on your settings and the default template within SSI Web.
We can edit any CVA task to preview how SSI Web displays the conjoint questions to respondents.
Highlight the CVAskybox_1 question on the list, and click Edit... (or, you can double-click the
CVAskybox_1 question to edit it).
776 SSI Web v8
The CVA question type has main four areas where you control supporting text: Header 1, Header 2, Or
Text, and the Footer. To see what your question looks like (using the default layout options), click
Preview.
Note that by default, we use "alternating colors" for the background color of the product concept areas.
The colors alternate between left and right in subsequent questions to give respondents a visual clue that
the aspects of the questions (the product characteristics) are changing. The colors that are currently
being used are those supplied automatically by the Style that has been selected by the survey. Later,
we'll show you how to select a different style.
CVA Help 777
The questionnaire we developed with Dr. Coldridge included the following text in the header: "Which of
these two luxury skyboxes would you prefer?". We need to change the default text provided by CVA to
include the text we intend to use.
Go ahead and close the browser window and return to edit the CVA question. Note that the default text
to change also includes some "commented-out" text intended for use with the single-concept at a time
format. (Note: the  tag in HTML tells the browser to ignore any text found within the tag.)
Change the Header 1 text to say: Which of these two luxury skyboxes would you prefer?
Click Preview, and the following is displayed:
You only need to edit one of the CVA questions to create the formatting as you desire. The changes
you make in one CVA question apply to all CVA questions in your survey.
used.
processing document. Assuming you really had such a document, you might use the Ctrl-C to copy,
778 SSI Web v8
Start:
Thank you for agreeing to participate in this study! We are asking prominent supporters of the
Wendell State football program to give us their opinions regarding a new expansion project to the
existing football stadium.
The expansion will involve additional seating, including about 25 to 30 luxury skybox suites.
These luxury suites offer premier views of the playing field, catering, and air-conditioned comfort.
Skyboxes range in size from 12-seat to 24-seat suites and cost from $15,000 to $32,500 per year
to lease (usually 6 or 7 home football games).
You should use your mouse to answer all questions. Click the Next button below to continue...
automatically opens. The Questionnaire Access and Passwords dialog is displayed::
CVA Help 779
as follows:
Thank you for agreeing to participate in this study! We are asking prominent
supporters of the Wendell State football program to give us their opinions regarding
a new expansion project to the existing football stadium.
The expansion will involve additional seating, including about 25 to 30 luxury

skybox suites. These luxury suites offer premier views of the playing field, catering,
and air-conditioned comfort. Skyboxes range in size from 12-seat to 24-seat suites
and cost from $15,000 to $32,500 per year to lease (usually 6 or 7 home football
games).
You should use your mouse to answer all questions. Click the Next button below to
continue...
780 SSI Web v8
your text to take greater control over the layout, font, and style later in this unit.
Click the OK button of the Preview window to close that window and return to the previous SSI Web
dialog.
After you are happy with the layout of the Start screen, click OK to return to the Write Questionnaire
dialog. If you need to move a question once it has been added to the List of Questions, simply highlight
the question to be moved and click the or buttons to move the questions within the list (you
can also highlight a question and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within
the current questionnaire, or even to another SSI Web study.)
Select-Type Questions
Attendance:
Thinking about last year's season, how often did you attend home football games at Wendell
State stadium?
Every home game

Most every home game
Some home games
No home games
on the list). At the Write Questionnaire dialog, click Add..., choose Select as the question type, and
type Attendance for the question name. Click OK and the Select Question dialog is shown.
CVA Help 781
Response Options tab. From the Response Options tab, use the Add button to add the four response
Now that you have seen how to specify Single Select question types in SSI Web, you have the tools you
need to specify the other two select-type questions for this questionnaire (Interest and Holdout--Please
refer to the questionnaire text for these questions as presented near the front of this unit).
The Holdout question layout poses some interesting challenges. The simplest approach is to format it
as a Select-type question with three columns (specify #columns on the Settings tab of the Select
Question dialog). The Holdout question formatted as a Select-type question looks like:
To do this, we formatted each option as a separate Response Option, with additional HTML text to force
hard returns and a blank line. For example, the first response option is formatted as:
782 SSI Web v8
However, you can make this holdout choice task look even nicer if you format it as a Grid-type question
(one row, three columns, radio buttons). On the Rows tab, make sure to un-check the Show Row Labels
box. From the Columns tab, add three Predefined List Members (List Member Text just as shown
directly above) to show each of the three product concepts. When you click Preview, the grid question
should look something like:
In addition to the Select questions we added to the questionnaire previously, there are four text-only
named IntroCVA1, IntroCVA2, IntroCVA3, and Finish. Even though the respondents aren't asked to
provide specific inputs to these, we refer to these as "questions" in the sense that we add them to the
questionnaire as if they were standard questions, and they are listed separately by their "question" name
in the List of Questions.
After the Attendance question follows the text-only instructions:

CVA Help 783
IntroCVA1:
These luxury boxes offer plenty of space for you and your guests to relax and enjoy the game.
These are fully enclosed suites, with floor-to-ceiling glass fronts for complete visibility of the
playing field.
Features include:
Air conditioning/heating
Fully carpeted, padded swivel chairs
Video monitors
Buffet table in the back
Valet parking and catering are also available.
introductory text into the Start question). There are more than three separate paragraphs here, there is
a bullet point list, and we'll need to deal with the additional element of bolding selected text.
Add a Text/HTML Filler question directly following the Attendance question in the questionnaire. To do
so, highlight the Attendance question, click Add…, and specify the Question Name as IntroCVA1 and
the question type as Text/HTML Filler.
The Text/HTML Filler dialog is displayed.
Earlier in this tutorial, when we used the Start question to format the opening page of the survey, we
placed each paragraph of text in a separate Header 1, Header 2, or Footer section. We saw that SSI
Web automatically places blank lines between text in these sections. However, with the text in the
IntroCVA1 question, there are more paragraph breaks to insert. We'll take the opportunity here to
introduce the concept of using a few simple HTML instructions within our survey text.
Browsers know how to interpret HTML instructions when displaying the page on the screen. If you know
HTML, you can use it whenever you wish within SSI Web surveys to accomplish your aims. If you don't
know HTML, it isn't very difficult to learn a few HTML tricks, or you can use the toolbar available within
the editor that appears when you click "Pencil" icons in SSI Web:
Either type or cut-and-paste the text for the IntroCVA1 question into the Header 1 text field (Any of the
784 SSI Web v8
and then (with the text in the first paragraph highlighted) click the "Paragraph" icon on the toolbar,
this inserts a <p> prior to the text and a </p> after the text:
<p> During this survey, we're going to ask you to consider features of luxury skyboxes. </p>
IntroCVA1 question.
Next, we need to add a bullet point list. The following HTML tags are used to create bullet point lists:
<ul> </ul> (open and close tags for an unordered bullet list)
<li> </li> (open and close tags for list elements within an unordered bullet list)
CVA Help 785
When you apply these HTML elements within the text for this question, you'll specify:
<p>Features include:
<ul>
<li>Air conditioning/heating</li>
<li>Fully carpeted, padded swivel chairs</li>
<li>Video monitors</li>
<li>Buffet table in the back</li>
</ul>
</p>
the toolbar. After you finish separating the paragraphs with <p></p> tags, applying the bullet list
instructions, and bolding the appropriate text with <b></b> tags, it should look something like:
786 SSI Web v8
When you add the Finish question at the end of the survey, make sure to add it at the end of the survey
as a Terminate/Link question type. On the Settings tab of the Terminate/Link question, click Terminate
Respondent to indicate that respondents that reach this question are finished. SSI Web may warn you
at this point that a terminating question must stand alone on its own "page" in the survey. This leads us
into our next discussion on page layout.
CVA Help 787
Notice that the preliminary questions (Attendance, IntroCVA1, IntroCVA2, and IntroCVA3) we've
added to the questionnaire are all currently arranged on the same page. Let's assume we wanted to
break these up, one question per page. You add a page break after the Attendance question by
highlighting Attendance on the list and clicking the Add Page Break button. Attendance is now
placed on its own page. Repeat this operation to place IntroCVA1, IntroCVA2, and IntroCVA3 on
separate pages. You may also choose to place Interest and Holdout questions on their own pages.
Make sure to place the final Terminate/Link screen Finish on its own (last) page.
788 SSI Web v8
On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Skybox
Local Test Server

CVA Help 789
Locally.
After you have edited the survey to the point that you are pleased with the look, content, and functionality
of your survey, we suggest you take the survey yourself under Local Test Server. You should answer
each question legitimately and carefully.
After you have taken the questionnaire carefully as suggested above, you should download the data
(downloading data is described in a separate tutorial called "Getting Started with SSI Web: A 45-minute
Hands-On Tour"), then compute and examine your part-worth utilities. The part-worths should very
closely reflect your preferences. If they don't, this suggests something may be wrong with your study
setup.
examine the resulting part-worths to make sure the data at least have face validity.
After you have tested the survey using the Local Test Server, you post the survey to the Web using an
account set up for you by an ISP (Internet Service Provider). (Setting up your survey on the web is
beyond the scope of this unit, but is described in detail elsewhere. Fielding options also include hosting
on your own company's server, or using Sawtooth Software's hosting service). You send an email to Dr.
Coldridge, with a link to your survey, and she browses to and takes the survey. After she give you some
feedback, you invite six other individuals in administration and planning take the survey. You download
the data and compute part-worth utilities. Due to the small sample size, the results are a bit noisy, but
the data seem to feel right.
Next, you recruit six respondents from the actual sampling frame to come to a central site to take the
survey. You watch silently as they take the survey. At the end of each session, you ask each
respondent follow-up questions to ensure that there weren't any parts that were difficult to understand or
just didn't make sense. You ask them specifically about the conjoint questions, making sure that they
understood what to do, and that they didn't feel overwhelmed by the task. After debriefing the test
respondents and analyzing their data to ensure that the results look reasonable, you make a few small
adjustments and proceed to field.
Data Management
790 SSI Web v8
To download the data from the respondents to the Skybox study, you can browse to the administrative
module for your study on the web site (again, we are speaking hypothetically, as for this tutorial study
there is no such site set up). Once at the administrative module, you would download your data (by
clicking Download from the main menu), making sure to save your data (named studyname_data.sqlite,
where STUDYNAME is your study name) to the same folder on your hard drive in which you developed
the SSI Web project. Alternatively, from the SSI Web interface, you can simply click Field | Download
Data... to download your data without logging into the Admin Module.
You can also use the Admin Module to tabulate results from your Select-type questions. For example,
you might be interested in the responses to the Holdout question. Once within the Admin Module, you
can request a tabulation of each question in the survey (the Marginals function). The tabulation for the
Holdout question for 50 hypothetical respondent records for this tutorial study is as follows:
Holdout
Option A: 12-person seating capacity... 9
18.0%
Option B: 18-person seating capacity... 20

40.0%
Option C: 24-person seating capacity... 21

42.0%
Total 50
100.0%
The report shows that 18% of respondents chose Option A, 40% chose Option B, and 42% chose
Option C.
To download the data from the 50 respondents to the Skybox study, you would browse to the
tutorial study there is no such site set up). Once at the administrative module, you would accumulate
and download your data (by clicking Download from the main menu), making sure to save your data
(named studyname_data.sqlite, where STUDYNAME is your four-character study name) to the same
folder on your hard drive in which you developed the SSI Web project.
Computing/Examining Utilities
Once you have downloaded the data, you are ready to estimate part-worths. If you try this with this
tutorial study you created, you will receive an error stating that you have no data within your project folder
to analyze. However, hypothetical data for this project are available as a sample CVA study in your SSI
Web installation. If you wish to follow along with the analysis steps below, open the CVA Tutorial project
we have provided by clicking Help | Sample Studies | Sample CVA.
To compute utilities, click Analysis | Calculate CVA Utilities Using Ordinary Least Squares…. Select
Settings… and the CVA OLS Settings dialog is displayed. These controls govern the OLS estimation.
The range of respondent answers is shown (as specified in your questionnaire). The recode method is
the dependent variable transformation. For most applications, we suggest a logit rescaling. It is beyond
CVA Help 791
the scope of this tutorial to discuss dependent variable transformations, but more information on that
subject is available in a section entitled: "CVA Ordinary Least Squares Settings". The range of the
respondent answers (1 to 9) and the scale direction are indicated. This pairwise questionnaire uses a
sliding scale with the lowest number on the left and the highest number on the right, so the correct scale
direction to choose is Highest Number on Right.
While still at the CVA OLS Settings dialog, click Show Advanced Settings, and then Edit... to edit the
utility constraints. Utility constraints impose any rank order of preference assumptions you have made
for the levels within attributes. By default, any a priori order settings you used when specifying the
attributes are also used as utility constraints. Currently, you have set the Location attribute as Worst to
best ranking, meaning the first level ("On about the 20-yard line") is worst and the last level ("Between
the 40- and 50-yard lines") is best. The Price attribute is set as Best to worst. These constraints will be
enforced during utility estimation. If you wish to modify these selections prior to calculating utilities, you
can do so from this screen. There is an additional area for specifying constraints that involves just pairs
of attribute levels. If some attributes are partially ordered, you may specify those constraints using the
Custom Constraints button. Click the Cancel button to return to the previous dialog, and then click the
Cancel button again to return to the main Compute Utilities dialog.
Once you are satisfied with your utility estimation settings, click Prepare for CVA OLS Computation.
SSI Web first scans your data file and gives a summary report of the conjoint information:
** Data Summary **
Number of respondents included = 50
Total tasks included across all respondents = 1050
The data will be written to the following file: c:\temp\cva2\cva2 OLS.utl
When you click Compute Utilities, SSI Web examines each respondent's data and computes a set of
utilities for each level in your study for each respondent. For more information about interpreting utility
values in conjoint analysis, please see Appendix B in this documentation, entitled: "Interpreting Conjoint
Analysis Data".
SSI Web then reports that the utilities have been saved to a file named STUDYNAME OLS.utl, for use in
market simulations using the separate SMRT software (Sawtooth Software's Market Simulator). But, the
information is also made available to you within a report. A report window is displayed in a spreadsheet
format with two sheets (tabs located at the bottom of the display): Summary and Individual Utilities. On
the Summary sheet, the average part-worth utilities (scaled according to the normalization procedure
called Zero-Centered Diffs) and attribute importances for the sample are displayed, along with standard
deviations.
792 SSI Web v8
The Summary Results report also contains diagnostic information about the utility run:
Tasks Included: All Pairwise

Recode Method: Logit
Additional Utility Constraints:

All Attribute Levels:
Attribute 2, Attribute 3
Successfully computed utilities for 50 respondents.

Average R-squared = 0.78849
Median R-squared = 0.81081
Number of constraint violations resolved:

Attribute 2 Level 3 preferred to Level 2: 1 Respondents
You are shown the average and median R-squared (fit) values for the data set. R-squared is bounded by
0 and 1.0, with 1.0 indicating perfect fit (note that the reported R-squared is the fit after imposing any
constraints on the data). Below that, SSI Web reports the reversed utility relationships that had to be
resolved. Respondents answer with some degree of error, and most data sets in practice will have many
reversals at the individual level, especially for attributes with relatively low importance. This tutorial data
set was artificially developed and has very few out-of-order relationships.
If you discover that a large proportion of respondents in your data sets have certain out-of-order
relationships, this suggests one or more of the following explanations:
1. The violated preference relationships are of low importance to respondents (minimal actual
CVA Help 793
difference in "true" part-worths),

2. Your assumptions regarding the utility relationships are faulty (you may need to change
your assumptions under the Constraints dialog, accessed by clicking Analysis | Calculate
CVA Utilities Using Ordinary Least Squares... | Settings... | Show Advanced Settings >> |
Edit...),
3. The respondents answered with a high degree of error,
4. Your design was inefficient or did not include enough questions to stabilize the utility
estimates,
5. You have made a data processing error.
If more than half of the respondents violate a particular rational preference relationship, it is definite cause
for concern.
If you open the Individual Utilities sheet (by clicking the Individual Utilities tab near the bottom of the
report), you can view the raw individual-level utilities for each respondent:
This report displays similar results as are saved to the STUDYNAME OLS.utl file (which is used for
simulations in the SMRT market simulation software). The Respondent number and R-Squared (actually,
R-Squared x 1000) are displayed in the first two columns. The part-worth utilities for each level within the
study start in column 4.
You can save results from this report to an Excel spreadsheet (containing both the Summary and
individual Utilities sheets) by clicking the Save As... button.
Moving the Data into SMRT
used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. We'll now
describe how you move data from your CVA project within the SSI Web Platform into SMRT.
When you calculated part-worth utilities using Analysis | Calculate CVA Utilities Using Ordinary Least
Squares... this process creates a text-only format file containing case IDs and part-worth estimates
called STUDYNAME OLS.utl.
To move the CVA part-worth utility data into SMRT for analysis:
Software SMRT.
794 SSI Web v8
folder and a studyname, making sure to specify that this is a CVA study type). You can
choose any folder or studyname that you want, as this new study functions independently of
your original SSI Web study.
3. Import the STUDYNAME OLS.utl file (file of respondent part-worth utilities) into your SMRT
following the prompts for importing CVA utilities (selecting CVA (Traditional Conjoint) Utilities
(*.utl) as the file type, and browsing to your .utl file). Once you import your utilities, you can
Click close to return to the main menu. Now that you have imported the utilities, the Mark et Simulator
menu item becomes active. From the main menu, click Analysis | Market Simulator.
You will note that there is a list of available Utility Runs (your recently computed run is available), and a
list of Simulation Scenarios (currently empty). Click Compute!, and a report of average utilities and
importances for your data set is displayed in the report window (these data should match the similar
report that was earlier displayed within SSI Web, directly following utility estimation):
Scenario:
Utility Run: CVA OLS Run
Average Utility Values

Rescaling Method: Zero-Centered Diffs
12-person seating capacity -26.57
18-person seating capacity 4.25
24-person seating capacity 22.32
On about the 20-yard line -52.30

On about the 30-yard line -0.13
Between the 40- and 50-yard lines 52.42
$15,000 per year 58.32

$22,500 per year 9.57
$32,500 per year -67.88
Average Importances
Seating 23.03
Location 34.91
Price 42.07
<<The above average utilities are rescaled utilities using the zero-centered "diffs" method. The diffs
method rescales utilities so that for each individual the total sum of the utility differences between the
worst and best levels of each attribute across attributes is equal to the number of attributes times 100.
CVA Help 795
For more about interpreting conjoint utilities, please see Interpreting Conjoint Analysis Data.
It is difficult to make firm conclusions from just these fifty respondents, but we might try to form some
tentative conclusions:
First, looking just at the report of average importances (the second section above), it appears that the
least important attribute (among the three attributes and the feature ranges we measured) is whether the
skybox has 12-person or 24-person seating capacity. Looking at average utility values, we see that if
everything else was held constant (price and location on the field), respondents on average would prefer
to have more seating capacity than less in the skybox.
Although you don't have much data to work with for segmentation purposes in the Skybox1 study (just
50 respondents), you should note that you can run market simulations by "banner" point. Banner points
are the segmentation variables that run across the columns of a cross-tabulation report. If you select a
Banner variable using the drop-down menu at the upper-left of the Market Simulator dialog, and then
click Compute!, you will note that the utility/simulation report is broken out by different groups of
respondents, depending on how they answered the Banner variable.
Many important effects due to the competitive context, respondent heterogeneity and correlations of
preferences among attribute levels can occur within a data set. These may be completely obscured by
viewing just average utilities and importances. Also, the concept of utility scores seems difficult to grasp
for many managers. For these reasons, running market simulations offers a more powerful and revealing
way to look at conjoint data.
Market Simulations
To open the Market Simulator, click Analysis | Market Simulator or double-click Mark et Simulator from
the Study Navigator window. The first time you open the Market Simulator for a study, the list of
Simulation Scenarios is empty. A Simulation Scenario is a particular "what-if" scenario, wherein certain
products are placed in competition with one another in a simulated marketplace.
Prior to fielding your study, Dr. Coldridge had conferred with the administration's planning group regarding
potential configurations for skyboxes. The result was three principal alternatives, which were used as a
holdout question in your CVA survey:
You may recall that the tabulation of the Holdout question showed that 18% of respondents preferred
the Bronze option, 40% preferred the Silver option, and 42% preferred the Gold option.
Notice that two of the levels of Price ($17,500 and $30,000) are in between the levels we measured in the
study. The market simulator lets us interpolate between levels, so this is not a problem. However, it will
be easier to specify these levels if we have assigned "Level Values" to each level.
To assign Level Values, from the main menu click Analysis | Market Simulator and then click the
Assign Level Values button at the very lower left-side of the dialog. The Assign Level Values dialog is
796 SSI Web v8
displayed. Using the drop-down box, select Price. Click the first row in the Value column
(corresponding to $15,000). Type "15000" Click the next two cells and enter "22500" and "32500."
Click OK to return to the Market Simulation menu.
Create a new simulation scenario by clicking Add....
When you click the Add... button, the Scenario Specification dialog is displayed. The first step is to
type a scenario name into the Name field, in the upper left-hand corner of the dialog. Type Base Case.
Next, you'll specify the three products for the first scenario. The area you type the products into looks
like the grid within a spreadsheet. The first column is where you type the product label. As you click
the various columns associated with the attribute levels, the level codes will appear in the window below
to remind you of the codes associated with each attribute's levels.
The attribute level codes for the three products for the planned skybox configurations were:
Base Case:
Specify these three products in the product entry grid. To add new rows to the grid (for additional
Hint: While working within the product specification grid, you can press the Enter key to advance to the
next cell. You can use the arrow keys to move the cursor within the grid. You can also cut-and-paste
information within the grid, or from a spreadsheet.
After you have specified these three products, you should choose a simulation method. The default
method is Randomized First Choice. We generally recommend this method for competitive market
scenarios like this one, so we'll use the default. Because we have already examined the utilities and
importances, un-check the Display Utilities box.
dialog.
Choose a utility run to use for your simulation. At this point, you should only have one run in the list, so
this is your only choice. If there were multiple utility runs (for example, another run from the CVA/HB
system), you could select that run by highlighting it in the list of Utility Runs.
name in the Simulation Scenarios list, and then click Compute!. SMRT examines each respondent's
record and predicts which product(s) each respondent would likely choose based on the product
specifications and each respondent's utilities.
Scenario: Base Case

Utility Run: CVA OLS Run

CVA Help 797

Model: Randomized First Choice
Total Sampling Iterations: 200000
Attribute Variability Multiplier: Autocalibrated Value (0.568351)
Product Variability Multiplier: 0
Exponent: 1
<<Above are the settings that govern this run. It is not necessary at this point for you to k now what
these settings mean (iterations, Multipliers, Exponent). These are explained in other areas of this on-line
help and also in the manual.>>
Seating Location Price

Bronze 1 1 17500
Silver 2 2 22500
Gold 3 3 30000
<<Above are the product level codes you specified for the three products in this simulation. Below are
the simulated shares of preference (choice) for these products, along with their standard errors.>>
Shares Std Err

Bronze 22.41 1.37
Silver 36.41 1.47
Gold 41.17 1.48
Though a total is not shown, the shares of preference sum to 100%. Standard errors are presented next
to each product's shares, which are useful for expressing confidence intervals around the share
estimates.
Tuning the Market Simulator to Predict Choice
Recall that when respondents were asked to choose among these three alternative skyboxes in the
Holdout question within the questionnaire, 18% preferred the Bronze option, 40% preferred the Silver
option, and 42% preferred the Gold option. The market simulator's predicted shares as displayed at the
end of the previous section show a very similar pattern of preference: the rank order is the same, but the
magnitude of the share differences is a bit compressed for the predictions relative to the actual choices.
When using a ratings scale with CVA questions (recall we used a 1 to 9 scale) and OLS estimation,
there is no guarantee that the resulting utilities will be appropriately scaled so that the market
simulations are closely linked to choice probabilities. However, the market simulator provides a way for
us to tune the scaling of the shares to better match our target holdout choice probabilities. We tune the
scaling of the shares using the "Exponent".
Tuning the market simulator to best fit holdouts requires some trial and error. The first step is to
compare the predicted results to the actual choice probabilities and determine a measure of fit. Let's
again compare our predicted shares of preference with actual choice probabilities as determined through
798 SSI Web v8
the Holdout question. The table below reflects the predicted shares when using the default settings for
Randomized First Choice (Exponent = 1):
(Exponent=1)
Predicted ActualAbsolute Difference
Shares Shares
(Predicted vs. Actual)
Bronze 22.41 18.00 4.41
Silver 36.41 40.00 3.59
Gold 41.17 42.00 0.83
------
Mean Absolute Error (MAE): 2.94
On average, our predictions are 2.94 share points different from the actual target holdout shares. There
are many ways to determine fit; the use of MAE (Mean Absolute Error) is one common measure.
Next, let's tune the exponent to achieve a better fit. From the Mark et Simulator dialog, highlight the
Base Case scenario on the list of Simulation Scenarios, and click Edit…. Then, click Advanced
Settings… and change the Exponent setting to 1.5. Choosing a higher exponent will accentuate the
differences among the choice probabilities, which will move us in the right direction to achieve closer fit to
the holdout probabilities. The new data are:
(Exponent=1.5)
Predicted Actual Absolute Difference
Shares Shares (Predicted vs. Actual)
Bronze 18.68 18.00 0.68
Silver 37.32 40.00 2.68
Gold 44.00 42.00 2.00
------
By adjusting the Exponent from 1.0 to 1.5, we have made the resulting share predictions "steeper" and
achieved a better overall fit to the holdouts. Our MAE has decreased from 2.94 to 1.79. By trial and
error, you will find that the MAE is minimized when the Exponent is about 1.6 (MAE = 1.71). The data
for this run are given below:
(Exponent=1.6)
Predicted Actual Absolute Difference
Shares Shares (Predicted vs. Actual)
Bronze 18.10 18.00 0.10
Silver 37.43 40.00 2.57
Gold 44.47 42.00 2.47
------
For the remainder of this tutorial, set the Exponent to 1.6 so that your results match ours. If you were
analyzing this data set in practice, you should use the Exponent of 1.6 for all subsequent market
simulations.
Note that in practice researchers usually include more than just one holdout choice task. The exponent
is adjusted to reduce the MAE across all holdout tasks. One does not choose a separate Exponent to
provide best fit to each separate holdout task.
CVA Help 799
scenarios. It is important to remember that the shares of preference resulting from conjoint predictions
are not equivalent to actual market shares, and often look quite different. Many other factors in the real
world influence market shares and cannot be measured and reflected solely by conjoint data. Conjoint
Sensitivity Analysis
Performing sensitivity analysis is one of the most common uses of the market simulator. Sensitivity
analysis involves first simulating a base case (as we previously did), representing an array of products in
competition with one another, then modifying one of the products and observing the change in share due
to the modification.
For example, the planning group at Wendell State may be interested to know how raising the price for
the most expensive option (Gold) would alter the relative preference among the three proposed
alternatives. You may recall from the previous sections in this tutorial that the base case scenario was
defined as:
The shares of preference for this base case configuration are:
Bronze 18.10%
Silver 37.43%
Gold 44.47%
The question is: how would the share of preference change if the Gold option were increased from
$30,000 per year to $32,500 per year? We can create a new simulation scenario to test this possibility.
You can save a lot of time in entering this scenario if you retain the settings for the first three products
when you create the new scenario. SMRT lets you do this. Click Add..., and from the Add Scenario
dialog use the drop-down control to choose Base Case as the starting template for the new scenario.
You can name this second simulation scenario Gold to $32,500. The level codes/values for the three
products for this simulation are:
Gold to $32,500:
After altering Gold's price, click OK to return to the Market Simulator dialog.
800 SSI Web v8
Before running the new Gold to $32,500 scenario, you may wish to clear the previous results from the
report window by clicking Clear and un-checking the box for the Base Case scenario. Check the Gold
to $32,500 scenario and click Compute!, and the new shares of preference are displayed. We've
summarized the previous shares of preference for the base case (when Gold was at $30,000) and the
new shares of preference (when Gold is increased to $32,500) in the table below:
Base Price
Case Increase
Shares Gold
Bronze 18.10 19.78
Silver 37.43 41.95
Gold 44.47 38.27
How might the relative shares of preference change if seating capacity or position relative to the field
were changed? You may wish to run additional scenarios to test those possibilities.
CVA Help 801
6.1.3 Is CVA the Appropriate Technique?

Many methods are available for collecting and analyzing conjoint data, and the researcher contemplating
a conjoint study must choose among them. We at Sawtooth Software have had many years of direct
experience with these methods, as well as the benefit of many conversations with users of our own and
other software. Based on that experience, we offer the following suggestions:
The Full Profile Method (such as used in CVA) was the first conjoint method introduced to the marketing
research community, and it is considered the original standard. Green and Srinivasan (1990)
recommended use of the full profile method when the number of attributes was six or fewer. Our opinion
is that this is a useful guideline, but there isn't a hard limit, and the number of attributes should depend
on the complexity of the attribute descriptions and respondent interest/knowledge of the product
category.
During the 1980s and early 1990s, we tended to favor ACA for projects involving about six or more
attributes. As choice-based approaches with HB estimation have captivated the industry and shown
their worth, we have shifted our attention toward CBC and Adaptive CBC. Adaptive CBC combines some
of the best aspects of ACA of CBC, and the early evidence suggests that it is a valuable new technique.
We have not conducted nor seen research that directly compares ACA to Adaptive CBC, but we expect
that ACBC should have important advantages when the number of attributes is between 5 and 12. When
the number of attributes exceeds about 15, and pricing research isn't a goal, we don't know which
method would prevail.
CVA is a "main effects only" model, and assumes there are no interactions among attributes. Many
conjoint practitioners agree that one must remain alert for the possibility of interactions, but that it is
usually possible to choose attributes so that interactions will not present severe problems. Like other
conjoint methods, CVA can deal with interactions in a limited way by collapsing two attributes into one.
For example, we could deal with an interaction between car color and body style by cross-classifying the
levels:
Red Convertible
Black Convertible
Red Sedan
Black Sedan
suspected of having interactions with many others, then collapsing attributes will not be enough. In that
case too many parameters must be estimated to permit analysis at the individual level, and the most
common solution is to evaluate interactions by pooling data from many respondents.
When CVA is executed well, it provides solid results for applications such as product design, pricing,
and segmentation work. It can work with the smallest of sample sizes, even in the case of revealing the
preferences for a few or even one individual. Even though the bulk of interest lately is in favor of CBC and
ACBC, those who have become expert in CVA methodology and CVA study execution will continue to
use the method confidently in appropriate contexts for many years to come.
For more information about selecting the appropriate conjoint method, see the article in the technical
papers library on our website entitled: "Which Conjoint Method Should I Use?" or use the Interactive
Advisor at http://www.sawtoothsoftware.com/products/advisor/.
802 SSI Web v8
6.2 Designing/Creating a CVA Study

6.2.1 Attributes and Levels
6.2.1.1 Defining Attributes and Levels
CVA projects allow up to 30 attributes, with up to 15 levels per attribute.
(Hint: you can select a list of attributes or levels from Word and paste into SSI Web using the Paste list
member(s) from the clipboard icon . This can save a great deal of time.)
In conjoint experiments, we show respondents product concepts that are described by different
combinations of attribute levels and ask them to somehow express their preferences for those concepts.
One way of thinking about conjoint analysis is that we are conducting a designed experiment for each
respondent to see how his/her preferences vary as we change attribute levels.
By observing how respondents evaluate products in response to changes in the underlying attribute
Ranking (Applies only to ACA or CVA-type conjoint studies)
For each attribute, you can specify whether it has known (a priori) order. If you know ahead of time that
respondents prefer low prices to high prices (all else equal) or fast speeds to slow speeds, you can avoid
having to ask what might seem to be an obvious question. For CVA studies, that means avoiding
showing products that are clearly superior on all aspects to another.
We strongly encourage you to specify an order for attributes that have known rational order, like price
and speed. When you specify a ranking order "worst to best" or "best to worst," you must be certain
that the rational respondent would agree with you, and that you have specified the correct order.
For example, the levels:
Level 1: $5
Level 2: $10
Level 3: $15
are arranged from "Best to Worst," meaning, the best level is first and the worst level is last.
Guidelines

CVA Help 803
overlapping meanings is wasteful and can lead to incorrect answers.
Furthermore, levels for related attributes may not combine naturally with one another. Though it can lead
to more realistic interviews, it is usually detrimental (and sometimes fatal) to prohibit levels from
occurring with others. (There are some intriguing exceptions to this that we will not discuss in this
documentation.)
Optional Features:
Sunroof
Extended warranty
a) Create an attribute with all potential combinations of these features. This results in an
eight-level attribute, if you include the option that none of these features is available.
Option (a) adds seven parameters to the model and forces the measurement of an explicit three-way
interaction. With the more complex model definition, we can investigate whether there are diminishing
returns by bundling the features. By splitting the options into three distinct binary attributes (Option b),
only three parameters are added to the model; however, interactions are not measured.

when considering prohibiting pairs.
Too many prohibitions can lead to imprecise utility estimation and unresolvable (confounded) effects and
the complete inability to calculate stable utilities. It is better to prompt respondents that they will see
combinations during the interview that are not yet available in the market or that seem unlikely. You can
urge respondents to answer as if these products were actually available today.
804 SSI Web v8
and package types:
Brand:
Sawtooth Spritz
Kong Kola
Splut
Package Type:
2-liter bottle

CBC (full-profile) and CVA are even more sensitive to too many attribute prohibitions than ACA. ACA
tends to be more robust in the face of attribute prohibitions given that a significant portion of the
information used to estimate part-worth utilities comes from the priors section, which is not affected by
attribute level prohibitions. And, partial-profile designs generally are less affected by prohibitions than
full-profile designs.
more levels tend to get more importance. The Number-of-Levels Effect is less problematic in ACA than
full-profile conjoint methods. Even so, we suggest you at least approximately balance the number of
levels across attributes.
7. In our experience, respondents have a difficult time dealing with more than about six or eight
attributes in full-profile conjoint methods like CBC or CVA. When faced with too much information,
respondents may resort to simplification strategies to deal with the difficulty of the task. Unless
CVA Help 805
respondents employ the same sort of simplification strategies when making real-world decisions, CVA
results may place too much emphasis on the few most important features.
Traditional CBC or CVA will work well for relatively small conjoint designs, and we generally suggest
other methods (such as Adaptive CBC or ACA) for larger problems.
806 SSI Web v8
6.2.1.2 Level Prohibitions and CVA Studies

Prohibitions should be used sparingly, or not at all. Specifying unnecessary or excessive prohibitions is
currently exists in the market or what the client is considering to offer. Sometimes a product is shown
with all the best features at a relatively low price; or two attribute levels that would not naturally occur in
the real world are paired together. The inclination is simply to prohibit such combinations. Newcomers
to conjoint analysis commonly think that by specifying prohibitions they can concentrate the
respondent's effort on the product combinations of most interest to the client, which should in turn
improve utility estimation for those most relevant combinations. This unfortunately is not always the
case with main-effects utility estimation, which CVA (like most conjoint approaches) employs.
Prohibiting certain combinations of levels from occurring often decreases the overall quality of the utility
estimates. In other words, by asking respondents to evaluate even those product combinations that the
client is unlikely to offer, we often improve the estimates of the product combinations that the client is
most likely to consider. We urge you to exercise restraint when considering prohibiting pairs. Please
see the discussion of prohibitions in the CVA Tutorial and Example section of the CVA documentation
for an example and guidance regarding using limited prohibitions to achieve specific objectives in CVA
designs.
Too many prohibitions can lead to imprecise utility estimation. It is better to prompt respondents that
they will see combinations during the interview that are not yet available in the market or that seem
unlikely. You can urge respondents to answer as if these products were actually available today.
Users commonly ask, "how many prohibitions can I specify without seriously damaging the results of my
study?" This cannot be answered without more information. It is not the sheer number of prohibitions
but the specific pattern of prohibitions that more directly affects the degree of correlation among the
attribute levels in the design matrix, and thus the design efficiency.
Assume that the researcher wants to specify three prohibitions between a two-level attribute and a three-
level attribute. There are just six possible combinations that can occur when combining those two
attributes. If the researcher prohibits three of those combinations from occurring, this eliminates half of
the possible combinations. There would probably be less damage to the efficiency of the design if three
total prohibitions were allocated across three separate attribute combinations (one per pair of attributes).
The researcher in this instance may decide to specify the three non-prohibited combinations of those
two attributes as a single attribute, rather than as two separate (and not independent) attributes. The
main drawback of this approach is that after combining these two attributes, the researcher cannot
compute the attribute importance or the relative part-worths of each attribute independently.
CVA Help 807
6.2.2 Designing the CVA Tasks (Questions)

6.2.2.1 Introduction to CVA Questionnaires
CVA offers two main questionnaire layout options: single concept (ranking or rating), and pairwise
comparison (ratings only). The approach you use is driven by which format best reflects decision-making
for the market you are studying, and the kinds of data analysis you need to perform.
Once you have entered your attributes and levels and specified a layout (single-concept or pairwise
comparison) you are ready to generate an experimental design (the conjoint questions). CVA's designer
uses a computer search algorithm that seeks to maximize D-efficiency, given the specifications for your
study (i.e. attributes and levels, and number of requested tasks). Designs with high D-efficiency support
independent and relatively efficient estimation of the part-worths.
Pairwise Presentation
CVA uses Pairwise Comparison as the default design and question presentation format. With pairwise
presentation, respondents are asked to compare two products, as shown below:
With small numbers of attributes, the paired comparison format has desirable characteristics. Paired
comparisons have a long history in the field of psychometrics, where they have been the method of
choice for the precise measurement of small subjective differences. When respondents compare two
products side-by-side, it can help them draw finer distinctions between products offering different
features. However, if two products are shown per question, respondents must evaluate twice as much
information as single concept designs (described below). All research we are aware of to date suggests
that relative part-worths are very similar whether estimated from single concept or paired comparison
conjoint questionnaire formats.
We suggest the pairwise design for most CVA-type conjoint projects. However, the pairwise
questionnaire only captures the relative differences in a respondent's preferences for attribute levels.
Pairwise questions never measure respondents' absolute level of interest in the product concepts.
Comparative information is all that is necessary for estimating relative part-worths and running
competitive market simulations. If you need to run "purchase likelihood" simulations within the market
simulator, you should use the single concept presentation, described next.
808 SSI Web v8
Single Concept Presentation
Single concept presentation is commonly known as "card-sort." Respondents can be shown one
product at a time and asked to rate how desirable each is or how likely they would be to purchase each.
An example is shown below:
This method is particularly useful if the "purchase likelihood model" is to be used in the simulator.
CVA's ordinary least squares (OLS) routine is appropriate for analyzing single concept ratings.
Alternatively, each of a set of product profiles can be printed on individual cards, and respondents can be
asked to sort the cards from most to least preferred. In this case, the responses are rank orders of
preference, and CVA's monotone regression routine is suggested for developing utilities. Ranking tasks
(like pairwise comparisons) only elicit comparative information.
CVA Help 809
6.2.2.2 Advantages and Challenges of CVA Questionnaires

Although CVA's question format has advantages in studies with small numbers of attributes, its task can
be challenging for the respondent. To be sure that your CVA questionnaire is appropriate for the study,
you must:
keep the number of attributes small,

pretest the questionnaire, and
conclude that the resulting utilities are reasonable,
before you proceed.
Any conjoint questionnaire can be difficult for the respondent, but CVA questionnaires can be particularly
so. Unlike our ACA System for Adaptive Conjoint Analysis, CVA does not use any strategy for
simplifying the respondent's task. Every question involves consideration of all the conjoint attributes.
CVA can create either single concept (card sort) or pairwise comparison (two concepts at a time). With
pairwise questions, the amount of information the respondent must consider is twice as great as with
single concept methods.
Unless the number of attributes is quite small, (no more than about six to eight) there is risk that
respondents may become overloaded, and that the quality of the data collected may suffer. For this
reason, we recommend other types of questionnaires if many attributes must be studied.
In summary, although the CVA question format can be very effective when the number of attributes is not
too large, we are concerned that inappropriate use of CVA may produce questionnaires that are too
difficult for respondents. As with any conjoint method, but particularly so in this case, it is essential to
pretest the questionnaire:
answer the questionnaire yourself, and analyze your own data to make sure the part-worth utilities
mirror your own values.
have others answer the questionnaire to report to you about whether it's too difficult.
have a sample of relevant respondents answer the questionnaire, and analyze their data. Be
particularly alert for "nonsense" results with respect to price. If many respondents have higher
utilities for higher prices, then the data are suspect.
If you do keep the number of attributes small, and if you do pretest the questionnaire and conclude that
the resulting utilities are reasonable, then you can proceed with confidence.
810 SSI Web v8
6.2.2.3 Selecting the Number of Tasks

After you have defined attributes and levels and decided which questionnaire format is best for your
study, you need to decide how many conjoint questions (tasks) to ask. This decision depends on how
many separate part-worth parameters need to be estimated. The more attributes and levels in your
study, the more part-worths to be estimated; and the more parameters to estimate, the longer the
questionnaire.
CVA examines the list of attributes and levels in your study and provides an initial recommendation
regarding the number of tasks (conjoint questions) to ask. The recommended number of tasks provides
three times the number of observations as the number of parameters to be estimated. The number of
parameters to be estimated is determined by the formula:
Total number of levels - number of attributes + 1
Although it is mathematically possible to estimate utilities by asking only as many questions as the
number of parameters to be estimated, we strongly recommend against this practice when using
standard estimation routines (OLS and Monotone Regression). Asking the recommended number of
tasks helps ensure enough information to calculate stable estimates for each respondent.
If your design includes many attributes and levels, you may notice that the recommended number of
questions exceeds what you could reasonably expect respondents to complete. This leads to a difficult
decision. Too few questions limit the degrees of freedom of the design (and the precision of the part-
worth estimates); but if you ask too many conjoint questions respondents may become fatigued or bored
and provide data of poor quality. In our experience, asking more than about 30 conjoint questions may
result in poor quality data. Testing your questionnaire with actual respondents can help you determine
the optimal number of questions to ask for your project.
CVA will not let you specify fewer questions than the number of parameters to be estimated, and will
warn you if you don't ask at least 1.5x the number of parameters to be estimated. A number of studies
in the literature suggest that some experienced conjoint analysts are willing to ask as few as 1.5x the
number of questions as parameters to estimate. Indeed, the use of HB estimation may slightly reduce
the number of questions you decide to ask respondents, as it estimates part-worths based on
information from the current respondent plus information from the other respondents in the same data
set.
Note: if using HB estimation, it is possible to ask respondents fewer questions than parameters to be
estimated, while still obtaining good aggregate results. (See Sparse Designs and Large Samples for
more info.) In that case, the questionnaire design still contains more tasks than parameters to estimate,
but each respondent is asked to complete just a random subset of the tasks. Please note that there is
loss in information at the individual level by taking this approach (relative to asking respondents to
complete all tasks). If having respondents skip tasks, you should use larger sample sizes than usual to
compensate for the loss in information .
CVA Help 811
6.2.2.4 Throw out "Obvious" Tasks

CVA can use the a priori preference relationships (i.e. "best to worst") specified when you provided your
attributes to avoid presenting conjoint profiles that would result in obvious answers. For example,
questions including a product that has many good features at a low price or many bad features at a high
price may be less efficient for deriving conjoint utilities. Questionnaires that avoid the most obvious
comparisons are more challenging for respondents and provide more useful information. A moderate
degree of balance in the expected utility of the concepts in CVA questionnaires is therefore desirable.
However, we caution that too much balance can increase the difficulty of the questionnaire and the error
in the responses, counteracting the benefits of improved utility balance.
CVA's method of discarding "obvious" tasks takes a conservative approach that eliminates only the most
extreme cases. Advanced researchers can further balance the design by specifying additional
prohibitions. This may be a reasonable approach so long as the researcher conducts substantial
quantitative pretests to ensure that the benefits from utility balance are not counteracted by increased
response error.
Recall that pairwise designs show a product concept on the left and another on the right. For each
attribute for which CVA has a priori order information, if the concept on the left has a higher utility than
the one on the right, a +1 is scored; if it has a lower utility, then it is scored -1. For single-concept
designs, only extreme levels are counted. Highest utility levels are scored +1, and lowest levels -1.
Non-a priori attributes are ignored, because CVA has no way of knowing which level is preferred. If the
imbalance exceeds some threshold, then the task is considered "obvious" and discarded. The threshold
used to determine "obvious" tasks is >75% of a priori attributes marked as +1.
The following table below shows the threshold used for each number of attributes (up to 10) to determine
"obvious" tasks:
"Obvious" Tasks Threshold
# TOTAL
Attributes Threshold
2 2
3 3
4 3
5 4
6 5
7 6
8 6
9 7
10 8
For example, for studies involving just 4 attributes, the threshold score must reach 3 before the task is
considered "obvious."
Because a modest degree of utility balance is a desirable quality, it may make sense in the design
phase to set any attribute to a priori for which you have a reasonably good idea about which levels are
preferred to others. For example, if some brands are generally preferred to others, you might specify
them in order from generally least to most preferred and then set the a priori setting to "worst to best." If
you take this approach, make sure to remove any a priori setting for all attributes that do not have
a clear order of preference before calculating utilities (under Additional Utility Constraints), or you
may impose constraints that are not consistent with some respondents' preferences.
812 SSI Web v8
6.2.2.5 CVA Design Strategy

After you have specified attributes, any a priori ordering, how many tasks to ask, and whether to use
"single" or "pairwise" concept tasks, you are ready to generate a design. The design specifies the
combination of attribute levels (profiles) shown in each conjoint question, for a single or multiple versions
of the questionnaire.
To generate a design using the defaults, click Generate Design.
Number of Tasks (Questions)
Based on the number of attributes and levels in your study, CVA provides an initial recommended
number of tasks (conjoint questions). The recommendation is based on asking three times as
many tasks as parameters to be estimated, where the number of parameters to estimate is equal
to:
Total number of levels - number of attributes + 1
The recommended number is an ideal number from a statistical standpoint that often is not used in
practice. The recommended number is often more than respondents can reasonably complete. Use
this as a guideline, then specify how many tasks you actually want to use within the software.
One of the most important decisions with a conjoint design is how many questions to ask. If you
don't ask enough questions, it may result in noisy part-worth estimates. If you ask too many
questions, you may overtax your respondents, leading to decreased data quality and/or abandoned
surveys. CVA warns you if you do not ask at least 1.5x as many tasks as parameters to be
estimated, and does not let you ask fewer tasks than the number of parameters to estimate. See
Selecting the Number of Tasks for more information.
Randomize Attribute Position within Concepts
Randomize Attribute Order

Randomize Attribute Order specifies whether to present the attributes in random presentation order
within a concept. If this is selected, the attribute list is randomized once per respondent, and all
tasks within a respondent's interview will display the attributes in that given order. This can be
useful to control order effects across respondents.
First Randomized Attribute

The first attribute in the range of attributes to be shown in random presentation order. Specify a "1"
if you wish all attributes to be randomized. If, for example, you want the first and second attributes
always to appear as the first two attributes in a product concept, specify a "2."
Last Randomized Attribute

The last attribute in the range of attributes to be shown in random presentation order. Specify the
last attribute number if you wish all attributes to be randomized. If, for example, you had a total of
five attributes in your study and you wanted the fourth and fifth attributes always to appear as the
last two attributes in a product concept, specify a "3."
Advanced Settings
When you click Show Advanced Settings, the following fields are available:
CVA Help 813
Number of Versions (default=10): A version of the questionnaire represents a single series (block)
of conjoint questions. If you want respondents to receive different sets of questions, you can
request multiple versions (up to 10). If you are conducting paper-and-pencil studies, you probably
do not want to manage more than a few different versions because of the increased hassle of
dealing with unique questionnaire versions. However, when you are able, additional questionnaire
versions decrease psychological order and context effects and thus improve your overall results.
See the section below entitled "A Single Version or Multiple Version?" for further guidance.
Design Seed (default=1): CVA's design generation algorithm requires a starting seed. You can use
any integer from 1 to 9999. If you repeat the analysis using a different starting seed, you will
usually obtain a slightly different (sometimes better) result.
Throw out Obvious Task s (default = yes): CVA can exclude product concepts that are clearly
better or worse than others from the questionnaire. To use this, you need to specify that certain
attributes have a priori preference order.
Task Pool Multiplier (default=10): When generating a questionnaire version, the Task Pool
Multiplier (multiplied by the number of requested tasks) controls how many unique tasks will be
used in the pool of candidate tasks to include. For example, if you request 18 tasks with a Task
Pool Multiplier of 10, 180 unique tasks will be searched among to find an optimal 18 tasks.
Version Pool Multiplier (default=10): This defines how many tries (passes) will be attempted from
different starting points. For example, if you are requesting 10 questionnaire versions and the
Version Pool Multiplier is 10, then 100 attempts will be made to find optimal versions of the
questionnaire. The top 10 versions (in terms of design efficiency) will be used in the final plan.
Hints: The defaults we've specified tend to work well and quick ly. But, you can often improve your
questionnaire's design efficiency by ask ing CVA to try harder. You can increase the Task and Version
Pool Multipliers to search deeper and longer for better solutions. You can also change the random
design seed to repeat the process from different starting points.
CVA Design Procedure
The conjoint design is a critical component to the success of any conjoint project. Attributes must vary
independently of each other to allow efficient estimation of utilities. A design with zero correlation
between pairs of attributes is termed "orthogonal." Level balance occurs if each level within an attribute
is shown an equal number of times. Designs that are orthogonal and balanced are optimally efficient.
In the real world, it might not be possible to create a perfectly balanced, orthogonal design for a particular
set of attributes and prohibitions consisting of a reasonable number of tasks. The CVA approach
produces high quality designs automatically, although they probably will not be perfectly balanced or
orthogonal. CVA lets you test the efficiency of a design before fielding your study. Testing the design
also lets you study the impact of including prohibitions, or asking fewer than the recommended number
of questions.
CVA provides an easy-to-use tool for generating well-balanced, "nearly-orthogonal" designs. CVA
generates a pool of potential conjoint questions (from which the final design will be chosen) using a
relatively simple procedure. If the total number of concepts to generate in the pool is greater than or
equal to the number of total possibilities, a complete enumeration is done of all non-prohibited
possibilities. If the pool represents a subset of all possible combinations, for each question to be
composed CVA does the following: For each attribute it picks a pair of levels randomly from among all
814 SSI Web v8
permitted pairs that have been presented the fewest times. A random decision is made about which
levels will appear on the left and on the right (for pairwise designs). No pair of levels will be repeated until
all other permitted pairs have been shown. Each pair of levels will be shown approximately the same
number of times, and each level from one attribute is equally likely to be shown with any level from
another attribute.
CVA's designer uses the following steps to select efficient designs, given the questionnaire
specifications:
1) CVA generates a pool of potential conjoint questions equal to, by default, 10 times the
requested number of questions (assuming that many unique questions exist).
2) The D-efficiency of the design is calculated for the pool, excluding one conjoint question at
a time. The one task that contributes least to the efficiency of the design is discarded, and
the process repeated until the desired number of tasks remains. (See Technical Notes about
the CVA Designer for more information about D-efficiency).
3) CVA then examines every potential 2-way swap of conjoint questions that remain with
those that were discarded or are available in the pool of potential conjoint questions. CVA
swaps any pairs of questions that result in an increased efficiency.
4) Next, CVA examines the frequency of level occurrences for each attribute. It investigates
changing levels that are over-represented in the design to levels of the same attribute that are
under-represented. Any changes that result in improved D-Efficiency (and are not prohibited)
are retained.
5) For pairwise designs, CVA flips left and right concepts to improve the left/right balance of
the design.
CVA repeats steps 1 through 5 multiple times and selects the best n solutions (where n is the total
number of versions of the questionnaire you'd like to use). CVA uses re-labeling (trading pairs of levels
within attributes across all tasks within a design) to create approximate aggregate level balance across
all versions. (Relabeling doesn't change the efficiency of each individual design.) An additional check is
performed to ensure that no two versions of the questionnaire are identical.
A Single Version or Multiple Versions?
Previous versions of our CVA software employed a single version (block) of the questionnaire design.
Each respondent received the same questionnaire (though the order of the tasks could be randomized).
From a statistical standpoint, a single version of the plan was all that was typically needed to ensure
precise estimates of main effects. However, researchers recognize that respondents are human, and
therefore the quality of the estimated utilities will depend on controlling psychological/order/context
effects. Randomizing the order of the tasks is a good way to control for context effects. However, these
effects may be further reduced by using more than one version of the questionnaire.
In addition, CVA/HB for hierarchical Bayesian analysis of part-worth utilities would seem to benefit from
multiple versions. Because HB pools information across respondents, having greater variation in the
design matrix (across all respondents) may improve individual estimates due to improved population
estimates.
Multiple versioning is nothing new for users of Sawtooth Software's tools. In our CBC software, the
default (when using computer interviewing) is to use 300 versions of the questionnaire. In reality, this is
CVA Help 815
probably overkill, since the vast majority of the benefits of multiple versioning are captured after the first
few versions. But, since CBC's algorithms for developing designs are generally fast and computer
interviewing automatic, it is as easy to develop and field 300 versions as 4. With CBC, there is another
good reason for including multiple versions of the questionnaire: choice-based designs can be stronger
than traditional conjoint for estimating interaction effects, and additional versions of the questionnaire
help stabilize those effects (assuming pooled estimation).
With CVA, there are benefits for including multiple versions of the questionnaire (to help reduce
psychological effects), but there are reasons to believe that employing a very large number (such as 300)
is not necessary.
1. CVA's search algorithm tries to find the optimally efficient set of questions given a
requested number of tasks. In contrast, CBC uses a "build up" approach that while
achieving very good design efficiency, doesn't attempt to find the optimal set of questions
given a requested number of tasks. Therefore, it can be expected that CVA is better
suited to choosing a smaller number of versions that are quite effective statistically.
2. CVA only estimates main effects (a single utility value for each level in the study). CBC
can estimate both main effects and first-order interactions. The goal of estimating main
effects suggests requiring fewer questionnaire versions.
3. CVA is generally a slower (and more thorough) way to generate designs than CBC.
Generating a very large number of versions by default (such as 300) could take a
significant amount of time.
Because of these arguments, CVA permits up to 10 different versions of the design. In our opinion, the
benefits for using more than 10 versions with CVA would be miniscule. For two decades, previous
versions of CVA have used single-version plans, with generally good results. Allowing a few additional
versions is convenient when interviewing via computer, and should lead to slight improvements.
When fielding multiple versions of the questionnaire, it is desirable (but in nowise a requirement) that
each version of the questionnaire be completed by approximately the same number of respondents.
When fielding your study on the Web, this happens automatically. Each respondent starting a CVA
survey is given the next available design. Once the last design is reached, the next respondent receives
the first design, and the process repeats.
Design Suggestions
We have stressed the importance of including enough conjoint questions to ensure efficient designs.
However, design efficiency is not the only reason for including two to three times as many questions as
parameters to be estimated. All real-world respondents answer conjoint questions with some degree of
error, so those observations beyond the minimum required to permit utility estimation are useful to refine
and stabilize utility estimates.
While there is generally a positive relationship between the number of conjoint tasks in the design and D-
efficiency, there are many exceptions. It is possible for designs with few or no degrees of freedom to
have 100% D-efficiency. This means that this design is optimally efficient for estimating main-effect
parameters. But this assessment ignores human errors. By increasing the number of tasks, you
provide more opportunities for respondent errors to cancel themselves out. Increasing the number of
tasks in this case (beyond the saturated orthogonal plan) will slightly reduce the reported D-efficiency
(from 100% to something slightly less than 100%), but the precision of the estimated parameters may be
significantly improved due to a greater amount of information provided by each respondent.
816 SSI Web v8
To summarize our point, don't focus solely on D-efficiency. Good CVA designs foremost should include
enough conjoint questions relative to the number of parameters to yield relatively precise estimates of
part-worths. Given an adequate number of conjoint questions, we next focus on selecting a design with
high D-efficiency.
Importing and Exporting Designs

You may import or export designs, as described in the section of this documentation entitled Importing/
Exporting CVA Designs. This is useful if you need to use a particular set of questions developed by a
colleague, using a different piece of software, or even from a design catalog. Importing designs is also a way
to add user-specified holdout questions.
CVA Help 817
6.2.2.6 Testing the CVA Design

After you have created a design (either through CVA's designer or by importing a design from a text file),
you can review the D-efficiency and receive additional diagnostic information by using CVA's Test Design
functionality. When you select this option, a report is shown on the screen:
CVA Design Efficiency Test

Tasks are 'Single Concept' using a seed of 1.

Includes 140 total tasks (14 per version).
Each task includes 3 attributes.
Design Version Efficiencies

-------------------------------------------------------------
Version Efficiency
1 0.97528
2 0.97528
3 0.97528
4 0.97528
5 0.97528
6 0.97528
7 0.97528
8 0.97528
9 0.97528
10 0.97528
Attribute Efficiency Coefficients (including all versions)

-------------------------------------------------------------
Attribute Efficiency
1 0.999
2 0.995
3 0.999
One-Way Frequencies (by version)

-------------------------------------------------------------
Level Total Ver1 Ver2 Ver3 Ver4 Ver5 Ver6 Ver7 Ver8 Ver9 Ver10
1 70 7 7 7 7 7 7 7 7 7 7
2 70 7 7 7 7 7 7 7 7 7 7
1 46 4 4 4 6 6 6 4 4 4 4
2 48 4 4 4 4 4 4 6 6 6 6
3 46 6 6 6 4 4 4 4 4 4 4
1 35 3 4 3 3 4 4 3 4 4 3
2 35 4 4 3 4 4 3 3 3 4 3
3 35 3 3 4 3 3 4 4 4 3 4
4 35 4 3 4 4 3 3 4 3 3 4
Two-Way Frequencies (all versions)

-------------------------------------------------------------
Att/Lev 1/1 1/2 2/1 2/2 2/3 3/1 3/2 3/3 3/4
1/1 70 0 23 24 23 17 18 18 17
1/2 0 70 23 24 23 18 17 17 18
2/1 23 23 46 0 0 12 12 11 11
818 SSI Web v8
2/2 24 24 0 48 0 12 11 13 12
2/3 23 23 0 0 46 11 12 11 12
3/1 17 18 12 12 11 35 0 0 0
3/2 18 17 12 11 12 0 35 0 0
3/3 18 17 11 13 11 0 0 35 0
3/4 17 18 11 12 12 0 0 0 35
The Test Design report shows you the D-efficiency of each design that was generated (CVA generates
10 design versions by default), which provides an indication of the overall quality of each design. The
highest possible number is 1.0, and the lowest is zero. For more information about D-efficiency, see
Technical Details on the CVA Designer.
Next, CVA reports attribute efficiencies (described in greater detail below), assuming all designs are
appended as an aggregate design plan. Low efficiencies for some attributes can be an indication that
some levels of that attribute might not be included enough times in the design, or that this attribute has
too many prohibitions (and is thus correlated with another attribute(s)). These coefficients can be useful
diagnostics for a conjoint questionnaire design during the pilot stages of a study. If the relative efficiency
is too low for a particular attribute, then additional questions should be introduced, or the design should
be modified to have less dependence among the attribute levels.
If you have low D-efficiency for some or all designs, or if some attributes have particularly low efficiencies,
you should consider removing constraints, asking more questions, or regenerating your design. Poor
efficiencies can compromise the quality of your utility estimates.
Next, the one-way frequencies are displayed, considering all designs together ("Total" column) and then
for each design separately ("Ver 1" through "Ver 10" columns). The frequencies reflect the number of
times each level occurs in the design. Following that are the joint frequencies, or the number of times
two levels appear together within the same product concept, across all designs. Although not shown
above (to conserve space), similar joint frequency tables follow for each separate design.
More Details on Attribute Efficiency Calculation
The Attribute Efficiency statistics describe the quality of the conjoint design. We are mainly interested
in the precision of estimates of differences in utility among levels of the same attribute. The relative error
variance of the difference is obtained from elements of the inverse of the correlation matrix among
independent variables. The relative error variance of the difference between two utility estimates is
obtained by summing those two diagonal elements of the inverse and subtracting twice the
corresponding off-diagonal element. We compute the relative error variances of all within-attribute
pairwise differences.
Suppose there were an orthogonal design for the same attributes and levels, and all the levels of a
particular attribute appeared the same number of times. (Such a design may or may not actually exist.)
Then we could compute similar relative error variances for that "ideal" design.
Our "Relative Efficiency" index is the ratio of the sum of theoretical relative error variances for such an
ideal design, divided by corresponding sum of relative error variances for the actual design used. The
overall relative efficiency index is reported, as is an index for each attribute. The best possible value is
1.000, although values that high may not actually be achievable for some combinations of numbers of
attributes and levels, since the required orthogonal designs may not exist.
CVA Help 819
6.2.2.7 Importing/Exporting CVA Designs

One of the strengths of the CVA system is its ability to generate experimental designs (the combinations
of attribute levels to show to respondents) in an efficient and easy manner. Also, multiple versions (up to
10) of the questionnaire may be generated, so different respondents see different combinations of
attributes within the conjoint tasks.
Some researchers may wish to export the design, to examine ahead of time the combinations that will
be asked of respondents (and perhaps to perform additional analysis regarding the design).
Other researchers may have developed questionnaire designs using other software and wish to import
those designs for fielding within the CVA system. They can do this through CVA's Import Design
capability. Another use of the Import feature is to add holdout conjoint questions ("fixed" CVA questions
that are asked exactly the same way across all questionnaire versions) in your study design.
For all purposes, the design file uses the same comma-separated (.CSV) layout. A .CSV file can be
directly opened and saved with Excel, and the examples below assume you are working with the
comma-separated file in Excel. (If using a text editor, the values in the file are separated by commas.)
If you import a design, any existing design will be overwritten.
Import/Export File Format
The file format is as follows.
Column 1
Indicates the questionnaire version.
Column 2
Indicates the task (conjoint question) number.
Column 3
Indicates the concept number. Designs showing one product concept at a time always have
concept #1. Designs showing two product concepts at a time (pairwise presentation) have concept
#1 and concept #2.
Columns 4 through n
Shows the attribute levels associated with the current concept, task, and version number.
Single-Concept Presentation:
In the example below, only two versions of the questionnaire are shown. There are three attributes in this
study.
820 SSI Web v8
CVA Help 821
Pairwise (Two-Concept) Presentation:
In the example below, only two versions of the questionnaire are shown. The design reflects pairwise
presentation layout (two concepts per question). There are three attributes in this study.
822 SSI Web v8
Additional Rules:
The version must start with 1 and proceed in that order.
The number of versions and number of tasks must match the number of versions and tasks specified in
the software.
The number of concepts must be constant across all tasks. The number of concepts must match the
number of concepts specified in the software.
Any prohibitions specified in CVA will not be checked against the imported design.
CVA Help 823
6.2.2.8 Including Holdout Choice Tasks in CVA Questionnaires

Holdout Choice Tasks
One of the reasons that Choice-Based Conjoint has gained in popularity over the years at the expense of
traditional full-profile conjoint is that the resulting CBC utilities are scaled based on the choice data
(rather than taking its scaling from the arbitrary choice of the scale chosen in CVA). When applied in
market simulations, these choice-developed utilities lead to shares of preference for competitive products
that are directly linked to probabilities of choice. In contrast, CVA utilities are estimated from rating
scales or rank-order data. These scales do not automatically lead to market simulations with
appropriate choice probability scaling.
CVA users can achieve approximate choice-probability scaling for utilities by adding holdout choice
tasks within their CVA surveys (as Select, Grid, or Free Format question types). By tuning the scale
factor (exponent) within Share of Preference or Randomized First Choice simulations to best fit holdout
choice probabilities, the simulator may be tuned to resemble choice-based scaling. The tutorial in this
online help describes the use of a holdout choice task. Additional information on holdout choice tasks
can also be obtained from a technical paper entitled "Including Holdout Choice Tasks in Conjoint
Studies" at www.sawtoothsoftware.com in the Technical Papers library.
Holdout choice tasks present a set of product profiles (usually three or four) described in full profile using
the same attributes and levels used in the CVA questionnaire. Respondents are asked to choose the
one alternative from the set that they most prefer. The analyst observes the probabilities of choice
across the competing alternatives by simply tabulating the selections. Then, these same product
profiles are specified as a scenario in the market simulator. Using a Share of Preference or Randomized
First Choice model, the analyst uses the CVA utilities to predict the probabilities of selection for the
same alternatives in the holdout task. If the conjoint data are working well and are scaled properly, the
predictions should closely match the observed choice probabilities. The analyst tunes the exponent
(scale factor) in the market simulator so that the predicted choice probabilities most closely fit the
observed probabilities of choice from the questionnaire.
Unless you have additional choice information by which to rescale your utilities for use in market
simulations, the scaling of shares of preference may not be appropriate when using CVA. We strongly
encourage you to use holdout choice tasks in every CVA study, and that you tune your market
simulations using the exponent to fit these holdout choices.
CVA-Looking Holdout Tasks
Some researchers may wish to include ratings-based holdout questions within their CVA studies that
look exactly like the other CVA questions. To do this, you can export your CVA design to a .csv file,
and then modify the design file to insert a few additional holdout tasks. Then, modify the design settings
to specify a new number of total questions (including the additional tasks you've added). Then, import
the new design.
When you conduct analysis, make sure not to include the holdout tasks within utility estimation. Within
the Settings dialog, use the Task s to Include filter to exclude the holdout tasks from utility estimation.
824 SSI Web v8
6.3 CVA/Web Paper and Pencil Studies

6.3.1 CVA/Web Paper and Pencil Studies
SSI Web lets you run your CVA studies via Web, over PCs (CAPI installation), or as paper-and-pencil
questionnaires. The process of running paper-based CVA studies is very similar to computer-based CVA
studies. You should set up your attributes, levels, and compose your conjoint tasks in the same manner
as if you were going to run a computerized study. However, you should probably choose plain black
(text) on white (background) for questionnaire colors, unless you plan to print the questionnaires in color.
By default, CVA's questionnaire designer assumes computerized interviewing with 10 questionnaire

versions (you can change the number of versions through the Advanced Settings). Because of the added
costs for managing multiple versions of the questionnaire in the field (and the added possibility of
introducing an error), you will probably not want to employ so many versions within paper-and-pencil
interviewing. You will create one or just a few versions (design blocks) of the questionnaire and assign
respondents (randomly) to take one of the different questionnaire versions. For more information about
the choice of how many versions to use in the questionnaire, please see the section on CVA Design
Strategies.
Saving the Questionnaire
When you click the Field | Create Paper & Pencil Interviews… button, each version of your
questionnaire is saved to a separate file within the Paper-and-Pencil folder within your study directory.
The files are named STUDYNAME_EXERCISENAME_V#.htm, where STUDYNAME is the four-character
or less study name for your project, and # is the version number of the questionnaire (always beginning
with version #1). These files are in HTML format and may be opened with most any recent word
processing software.
You should do any additional formatting you like to make the questionnaire appear as you wish prior to
printing.
Note: It is helpful to print the version# and task# with each task in a paper-and-pencil study. This gives
an additional reference while performing data entry, helping ensure that you do not introduce errors when
specifying the version numbers seen by respondents. An easy way to automatically label your choice
tasks is to add a function [%CVAVersion( )%] in the header or footer of the choice task. When you do
this, each task includes a label such as D - 1, T - CVA_1, meaning "Design #1, Task CVA_1."
Fielding the Questionnaire
Make sure when fielding the questionnaire that you are able to match the different questionnaire versions
correctly with respondent answers. You should also try to randomly distribute the questionnaire versions
among respondents, so that roughly equal numbers of respondents complete each version.
Data Entry
After you have collected the data, you prepare a .csv file containing the respondent answers. This file
follows a required .csv (comma separate values) format, with specific labels in the first row of the file. To
see that format, prepare a template for yourself by clicking Field | Create Accumulated Data Template
File.... A file named Accumulated Data.csv is saved to your study folder. Use this template to arrange
your data, and make sure to use the prescribed header row within your data file.
CVA Help 825
The layout is:
Column 2: Version#
Column 3: CVA answer #1, etc.
Any missing data are coded as blank (meaning, if you viewed the data file using Excel, the cell would be
empty; if you viewed the file with a text editor, the missing value would look like two consecutive
commas with no text in between).
Ranking Data Formats
If you have used single-concept presentation and asked respondents to rank the cards from best to
worst, there are two possible formats you can use for recording the data. You use a comma-separated
file, as described directly above. However, you can either record the data as:
Answers are ratings/rankings

Answers are card numbers
If the answers are ratings/rankings, the third field in your data file contains the ranking assigned to Card
#1 in your design. The fourth field in your data file contains the ranking assigned to Card #2, etc.
If the answers are card numbers, then the third field in your data file contains the card number for the
card chosen as "best" by the respondent. The fourth field would contain the card number for the card
chosen 2nd best, etc.
Accumulating the Data
When you have prepared the data as described above, you are ready to Accumulate Paper & Pencil
Data. Click Field | Accumulate Conjoint Paper & Pencil Data.... Then, browse to your comma-
separated data file (should carry a *.csv extension) that contains the respondent answers.
Once you have accumulated the respondent data, you are ready to compute utilities using the standard
dialogs used for any CVA study.
826 SSI Web v8
6.4 CVA Advanced Designs

6.4.1 Conditional Display for CVA
Conditional display may be used for showing a graphic to represent the occurrence of multiple attribute
levels appearing together within a concept in a CVA task. For example, you may have two separate
attributes for brand and package size, but a single picture reflects the joint brand/package size concept.
Where the text for a particular brand and package size might appear in a CVA concept, you may wish
to replace that with a graphic.
But conditional display is not limited to just graphics, as it simply adds (or replaces, depending on your
Attribute Visibility setting) the existing level text. Other possibilities include:
Multi-media files
Text
JavaScript
Attribute Label (Optional)
If you are using Conditional Display and have chosen to show the graphic (or other conditional display
"element") at the top or bottom of the conjoint question, you can optionally specify an attribute label to
be shown in the conjoint question (on the left-hand side of the task). This attribute label is only used
(and is only applicable, otherwise it is "grayed out") if you have specified that attribute labels are to
appear in the choice task. You specify that attribute labels should appear within the choice task by
clicking Compose | Conjoint Settings | CVA Settings… and selecting the Format tab. Then, check the
Include Attribute Label in Tasks box.
Note: if in Step 3 of the Conditional Relationships Wizard, you specified that the conditional display
element is to replace the text of a certain attribute within the task, then the attribute label used for the
conditional element is the original attribute label for the replaced attribute text (that you originally typed
within the Specify Attributes and Levels dialog).
CVA Help 827
6.4.2 Conditional Pricing for CVA

Warning: this is an advanced area. Users should pay special attention to the fact that CVA does
not automatically support the estimation of interaction effects. To reduce the likelihood that
interaction effects may be required, it is important to construct conditional pricing tables in a
proportional manner. Even if using "proportional" lookup tables, CVA may still fail to capture
interaction effects important to modeling buyer behavior. Conditional pricing increases the
likelihood that significant interaction effects may occur in the data. CBC (with its similar
conditional pricing capability) may be the better choice in such situations, because it is designed
to detect and model interaction effects. Please also see the section entitled Modeling Interaction
Effects with CVA for more information regarding these issues.
In pricing research, it is sometimes very useful if prices for product concepts are made to depend on
other attribute levels (such as brands). The first versions of CVA could handle this in a limited way by
using prohibitions. For example, one could prohibit high prices from being shown for some products, and
prohibit low prices from being shown with others. Such prohibitions, however, can lead to very inefficient
designs.
There are some work-arounds for dealing with situations where prohibitions with price seem to be
needed. For example, if package size is an attribute, prices for the 64 oz package would be much
higher than prices for the 16 oz size. One approach is to express price as a percentage above or below
the "normal price" for that package size, such as "10% above the average price." Another is to show
unit prices, rather than prices per package. For example, with detergent we might study 16, 32, and 64
ounce packages, but present prices in terms of "cents per ounce." These are ways of making a single
set of price levels work for all package sizes without specifying any prohibitions, but it would be
preferable to display actual prices appropriate for each package size.
The conditional pricing option lets you create a look-up table to determine the prices to show for specific
combinations of attributes. Drawing upon the example above, assume we have three attributes as
follows to describe detergents:
BRAND
Brand A
Brand B
Brand C
PACKAGE
16 oz. package
32 oz. package
64 oz. package
PRICE
"Low price"
"Medium price"
"High price"
The text for the price levels above is not shown during the interview, and serves as a place-holder only.
The price levels displayed during the interview vary according to the package sizes as follows:
828 SSI Web v8
Conditional Pricing Lookup Table
Low Price Medium Price High Price

16 oz. package $1.49 $2.09 $2.69
32 oz. package $2.79 $3.99 $5.19
64 oz. package $4.89 $6.99 $9.09
To construct this lookup table, we started with average prices for the three package sizes (middle
column). To determine the prices in the "Low Price" column, we decreased the average price by 30%
(and rounded to the nearest 9 cents). The "High Price" was calculated by increasing the average price
by 30% (and again rounding to the nearest 9 cents).
The example above illustrates how to make price dependent on a single variable (package size). CVA
lets you make price conditional on up to 3 attributes.
You will need to deal with an additional level of complexity when analyzing the results of studies that
involved conditional pricing. In the example above, even though nine unique prices were shown to
respondents, CVA's analysis programs by default still regard these as just three levels of price: Low,
Medium and High. However, we interpret the results keeping in mind that larger package sizes were
displayed with higher prices on average. We interpret the part-worth of the 32 oz. package taking into
account that it was shown at an average price roughly $1.90 cents higher than the 16 oz. package. The
main effect contains information not only about how desirable one package size is versus another, but
also about the relative price levels at which they are offered. Therefore, it is perfectly legitimate if the part-
worth utility for the 32 oz. package is lower than the 16 oz. package. This would indicate that
respondents on average did not feel the larger size was worth the extra $1.90.
The conditional pricing table must also be taken into account when using the market simulator. Unless
the conditional pricing table is explicitly used in the market simulator, to simulate a 32 oz. package at
$3.99, level 2 for price is specified, etc. However, with the most recent releases of SMRT, when you
prepare the data for analysis in SMRT (the utility file and .ATT file), the conditional pricing table is also
exported for use in the SMRT market simulator. The conditional prices are automatically read from a
special footer section in the .ATT file into the Assign Level Values area of the SMRT market simulator
(please double-check them to verify that the information was interpreted correctly, as we parse text
strings to extract the price values). Once the conditional prices are within the SMRT market simulator,
you simply specify the actual conditional price during simulations. For example, to simulate a 32 oz.
package at $3.99, instead of specifying level 2, you specify a level value of 3.99.
Users should note that the use of Randomized First Choice (or the historical Share of Preference with
Correction for Product Similarity) can be problematic with conditional pricing. These methods assume
that products sharing the same attribute levels are identical on those levels and require some downward
correction in share. But, with conditional pricing tables, even though two brands may share the same
"level" for price, the conditional pricing table may have actually resulted in unique prices for brands within
the questionnaire. Therefore, RFC may implement a correction for product similarity that isn't actually
warranted, assuming that the brands' prices were identical when in fact they were not. To avoid this
outcome, we suggest turning off the correction for product similarity with respect to price within the RFC
Method Settings dialog.
In our previous example, we used (near) constant deviations from average prices to build a proportional
conditional table. Some researchers choose to use constant absolute price differences between levels
of the conditional attribute(s) instead of proportional changes. In either case, the tables have symmetric
CVA Help 829
properties, which are desirable from an analysis standpoint. Proportional or symmetric conditional
pricing tables make it more likely that main effects utilities will fit the data well. Since CVA can only use
main effects models, this is a very desirable outcome. When using conditional pricing (especially with
asymmetric price tables), specification of interactions may be necessary to fit the data properly, and
CVA only can manage this in a limited manner (see Modeling Interaction Effects with CVA).
Specifying Conditional Pricing
Before you can specify conditional prices, you first must have defined your list of attributes and levels.
Instead of specifying the text for actual prices for your price levels, you should have used placeholder
text such as "low price," "medium price," and "high price." When the questionnaire is administered, the
placeholder text you assigned is overwritten by the text string you specify in the conditional pricing table.
When you create a conditional price lookup table in CVA, you must specify the number of attributes that
participate in the relationship (there should be at least two). If price is dependent on one other attribute
(such as package size), two attributes participate in the relationship: package size and price. Click
Compose | Conjoint Settings | Conditional Relationships… (or click the Conditional
Relationships… button from the CVA Settings dialog). A dialog appears in which you specify attributes
involved in the relationship. First, select your price attribute. The level text for this attribute is overwritten
by the prices that you specify in the conditional pricing table. Then, select the attribute(s) upon which
price is dependent.
The conditional pricing table is displayed, with as many rows in the table as required to completely
define the prices for all combinations of price and the dependent attribute(s). You can cut and paste
conditional pricing information directly from a spreadsheet program into the conditional pricing grid.
There can be a maximum of one conditional pricing table per study. However, some users have
discovered that they can use conditional display tables just as they would use conditional pricing tables.
In that case, more than one conditional pricing variable may be included. The only difference is that the
conditional prices specified as conditional display variables are not automatically carried forward to the
SMRT market simulator. In that case, you may need to use relative indices (such as 0.8, 1.0, 1.2, etc.)
when defining products in the market simulator, rather than having SMRT automatically use the
conditional pricing table within the simulator.
830 SSI Web v8
6.4.3 Modeling Interaction Effects with CVA

This is an advanced area. We assume the reader understands experimental design principles, is
familiar with dummy-coding for part-worth attributes, is familiar with tuning simulators via the
Exponent to best fit holdout data, and is able to do some additional data processing.
Background
As a traditional full-profile conjoint approach, CVA is a "main effects only" model, and assumes there are
no interactions among attributes. Many conjoint practitioners agree that one must remain alert for the
possibility of interactions, but that it is usually possible to choose attributes so that interactions will not
present severe problems. Like other conjoint methods, CVA can deal with interactions in a limited way
by defining composite variables. For example, we could deal with an interaction between car color and
body style by cross-classifying the levels:
Red Convertible
Black Convertible
Red Sedan
Black Sedan
suspected of having interactions with many others, then composite attributes will not be enough. In that
case too many parameters must be estimated to permit analysis with a purely individual-level model, and
the most common solution is to evaluate interactions by pooling data from many respondents. For
example, CBC (Choice-Based Conjoint) is considered a stronger approach for estimating interaction
terms. It can automatically estimate interaction effects between all attributes, not necessarily just those
that the analyst identifies prior to designing the study.
CVA can accommodate up to 15 levels for each attribute. Therefore, you can use the composite
attribute approach to model interactions between two attributes having as many as 5 and 3 levels,
respectively (such that the product of the levels between the attributes does not exceed 15).
Designing CVA Studies for Limited Interactions
In the previous section of this documentation, we introduced the advanced design strategy of conditional
pricing tables. We warned that these should be built with near-proportional pricing across the price tiers.
We also warned that specifying conditional pricing tables might increase the likelihood that interactions
would be needed to properly model the data. The following instructions provide a framework for advanced
analysts to develop designs that formally support estimation of limited interactions and also examine
whether those interaction terms are useful for a given data set. Rather than use the Conditional Pricing
facility in CVA (which assumes main effects), this example also assumes customization of price ranges
for each brand. For this example, we will assume a conjoint study where two of the attributes are brand
(4 levels) and price (3 levels). (Note that the steps below may be used for any two attributes, whether
involving price or not.)
1. When designing the study, create a single composite attribute reflecting all combinations of
brand and price (4 x 3 = 12 levels). This will increase the number of parameters that need to be
estimated relative to the main effects plan. With main effects, there are (4-1) + (3-1) = 5
parameters to fit to account for the main effects of brand and price. With the composite factor
accounting for all combinations of brand and price, there are 12 - 1 = 11 parameters to fit. Thus,
the interaction design increases the number of parameters to estimate by 11 - 5 = 6. Given the
standard rule-of-thumb for designing CVA questionnaires (field twice as many questions as
CVA Help 831
parameters to estimate), this will increase the number of cards that each respondent should
evaluate by 2 x 6 = 12 cards. Of course, with CVA/HB estimation, you may do just as well with
fewer cards than this recommendation.
2. Add CBC-looking fixed holdout questions at the end of the questionnaire (as Select-type or Grid
questions). Three or four scenarios, each having 4 concepts (one for each brand) should be
sufficient. Don't include a "None" option. These holdouts are useful not only for tuning the CVA
part-worths to have the appropriate scale factor for predicting choice probabilities, but for
investigating whether interaction effects improve the accuracy of the model.
3. Estimate the part-worth utilities (preferably with CVA/HB).
4. Plot the mean part-worths for the brand x price composite factor (pseudo demand curves). If
reversals are present, you may consider re-running the estimation with constraints in place.
Examine the resulting curves for face validity.
5. Specify the holdout scenarios each as a separate simulation scenario within the market
simulator. Tune the Exponent setting (not independently for each simulation scenario, but using
the same scale factor across all simulation scenarios) so that the simulated shares most closely
fit the actual observed choices for the holdouts (Mean Absolute Error, or Mean Squared Error).
6. Repeat the analysis using main effects only. To do this, save a copy of the study within SSI
Web and work from this duplicate study. Export the original design to a .csv file. Modify the
design file to separate the brand x price composite factor into two attributes. Recode the level
numbers accordingly to reflect main effects. Modify the attribute list to separate brand and price as
attributes, to match your modified design. Import the recoded design.
7. Import the respondent data as if the study were conducted via paper-and-pencil.
8. Repeat steps 3 through 5 with the main effects model. Compare the plot of utilities and also the
fit to holdouts. If interaction effects have greater face validity (visual plot) and noticeably stronger fit
to holdouts, then this is evidence that interaction effects are useful. If the two models appear
equally useful, we suggest using the more parsimonious model (main effects).
832 SSI Web v8
6.4.4 Sparse Designs and Large Samples

With typical CVA studies, one assumes OLS estimation at the individual level and therefore must ask
each respondent to complete as many questions as parameters to be estimated, equal to (N-n+1),
where N is the total number of levels in the study and n is the total number of attributes. However, with
HB estimation, it is possible to achieve better results than OLS with the same number of (or even fewer)
questions. Because HB borrows information across the sample, it stabilizes parameters (part-worth
utility estimates) even when the data are very sparse at the individual level (perhaps involving fewer
questions than parameters to estimate).
Sometimes, researchers have the luxury of very large sample sizes (such as 1000+), and may willing to
sacrifice precision at the individual level to dramatically shorten the length of the survey. For example, if
a researcher was comfortable using a standard CVA questionnaire with 18 choice tasks and 500
respondents, it would seem reasonable that the researcher could obtain quite similar overall results by
using 1000 respondents where each respondent answered 9 questions, or 1500 respondents where each
respondent answered 6 tasks. In all three cases, there are 9,000 answers to conjoint tasks, and
sampling error is reduced if increasing the sample sizes. And, HB has proven to do a creditable job with
estimating useful part-worth utilities for conjoint studies with as few as 6 tasks per person.
CVA allows you to randomly skip a subset of the conjoint questions for each respondent. Of course, if
the number of tasks answered by the respondent is fewer than the number of parameters to be
estimated, then OLS estimation cannot be used. We only recommend this approach for situations where
you have very large sample sizes and where market simulation results are the key output, rather than
individual-level classification. If you want to purposefully use sparse designs (within each individual), we
suggest developing the questionnaire using standard rules (generate 2x to 3x as many tasks as
parameters to estimate, and multiple design versions).
Aggregate Estimation via OLS: The simplest method of analysis is to compute the mean ratings for
conjoint questions across the population (or subgroups of the population). Then, use the paper-and-
pencil methodology to import those means from a text (.csv) file into CVA for OLS estimation. Rather
than each record in the .csv file representing a unique person, format just a single record to represent the
total population (or, for subgroup analysis, format a record for each subgroup to be analyzed). The rating
for each card is the mean rating for the population (the paper-and-pencil import supports decimal places
of precision).
HB Estimation: HB estimation can be used even with sparse data. If the data are quite sparse at the
individual level relative to the number of parameters to be estimated, we would recommend decreasing
the Prior Variance assumption (and potentially increasing the strength of the prior assumptions by
increasing the Degrees of Freedom) in the Advanced Settings so that much more "shrinkage" to the
population mean occurs. Otherwise, individual respondents may display wild deviations from reasonable
individual-level utilities. You may also need to significantly increase the number of initial and used
iterations to obtain convergence.
Example:
Assume each version of your questionnaire has 21 CVA questions. Further assume you wanted each
respondent to complete a random 6 out of 21 tasks (you plan to use HB estimation).
First, make sure each of your CVA questions (CVA1_1 to CVA1_21) is located on its own page.
Randomize the pages, by clicking Randomize... | Pages, and by selecting your first and last CVA
questions (CVA1_1 and CVA1_21) as the range.
CVA Help 833
To randomly skip a subset of questions, add a new HTML/Filler question directly beneath CVA1_1 (on
the same page as CVA1_1). Call that new question something like Skip1. Within Skip1, click the Sk ip
Logic tab, add a skip (Post Skip, with a destination question being the next question directly following
the last CVA question), and specify the following skip logic:
if (
SHOWN("CVA1_1")+
SHOWN("CVA1_2")+
SHOWN("CVA1_3")+
SHOWN("CVA1_4")+
SHOWN("CVA1_5")+
SHOWN("CVA1_6")+
SHOWN("CVA1_7")+
SHOWN("CVA1_8")+
SHOWN("CVA1_9")+
SHOWN("CVA1_10")+
SHOWN("CVA1_11")+
SHOWN("CVA1_12")+
SHOWN("CVA1_13")+
SHOWN("CVA1_14")+
SHOWN("CVA1_15")+
SHOWN("CVA1_16")+
SHOWN("CVA1_17")+
SHOWN("CVA1_18")+
SHOWN("CVA1_19")+
SHOWN("CVA1_20")+
SHOWN("CVA1_21") >=6
)
{
return 1;
}
else
{
return 0;
}
End Unverified
Finally, copy the Skip1 question directly beneath CVA1_2, CVA1_3, etc. It will
automatically be renamed Skip2, Skip3, etc. Test your survey to ensure it works
properly.
834 SSI Web v8
6.5 Data Analysis

6.5.1 Estimating Part-Worth Utilities
Once you have downloaded your data (saving the studyname_data.sqlite file within your study folder) or
read your paper-and-pencil data into SSI Web, you are ready to estimate part-worth utilities. CVA
employs different statistical methods for estimating the separate part-worths for the attribute levels in
your study. A unique set of part-worths is estimated for each individual, and the utility run is saved to a
.UTL file for importing into the SMRT Market Simulator.
Ordinary Least Squares and Monotone Regression
When you click the Analysis menu, you can select whether to estimate utilities via Ordinary Least
Squares (OLS) or Monotone Regression . (If your system includes the CVA/HB module, you can
additionally choose HB estimation.)
OLS is the method of calculation traditionally used in most conjoint studies. However, OLS is not
appropriate for conjoint data consisting of rank orders.
For OLS to be appropriate, we must assume the data are "scaled at the interval level." By this, we mean
that the data are scaled so that real differences in the things being measured are communicated by the
arithmetic differences in their values. Fahrenheit temperature, for instance, has an interval scale. The
difference between 70 and 80 degrees is exactly as large as the difference between 80 and 90 degrees.
In the social sciences and in marketing research we are usually willing to assume that rating scale
values possess this kind of scaling.
However, it is usually not reasonable to make such an assumption about rank order data. Suppose a
respondent were to rank 30 concept statements in terms of his likelihood of buying each concept. In the
absence of other information, we would probably expect the concepts to have a normal distribution of
buying likelihood. If so, then we would expect there to be larger "real" differences between concepts with
extreme ranks (such as 1 versus 2, or 29 versus 30) than those in the center of the distribution (such as
ranks 14 and 15).
When the data are rank orders, it is more appropriate to use a method of calculation that does not
assume that the data represent anything more than rank orders. That is the case with nonmetric
methods, and in particular with the monotone regression method provided in CVA.
There is also another reason why we have provided a nonmetric method of calculation within CVA: With
such methods it is easier to constrain calculated utilities to conform to the researcher's expectations.
Conjoint utilities are often observed to violate principles of common sense. For example, in pricing
studies it sometimes turns out that respondents seem eager to pay higher prices rather than lower
prices. This may accurately reflect some respondents' behavior; price is sometimes taken as an
indicator of product quality, and respondents may suspect that a low price reflects poor quality.
However, it is customary to explain to the respondent that "everything else is equal," and that the
attributes are to be considered independently of one another. Under those conditions, if a high price level
receives a higher utility then a low price level, we are likely to conclude that the respondent was simply
confused. Rather than discard data for that respondent, it is often useful to provide additional information
to the calculating program in the form of "constraints." For example, we may tell the calculating program
that the utility for $1.00 must be no lower than the utility for $1.25.
CVA Help 835
With nonmetric methods it is easy to enforce such constraints. Since these methods are iterative, all
that is necessary is to insure that the successive estimates at each stage obey the specified
inequalities.
With OLS it is much more difficult to enforce such constraints. CVA provides that capability in a limited
way: after the least squares solution is computed, then it is adjusted by "tying" values that violate
specified inequalities. However, this is an inelegant way of solving the problem. When the data contain
many violations of common sense relationships, then the nonmetric method provides a better way of
enforcing desired constraints. However, more recent research has shown that HB estimation provides
yet an even better way for estimating utilities that are less prone to reversals and for additionally
enforcing utility constraints.
Additional Information
We have not provided details regarding utility estimation in this section. Please see How CVA
Calculates Utilities for more information about OLS and Monotone Regression. Also see an introductory
paper available at www.sawtoothsoftware.com in the Technical Papers library entitled "Analysis of
Traditional Conjoint Using Excel: An Introductory Example." For information regarding HB estimation,
please see two papers also available in the on-line Technical papers library: "The CVA/HB Technical
Paper" and "Monotonicity Constraints in Choice-Based Conjoint with Hierarchical Bayes."
Settings for Utility Estimation
From Analysis | Calculate CVA Utilities using <OLS/Monotone>... if you click the Settings… button,
you can review or change the settings for the estimation. The controls depend slightly whether you are
using Ordinary Least Squares or Monotone Regression Estimation:
OLS Settings
Monotone Regression Settings
Additional Utility Constraints…: When you click this button, a new dialog opens in which you can
specify Utility Constraints or override previously defined a priori orders for attributes. Utility constraints
are useful if you have attributes that have a specific order such as price. It usually doesn't make sense
that a higher price would be preferred to a lower price. However, utilities (especially when calculated at
the individual level) sometimes include such "reversals." The researcher can smooth some of the noise
out of conjoint data by providing additional information about the order of attribute levels. However,
researchers often find that constraining utilities will slightly hurt share prediction accuracy. The benefit of
adding constraints is that they can improve individual-level classification and prediction.
Important Note: If you specified a priori order when you specified attributes and levels, CVA will
automatically constrain utilities of these ordered attributes to have no reversals. If you do not wish to
apply constraints, you should override these a priori specifications by editing the Constraints.
When you are satisfied with your settings, close the dialog and click the Prepare for CVA OLS
Computation... (or Prepare for CVA Monotone Regression Computation...) button. CVA
automatically calculates part-worths for each respondent and provides a summary report on the screen.
Respondents with no variation in the dependent variable or too many missing answers are not included in
the utility run. The utility run is saved to a .UTL file for use in the SMRT Market Simulator.
Moving the Data into SMRT

836 SSI Web v8
used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. We'll now
describe how you move data from your CVA project within the SSI Web Platform into SMRT.
When you calculated part-worth utilities using Analysis | Calculate CVA Utilities Using Ordinary Least
Squares... (or via Monotone regression or HB) this process creates a text-only format file containing
case IDs and part-worth estimates called STUDYNAME OLS.utl.
Software SMRT.
3. Import the STUDYNAME OLS.utl file (file of respondent part-worth utilities) into your SMRT
following the prompts for importing CVA utilities (selecting CVA (Traditional Conjoint) Utilities
(*.utl) as the file type, and browsing to your .utl file). Once you import your utilities, you can
Click close to return to the main menu. Now that you have imported the utilities, the Mark et Simulator
You will note that there is a list of available Utility Runs (your recently computed run is available), and a
list of Simulation Scenarios (currently empty). Click Compute!, and a report of average utilities and
importances for your data set is displayed in the report window (these data should match the similar
report that was earlier displayed within SSI Web, directly following utility estimation).
CVA Help 837
6.5.2 CVA Ordinary Least Squares Settings

This dialog controls the settings used for estimating part-worths using OLS (Ordinary Least Squares)
Regression. OLS is recommended for CVA questionnaires involving ratings data, where respondents
rated individual conjoint cards or compared two concepts at a time using a sliding rating scale. Please
see the section entitled How CVA Calculates Utilities for more information about OLS Regression.
Minimum Scale Value: If you chose either Logit or Zero-centered recoding methods, you must specify
the minimum scale value. If you used a 0 to 100 purchase likelihood scale, the minimum is 0. If you
used a 1 to 9 scale with a pairwise questionnaire and used 0 as the missing value, the Minimum Scale
Value is 1.
Maximum Scale Value: If you chose either Logit or Zero-centered recoding, you must specify the
maximum value.
Scale Direction: We assume that most users field ratings-based CVA questionnaires. For single-
concept designs, we have made "Highest Number Best" the default, meaning that higher numbers
indicate higher preference. However, for ranking tasks, lower numbers frequently imply higher utility. In
that case, "Lowest Number Best" should be chosen. For pairwise comparison designs, "Highest
Number on Right" should be chosen if the values run from lower numbers on the left to higher numbers
on the right. If you specify this field incorrectly, the part-worths will appear opposite to how you expect
them (i.e. higher prices will be preferred, etc.).
Recode Method:
Sometimes it is desirable or even necessary to recode the CVA responses. CVA provides three options:
None, Logit (default) and Zero-centered. Each of these is described below. If you only plan to use the
First Choice model in the CVA market simulator, whether you center or not does not impact your results.
If you plan to use the Purchase Likelihood model, you must use single concept presentation and specify
a Logit recoding to ensure meaningful results. If you plan to use the Share of Preference option or
Randomized First Choice, it is probably helpful to specify a zero-centered or a logit rescaling. Scaling of
part-worths can have a significant impact upon Share of Preference results. The scaling can be adjusted
later in the Market Simulator using the Exponent setting.
None. This setting uses the responses in the data file as-is. If you only plan to use the First Choice
model in the CVA market simulator, you may choose this option.
Logit. It is common in "single concept at a time" questionnaires for respondents to rate concepts on
"Level of interest" or "likelihood of buying" scales. The scale could be three digit (from 0 to 100) or single
digit (from 1 to 5). In those cases it is standard procedure to calculate part-worths using the actual
ratings as dependent variables.
Consider a 0 to 10 purchase likelihood scale. If we calculate part-worths using the original scale values,
we will be assuming that the difference between ratings of say, 5 and 6, is as meaningful as the
difference between ratings of 9 and 10. Yet it would seem reasonable to believe that the difference
between 9 and 10 is a larger subjective difference for the respondent than the difference between 5 and 6.
When data are probabilities, this problem is often handled by transforming the data to a new scale that
shrinks the differences in the center of the scale and stretches differences at the extremes of the scale.
This is called the "logit" transformation, given by the formula:
Recoded value = ln[p/(1-p)]
where p is equal to the probability of purchase.

838 SSI Web v8
When using the Purchase Likelihood model, the Market Simulator requires that utilities be scaled so that
antilogs of their sums are equal to expected likelihoods of purchase. Thus, the data should be recoded
into logits. Here is an example of a logit transformation for a 0 to 10 scale:
Scale Interpreted Logit

Value Probability Value
0 .0 -3.000
1 .1 -2.197
2 .2 -1.386
3 .3 -0.847
4 .4 -0.405
5 .5 0.000
6 .6 0.405
7 .7 0.847
8 .8 1.386
9 .9 2.197
10 1.0 3.000
Notice that in the middle of the scale the difference between adjacent logit values is only 0.405, but the
difference between logit values for ratings of 8 and 9 is 0.811. Logit values are undefined for probabilities
of 0 and 1, but approach plus and minus infinity. To keep things under control, we arbitrarily limit the
logit values to the range of plus or minus 3.
The example above shows a simple way for converting numeric responses on a 1 to 9 scale first to
probabilities, and then to logits. The method CVA actually employs for the logit recode is only slightly
different. Because the logit transform is very sensitive at high and low probabilities, we first map numeric
responses to probabilities using the following formula:
p = (y-min+1)/(max-min+2)
where: p=probability
y=answer
min=minimum scale value
max=maximum scale value
After calculating the probability p using the above formula, we again use the formula:
Recoded value = ln[p/(1-p)]
Researchers who wish to use other recoding schemes can do so by using the paper-and-pencil method
of importing respondent answers and directly modifying the values in the respondent answer file.
Logit recodes can also be useful in "pairwise" questionnaires, where the rating scale may run from 1 to 9
with 5 meaning "no preference."
Zero-centered. Many researchers find that zero-centering the response scale yields more interpretable
utilities than when the response scale features only positive values. Whether you adopt this option or
use no recoding method has no impact in the reported average utilities from the simulator or the
competitive simulation models. But, if you plan to export and use the raw utilities in another way, you
may want the resulting utilities to be based on a zero-centered response scale
We suggest zero-centered or logit scaling for paired-comparison questionnaires. If we asked paired-

CVA Help 839
comparison questions using CVA's default 1 to 9 point scale, the recode values would be: -4 -3 -2 -1
0 1 2 3 4.
Notice that these recoded values fall roughly in the same zero-centered range as the logit transformation
for purchase likelihood models, but that the zero-centered rescaling assumes equal increments instead
of "stretching" the scale at the ends.
840 SSI Web v8
Tasks to include:
This lets you specify which tasks (conjoint questions) should be included when estimating part-worths.
By default, all tasks are included. Generally, you should include all conjoint tasks in estimation.
However, if you have placed holdout tasks within the survey (by exporting the design, modifying it to
include additional holdout tasks, and re-importing the design), you may decide to omit those holdouts
from the part-worth estimation. Please see the section entitled Importing Designs into CVA for more
information about including holdout cards in your CVA studies.
Constraints:
The Edit... button lets you specify that certain levels within certain attributes have known a
priori preference order and that the part-worths should be constrained accordingly. All rank-
order relationships you originally specified when entering your list of attributes and levels are
carried forward to the Additional Utility Constraints dialog. But, you can modify those
selections.
CVA Help 841
6.5.3 CVA Monotone Regression Settings

This dialog controls the settings used for estimating part-worths using Monotone Regression. Monotone
Regression is recommended for card-sort designs involving rank-order data, where respondents sorted
individual conjoint cards from best to worst. Please see the section entitled How CVA Calculates
Utilities for more information about Monotone Regression.
Minimum Scale Value:
Specify the lowest value used in the rank-order scale. This is usually 1 when using card-sort
data. If using pairwise comparison presentation, it is the lowest value used in the response
scale.
Maximum Scale Value:
Specify the highest value used in the rank-order scale. This is usually equal to the number of
cards (conjoint questions) in your study. For example, if you have 18 cards in your study,
then specify 18. If using pairwise comparison presentation, it is the highest value used in the
response scale.
Scale Direction:
Single-Concept (Card-Sort) Designs: If using card-sort data, there are two choices: "Lowest
Number Best" and "Highest Number Best." With rank-order data, if a "1" in your data set
indicates the best card, you should choose "Lowest Number Best." If you used CVA to import
rank-order data from a text file and specified that the values were card numbers, CVA
internally coded the values so that a "1" is associated with the best card, a "2" with the next-
best card, etc. If higher numbers indicate greater preference, choose "Highest Number Best."
Pairwise Comparison Designs: It is unusual to use Monotone Regression in pairwise

comparison CVA designs, but if you do wish to try it, you should specify the correct scale
direction. The selections are: "Highest Number on Right" and "Lowest Number on Right."
Monotone regression only retains the ordinal information about which concepts are preferred to
others. The magnitudes of the differences are ignored. For this reason, part-worth estimation
from pairwise rating questionnaires using monotone regression may result in noisier estimates
than when preserving the additional information of the ratings scale under OLS.
Task s to Include:
This lets you specify which tasks (conjoint questions) should be included when estimating
part-worths. By default, all tasks are included. Generally, you should include all conjoint
tasks in estimation. However, if you have placed holdout tasks within the survey (by exporting
the design, modifying it to include additional holdout tasks, and re-importing the design), you
may decide to omit those holdouts from the part-worth estimation. Please see the section
entitled Importing Designs into CVA for more information about including holdout cards in your
CVA studies.
Constraints:
The Edit... button lets you specify that certain levels within certain attributes have known a
priori preference order and that the part-worths should be constrained accordingly. All rank-
order relationships you originally specified when entering your list of attributes and levels are
carried forward to the Additional Utility Constraints dialog. But, you can modify those
842 SSI Web v8
selections.
CVA Help 843
6.6 Technical Details

6.6.1 The CVA Designer
The CVA designer uses a computer search technique to find D-efficient designs, given a certain conjoint
design, any prohibitions (for level by level combinations or due to utility balance constraints), and a
requested number of conjoint profiles.
D-efficiency is described in the article by Kuhfeld, Tobias, and Garratt (1994), "Efficient Experimental
Design with Marketing Research Applications," Journal of Mark eting Research, 31 (November), 545-557.
Paraphrasing the article, D-efficiency is a function of the determinant of the X'X matrix, given by the
formula:
where ND = number of tasks

p = number of attributes
X = the design matrix using orthogonal coding
If a design is orthogonal and balanced (each level within an attribute shown an equal number of times),
then it has optimum efficiency.
The D-efficiency measures the goodness of the design relative to the hypothetical orthogonal design. A
perfect design will be both orthogonal and balanced and will result in an efficiency of 1.0. However, an
orthogonal design is not always possible, given the number of attribute levels and requested number of
tasks. A final efficiency of less than 1.0 may still indicate a satisfactory design.
User-Specified Designer Controls
CVA generates an initial pool of conjoint profiles equal to ten times the requested number. It repeats the
search process within that candidate pool equal to ten times as many questionnaire versions that are
requested. Some users may wish to over-ride these limits to either speed up the process, or to let CVA
spend an even greater amount of time searching for the most efficient design. You can override the
defaults by clicking the Show Advanced Settings button and modifying the fields that appear. For
larger CVA designs, a task pool multiplier of 100 or more often will lead to finding more efficient designs
faster.
844 SSI Web v8
6.6.2 How CVA Calculates Utilities

Ordinary Least Squares (OLS)
This section assumes the reader has basic knowledge of linear regression. However, understanding of
the technical details in this section is not essential for using CVA.
The OLS calculator does ordinary least squares linear regression using a "dummy variable" approach. A
vector of independent variables for each observation (conjoint question) is built using information from the
experimental design. The vector has elements of 1, 0, or -1, depending on whether respective attribute
levels appear in that question, and whether they appear on the left-hand (-1) or right-hand side (+1) of the
pairwise questions.
The dependent variable for each observation is obtained by applying the indicated recoding transformation
to the corresponding data value. If the data value has the code reserved for missing data, then that
observation (conjoint question) is not included in the calculation.
When using regression to estimate conjoint utilities, it is customary to delete one level of each attribute
from the computation. Otherwise, there is a linear dependence among the variables describing levels of
each attribute, which leads to indeterminacy in the computation. Omitting one level of each attribute
from the computation is equivalent to setting its utility at zero, with the other levels measured as
contrasts with respect to zero. The OLS calculator omits the first level of each attribute from the
regression computation. Thus, if there are k attributes with a total of n levels, the regression is done with
only n-k independent variables. The indeterminacy could also be handled by adding side conditions,
such as requiring that the utilities for each attribute sum to some constant. However, our approach has
the advantage of greater computational speed.
An intercept term is also computed by CVA, but it is not reported separately. Since the utilities will
most often be used by adding up sums consisting of one value from each attribute, the intercept has
been divided by the number of attributes and that fraction has been added to every utility value. Thus the
first level for each attribute, which would otherwise be zero, will be equal to the intercept divided by the
number of attributes.
The "r squared" value for each respondent does not contain any correction for degrees of freedom. If the
number of observations is equal to the number of parameters being estimated (levels - attributes +1),
then the r squared value will be unity.
If the design is deficient -- containing either too few observations to permit estimation or insufficient
information for a particular attribute level -- then a message to that effect will appear on the screen and
utilities will not be estimated for that respondent.
If there are degrees of freedom available for error, then descriptive data will be written to a log file with
information about the precision of estimation.
A statistic ("rms cor") is provided for each respondent, which describes the amount of correlation among
the independent variables. It is the "root mean square" of off-diagonal elements of the correlation matrix
for the n-k independent variables. Subtle relationships among variables can easily lead to faulty designs
that would not be detected, and therefore we caution against paying much attention to this statistic. We
include it only because our users may be accustomed to similar statistics in other software packages.
In an orthogonal design with no missing data, this value will be either zero or a small positive number.
(Orthogonal designs have correlations within attributes.) Orthogonal designs are sometimes altered to
eliminate "nonsense" questions, and this compromises orthogonality. Also, some design procedures
CVA Help 845
(CVA's questionnaire design module, for example) produce well-balanced but not perfectly orthogonal
designs.
For each individual, standard errors of utility estimates are provided (within a log file), except for the first
level of each attribute, which is assumed to have utility of zero. These standard errors may also be
regarded as standard errors of differences between each level and the first level of that same attribute.
These standard errors can be of diagnostic value. Attribute levels with large standard errors should be
given more attention in questionnaire design. They may appear in too few questions, or they may occur
in patterns that compromise the level of independence necessary for good estimation.
For more information on utility estimation, see Avoiding Linear Dependency .
Monotone (Nonmetric) Regression
This option for calculating utilities uses a method similar to that described by Richard M. Johnson in "A
Simple Method of Pairwise Monotone Regression", Psychometrika, 1975, pp 163-168.
The method is iterative, finding successive solutions for utility values that fit the data increasingly well.
An initial solution is developed, either randomly or using information in the experimental design. Two
measures of goodness of fit are reported: theta and tau.
Tau
Suppose the conjoint questionnaire presented concepts one at a time and asked for a rank order of
preference. Although there would have been many concepts in the questionnaire, consider just four of
them, concepts P, Q, R, and S. Suppose the respondent ranked these concepts 7, 9, 13, and 17,
respectively, and at some intermediate stage in the computation, utilities for these concepts are
estimated as follows:
Estimated Preference
Concept Utility Rank
P 4.5 7
Q 5.6 9
R 1.2 13
S -2.3 17
We want to measure "how close" the utilities are to the rank orders of preference.
One way we could measure would be to consider all of the possible pairs of concepts, and to ask for
each pair whether the member with the more favorable rank also has the higher utility. Since these are
rank orders of preference, smaller ranks indicate preference, so we know that:
Preference Utility Squared

Difference Difference
P is preferred to Q -1.1 1.21
P is preferred to R 3.3 10.89
P is preferred to S 6.8 46.24
Q is preferred to R 4.4 19.36
Q is preferred to S 7.9 62.41
R is preferred to S 3.5 12.25
--- -----
Total 152.36
846 SSI Web v8
Of the six pairs, five have utility differences with the correct signs (the preferred product has the higher
utility), and one pair has a utility difference with the wrong sign.
Kendall's tau is a way of expressing the amount of agreement between the preferences and the
estimated utilities. It is obtained by subtracting the number of "wrong" pairs from the number of "right"
pairs, and then dividing this difference by the total number of pairs. In this case,
tau = (5 - 1) / 6 = .667
A tau value of 1.000 would indicate perfect agreement in a rank order sense. A tau of 0 would indicate
complete lack of correspondence, and a tau of -1.000 would indicate a perfect reverse relationship.
Tau is a convenient way to express the amount of agreement between a set of rank orders and other
numbers, such as utilities for concepts. However, it is not very useful as a measure on which to base an
optimization algorithm. As a solution is modified to fit increasingly well, its tau value will remain constant
and then suddenly jump to a higher value. Some other measure is required that is a continuous function
of the utility values.
Theta
For this purpose we use the statistic "theta." Theta is obtained from the squared utility differences in the
last column of the table above. We sum the squares of those utility differences that are in the "wrong
order," divide by the total of all the squared utility differences, and then take the square root of the
quotient. Since there is only one difference in the wrong direction,
theta = square root(1.21/152.36) = .089
Theta can be regarded as the percentage of information in the utility differences that is incorrect, given
the data. The best possible value of theta is zero, and the worst possible value is 1.000.
Now that we have defined theta, we can describe the nature of the computation.
The process is iterative. It starts with random values as estimates of the partworths. In each iteration a
direction of change (a gradient vector) is found which is most likely to yield an improvement in the
partworths. A number of small changes are made in that direction, which continue as long as theta
improves. Each iteration has these steps:
1. Obtain the value of theta for the current estimates of partworths and a direction (gradient) in
which the solution should be modified to decrease theta most rapidly.
2. Try a small change of the partworths in the indicated direction, which is done by subtracting
the gradient vector from the partworth vector and renormalizing the partworth estimates so as
to have a sum of zero within each attribute and a total sum of squares equal to unity. Each
successive estimate of utilities is constrained as indicated by the a priori settings or additional
utility constraints.
3. Re-evaluate theta. If theta is smaller than before, the step was successful, so we accept
the improved estimates and try to obtain further improvement using the same procedure again,
by returning to step (2). If theta is larger than before, we have gone too far, so we revert to the
previous estimate of partworths and begin a new iteration by returning to step (1).
If any iteration fails to improve theta from the previous iteration, or if theta becomes as small as 1e-10,
CVA Help 847
the algorithm terminates. A maximum of 50 iterations are permitted, and within any iteration a
maximum of 50 attempts at improvement are permitted. In theory, the iterations could continue almost
indefinitely with a long series of very small improvements in theta. For this reason it is useful to place a
limit on the number of iterations.
To avoid the possibility of stumbling into a bad solution due to a poor starting point, the process is
repeated 5 separate times from different starting points. For each respondent, the weighted average of
the five resulting vectors of part-worths is computed (weighted by Tau, where any negative Tau is set to
an arbitrarily small positive number). A weighted Tau is also reported with this final estimate of part-
worth utilities.
How CVA Utilities Are Scaled
Monotone regression
CVA's monotone regression utility calculator scales utilities in a way that is easy to describe and to
understand. For each respondent, the values for each attribute have a mean of zero, and their sum of
squares across all attributes is unity. Here is an example, assuming two attributes, one with 3 levels
and one with 2 levels:
utility square
Attribute One Level 1 .50 .25

Level 2 .10 .01
Level 3 -.60 .36
--- ---
0.00
Attribute Two Level 1 .44 .19

Level 2 -.44 .19
---- -----
0.00 1.00
OLS regression
CVA's OLS utility calculator scales utilities in a way that depends upon the data, and upon the
researcher's use of the recode capabilities. The calculation has these steps:
1. If automatic recoding was specified, then the data are automatically recoded. If no recode
was specified, the values in the data file are used without modification.
2. An array of "dummy" variables is constructed with a row for each conjoint question and a
column for each attribute level. Each cell of this array has a value of 1, 0, or -1, depending on
the experimental design. For single-concept presentation, the values are either 1 or 0. If the
level appears in the concept it is coded as 1, and if absent it is coded as 0. For pairwise
presentation, if a level appears in the left-hand concept it is coded as -1, or 1 if in the right-
hand concept. If an attribute level does not appear in either profile, then the corresponding
array element is 0.
3. The first level (column) for each attribute is omitted temporarily from the design, which
avoids technical problems of indeterminacy in the solution. (See Avoiding Linear Dependency)
848 SSI Web v8
4. OLS regression is used to predict the transformed data values from the surviving columns
of the array (variables). A regression coefficient is computed for each variable, as well as a
single intercept. The regression coefficients for the omitted variables are assumed to be zero.
5. The intercept is divided by the number of attributes, and the quotient is added to every
regression coefficient, including those previously assumed to be zero. The resulting values are
reported as utilities for the attribute levels. (The intercept is handled in this way to make it
easy to calculate total utilities for products during simulations. Since each product to be
simulated will have exactly one level from each attribute, the simulator will be able to include
the intercept automatically just by adding the utilities of its attribute levels.)
As can be seen from this explanation, with the OLS calculator the scaling of the utilities is completely
under the control of the researcher. Other things being equal, if the data are collected on a 100-point
scale, the utilities will be about ten times the magnitude as they would be if the data were collected on a
10-point scale.
CVA Help 849
6.6.3 Avoiding Linear Dependency

CVA's OLS regression utility calculator deletes one level of each attribute from the utility computation.
That is done to avoid linear dependence among attribute levels, which would result in indeterminacy in
the solution. The purpose of this section is to explain why that precaution is required, when it is
appropriate, and when it is useful but not essential.
We consider two popular types of conjoint designs:
Full-profile single concept

Paired comparison
In each case we will consider a design matrix with a row for each question and a column for each
attribute level. Suppose that the total number of attributes is n and the total number of levels (columns)
is N.
Full-profile Single Concept
In these designs, each question presents a single concept. Each concept has a specified level of each
attribute. Each row of the design matrix has 1's in the columns corresponding to the attribute levels in
that concept, and other elements of 0 (zero). Since each row contains exactly n 1's and N -1 zeroes,
the sum of each row is n.
CVA computes an intercept term, which is equivalent to adding an imaginary N + first column to the
design matrix in which every element is a 1. That column would be linearly dependent on the sum of the
N existing columns, which would cause the solution to be indeterminate.
However, the problem is greater than that. Consider just the subset of columns corresponding to levels
of one attribute: Every row will have a single 1 and other elements of 0. Therefore, within that attribute
the sum of every row is equal to 1. This problem generalized to every attribute. The most popular
solution is to delete one level from each attribute, which is equivalent to assuming that that level has a
utility of 0. If this is done, then the rows no longer have equal sums and the indeterminacy is resolved.
CVA arbitrarily deletes the first level of each attribute.
Paired Comparisons
In these designs each question presents two concepts and the respondent is asked to express a
preference. Suppose one concept is presented on the "left" and one on the "right" and that the response
scale is arranged so that larger responses indicate greater preference for the concept on the right. Also,
we assume that if a particular attribute appears in one of the concepts, it will also appear (probably with
a different level) in the other concept.
The design matrix has a row for each concept and N columns. Each element is 1 if that attribute level
appears in the concept on the right, -1 if it appears in the concept on the left, or 0 if that attribute level
does not appear in either concept. (If an attribute level should happen to appear in both concepts it
should not have an effect on respondent preference, and is given a value of zero.)
Consider just those columns corresponding to levels of one attribute. If an attribute appears in either
concept, it will also be represented in the other concept. Therefore, each value of 1 has a corresponding
value of -1, and the sum of values will be 0 for every row within the columns corresponding to each
attribute. Thus paired comparison designs (even those that present concepts specified on only subsets
850 SSI Web v8
of attributes) are subject to the same indeterminacy as full-profile single concept designs. CVA handles
the problem of indeterminacy in exactly the same way as with full-profile single concept designs, by
deleting the first level of each attribute and assuming its utility to be 0.
CVA Help 851
6.7 The CVA/HB Module for HB Estimation

6.7.1 What is CVA/HB (hierarchical Bayes for CVA)?
Introduction
The CVA/HB Module is an add-on component within SSI Web that uses hierarchical Bayes (HB) to
estimate part-worths for ratings-based full-profile conjoint analysis (CVA) studies. HB considers each
individual to be a sample from a population of similar individuals, and "borrows" information from the
population in estimating part-worths for each respondent. With HB, CVA users can often achieve
equivalent results (relative to OLS) using fewer tasks per person and/or fewer total respondents.
Precisely how much improvement HB estimation offers over the standard estimation techniques depends
on the project.
If using the CVA/HB module, it is possible to obtain good part-worth utility estimates even though
respondents have evaluated a random subset of the questions within a questionnaire version (even fewer
questions than parameters to be estimated). This approach can significantly reduce the number of
questions each respondent is asked. However, we recommend you not do this unless sample sizes are
quite robust.
HB estimation takes considerably longer than OLS or Monotone regression. Computation time will
usually vary from about 3 to 20 minutes for most CVA data sets.
Although the CVA/HB module will produce results for very small sample sizes, we caution against using
it in those instances. CVA/HB will produce results even with data from a single respondent, but with very
small sample sizes it will have difficulty distinguishing between heterogeneity and error. How many
respondents are required for robust HB estimation depends on the study design and the nature of the
sample. We have seen HB perform well with samples as small as 80 respondents for ratings-based
conjoint studies. HB may perform consistently well with even smaller sample sizes, though we know of
no series of studies to substantiate that claim.
In any case, we suggest not using HB blindly. It is prudent to design holdout choices within your CVA
questionnaires so that you can assess the performance of alternative part-worth estimation methods.
Background
The first traditional conjoint analysis applications in the early- to mid-1970s used non-metric estimation
or OLS to derive part-worths. These techniques served the industry well over the first few decades of
conjoint analysis practice. Even so, conjoint researchers have always faced a degrees of freedom
problem. We usually find ourselves estimating many parameters (part-worths) at the individual level from
relatively few observations (conjoint questions). It is often challenging to get respondents to complete
many conjoint tasks, so researchers may sacrifice precision in the part-worth estimates by reducing the
number of conjoint profiles. It is precisely in those cases that HB can be most useful.
HB became available in about the mid 1990s to marketing researchers. HB significantly improves part-
worth estimates and produces robust results when there are very few or even no degrees of freedom.
Several articles (see for example Lenk, et al. 1996 and Allenby, et al.1998) have shown that hierarchical
Bayes can do a creditable job of estimating individual parameters even when there are more parameters
than observations per individual.
It is possible using CVA/HB to estimate useful part-worths for an individual even though that respondent
has answered fewer tasks than parameters to estimate. This can occur if respondents quit the survey
852 SSI Web v8
early. However, the researcher may choose this approach by design. The researcher might randomly
assign respondents a subset of a larger CVA design, so that across all respondents each task has
roughly equal representation.
The CVA/HB Module estimates a hierarchical random coefficients model using a Monte Carlo Markov
Chain algorithm. In the material that follows we describe the hierarchical model and the Bayesian
estimation process. It is not necessary to understand the statistics of HB estimation to use this module
effectively. The defaults we have provided make it simple for researchers who may not understand the
statistics behind HB to run the module with consistently good results.
We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have
been helped by several sources listed in the References. We have benefited particularly from the
materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing
Association's Advanced Research Techniques Forum.
The Basic Idea behind HB
CVA/HB uses Bayes methods to estimate the parameters of a randomized coefficients regression
model. In this section we provide a non-technical description of the underlying model and the algorithm
used for estimation.
The model underlying CVA/HB is called "hierarchical" because it has two levels. At the upper level,
respondents are considered as members of a population of similar individuals. Their part-worths are
assumed to have a multivariate normal distribution described by a vector of means and a matrix of
variances and covariances.
At the lower level, each individual's part-worths are assumed to be related to his ratings of the overall
product profiles within the conjoint survey by a linear regression model. That is to say, when deciding on
his preference for a product profile, he is assumed to consider the various attribute levels that compose
that product, and add the value of each level to come up with an overall rating for the product concept.
Discrepancies between actual and predicted ratings are assumed to be distributed normally and
independently of one another.
Suppose there are N individuals, each of whom has rated conjoint profiles on n attribute levels. If we
were to do ordinary regression analysis separately for each respondent, we would be estimating N*n
part-worths. With the hierarchical model we estimate those same N*n part-worths, and we further
estimate n mean part-worths for the population as well as an n x n matrix of variances and covariances
for the distribution of individuals' part-worths. Because the hierarchical model requires that we estimate
a larger number of parameters, one might expect it would work less well than ordinary regression
analysis. However, because each individual is assumed to be drawn from a population of similar
individuals, information can be "borrowed" from other individuals in estimating parameters for each one,
with the result that estimation is usually enhanced.
For more information, please read the CVA/HB Technical Paper, available for download from our
technical papers library at www.sawtoothsoftware.com.
CVA Help 853
6.7.2 Running CVA/HB

Once you have collected conjoint data for a CVA project, you are ready to estimate part-worths with HB.
Click Analysis | Estimate CVA Utilities - HB.... Click the Settings... button to review or change the
default settings:
Estimation Parameters:
Number of Iterations before Using Results: This describes how many "burn-in" iterations will be
used, prior to assuming convergence. 10,000 iterations usually provides enough initial iterations so
that the part-worth utilities "converge." Convergence is assessed typically by viewing the visual
display of the history of part-worth estimates. If the lines in the display tend to be wobbling
horizontally, with no noticeable trend, then this is evidence of convergence. If the parameters do
not seem to converge, then you can increase the number of initial iterations.
Number of Draws to be Used for Each Respondent: This describes how many "draws" or
candidate betas will be used (averaged to form a point estimate) for each respondent. 10,000
draws tends to give quite good precision for each respondent. However, you can obtain slightly
better precision by increasing the number of used draws.
854 SSI Web v8
Scale:
The Scale Settings (Minimum Scale, Scale Direction, and Recode Method) are similar to those
described for OLS estimation
Advanced Estimation Parameters:
Prior Degrees of Freedom: This value is the additional degrees of freedom for the prior covariance
matrix (not including the # parameters to be estimated). The higher the value, the greater the
influence of the prior variance and more data are needed to change that prior. The scaling for
degrees of freedom is relative to the sample size. If you use 50 and you only have 100 subjects,
then the prior will have a big impact on the results. If you have 1000 subjects, you will get about the
same result if you use a prior of 5 or 50. As an example of an extreme case, with 100 respondents
and a prior variance of 0.1 with prior degrees of freedom set to the number of parameters estimated
plus 50, each respondent's resulting part-worths will vary relatively little from the population means.
We urge users to be careful when setting the prior degrees of freedom, as large values (relative to
sample size) can make the prior exert considerable influence on the results.
Prior Variance: The default is 2 for the prior variance for each parameter, but users can modify this
value. Increasing the prior variance tends to place more weight on fitting each individual's data, and
places less emphasis on "borrowing" information from the population parameters. The resulting
posterior estimates are relatively insensitive to the prior variance, except 1) when there is very little
information available within the unit of analysis relative to the number of estimated parameters, and
2) the prior degrees of freedom for the covariance matrix (described above) is relatively large.
Random Starting Seed: HB requires a random number generator, and therefore a starting seed is
required. By default, the random seed is 1, but users can specify a specific seed to use (integers
from 1 to 32000), so that results are repeatable. When using different random seeds, the posterior
estimates will vary, but insignificantly, assuming convergence has been reached and many draws
have been used. If you use a seed of "0", a seed is drawn based on the computer's clock, so a
different seed will be chosen if you repeat the analysis.
Results
After the computation finishes, the new HB run is automatically saved to a STUDYNAME.hbu file within
your study director. That file may be imported within SMRT for use in the market simulator.
Software SMRT.
3. Import the STUDYNAME.hbu file (file of respondent part-worth utilities) into your SMRT study
by clicking (from the SMRT software menu) Analysis | Run Manager | Import… and following
the prompts for importing CVA utilities (selecting HB Utilities (*.hbu, *.hbc) as the file type,
CVA Help 855
and browsing to your .hbu file). Once you import your utilities, you can select Analysis |
Market Simulator and begin simulations.
Click close to return to the main menu. Now that you have computed utilities, the Mark et Simulator
856 SSI Web v8
6.7.3 Using Constraints in CVA/HB

Conjoint studies frequently include product attributes for which almost everyone would be expected to
prefer one level to another. However, estimated part-worths sometimes turn out not to have those
expected orders. This can be a problem, since part-worths with the wrong relationships (especially if
observed at the summarized group level) are likely to yield nonsense results and can undermine users'
confidence.
CVA/HB provides the capability of enforcing constraints on orders of part-worths within attributes. The
same constraints are applied for all respondents, so constraints should only be used for attributes that
have unambiguous a priori preference orders, such as quality, speed, price, etc.
Evidence to date suggests that constraints can be useful when the researcher is primarily interested in
individual-level classification or the prediction of individual choices, as measured by hit rates for holdout
choice tasks. However, constraints appear to be less useful, and indeed can be harmful, if the
researcher is primarily interested in making aggregate predictions, such as predictions of
shares of preference. Most research is principally concerned with the latter. Another concern is that
constraints can bias the apparent importances of constrained attributes in market simulations, relative to
unconstrained attributes.
If you check the Use Constraints box on the CVA/HB Settings dialog, CVA/HB automatically constrains
the part-worths if you have specified a priori orders (e.g. Best to Worst or Worst to Best) when you
specified your attributes, or if you specify additional utility constraints by clicking Edit... and supplying
additional constraints using the Custom Constraints dialog.
CVA/HB employs a technique called Simultaneous Tying. In a paper available on the Sawtooth Software
Web site (Johnson, 2000), the author explored different ways of enforcing constraints in the HB context.
He found the method of simultaneous tying to perform best among the techniques investigated.
Simultaneous tying features a change of variables between the "upper" and "lower" parts of the HB
model. For the upper model, we assume that each individual has a vector of (unconstrained) part-worths,
with distribution:
ßi ~ Normal(α, D)
where:
ßi = unconstrained part-worths for the ith individual

α = means of the distribution of unconstrained part-worths
D = variances and covariances of the distribution of unconstrained part-worths
For the lower model, we assume each individual has a set of constrained part-worths, bi where bi is
obtained by recursively tying each pair of elements of ßi that violate the specified order constraints.
With this model, we consider two sets of part-worths for each respondent: unconstrained and
constrained. The unconstrained part-worths are assumed to be distributed normally in the population,
and are used in the upper model. However, the constrained part-worths are used in the lower model to
evaluate likelihoods.
We speak of "recursively tying" because, if there are several levels within an attribute, tying two values to
satisfy one constraint may lead to the violation of another. The algorithm cycles through the constraints
repeatedly until they are all satisfied.
CVA Help 857
When constraints are in force, the estimates of population means and covariances are based on the
unconstrained part-worths. However, since the constrained part-worths are of primary interest, we plot
the constrained part-worths to the screen. Only the constrained part-worths are saved to the utility run
for use in the market simulator.
When constraints are in place, measures of fit (average r-squared) are decreased. Constraints always
decrease the goodness-of-fit for the sample in which estimation is done. This is accepted in the hope
that the constrained solution will work better for predictions in new choice situations. Measures of scale
(Avg. Variance and Parameter RMS), which are based on unconstrained part-worths, will be increased.
858 SSI Web v8
7 Appendices for SSI Web

7.1 A: Interpreting Conjoint Analysis Data
Conjoint analysis provides a number of outputs for analysis including: part-worth utilities (or counts),
importances, shares of preference and purchase likelihood simulations. This appendix discusses these
measures and gives guidelines for interpreting results and presenting findings to management. Before
focusing on conjoint data, we'll review some fundamentals for interpreting quantitative data. The
definitions below are adapted from Statistics for Modern Business Decisions, Fourth Edition, by
Lawrence L. Lapin.
The Nature of Quantitative Data:
There are four general types of quantitative data:
1) Nominal data are those wherein the numbers represent categories, such as 1=Male,
2=Female; or 20=Italy, 21=Canada, 22=Mexico. It is not appropriate to perform mathematical
operations such as addition or subtraction with nominal data, or to interpret the relative size of the
numbers.
2) Ordinal data commonly occur in market research in the form of rankings. If a respondent ranks
five brands from best "1" to worst "5," we know that a 1 is preferred to a 2. An example of an
ordinal scale is the classification of the strength of tornados. A category 3 tornado is stronger and
more damaging than a category 2 tornado. It is generally not appropriate to apply arithmetic
operations to ordinal data. The difference in strength between a category 1 and 2 tornado is not
necessarily equal to the difference in strength between a category 2 and a 3. Nor can we say that a
category 2 is twice as strong as a category 1 tornado.
3) Interval data permit the simple operations of addition and subtraction. The rating scales so
common to market research provide interval data. The Celsius scale also is interval scaled. Each
degree of temperature represents an equal heat increment. It takes the same amount of heat to
raise the temperature of a cup of water from 10 to 20 degrees as from 20 to 30 degrees. The zero
point is arbitrarily tied to the freezing point of distilled water. Sixty degrees is not twice as hot as
30 degrees, and the ratio 60/30 has no meaning.
4) Ratio data permit all basic arithmetic operations, including division and multiplication.
Examples of ratio data include weight, height, time increments, revenue and profit. The zero point
is meaningful in ratio scales. The difference between 20 and 30 kilograms is the same as the
difference between 30 and 40 kilograms, and 40 kilograms is twice as heavy as 20 kilograms.
Conjoint Utilities (Part-Worths):
Conjoint part-worths are scaled to an arbitrary additive constant within each attribute and are interval
data. The arbitrary origin on the scaling within each attribute results from dummy coding in the design
matrix. When using a specific kind of dummy coding called
"effects coding," utilities are scaled to sum to 0 within each attribute. A plausible set of part-worth
utilities for miles per gallon might look like:
30 MPG -1.0
Appendices for SSI Web 859
40 MPG 0.0
50 MPG 1.0
Just because 30 MPG received a negative utility value does not mean that this level was unattractive. In
fact, 30 MPG may have been very acceptable to all respondents. But (all else being equal) 40 MPG and
50 MPG are better. The utilities are scaled to sum to 0
within each attribute, so 30 MPG must receive a negative utility value. Other kinds of dummy coding
arbitrarily set the part-worth of one level within each attribute to zero and estimate the remaining levels as
contrasts with respect to zero. Again, the same cautions regarding interpretation apply. Whether we
multiply all the part-worth utilities by a positive constant or add a constant to each level within a study,
the interpretation is the same. Suppose we have two attributes with the following utilities:
Blue 30 Brand A 20
Red 20 Brand B 40
Green 10 Brand C 10
The increase in preference from Green to Blue (20 points) is equal to the increase in preference between
Brand A and Brand B (also 20 points). However, due to the arbitrary origin within each attribute, we
cannot directly compare values between attributes to say that Red (20 utiles) is preferred equally to
Brand A (20 utiles). And even though we are comparing utilities within the same attribute, we cannot say
that Blue is three times as preferred as Green (30/10). Interval data do not support ratio operations.
Counts:
When using Choice-Based Conjoint (CBC), the researcher can analyze the data by counting the number
of times an attribute level was chosen relative to the number of times it was available for choice. In the
absence of prohibitions (orthogonal plans), counts proportions are closely related to conjoint utilities. If
prohibitions were used, counts are biased. Counts are ratio data when compared within the same
attribute. Consider the following counts proportions:
Blue 0.50 Brand A 0.40

Red 0.30 Brand B 0.50
Green 0.20 Brand C 0.10
We can say that Brand A was chosen 4 times as often as Brand C (.40/.10). As with conjoint utilities,
we cannot report that Brand A is preferred to Red.
Conjoint Importances:
Sometimes we want to characterize the relative importance of each attribute. We do this by considering
how much difference each attribute could make in the total utility of a product. That difference is the
range in the attribute's utility values. We percentage those ranges, obtaining a set of attribute importance
values that add to 100, as follows:
Percent
Range Importance
Brand (B - C) 60 - 20 = 40 26.7
Color (Red - Pink) 20 - 0 = 20 13.3
Price ($50 - $100) 90 - 0 = 90 60.0
---- ----
150 100.0
860 SSI Web v8
For this respondent, the importance of Brand is 26.7%, the importance of Color is 13.3%, and the
importance of Price is 60%. Importances depend on the particular attribute levels chosen for the study.
For example, with a narrower range of prices, Price would have been less important.
When summarizing attribute importances for groups, it is best to compute importances for respondents
individually and then average them, rather than computing importances from average utilities. For
example, suppose we were studying two brands, Coke and Pepsi. If half of the respondents preferred
each brand, the average utilities for Coke and Pepsi would be tied, and the importance of Brand would
appear to be zero!
When calculating importances from CBC data, we suggest using utilities resulting from Latent Class
(with multiple segments) or HB estimation, if there are attributes on which respondents disagree about
preference order.
Importances are ratio data. An attribute with an importance of 20 (20%) is twice as important as an
attribute with an importance of 10.
Shares of Preference:
Conjoint part-worths and importances are often very difficult for non-researchers to understand. Many
presentations to management have gone awry when the focus of the conversation turned to explaining
how part-worths were estimated and, given the scaling resulting from dummy coding, how one can or
cannot interpret them.
We suggest using market simulators to make the most of your data and for communicating the results of
the conjoint analysis project to non-researchers. When two or more products are specified in the market
simulator, we can estimate what percent of the respondents would prefer each product. Shares of
preference are ratio data. Even so, we recognize that noise inherent in the data, the exponent (scaling
multiplier if using logit simulations) and the simulation model used can dramatically affect the scaling of
shares of preference. A product that captures twice as much share as another in a first choice simulation
(or using a large exponent) may capture considerably less than twice the share using the share of
preference (probabilistic) model.
Purchase Likelihoods:
During some conjoint interviews such as ACA or traditional full-profile ratings-based conjoint (CVA),
respondents may be asked to rate individual products on a 0 to 100 point purchase likelihood scale. This
is very helpful to gauge respondent interest in the product, and for scaling the data for use in purchase
likelihood simulations. Once we have scaled conjoint data to reflect purchase likelihoods, we can predict
how respondents would have rated any combination of attributes included in the study in terms of
purchase likelihood.
Purchase likelihoods should not be considered as strictly ratio data. A respondent may not truly be
twice as likely to purchase a product he rated a 50 versus another he rated a 25. Even so, it is quite
common to state that a product with a purchase likelihood of 55 represents a 10% relative increase in
purchase likelihood over a product that received a 50.
7.2 B: Holdout Choice Tasks in Conjoint Studies

We think it is wise to include holdout choice tasks in conjoint interviews, even though they may not
appear to be needed for the main purpose of the study. They almost always turn out to be useful, for
these reasons:
They provide a proximal indication of validity, measured by the utilities' ability to predict choices
not used in their estimation.
They provide a check on the scaling of the utilities. If the most popular concepts are over-
predicted, then the scale parameter should be reduced. If the predictions are too flat, then the
scale parameter should be increased.
They permit identification and removal of inconsistent respondents.
They can be used for testing specific product configurations under consideration. Much value
can be added by direct measurement of these concepts.
It's hard to design good holdout concepts without some prior idea of respondent preferences. There's no
point in asking people to choose among concepts where one dominates in the sense that everyone
agrees which is best. And, similarly, it's good to avoid presenting concepts that are equally attractive,
since equal shares of preference would be predicted by a completely random simulator. If you present
triples of concepts, it's probably best if their shares of choices are somewhere in the neighborhood of
50/30/20.
When conducting CBC studies, if you plan to do segmentation with latent class analysis, it's wise to
consider the kinds of groups you expect to get, and to design products in holdout choice sets so that
one alternative will be much more preferred by each group. This maximizes your ability to confirm the
validity of the multi-group Latent Class simulator.
It isn't necessary to have many holdout sets to check the validity of your utilities, or their scaling.
However, if you want to use those choices to identify and eliminate inconsistent respondents, you need
several choice sets. For ACA studies, holdout concepts can be included in the Web-based interview
using CiW question types. For CBC, they can be included by adding "Fixed" choice tasks to the
design.
We've shown an example of a holdout choice task below:
If you were shopping for a credit card and these were your only options,
Visa MasterCard Discover
No Annual Fee $30 Annual Fee $60 Annual Fee
15% Interest Rate 12% Interest Rate 9% Interest Rate
Frequent Flier Program No Frequent Flier Program Frequent Flier Program
$4,000 Credit Line $6,000 Credit Line $2,000 Credit Line
It is probably not very useful to include a "None" option in holdout choice tasks, particularly when these
862 SSI Web v8
are paired with traditional conjoint exercises which don't have a "None" option. The exception to that rule
would be if modeling "None" and using it in the market simulator was of interest.
Finally, if you do have several choice sets, it's useful to repeat at least one of them so you can obtain a
measure of the reliability of the holdout choices. Suppose your conjoint utilities are able to predict only
50% of the respondents' holdout choices. Lacking data about reliability, you might conclude that the
conjoint exercise had been a failure. But if you were to learn that repeat holdout tasks had reliability of
only 50%, you might conclude that the conjoint utilities were doing about as well as they possibly could,
and that the problem lies in the reliability of the holdout judgements themselves.
864 SSI Web v8
8 SMRT (Market Simulator) Help

8.1 What Is SMRT?
SMRT stands for "Sawtooth Software Market Research Tools" and is a separate program from the SSI
Web system. The SMRT Market Simulator is used for estimating the likely acceptance or interest in
different product concepts defined using the attributes and levels included in your conjoint analysis
study. You install SMRT separately from the CD-ROM or from our website.
SMRT was released as a software system prior to SSI Web. It is our older Windows-based software for
conducting ACA, CBC and traditional conjoint (CVA) analysis. The system includes market simulation
capabilities (supporting CVA, ACA, CBC, and ACBC--which you tell SMRT to treat as it it were a CBC
study) and additional tools for data management and analyzing CBC data, including Counts, logit
analysis, and latent class estimation. However, SMRT cannot create web-based projects. Interviewing
with SMRT is done via floppy disks, from a hard drive on a local or networked machine, or via paper-and-
pencil (in the case of CBC and CVA).
When the internet increased in popularity for market research interviewing, Sawtooth Software created a
new platform for Web-based surveys called SSI Web rather than try to retrofit the older SMRT platform for
web-based interviewing. Since the market simulator (and other analytical tools for CBC) already existed
under SMRT, Sawtooth Software elected not to rebuild those same tools within the new SSI Web
platform. (We are building market simulation tools to be integrated into SSI Web for the upcoming v9
release!).
The SMRT platform can import data and utility runs developed within the SSI Web system. You can also
export segmentation data from SSI Web to a text-only format and import those data into SMRT for
segmentation purposes. When you analyze data using SMRT, you need to create a new project within
the SMRT software system. You may store this new project in the same folder as your SSI Web
project, or you may install it in a separate folder.
SSI Web and SMRT are independent software systems, but they can "speak" to one another through
Import, Export and Merge processes.
A special version of SMRT may be installed by clients (the Client Conjoint Simulator). This way, clients
can perform market simulations independent of consultants.
Sawtooth Software also has a web-based marketing simulator, if you need a less feature-filled and more
user-friendly market simulator than SMRT.
SMRT (Market Simulator) Help 865
8.2 Data and Utility Run Management

8.2.1 Importing Utility Runs
Overview
SMRT can automatically import conjoint part-worth data generated by any of Sawtooth Software's current
conjoint software systems by clicking Analysis | Run Manager | Import…. (The number of attributes
and levels per attribute to be imported cannot exceed the limitations of your system). It can also import
part-worths that you have generated in most any other standard way, as long as you format those data
as an .HBU file and optionally create a file describing the coded linear terms (.VAL) if they apply.
The formats automatically imported by the Market Simulator are:
HB Utilities (*.hbu, *.hbc)

ACA or ACA v4.x (*.utl)
CVA (Traditional Conjoint) Utilities (*.utl)
Latent Class Utilities (*.pxx)
Generic Conjoint File (*.hbu)
Latent Class, HB Utilities, and the Generic Conjoint *.hbu formats are the most complex, allowing for a
great deal of flexibility in the inputs, including first-order interactions and linear term specifications of
utilities.
Importing Utilities into the Market Simulator
The first step in importing utilities into the Market Simulator is to create a new study or open an existing
study. Open the SMRT software by clicking Start | Programs | Sawtooth Software | Sawtooth
Software SMRT. To create a new study, select File | New, and browse to the directory in which you
want to save all study-related files.
Remember: The SMRT and SSI Web systems work independently, so you need to establish a
new project name and project file for the SMRT system to analyze your conjoint data.
Specify a study name, and the extension ".SMT" will automatically be added. If you own more than one
conjoint analysis license, you are asked to specify the study type (i.e. ACA, CVA or CBC). (If you are
have an ACBC project; just indicate to SMRT that it is a CBC project--CBC and ACBC are
interchangeable as far as SMRT market simulations are concerned.)
To open an existing SMRT study (not an SSI Web study), select File | Open, and browse to the
directory containing the studyname.SMT file. Double-click the studyname.SMT file. If attributes and
levels definitions exist in the study, they must match the attributes and levels for the conjoint data to be
imported.
After you have opened the study, choose Analysis | Run Manager. The Import option on the Run
Manager dialog lets you import conjoint utility (part-worth) data from text-only files for use within the
Market Simulator. You can import multiple sets of conjoint utilities which become separate selectable
runs. The utility data can come from a number of formats or sources:
HB Utilities (*.hbu) Hierarchical Bayes utilities generated by CBC/HB, ACA/HB, or

Adaptive CBC (see the HB manuals for data layout). If
importing an .HBU file, the attribute labels are read from the
866 SSI Web v8
header section of the .HBU file.
ACA (*.utl) part-worth utilities from the ACA system (see the ACA manual
for data layout) or ACA v4.x. If you import a studyname.UTL
file, labels for the attribute levels are imported from the
studyname.ACA file. If a studyname.ACA file does not exist,
default attribute and level labels are assigned.
CVA Utilities (*.utl) part-worth utilities from the CVA v2.x system. If you import a
studyname.UTL file, labels for the attribute levels are imported
from the studyname.CVA file. If a studyname.CVA file does
not exist, default attribute and level labels are assigned.
Latent Class (*.pxx) Latent Class utilities generated by the CBC Latent Class v2
Module (see the Latent Class manual for data layout). When
importing Latent Class utilities, the data are read from two files:
*.Pxx (e.g. studyname.P05 for a 5-group solution), which
includes information about respondents' probabilities of
membership in each group; and the *.LCU file which contains
the average utility estimates for each group. Note that the .Pxx
file is not generated by default, but must be requested by a
parameter in the .PAR file prior to generating the Latent Class
run. During the import process, individual-level utility data are
generated by multiplying the probabilities of membership by the
vector of utilities for each class. If importing Latent Class data,
the attribute levels are read from the header section of the .LCU
file. If linear specification of quantitative attributes has been
used, there must be an accompanying .VAL file in the same
directory.
You can also merge the latent class membership for use as
banner points and/or filters. To merge group (class)
membership, select File | Merge Variables | Add... and then
choose Latent Class 2.x Group Membership (*.Pxx). SMRT
assigns each respondent to the group for which he/she has the
highest probability of membership (ties broken randomly). To
use group membership from latent class as a banner point, you
must first specify the merged variable as a Custom Segment.
Generic Conjoint (*.hbu) With the Generic Conjoint .HBU format, you can import
conjoint utility data generated in any other way or software
program, including main effects, first-order interactions or linear
term specifications, as long as the attributes and levels do not
exceed the capacity of your system and the data are formatted
as an .HBU file. The generic .HBU format assumes one case
per respondent (not "Draws" as can be an option for
Hierarchical Bayes estimation).
Data from text-only files are read with 15 significant digits of precision.
When utility run data are imported, four files are modified or created:
Studyname.UCS (a binary data file containing respondent numbers and conjoint utilities)
Studyname.QNR (a binary data file containing attribute level labels and questionnaire settings)
Studyname.DAT (a binary data file containing respondent numbers and any segmentation
information)
Studyname.IDX (a binary index file describing the layout of the Studyname.DAT file).
If your study had no respondent data prior to importing a utility run, a list of respondent numbers
(corresponding to your utility run) is created in a Studyname.DAT file. If your study already included
respondent numbers and data, then the conjoint part-worths are written to the .UCS file and it is
assumed that corresponding respondent records are available in the .DAT file (if applying banner points,
filters or weights). If no attribute labels are provided, SMRT provides default labels. If conjoint attributes
and level labels exist in your study prior to importing conjoint utility data, those definitions are preserved
and the utilities to be imported must match that layout. If there is a mis-match between the number of
imported attribute levels and the number of existing levels in the study, SMRT will issue a warning
message.
If your text-only conjoint data file included segmentation information you want to import, you must merge
those data separately using File | Merge Variables.
Exporting Utilities into Text-Only Format
The Export option lets you export conjoint utility data from the binary .UCS file into a text-only .HBU
format. Data are written with up to 5 decimal places of precision.
Text-only conjoint data in an .HBU format can be imported for use in the Market Simulator.
868 SSI Web v8
8.2.2 Merging Segmentation Data

Introduction
SMRT includes a Merge Wizard to lead you through the process of merging additional descriptive or
segmentation variables into that same data file. These variables might be demographics, such as
gender, household size or age, or information about a firm such as company size, volume purchased, or
industry classification. You can also merge continuous variables to be used as respondent weights.
The type of file the typical user will merge will probably be in text-only (DOS Text) format as created by
SSI Web. The data file to be merged must include a respondent classification (Case ID) number. All
merged data must be numeric only.
After you have merged additional variables into your data set, these are automatically available to you as
"stub variables" for use in the Tables program or as respondent weights in either the Tables or the
Market Simulator. If you undertake the additional step of defining Custom Segmentation Variables based
on these merged variables, you can also use the merged information as filters or "banner points" during
simulations if using individual-level utilities.
The next few sections include basic information about merging segmentation information and defining
Custom Segmentation Variables. Additional help and details are available in the on-line help within
SMRT, accessed by pressing F1 at any time during the merge process.
Merging Segmentation Data from Text-Only Files
Often users merge segmentation information (demographics, usage data, firmographics, etc.) into their
market simulators for use as segmentation, filters or weighting variables. These data can come from any
source (such as SSI Web), as long as the data have been formatted as a text-only file (either fixed-
column or delimited). The respondent numbers in the file(s) to be merged must be numeric and must
match the respondent numbers in the conjoint data file. The cases do not necessarily need to be in the
same sort order.
The Merge program in the Market Simulator has a "Merge Wizard" that leads you through the entire
process.
For the Merge program to work properly, you must have unique respondent numbers that match in both
the source and destination data files (numeric data only). You should not have duplicate respondent
numbers in either file. Any respondent in the file of conjoint part-worths not found in the file to be merged
will receive a "missing" value for the selected merge variables.
Delimited Files
When you specify to merge information from a delimited text-only file into your data file (*.dat), you need
to instruct the software on how to interpret the data file. You do so by specifying the delimiter (the
character that separates data fields, such as a space, tab or comma) and specifying what value (if any)
is used to designate missing values. This value may be any string of alphanumeric characters or
symbols.
The Merge Wizard asks you to specify whether respondent data is on one line (example below, with
three respondents numbered 1001 to 1003) or otherwise.
1001 3 2 5 7
1002 2 3 1 5
1003 1 3 3 7
In the example above, each respondent record occupies a single line, and the data are separated by
blank spaces. This layout is an example of "one line per respondent record."
Below is an example in which each respondent's data span more than one line:
1001
3 2 5 7
1002
2 3 1 5
1003
1 3 3 7
In the example above, each respondent record is spread across two lines. In order to understand how to
read this layout, the Merge Wizard asks you how many fields per respondent record there are. In this
case, there are five (note that this number includes the respondent number field).
The Merge Wizard asks you to indicate in which field the respondent number is located. Note that this
is not the "column number" where each column is a single character, but the field in which fields are
separated by the delimiter. In the examples above, respondent number is located in field #1.
To avoid misinterpretation of text strings (which may contain delimiters), text in the data file may be
enclosed in quotation marks. Single (') or double (") quotation marks are supported.
The following is an example of how Merge reads data depending on the user-specified parameters for
handling string variables:
Text in Delimiter Quotes? Data Read as String read as

Comma Yes 5,6,"B K O",32 4 fields B K O (1 string)
Comma No 5,6,"B K O",32 4 fields "B K O" (1 string)
Space Yes 5 6 "B K O" 32 4 fields B K O (1 string)
Space No 5 6 "B K O" 32 6 fields "B K O" (3 strings)
Fixed Text-Only Files
When you are merging data from a fixed column text-only file, the Merge Wizard asks you to specify in
which columns the data are located and the length of each field. A column is the width of a single
character. Many text editors (such as Microsoft's DOS editor, which you can access from the DOS
prompt by typing EDIT and pressing the ENTER key) provide a column counter at the top or bottom of
the edit window.
With fixed column ASCII files, the data are aligned in columns when viewed with a non-proportional font
(such as Courier) such as follows (the first two lines below are a reference for counting columns, they
would not appear in the data file):
1234567890123456789012345678901234567890
----------------------------------------
6997 3 25 2
25283 23 1
234 9 6 4
870 SSI Web v8
Assume that the first variable is the respondent number (though it is not necessary that it be the first
variable in the file). It is right-justified within an area that begins in column 1 and has a length of 6. The
next variable starts in column 15 and has a length of 2. The third variable starts in column 28 and has a
length of two (note that the second case has a missing value). The final variable starts in column 34 and
has a length of 1.
8.2.2.1 Custom Segmentation Variables

The variables you merge into your data set (i.e. from SSI Web or a Text-Only source) though valid
automatically as weights are not available as banner points and respondent filters until you first create
new variables based on those data called Custom Segmentation Variables.
A Custom Segmentation Variable is one you create based on the values of one or more other
merged variables. Custom Segmentation Variables can be used as banner points and respondent
filters in the Tables and the Mark et Simulator programs.
You can create Custom Segmentation Variables by selecting Analysis | Custom Segments. The
details for specifying these variables is available in the on-line help by pressing F1 when you are at the
Custom Segments dialog.
As an example, we could create a Custom Segmentation Variable based on gender and income.
Suppose we merged a variable called Q31 from an SSI Web questionnaire that classified a respondent's
gender (Male=1, Female=2), and Q32 that reported a respondent's income (continuous variable). These
variables would not be available for choosing as banner points or respondent filters in the simulator until
we had created a custom segmentation variable. We might call this custom variable Gender_Income,
and indicate the following definitions:
Definitions for Custom Segmentation Variable: Gender_Income
Segment Label Logical Definition

Female Low Income (Q31 = 2) & (Q32 <= 25000)
Female High Income (Q31 = 2) & (Q32 > 25000)
Male Low Income (Q31 = 1) & (Q32 <= 25000)
Male High Income (Q31 = 1) & (Q32 > 25000)
After creating a new Custom Segmentation Variable called Gender_Income, these segments are
available for selection as banner points and respondent filters.
Though our example above used two variables to define a Custom Segmentation Variable, you could use
only one variable, or many more variables. The Market Simulator supports sophisticated ways for
defining Custom Segmentation Variables, including the following operators: = , >, >=, <, <=, != (not
equal to), & (and), | (or), ! (not), $MISSING (missing value). You can include complex statements that
include parentheses for controlling the flow of logical operations.
872 SSI Web v8
8.2.2.2 Weighting Respondents

Weighting lets some respondents have more impact on summary statistics than others. For example, a
respondent with a weight of 2.0 is counted twice as much as another respondent with a weight of 1.0.
Weighting is useful for adjusting a sample to reflect known population characteristics. Weights can be
applied during all stages of analysis.
There are two ways to define weights:
1. Use a merged variable value as a weight. You may have calculated a weight for each
respondent and placed those weights along with the respondent numbers in a text-only file for
merging with your part-worth data.
2. Assign weights to segmentation categories. In this case, the weighting variable must be
defined as a Custom Segmentation Variable. For example, assume we wanted to weight our
sample based on Gender. The table below shows the weight that needs to be given to
respondents to achieve the target gender representation.
Actual Target
Proportion Proportion Weight
Male 0.35 0.49 0.49/0.35 = 1.4000
Female 0.65 0.51 0.51/0.65 = 0.7846
If we apply these weights, the average weight will be 1.0, and the unweighted number of respondents will
be equal to the weighted number of respondents. Whether you use a merged variable value as the
weight or assign weights to categories of a segmentation variable, we strongly suggest your weights
have this property. If not, some statistics in Tables will be incorrect.
8.2.3 Data Management Tools

Introduction
The Market Simulator includes some data management tools that can be useful for some situations.
These tools are accessed from the Tools menu, and include:
View/Edit Data
Index
Data Doctor
Optimize Study files
This section describes the purpose and use for these data management tools.
View/Edit Data
The Market Simulator lets you view the respondent numbers and any segmentation variables that have
been merged into the conjoint part-worth data. (You cannot view the part-worths themselves through this
option.)
When you click Tools | View/Edit Data, a dialog is displayed with three separate tabs: Respondents,
View Respondent Data and Edit Respondent Data.
Respondents Tab
When you click the Respondents tab, SMRT reads the .IDX file located in the study directory, which
provides an index of the .DAT file (pointers to the location of data segments within a .DAT file) including a
list of the respondent numbers. Respondent numbers are initially displayed in sorted order (by default)
and an asterisk following a number indicates a gap in the sequence.
Note that SMRT does not read the studyname.DAT file to generate the report on this tab. Therefore, if
your .IDX file for some reason has become corrupted or for some reason does not match (pertain to) the
.DAT file, it is possible to see information on this tab that is really not in the .DAT file. To ensure that the
information in the .IDX file matches the .DAT file, you should run either Tools | Index, or Tools | Data
Doctor.
Close Closes the current dialog.

Print Prints the information displayed on the screen.
New Disk Lets you view or edit data in another directory or floppy disk.
Filter (Drop down list). This drop-down list lets you filter the respondents to be
displayed by complete/incompletes or a particular disposition. It also lets you
view summary counts of the number of respondents by disposition or status
(these disposition and status codes are a carryover from Sawtooth Software's
Ci3 system, and are not relevant to the Market Simulator).
Sort By default, the respondent numbers in this window have been sorted in
ascending order. Note that the actual order of the records in the file will likely
be different. If you uncheck the Sort box, the actual order of the respondent
records is displayed. Note that this check box only changes the order in
which the records are displayed on the screen, it doesn't actually change their
order in the data file.
Disposition Displays the disposition codes (applies to Ci3 data only—see the Ci3 manual
for more details) for each respondent record.
874 SSI Web v8
View Respondent Data Tab
This dialog reads the information from the studyname.DAT file and displays any segmentation data that
you may have merged with the conjoint part-worths.
If not all of the merged information can be displayed within the window, you can use the scroll bar at the
right of the dialog box to see values for later variables. You can scroll through data for different
respondents by clicking on the double left-or-right arrows at the bottom of the dialog box. Alternatively,
you can select a different respondent by clicking on that respondent's number at the left of the screen.
Close Closes the current dialog.

Print Prints the information displayed on the screen.
<< and >> These buttons let you move between adjacent records in your data file.
Edit Respondent Data
This dialog shows detailed information on each respondent, permitting editing the values for merged
information or changing the respondent number.
The Answer: field lets you change the value of any response.
You can make tentative editing changes to answers that will not become permanent until you say that
you want to retain them.
If you click the Change Respondent Number button in the upper right hand corner, a box appears in
which you can provide a different respondent number. You change a respondent number by typing in a
different respondent number, and then clicking on OK. Unlike segmentation variable values that you
change, a respondent number change is made immediately, and you will not have the opportunity to
decide that this change should be ignored.
You can also delete this respondent if you like. You do that by clicking the box labeled Delete This
Respondent. A check mark appears in the box, indicating that the respondent will be deleted. If
individual-level utility runs have been saved, the utility data will also be deleted for this respondent. If you
change your mind you can click that box a second time and remove the check mark.
You can move from one merged segmentation variable to the next by scrolling with the double arrows at
the bottom of the dialog box, or scrolling in the box in which the question (variable) name is displayed.
When you have finished with this respondent, click the Respondents Tab at the top of the dialog box to
again see the list of respondents with data in this file, and select another respondent number. Then click
again on the Edit Respondent Data tab, and you will be able to edit that respondent's data.
When you make changes to the data file, they are recorded in a file named questionnaire.log. That file is
in text-only format and can be viewed by any editor or word processor.
Close When you click the Close button, any changes you made to the data are
made permanent. You can make modifications to multiple records before
clicking the Close button.
<< and >> These buttons let you move between adjacent records in your data file.
Delete This Respondent

Deletes the current respondent (when you click the Close button).
Change Respondent Number

Lets you change the case identification number. This change is made
immediately rather than when you click the Close button.
Index…
The data file (studyname.DAT) used by SMRT is always accompanied by an Index file (studyname.IDX).
This file contains information about where each respondent's data are located within the file.
If your studyname.IDX file gets lost or corrupted, you can easily regenerate it by clicking this option. By
default, the software will re-index the file within your study directory. You can also regenerate lost or
corrupted .IDX files in other directories (or your floppy drive) by specifying a different path.
Data Doctor…
The Data Doctor is a utility for repairing data files (studyname.dat) that have become corrupted.
Occasionally, sectors on disks fail or hardware doesn't properly record the data. With very large data
sets and tens of thousands of respondents, data corruption or loss becomes increasingly likely.
Sometimes a single corrupted record in the middle of a data file makes it impossible for the Market
Simulator to understand how to deal with the subsequent respondents. Other times, the information at
the top of the data file is corrupted so that the Market Simulator doesn't even know how to begin.
The Data Doctor analyzes a .DAT file and checks it for validity, skipping non-critical sections of the file if
it finds errors in them. The user is prompted as to what action to take for each error encountered. This
process can be slow with large data files.
The Data Doctor can operate on the accumulated data file in the study directory, or on any other drive/
directory.
Optimize Study Files…
This tool is used to optimize the study files (.SMT, .QNR and .UCS), removing information no longer
needed. For example, when you delete a utility run (from the .UCS), the section is only marked for
deletion until you choose Optimize Study Files.
Optimizing study files will help speed access to information in those files.
876 SSI Web v8
8.2.4 Tables Program

Overview
The functionality and output of the Tables program resembles the cross-tab capabilities found in many
statistical and cross-tab packages today. However, the SMRT Market Simulator is primarily concerned
with conjoint analysis; the Tables program is basic and relatively inflexible. If you need to analyze the
results with a more sophisticated package, SMRT or SSI Web can export data for use in other software
systems.
Analyzing Nominal/Ordinal Data
Nominal data are numeric values, where the numbers refer to categories or classifications. For example,
one might code gender as:
Male = 1
Female = 2
An example of a type of survey question that might be analyzed is shown below:
Which of the following best describes the

highest level of education you achieved?
Some high school
Graduated high school
Some college
Graduated college
Some post-graduate studies
Post-graduate degree
Doctoral degree
If we assign the following codes:
Some high school = 1

Graduated high school = 2
Some college = 3
Graduated college = 4
Some post-graduate studies = 5
Post-graduate degree = 6
Doctoral degree = 7
it is true that larger numbers reflect greater education (most would agree on this point). In contrast to the
previous example where the coded value really had no quantitative meaning other than nominal
classification, these data reflect ordinal scaling. However, as with nominal data, it still doesn't make
sense to apply mathematical operations such as addition, subtraction, multiplication or division to
ordinal data. For example, a person with a "post-graduate degree" (code 6) doesn't necessarily have
twice as much education as a person who completed "some college" (code 3).
The Table Display
When you analyze survey responses using Tables, a table of results (in this case, frequencies) is
generated and displayed in the report window:
Total
Education
Some high school 3
Graduated high school 35
Some college 68
Graduated college 73
Some post-graduate studies 18
Post-graduate degree 11
Doctoral degree 4
Total 212
Missing -
This simple table displays summary frequencies for the entire sample. Frequencies represent the
number of times respondents answered (or were classified) in a particular way. Every time you click
Compute!, a new table (or set of tables if you have selected multiple variables) is appended to the report
window. You clear the window by clicking Clear. In addition to frequencies, there are other statistics
you can include (by clicking Statistics) in the table appropriate for nominal or ordinal variables:
Column percent
Row percent
Table percent
Chi-Square
These statistics are described in any good statistics textbook.
You can also analyze the results by another variable, such as Gender. In the example below, Gender is
the banner (column) variable, and Education the stub (row) variable. This display is often referred to as a
cross-tabulation.
Education by Gender
Gender
Male Female Total
Education
Some high school 3 - 3
Graduated high school 18 17 35
Some college 38 30 68
Graduated college 33 40 73
Some post-graduate studies 8 10 18
Post-graduate degree 6 5 11
Doctoral degree 3 1 4
Total 109 103 212

Missing - - -
878 SSI Web v8
This display lets us compare educational achievement by gender. For example, 33 males went as far as
graduating college (but no farther) vs. 40 females. If we include column percentages, we make better
sense of the results, since there are an unequal number of males and females in the sample.
Education by Gender
Gender
Male Female Total
Education
Some high school 3 - 3
3% - 1%
Graduated high school 18 17 35

17% 17% 17%
Some college 38 30 68
35% 29% 32%
Graduated college 33 40 73
30% 39% 34%
Some post-graduate studies 8 10 18

7% 10% 8%
Post-graduate degree 6 5 11
6% 5% 5%
Doctoral degree 3 1 4
3% 1% 2%
Total 109 103 212

100% 100% 100%
Missing - - -
Analyzing Continuous Variables
You can also analyze continuous variables with the Tables program.
The following statistics are available in Tables and appropriate for analyzing continuous data:
Mean
Standard deviation
Variance
Minimum
Maximum
Standard Error of Mean
These statistics are described in any good statistics textbook.
Continuous variables can only be specified as stub variables; they cannot be used as a banner variable
unless you recode them to discrete categories by creating a custom segment.
If a continuous variable you are analyzing has many unique values, the table can become extremely
large. A simple remedy is to turn off frequencies and only request summary statistics, such as the
mean, minimum, maximum, standard deviation and standard error.
880 SSI Web v8
Some Notes on Respondent Weighting
The Tables program lets you weight respondents differentially. Weighting lets some respondents have
more impact on the statistical summaries than others. For example, a respondent with a weight of 2.0 is
counted twice as much as another respondent with a weight of 1.0. Weighting is useful for adjusting a
sample to reflect known population characteristics.
Weights affect every statistic mentioned earlier in this section, except for Minimum and Maximum.
When you request weighted tables, both weighted and unweighted totals are reported. We strongly
suggest you assign weights so that the average weight is equal to 1.0. If the weights do not average 1.0,
the total unweighted number of respondents will not equal the total weighted respondents. More
critically, many statistics are incorrect if the average weight is not equal to 1.0, including:
Standard error
Chi Square
You will also see some inaccuracy in the following statistics because of the division by (n-1), especially
if you have small sample sizes:
Standard deviation
Variance
8.3 Using Simulators to Answer Strategic Questions

8.3.1 Overview
The Market Simulator is usually considered the most important tool resulting from a conjoint project.
The simulator is used to convert raw conjoint (part-worth utility) data into something much more
managerially useful: simulated market choices. Products can be introduced within a simulated market
scenario and the simulator reports the percent of respondents projected to choose each. A market
simulator lets an analyst or manager conduct what-if games to investigate issues such as new product
design, product positioning, and pricing strategy.
A Warning about Interpreting the Output of Market Simulators
Under very controlled conditions (such as markets with equal information and distribution), market
simulators often report results that closely match long-range equilibrium market shares. However,
conjoint part-worth utilities cannot account for many real-world factors that shape market shares, such
as length of time on the market, distribution, out-of-stock conditions, advertising, effectiveness of sales
force, and awareness. Conjoint analysis predictions also assume that all relevant attributes that
influence share have been measured. Therefore, the share of preference predictions usually should not
be interpreted as market shares, but as relative indications of preference.
Divorcing oneself from the idea that conjoint simulations predict market shares is one of the most
important steps to getting value from a conjoint analysis study and the resulting simulator. While
"external effect" factors can be built into the simulation model to tune conjoint shares of preference to
match market shares, we suggest avoiding this temptation if at all possible. No matter how carefully
conjoint predictions are calibrated to the market, the researcher may one day be embarrassed by
differences that remain.
882 SSI Web v8
8.3.2 What Is a Market Simulation?

A conjoint study leads to a set of utilities (part-worths) that quantify respondents' preferences for each
level of each attribute. These part-worths can be analyzed in a number of ways. You can examine each
respondent's part-worths (but this task could become overwhelming). You might summarize the average
part-worth utilities, or compute average importances. You could create graphs and charts to display that
information, but to many it might seem somewhat abstract and difficult to grasp. Examining average
responses could also fail to detect important segments of the market that have unique and targetable
preferences.
A good market simulator is like having all of your respondents gathered in one room for the sole purpose
of voting on product concepts and competitive scenarios (defined in terms of the attribute levels you
measured) you show them. You walk into the room, show them a market scenario (i.e. products A, B
and C), and they vote for the one(s) they prefer. Millions of potential products and market situations
could be evaluated, and your captive audience would never get tired, ask for lunch breaks, or require you
to pay them by the hour.
How does a market simulator work? Let's suppose we had a way (such as through conjoint or choice
analysis) to quantify how much people liked the different qualities of ice cream cones. Let's refer to
those preferences as part-worth utilities, and assume the following values for a given respondent:
Utility
Chocolate 0
Vanilla 30
Strawberry 40
$0.60 50
$0.80 25
$1.00 0
Using those utility values, we could predict how he would choose between a vanilla cone for $0.80 or a
strawberry cone for $1.00.
Vanilla (30 utiles) + $0.80 (25 utiles) = 55 utiles

Strawberry (40 utiles) + $1.00 (0 utiles) = 40 utiles
We'd predict he would prefer the vanilla cone. If we had data for 500 respondents, we could count the
number of times each of the two cones was preferred, and compute a "Share of Preference," also
referred to as a "Share of Choice":
Share of Choice
Vanilla @ $0.80 300/500 = 0.60
Strawberry @ $1.00 200/500 = 0.40
In our hypothetical market simulation, 60% of the respondents preferred the vanilla, and 40% the
strawberry cone. This illustrates the most simple simulation approach, referred to as the First Choice
model.
8.3.3 Why Conduct Market Simulations?

Looking only at average preferences (part-worth utilities) can mask important market forces caused by
patterns of preference at the segment or individual level. Marketers are often not interested in averages,
but in the targetable, idiosyncratic behavior of segments or individuals.
For example, consider the following three respondents, and their preferences (utilities) for color:
Utilities for Color
Blue Red Yellow

Respondent A 50 40 10
Respondent B 0 65 75
Respondent C 40 30 20
Average: 30 45 35
Looking only at average preferences, we would pronounce that red is the most preferred color, followed
by yellow. However, if one of each color was offered to each respondent, red would never be chosen
under the First Choice model, yellow would be chosen once, and blue twice — the exact opposite of
what aggregate part-worth utilities suggest. While this is a hypothetical example, it demonstrates that
average part-worth utilities do not always tell the whole story. Many similar, complex effects can be
discovered only through conducting simulations.
Some reasons for conducting conjoint simulations include:
1. Conjoint simulations transform raw utility data into a managerially useful and appealing
model: that of predicting market choice (Share of Preference) for different products.
Under the proper conditions, shares of preference quite closely track with the idea of
market share — something most every marketer cares about.
2. As demonstrated earlier, conjoint simulations can capture idiosyncratic preferences
occurring at the individual or group level. These "hidden" effects can have a significant
impact on preference for products in market scenarios. When multiple product offerings
have been designed to appeal to unique segments of the market, capturing such effects
is especially important for accurately predicting preference.
3. Conjoint simulations can reveal differential substitutability (cannibalism/cross-elasticity
effects) between different brands or product features. If two brands are valued highly by
the same respondents (have correlated preferences), these brands will tend to compete
more closely. Product enhancements by one of these brands will result in more relative
share being lost by the correlated brand than by other less similar brands within the
same simulation. Examining aggregate part-worth utilities cannot reveal these important
relationships.
4. Conjoint simulations can reveal interaction effects between attributes. If the same
respondents that strongly prefer the premium brand are also less price sensitive than
those who are more likely to gravitate toward a discount brand, sensitivity simulations will
reflect a lower price elasticity for the premium relative to the discount brand. A similar
interaction effect can occur between many other types of attributes: such as model style
and color.
Note when using CBC data: It is important to note that complex effects other than two-way interactions
such as cross-effects cannot be reflected using the model of aggregate-level logit offered by our CBC
system. Latent Class is a technique for estimating part-worth utilities and reflecting respondent
differences at the group/segment level, and CBC/HB (Hierarchical Bayes) is a way to estimate utilities at
the individual level for CBC data. Because they are built on individual-level preferences, simulators based
884 SSI Web v8
on HB models are able to reflect the important and complex behaviors mentioned earlier. It is not
surprising that Latent Class and CBC/HB have been shown to outperform similarly-defined aggregate
level logit models in terms of predictive validity.
ACA (Adaptive Conjoint Analysis) and CVA (Full-Profile Conjoint Analysis) capture respondent-by-
respondent preferences and are thus very useful inputs to this and other market simulation models.
8.3.4 Typical Questions for Conjoint Simulators

There are many marketing strategies that can be investigated with the conjoint simulator. Here are three
of the most common:
1. Given a current competitive environment, what product should I offer to maximize

interest in my offering? How can I modify an existing product to capture more relative
demand? A Market Simulator lets you input multiple products and place them in simulated
competition one with another. Each product is defined using the attribute levels measured in the
conjoint study (brands, colors, prices, speeds, warrantees, etc.). Therefore, if you have measured
the relevant brands and features offered in the market, you can simulate a realistic market scenario
within the Market Simulator. Within that market scenario, you can add a new product and see how
well it competes. If the goal is to maximize share, offering the best features at the lowest price is
often the trivial solution. The Market Simulator focuses on the demand side of the marketing
equation; but it is also important to pay attention to the supply side and take the costs of
producing different products/services into consideration. If you have cost information available to
you, the Market Simulator permits you to investigate the incremental benefits of different features of
a product relative to the cost of offering them. (SMRT does not provide an automatic way to input
cost information, but you can divide the incremental gain in share of preference by the incremental
cost to produce a profitability index.)
2. What is the relative price sensitivity of different brands? If I raise my price by 10%, how
will it affect my brand? How will it affect competitors' brands? You can conduct "sensitivity
analysis" for attributes such as price using the Market Simulator to generate relative demand
curves. The approach involves holding all other brands at a constant price and changing the price
of a single brand, recording the relative share at each point for that brand along the price
continuum. If your conjoint data reflect respondent differences (all models offered by Sawtooth
Software except for aggregate CBC logit), differential cross-elasticities can be investigated through
this approach.
3. What portfolio of products can I offer to appeal to different market segments and
maximize overall share? The market simulator lets you merge additional segmentation
variables (such as demographics or firmographics) with your conjoint part-worths. If you have
segmentation information, you can investigate product formulations that appeal to different groups
of respondents. It is likely that by designing products that appeal uniquely to targetable segments
that you can increase overall share for your product line or occupy a niche that is not currently
being served.
These three strategic questions and the simulation strategies for responding to them are illustrated later
within this documentation.
886 SSI Web v8
8.4 Market Simulation Theory and Models

8.4.1 Base Case Scenario
Introduction
Market Simulations provide a very useful way to use conjoint/choice data. Simulations provide an
intuitive tool to move from the esoteric realm of part-worth estimates/effects toward the practical world of
predicting buyer behavior for specific market situations. Before using the Market Simulator within SMRT,
you should become familiar with some terminology, issues and theory.
This section will discuss setting up a base case scenario, the scaling of simulation results, the Red-Bus/
Blue-Bus problem, and the five simulation models offered in the Market Simulator. Finally, External
Effects and the use of the None weight are introduced.
Base Case Scenario
Usually the first step in using the market simulator is to define a "Base Case" scenario. A base case
typically reflects a current (or future) market scenario: your brand vs. the relevant competition. If there is
no relevant competition, or your conjoint study was designed to model only your product, the base case
may be a single product, reflecting a likely configuration.
The Market Simulator lets you input the market scenario in a grid format, where the products are rows
(you can have up to 100), and the attributes are columns (you can have up to 30).
Attribute 1 Attribute 2 Attribute 3

Product (Brand) (Package) (Color)
Product A 1 2 3
Product B 2 1 2
Product C 3 3 1
You provide text labels in the first column to identify the products. In the attribute cells, you type the
numeric value associated with different attributes. For example, Product B is defined by level 2 of Brand,
level 1 of Package and level 2 of Color. (The Market Simulator displays the list of attributes and the
codes associated with each level as you enter values in the grid.)
After defining the market scenario, you should decide which simulation method is appropriate for your
data and the types of strategic questions you intend to answer. Later, we'll describe all five available
models in the Sawtooth Software market simulator and provide some notes and recommendations for
each.
After you have chosen the appropriate simulation technique, you can begin conducting simulations.
Typically, one first examines the shares of preference (or choice) given to the products in the base case.
Then, modifications to the base case are investigated by altering the base case itself and rerunning the
analysis, or by adding additional "scenarios." A scenario is just another name for a defined set of
competitive products, and setting up each subsequent scenario feels just like defining the first base case
scenario. The market simulator lets you input many simulation scenarios, and stores these for your
convenience.
Prior to introducing the different models of choice offered by the Market Simulator, it is instructive to
cover two topics: the Exponent and the Red-Bus/Blue-Bus problem.

888 SSI Web v8
8.4.2 The Exponent (Scale Factor)

Assume that you set up a simulation as defined in the previous section with three products: A, B and C.
Also assume that you are simulating the projected choice for just one individual under a Share of
Preference model (described in greater detail later). After clicking Compute!, simulation results for this
individual might come back as follows:
Product Share of Choice

A 10.8%
B 24.0%
C 65.2%
Total 100.0%
Note that in conjoint simulations, the resulting shares are normalized to sum to 100%. We interpret
these results to mean that if this respondent was faced with the choice of A, B, or C, he would have a
10.8% probability of choosing A, a 24.0% probability of choosing B, and a 65.2% probability of choosing
C. Note that B is more than twice as likely to be selected as A, and C is more than twice as likely to be
chosen as B.
Let's suppose, however, that the differences in share seen in this simulation are really greater than what
we would observe in the real world. Suppose that random forces come to bear in the actual market (e.g.
out-of-stock conditions, buyer confusion or apathy) and the shares (probabilities of choice) are really
flatter. We can often tune the results of market simulations using an adjustment factor called the
Exponent.
The table below shows results for the previous simulation under different settings for the Exponent:
Share of Choice under

Different Exponent Values
0.01 0.5 1.0 2.0 5.0
Product A 33.0% 20.2% 10.8% 2.4% 0.0%
Product B 33.3% 30.1% 24.0% 11.6% 0.7%
Product C 33.7% 49.7% 65.2% 86.0% 99.3%
Total 100% 100% 100% 100% 100%
The exponent is applied as a multiplicative factor to all of the utility part-worths prior to computing
shares. As the exponent approaches 0, the differences in share are minimized, and preference is divided
equally among the various product offerings. As the exponent becomes large, the differences in share
are maximized, with nearly all the share allocated to the single best product. (Given a large enough
multiplier, the approach is identical to the First Choice model, with all of the share given to a single
product.)
If you have solid external information (such as existing market share data) and have reason to expect
that conjoint shares should resemble market shares (see earlier assumptions), you may want to tune the
exponent within simulations. Or perhaps you have choice data from a holdout choice scenario included
in your conjoint survey. You may decide to tune the exponent so that simulated shares resemble these
holdout shares. If you do not have solid external information, you probably should not change the
exponent from the default value of 1.
Once the exponent is "set" for a data set, one typically does not change it from one simulation to the
next.
8.4.3 The Red-Bus/Blue-Bus Problem

While market simulators have proven eminently useful for simulating buyer behavior, one of the most
common simulation models (the Logit or Share of Preference model) has displayed a problematic result
as characterized by the oft-cited Red-Bus/Blue-Bus problem. The underlying property leading to this
problem is termed IIA, which is shorthand for "Independence from Irrelevant Alternatives." The basic idea
of IIA is that the ratio of any two products' shares should be independent of all other products. This
sounds like a good thing, and at first, IIA was regarded as a beneficial property.
However, another way to say the same thing is that an improved product gains share from all other
products in proportion to their shares; and when a product loses share, it loses to others in proportion to
their shares. Stated that way, it is easy to see that IIA implies an unrealistically simple model. In the
real world, products compete unequally with one another, and when an existing product is improved, it
usually gains most from a subset of products with which it competes most directly.
Imagine a transportation market with two products, cars and red busses, each having a market share of
50%. Suppose we add a second bus, colored blue. An IIA simulator would predict that the blue bus
would take share equally from the car and red bus, so that the total bus share would become 67%. But
it's clearly more reasonable to expect that the blue bus would take share mostly from the red bus, and
that total bus share would remain close to 50%.
It is important to note that some degree of IIA is appropriate and useful within market simulations. In
many markets, there is some degree of randomness to buyer behavior. It is not that people are
irrational, but that buyers must balance the costs of making a utility maximizing decision against the
costs of taking the time to make perfect decisions. It is quite reasonable for rational buyers to make
what on the surface may seem as haphazard decisions — especially for low-involvement purchases. A
similar or even duplicate offering could thus be expected to capture more share in the real world than a
rational simulation model might suggest.
In general, market simulation models based on disaggregate models of preference (utilities estimated at
the individual level) are more immune to IIA difficulties than aggregate models of preference (aggregate
logit, as offered by our CBC System). In addition to modeling respondent preferences at the individual
level, there are market simulation methods that help deal with IIA. These are described in the next
sections.
890 SSI Web v8
8.4.4 Market Simulator Models

The Market Simulator offers five models:
1. First Choice
2. Share of Preference
3. Share of Preference with Correction for Similarity
4. Purchase Likelihood
5. Randomized First Choice
This chapter provides a brief introduction to the models used in SMRT's market simulator. More detail is
provided in the section entitled: "Technical Details for Simulations."
First Choice
This option is the simplest and is sometimes referred to as the "Maximum Utility Rule." It assumes the
respondent chooses the product with the highest overall utility. The results for this option are invariant
over many kinds of rescalings of the utilities. In particular, one could add any constant to all the levels
for an attribute and/or multiply all part-worth utilities by any positive constant without affecting the shares
for the simulated products.
The First Choice model requires individual-level utilities, such as those generated by ACA, CVA, ACBC,
or CBC/HB. The First Choice model cannot be used with Latent Class or Logit runs for CBC.
The First Choice model is very intuitive and simple to implement. Its principal strength is its immunity to
IIA difficulties (red-bus/blue-bus problem). In other words, the First Choice rule does not artificially inflate
share for similar (or identical products). This property is especially important for product line simulations
or situations in which some product offerings are quite similar to others in the competitive set.
Its principal weakness is that the share of preference results are generally more extreme than the other
simulation models and one cannot adjust the steepness of the model using the exponent multiplier. We
have seen evidence that the First Choice model's predictions can often be more extreme (especially
when using CVA or ACA utilities) than market shares in the real world — especially for low involvement
purchases.
Another weakness is that it reflects information only about the respondent's first choice. Information
about the relative preference for the remaining products in the simulation is lost. As a result, standard
errors for the First Choice model are generally higher than with the other models offered in the Market
Simulator. Sample sizes need to be larger for First Choice modeling than the other approaches to
achieve equal precision of estimates.
We recommend using the First Choice model with ACA or CVA utilities if you have large sample sizes
and have determined through holdout choice validation or, preferably, through validation versus actual
market choices that the First Choice model accurately predicts shares better than the other approaches.
Share of Preference Models
The Share of Preference models (both with and without correction for product similarity) use the logit rule
for estimating shares. The product utilities are exponentiated and shares are normalized to sum to
100%.
The Share of Preference models result in "flatter" scaling of share predictions than the First Choice
model. In general, we expect that this flatter scaling more closely matches what occurs in the real
world. The Share of Preference models capture more information about each respondent's preferences
for products than the First Choice method. Not only do we learn what product is preferred, but we learn
the relative desirability of the remaining products. This means that standard errors of share predictions
are lower than the First Choice shares.
The Share of Preference model (without correction for product similarity) is subject to IIA, and can
perform poorly when very similar products are placed in competitive scenarios (e.g. line extension
simulations) relative to other less similar items within the same set. If using CBC under aggregate logit
simulations, the IIA problem is intensified. Under Latent class, the problem is somewhat reduced. With
individual-level utility models (ACA, CVA, ACBC, or CBC/HB), the problem is greatly reduced, but
nonetheless can still be an issue.
The Share of Preference with Correction for Product Similarity model can result in more valid predictions
when the competitive set includes products that have significant differences in similarities. However, this
model is not as theoretically complete as the Randomized First Choice method and can give unexpected
results, especially when conducting sensitivity simulations. The Randomized First Choice method has
been shown to handle product similarity issues in conjoint simulations better. For this reason, we
generally don't suggest using the Share of Preference with Correction for Product Similarity. It remains
an option in the Sawtooth Software simulator mainly for historical purposes.
Purchase Likelihood Model
The purchase likelihood model estimates the stated purchase likelihood for products you specify in the
simulator, where each product is considered independently. The likelihood of purchase projection is
given on a 0 to 100 scale.
If you intend to use the Likelihood of Purchase option in the Market Simulator, your data must be
appropriately scaled. The following estimation methods result in data appropriate for the purchase
likelihood option:
1. ACA, if calibration concepts have been asked and used in utility estimation.
2. CVA, if single-concept presentation was used, and the logit rescaling option used with OLS
regression.
3. CBC/HB, if calibration concepts have been asked and the Tools + Calibrate Utilities program
(from the CBC/HB standalone program) is used to rescale the utilities.
Any other procedure will result in simulations that are not an accurate prediction of stated purchase
likelihood. Also keep in mind that the results from the Purchase Likelihood model are only as accurate
as respondents' ability to predict their own purchase likelihoods for conjoint profiles. Experience has
shown that respondents on average exaggerate their own purchase likelihood.
You may use the Purchase Likelihood model even if you didn't scale the data using calibration concepts,
but the results must only be interpreted as a relative desirability index. Meaning: a value of "80" is higher
(more desirable) than a value of "60," but it doesn't mean that respondents on average would have
provided an 80% self-reported likelihood of purchase for that particular product.
The purchase likelihoods that the model produces are not to be interpreted literally: They are meant to
serve as a gauge or "barometer" for purchase intent. Under the appropriate conditions and discount
892 SSI Web v8
adjustments (calibration), stated intentions often translate into reasonable estimates of market
acceptance for new products.
Randomized First Choice
The Randomized First Choice (RFC) method combines many of the desirable elements of the First
Choice and Share of Preference models. As the name implies, the method is based on the First Choice
rule, and can be made to be immune to IIA difficulties. As with the Share of Preference model, the
overall scaling (flatness or steepness) of the shares of preference can be tuned with the Exponent.
Most of the theory and mathematics behind the RFC model are nothing new. However, to the best of our
knowledge, those principles have never been synthesized into a generalized conjoint/choice market
simulation model. RFC, suggested by Orme (1998) and later refined by Huber, Orme and Miller (1999),
was shown to outperform all other Sawtooth Software simulation models in predicting holdout choice
shares for a data set they examined. The holdout choice sets for that study were designed specifically
to include product concepts that differed greatly in terms of similarity within each set.
Rather than use the part-worth utilities as point estimates of preference, RFC recognizes that there is
some degree of error around these points. The RFC model adds unique random error (variation) to the
part-worth utilities and computes shares of choice in the same manner as the First Choice method.
Each respondent is sampled many times to stabilize the share estimates. The RFC model results in a
correction for product similarity due to correlated sums of errors among products defined on many of the
same attributes.
The RFC model is very computationally intensive, but with today's fast computers speed is not much of
an issue. With the suggested minimum of 100,000 total sampling iterations for a conjoint data set, it
takes only a few moments longer than the faster methods to perform a single simulation. According to
the evidence gathered so far on this model, we think it is worth the wait. The RFC model is appropriate
for all types of conjoint simulations, based on either aggregate- or individual-level utilities.
The most complete use of the RFC model requires tuning the appropriate amount of attribute- and
product-level error. By default, only attribute-level error is used in the simulator. This setting assumes
no product share inflation for identical offerings. If you have questions regarding tuning the RFC model
read the section covering the details of RFC or read the technical paper entitled "Dealing with Product
Similarity in Choice Simulations," available for downloading from our home page: http://
Note: By default, a correction for similarity (correlated attribute error) is applied to all attributes;
but starting in SMRT v4.14, the user can specify that certain attributes should not involve a
correction for similarity. We recommend you remove the correction for similarity for any Price
attribute. You do that under the Method Settings... button on the Scenario Specification dialog.
8.4.5 External Effects

The factors evaluated in a conjoint analysis study usually focus on product attributes, ignoring other
factors that affect market share. Consequently, the Market Simulator calculates a "preference share"
rather than a "market share," emphasizing that important factors for calculating market share are
missing from the model. Such missing factors include: the level and effectiveness of advertising, the size
and effectiveness of the sales force, the number of outlets where the product is sold, the length of time
the product has been on the market, and whether the product was first on the market.
The principal value of conjoint simulators is to indicate the kinds of changes that would make the most
favorable differences in customer preferences. There is no real need for them to have the appearance of
market shares. All the same, it is sometimes disconcerting for those viewing results to see shares of
choice that are vastly different from known market shares.
The External Effect option of the Market Simulator helps to account for factors outside the model. When
used properly, External Effects can lead to more realistic predictions. Even so, we recognize that the
method used in our simulator to adjust for external effects is a simple approach with certain
weaknesses. External Effects are introduced into the Market Simulator by applying a multiplicative
External Effects factor to each product's preference share. The factor ranges from 0 to 9999. A value of
1 introduces no effect, a value greater than 1 increases a product's preference share above what the
model would otherwise predict, a value less than 1 decreases the predicted preference below this value,
and a value of 0 eliminates the product from the model.
You set External Effect factors from the Scenario Specification dialog, by checking Apply External
Effects. When the Apply External Effects box is checked, an additional column in the product
specification grid appears (at the far right) in which you can type external effects.
Setting the factor for each product is subjective. The following procedure minimizes the level of
subjectivity:
1. Start by running a simulation on products currently on the market for which market
shares are known or can be estimated. Set the External Effects factor to 1 for all
products in this simulation.
2. Divide the actual market share for each product by the preference share predicted by the
model. This number becomes the External Effects factor for that product.
3. Re-run the Market Simulator with these factors applied. Check that the results now
reproduce the actual market shares for the products.
4. Set the External Effects factor for new products, using the External Effects factor for the
current products as benchmarks. A new product should have an External Effects factor
close to the ones for similar existing products (similar with respect to the "missing"
factors listed above).
5. The Market Simulator produces shares that new products are expected to achieve when
they have fully penetrated the market. The External Effects factor can be used to
estimate shares before full penetration is achieved. One way to do this is to estimate,
from past experience, the extent to which new products have reached full penetration at a
time t after their introduction. When you want a simulation for time t, multiply this
penetration factor by the one you have calculated using the steps above, and use this
result as the External Effects factor.
894 SSI Web v8
For example, suppose you are a car manufacturer and you are introducing a new sedan. And suppose
from your past sales you know the following penetration rate (fraction of expected monthly sales) for a
sedan:
Months After Introduction

Time 0 3 6 9 12 15 18
Market Penetration 0.05 0.20 0.50 0.80 0.90 0.95 1.00
Then, if the External Effects factor at full market penetration is expected to be 1.2, the External Effects
factor used as a function of time t would be:
Months After Introduction

Time 0 3 6 9 12 15 18
Market Penetration 0.06 0.24 0.60 0.96 1.08 1.14 1.20
As a final note, if you do need to apply external effect factors, we suggest you first investigate tuning the
Exponent to best fit target shares prior to invoking external effect adjustments.
8.4.6 Interpolating between Levels

Sometimes you may want to simulate a product that has a level in between two levels you measured.
This is called interpolation, and it is usually a reasonable thing to do when dealing with attributes that are
quantitative in nature (price, speed, etc.). The market simulator uses a straight-line interpolation
between adjacent part-worths.
Consider the following attribute:
Level # Level Text

1 5 pages per minute
The simulator lets you specify a level code between two levels you measured. For instance, level 2.5
represents the point half-way between 10 and 15 pages per minute, or 12.5 pages per minute. Twelve
pages per minute represents a value 40% of the way between 10 and 15 pages per minute. Therefore,
level 2.4 corresponds to 12 pages per minute. But there is a much easier way to interpolate between
levels. You may find it more convenient to refer to levels of quantitative attributes in the Market Simulator
using values other than the original level numbers. When you click Assign Level Values, you can
assign values to represent levels, such as:
Level # Level Value Level Text

1 5 5 pages per minute
It is much easier to specify products in the market simulator after recoding level numbers to values that
match the pages per minute. To simulate a product at 12 pages per minute, we would now specify "12"
for speed.
Three additional points are worth mentioning regarding recoding attribute levels for use in the simulator:
1. Interpolation only works correctly if all values are either ascending or descending for an
attribute.
2. If you want to omit an attribute from simulations, you must omit it for all products in the
simulation. You omit a level by specifying "N/A" (you type these characters) instead of a
numeric code.
3. If linear terms (coefficients) were estimated instead of part-worths for a quantitative

function such as price, you should make sure not to change the level values from those
that your consultant provided. For example, if the coded values for a linear price term
were -2, 0 and +2 (corresponding to zero-coded values representing $2, $4, and $6), it
would not be appropriate to change those to 200, 400, 600, as a utility effect 100 times
the appropriate effect would be applied in simulations. The Market Simulator zero-
centers (but doesn't standardize) all coded values for linear terms before multiplying by
the utility coefficients. Thus, values under Assign Level Values... of 2, 4, 6 would be
applied the same as 100, 102, 104 or 52, 54, 56. After zero-centering, these three
different coding schemes would be applied as -2, 0, +2 in simulations.
For more information about how the Market Simulator interpolates between levels when interaction terms
896 SSI Web v8
are involved, please see the Appendix entitled: "How the Simulator Interpolates between Levels."
8.4.7 The "None" Weight

If you are conducting simulations for a CBC study and the questionnaire contains choice tasks including
a "None" option, then the option of None can be included in simulations. However, the share of
respondents predicted to choose None will only be correct if the number of products in the simulation is
the same as the number of products in the CBC questionnaire. This is another problematic outcome of
the IIA rule and especially affects simulators built upon aggregate logit runs. With individual-level
modeling under CBC/HB, we have seen that the share of "None" is approximately correct irrespective of
the number of products used in the simulator (within a reasonable range of product alternatives).
The market simulator lets you specify a "None" weight from Scenario Specification dialog by clicking
the Advanced Settings... button. By default, the None weight is set to "0," which means that we do not
report a None percentage and we assume that all respondents are in the market and must "buy" a
product. To turn the None weight "On" with its default weight (as estimated via the respondent choices
in the CBC questionnaire), set the value to 1.0.
If you are using an aggregate logit run in your market simulations and are using a different number of
products than were reflected in your CBC questionnaire, you may need to consider the suggestions
below to deal with the problems of IIA and the "None" weight. If you are using individual-level utilities
from CBC/HB, then you these considerations are less an issue for you.
None Calibration for Aggregate Logit Models
Logit (Share of Preference) models tend to give too much share to products that are similar to one
another, and to penalize products that are unique (this is especially the case with aggregate logit
solutions). The None option does not have specified levels on any of the conjoint attributes, and is
therefore unique as compared to the products in the simulation.
If you do a simulation with a few products plus None, and then try another that includes those same
products plus others, you will find that the share predicted for None will be smaller when there are more
products. That may not be an accurate reflection of reality. If the respondents who chose None did so
because they would never buy a product in that category, then it would clearly be incorrect to assume
that fewer would choose none just because they are offered an array of more products. On the other
hand, if respondents are really candidates for those products, then one would expect the share choosing
None to decrease when respondents are offered a richer set of choices.
We know of no way to tell how the share choosing None should vary as the number of products in the
simulation changes, although we think some allowance should be made in those cases where more
products are in the simulation than in the original choice tasks.
For that reason, we provide the user with the capability of adjusting the percentage choosing None,
through multiplication by a constant supplied by the user. In general, we think the None weight should
be somewhere between 1.0 and the fraction n/t, where n is the number of products in the simulation and
t is the number of products in the average choice task. With individual-level utilities, less (or no)
adjustment for the None may be needed. With aggregate utilities, more adjustment may be appropriate.
898 SSI Web v8
8.5 Using the Market Simulator

8.5.1 The Market Simulator Dialog
Introduction
The previous two sections have dealt mainly with theory. This section discusses how to use the
software to simulate market choices after you have computed part-worth utilities or have imported utilities
from another system. We should note that context-sensitive on-line help for all screens and dialogs
within the software is available by pressing F1. You can access general help within SMRT by clicking
Help | Help Topics.
The Market Simulator Dialog
To start SMRT, click Start | Programs | Sawtooth Software | Sawtooth Software SMRT.
The first step in running SMRT's Market Simulator is to open a study. (We'll assume you have already
created a study by importing part-worth data into SMRT, as described earlier.) To open an existing
study, select File | Open, and browse to the directory containing the studyname.SMT file. Double-click
the studyname.SMT file. The main program menu is displayed.
To open the market simulator, click Analysis | Market Simulator. The Market Simulator dialog is
displayed:
This dialog contains two main lists: the list of available Utility Runs (which you cannot modify) and the
list of Simulation Scenarios (which you can add to and modify). A description of each of the settings and
buttons is provided below:
Banner If you are using part-worth utilities from a method other than logit, you
can specify the banner variable to use in reporting the results of the
simulation. For example, if you want to see the results split out
between large and small business, you can choose that variable as the
banner point.
Utility Runs A utility run contains a set of part-worths utilities estimated in a

particular way for a given number of respondents. For example,
perhaps you imported part-worth data from two separate HB runs for
use in the Market Simulator. The first set involved main effects only
and the second set additionally involved some interaction effects.
When you imported these two separate data sets, you provided a label
for each utility run by which it is referenced and selected during
simulation analysis. For example, you may have referred to the first
run as "Main Effects Only" and the second run as "With Interactions."
By default, the first utility run on the list is highlighted. Highlight the
900 SSI Web v8
utility run you want to use during simulations by clicking with the
mouse. Only one utility run can be selected at a time.
Simulation Scenarios When you first set up a study, the list of Simulation Scenarios is
blank. To specify a new scenario, click Add…. To choose an existing
scenario, check the scenario box with the mouse. You can check
more than one simulation scenario at a time. If you choose multiple
scenarios, all of the simulations will be conducted at the same time (in
batch mode). To edit an existing scenario, highlight it on the list and
click Edit.... To delete an existing scenario, highlight it on the list and
click Delete.
Output Precision This setting controls how many decimal places of precision are used
when reporting results.
Assign Level Values This button lets you define the numeric code to be applied to different
levels, such as for levels of price, speed or quantity. Assigning level
codes can make it easier to define products in simulation scenarios
and interpolate between measured levels.
Compute! Click this to perform a simulation for the selected Utility run and Simulation
Scenarios. The simulation results are displayed in the report window. If no
products have been specified in a Simulation Scenario, average part-worth
utilities and importances are displayed.
Clear Click this to clear the results from the report window.
Print… Prints the results displayed in the report window.
Save As… Saves the results displayed in the report window to a text file.
Close Exits the Market Simulator dialog.

8.5.2 Adding a Simulation Scenario

Usually the first step in using the market simulator is to define a "Base Case" scenario. A base case
typically reflects a current (or future) market scenario: your brand vs. the relevant competition. If there is
no relevant competition, or your conjoint study was designed to model only your product, the base case
may be a single product.
If you have defined a base case scenario or other scenarios, when you open the Market Simulator dialog,
these will appear in the list of Simulation Scenarios. You can view or edit an existing simulation
scenario by highlighting that scenario and clicking Edit….
To add a new scenario, click Add…. If a scenario or scenarios have already been defined, you are next
asked whether the new scenario should be a copy of an existing scenario (used as a starting point, so
that you don't have to re-enter common information again) or whether you want to start with a blank
template.
The Scenario Specification dialog is then displayed:
The cursor is initially active in the Name: field. You must provide a name to identify the scenario. This
name will appear in the Simulation Scenarios list in the Market Simulator dialog.
The Market Simulator lets you input the market scenario in a grid format, where the products are rows
(you can have up to 100), and the attributes are columns (you can have up to 30). For example, you
902 SSI Web v8
may have in mind to specify the following competitive scenario for a study that has three total attributes
with three levels each:
Attribute 1 Attribute 2 Attribute 3

Product (Brand) (Performance) (Price)
Product A 1 2 3
Product B 2 1 2
Product C 3 3 1
If you started with a blank template, only one line in the grid is displayed. A default label of "Product 1"
is provided, which you can edit by clicking this field with the mouse. To specify the attribute levels that
make up a product, click each cell within the grid and specify a level code. The attribute level labels and
level codes are displayed automatically in the Attribute Level View window as an aid. You must provide
a level value or an "N/A" for each cell in the grid under the attribute columns.
After you have fully defined the first product (row) of the grid, you are ready to add another product. You
can add and delete products from the simulation scenario using the Insert Product and Delete Product
buttons.
Insert The products in the market scenario are specified in the grid central to this
dialog.
Product Each row of the grid represents a product concept. When you click Insert
Product, a new row is added to the grid. You define a product concept by
typing in level values directly into the grid for the attributes describing that
product.
Delete Deletes the highlighted product concept.

Product
OK Saves any modifications you've made to the current dialog and returns you to
the main market simulation dialog.
Cancel Returns you to the main market simulation dialog without saving any changes.
8.5.3 Other Controls in Simulation Scenarios

Besides formulating the different competing products in the simulation scenario, there are a number of
other important decisions to be made. You must decide on the simulation method (the mathematical
model that converts respondent preferences into simulated shares of choice or purchase likelihood
estimates), respondent settings (if using individual-level data), the scaling of shares and output display
options.
Simulation Method Select the method used for simulations for the current scenario. The method
is Randomized First Choice by default. (See the section entitled,
"Market Simulator Models" for more details about the different
simulation methods.)
Operating Mode The default mode is Simulation. You can specify Sensitivity mode,
which is a way of processing in batch multiple simulation runs that
are variations of the current scenario. For example, you may wish to
see how varying all levels of price for Product 1 affects its share
relative to a fixed set of competitors. Rather than run the same
simulation over and over again for all levels of price for Product 1,
you can specify Sensitivity mode, and choose Product 1 as the
Sensitivity Product, and Price as the Sensitivity Attribute.
Respondents to Include If you are using part-worth utilities from a method other than logit,
you can specify the respondents to include in the simulation. If you
are using aggregate utilities (from Logit), this option is not available.
Respondent Weights If you are using individual-level utilities (from ACA, CVA, ACBC or
CBC/HB), you can specify the respondent weights to be applied in
the simulation. If you are using aggregate utilities (from Logit or
Latent Class), this option is not available.
Apply External Effects In some situations, you may want to adjust the share given to a
particular product by some external factor. For example, perhaps
that product is only available in half of the markets relative to the
other products. In that case, you can check the Apply External
Effects box. When this box is chosen, a new column appears in
the product specification grid labeled "External Effect." All external
effects are initially set to unity (1). To adjust the shares to account
for a given product only being available in half of the markets, you
might specify an external effect of 0.5 for that product, and leave all
other products with external effects of unity (1).
Advanced Settings When you click this button, a dialog appears that lets you define
additional settings:
Correlation Cutoff If you are using individual-level utilities generated by ACA,

CVA, ACBC or CBC/HB, a measure of fit is provided
indicating the relative consistency of each respondent's
utilities. You can select a value used as a cutoff for
excluding respondents that are at or below that threshold
for consistency.
904 SSI Web v8
Exponent The exponent tunes the overall "flatness" or "steepness" of the share results (see
discussion earlier regarding the Exponent). It has effect
only for the Share of Preference models and Randomized
First Choice.
"None" Weight If you are analyzing data for a CBC study that included a "None" option, this
lets you specify the weight applied to the None utility. By
default, it is zero (no "None" share computed). (See
discussion earlier regarding the None weight for more
direction.)
Unacceptable If you are using data from an ACA study and if you asked the
Extrapolation "Unacceptables" section, you can tune how low the utility should be for
unacceptable levels. This percentage determines the
degree of extrapolation when assigning the utility of
"Unacceptable" levels. The extrapolation is customized
for each respondent as a projection from the worst level of
each attribute. The constant subtracted from the worst
level within that attribute is determined as a percentage of
the average differences between best and worst levels
across all attributes for that respondent. The default is
20%. To use this setting properly, you should tune the
extrapolation to best fit holdout choices/observations.
Please note that the Unacceptable Extrapolation you set
has a direct bearing on the importance of each attribute,
and also alters the part-worth utilities.
Output Options
Individual Results to File
If you are running simulations using part-worth utilities generated by a

method other than logit, you can save the estimated shares of preference
for each individual to a text-only file for analysis in another software
package. This can be helpful, for instance, if you want to identify
particular respondents that prefer (or do not prefer) a certain product.
Display Utilities
Controls whether the utilities for each attribute level (or interaction terms)
are displayed in the output. The utilities displayed are rescaled by a
method called Zero-Centered Diffs. The diffs method rescales utilities so
that the total sum of the utility differences between the worst and best
levels of each attribute across attributes (main effects) is equal to the
number of attributes times 100. Note: the average part-worth attribute
utilities are influenced by the number of respondents in the simulation and
respondent weighting, but are not affected by the product specifications
you enter. Please see an earlier section entitled, "Interpeting Conjoint
Analysis Data" for more information about interpreting utilities.
Display Importances
Checking this option tells the Market Simulator to include a summary of
attribute importances in the simulation output. The importance of an
attribute is defined as its weight, or the maximum influence it can have on

product choice, given the range of attribute levels defined in the study. In
other words, an attribute's importance is an indicator of the amount of
influence an attribute may have in simulations. Attribute importances are
only accurate when individual- or segment-level utility data are available
(such as computed under Latent Class, ACA, CVA, ACBC or Hierarchical
Bayes). Aggregate importances from logit can be misleading if
respondents disagree about the order of preference of levels within an
attribute. Please see an earlier section entitled, "Interpeting Conjoint
Analysis Data" for more information about interpreting importances.
906 SSI Web v8
8.5.4 Running the Simulation

After you have specified the products and chosen the desired settings for your simulation scenario, you
are ready to compute the results. From the Market Simulator dialog, click Compute! Depending on
the model you choose and the number of respondents, it can take from a few moments to a few minutes
to compute. The results are displayed in the text report window. You can scroll up and down across the
report within this window. You can also cut-and-paste results into a word processor or spreadsheet
package.
Every time you click Compute!, results are appended to any existing data in the report window. Click
Clear to delete the contents of the window.
8.6 Practical Simulation Examples

8.6.1 Introduction
In this section, we'll discuss how to use the market simulator to answer some common strategic
questions introduced earlier:
Given a current competitive environment, what product should I offer to maximize interest
in my offering?
What is the relative price sensitivity of different brands? If I raise my price by 10%, how
will it affect my brand?
What portfolio of products can I offer to appeal to different market segments and maximize
overall share?
For the examples in this section, we'll assume the following three attributes, measured using three levels
each:
Brand:
1 BrandA
2 BrandB
3 BrandC
Style:
1 StyleA
2 StyleB
3 StyleC
Price:
1 $100
2 $150
3 $200
The results reported in this section are fictitious and not based on any particular data set.
908 SSI Web v8
8.6.2 New Product Introductions

Let's assume that your company is interested in entering a market that currently consists of just two
competitors. There are just three attributes that adequately describe the products and account for
preference in the market: Brand, Style and Price. The two players are:
1) BrandA, StyleA, $100

2) BrandB, StyleB, $200
Your company has developed a new style (StyleC) that you think may appeal to buyers, and you want to
investigate its potential with respect to the two existing products.
The first step, typically, is to simulate the existing market scenario. You use the market simulator to
define the two existing products:
Product Specifications:
Product Name Brand Style Price
"BrandA" 1 1 1
"BrandB" 2 2 3
For this simulation scenario, there are no similarities (in terms of common shared attribute levels)
between the products in the simulation. Therefore, the Share of Preference model is appropriate (though
it would also be appropriate to use the Randomized First Choice method). When you click Compute!,
the following shares of preference are displayed:
Shares of Preference for Products:

BrandA 64.3
BrandB 35.7
Note that the buyers in the simulation are all assumed to choose a product, so the shares of preference
across products in the simulation sum to 100%.
Let's assume that you have actual market share information about these two brands. You note that the
shares reported above do not necessarily match the actual market shares. You accept this, however,
recognizing that many factors influence market shares in the real world that cannot be captured through
conjoint analysis. You are principally interested in relative preferences, assuming that the marketplace
is an equal playing field: equal distribution, awareness, effectiveness of sales force, and equilibrium long-
range demand.
In the second stage of this simulation example, we'll define a new scenario that includes your company's
proposed product: BrandC, StyleC, $150. You add another row to the simulation specification grid:
"BrandA" 1 1 1
"BrandB" 2 2 3
"BrandC" 3 3 2
After clicking Compute!, the following shares are displayed:

BrandA 42.5
BrandB 21.3
BrandC 36.2
You note that BrandA is still the most preferred product, but that your brand is preferred to BrandB.
Like any market research statistic computed from samples, shares of preference are not estimated
without error. It is common to estimate a confidence interval, to gain a feel for the degree of uncertainty
due to sampling and measurement error associated with a given share of preference. If your simulator is
based upon part-worth utilities generated by a method other than logit, the Market Simulator lets you add
the standard errors of the shares of preference to the simulation report. Let's assume that the standard
error reported for BrandC for the simulation above was 1.53. The 95% confidence interval is computed by
adding plus and minus 1.96 times the standard error to the estimated share of preference. In this
example, the 95% confidence interval is 36.2 plus and minus (1.96)(1.53) = 3.0 share points, or the
interval [33.2, 39.2].
You next may ask yourself what price you would need to charge to capture the same relative preference
as BrandA. To simulate this, you lower the price slightly for your brand. The market simulator lets you
interpolate between levels, so you can investigate even the smallest of price changes. As a first step,
you decide to lower the price to $130 for BrandC (while holding the specifications for BrandB and BrandA
constant). The new simulated shares are:

BrandA 39.2
BrandB 19.0
BrandC 41.8
You have overshot the mark (BrandC's share exceeds BrandA's share), so you try a slightly higher price
than $130 and click Compute! once more. You make repeated attempts until BrandA and BrandC's
shares are equal. Let's assume that after a few more attempts, you discover that the price that makes
your company's offering match the share of preference of the market leader is $136. Another way of
thinking about this finding is that your proposed product commands a $136 - $100 = $36 premium over
BrandA's product. (Respondents are indifferent between BrandA, StyleA at $100 and BrandC, StyleC at
$136).
910 SSI Web v8
8.6.3 Estimating Demand Curves and Elasticities

Before outlining simulation strategies for pricing research, we should give a few notes of caution. The
ACA system is often not a very good pricing research tool; CVA and particularly the CBC system are
considered stronger approaches for most markets.
If using CBC for pricing research, we recommend the use of Latent Class or especially individual-level
computation methods, such as the CBC/HB (Hierarchical Bayes) module.
For estimating demand curves using market simulations, the Share of Preference method is appropriate.
Randomized First Choice may also be selected, if you believe some correction for product similarity
should result if products carry the same prices (however, the use of Randomized First Choice for
generating demand curves can often result in strange shifts in the curves around the "average" price, if
brands are initially set at the same average price).
We will build upon the previous example during this section. We previously computed shares of
preference for three products defined using the following attribute level codes:
"BrandA" 1 1 1
"BrandB" 2 2 3
"BrandC" 3 3 2
You may recall that price levels 1, 2 and 3 correspond with the following prices $100, $150, and $200.
The shares of preference for the products as defined above were:

BrandA 42.5
BrandB 21.3
BrandC 36.2
Let's assume we wanted to estimate a demand curve for your company's offering: BrandC, in the context
of the current competition and prices. We do this through "sensitivity analysis." The Market Simulator
offers an automatic way to conduct sensitivity analysis, but so you understand the process, we'll
describe it in separate steps.
Recall that we measured three distinct levels of price $100, $150 and $200. Note that we've already
computed the share of preference for BrandC when it is offered at $150 (21.3).
To estimate the demand curve for BrandC, we'll need to conduct two additional simulations: a simulation
with BrandC at the lowest price ($100), and a simulation with BrandC at the highest price ($200). For
each of these simulations, we'll hold the BrandA and BrandB product specifications constant.
To estimate BrandC's share at the lowest price ($100), we use the following product specifications:
"BrandA" 1 1 1
"BrandB" 2 2 3
"BrandC" 3 3 1
We click Compute!, and the following shares are reported:


BrandA 33.9
BrandB 15.6
BrandC 50.5
We record BrandC's share (50.5), and proceed to the next step. To estimate BrandC's share at the
highest price ($200), we use the following product specifications:
"BrandA" 1 1 1
"BrandB" 2 2 3
"BrandC" 3 3 3
We click Compute!, and the following shares are reported:
Shares of Preference for Products

BrandA 49.2
BrandB 26.9
BrandC 23.9
From these three separate simulation runs, we now have all the information we need to plot a demand
curve for BrandC, relative to the existing competitors and prices. Assuming that BrandA and BrandB are
held constant at current market prices, the relative shares of preference for BrandC at each of the price
points within the measured price range are:
Share of
Price Preference
$100 50.5
$150 36.2
$200 23.9
We have demonstrated how to estimate a demand curve for BrandC, relative to the existing competitors
at current market prices. If the goal is to estimate demand curves for all brands in the study, the usual
procedure is to record the share for a brand at each price level while holding all other brands at the
average (middle) price. It is often interesting to plot these demand curves and look at the patterns of
price sensitivity between brands and the different slope of the curves from one segment of the curve to
the next. It is also common to want to characterize the degree of price sensitivity using a single value,
referred to as an elasticity. The Price Elasticity of Demand (E) is defined as:
E= % Q
% P
where,
"% " means "percent change in,"
Q is defined as quantity demanded, and
P refers to the price.
If the brand or product follows the law of supply and demand (most products do), price increases lead to
decreases in quantity demanded, and the elasticity is negative. The larger the absolute value of the
elasticity, the more price sensitive the market is with respect to that brand or product.
Using the "midpoints" formula, we can compute the average price elasticity of demand across the
demand curve for BrandC:
912 SSI Web v8
E = q2 – q1 ÷ p2 – p1
(q1+q2)/2 (p1+p2)/2
23.9 – 50.5 ÷ 200 – 100

(50.5+23.9)/2 (100+200)/2
-0.715 ÷ 0.667
= -1.073
Another way to compute the average price elasticity of demand (which can be more accurate if more
than two price points along the curve have been estimated) is the "log-log" regression. One takes the
natural log of prices and shares and regresses the log of share on the log of price (you can do this within
a spreadsheet). The resulting beta is the average price elasticity of demand.
As with all conjoint simulation results, the resulting elasticities from conjoint simulators are interpreted
bearing in mind some assumptions. In particular, the degree of noise within the conjoint data is
particularly relevant. For example, if the respondents to the conjoint survey answered in a more
haphazard way compared to buyers in the real world, the price elasticities estimated from conjoint
simulations may be uniformly understated (too insensitive). Also, you should recognize that changing
the Exponent will shift the elasticities. Even if this is the case, the relative price sensitivities for brands
are still useful.
8.6.4 Designing Products to Appeal to Unique Market Segments

Customizing products to appeal to target segments or even individuals is a common theme in marketing.
Many companies dedicate significant resources to developing a portfolio of products that it hopes will
appeal to unique segments. For line extensions, the challenge for any company is to design new
product(s) that take share from its competitors without stealing an unacceptable amount of share from
products within its existing line.
One common approach to designing an effective line extension is to use the conjoint data to segment
the market into latent (not observed) market segments (sometimes referred to as clusters) that have
similar preferences. These segments are termed "latent," because they are not simply delineated based
on an explicit variable such as gender, income or company size. Rather, the underlying segments are
revealed through a statistical segmentation technique such as cluster analysis or latent class modeling.
Segments are formed with the goal of maximizing the differences in preference between groups while
minimizing the differences in preference within groups. Once these latent segments have been identified,
one can profile them in terms of other variables in the survey (i.e. demographics, usage or media habits).
If simulations are based on Latent Class (for CBC data) or if you used a cluster technique on individual-
level part-worths, you can conduct simulations "by segment." If using a Latent Class utility run, this
happens automatically. If based on a cluster analysis, the segment membership information can be
made available to you as a "banner point."
For example, let's assume that a cluster analysis revealed three relatively different segments for the
hypothetical example we've been using throughout this section. Let's also assume that you merged a
variable containing the segment membership information into the data set as a "banner" variable called
"SEGMENT." If you select SEGMENT as the banner variable and click Compute!, the market simulator
displays the part-worth utilities and importances for each segment.
By examining the part-worths and importances for each group, you can gain insight into the product
features that might appeal to each. You also should bear in mind the size of each segment, as this
represents its demand potential. Consider the following part-worth utility preferences:
Part-worth Utility Preferences
Segment1 Segment2 Segment3

n = 128 n = 283 n = 216
BrandA 39 -51 -44

BrandB 5 39 -29
BrandC -44 12 73
StyleA 61 -52 -34

StyleB -23 45 -9
StyleC -38 7 43
$100 56 55 50
$150 7 2 6
$200 -63 -57 -56
914 SSI Web v8
We can study the part-worths to learn about the differences among the segments. We can also use
these preferences to simulate market choices for the following market scenario:
"BrandA" 1 1 1
"BrandB" 2 2 3
"BrandC" 3 3 2
Shares of Preference for Products, by Market Segment:

Segment1 Segment2 Segment3 Total
n = 128 n = 283 n = 216 627
BrandA 84.8 21.5 22.2 34.7

BrandB 7.4 40.0 14.2 24.5
BrandC 7.8 38.5 63.6 40.8
(Note that these shares do not match the shares reported for earlier examples in this section. Since
these results are for illustration only, no significance should be attached to this difference.)
Let's assume your company produces BrandC with StyleC at $150. Your total share of preference is
40.8%. We see from the simulation by segment that yours is the most preferred product within Segment
3, and the second-most preferred product in Segment 2. BrandA clearly dominates Segment1 (the
smallest segment).
Let's assume that your company was interested in offering an additional product. We could examine the
table of part-worth utilities presented earlier as a first step in formulating hypotheses about what
additional product might be successful.
Starting in order, you may first consider Segment 1, but this segment does not seem to offer many
opportunities for your brand. BrandA offering StyleA at a low price has got this relatively small segment
nearly wrapped up, and this segment doesn't seem very receptive to your brand: BrandC.
You next consider Segment 2, which seems to represent a better opportunity for your brand. It is a
relatively large segment that prefers BrandB, but also seems receptive to BrandC. Note also that
Segment 2 strongly prefers StyleB, but your company currently offers only StyleC. By offering a StyleB
product, you might be able to convert some current BrandB customers from within Segment 2 to your
product line.
You currently dominate Segment 3 and should probably not consider designing another product to
appeal to this segment, since a good deal of the possible share to be gained from a new product would
be taken from your existing product within that segment.
Let's simulate what happens if in addition to your current product (BrandC, StyleC, $150), you offer
another product (BrandC, StyleB, $200). Note that since this simulation involves products that have
similar definitions (there are two BrandC products), you should probably use a technique that adjusts for
product similarities, such as Randomized First Choice.
Shares of Preference for Products, by Market Segment:

Segment1 Segment2 Segment3 Total
n = 128 n = 283 n = 216 627
BrandA 82.2 17.2 18.6 31.0
BrandB 7.2 32.0 11.9 20.0
BrandC (old) 6.8 27.7 47.8 30.4
BrandC (new) 3.8 23.1 21.7 18.7
The new product has somewhat cannibalized the existing product, reducing its share from 40.8 (see the
previous simulation) to 30.4, but has resulted in a relative overall gain of 1 - [(30.4 + 18.7)/40.8] = 20% in
preference.
For line extension simulations you conduct, the answer will likely not be so clear and the process so
direct as we've shown here. You'd certainly want to investigate other product configurations to make
sure you weren't overlooking even better opportunities to enhance share. You would also want to
consider the cost implications of different options for line extensions. Also, you would probably want to
conduct sensitivity analysis for the new product with respect to price, to determine a strategic price point
(given your costs and market share goals).
Viewing the preferences and shares by segment is not required in designing an effective line extension.
However, viewing the separate market segments can help you more quickly recognize patterns of
preference, size the different segments of the market, and thus more easily arrive at a good solution.
This exercise of viewing segment-based preferences and designing products to fill heterogeneous needs
is a useful approach. However, it would seem more efficient to let an automated search algorithm find an
optimal product or set of products rather than to proceed manually. Sawtooth Software's SMRT
Advanced Simulation Module includes automated optimization search capability.
916 SSI Web v8
8.7 Technical Details for Simulations

8.7.1 First Choice Method
This option is the simplest and is often referred to as the "Maximum Utility Rule." It assumes the
respondent will choose that product with the highest overall utility. All other less preferred products
receive zero share for this individual. The results for the First Choice model are invariant over many kinds
of rescalings of the utilities.
If two or more products are exactly the same in terms of utility (and more preferred than any other
product in the simulation scenario), the respondent's choice is split evenly amongst them.
8.7.2 Share of Preference Options

These options do not assume that the respondent always chooses the product with highest utility.
Instead, they estimate probability of choosing the simulated product, arriving at a "share of preference"
for the product.
This is done in two steps:
1. Subject the respondent's total utilities for the product to the exponential transformation (also
known as the antilog): s = exp(utility).
2. Rescale the resulting numbers so they sum to 100.
Suppose two products, A and B, have total utilities 1.0 and 2.0. Then their shares of preference would
be computed as follows:
product share of
utility exp(utility) preference
A 1.0 2.72 26.9
B 2.0 7.39 73.1
(For the Share of Preference with Correction for Product Similarity, shares of preference are modified by
further computations.)
Unlike the First Choice option, the scaling of utilities can make a big difference with the Share of
Preference Options. Consider what happens if we multiply the utilities by constants of 0.1 or 10.0:
For multiplier of 0.1

product share of
A 1.0 1.105 47.5
B 2.0 1.221 52.5
For multiplier of 10.

product share of
A 10.0 22026 100.005
B 20.0 4.85E08 99.995
If we multiply the utilities by a small enough constant, the shares of preference can be made nearly
equal. If the utilities are made small enough, every product receives an identical share of preference,
irrespective of the data.
If we multiply the utilities by a large enough constant, the shares of preference can be made equivalent to
the First Choice model.
It is apparent that scaling of the utilities can make a big difference with the Share of Preference Models.
Most of Sawtooth Software's utility estimation methods result in utilities appropriate for use with Share of
Preference models. We strongly suggest you include holdout (fixed) choice tasks in your conjoint
questionnaires to use for calibration, or check the simulation predictions against actual market share.
The Market Simulator lets you scale the utilities within the Share of Preference option at the time the
simulation is done. This is accomplished by a parameter called the "exponent" that you can set when
918 SSI Web v8
preparing for simulations. The default value of the exponent is 1. It has the same effect as the multiplier
illustrated immediately above. The exponent can be used to adjust the sensitivity of the simulation
results so that it more accurately reflects holdout choices, or actual market behavior.
A smaller exponent causes small shares to become larger, and large shares to become smaller — it has
a "flattening" effect. In the limit, with a very small exponent every product receives the same share of
preference.
A large exponent causes large shares to become larger, and small shares to become smaller — it has a
"sharpening" effect. In the limit, a very large exponent produces results like those of the First Choice
option.
If you have solid external information (such as existing market share data) and have reason to expect
that conjoint shares should resemble market shares, you may want to tune the exponent within
simulations. If you do not have solid external information, you probably should not change the exponent
from the default value of 1.
Share of Preference Model with Correction for Product Similarity
Share of preference models that do not consider similarities among products have a serious problem: if
an identical product is entered into the simulation twice, it can receive up to twice as much total share of
preference as it would when entered only once. Although no researcher would make the mistake of
entering the same product twice, the principle is still troublesome. Products differ in similarity to one
another, and "plain" share of preference models tend to give too little share to relatively unique products.
The third choice model in the Market Simulator includes a correction to prevent the preference shares of
similar products from being overstated. The correction is based on each product's total similarity with
other products. The basic idea behind this correction was suggested by Richard D. Smallwood of
Applied Decision Analysis, although we are responsible for the details of
its implementation.
The procedure is:
1. For the n products in a simulation, an n x n similarity matrix is computed, with 1's indicating
complete similarity, 0's indicating total lack of similarity, and fractional values for differing
degrees of similarity:
First a "dissimilarity" matrix is computed. Consider a scale for each attribute where its levels
are coded 10, 20, 30, and so on. The dissimilarity of a pair of products on an attribute is taken
as the absolute difference between their codes, but with a maximum of 10. (This allows
"continuous" and "categorical" attributes to be treated in the same way.) The total
dissimilarity between two products is the sum of their dissimilarities over all attributes. Two
products differing by an entire level on each attribute are maximally dissimilar.
Next, total dissimilarities are rescaled by a constant so the maximum possible is 3.0, rather
than 10 times the number of attributes.
Dissimilarities are then converted to similarities by a negative exponential transformation. At

this point the minimum possible similarity is exp(-3) ~ .05, achieved if two products have
completely different levels on every attribute, and the maximum possible similarity is exp(0) =
1, achieved only if they have identical levels on all attributes.
Finally, the similarities are subjected to a further rescaling that sets the minimum to 0 and the
maximum to 1.
2. Column totals of the similarity matrix are calculated to get a vector of "total similarities." The
smallest possible value is 1.0, which is found if a product is maximally dissimilar to all others
and similar only to itself. If two products are identical but maximally dissimilar to all others,
those products each have values of 2.0.
3. For each respondent, shares of preference are divided by corresponding "total similarities,"
and then renormalized to have sum of unity. This has the effect of reducing shares for
products that are relatively similar to others, and increasing shares for products that are
relatively unique. In the limit, where two or more products are identical and totally unlike any
others, they divide among themselves the share that each product would have if the others
were not present.
920 SSI Web v8
8.7.3 Purchase Likelihood Option

The purchase likelihood option estimates the stated purchase likelihood for products you specify in the
simulator. Each product is considered independently. If you intend to use the Likelihood of Purchase
option in the Market Simulator, your data must be appropriately scaled. The following estimation
methods result in data appropriate for the purchase likelihood option:
1. ACA, if calibration concepts have been asked.
2. CVA, if single-concept presentation was used, and the logit rescaling option used with OLS
regression.
3. CBC/HB, if calibration concepts have been asked and the Tools + Calibrate Utilities program
(from the CBC/HB standalone program) is used to rescale the utilities.
Any other procedure will result in simulations that are not an accurate prediction of stated purchase
likelihood. Also keep in mind that the results from the Purchase Likelihood model are only as accurate
as respondents' ability to predict their own purchase likelihoods for conjoint profiles. Experience has
shown that respondents on average exaggerate their own purchase likelihood.
You may use the Purchase Likelihood model even if you didn't scale the data using calibration concepts,
but the results must only be interpreted as a relative desirability index.
The purchase likelihoods that the model produces are not to be interpreted literally: They are meant to
serve as a gauge or "barometer" for purchase intent.
This model provides a means of simulating a product category with only a single product. The other
three choice models are comparative models that require at least two products to be specified in a
simulation. Likelihoods are estimated for product concepts by summing scaled utilities and estimating
probabilities with the following transformation:
p= eu
_____
1 + eu
where,
p = probability of purchase
e = the constant e
u = product utility
8.7.4 Randomized First Choice

The Randomized First Choice (RFC) method combines many of the desirable elements of the First
Choice and Share of Preference models. As the name implies, the method is based on the First Choice
rule, and can be made to be immune to IIA difficulties. As with the Share of Preference model, the
overall scaling (flatness or steepness) of the shares of preference can be tuned.
Most of the theory and mathematics behind the RFC model are nothing new. However, to the best of our
knowledge, those principles have never been synthesized into a generalized conjoint/choice market
simulation model. RFC, suggested by Orme (1998) and later refined by Huber, Orme and Miller (1999),
was shown to outperform all other Sawtooth Software simulation models in predicting holdout choice
shares for a data set they examined. The holdout choice sets for that study were designed specifically
to include product concepts that differed greatly in terms of similarity within each set.
Rather than use the utilities as point estimates of preference, RFC recognizes that there is some degree
of error around these points. The RFC model adds unique random error (variability) to the utilities and
computes shares of preference in the same manner as the First Choice method. Each respondent is
sampled many times to stabilize the share estimates. The RFC model results in a correction for product
similarity due to correlated sums of errors among products defined on many of the same attributes. To
illustrate RFC and how correlated errors added to product utilities can adjust for product similarity,
consider the following example:
Assume two products: A and B. Further assume that A and B are unique. Consider the following
product utilities for a given respondent:
Avg. Product Utilities

A 10
B 30
If we conduct a first choice simulation, product B captures 100% of the share:
Avg. Product Utilities Share of Choice

A 10 0%
B 30 100%
However, let's assume that random forces can come to bear on the decision for this respondent.
Perhaps he is in a hurry one day and doesn't take the time to make the decision that optimizes his
utility. Or, perhaps product B is temporarily out-of-stock. Many random factors in the real world can
keep our respondent from always choosing B.
We can simulate those random forces by adding random values to A and B. If we choose large enough
random numbers so that it becomes possible for A to be sometimes chosen over B, and simulate this
respondent's choice a great many times (choosing new random numbers for each choice iteration), we
might observe a distribution of choices as follows:

A 10 25.0%
B 30 75.0%
(Note: the simulation results in this section are for illustration, to provide an intuitive example of RFC
modeling. We assume shares of preference are proportional to product utilities.)
922 SSI Web v8
Next, assume that we add a new product to the mix (A'), identical in every way to A. We again add
random variability to the product utilities so that it is possible for A and A' to be sometimes chosen over
B, given repeated simulations of product choice for our given respondent. We might observe shares of
preference for the three-product scenario as follows:

A 10 20.0%
A' 10 20.0% (A + A' = 40.0%)
B 30 60.0%
Because unique (uncorrelated) random values are added to each product, A and A' have a much greater
chance of being preferred to B than either one alone would have had. (When a low random error value is
added to A, A' often compensates with a high random error value). As a simple analogy, you are more
likely to win the lottery with two tickets than with one.
Given what we know about consumer behavior, it doesn't make sense that A alone captures 25.0% of the
market, but that adding an identical product to the competitive scenario should increase the net share for
A and A' from 25.0% to 40.0% (the classic Red Bus/Blue Bus problem). It doesn't seem right that the
identical products A and A' should compete as strongly with one another as with B.
If, rather than adding uncorrelated random error to A and A' within each choice iteration, we add the
same (correlated) error term to both A and A', but add a unique (uncorrelated) error term to B, the shares
computed under the first choice rule would be as follows:

A 10 12.5%
A' 10 12.5% (A + A' = 25.0%)
B 30 75.0%
(We have randomly broken the ties between A and A' when accumulating shares of choice). Since the
same random value is added to both A and A' in each repeated simulation of purchase choice, A and A'
have less opportunity of being chosen over B as compared to the previous case when each received a
unique error component (i.e. one lottery ticket vs. two). The final utility (utility estimate plus error) for A
and A' is always identical within each repeated first choice simulation, and the inclusion of an identical
copy of A therefore has no impact on the simulation result. The correlated error terms added to the
product utilities have resulted in a correction for product similarity.
Let's assume that each of the products in this example was described by five attributes. Consider two
new products (C and C') that are not identical, but are very similar—defined in the same way on four out
of five attributes. If we add random variability to the part-worths (at the attribute level), four-fifths of the
accumulated error between C and C' is the same, and only one-fifth is unique. Those two products in an
RFC simulation model would compete very strongly against one another relative to other less similar
products included in the same simulation. When C received a particularly large positive error term added
to its utility, chances are very good that C' would also have received a large positive error term (since
four-fifths of the error is identical) and large overall utility.
RFC Model Defined
We can add random variability at both the attribute and product level to simulate any similarity correction
between the IIA model and a model that splits shares for identical products:
Ui = Xi (ß + Eattribute) + Eproduct
where:
Ui = Utility of product i for an individual or homogenous segment at a moment in time

Xi = Row of design matrix associated with product i
ß = Vector of part-worths
Eattribute = Variability added to the part-worths (same for all products)
Eproduct = Variability (Gumbel) added to product i (unique for each product)
Repeated draws are made to achieve stability in share estimates, computed under the First Choice rule.
In RFC, the more variability added to the part-worths, the flatter the simulations become. The less
variability added to part-worths, the more steep the simulations become. Under every possible amount of
attribute variability (and no product variability), shares are split exactly for identical products, resulting in
no "inflation" of net share. However, there may be many market scenarios in which some share inflation
is justified for similar products. A second unique variability term (distributed as Gumbel) added to each
product utility sum can tune the amount of share inflation, and also has an impact on the flatness or
steepness of the overall share results. It can be shown that adding only product variability (distributed as
Gumbel) within the RFC model is identical to the familiar logit model (Share of Preference Model).
Therefore, any degree of scaling or pattern of correction for product similarity ranging between the First
Choice model and Share of Preference can be specified with an RFC model by tuning the relative
contribution of the attribute and product variability.
The exponent also can play a role in RFC, similar, but not identical to, product variability. Decreasing
the exponent (multiplying the utility estimates by a value less than unity) decreases the variance of the
utility estimates relative to the variance of the random variation added within RFC simulations, in turn
making simulated shares flatter. There is a subtle difference between increasing product variability and
lowering the exponent, though both result in a flattening of shares. If only attribute variation is being used
in an RFC simulation, decreasing the exponent flattens the shares, but the overall model still does not
reflect the IIA property. Adding product variability, however, flattens the shares and causes the RFC
model to reflect at least some degree of IIA behavior. Though the exponent is not required to simulate
different patterns of correction for product similarity and scaling, in the next section we show that it is
useful to retain the exponent adjustment from an operational point of view.
The RFC model is very computationally intensive. With the suggested minimum of 100,000 total
sampling iterations for a conjoint data set, the results tend to be fairly precise. But, if you have dozens
of products in the simulation scenario, some product shares can become quite small, and greater
precision would be needed. You can increase the precision by increasing the number of sampling
iterations. The RFC model is appropriate for all types of conjoint simulations, based on either aggregate-
or individual-level utilities. It provides the greatest benefits in the case of aggregate (logit and Latent
Class) models, which are more susceptible to IIA difficulties than individual-level models.
If you plan to use the RFC model with individual-level utilities to compute reasonably stable estimates of
share at the individual level, you should sample each respondent at least 5,000 times, and preferably
924 SSI Web v8
more. This can take a good deal of computing time. The software asks how many total sampling
iterations to use, where the number refers to the total iterations across all respondents. If you have 500
respondents and want each respondent sampled 5,000 times, you should specify a total of 2,500,000
iterations.
The greatest complexity of the RFC model from an operational point of view is that the magnitude of the
attribute variability multiplier must be adjusted whenever the number of products or number of attributes
on which products differ changes across simulations to maintain comparable scaling of shares of
preference. Our implementation of the RFC model includes an auto-calibrating attribute variability
multiplier (developed through Monte-Carlo simulations) so that this issue is transparent to the user.
Ideally, you will provide your own validation data, in the form of holdout concepts or actual market shares,
to permit calibration of the attribute and product variability terms. This allows you to best fit the scaling
of shares and degree of correction for product similarity appropriate for the specific market you are
modeling. If you are using the auto-calibrating attribute variability multiplier, you can adjust the relative
contribution of attribute and product variability by manipulating only the exponent and the product
variability multiplier. For example, if you want to decrease the correction for product similarity by adding
some product variability and adding less attribute-level variability to the utilities, you could increase the
product variability multiplier (say, from 0.0 to 0.2). After doing so, the resulting shares of preference
would be flatter than before the adjustment. (The more variation added to utilities, the flatter the resulting
shares.) To "sharpen" the shares again, you would adjust the exponent upward. To adjust the amount
of correction for similarity, you should have outside information about choices for holdout choice tasks, or
actual market share, where the product concepts involved had significantly different degrees of product
similarity within each set.
For more information about the RFC model, please refer to Huber, Orme and Miller's paper entitled,
"Dealing with Product Similarity in Choice Simulations," available for downloading from our home page:
http://www.sawtoothsoftware.com.
8.8 Appendices for Simulation Topics

8.8.1 Importing Conjoint Data from Text-Only Files
The Market Simulator can import conjoint part-worth utility data from any of the files generated by
Sawtooth Software's current conjoint analysis systems. If you are importing data that you generated
yourself, you should create a text-only file matching the ".HBU" format. If your utility file includes linear
terms, you must also provide a .VAL file. The formats for these two files are described below.
The ".HBU" Import Format
This format gets its name from the CBC/HB system, which creates text-only files containing conjoint
part-worth estimation with the extension ".HBU". It is the most flexible of the Sawtooth Software part-
worth file formats, accommodating:
Part-worth or linear estimation of attribute effects

Main effect and optional first-order interactions
Individual-level records, with a measure of fit (e.g. correlation) that can be used as a filter for
exclusion
The .HBU file should be created as a text-only, space or tab-delimited format.
A Simple, Main Effects Example
Here is an example file for a simple study with 3 attributes (main effect part-worth estimation only) and a
total of 2 respondents:
3 0 11 1 1
3 4 4
1 0 0
0 1 0
0 0 1
1 1 Brand A
1 2 Brand B
1 3 Brand C
2 1 Style A
2 2 Style B
2 3 Style C
2 4 Style D
3 1 $8
3 2 $10
3 3 $15
3 4 $20
1001 778 0 11 0
-1.323 0.667 0.656 -.520 0.836 0.310 -0.626 1.271 0.652 -.529
-1.394
1003 540 0 11 0
-1.042 0.235 0.807 -.239 -0.211 0.310 0.140 1.060 0.254 -.095
-1.219
The data structure consists of a header section (the first 16 lines in this example), followed by a
respondent header (one per respondent) and preference values for each respondent.
926 SSI Web v8
The first line contains the number of attributes (3), whether a "None" utility is included (1 if yes, 0 if no),
the total number of parameters estimated for each individual (11), and the numbers 1 and 1. (These last
two values are just to conform to the file format for the Latent Class module, which uses the same
header format.)
The second line contains the number of estimated terms for the main effects for each attribute (3, 4, 4).
Following is a line for each attribute, each with as many entries as there are attributes. This is an
attribute-by-attribute matrix of ones and zeros (or minus ones) that indicates which effects were
estimated. This example represents main effects only and is indicated by ones on the diagonal. If
interactions were included (they weren't for this example), they would be indicated by a one in other
positions. Linear variables (there weren't any for this example) are indicated by negative ones on the
diagonal.
Following are labels, one for each parameter estimated. There are two leading space-delimited integers
on each line, indexing the attributes and levels. These attribute/level labels are in the same order as the
parameter estimates that follow in the file, and serve to identify them. If interaction parameters were
estimated, then this list will include a label for each term.
The record for each respondent starts with a line that contains:
Respondent number (respondents 1001 and 1003 are found in this example). Respondent numbers
are limited to 9 total characters.
A measure of fit such as R-squared or RLH can be included and used as a filter in simulations.
The range of acceptable values is from 0 to 1000. If no measure of fit is available, you can set this
to an integer within that range such as "0".
A value of zero (this is a carry-over from other Sawtooth Software formats)
The total number of parameter estimates per respondent (in this case, 11 part-worths)
A value of -1 if a "none" utility exists, or a zero otherwise.
This respondent header is followed by the parameter values for that respondent, in the same order as the
labels in the header.
Even though the example above uses utility values with just three decimal places of precision, up to 15
significant digits of precision may be used. The utility values can be formatted on a single or multiple
lines (hard return characters within a respondent record are ignored). However, each respondent's record
should begin a new line.
A Complex .HBU File Example
If your conjoint part-worths include linear terms (such as a single coefficient representing the main effect
for Price), you must supply two files: .HBU and .VAL (format described below). The example below
illustrates a linear term and the inclusion of a first-order interaction.
3 1 12 1 1
3 4 1
1 0 1
0 1 0
1 0 -1
1 1 Brand A
1 2 Brand B
1 3 Brand C
2 1 Style A
2 2 Style B
2 3 Style C
2 4 Style D
3 1 Price
Brand A x Price
Brand B x Price
Brand C x Price
NONE
1003 368 0 12 -1
-0.099 1.082 -0.983 -0.070 2.106 -3.585 1.549 -1.463 0.391 0.537
-0.928 1.004
1004 708 0 12 -1
7.197 4.854 -12.051 7.022 2.581 -4.098 -5.504 -0.735 -1.496 -2.280
3.776 3.872
1005 497 0 12 -1
3.314 1.854 -5.169 3.656 1.736 -3.652 -1.740 -0.726 -0.114 -0.408
0.522 2.709
Note in this example that both linear effects and interactions have been specified. Because only one
coefficient is estimated to represent the main effect for Price, note the "1" in the second line, third
position. We've copied lines three through five below and added bolding and underlining for further
illustration:
1 0 1
0 1 0
1 0 -1
The elements along the diagonal (in bold) correspond to the main effects. All main effects are included,
but attribute 3 is represented by a single coefficient (linear term). The linear terms are indicated by "-1"
in the design matrix. Off-diagonal elements reflect interaction terms. There is one interaction
(underlined) specified: attribute 1 x attribute 3 (note that it is indicated in both the lower and upper halves
of the matrix).
The order of utility values must be as follows:
Main effects
Interactions, in the following order (assuming complete specification of interactions):

Attrib1 x Attrib2 . . . Attrib1 x Attribn, Attrib2 x Attrib3 . . .Attrib2 x Attribn, etc.
None (only one value)
Important Note: The Market Simulator assumes that the independent variables used for linear
variables were zero-centered. If you used Sawtooth Software's CBC/HB or Latent Class systems,
this happened automatically during utility estimation. If you are providing utility files from your
own estimation procedure, you should make sure your linear codes are zero-centered.
928 SSI Web v8
The .VAL File
If your .HBU file includes linear terms, you must provide an accompanying .VAL file. Let's assume that
you estimated a linear term for Price, and that the dollar values were $8, $10, $15 and $20. The Market
Simulator assumes that you zero-centered the codes for linear variables for estimation. Therefore, you
may have coded these prices as -5.25, -3.25, 1.75, 6.75 in your independent variable matrix. In that
case, the accompanying .VAL file should be as follows:
Brand
1 2 3
Style
1 2 3 4
Price
-5.25 -3.25 1.75 6.75
There are two lines for each attribute: the first is a short attribute labels and the next contains blank-
delimited values to be assigned to that attribute's levels for use in defining products in the simulator.
There must be one value for each level. In this example the first two attributes have values of 1, 2 and 3,
which are used as nominal level codes for these attributes. The third attribute has values of -5.25, -3.25,
1.75 and 6.75.
When you import the .HBU and .VAL file into the Market Simulator, these numeric codes will be
assigned to the different levels of price. However, you can always add back the constant to change
these values to 8, 10, 15 and 20 (the original prices in our example) for ease of use in the simulator by
clicking Assign Level Values. Adding back the constant (that was subtracted for zero-centering) is
appropriate as during simulations the Market Simulator always zero-centers level values for linear terms
before applying them to the coefficients in the utility file.
8.8.2 How the Simulator Interpolates between Levels
When dealing with main effects, the Market Simulator does linear interpolation between adjacent levels.
That is a straightforward procedure that requires no explanation. However, when dealing with two-way
interactions, the situation is considerably more complex. This appendix explains and justifies the
procedure used in two-way interpolation.
With one-dimensional interpolation, as for main effects, we first find a line containing the two extreme
points, and then choose a result from that line. With two-dimensional interpolation the situation is
different. We have a total of four points, composing the four corners of a square. For convenience we
might label those points as follows:
Attribute 2
Level 1 Level 2
Level 1 A B
Attribute 1
Level 2 C D
In this scheme, point A represents the combination of the first level for both attributes, D represents the
combination of level 2 for both attributes, and B and C each represent combinations of level one on one
attribute and level 2 on the other.
By analogy with the one-dimensional case, we might try to fit a plane to all four points, and then do our
interpolation within that plane. Although a plane can always be found to fit three points, it will not
generally be true that a plane will fit four points very well.
Our circumstance is particularly troublesome because the surface required to fit all four points would be
quite warped. If both attributes happen to have only two levels, then the row and column sums of the 2 x
2 array are zero, implying that A = -B = D = -C. If both dimensions have more than two levels the
relationship is not so precise, but the surface containing the four points still tends to be saddle-shaped.
Accordingly, we use a nonlinear interpolation procedure, which we describe with a numerical example.
Suppose a product has specifications of 1.2 for attribute 1, and 1.6 for attribute 2. Also suppose the logit
estimates for the four points are:
Attribute 2
Level 1 Level 2
Level 1 A= 1.0 B=-1.5
Attribute 1
Level 2 C=-0.5 D= 2.0
Finally, let p1 = the proportion of the distance between the two levels of Attribute 1 for which we would
interpolate (0.2 in this example) and p2 = the proportion of the distance between the two levels of
Attribute 2 for which we would interpolate (0.6 in this example).
We interpolate by forming the convex sum of four corner points:
Result = (1-p1) * (1-p2) * A

+ (1-p1) * p2 * B
+ p1 * (1-p2) * C
+ p1 * p2 * D
930 SSI Web v8
This procedure describes a surface that includes all four corner points as well as their centroid. Notice
that if p1 = p2 = 0, then the result is equal to A. If p1 = p2 = 1, the result is equal to D. If p1 = 0 and p2
= 1, the result is equal to C, and if p1 = 1 and p2 = 0, the result is equal to D. If both attributes have
integer specification, this method of interpolation therefore gives the same answer as if no interpolation
were used.
This procedure has the desirable property that there are no discontinuities in the regions of the four
corner points, as there would be if we tried to fit the points with a plane.
If one attribute has an integer specification but the other requires interpolation, then interpolation is
automatically linear for the one attribute requiring it. For example, if p1 = 0, then the result is (1-p2)*A +
p2*B, a linear interpolation between A and B. Similarly, if p1 = 1, then the result is a linear interpolation
between C and D.
Also, if p1 = p2 = 0.5, the result is (A+B+C+D)/4. Therefore the result at the center of the square is the
average of the four corners. If both attributes require interpolation, then the surface is quite regular, with
only as much curvature as required to fit the four corners.
For our numerical example, the result is -0.2. The center of the square has a value of 0.25, but the
location corresponding to p1 = 0.2 and p2 = 0.6 is upward and to the right of the center, toward the
corner that has a value of -1.5.
Since our procedure is nonlinear, it would be hazardous to extrapolate beyond the known points.
Therefore, the simulator does not permit the user to extrapolate beyond measured levels of the attributes
for those involved in interaction effects.
8.8.3 How the Simulator Uses Latent Class Data
Sawtooth Software provides Latent Class analysis software for use with our CBC System for Choice-
Based Conjoint. With Latent Class, rather than computing a set of part-worths (utilities) for each
respondent, the algorithm finds groups of respondents with similar preferences and estimates average
part-worths within these segments.
As when using cluster analysis, the Latent Class analyst specifies how many groups to use in the
segmentation. In contrast to cluster analysis, respondents are not assigned to different segments in a
discrete (all-or-nothing) manner under Latent Class analysis, but have probabilities of membership in
each segment that sum to unity. The sum of the probabilities of membership across respondents for
each group defines the total weight (class size) of that segment.
One can conduct overall market simulations with Latent Class results by computing shares of preference
within each segment and taking the weighted average of these shares across segments. This is the
approach used in Sawtooth Software's Latent Class Simulator in version 1 of the software (LSIM.EXE).
Another way to use Latent Class data is to convert the segment-based results into individual-level
estimates. While these estimates are not as accurate at characterizing respondent preferences as
Hierarchical Bayes analysis, they are an appropriate extension of the Latent Class model.
The Market Simulator converts the group-based Latent Class part-worths into individual-level part-worths
in the following way: For each respondent, a weighted combination of the group part-worth vectors is
computed, where the weights are each respondent's probabilities of membership in each group.
Converting the Latent Class utilities to individual-level part-worths provides added flexibility for market
simulations. It lets the analyst apply segmentation variables as filters, banner points or weights without
requiring that a new Latent Class solution be computed each time.
However, creating individual-level utilities from a segment-based solution slightly alters the results when
comparing the output of Sawtooth Software's Latent Class module to the same data used within the
Market Simulator. While the overall shares of preference for the market are nearly identical, the within-
class results reported in the Market Simulator output are slightly less differentiated between segments
(pulled toward the overall market mean). That is because for the purpose of banner points and filters, the
Market Simulator assigns respondents fully into the latent class for which they have the greatest
probability of membership. For example, consider a respondent whose preferences are characterized as
80% like group 1 and 20% like group 2. His contribution to the mean values reported in the group 1
column (banner point) includes some group 2 tendencies.
The differences between the within-class means reported by Latent Class and the Market Simulator are
not usually very great since respondents' probabilities of membership in classes usually tend toward zero
or one. The smoothing that occurs when reporting the by-segment results in the Market Simulator will
probably not substantially change your interpretation of the results. If the differences concern you, you
can always refer to the original Latent Class output.
932 SSI Web v8
Index ACBC Sections and Flow 463

ACBC Specifications 458
-A- ACBC Testing and Validation 579
ACBCBYOLabel 185
A priori order 379, 720, 802 ACBCBYOValue 185
Abs (Absolute Value) 175 ACBCCurrentCalibration 185
ACA 671, 730, 883, 890, 903, 910, 920 ACBCCurrentChoiceTask 185
ACA Algorithms 738, 741, 743 ACBCCurrentScreener 185
ACA Calibration Concepts 714 ACBCIsMustHave 185
ACA Control Parameters 724 ACBCIsUnacceptable 185
ACA Customizing Importance Question 736 ACBCNumCalibrations 185
ACA Data Storage 727 ACBCNumChoiceTasks 185
ACA Dropping Importance Question 734 ACBCNumScreenedInConcepts 185
ACA Functions 181 ACBCNumScreeners 185
ACA Interview Sections 714 ACBCWinnerLabel 185
ACA Overview 673 ACBCWinnerValue 185
ACA Staying Out of Trouble 732 Accumulate CAPI Files 279
ACA Tutorial 674 Accumulate Data 304
ACA/HB 745 ACD File Format 746
ACAAttribute 181 ADD 168
ACABest 181 Add Merged Variables 316
ACACalMax 181 Adding Questions 42
ACACalMin 181 AddSorted 168
ACAImportance 181 Admin Module 284
ACAImportance Question 714 Admin Module Customization 301
ACALeastImpAttLabel 181 Aggregation in CBC 327
ACALeastImpAttValue 181 AIC 168
ACAMostImpAttLabel 181 AIE 168
ACAMostImpAttValue 181 AIELoopList 176
ACAPair Question 714 AIG 168
ACAPriorsBestLevelLabel 181 AIGLoopList 176
ACAPriorsBestLevelValue 181 AIL 168
ACAPriorsWorstLevelLabel 181 AILLoopList 176
ACAPriorsWorstLevelValue 181 Allocation Questions 65
ACARating Question 714 Alternative-Specific Designs 414
ACAStrictImportance 181 Analyzing Conjoint Data 858
ACAUtility 181 ANC 168
ACAWorst 181 Anchored MaxDiff 645
ACBC Attributes and Level Setup 473 ANE 168
ACBC Constructed Attribute Lists 532 ANELoopList 176
ACBC Design Algorithm 526, 531 Answered 171, 187
ACBC Designing a Study 475 AreAllQuotaCellsClosed 174
ACBC Estimation 551 AreAllQuotasClosed 174
ACBC Functions 185 AreAnyQuotasClosed 174
ACBC Hands-On Tutorial 485 AreQuotaCellsClosed 174
ACBC Motivation 461 Assign Level Values 898
ACBC Overview 459 Assumptions of Market Simulators 881
ACBC Price Variable 482 Attribute Interactions 385
Index 933
Attributes 389 CHO File Format 452

Attributes and Levels 379, 720, 802 CHS File Format 454
Auto Calibrating Attribute Variability Multiplier 920 CiW Tutorial 4
Cluster Analysis 913
-B- Composing Constant Sum Questions 65
Back Button Disable 115 Composing Grid Questions 62
Backing up 138 Composing Numeric Questions 55
Bad Respondents 638 Composing Open-end Questions 58
Banner 898, 913 Composing Questionnaires 39
Base Case 886 Composing Questions 42
Best-worst CBC 386 Composing Ranking Questions 68
BlockPosition 171 Composing Select Questions 48
Boolean functions 187 Composing Start Question 46
BooleanToNumber(Boolean) 171 Composing Text Questions 61
Branching 131 Conditional Display in CBC 413
Browser 115, 171 Conditional Display in CVA 826
BYOCondText 185 Conditional Graphics in ACBC 538
Conditional Pricing in CBC 410
-C- Conditional Pricing in CVA 827
Confidence Interval 908
CALIB Program 890, 920 Constant sum CBC 386
Calibration Concepts 890, 920 Constant Sum Questions 65
Cannabilism 883 Constraints in CVA/HB 856
CAPI Deleting Sensitive Data 280 Constructed Lists 140, 168
CAPI Interviewing 276 Cookies 234
CBC 883, 885, 889, 897, 898, 910, 913 Copying Questions 42
CBC Functions 179 Copying Questions across Studies 39
CBC Introduction 327 Correlation Cutoff 903
CBC Setup 395 Cost Data 885
CBC Tutorial 330 Counting analysis 434
CBC/HB 657, 890, 903, 920 Counting Analysis for ACBC 548
CBC: Overview 325 Counting Analysis in MaxDiff 631
CBCAttributeLabel 179 Counting Data Fields 98
CBCAttributeValue 179 Counts 858
CBCCurrentTask 179 Creating Passwords 224
CBCDesignConceptValue 179 Cross-Elasticities 885
CBCDesignLevelText 179 Cross-Elasticity Effects 883
CBCDesignLevelValue 179 CSS 101
CBCNone 179 CSS hooks 101
CBCTotalTasks 179 Custom CBC Questions 424
CBCVersion 179 Custom Pricing 410, 827
Ceiling 175 Custom Segmentation Variables 868
Changes After Fielding Begins 261 Customized MaxDiff Questionnaires 649
Check-Box Questions 48 Customizing Admin Module 301
CheckSelect 171, 190 CVA 883, 890, 903, 910, 920
Chip Allocation Questions 65 CVA Design Strategy 812
CHO File 433 CVA Designer 843
Cho file export ACBC 577 CVA Functions 183
934 SSI Web v8
CVA Tutorial and Example 760 ErrText 178

CVA Utility Estimation 844 ErrTotal 178
CVA/HB (Hierarchical Bayes) for CVA 851 Exp (Exponentiate) 175
CVAVersion 183 Exponent 888, 890, 893, 903, 910, 917
Export Job Dialog 309
-D- Export Select File Type 305
Data 281 Export Selecting Data 312
Data Doctor 873 Exporting CBC Designs 407
Data Fields 98, 303 Exporting CVA Designs 819
Data Generator 263 Exporting Open-End Data 315
Data Management (in Field) 284 External Effects 881, 893
Database 281
Database tables 281
-F-
Debug 171 Fatigue 809
Decimal Point Character 119 Files created by SSI Web 322
D-Efficiency in CVA 812 Filters 306
Deleting Data From CAPI 280 First Choice Model 882, 888, 890, 916
Demand Curves 885, 910 Fit Statistic in MaxDiff 638
Design Efficiency 399, 662 Fixed Choice Tasks 393
Designing a CBC Study 372 Floor 175
Designing MaxDiff Studies 624 Font Colors 111
Displaying a Progress Bar 113 Foreign Language Studies 220
Displaying Previous Responses 165 FormatListLabels 168
DisplayListLabels 168 FormatNumber 175, 190
DisplayTotal 171, 190 Free Format CBC 424
Double-Byte Characters 220 Free Format Questions 78, 80, 81, 82, 83, 84, 85
Downloading Data 289 Full Profile Conjoint 801
Downloading Password Reports 296
Dual-Response None 391 -G-
Dummy Coding 849 Get Data 304
Getting Data 289
-E- Getting Started 1
Effects coding 438 Getting Started with ACA/Web 669
Efficiency 399 Getting Started with CVA 756
Elasticities 910 GetValue 171
Email 271, 272 Global Font Colors 111
Email Invitations 272 Graphical buttons 109
EncodeForURL 171 Graphics 128, 139
Ending a Survey 75 GraphicsPath 171
Ending Surveys 100 Grid Questions 62
ErrCursum 178
ErrField 178 -H-
ErrMax 178 HB Estimation 745
ErrMin 178 HB estimation for ACBC 556
Error Functions 178 HB Estimation in MaxDiff 632
Error Messages (for the Survey Respondent) 116 HBU Format 925
ErrQName 178 Head Tag 112
Index 935
Heterogeneity 883
Hidden Variables 86 -L-
Hierarchical Bayes 433, 745, 883 Label 171
Hierarchical Bayes Analysis 657 Latent Class 433, 659, 883, 913, 931
Holdout Choice Tasks 393, 861 Length 177
Holdout Tasks in CVA 823 Level Prohibitions 382, 723
Holdouts 888, 890 Level Values 895, 898
Holdouts in CVA 819 License Sizes 98
HTML Cheat-Sheet 124 Limbo data 138, 281
HTML Elements 122 Line Extensions 913
Linear Dependency 849
-I- Linear Terms 895
ICE 883, 903, 920 LinkButton 171
IIA 889, 890, 897 Linking Surveys 274
Images 128, 139 Linking to other surveys 75
Importance Question 714 List Building 140, 163
Importance Strict 190 List Building Functions 168
Importances 858, 903 ListHasParentItem 187
Importing CBC Designs 407 ListHasParentMember 168
Importing Conjoint Data 925 ListLabel 168
Importing CVA Designs 819 ListLabelsArray 168
Importing Passwords from Text File 229 ListMax 168
Importing Utility Runs within SMRT 865 ListMin 168
Incompletes Report 297 ListValue 168
Inline Scripting 163 ListValuesArray 168
INSERT 168 Ln (Natural Log) 175
Installing Survey on Web 250, 257 Log10 (Log Base-10) 175
Interaction Effects 883 Logging On 287
Interaction Effects in ACBC 559 Logit analysis 438
Interaction Effects in CVA 830 Logit Model 897
Interactions 385 Loop functions 176
International Surveys 220 LoopIteration 176
Internationalization 220 LoopLabel 176
Interpolation 895, 929 Loops 136
Interpreting Conjoint Results 858 LoopValue 176
Invitations 271, 272
IPAddress 171 -M -
IsAlphaNumeric 177 Marginals 292
Isnumber 177, 187 Market Share 883, 888, 893
IsQuotaCellOpen 174 Market Shares 858
Item List in MaxDiff 621 Market Simulator 754
Market Simulator Dialog 898
-J- Math Functions 175
Javascript 87, 171, 187 MaxDiff Designs 662
Javascript Turning Off 119 MaxDiff Functions 184
JavaScript Verification 87 MaxDiff Introduction 585, 586, 589
JQuery 87 MaxDiff Items 621
936 SSI Web v8
MaxDiff Latent Class Analysis 643

MaxDiff Logit Analysis 640 -P-
MaxDiff Motivation 586 Page Breaks 45
MaxDiff Paper-and-Pencil Studies 655 Page Headers and Footers 112
MaxDiff Questions 617 PageNumber 171
MaxDiff Scores on-the-fly 654 PageTime 171, 190
MaxDiff Tutorial 590 Paired Comparisons MPC 586
MaxDiffDesignLabel 184 Pairs 741
MaxDiffDesignValue 184 Pairs (how many) 724
MaxDiffRankAttLabel 184 Pairwise Comparison in CVA 756, 807
MaxDiffRankAttValue 184 Paper-and-Pencil in CBC 431
MaxDiffScore 184 Paper-and-Pencil Studies in CVA 824
Merged fields 224, 228 Part Worths 728, 858
Merging Segmentation Data 868 Partial-Profile Designs 418
Method of Paired Comparisons 586 Part-worth Utilities 882
Mid 177 Pass-In Fields 224, 233
Midpoints Formula 910 Password Importing from Text File 229
Mixed Approach 526 Password Report 296
Monotone regression for ACBC 568 Passwords 224, 226, 281, 286
Monotone Regression in CVA 841 PDF File 223
Moving Data into SMRT 450, 752 Perl 198
Must-Haves (ACBC) 548 Piping 140, 165
Popup 171, 190
-N- POWER 175
Naming Questions 42 Practice Data 258
New Product Introduction 908 Precision 890, 908, 921
Next and Previous buttons 109 Predefined Lists 140
None Concept 391 Predictive Quota Control 244
None threshold in ACBC 574 Pre-Qualification 246
None Weight 897, 903 Price Adjustments in ACBC 536
NumberToBoolean 171 Price in ACBC 482, 552
NumberToString 171 Pricing Studies in CBC 410
NumChecked 171 Pricing Studies in CVA 827
Numeric Questions 55 Print Survey 223
Progress Bar 113
-O- ProgressBar 171
OLS in CVA 837 ProgressBarOff 171
One-Click Access to Survey 272 ProgressBarSet 171
Online Simulator 317 Prohibitions 379, 382, 720, 723, 802
Open-End Questions 58 Prohibitions for CVA 806
OperatingSystem 171 Purchase Likelihood Model 890, 920
Optimization 913 Purchase Likelihoods 858
Optimize Study Files 873 Purging Files 280
Other Specify Responses 48
Output Precision 898
-Q-
Question Names 42
QuestionName 171
Index 937
Quota Control 236, 240, 243, 244, 246 Sensitivity Analysis 885, 903
Quota Control Functions 174 Server Installation 250, 257
QuotaCellCompletes 174 Server-Side Problems 259
QuotaCellLimit 174 SetListLength 168
QuotaCellName 174 Setting Page Breaks 45
QuotaCellRemaining 174 Setting Passwords 286
QuotaCellValue 174 SetValue 171, 190
Quotas Changing During Fielding 247 Share of Preference Model 888, 890, 917
Share of Preference with Correction for Product Similarity
-R- 890
Shares of Preference 858
Radio Button Questions 48
Shelf-Facing Display 421
RadioSelect 171, 190
Shown 171, 187
RandNum 171, 197
Simple Browser 115
Random Numbers 197
Simulation Scenario 898, 901
Random Subset of Questions 129, 131
Simulations 754
Randomize 168
Single Concept in CVA 756, 807
Randomized First Choice Model 890, 921
Skip Logic 131, 163
Randomizing Blocks of Questions 129
Skip Sections 541
Randomizing Pages 129
SMRT 318, 450, 752, 864
Randomizing Question Order 129
Software Requirements 28
Ranking Questions 68
SortByLabel 168
Recode 314
SortByValue 168
Red-Bus/Blue-Bus Problem 889, 890
Sparse Designs in CVA 832
Remove 168
SPSS File Export 310
RemovePrevious 171
Sqrt (Square Root) 175
Resetting Survey 300
SSI Script 163
RespNum 171
Standard Error 890, 908
Respondent Verification 178
Start question 46, 225
Restart 46, 75, 224, 272, 274, 289, 298
Strict Importance 190
Restrictions 379, 382, 720, 723, 802
String Functions 177
Reverse 168
StringToNumber 171
RIC 168
Study Management 29
ROUND 175
Studyname 171
ROUNDTONUMBER 175
Styles 101
Styles Modifying 101
-S-
SysRand 171, 197
Sample Size 890 System Functions 171
Sample Size for MaxDiff 632 System Variables 303
Scale Factor 888 SystemTime 171
Scenarios 886
Score Estimation in MaxDiff 632 -T-
Screener Questions 236
Tables program 876
Segmentation 659, 885, 913
Tabulating Data Online 292
Segmentation data into SMRT 450, 752
Tabulating segmentation data 876
Segments 306
Technical Support 1
Select Questions 48
Terminating Question 75
Semantic Differential 71
938 SSI Web v8
Test Data 263 Weights 306, 903

Test Design (ACBC) 479 Whats New? 30
Test Survey 298 Write Questionnaire Dialog 39
Testing CVA Designs 817
Testing Designs in CBC 399 - Z-
Testing Questionnaire 248 Zero-Centered Diffs 903
Test-Retest Validity 861
Text/HTML Filler Questions 61
TextExists 177
Throw Out Obvious Tasks 811
ToLower 177
Tool Tip 171, 190
Tool Tips 171, 190
ToolTip 171, 190
TotalPages 171
ToUpper 177
Tuning the Simulator 881, 888, 917, 921
-U-
Unacceptables (ACBC) 548
Unverified Perl 198
Unverified Perl in ACBC 546
Updating in ACA 743
Uploading Survey 250, 257
URL link 75
UserAgent 171
User-defined passwords 234
Utilities 728, 858
Utilities: Feedback 751
Utility Constraints in ACBC 565
Utility Estimation in CVA 844
Utility Runs 898
UTL File Format 746
-V-
Validation 393
Validity 861, 883, 890
Value 171, 174
Verification 116
Verification (Custom JavaScript) 87
Versions in CVA 812
Viewing Data on Server 290
Viewing Editing Data 316
Volumetric CBC 429
-W-
Weighting 872

SSI Web Documentation PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SSI Web Documentation PDF

Uploaded by

Copyright:

Available Formats

SSI Web v8.

Software for Web Interviewing

(Updated 2 June 2015)

Sawtooth Software, Inc.

SPSS is a trademark of SPSS, Inc.

WS_FTP is a trademark of Ipswitch.

We express our gratitude to the Perl community.

Other acknowledgements regarding copyrighted work are in the Readme.rtf file.

Bryan Orme, Editor

Adaptive CBC (ACBC) Help

Appendices for SSI Web

SMRT (Market Simulator) Help

1 General Web Interviewing (CiW) Help

Steps to Creating a Web Survey

2. Create a new study in SSI Web.

at support@sawtoothsoftware.com or +1-801-477-4700 if you cannot find the answer to your problem or

Privacy and Research

1.1.2 Getting Started: 45-Minute Tour of SSI Web

SSI Web stands for "Sawtooth Software

The Big Picture

If you don't want to deal with hosting issues or

Starting a New SSI Web Study

Writing a Simple Survey

Thank you very much. That's all we need to know today.

The Questionnaire Access and Passwords dialog is shown:

Create User Names and Passwords

Next, click the Preview button at the bottom of the dialog.

The Select Question dialog is displayed:

Next, let's specify the response options:

From SSI Web, Click OK to return to the Write Questionnaire dialog.

This is a "Numeric" question

Next comes Q3:

This is a "Select" type question

Next comes Q4:

This is an "Open-end" type question

Thank you very much. That's all we need to know today.

This is a "Terminate/Link" type question

Page Layout and Skip Patterns

Q1 to be followed by a page break

From Q1, Skip To Finish If Q1=1

Previewing the Questionnaire

Test Running Your Survey Using "Local Test Server"

Local Test Server

run the survey again, click the Test Survey button .

Adding Polish and Style

The Admin Module

With the Admin Module, you can…

Check how many completed records you have

Running the Admin Module

Exporting (Converting) Data to Excel(TM) Format

Here are our results (yours will be different):

1.1.3 Software Requirements: Users, Servers and Respondents

PC running under Windows XP Service Pack 2, Windows Vista, Windows 7 or later.

The Internet/Intranet Server:

Support Perl Version 5.007 or later (capable of running CGI scripts)

For the Respondent:

1.1.4 Creating/Opening a Study

To create a new study:

To open an existing study:

To copy the current study to a new path or name:

1.2 What's New in v8.4.0?

Updates for v8.3.0

Not Applicable (Missing) Levels in Conditional Display If you have an Alternative-Specific or