Testing the prototype(s)

1. Our primary goal in a test interview is to learn, not to give reasons for or sell the

prototype(s)*.

2. This is why we don’t explain (too early) how it works.

3. We ask for stories and situations in which our potential customers might have

needed the prototype.

4. Whenever possible, we collect and analyze quantitative data to validate the

qualitative results.
prototype(s)*.=> one model or few versions
 Test preparation Record data &
Where, when, how Conducting test in document Infer findings
real settings

Feedback grid,
other templates
Question map
Data collection methods
Test Scenario Face to face, telephone, online
Structured questions vs. un-stru. Q
Who, where, when
Testing versions: A B testing
Multi-variate testing ….which involves more than one variable
Question map
1. Start with real scenario
2. Ask what they would want to do
3. Would they prefer to use something like this prototype?
4. Ask q in area where we want to gain more insights –
form of final product, time of use, duration of use,
supporting factors in ecosystem,
challenges, handleability, storage, usability etc. (not pricing matters)

1. Less important aspects can be omitted for detailed investigation …after quick check
2. Whole testing/investigation time should be quick
3. Must be preceded by rapport building
4. Recording data must be real time/simulataneously.

Test Scenario: sequence of activities, describing the situation of the person

being tested with, plan for refraining from telling “why prototype was made”
Data collection methods Guidelines
Face to face, telephone, online Natural environment
Structured questions vs. un-stru. Q Potential users
A B testing Create the right scenario
Multi-variate testing Be unobtrusive and sympathetic
Have a team …pilot study
Observe, note and cross question
Show variants ….settings + adv vs.
dis-adv.
Ask solutions
Get permission for re-connection
Get more links for respondents *

*specially important in few cases

Making a Qs map….some of these q will be answered based on observations or from indirect hints
1. What problem does your idea solve? Use this template for exercise in next session
2. How are users solving this problem currently?
3. Can your user think of another product that does something similar?
4. How have previous solutions failed?
5. Do users understand what this product or service does?
6. How do users feel about the product or services?
7. Which stakeholders accompany product’s use? How would they react?
8. Who is your competition?.......(hint)
9. What is the app/site for and what can users do there?
10. Does your target market have a need for this product?
11. What devices do users imagine themselves using when they interact with this product?
12. What scenarios can they picture themselves using it in?....time of day, temperature, humidity, ambience etc.
Some questions could be asked upfront

1. How would they expect it to look?

2. Once you show them the prototype, do users understand what it does?
3. How does it measure up to their expectations?
4. What features are missing?
5. Does anything seem out of place or unnecessary?
6. How do users feel when using the prototype?
7. If users had a magic wand, what would they change about the product?
8. How likely or unlikely would they be to use this product once it’s finished?
Landing page versions
= Prototype versions
Documenting results & inferring insights/learning

Current negative features

Current positive features

Future product
Grene redesigned its mini cart and saw a 2x increase in overall purchase quantity
Grene is a highly recognized eCommerce brand headquartered in Poland that sells a comprehensive variety of

agriculture-related products.

Þ Has run many successful A/B tests,

Þ One of which was revamping its mini cart page to add prominence to in-page elements.

While analyzing their mini cart page, Grene’s team found that users were:

1.assuming the “Free Delivery” USP to be a clickable button, hoping to find some extra details. This was creating a lot of

friction.

2.finding it difficult to see the total of each item in the cart.

3.scrolling to the bottom of the cart page to click on the “Go To Cart” CTA button.
https://vwo.com/blog/ab-testing-examples/
Changes were:

1.added a CTA (call to action) button at the top of the mini cart to help users
quickly transit to the main cart page.
2.added a ‘remove’ button on the right side of each item (to avoid any unwanted
clicks) and the total value of each product.
3.increased size of the “Go To Cart” CTA button => prominently visible.
Steps to measure the results:

1. Identify the variables to be measured

…like site visits, time spent, click through rate, purchase values
2. Find the scale to measure the variable ….frequency, averages
….category/product wise
3. Compare ‘Control group’ result with ‘experimental group’ result