12/12/13 Heuristic Evaluation

Heuristic Evaluation Evaluation

706.021 Mensch-Maschine-Kommunikation 3VU SS 2010

Gruppe 3-09
Gerwald Tschinkel
Luis Dean Kals
Rene Langgartner

Heuristic Evaluation of Website

http://www.mediamarkt.at

Report of 23 Apr 2010

1 Executive Summary

1.1 Product and Goals

This report documents a heuristic evaluation of http://www.mediamarkt.at, the website of a local market for electric
appliances. The goals of this heuristic evaluation were to evaluate the usability of the website using a set of heuristics
and establish a prioritized list of issues for the developers' attention. The findings along with the suggestions will
guide future changes in the software's interface.

1.2 Methods
Reseachers utilized "Andrews General Usability Heuristics". Each team member independently performed tasks multiple
times, recorded usability issues and which heuristic violations occurred and then rated each issue in terms of severity.
Researchers later consolidated and prioritized usability issues according to severity.

1.3 Findings
This evaluation identified 26 usability issues in total. We described 5 usability issues in detail and proposed solutions
for each of them in the report. The following are the most severe issues outlined in the report:

1. Sorting Problems:

It is impossible to sort items based on the price, which is a standard feature in most online stores.

2. Webcode:

No real description on how to reach or recieve a webcode, yet it is listed on every link featuring a product (and
some other subsites).Couldn't find a webcode for various products, couldn't find one at all. It says it navigates
to a specific MediaMarkt website however, we've never noticed a webcode and based on the facts its primarily
under products, we guess its meant to help with product search. However no product lists one in any findable
way.

3. Inconsistent naming "Wunschliste" and "Einkaufsliste":

After adding a product to your wishlist, it gets directly placed into the shopping list.

4. Registering confirmation email:

Registering a user account doesn't send the confirmation email in reasonable time and already logged in, but
without confirming the registration email, the costumer cannot do anything.

5. Search does not find specific existing page:

There is a Service called "K�chenger�te Einbaucheck". If you type this exact string into the search, you will find
a lot of "K�chenger�te", but not the specific page.

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 1/11

12/12/13 Heuristic Evaluation
More detailed information is given in the Heuristic evaluation.

2 Introduction
Reseachers utilized "Andrews General Usability Heuristics", which is based on Nielsen's ten usability heuristics to
evaluate the website. The ten heuristics can be seen in Table 1.

1. Individual work:

Each team member independently performed tasks multiple times, recorded usability issues and which heuristic
violations occurred, they rated each issue in terms of severity.

2. Work in group:

Later, the researchers discussed their independent findings. Afterwards they consolidated and prioritized
usability issues according to severity.

No Usability Heuristics
1 The system should give appropriate feedback within an appropriate time. (Feedback)
2 The system should speak the user's language. (Speak the User's Language)
3 The user should be free to explore the system without penalty. (Reversible Actions)
4 The same word, phrase, action, or situation should always mean the same thing. (Consistency)
5 Prevention is better than cure. (Error Prevention)
6 Place knowledge in the world. (Recognition Rather Than Recall)
7 Provide accelerators for frequent users, unseen by novice users. (Flexibility and Efficiency of Use)
8 "Less is more". (Aesthetic and Minimalist Design)
9 A good error message helps users recognize, diagnose, and recover from errors. (Good Error Messages)
10 It may be necessary to provide help and documentation. (Help and Documentation)

Table 1: Ten heuristics.

2.1 User Profiles

1. Product Surfers:

Users that want to get informed about a product (read reviews, etc)

Users that just want to "browse" through the page / through the products

2. Online Customers:

Users that want to shop online

3. Potential Employees

4. Potential Store Customers:

Users that just want to get informed about, how to get to the store, when they have opened, etc.

Users that wants to know about the current "special offers"

users that are looking for a specific type of product (eg what laptops do they offer, ...)

Users that are looking for a specific product, for availability / cheapest price

2.2 Extent of Evaluation

The reseachers analysed the parts of the User Interface which are accessible for a common visitor of the website and
which she/he uses during her/his visit. The researchers did not check for failures in the sourcecode or anything of the
like. The main exercise was to simulate all types of visitors (see 2.1) and how these types use the website, trying to
detect as many failures/problems as possible. The goals of this heuristic evaluation were to evaluate the usability of
the website using a set of heuristic and disclose a prioritized list of issues to developer's attention. The findings along
with the suggestions will guide future changes in the software's interface.

3 Evaluation Environment

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 2/11

12/12/13 Heuristic Evaluation
The hardware and software configurations are listed in Table 2.

Evaluator Luis Kals Gerwald Tschinkel Rene Langgartner

Age 22 29 21
Sex male male male
Web Browser Mozilla/5.0 en IE 8.0 en Opera 10.10 de
Operating System Ubuntu 9.10 Windows 7 Ultimate Ubuntu 9.10
Internet Connection 8 mbps, AON, cable 8 mbps, UPC, dsl 10 mbps, cello, cable
Monitor Size 15" 13,3" 15,4"
Monitor Colours 32-bits 32-bits 32-bits
Monitor Resolution 1280 x 800 1280 x 800 1366 x 768
Browser Resolution 1280 x 800 1280 x 800 1366 x 768
Date of Evaluation 2010-04-11 2010-04-11 2010-04-13
Time of Evaluation 18:00-21:30 18:00-22:00 10:00-11:00

Table 2: Evaluation Environment

4 Positive Impressions

4.1 Professional Photos of Products

The appearance of the products is very professional with good highlighting as can be seen from the figure 1.

Figure 1: Professional Photos (here a notebook)

The researchers also gave positive comments for the good overview on the searching process. Also the visible state of
actuality is very positive. These impressions were highlighted with red circles in figure 1.

4.2 Category Kontakt

All the needed information is listed (Store Opening Date; Address and Owner; etc.). A Telephone, Fax and Email
address are listed. A feedback form is also included for questions. It also includes a functional google-earth map that
shows the location. 2

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 3/11

12/12/13 Heuristic Evaluation

Figure 2: Category Kontakt

4.3 Das Interessiert Andere Kunden

When looking for a specific range of products, it lists the products in the same category that others were looking for.
The font dictating how often the product was searched/how popular. In Figure 3 "Das Interessiert Andere Kunden" is
highlighted with a red circle.

Figure 3: Das Interessiert Andere Kunden

The researchers also mentioned the fast loading speed.

5 Analysis of Main Problems

The 5 most severe problems from the helist are described in the following chapter.

5.1 Sorting Problems

It is impossible to sort items based on the price, which is a standard feature in most online stores. 4

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 4/11

12/12/13 Heuristic Evaluation

Figure 4: Sorting Problems

5.2 Webcode
There is no real description on how to reach or recieve a webcode, yet it is listed on every link featuring a product (and
some other subsites). The researchers couldn't find a webcode for various products, couldn't find one at all. It says it
navigates to a specific MediaMarkt website however, we've never noticed a webcode and based on the facts its primarily
under products, we guess its meant to help with product search. However no product lists one in any findable way. 5

Figure 5: Problems with the webcode

5.3 Inconsistent naming "Wunschliste" and "Einkaufsliste"

After adding a product to your wishlist, it gets directly placed into the shopping list. 6

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 5/11

12/12/13 Heuristic Evaluation

Figure 6: After adding a product to your wishlist, the shopping list

appears from nowhere.
The shopping list is highlighted with a red circle.

5.4 Registering Confirmation Email

Registering a user account doesn't send the confirmation email in reasonable time and already logged in, but without
confirming the registration email, the costumer cannot do anything. 7

Figure 7: Here is the confirmation message on the website, but the

email is not sent.

5.5 Specific Existing Page is not found

There is a Service called "K�chenger�te Einbaucheck". If you type exactly this string in the search, you will find a lot of
"K�chenger�te", but not the specific page. 8

Figure 8: Specific page not found

6 List of All Found Problems

Found

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 6/11

12/12/13 Heuristic Evaluation
No. Short Title Description Screenshot Heuristic How Reproducible
LK GT RL
There is a Service
called
"K�chenger�te
Einbaucheck". If
Search does you type exactly
tschinkelgerwald-
1 not find this string in the Minimalist Design search y y -
prob12.png
specific page search, you will
find a lot of
"k�chenger�te",
but not the
specific page.
Impossible to sort
items based on
KalsLuis- Flexibility and Efficiency of
2 Sorting Items Price [A standard Product Search y y y
prob03.png Use
feature in most
online stores].
No real description
on how to reach or
recieve a webcode,
yet it is listed on
every link featuring
a product (and
some other
subsites).Couldn't
find a webcode for
various products,
couldn't find one
at all. It says it
navigates to a
KalsLuis- Aesthetic and Minimalist Product Search / Product
3 Webcode specific y y -
prob04.png Design Descriptions, etc
MediaMarkt
website however,
I've never noticed
a webcode and
based on the facts
its primarily under
products, I guess
its meant to help
with product
search. However
no product lists
one in any findable
way.
Registering a
user account And I am logged
doesn't send in, but i cannot do
the anything, as i tschinkelgerwald- klick on Registrieren, and
4 Good Error Messages y y -
confirmation haven't confirmed prob05.png fill out the data.
email in the registration
reasonable email.
time.
Although officially
logged in on the
Account website it still
Creation - Still denies me basics KalsLuis-
5 Good Error Messages General Item Comments y - -
not officially and says I am not prob02.png
Logged in logged in. However
wishlists work.

Inconsistent After you added a

naming product to your go on an product details
tschinkelgerwald-
6 "Wunschliste" wishlist, it gets Consistency and Standards page, and click on "auf - y y
prob06.png
and directly placed into meine Wunschliste"
"Einkaufsliste" your shopping list.
If you are not sure
Search does
to which category
not have an
a product belongs langgartnerre- Flexibility and Efficiency of
7 �Alle search - - y
have to try several prob01.png Use
Kategorien�
til you find the
file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 7/11
12/12/13 Heuristic Evaluation
Kategorien�
til you find the
option right one

Click on go on any product

Register leads Seems to be a tschinkelgerwald- details, then at the end,
8 Bug - y -
to an empty clicking-bug. prob04.png click on "Jetzt f�r "Mein
page MediaMarkt" registrieren."
When adding an
object to a
wishlist, I take for
granted I can
choose when
pressing on 'add'
my wishlist when
clicking. However I
have to choose
Wunschlisten
that wishlist as KalsLuis- Product Info / when
9 - Choosing Consistency y - -
active beforehand prob06.png Logged in
lists
however the
wishlist is right at
the bottom of the
screen and thus
silly to keep
scrolling down -
activating and
then choosing the
product.
�Software� vs.
�Software �
Intransparent
Spiele und langgartnerre-
10 categories of Minimalist Design search - - y
Anwendungen� prob01.png
products
where is the
difference?

There are two

different ways of
displaying a
Inconsistent click on "Honeywell" in
product short
display of tschinkelgerwald- the Tag Cloud, than click
11 overview. Compare Consistency and Standards - y -
product prob03.png on the product, than click
tschinkelgerwald-
preview on "more from honeywell"
prob02.png and
tschinkelgerwald-
prob03.png
When swapping
between two
wunschlisten, it
took the second
one and labeled it
aktiv. It was badly
listed. However I
had to press the
same button again
to aktivate the
Einkaufsliste,
However I received
no feedback on the
Wuenschlisten change (It still KalsLuis- Reversible
12 Wuenschliste y - -
� Swapping labeled Musik as prob07.png Actions/Feedback/Consistency
being active), It
was also the same
button i used to
active the other
list. If I didn't
refresh the page,
it didn't show the
change. Aktivieren
looks like an
einkaufsliste using
the same red
arrow and is a
design flaw.
When swapping
file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 8/11
12/12/13 Heuristic Evaluation
When swapping
between more
'wuenschlisten',
one vanishes and
it states
einkaufsliste is
aktiv (and not the
one activated). It
seems to always
Wuenschlist -
be the one with KalsLuis- Reversible
13 More than two Wuenschliste y - -
the lowest prob08.png Actions/Feedback/Consistency
lists
alphabetical name.
Refreshing shows
the correct active
list but the final
vanishes again on
changing / still
same problem as
above when
changing.

Difficult to add a
product to
langgartnerre- Product Search / Product
14 Wunschliste "Wunschliste", Minimalist Design y - y
prob02.png Descriptions, etc
because of bad
placing and design
At each product
you see the text
"Angebot g�ltig
am 10.04.2010,
MediaMarkt
Seiersberg" But
the problem is,
that now (test-
date) we have the
Validity of tschinkelgerwald-
15 11.04.2010 ? product details - y -
prices prob09.png
(=Sunday) - so the
prices are only
valid in the past.
My expectation
would have been,
that the prices are
at least valid until
the next day the
shop has opened
Although logged
in, it says I can
MediaMarkt register for Mein KalsLuis-
16 Bug Top 10 y - -
top 10 Mediamarkt. prob05.png
However I just
registered!

The result is not

arranged very well.
Confusing very small, with a
arrangement lot of additional tschinkelgerwald- click on "Honeywell" in
17 Minimalist Design - y -
of search text around it, that prob02.png the Tag Cloud
results is not as
neccessary.

Top offers are

There are no links
just langgartnerre- Flexibility and Efficiency of
18 to detailed product Top 10 y - y
announced as prob05.png Use
informations
screenshots
When using the
search without Press "Fotoservice"; Use
entering a langgartnerre- searchengine without a
Screen gets message and prob06.png, term and category; If you
19 Bug - - y
stuck without selecting a langgartnerre- press "Service" (or any
category the prob07.png other category) the site
screen gets does not change
�stuck�.

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 9/11

12/12/13 Heuristic Evaluation

When you want to

delete something
from the product
Deleting items comparison list,
from the there is a checkbox
an a link at each tschinkelgerwald-
20 product Consistency and Standards product comparison - y -
product. if you prob08.png
comparison
list click on the link,
nothing happens.
you first need to
click on the
checkbox and than
click on the link.
People, who want
Bad
to find the job langgartnerre- Flexibility and Efficiency of
21 positioning of Bottom of mainpage - - y
category may have prob04.png Use
job exchange
some problems
When using the
webform to send a
request; too many
various options are
listed for various
Too much places. If checking
KalsLuis- Aesthetic and Minimalist
22 information in from the registration/newuser.html y - -
prob01.png Design
Contact Form Seiersberg website
this is unneeded
because you only
need a general
requiry or a requiry
to the store.

The text "aus

Wrong layout dieser liste tschinkelgerwald-
23 Bug "Wunschliste" - y -
of text entfernen" has a prob11.png
wrong layout (bug)
The Google Maps
Popup shows all
Contact
Contact page information, but
- opening because of a tschinkelgerwald- Contact page (Start ->
24 Minimalist Design - y -
times not very technical problem, prob01.png click on "Kontakt")
readable some text is not in
the popup and is
therefore not very
readable.
On the contact
Obsolete page, the
tschinkelgerwald- Contact page (Start ->
25 information on christmas opening Minimalist Design - y -
prob01.png click on "Kontakt")
contact page times are still
available (in April)
On the right side
of the products
details page, you
can find a list, that
shows you in which
Confusing
product category
icon, on the tschinkelgerwald- go an a products details
26 you are. the sign Consistency and Standards - y -
list "in which prob07.png page
next to the current
category am i"
category is an red
X � more known
as when it comes
to deleting
something.

Table 3: Aggregated list of all problems found, in descending order of average severity.

Code Meaning

LK Luis Kals

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 10/11

12/12/13 Heuristic Evaluation
GT Gerwald Tschinkel
RL Rene Langgartner
y Found by this evaluator

Legend: Name Codes.

Severity Meaning
4 Catastrophic problem
3 Serious problem
2 Minor problem
1 Cosmetic problem
0 Not a problem
Av Average severity

Legend: Severity Ratings.

Appendix A: Heuristics Used by the Evaluators

The evaluators used the set of heuristics found here heuristiken.pdf . In the hardcopy, you will find a printout of the
heuristics.

Appendix B: Individual Evaluation Logs

The raw, individual evaluation logs of each evaluator. The hardcopy includes printouts of each of the evaluator logs.

Name Log File

Luis Kals log-kals.txt

Gerwald Tschinkel log-gtschinkel.txt

Rene Langgartner log-langgartner.txt

Appendix C: Spreadsheet
The filled-in spreadsheet (heliste.xls) of all problems including severity ratings.

The hardcopy includes a printout of the spreadsheet.

XHTML 1.1 C SS2

file:///C:/Users/ibm/Desktop/Heuristics Evaluation/Sample Heuristic Evaluation by a student/HE.htm 11/11