Heuristic test:

Consistency:

- Pros: Internal consistency for the circled icon on the top, with a blue ring occurred when
certain functions are activated.

-
- Cons: Inconsistency for the “sharing screen” icon compared with ones with same purpose in
Zoom and Tencent meeting, where there is an arrow in the block to indicate an action of
uploading to share in the meeting.
Similar cons for sharing the camera. The camera icon can be interpreted as open the camera
of the laptop instead of sharing the video in the room.

Mapping

- Pros: Intuitive mapping for icons such as microphone to indicate whether the mic of the
laptop is turned on.
- Cons: Share screen icon and create icon are hard to understand.

Minimalism:

- Pros: The interface looks nice and neat.

- Cons: So simple that it can increase the difficulty in interpreting the icon message such as
“share screen”, or the difficulty in moving the created object because no assistance tool or
coordination is provided.

Freedom:

- Pros: Options to read documentation help or start tour for beginner

- Cons: No freedom in managing the existed object in the room. Need to edit one by one

- Cons: No step choice for tours, stuck at “share” the room when no one is here

Flexibility:

- Cons: No accelerator or shortcut observed.

Recognition:
 - No annotation of words for icons. Text box will only occur if the mouse arrives at the icon.
This deteriorates the difficulty in interpreting the meaning of the icons especially as some of
them are externally inconsistent such as video sharing and so on.

Visibility:

- Pros: Count down of seconds when shooting an image.

Error Prevention:

- Cons: No observed error prevention

Error Recovery:

- Pros: Accidental sharing or accidental creation of object can both be deleted with provided
options.
- Cons: Ctrl + Z action to undo is not provided for error correction

Help:

- Pros: Documentation and interactive instruction

- Cons: No Q&A to solve ad-hoc problem

Functionality:

- Cons: Poor Internet compatibility. Users in mainland China suffered from severe difficulties
simply in entering the room
- Cons: Poor functions realization. There were cases where several group members in a same
room can’t see each other or some of group members can’t hear each other while others
can.
- Cons: Actions such as writing with a pen, rotate the view through dragging is practically hard
by mouse.
- Pros: Rotate view by Q and E keyboard is much easier to operate.

Usability Testing:

Task Design:

- Goal: To discuss and hopefully come up with available ideas for course projects in
COMP4461.
- Look for: How well does the brainstorming task flow work?
- Context of use: The users are expected to be under pressure as the timeline is short, non-
interrupted as they are having the meeting. They are also expected to be at home with
available microphone and speaker to interact in the task.
Evaluation Metrics:

- Realization: What does the user want to do during the brainstorm, and can the user identify
relevant function to satisfy such needs?
- Issue: Identify issue and its severity, if any.
- Self-reported feedback on user experiences

Recording method:

- Video recording

Users:

- Target user profile: Users that expect Mozilla Hub as a place for virtual meeting to discuss
issues and conduct brainstorming.
- Who: 5 college students from HKUST, with age from 19 to 22. They would discuss about the
course project ideation in the Hub, which is expected to be a match for the target user.

Experiment findings:

- Users encounter severe Internet connection issue during the experiment. The majority of
the users spent over 15 minutes simply for entering the room. Some teammates can’t hear
other voices with no hardware and software problem in their own devices.
- Users attempted to communicate with each other to start the meeting, but the
aforementioned problem hindered meeting from getting started.
- The imitated effect of distance to voice volume interrupted the meeting for several times
because members can’t hear each other clearly.
- Users attempted to share notes but found no available effective functions in the Hub. Pen
function is impracticable for taking notes.
- Users were not aware of the screen sharing supported in the Hub.