Dissemination Event

February 2023
 ISO IEC JTC1 SC35
User Interfaces

WG9: Natural user interfaces and

interactions
• Natural User Interface(NUI): the user operates the interface
through an intuitive manner related to everyday human behaviors
in a real world [SOURCE: ISO/IEC 4944]
• “Naturalness”: user's expectations, behavior and feelings that
are involved in interacting with an NUI.
• Everyday human behaviors: actions which can be performed
without the need for learning and training
• Natural Interactions: performed with NUI including (but not
limited) to touch, gestures, facial expressions, and/or auditory
interactions.

JTC 1 SC35 2
 ISO IEC JTC1 SC35
User Interfaces

JTC 1 SC35 3
 ISO IEC JTC1 SC35
User Interfaces

NUI in a real world

• NUI: a user interface that is effectively invisible and

remains invisible as the user continuously learns
increasingly complex interactions. (Wikipedia)
• A user can quickly change from novice to expert.
• “Natural User Interfaces and Interactions" refer to a
goal in the user experience – that the interaction
comes naturally, while interacting with the technology,
rather than that the interface itself is natural.

JTC 1 SC35 4
 ISO IEC JTC1 SC35
User Interfaces

Scope of WG 9

• Natural User Interface

• Gestures based User Interfaces (ISO/IEC 30113 Series)
• Voice User Interface for Personal Mobility Services (CD
7818)
• Interactions
• Unifying Input Actions across Devices (PWI 4933)
• User Experience(UX)
• Evaluating NUI Usability (DIS 4944)

JTC 1 SC35 5
 ISO IEC JTC1 SC35
User Interfaces

Published Standards

• ISO/IEC 30113-1 “Gesture-based interfaces across devices and

methods - Part 1: Framework”
• ISO/IEC 30113-5 “Gesture-based interfaces across devices and
methods - Part 5: Gesture Interface Markup Language (GIML)”
• ISO/IEC 30113-11 “Gesture-based interfaces across devices and
methods - Part 11: Single-point Gestures for common system actions”
• ISO/IEC 30113-12 “Gesture-based interfaces across devices and
methods - Part 12: Multi-point Gestures for common system actions”
• ISO/IEC 30113-60 “Gesture-based interfaces across devices and
methods - Part 60: General guidance for screen readers”
• ISO/IEC 30113-61 “Gesture-based interfaces across devices and
methods - Part 61: Single-point gestures for screen readers”

JTC 1 SC35 6
 ISO IEC JTC1 SC35
User Interfaces

(1) Gesture-based UI (ISO 30113 Series)

7
 ISO IEC JTC1 SC35
User Interfaces

Gesture Interface Markup Language

ISO 30113-5: 2019  XML for defining Gestures

8
 ISO IEC JTC1 SC35
User Interfaces

Common System Actions – Single POI

ISO 30113-11: 2017  Single Point Gestures

9
 ISO IEC JTC1 SC35
User Interfaces

Common System Actions – Multiple POI

ISO 30113-12: 2019  Multiple Point Gestures

2-Point Right

3-Point Left

10
 ISO IEC JTC1 SC35
User Interfaces

General guidance for Screen Readers

ISO 30113-60: 2020
 (A): Apple
 (G): Google
 (S): Samsung
 (O): Orange

ISO 30113-61: 2020

 Single POI

ISO 30113-62
 Multiple POI

11
 ISO IEC JTC1 SC35
User Interfaces

(2) Voice UI for PMS (CD 7818)

• Personal Mobility Service

• Navigation: finding a way to destination
• Environment: providing road, traffic,
weather information
• Guidance: providing proper information
and guidance for commuters, athletes, tourists, etc.
• System Status: reporting time, distance, speed, battery, etc.
• Emergency: handling emergency situation

12
 ISO IEC JTC1 SC35
User Interfaces

Conceptual Overview

Recognizing Voice
Commands
PM Device
Location-based
Voice Information (GPS)
Commands PM
User Voice
Service Internet
(Rider Feedback & UI
App Communication
) Information

Database Access
(Command &
Information)

13
 ISO IEC JTC1 SC35
User Interfaces

Requirements and Recommendations (1)

• PMS should be used by users through VUI with earphones
and microphone for safety, performance and convenience.
• PMS should have a capability to be controlled by voice
commands.
• The voice commands shall be used only for the functions
that are not related to safety.
• The voice commands should be able to be received against
ambient noise.

14
 ISO IEC JTC1 SC35
User Interfaces

Requirements and Recommendations (2)

• PMS should return feedback of acceptance or

rejection of voice command to the user.
• The feedback shall be provided to user without
disturbing riding/driving PMD.
• The PMS shall be activated before the users control
the applications using the voice commands.
• The applications shall be able to recognize and execute
the voice commands.

15
 ISO IEC JTC1 SC35
User Interfaces

Requirements and Recommendations (3)

• Speech recognition of voice commands shall be tested as
specified in ISO/IEC 30122-2 such as attributes of the
commands.
• The test results, “sentence recognition rate”, “word
recognition rate” and “word accuracy” shall be high
enough to achieve the user goals.

16
 ISO IEC JTC1 SC35
User Interfaces

(3) Evaluating Usability of NUI (DIS 4944)

• Framework of evaluating usability of NUI

1. User Interfaces for ‘tasks’

 Focusing on tasks performed by users (a general way of the evaluation)

 Evaluating extent of performing task of users to achieve their goals

 Evaluating extent of expressing information for users to accomplish their goals

2. ‘Naturalness’ of interactions
 Focusing on the naturalness

 Evaluating in terms of expectation, behavior and feeling of users with NUI

 Evaluating extent of required learning and training for interacting through NUI 17
 ISO IEC JTC1 SC35

ISO 9241: Usability User Interfaces

Extent to which a system, product or service can be used by

specified users to achieve specified goals with effectiveness,
efficiency and satisfaction in a specified context of use

Effectiveness
Measuring Usability Accuracy and Completeness with which
users achieve specified goals.

Effectiveness
Efficiency
Resources expended in relation to the
accuracy and completeness with which
users achieve goals. Time used, Human
Usability effort expended, etc.

Satisfaction
Efficiency Satisfaction Freedom from discomfort and positive
attitudes towards the use of the product
18
 ISO IEC JTC1 SC35

Framework for the Evaluation

User Interfaces

19
 ISO IEC JTC1 SC35
User Interfaces

Usability Components
• Effectiveness:
 Accuracy: an actual outcome matches an intended outcome.
 Completeness: users can achieve all intended outcomes.
• Efficiency:
 Time used in attempting to achieve a goal.
 Human effort: mental and physical effort to complete a task.
 Financial resources: such as wages, the cost of energy or connectivity.
• Satisfaction:
 Physical responses: include feelings of physical comfort or discomfort.
 Cognitive responses: include attitudes, preferences, and perceptions.
 Emotional responses: result from experience while using the object of
interest.
20
 ISO IEC JTC1 SC35
User Interfaces

Evaluation of Usability (1)

• Suitability for the user's tasks: The users perform tasks in a reasonable
manner to how they would perform in the real world with everyday interactions (if
the computer was not involved).

• Self-descriptiveness: The system with NUI presents appropriate information to

the user in a manner that is related to how the information exists in the real world
(rather than in a manner that is optimized for the computer).

• Conformity with user expectations: The system with NUI recognises and
processes predictable user interactions in a predictable manner that is consistent
with the real world (rather than designed based on typical human-computer
interaction styles).

• Learnability: The system with NUI supports the users in the discovery and
exploration of its capabilities and information (without needing for formal training
regarding the user interface and interactions before they can use the system).

21
 ISO IEC JTC1 SC35
User Interfaces

Evaluation of Usability (2)

• Controllability: The system with NUI allows the user to interact with the NUI
in the manner and sequence that is most natural to their preferred manner
(rather than in a single prescribed manner).

• Use error robustness: The system with NUI allows the users to try
different interactions while avoiding and minimizing any resulting use errors
and by providing suitable error recovery where needed.

• User engagement: The system with NUI invites and motivates the users to
interact with the NUI in a similar manner to everyday human behaviors in the
real world.

22
 ISO IEC JTC1 SC35
User Interfaces

Evaluation based on use

• Identify usability defect and potential usability problem:

• Checks if the system satisfies the user requirements

• Observation of users:

• Observes user behaviours, performance and reactions

• User surveys:

• Qualitative and quantitative studies including interviews

23
 ISO IEC JTC1 SC35
User Interfaces

Usability Evaluation Report

The report can include the followings: (ISO 25066)

1. Executive summary
2. Description of the object evaluation
3. Purpose of evaluation
4. Method
5. Procedure
6. Results
7. Interpretation of results and recommendations (opt.)

24