NAME LAU EE LIN

ID NUMBER TP061698

CLASS & CODE APD2F2305ID

COURSE BACHELOR OF ARTS (HONOURS) IN INDUSTRIAL DESIGN

MODULE NAME & CODE PRODUCT DESIGN

LECTURER CHRISTINE LIM PEI SHIN

CENTER APIIT
Table of Contents
VISUAL IMPAIRMENTS ....................................................................................................................................................... 3

VISUAL/SIGHT ...................................................................................................................................................................................... 3
BEHAVIOURS & NEEDS ........................................................................................................................................................................... 3
PROBLEMS ........................................................................................................................................................................................... 3
TOOLS ................................................................................................................................................................................................ 4

RESEARCH ......................................................................................................................................................................... 4

ISSUES................................................................................................................................................................................................. 4
Struggle faced by Visual Impairments ........................................................................................................................................ 4
Ways to Improve .......................................................................................................................................................................... 5

EXISTING PRODUCTS ......................................................................................................................................................... 5

ASSISTIVE MOBILE INFORMATION ROBOT (AMIR) ...................................................................................................................................... 5

General Robot Type: .................................................................................................................................................................... 7
Shopping Robot Cart.................................................................................................................................................................... 8
Technologies .............................................................................................................................................................................. 10
ENVIRONMENT AND INTERIOR ............................................................................................................................................................... 13

ACCESSIBLE SHOPPING SYSTEM FOR VISUAL IMPAIRED ......................................................................................................14

........................................................................................................................................................................................14

LAYOUT ............................................................................................................................................................................................. 14
MATERIALS AND FINISHES ..................................................................................................................................................................... 17

TECHNOLOGIES ................................................................................................................................................................17

ARTIFICIAL INTELLIGENCE (AI) ............................................................................................................................................................... 17

Type of AI.................................................................................................................................................................................... 18
Examples of AI: .......................................................................................................................................................................... 18
RFID ................................................................................................................................................................................................ 18
NFC ................................................................................................................................................................................................. 18
Application: ................................................................................................................................................................................ 18
ULTRASONIC SENSOR ........................................................................................................................................................................... 18
Applications: .............................................................................................................................................................................. 18
SLAM .............................................................................................................................................................................................. 19

DESIGN CRITERIA..............................................................................................................................................................19

PRIMARY: .......................................................................................................................................................................................... 19
SECONDARY ....................................................................................................................................................................................... 19

REFERENCES ....................................................................................................................................................................20
Visual Impairments
Visual/Sight
Typically people think of full blindness when discussing web accessibility for the visually impaired. There are many different visual challenges
people face ranging from full blindness to tunnel vision or the inability to see colour or certain colours. Over time as we age our vision
deteriorates, how many of us require prescription glasses? I certainly do and occasionally browse online without them because I’ve misplaced
them or they are in another room browsing online then becomes harder on poorly designed websites. Finally, even those with good vision can
be affected if they are in poor lighting conditions, think about glare from the sun.

Blindness — loss of vision in both eyes

Central field loss — only seeing the edges of the visual field
Clouded vision — like looking through a foggy window
Colour blindness — inability to perceive any colour or certain colours
Contrast sensitivity — the contrast between objects and their backgrounds is reduced
Poor acuity — vision that is not sharp
Poor lighting — typically when outdoors in the sun
Tunnel vision — only seeing the middle of the visual field

Diabetes Retinopathy Cataract

Full. Vision
Visiondisbetidis s

Retinitis Pigmentosa Only light vision

AMD

Behaviours & needs

Visually impaired users require a significant amount of trust in people and technology to feel comfortable. When using digita l displays they
may only be able to see a small portion of the content at a time. To recreate this try reading this section by lightly clenching your fist to leave
a small hole and putting it up to your eye, now read on.
Customising settings for fonts, colours, and spacing is important, especially enlarging text sizes and images, in some cases up to 1200%
magnification. This allows them to cater the interface to their needs. They also expect a common layout to remain oriented as they will start
in the top left corner and work left to bright, top to bottom. Users with severe visual issues may not use a monitor at all and need all tasks to
be accomplishable with a keyboard & screenreader. This means tabbing orders and skip links need to work correctly, it also means the correct
labels have been added in the code for screen readers to use and perform commands.

Problems
A lack of full keyboard support is still a big issue, as well as interactions that are not executable by other tools that they use. Define the tools
these users need and test that they work for every feature implemented.
Other problems include alt text that is missing on images and controls or hasn’t been updated, providing incorrect or missing information to
screen readers. Content that prevents resizing or that loses information when resized, for example, content overlapping or hidden off-screen.
A lack of visual orientation cues like headings, navigational aids, and background colour changes which prevent users from finding their way
around the interface. Poor foreground and background contrast on text and media content as well as missing textual or audio descriptions on
videos. Lastly, interactions that don’t provide feedback on the component itself but elsewhere on the interface outside of thei r visual field
causing confusion and them to question: “Did that do anything?”.

Tools
Some of the tools used include:
Audio descriptions
Keyboard only
Magnification (zoom)
Refreshable braille display
Screenreader
Text-to-speech
Voice recognition

Research
Issues
Struggle faced by Visual Impairments
- Negative behaviour from salespeople
- Have trouble distinguishing prices & size of garment
- Navigating
- ambient factor like lighting, colour perception and signage
- self checking-out
- locating mall entrance
- store is not pet-friendly
A study by the National Council of the Blind Ireland (NCBI) estimated the barriers in grocery stores as follows:
• 79% stated that the shop is not easy to navigate through the different zones
• 96% had difficulties accessing the information on signs
• 95% had difficulties accessing the information on labels
• 73% face obstacles in the aisles
• 89% mentioned that to improve their shopping experience they want their bill to be read to them.

Ways to Improve
- wide walkways
- keep aisles as free as possible from any obstacles
- dark tiles outlining white tiles
- good lighting & signage （esp at important places

- natural lightning
- Braille signage
- Ceiling hung signage
- step counting feature
- different flooring to differentiate different department
- use audio

‘Standardise the bag location. Use different flooring material. Signage. Put a line in the flooring in the aisles section to h elp know that we are
heading forward and not in a zigzag direction.’ Blind Participant

‘Standard floor layout. Different floor colours with different materials. Staff training. Remove random displays from the middle of aisles.’ Visually
impaired participant M

‘A good layout design, wider aisles, good lighting. Auditory signs.’ Visually impaired participant N

‘Contrast in colours. Lighting. Wide aisles to accommodate a guide dog. Auditory signs to alert the person to what is in the aisle.’
Accessibility Specialist

Existing Products
Assistive Mobile Information Robot (AMIR)
Robotic shopping assistants are mostly utilised in navigation activities to satisfy the shopping needs/interests of the user, which are frequently
represented by a shopping list. A customer's ability to navigate a supermarket in search of products is not always easy. Fin ding the required
department or aisle frequently consumes the majority of the time that shoppers spend in a store. This difficulty can be solved by using robotic
platforms, saving the buyer’s time and energy.

The robotic system consists of a guide robot, a cart robot, and cameras. The guide robot is an autonomous mobile robot with a localization
function, followed by the cart robot. The cameras are used to detect obstacles and people around the robot. The obstacle detection system
uses 32 sets of ultrasonic sensors connected in series. In addition, the platform has a second detection system consisting of three sets of laser
rangefinders.
These classified assistive solutions into three categories: tag-based systems (usually, RFID and NFC tags), computer vision-based systems, and
hybrid systems.This platform looks like a hand cart. The robot’s hardware contains radio frequency identification (RFID) tags , a platform
microcontroller, and a laser rangefinder. RFID tags can be attached to any product or clothing and do not require an external power source.
In addition to this, it is a simple and cheap solution. The device software consists of three components: a user interface, a route planner, and
a behavior manager. The route planner and the behavior manager partially implement spatial semantic hierarchy. Following it, information
about space is divided into four levels: control level, causal level, topological level, and metric level. RoboCart has the t wo following
disadvantages, namely difficulty in turning around in aisles and limited spatial sensing (50 cm from floor level), making difficult the detection
of billboards installed on shelves.

Computer vision-based systems identify objects without RFID or NFC tags, directly utilizing information about features of the objects. The
disadvantage of this approach is that additional devices are often prerequisite for the system to function. Hybrid systems co mbine strong
points from both approaches. For example, a smartphone camera is used to identify QR-codes on product shelves and RFID tags to navigate
through a store. These devices can automatically count the contents of the carts wireless sensor network (WSN) to automatize invoice
processing. An article demonstrated the successful use of an ultra-high frequency (UHF) RFID system mounted on a smart shopping cart for
the same purpose.
General Robot Type:

NAME Description
Robot Waiter A robot waiter is exactly what it
sounds like: a robot that takes
orders and delivers food to diners at
a restaurant. While these robot
waiters generally do not take
orders, they deliver food to
customers and even take away dirty
dishes.

Features:
-trays
-touchscreen
-infrared camera
Navigation Robot Mini Cheetah
a robot dog to help in ways similar
to real guide dogs.

Features:
-path generator
-navigation
-obstacle detection
-map creation (SLAM)
- human tracking depth camera

Budgee Robot Features:

Carry up to 50lbs
Obstacle Avoidance

Technologies:
Sonar sensor

Budgee follows a transmitter that

you carry or wear. The transmitter
can be attached to a wheelchair,
scooter or belt.
 Mechanical:
Weight 22lbs
Lithium ion battery
Lasts 10 hours
Recharge in 3 hours

Shopping Robot Cart

NAME Description Pros and Cons

DASH Features: Pros:
Navigation Auto charging
Path planning Equipped with shopping list
Obstacle avoidance On-board check out
On-board checkout Built in scale
docking station & automatic recharging
Equipped with shopping list Cons:
Customisable intuitive user interface Limited capacity
Touchscreen
Parts:
Screen
Barcode reader
Cart
Machine body

Technologies :
AI (computer vision algorithm)
Sensor fusion
Alexa shopping list

How to use:
scan a QR code in the app with the reader
on the cart.
place your one or two bags in the cart and
begin shopping.
As put an item, need to wait to hear a beep

The cart’s main feature is a sensor array in

the rim that uses AI-powered cameras and
barcode scanners to identify whatever you
put in (or take out) and create a live receipt.
A touchscreen lets you check how much
you’ve spent and a built-in scale lets you
weigh loose items.
Gitamini Features: Pros:
Cargo-carrying Pairing with user (follow mode)
identify and follow its owner
Carry up to 20 pounds for 21 miles Cons:
Parts: Small capacity
two wheels Limited features:
a central trunk No self checkout
a machine vision system No weight scaling

Technologies:
array of cameras
Sensors
Radar

To activate follow mode, stand in front of

the Gitamini and tap a pairing button. The
robot will then lock on to you using vision
only (no GPS or Bluetooth are utilized) and
will follow you at speeds of up to 6mph.

Weight: 28 pounds

ELI Self-driving Robot Features: Pros:

Obstacle avoidance Auto rechargeable
Human Voice Recognition Automatic payment
Automatic payment Human voice recognition
Auto recharging
Cons:
Technologies: Touchscreen
Smart sensor Too bulky
Liquid-crystal display screen
AI technology

Mechanical：

Weight 140kg
Carry up to 80kg
2.5 m/s
Recharge <2h

Shopic - smart cart The Shopic device, a clip-on unit that Pros:
shoppers take from a rack and attach to an Compact size
ordinary cart, uses computer vision Easy to Use
algorithms. The AI can identify every item
placed in the cart and detects when an item Cons:
is added or removed from the cart with Touchscreen
99.4% accuracy Discover items
AI detection in cart
 Features:
Scan and go (self checkout payment)
Navigation
Asking for customer service
Discover items
Personalised shopping experience

Parts:
Charger-shelf unit
Plastic mounts on existing carts
Screen body

Technologies:
AI
Camera
Touch screen display
RFID
Barcode reader

How to use:
Grab a cart
Attach a clip-on
Pay on cart and go

Amazon Smart Trolley
The Dash Cart is a smart cart that makes Pros:
grocery trips faster and more personalized Built in scale
than ever. Find items quickly and easily. Add,
remove, and weigh items right in your Dash Cons:
Cart. When you’re done shopping, skip the Touchscreen
checkout line and roll out to your car.

Features:
Built-in weight scale
All-day battery life
Alexa shopping list

Technologies:
AI-powered camera
Barcode scanner
Load sensor

Technologies
Varieties Description Application Pros & Cons
Artificial Intelligence (AI) Type 2 AI : Limited memory. These AI Shopic - smart cart
systems have memory, so they can
use past experiences to inform
future decisions. Some of the
decision-making functions in self-
driving cars are designed this way.

Amazon Smart Trolley

Application:
AI in business
AI in media (targeted advertising)
AI in transportation
Radio Frequency Identification a technology whereby digital data Tap-and-go credit card payment Pros:
(RFID) encoded in RFID tags or smart labels Can be read outside the
(defined below) are captured by a Retail sales line-of-sight
reader via radio waves.
Stock tracking Time saving
Application:
Asset tracking Data can be updated in real
ID Badging time
NFC
Read by default and billing
information is generate

Cons:
Require power source
Barcode reader A barcode reader, also called a price Supply chain to track product Pros:
scanner or point-of-sale (POS) Reduce pricing error
scanner, is a hand-held or stationary
input device used to capture and No power source required
read information contained in a
barcode. Cons:
Need a clear line of sight

Need for barcodes to be

printed on labels
 Barcode reader is slow
(read one product at a
time)

Data can’t be changed

Near Field Communications A short-range wireless technology Handle mobile payment Pros:
(NFC) that allows your phone to act as a Two-ways commutation
transit pass or credit card, quickly Digital cards
transfer data, or instantly pair with Store more information
Bluetooth devices like headphones Digital keys
and speakers. Cons:
Short range

Require actuon by user

Load Cell An electronic device that converts Bag filling Pros:
tension and compression forces into Accuracy
a corresponding electrical signal. To Supermarket scales
determine the weight of an object. Easy available

Application: Cons:
Conveyor belt scales Bulky in size
Smart trolley
Infrared Sensor A simple electronic device which Flame detector Pros:
emits and detects radiation in order Item counter Low power requirement
to find out object or obstacle in its Do not require contact
range, it’s features are heat and
motion sensing. Cons:
Require line of sight
Range: 0.75 to 1000um Limited range

Application:
Proximity sensor
Item counter
Human body detection
Wide beam sensor See obstacle over a wider field of Unmanned aerial vehicle Cons:
view and prevent it from getting too Field of detection is too
close to any one target wide

Application:
Cars obstacle detection
 Ultrasonic sensor An ultrasonic sensor emits sound Vehicle detection Pros:
waves toward an object and People detection Can detect liquids level
determines its distance by detecting Liquid detection
reflected waves Able to detect transparent
object
Application:
Proximity sensor (Self-parking) Able to tell how far the
Finger print sensor object is

More accurate

More compact

Cons:
Not as susceptible to
interference of smoke and
gas

Slower compare to light

SLAM(simultaneous localization
and mappin)
a method used for autonomous Robot vacuum
vehicles that lets you build a map
and localize your vehicle in that map
at the same time. SLAM algorithms
allow the vehicle to map out
unknown environments.

From the research mapping above, AI is the crucial element for the design as it able to generate the best shopping plan and route for the visual
impairments. Besides that, AI can count the steps automatically throughout the store. AI can also use audio to tell the user what is placed in
to the cart.

RFID is the ideal solution among RFID, barcode, and NFC as it is faster. RFID allow the design recognise, read and give out information like name,
weight and price of the product.

Load cell also works well when comes into this topic. As it can be used in the design to weight the product.

Among the other sensors system, ultrasonic has the most accuracy and it can detect transparent objects although it works slower than light.
Ultrasonic sensor is use to detect obstacles in front of the user.

The aim of this project is to develop a solution that supports people with visual impairments when shopping in grocery stores. The solution
should support the person in parts of the shopping experience, which consists of navigation, item-identification, and payment.

Environment and Interior

- wide walkways
- dark tiles outlining white tiles
- good lighting & signage
- natural lightning
- Braille signage
- Ceiling hung signage
- step counting feature
- different flooring to differentiate different department
- pet-friendly
- no touchscreen

Accessible Shopping System for Visual Impaired

Layout
Breaking these up into shorter sections of varying heights will make it much easier for these customers.
Having higher shelves around the perimeter of the shop with the lowest ones in the centre of the space will allow for a clear view around the
store. Customers with a cognitive or visual impairment should also be able to identify the size and layout of the store from the entrance and
be directed around the store by focussing their vision towards key visual merchandise displays.

Colours wrap around on the floor between the end of aisle signs, helping those who rely on their peripheral vision to clearly identify which
aisle they’re going into. The use of bright colours and large numbers aid the effective communication and wayfinding process for all.

Beside that, eye-level signage is used to divide each aisle. Where previously, these were commonly small signs above the to shelf, this new
design makes it easy to see how far down the aisle you have to go to look for the item you are after.

Ensuring similar products are grouped together to help people find what they need quickly will minimise confusion and make fo r an easier
shopping experience.

Clear aisles with a mixture of short, low displays and large perimeter bays
Colours wrap around on the floor between the end of aisle signs.

Eye-level signage is used to divide each aisle.

Materials and finishes
Natural materials may be more recognisable for those with dementia too. Evidence suggests that using a palette of natural materials such as
timber, cork, rubber and porcelain can satisfy the needs of those with shoppers who prefer more tactile, rougher surfaces. For example, nature
stuff like vegetable and fruits department use timber as materials able to let the shopper more understand with it.

Floors should have a consistent tone throughout so that for someone with a visual impairment it is clear that it’s one level surface. A bold
change in colour can be perceived as a step or a change in level and this can result in unnecessary falls.

Similarly, shiny and reflective materials should be avoided because they can cause confusion for both the visually impaired and those living
with a cognitive condition such as dementia. Shiny surfaces often look like they are wet and can cause discomfort for anyone having to walk
through the space.

Having a clutter-free ceiling can help visually impaired customers navigate more easily. This is because, as the least chaotic space, many focus
on it to help them determine the size and shape of a room.

Natural materials like timber can be calming for customers with a cognitive impairment

Technologies
Artificial intelligence (AI)
AI is important for its potential to change how we live, work and play. It has been effectively used in business to automate tasks done by
humans, including customer service work, lead generation, fraud detection and quality control. In a number of areas, AI can perform tasks
much better than humans. Particularly when it comes to repetitive, detail-oriented tasks, such as analyzing large numbers of legal documents
to ensure relevant fields are filled in properly, AI tools often complete jobs quickly and with relatively few errors. Because of the massive data
sets it can process, AI can also give enterprises insights into their operations they might not have been aware of. The rapidly expanding
population of generative AI tools will be important in fields ranging from education and marketing to product design.

Type of AI
o Reactive Machines
o Limited Memory
o Theory of Mind

Examples of AI:
o Siri, Alexa and other smart assistants
o Self-driving cars
o Google search
o Conversational bots
o Email spam filters
o Netflix’s recommendations

RFID
RFID belongs to a group of technologies referred to as Automatic Identification and Data Capture (AIDC). AIDC methods automatically identify
objects, collect data about them, and enter those data directly into computer systems with little or no human intervention. R FID methods
utilize radio waves to accomplish this. At a simple level, RFID systems consist of three components: an RFID tag or smart lab el, an RFID reader,
and an antenna. RFID tags contain an integrated circuit and an antenna, which are used to transmit data to the RFID reader (also called an
interrogator). The reader then converts the radio waves to a more usable form of data. Information collected from the tags is then transferred
through a communications interface to a host computer system, where the data can be stored in a database and analyzed at a later time.

NFC
A short-range wireless technology that allows your phone to act as a transit pass or credit card, quickly transfer data, or instantly pair with
Bluetooth devices like headphones and speakers.
NFC is based on RFID technology, but has a much lower transmission range. NFC, however, only has a maximum range of a few centimeters, at
most. And in most smartphone-related applications, you’ll find that the software will only initiate communication if there’s physical contact.
This is to prevent accidental triggers — especially important now that the technology is used for transferring sensitive data.
Another noteworthy point is that devices can act as either an NFC reader or tag. This bidirectional capability allows you to use one piece of
hardware — such as your smartphone — for all kinds of different applications.

Application:
- mobile payment app

Ultrasonic Sensor
Applications:
o Liquid level control
o Full detection
o Thread or wire break detection
o Robotic sensing
o Stacking height control
o People detection for counting
o Vehicle detection for car wash and automotive assembly
SLAM
SLAM can use information such as the number of wheel revolutions and data from cameras and other imaging sensors to determine the
amount of movement needed. This is called localization. The robot can also simultaneously use the camera and other sensors to create a map
of the obstacles in its surroundings and avoid going the same area twice. This is called mapping.
SLAM is useful in many other applications such as navigating a fleet of mobile robots to arrange shelves in a warehouse, parking a self-driving
car in an empty spot, or delivering a package by navigating a drone in an unknown environment. MATLAB and Simulink provide SLAM algorithms,
functions, and analysis tools to develop various applications. You can implement simultaneous localization and mapping along with other tasks
such as sensor fusion, object tracking, path planning and path following.

Design Criteria
Primary:
1. Blind Operation
2. Audio Description
3. Navigation
4. Depth Camera
5. Canesitter

Secondary
1. Weight Scaling
2. Auto Check-out
3. Auto charging
References
Burns, E., Laskowski, N., & Tucci, L. (2023). artificial intelligence (AI). Enterprise

AI. https://www.techtarget.com/searchenterpriseai/definition/AI-Artificial-Intelligence
DASH robotic shopping cart. (2018, January 14). 5elementsrobotics. https://5elementsrobotics.com/dash-robotic-shopping-
cart/
Eli robot: smart service robot. (n.d.). https://robot.cfp.co.ir/en/newsdetail/11174
Gil Press. (2023, February 21). Shopic To Deploy 2000 AI-Powered Smart Shopping

Carts. Forbes. https://www.forbes.com/sites/gilpress/2023/02/21/shopic-to-deploy-2000-ai-powered-smart-

shopping-carts/?sh=1c886641bdd1
Hanna, K. T. (2022). barcode reader. WhatIs.com. https://www.techtarget.com/whatis/definition/barcode-reader-POS-
scanner-bar-code-reader-price-scanner
Lewis, V. (2023, April 14). Making Clothing Stores Accessible For Low Vision. Veroniiiica. https://veroniiiica.com/clothing-stores-
accessible-low-vision/
Platform - Shopic. (2023, January 24). Shopic. https://www.shopic.co/platform/
Robotnik. (2023, March 8). Eli Robot - Autonomous Shopping Cart | Robotnik®. https://robotnik.eu/products/customization/eli/
Ryumin, D., Kagirov, I., Axyonov, A., Pavlyuk, N., Saveliev, A., Kipyatkova, I. S., Železný, M., Mporas, I., & Karpov, A. (2020). A multimodal user

interface for an assistive robotic shopping cart. Electronics, 9(12), 2093. https://doi.org/10.3390/electronics9122093
Shetty, A., & Shetty, A. (2018). Infrared Sensor – How it Works, Types, Applications, Advantage &

Disadvantage. electricalfundablog.com. https://electricalfundablog.com/infrared-sensor/

Test, T. (2019). Wide Beam Ultrasonic Sensor For Obstacle Detection. MaxBotix. https://maxbotix.com/blogs/blog/wide-beam-
ultrasonic-sensor#:~:text=A%20wide%20beam%20sensor%20is,close%20to%20any%20one%20target .
Vincent, J. (2021, September 22). Gitamini is a cute, compact, cargo-carrying robot that will follow you around like a dog. The

Verge. https://www.theverge.com/2021/9/22/22685649/gita-gitamini-cargo-carrying-robot-piaggio
What is a Load Sensor? (2019, October 8). Fierce Electronics. https://www.fierceelectronics.com/sensors/what-a-load-sensor
What is an Ultrasonic Sensor? (2019a, October 7). Fierce Electronics. https://www.fierceelectronics.com/sensors/what-
ultrasonic-sensor#:~:text=An%20ultrasonic%20sensor%20is%20an,sound%20that%20humans%20can%20hear ).