Int. J.

Human–Computer Studies 170 (2023) 102966

Contents lists available at ScienceDirect

International Journal of Human - Computer Studies

journal homepage: www.elsevier.com/locate/ijhcs

Eyes can draw: A high-fidelity free-eye drawing method with unimodal gaze
control
Lida Huang a ,∗, Thomas Westin a , Mirjam Palosaari Eladhari a , Sindri Magnússon a , Hao Chen b
a
Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden
b
People’s Hospital of Guangxi Zhuang Autonomous Region, Nanning, China

ARTICLE INFO ABSTRACT

MSC: EyeCompass is a novel free-eye drawing system enabling high-fidelity and efficient free-eye drawing through
00-01 unimodal gaze control, addressing the bottlenecks of gaze-control drawing. EyeCompass helps people to draw
99-00 using only their eyes, which is of value to people with motor disabilities. Currently, there is no effective
Keywords: gaze-control drawing application due to multiple challenges including involuntary eye movements, conflicts
Free-eye drawing between visuomotor transformation and ocular observation, gaze trajectory control, and inherent eye-tracking
Eye tracking errors. EyeCompass addresses this using two initial gaze-control drawing mechanisms: brush damping dynamics
Unimodal gaze control
and the gaze-oriented method. The user experiments compare the existing gaze-control drawing method and
EyeCompass, showing significant improvements in the drawing performance of the mechanisms concerned.
The field study conducted with motor-disabled people produced various creative graphics and indicates good
usability of the system. Our studies indicate that EyeCompass is a high-fidelity, accurate, feasible free-eye
drawing method for creating artistic works via unimodal gaze control.

1. Introduction
Free-hand drawing is the primary approach employed in able-
bodied drawing and an important component of creative and artistic
activities, especially in the first conceptual phase. Free-hand drawing
stimulated by eyes is known as ‘‘free-eye drawing’’ (Tchalenko, 2001).
For motor-disabled people, free-eye drawing enables them to express
their ideas visually; thus, it is important to develop accessible free-eye
drawing methods for the motor disabled.
However, currently, no research has been able to facilitate gaze-
control or free-eye drawings with high-fidelity graphics and system ac-
cessibility. There are many challenges related to drawing high-fidelity
graphics via gaze control. For instance, early gaze-control drawing
applications cannot distinguish between viewing and drawing, as the
gaze signal is continuous (Gips and Olivieri, 1996; Heikkilä, 2013a,b;
Hornof et al., 2003; Meyer and Dittmar, 2009; Van der Kamp and
Sundstedt, 2011). In addition, unintentional eye movements and eye
jitters can cause involuntary selections, the so-called ‘‘Midas-Touch’’
problem (Chin et al., 2008; Istance et al., 2008; Jacob, 1991). As the
current gaze-control drawing method generates drawn lines from the
position of the user’s gaze points on screen in real time, any involuntary
saccades will cause drawing errors. Furthermore, drawing by hand
is a dual task, where the eyes are used for viewing and the hands
for drawing. By contrast, gaze-control drawing involves performing
a dual task with the eyes only, causing conflict between visuomotor
transformation and ocular observation (Gips and Olivieri, 1996; Hornof
et al., 2003).
Fundamentally, free-eye drawing requires emulation of the click-
and-drag mouse event. To the best of the authors’ knowledge, there
is a scarcity of recent studies focusing on the interactive modes or
methods to facilitate accurate control of gaze trajectories in real time.
Consequently, there is neither an effective gaze-control interaction
method nor an application for conducting drawing and related graphic
arts. Lastly, eye-tracking techniques make unavoidable errors when
estimating the actual gaze position, causing a disparity between what
the user’s gaze produces and what they are seeking to achieve.
Gaze-control modality is also a challenge for free-eye drawing.
Unimodal gaze control has many advantages; for example, inclusion
for motor-disabled people. As a single modality, it is also convenient
to integrate with additional input devices to extend its interaction.
However, unimodal gaze control is limited in that it cannot facilitate
simultaneous viewing and drawing. Many related studies have em-
ployed multimodal gaze control (Van der Kamp and Sundstedt, 2011;
Dziemian et al., 2016; Creed et al., 2020) to allow eyes to accomplish
the dual task. However, multimodal gaze control is less accessible as
it excludes severely motor-disabled people whose hand or foot use is

∗ Corresponding author.
E-mail address: lyyda@hotmail.com (L. Huang).

https://doi.org/10.1016/j.ijhcs.2022.102966
Received 24 January 2022; Received in revised form 4 November 2022; Accepted 13 November 2022
Available online 21 November 2022
1071-5819/© 2022 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
L. Huang et al.
impaired (Cecotti, 2016; Dziemian et al., 2016; Parisay et al., 2021). It
also has a greater requirement for equipment.
Almost a decade has passed since the latest gaze-control drawing
literature was published in 2013 (Heikkilä, 2013a,b), there is currently
neither a solution nor a viable free-eye drawing system to address
the aforementioned problems (i.e., involuntary eye movements, the
conflict between visuomotor transformation and ocular observation,
gaze trajectory control, inherent eye-tracking errors, and unimodal gaze
control). Unfortunately, existing gaze-control methods reported in the
earlier literature (Gips and Olivieri, 1996; Heikkilä, 2013a,b; Hornof
et al., 2003; Meyer and Dittmar, 2009; Van der Kamp and Sundstedt,
2011) are feasible neither for visual creativity at a free-hand drawing
level nor with modern technological advancements.
This paper therefore proposes a novel free-eye drawing system
(EyeCompass) that enables high-fidelity and efficient free-eye drawing
via unimodal gaze control. This system aims to address the bottlenecks
of accurate click-and-drag for gaze interaction and innovate gaze-
control drawing methods. There are three important components in the
techniques used by EyeCompass: (1) the ability of the eyes to ‘acquire’
the brush and navigate the input, rather than acting as direct input;
(2) the use of visual cursors in the system interface; and (3) the way
in which smooth pursuit is triggered and employed in gaze interaction.
These three components integrate within EyeCompass and can also be
generalized to universal eye-tracking applications.
The structure of this paper is as follows. First, the previous literature
in gaze control drawing is reviewed in Section 2. Then, the rationale
of gaze control drawing is discussed in Section 3, while the design
principles and the techniques used in EyeCompass are introduced in
Sections 4 and 5. Then, in Sections 6 to 8, the implementation of
two user experiments is presented to explore how our method affects
graphical fidelity. In Section 9, a field study among motor-disabled
people demonstrating the usability of EyeCompass in context by its
intended users is reported. Finally, the implications of this study’s
findings for future gaze-control interaction design are discussed in
Sections 10 and 11.
As stated, the overall aim of the work presented in this paper is to
create a system that can help people to draw using their eyes instead
of using their limbs. In particular, the paper makes the following five
contributions:
(1) A retrospective review of free-eye drawing techniques across dis-
ciplines, including eye physiology, eye movement patterns in drawing,
and eye-tracking technologies, and systematic identification of multiple
challenges in the field in terms of the usability of existing free-eye
drawing techniques.
(2) A novel and open-source unimodal gaze-control drawing system
and drag- and-drop emulation that can be generalized to common
graphics software and similar gaze interaction. The system has been
made publicly available to enable future researchers to build new
functionalities on top of our results.
(3) Brush-damping dynamics that can stabilize gaze-trajectory con-
trol, enhance drawing accuracy, and enable the user to create smooth
curves that were previously unattainable.
(4) A gaze-oriented mechanism leveraging smooth pursuits in an
innovative way so that the user can control the visual stimulus, reduce
involuntary eye movements, and navigate the gaze point freely.
(5) A comparison of our methods to the existing free-eye drawing
methods in terms of: a) drawing performance and system robustness,
and b) usability and accessibility among motor-disabled people.
2. Related research
2.1. Unimodal gaze-control drawing methods
The earliest gaze-control drawing function was created in the Ea-
gleEyes system (Gips and Olivieri, 1996) in 1996. This system employed
2
International Journal of Human - Computer Studies 170 (2023) 102966
a simple free-eye drawing (Tchalenko, 2001) function using electroocu-
lography (EOG) to translate gaze-trajectory signals into colored lines.
The study identified two significant problems with the free-eye drawing
method. First, it could not distinguish viewing and drawing, as the eye-
tracking signal was continuous. Second, the study noted the difficulty
in accurately controlling the gaze for drawing. Further studies of free-
eye drawing subsequently stagnated, as the approach was considered
error-prone and impracticable.
To address the conflicts between viewing and drawing, EyeDraw
proposed a state transition between ‘‘Looking’’ and ‘‘Drawing’’ for
eyes to enable ocular observation (Hornof et al., 2003). However,
precise control of the cursor was indicated to be problematic in the
user feedback on EyeDraw (Hornof et al., 2003). Later, Yeo and Chiu
(2006) classified the viewing and drawing involved in eye drawing
by interpreting natural eye movement patterns and distinguishing the
distances between gaze shifts and eye-behavior jitters; however, they
did not report their results.
Due to the lack of control of gaze trajectories, previous researchers
halted exploration of the free-eye drawing approach and adopted other
eye drawing strategies, the most common of which provided a set of
basic shapes to be applied to a canvas. For example, the EyeDraw draw-
ing tool was based on a system of built-in geometries, namely straight
lines, rectangles, and ellipses, as well as colors. Another application,
EyeArt (Meyer and Dittmar, 2009) extended the preset geometries
to include, for example, circles, text, polylines, and polygons. It also
supported broader canvas editing functions, such as eraser, color fill,
and undo. The most recent gaze-control drawing prototype was Eye-
Sketch (Heikkilä, 2013a,b), developed in 2013, which utilizes a similar
drawing tool to EyeDraw for rectangles, ellipses, and lines.
There are, to our knowledge, none of the developed applications
in the previous studies are available. Currently, the most used gaze-
control drawing applications among people with motor disabilities are
traditional graphics software, such as Photoshop, Windows Paint, and
Inkscape, as revealed by the latest gaze interaction research among
individuals with motor disabilities (Huang et al., 2022). As conven-
tional software interfaces cannot support gaze control directly, motor-
disabled people control these applications via an indirect control,
specifically, emulating mouse and keyboard devices through a toolbar
at the desktop borders and replicating mouse and keyboard events
using dwell click (Menges et al., 2019). Tobii Windows Control1 and
Optikey2 are typical indirect gaze control tools available in the market.
Both the results published for previous applications and indirect
gaze control in practice have shown a lack of precision and creative
flexibility (Huang et al., 2022). The reason for this is that the existing
gaze-control drawing methods are all based on a common form of
gaze-control mechanism: ‘‘what you look at is what you get’’ (Jacob,
1991). Specifically, this means that the gaze position of the user is
where the pixel is generated on the canvas. Recently, Streichert et al.
(2020) compared the drawing performance via this type of gaze-control
modality to the performance via touch input. The error rate with
the eye tracker (mean=0.274 ± 0.148) was significantly higher than
the error rate with touch input (mean=0.098 ± 0.020), revealing the
infeasibility of precise drawing tasks via current modalities.
In summary, the common problem of existing studies is the inability
to replicate free-hand drawing and control real-time gaze trajectories
accurately. To the best of the authors’ knowledge, there have been no
further improvements in gaze-based free-eye brush control, especially
for drawing high-fidelity sketches via gaze-control drawing.
1
https://www.tobiidynavox.com/products/windows-control/
2
http://www.optikey.org/
L. Huang et al.
2.2. Multimodal gaze-control drawing methods
In 2011, Van der Kamp and Sundstedt (2011) introduced a multi-
modal drawing program managed via gaze and voice input. The voice
commands started and stopped the drawing, while the gaze commands
determined the cursor position. This approach avoided activating draw-
ing using dwell click and aimed to negate the ‘‘Midas touch’’ problem.
The program included six features: line, rectangle, ellipse, polyline,
flood fill, and curves. However, this multimodal approach raised system
demands, including voice recognition and hardware issues, namely
with the microphone and graphics processing unit (GPU). It also ex-
cluded severely motor-disabled people who are incapable of speech.
Nevertheless, this approach did address the aforementioned difficulties
with gaze-controlled drawing.
Dziemian et al. (2016) presented a gaze-control robotic arm for
writing and drawing in a large three-dimensional space, 87 cm from
the eye tracker, and with an approximately 84 × 59 cm workspace
area. The design utilized a 3D eye-movement data flow to control the
drawing task and gaze positions to determine the robot’s endpoints on
the canvas. The attachment or detachment of the robotic arm from
the canvas was executed using displacements of eye-tracking data via
tilting the head forward (toward) and backward from the canvas. This
technology demonstrated that eye-tracking is a powerful tool for assist-
ing motor-disabled people in performing graphic activities. However,
the high-end nature of the gaze-based tele-prosthetic (UR10 robot arm)
used meant that it was unavailable to ordinary users. The authors also
noted that the need to move the head was a disadvantage that would
particularly apply to severely paralyzed people.
In terms of the other similar extant research on gaze-control visual
design, Creed et al. (2020) proposed a novel creative design application
called Sakura that used multimodal gaze interaction, in which gaze
control cooperated with a mechanical switch, and included a gaze-
based interface with 100 pixels padding and button size. Its use and
functions included adding texts, shapes (squares, circles, lines, and vec-
tors), importing images, scrolling the canvas, alignment, and preview.
Nevertheless, Sakura was not tailored for specialist drawing functions,
and it could not facilitate free-hand design. The results indicated that
it focused on interaction design, such as webpages and vector-based
wireframing.
In summary, to date, multimodal gaze drawing methods have not
been able to overcome the obstacles associated with gaze-control draw-
ing methods. Multiple modalities also increase the complexity of gaze
interaction and create barriers for some severely motor disabled people
to accessing the applications.
2.3. Summary
The aforementioned studies have addressed the lack of technologies
available in gaze-control drawing applications, yet challenges remain
in conducting graphic tasks at high fidelity and a creative drawing
level. Table 1 compares the capacities of previous gaze-control drawing
studies and the differences between approaches in this field of research.
3. Rationales for gaze control drawing
This section will discuss the rationales behind gaze-control drawing,
including the eye-movement patterns and eye physiology in eye-hand
drawing to comprehend in detail how gaze-control drawing functions
and how this corresponds to the solutions provided in EyeCompass.
3
International Journal of Human - Computer Studies 170 (2023) 102966
3.1. Voluntary eye movements
To draw via the eyes, the drawing system inputs gaze trajectories
and outputs pixels corresponding to those trajectories; therefore, free-
eye drawing depends on voluntary eye movements to constitute the
gaze trajectories and conduct drawing.
There are two types of voluntary eye movements: saccades and
smooth pursuits (Duchowski and Duchowski, 2017; Verghese and Beut-
ter, 2002). Saccades are rapid, step-like changing and ballistic move-
ments in very high velocities (e.g., 300◦ per second) (Verghese and
Beutter, 2002). Saccadic scan paths are polylines, and the endpoints
(fixations) naturally leap toward task-relevant objects (Jacob, 1991;
Kasprowski et al., 2016). In contrast, smooth pursuits are smooth
eye movements that follow moving objects; hence, the velocities and
positions of the eye movements vary according to the visual tar-
gets (Duchowski and Duchowski, 2017; Verghese and Beutter, 2002).
Existing gaze-control drawing methods depend on saccades to con-
duct drawing. Previous free-eye drawing methods can be described as
‘‘where you gaze is where you draw’’ (Jacob, 1991), in which the drawn
lines are exactly the saccadic gaze path. Therefore, these methods only
generate irregular polylines. Due to eye physiology, it is impossible
to draw smooth curves without seeing the target stimulus, as smooth
pursuits are not involved. Additionally, ballistic saccades cannot draw
pixel-level figures. Human eyes are the organ of perception. Voluntary
eye movements are naturally triggered by the need to deposit visual
information into the foveal region (Duchowski and Duchowski, 2017;
Snowden et al., 2012). If the eyes look at an object within the foveal
region, the eye movements become subtle (Majaranta and Bulling,
2014). For instance, the eyes cannot make saccadic movements when
drawing pixel-level figures.
Studies show that the paths of smooth pursuits are fluent curves
when the eyes are pursuing a smooth motion (Soechting et al., 2010).
Therefore, this paper seeks a solution using smooth pursuits as eye
movements in the gaze-control drawing system. In Section 4.2, a gaze-
oriented mechanism, namely, the ‘‘Compass’’, is proposed to trigger
smooth pursuits based on what the user wants to draw, so that the user
can achieve fluent and high-fidelity graphics.
3.2. The stimulus-driven pattern
Characterized by their perceptive nature, eye movements follow a
stimulus-driven pattern; in other words, a goal-driven modality (Hay-
hoe and Ballard, 2005; Yantis, 1998). The occurrence of fixation is
intimately related to the visual stimulus (Hayhoe, 2000; Hayhoe and
Ballard, 2005; Roelfsema et al., 2000; Ullman, 1987). Similarly, Slo-
bodenyuk (2016) argued that eye movements toward the target are
motivated by the need to see external objects and are intimately related
to contextual stimuli. Furthermore, eyes are more sensitive to a moving
target, as the peripheral area of the fovea is physiologically sensitive to
motion. In vision, moving targets are ascribed a greater degree of visual
attention than static targets (Johansson et al., 2001). For example, a
viewer’s gaze can be attracted quickly by a sudden movement in their
periphery (Hillstrom and Yantis, 1994; Johansson et al., 2001). In the
case of free-eye drawing, the drawn lines and the moving brush cursor
are target stimuli. It is the perceptive nature of eyes to saccade at these
target stimuli in the drawing process; the subsequent involuntary eye
movements cause unexpected lines to be drawn. It is also challenging
for the user to move their gaze away from the target stimuli to a to-
be-drawn position in the blank space, due to the lack of gaze-related
targets. Therefore, drawing with the eyes cannot be physiologically
controlled as freely as with the hands.
In order to remove the visual distractions and enhance controlla-
bility, the current project built a cursor interface that adhered to the
brush point (described in full in Section 4.2). The cursor interface plays
a role in a moving target stimulus into the top priority position, above
the canvas elements, and enables the user to focus their visual attention
on the correct to-be-drawn position by naturally following its path in
the drawing process.
L. Huang et al.
Table 1
Comparison of the related research.
Project Drawing Functions
EagleEyes (Gips and Olivieri, Free-eye drawing
1996)
EyeDraw (Hornof et al., 2003) Adding straight lines, rectangles,
and ellipses
EyeArt (Meyer and Dittmar, Adding straight lines, rectangles,
2009) ellipses, circles, text, polylines,
and polygons
EyeSketch (Heikkilä, 2013a,b) Adding straight lines, rectangles,
and ellipses
Van der Kamp and Sundstedt Adding straight lines, rectangles,
(2011) ellipses, polylines, and curves
Dziemian et al. (2016) Free-eye drawing
Sakura (Creed et al., 2020) Adding texts, squares, circles,
lines, vectors, and images
3.3. Gaze pattern in eye-hand drawing
Those involved in gaze-control drawing research must understand
the eye-hand interaction involved when drawing by hand. Tchalenko
(2007) discovered that there are two eye movement modes in eye-
hand drawing, namely ‘‘close pursuit’’ and ‘‘target locking’’. Close
pursuit characterizes the fixations that fall within the vicinity of the
pen tip, coinciding with small saccades around the paper (canvas) to
help situate the drawn lines on the correct locations. Target locking
characterizes consistent fixations that chase the drawn line’s endpoint
when the line is drawn. He also found that the eyes played two roles
during the drawing process: (1) capturing and cognizing visual data
and determining the starting point on the canvas, before positioning the
pen; and (2) assisting the hand in positioning the pen and executing the
drawing precisely. Similarly, saccades between drawn lines and hands
were observed by Gowen and Miall (2006), whose findings indicated
that eyes intuitively fixate on the drawn lines in observation, and
integrate saccading to capture external visual information, with the
fixation paths not usually overlapping the drawn lines.
The eyes are naturally inclined to perceive information during eye-
hand drawing. Previous free-eye drawing methods were error-prone,
as they required the eyes to carry out conflicting tasks simultaneously
(i.e., visuomotor transformation and ocular observation). Furthermore,
those methods translated the user’s fixations directly into brush tra-
jectories in real-time. For example, suppose the user tries to observe
the to-be-drawn position with, for instance, ‘close pursuit’ or ‘target
locking’. In that case, the program recognizes the eye movements
as drawn lines, causing involuntary strokes. To prevent unexpected
strokes, the user must continuously draw without saccades and breaks,
and cannot encode the structures of the to-be-drawn graphics to omit
observational eye movements. Hence, this drawing mechanism opposes
natural drawing activities.
To preserve the observational eye interaction and provide a po-
sitional tolerance of the eye movements necessary in drawing, we
introduced brush-damping dynamics (described in full in Section 4.1).
Our method enables the user to drag the brush remotely with eye
movements via a damping function, thus providing a time latency with
which to make observations and encode the to-be-drawn position on
the canvas for the user.
3.4. Eye-tracking errors
Multiple factors cause eye-tracking errors; these can be categorized
into device-related factors, such as pupil detection and mapping algo-
rithms that map the pupils to the screen coordinates (Lander et al.,
2018), and human-related factors (Kasprowski et al., 2016), such as
angle kappa, cornea curvature, and relative head position and orienta-
tion to the eye trackers (Hansen and Ji, 2009). Thus, eye-tracking errors
4
International Journal of Human - Computer Studies 170 (2023) 102966
Free-hand Gaze-control Modality
Yes Unimodal (Gaze)
No Unimodal (Gaze)
No Unimodal (Gaze)
No Unimodal (Gaze)
No Multimodal (Gaze and
Voice Input)
Yes Multimodal (Gaze and
Robotic Arm)
No Multimodal (Gaze and A
Mechanical Switch)
vary in different gaze control cases. Even when repeatedly implement-
ing rigorous calibration, eye-tracking errors cannot be eliminated when
the eye coordinates are computed (Bozomitu et al., 2017; Feit et al.,
2017).
There are several free-hand drawing problems associated with in-
herent eye-tracking errors. Specifically, there is always a deviation
between the actual and estimated gaze points. When the user draws
using the raw free-eye drawing method, the position where the user fix-
ates differs from where the brush draws on the canvas. As the inherent
eye-tracking errors in the eye-tracking program are unpredictable due
to their multiple causes, it is challenging for the user to know where
the endpoint of drawn lines will be located.
In order to alleviate the effects of inherent eye-tracking errors, this
paper proposes a solution that employs a user interface (UI) offsetting
method in the EyeCompass system design.
4. System design
Based on the findings of previous related research, this paper initi-
ated a hybrid approach that enhanced the flexibility of drawing, and
optimized the graphic fidelity and accuracy of gaze control. First, the
previous free-eye drawing dynamic, in which viewing and drawing are
conjoined, was updated by adding a time lapse between the gaze point
and the brush velocities. Second, a gaze-oriented drawing interface
leveraging smooth pursuits was employed to navigate the eye move-
ments, thereby overcoming the aforementioned issues associated with
free-eye drawing. Unimodal gaze control was selected as it provided
the most straightforward mechanism and optimal compatibility for
combining external sensors, such as a foot paddle, haptic touch bar,
and voice input. The resultant accessibility of this design is discussed
in the following sub-sections.
4.1. Damping function
A brush-damping function was employed on the brush point so
that the positions and velocities of the brush point and the gaze point
were differentiated. The gaze position was not used as a direct input,
as is usually the case, but as a remote controller to pilot the brush
point. Based on Eq. (1), the brush velocity is an instantaneous speed
determined by the gaze position in real time. When the user moves
the gaze point, the brush moves toward that point with its velocity
decreasing linearly, following the decreasing distance between the
brush point and the gaze point (Fig. 1). This type of brush dynamics
emulates the eye-hand drawing motion, as the brush can be freely
accelerated, decelerated, and ceased. The damping ratio 𝜕 provides a
buffer for the user to think and observe the ongoing sketching, which
mimics the intuitive eye movement patterns of eye-hand drawing.
L. Huang et al.
Fig. 1. Description of the damping function. It is evident that the brush velocity is proportional to the distance between the brush point and the gaze point.
In traditional eye-tracking programs, the Exponentially Weighted
Moving Average (EWMA) is employed to filter out eye jitters (Van
der Kamp and Sundstedt, 2011; Zhang et al., 2020; Stawicki et al.,
2017). This algorithm uses time series gaze-position data to smooth out
short-term fluctuations and applies exponentially decreasing weighting
factors (Hunter, 1986). Our drawing system did not employ EWMA,
as it is affected by the older datum points. Instead, another damping
ratio 𝜎 (Eq. (1)) was applied to the velocity of the eye position, which
meant that the gaze point had a minimum time lag and damping from
the actual eye movement. This damping ratio functions as a filter to
soften eye jitters.
{ functions or a linear combination of their integrals. Any shape could,
𝑣brush(𝑥,𝑦) = 𝜕(𝑥gaze − 𝑥brush , 𝑦gaze − 𝑦brush ), 0 < 𝜕 < 1
(1)
𝑣gaze(𝑥,𝑦) = 𝜎(𝑥eye − 𝑥gaze , 𝑦eye − 𝑦gaze ), 0.1 < 𝜎 < 1
where 𝑣brush (𝑥,𝑦) is the vector of brush velocity; (𝑥gaze , 𝑦gaze ) is the
gaze point coordinates; (𝑥brush , 𝑦brush ) is the brush point coordinates;
𝑣gaze(𝑥,𝑦) is the vector of gaze velocity; (𝑥eye , 𝑦eye ) is the eye position;
and, 𝜕 and 𝜎 are the damping ratios. The value interval (0.1, 1) for 𝜎
is an imperceptible range of the time lapse. Detachment between the
two points is evident when the damping ratio is less than 0.1 (see
Section 7.6.2).
4.2. The compass cursor design
The interface of the brush is composed of a circular cursor (CC), of
which the circle center is the brush point, and a directional needle (DN)
pointing from the brush point (circle center) to the gaze point, namely
the direction of the brush movements (Fig. 2). The interface functions
similarly to a compass, which is why it was named ‘‘Compass’’. It has
three novel functions.
First, the Compass serves as a prior visual stimulus on the canvas.
With the damping function, the brush obtains velocity when there is
a positional difference between the gaze point and the brush point
(Eq. (2)). The brush velocity is proportional to the distance between the
brush point and the gaze point, according to Eq. (1). Thus, the brush
is able to move at a constant speed when the user’s gaze follows the
outline (the radius) of CC (Eq. (3)). Then, triggered by the stimulus-
driven pattern, the eyes will continue to smoothly pursue the outline
of CC. This method follows a gaze-oriented mechanism and minimizes
the uncontrollable saccadic movements and unintentional gaze of the
eyes during free-eye drawing.
‖𝑝gaze − 𝑝brush ‖ = 𝑟
𝑣brush ∝ 𝑟
International Journal of Human - Computer Studies 170 (2023) 102966
where 𝑝gaze is the gaze position; 𝑝brush is the brush position; 𝑟 is the
radius of the Compass; and 𝑣brush is the brush velocity.
Second, the circle engages with the range of shapes with ease, using
the Fourier Transform (FT) (Boashash, 2015; Bochner et al., 1949;
Hassanieh et al., 2012). Unlike drawing via the former gaze-control
drawing method, i.e., strenuously saccading anywhere on the canvas,
users can create various shapes simply by fixating on the outline of the
circle when using CC (Fig. 2). The graphics drawn can, thus, be de-
fined as a figure by coordinates in two-dimensions, namely an integral
function dependent on space. In the FT, the brush trajectories can be
decomposed to a function of spatial frequency, that is, trigonometric
therefore, be drawn by tracing the circular outline in different time lags
(Fig. 3).
Third, the DN enables the precise control of the brush direction.
Since the brush point is separated from the gaze, the brush may fall
beyond the view area of fixation. The DN displays a line segment
pointing to the gaze point from the brush center, preventing the user
from glancing back at the brush center for the drawing direction and
causing unwanted sketches. The line segment signifies the direction of
the brush movement and can accurately create vertical and horizontal
lines and specific angles (e.g., 30◦ , 60◦ ).
4.2.1. Eye-tracking error offset
The integration of the brush damping dynamics and the gaze-
oriented mechanism enabled the user to identify the correct spatial
position of the shape to be drawn by observing information from the
Compass. In an environment without eye-tracking errors, CC radius was
able to generate a constant brush speed (Eq. (3)), while the DN showed
the brush direction.
However, eye-tracking errors continued to exist even after careful
calibration (Section 3.4). The bias of the estimated gaze position can
mislead the gaze and cause deviation in gaze trajectory. Fig. 4 illus-
trates how the eye-tracking error causes drawing mistakes and the bias
of brush velocity 𝑒⃗. As shown in Fig. 4, gazing horizontally at the
Compass did not achieve horizontal segments; instead, it produced a
slanting segment connecting the brush point and the estimated gaze
point. The deviation is generated because the system computed the
brush velocity with the estimated gaze position (Eq. (4)).
𝑣⃗ = 𝑣⃗′ + 𝑒⃗ (4)
where 𝑣⃗ is the brush velocity driven by the real gaze position; 𝑣⃗′ is
the brush velocity driven by the estimated gaze position; 𝑒⃗ is the brush
(2) velocity bias from the eye-tracking accuracy error between 𝑣⃗ and 𝑣⃗′ .
Based on Eqs. (3) and (4), the brush speed changes as the gaze
(3) direction changes; the bias of brush speed will be highest when the
5
L. Huang et al. International Journal of Human - Computer Studies 170 (2023) 102966

Fig. 2. Description of the Compass. The dashed circle denotes a circumference of 360◦ , and DN denotes the angle of the brush velocity. When the gaze rests on the outline of
the circle, the brush obtains a velocity toward the gaze point. This mechanism enables the user to manipulate the brush and sketch more accurately.

Fig. 3. Description of drawing different wavy lines using the Compass. For example, by constraining their gaze to the Compass at the location of the red dots, the user can create
smooth curves.

Fig. 4. The drawing effects with a hypothetical deviation of the estimated gaze point. (a) If the user gazed at the right side of the CC horizontally, the brush would move slower
than the actual speed and go toward the upper right; (b) if the user gazed at the left side horizontally, the brush would move faster than the actual speed and go toward the
upper left.

gaze direction and the eye-tracking error are equiangular (Fig. 5).
Consequently, the CC cannot control the brush with a constant speed,
and the brush movement no longer aligns with the DN.
Based on the stimulus-driven pattern (Section 3.2), we used two
transformed UIs to orient the gaze to make a visual adjustment: an
Archimedean spiral cursor (SC) to offset the speed bias, and a direc-
tional needle array (ADN) to offset the angular bias. Generally, in
practice the eye-tracking errors were integrated with both speed bias
and angular bias. A combination of the SC and ADN was able to resolve
the mixed errors, e.g., the case shown in Fig. 4.
(1) The Spiral Cursor (SC). (1) The spiral cursor (SC). As shown
in Fig. 5, the consequence of the bias was that the brush would be
almost still in a particular direction while moving extremely fast in the
opposite direction. The loop of an Archimedean spiral can compensate
for the difference of 𝑟 (Eq. (2)) between the two opposite directions

6
L. Huang et al.
Fig. 5. The bias of brush speed with an eye-tracking error. When the gaze direction and the eye-tracking error are equiangular, the bias of brush speed is the highest. (a) The
brush speed is particularly low; (b) the brush speed is particularly high.
Fig. 6. Description of the spiral cursor theory. It is represented by a hypothetical deviation of the estimated gaze point and applied to all eye-tracking error scenarios.
(𝜃=180◦ ). The SC is denoted by the polar coordinates (𝑟, 𝜃) (Eq. (5)).
𝑟 = 𝑝brush + 𝑏𝜃
where 𝑝brush is the brush center position on the screen; 𝑟 is the radius
of the spiral cursor; 𝜃 is the polar angle in the polar coordinate; and 𝑏
is the parameter controlling the distance between loops. In the system,
the 𝑏 value follows Eq. (6).
𝑏 = 2 △ 𝑟∕𝜋 ≈ 0.64 △ 𝑟
where △𝑟 is the deviation of the estimated gaze position from the real
gaze position due to inherent eye-tracking errors. The calculation of the
𝑏 value is shown in Fig. 6.
(2) The Directional Needle Array (ADN). As shown in Fig. 4,
the user tried to gaze horizontally; however, the DN went to the
upper right, due to the angular bias between the actual gaze and the
estimated gaze position. Therefore, the user needs to make a visual
change when gazing at the DN if the brush direction is horizontal.
Since eyes have a small high visual acuity region (foveal region of
International Journal of Human - Computer Studies 170 (2023) 102966
approximately 2◦ ) (Snowden et al., 2012), a single DN would fall
outside the fixation area if the eye-tracking error was considerable. A
(5)
user would, therefore, lose the visual target and the information for
conceiving the next drawing step. Thus, as a visual stimulus beyond
the fixation area, the deviated DN is likely to distract visual attention
and trigger involuntary eye movements.
In light of the stimulus-driven attention principle of eyes, again, the
(6)
ADN aims to retain the DN within the perception area of the eyes when
the user makes a visual change, and prevent the user’s eyes from falling
outside the oriented cursor. Instead of directly gazing at the required
angle, the user develops drawing strategies by resting their gaze on a
particular line of the ADN until the DNs perceive the desired direction
(Fig. 7). With the gaze-oriented array, the user can easily change their
gaze position. This method removes the limitation inherent in the
notion of ‘‘where you gaze is where you draw’’, ultimately maintaining
the ocular observation and reducing the angular bias in drawing.
7
L. Huang et al.
Fig. 7. Description of the directional needle array.
5. System implementation
The EyeCompass system was built with Unity using C# and To-
bii API3 and runs in full-screen kiosk mode with a resolution of
1920 × 1080 pixels. It supports Tobii gaming eye trackers and can
support other types of eye trackers by using their APIs.
In the system, the dwell click was adopted to initiate commands,
including ‘start drawing’ and ‘end drawing’, and to click on the buttons,
as previous research has shown this to be the most sophisticated gaze
selection approach (Lankford, 2000; Menges et al., 2019; Urbina and
Huckauf, 2010; Zhang et al., 2020). After dwell clicking on a certain
position on the canvas, the brush is activated and the compass circus
appears at the brush point. The system adopts a default dwell time of
500 ms.
The system interface comprises the fundamental free-hand drawing
functions such as pen, eraser, undo, redo, brush settings, grid settings,
zooming, and nudging (Fig. 8). It also contains three unique functions
comprising the compass mechanism: the brush speed setting (a medium
brush speed of 1x and a decelerated speed of 0.5x); the cursor UIs (CC
and SC), and the directional needle styles (DN and ADN). This enables
the user to draw at different speeds and with four compounded compass
UIs.
Our system maintains two modes for drawing: a drawing mode
(default) and a viewing mode (Fig. 8), and the user can switch between
modes through the left-hand menu button View/Draw. In the Draw
mode, the user can draw on the canvas and access the brush settings
in the middle of the right-hand menu. In the View mode, the user
cannot draw but can nudge the canvas (left, right, up, and down)
and examine the sketches. The two modes help the user differentiate
between ocular observation and drawing, which prevents the user from
activating the brush when observing the drawing (i.e., the ‘Midas-
touch’ problem (Chin et al., 2008; Jacob, 1991; Velichkovsky et al.,
2014). The menus are built with gaze-sensitive buttons in width and a
height of 100 pixels (Hansen et al., 2018) for gaze accuracy.
Gaze-selection mistakes are a common problem for users of eye-
tracking interaction applications, and hence are addressed in our design
and implementation. Among previous studies, Creed et al. (2020) iden-
tified a button-selection issue leading to users clicking on the wrong
buttons where they are placed in close proximity on an interface. Sim-
ilarly, Zhang et al. (2020) discovered that increasing the key gap from
0.2 cm to 1.2 cm between frequently typed keys in a 23-inch screen
significantly improved (nearly doubled) typing speed and reduced the
error rate (decreased to one-third to one-fifth). In order to address this
challenge, we placed the most frequently selected pairwise controls,
i.e., Undo and Redo, Zoom In and Zoom Out, at the left and right-side
corners of the screen, respectively. The intention of this design was to
minimize the selection error rate.
3
https://developer.tobii.com/pc-gaming/unity-sdk/api-overview/
8
International Journal of Human - Computer Studies 170 (2023) 102966
6. Pilot study
Before conducting the drawing experiments, we tested the error
distributions after careful calibration with a Tobii 4C eye tracker to
ground the accuracy for this particular eye tracker in the specific study
setup.
The data was collected via a Tobii 4C eye tracker with a 27-inch
monitor at 1920 × 1080 pixels (81.59 PPI). The lighting environment
was a windowless room with 300 lux.4 The participants (N=100) sat
70 cm from the screen. None of them had any eye-related diseases or
visual deficiencies. Before the test, the participants were informed of
the goal of the experiment and were asked to sign a consent form. Then,
they were given a five-minute instruction on eye-tracking calibration.
After calibration, the participants were asked to gaze at the screen at
night dots (benchmarks) for two seconds each.
The results showed that the average error of Tobii 4C was 1.33◦
(52.06 pixels). As Fig. 9 shows, the eye-tracking errors were lower than
the foveal region of human eyes (approximately 2◦ (Snowden et al.,
2012)), implying that the eyes cannot perform subtle control within
this degree of visual angle.
7. Experiment - User study 1
We carried out two experiments (User Studies 1 and 2) to compare
our method to the existing free-eye drawing method based on the
paradigm of ‘‘where one looks is where one draws’’ (Jacob, 1991; Gips
and Olivieri, 1996; Streichert et al., 2020).
In User Study 1, we compared the performance of the free-eye
drawing method with one of the mechanisms introduced in the Eye-
Compass: the damping dynamics. This experiment aimed to explore
the hypotheses that (1) the brush-damping dynamics allow users to
create figures of a higher fidelity than the previous free-eye drawing
method; (2) the brush dynamics render the system more intuitive and
efficient by providing time lapses for drawing observation and reducing
involuntary eye movement errors. Two research questions were set:
RQ1: Do the damping ratios improve the drawing performance
compared to the existing free-eye drawing method?
RQ2: What are the optimal damping ratio settings?
7.1. Apparatus
The user study was carried out in the office of the Department of
Computer and System Sciences, Stockholm University. We used the
low-cost Tobii Eye Tracker 4C as this (or similar trackers) is what
consumers would use. The eye tracker was mounted on the bottom of
the monitor. The participants sat in front of the monitor at a distance of
69 ± 15 cm. The system ran on a desktop computer with an Intel core
i5-9400f CPU @ 2.90 GHz (processor) using 16 GB RAM and Gigabyte
GeForce GTX 1660 Super Gaming Oc 6G (Graphics Card) and Windows
10. The interface was displayed on a 27-inch monitor with a 16:9 aspect
ratio and a resolution of 1920 × 1080 pixels.
7.2. Participants
In total, 60 participants (28 females and 32 males, aged between
20–35 years, non-disabled) from Stockholm University, KTH Royal
Institute of Technology, and Karolinska Institutet were recruited via
email, word of mouth, and social media. None of them had any eye-
related diseases or visual deficiencies; 38 of them had normal visual
acuity, 16 wore glasses, and 6 wore contact lenses. Each participant
was given 100 SEK (∼9 USD) as a token of appreciation for their
participation. All of the participants in the drawing tasks had drawing
4
EN 12464-1:2021 (Light and lighting. Lighting of work places —
Classroom)
L. Huang et al. International Journal of Human - Computer Studies 170 (2023) 102966

Fig. 8. EyeCompass interface overview. CC stands for the circular cursor; SC stands for the spiral cursor; DN stands for the directional needle; ADN stands for the directional
needle array.

Fig. 9. The eye-tracking error distribution after calibration (in degrees of visual angle and pixel). The pixel coordinates of the eight benchmarks on the screen are (180, 180), (180,
530), (180, 880), (960, 180), (960, 530), (960, 880), (1740, 180), (1740, 530), (1740, 880). The heatmap denotes the overlap percentages of the gaze points in the distribution
area and shows that most gaze points fell at the center of the benchmarks.

experience but had no experience using eye trackers. None of the partic-
ipants in the drawing task participated in both studies (e.g., User Study
1 and 2). Five external drawing experts (beyond the 60 participants)
were recruited via the same approach; they were responsible for rating
the sketches drawn via Mean Opinion Score (MOS) (Streijl et al., 2016).
7.3. Experimental setup
A between-subjects experiment was conducted to compare the per-
formance of the damping function to the existing free-eye drawing
method. If a participant is exposed to multiple drawing conditions,
this will cause a carryover effect, such as practice or learning effects,
in subsequent tests (Maxwell et al., 2017). Carryover effects threaten
the internal validity of the drawing task results. Thus, a between-
subjects design was adopted to minimize the learning and transfer
across groups.
There were four groups, each comprising of 15 participants. The
independent variable was the damping ratios between groups. Group
1 was the control group, which used the existing free-eye drawing
method (in the form of drawing where you look) (Jacob, 1991). Groups
2, 3, and 4 were experimental groups with damping ratios of 10−1
(a fast brush speed), 10−2 (a medium brush speed), and 10−3 (a slow
brush speed), respectively. The drawing task was to draw five types of
geometric primitives (Barasch, 2013; Goodrich, 1954; Ruskin, 1876):
a circle, a square, a wavy line, a zigzag line, and a five-pointed star.
The circle and the wavy lines were included to study the feasibility of

9
L. Huang et al.
Fig. 10. A sample of the experiment implementation with the Tobii 4C eye tracker
below the screen.
controlling direction and drawing smooth curves. The square and the
five-pointed star were included to study the feasibility of controlling
direction and drawing corners.
The 60 participants were randomly assigned to one of the groups.
All participants were informed of the goal of the experiment and were
asked to sign a consent form. Each participant was given a 10-minute
training session, during which they were instructed on how to use
the eye tracker in the test version for free-eye drawing, including
how to use the eye-tracking calibration, gaze-control modes, and free-
eye drawing commands based on dwell click and dragging the cursor
(Fig. 10). After a 2-minute break, the participant performed a 7-point
eye-tracking calibration and completed the drawing task using only
their eyes. The drawing processes were recorded using a Windows
10 built-in screen recorder. The drawing results were taken as the
final images when the participants confirmed that they had finished
drawing.
At the end of the experiment, each participant participated in a
semi-structured interview with open-ended questions, such as ‘‘What
did you think?’’ and ‘‘How did you get on with the drawing?’’ Partici-
pants were asked to give feedback on the method used and their user
experience. The interview last for five minutes for each participant.
7.4. Data analysis
The drawing performance and graphical fidelity of the results were
evaluated using Mean Opinion Score (MOS), a well-known method
of quantifying the subjective quality of visualization (Aldridge et al.,
1995; Cranley et al., 2006; Sun and Huang, 2011; Vázquez and Stein-
feld, 2014). The drawing results were randomly shuffled and rated by
a blind reviewer based on an 11-point MOS standard ( Table 2). The
reviewers were fully aware of the evaluation standard and were asked
to evaluate which painting most resembled the shape sample, without
knowing the method used to create it. Each figure was reviewed by
the five reviewers, and an average score was calculated. The MOSs
of each shape were analyzed using one-way ANOVA and descriptive
analysis. The drawing outputs were re-evaluated by post-hoc analysis
of the screen recordings of the drawing processes. The system auto-
matically recorded the length of each line and the task completion
time, defined as the duration between ‘brush activated’ to ‘brush end’.
The drawing speed of each group was calculated and analyzed using
one-way ANOVA. The interview records were assessed using content
analysis.
7.5. Results
7.5.1. Quantitative results
As shown in Fig. 11, the average MOS of each shape increased
significantly with the decreasing damping ratio, reaching the highest
10
International Journal of Human - Computer Studies 170 (2023) 102966
value when the damping ratio equaled 10−3 . The MOS of each shape
produced via the method of ‘‘where you look at is what you get (draw)’’
was the lowest within the four groups. Specifically, the MOSs of the
circle and the wavy line were the lowest (approx. 2). The square, the
zigzag, and the star obtained higher scores (around 4 to 5.5). For all
the shapes, the mean MOSs increased when the participants used our
method. The scores increased significantly when the damping ratio was
10−1 (a high brush speed) and 10−2 (a medium brush speed), and
increased gradually when the damping ratio was 10−3 (a slow brush
speed).
Fig. 12 shows that the drawing speeds and the standard deviations
(SDs) of the previous method was significantly high among all the
tested figures. The drawing speed decreased when our method was
used, decreasing exponentially with the decreasing damping ratio. The
SD of the drawing speed also decreased with the decreasing damping
ratios. The drawing speeds were almost the same for the different
shapes drawn, particularly when the damping ratios were 10−2 (a
medium brush speed) and 10−3 (a slow brush speed).
7.5.2. Qualitative results
Fig. 13 represents the drawing performances of different groups.
The drawing outputs improved with each group with the increased
damping ratio. Specifically, all the shapes drawn via the previous
method were distorted and had many sharp corners. The shapes im-
proved with each group using the damping function and were almost
as well-proportioned and smooth as the reference shapes once the
damping ratio reached 10−3 (a slow brush speed). The images drawn
using a lower damping ratio had fewer involuntary lines, more gliding
details, and higher fidelity. In general, it could be concluded that the
drawings that most closely resembled the reference images were drawn
by the group using our method with a damping ratio of 10−3 (a slow
brush speed).
During the interviews, all the participants who used the previous
method reported extreme difficulty in controlling the brush and unsat-
isfactory drawing results, as can be seen by the following quotes: ‘‘I
could not control eye movements without looking around’’, and ‘‘The brush
was totally out of control’’. They were also annoyed by serious brush
jitters.
The group with the damping ratio of 10−1 gave similar comments
to those who used the previous method. In contrast, in the groups with
the damping ratio of 10−2 (a medium brush speed) and 10−3 (a slow
brush speed), the participants stated that they had felt comfortable
with the time lapse between the brush and the gaze: ‘‘The speed was
useful to control the brush as I could direct the brush easily’’. They also
highlighted that they were able to glance at the drawn lines within the
time lapse between the brush point and the gaze. However, most of the
participants in the group with a damping ratio of 10−3 reported that
the slow brush speed caused severe eye strain: ‘‘My eyes were exhausted
when I gazed at the screen for such a long time’’.
Among the four groups, some of the participants suffered from a
‘‘cursor drifting’’ problem: ‘‘I could not gaze straight at the brush point,
or the brush would move away. In this way, the cursor kept drifting and
my gaze could never reach it ’’, and ‘‘I found the brush was not in the right
position, wherever I looked’’. Drawing mistakes caused by involuntary
eye movements were also reported: ‘‘When I looked at what I draw, it
produced redundant lines on the canvas’’.
7.6. Discussion
7.6.1. The effect of damping function
The shapes drawn using the previous method could hardly be
recognized. Notably, smooth curves were very difficult to draw. Thus,
the previous method for gaze-control drawing is evidently not feasible.
In contrast, our method using the damping function successfully cre-
ated smooth and geometrically well-drawn shapes. Thus, the damping
ratios helped to obtain positive drawing results in free-eye drawing
L. Huang et al. International Journal of Human - Computer Studies 170 (2023) 102966

Table 2
The evaluation standard for the drawing results.

Which painting is more similar to the shape sample? MOS

The shape drawn is geometrically perfect, almost coincides >8, ≤10

with the reference image, and its details are drawn precisely
(smooth and gliding lines, sharp corners, well-proportioned
circles, and horizontal or vertical-sided squares).
It is easy to recognize the shape drawn, the shape is >6, ≤8
well-proportioned, the curves are smooth, and the straight
lines are horizontal or vertical-sided.
The shape is skewed or distorted slightly. The drawn lines in
the shape are tidy and gliding.
The shape can be recognized correctly, but its details are not >4, ≤6
well drawn.
The shape is asymmetrical, twisted, skewed, or distorted.
There are small amounts of irrelevant lines in the shape.
The shape can just about be recognized, but the details are >2, ≤4
cluttered. The shape is twisted, skewed or distorted
significantly, and there are many irrelevant lines in the
shape.
It is very difficult to recognize the shape and the quality of >0, ≤2
its details is inferior.
The shape drawn cannot be recognized at all. =0

A higher MOS indicates a better image quality. A score lower than 4 signifies an unrecognizable and low-fidelity image, and a
score higher than 6 signifies a recognizable, high-fidelity image. This MOS standard was used to quantify the drawing outputs
in User Study 1 and User Study 2.

Fig. 11. The Mean Opinion Scores (MOS) of the groups. The 𝑥-axis denotes the four groups: (1) the previous free-eye drawing method, (2) a damping ratio of 10−1 (a fast brush
speed), (3) a damping ratio of 10−2 (a medium brush speed), (4) a damping ratio of 10−3 (a slow brush speed).

Fig. 12. The drawing speed of the four groups. The 𝑥-axis denotes the four groups: (1) the previous free-eye drawing method, (2) a damping ratio of 10−1 (a fast brush speed),
(3) a damping ratio of 10−2 (a medium brush speed), (4) a damping ratio of 10−3 (a slow brush speed).

and improved the drawing performance remarkably. Specifically, two
factors related to the damping function were shown to be effective in
controlling the brush and refining the lines drawn in free-eye drawing.
First, this method suits the perceptional nature of eyes when drawing.
Second, the brush damping dynamics stabilize the brush by alleviating
the influence of involuntary eye movements.
(1) Emulation of Eye Movement Patterns in Eye-Hand Drawing.
The inability to observe the sketches during the drawing process was
noted as an issue in the previous gaze-control drawing research (Gips
and Olivieri, 1996; Hornof et al., 2003). This issue originates from natu-
ral eye behavior when drawing. In a real-world drawing process using
hands, the eyes fulfill two tasks (Tchalenko, 2007): one is obtaining
visual information to help the painter decide how and what lines to
draw, and the other is assisting the hand movements to execute the
drawing precisely. To implement the dual-task, the eyes engage in two
primary activities throughout the drawing process: ‘‘close pursuit’’ and
‘‘target locking’’ (Tchalenko, 2007). When a brush point approximately
equals the gaze point, as in the previous method, the user cannot make
saccades to observe and conceive the next position to draw as the gaze
trajectories are transmitted immediately to drawn lines. This type of
drawing mechanism goes against physiological eye movement patterns
and hinders usability.
Furthermore, it is rather difficult for the user to fix their eyes
precisely on a blank space on the canvas. Natural eye movements
happen in response to contextual stimulus in external environments,

11
L. Huang et al.
Fig. 13. Comparison of the drawing results between groups. Each row was sampled respectively by one subject’s results whose MOS was similar to the average MOS in the
corresponding group.
following a stimulus-driven pattern (Kasprowski et al., 2016; Slobo-
denyuk, 2016). In this case, the drawn lines are the only contextual
stimulus on the canvas. Consequently, the user’s eyes are likely to be
instinctively attracted to the drawn lines and, thus, cause involuntary
lines while drawing.
Our method solves this issue by separating the positions of the
brush and the gaze point and using the damping ratio to constrain
the displacement of the brush position toward the gaze point. In our
method, the gaze path of observation is not immediately outputted
as pixels on the canvas. Minor eye movements, for example, close
pursuit (Tchalenko, 2007), are filtered by the damping functions so that
the user can observe the drawn line while drawing. The experimental
results show that a relatively low damping ratio generates a consider-
able time lag between the brush point and the gaze and provides the
user with more chance to observe the drawing outputs. The interview
feedback confirmed that an appropriate time lapse facilitates the dual-
task of the eyes (‘‘close pursuit’’ and ‘‘target locking’’ (Tchalenko,
2007)) by enabling the user to make observations through glances
within a small range and to make decisions on the next drawing steps
within the brush latency.
(2) Brush Stabilization. The lower the damping ratio, the lower
the brush speed and the better the drawing performance. This is demon-
strated by the results: the groups using fast brush speeds lacked control
over the direction of the brush, leading to skewed lines. In contrast, the
results obtained by participants using slower brush speeds were better
proportioned.
The SD trend of the brush speed explains why the damping function
can enhance brush control. The brush speed is entirely synchronized
with eye movement in the previous method. Thus, it is extremely
affected by saccades and eye jitters (as indicated by the high SD),
which make it unsteady. This decreases exponentially with a decreasing
damping ratio, and the SD of the brush speed decreases correspond-
ingly. The decreasing brush speed and SD show that the damping ratio
reduces the negative impact of unintentional eye movements and eye
jitters on brush stability. Thus, the damping function enables the user
to control the brush at an appropriate and stable speed and enhances
the competence of direction control.
12
International Journal of Human - Computer Studies 170 (2023) 102966
Additionally, the damping function acts as a noise filter, reducing
redundant lines caused by eye jitters. The images drawn indicate that
drawing with a damping ratio of 10−2 (a medium brush speed) and
10−3 (a slow brush speed) effectively improves drawing performance
and reduces drawing jitters compared to the previous free-eye drawing
method. The errors in the experimental results decreased significantly
with lower brush speeds, validating the plausibility of the damping
function.
7.6.2. The optimal damping ratio
According to the results based on the MOS standard, the shapes
drawn using the damping ratios of 10−2 and 10−3 were recognizable,
which indicates that the damping ratio should not be higher than 10−2
to achieve acceptable drawing results. However, there are several issues
with a damping ratio of 10−3 . Although involuntary sketches were
reduced with a 10−3 damping ratio (a slow brush speed), they did
not significantly differ from those obtained using a damping ratio of
10−2 . Nevertheless, the task completion time increased remarkably.
Furthermore, the participants drawing with the lowest brush speed
reported severe eye fatigue as they needed to fixate on a point for a long
time until the brush moved toward their gaze point. Most importantly,
according to Fig. 12, the drawing speed when using a damping ratio of
10−3 is lower than the normal free-hand drawing speed, varying from
approximately 28 mm/s to 150 mm/s (Tchalenko, 2007), while the
drawing speed when using a damping ratio of 10−2 is within the range
of the optimal ratio for brush damping dynamics and is helpful for
drawing performance, eye comfort, dwell time command, and human
drawing nature. Hence, it can be concluded that the optimal damping
ratio most effective in filtering out eye jitters is around 10−2 .
The damping dynamic can also be generalized by assigning non-
linear damping ratios to control the brush speed under different brush-
gaze distances. For example, in Fig. 14 the damping ratio is 0 when
the brush-gaze distance is less than 97 px (a gaze-based precise limit
(Hansen et al., 2018)) and is 10−2 when the brush-gaze distance is
more than 97 px. In this case, the brush would be still when the user
makes close pursuits (Tchalenko, 2007) on the canvas, stimulating the
start-stop action inherent to real-world drawing scenarios.
L. Huang et al.
Fig. 14. A non-linear brush damping dynamic.
Fig. 15. The cursor drifting problem, which hypothesizes that the eye-tracking program locates the user’s gaze at the point pgaze while the user is looking at the position of preal
with an eye-tracking error. (a) The bias from the inherent eye-tracking error. (b) When the user’s eyes look at the brush point, the eye-tracking program computes the offset as
fixation moving from 𝑝gaze to 𝑝gaze′ .
The damping function (Eq. (1), Section 4.1) summarizes a universal
type of gaze-control dynamics and can describe many of the existing
gaze-control mechanisms. It can be generalized to various eye-tracking
applications and helps adjust the dynamics of the cursor to control
the gaze. For example, common mouse selection events (selection and
command) (Ramirez Gomez and Gellersen, 2019; Velloso and Carter,
2016) in eye-tracking games use a damping ratio of 0, i.e., the cursor’s
position and the speed are equal to the position and speed of the gaze.
In some aiming games where the gaze is used for navigation or ori-
entation control (Smith and Graham, 2006), the game camera (field of
vision) follows where the eyes look (Nielsen et al., 2012). The damping
function makes gaze-adaptive interaction feasible by controlling the
speed and trajectories of vision transformations in game scenes.
7.6.3. Cursor drifting
The participants frequently reported that the cursor drifted in a
specific direction when their eyes pursued the brush cursor. This issue
points to an eye-tracking error, highlighted by Hornof et al. (2003),
where the user’s eyes chase their fixation (the cursor) on the screen
and cause the gaze cursor to drift.
According to the research on natural eye movement patterns, eyes
are target-oriented and visual attention is sensitive to moving targets.
When the user is drawing, the moving brush is a significant target
stimulus that appeals to the user’s visual attention. Furthermore, on a
blank canvas without auxiliary UI, the user must gaze at the endpoint
of the drawn lines to encode the to-be-drawn contents. However, due
to inherent eye-tracking errors, the estimated gaze point and the actual
gaze position is consistently offset. In the previous gaze-control drawing
method, where the brush point coincides with the gaze, the brush point
equals the estimated gaze point, but not the actual point the user gazes
at (Fig. 15). When the user tries to look at the brush point, the system
will interpret the deviation between the estimated gaze point and the
real gaze point as moving the gaze; consequently, the brush will drift
away. This means the user is always chasing the drifting brush cursor
when drawing, much like a person chasing their shadow.
Common eye-tracking applications use selective-based interfaces de-
signed with a positional tolerance for the deviation between the actual
gaze position and the estimated gaze point (cursor) (Hansen et al.,
2018; Menges et al., 2019). In addition, UI elements for dwell clicks
are fixed so the user can adjust their gaze to a proper position where
the fixed buttons can be clicked on. However, in click-and-drag-based
interfaces (e.g., free-eye drawing), a moving cursor (e.g., the brush) has
less positional tolerance for the eye-tracking deviation. For example,
the user needs to gaze at the brush to observe what they are drawing
and conceive the next actions. Meanwhile, their gaze trajectories are
13
International Journal of Human - Computer Studies 170 (2023) 102966
translated to drawn lines) in real time. Even a tiny deviation can affect
the user’s judgement and create performance inaccuracy. This issue will
be addressed using the gaze-oriented method (the Compass) in User
Study 2.
8. Experiment-User study 2
This study tested the hypothesis that using the Compass could
improve the graphical fidelity of free-eye drawing and mitigate the
negative impact of eye-tracking errors and involuntary eye behaviors
resulting from the eyes’ stimulus-driven pattern. It will also examine
whether the gaze-oriented mechanism is aligned with the eye’s per-
ceptive nature and its physiology. The experiment compared graphical
fidelity, drawing performance, brush controllability, and usability of
our gaze-oriented method (i.e., different combinations of the Compass)
with the existing free-eye drawing method and validated the results of
User Study 1 with the damping ratio dynamics.
8.1. Apparatus
The study was conducted in a laboratory at Guangxi University,
China, for the convenience of recruiting large number of participants.
The eye tracker used was Tobii Eye Tracker 4C, and the interface was
displayed on a 27-inch monitor with a 16:9 aspect ratio and a resolution
of 1920 × 1080 pixels (the same apparatus as in User Study 1).
Participants sat in front of the monitor at a distance of 73.4 ± 10.1 cm.
The drawing system ran on a desktop computer with an Intel core i5
10400F CPU @ 2.90 GHz using 16 GB RAM and ASUS DUAL-RX580-8G
(graphics card) and a Windows 10 operating system.
8.2. Participants
Participants (N=90, 47 females and 43 males, aged between 25–35,
nondisabled) were recruited from the students of Guangxi University,
China. None of the participants had any eye-related diseases or visual
deficiencies; 45 of them had normal visual acuity, 32 wore glasses, and
13 wore contact lenses. Each participant was given 100 CYN (∼15.5
USD) as a token of appreciation for their participation in the study.
The participants all had drawing experience, but were inexperienced
in using eye trackers. To maintain the same individual factor level, the
MOS reviewers were the same as those in User Study 1 and the MOS
standard was the same standard used in User Study 1.
L. Huang et al.
8.3. Experimental setup
A between-subjects study was conducted. The 90 participants were
randomly divided into six groups, which were tested under different
conditions. There were two control groups without the Compass: the
first group used the existing free-eye drawing method ‘‘what you gaze
at is what you get (draw)’’; the second group used the conclusive
damping ratio of 10−2 without the Compass; the experimental groups
used the damping ratio of 10−2 and different Compass UIs, namely:
(1) the circular cursor with a directional needle (CC+DN): (2) the
circular cursor with the directional needle array (CC+ADN): (3) the
spiral cursor with a directional needle (SC+DN); and (4) the spiral
cursor with the directional needle array (SC+ADN), respectively.
To eliminate errors caused by the differences in individual drawing
skills, the participants were assigned to draw universal figures such as
English letters (A, B, C, a, b), Arabic numerals (1, 2, 3), and symbols
(%, &, ⊠) and four simple icon-like figures (a smiley face, a piece
of leaf, a piece of cloud, and a house). A handwriting font (Lucida
Handwriting) was chosen for the letters and numerals, as this font most
closely resembles human handwriting. Each participant was shown the
reference images and asked to reproduce the images to the best of their
ability using free-eye drawing. The figures contained the geometric
compositions of the figures used in Experiment 1, that is: curved,
straight, horizontal, vertical, zigzag, and diagonal lines but comprising
more variables that rendered them more similar to hand-drawn figures.
Before the test, the participants were informed of the goal of the
experiment and asked to sign a consent form. Before the test session, the
participants were introduced to the drawing system and the Compass
UI they would use, depending on their assigned group. They each spent
10 min practicing using the UI tool. Before starting drawing, each
participant performed a seven-point eye-tracking calibration. To ensure
that they were familiar with the figures used, they were asked to imitate
the reference images on paper by hand before they completed the test.
After a 1-minute break, each participant re-calibrated the eye tracker
and initiated the drawing test using only their eyes. The drawing
process was recorded using the Windows 10 built-in screen recorder.
The drawings were taken as final images when the participants con-
firmed that they had finished drawing. Following the drawing tests,
each participant was asked to take part in a 5-minute semi-structured
interview, in which they were asked open-ended questions, including
about drawing competence, accuracy, and the challenges they faced,
such as ‘‘Did you make mistakes when you were drawing? If yes, why?’’
and ‘‘What do you think of the method, and why?’’ Lastly, a System
Usability Scale (SUS) test (Brooke, 1996) for the drawing usability was
conducted with each participant.
8.4. Data analysis
The drawn images were assessed by a blind review based on the
evaluation standard ( Table 2 in Section 7.4) by the same reviewers
used in User Study 1. The figures drawn by each participant were rated
with one MOS by the five reviewers and the mean MOS was calculated.
The drawing outputs and drawing process recordings were re-examined
and analyzed. The results of the interview were assessed using content
analysis.
8.5. Results
8.5.1. Quantitative results
Fig. 16 shows the MOS and SUS scores produced by the mean and
SD for the different groups. The average MOS using the previous free-
eye drawing method was the lowest (approx. 2), which is deemed
unrecognizable according to the MOS standard (see Table 2). The
mean MOS increased significantly with a damping ratio of 10−2 (a
medium brush speed) (approx. 5). The experimental groups using the
Compass obtained higher MOS values. The means of MOS were higher
14
International Journal of Human - Computer Studies 170 (2023) 102966
than 6 with the Compass, which denotes that the figures are highly
recognizable according to the MOS standard.
The mean MOS obtained using the Compass with CC+DN was ap-
prox. 6, and increased significantly in the CC+ADN and SC+DN groups.
The SC+ADN group obtained the highest MOS and only increased
slightly compared to the results obtained by the groups using CC+ADN
and SC+DN.
The upward trend of SUS was aligned with the trend of MOS. The
average SUS score with the previous gaze-control drawing method was
the lowest (approx. 46), indicating poor usability according to the
usability rankings (Bangor et al., 2009). The mean of SUS using no
Compass but a damping ratio of 10−2 increased slightly (approx. 57),
which is considered ‘‘OK’’ in the ranking of SUS (Bangor et al., 2009).
In contrast, the SUS scores of the drawings made using the Compass
increased significantly. The group using CC+DN obtained an average
usability score of 75, which is rated ‘‘Good’’ (Bangor et al., 2009).
The other three groups using different combinations of the Compass
obtained usability scores higher than 80 (rated ‘‘Excellent’’ (Bangor
et al., 2009) The highest SUS score was obtained by the group using
SC+ADN (approx. 83).
8.5.2. Qualitative results
Fig. 17 represents the drawing results of the different groups. The
drawings were retrospectively compared with the interview results to
evaluate the drawing performance as below.
(1) Previous Free-Eye Drawing Method. The results were severely
distorted and out of shape; they also contained a large number of
redundant lines. Most of the figures were unrecognizable, for example,
the figures of ‘‘C’’, ‘‘3’’, the leaf, the cloud, and the house.
The participants found it ‘‘very difficult to draw’’. They emphasized
the difficulty of moving their gaze precisely to the desired point,
highlighting the fact that the interaction was unnatural. The other
notable issue was the cursor drifting: ‘‘the position of the lines was not
the place I gazed at, and every time I gazed at the brush, there was a wrong
sketch.’’
(2) No Compass. The results showed a slight improvement on those
obtained with the previous method. However, the results still revealed
many ill-drawn strokes, obvious distortion, and significant skewness,
especially in the curved and round figures, such as the figures of ‘‘a’’,
‘‘3’’, the smiley face, and the leaf.
The participants reported drawing mistakes due to unintentional
strokes: ‘‘I could not keep myself from looking at my sketches in the
middle of the drawing.’’ Another factor causing unintentional drawing
mistakes was cursor drifting: ‘‘When I looked at the endpoint of the lines
to examine what I had drawn, the endpoint drifted away and drew lines
everywhere’’. Notably, several participants expressed their desire to have
some assisted tool to guide the gaze: ‘‘It was difficult to stabilize my gaze
without anything to gaze at ’’, and ‘‘I wish there was a grid to help me look
at the canvas’’.
(3) CC+DN. The drawn figures showed fewer mistakes and less
redundancy. The lines were tidier than those drawn by the first two
control groups. All the figures were recognizable, though they were
slightly distorted with twisted strokes and skewness.
The participants expressed their appreciation of the interface: ‘‘It
was very helpful to navigate the brush’’. Two major problems were
reported. One was distortion: ‘‘The circle I drew often turned out to be
spiral, the beginning and the end points of the circle could not close up’’,
and ‘‘When I drew by gazing at the edge of the circular cursor, the brush
always moved faster and was too sensitive on the right-hand side and could
hardly move on the left-hand side’’. Another problem was the angular bias
for straight lines: ‘‘The vertical lines were skewed when I draw the square’’.
(4) CC+ADN. There was a significant improvement compared to
the results in the three aforementioned groups. All the figures were
well-proportioned and had tidy lines. There were no twisted strokes
or distortion in the figures. Some figures had short redundant lines at
L. Huang et al. International Journal of Human - Computer Studies 170 (2023) 102966

Fig. 16. (a) MOS of the different groups. (b) SUS score of the different groups. In the two figures; (1–6) denotes the six groups: (1) the existing free-eye drawing method ‘‘what
you look at is what you get (draw)’’; (2) no Compass and with the damping ratio 10−2 ; (3) the circular cursor with a directional needle with a damping ratio of 10−2 (CC+DN);
(4) the circular cursor with the directional needle array with a damping ratio of 10−2 (CC+ADN); (5) the spiral cursor with a directional needle with a damping ratio of 10−2
(SC+DN); and (6) the spiral cursor with the directional needle array with a damping ratio of 10−2 (SC+ADN).

Fig. 17. Comparison of the drawing results between groups. Each row respectively samples one subject’s results whose MOS was similar to the average MOS in the corresponding
group.

the endpoint of the strokes, but the redundant lines did not hinder the
overall graphical structure.
The participants revealed a distortion issue similar to the group with
CC+DN. However, they did not report any errors related to angular
bias.
(5) SC+DN. The figures presented a similar completion level to
those drawn by the CC+ and group. The arcs were slightly rounder,
for example, in the figures of ‘‘B’’, ‘‘3’’, ‘‘&’’, and the smiley face.
The participants commented that SC made it easy to draw circles
and arcs: ‘‘The spiral was a useful tool to draw full circles, as I could
shift the speed by moving my gaze between the inner loop and the outer
loop’’. However, they reported difficulty in drawing straight lines: ‘‘I
always looked along the loops of the spiral cursor subconsciously when I
was drawing straight lines, then the straight lines often bent.’’
(6) SC+ADN. The figures were well-proportioned, with gliding lines
and the highest level of completion out of the experimental groups. The
details in the figures closely reflected the reference image.
The participants reported very few, if any, drawing mistakes by the
interface: ‘‘It [the Compass] is a smart design, I have never imagined how
people can draw with eyes before’’, and ‘‘I figured out some skills in how to
gaze through the Compass to draw satisfactory figures.’’
8.6. Discussion
The drawing tests and the interview validated the hypothesis that
the gaze-oriented compass cursor can significantly improve brush con-
trol, graphical fidelity, and user experiences in free-eye drawing.
8.6.1. Methods without gaze-oriented interfaces
The previous free-eye drawing method is the mechanical form of
‘‘what you look at is what you get (draw)’’. The drawn result was
similar to that obtained in Experiment 1, with the conclusion being
that it is challenging to create precise figures using this method. In
the drawing results, the drawn images included many distortions,

15
L. Huang et al.
skewness, and redundant lines at the endpoints of the strokes. Most of
the drawn images with the previous gaze-control drawing method were
unrecognizable. Furthermore, this method demonstrated poor usability
and unnatural interaction.
The second group drawing using the damping ratio of 10−2 vali-
dated the conclusion drawn in Experiment 1 that the damping ratio
effectively improves free-eye drawing accuracy. Notably, the results
using this damping ratio revealed different scores in the two experi-
ments. There score in Experiment 2 was lower as the figures tested in
Experiment 2 were compounded and consisted of many strokes, while
the figures tested in Experiment 1 were mostly completed with one
stroke; thus, the figures tested in Experiment 2 required more precise
control of the brush position, direction, and graphical proportion.
In the figures drawn by the two groups without the Compass, the
polyline-alike strokes were similar to the scanpath of saccades (Jacob,
1991). This phenomenon verifies that eye movements are mainly sac-
cades when drawing without gaze-oriented visual stimulus. Thus, it
is challenging to obtain smooth and well-proportioned shapes, as the
user finds it difficult to locate the gaze precisely on the desired spatial
position. The user is also likely to look at what they have drawn and
the endpoint of the stroke, leading to issues such as cursor drifting and
involuntary eye movements that produce redundant lines.
At the commencement of free-eye drawing on a blank canvas, there
are several constant target stimuli for eye movements: the endpoint of
the brush (moving target, prior to visual attention) and the drawn lines
(still target). The eyes naturally fix on the brush point (moving target)
and the drawn lines (target involving specific information) (Hayhoe and
Ballard, 2005; Johansson et al., 2001; Yantis, 1998) as eye movements
are physiologically stimulus-driven. Hence, free-eye drawing is error-
prone via the previous method, as the user must force their eyes to
move against the eye’s perceptive nature and gaze away from the visual
stimuli. The other challenge is that in the previous method, the user
cannot observe what they have drawn as the gaze trajectories are
generated as pixels in real time. This is why the participants using
this method intuited the need for visual aids for gaze control in the
follow-up interview. Therefore, the method without a gaze-oriented
target stimulus in sight is infeasible when seeking to draw complicated
(compounded) figures at high fidelity and to a precise level.
8.6.2. Use of the Compass
The experimental groups using the Compass successfully achieved
precise and high-fidelity figures, including variable figures. The us-
ability and graphical fidelity of free-eye drawing with the Compass
increased significantly and the participants in these groups reported
having to make less effort and experiencing less frustration than the
participants in the previous groups. Moreover, the cursor drifting issue
mentioned by the previous two groups was not reported by any of the
users drawing with the Compass. There were rare reports of loss of
observation and mistakes in the form of unintentional strokes.
The basic UI combination was a circular cursor with a single direc-
tional needle. When the results are compared to those achieved without
using gaze-oriented interfaces, it is evident that this type of UI notably
improves free-eye drawing. However, the drawing performance of this
basic UI combination was the lowest among the Compass groups. The
images drawn via this method had twisted strokes and slight skewness.
According to the user feedback using CC, the brush velocity when
gazing to the left-hand side of the cursor was different from that of
the right-hand side. This meant that the lines drawn toward the left
and right sides were different lengths, even when the user’s gaze was
the same duration and distance from the brush center. Conclusively,
the basic UI is usable only when the eye-tracking error is minor and is
feasible for use to draw smooth curves.
In comparison to the groups using the circular cursor or a single
directional needle, the users drawing with a spiral cursor and the
directional needle combination achieved the highest competence in
free-eye drawing. The spiral cursor made it significantly easier for the
16
International Journal of Human - Computer Studies 170 (2023) 102966
user to correct distortion and skewness and draw well-rounded circles
and arcs when the eye-tracking error was relatively severe, while the
directional needle array enhanced directional control and gave the user
the ability to correct graphical angular bias caused by the eye-tracking
error. Hence, there was less graphical skewness in the drawn lines in the
results produced by the groups using the spiral cursor and directional
needle array.
Specifically, the Compass works as a moving target triggering the
eyes’ smooth pursuit and generating fluent gaze trajectories (Soecht-
ing et al., 2010). This mechanism is based on the perceptive nature
of eye movements and their stimulus-driven pattern (Duchowski and
Duchowski, 2017; Verghese and Beutter, 2002). Since the paths of
smooth pursuits closely align with the trajectories of a target-oriented
stimulus (Soechting et al., 2010), the gaze trajectories are affected
primarily by the outlines of the cursor’s UI. This is why the directional
needle and directional needle array are better at guiding the user to
draw straight lines, while the circular and spiral cursors enable the user
to draw curves and circles. It was also demonstrated that the Compass
enabled the user’s eyes to correct the observational error associated
with eye-tracking bias.
Additionally, the Compass is crucial to guide the user in drawing
information, including the brush direction and brush trajectories, so
that the user is aware of what they have drawn and what needs to
be drawn next, making it possible for the user to encode the correct
spatial position of the to-be-drawn shape without gazing toward the
brush point. Therefore, the Compass can eliminate the cursor drift-
ing in free-eye drawing and achieve accurate graphics, and different
UI combinations can be tailored to and used in different drawing
scenarios.
8.6.3. Smooth pursuit-based gaze interaction
Smooth pursuit eye movement has been the subject of studies for
many years (Robinson, 1965), but smooth pursuit-based gaze interac-
tion is a fairly new focus of study. Previous research has primarily
employed smooth pursuits as a visual guide for swift target selection
and has demonstrated smooth pursuits as a viable method to improve
selection accuracy (Lohr and Komogortsev, 2017; Pantanowitz et al.,
2021; Velloso et al., 2017; Vidal et al., 2015). The principle behind the
smooth-pursuit-based selection is categorized as ‘‘motion correlation’’:
to represent a target by motion and have users select the target by
replicating its motion (Velloso et al., 2017).
The present studies mostly focus on target selection, in which the
smooth pursuit motions and the target locations of these methods are
pre-determined. Additionally, smooth pursuit eye movements in these
methods rely on an externally driven stimulus. However, gaze-control
drawing is a completely different interaction that requires a swipe
mouse motion, namely, click-and-drag. In the free-eye drawing zone,
it is impossible to list all potential target locations for the brush cursor.
Our method leverages a novel mechanism with gaze interaction,
i.e., the self-driven smooth pursuits. In detail, the Compass enables
the user to self-drive the stimulus while also following it in a smooth
pursuit fashion. Smooth pursuits are used to control the gaze trajecto-
ries in the click-and-drag mouse event. To the best of our knowledge,
this type of smooth pursuit-based interaction has not been discussed in
eye-tracking literature to date. We prove that smooth pursuits can be
applied in gaze interaction that requires drag-and-click, thus bridging
the gap in the gaze-control modality.
Our method also provides a generalizable smooth pursuit-based
paradigm using a constant geometric interface and eliminates the need
to predetermine the interfaces in different scenarios. Specifically, based
on the Fourier Transformation (FT) theory (Boashash, 2015; Bochner
et al., 1949; Hassanieh et al., 2012), in which various figures can
be deformed into a function depending on space (gaze position) or
time (gaze duration), the gaze position and duration along with the
outlines of the Compass can create unlimited shapes. The drawing
results represent the competence of the Compass to handle creating
varied shapes, including but not limited to letters, icons, and graphical
figures, via free-eye drawing.
L. Huang et al.
9. Field study
This section evaluates the system usability among disabled people.
To achieve this, we arranged a field study that involved eleven motor-
disabled participants who could not use their hands and used eye
trackers to access computers. This study was conducted at two loca-
tions simultaneously, namely, Guangxi Province and Hubei Province,
China, in collaboration with the People’s Hospital of Guangxi Zhuang
Autonomous Region, the local day centers, and the Disabled Persons
Federation.
9.1. Apparatus
We used the same hardware in both locations to maintain study
consistency. A Lenovo Legion R7000 laptop with AMD Ryzen™ 5 4600H
3.0 GHz, Gigabyte GeForce RTX 2060 OC HDMI 3xDP 6 GB, 15.6-inch
display, and 16 GB RAM. The laptop had Tobii Eye-Tracking software,
EyeCompass, and Windows 10 installed. The screen resolution was
1920 × 1080 pixels. The laptop was set on a table and adapted to
the participants’ needs. The eye tracker used was Tobii 4C, which was
mounted at the bottom of the laptop screen. The participants sat at an
approximate distance of 65 cm from the laptop.
9.2. Participants
Eleven non-paid volunteers (motor-disabled, three females and eight
males, aged 25 to 70 years) with a hand or total body physical im-
pairment were recruited via partnerships with local hospitals, day
centers, and the Disabled Persons’ Federation. Four participants had
had strokes, three had cerebral palsy, two had repetitive strain injuries
in their wrists, and two had high paraplegia (severe motor disability).
None of them had any eye-related diseases or visual deficiencies. Five of
them had normal visual acuity, and six wore glasses. All the participants
were experienced in using eye trackers to control a computer. Three
participants were skilled in computer graphics, four were amateurs, and
the other four were novices. Among those experienced in drawing, they
commonly used Adobe Photoshop or Window Paint controlled via the
widgets of eye-tracking software.
9.3. Implementation
This study was approved by the Institutional Review Board (IRB).
The study was implemented in either the participants’ residences,
workplaces, or local day centers. Before the test, the participants were
informed of the procedures to be used and were asked to sign a
consent form. Then, demographic data, including their use of eye
trackers, drawing skills, and computer skills were collected. Next, the
participants underwent the eye-tracking calibration process and were
introduced to the EyeCompass interface in a 10-minute training session.
They were also tutored on the use of the dwell click and the manipula-
tion of the EyeCompass interface to ensure that they were familiar with
unimodal gaze control and the system.
The test commenced after the participants had confirmed they had
learned how to control the system, following a second eye-tracking
calibration. During the test session, they were encouraged to draw
simple sketches they had designed themselves. The drawing novices
were permitted to look up reference images on the Internet. During
the drawing process, they were free to edit the figures using all of the
functions (e.g., Draw, Erase, Undo, Redo, Move, Grid) and different
Compass interfaces of the system, and could modify or withdraw the
figures at any time. The participants were also allowed to take short
breaks (approx. 5 min) during the drawing test. Following each break,
they were asked to re-calibrate the eye trackers before starting drawing
again. The drawn images were saved at the end of the drawing process
once the participants were satisfied with the results.
17
International Journal of Human - Computer Studies 170 (2023) 102966
The system usability was tested using the SUS (Brooke, 1996)
questionnaire in the Chinese version. Following the usability test, a 10-
minute semi-structured interview was conducted with each participant,
which included open-ended questions such as ‘‘What challenges did you
have while drawing?’’ and ‘‘How do you feel that this system compares
to current eye-tracking tools and related applications?’’ The duration
of the entire process varied from 35 to 180 min in total, depending on
the length of time it took the participant to finish the drawing.
9.4. Results
9.4.1. System usability
The average SUS score of the participants in this experiment was
82.5 (SD=4.47), which, according to the SUS rating scales, falls within
the ‘‘Excellent’’ usability bracket (Bangor et al., 2009). All the partic-
ipants rated the system positively. Eight participants achieved scores
above or equal to 80.0. The participant who obtained the highest score
(90.0) showed a great interest in the system and stated: ‘‘I am very
curious about how to draw with eyes using the system, and it is amazing
for disabled people to draw’’. The lowest score obtained was 77.5 (which
falls within the ‘‘Good’’ usability bracket (Bangor et al., 2009) by the
three participants who were drawing novices. The participants com-
mented that the gaze-control methods available in EyeCompass were:
‘‘very helpful for people who have special needs’’, ‘‘novel’’, and ‘‘innovative’’
and that the gaze-control mechanisms of the system were ‘‘emerging ’’
compared to the eye-tracking applications they had used. For example,
‘‘I’ve never seen this method before, most eye-tracking applications can only
make selections through dwell clicks. It has been a profound experience to
create a drawing with my eyes.’’ One of the three drawing experts stated:
‘‘I used to draw with Photoshop via Tobii Windows Control, but it was
challenging to draw accurately with the eyes. In comparison, this system
allowed me to draw in a comfortable and accurate way.’’ Another drawing
expert stated: ‘‘The drawing software I have used is Windows Paint, which
has to be accessed via the eye-tracking widget Tobii Computer Control. This
system created better graphics with more ease and fidelity than the existing
approach.’’
9.4.2. Interface
The participants had positive things to say about the interface,
including that it was ‘‘intuitive’’, ‘‘straightforward’’, ‘‘easy to learn’’, and
‘‘worked well’’. One of the participants also highlighted that ‘‘The flat
design of the interface is concise and professional.’’ Additionally, the
participants all preferred to draw with the visually assistive grid on
the canvas: ‘‘It was more comfortable to control the gaze than a blank
canvas’’, especially when executing dwell clicks (e.g., ‘‘I could at least
gaze at the grid when I wanted to start or end the brush, or I could
not stabilize my gaze on the canvas without looking at anything ’’). The
users had different preferences regarding the possible combinations
offered by the Compass. They also found the customized settings for
the different combinations of the Compass and the brush speed setting
‘‘(very) convenient ’’, ‘‘mindful of users’’ and ‘‘helpful’’.
9.4.3. Drawing results
All the participants were able to command the interface and suc-
cessfully complete free-eye drawn graphics with relatively high fidelity.
Remarkably, the drawing experts and amateurs created paintings of
an artistic level (Fig. 18); these drawings were created with evident
individual style. Additionally, one of the drawing experts commented
that the system enabled the creation of various styles of illustrations,
such as one-line-drawings, scribbles, and abstract arts.
L. Huang et al.
Fig. 18. The graphics drawn by the participants.
9.5. Discussion
9.5.1. Usability of EyeCompass
EyeCompass represents a high-fidelity and effective free-eye draw-
ing method for motor-disabled people. In the field study, the overall
SUS score and user feedback indicated that the system has good us-
ability and high acceptance among the participants. All the participants
were able to create simple artistic drawings through the guidance of the
Compass with unimodal gaze control. When compared to the drawings
created via gaze-control that were reported in the literature (Gips
and Olivieri, 1996; Heikkilä, 2013a,b; Hornof et al., 2003; Meyer and
Dittmar, 2009; Streichert et al., 2020; Van der Kamp and Sundstedt,
2011; Yeo and Chiu, 2006), it is apparent that our system was able
to obtain results that more closely represented drawings made by
hand. Specifically, previous free-eye drawing approaches (Gips and
Olivieri, 1996; Streichert et al., 2020) have been unable to make a
distinction between ocular observation and drawing gaze movements,
leading to inaccuracies that reflect a lack of control over the brush
(gaze). The other gaze-control drawing applications (Heikkilä, 2013a,b;
Hornof et al., 2003) facilitate the reproduction of preset essential
shapes (e.g., circle, ellipse, square, straight line, rectangle) rather than
free-eye drawing, with the results featuring polylines and lacking detail
and variety. In these studies, the drawings obtained and the user
experiences uncovered issues including lack of brush control, a conflict
between ocular observation and visuomotor transformation, inaccuracy
of the gaze trajectories, and eye fatigue.
By integrating brush-damping dynamics and gaze-oriented mech-
anism, our system addressed these challenges (i.e., involuntary eye
movements, the conflict between visuomotor transformation and ocular
observation, and inherent eye-tracking errors) in gaze-control drawing,
especially free-eye drawing, enhancing the control of gaze and the
ability to create more intricate figures. The participants with a motor
disability were thus able to achieve illustrations with a higher graphical
fidelity using our system.
Furthermore, the gaze-based interface of EyeCompass provides
straightforward and effective access to drawing functions and addresses
the gaze-control problem in manipulating interfaces with pixel-level
elements and multiple hierarchies, such as Photoshop. The user ex-
periences also show that the customized options for the drawing
mechanism parameters are efficient and mindful of different users’
needs (Fig. 19). In particular, the users predominantly preferred draw-
ing with the grid on. Our experiment validated the effectiveness of the
gaze-oriented mechanism and proved that appropriate visual aids on
the canvas that follow the stimulus-driven pattern for gaze control are
helpful.
9.5.2. Generalization of EyeCompass mechanisms
Digital graphics generally depend on click-and-drag mouse interac-
tion. Implementing the direct click-and-drag function via gaze control
remains a challenge at present. The latest contribution of the gaze-
control click-and-drag is Gaze+Hold, which emulates click-and-drag by
closing one eye (Ramirez Gomez et al., 2021). Specifically, Gaze+Hold
can move an object on the screen in four steps: gazing at it, closing one
eye to pick it up, gazing at a target location, and then opening the eye
to drop the object. Unfortunately, Gaze+Hold is not feasible for digital
graphics as it cannot control gaze trajectories in real time. To the best
18
International Journal of Human - Computer Studies 170 (2023) 102966
of our knowledge, there is scant research on a gaze interaction mode
that can accurately control gaze trajectories during mouse dragging.
The high usability and good drawing performance achieved by
EyeCompass points to a feasible solution for general graphics software
that requires gaze trajectory control. The series of user studies verified
the robustness of the novel gaze-control mechanisms, i.e., the brush-
damping dynamics and the gaze-oriented mechanism, and indicated
that a combination of these mechanisms could accomplish the click-
and-drag mouse event with gaze accuracy. On the one hand, the
damping dynamics stabilized the gaze trajectories and filtered the noise
from gaze movements, thus enabling the user to control the gaze point
(mouse) dynamics quantitatively and precisely with the damping ratio.
On the other hand, the gaze-oriented Compass enabled the user to con-
trol the gaze naturally and accurately by using the eyes’ physiological
features, i.e., smooth pursuits and the stimulus-driven pattern. Various
gaze trajectories could be obtained by adapting different visual stimuli
as the cursor interface. The hybrid of these mechanisms is promising for
use with creative design applications other than drawing, for example,
computer-aided design (CAD) software.
9.5.3. Generalization of the unimodal gaze control
EyeCompass, with its unimodal gaze-control function, was demon-
strated as being accessible for severely disabled participants in this field
study. Other multimodal gaze-control applications for drawing and re-
lated creative programs have been created in the past. For example, Van
der Kamp and Sundstedt (2011) introduced a multimodal drawing
program managed via gaze and voice input. The voice commands
started and stopped the drawing while the gaze commands determined
the cursor position. Dziemian et al. (2016) presented a gaze-controlled
writing and drawing robotic arm that could be controlled by tilting the
head forward toward and backward from the canvas to indicate the
start and end of the drawing task. Creed et al. (2020) proposed a novel
creative design application called Sakura that used multimodal gaze
interaction, within which the gaze control modality collaborated with
a mechanical switch. These multimodal studies highlighted that there
was an access barrier for severely motor-disabled users. The unimodal
EyeCompass bridges this gap and provides better accessibility to people
with motor disabilities. Furthermore, the unimodal gaze-control system
can easily integrate with other control modes, such as voice input, foot
paddles, and QuadSticks, to cater to various individual needs.
10. Limitations
10.1. Remaining problems in free-eye drawing
In this paper, we focus on addressing some particular problems of
free-eye drawing, including the lack of gaze-control accuracy and the
unavailability of gaze-control drawing methods in practice. Although
our method has improved gaze-control drawing significantly, some
problems remain. One limitation in our study is the brush tailing
observed during the drawing tests: ending the brush via dwell clicks
would lead to redundant lines of varying degrees at the endpoint of
the drawn line as the user waited for the end of the brush. This issue
originates from the constant eye jitters in gaze data due to the nature of
eyes (Murata and Karwowski, 2018; Skovsgaard et al., 2010). Although
the damping function relieved eye jitters, the system also employed
multiple other strategies (i.e., a progress bar for the dwell click, a
L. Huang et al.
Fig. 19. An example of a drawing made using the customized options. The mechanisms were set to be the spiral cursor with a customized brush speed (0.5x) and the grid turned
on.
shortened dwell time); brush tailing could not be eliminated as it was
rooted in the dwell click interaction.
Therefore, alternative approaches to end the brush rapidly and
robustly are necessary for improved gaze-control drawing. In our ex-
tended research, a trial version using eye closure to activate the draw-
ing command was tested with a small group of people. Still, the
approach proved to be problematic as the brush point went downward
with the closing of the eyelids, as usually occurs during eye closing.
The consequence was a plethora of involuntary lines. Due to the time
limits and the demands on the participants, a further user study was
not conducted. However, it is imperative to determine a quicker and
more precise approach to exiting drawing execution in the future.
10.2. System capability
Another limitation is that the system’s capability is vulnerable to
factors such as algorithmic issues arising from eye trackers, interac-
tion environments, and perceptual problems. Drawing with the eyes
is conducted in a more complex environment than drawing with the
hands. Even a perfect free-eye drawing system will fail due to multiple
impact factors, including algorithmic issues related to eye trackers, eye-
tracking calibration, the user’s gaze interaction skill, and the perceptual
abilities of the user. For example, the gaze trajectories will not be
correctly generated if the gaze data is not correctly captured by the
devices.
Furthermore, the system involves different combinations of the
Compass and brush speed settings capable of meeting different drawing
needs. Mastering the tricks of conceiving what and how to draw via
the Compass demands significant learning time and exposure to the
system. This was evident when participants in the field study who were
drawing novices were less perceptive of the tricks of the Compass. More
sophisticated methods need to be explored to simplify the learning
processes of free-eye drawing.
Therefore, although the usability of free-eye drawing was improved
significantly by EyeCompass, there remains a need to keep pace with
the capabilities and conveniences of drawing by hand. Employing tech-
nologies such as machine learning and deep learning could potentially
reduce the problematic impact factors aforementioned, pointing toward
interesting directions for future research.
10.3. System function design
EyeCompass is a basic drawing system that focuses on solving the
critical problems in current gaze-control drawing methods. The main
goal of EyeCompass is to enable high fidelity and efficient free-eye
drawing via unimodal gaze control. As such, it has a limited set of
drawing functions. When it is generalized to drawing applications in the
future, additional editing functions (e.g., rotation, transformation, pixel
selection, brush styles, multiple views) would be desirable, as would be
a greater amount of interaction design via gaze control. The extension
of the solution’s functionality requires rigorous and longitudinal user
tests based on the principles of gaze-control drawing (Section 3) and
gaze-based interface design (Hansen et al., 2018).
19
International Journal of Human - Computer Studies 170 (2023) 102966
11. Conclusion and future work
Previous gaze-control drawing research cannot facilitate gaze-
control drawing with high-fidelity graphics and good system accessibil-
ity. In terms of gaze-control drawing for motor-disabled people, there
is currently no sophisticated application on the market, nor has any
literature been produced on the matter in recent years. The research
that does exist was published some time ago and is therefore outdated,
due to its use of simplistic gaze-control drawing methods and super-
seded interfaces (Gips and Olivieri, 1996; Heikkilä, 2013a,b; Hornof
et al., 2003; Meyer and Dittmar, 2009; Van der Kamp and Sundstedt,
2011). Moreover, the graphic effectiveness and system usability of these
studies was rarely verified in the articles concerned.
Several recent studies have adopted multimodal approaches to de-
veloping gaze-control drawing and achieved a promising degree of
usability. However, multimodal gaze control excludes some severely
motor-disabled people from using the application; unimodal gaze con-
trol is the most accessible approach for all physical abilities, including
individuals with motor disabilities.
This paper, therefore, proposed a free-eye drawing method that
enables high-fidelity and efficient free-eye drawing via unimodal gaze
control. The drawing system (EyeCompass) was developed based on
cross-disciplinary bodies of work, including the physiological eye move-
ment patterns involved, the nature of the eye in eye-hand drawing,
and eye-tracking technologies. EyeCompass consists of two novel gaze-
control mechanisms: brush-damping dynamics and gaze-oriented inter-
faces. This paper postulated two hypotheses to examine the system’s
usability and fidelity: (1) brush-damping dynamics enable users to draw
intuitively, reduce involuntary eye movement error, and stabilize the
brush speed; and (2) the gaze-oriented Compass aligns with the eyes’
target-oriented perceptive nature and makes it feasible to draw various
figures with high fidelity and efficiency.
The above hypotheses were tested via a series of user studies
conducted in laboratories and fieldwork among motor-disabled people.
User Study 1 verified that the brush-damping dynamics were able to
significantly improve brush stability and controllability and to reduce
noise from involuntary eye movements and eye jitters to a considerable
degree. The user feedback indicated that the damping function con-
formed to the real-world drawing interaction (Gowen and Miall, 2006;
Tchalenko, 2007; Tchalenko et al., 2014) by providing a time lapse for
eye observation and cognition. Moreover, a damping ratio of 10−2 was
demonstrated to be the optimal value for both drawing performance
and efficiency. User Study 2 demonstrated a significant improvement in
drawing performance via the Compass. By gazing through the Compass’
cursors, users could draw high-fidelity figures with greater flexibility
and lower error rates than using the previous gaze-control drawing
methods. Additionally, the extended interface design of the Compass
could compensate for the inherent eye-tracking bias.
Finally, the field study investigated the system usability among
motor-disabled people and the interface preferences of the users. It
was found that EyeCompass has a superior usability to previous gaze-
control drawing approaches and produced optimistic reports of user
L. Huang et al.
experiences among motor-disabled people. The drawing outputs in the
field study also emphasized the capability of EyeCompass to produce
flexible graphics via unimodal gaze control.
In conclusion, EyeCompass helps people to draw using only their
eyes, which is of value to the motor disabled. The system developed
by this study has filled the void of free-eye drawing research in recent
years and addressed the main challenges of current gaze-control draw-
ing, especially free-eye drawing, including involuntary eye movements,
the conflict of the dual-task (observation and drawing) for eyes, and
the effect of eye-tracking errors. The system developed in this study
achieved hand-sketch level drawing with high fidelity and creativity,
showing the technique’s ability to complement real-world drawing
tasks. The mechanisms behind the system, namely the gaze-oriented
Compass and brush-damping dynamics, also make a scientific contribu-
tion that advances understanding of the kind of gaze interaction that is
comfortable for the eyes. Furthermore, the combination of the Compass
and brush-damping dynamics is generalizable for the typical click-and-
drag mouse event. Therefore, the gaze interaction of EyeCompass is
of compelling value for future gaze-interaction research, not only for
gaze-controlled drawing but also for other application scenarios, such
as digital design software, gaze-controlled games, and desktop-control
eye-tracking programs.
Moreover, this system has fulfilled the need for universal access
via its unimodal gaze control, which is compatible with other multi-
modal devices, such as voice input, foot paddles, and QuadStick. As
human–computer interaction becomes increasingly ubiquitous, interac-
tions coupling multiple modalities will provide motor-disabled people
with greater adaption and accessibility.
There are multiple directions for further research: (1) the explo-
ration of a substitute for dwell click for start-drawing and stop-drawing
commands; (2) the exploration of artificial intelligence methods to
assist in free-drawing, such as automatic revision and generation of
sketches; (3) the expansion of the drawing functions and connections
to common drawing applications, such as Adobe Photoshop and Win-
dows Paint; and (4) further development and improvement of gaze
interaction based on eye physiology, and the enablement of more
motor-disabled people to produce creative works.
CRediT authorship contribution statement
Lida Huang: Conceptualization, Methodology, Software, Investi-
gation, Formal analysis, Writing – original draft. Thomas Westin:
Proofreading. Mirjam Palosaari Eladhari: Writing – review & edit-
ing, Supervision. Sindri Magnússon: Supervision. Hao Chen: Data
curation, Visualization, Investigation, Validation.
Declaration of competing interest
One or more of the authors of this paper have disclosed potential or
pertinent conflicts of interest, which may include receipt of payment,
either direct or indirect, institutional support, or association with an
entity in the biomedical field which may be perceived to have potential
conflict of interest with this work. For full disclosure statements refer
to https://doi.org/10.1016/j.ijhcs.2022.102966. The first author has
a survey paper entitled "A Study of the Challenges of Eye Tracking
Systems and Gaze Interacton for Individuals with Motor Disabilities"
(Cited as Huang L, Xu C, Westin T, et al. A Study of the Challenges
of Eye Tracking Systems and Gaze Interacton for Individuals with
Motor Disabilities[C]//International Conference on Human-Computer
Interacton. Springer, Cham, 2022: 396- 411.) published by the Inter-
national Conference on Human-Computer Interacton, Springer. In this
conference paper, the challenge of gaze-control drawing was discovered
among motor-disabled people. However, this conference paper does
not conflict with our submitted paper to your journal. It has some
background information about gaze-control drawing practical examples
and is referred to by this journal paper. Therefore, we report this
conference paper to avoid any possible interpretation.
20
International Journal of Human - Computer Studies 170 (2023) 102966
Data availability
The data of the user studies and the field study are confidential as
it involves personal information. The code of EyeCompass is available
on request.
References
Aldridge, R., Davidoff, J., Ghanbari, M., Hands, D., et al., 1995. Measurement of scene-
dependent quality variations in digitally coded television pictures. IEEE Proc.-Vision
Image Signal Process. 142 (3), 149–154.
Bangor, A., Kortum, P., Miller, J., 2009. Determining what individual SUS scores mean:
Adding an adjective rating scale. J. Usability Stud. 4 (3), 114–123.
Barasch, M., 2013. Theories of Art: 3. from Impressionism to Kandinsky. Routledge.
Boashash, B., 2015. Time-Frequency Signal Analysis and Processing: A Comprehensive
Reference. Academic Press.
Bochner, S., Chandrasekharan, K., Chandrasekharan, K., 1949. Fourier Transforms. (19),
Princeton University Press.
Bozomitu, R., Păsărică, A., Cehan, V., Rotariu, C., Costin, H., 2017. Methods of control
improvement in an eye tracking based human-computer interface. In: 2017 IEEE
23rd International Symposium for Design and Technology in Electronic Packaging.
SIITME, IEEE, pp. 300–303.
Brooke, J., 1996. Sus: a ‘‘quick and dirty’’ usability. Usability Eval. Ind. 189.
Cecotti, H., 2016. A multimodal gaze-controlled virtual keyboard. IEEE Trans.
Hum.-Mach. Syst. 46 (4), 601–606.
Chin, C.A., Barreto, A., Cremades, J.G., Adjouadi, M., 2008. Integrated Electromyogram
and Eye-Gaze Tracking Cursor Control System for Computer Users with Motor
Disabilities. Electrical and Computer Engineering Faculty Publications.
Cranley, N., Perry, P., Murphy, L., 2006. User perception of adapting video quality.
Int. J. Hum.-Comput. Stud. 64 (8), 637–647.
Creed, C., Frutos-Pascual, M., Williams, I., 2020. Multimodal gaze interaction for
creative design. In: Proceedings of the 2020 CHI Conference on Human Factors
in Computing Systems. pp. 1–13.
Duchowski, A.T., Duchowski, A.T., 2017. Eye Tracking Methodology: Theory and
Practice. Springer.
Dziemian, S., Abbott, W.W., Faisal, A.A., 2016. Gaze-based teleprosthetic enables
intuitive continuous control of complex robot arm use: Writing & drawing. In: 2016
6th IEEE International Conference on Biomedical Robotics and Biomechatronics
(BioRob). IEEE, pp. 1277–1282.
Feit, A.M., Williams, S., Toledo, A., Paradiso, A., Kulkarni, H., Kane, S., Morris, M.R.,
2017. Toward everyday gaze input: Accuracy and precision of eye tracking and
implications for design. In: Proceedings of the 2017 Chi Conference on Human
Factors in Computing Systems. pp. 1118–1130.
Gips, J., Olivieri, P., 1996. An eye control system for persons with disabilities. In:
COMPUTER SCIENCE DEPARTMENT. the Eleventh International Conference on
Technology and Persons with Disabilities. Los Angeles, California.
Goodrich, D., 1954. Imaginative design: Abstract drawing projects for beginners in art.
Design 55 (3), 120–142.
Gowen, E., Miall, R.C., 2006. Eye–hand interactions in tracing and drawing tasks. Hum.
Mov. Sci. 25 (4–5), 568–585.
Hansen, D.W., Ji, Q., 2009. In the eye of the beholder: A survey of models for eyes
and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32 (3), 478–500.
Hansen, J.P., Rajanna, V., MacKenzie, I.S., Bækgaard, P., 2018. A fitts’ law study of
click and dwell interaction by gaze, head and mouse with a head-mounted display.
In: Proceedings of the Workshop on Communication By Gaze Interaction. pp. 1–5.
Hassanieh, H., Indyk, P., Katabi, D., Price, E., 2012. Simple and practical algorithm for
sparse Fourier transform. In: Proceedings of the Twenty-Third Annual ACM-SIAM
Symposium on Discrete Algorithms. SIAM, pp. 1183–1194.
Hayhoe, M., 2000. Vision using routines: A functional account of vision. Vis. Cogn. 7
(1–3), 43–64.
Hayhoe, M., Ballard, D., 2005. Eye movements in natural behavior. Trends in Cognitive
Sciences 9 (4), 188–194.
Heikkilä, H., 2013a. Eyesketch: a drawing application for gaze control. In: Proceedings
of the 2013 Conference on Eye Tracking South Africa. pp. 71–74.
Heikkilä, H., 2013b. Tools for a gaze-controlled drawing application–comparing gaze
gestures against dwell buttons. In: IFIP Conference on Human-Computer Interaction.
Springer, pp. 187–201.
Hillstrom, A.P., Yantis, S., 1994. Visual motion and attentional capture. Percept.
Psychophys. 55 (4), 399–411.
Hornof, A., Cavender, A., Hoselton, R., 2003. Eyedraw: a system for drawing pictures
with eye movements. ACM SIGACCESS Accessibility Comput. (77–78), 86–93.
Huang, L., Xu, C., Westin, T., Dupire, J., Le Lièvre, F., Shi, X., 2022. A study of the
challenges of eye tracking systems and gaze interaction for individuals with motor
disabilities. In: International Conference on Human-Computer Interaction. Springer,
pp. 396–411.
Hunter, J.S., 1986. The exponentially weighted moving average. J. Qual. Technol. 18
(4), 203–210.
L. Huang et al.
Istance, H., Bates, R., Hyrskykari, A., Vickers, S., 2008. Snap clutch, a moded approach
to solving the Midas touch problem. In: Proceedings of the 2008 Symposium on
Eye Tracking Research & Applications. pp. 221–228.
Jacob, R.J., 1991. The use of eye movements in human-computer interaction tech-
niques: what you look at is what you get. ACM Trans. Inf. Syst. (TOIS) 9 (2),
152–169.
Johansson, R.S., Westling, G., Bäckström, A., Flanagan, J.R., 2001. Eye–hand
coordination in object manipulation. J. Neurosci. 21 (17), 6917–6932.
Kasprowski, P., Harezlak, K., Niezabitowski, M., 2016. Eye movement tracking as a new
promising modality for human computer interaction. In: 2016 17th International
Carpathian Control Conference. ICCC, IEEE, pp. 314–318.
Lander, C., Löchtefeld, M., Krüger, A., 2018. Heyebrid: A hybrid approach for mobile
calibration-free gaze estimation. Proc. ACM Interact. Mob. Wearable Ubiquitous
Technol. 1 (4), 1–29.
Lankford, C., 2000. Effective eye-gaze input into windows. In: Proceedings of the 2000
Symposium on Eye Tracking Research & Applications. pp. 23–27.
Lohr, D.J., Komogortsev, O.V., 2017. A comparison of smooth pursuit-and dwell-
based selection at multiple levels of spatial accuracy. In: Proceedings of the 2017
CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp.
2760–2766.
Majaranta, P., Bulling, A., 2014. Eye tracking and eye-based human–computer
interaction. In: Advances in Physiological Computing. Springer, pp. 39–65.
Maxwell, S.E., Delaney, H.D., Kelley, K., 2017. Designing Experiments and Analyzing
Data: A Model Comparison Perspective. Routledge.
Menges, R., Kumar, C., Staab, S., 2019. Improving user experience of eye tracking-
based interaction: Introspecting and adapting interfaces. ACM Trans. Comput.-Hum.
Interact. 26 (6), 1–46.
Meyer, A., Dittmar, M., 2009. Conception and development of an accessible application
for producing images by gaze interaction-EyeArt.
Murata, A., Karwowski, W., 2018. Automatic lock of cursor movement: Implications
for an efficient eye-gaze input method for drag and menu selection. IEEE Trans.
Hum.-Mach. Syst. 49 (3), 259–267.
Nielsen, A.M., Petersen, A.L., Hansen, J.P., 2012. Gaming with gaze and losing
with a smile. In: Proceedings of the Symposium on Eye Tracking Research and
Applications. pp. 365–368.
Pantanowitz, A., Kim, K., Chewins, C., Rubin, D.M., 2021. Gaze tracking dataset for
comparison of smooth and saccadic eye tracking. Data Brief 34, 106730.
Parisay, M., Poullis, C., Kersten-Oertel, M., 2021. Eyetap: Introducing a multimodal
gaze-based technique using voice inputs with a comparative analysis of selection
techniques. Int. J. Hum.-Comput. Stud. 102676.
Ramirez Gomez, A.R., Clarke, C., Sidenmark, L., Gellersen, H., 2021. Gaze+ hold: Eyes-
only direct manipulation with continuous gaze modulated by closure of one eye.
In: ACM Symposium on Eye Tracking Research and Applications. pp. 1–12.
Ramirez Gomez, A., Gellersen, H., 2019. Looking outside the box: reflecting on
gaze interaction in gameplay. In: Proceedings of the Annual Symposium on
Computer-Human Interaction in Play. pp. 625–637.
Robinson, D.A., 1965. The mechanics of human smooth pursuit eye movement. J.
Physiol. 180 (3), 569–591.
Roelfsema, P.R., Lamme, V.A., Spekreijse, H., 2000. The implementation of visual
routines. Vis. Res. 40 (10–12), 1385–1411.
Ruskin, J., 1876. The Elements of Drawing; in Three Letters to Beginners.
Skovsgaard, H., Mateo, J.C., Flach, J.M., Hansen, J.P., 2010. Small-target selection with
gaze alone. In: Proceedings of the 2010 Symposium on Eye-Tracking Research &
Applications. pp. 145–148.
21
International Journal of Human - Computer Studies 170 (2023) 102966
Slobodenyuk, N., 2016. Towards cognitively grounded gaze-controlled interfaces. Pers.
Ubiquitous Comput. 20 (6), 1035–1047.
Smith, J.D., Graham, T.N., 2006. Use of eye movements for video game control. In:
Proceedings of the 2006 ACM SIGCHI International Conference on Advances in
Computer Entertainment Technology. pp. 20–es.
Snowden, R., Snowden, R.J., Thompson, P., Troscianko, T., 2012. Basic Vision: An
Introduction to Visual Perception. Oxford University Press.
Soechting, J.F., Rao, H.M., Juveli, J.Z., 2010. Incorporating prediction in models for
two-dimensional smooth pursuit. PLoS One 5 (9), e12574.
Stawicki, P., Gembler, F., Rezeika, A., Volosyak, I., 2017. A novel hybrid mental spelling
application based on eye tracking and SSVEP-based BCI. Brain Sci. 7 (4), 35.
Streichert, A., Angerbauer, K., Schwarzl, M., Sedlmair, M., 2020. Comparing input
modalities for shape drawing tasks. In: ACM Symposium on Eye Tracking Research
and Applications. pp. 1–5.
Streijl, R.C., Winkler, S., Hands, D.S., 2016. Mean opinion score (MOS) revisited:
methods and applications, limitations and alternatives. Multimedia Syst. 22 (2),
213–227.
Sun, H.-M., Huang, C.-W., 2011. The effect of media richness factors on
representativeness for video skim. Int. J. Hum.-Comput. Stud. 69 (11), 758–768.
Tchalenko, J., 2001. Free-eye drawing. Point: Art Des. Res. J. 11, 36–41.
Tchalenko, J., 2007. Eye movements in drawing simple lines. Perception 36 (8),
1152–1167.
Tchalenko, J., Nam, S.-H., Ladanga, M., Miall, R.C., 2014. The gaze-shift strategy in
drawing. Psychol. Aesthet. Creativity Arts 8 (3), 330.
Ullman, S., 1987. Visual routines. In: Readings in Computer Vision. Elsevier, pp.
298–328.
Urbina, M.H., Huckauf, A., 2010. Alternatives to single character entry and dwell time
selection on eye typing. In: Proceedings of the 2010 Symposium on Eye-Tracking
Research & Applications. pp. 315–322.
Van der Kamp, J., Sundstedt, V., 2011. Gaze and voice controlled drawing. In:
Proceedings of the 1st Conference on Novel Gaze-Controlled Applications. pp. 1–8.
Vázquez, M., Steinfeld, A., 2014. An assisted photography framework to help visually
impaired users properly aim a camera. ACM Trans. Comput.-Hum. Interact. 21 (5),
1–29.
Velichkovsky, B.B., Rumyantsev, M.A., Morozov, M.A., 2014. New solution to the midas
touch problem: Identification of visual commands via extraction of focal fixations.
Procedia Comput. Sci. 39, 75–82.
Velloso, E., Carter, M., 2016. The emergence of eyeplay: a survey of eye interaction
in games. In: Proceedings of the 2016 Annual Symposium on Computer-Human
Interaction in Play. pp. 171–185.
Velloso, E., Carter, M., Newn, J., Esteves, A., Clarke, C., Gellersen, H., 2017. Motion cor-
relation: Selecting objects by matching their movement. ACM Trans. Comput.-Hum.
Interact. 24 (3), 1–35.
Verghese, P., Beutter, B.R., 2002. Motion processing. In: Ramachandran, V. (Ed.),
Encyclopedia of the Human Brain. Academic Press, New York, pp. 117–135.
Vidal, M., Bulling, A., Gellersen, H., 2015. Pursuits: Spontaneous eye-based interaction
for dynamic interfaces. GetMobile: Mob. Comput. Commun. 18 (4), 8–10.
Yantis, S., 1998. Control of visual attention. Attention 1 (1), 223–256.
Yeo, A.W., Chiu, P.-C., 2006. Gaze estimation model for eye drawing. In: CHI’06
Extended Abstracts on Human Factors in Computing Systems. pp. 1559–1564.
Zhang, S., Tian, Y., Wang, C., Wei, K., 2020. Target selection by gaze pointing
and manual confirmation: performance improved by locking the gaze cursor.
Ergonomics 63 (7), 884–895.