Professional Documents
Culture Documents
ARTÍCULO The Effect of Spatial Ability On Immersive 3D Drawing
ARTÍCULO The Effect of Spatial Ability On Immersive 3D Drawing
Figure 1. 3D drawings made by high and low spatial ability participants while walking in study 1.
173
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
spatial ability. Previous work has already shown that spatial Another factor affecting 3D drawing is the correct
ability influences 3D modelling in CAD programs [9, 14] positioning of a stroke in the 3D scene, as the user needs to
and on 2D drawings [39]. However, no other work has consider the spatial relationships between objects while
studied the effect of the user’s spatial ability on 3D drawing [4]. This task relies on the user’s spatial ability [9,
immersive drawing and how it affects the final sketch. 17], which consists of three elements: a) spatial
Through understanding users’ behaviours during the stroke visualization, the mental ability to manipulate 2D and 3D
planning phase, our work extends previous work that figures, b) spatial orientation, the ability to rotate mental
analyzes the stroke creation process [2, 57]. In our first representations of 2D and 3D objects, and c) spatial
study, we investigate how participants create 3D sketches perception, the ability to be aware of one’s relationship with
using freehand drawing while standing or walking. We the environment. A person’s spatial ability is dependent on
found that the shape likeness of the drawing is affected by many factors, such as training [14, 46]. Previous work has
the user’s spatial ability, but not the line precision, with found that individuals utilize all elements of their spatial
sketches drawn by high-spatial-ability participants being ability [9, 39, 46] and their spatial memory of the scene [47]
more like the target shapes than sketches drawn by low- while drawing. For example, Orde [39] found that people
spatial-ability participants (Figure 1). We found no with more developed spatial skills are more capable of
difference between standing and walking, but when only converting an abstract mental picture into a concrete product
consider walking the high-spatial-ability user’s movement and that the higher their spatial skills are, the more
was more systematic than the low-spatial-ability user’s sophisticated their representation is. Samsudin et al. [46]
movement. This difference affects their shape likeness. In also found that users with high spatial orientation can better
our second experiment, we further investigate the effect of conceptualize the world around them and thus can better
viewpoint change on the stroke planning phase. We found solve orthographic drawing tasks. For 3D content creation,
that, when walking, the participant’s shape likeness and line Branoff and Dobelis [9] found a relationship between the
precision was better than when using hand-based view students’ spatial abilities and their final scores in a university
control. We also found that choosing the correct viewpoint modelling course, and showed that students with high spatial
to draw had a positive effect on the final sketch. ability got better scores on their 3D modelling test than low
spatial ability students.
RELATED WORK
3D Drawing Tools
Challenges of 3D Drawing
3D stroke creation using VR devices is less accurate than 2D Creating a user interface that lets users accurately draw in a
stroke creation using pen and paper [2, 57]. For example, 3D virtual environment has been an open area of research for
Wiese et al. [57] found a difference in line precision and decades. Earlier systems such as HoloSketch [13] or
completion time between 2D and 3D sketches. They found CavePainting [29], showed the possibilities of directly
that 3D drawing requires higher manual effort and imposes drawing in 3D by using a straight one-to-one mapping of
higher cognitive and sensorimotor demands than 2D body movements to strokes. This technique, called freehand
drawing. This is a consequence of the need to control more drawing, is easy to learn and use [56] and is the basis of most
degrees of freedom during movement (6DOF instead of current commercial systems like FreeDrawer [56], Quill [16]
2DOF). Arora et al. [2] also found that users’ shape likeness and Tilt Brush [20]. However, the accuracy of freehand
for VR decreased by 148%, using the metric of overall mean drawing is reduced by the stroke creation challenges
deviation from a target stroke. One possible explanation for presented above. Previously proposed user interfaces for
this difference is the lack of a physical surface, since it forces immersive 3D drawing have tried to increase the user’s line
users to rely only on their eye-hand coordination to control precision during the stroke creation process. Some tools like
the stroke position. However, previous work by Tramper and Lift-Off [27] and Drawing on Air [30] use novel metaphors
Gielen [53] on eye-hand coordination has found a difference to constrain the stroke creation process by reducing the
in the dynamics of visuomotor control for version and demands on the user’s visuomotor skills. Other tools reduce
vergence. In this context, we understand version as coupled the user manual effort by beautifying strokes, i.e. Fiorentino
eye movements in the same lateral/vertical direction, and et al. [18] and Multiplanes [5]. Beautification is the process
vergence as independent eye movements in opposite of transforming informal and ambiguous freehand input to
directions for depth accommodation. Arora et al. [2] also more formal and structured representations [37]. There are
identify the depth perception problems associated with also tools that let users draw on a plane to reduce the effect
stereo displays as a likely reason for the inaccuracy of 3D of depth perception errors in stereo display systems. These
immersive drawing. Such problems include the under- include Digital Tape Drawing [22], ImmersiveFiberMesh
estimation of distances [42] and the targeting accuracy [40], Multiplanes [5], and SymbiosisSketch [3].
differences between movements in the vergence and the Immersive 3D drawing tools also let users easily change
version direction [6, 7]. their viewpoint [3, 5, 16, 20] by walking around their
drawing. This characteristic is important since viewing a 3D
174
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
object from multiple viewpoints helps users create a mental shape likeness of the final drawing. We expect that users
model of the object they are drawing [45, 50]. Some tools let with higher spatial abilities will draw shapes that are more
users employ the grab the air interaction technique [44], or similar to the example than those with lower abilities.
a variation of it, to further manipulate their drawings. For
Effect of viewpoint change while 3D drawing.
example, ImmersiveFiberMesh [40] and Digital Tape Previous work has found that when comparing walking and
Drawing [22] use two-hand pan and zoom interaction, in standing while doing a data analysis task [35], users’ spatial
which hand movement direction specifies the camera abilities influence their performance. Similar to 3D drawing,
movement. Lift-Off [27] lets users grab the drawing with data analysis is also a visually-demanding 3D task [35].
their non-dominant hand and reorient it by moving that hand. Based on this, we hypothesize (H2) that a user’s line
Previous work has studied the effect of changing the precision and shape likeness diminish when they change
viewpoint on 3D CADs and found that it may lead to their viewpoint while drawing. We expect that when drawing
disorientation and frustration and suggested the use of while standing the user’s line precision and shape likeness
widgets to help users [19, 32]. However no previous work will be higher than when they walk while drawing.
has studied this effect on 3D immersive drawing.
Effect of the user’s behavioural differences based upon
MOTIVATION their spatial ability.
In this paper, we aim to identify the user behaviours during Two important elements of spatial ability are spatial
the stroke-planning phase while drawing in VR depending orientation and spatial visualization, both of which are
on their spatial ability. We focus on the stroke-planning distinct abilities and affect the way the user understands the
phase since in this stage users mentally plan the next stroke 3D environment [23, 34, 41]. Based on this relationship we
based on the current drawing [28], choosing the viewpoint hypothesize (H3) that a user’s spatial ability affects drawing
and the hand position from where to draw the new stroke. behaviours in 3D regarding their hand position in space,
When selecting the viewpoint, the user shape likeness may viewpoint orientation and hand movement direction. We
be affected by the disorientation caused by low spatial expect that high-spatial-ability users draw differently than
updating [43, 52], which is the ability to create and update a low-spatial-ability users.
mental model of the environment. This problem can be
enhanced by the method used to control the view [8, 55]. USER STUDY 1
When choosing the correct hand position, the user shape This study aims to establish a baseline for freehand 3D
likeness and line precision may be affected by their depth immersive drawing depending on the user’s spatial ability. It
perception [31], which depends on the display type [6, 7], also evaluates the differences between staying in the same
and each individuals’ vision system [24]. We also aim to position and walking while drawing a single 3D object in
quantify the effect of the user’s spatial ability on their final VR.
3D drawings. Understanding the effect of the spatial ability
Methodology
is important because planning the next stroke relies on the
understanding of the correct relationships between the Participants
elements inside the 3D virtual environment, which heavily We recruited 12 participants from the university community
depends on the user’s spatial ability [11]. (7 female). 10 of participants were between 18-24 years old
and 2 between 25-34 years old. Only one participant was left
Previously proposed user interfaces for 3D immersive handed. The participants’ frequency of drawing with pen and
drawing have focused on helping users create precise lines paper was, 1 drawing every day, 5 a few times a week, 1
by solving the challenges of the stroke creation process. once a week, 4 a few times a month, and 1 once a month. For
However, few tools have focused on solving the challenges drawing in VR, 8 of our participants had never drawn in VR
of the stroke-planning phase, which is less understood. before, 2 had drawn 2-4 times, and 2 had drawn between 5-
Helping users plan their next stroke is important: Israel et al. 9 times.
[26] showed that users expect that directly drawing in an
immersive 3D environment will improve their spatial
thinking, specifically improving their ability to draw 3D
objects that have proper proportions relative to the user’s
bodies. Our final goal is to identify opportunities to develop
better user interfaces for immersive 3D drawing based on the
behavioural differences between spatial ability groups.
Effect of spatial ability on immersive 3D drawing.
No previous work analyzes how the user’s spatial abilities
affect immersive 3D drawing. However, based on previous
results on 2D drawing [39] and 3D modelling [9, 14], we Figure 2. a) Physical experimental setup with walking area. b)
hypothesize (H1) that a user’s spatial ability affects the 3D environment of the experiment
175
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
176
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
177
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
Figure 5. Experiment 1 results, 1) total time, 2) drawing time, 3) line straightness, 4) shape likeness, and 5-12) drawing behaviours.
marginally significant effect on line-precision scores
between HSA and LSA participants (F1, 34 = 3.68, p = 0.06).
HSA participants have better line precision than LSA
participants (Figure 5.5).
Movement paths: We analyzed the participant’s head
position over time to see how the participants moved while
drawing. To get this data, we first identified head positions
that were at least 30 cm from the previous position to
eliminate head movements without walking. Then we used a
red-to-blue gradient to visualize which positions were used
Figure 6. Allegory movement paths for standing position from
first and which last (Figure 6). Shades of red represented the
a top view. Green areas are the user standing positions, red-to-
blue gradient dots represent the participant’s movement.
first positions and blue the last. We found that some
participants orbit around part of their drawing before moving
Standing positions while drawing: We identified the to another part, and we called this behaviour rocking
different positions where the participants stood while movement. There is a statistically significant in the number
drawing to identify the number of viewpoints from which of rocking movements between HSA and LSA participants
they saw their drawing. To get this data, we created heat (F1, 34 = 11.45, p = 0.0001). Cohen’s d = 0.59 identifies a
maps of the participants’ head positions while drawing. In large effect size. HSA participants did more rocking
the heat maps, each head position had a weight of 0.1 pts and movements than LSA participants (Figure 5.7).
a circular area of 10 cm radius; see Figure 6. From these heat
maps, we identified three different types of patterns: two We also analyzed the shape-likeness and line-precision
view positions, circular movement paths, and semi-circular results between participants that used rocking and those that
movement paths. The two-view pattern was used only by two did not. There is a statistically significant effect on the shape-
HSA participants for shape 1, so we removed this pattern likeness scores between participant that rocked and those that
from the following analysis. For the remaining 34 drawings, not (F1, 30 = 11.6, p = 0.002). Cohen’s d = 0.57 identifies a
19 follow a circular movement path (Figure 6a), and 15 large effect size. Participants that used rocking achieved
follow a semi-circular path (Figure 6b). better shape-likeness scores than those participants that did
not (Figure 5.9). There is also a statistically significant effect
When analyzing the movement patterns, we found a on line precision between participant that rocked and those
statistically significant effect on shape-likeness scores that not (F1, 30 = 10.58, p = 0.002). Cohen’s d = 0.55 identifies
between HSA and LSA participants (F1, 34 = 11.7, p = a large effect size. Here, participants that used rocking got
0.0001). Cohen’s d = 0.29 identifies a medium effect size. better line-precision scores than those participants that did
Overall, HSA participants got better shape-likeness scores not (Figure 5.8). However, we did not find a difference in
than LSA participants for both movement types (Figure 5.6). terms of line precision or shape likeness based on the
There is also a marginally significant effect on shape likeness participant’s spatial ability, nor an interaction between
between the participant’s spatial ability and movement type spatial ability and rocking.
(F1, 34 = 2.92, p = 0.09). Both participant groups achieved
better shape-likeness scores when using the circular pattern Hand starting position: We analyzed the hand’s starting
than the half-circle pattern, but in both conditions’ HSA position for each stroke to identify whether the users’ spatial
shape-likeness scores were higher. Finally, there is a ability affected their arm movement relative to their head
178
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
position. To get this data, we manually tagged the screen followed the hand. We could only identify a marginally
capture of the session and recorded the percentage of strokes significant interaction between spatial ability and head
that were part of different groups. We found two different movement (F1, 10 = 3.4, p = 0.07). More participants kept their
types of hand starting position, middle and side. For a hand- head static while drawing, but for the participants that moved
starting-position in the middle, participants started the stroke their head, LSA participant moved their head while drawing
near the vertical midline of their bodies and moved their hand more than HSA participants (Figure 5.12).
away from the center. In a side-hand-starting position,
Discussion
participants started the stroke on one side of their body and Shape likeness: Overall, HSA participants draw shapes that
moved their hand across their body. There is a statistical are more like the 3D models than LSA participants. For
significance in the percentage of strokes that were part of example, for Shape 2 the difference is almost 1.9 pts in
each group (F1, 10 = 52.31, p = 0.0001). Cohen’s d = 1.5 ratings between the groups (HSA = 7.7 pts. vs. LSA = 5.8
identifies a large effect size. A majority of the participants pts.). The difference in scores is also visible when just
started a stroke in the center of their body rather than on the considering the walking condition. These findings support
side of their body (Figure 5.10). However, there was neither our hypothesis H1, as participants with high spatial ability
a significant difference between HSA and LSA participants achieved better scores for shape likeness compared to low
nor an interaction between spatial ability and hand position. spatial ability participants. One probable reason behind a
Drawing direction: We analyzed the way each participant difference in drawing scores is the participants drawing
moved their hand when drawing strokes, based on their body experience, however, when analyzing the 2D drawing
position, to identify how frequently participants drew strokes experience between both groups we found that participants
perpendicular to the view plane. Other drawing movements had similar drawing experiences (F1,11 = 0.19, p = N.S.).
might be lateral/vertical in the drawing plane or a mixture of When analyzing the effect of the different types of
both, being diagonal in depth. To analyze drawing directions, movements, we did not find a significant effect on the shape
we manually tagged the screen capture of each session and likeness to the 3D model. The observed lack of impact of
recorded the percentage of strokes that were part of each walking on shape likeness is the opposite of what previous
group. We used the participant’s current view to identify literature on the cognitive effort of walking reports [59].
their drawing direction. We did not use the head position, as Line precision: The overall line precision for both standing
this may not represent the body orientation accurately. We and walking conditions was not statistically different
found three different types of drawing directions: a) lateral, between the HSA and LSA groups. There were significant
b) diagonal, and c) perpendicular. In the lateral drawing low scores for line straightness in the standing condition, but
direction participants moved their hand approximately those can be related to the naturally curved arm movements
parallel to the view plane, in diagonal participants moved of humans, which have been discussed before by Arora et al.
their hand approximately at a 45° angle from the view [2] and are a consequence of the absence of a physical
direction, and in the perpendicular drawing direction surface. One probable reason behind the similar scores is the
participants moved their hand approximately along the view difference in total time between movement types, but we
direction, i.e., perpendicular to the view plane. consider this to be a consequence of the time spent moving
There is a statistically significant difference in the percentage in the walking condition. The statistical difference in
of strokes that belong to each category (F2, 20 = 38.03, p = drawing time means that participants spent less time creating
0.0001). Cohen’s d = 1.1 identifies a large effect size. A strokes while standing. We do not have enough information
Tukey-Kramer post-hoc test showed that the lateral direction to identify a reason for this difference, however we
was more frequently used than diagonal and perpendicular hypothesize that participants could not see the mistakes they
directions (p = 0.0001), for 49% of all drawing movements. were making, as they used only a single viewpoint. Overall,
The diagonal direction was also more frequent than the our results do not support hypothesis H2. Interestingly, when
perpendicular direction (p = <.0001), with 37% of all the only evaluating drawings created when walking, the line
drawing movements. Finally, the perpendicular direction precision was significantly different between spatial ability
was used infrequently, with only 14% of the drawing groups. Notably, we found that the HSA participant’s line
movements (Figure 5.11). We could not identify a difference precision was better than that for the LSA participants. The
in the percentage of strokes that belong to each category HSA participant’s higher (3D) line precision scores while
between HSA and LSA participants, nor an interaction walking confirm the findings of previous work for 2D
between spatial ability and drawing direction category. drawings [39].
Head movement while drawing: We analyzed whether the User’s behaviours: To better understand how a user plans
head followed the hand while doing the stroke or not, to their next stroke, we also analyzed the walking condition of
identify if there are differences in eye-hand coordination our experiment in detail to identify the participants’ drawing
between HSA and LSA participants. To get this data, we behaviours while drawing in 3D. In this condition, the
manually tagged the screen capture of the session and participants were free to walk in any direction while drawing.
recorded the percentage of strokes for which the head First, we analyzed the users’ standing positions around the
179
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
drawn object. Most participants followed either a circular or The last two measures we evaluated were hand start
a semi-circular movement pattern. Although there is only a positions for a stroke and head movement while drawing. For
marginally significant difference between these patterns hand position, there was a significant difference on the
regarding outcomes, there seems to be a tendency where a percentage of stroke that started at each position between the
user’s movement pattern and spatial ability affects the shape- categories (side = 0.23% vs center = 0.76%), but not between
likeness score. For example, HSA participants that followed HSA and LSA participants. For head movement types there
a circular pattern had a slightly higher score (7.9 pts.) than was no difference between spatial ability groups. These
the participants that followed a half-circle pattern (7.7 pts.). results lead us to believe that the participant’s arm movement
This effect is more pronounced for LSA participants (circle and their eye-hand coordination do not strongly correlate
= 6.6 pts. vs half-circle = 5 pts.). In general, the circular with the user’s spatial ability. In conclusion, the participant’s
pattern has a larger number of distinct viewpoints than the movement while drawing and the benefits of different
half-circle pattern. These results support previous work, movement patterns depends on the participant’s spatial
where multiple views of an object helped users understand a ability. These findings support our hypothesis H3. However,
3D object better than with mental rotation [45], which is an we found no effect of a user’s spatial ability on other
important step of planning the next stroke. behavioural methods used for 3D drawing, like drawing
direction, hand starting positions and head movement while
We also analyzed the movement paths while drawing. Here
drawing.
we found that some participants performed the rocking
movement explained before. This rocking movement allows General Discussion: Our first hypothesis was that a user’s
participants to change perspectives continuously before spatial ability affects the shape of their final sketch. We were
making a stroke, which may help them plan that stroke better. able to confirm our H1, as the shape likeness of the drawing
In contrast, for both spatial ability groups, participants with is affected by the user’s spatial ability. However, we did not
low shape-likeness scores remained in the same position find a difference in line precision. These results lead us to
more often than high-score participants. Even if they used believe that the higher cognitive and sensorimotor demands
similar viewpoints, their movements around the drawing are of 3D drawing [57] affect HSA users less than LSA users.
also more chaotic (Figure 7). Using a rocking movement has Our second hypothesis was that the shape likeness and line
a positive effect on the overall likeness of the score (Present precision of users diminishes when they change their
= 7.7 pts. vs. Absent = 5.8 pts.) and on the line precision viewpoint while drawing. However, we found that the user
(Present = 5.7 pts vs Absent = 4.7 pts). line precision increased when walking, and we found no
difference on shape likeness, which does not support
When analyzing the hand movement direction, we found a
hypothesis H2.
difference in the number of strokes each participant did in
each category, with most participants using more lateral and While our results were only measured for geometrical
diagonal drawing movements and fewer perpendicular shapes, we hypothesize that they also hold for more general
movements. These results together with the movement drawings. It is as difficult to draw a straight line in an
patterns and the rocking movements lead us to believe that otherwise free-form drawing as it is to draw one that is part
most participants were mostly performing planar drawings. of a geometric shape. We performed a post hoc power
When they needed to draw in depth, they preferred to move analysis for each parameter, based on the mean, between-
around rather than to draw with hand motion perpendicular groups and within-groups comparison, and the effect size. It
to their view. This is an important finding because it shows showed limited statistical power because of the modest
that participants were actively avoiding perpendicular sample size in the present study (N = 12) for the following
movements, even though the freedom to do this has been results: drawing time (.40), shape likeness (.60), line
claimed as an advantage of 3D drawing [49]. Based on precision (.24) for the walking condition, and shape likeness
previous work we hypothesize that the reasons behind this (.53) for the movement paths analysis. All other statistical
deliberate behaviour are to work around depth perception results obtained statistical power at the recommended .80
problems [1, 6, 7, 42] and biomechanical limitations [36]. level.
Based on these mixed results, and our analysis of the
participant’s drawing behaviours, we believe that user’s
spatial ability only affects certain classes of 3D drawing
behaviours and that different classes of 3D drawing
behaviours affect different parts of the stroke planning
process and the stroke execution process. Those related to
user movement while drawing, i.e., movement pattern and
Figure 7. Movement paths with rocking vs no rocking. The red
movement path, are correlated with shape likeness, as we
shapes show areas where the user stood to draw, and the
yellow lines show walking paths. The blue lines show rocking
found a significant difference between the shape-likeness
movements. scores of participants that use the rocking movement and
those that do not. These behaviours are related to identifying
180
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
an appropriate viewpoint to draw the next stroke. They are Apparatus & Procedure
also related to the user’s spatial ability, as HSA participants The hardware setup was identical to study 1. The 3D
used such behaviours more than LSA participants. Those software was updated to allow users to rotate their sketches.
drawing 3D behaviours related to hand movements and eye- As in the previous experiment, users used their dominant
hand coordination, i.e., hand starting position and head hand to draw the strokes with the freehand drawing
movement, are not correlated to either the line precision or technique. The experiment procedure was identical to study
the shape likeness. These behaviours are related to 1, and the target shapes were Shape 2 and 3 of study 1 (Figure
positioning the hand in the correct place to draw the stroke. 3).
They are also individual to each participant, as the user’s Design
spatial ability did not influence them. Finally, the hand The study used a 3x2x2 mixed design. The within-subjects
movement direction is part of the stroke creation process, not independent variables were the movement type (walking,
the planning phase. But it shows that the users carefully plan OHR, THR) and the drawing shape (2, 3). The between-
their standing position to avoid movements in depth. subjects independent variable was the user’s spatial ability
USER STUDY 2 (low vs. high). In total, we collected 72 drawings, 6 for each
In user study 1, we found that there are specific behaviours participant. Because there were the same number of
that help users select the correct viewpoint to draw the next participants in both ability groups, our design was balanced
stroke. Interestingly, one of our findings contradict previous between factors. The order of conditions across within-
work that changing the viewpoint through walking does not subject dimensions was counter-balanced across participants
impact the shape-likeness scores. This result motivated us to following a Latin-square design. The collected measures
further study the effect of changing the viewpoint while were drawing time (seconds), total time (seconds), the stroke
drawing in VR. Based on this, we hypothesize (H4) that a objects in Unity3D, and the participant’s head and hand
user’s line precision and shape likeness is affected by the position. We also recorded video of the participants and
viewpoint method used while drawing. We expect that created a screen video of the participants’ view while
drawing while walking improves the user’s line precision drawing.
and shape likeness compared with other viewpoint-control Results
methods. Therefore, we evaluated the effect of using two The results were analyzed using repeated measures ANOVA
common hand-based viewpoint control methods currently with α=0.05. All the data were normally distributed, except
used in 3D immersive drawing systems: for drawing time and shape likeness. To normalize that data
Two-hand rotation (THR): We implemented a variation of we used ART [58] before the ANOVA. Statistical results are
the grab the air interaction technique [44], where the user reported in Table 2, where “***” marks results with p <
uses both controllers to grab the world and rotate it. Our 0.001, “**” for p < 0.01, “*” for p < 0.05, “M.S.” for
implementation uses the position between both controllers as marginally significant, and “N.S.” for not significant. Figure
the rotation pivot. As we are comparing this method with 8 shows some of the resulting 3D drawings done by our
walking, we only allow users to do a 1DOF rotation around participants.
the world up vector. Our technique also allowed for a 3DOF Total Time: There was no significant difference between
translation of the object. spatial ability groups or movement types for total time.
One-hand rotation (OHR): We mapped the sketch rotation to Drawing time: There was no significant difference between
the left controller pose and let users rotate it and translate it spatial ability groups or movement types on drawing time.
by moving the controller. Again, we only allowed 1DOF
rotation to emulate walking.
METHODOLOGY
Participants
We recruited 12 participants from the university community
(4 female). 5 of the participants were between 18-20 years
old, 3 were between 21-24 years old, and 4 were between 25-
30 years old. None of the participants took part in study 1.
The participants’ frequency of drawing with pen and paper
was that 4 draw a few times a week, 2 every week, 3 a few
times a month, 1 once a month, and 2 less than once a month.
For drawing in VR, only 2 participants had drawn in VR
before. We measured the participants’ spatial abilities before
they drew as we did in the first study; the participants were
equally divided between high and low spatial ability.
Figure 8. Study 2 results.
181
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
Line precision: We coded each drawing using the same study 1 are a consequence of the positive effects of physical
method as in study 1. There was a significant main effect of moving on spatial updating, which helps users to remain
movement type on the matching of line pairs (F2, 20 = 4.8, p oriented in space [33]. Thus, users can more easily find the
= 0.019), but not between spatial ability groups. Cohen’s d = correct viewpoint to draw their next stroke. Finally, the
0.19 identifies a small effect size. A post-hoc analysis interaction between movement type and spatial ability,
showed a difference between using THR and walking (p = where the movement type affected HSA less than LSA
0.02), but not between OHR and walking or OHR and THR participants, verifies previous work on spatial ability [23,
(Figure 9.1). There was also a significant main effect of 34].
movement type on the stroke’s degree of deviation (F2, 20 =
Line precision: We found that the movement type has a
6.5, p = 0.006), but not for spatial ability. Cohen’s d = 0.31
significant effect on line precision for the matching of line
identifies a medium effect size. A post-hoc analysis
pairs and the stroke degree of deviation. These line precision
identified a difference between using THR and walking (p =
categories are related to positioning the stroke in place,
0.0056), but not between OHR and walking or OHR and
which confirms that depth perception issues affect 3D
THR (Figure 9.2). For the rest of the line-precision
drawing. It also makes us hypothesize that spatial orientation
categories, there was no significant main effect between
also affects line precision. The post hoc analysis shows that
movement types and spatial ability groups.
walking is better than THR. The lack of significance between
Shape likeness: We again coded each drawing using the walking and OHR made us look at the data in more depth.
same method as in study 1. There was a significant main We found that participants used a similar number of scene
effect on shape-likeness scores between LSA and HSA rotations in the OHR and THR conditions (F1, 10 = 0.01, p =
participants (F1, 10 = 6.5, p = 0.02). Cohen’s d = 0.6 identifies N.S.), but when looking at the screen capture videos, the
a large effect size. Shape-likeness scores for HSA OHR rotations seem to have involved smaller angles than the
participants were significantly higher than LSA participants’ THR rotations. Based on this, we speculate that these small
scores (Figure 9.3). There was also a significant main effect OHR rotations might have been less disorienting than the
of movement type on shape likeness (F2, 20 = 32.2, p < 0.001). larger THR rotations. Overall, these results show that for
Cohen’s d = 1.77 identifies a large effect size. A Bonferroni immersive 3D drawing walking is better than hand-based
correction post-hoc analysis identified each movement type view control.
in a different group, where walking was better than THR, and
THR was better than OHR. There was a significant main
effect on shape likeness between movement type and spatial
ability (F2, 20 = 3.6, p = 0.047). For HSA participants a post-
hoc analysis identified a difference between using OHR and
walking (p = 0.0006). For LSA participants, all rotation
methods were statistically significantly different from each
other (Table 2 and Figure 9.3).
SPATIAL ABILITY MOVEMENT MOVEMENT ⨉ Figure 9: Experiment 2 results, 1) matching of line pairs, 2)
SPATIAL ABILITY degree of deviation, and 3) shape likeness.
MEASURE F (1, 10) p F(2, 20) p F (2, 20) p
TOTAL TIME 3.13 N.S. 3.07 M.S. 1.8 M.S. General Discussion: We analyzed the effects of viewpoint
DRAWING TIME 0.63 N.S. 2.34 N.S. 0.016 N.S.
LINE change on stroke planning. We found that users achieved
0.46 N.S. 1.48 N.S. 0.54 N.S.
STRAIGHTNESS better shape likeness and line precision scores when walking
MATCHING OF
LINE PAIRS
2.8 N.S. 4.8 * 2.9 M.S. than when using hand-based viewpoint control. We
DEGREE OF
0.13 N.S. 6.5 ** 1.26 N.S.
hypothesize that this difference is a consequence of better
DEVIATION
CORRECTIVE
spatial updating when walking [33], which help users better
0.13 N.S. 0.24 N.S. 2.2 N.S.
MOVEMENTS orient themselves in space than when using hand-based
SHAPE LIKENESS 6.5 * 32.2 *** 3.56 *
viewpoint control methods. These results validate previous
Table 2. User study 2 statistical results work, where the viewpoint control method has an effect on
Discussion user performance [35, 55], but also extends this work to 3D
Shape likeness: We found a difference between movement immersive drawing. Our results also show the importance of
types, where walking was better than THR, and THR was choosing the correct viewpoint control method, as we found
better than OHR. There were no significant differences for that both line precision and shape likeness are affected by it.
drawing or total time, which shows that the time spent We hypothesize that the reason behind this is related to the
drawing is unlikely to be a cause for this difference. These selection of a wrong viewpoint due to disorientation. Based
results show that the viewpoint control method has an effect on this, the higher degree of spatial orientation of HSA
on user performance. Specifically, it shows that walking is participants compared to LSA participants in study 1 may be
better for keeping the shape likeness of the sketch than hand- the reason for their high line precision scores in the walking
based viewpoint control methods. Based on this, we condition. However, such speculations need to be verified in
hypothesize the similar results of standing and walking in future work. We were also able to confirm the outcomes of
182
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
our study 1 results, i.e. that the shape likeness of the drawing strokes, better, e.g., by encouraging users to do rocking
is affected by the user’s spatial ability, but not line precision. movements between drawing strokes.
This confirms our H1. We were also able to show that hand-
Give users tools to maintain their orientation while changing
based viewpoint control methods affect the user’s line the viewpoint
precision and shape likeness, which confirms our H4. In study 2 we found that one advantage of walking over other
However, we found that the user’s spatial ability can viewpoint control methods is its ability to help users keep
diminish this effect, based on the interaction between spatial themselves oriented in space. This is related to spatial ability,
ability and movement type on shape likeness. In conclusion, as HSA users achieved were less affected by the negative
the results of study 2 further support the findings of study 1 effects of using hand-based control methods than LSA users.
on the user’s behaviours, as selecting the correct viewpoint However, it is not always possible to physically move to the
is not only important for the shape likeness, but also for the correct viewpoint. Therefore, a 3D drawing user interface
sketches’ line precision. Finally, limited statistical power should help LSA users better understand the spatial
because of the modest sample size in the present study (N = relationship between strokes, such as orthogonality between
12) may have played a role in limiting the significance of strokes, by providing strong orientation cues or navigation
some of the statistical comparisons conducted. A post hoc aids that show the probable position of new strokes in
power analysis, based on the mean and the between-groups relation to the current strokes.
and within-groups comparison effect size, revealed that the
following statistical results had a limited statistical power: Give users tools to draw in depth
the matching of the line pairs (0.25), degree of deviation In study 1, we found that most users preferred lateral hand
(0.58), line precision (0.51) and shape likeness (0.50). Other movements over depth movements. However, using only
results obtained a statistical power at the recommended .80 lateral hand movements negates one of the advantages of
level. drawing in 3D, the ability to draw in depth. Therefore, we
suggest giving users tools to improve depth perception to
USER INTERFACES TO SUPPORT IMMERSIVE 3D encourage depth hand movements. Another advantage of
DRAWING
providing extra tools to better perceive depth is that it may
Based on the results of our two studies, we present several help users identify the correct hand position in space, which
recommendations for 3D immersive drawing interfaces: may improve line precision as found in study 2. Previous
Encourage users to change their viewpoint. work [2] suggested the use of planar surfaces to avoid
Moving is critical for 3D drawing, since it allows users to problems related to depth perception and motions while
view their drawing from different viewpoints and better plan drawing in VR. We complement their advice, by suggesting
their next stroke. In study 1 we identified that walking avoids the use of widgets and visual guides inside the virtual
the creation of accidentally curved strokes regardless of user environment to provide extra depth cues.
spatial ability, which Arora et al. [2] identified to be a
CONCLUSION
problem with drawing when standing. In study 2 we This paper quantifies the effect of spatial ability on 3D
identified that the viewpoint control method affects the shape drawing. Our findings show that the user’s spatial ability
likeness, because when users changed their view using hand- affects the shape likeness of their sketches but not their line
based viewpoint control methods their shape-likeness scores precision, as high-spatial-ability users achieve better shape-
were lower than when they walked around their drawing. likeness scores than low-spatial-ability users. This is
This is related to spatial ability, as HSA users achieved better particularly interesting since previous literature [14, 39] has
shape-likeness scores than LSA users. Therefore, a 3D not identified such an effect. More importantly, we found
drawing user interface needs to encourage users to walk different types of user behaviours while drawing in 3D in
around their drawings to make it easier to identify the “real” study 1. Those related to the shape likeness are about
stroke shape in 3D. identifying the correct viewpoint to draw the next stroke.
Help users identify the spatial relationship between strokes Other behaviours are related to line precision and help users
Previous work [30] found that repositioning the view correctly position their hand in space to start a stroke. In
increases 3D understanding of the shape. Study 1 and 2 study 2, we found that choosing the correct viewpoint also
complement these results by showing that understanding the has a positive effect on line precision. In future work, we plan
global shape of the sketch is related to systematically moving to explore the effect of disorientation on line precision and
around the drawing while maintaining a good spatial shape likeness further.
orientation. More importantly, we identified that regardless
ACKNOWLEDGMENTS
of the user’s spatial ability, using rocking movements around This research was sponsored in part by Adobe Research,
the drawing improved their shape-likeness scores, as such CONACYT and NSERC.
rocking helps to perceive the correct spatial relation of a new
stroke relative to existing content. Therefore, a 3D drawing REFERENCES
user interface should help users understand the spatial [1] Armbrüster, C. et al. 2008. Depth Perception in
relationship between strokes, such as orthogonality between Virtual Reality: Distance Estimations in Peri- and
Extrapersonal Space. CyberPsychology &
183
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
184
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
design—Results from expert discussions and user Sketch-Based Interfaces and Modeling (SBIM ’09),
studies. Computers & Graphics. 33, 4: 462–473. 69. https://doi.org/10.1145/1572741.1572754.
https://doi.org/10.1016/j.cag.2009.05.005. [38] Oculus 2016. Rift.
[27] Jackson, B. and Keefe, D.F. 2016. Lift-Off: Using [39] Orde, B.J. 1997. Drawing as Visual-Perceptual and
Reference Imagery and Freehand Sketching to Spatial Ability Training. Proceedings of Selected
Create 3D Models in VR. IEEE Transactions on Research and Development Presentations at the
Visualization and Computer Graphics. 22, 4: 1442– 1997 National Convention of the Association for
1451. https://doi.org/10.1109/TVCG.2016.2518099. Educational Communications and Technology.
[28] Jin, Y. and Chusilp, P. 2006. Study of mental [40] Perkunder, H. et al. 2010. Shape Modeling with
iteration in different design situations. Design Sketched Feature Lines in Immersive 3D
Studies. 27, 1: 25–55. Environments. Proceedings of the Eurographics
https://doi.org/10.1016/j.destud.2005.06.003. Symposium on Sketch-Based Interfaces and
[29] Keefe, D.F. et al. 2001. CavePainting: A Fully Modeling (Expressive’10), 127–134.
Immersive 3D Artistic Medium and Interactive [41] Pittalis, M. and Christou, C. 2010. Types of
Experience. Proceedings of the ACM Symposium on reasoning in 3D geometry thinking and their
Interactive 3D Graphics (I3D '01), 85–93. relation with spatial ability. Educational Studies in
https://doi.org/http://doi.acm.org/10.1145/364338.3 Mathematics. 75, 2: 191–212.
64370. https://doi.org/10.1007/s10649-010-9251-8.
[30] Keefe, D.F. et al. 2007. Drawing on Air: Input [42] Renner, R.S. et al. 2013. The perception of
Techniques for Controlled 3D Line Illustration. egocentric distances in virtual environments - A
IEEE Transactions on Visualization and Computer review. ACM Computing Surveys. 46, 2: 1–40.
Graphics. 13, 5: 1067–1081. https://doi.org/10.1145/2543581.2543590.
https://doi.org/10.1109/TVCG.2007.1060.
[43] Riecke, B.E. et al. 2007. Spatial Updating in Virtual
[31] Kenyon, R. V. and Ellis, S.R. 2014. Virtual Reality Reality: The Sufficiency of Visual Information.
for Physical and Motor Rehabilitation. Springer Psychological Research. 71, 3: 298–313.
New York. https://doi.org/10.1007/978-1-4939- https://doi.org/10.1007/s00426-006-0085-z.
0968-1.
[44] Robinett, W. and Holloway, R. 1992.
[32] Khan, A. et al. 2008. ViewCube. Proceedings of the Implementation of flying, scaling and grabbing in
2008 symposium on Interactive 3D graphics and virtual worlds. Proceedings of the 1992 symposium
games (SI3D ’08), 17. on Interactive 3D graphics (SI3D ’92), 189–192.
https://doi.org/10.1145/1342250. 1342253. https://doi.org/ 10.1145/147156.147201.
[33] Klatzky, R.L. et al. 1998. Spatial Updating of Self- [45] Rock, I. et al. 1989. Can we imagine how objects
Position and Orientation During Real, Imagined, look from other viewpoints? Cognitive Psychology.
and Virtual Locomotion. Psychological Science. 9, 21, 2: 185–210. https://doi.org/10.1016/0010-
4: 293–298. https://doi.org/10.1111/1467- 0285(89)90007-8.
9280.00058.
[46] Samsudin, K. et al. 2016. Training in Mental
[34] Kozhevnikov, M. and Hegarty, M. 2001. A Rotation and Spatial Visualization and Its Impact
dissociation between object manipulation spatial on Orthographic Drawing Performance. Journal of
ability and spatial orientation ability. Memory and Educational Technology and Society. 14, 1:179-
Cognition. 29, 5: 745–756. 186.
https://doi.org/10.3758/BF03200477.
[47] Shelton, A.L.M.Y.L. and McNamara, T.P. 1997.
[35] Lages, W.S. and Bowman, D.A. 2018. Move the Multiple views of spatial memory. Psychonomic
Object or Move Myself? Walking vs. Manipulation Bulletin & Review. 4, 1: 102–106.
for the Examination of 3D Scientific Data. https://doi.org/10.3758/BF03210780.
Frontiers in ICT. 5: 1–12.
https://doi.org/10.3389/fict.2018.00015. [48] Shepard, R.N. and Metzler, J. 1971. Mental
Rotation of Three-Dimensional Objects. Science.
[36] McDonald, A. et al. 2012. Spatial dependency of 171, 3972: 701–703.
shoulder muscle demands in horizontal pushing and https://doi.org/10.1126/science.171.3972.701.
pulling. Applied Ergonomics. 43, 6: 971–978.
https://doi.org/10.1016/j.apergo.2012.01.005. [49] Tano, S. et al. 2013. Truly useful 3D drawing
system for professional designer by “life-sized and
[37] Murugappan, S. et al. 2009. Towards beautification operable” feature and new interaction. Lecture
of freehand sketches using suggestions. Notes in Computer Science (including subseries
Proceedings of the 6th Eurographics Symposium on Lecture Notes in Artificial Intelligence and Lecture
185
Session 4: Images + Art C&C ’19 June 23–26, 2019, San Diego, CA, USA
Notes in Bioinformatics). 8117 LNCS, PART 1: [55] Usoh, M. et al. 1999. Walking -> Walking-in-Place
37–55. https://doi.org/10.1007/978-3-642-40483- -> Flying, in Virtual Environments. Proceedings of
2_3. the Conference on Computer Graphics and
[50] Tarr, M.J. et al. 1998. Three-dimensional object Interactive Techniques (SIGGRAPH ’99), 359–364.
recognition is viewpoint dependent. Nature https://doi.org/ 10.1145/311535.311589.
Neuroscience. 1, 4: 275–277. [56] Wesche, G. and Seidel, H.-P. 2001. FreeDrawer: A
https://doi.org/10.1038/1089. Free-Form Sketching System on the Responsive
[51] Tchalenko, J. 2009. Segmentation and accuracy in Workbench. Proceedings of the ACM Symposium
copying and drawing: Experts and beginners. Vision on Virtual Reality Software and Technology (VRST
Research. 49, 8: 791–800. ’01), 167. https://doi.org/10.1145/ 505008.505041.
https://doi.org/10.1016/j.visres.2009.02.012. [57] Wiese, E. et al. 2010. Investigating the learnability
[52] Teramoto, W. and Riecke, B.E. 2010. Dynamic of immersive free-hand sketching. Proceedings of
visual information facilitates object recognition the Seventh Sketch-Based Interfaces and Modeling
from novel viewpoints. Journal of Vision. 10, 13: Symposium (SBIM’10), 135–142.
1–13. https://doi.org/10.1167/10.13.11.Introduction. [58] Wobbrock, J.O. et al. 2011. The aligned rank
[53] Tramper, J.J. and Gielen, S. 2011. Visuomotor transform for nonparametric factorial analyses
coordination is different for different directions in using only anova procedures. Proceedings of the
three-dimensional space. The Journal of SIGCHI Conference on Human Factors in
Neuroscience. 31, 21: 7857–7866. Computing Systems (CHI ’11), 143.
https://doi.org/10.1523/JNEUROSCI.0486-11.2011. https://doi.org/10.1145/1978942.1978963.
[54] Ullman, D.G. et al. 1990. The importance of [59] Yogev-Seligmann, G. et al. 2008. The role of
drawing in the mechanical design process. executive function and attention in gait. Movement
Computers & Graphics. 14, 2: 263–274. Disorders. 23, 3: 329–342.
https://doi.org/ 10.1016/0097-8493(90)90037-X. https://doi.org/10.1002/mds.21720.
186