Research on the Narrative Design of Human-Computer Interaction
in Network Virtual Museum
Yin Jianqiang
Macau University of Science and Technology
Faculty of Humanities and Arts
Macao, China
935167947@qq.com
Abstract
The content of this article is the design of the narrative content of
the network virtual museum based on digital media technology.
The research method of this article is based on the theory of
narratology. It introduces the interactive relationship between
virtual museums and narratology from the perspective of human-
computer interface technology. Through case analysis,
bibliometrics, questionnaire surveys and, other methods, the
Forbidden City is selected. The virtual museum project launched
by the museum is the object of analysis. It discusses the design and
application of narrative thinking and emotional experience in
virtual museums and extends the research on interactive narrative
structures in games and film art. It finds similarities and differences
and chooses the essence of it, which solves the current virtual
museum. The shortcomings of the interactive narrative content,
such as selectivity, unity and, participation, create a new formal
language and provide strong practical support for the interactive
narrative method of the virtual museum.
CCS CONCEPT
Post-epidemic era, Virtual museum; Human- computer interaction,
Scene analysis
Keywords
Post-epidemic era; virtual museum; interactivity; narratology
1. Introduction
2020 must be an unforgettable year. The sudden outbreak of the
new crown epidemic will bring a heavy blow to all industries.[1] To
meet people's spiritual and aesthetic needs during the epidemic,
online virtual museums have developed rapidly. In the
"Encyclopedia Britannica", the definition of a virtual museum is
given: that is, "digitally recorded images, sounds, files and other
data collections of historical, scientific or cultural interest accessed
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for
components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to
post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
ICCIR 2021, June 18–20, 2021, Guangzhou, China
© 2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-9023-1/21/06…$15.00
182
https://doi.org/10.1145/3473714.3473744
through electronic information media. Electronic media. Virtual.
The museum does not contain actual objects, so the permanence
and uniqueness of the museum are lacking in the institutional
definition of the term.[2] "With the continuous advancement of
science and technology, the exhibition methods of virtual museums
are also constantly enriched, especially in the context of the post-
epidemic era, the specific environment, media strategies, and the
integration of major network platforms are closely related.
The interactive narration studied in this article refers to
narrative expressions based on Internet technology using
multimedia interactive functions, such as interactive games in
virtual museums, interactive cultural and creative products,
interactive virtual exhibition halls, interactive audio-visual, etc.
The research specifically focuses on a series of activities such as
"Traveling in the Forbidden City", "Audio-visual Museum", and
"Panoramic Forbidden City" launched by the Palace Museum,
exploring the technical application of narrative thinking in virtual
museums. The expression of interactive narrative often differs
from the previous "linear streamline" in its functional attributes in
the process of use, and the power of use is equal to or even greater
than the power of the operator. At the end of the study, through
case studies in the game and film industries, how to construct
interactive narrative forms for virtual museums under multimedia
technology, and put forward design ideas. In particular, the virtual
museum referred to in this article is not a virtual exhibition hall in
a physical museum, but a virtual space browsed on the mobile
terminal through network technology.
2. Interactive Narrative
2.1. The definition of interactive narrative
The interactive narration has become a hot topic in various fields
such as literature, art, film, games, and virtual reality. With the
continuous development of narratology, interactive narrative can
be roughly divided into interactive text narrative, interactive
electronic media narrative. There are three categories of interactive
digital media narratives. This research focuses on interactive
digital media narratives.
The interactive digital media narrative is a new form developed
from narratology under the background of postmodernism. It
inherits the context of Aristotle’s traditional narrative. It is the
integration and development of electronic media narrative and text
narrative. Its characteristics are diverse., Non-linear, to provide
users with a virtual interactive space that can be participated,
interacted with, communicated, watched, and browsed. Its greatest
significance lies in plot participation and role-playing, giving users
the greatest satisfaction. The contemporary design field has
absorbed and borrowed narrative methods from different levels. In
the interactive design of virtual museums, narrative thinking has
officially developed from the narrative in the early literature field
and the narrative in the electronic media field to the narrative in the
multimedia digital virtual reality field.
2.2. The development and characteristics of
interactive narrative theory
The concept of interactive narrative is that Chris Crawford’s article
"Chris Crawford in Interactive Storytelling" proposed that
interactive narrative is divided into two parts: interaction design
and narrative design.[3] The interactive narrative theory is a new
narrative theory born with the development of multimedia, which
is mainly used to generate new narrative forms in the new era. For
example, the phenomenon of user interaction with the scene
generated by the use of multimedia art in the virtual museum,
which in turn stimulates the user's active participation and sense of
experience.[4]
Traditional narration is a closed fixed space with subjective
conjecture constructed by the creator according to his thoughts. Its
main feature is linear structure narration, and users are passive. In
interactive narratives, creators hand over power to users so that
users can interact with creators, interact with each other, interact
with stories, and even become characters in the story while
participating in direct interaction between characters.
Its characteristics mainly include the following aspects:
Non-linear structure narrative. In the past, the narrative
methods were based on linear narratives, and users could only
passively accept the various results preset by the creator, and could
not have any effect on the results.[5] First, in the non-linear structure
narrative, the user has a certain degree of initiative, from the
predictable results in the linear narrative development to the
unpredictable results, resulting in many possibilities, thus
constructing a "creator-story narration-multimedia media — —
User participation" narrative process.
Narrative based on the concept of interaction. Through the data
integration of text, graphics, and audio-visual data in the virtual
space, diverse information is presented, which causes users to
actively participate in the selection and form manipulation of
narrative text. In terms of content expression, the interactive
narrative is an immersive experience of diverse choices of user
roles, exploration of story content, and interactive design.[6]
electronic reading, from digital animation to film interaction,
digital media technology has made the original one-way
communication route developed into a two-way interaction.
In literary works, we often use rhetorical techniques such as
"borrowing objects to express people" and "borrowing scenes to
express emotions." The same concepts can also be used for
narration in film and television works, architectural design, and
multimedia art. The construction of virtual museums is the same as
literary works and architectural art. The virtual museum based on
digital media technology provides users with a participatory,
interactive, and browsing virtual space. The user is rooted in the
virtual space and explores the route of the story. What the user does
in the process The choices made and the imagination and sentiment
that inspired are all part of the storyline.
3.1. Active participation
Active participation is an inevitable link in interactive narratives.
Creators are conceiving how users can use the interface platform
to enter the virtual museum, design the narrative path from the
user’s perspective, and give more consideration to the diversity that
users can choose. The initiative is placed in the hands of users,
allowing users to actively participate in, and then achieve the ideal
interactive narrative effect.
For example, The" Reconstructing Cloud Classroom" project
launched by the Suzhou Museum's official account. Users use the
online mini program to view the exhibition while choosing the
content of the exhibition according to their interests. The Suzhou
Museum uses the "DIY screen" mini-game, Pull the relationship
between the user and the interface, and actively participate in the
production process. (as shown in Figure 1 below)

3. The relationship between virtual museums

and interactive narration
The interactivity of the virtual museum is the interaction in the
sense of computer science, sociology, psychology, design, and Figure 1: Reinventing the Cloud Classroom (Picture source:
communication-based on digital media technology. As a medium, The author repainted according to the official account of
digital media inherits and develops text communication and Suzhou Museum)
electronic media communication. From books and periodicals to

183
 and "VR Forbidden City" are using interactive narrative thinking
3.2. Immersive experience based on the application of digital media technology in virtual
Interactive narration presents a lifelike virtual environment for museums to attract the public to participate in the interaction.
users, enabling users to experience immersive feelings and The first item: Wandering around the Forbidden City.
cognitive space in a state of immersion. In the form of webcasting, the Palace Museum is cooperating
For example, Google Art Project (GAP) jointly launched by with Xinhuanet, People's Daily, Douyin, and many other
Google and many famous art museums around the world, Goole multimedia networks to simultaneously live broadcast. The live
uses GigaPixel technology to scan the exhibits of major museums broadcast route is divided into three separate explanations. The
in ultra-high definition, with a photo level of up to 7 billion pixels, three online live broadcasts are connected in different forms. Users
and uses Google Street view technology reproduces the internal can choose the route they are interested in. Each route is equipped
scenes of the museum, and viewers can watch 360-degree with a professional commentator, which is very different from the
panoramic videos online, providing users with an immersive way we visit the exhibition. Users follow the ideas of the
experience. (as shown in Figure 2 below) commentator. I found a lot of interesting things about the
Forbidden City that I didn't notice, and I can interact with the
narrator online. The whole explanation process of the narrator from
the beginning to the end is the story. The user interacts with the
narrator through the platform, which can influence or even change
the narrative path. Narrative thinking greatly enhances the user
experience.
For the fitness check-in activity, the Palace Museum and China
Telecom launched the "Traveling in the Forbidden City" fitness
check-in project. The whole process is divided into ten sites
through a virtual path. Each site will have a knowledge question
and answer session and corresponding steps. Counting can get
Figure 2: Google Arts Project (GAP) points and medals as rewards. This form of check-in is like a
(Picture source : https://artsandculture.google.com/) narrative game, which drives users' thinking while feeling cultural
charm. (as shown in Figure 3 below)
4. The application and technical exploration of
interactive narration in virtual museums
In the post-epidemic era, physical museums and virtual museums
together constitute the organic system of the museum industry,
which is continuously integrated and developed. In the past,
whether it was a physical museum or a virtual museum, most of its
exhibits were displayed with an emphasis on the exhibits
themselves, without narrative festivals and immersive experiences,
and mainly displayed items, accompanied by an academic word
explanation, which made people feel boring. Today's museums are
no longer the mode of visiting exhibitions simply through exhibits
or pictures. With the advancement of science and technology,
many new technologies have been applied to museum research
around the world. For example, Augmented Reality (AR), Virtual
Reality (VR), 3D imaging technology, etc. It is precise because of
the continuous development of virtual museums that the trend of
"website-digital/virtual-smart" museums has gradually formed,
laying the foundation for the narrative design and enhancing the
user experience of viewing the exhibition.

4.1. Application of interactive narrative

The new crown epidemic has caused the Palace Museum in Beijing
to be closed from January 25 to May 1, 2020. In this special period,
Figure 3: Wandering around the Forbidden City
to meet the public’s demand for tourism and culture, the Palace
(Picture source: Wandering the Forbidden City Mini
Museum keeps up with the pace of the times and accordingly
Program)
launched the "Wandering Tour". The creators of a series of
activities such as "Forbidden City", "Panoramic Forbidden City"

184
 The second item: the panoramic view of the Forbidden City. A
panoramic tour of the Forbidden City Museum, a virtual Palace
Museum is depicted in detail, using 360-degree photography to
show all the visitable parts of the Forbidden City one by one in the
virtual museum. The user can choose the angle of view, route, and
location in the scene at will. It will be equipped with music and text
commentary, and the exquisite pictures will all make users amazed
and immersed in the scene. (as shown in Figure 4 below)

Figure 6: VR Palace
(Picture source:
https://pano.dpm.org.cn/gugong_app_pc/index.html)

The research uses questionnaire survey methods to summarize

and sort out the post-use evaluations of online virtual museums for
different age groups to understand the impact and needs of virtual
museums on users during the epidemic. The questionnaire was
randomly distributed in a WeChat mini program, including
Figure 4: Panorama Forbidden City classmates, friends, family members, teachers, social people, etc.
(Picture source: After 8 hours of collecting information, 103 valid questionnaires
https://pano.dpm.org.cn/gugong_app_pc/index.html) were finally collected. Among them, 66.02% said that online
virtual museums can meet the needs during the epidemic. (as
The third item: VR Palace. The Palace Museum uses three-
shown in Figure 7 below)
dimensional data visualization technology to highly restore the
Forbidden City complex. Users can select VR glasses mode and
naked eye mode through the interface, deeply analyze each
landscape and exhibit, and open the door to cultural relic
exploration in the virtual world. In addition, a cultural relic
restoration link is also set up. During the interaction with the virtual
characters in the scene, it will tell the user the history, materials
and restoration methods of the cultural relics, and the user can
personally experience the process of restoring the cultural relics in
the scene. This allows users to actively participate in the design,
taking into account the psychological needs of users, which is very
attractive. (As shown in Figures 5 and 6 below) Figure 7: Questionnaire survey on the use of online virtual
museums during the epidemic (Picture source: the author's own
drawing)

Through some virtual exhibition projects launched by the

Palace Museum, the time and space generated by users in the
virtual museum and the narrative festival coincide. The interactive
narrative makes the experience a jumping behavior that transcends
time and space. The audience can choose any node to enter the
narrative, and then meet any node in the narrative. This kind of
jumping thinking is more suitable for the information and memory
of the human brain.[7] Increased the interest of users and met the
exhibition needs during the epidemic.
Figure 5: VR Palace
(Picture source :
https://pano.dpm.org.cn/gugong_app_pc/index.html） 4.2. Exploration of interactive narrative
technology
Based on the above interactive research on the virtual museum of
the Palace Museum, it is found that there are still shortcomings:
Although the live broadcast form of the "Forbidden City" online
platform has driven users' enthusiasm for participation and viewing,
the live broadcast has a time limit. If you do not participate within

185
the specified time, you cannot watch it. Even if there is a recording, and making corresponding judgments based on the needs of the
you cannot interact with the interface. movie plot.
The mode of the "Traveling in the Forbidden City" game is too For example: "Black Mirror: Bandersnatch", released in
simple in the narrative. The user cannot be immersed in the game. December 2018, is an interactive game-type movie with
The rules are completely specified by the creator, and the user lacks experimental challenges. This movie has different settings in each
the right to choose. plot. Option, the movie will continue to be shown according to the
The "Panorama Palace" project has a certain sensory effect, but choice, if you choose to abstain, there will be another ending.
the changes are singular in the course of use. In addition to music Based on high-tech film and television interaction, it brings a
and commentary, it lacks interactive plots, which can easily cause different experience to the audience. The power and form of this
users' visual fatigue. choice will make the audience psychologically change and shift
The overall experience of the "VR Forbidden City" project is their thinking to the plot of the movie. (as shown in Figure 9 below)
good, but the open areas are only three places: Yangxin Hall,
Lingzhaoxuan, and Tiongqinzhai, and users have fewer choices.
However, the most representative application of interactive
narrative is the game and movie industry. Online virtual museums
can innovate and develop interactive narratives in games and
movies, close the relationship with users, and give birth to new
storylines.
In interactive games, different narrative environments
constitute diversified narrative forms. The game has strong
interactive characteristics. Various game types, roles, and plots
have laid the foundation for interactive narrative. Players can
participate in the narrative of the scene by manipulating the
machine to create an individual. Narrative path to complete the
narrative structure.
For example, The sci-fi character game "A Chinese Ghost
Story". This mobile game is very attractive in terms of plot,
appearance, characters, social interaction, etc. Players can create
their real selves in the game by choosing virtual characters. You
can customize your role through your own life experience,
professional skills, personality characteristics, behavior style, age,
hobbies, gender, etc., to create a virtual self, which is strongly
subjective. In addition, in the virtual space, players can
communicate, alliance, and vent their emotions with each other. In
short, one of the most user-friendly games. (as shown in Figure 8
below) Figure 9: Black Mirror: Bandersnatch
(Picture source:
https://baike.baidu.com/tashuo/browse/content?id=90920d34e
84cb5d8f153175e&fr=vipping)

In summary, the interactive narrative technology of games and

Figure 8: A Chinese Ghost Story Game
(Picture source:
http://www.icantw.com/news/news_ican/54834.html)
In interactive movies, the story path is set as a parallel storyline,
and interactive narration gives users a certain amount of initiative,
instead of passively accepting the development of the movie plot,
movies is embedded in the design of virtual museums. Whether it’s
role-playing or visual changes, games and movie spaces are similar
to virtual museum spaces. While interacting with experience, they
interact with the user interface through video narrative expressions.
Within a limited time, various narrative arrangements can be used
to transform the reality of people. Integration of thinking and plot
to establish a new narrative structure. Pay attention to the core
position of "people", design different narrative scenes by studying
people's sensory behavior, psychological changes, and cognitive
activities to achieve a full range of immersive interactive
experiences. According to different choices of users, various
endings are presented to achieve the idealization of interactive
narration.

186
 [8] Wang Zhigang, and Wang Ye. "Gu Sijin Translation: "Everything is animist"
Tsinghua University Cultural Heritage Protection and Innovation Research
Achievements Exhibition Sidelights." Decoration.01(2019):36-43.
5. Conclusion doi:10.16272/j.cnki.cn11- 1392/j.2019.01.009.
Today, interdisciplinary research plays a key role in the
development of innovation. Interactive narratives are involved in
current literature, art, movies, games, virtual reality, and other
fields. The interactive narrative thinking of virtual museums should
be combined with narrative thinking in other fields. Innovation and
development. Virtual museums make up for the shortcomings of
physical museums, break through the dimensions of time and space,
and present a unique exhibition experience and visual enjoyment
during the epidemic. The contextualization, interactivity, and
immersion presented in the virtual space give a dynamic
experience to the static display content of the original physical
museum and meet the diverse needs of users.[8]
With the continuous improvement of science and technology,
virtual museums have a broad development space. The interactive
narrative concept makes the communication of virtual museums
more personalized and targeted, and realizes the effective
dissemination of information; through the active participation of
users, the users themselves are deeply affected. To create a virtual
space belonging to the user, and then change the route of
participation and communication effects; diversified forms of
expression attract more users to participate in the experience,
connect users, and realize the communication intention of the
virtual museum. The development of the museum has gradually
formed a model from physical static-virtual dynamic, passive
viewing-active participation, looking from the air-immersive, from
human-computer interaction to human-computer integration.
In the post-epidemic era, the digital form of museums has
become an inevitable trend for future museum development. The
era of virtual museums means that the Internet carries
corresponding responsibilities. The museum has always been an
educational institution for the collection of human, natural,
tangible, and intangible cultural heritage. How to use the Internet
to tap more advantageous resources, establish mature online
exhibitions and communication methods, adapt to the rapid
development of society, and perfectly unite the online and offline
development model, which is integration A topic worth pondering
in the museum industry.

References
[1] Ge Xiuzhi. "Museum curation in the post-epidemic era." Fine Arts
Research .04(2020):131-133. doi:10.13318/j.cnki.msyj.2020.04.021.
[2] The web version of Encyclopedia Britannica
https://www.britannica.com/topic/virtual-museum
[3] Chris Crawford. Chris Crawford talks about game design. New Riders, 2003.
[4] Zhang Wen, Liu Yuan, and Zhang Wenli. "The Digital Cultural
Communication Design of Jiaoshan Inscriptions Based on Interactive
Narrative Theory." Packaging Engineering 41.16(2020):188-192+204.
doi:10.19554/j.cnki.1001-3563.2020.16.028 .
[5] Sun Lijun, and Liu Yuejun. "The Research and Development of the Third
Generation Film in the Digital Interactive Era." Journal of Beijing Film
Academy .04(2011):45-50. doi:.
[6] Li Yijun. "Human's separation and integration of reality. Thinking of
interactive experience in virtual reality." New Art 37.04(2016):122-127. doi:.
[7] Miao Ling. "Exploration of the Application of Virtual Reality Technology in
the Narrative Design of Museums." Packaging Engineering 39.04(2018):15-
18. doi:10.19554/j.cnki.1001-3563.2018.04.004.

187