I.

Overview of Livestream industry

1.Livestream
According to Wikipedia, Livestream - refers to the live transmission of a
recorded object and its output on another device at the same time . 1To put it simply,
livestream is a live broadcast of everything that is happening now, in real time.
Besides, users can also interact directly with each other through streaming media
provided by the provider.
● Livestream sales
Livestream sales is a marketing and sales strategy that businesses use to introduce
their products or services through live video. During this process, viewers can place
orders and interact directly with businesses through the livestream platform.

2. Market demand

● Livestream market

Since social life was disrupted in 2020 due to the COVID-19 pandemic. Many
businesses and users have turned to online sales to maintain their businesses. And
from there, livestream became popular with people across the country extremely
quickly and became a new shopping habit for many people 2.

According to market research company Adsota, writing in the publication Adapting to

COVID19: Retail and E-commerce, “It seems that livestream sales are truly
becoming a leading marketing trend in terms of realizability. competition and
efficiency in the e-commerce industry" .

According to forecasts by Grand View Research - a consulting and research company

in the US, within 7 years, the value of the livestream sales market is expected to
triple, from $70 billion in 2021. to nearly $224 billion by 2028 . This is a large
growth rate and worthy of expectation. In recent years, big brands have also been
taking advantage of livestream to sell and earn trillions.

● The growth of livestream sales sessions is exploding as shown by the

following statistics 3:
- 82% of viewers prefer livestream videos from a brand over social media posts.

1People who contribute to Wikimedia projects. (2023, October 11). Stream live.
https://en.wikipedia.org/wiki/Ph%C3%A1t_tr%E1%BB%B1c_ti%E1%BA%BFp
2Phuong Thao (2022, October 31). Livestream sales - a new trend of the e-commerce industry. Subiz
Blog. Retrieved November 27, 2023, from https://subiz.com.vn/blog/livestream-ban-hang-xu-huong-
moi-cua-thuong-mai-dien-tu.html
3Trang Vu (2023, January 4). 11 livestream sales brands achieve huge revenue. Subiz Blog.
Retrieved November 27, 2023, from https://subiz.com.vn/blog/livestream-ban-hang-dat-doanh-thu-
khung.html
 - Videos and livestreams directly on the landing page (brand website) can
increase conversion rates by 80% or more.

- 73% of B2B businesses using livestreaming report positive results in their ROI.

- 28% of marketers are investing more in livestream sales: Investing in

livestream early gives your business a chance to gain more market share in the
livestream market.

- Consumers who watch livestream sales are 81% more likely to make a
purchase: The livestream sales model is especially popular in industries like
apparel, fashion and beauty... Seeing an influencer try out products This
product can stimulate shopping demand and make a big difference in
conversions.

- Consumers are 39% more likely to share livestream content because real-time
is more engaging.

- 63% of people aged 18 - 34 regularly watch livestream sessions. If the target

customers are in this age group, applying livestream sales will be very useful.

● Some examples of huge sales increases for brands from livestream sales
sessions

Taking advantage of the advantages of livestream, brands are quickly making it a

part of their marketing strategy, thanks to which they have been reaping amazing
results.

- YODY: YODY is a famous fashion brand that has been covering livestreams
on many platforms such as: Facebook, Shopee,... In 2021, YODY CEO - Mr.
Nguyen Viet Hoa and his family went on air to livestream, respectively.
Collaborate and share about the "Pandemic Season Subsidy" campaign. During
this 90-minute livestream, both sharing and interacting, YODY received more
than 1,000 orders and sales reached over 500 million. These are desirable
numbers for any brand.

- Welden: During the two days of Black Friday, the first time accessing sales via
livestream, Welden's bag sales reached 350 thousand dollars (more than 8.2
billion VND). Now the Welden brand sells 20% of its bags through the
livestream marketplace.

- Tiktoker Vo Ha Linh: she is famous on tiktok with a large number of fans.

Thanks to that, on March 15, 2023, in just 1 hour of livestream, she sold all the
 products of 2 factories and 1 warehouse, with total estimated sales of 20 billion
VND 4.

And many other large and small brands in Vietnam and around the world have been
using livestream as a tool for market expansion, brand coverage, and sales growth.
They invest more money in livestream sessions and hire professional livestreamers.

II. Problems need to be slove

However, in order to conduct effective livestream sessions and interact and support
the right industry and business field, streamers, KOLs/KOCs and businesses
themselves must ensure:

- Product quality: Products need to be introduced clearly and realistically on

livestream. Ensure that all product information is provided accurately and
completely.
- Fast and accurate support: Responding to questions from customers in real
time can create more trust and friendliness. This can increase sales
opportunities and build customer trust.
- Increase the number of livestream sessions: Conducting multiple livestream
sessions a day can help increase your chances of reaching different customers,
while also increasing your chances of brand recognition.
- Customer support spirit: Always maintain a supportive and friendly spirit.
This helps build better relationships with customers and creates engagement
with your brand.

To ensure these factors, businesses must carefully invest in choosing KOCs/KOLs

suitable for their specific industry, to increase efficiency during the livestream
process. Furthermore, responding quickly and interacting flexibly with customers
often requires conducting multiple livestream sessions, while also reaching out and
supporting new customers. However, implementing all of these activities can require a
significant financial investment on the part of the business to maintain continuous and
effective operations. Therefore, these factors directly affect three main issues: Cost,
Human Resources and Time. Fortunately, in the current era of globalization, AI -
Artificial Intelligence is developing very strongly, and using AI to create virtual
servers and virtual characters to perform Livestream sessions with similar tasks. real
people. And they are completely capable of solving three major problems still being
encountered in the livestream sales market today.

4Bang Hao (2023, March 20). Many businesses revived thanks to "order closing livestream" .
Electronic Magazine. Retrieved November 27, 2023, from https://vneconomy.vn/nhieu-doanh-nghiep-
hoi-sinh-nho-livestream-chot-don.htm
III. Competitive advantages

1.Compared with traditional livestreaming

The table below summarizes the USPs (Unique Selling Proposition) - outstanding
advantages of the livestream method using AI compared to the traditional livestream
method with real people.

Livestream using AI Livestream with real people

Optimize marketing costs and control Expensive commission costs, must

revenue better design policies for KOL's fans

Capable of using multiple languages It costs money to invite KOLs from

many different countries

Work continuously without needing to Cannot work continuously with high

rest, eat, or be affected by emotions intensity, need to pay attention to
emotions to avoid scandals

Unlimited human resources, diverse Human resources are dependent on

fields KOC/KOL

1.1. Livestream using AI helps businesses optimize marketing costs and

control revenue better
Hugo Huang - founder of the company providing virtual characters Sansongshuzi said
the cost of a virtual host to run an online marketing campaign is much lower than
hiring a human to do the same job. Huang's company specializes in providing virtual
hosts for about 70 USD/month to appear in short videos or sales livestreams.
Meanwhile, the usual cost to hire a real streamer and studio for livestreaming costs
about 6,900 USD/month. 5Therefore, AI streamer will be a lifesaver for small
businesses that want to expand their brand and media influence but do not have
enough money to invite KOLs.
Besides cost issues, livestreaming with AI helps brands have greater decision-making
power on the number of products and selling prices of products during the livestream.
When a global brand sells products through KOL live streaming, powerful KOLs will
look for discounted prices to attract fans and maintain their fan base. For example, a
KOL on Douyin asked a European brand to offer a 40% discount on products sold
during a livestream. Therefore, it is the KOL, not the brand, who decides the selling
price or quantity during the KOL's live stream. On the contrary, if a global brand

5 China uses AI livestream models to sell products. (2023, July 25). Thanh Nien Magazine.
https://thanhnien.vn/trung-quoc-dung-nguoi-mau-ai-livestream-ban-hang-18523072422550088.htm
decides to use live streaming AI, then AI can be considered a promotional tool and the
brand is the one making all the decisions.
Next is the problem "How to divide the revenue from the livestream? ” In cooperation
with KOLs, brands and KOLs jointly enter into a revenue sharing contract. That is, the
KOL will take a percentage of the revenue from the live stream as a commission fee.
A recent survey shows that KOLs typically charge at least 20% commission on sales
(The Information, 2022). In contrast to MC AI, all proceeds from live streaming
belong to the brand.6

1.2. MC AI can livestream in many different languages

LA-based HeyGen has launched a tool that translates videos into seven different
languages, transcribing the speaker's voice and syncing the speaker's lips so
everything looks natural, and produces consistently good results. surprising way. With
MC AI being able to talk to customers in multiple languages, companies with a
localization strategy combined with global sales can cut the cost of inviting
appropriate KOLs from each country to advertise. promote their products in that
country. All businesses need is their AI avatar, and that persona will communicate
with customers in their native language. Xiaoice and Quantum Planet are now
working together to introduce these MC AIs to customers in China. Their virtual
characters can speak 129 languages, including English and several Southeast Asian
languages, such as Vietnamese, Thai and Indonesian. In March, they used Thai-
speaking MC AI to sell furniture for a Chinese company for the first time and sold
$2,000 worth of products in an hour.7

1.3. MC AI can livestream 24/7 without getting tired

A typical example is Sales Ambassador Diem Hang AI of Vinapharma Group
Pharmaceutical Joint Stock Company. With advanced programming technology, Sales
Ambassador Diem Hang AI is calculated to save 70-90% of costs, time, filming
personnel, no need to appear live, no need to record, no need to worry about payment.
wrong words, no fuss about private images and has the ability to livestream 24/7. In
fact, Sales Ambassador Diem Hang AI has been livestreaming for 7 days and 7 nights
but can still interact smoothly with customers based on scripts and product keywords
without being affected by emotional fluctuations or other factors. physiological and
psychological changes. 8Real streamers operate within a certain time frame. They
cannot operate continuously for such a long time. They will feel tired and

6 Could AI livestream perform better than KOL in cross-border operations? ScienceDirect.

https://www.sciencedirect.com/science/article/abs/pii/S1366554523001187
7 The deepfake avatars who want to sell you everything. MIT Technology Review.
https://www.technologyreview.com/2023/09/20/1079885/chinese-deepfake-livestream-foreign-language/
8 Diem Hang AI - the first AI employee in Vietnam to livestream sales 7 days and 7 nights. (2023, November
8). Tuoi Tre Newspaper. https://tuoitre.vn/diem-hang-ai-nhan-vien-ai-dau-tien-tai-viet-nam-livestream-ban-
hang-7-ngay-7-dem-20231108125256253.htm
uncomfortable when having to maintain emotions and the ability to interact with
customers during that time. long so they need rest or time to do other work.
Businesses that hire live streamers to sell their products may also be exposed to risks
or potential scandals due to the actions and personal lives of real-life streamers. Thus,
using AI as an MC, businesses do not need to care about whether their "employees"
are tired or not, they can let their "employees" go on air at any time without having to
worry about losing performance or not. Exhaustion during intense livestreaming; Or
there will be a group of customers who will boycott the company's products because
they chose the wrong streamer with a bad reputation.

2. Evaluate the advantages of MC AI in optimizing user experience.

2.1. The effectiveness of the livestream using AI virtual characters
MC AI's livestream is almost as effective as a human livestream. Tok Time Digital
Human, driven by AI, can automatically conduct intelligent live streaming throughout
the day, achieving real-time online presence at minimal cost. It has different
interaction modes during live streaming, can recommend products, answer questions,
and give personalized recommendations to customers.9
Another example is Baidu AI Cloud's Xiling platform - an all-in-one 2D digital live
streaming content creation engine for humans built on a large language model. It
combines human digital avatar and content editing capabilities, including avatar
creation, short video production, AI live streaming, intelligent live Q&A, customer
relationship management customers and create content for marketing and livestream
tasks, creating an AI Brand Ambassador who can livestream sales like a real person.
The platform significantly reduces the cost of digital personas and content production,
while improving live streaming efficiency, paving the way for innovative live
streaming scenarios. Thanks to this, during live streams, audiences can interact with
AI Brand Ambassadors in real time through comments, likes and gifts, enriching the
viewer experience.10
In fact, the ADT Global team has successfully created and developed the first AI
Digital Human in Vietnam - AI Ha Vy with full body shaping, from hair, face, eyes...
all very realistic. AI Digital Human Ha Vy impresses with her ability to decide her
own expressions and chat in real-time, communicating fluently in Vietnamese.
Besides, Ha Vy can also proactively create content, self-study and provide answers in
Vietnamese when receiving questions. In particular, Ha Vy once broke many records
in: Speech to text processing speed (Speech to text) of 0.1 second for 50 words; Text
to speech speed is less than 2 seconds for 50 words; Able to recognize context and
9 Jones, E. (2023, July 14). Tok Time Digital Human Live Streaming E-commerce Leading the Global Trend.
OTS News. https://www.otsnews.co.uk/tok-time-digital-human-live-streaming-e-commerce-leading-the-global-
trend/
10 PLTFRM. (2023, June 16). PLTFRM and Baidu AI Cloud Digital Human Center have signed a strategic
partnership to bring AI digital humans to brand promotions and livestreaming. LinkedIn.
https://www.linkedin.com/pulse/pltfrm-baidu-ai-cloud-digital-human-center-have-signed-strategic/
achieve the fastest response speed: 0.1 seconds to respond to 50 contextual words
(average 3-5 characters per word). These are evidence that AI can almost completely
simulate humans, and can replace humans to take on tasks such as customer care, self-
service, brand ambassadors, KOLs, models, etc. MC AI, tour guide AI... even
livestream sales 24/7 on multiple platforms.11
With the feasibility of virtual character building technology and the ability to
customize the character's design to suit the brand's purpose or message, businesses can
easily create virtual streamers according to their needs with low cost.

2.2. Customer interaction and support capabilities

With virtual streamers having a high degree of lifelikeness, i.e. being able to imitate
human voices, speech, expressions and gestures can make consumers believe that they
are real. When consumers feel the virtual streamer is highly animated, they will see it
as interacting with real people who have the ability to build deep connections and
evoke emotions in them, like shopping in the world. authentic, facilitates positive
attitudes and enhances purchase intention (Balakrishnan and Dwivedi, 2021).
MC AI is also focused on its ability to respond to customer questions, such as
providing detailed product information and promotion information, making users feel
like they are being provided with useful information. for purchasing decisions and
appreciate MC AI's interaction quality and service quality.12
1. Customize sales content
This virtual server can perform its livestream work in many different fields by, you
can customize your sales content (from different expressions to different fields).
2. Not through training
For real employees, they must go through the training process to be able to adapt and
perform their jobs and tasks well, especially when moving to another field, which
takes a lot of time to learn and understand. only have the ability to advise customers.
Meanwhile, for the virtual server, you only need to replace the sales content, it has the
ability to learn from that content and respond quickly. In addition, it has the ability to
automatically create sales content according to each field of each user's request

III.Product

- Interactive interface: the virtual server interacts directly with the user through
a specific character, it will imitate the operating status, interactive talking, hand
gestures like a real streamer and Through this interface to consult and answer
customers' questions

11 Vietnam's first AI Digital Human attracts attention. (2023, November 24). Capital Tuoi Tre Newspaper.
https://tuoitrethudo.com.vn/ai-digital-human-dau-tien-cua-viet-nam-gay-chu-y-239105.html
12 How do virtual streamers affect purchase intention in the live streaming context? A presence perspective.
(2023, June 16). ScienceDirect. https://www.sciencedirect.com/science/article/pii/S0969698923001030#sec3
 - Product description and customer support interaction : Responsible for
product description and direct support interaction with customers via
livestream to help them better understand the industry they are doing business
in to make quick purchasing decisions. quickly
- Livestream 24/7 without getting tired: perform continuous livestream
sessions without getting tired, ensuring timeliness and always having customer
support available at any time, helping to increase the conversion rate of
customers as well as Brand recognition and coverage on social networking
platforms.
- Can be applied in many different languages: For our project, the virtual
server can perform descriptions and interact with customers in almost all
popular languages. Then you won't have to worry about hiring many people
from many different countries to conduct this livestream session. At the same
time, foreign businesses can also access and use this technology.
- Customize sales content: For virtual servers, not fixed by any specific field.
For people and businesses that sell different products, they must hire many
different people to ensure their knowledge is appropriate to the business field
of the business. As for virtual servers, as long as you change the content of the
sales field, the virtual server will self-learn and perform the job like a truly
knowledgeable livestreamer.

IV. System review

4.1 Analytical data
Data used in this section includes:
● A doc file contains sales content, which is the foundation for the machine to
learn and give answers based on that content. Content_Tho_food.docx
● An excel file is used for testing and analysis in this section Data_metrics_process
Includes 440 rows and columns:
- Topic: topic of the question
- Comment: question
- answer True: answer contains key word
- answer_system: system answer
- check_answer: The group checks whether the system's answer is correct or not.
If correct, type 1 and incorrect, type 0 in this column
- Time_remove_non_UFT8: time to process and remove special characters in
comments
- Time_teencode_to_VN: time to convert comments containing teencode to
normal comments
- Time_model_NLP: time for model_NLP to process comments to provide an
answer
 - Time_text_to_speech: Time for text_to_speech mode to process the answer to
output the audio file
- Time_rep: Time from the time the system receives that comment to the time it
produces the audio file. It can be said that this is the time to let the system
prepare to answer the question
- Response time = Crawl time + Time_rep + R
In which: R is the remaining time to complete broadcasting a current sales
content.
4.2 Evaluation results

4.2.1 Accuracy

Dashboard of answer accuracy is set by AI based on sales content

● Correct response rate

Based on available sales content, we program it to provide automatic answers every
time someone comments on the livestream post. The image above shows the
accuracy rate of these answers, the accuracy is 79.5%, which can be considered low
when used in practice.
Review and evaluate why the answers are not correct. Give the following main
reasons:
- AI is still quite rigid, does not understand the hidden content inside the
question. For example, when someone asks "How much is dried chicken?" - I
want to ask here about the price of dried chicken. But the answer given is
"Hello, Tho food's beef jerky has a weight of 500gr." , is a description of
product volume. When the question does not have the keyword "price" but
only uses the word "how much" , the artificial intelligence is very likely to give
an incorrect answer. For natural questions, often used in spoken language, AI
also has more difficulty understanding the meaning of the question and giving
focused answers.
- AI also uses other available knowledge to give answers, thereby causing
some answers to be distorted, misunderstood, and answered incorrectly. When
using an external service, here OpenAI's API, to train data, you will easily
encounter this situation.
- Shortcomings in sales content . When writing sales content, users may miss a
few points without paying attention. If someone comments on that content, the
answer will be an apology or creating a wrong answer. In the future, the
software will also be more developed so that sellers can select items in the sales
content and AI will automatically write that sales content without the need for
users to write it manually. This can help reduce content omission rates and
reduce the effort users have to write row content themselves.
Overall, with this model, the accuracy of 79.5% is not satisfactory compared to the
KPI given for accuracy of 93%. The team will continue to standardize the data and
develop the model for greater accuracy, which will be presented later.
● Accuracy rates according to question length

With the pink column chart, the horizontal axis represents the question length, from
1 hour to the longest question of 24 hours. The most common questions are
questions containing 5-11 words, but the accuracy in this part is also nearly the
lowest.
The longer the question, the higher the accuracy because there is more data in the
question to know clearly what the question is. This is shown by the blue line graph.
It can be seen that users often do not tend to comment too long during livestreams,
but this is also a difficulty when using artificial intelligence. The less information in
the question, the more difficult it is for artificial intelligence to give an accurate
answer. Because when using available natural language models, causing confusion
and mixing information is unavoidable. To overcome this, it is necessary to have
more training data of livestream sessions, from which the model can develop and
become familiar with processing data in livestream sales, with common trading
words. day, teen code words of young people,..
● Exact rate according to topic

This part depends a lot on the sales content being trained, the clearer the content,
the more accurate the answer. The easier the content to guess and the fewer
synonyms, the more accurate the answer.
In the ingredients, flavor and origin sections, users often use those exact words in
their questions. This helps reduce confusion and makes it easier to give focused
answers.
In the preservation, volume and description sections, most of the reasons for
discrepancies are due to misunderstanding of the question. As mentioned above,
when using external natural language resources, they will also use their other
knowledge to give answers. Here, when users ask about preservation methods or
volumes and descriptions, they give general answers based on their vast knowledge
base and do not always answer the exact focus of the content. saleman's. For
example, when asked “Is the beef crispy?” - the answer is “Hello, Beef jerky can
be made crispy depending on how it is prepared and how long it is dried. If done
correctly and dried long enough, beef jerky will be crispy and chewy. However, if
not done correctly or not dried enough, beef jerky can become hard and difficult to
chew." , this is the answer for how to prepare beef jerky in general for cooks.
The shipping fee part also has low accuracy because this part is not provided with
enough information in the sales content. This leads to users not being able to give
accurate answers when asked.
 Regarding the accuracy rate of answers according to the question topic, the clearer
and more accurate the sales content will be, the better this rate will be. This can be
considered not a big problem for the project.

4.2.2 Response time

Dashboard of AI response times to trained questions

● Average response time: 8.365s

Based on pre-set question sets, AI can answer questions within 8,365 seconds on
average. This is quite a slow response time, compared to other AI systems. Therefore,
it is extremely necessary to further improve the quality of AI training as well as
improve the question set to help enhance user experience.

- Average time for each component :

According to the chart we can see, the average response time for each
component has a clear difference. Detail:
+ Time_remove_non UFT8: Time to process and remove special
characters in comments is 0.000s. Showing that this result is good, AI
can process and remove special characters in comments very quickly.
+ Time_teencode_to_VN: time to convert comment containing teencode
to normal comment is 0.051s. This result is quite positive, quite good in
less than 1s. However, to enhance the customer experience even better,
as well as to be able to compete well in the market
+ Time Text_to_speech: The time for text_to_speech mode to process the
answer to produce an audio file is 3.171s. This is a pretty good and
quick result. However, continuous monitoring and improvement can
help ensure that the AI system maintains an optimal performance as well
as increasing the customer experience is a priority to make the system
more optimized.
+ Time model NLP: the time for model_NLP to process comments to give
an answer is 5.094s. This is not a very good result and is considered
quite slow because LIVA software is aiming to provide immediate
feedback, in real time, so the time of 5,094 seconds can be considered
slow. Compared to applications such as chatbots or other online help
systems, this can also be considered a disadvantage, as users can expect
a shorter response time. So this is a point that needs to be improved.
+ Time_rep: Time from the time the system receives that comment to the
time it produces the audio file. It can be said that the time it takes for the
 system to prepare to answer the question is 8.365s. To be able to
livestream sales in real time, it takes 8.365 seconds which is not a good
result. Because customers usually want their questions answered quickly
after commenting, because this is also a point where the model needs to
be improved more to bring a good experience to users.

- Box-and-whisker plot analysis of Time model NLP, Time Text_to_speech

and Time_rep:

According to the dashboard above, we can see:

Box-and-whisker plots of Time_model_NLP : with data collected as follows:
Lower Whisker = 1.32; Lower Hinge = 3.15; Median = 4.97; Upper Hinge =
6.75; Upper Whisker = 10.73.
+ The median average is 4.97s. It's also a pretty fast time, but it also needs to be
improved
+ The data tends to skew slightly above the mean.
+ The range of molecules is: 6.75 - 3.15 = 3.6
+ The distance between the largest and smallest value is: 10.73 - 1.32 = 9.41
Box-and-whisker plots of Time Text to speech : with data collected as follows:
Lower Whisker = 1.17; Lower Hinge = 2.29; Median = 3.13; Upper Hinge =
3.82; Upper Whisker = 5.36.
+ The median average is 3.13
+ The data tends slightly above the mean.
+ The range of molecules is: 3.82 - 2.29 = 1.53
+ The distance between the largest and smallest value is: 5.36 - 1.17 = 4.19
Time Rep's box-and-whisker plots : with the following data:
Lower Whisker = 2.75; Lower Hinge = 5.54; Median = 8.37; Upper Hinge =
10.70; Upper Whisker = 18.27.
+ The median average is 8.37
 + The data tends slightly above the mean.
+ The range of molecules is: 10.7 - 5.54 = 5.16
+ The distance between the largest and smallest value is: 18.27 - 2.75 = 15.52

In general, it can be seen that the processing time of Text-to-speech is better than the
Time model NLP. In Time Rep's box-and-whisker plots, there are two outliers, 34.13s
and 35.25s, showing that the chart is not good because it takes a lot of time.

Specifically, the question " Does dried chicken smell like fresh chicken?" ” with the
answer being “ Hello, Dried chicken has the typical aroma of fresh chicken, but the
taste of fresh chicken will not be the same as when the chicken is still alive. The
drying and marinating process will make dried chicken have a rich, attractive and
distinct flavor. ” in the Flavor section takes 34.13s time_rep and the question “ I
want to ask about the origin of the beef jerky the shop is selling?” with the answer "
Hello, Tho food's Raw Beef Jerky is made from native grass-fed beef that grazes
naturally on the hills and mountains of the Central Highlands. The beef is carefully
selected, marinated with rich spices, mildly spicy, and irresistibly delicious. The
product has a certificate of food safety and hygiene, ensuring the health of consumers.
” in the Origin section takes 35.25s time_rep . Shows the need to improve the quality
of questions because these two questions are quite unclear, lost and not in focus
compared to the topic. Response times need to be reduced by optimizing processing
and fine-tuning the NLP model as well as ensuring that the response is formatted so
that it is easy to understand and ensuring that grammar and sentence structure are
clear. . Besides, it is also necessary to consider the length of the answer and consider
whether such a long answer is appropriate for the context.

Not only that, all three charts show that calculating the median score is better than
calculating the average score, and the response time will be faster and more effective.
Because the median is no more affected by two outliers than the average. In box-and-
whisker plots, this makes sense for more efficient evaluation because it helps evaluate
the data distribution without the noise of outliers like 34.13s and 35.25s time_rep.

● Average response time Time_rep by topics:

Based on the chart above, it can be seen

For this part, the effectiveness of response speed also depends heavily on the
quality of the trained sales content. With higher clarity, the answer will be more
accurate and the response will be faster. than. Based on the chart above, it can be
seen that the average response time for each question and topic is different.

The topic with the fastest response time is Volume. We can see that with a total of
50 questions, it takes an average of about 5,987 seconds to answer a question in the
Volume section.

The topic Price - Raw Materials has the second fastest response, we can see that
with a total of 100 questions, equivalent to an average of about 7.67 seconds to be
able to answer a question in the Price - Raw Materials section.

In the topic of Shipping Fees, we can see that with a total of 50 questions,
equivalent to about 8,011 seconds to answer a question in the Shipping Fees
section.

In the Origin topic, we can see that there are a total of 75 questions, equivalent to
about 8,786 seconds to answer a question in the Origin section.

In the Flavor topic, we can see that there are a total of 66 questions, equivalent to
about 9,039 seconds to answer a question in the Flavor section.

In the topic of Preservation, we can see that with a total of 50 questions, equivalent
to about 9,072 seconds to answer a question in the Preservation section.

The topic Description is ranked last because we can see that with a total of 50
questions, it takes about 9,929 seconds to answer a question in the Description
section. This is a relatively long time compared to a real-time livestream software
like LIVA. Therefore, the software needs to improve response time, as well as
review trained questions carefully and thoroughly and prepare better quality
questions to optimize efficiency.

It can be seen that in the topics Flavor, Preservation and Description, the time to
respond to customer questions is much longer than the remaining topics.
Importantly, the quality of the content for each topic affects the response rate. The
higher the clarity of the content, the more accurate and faster the answer will be.
Therefore, it is important to optimize the training process and control the quality of
input data to improve response time. This may include improving question format,
using clearer wording, and ensuring that input is of high quality. Volume topics are
 the topics that receive answers in the fastest time, an average of 5,987 seconds per
question. However, compared to other AI software, this is also not a good time and
needs further improvement to enhance better customer experience. Therefore, it is
necessary to continuously improve to optimize customer experience. Shorter
response times not only provide user comfort, but can also increase customer
satisfaction and trust.

● Average response time Time_rep by Number of words in question:

Based on the chart, it can be concluded that the number of words in the
question does not greatly affect LIVA's response time. Specifically, in the case
of questions with a number of words between 14-16 words, the average
response time is 12,047s, longer than questions with a number of 16-18 words
with an average response time of 9,007s. . Therefore, it can be concluded that
the number of words in the question does not affect the speed of the software
answering questions. Answering speed depends largely on the quality of the
input training questions, not on the length or shortness of the questions.

For the two outliers analyzed above, for the question with time_rep 34.13s
there are 9 words in the question and the question with time_rep 35.25s has 15
words in the question. Also at two fairly high rates in the graph of Time_rep by
word count in the question above.

IV. Result.
Compared to the OKRs that the team came up with from the course for this project:
- The accuracy of the entire system: is 93%, which means the ability to process
and respond to comments, feedback, and questions from customers has an
 accuracy of up to 93%, however during the implementation process And
evaluating the effectiveness, our group got 79.5% correct.
- Time to receive and respond to customer comments: 5 seconds. However,
through evaluation, this system processes comments in nearly 8.4 seconds,
longer than the set goal.
In addition, to increase authenticity and attract more users, we will continue to
develop the interface, adding necessary features in the interface so that users (business
owners) can easily perform on the system. our system. Besides, we also add more
characters and can develop a virtual human model similar to real people (from their
voice, face, to their activities). ) to increase the authenticity as much as possible.
In addition, in order for this system to be deployed to the market, legal constraints
must be required, ensuring that the content in the livestream sessions does not violate
the law or other violations. required of the livestream platform.