Chat Gpt-4

vs

researcher
6 deliverables

Swipe
We’re Going to Compare:

1 UX persona

2 User interview questions

3 Analyzing interviews

4 Competitor research

5 Requirement gathering & UX plan

6 MVP Prioritization

Repost
People wonder whether ChatGPT will

take their job.

Let’s see how AI does with

UX research.

To draw the conclusions, we

compared multiple UX deliverables

our team have created (for different

projects) with what ChatGPT-4 did.

Here’s what I learned --->

Repost
1. UX persona
UX persona is a set of information
and insights about a representative
of the userbase, who is the probable
main type of user (main persona)
and few others if needed.

Purpose of a persona sheet is to

incite empathy and perspective for
the reader (usually a UX designer).
Here’s the comparison --->

Repost
1. UX persona

by Researcher
Still viable options for She doesn’t come in
background contact with students
H directly, at all even

Skills dep
Engineering+managem They mostly talk to
en managers
Training experience

There is some level of

“laziness” about the
They speak in KPI
They like oversight
over people

(High priority)
Easy tools
integration
API
integration

Repost
1. UX persona
by ChatGPT
Name: Samantha Green

Age: 42

Occupation: Training Director

Location: Chicago, IL

Bio: Samantha Green is a highly experienced Training Director who has been working in
the corporate training and development field for the past 15 years. She is responsible
for planning, implementing, and evaluating employee training programs in her
organization. Samantha is passionate about helping employees improve their skills and
reach their full potential.

Goals
Design and implement effective training programs that cater to diverse learning
styles and skill levels
Continuously evaluate the success and impact of training initiatives to ensure they
meet business objectives and employee needs
Utilize technology and innovative methods to deliver engaging, efficient, and cost-
effective training solutions.

Challenges
Balancing the need for customized training solutions with budget constraints and
time limitations
Ensuring the training material remains up-to-date, relevant, and engaging for
employees
Encouraging employee participation and engagement in training programs.

Repost
1. UX persona
by ChatGPT
Technology Use:

Samantha is comfortable using technology in her work and personal life. She uses a
mix of desktop and mobile devices to manage her daily tasks, including email, calendar
management, project management, and online research. She is open to exploring new
technologies and tools that can enhance her training programs and improve employee
learning experiences.

Quote:

"Helping employees grow and develop is what drives me. I believe that the right
training can unlock an individual's potential and contribute to the overall success of the
organization."

UX Preferences
Intuitive and easy-to-use software with a clean and straightforward interface
Access to a comprehensive library of training materials and resources that can be
easily customized to fit specific needs
The ability to track and measure the success of training initiatives through analytics
and reporting tools
Integration with other tools and platforms that she uses for project management
and collaboration
Support for various training delivery methods, including in-person, online, and
blended learning options.

Repost
1. UX persona
by ChatGPT
Need: Efficient task and project organization. 

Monday.com Feature: Boards and Workspaces - Project managers can create and
organize tasks within boards and workspaces, allowing them to visualize projects
and keep track of progress easily
Need: Seamless team collaboration. 

Monday.com Feature: In-app communication and mentions - Project managers can
communicate with their team members directly within tasks or updates, using
mentions to ensure everyone stays on the same page
Need: Clear assignment of responsibilities. 

Monday.com Feature: Assigning tasks to team members - Project managers can
easily assign tasks to specific team members, ensuring everyone knows their
responsibilities and deadlines
Need: Effective time management and scheduling. 

Monday.com Feature: Timeline and calendar views - Project managers can use
timeline and calendar views to visualize project schedules and deadlines, helping
them manage time and resources efficiently
Need: Tracking project progress and performance. 

Monday.com Feature: Dashboards and reporting - Project managers can create
custom dashboards to track project progress, performance metrics, and team
workload, giving them a high-level overview of their projects' status
Need: Flexibility and customization. 

Monday.com Feature: Customizable columns and automations - Project managers
can tailor their boards with various column types, like status, text, numbers, and
more, while automations help automate repetitive processes, saving time and effort
Need: Integration with other tools and platforms. 

Monday.com Feature: Integrations - Project managers can connect Monday.com
with other tools they use, such as Slack, Google Drive, or Microsoft Teams,
streamlining their workflow and ensuring seamless data flow across different Repost
platforms.
1. UX persona

Optimal prompts
How is a day in the life of a {UX
Persona}?
What are the main challenges 

a {UX Persona} encounters on
their day to day work?
What are their main skills
What are their main motivations?
Create a UX Persona for a {UX
Persona}
Write a short bio for {UX Persona}.
Repost
1. UX persona

Researcher
It takes us anywhere between 1-2 weeks
to create a baseline persona (the first
takeaways can be created in 1-2 hours of
a workshop session
With every new finding, the persona files
gets update
The end-result is high-qualit
Uses deeper understanding of client and
userbase context

Repost
1. UX persona
ChatGPT
Only takes about 5-10 minute
The answers typically match the initial
findings and early stakeholder interviews
by about 75
Cannot reliably connect user needs and
how they are served by a produc
Cannot prioritize needs for particular
product strategy (lack of client's vision
Sharing sensitive company information
with external system would require great
security if persona generation could be
improved

Repost
1. UX persona

The Verdict:

Somewhat Viable

ChatGPT does ok with creating a

baseline persona.

It can quickly create short bios that

can boost empathy towards users,

improving accuracy of design work.

From the second week onward, the

persona file a researcher creates is a

lot more useful and in-depth.

Repost
2. User interview
questions
User interview questions are used
for gathering information about
needs, wants, pains and
expectations of users, or potential
users of a given service/product.

Here’s the comparison --->

Repost
2. User interview questions

by Researcher
(excerpt
Could you comment on your overall experience with
RETROFY

What are you missing in this solution? Why is that

important for you

Could you show us how you use this tool and what
features or parts you use the most?

Could you comment on why you use those features

the most?

What challenges have you encountered while using

the tool? Why are those challenges essential for you

What are those things this tool does well or very well
according to you?. Why are those important

What value brings this tool for you personally

What about what value brings to your team?

Repost
2. User interview questions

by ChatGPT
Create user interview questions for the following goals:



- Understand what are the user’s challenges around

retrospectives (i.e., cause discomfort, lack of engagement,
annoy them) 

- Understand what are retrospectives’ success metrics for
each user persona involved

- Explore user expectations before, during, and after
joining the retrospective.

- Understand the user’s perception of RETROFY solution

- Identify challenges/pain points reported by RETROFY
users (if any)

Categorize the questions accordingly.

Repost
2. User interview questions

by ChatGPT

Notice we already defined goals and

objectives of the interview before

writing the prompt.

Defining those objectives is crucial

to success of a any research project.

(see the result) --->

Repost
2. User interview questions

by ChatGPT

Goal: Understand the user's perception of RETROFY solutio

Initial Impressions: a. What were your initial thoughts when you first started using

RETROFY? b. How did your perception of RETROFY change as you continued to use

it

Features and Functionality:

Which features of RETROFY do you find most useful

Are there any features you find lacking or that could be improve

How does RETROFY compare to other retrospective tools you have used in the

past

User Interface and Usability

How easy is it for you to navigate and use RETROFY's interface

Is there anything about the design or layout that you find particularly helpful or

hindering

Have you encountered any difficulties or frustrations while using RETROFY

Collaboration and Team Dynamics

How has RETROFY affected your team's communication and collaboration during

retrospectives

Do you feel that RETROFY has made it easier for team members to share their

thoughts and opinions openly

Has RETROFY had any impact on your team's ability to address conflicts and

reach consensus

Overall Satisfaction

On a scale of 1-10, how satisfied are you with RETROFY as a retrospective tool

What do you like most about RETROFY, and what areas do you think need

improvement

Would you recommend RETROFY to other teams or colleagues? Why or why not?

Repost
2. User interview questions

UX Researcher
Can adapt and change question progression during
interview
Can use social skills to build rapport and trust
Can apply more complex frameworks for gathering
insights
ChatGPT
Can define what questions to ask within given
domain
Cannot detect which questions are possible to be
answered (eg. company secrets)
Cannot adjust questions for client’s context (priority
and sensitivity)

Repost
2. User interview questions

The Verdict:
Somewhat Viable

ChatGPT can give you an outline of

many important questions within
given industry and role of the
participant. However it falls short in
formulation of said questions,
defining which questions participant
might be permitted (or not!) to
answer, as well as overall context
and style of questioning (eg. due to
low UX maturity).

Repost
3. Analyzing interviews
Collecting data is one thing, but
analyzing it and finding insights is
another.

Research involves transforming data

into actionable insights.

Here’s the comparison --->

Repost
3. Analyzing interviews
by Researcher
Pains

-> Team friction

“Aon the team kind of felt he was unreliable. And a lot of that had
been coming to me directly as the manager and kind of putting me
as the middleman between the other people on the team and this
individual.”  - D., Expert Facilitator

->Items importance

“I would say that the the team leads probably take a higher
percentage of the action items, maybe they delegate it afterwards.
But just to give it the importance that it needs.” J., Leader/
Participant

“how do you prioritize the action items that come out of the retro?
Definitely, that's a challenging piece that just has to be taped. I've
rarely seen it that you just have to find time to do it… People need to
to realize that you're doing that so that more regular work can be
done - improve efficiency & quality - which will make it so you spend
less time fixing bugs later down the line. Right? t's all about the
opportunity cost of time.” J., Leader/Participant

Repost
3. Analyzing interviews

by ChatGPT
What are the top 5 UX insights that can be applied to a digital product

designed to address those challenges:

(Interview transcript)

(see the result) --->

Repost
3. Analyzing interviews

by ChatGPT
Facilitate open communication: Create features that encourage participants

to share their thoughts, concerns, and ideas openly. This may include

anonymous feedback mechanisms, structured discussion formats, or easy-

to-use commenting and collaboration tools

Conflict resolution support: Provide features that help teams address and

resolve conflicts effectively. This could include dedicated spaces for

discussing sensitive issues, guidelines for constructive feedback, or even

integrating third-party conflict resolution resources or experts

Prioritization and delegation tools: Implement features that help teams

prioritize and delegate action items. This could include visual aids like drag-

and-drop boards, built-in prioritization methods (e.g., MoSCoW or ICE

scoring), and easy assignment of tasks to specific team members with

deadlines

Progress tracking and follow-up: Design the digital product to enable easy

tracking of action items and their progress. This could include visual

representations like Kanban boards, reminders or notifications for upcoming

deadlines, and the ability to mark tasks as completed, in progress, or

blocked

Asynchronous collaboration support: Recognize the needs of remote and

distributed teams by providing tools that support asynchronous

collaboration. This could include allowing users to leave comments, submit

feedback, or contribute to discussions at their convenience, and

summarizing key points for later review during synchronous meetings.

Repost
3. Analyzing interviews

UX Researcher
Can analyze method of delivery and emotional state
of the participant
Can use previous experience from social
interactions to detect red flags, patterns of behavior
Can apply more complex frameworks for analysis
(including perspective-taking and empathy)

ChatGPT
It cannot detect or focus potential lies and self-
deception
Cannot use empathy and previous communication
as basis for analysis
Cannot reliably adjust for multiple facets of the
client’s context at the same time (UX matuirty,
emotional state, organization culture etc.)

Repost
3. Analyzing interviews

The Verdict:

Totally Unviable

Despite its best efforts, AI, as of

now, can’t replicate empathy and
perspective-taking in a way that
allows a thorough analysis that can
at least attempt to go past what is
overtly seen in the results.

It is far too easy to be fooled, either

by perceptions and assumptions of
others, or its own shortcomings.

Repost
4. Competitor research
Competitor research allows an in-
depth look at the landscape of other
companies that offer similar
products. It is a method of learning
about established practices, design
patterns and current trends in the
given industry.

Here’s the comparison --->

Repost
4. Competitor research

by Researcher

Repost
4. Competitor research

by ChatGPT
Create a list of features of chess.com from UX perspective

Here are some features of chess.com from a user experience (UX) perspective
Easy Navigation: Chess.com has a clean and easy-to-use interface with
clear navigation options, making it easy for users to find what they need
User Profile: Users can create a detailed profile on Chess.com, including
their ratings, game history, and personal information. The profile can also
be customized with a profile picture and background
Game Modes: Chess.com offers various game modes, including live chess,
turn-based chess, and puzzle rush. Users can easily switch between game
modes with just a few clicks
Tutorials and Learning Resources: Chess.com provides a wide range of
tutorials and learning resources for users to improve their skills, including
training exercises, videos, and articles
Computer Analysis: Chess.com offers a computer analysis feature that
allows users to analyze their games and learn from their mistakes. The
feature provides in-depth analysis of each move, along with alternative
moves and suggestions
Community Features: Chess.com has a large and active community of
players from all around the world. Users can interact with each other
through private messaging, chat rooms, and forums....

Repost
4. Competitor research

UX Researcher
It typically takes about a week for a first
competitor draft
Can use their previous industry
experience in assessing viability of
competitor’s features
The file focuses both on visual and
functional aspects of competition
ChatGPT
With external help, not more than 5 min
Saves time on comparative analysis of all
available features

Repost
4. Competitor research

Optimal prompts
Use the prompts in this order
Define industry and type of produc
Create a list of competitors of said produc
Create a list of features for said produc
Create a table with comparison of features
of competitors of said product

Repost
4. Competitor research

The verdict:

Mostly Viable

If you can help ChatGPT standardize

feature names, it helps to map which

features competitors have that you

don’t and vice versa.

Repost
5. Requirement gathering
& UX plan
It is a preliminary set of activities
that are aimed at creating a list of
priorities, objectives and tasks that
need to be taken into account during
the course of a UX project.

Here’s the comparison --->

Repost
5. Requirement gathering

by Researcher

Repost
5. Requirement gathering
by ChatGPT

Repost
5. Requirement gathering

UX Researcher
It typically takes about a week or two
depending on the project complexity
Requires interviewing stakeholders and
eliciting acceptance criteria
ChatGPT
Takes a minute
Cannot incorporate USP and other
specifics into a planning
Does not operate on customer-specific
information to prioritize
It is prone to errors and forgetting to share
specific information
Repost
5. Requirement gathering

The Verdict:


Totally Unviable

We didn’t give you prompts, because

no matter what prompts you use, the

results (such as rough estimates and

acceptance criteria) are unreliable.

Repost
6. MVP Prioritization
MVP feature prioritization helps
define in what order they should be
designed, what are the underlying
variables affecting this progression
and how to optimize design process.

Here’s the comparison --->

Repost
6. MVP Prioritization

by Researcher
Items Desirabi Feasibility Viability Total

5 8 8 21

8 7 9.5 24.5
5 5 5 15
8 8 9.5 25.5
8 7 7 22
8 7 9.5 24.5

5 7 7 19
Completion can take few
months and take a lot of 5 7 7 19
resources

5 7 7 19
5 7 7 19
5 7 7 19
Business plan is dynamic/ 6.5? 7 10 23.5
LIVE (unique value prop)

Business Plan
9.5 7 10 26.5

Funding timeline
9.5 5 9.5 24
Visualizations 7 8 7 22
Goes online after business Management presentation 9.5 7 10 26.5
plan is completed
9 5 9.5 23.5

10 5 10 25

Ultimate goal for users to

come to the platform

Repost
6. MVP Prioritization
by ChatGPT

Repost
6. MVP Prioritization

UX Researcher
Can include client context and various points of view
Can recommend best prioritization method for the
given case
Can provide very flexible timeframe adjustments for
different project constraints

ChatGPT
Performs only requested method of prioritization
(difficult to adhere to client’s wishes and past
experience)
Attempts to generalize and normalize timeframe
values
Cannot predict time cuts for tasks based on team
composition/skills/client information

Repost
6. MVP Prioritization

The Verdict:

Somewhat Viable

If you have major difficulties making the

decision of where to start, everything seems
very important - the AI can help you apply a
framework of your choice to make
prioritization simpler. If you are short on time,
this can also help in giving you a basic
solution. But what is important, despite its
high efficiency, it won’t incorporate the
essential part of UX process - careful
adjustment for specific client context (be it
organizational, business, marketing or even
interpersonal).

Repost
Disclaimer
This post is a part of our ongoing
attempts to experiment with AI.
We've concluded that it excels at
comparison and repetition but is
lacking the underlying empathy that
is needed for fully applying UX
processes
Quality aside, ChatGPT-4 does have
other practical issues such as lack of
source citations.

Repost
However, we believe that:

ChatGPT-4 has potential for

secondary research at the early

stages of research efforts.

The tool also shows promise for

summarising information

Ultimately, the tool definitely has

applications for UX professionals:

junior and senior alike, as long as

the users are aware of ChatGPT’s

current flaws.

Repost
Special thanks to:

Adam Fedyniuk

For his crucial role on this research.

Repost
That’s a wrap!

Repost
Don't miss any updates,
Follow and

hit the bell icon.