ASSESSMENT GUIDE

MARK3088
Product Analytics

Term 2, 2024
Assessment Summary
Assessment Task Weighting Due Date* Learning Outcomes
Assessment 1: Quizzes 30% Weeks 3 & 5 CLO1, CLO3
There are two quizzes during the term, and you will receive automatic feedback/feedforward after
completing the task.
Assessment 2: Case study analysis 40% Week 7 CLO1, CLO2, CLO3
Each student will write a report to investigate a business problem about new product development in the
digital space. The case will enhance students' analytical, qualitative, and strategic decision-making skills as
well as their knowledge of multidisciplinary marketing approaches.

(BCom students: myBcom course points for PLO3)

Assessment 3: Group project 30% Week 10 CLO2, CLO3, CLO4
You will develop a new data product with your group members. This project provides you with an
opportunity to take your knowledge and skills of the product analytics learned in the course and apply them
to a real-world problem.

(BCom students: myBcom course points for PLO4)

* Due dates are set at Australian Eastern Standard/Daylight Time (AEST/AEDT). If you are located in a different time-zone, you can use the time and date converter.

Course Learning Outcomes (CLO)

1. CLO1: Develop new product idea generation employing relevant product and customer data analytics. [PLO 1, 2]
2. CLO2: Create model-based, data-driven product/service decisions using advanced data analytics to provide effective solutions to meet customers' needs.
[PLO 1, 2, 3]
3. CLO3: Combine information and skills to effectively communicate the value of new business data products in oral and written formats in individual and group
settings. [PLO 2, 3, 4]
4. CLO4: Operate collaboratively in a culturally diverse team and contribute towards achieving desired results. [PLO 4, 5]
UNSW Business School 1
Assessment Details
Icon legend

Due Date Weighting Format Length/Duration Submission

Turnitin
Turnitin is an originality checking and plagiarism prevention tool that enables checking of submitted written work for
improper citation or misappropriated content. Each Turnitin assignment is checked against other students' work, the
Internet and key resources selected by your Course Coordinator.
If you are instructed to submit your assessment via Turnitin, you will find the link to the Turnitin submission in your
Moodle course site. You can submit your assessment well before the deadline and use the Similarity Report to
improve your academic writing skills before submitting your final version.
You can find out more information in the Turnitin information site for students.

Late Submissions
The parameters for late submissions are outlined in the UNSW Assessment Implementation Procedure. If you submit
your assessments after the due date, you will incur penalties for late submission unless you have Special
Consideration (see below). Late submission is 5% per day (including weekends), calculated from the marks allocated
to that assessment (not your grade). Assessments will not be accepted more than 5 days late.

Extensions
You are expected to manage your time to meet assessment due dates. If you do require an extension to your
assessment, please make a request as early as possible before the due date via the special consideration portal on
myUNSW (My Student profile > Special Consideration). You can find more information on Special Consideration and
the application process below. Lecturers and tutors do not have the ability to grant extensions.

Special Consideration
Special consideration is the process for assessing the impact of short-term events beyond your control (exceptional
circumstances), on your performance in a specific assessment task.

What are circumstances beyond my control?

These are exceptional circumstances or situations that may:
• Prevent you from completing a course requirement,
• Keep you from attending an assessment,
• Stop you from submitting an assessment,
• Significantly affect your assessment performance.

Available here is a list of circumstances that may be beyond your control. This is only a list of examples, and your
exact circumstances may not be listed.
You can find more detail and the application form on the Special Consideration site, or in the UNSW Special
Consideration Application and Assessment Information for Students.

UNSW Business School 2

Assessment 1: Quizzes
Weeks 3 & 5

30%

Tests

1 hour for each quiz

Via Moodle course site

Note: If using Turnitin include whether similarity report will be made available to students

Description of assessment task

ONLINE QUIZ 1 (10%)

Description: This is an online multiple-choice test, consisting of 10 questions covering week 1-3
Timing: The Online Quiz will be available on Week 3 Tuesday 5pm, and you must finalize your attempts to complete it
by 11.59pm on 16th June Sunday. The quiz will be disabled after this time so make sure that you have completed it
before then. My advice is to complete it a few days early to avoid any glitches that may occur.

ONLINE QUIZ 2 (20%)

Description: This is an online multiple-choice test, consisting of 20 questions covering week 1-5.
Timing: The Online Quiz in Week 5 will be available on 25th June Tuesday, and you must finalize your attempts to
complete it by 11.59pm on 30th June Sunday. The quiz will be disabled after this time so make sure that you have
completed it before then. My advice is to complete it a few days early to avoid any glitches that may occur.

Approach to the assessment task

Read the questions carefully and select the correct answer. You may attempt this quiz as many times as you choose
(on each attempt questions will be sampled from a bank in a randomized order). The highest mark will be recorded.
You will receive your mark and right answers right after the quiz closes.

Structure
This assessment task constitutes 30% of your overall mark for this course. Each question is worth 1 mark. You will
receive 1 mark for answering a question correctly. Feedback and results will be provided through the Moodle quiz
activity once the quiz window closes and after you have attempted the quiz.

Submission instructions
Remember you can take the quizzes as many times as you like until the quiz closing date. The highest mark will be
recorded as the final mark.

UNSW Business School 3

Assessment 2: Case Study Analysis
Week 7

40%
(BCom students: myBcom course points for PLO3 Business Communication)
Written response
Limit your response per question to a maximum of 2 A4 pages

Via Moodle course site

Note: If using Turnitin include whether similarity report will be made available to students

Description of assessment task

Each student will write a report to investigate a business problem about new product development in the digital space.
The case will enhance students' analytical, qualitative, and strategic decision-making skills as well as their knowledge
of multidisciplinary marketing approaches.
Specifically, limit your response per question to a maximum of 2 A4 pages. In total, no more than 6 pages. For each
specific task conducted, interpret the results and transform the results into words (findings) i.e. based on your results,
what further information and recommendations (e.g. managerial actions) you can provide to Colgate or LDC.

Approach to the assessment task

1. Sentiment analysis
Description of the dataset
With total 1000 rows, the dataset consists of 500 positive and 500 negative rows Colgate product reviews. All the
rows have been generated through prompts given to ChatGPT-4. There is no redundancy/duplication in the rows since
GPT generated only max 50 rows in one-go. The rows were carefully inspected to remove any duplicate found at run-
time. This dataset can be used for basic sentimental analysis as this dataset mirrors real-world customer product
reviews.
Here, sentiments on toothpaste reviews are collected and stored as .csv file, consisting of 2 attributes:
• Review: review of Colgate toothpaste.
• Sentiment: sentiment related to the review, i.e., positive or negative.
Instructions of this analysis
Perform the following tasks:
a) Train a natural language model from scratch (you can choose the complexity of your model) that predicts the
sentiment score (positive or negative) of a given review. To perform this task, the dataset needs to be split a
priori in a 80-20 train-validation dataset, respectively, at random. The inputs of your natural language models
can be either “Document embeddings” or “Bag of words”, depending on your personal preferences. Remark:
To perform this task, it can be that you need to perform some pre-processing steps.
b) Evaluate the performance of your trained model by calculating the precision, recall and F1-score
measurements on the validation dataset.
c) Predict sentiment from text. Visualise the Sentiment Analysis using either Scatter Plots or Heat Map (your
choice of method and variables).

2. Topic modelling
Description of the dataset
UNSW Business School 4
Product reviews are important for brands to not only develop new product features, but also continuous improvement
to achieve product superiority over their competitors, command higher market shares and a premium price relative to
their competitors. Here, a corpus of toothpaste product reviews published in a British supermarket in the United
Kingdom. It is stored as .csv file, consisting of 6 attributes:
• Rating: between 1 to 5
• ReviewText: Text of the toothpaste review in English
• SubmissionTime: Date of publishing for the product review in YYYY-MM-DDTHH:MM:SS.000+00:00 format
• Title: Text title of each review in English
• UserLocation: Location of the reviewer in city, region, country format
• UserNickName: Reviewer's nickname
Instructions of this analysis
Perform the following tasks:
a) Apply the Latent Semantic Analysis and Latent Dirichlet Allocation technique to study the topic focus of
toothpaste’s reviews. Characterize them by exploring the most frequent words in each topic.
b) How do these topics evolve through time in the product review?
Remark: To perform this task, it can be that you need to perform some pre-processing steps.

3. A/B Test
Description of the dataset
Reviewing the LDC case study again together with both Week 1 and Week 3 lecture slides as well as Week 4 tutorial
slides. LDC tested its product at each stage of the customer journey. LDC has implemented experimental design for
InstaMoney. The meaning of each variable is explained through the spreadsheet called “Variable Legends”.
Instructions of this analysis
Perform InstaMoney Prototype A/B testing results (both conversion and lift) for the following features:
a) Generating Prospects
b) Onboarding
c) Credit Assessment
d) Payment Reminder
e) Query Resolution .
Based on the A/B testing results, how can LDC incorporate the best feature into InstaMoney?

Submission instructions
• Questions 1 and 2 are designed for Orange, consisting of a clear workflow so that the steps and conclusions
you make can be easily followed and verified.
• Question 3 is designed to be answered using excel although Orange also allows you to select the plotting
method in Scoring method to conduct statistical analysis, such as t-test and ANOVA. Both tools are
acceptable. The key is to draw conclusion to ensure the best product features.
• Create a report using Microsoft Word, including cover page, index, headings, and visuals.
• Screenshot of your completed workflow from Orange.
• Include in your report answers to the questions 1-3 outlined above. Limit your response per question to a a
maximum of 2 A4 pages (single space, 12 fonts, 2.5cm margins, Calibri). Hence, a total number of 6 pages
can be submitted.
• Always provide rational to the insights you have gathered.
• Feel free to customize the report based on specific findings and nuances from your analysis.
• N Name your report as Tutorial_PA_Orange_zID.doc (or .docx). For example:
W14A_PA_Orange_z1234567.doc.
UNSW Business School
• Name your Orange workflow as Tutorial_PA_Orange_QuestionID_zID.ows. For example:
W14A_PA_Orange_Q1_z1234567.ows. OR W14A_PA_Orange_Q2_z1234567.ows.
• For Q3 and excel is chosen, name your file as W14A_PA_Excel_Q3_z1234567.xls (or .xlsx).
• This assignment contains four submission parts: report, workflow of question 1, workflow of question 2,
workflow or excel of question 3. Submit the files via the corresponding upload links on Moodle by 14th July
Sunday 11:59 pm.

• Late submission results in a 5% penalty for every day late, with a maximum of 5 days, impacting the grade
negatively.

UNSW Business School

A2 RUBRIC – Case Study Analysis
Criteria % Fail
Research and Data 25% Did not select a sufficient
Collection time period for their analysis.
Did not use segments,
advanced filters, or
secondary dimensions to
filter and analyse the data.
Application and 20% Did not use the appropriate
Data Visualisation table or graph type to visualize
their data.
Did not highlight the most
important data points in their
visualizations.
Provided High 45% Their recommendations were
Quality not relevant to the questions
Recommendations that they were asked to
answer.
Their recommendations were
not supported by the data that
they had analysed.
Their recommendations were
not based on a clear
understanding of the data.
Report 10% Writing was unclear,
Presentation disjointed, and difficult to read.
Writing was informal and
unprofessional.
Report was too long or too
short.
Did not cite their sources
correctly.
Did not use appendices or
used them incorrectly.
UNSW Business School
Pass
Selected a time period that was
long enough to capture the
trends and patterns that they
were interested in.
Used segments, advanced
filters, or secondary dimensions
to filter and analyse the data.
Used the appropriate table or
graph type to visualize their
data.
Highlighted the most important
data points in their
visualizations.
Their recommendations were
relevant to the questions that
they were asked to answer.
Their recommendations were
supported by the data that they
had analysed.
Their recommendations were
based on a clear understanding
of the data.
Writing was clear, concise, and
easy to read.
Writing is formal and
professional.
Report is the appropriate length.
Cited their sources correctly.
Used appendices correctly.
Credit
Selected a time period that was
appropriate for the scope of
their analysis.
Used segments, advanced
filters, or secondary dimensions
in a way that was effective in
answering their research
questions.
Used the appropriate table or
graph type to visualize their
data in a way that was clear,
concise, and easy to
understand.
Highlighted the most important
data points in their
visualizations in a way that was
clear and concise.
Their recommendations were
well-supported by the data that
they had analysed.
Their recommendations were
based on a sound
understanding of the problem.
Their recommendations were
supported by sound reasoning.
Writing is well-written and
engaging.
Writing is clear, concise, and
easy to read.
Report is longer than the
minimum requirements.
Cited their sources correctly
and thoroughly.
Used appendices correctly and
effectively.
Distinction
Selected a time period that was
well-chosen and allowed them
to draw meaningful
conclusions.
Used segments, advanced
filters, or secondary dimensions
in a way that was creative.
Used the appropriate table or
graph type to visualize their
data in a way that was creative
and engaging.
Highlighted the most important
data points in their
visualizations in a way that was
creative and engaging.
Their recommendations were
based on a sound
understanding of the problem
and were feasible to implement.
Their recommendations were
feasible to implement.
Their recommendations were
based on a deep understanding
of the problem.
Writing is polished.
Report complies with the length
requirements.
Cited their sources correctly
and creatively.
Used appendices correctly and
creatively.
High Distinction
Selected a time period that was
exemplary and allowed them to
make significant contributions
to the field.
Used segments, advanced
filters, or secondary dimensions
in a way that was innovative
and led to new insights.
Used the appropriate table or
graph type to visualize their
data in a way that was
innovative and led to new
insights.
Highlighted the most important
data points in their
visualizations in a way that was
innovative and led to new
insights.
Their recommendations were
innovative and led to new
insights.
Their recommendations were
both feasible and cost-saving to
implement.
Their recommendations were
based on a synthesised
understanding of the problem.
Writing polished and highly
developed.
Report complies with the length
requirements and information is
clearly signalled.
Cited their sources correctly
and follows the Harvard
reference system closely.
Used appendices correctly and
creatively. Appendices support
the report’s findings in a clear
way.
7
Assessment 3: Group Project
Week 10

30%
(BCom students: myBcom course points for PLO4
Teamwork)
Oral and written response

Max 15 minutes
Lecture on 29th July

Description of assessment task

You will develop a new data product with your group members. This project provides you with an opportunity to take
your knowledge and skills of the product analytics learned in the course and apply them to a real-world problem. The
purpose of group presentation is to communicate what you are building, who it is for, and how it will deliver value to
end users and to the business.

Approach to the assessment task

This assessment provides you with the opportunity to apply the concepts learned in class to develop and
communicate new data products. Your group will identify customers’ pain points and unmet needs in each stage of
the customer journey and suggest potential solutions. Then, your group will plan a new product measurement
framework, develop a tracking plan, and test your product. Finally, your group will communicate your work using video
recording. In classes, all of us will watch the recorded videos and then discuss them together.

Milestones, scaffolding, informal feedback opportunities during tutorials.

• Week 3: Group formation with each group of 4-5 students. Elect the roles of CEO (group leader), developer,
product manager, product marketing manager, UX designer. Confirm the app product or app service. (Keep a
diary re: teamwork challenges, solutions and skills developed)
• Week 4: Part 1 Creating “Customer Journey Map”
• Week 8: Part 2 Creating “Measurement Framework”
• Week 9: Part 3 Creating “Tracking Plan”
Week 10: Presentation

Submission instructions
As you are aware now, there are a number of delivery dates for this project. Although they are formative coursework
which does NOT count towards the final mark, completing the tasks in the respective week helps to progress your
group project. Make sure you work with your group and individually to format and complete the delivery following
these requirements:
• Always provide rational to the insights you have gathered and use academic support to strengthen your
arguments.

UNSW Business School 8

• For presentations, make sure you work on a clear and professional presentation. Remember that you do not
need to include all the materials you have gathered. You will need to make a group decision to organise the
most important materials.
• As part of your slide presentation, please add speaker notes to each slide. These notes are important because
they provide context, explanations, and additional details that may not be evident from the slide content alone.
The notes help you convey your message effectively to the audience and ensure a comprehensive
understanding of your presentation.
• All group members must be involved in the presentation.
• Name your presentation as Tutorial_PA_zID-groupleader.ppt (or .pptx). For example: T11A_PA_z1234567.ppt.
Note that only ONE member of the group submits the deck of slides via Turnitin.
• Submit the file via an upload link on Moodle by 29th July Monday noon 12:00 pm. Late submission results in a
5% penalty for every day late, with a maximum of 5 days, impacting the grade negatively.

UNSW Business School

Supporting resources and links
Part 1 Creating customer journey maps
Customer journey map is a visual representation of the steps a user takes while using your product and interacting
with your company. Put yourself in your customers’ shoes to find the problems in your user experience (and uncover
opportunities to solve them). The below templates (Free Customer Journey Map Template (productschool.com))
serve as a starting point, and you are welcome to create your own maps that help you align your product business
with customer needs and expectations.

UNSW Business School 10

Part 2 Creating a measurement framework
Mixpanel’s measurement framework is composed of a Focus metric, which typically is the metric that is slightly more
important than the rest; as well as Level 1 metrics that either directly contribute or act as a check to the focus metric.
More specific metrics can also be further added to drive the Level 1 and Focus metrics.

Part 3 – Creating a tracking plan

Your task is to build out a tracking plan with events, event properties and user properties that can help you keep track
of metrics and answer product questions. Use what you have learned and build a tracking plan with this in mind. For
instance, the product you will be developing a tracking plan for is a podcast player. The app works like a usual podcast
player where users can listen, subscribe and save/share podcasts they like. The service includes a free plan and a
paid premium plan. The paid premium plan provides users with extra features for an even better listening experience.
Use the outline of metrics below as the input for your tracking plan.

Metrics to track:
• What % of new sign-ups upgrade to a paid subscription?
• What is the average podcast listening time per week per user?
• What is the activation funnel conversion from signing up, listening to a podcast, and then subscribing to it?
• What is the average number of podcasts users are subscribed to?
• What are the top 10 podcasts that users subscribe to?

UNSW Business School

In addition to answering the questions, it is important to analyse measures in the following dimensions:
• The date the user signed up for an account
• The date the user upgraded to the paid premium plan
• The country that the user is in
• The plan the user is on (free or premium)

Use the tracking plan template provided in the course and add the additional events, event properties and user
properties necessary.

UNSW Business School

 A3 RUBRIC – Group Project
Criteria
Research evidence, Identify
key points and logical
analysis
Ability to home-in on key
issues
Engaging the listener as an
individual
Ability to engage the
audience (including Q&A)
Communicates clearly and
concisely
The clarity of slides
Coherence of the talk (in
relation to presentation flow
and overall big idea)
% Fail Pass Credit Distinction High Distinction
45% Development of ideas is not clear Identifies key points and attempts Development of ideas is generally Development of ideas is clear and In addition to meeting the
or logical. to explain them. clear and logical but could be logical; key points are clearly distinction criteria,
No application of course concepts, Attempts to develop ideas relevant more consistent. Good application communicated. research and analysis are
models, or tactics. to the part but lacks depth and of course concepts, models, or Very good application of course insightfully and effectively used to
No research and analysis focus. tactics. concepts, models, or tactics. draw conclusions and to provide a
Little or no mention of why your Some application of course Good research and analysis and Research and analysis are strong connection to other parts.
part is important and how does concepts, models, or tactics. used effectively to draw and used effectively to draw
your part integrate with other Some research and analysis and conclusions. conclusions and to provide a Explains very well how your part
parts. used effectively to draw Explains how your part integrate connection to other parts. integrate with other parts.
conclusions. with other parts. Explains well how your part
Attempt to explains how your part integrate with other parts.
integrate with other parts.
35% Poor eye contact with audience. A Some eye contact with audience. Adequate eye contact with Good eye contact with audience. Particularly good eye contact with
lot of reading. Poor body language.Some reading from script. Normal audience. Satisfactory level of Good body language. Prominent audience, and exceptionally good
Poor visual aids. Difficult to follow. body language. Some difficulty to interest from audience. Engaged level of interest from audience. body language. Particularly good
Reads from script. No or little follow. Normal visual aids. audience adequately. No script Have impact on audience. Some visual aids. Showed creativity and
variation of tone or expression. Some variation of tone and reading. lapses in attention noted. flair. Delivery information
expression to make the Engaging presentation which uses engagingly and memorably.
Does not express (or explain presentation engaging. animated expression and variation Engaging presentation which is
where necessary) complex ideas Does not consistently, expresses in tone. animated and with some creativity. Engaging and creative
and information clearly in (and explains where necessary) Generally, expresses and explains Expresses (and explains where presentation style.
language appropriate for the complex ideas and information in where necessary complex ideas necessary) complex ideas, In addition to meeting the
intended audience and purpose language mostly appropriate for and information clearly in arguments, and information clearly distinction criteria, the student
(using own words where possible). the intended audience and language appropriate for the and concisely in language uses fluent, accurate expression/
Word choice makes meaning purpose (using own words as intended audience and purpose appropriate for the intended grammar.
unclear. much as possible). (using own words as much as audience and purpose (using own
Time management is ok but needs possible). words as appropriate). Seamless time management.
Poor time management. to be managed better. Good time management. Good time management.

Teamwork and Leadership
Demonstrates proactive
collaboration skills and
effective conflict
resolution, actively
seeking assistance when
necessary.
Presents a project output
that is well-integrated.
Demonstrates the
practice of reflective
thinking.
20% Fails to proactively collaborate, Displays limited initiative in
resolve conflicts, or seek collaboration, seldom seeking
assistance, resulting in a lack of assistance or addressing
meaningful teamwork. conflicts. Poor understanding of
collaboration or lack of conflict
Fails to present a cohesive group resolution skills.
effort, lacking integration, logical
sequence, and overall coherence Project provides evidence of
within the project. some work on integration of
materials. Project is not
Fails to provide any evidence of consistent and lacks
reflective thinking. Little to no cohesiveness. Lacks logical
evidence of self-assessment or flow.
acknowledgment of the group's Reflective thinking is lacking,
performance. with little to no evidence of self-
assessment or acknowledgment
of the group's performance.
Shows some initiative in
collaboration, seeks assistance,
or tries to address conflicts in an
effective way.
Demonstrates some evidence of
integration, but notable
inconsistencies in wording, tone,
or style disrupt logical flow.
Occasionally engages in
reflective thinking, mostly
identifying strengths while
offering minimal
acknowledgment of areas for
improvement.
Displays effective collaboration
skills most of the time, seeking
assistance and resolving conflicts
when required.
Achieves good integration, albeit
with minor inconsistencies in
wording, tone, or style, yet
maintains a strong logical flow.
Frequently engages in reflective
thinking, highlighting an ability to
identify strengths, albeit with
limited recognition of areas for
improvement.
Exhibits exceptional proficiency in
collaboration skills, consistently
resolving conflicts adeptly and
seeking assistance as needed.
Achieves exceptional integration,
maintaining impeccable
consistency and logical coherence
across all project components.
Consistently engages in reflective
thinking, adeptly identifying
strengths and highlighting areas
for potential improvement and
future applications of gained
skills/tools.

UNSW Business School 13