Running head: COMPOSTING PROTOTYPE

Inspecting Prototype: Composting

Crystal Chavez

California State University, Monterey Bay

IST 622: Assessment and Evaluation

Professor Su

July 25, 2017

COMPOSTING PROTOTYPE 2

Table of Contents

Introduction ................................................................................................................................... 3

Methodology ................................................................................................................................. 3

Prototype ................................................................................................................................ 3

Learners .................................................................................................................................. 4

Tryout Conditions and Process .............................................................................................. 4

Results ........................................................................................................................................... 5

Entry Conditions .................................................................................................................... 5

Instruction .............................................................................................................................. 5

Outcomes ............................................................................................................................... 6

Recommendations .................................................................................................................. 7

Summary ....................................................................................................................................... 8

Appendices .................................................................................................................................... 9

Appendix A Introductions to Composting: Browns and Greens Pre-Quiz ......................... 9

Appendix B Section 2 ................................................................................................... 12

Appendix C Answer Bank .......................................................................................... 13

COMPOSTING PROTOTYPE 3

Introduction

As consumers continue to move towards eco-friendly decisions, many are considering

the benefits of composting. Introduction to Composting: Browns and Greens is a prototype

designed for the county of Santa Clara. This training module encourages consumers to reduce

the amount of garbage they generate. It helps beginner composters create a healthy compost bin

at their home, while also giving brief insight into more advanced techniques that they want to

avoid during their learning phase.

Methodology

Prototype

Introduction to Composting: Browns and Greens was created during the Interactive

Multimedia for Instruction (IST 526) course offered at California State University, Monterey

Bay. The prototype was built using Adobe Captivate software and the information provided in it

was received from Melissa Solorzano. The prototype is available on all platforms: desktop,

laptop, tablet, and phone. It is accessible anywhere and anytime through the click of a shared

link.

The eProduct is meant to help beginner composters learn how to setup and maintain a

compost bin at home. It engages the user by having a narrator explain much of the information,

while graphics appear on screen. Important numbers and vocabulary appear when appropriate.

Users interact with the prototype by engaging in drag and drop activities and answering practice

questions while continuously receiving instant feedback.

COMPOSTING PROTOTYPE 4

Learners

Participants in this study ranged from the ages of 18-62. Eight participants were

recruited through the use of social media and text messages. Many people in California are eco-

conscious, but we were also able to recruit participants from many different states. This helped

to ensure the product was successful regardless of whether or not the individual cares about the

state of the environment.

The ideal user would be someone with the interest of starting their own compost bin, but

without any idea how to develop and maintain a healthy one. They may have a basic

understanding of what can go in a compost bin, but might not necessarily know how they are

supposed to be kept. Since the eProduct was developed for Solorzano to employ in her capstone

while working for the county of Santa Clara, the anticipated future users would be residents of

that county. The demographic of my participants varies widely in prior knowledge of

composting, in age, and in computer-literacy.

Tryout Conditions and Process

All functionality of the prototype was tested fully before submission the previous

semester. The focus was on making sure all buttons led to where they were supposed to lead and

the drag and drop activities were fully functional on a tablet or other handheld device. An issue

reported back to be was that the arrow buttons were not working, however they were timed to

become actives after the narration had finished. So participants who had their volume all the

way down would not have known words were being spoken. This was addressed before heading

into user testing by reminding participants that audio/closed captioning was a necessary

component for completing the lesson.

COMPOSTING PROTOTYPE 5

Results

Entry Conditions

Participants were given the following message:

Help Crystal complete her Term project for her class. Click on the google form

and take the pre-quiz, watch an interactive video, and then take the post quiz. No

personal data will be recorded. She needs at least 10 volunteers. Itll take 10-15

minutes max. Compatible with phone, tablet, or computer.

https://goo.gl/forms/BgjQDEWh6Q941EG43

The link then leads participants to a Google Form (Appendix A). After section one (of three) is

completed, the pre-test, they are taken to section two and are directed to click on a link

(Appendix B) and press ok on the pop-up box.

Most of the participants were able to proceed without any questions. From the feedback

I was given by willing participants, there was no trouble getting the form and tutorial on

composting to play while using their mobile phones.

Instruction

All user activity was anonymous and the pre- and post-quizzes were manually graded.

Participants were instructed to answer all questions on the pre- and post- quiz. Thanks to

Google Forms, I was able to enforce that requirement. Feedback was made optional, and I did

not expect to receive feedback on the pre-quiz, but I was surprised to receive absolutely no

feedback at the end of the post-quiz. Of the users who wanted to share information, they

reported that the totally time needed to complete the exercise ranged from 10 to 15 minutes.
COMPOSTING PROTOTYPE 6

Outcomes

The first question surveyed how confident they were in their ability to create and

maintain a compost bin. This same question was revisited during the post-quiz to see if any

changes in confidence occurred. Without looking at the complete test results, 75% of users

reported a higher confidence level on their post-quiz. Of those 75%, 25% included a significant

increase in their confidence levels.

Pre-Quiz 2 1 1 1 2 4 1 3
Post-Quiz 5 5 1 4 2 5 2 4
Table 1. Individually Reported Confidence Levels

The average score on the pre-quiz was 8.5 correct answers, whereas the average on the

post-quiz was 12 correct answers. Initial findings indicate that the users gained some knowledge

by watching and participating in the Composting tutorial. Full results are below in table 2.

Pre-Quiz 6 10 8 8 6 8 10 12
Post-Quiz 14 14 8 12 8 14 12 14
Table 2. Correct Number of Answers per User

I hypothesized that users would do better on the post-quiz after watching the useful

video. Using the p-value from the one tailed test, 0.00309%, we can assume that the likelihood

of lucky randomized answers is low. Since -3.861740991 (t-stat value) is less than the critical

value of 1.894578605, we can reject the null hypothesis.

COMPOSTING PROTOTYPE 7

t-Test: Paired Two Sample for Means

Pre-Quiz Post-Quiz
Mean 8.5 12
Variance 4.285714286 6.857143
Observations 8 8
Pearson Correlation 0.421637021
Hypothesized Mean Difference 0
df 7
t Stat -3.861740991
P(T<=t) one-tail 0.00309876
t Critical one-tail 1.894578605
P(T<=t) two-tail 0.006197521
t Critical two-tail 2.364624252
Table 3. Statistical analysis of the Pre and Post Quiz scores

The biggest takeaway message is that no users reported feeling less confident in their

skills and no students received a lower score on the post-quiz.

Recommendations

Introduction to Composting: Browns and Greens seem to be ready to be included as one

component in a larger module. I recommend keeping the pre-quiz and post-quiz as a means to

show your supervisor that the product is effective in teaching the community some of the basics

of composting.

Future research should give the participants the chance to leave some personal

information so that we can contact them about their experience. It was difficult to know if

something was going wrong during the user testing phase, luckily, all the participants of this

study could call or message me for advice whenever they ran into a problem.

The results were positive. Thanks to user-testing I feel confident in the prototype I built

with the help of the designer, Melissa Solorzano. Confidence levels grew among the users, but it

is too early to know whether any of them will start their own compost bin.
COMPOSTING PROTOTYPE 8

Summary

The prototype was a success! Many of the potential roadblocks were found out before

sending the eProduct out for user testing. Although participation was anonymous and no

personal records were received or stored, many of the participants contacted me afterwards.

They were very joyful in their results and felt confident enough to brag about their experience. I

do not know where this portion of the composting training will land, but it seems like a good

building block.
COMPOSTING PROTOTYPE 9

Appendices

Appendix A Introductions to Composting: Browns and Greens Pre-Quiz

*Required
1. Q1: How confident are you in your ability to maintain your own
compost bin? * Mark only one oval.

Not confident at all - 1 2 3 4 5 completely confident

00000
2. Q2: What is the best way to maintain a healthy compost bin? * Mark only one
oval.

0Adding compost to it every day

0Keeping a balanced brown to green ratio

0Watering it daily
0Maintain an adequate worm supply inside it

3. Q3: What are considered the browns of composting? (Select all that
apply) * Check all that apply.

Dfresh grass clippings

D dead leaves
D Nitrogen rich materials
D Carbon rich materials

4. Q4: What are considered the ""greens"" of composting? (Select all that
apply) * Check all that apply.

D fresh grass clippings

D dead leaves
D Nitrogen rich materials

D Carbon rich materials

COMPOSTING PROTOTYPE 10

5. Q5: What might happen if you add too many materials rich in Carbon to your
compost bin? * Mark only one oval.

0It will slow the breakdown of materials

0The bin will start to smell
0It will attract unwanted animals
0Nothing. The Carbons will neutralize the Nitrogens

6. Q6: Adding too many Nitrogen-rich materials into the compost bin will
result in what? * Mark only one oval.

0The bin will breakdown materials slower

0Dehydration. You will need to water the bin.

0Nothing. The Nitrogens neutralize the Carbons

0The compost pile will start to smell

7. Q7: How should you organize the organic materials into your
compost bin? * Mark only one oval.

0Dump in browns and greens all together

0Put in only greens/Nitrogens

0Put in only browns/Carbons

0Have equal layers of browns and greens
8. QS: What organic materials are not suitable for a beginners home
compost bin? (Select all that apply) * Check all that apply.

D dead leaves
D diseased leaves
D olive oil

D raw meat
D milk
COMPOSTING PROTOTYPE 11

9. Q9: Match the item to the category it belongs to.*

Mark only one oval per row.

Trash Browns Greens

Newspaper 0 0 0
Coffee grounds 0 0 0
Fatty foods 0 0 0

10. Q10: What is the proper Carbon to Nitrogen ratio? *

Mark only one oval.

030 carbons : 1 nitrogen

01 carbon: 30 nitrogens
01 carbon : 1 nitrogen

01 carbon : 5 nitrogens
05 carbons: 1 nitrogen

11. Feedback?
COMPOSTING PROTOTYPE 12

Appendix B Section 2

Interactive Tutorial
Please go to the following link and perform the exercise:
http://itcdland.csumb.edu/~crchavez/ist526/compost_v5_final/index_scorm.html

Pop-up will occur, press OK to continue.

Figure 1. Image of introductory welcome screen.

12. I completed the Introduction to Composting: Browns and Greens *

Mark only one oval.

0 Yes, take me to the post-quiz Skip to question 13.

0 No. Skip to question 12.

COMPOSTING PROTOTYPE 13

Appendix C Answer Bank

Introduction to Composting:Browns and Greens Post-Quiz

1. Q1: How confident are you in your ability to maintain your own compost
bin? * Mark only one oval.

Not confident at all - 1 2 3 4 5 completely confident

00000
14. Q2: What is the best way to maintain a healthy compost bin? *
Mark only one oval.

0Adding compost to it every day

0Keeping a balanced brown to green ratio

0Watering it daily
0Maintain an adequate worm supply inside it

15. Q3: What are considered the browns of composting? (Select all that
apply) *Check all that apply.
Dfresh grass clippings
D dead leaves
D Nitrogen rich materials
D Carbon rich materials

16. Q4: What are considered the ""greens"" of composting? (Select all that
apply) * Check all that apply.

D fresh grass clippings

D dead leaves
D Nitrogen rich materials

D Carbon rich materials

COMPOSTING PROTOTYPE 14

17. Q5: What might happen if you add too many materials rich in Carbon to your
compost bin? * Mark only one oval.

0It will slow the breakdown of materials

0The bin will start to smell

0It will attract unwanted animals
0Nothing. The Carbons will neutralize the Nitrogens

18. Q6: Adding too many Nitrogen-rich materials into the compost bin will
result in what? * Mark only one oval.

0The bin will breakdown materials slower

0Dehydration. You will need to water the bin.

0Nothing. The Nitrogens neutralize the Carbons

0The compost pile will start to smell

19. Q7: How should you organize the organic materials into your
compost bin? *Mark only one oval.

0Dump in browns and greens all together

0Put in only greens/Nitrogens

0Put in only browns/Carbons

0Have equal layers of browns and greens

20. Q8: What organic materials are not suitable for a beginners home compost
bin? (Select all that apply) *Check all that apply.

D dead leaves
D diseased leaves
olive oil
D
D raw meat
D milk
COMPOSTING PROTOTYPE 15

21. Q9: Select the category to which the Item belongs.*

Mark only one oval per row.

Trash Browns Greens

Newspaper 0 0 0
Coffee grounds 0 0 0
Fatty foods 0 0 0

22. Q10: What is the proper Carbon to Nitrogen ratio? *

Mark only one oval.

030 carbons: 1 nitrogen

01 carbon : 30 nitrogens
01 carbon: 1 nitrogen

01 carbon: 5 nitrogens
05 carbons:1 nitrogen

23.Feedback?

Powered by
IIGoogle Forms