Iormalive IvaIualion IIan

Georgia 4-H Manners Worlh MiIIions I-Learning DeveIomenl Iro|ecl

2/25/2014

Team Avesome:
Russ IaImer
HoIIy Sanders
Sarah Marslon
MichaeI Smilh
Lucy DaigIe


Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 1

/012345
CheryI Varnadoe, Ixlension 4-H SeciaIisl
324 Hoke Smilh Annex
Alhens, GA 30602-4356
706/542-4444
cv4h+uga.edu

(346789:41735
This documenl describes lhe formalive evaIualion Ian lo assess lhe usabiIily and effecliveness
of lhe Georgia 4-H Manners Worlh MiIIions lraining moduIes deveIoed by Team Avesome.
These moduIes focus on roer dining eliquelle as delaiIed in lhe Georgia 4-H Manners Worlh
MiIIions curricuIum.
The formalive evaIualion Ian describes lhe background, urose, and Iimilalions of lhe
evaIualion, lhe slakehoIders and arlicianls of lhe evaIualion, lhe decisions and queslions lo
be addressed by lhe evaIualion, lhe melhods, inslrumenls, and rocedures lo be used in lhe
evaIualion, and lhe Iogislics, limeIine, and budgel for lhe evaIualion. This evaIualion Ian has
been reared by Team Avesome as a requiremenl for lhe IDIT 6210 course in lhe
InslruclionaI Design and DeveIomenl (IDD) rogram al lhe Universily of Georgia.

;<:=>679385
Georgia 4-H is an organizalion lhal seeks lo offer a variely of Ieadershi and ersonaI skiIIs lo
young eoIe from age 9 19. These skiIIs refIecl lhe four H's of Head, Hearl, Hands, and
HeaIlh. Our cIienl has noled a decrease of eliquelle and manners avareness in lhe young
eoIe she vorks vilh. As she noled, lhese sofl skiIIs are rareIy laughl in schooI, and may nol
be Iearned al home, yel lhey are crilicaI lo successfuI inleraclion vilh eers and aduIls aIike.
The currenl Manners Worlh MiIIions conlenl is simIy a sel of besl raclice highIighls and
buIIel oinls, Ieaving significanl room for lhe crealion of a more robusl inslruclionaI soIulion.
One arl of lhe Manners Worlh MiIIions curricuIum covers dining eliquelle. A recenl Nev York
Times arlicIe made lhe oinl lhal vilh lhe increase in lhe number of eoIe hoIding more lhan
one |ob, meaIs oflen come in lhe form of fasl food, and lhe eliquelle associaled vilh and oflen
Iearned from having meaIs logelher al home may nol haen. The foIIoving quole suorls
lhis oinl: Mrs. Mason bIames a fasler ace of Iife and lhe demise of lhe home-cooked famiIy
Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 2

meaI. 'You don'l need lo knov aII your sociaI graces lo sil dovn al McDonaId's and eal a burger
and fries,' she said. (Severson, 2011).
Our cIienl agrees, slaling lhal |kjids don'l sil dovn al lhe labIe vilh lheir famiIies as oflen as
lhey used lo and slales lhal one of her goaIs vilh lhis curricuIum is lrying lo bring famiIies
back lo lhe labIe vhere lhey can raclice good manners. Team Avesome has been chosen lo
deveIo e-Iearning aIigned vilh lhe exisling Manners Worlh MiIIions curricuIum lhal viII be
used by agenls al meelings and evenls lhroughoul Georgia lo address lhis robIem.

.96?7@25
The urose of lhis evaIualion is lo rovide feedback lo lhe deveIomenl leam on lhe quaIily of
lhe Georgia 4-H Manners Worlh MiIIions lraining moduIes and lo assess lhe usabiIily and
effecliveness of lhe moduIes.
In order lo oblain lhis informalion, lhe evaIualion viII:
erform a usabiIily lesl in order lo delecl fIavs in lhe design of lhe moduIes
oblain feedback on lhe lraining moduIes from exerl revievers in order lo modify lhe
design of lhe moduIes lo imrove quaIily and beller meel inslruclionaI goaIs
assess lhe inslruclionaI effecliveness of lhe conlenl for lhe Iearning ob|eclives
oblain feedback on lhe lraining moduIes from lhe cIienl and from reresenlalives of lhe
largel audience in order lo modify lhe design of lhe moduIes lo increase quaIily and
beller aIign vilh lhe needs of lhe Georgia 4-H rogram

A4<=2B70826@5
The rimary slakehoIders for lhis evaIualion incIude CheryI Varnadoe and olher slaff of lhe
Georgia 4-H organizalion as veII as lhe membershi of Georgia 4-H.
The deveIomenl leam for lhe lraining moduIes incIudes Lucy DaigIe, Sarah oIen Marslon,
Russ IaImer, HoIIy Sanders, and MichaeI Smilh.

Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 3

C2:1@173@5
Since lhis is a formalive evaIualion, lhe rimary decision lo be addressed by lhis evaIualion is
hov lhe usabiIily and effecliveness of lhe lraining moduIes may be imroved.

D92@4173@5
In order lo suorl informed decision making aboul lhe usabiIily and effecliveness of lhe
lraining moduIes, lhe foIIoving queslions viII be addressed during lhis evaIualion:
Are lhe lraining moduIes easy lo use`
Do exerl revievers feeI lhal lhe lraining moduIes are easy lo use`
Do Iearners reresenlalive of lhe largel audience feeI lhal lhe lraining moduIes are easy
lo use`

Are lhe lraining moduIes effeclive`
Do exerl revievers feeI lhal lhe lraining moduIes are effeclive`
Do Iearners reresenlalive of lhe largel audience shov an increase in knovIedge of
dining eliquelle uon comIeling lhe moduIes`

Are lhe lraining moduIes engaging`
Do Iearners feeI lhal lhe lraining moduIes are engaging`


%24B78@5
A mixed melhods aroach viII be used lo address lhese queslions. Dala coIIeclion slralegies
viII incIude:
1. IvaIualor rofiIe
2. Ixerl reviever usabiIily lesl
3. Learner usabiIily lesl
4. Learner re- and osl-lesls
5. Training moduIe scores

Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 4

&0?B< '2@45
The urose of lhe aIha lesl is lo:
1. erform a usabiIily lesl in order lo delecl fIavs in lhe design of lhe moduIes
2. oblain feedback on lhe lraining moduIes from exerl revievers in order lo modify lhe
design of lhe moduIes lo imrove quaIily and beller meel inslruclionaI goaIs

.<641:1?<34@5
Iarlicianls in lhe aIha lesl viII incIude a seIeclion of IDIT 6200 sludenls from lhe IDD
rogram.

.67:289625
The evaIualion leam viII conlacl five evaIualors from lhe IDIT 6200 cIass via emaiI vilh
inslruclions for comIeling lhe aIha lesl. Uon comIelion of lhe moduIes, each evaIualor viII
fiII oul an evaIualor rofiIe form and usabiIily lesl.

&@@2@@E2345
AII evaIualors viII be asked lo comIele a usabiIily lesl uon comIelion of lheir reviev of lhe
ro|ecl. The usabiIily lesl viII consisl of 12 - 15 queslions reIaled lo conlenl, inlerface, and
navigalion, evaIualors viII resond lo lhe queslions using a Likerl scaIe, vilh olions ranging
from SlrongIy Disagree lo SlrongIy Agree.
Survey loics viII incIude:
Conlenl
Navigalion
Inlerface
IunclionaIily
Robuslness
UsabiIily
InslruclionaI slralegies

Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 5

;24< '2@45
The urose of lhe bela lesl is lo:
1. assess lhe inslruclionaI effecliveness of lhe conlenl for lhe Iearning ob|eclives
2. oblain feedback on lhe lraining moduIes from samIe Iearners reresenlalive of lhe
largel audience in order lo modify lhe design of lhe moduIes lo increase quaIily and
beller aIign vilh lhe needs of lhe Georgia 4-H rogram

.<641:1?<34@5
Iarlicianls in lhis evaIualion viII incIude a seIeclion of Iearners reresenlalive of lhe largel
audience.

.67:289625
The cIienl and/or lhe evaIualion leam viII conlacl 4 6 samIe Iearners reresenlalive of lhe
largel audience vilh inslruclions for comIeling lheir orlion of lhe bela lesl. The samIe
Iearners viII comIele re- and osl-lesls before and afler revieving lhe lraining moduIes, in
addilion lo a shorl usabiIily lesl.

&@@2@@E2345
efore revieving lhe moduIes, lhe samIe Iearners viII comIele a re-lesl of 12 - 15 muIliIe-
choice queslions uIIed from lhe moduIes in order lo delermine a baseIine of knovIedge. Afler
revieving lhe moduIes, lhe samIe Iearners viII comIele a osl-lesl of lhe same queslions in
order lo measure Iearning effecliveness. SamIe Iearners viII aIso comIele a shorl usabiIily
lesl, vhich viII consisl of 10 - 12 queslions reIaled lo conlenl, inlerface, and navigalion,
evaIualors viII resond lo lhe queslions using a Likerl scaIe, vilh olions ranging from
SlrongIy Disagree lo SlrongIy Agree. Survey loics viII incIude:
Conlenl
Navigalion
Inlerface
UsabiIily
Ingagemenl

Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 6

+1E14<4173@5
'1E25
As lhis is a schooI ro|ecl, ve are conslrained by lhe lime Iimils of lhe course. Design,
deveIomenl, and evaIualion viII lake Iace over lhe course of one semesler, modificalions
based on feedback from lhe evaIualion viII be Iimiled lo vhal is feasibIe given lhe due dale of
lhe finaI ro|ecl.
;98>245
In-delh evaIualions can be coslIy, since no budgel has been rovided for lhis ro|ecl, ve viII
be Iimiled lo lhe looIs and lechniques lhal ve can imIemenl for free.
/7EE931:<41735
Communicalion vilh aII arlicianls viII mosl IikeIy be conducled eIeclronicaIIy, since lhe
evaIualion leam, 4-H slaff, and samIe Iearners may be searaled by dislance. Ivery efforl viII
be made lo faciIilale communicalion, evaIualion, and dala coIIeclion by making resources
avaiIabIe onIine.

+7>1@41:@5
Russ IaImer and MichaeI Smilh viII coordinale lhe imIemenlalion of lhis evaIualion Ian
(incIuding scheduIing, dala coIIeclion, and dala reorling) vilh lhe cIienl. AII dala coIIecled viII
be rocessed, anaIyzed, and inlerreled by lhe evaIualion leam, vho viII lhen creale a finaI
reorl of lhe resuIls. The finaI reorl viII be rovided lo CheryI Varnadoe and olher
slakehoIders (as delermined by CheryI Varnadoe and Russ IaImer). Secific delaiIs aboul lhe
scheduIing of lhe evaIualion, dala coIIeclion, and dala reorling may be found in lhe TimeIine
seclion of lhis Ian.

'1E201325
This evaIualion viII lake Iace according lo lhe foIIoving eslimaled limeIine:
IvaIualion IIan Ieb 25
AIha Tesling Ar 01 Ar 07
ela Tesling Ar 08 Ar 14
Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 7

IvaIualion ResuIls AnaIysis Ar 15 Ar 21
IvaIualion Reorl Ar 22

The finaI reorl viII be submilled lo CheryI Varnadoe and olher inleresled slakehoIders no Ialer
lhan 5:00 .m. on AriI 22.

;98>245
No budgel has been aIIocaled for lhe deveIomenl or evaIualion of lhe Manners Worlh MiIIions
lraining moduIes, since lhis ro|ecl is being deveIoed in order lo fuIfiII lhe requiremenls of lhe
IDD rogram's IDIT 6210 course. The cosls for lhis evaIualion viII consisl excIusiveIy of
ersonneI cosls, lhe buIk of vhich viII be rovided ro bono by lhe evaIualion leam.
AddilionaI ersonneI cosls viII incIude lhe lime and efforl senl by slakehoIders (4-H slaff and
samIe Iearners) on consuIling, scheduIing, exerl reviev, inlervievs, and surveys. In order lo
kee cosls dovn, aII inlervievs and surveys viII be rovided onIine, and lhe resuIls viII be
dislribuled eIeclronicaIIy.

&??2381F &5 #61>13<0 /01234 .67?7@<0
We are vorking on a nev manners curricuIum for youlh and vanl an inleraclive ro|ecl lo go
aIong vilh il. Ierhas a age vilh a Iace selling according lo a given menu lhal you can drag
and dro lo roerIy sel a labIe lo malch lhe menu, a manners quiz bovI queslion and ansver
game, and any olher inleraclive eIemenls lhal leach youlh aboul manners and eliquelle.

Iebruary 25, 2014 "#$%&'()* *)&+,&'(#- .+&-

Task AnaIysis Iage 8

/01234 &94B761G<41735

IvaIualion IIan for Georgia 4H Manners Worlh MiIIions I-Learning ModuIe Iro|ecl:
Afler revieving lhe above maleriaIs, I consenl lo lhe ro|ecl delaiIs and granl ermission lo lhe
Universily of Georgia inslruclionaI design leam lo conlinue vilh lhe ro|ecl deveIomenl.



CheryI Varnadoe

_________________________________________ ______________________
Signalure Dale




Russ IaImer

_________________________________________ ________________________
Signalure Dale