Executive Summary (1 -1.

5 pg)
The State of Missouri has a valuable resource for its residents in Mizzou K-12
Online. The origins of the program lie in former correspondence courses that
could be completed by elementary and secondary age students throughout the
state as a means to work towards achieving a high school diploma. With the
advent of online learning in recent years the Mizzou K-12 Online program was
born to take correspondence courses to the next level and capitalize on the
power of the internet.
Many of the courses offered by Mizzou K-12 Online are simply former

correspondence courses gone digital. While this provides learners with a

potentially easier access point it really doesnt utilize technology as more than a
substitutionary mechanism of content delivery. Dr. Reuben Penteduera
advocates for higher level technology integration in a manner that allows for
redefinition of what is possible as a result of technological progress. (see SAMR)
Dr. Holly Henry, program director for Mizzou K-12 Online, has realized the
potential that the offered courses have to be more than just a substitute for a
traditional correspondence course. In fact, while this evaluation will focus
exclusively on the semester based world history course that is offered, her hope
is to review the entirety of the program in hopes of making the offerings more
relevant, robust, and revolutionary while also maintaining their academic integrity.
In order to transform these courses into truly redefined learning experiences it is
essential to evaluate what the current model is from several vantage points. The
goal of this evaluation is to be able to provide substantive data from students to
online learning experts regarding the state of the program as it is currently
implemented and to use that data to make informed decisions and
recommendations for improvements to the course specifically and the program
generally for the benefit of future learners enrolled in Mizzou K-12 Online.
Questions that are addressed in the evaluation protocol are raised in response to
the decisions the client will need to make in regards to moving forward with the
improvement of the Mizzou K-12 Online semester based world history course.
All supporting material and resources can be found in the appendices.

Client Assumptions (1 paragraph)

Mizzou K-12 Online currently serves the state of Missouri as an online option for
distance learning for the state's K-12 students. Currently, the entire program is up

for review. The original online courses were simply converted from former
correspondence courses with no real functional change other than moving into
an online format. Our client is working under the assumption that the current
state of online learning is much more advanced and offers many more
advantageous options for students than when the online courses were originally
constructed. These courses are admittedly out of date and in need of redesign
based on contemporary research involving how people learn online. Our client
understands that this is a huge transition process and that there is an
incremental nature to this rather than wholesale change across the entire
program. Courses identified as the primary recipients of research based
recommendations into their structure and process include Personal Finance and
the course addressed by this study, World History. Our client does not expect this
study to contribute to an update of actual content but that it will primarily focus on
content delivery and student interactions.

Introduction (2-3 paragraphs)

This document is the report on the Mizzou K-12 Online term-based World History
course. The course is divided into two units, both 26 weeks in duration and
offered to students in grades 10-12. The course covers the history of the world
from the Stone Age through 1776 in the first unit and the history of the world from
1776 through the Persian Gulf War in the second unit. The delivery mechanism
of the online curriculum will be evaluated over an 8 week process as a research
assignment by four students (Karen Whelan, Stephanie Bossaller, Emily Millikan,
and Scott Vonder Bruegge) enrolled in Formative and Summative Assessments,
a course that is part of the University of Missouri School of Information Science
and Learning Technologies.
The sections of this report will focus on several areas. These main areas will be
purpose, methodology, and results. The purpose section will include information
on the background, audience, and decisions affected by the results. As part of
the section on methodology, this report will include information on questions to be
asked, the format and delivery mechanism for those questions, strategies for
obtaining data, and definition of the research sample. Lastly, the report will
include a summary of the findings and include recommendations based off
research conducted.
Dr. Holly Henry, of Mizzou K-12 Online, has initiated this evaluation of their
program. Through the assistance of Dr. Julie Caplow at the University of
Missouri, the research team of Karen Whelan, Stephanie Bossaller, Emily
Millikan, and Scott Vonder Bruegge has been assembled. Emily Millikan serves a

team leader.

Background (1-2 pages)

Describes any information needed to provide the reader with an understanding of
the background and context of the interactive learning (instructional material)
system being evaluated.
The University of Missouri High School has been providing education to
high school students for nearly 100 years. Given the changes in education and
the needs of learners, The University of Missouri founded K-12 Online Education
to provide even more learning opportunities to a variety of users and serves
3,805 students from around the world. More information about the Mizzou K-12
Online Program can be found at: here
The original K-12 Online Courses were designed directly from
correspondence courses, which consisted of accessing materials online or in
print and using US mail to mail assignments between learner and instructor.
Therefore, the online courses were designed as mainly read and respond
courses. The students read material and responded to their instructors through
assignments. Those assignments were sent to the teacher for feedback and then
returned to the student. The main difference between the correspondence and
online courses is that all work and communication is now completed in an online
environment, decreasing wait time for replies and feedback. The online course
environment also offers more opportunities for a variety of activities and
increased student interaction.
Mizzou K-12 Online provides flex and scheduled courses. The flex
courses are still very similar to the original correspondence courses. There are
more opportunities for discussion, group activities, and simulations in the
scheduled courses. For the purposes of this evaluation, a World History
scheduled course, delivered through the LMS Canvas, will be evaluated, and
feedback will be provided.
The variety of courses offered allow students many opportunities to
complete their high school career while being part of other commitments
including, but not limited to: acting, athletics, travel, medical disabilities, early
graduation, AP course needs, alternative students, and course recovery. These
courses are also offered to students in other countries including China, Vietnam,
and Brazil, offering students all over the world access to quality education.

In order to ensure these students are provided with quality learning

environments and the same opportunities their peers are in the face-to-face
classroom, Dr. Holly Henry and her colleagues have requested evaluations on
two of their courses.
Through this evaluation, content will not be examined. Mizzou K-12 Online
has designated content specialists to determine what content needs to be
provided to students. This evaluation will examine the delivery of materials,
interactions between users, use of various technologies, and other aspects
needed to bring this course up-to-date with current opportunities.
The courses are currently in an outdated state, with no changes made in
the last several years. Theory and learning styles have changed drastically, so
the needs of the online environment need to change with them. The clients have
requested that their courses become more lively and up to date in order to create
a cohesive learning environment for 21st century learners. It is also important for
the evaluation to take into consideration any future changes and the flexibility
needed for learners and educators in this environment.

Purposes (1-1.5 pages)

Delineates the purpose of the evaluation. A single plan can address a variety of
purposes, but all must be delineated clearly. Evaluation is always a political
process and all parties must accept the purposes of the evaluation to be
successful. Include in this section the purposes for BOTH the formative and
summative components of the overall evaluation plan.

Formative Purpose(s)
Summative Purpose(s)
Per the request of the client, Dr. Holly Henry, the purpose of this evaluation
will be to primarily look at the structure and organization of the on-line
presentation of the scheduled Mizzou K-12 On-line World History Course and
make determinations about the effectiveness of the learning environment in order
to provide formative feedback which will allow for improvement in the
presentation style of the content of the course. Dr. Henry hopes this evaluation
will determine a structure and delivery system that is engaging and results in
learners' increased mastery.
A successful, well designed on-line course contains a variety of
components which engage learners on many levels and promote active and
interactive involvement. Material that is presented via multi-media methods has
been shown to improve learner outcomes and improve the learners processing

and critical, higher order thinking skills. This in turn increases learner satisfaction
and motivation which can then transfer to greater marketability of the on-line
product.
The World History course will be evaluated for features that make the
learning environment as effective as possible. The evaluation tools will examine
the balance between passive versus interactive learning, individual study versus
collaborative interactions and will notate the inclusion or exclusion of instructional
activities that promote interactivity such as on-line discussions, video/audio
conferencing and other types of collaborative assignments. The evaluation team
will be looking at whether there are learning resources within the course, such as
illustrations, photographs, videos, animation or other forms of interactive media,
which are being used to present facts and reinforce concepts that allow for
compatibility with learners different interests, abilities, and learning styles. An
investigation will take place as to whether the course contains links to resources
that allow for exploratory learning beyond the scope of the immediate content
being presented and whether constructive, relevant, and frequent feedback is
provided to promote clarification, elaboration, and transfer of knowledge.
Ultimately, the results of this evaluation will give summative data to the
client that indicates whether the course in its current format is effective, not so
much in its presentation and acquisition of content, but in the satisfaction and
engagement level of the learners.

Audience/Stakeholders (1-2 paragraphs)

Specifies the clients as well as all the primary and secondary audiences or
consumers of the evaluation. In general, it is recommended to open up the
evaluation to as many people or agencies as the client will allow. This is where
you will identify the relevant stakeholders in the evaluation--those are the people
who have a direct and indirect interest in the results of the evaluation. Be sure to
remember that you may have both different and similar audiences for the
summative and formative components of the overall plan.

-Audience for Formative Component

-Audience for Summative Component
Our client, Dr. Holly Henry, an instructional designer-expert for Mizzou K-12 Online has
requested an external evaluation of one of their World History scheduled semesterbased high school course. The primary audience and/or stakeholders for this initial
formative evaluation report component are the administrators and staff of the College of
Education Mizzou K-12 Online and MU High School. This report will provide
supplementary information as they engage in an entire program re-design.

Potential indirect audiences of the summative evaluation component could possibly be:
The University of Missouri College of Education administration and
faculty;
Program accrediting agencies: AdvancED and the North Central
Association Commission on Accreditation and School Improvement; and
The distance education community, through dissemination and publishing
of report findings to conferences and journals.

Decisions (1-2 paragraphs)

Specifies the anticipated decisions that will likely be influenced by the evaluation.
This section is probably the most difficult, but it should be included in the
evaluation - i.e., what impacts the results of the evaluation may have on
decision-making. Most developers do not wish to anticipate negative outcomes,
but these too must be considered. This section should included anticipated
decisions for BOTH the formative and summative components of the plan.

-Decision(s) for Formative Component

-Decision(s) for Summative Component
This evaluation is preliminary analysis for the Mizzou K-12 Online instructional designers
as they engage in a complete program re-design. We are reviewing one scheduled high
school course, World History, with the goal of investigating ways to increase usability
and encourage student participation. Decisions that could result or be influenced by the
results of this evaluation, could be:
1. Improvements and modifications to current multi-media methods;
2. Strategies for increasing student engagement within this course
encouraging a more interactive learning experience; and
3. The implementation feasibility of a multi-media and collaborative
framework built into other scheduled courses program wide.
Those recommendations found to be pedagogically sound and feasible could result in
long term program assessment. That assessment could review if enrollment numbers
within the scheduled courses were affected; and to measure learners satisfaction and
engagement within the program.

Questions (.5 - 1 page)

Clarifies the questions to be addressed by the evaluation design and data
collection methods. The clearer and more detailed these questions are, the
more likely that you will be able to provide reliable and valid answers to them.
Include the questions to be used for BOTH the formative and summative
components of the plan. You can use parentheses after each question
(formative) or (summative) to indicate which component will be addressed by the
question OR have two separate sub-sections under the Questions section; one
sub-section for formative questions and one sub-section for summative
questions.

-Questions for Formative Component

-Questions for Summative Component
The questions documented below will help establish the viability of achieving the
purpose and goals of this evaluation.
1. Will the current structure of the course be able to incorporate
collaborative elements (i.e. group projects, blogs, wikis, discussion boards, etc.)
while still meeting the learning objectives? [F]
2. Will the students be able to understand the collaborative environment (i.e.
discussion board, voice thread, wikis, video meetings)? [F]
3. Have current scheduled students indicated they would like more
interactivity within their courses? [F]
4. Have prospective students indicated they were looking for courses with
interactivity when evaluating a course to enroll in? [F]
5. Would there be any additional technical requirements for adaptation of
collaborative elements? [F]
6. Are the instructions clearly stated for ease of use for these newly
incorporated collaborative elements? [F]
7. Would current flex students be interested in more interactive scheduled
courses? [F]
8. Are instructors adept enough to maintain and foster course collaborative
and/or interactivity efforts within their course to ensure momentum isnt lost?
[F/S]

9. What current innovations have subject matter experts or course

instructors instituted within their course to encourage student participation? Has
this worked? [F]
10. Has the increased usage of collaborative elements led to increased
student enrollment? [S]
11. Does the average age or maturity of the enrolled student allow for
collaborative features within the course (i.e. voice thread, discussion board,
etc.)? [S]
12. Has the use of external applications (i.e. wordle, wikis, audio notes,
thinkbinder, etc.) increased student participation beyond the requirement making
the course richer with student interactivity? [S]
13. Are students more likely to continue to enroll in scheduled courses after
successfully completing one with more collaborative elements? [S]
__________________
[F]ormative component
[S]ummative component

Method (1-3 pages)

Describes the evaluation design and procedures. There are scores of designs
and hundreds of procedures that can be used. The keys to success are
matching these options with the purposes and questions of the plan and keeping
within the budget and timelines of the study. Identify the methods for BOTH the
summative and formative components of your plan. (Note: This is a good
section in which to use tables (one formative component, one for summative
components that has your question in one column and your methods or
information collection strategies (e.g., surveys, focus groups, etc.) in columns
with an X to designate which method is tied to which question.)

Formative Method
Summative Method

Sample (.5 - 1 page)

Specifies which learners, instructors, and other personnel will participate in the
evaluation. A rationale for the sample sizes should be included. Identify the
samples for BOTH the summative and formative components of the overall plan.

Formative Sample
Summative Sample

Instrumentation (1-2 pages for the description)

Describes ALL the evaluation instruments and tools to be used. Actual
instruments should be included in appendices for review and approval. For the
purposes of these projects, you should include a description of all the
instruments and tools you will use for BOTH the summative and formative
components of the plan. In addition, you need to develop ONE of the
instruments that you would use for your Summative Evaluation and ALL of the
instruments that you will use for your Formative Evaluation. These instruments
should be included in your plan as appendices.
Since you are implementing the Formative Evaluation part of your plan, you need
to include ALL the instruments that you would use.

Formative data collection

Summative data collection
At least ONE instrument for Summative evaluation component and ALL
instruments for Formative Evaluation should be included as appendices
(separate appendix for each instrument)

Limitations (1/2 page (is that .5 pages?))

Spells out any limitations to the interpretation and generalizability of the
evaluation results. It should also describe potential threats to the reliability and
validity of the evaluation design and instrumentation.

limitations of the Summative evaluation component results

limitations of the Formative evaluation component results

Logistics (1-2 paragraphs)

Specifies who will be responsible for the various implementation, analysis, and
reporting aspects of the evaluation.

Summative Component of Plan

Timeline ( page in list/bullet form)

Presents the schedule for implementation, analysis and reporting of the
evaluation.

Summative Component of Plan

Budget (.5 - 1 page)

Clarifies the finances for the evaluation. Personnel time is usually the major cost
factor. Other significant cost factors are travel, data preparation (eg.,
transcribing taped interviews), and document duplication. Unfortunately, many
evaluations lack sufficient funding,

Summative Component of Plan