Farnoush H.

Davis

Fall 2010
EDTECH 505-4172
Reflection
If I want to highlight the most important things I have learned about evaluation, I
would mention data analysis, the evaluator’s program description (EPD), and the
difference between research and evaluation. Learning about evaluation and how
its role is different than research I found evaluation to be a more interesting
process than just a test or final exam; it is a constant process to examine your
target program in order to reject or revise it. Data analysis is interesting and
important to me because I think it is the constructional step of any evaluation. The
conclusion you will draw for your evaluated program is based on the data we
gather. Therefore, unreliable data would result in a wrong decision.
 
Data collection is a key step in language learning to find what should be taught
and how. And evaluation helps to find if the objectives have been met.
Rationale for Evaluation
The reason for evaluation is to collect data and - based on the goal set of a
program, project, or product - make a decision about choosing or improving what
is being evaluated, to see whether the program is still aligned with the objectives.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/cYJkuG
Evaluation and EdTech
In educational technology, evaluation plays a more important role because it is
the best tool for instructors to stay aligned with their educational goals and
objectives. In long distance learning, evaluation replaces the gap of physical
attendance to provide information on instructional progress and condition.   

Credit to: http://bit.ly/cHQ9ok

Research and Evaluation
Evaluation tries to answer specific questions in a limited situation and only for
specific purposes; however, research is a broad study on a general question with
no specific right/wrong answer. Research is often intended to generalize results
and expand them to similar activities. 

Credit to: http://bit.ly/aoBslo

Efficiency, Efficacy, and Impact
Evaluation tries to answer specific questions in a limited situation and only for
specific purposes; however, research is a broad study on a general question with
no specific right/wrong answer. Research is often intended to generalize results
and expand them to similar activities.
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/dibwXh and http://bit.ly/drz3Qq 
Impact of Evaluation
Decision-making is the main reason for conducting an evaluation. An appropriate
evaluation process collects sufficient data to see if the program’s objectives still
meet the main goal and are achieving desirable results. Evaluation has a direct
influence on decision-making as a final step to reject or accept a program.    
 
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/bqlm8T
Decision-Making Model
This model does not concentrate on the current performance of a program, but
instead focuses on the outcomes to decide the future of a program. Since the
purpose of this model is to make future decisions, both quantitative and
qualitative data can be useful to the decision process.
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/b3LN2R
Goal-Based Model
The focus of the goal-based model is not on the program itself but on the
participants. It asks indirectly if the program was effective, by asking if those who
are involved in a program have changed, learned, or grown.
 
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/dx1Y2a
Quantitative vs. Qualitative
Quantitative data focuses on numerical quantities and measurements while
qualitative data focuses on reasoning, logical thought, and perceptions. Both
present different types of information, however, there are similarities in a way they
are collected, analyzed, and used.
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/bbTVce and http://bit.ly/cpPMd8
Levels of Data

Nominal is when data ordering is arbitrary. Ordinal is logical ordering to a set of

scales from one extreme to another. Interval is a continuous data where the
distance between two scores on a scale is always equal with no absolute zero.
Ratio is a continuous data where two differences can be specified with an
absolute zero in the scale.

Credit to: http://bit.ly/9cawpm , http://bit.ly/cxfpNR , http://bit.ly/c2HKlt  http://bit.ly/c7odny

Data Instruments
Interview collects data through asking and answering specific questions.  Scales
are measure values in terms of numbers, words, or feelings. Sentence completion
is a tool with different variations which is used to measure communication by
having the respondents complete a sentence or solve a problem from a list of
possible choices. Tests are standardized in values to provide numerical results.
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/cif6x5 , http://bit.ly/d80USC  , http://bit.ly/cpBYLx , http://bit.ly/cTod2R
Formative vs. Summative
Formative evaluation tries to find out if the results meet the goals of a program in
order to improve or change the process and put it back in line. However,
summative evaluation is to find out the results, numbers, and outcome of a
program.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/dqtTdb
 Samples and Populations
Populations are groups of individuals which can be large or small. Samples are
actually small portions and parts of a population.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Credit to: http://bit.ly/dBtzRS
 Validity and Reliability

Validity is the extent that a concept is in alignment with what is understood and
what is shown to be reality - or accepted reality. Reliability is the degree to which
something can perform its given function under normal circumstance. 

Credit to: http://bit.ly/bU9lqj

Independent and Dependent

The dependent variable is the variable or value that is the result of the change in
the independent variable. The independent variable is the variable or value that is
being changed. 

Credit to: http://bit.ly/bmE6U5

Criterion-referenced vs. Norm-referenced
Norm-referenced tests compare the results against the cumulative results of
those who have taken the test before.  Criterion-referenced tests do not rely on
just a cut-off score, but are an assessment of one's knowledge in a particular
subject area.  

Credit to: http://bit.ly/99xqV6

Activity Evaluation
This was my second time completing a "visual metaphor" activity. This is a good
practice to look at things and think of their meanings from different perspectives.
For me, this type of activity is more effective than memorizing a list of terms,
because it helps to visualize concepts and ideas that are a part of the meaning of
each term. 
 
I think this particular “test” is a good example of how to put into practice the
concepts of evaluation which we have been learning through the last few weeks.
Visual metaphor activities are effective for those learners, including me, who tend
toward visual learning to understand the concepts better. It was interesting that
we were given an exercise to relate and learn concepts using abstract images,
and were then asked to evaluate this activity with using the same concepts we
had just learned. I suppose the question of its effectiveness is if we can use our
new knowledge to see the value of exercises like these.