Running head: Experimental and Ex Post Facto Design

PhD in International Management

Submitted by: Sohil Sharma

Horizons University, Paris

PHD 650: Experimental and Ex Post Facto Designs

Facilitator: Dr. Perry Haan.

Date of Submission: 28th April 2019

A Quantitative Ex Post Facto Study for one-to-one mobile technology
2

Table of Contents

Introduction 3

Experimental design in conjunction with history 4

Ex-post Facto in conjunction with history 7

Sample using experimental design 9

General layout for an Experimental Design 10

Example of an Experimental Design Diagram 11

Ex Post Facto Design Paper 12

Overview of the Research Method 13

Research Questions 14

Hypotheses 15

Population 16

Conclusion 17

References 18
A Quantitative Ex Post Facto Study for one-to-one mobile technology
3

Introduction

The aim of this paper is to provide an overview of the principles underlying experimental

research and Ex post Facto Design advantage and disadvantage followed by a case study that

utilizes ex-post factor design for examine the impact a districtwide high school one-to-one mobile

technology initiative. This paper will be focusing on how the design was implemented including

historical foundation with target populations. The objective of this research paper is to introduce

the reader about the different designs and giving examples of when they may be used. To

determine when either experimental or ex-post design is an appropriate research method for a

specific research question. The importance of relationships allows for prediction from one

variable to another which can examine whether one or more pre-existing conditions could

possibly have caused subsequent difference in groups of subjects or comparing two or more

groups on one or more measures. To further the discussion of different research, it is important to

understand the basic terminology. The starting point here is to identify what variable means. A

variable is described as anything that has a quantity or quality that varies accordingly.

Independent variable, the variable that is controlled in a scientific experiment by representing the

cause of an outcome. It may be a characteristic that a researcher assumes we’ll make a difference.

Another key terminology is dependent variable; these variables are expected to change because of

the outcome that are conceived through manipulating the independent variable. The variable

depends on other factors that are measured by exploitation. Experiment group, these are bunch of

people that are for monitoring and investigation which receives the treatment. Last one is Control

group, we use these group mainly to compare against experiment group. It remains same

throughout the research which is then used as a benchmark to measure how the other groups or

tested subject are performed.

A Quantitative Ex Post Facto Study for one-to-one mobile technology
4

Experimental design in conjunction with history

According to Mary (2009), experimental research designs are the primary approach used

to investigate causal (cause/effect) relationships and to study the relationship between one

variable and another. For example, if a cardiology student conducts experimental research on the

effect of eating junk food on cholesterol and conclude that most hear patient have diabetes then

there are aspects (causes) which can cause heart attack (effect). This is a traditional type of

research that is quantitative in nature. History begins when Ronald A. Fisher devolved the idea

and methodology underlying principles of experimental design by joining Rothamsted

Experimental Station at Galton laboratories in the 1920s and 1930s, which was an agricultural

research station. According to Committee on National Statistics (1995), the use of experimental

design methods in the chemical industry was promoted in the 1950s by the extensive work of

Box and his collaborators on response surface designs. According to Box and Draper, 1987, over

the past 15 years, there has been a tremendous increase in the application of experimental design

techniques in industry.

According to Bedard (2016), the first experiments can be seen in early 1747 by James

Lind, while serving as surgeon on HMS Salisbury, Lind conducted experiments on individuals

with scurvy. He took 12 of them and separated them into six pairs. Each pair was assigned to eat

a different diet. Some had to eat garlic, mustard, and horseradish while others had to eat orange

and lemons. Other had anything from cider to seawater. The results showed that individuals with

the fruit diet recovered from scurvy. Experimental Research is one of the methods of quantitative

research methods where a set of variables are constant while the rest of the other set of variables

are measured as the subject of experiment with a scientific approach. If there is a change
A Quantitative Ex Post Facto Study for one-to-one mobile technology
5

happened in dependent variable due to manipulation of the independent variable, it’s considered

to be successful experimental research.

Time plays a vital factor for building connection between cause and effect when comes to

conducting experimental research. There are three primary types of experimental research design

that could varies according to various conditions and groups.

1. True Experimental Research Design: It’s the most accurate form because it works on

the principle of statistical analysis to prove or disprove a hypothesis. It is commonly

used in physical sciences experiments as it has the power to establish cause-effect

relationship within a group/s in true sense. To be complete, there are three factors

which need to be satisfied from random distribution to variable that can be

manipulated and control plus experimental group.

2. Pre-Experimental Research Design: Easiest form of experimental design where the

various group are observed after considering factors of cause and effect. Purpose of

design is to understand if there is a need of further investigation carried out on the

target group/s which could be cost-effective.

3. Quasi-Experimental Research Design: Quasi word indicates similarity, in our case is it

somehow like experimental research but not exactly. According to Bhat (2015), the

difference between the two the assignment of a control group. This research design, an

independent variable is manipulated but the participants of a group are not randomly

assigned as per conditions. The independent variable is manipulated before calculating

the dependent variable and so, directionality problem is eliminated accordingly.

Quasi-research is used in field settings where random assignment is either irrelevant

or not required.
A Quantitative Ex Post Facto Study for one-to-one mobile technology
6

One of the biggest advantages of having experimental research is the investigator have a

stronger hold on variables to obtain desired results which are extremely specific. By providing

better results researchers confidence also boosted with great transferability. It has the power to

derive cause and effect of a hypothesis where result can be applied to various other similar

aspects. Results can be duplicated when the same variable is controlled by others as it provides

high level of control to researchers. Other vital benefits are related to cross-border boundaries,

there is no limit to a specific subject or industry example teachers might use experimental

research to identify if new methodology of teaching is better than the older one or pharmaceutical

companies to determine the feasibility of new products.

On the other hand, disadvantage of having experimental research could be possibility of

producing artificial results which can occur due to lesser human response that is difficult to

measure or controlling irrelevant variable on certain occasion, and it is time-consuming process.

Non-identical life experience can cause the participant to reach to the same situation in a different

and dissimilar way. This research required specific levels of control variable that is why there is

high risk of any human error at some point which can reveal information about other variables

causing failure in the validity of the experiments. Several ethical reasons can hamper experiments

from every getting started such as environmental influence around participants due to extraneous

variables or participant some go along with what they think a researcher wants to see rather than

suppling an honest and truthful response.

Advantages and disadvantages of experimental research conclusion confirm it is useful to

use it, but we need to be careful when controlling the variable to be beneficial. It is easy to be

influenced by internal or external influences so variable should be tightly controlled as results can

be replicated quickly. By taking the above points into consideration, it becomes realistic to
A Quantitative Ex Post Facto Study for one-to-one mobile technology
7

monitor if this research process is workable and sensible for next theory or idea. The main aims

of experimental research are to predict phenomena with suitable study period.

Ex-post Facto in conjunction with history

Sometimes it is impossible control certain variables to scrutinize their future influence on

dependent variable example experimentation is not feasible when we ask the parent to abuse their

children for introducing a new virus. In these scenarios, ex post facto design plays a vital role. It

basically means how we can perform impact analysis on existing data. The first ex post facto

design was invested and use by Giddings’s student Chapin in 1937 in the social sciences field.

According to Kerlinger (1964), ex post facto research defines as: that research, in which the

independent variable or variables have already occurred and in which the researchers start with

the observation of a dependent variable or variables. He then studies the independent variables in

retrospect for their possible relations to, and effects on, the dependent variable or variables. Later,

after couple of other people suggestion and investigation Tuckman (1972) defined the term ex

post facto to be: an experiment in which the researchers examine the effects of a naturalistically

occurring treatment after that treatment has occurred rather than creating the treatment itself. The

experimenter attempts to relate this after-the-fact treatment to an outcome or dependent measure.

The term ex post facto means ‘from what is done afterwards’ where independent variable

is not manipulated which has already occurred. It’s a quasi-experimental study which explores

how an independent variable, present earlier to the study in the participants, influence a

dependent variable. There could be more difficulty in establishing cause-effect relationship if we

compare it with experiments however it is flexible by nature including less costly. Researchers

has little to no control over independent variable buy it less time-consuming to conduct. An
A Quantitative Ex Post Facto Study for one-to-one mobile technology
8

example of non-manipulated independent variables can be age, sex, family environment,

socioeconomic status (SES) etc.

Advantage of ex post facto research is very important where the rigorous experimental

approach is not possible. It is useful to understand the study which says what goes with what and

under what conditions allowing the researchers to explored simple cause-and-effect relationships

and allowing them to avoid artificially in the research process. However, it is not possible to

manipulate necessary factors to establishing the relationship directly with cause-and-effect. This

research provides a sense of direction with the source of hypotheses which have the power to be

tested subsequently by other rigours experimental methods. Ex post facto research is serviceable

when conducting an experiment is unethical, costly, expensive and impractical and where the

independent variable lies outside the boundaries of researcher’s controls.

On the other hand, one of the biggest disadvantages of this research is conclusion are

based on limited sample as researchers need to match groups on key variables forcing them to

shrink various samples. It provides lack of control over randomizing the subject and independent

variable keeping in mind that one cannot know whether the causative factor has been located or

discovered in the approach or not. In simple words there are two major concerns. The first one is

inability for the researchers to manipulate the independent variable; and second, researcher’s

deficit power to allocate the subject randomly to group levels of the variables accordingly. Some

of the difficulties in ex post facto research that is been faced by the researchers are randomization

is impossible with correlation does not equal cause and governing crucial variable is impossible

to isolate. Even after all this, there are various ways through which we can somehow controls in

ex post facto research like at the time of selecting sample that are as homogeneous make sure that

they are as homogenous as possible on a given variable, match the subjects in control group and
A Quantitative Ex Post Facto Study for one-to-one mobile technology
9

in the experimental for the design which is causal-comparative, In design build extraneous

independent variable and use variance technique for analysis.

There are eight stages involved in the design and procedures of ex post facto while

investigating. First step is to define the problem statement and focus on the survey literature, then

in second step we need to state the hypotheses taking into assumptions on which the research

procedures are based. After that the third stage is to select the subject like a sample by identifying

the methods which you will use for collecting the data. Later, in the fourth stage we will identify

the categories and criteria for arranging the data to fit the purposes of the study. In stage five, the

purpose is to gather data as well as discard the data based on factors which are always present in

which the given outcome occurs with those factors that are not always present. Then we have

stage six where we gather data on where the given outcomes didn’t occur even after present of

those factors. Seven stages are to compare two sets of data i.e. subtract stage five from stage fix

to detect the root causes that are responsible for the occurrence of the outcome and the last stage

is to interpret and report finding with analyzing.

Sample using experimental design

To understand the effect of various types of insulation wrapped over a jar that can cause

variance in temperature of water, we have used experimental design for continuous investigation

on this topic. To, progress on experimental design we have drafted an example with the

specification accordingly. The end goal is to measure the result of different temperature of water

in a jar due to insulation wrapped over the jar. Experiments can have various test differences,

which could be carried out by the non-identical experimenter with the starts and finishes factor

that changes while performing the experiments. There are nine steps that we must follow before

moving forward for conducting experimental research specially regarding those topics where you
A Quantitative Ex Post Facto Study for one-to-one mobile technology
10

are analyzing the effect of one entity on another. First and the most important one is to select the

area of your interest. There is no point moving ahead if you are not satisfied with keen interest of

finding the result or outcome that you are expecting somehow from your experimental research.

In our research we have chosen area of interest that is linked to reaction of water temperature.

Then second step is to identify the research problem by establishing question and known

problems. Is it something which is already researched by someone or trying to gather information

regarding the known problems? Then in the next step we need to conduct a literature search

before proceeding to design the experiment. It the most important phase where history is been

thoroughly studied to get prescribe results. The fourth stage is to create IF and Then statement to

construct a hypothesis. Hypothesis helps in establishing more deeper causes and effects of doing

research. Later we must determine the design of the research and the research methods to

proceed. Moving forward we have another vital step to begin conducting the research. Once the

research is ongoing, we must test the hypothesis that we defined at the fourth stage. Last step is to

analyze the data and formulate conclusions.

General layout for an Experimental Design

Title: The effect of (independent variable) on (dependent variable)

Independent Variable: A variable whose variation does not depend on another

Dependent Variable: Opposite of the independent variable.

Scientific Question: To determine the purpose.

Hypothesis: If (planned change in the independent variable), then (predicted change in

dependent variable)

Material user: To determine the different material that can be used.

A Quantitative Ex Post Facto Study for one-to-one mobile technology
11

Level of Independent variable with number of repeated trails: Level 1 – No of trails

(control), Level 2 – No of trails, etc.

Constants: Situation which can fluctuate over a period

Example of an Experimental Design Diagram

Title

Reaction on temperature of water in a jar when different types of insulation is wrapped over it.

Independent Variable

Different types of insulation

Dependent Variable

Temperature of water in jar changes which is measured in degree Celsius

Scientific Question (purpose)

Does types of insulation wrap plays a vital role effecting the temperature of water in jar

irrespective any other violations.

Hypothesis

If we put the water jars below the sub and then tried to wrap with different types of insulation, the

level of water temperature will fluctuate by various amounts.

Materials used

Cotton, Wool, Nylon, No insulation.

Levels of Independent variable and repeated trails

With Wool, Cotton, Nylon: No of repeated trails, three jars

With No insulation (control): No of repeated trails, three jars

Constants

a. Jars are identical.

A Quantitative Ex Post Facto Study for one-to-one mobile technology
12

b. Jars half-filled with water

c. Jars placed directly below sunlight

d. Jars filled with same plastic lids.

Ex Post Facto Design Paper

Investigation of this study was conducted on the parameter of one of the quantitative

researches that utilizes ex post facto design for examined if districts school can improve their

secondary student test scorer which is standardized when provided the opportunity to use one-to-

one mobile technology. This study scrutinizes mobile technology that had on 10th grad students

in the Forney Independent School District (ISD) (Texas) who passed their standardized TAKS

tests during the 2010 academic year. Discussing about historical foundation of this study, the data

were obtained from the Texas Education Agency (TEA) website.

According to Aquil (2016), the study compared the 2009 academic year when 10th grade

students (n=520) did not have one-to-one mobile technology and the 2010 academic year when

10th grade students (n=530) were able to use one-to-one mobile technology as their personal

school and home device. The purpose of this study for the researchers is to understand if there

was a significant relationship between one-to-one mobile technology with the total number of 10th

grade student who were able to pass their standardized Texas Assessment of Knowledge and

Skills i.e. TAKS Test during 2010 academic year. Researchers first gathered macro view to

understand how schools are currently implementing one-to-one mobile technology then focused

specifically on 10th grade student beginning from 2010 academic year.

The background of the problem arises when the U.S. is declining academically when

compared to other developed nations According to U.S. Department of Education, 2010. For

example, standardized test scores in the U.S. education system have been a cause for concern.
A Quantitative Ex Post Facto Study for one-to-one mobile technology
13

Other nations have continued to perform better than the U.S. on standardized reading, science,

and math tests administered to 15-year old globally (OECD, 2010; Schleicher and Davidson,

2013). According to Davies (2011), the U.S. federal government requires public schools to

integrate technology into the school curriculum because current research indicates that

technology enhances the learning process.

Overview of the Research Method

The study confirms a statistically significant difference between the independent and

dependent variable. Th independent variable is introduction of one-to-one mobile technology

whereas dependent variables are no of 10th grade students who passed the TAKS tests for each

individual subject. Researchers used quantitative method because numerical public archival

academic performance data from Forney ISD (Texas) was collected from the Texas Education

Agency (TEA) and it is easy to collect data from previous occurrences. Searcher used chi-square

test to examine the relationship between various variables. According to Creswell (2008),

quantitative research tests objective theories by relating variables that can be measured with

instruments that produce numerical data that can be statistically analyzed. Number of students in

10th grade who passed the standardized tests during 2009 years without one-to-one mobile

technology was consider as control of the study. According to Aquil (2016), A qualitative study

would not have been appropriate in this case because qualitative data takes the form of

generalizations, perceptions, and/or the “why” of the study without concrete numerical data to

support the study. Ex post facto study was appropriate for this research since the researchers

analyzed numerical public archival data to determine how the integration of one-to-one mobile

technology affected the number of 10th grade students who passed the TAKS tests.
A Quantitative Ex Post Facto Study for one-to-one mobile technology
14

Research Questions

Public archival data were utilized for retrieving the examined 10 grade student

information. Below are the research questions that were used by Aquil (2016) during

investigation phase. The research questions were:

1. Is there a statistically significant difference between 10th grade students who

passed the TAKS English Language Arts test during the 2010 academic year

with access to one-to-one mobile technology and 10th grade students who did

not have access to one-to-one mobile technology during the 2009 academic

year?

2. Is there a statistically significant difference between 10th grade students who

passed the TAKS Math test during the 2010 academic year with access to one-

to-one mobile technology and 10th grade students who did not have access to

one-to-one mobile technology during the 2009 academic year?

3. Is there a statistically significant difference between 10th grade students who

passed the TAKS Science test during the 2010 academic year with access to

one-to-one mobile technology and 10th grade students who did not have access

to one-to-one mobile technology during the 2009 academic year?

4. Is there a statistically significant difference between 10th grade students who

passed the TAKS Social Studies test during the 2010 academic year with access

to one-to-one mobile technology and 10th grade students who did not have

access to one-to-one mobile technology during the 2009 academic year?

5. Is there a statistically significant difference between 10th grade students who

passed all their TAKS tests during the 2010 academic year with access to one-
A Quantitative Ex Post Facto Study for one-to-one mobile technology
15

to-one mobile technology and 10th grade students who did not have access to

one-to-one mobile technology during the 2009 academic year?

Hypotheses

There are variable hypotheses included from null (H0) to alternative hypotheses (HA) that

determined technology affect on 10th grade student. According to Aquil (2016), a couple of

hypotheses are given below:

H01: There is no statistically significant difference in the number of 10th grade students who

passed the TAKS English Language Arts test during the 2010 academic year when provided with

one-to-one mobile technology.

HA1: There is a statistically significant difference in the number of 10th grade students who

passed the TAKS English Language Arts test during the 2010 academic year when provided with

one-to-one mobile technology.

H02: There is no statistically significant difference in the number of 10th grade students who

passed the TAKS Math test during the 2010 academic year when provided with one-to-one

mobile technology.

HA2: There is a statistically significant difference in the number of 10th grade students who

passed the TAKS Math test during the 2010 academic year when provided with one-to-one

mobile technology.

Addressing each hypothesis and research question was important because it allowed the

researchers to determine whether there were any associations between 10th grade students using

one-to-one mobile technology and 10th grade students who passed their TAKS tests in Forney

ISD (Texas) during the 2010 academic year. According to Simon and Goes (2013), when a study

does not allow for true experimental research to be conducted, ex post facto research can be ideal
A Quantitative Ex Post Facto Study for one-to-one mobile technology
16

since the hypotheses, cause-and-effect relationships, and variables can be analyzed without

manipulation of the variables.

Population

According to Aquil (2016), The population of this study consisted of 10th grade students

in Forney ISD (Texas) from the 2009 (n=520) and 2010 (n=530) academic years. The P-12th

grade population for the 2009 academic year was 7,413 students and the P-12th grade population

for the 2010 academic year was 7,795 students. The researchers obtained the archival data of the

population from the TEA website. The data consisted of the school district AEIS information

from the Forney ISD (Texas) academic years of 2009 and 2010. For the data collection

individuals were not contacted, so there was no need of informed consent. Data collection for this

study was from public anonymized secondary sources provided by the Texas Education Agency.
A Quantitative Ex Post Facto Study for one-to-one mobile technology
17

Conclusion

As over the topic we have discuss about the purpose of this quantitative, ex post facto

study was to determine the impact on TAKS test scores with implementation of one-to-one

mobile technology. The schools were indirectly preparing students to start using technology to be

on 21st century pace. According to Aquil (2016), The TAKS, ELA (English Language Arts) pass

rate increased by 5% (92% to 97%) for 10th grade students in 2010 when one-to-one mobile

technology was implemented. There are many research questions mainly called as hypothesis

which was used for the study by researchers for example:

 Is there a statistically significant difference between 10th grade students who

passed the TAKS English Language Arts test during the 2010 academic year with

access to one-to-one mobile technology and 10th grade students who did not have

access to one-to-one mobile technology during the 2009 academic year?

Various ways also include supporting the results of chi-square test like by providing

graphical data. Proven graphical picture was provided by searchers as a proof. Question was

answered with the results of chi-square test that was positive. The results of the expected

outcome and actual outcome of all the alternative hypothesis were yes. For example, for the

above research question the graphical data were showing that supports the chi-square statistic

proving The TAKS Math pass rate increased by 18% (65% to 83%) for 10th grade students in

2010 when the schools implemented one-to-one mobile technology. Schools and school districts

can improve their secondary student standardized test scores when afforded the opportunity to

use one-to-one mobile technology in the public-school setting. Results from this study confirmed

the significant role that technology plays in a school district allowing each student to use

technology at any time.

A Quantitative Ex Post Facto Study for one-to-one mobile technology
18

References

Aquil F. Bayyan, Sr. (2016). One-to-one mobile technology and standardized testing: a

quantitative ex post facto study. Retrieved from

https://www.forestoftherain.net/uploads/3/5/8/2/3582998/aquil_f._bayyan.pdf

Bhat, A. (2015). Experimental research- definition, types of designs and advantages. Retrieved

from https://www.questionpro.com/blog/experimental-research/

Bedard, H. (2016). The history of experimental design. Retrieved from

https://prezi.com/3tnggcvonecq/the-history-of-experimental-design/

Box, G.E.P., and N.R. Draper (1987). Empirical Model-Building and Response Surfaces. New

York: Wiley

Committee on National Statistics. (1995). Statistical Methods for Testing and Evaluating Defense

Systems. Retrieved from https://www.nap.edu/read/9074/chapter/13

Creswell, J. (2008). Research design: qualitative, quantitative, and mixed methods approaches.

Thousand Oaks, CA: SAGE Publications, Inc.

Davies, R. S. (2011). Understanding technology literacy: A framework for evaluating educational

technology integration. TechTrends, 55(5), 45-52. Retrieved from

http://dx.doi.org/10.1007/s11528-011-0527-3

Kerlinger, F. N. (1964). Foundations of behavioral research. Retrieved from

https://files.eric.ed.gov/fulltext/ED090962.pdf

Mary, M. (2009). Overview of Experimental Research. Retrieved from

https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/overview
A Quantitative Ex Post Facto Study for one-to-one mobile technology
19

OECD. (2010). PISA 2009 Results: What students know and can do: student performance in

reading, mathematics and science. Retrieved from

http://www.oecd.org/dataoecd/54/12/46643496.pdf

Schleicher, A., & Davidson, M. (2013). PISA 2012 Results: Country Note: United States.

Retrieved from http://www.oecd.org/pisa/keyfindings/PISA-2012-results-US.pdf

Simon, M., & Goes, J. (2013). Dissertation and scholarly research: Recipes for success. Retrieved

from http://www.dissertationrecipes.com/wp-content/uploads/2011/04/ExPost-Facto-

research.pdf

Tuckman, B.W. (1972). Conducting educational research. Retrieved from

http://ww2.odu.edu/~jritz/attachments/coedre.pdf

U.S. Department of Education. (2010, December). Secretary Arne Duncan's remarks at OECD's

release of the program for international student assessment (PISA) 2009 results. Retrieved

from: http://www.ed.gov/news/speeches/secretary-arne-duncans-remarksoecds-release-

program-international-student-assessment-