Running head: LANDSCAPE IRRIGATION SCHEDULING MODULE

Evaluation of the Landscape Irrigation Scheduling Module

Michael Johnson

California State University Monterey Bay

IST 622 Assessment and Evaluation

Dr. Bude Su

July 23, 2019

LANDSCAPE IRRIGATION SCHEDULING MODULE 2

Table of Contents

Section I .............................................................................................................................. 3

Introduction ..................................................................................................................... 3

Section II ............................................................................................................................. 3

Methodology ................................................................................................................... 3

Prototype ..................................................................................................................... 3

Learners....................................................................................................................... 5

Tryout Conditions ....................................................................................................... 5

Process ........................................................................................................................ 6

Section III............................................................................................................................ 6

Results ............................................................................................................................. 6

Entry Conditions ......................................................................................................... 7

Instruction ................................................................................................................... 7

Outcomes .................................................................................................................... 8

Recommendations ....................................................................................................... 9

Section IV ......................................................................................................................... 10

Summary ....................................................................................................................... 10

Section V ........................................................................................................................... 11

Appendices .................................................................................................................... 11

Appendix A ............................................................................................................... 11

Appendix B ............................................................................................................... 13

Appendix C ............................................................................................................... 14
 LANDSCAPE IRRIGATION SCHEDULING MODULE 3

Section I

Introduction

The Landscape Irrigation Scheduling (LIS) learning module consists of a mAuthor

interactive educational course designed to train students how to create climate-appropriate

landscape irrigation schedules for the California Central Coast region. The prototype used for

this evaluation is an interactive, multimedia e-learning product with narrated slides and scenario-

based activities that will be accessible asynchronously through the Canvas learning management

system (LMS). Fifteen learners participated in the evaluation of the learning module. The

purpose of the prototype test was to determine the effectiveness of the learning module based on

pre-and post-test scores.

Section II

Methodology

Prototype

The prototype tested is the first part of a series of learning modules that comprise a

comprehensive interactive, multimedia e-learning product designed to guide learners through a

systematic process for creating climate-appropriate landscape irrigation schedules. This

introductory module includes information about irrigation system application rates, soil

infiltration rates, and climate-appropriate plant materials for the Central Coast California region.

The prototype is designed to be accessed through the Canvas LMS via laptop, desktop

computer, tablet, or phone. Learners should complete the learning module in approximately

twenty minutes. The module begins with a brief introduction to current water use issues in
 LANDSCAPE IRRIGATION SCHEDULING MODULE 4

California, including water use regulations, limited water resources, and cyclical periods of

drought to gain learner attention and convey relevance of the topic to learners. Then three main

concepts that are necessary for understanding how to create climate-appropriate landscape

irrigation schedules are introduced: (1) irrigation system application rates, (2) distribution

uniformity of the systems, and (3) plant water requirements (Figure 1). Narrated slides and

interactive activities keep learners engaged while activating short term memory.

Figure 1. Landscape Irrigation Scheduling learning module screenshot.

The LIS learning module will help learners develop a clear understanding of interactions

amongst different soil types (infiltration rates, water holding capacities) with various irrigation

system components (drip emitters, sprinklers, bubblers) while beginning to recognize how

irrigation scheduling decisions affect the soil and plant materials in a landscape. The learners

will be working to learn objective standards, particularly upper limits for irrigation water use in
 LANDSCAPE IRRIGATION SCHEDULING MODULE 5

different scenarios and the consequences for exceeding those limits. Real-world problems are

incorporated in the learning module to tap into the learner’s “innate curiosity about the world and

how things work” (Harasim, 2017, p. 71).

Learners

The Landscape Irrigation Scheduling module was created for students in the Irrigation

Systems Design & Management (HORT 58) course at Cabrillo College in Aptos, California.

Typically, these learners have little or no prior knowledge, skills, or experience with landscape

irrigation scheduling procedures. Many of the students are concurrently enrolled in one to three

other classes in the Horticulture Department. Some of the learners work or have previously

worked in the landscape, nursery, or agriculture industries locally.

For evaluation purposes, learners were chosen to represent a sample population of the

intended audience. The learners were current (Summer 2019) students in the Cabrillo College

Horticulture Department. Six out of fifteen learners indicated that they intend to enroll in HORT

58 during the Spring 2020 semester. All learners were familiar with the Canvas LMS, having

used it in other courses at Cabrillo College.

Tryout Conditions

The effectiveness test was administered under similar conditions as the intended delivery

conditions for the prototype. Learners accessed the learning module with web-enabled devices

including desktop and laptop computers and phones on campus. The participants used a link to

access the LIS learning module webpage that was provided in an email. Ultimately the learning

module will be accessed via Canvas, however that option was not available at the time of the

effectiveness test.
 LANDSCAPE IRRIGATION SCHEDULING MODULE 6

Learners participated in four separate activities during the effectiveness test: (1) pre-test

questionnaire (paper-based), (2) LIS learning module (HTML5-compatible, SCORM package),

(3) post-test questionnaire (paper-based), and (4) satisfaction evaluation (paper-based).

Process

The pre-test questions (Appendix A) were designed to test learners’ prior knowledge,

measure confidence in calculating the application rate of an irrigation system, and address

content delivered in the learning module. The pre-test and post-test questions were the same.

Learners were asked to indicate which, if any, classes they had previously taken in the Cabrillo

College Horticulture Department and if they intended to take HORT 58 during the Spring 2020

semester.

After completing the learning module, learners filled out a post-test questionnaire

(Appendix B) and satisfaction evaluation (Appendix C). The post-test questions were identical to

the pre-test questions to accommodate evaluation of the effectiveness of the learning module and

so a paired analysis could be performed. The satisfaction evaluation was intended to gather

information about learners’ opinions on the ease of use of the learning module and its effects on

familiarizing learners with the process of creating climate-appropriate landscape irrigation

schedules. Statements like, “The learning module…was presented in an easy to use format” and

“The learning module…assisted me in developing skills to create climate-appropriate landscape

irrigation schedules” were used to rate learner agreement on a scale of strongly disagree to

strongly agree.

Section III

Results
 LANDSCAPE IRRIGATION SCHEDULING MODULE 7

Entry Conditions

The effectiveness test was conducted in person with a classroom of fifteen students in the

HORT 72 Organic Food Production course in the same room that HORT 58 is convened.

Learners received an email with a link to the learning module webpage and easily accessed the

LIS module. The entire course of action was intended to take less than 30 minutes and every

learner completed the process within that timeframe.

Instruction

Intended Instruction

The observed instruction was similar to the intended instruction, however there were

some key differences. The LIS learning module is designed to be used asynchronously via the

Canvas LMS, however during the effectiveness test the module was not yet available through

Canvas. Learners will typically use their own devices to view the LIS module. For the

effectiveness test nine learners used their own devices and six learners used desktop computers

in the classroom.

Observed Instruction

All learners were able to follow instructions and complete the LIS learning module

without complications within the expected timeframe. One learner initially experienced a

problem with the volume of the narration while wearing earbuds, but she was able to lower the

volume to an adequate level easily. Another learner did not see the “play” button for the

narration on three separate slides and correspondingly did not interact with that feature on those

slides.
 LANDSCAPE IRRIGATION SCHEDULING MODULE 8

Outcomes

Learning Gains

Learners scored significantly higher on the post-test questions compared to the pre-test

questions, indicating that learners achieved significant learning gains. A paired-sample t-test was

performed on the pre-and post-test mean scores to test the hypothesis that viewing the learning

module would result in a statistically significant increase in the test scores. Based on the t-test

statistic of -9.45929, at df = 14, the critical value (for α = 0.05; one-tailed distribution) is 1.76131

(Table 1). The absolute value of the t-stat is greater than the t-critical value (1.76131) so it was

determined that viewing the LIS learning module resulted in learning gains.

Table 1. Paired Two Sample for Means.

t-Test: Paired Two Sample for Means

Pre-test score Post-test score

Mean 1.466666667 4.266666667
Variance 0.695238095 0.780952381
Observations 15 15
Pearson Correlation 0.109862776
Hypothesized Mean Difference 0
df 14
t Stat -9.459294946
P(T<=t) one-tail 9.25668E-08
t Critical one-tail 1.761310136
P(T<=t) two-tail 1.85134E-07
t Critical two-tail 2.144786688

The pre-and post-test data were collected and transferred to a spreadsheet. Excel was

used to run statistics for the data set. The pre-and post-test scores are displayed in the chart

below (Figure 2). There were three total questions for the pre- and post-test, with a possible high
 LANDSCAPE IRRIGATION SCHEDULING MODULE 9

score of five points. All learners scored higher on the post-test. The mean score for the pre-test

was 1.47 out of five or 29%. The mean score for the post-test was 4.27 out of five or 85%.

Figure 2. Pre- and post-test scores.

Pre-test and Post-test scores

5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Learner Reactions

Overall learners considered the learning module to be an effective instructional tool. Prior

to viewing the LIS learning module eight people, or 53% disagreed or strongly disagreed with

the statement that they were confident in their ability to calculate the application rate of an

irrigation system. After viewing the module, none of the learners disagreed or strongly disagreed

with the statement. Learner confidence levels increased significantly post-test with eleven or

73% agreeing with the statement and three or 20% strongly agreeing with the statement.

Recommendations
 LANDSCAPE IRRIGATION SCHEDULING MODULE 10

Based on the evaluation of the Landscape Irrigation Scheduling learning module, the

following recommendations will be considered:

 Automate the audio narration. With one exception, all learners used the narration

feature. However, several learners suggested that the narration should be

automated to begin at the start of each slide, instead of beginning with pressing

the play button.

 Include more interactive features. Five learners suggested adding more interactive

features, including a video, another scenario-based activity, and more quiz

questions.

 Confirm appropriate order of slides. Three learners suggested rearranging a

couple of the slides to confirm that information contained in the quiz questions

has already been addressed in previous slides.

Section IV

Summary

The research hypothesis for this prototype effectiveness test was that that viewing the

Landscape Irrigation Scheduling learning module would result in a statistically significant

increase in the test scores. The results obtained support the research hypothesis. Learners

reported that viewing the LIS learning module increased their confidence in performing

important tasks involved with creating a landscape irrigation schedule, specifically calculating

the application rate of an irrigation system. The results of this prototype effectiveness test will be

used to develop more effective learning modules for my Capstone project.

 LANDSCAPE IRRIGATION SCHEDULING MODULE 11

Section V

Appendices

Appendix A

Landscape Irrigation Scheduling Module Pre-Viewing

Questionnaire

Your feedback is important! Please answer the following questions. When you have competed
the questionnaire, begin viewing the module. Thank you for sharing your thoughts.

Neither
Before viewing the Landscape Strongly Agree nor Strongly
Disagree Agree
Irrigation Scheduling Module: Disagree Disagree Agree

1. I'm confident in my ability to 1 2 3 4 5

calculate the application rate of an
irrigation system.

Place a check mark beside each course that you have taken in the Horticulture Department at
Cabrillo College.
 HORT 1A Basic Horticulture
 HORT 1B Basic Horticulture: Crop Production
 HORT 2 Soil Science and Management
 HORT 52 Greenhouse Design and Operation
 HORT 54 Business Aspects of Horticulture
 HORT 58 Irrigation Systems Design and Management
 HORT 62 and/or HORT 63 Plant Materials and Design
 HORT 70 Organic Agriculture
 HORT 71 Organic Food Production 1
 HORT 72 Organic Food Production 2
 HORT 100A Plant Propagation
 HORT 125 Hydroponic and Substrate Production
 HORT 150 Pest Management
 HORT 65 Landscape CADD and Surveying and/or HORT 66 Landscape Design
 HORT 164 California Native Plants & Plant Communities
 HORT 172 Arboriculture
 HORT 160B Edible Landscaping
 HORT 176 Permaculture Design
 Other _________________________________________________

I intend to take HORT 58 Irrigation Systems Design and Management course within the next
year (Spring 2020)
o Yes
o No
 LANDSCAPE IRRIGATION SCHEDULING MODULE 12

Circle the best answer:

1. The application rate of an irrigation system is usually measured in

a. Liters per hour
b. inches per minute
c. inches per hour
d. gallons per hour
e. gallons per minutes

2. The application rate of an irrigation system should not exceed the ________________
rate of the soil.
a. Erosion
b. Compaction
c. Application
d. Infiltration
e. Leaching

3. Select the items that you need to know to create an efficient irrigation schedule
a. Lateral line pipe sizes
b. Dynamic pressure of the piping system
c. Application rate of the irrigation system
d. Plant water requirements
e. Elevation change throughout the landscape and irrigation system
f. Distribution uniformity of the irrigation system
g. Friction loss through the mainline pipes

After you have answered these questions, please proceed to the learning module.
 LANDSCAPE IRRIGATION SCHEDULING MODULE 13

Appendix B

Landscape Irrigation Scheduling Module Post-Viewing

Questionnaire
Neither
After viewing the Landscape Irrigation Strongly Agree nor Strongly
Disagree Agree
Scheduling Module: Disagree Disagree Agree

1. I'm confident in my ability to 1 2 3 4 5

calculate the application rate of an
irrigation system.

Circle the best answer:

1. The application rate of an irrigation system is usually measured in

a. Liters per hour
b. inches per minute
c. inches per hour
d. gallons per hour
e. gallons per minutes

2. The application rate of an irrigation system should not exceed the ________________
rate of the soil.
a. Erosion
b. Compaction
c. Application
d. Infiltration
e. Leaching

3. Select the items that you need to know to create an efficient irrigation schedule
a. Lateral line pipe sizes
b. Dynamic pressure of the piping system
c. Application rate of the irrigation system
d. Plant water requirements
e. Elevation change throughout the landscape and irrigation system
f. Distribution uniformity of the irrigation system
g. Friction loss through the mainline pipes
 LANDSCAPE IRRIGATION SCHEDULING MODULE 14

Appendix C

TRAINING SATISFACTION EVALUATION

Your feedback is important! Circle the number based on this scale that most accurately reflects your
opinion. Thank you for sharing your thoughts.

Neither
Strongly Agree nor Strongly
The training I received today: Disagree Agree
Disagree Disagree Agree

1. Assisted me in developing skills to 1 2 3 4 5

create climate-appropriate
landscape irrigation schedules.

2. Was delivered at a comfortable 1 2 3 4 5

pace so I could understand the
content.

3. Provided useful information. 1 2 3 4 5

4. Incorporated learning activities that 1 2 3 4 5

were engaging.

5. Was presented in an easy to use 1 2 3 4 5

format.

6. Improved my confidence in my
ability to calculate the application
rate of an irrigation system.

7. Helped me become familiar with 1 2 3 4 5

the process of creating a climate-
appropriate landscape irrigation
schedule.

What did you like best about the learning module?

What should we consider doing, or not doing, again in this learning module?