Running head: EDU 861 ASSESSMENT PROJECT PROPOSAL 1

EDU 861 Assessment Project Proposal

Randy Cloke

Salem State University

ASSESSMENT PROJECT PROPOSAL 2

This assessment will consider the effect of incentivizing students in Salem State First

Year Seminar courses to complete the Fall Mapworks survey, along with how lessened

incentivization, due to lower funding, affects completion rates. It will also provide an

understanding of whether the resources used for incentivization are effective in their intended

goal.

Context

The usage of Skyfactor Mapworks as a retention and early warning platform was

implemented by Salem State University as part of a larger effort to retain students and connect

them with resources that can help them succeed. Also created was the First Year Experience

office, which sought to support first-year students. Housed in Academic Affairs, First Year

Experience helped create First Year Seminar courses which helps to introduce to potential

academic areas of interest, connected faculty, and aids the transition of first-year students to

college generally and Salem State specifically.

Assessment Need

This assessment is valuable as it will add to Mapworks’ continued effort to maintain or

raise its level of funding to meet the need of incentivizing First Year Seminar students to

complete the Fall survey. Given the multi-year long effort Salem State University has made to

learn the student experience to provide assessment of university student services, it is logical to

maintain the funding to provide incentivization to gather as much data from students as possible

to best inform actions to be made and assessment of actions that have been made.

This assessment is being conducted to evaluate the effect that the removal of incentives

for First Year Seminar students to complete the Mapworks Fall survey. Given the shifting level
ASSESSMENT PROJECT PROPOSAL 3

of funding that Mapworks as an office has faced as of the 2017-18 academic year, expected

expenses has necessitated that the office eliminate the purchase of incentives for First Year

Seminar students to complete the Mapworks survey. Given the high level of completion rates in

previous years, it is worth assessing the effect that those incentives had compared to 2017-18,

when incentives have been eliminated.

Purpose

The purpose of this assessment is to help determine whether part of Mapworks’ limited

funding is well spent on incentivizing First Year Seminar students to complete the Mapworks

Fall survey. This assessment will provide evidence for requesting additional funding for the

office to continue or further its efforts to gain as much data from students as possible, especially

from first-year students. First-year students are a critical focus for efforts of retention, and given

that all first-year students are required to take a First Year Seminar, it is logical to explore

options to ensure as high of a completion rate as possible for those courses.

Key Assessment Questions, Goals, and Objectives

The key questions that frame the assessment are numerous. Firstly, a question regarding

completion rates for First Year Seminar courses prior to incentivization is important to set an

initial baseline against which to compare future years. A following question would be to assess

whether providing incentives increased the rate of completion for First Year Seminar courses in

the following years. And then, lastly, a key question is now, in the 2017-18 Fall survey period, if

the lack of incentivization either correlates to a dip in lower completion rates or if there is no

significant decrease in these rates.

Further, it must be asked what the overall cost of incentivization has been in pure dollar

value, comparing that figure to amount of resources from which that dollar value can be pulled,
ASSESSMENT PROJECT PROPOSAL 4

and then using those figures against the differences between completion rates. What is the fiscal

cost? How many students do we reach by providing additional incentives? If incentives were

able to be offered again, how would their effectiveness be measured to ensure they are worth the

resources used to afford them? These questions will be key to framing the assessment going

forward.

Clients and Stakeholders

In this assessment, there would be numerous stakeholders which would benefit from the

assessment’s findings. Chiefly is the Mapworks office, which can use this assessment to

determine whether the cost of First Year Seminar incentivization is worthwhile with an

increasingly limited budget. This assessment would also provide a benchmark against which

similar initiatives used by other offices on campus, including First Year Experience, utilize for

survey completion, whether Mapworks or their own, internal, surveying conducted. The Office

of the Executive Vice-President would also be a stakeholder in the findings of this assessment, as

it would provide additional information that can be used when determining funding allocations

for the Mapworks office with the added factor of the cost of incentivization as part of that

allocation process. Students would also be a stakeholder in the findings of this assessment, as

they are directly benefiting from the immediate issuance of an incentive for survey completion,

but also from the increased understanding of campus resources and giving of feedback to the

university that Mapworks survey completion can provide.

Methodology: Design

In determining whether providing funding for survey completion incentivization for First

Year Seminar students to complete the Fall Mapworks survey, this assessment will provide for

an analysis of whether such funding is well spent on such a measure or does not have the
ASSESSMENT PROJECT PROPOSAL 5

intended effect of greater rates of response. The assessment will center on whether the financial

resources used for survey completion are effective in their intended goal based on the completion

rates measured (Henning & Roberts, 2016, p. 76).

Methodology: Sample

The data sample used in this assessment will be the Fall Mapworks survey completion

rate for First Year Seminar courses in Fall 2014, Fall 2015, Fall 2016, and Fall 2017. The rate

will be specific for each course, and will be compiled together, in aggregate, with completion

rates from all First Year Seminar courses.

Methodology: Collection Methods

The data samples will be collected via the Mapworks online platform, wherein

completion reports will be run for each First Year Seminar which has run during the Fall of the

respective sample year. This process will be done for each Fall semester survey. Each semester’s

data set will also include the number of First Year Seminar courses and the number of students,

in aggregate, enrolled in those courses during the survey period. This method of collection is

direct, and does not involve interviews, surveys, or solicitation of data, whether qualitative or

quantitative, from human subjects.

Methodology: Analysis Methods

Data analysis is entirely quantitative, and will compare the above described data sets

against one another in cohorts of “pre-incentivization”, “with incentivization” and “post-

incentivization.” Including levels of incentivization, this comparison will be a regression analysis

to determine correlation between the inclusion of incentivization and the rates as which First

Year Seminar students complete the Fall Mapworks survey

Timeline
ASSESSMENT PROJECT PROPOSAL 6

Within the next two weeks, I will continue to hone the proposal as described here. While

the topic is set, I may continue to seek advice and additional data points for comparative

purposes. The Fall survey period ends on October 17th, and after this point I will be able to run

completion reports for each First Year Seminar which has run this Fall. I will run the same

reports for previous Fall semesters, including when incentives were offered and prior to the

period where incentives were offered.

In the weeks following, I will also seek literature describing what, if any, links exist

between incentivization and survey completion to further frame the assessment that will be

conducted for this proposal. I will conduct a literature review, if necessary, that will contribute to

the final report.

In the nearest term, I may inquire to see whether it is appropriate to conduct, either in

place of this proposal or including it, additional assessment of the greatly lower incentivization

offered overall for all students for this year’s Fall survey period compared to prior years. This

may not be necessary, however, as further progress is made on this current proposal.

Most critically, it is becoming more readily apparent that I need to do a much deeper dive

into assessment terminology and methods. While I have conducted a light overview previously, I

will successfully complete course or assignment objectives without additional knowledge

attainment. While I do believe that I have a workable and valuable program or initiative which is

worthy of assessment, I will not be able to do this well without furthering my understanding of

assessment procedure.

Conclusion

This assessment is not the most thorough, in-depth, or assessment put forth in any setting.

What it does provide for, however, is important data that should be part of a calculus when
ASSESSMENT PROJECT PROPOSAL 7

Mapworks, as an office, maps out its spending priorities, along with potential bolstering of

arguments of additional funding to improve rates, collect more data, and provide greater

assessment of the Salem State student experience overall. Moreover, it will provide me,

personally, with a greater knowledge of assessment that can be done easily across initiatives and

projects to ensure that what I am attempting to do is working as well as intended or not. Ideally,

this assessment would be done longitudinally with responses from first year students who have

taken the survey indicating their rationale or impetus for completing the survey, whether

compulsory (like being included in a course syllabus), genuine interest in how Mapworks can

help the student individually, or due to the possibility of winning one, or any, of a number of

incentives. This proposal will likely need further revision, but I am hopeful that it will not be in a

place where it is unworkable.

ASSESSMENT PROJECT PROPOSAL 8

References

Henning, G., & Roberts, D. M. (2016). Student affairs assessment: Theory to practice. Sterling,

VA: Stylus Publishing, LLC.