The document provides guidance on creating effective surveys, with the following key points:
[1] The steps to survey design include determining the problem and research questions, identifying respondents, planning deployment and format, and crafting good questions.
[2] When designing questions, best practices are to ask one question at a time, avoid bias and negatives, and use response scales like Likert scales for nuanced answers.
[3] Resources for survey design include Survey Monkey accounts through Mission College as well as external guidelines on question wording, platform best practices, and compliance.
The document provides guidance on creating effective surveys, with the following key points:
[1] The steps to survey design include determining the problem and research questions, identifying respondents, planning deployment and format, and crafting good questions.
[2] When designing questions, best practices are to ask one question at a time, avoid bias and negatives, and use response scales like Likert scales for nuanced answers.
[3] Resources for survey design include Survey Monkey accounts through Mission College as well as external guidelines on question wording, platform best practices, and compliance.
Original Description:
surveys
Original Title
Creating Effective Surveys All College Day Spring 2018 20180126 Final
The document provides guidance on creating effective surveys, with the following key points:
[1] The steps to survey design include determining the problem and research questions, identifying respondents, planning deployment and format, and crafting good questions.
[2] When designing questions, best practices are to ask one question at a time, avoid bias and negatives, and use response scales like Likert scales for nuanced answers.
[3] Resources for survey design include Survey Monkey accounts through Mission College as well as external guidelines on question wording, platform best practices, and compliance.
The document provides guidance on creating effective surveys, with the following key points:
[1] The steps to survey design include determining the problem and research questions, identifying respondents, planning deployment and format, and crafting good questions.
[2] When designing questions, best practices are to ask one question at a time, avoid bias and negatives, and use response scales like Likert scales for nuanced answers.
[3] Resources for survey design include Survey Monkey accounts through Mission College as well as external guidelines on question wording, platform best practices, and compliance.
All College Day January 26, 2018 Inge Bond, Brian Goo, Erik Hou Office of Research, Planning, and Institutional Effectiveness Session Objectives: • Steps to survey design • Getting your questions right • Response options • Survey resources at Mission • Survey resources, external Steps to Survey Design: Rationale • What problem are you trying to solve? • Ask yourself: Can I take action based on what I learn? • What question(s) are you trying to answer? • Is there another way to get this information? Steps to Survey Design: Respondents • Who has the information needed to answer your research question(s)? • When thinking about your audience: • Consider equity issues – is your group representative of the population? • Are there questions you can ask that will help you to identify pertinent subgroups? • Demographic information • Habits (e.g. drivers versus cyclists) Steps to Survey Design: Deployment • Online (electronic) surveys: • Benefits: respondents can take their time, feel less recognizable in their responses • Drawbacks: harder to gather responses • In person (especially in-class): • Benefits: captive audience – participation is likely to be high • Drawbacks: respondents may not be as candid as they would be in an electronic survey Steps to Survey Design: Survey Format • Explanation and Consent: • Respondents must be competent and willing to answer the questions • Make the purpose of your survey clear to the respondents • Describe your anonymity/confidentiality policies and procedures • Keep it short and simple • Shorter surveys = higher response rates • Arrange questions in a logical order • Incentives • Increase response rates • Pre-test your survey • Have 3-5 colleagues test your survey out to look for grammatical errors, branching issues, and overall clarity Steps to Survey Design: Questions • Make sure that every question is necessary • Identify the data you need to collect, then write your questions • Use language that is simple and familiar to your respondents • Avoid acronyms or other “edu-speak” (know your audience) • Ask one question at a time • Expecting a single answer to a question that has multiple parts – muddies the responses and can confuse/frustrate respondents • and is a red flag of a double-barreled question • Example question: “Rate the effectiveness of Basic Skills STEM courses in student success” Steps to Survey Design: Questions • Avoid leading or biased questions • Look at your adjectives and adverbs – if not needed, take them out • “How much do you like…” • Avoid prestige bias • Some will exaggerate where they think a response is desirable (such as working long hours). • Avoid negative questions • Easily misinterpreted • not, prohibit, impossible are red flags of negative questions Steps to Survey Design: Questions, cont’d • Open-ended questions: • Enable respondents to provide information beyond what a fixed set of response options would • Entail more complex data analysis • Close-ended questions: • Greater uniformity of responses • Enable scaled responses • Simplify data analysis Steps to Survey Design: Likert Scale Questions • Use response scales whenever possible: • Allows for nuance and a more accurate data than binary response • Ideally, between five and seven points is best • In general, providing a middle category provides better data • Only caveat is if you want to make respondents give a non- neutral answer Steps to Survey Design: Likert Scale Questions • If possible, avoid agree/disagree questions, as respondents can be biased toward the “agree” option(s) (alternatives include measures of likelihood or satisfaction) • Example scaling options • https://www.uc.edu/content/dam/uc/sas/docs/Assessment/likert- type%20response%20anchors.pdf • Types of scales • Satisfaction • Awareness • Importance • Support/Opposition Steps to Survey Design: Creating the Survey/Survey Platforms • Survey Monkey – The college has established accounts • Advanced data analysis and survey export • Office365/One Drive Forms – New-ish. Comes with the college’s subscription to Office365 • For surveys of people with WVM id’s this can be a good way to limit participation • Google Forms – Most established open survey platform Survey Resources at Mission College • ORPIE is here to help! • Survey schedule: http://missioncollege.org/research/documents/surveys/Surveys_Scheduled_ 2017_2018.pdf • Avoiding survey fatigue • Instrument review • Survey Monkey accounts: • ORPIE (no public access – used for surveys with high need of confidentiality) • Office of Instruction (access through Patty E.) • Office of Student Services (access through Zita M.) Survey Resources, External: • Harvard University Program on Survey Research Tip Sheet on Question Wording • Survey Monkey survey guidelines • Qualtrics Research Success Kit (requires registration) • FERPA compliance Questions? Comments?