You are on page 1of 77

eBook Shahid Hassan

Design and Development of Survey Questionnaire Using Likert Scale


From Validation to Usability

What is in the eBook Authored by Shahid Hassan?


Design and Development of Survey Questionnaire Using Likert Scale:
From Validation to Usability
No The current eBook is designed and developed to,

1 Provide readers a direct reading option to download PDF access with podcast message and in-
built media message.

2 Address the research issues more often received as consultancy request, on validation of
questionnaire albeit, a newly created or borrowed from literature questionnaire to modify. I
experienced learners’ gaps in knowledge in this field and prompted to write this eBook that
will serve as resource learning material to follow through Micro-credential course.

3 Describes, what do learners need help with, or something that they need more to know
information on validation of questionnaire-survey research.

4 Build an inside map-out of eBook provided with condensed and yet within the need assessment
of readers described in an attractive back-to-back 7 chapters with clear, colorful and engaging
figures, tables flow chart and appendix.

5 Explain step-by-step procedural knowledge to implement just acquired declarative knowledge


and skills accomplished to practice tasks provided in exercise section in the end.

6 Develop readers perception in creating of questionnaire items in designing a Google form, face
and content validity index (CVI), pilot to full case study, reliability process and principal
component analysis (PCA) in a reader-friendly learning environment.

7 Adopt to microlearning style in dealing with whole validation process into chunking-in easy
steps of developing validation skills such as coding of raw data, recoding of negatively
connotated items, cleaning of data from unengaged respondents, missing values, transfer of
data from Excel to SPSS worksheet.

8 Readers, factual knowledge in research and design of research has also been covered with just-
on time relevant to know content on research type with better understanding of Likert scale in
a questionnaire survey

9 Discuss in addition to analyze and interpret data, simply based on outcome parameters of mean
and standard deviations, towards its usability of questionnaire outcome data.

10 Promote research and publish besides, additional responsibility to teach and assess a faculty is
entrusted during the Covid pandemic is explored in current eBook specially design to meet
the demand of online research marked with restricted and controlled movement with its impact
on education.

Shahid Hassan 11 March 2022


DEDICATED TO

MY FATHER

Syed Alaul Haque


PROLOGUE
It gives me an immense pleasure to write an introductory note for this eBook written by Prof.
Shahid Hassan on, “Design and Development of Survey Questionnaire Using Liker Scale:
From Validation to Usability”.

The merits of sample survey have allowed for it to become one of the prevalent social science
methods for conducting research across varying fields. The extensive and expanding literature
on the topic is an indication of its popularity, application, and improvement. However, there
may not be enough published research or sample questionnaires to explore some novel and
unique research issues educators have likely experienced in a fast-changing learning
environment due to Covid-19. This book specifically looks to offer insight into one such matter
by looking at online surveys using Likert-scale.

The book comes as a solution for those academicians who apart from teaching and assessing
students had additional responsibilities of researching and publishing amid unprecedented
challenges of the covid pandemic. One of the results of the many trials of the pandemic for
academicians saw an increase in validated questionnaire to collect data to conduct research.
For those unaware, validation is an integral part of a newly designed questionnaire. It is
mandatory to test the questionnaire for validity and reliability before it is used for data
collection. Validation is even required for those authentic and published questionnaires used in
a different socio-culture environment and population. Thus, while its importance is apparent it
remains a skill that isn’t considered a mandatory acquirement by researchers. As a consequence
of this gap, the author feels it is the right time for researchers in education to develop expertise
in designing and developing their own validated questionnaire instead of relying on those
already in circulation. The eBook will teach readers how to create a validated questionnaire
from items design to validation and focus on its usability in research. The emphasis is to train
readers to develop and validate items in a questionnaire-based survey. Visual aids in the form
of flow chart have also been developed by the author to summarise the entire process of
validation from creation to usability of questionnaire. Special feature of this eBook is the series
of fictitious data created and utilized in the generation of output tables that assist in explaining
content validity and reliability indices. Principal Component Analysis output matrices as an
essential part of questionnaire validation is also demonstrated and discussed.

This eBook will provide comprehensive knowledge and skills for those interested in the
validation process in designing a questionnaire. The acquirement of such a skill will not only
streamline the implementation of their novel research ideas and render the process seamless
but also reduce the frustration that comes with relying on the limited pool of available
questionnaires. Readers will find this book worth reading and appreciate the step-by-step
procedural guide. It is my opinion that the current eBook will provide readers all relevant
knowledge and the skill required to create a validated questionnaire for survey research.

Prof. Dr. Abdul Aziz Baba


Vice-Chancellor and CEO, IMU Education; President, IMU Health, International Medical
University Malaysia.
I
FOREWORD

In the questionnaire lies the heart of the survey. The questionnaire is the source of the collected
data on which the survey results depend crucially. It is vital that questionnaires be constructed
in a manner to minimise response errors. This necessitates that the researcher be knowledgeable
of all the possible outcomes of the many decisions they take in survey design, the impact these
decisions can have on the quality of the questions, and subsequently the responses of the
participants. Therefore, it comes as no surprise that the ‘art’ of designing and validating a
questionnaire has become the domain of ‘survey specialists’ and many researchers opt to avoid
the perilous path of designing their own and adapt one that is already in circulation. This book
is written with the intention to remedy this. It serves as an aid (and motivator) for researchers
(undergraduates and beyond) in both, designing and validating a questionnaire best suited for
their survey. It addresses, both typical and varied issues faced by students during their research.
The author with his over 20 years of experience in research consultancy and medical education,
has further sifted these issues to ensure they are relevant to current times. This ensures a concise
yet fruitful involvement from the reader.

The author to the book is an active teacher and researcher who has made significant
contributions to both academic and clinical field in medicine. Knowing the author for many
years since he joined University Sains Malaysia, when I was the Dean back around in 2000.
His journey through Surgery to Medical Education, which he formally qualified with MHPE
from Maastricht University has benefitted many medical Schools in Malaysia. Another salient
feature of the book is that it builds towards the quintessential problems faced by researchers in
a layered fashion where the complexity of issues rises as the reader delves deeper into the
chapters. So while the primary motive of this book is to deal with the problems faced by
researchers this has not deterred the author from attempting to give the reader a comprehensive
experience by dedicating the first 2 chapters to providing a concise review of the types of
researches there are and the steps involved in conducting them.

Most importunately the book explains all the processes involved in designing a questionnaire
to the point of validating it in a step by step manner. This breakdown of information into smaller
chunks creates a stress-free experience for the reader and is the next best option to having a
teacher explain in real time. The author seamlessly addresses issues faced in and the skills
needed to understand topics like face and content validity, reliability process, principal
component analysis and questionnaire usability in the same step by step process ensuring a
reader friendly environment. The use of clear, colourful flowcharts, step by step screenshots of
google forms and tables has been adopted to ensure an engaging experience wherever possible.
Furthermore, electronic components have also been added to the digital experience of the book
by including podcasts, inbuilt media message and video links where deemed appropriate.
Having known the work of the author, both clinically and in medical education for the last 20
years, I have no doubts this content will deliver what the title promises in the most efficient
manner. It is my hope and expectation that this book will provide an effective learning
experience and referenced resource for all researchers in validating a questionnaire.

Prof. Dr. Zabidi A. M. Hussin


Pro Vice-Chancellor Academic, International Medical University Malaysia.ii
PREFACE

It is crucial for the faculty in teaching institution to research and publish during Covid
pandemic besides, teaching and assessing in medical education. It is an opportunity for faculty
to conduct questionnaire based research and learn about survey based research design and its
validation. It has been found that most of the researchers rely on published questionnaire
though, designing a questionnaire is a good research practice to start with. It may pay off the
researchers in many ways in future. Keeping with researchers difficulties that they may
encounter in creating questionnaire-based research and its usability, author is prompted to write
this short book and share his knowledge and skills with those interested to learn, how to
produce validated instruments for survey-based research. The objective of this eBook is, to
learn how to create a validated questionnaire from design to validation process and its usability
in research.

Emphases on questionnaire-based research in this eBook is on creation of items in a


questionnaire using survey design and its validation using a fictitious data. Various chapters in
this book will discuss the entire steps of validation of questionnaire-based research and its
usability in detail. A flow chart developed in the process portrays the entire process of creation,
validation and usability of questionnaire. The author has comprehensively described, how to
create a questionnaire based survey-research and its validation in detail, which may help
readers and new researchers to understand descriptive type of research using survey design,
specially explored for faculty interested to develop their research skills during Covid-19
pandemic and share their experiences with global community.

Writing this eBook author also intend to complement the micro-credential course subsequently
will be developed on validation of questionnaire based survey using the Likert scale. Validation
of instruments in survey research is fundamentals of research that every researcher should be
well conversant in research methodologies. A sudden move from on-campus to emergency
remote teaching and assessment has change the entire perceptions of clinical education in the
eyes of students and faculty and raised many questions that needs to be researched through
simply exploring the perception and many other untoward effects of Covid-19 pandemic on
teaching and learning and its assessment.

A validated instrument refers to a questionnaire and its scale that has been available often in
literature and are designed and developed to be administered among the intended respondents.
If no existing questionnaires can be found in literature in a fast changing situation during Covid
pandemic, or those not found adequate to explore a research question, it is appropriate to
structure a new questionnaire and contribute to literature. It is mandatory to test
the questionnaire for validity and reliability including those validated instruments, which
were validated in a different socio-culture environment and population.

Prof. Dr. Shahid Hassan Author


IMU Centre for Education (ICE) and School of Medicine, International Medical University,
Malaysia.
ACKNOWLEDGEMNT

I would like to thank my wife Dr. Gauhar Hassan for her all-out support in helping me to
compile this book and without her support it wouldn’t have been possible. Besides, her physical
support, she also bearded with me in my long hours working to have accomplished writing
this book in stipulated time. Her big support was extremely important for me being in locked
down during the Covid pandemic and needing support to help use technology in developing
podcast and media messages and to move forward smoothly to run teaching and learning and
research activities.

My further thanks go to my three daughters namely Sabika Gul Hassan, Rabika Gul Hassan
and Nabiha Gul Haasan for their moral support and encouragement to continue my good
writing practices and their on-time support to review draft writing as and when asked. During
all this while we have been departed from each other and have not been able to meet due to
Covid-19 travelling restriction in 2020 and 2021. And special thanks to Rabika for her time
given to proof read and technical help in digital creation of this book.

My thank also goes to Mr. Hasnain Zafar Baloch in motivating me to consider publishing iBook
and developing micro-credential course on this book topic. His support was just-in-time in
acquisition of eLearning tools skills associated with emergency remote teaching and learning
and assessing.

Last but not the least my students in School of Medicine, who chose to research validation of
newly created questionnaire-based survey through a Joint Research and Ethic Committee
approved research proposal on students’ perceptions of online teaching and learning during the
Covid-19 pandemic. It was their interaction with me during the supervision of research
proposal that motivated me to document my knowledge and skills on validation of
questionnaire as my 1st iBook to publish and share with the global community.
MEDIA MESSAGE
Link: https://youtu.be/DJhGuOy9vhI

PODCAST Message

Link: https://anchor.fm/shahid-hassan7
Table of Contents
Design and Development of Survey Questionnaire Using Likert Scale ................................. 2
DEDICATED TO ................................................................................................................... 3
PROLOGUE......................................................................................................................... 4
FOREWORD ....................................................................................................................... 5
PREFACE ............................................................................................................................ 6
ACKNOWLEDGEMNT .......................................................................................................... 7
MEDIA MESSAGE ............................................................................................................... 8
PODCAST Message............................................................................................................. 8
CHAPTER NO: 1 ................................................................................................................ 11
INTRODUCTION ........................................................................................................................11
CHAPTER NO: 2 ................................................................................................................ 16
TYPES AND DESIGNS OF RSEARCH .............................................................................................16
CHAPTER NO: 3 ................................................................................................................ 21
DESIGNING AND DEVELOPING QUESTIONNAIRE ITEMS ............................................................21
CHAPTER NO: 4 ................................................................................................................ 25
CREATING ONLINE SURVEY FORM .............................................................................................25
CHAPTER NO: 5 ................................................................................................................ 37
VALIDATION PROCESS ...............................................................................................................37
CHAPTER NO: 6 ................................................................................................................ 42
VALIDITY AND RELIABILITY OF THE QUESTIONNAIRE .................................................................43
CHAPTER NO: 7 ................................................................................................................ 47
CONSTRUCT VALIDITY PRINCIPAL COMPONENT ANALYSIS ........................................................47
CHAPTER NO: 8 ................................................................................................................ 51
QUESTIONNAIRE USABILITY ......................................................................................................51
FIGURES ........................................................................................................................... 54
Validation of Questionnaire Based Survey Using Likert Scale: Figures .......................................54
TABLES............................................................................................................................. 57
Validation of Questionnaire Based Survey Using Likert Scale: Tables ........................................58
APPENDIX ........................................................................................................................ 63
Validation of Questionnaire Based Survey Using Likert Scale ....................................................64
REFRENCES ...................................................................................................................... 65
INDEX .............................................................................................................................. 67
TAKE-HOME EXERCISE...................................................................................................... 69
TAKE-HOME EXERCISE...................................................................................................... 70
ABOUT AUTHOR .............................................................................................................. 76
CHAPTER NO: 1

INTRODUCTION

In an unprecedented challenging situation of Covid-19 pandemic that we all have been facing
in delivery of education globally, teaching faculty has the additional responsibilities of
researching and publishing besides, teaching and assessing in medical education [1]. Multiple
subject areas are opened for research in teaching and learning and often new researchers are
inclined to perform questionnaire-based research using the inventories borrowed from
published literature. However, researchers find difficulties in establishing suitability of those
inventories in a changing socio-cultural environment and other associated issues. The problem
further mounts for researchers when those established in literature questionnaires are to be
modified to determine the variability of the scale inventory for a targeted population. Instead
of research conducted using established questionnaire that often has the issue of granting
permission from the authors or journal administration, a teacher might research to establish his
own validated survey instruments. Keeping with researchers incentives, knowledge and
difficulties encountered to create questionnaires and conduct research using questionnaires,
author is obliged to write this paper and share his knowledge and skills with global community
in learning how to produce validated instruments for survey-based research.

Validation of a questionnaire created for survey-based research involves multiple steps from
its inception of creative idea to design, development and validation of questionnaire. Validation
process of a questionnaire is a standard procedure (see figure 1) that must be followed through
in a questionnaire accepted for valid research and data collection. This model is used here to
describe each aspects of validation in a friendly user manner to encourage researchers to take
this initiative of producing validated instrument for research and sharing of knowledge during
the Covid pandemic. Authors got involved into this situation after a number of consultancy
sessions with faculty seeking review of their research proposals based on published
questionnaire with a number of issues not realised by researchers. This may include authors or
journal permission, suitability for targeted population, revalidation, analysis outcome and
interpretation of data obtained from those questionnaire-based survey.

In survey based research data is collected using a questionnaire with multiple items measured
either on a dichotomous option of, “Yes or No” or on a Likert scale from, “1 to 4”, where 1
represents strongly disagree, 2 disagree, 3 agree and 4 represents strongly agreed (see
appendix). The original Likert scale is a set of statements (items) offered for a real or
hypothetical situation under study [2]. Participants surveyed are directed through an instruction
to indicate their level of agreement (from strongly disagree to strongly agree) with every
question (items) on a metric scale. The statements (items) in combination reveal the specific
dimension of the construct The statements (items) in combination reveal the specific dimension
of the construct inter-linked with each other [3].The issue here may arise, how to quantify these
subjective preferential choices in a validated and reliable manner and that is what has been
offered by a questionnaire based survey designed with a Likert scale [4, 5].

Preparation and implementation of a quantitative questionnaire involve multiple steps and rules
that must be carried out meticulously6. The first important step is to define the survey objectives
which gives us the insight of the information we inquire to collect and give a sense of purpose
to the survey [6]. Other than that, validation of an instrument is one of the important steps in
questionnaire development [6].
Validity of a questionnaire can be explained as the extent of the questionnaire measures what
it intends to measure [7] while reliability refers to the scale of the developed questionnaire able
to be reproducible [8]. In questionnaire validation, the validity is achieved through face and
content validity where a group of experts involve in reviewing and analysing the questionnaire.
The purpose is to assess whether the items of each domain are practical, suitable and applicable
to the intended target population in this study [8]. Meanwhile, reliability of this questionnaire
is established by using a pilot test where the data collected is analysed using SPSS software.
This is to measure the internal consistency of the questionnaire to know the degree of the items
in the questionnaire are measuring the same thing for each respondent [9].

The process of creating a questionnaire using a Likert scale primarily will require validation in
terms of content validity index (see figure 2) and a factor reduction technique in order to
identify relevant items in relevant factors (construct) called factor analysis or more precisely
Principal Component Analysis (PCA). This process analyses the data for initial extraction and
subsequent rotation to identify factors with their respective items, which may not be the same
a researcher might have created. To get started with selecting a rotation method one has to start
with factor analysis procedure for our all the items in questionnaire designed, either as one
factor or assigned to different factors with certain number of items in each factor. Current paper
is an effort to provide an easy training to those interested in questionnaire based survey
research. The process of creating and validating a newly created questionnaire are described as
following.

Let’s begin with research design for its three basic flavours of being exploratory, descriptive
and explanatory depending on intent of researcher. The difference in three types are:

1. Where one aims to explore in an area that one knows nothing or have little knowledge.
2. Where one wants to describe further with accuracy about something one already knows. 3.
Where researcher wants to explain causal/predictive relationship among the variables. Under
these three types of research we may have research design called experimental, survey,
comparative, case study, observational, action research or mixed method research. Survey
research falls primarily under descriptive research. The basic idea in this type research is to use
or create a standardised instrument to collect data from a large number of respondent using
interview-based, internet-based or questionnaire-based survey. Survey research is used for
descriptive purposes although implication of this kind of research may be causal or predictive
however, process of research is descriptive in nature.

After accomplishing the literature search relevant to subject area, researcher may start writing
the items that covers every aspect in terms of content of the questionnaire for survey. First thing
is to think of design of survey questionnaire based on the objective. If the objective is to gather
the data for descriptive analysis, it will basically require to look at mean and standard deviation
or percentage/proportion for data interpretation of items in different constructs depending on,
whether questionnaire is one or multiple factors survey.

Validation of a Questionnaire-based Survey Using a Likert Scale:

Validation of questionnaire-based survey is often taken lightly by the researchers unless queries
come from the reviewers and validation of questionnaire is seriously and effectively considered
important. This is not an easy process and needs strategies to be set in the beginning rather than
waiting for suggestion to validate questionnaire in your survey. A questionnaire due for
validation is not the one created first time by the researcher but it is also required to validate a
survey borrowed from literature and either modified or translated to different language. One
should not wait to add validation to study when urged by the reviewers. In social sciences and
medical education research it is very difficult to find a faculty trained in validation of
questionnaire to help validate the survey items. Author here has compiled a comprehensive
strategy to validate survey questionnaire and share with community interested in questionnaire
-based survey (see figure 2, the flow chart).

What to Validate in a Survey Questionnaire?

A survey is one of the research designs in quantitative or qualitative research that often is used
descriptive purposes however, can also be used for inferential statistics depending on nature of
data collected and scope of the research. A Likert scale used in questionnaire is a psychometric
scale is most widely used scale in survey research often used interchangeably with rating scale.
Validating a survey refers to the process of evaluating a survey items for its validity and
reliability. This needs a step-by-step process, which should be meticulously done to be
defensible and credible for the entire research will depend how dependable the survey questions
were.

Why to Validate of a Survey Questionnaire?

As indicated an inventory or questionnaire survey needs validation to ensure that instrument


is measuring what it is supposed to measure. All newly created or modified questionnaire
survey must therefore be validated prior to its use to ensure the reliability and validity of survey
research to obtain relevant information in terms of consistency and accuracy of survey.

When to Validate a Survey Questionnaire?

Any survey needs the face and content validity and reliability to be ensured to view the
respondent’s opinion has measured what it was intended to measure. Any research
questionnaire whether created or borrowed from literature and modified or translated to a
different language should fully or partially be undertaken for validation process for research
output to be trust worthy. A good survey based on questionnaire should have content validity
in the eyes of experts and reliability measured through statistical test and that it covers all the
aspects of constructs being measured.

How to Validate a Survey Questionnaire?

Author recommends a seven steps validation process that will help both, those new in survey
research or those intended to create inventories or questionnaires as part of their data collection
instrument. After having created the questionnaire with or without the identified constructs in
a survey or one modified from literature and used to collect data following seven steps are
mandatory to practice as step-by-step method of validation of survey.

Step 1. Ensuring the Face and Content Validity:

This is going to be a two-step method that will involve first, at least 2 experts one from the
field of expert and other an experience person in constructing the items questionnaire survey
to advise on overall dependability in terms of face validity. Secondly a panel of subject experts,
educators, researchers or statisticians as the need may be. Content validity should look into
clarity and relevancy of items to the theme or constructs if items are divided into
initial extraction and subsequent rotation to identify factors with their respective items, which
may not be the same a researcher might have created.various components or factors. A detailed
method of calculating the Content Validity Index (CVI) is given in the relevant section of this
book. Both in face and content validity items with negative connotations, guided, confusing or
more than one object in a question asked are taken care of, as reviewed by the experts. initial
extraction and subsequent rotation to identify factors with their respective items, which may
not be the same a researcher might have created.

Step 2. Limited Case Survey for Pilot Study:

A pilot test should be run taking at least 10% of expected subject in intended survey in ultimate
research to identify language, grammar and clarity and understanding issues related to original
survey items. Participants in pilot test should be those from the institution and or
Alumni. Data collected from the pilot study should be reviewed by the experts involved in
research. This process will fine tune the questions for its weakness and relevancy and their
placement in to the relevant construct if survey has used multiple constructs or components.

Step 3. Full Case Survey for Embarkation Study:

Another case study for validation of survey involves a sample data calculated in keeping with
factor analysis assumptions of ideally 10 subjects per item however, a minimum of 5 subjects
per items are also be allowed and acceptable. The data collected after administering the
questionnaire to a sample size calculated accordingly are used for reliability tests and factor
analysis using Principal Component Analysis method. However, prior to establishing the
reliability primarily using Cronbach’s alpha and PCA data is subjected to cleaning for a number
reasons in step 4 below.

Step 4. Data Collection and Cleaning:

In data cleaning, a number of procedures are carried out after developing a spread sheet both
on Microsoft Excel and SPSS step-by-step. Data entry is carefully done by at least a couple of
researchers involved in study to have double check and minimizing the errors. After the work
sheet first developed on Microsoft Excel as the raw data, first we do the coding of data into its
numeric values as in Likert scale. Next step in data cleaning is to perform the recoding of items
with negative connotations. After this cleaning of data of unengaged respondents is checked
out for those participants who casually respondent the questionnaire by selecting one option
for all items and this is likely when respondents have moral or compulsive pressure to respond
in case study step 3 of the validation process. At this point one may also check for maximum
or minimum if the survey was administered to respond using 1-4/5 point Likert scale any data
point with number 6 or zero for example should be cleaned from the data by removing them.
Yet another step in cleaning the data is to find the missing values and that one is best done on
SPSS after having transferred the data to create SPSS data sheet. Once the data is transferred
to SPSS it is analyzed for normality of distribution before reliability is checked using couple
of methods in SPSS.

Step 5. Run Principal Component Analysis (PCA):


Principal Component Analysis is one of the methods of component analysis that is used in
validation process to identify the underlying components or construct (factors) being measured
by the survey items. Factor analysis is about how the items are divided in to various component
based on the similarity to represent the same construct called factor loading in which same
items should load into the same factor. Factor loading scale runs between -1 to 1.0 and strong
loading of an items has a factor loading of 0.6 and above. In PCA items should have a
correlation of 0.3 and above. Below 0.3 factor analysis may not be valid. If a question is not
loaded into any factor and this might happen then such an item should be removed from the
questionnaire. The process of defining factors in the statistics is referred to rotation. Using
rotation loading pattern becomes clear and easy and a more pronounced.

Step 6. Check Internal Consistency of Items

Cronbach’s alpha is a good test of reliability for the internal consistency of items in a scale in
questionnaire survey. Closer the internal consistency of Cronbach’s alpha coefficient is to 1.0,
the greater is the internal consistency of items in a scale. Another way to look at reliability is
the correlation coefficient of items in a factor that ensures that the survey items are consistent
and measure the same construct indicated by a factor loading. Checking the correlation with
Pearson’s or Spearman’s test depends on, whether the data is normally distributed or not
respectively.

Step 7. Revise Questionnaire for Reformatting:

Ultimately the PCA and reliability coefficient alpha is used to revise the survey based on
information gathered from reliability alpha and principal component analysis. I f we find an
item that do not load onto any factor can be removed from the questionnaire however, if the
question is an important one, it is not deleted and can be retained as an entity and analyzed
separately. After readjusting the questionnaire based on the findings from the PPCA and
reliability and other minor changes, the questionnaire can be considered dependable for
administration to collect data in a survey research. Validation is an important process to claim
a questionnaire being dependable to collect and analyze a data. A survey with good face and
content validity and tested with principal component analysis and reliability measure of
Cronbach’s alpha is considered validated from psychometric aspect and is ready for use to
collect data. Further usability of a validated instrument will depend upon research methodology
and intention to have descriptive or inferential statistical approach. A validated questionnaire
adds to quality of research outcome and its generalizability and acceptable to journals
administration for publication that one contributes to literature for further use by other
researchers with or without modification seeking author’s permission.
CHAPTER NO: 2

TYPES AND DESIGNS OF RSEARCH

Research is a creative and systemic work undertaken to explore an issue with scientific
evidences that involves collection, organization and analysis of information in a predefined
study design. According to Bradenton research has three basic ingredients, the exploratory,
descriptive and explanatory research and under these three categories there could be various
types of research designs.

1. Exploratory Research: In exploratory research we explore a research problem that we


know nothing or very little about it and this is often a qualitative research that
provides basis for further research.

2. Descriptive Research: In descriptive research, we know about the subject matters and
want to research further with more accuracy and precision. Descriptive studies are
often quantitative but it can be qualitative as well. In explanatory research.
3. Explanatory Research: In explanatory research is a causal or predictive research that
aims to explain the cause-and-effect relationship among the independent and
dependent variables.

Moving from the basic three categories there are seven types of research or study designs.
These are,

1. Survey Design: In survey design research we have validated instruments created to


collect standardized data from a large group of respondents. Survey research can use
interview, questionnaire-based, internet-based or even telephone-based surveys to
collect data, which is standardized using a reliable and validated instrument. Surveys
are typically descriptive research though the intent can be exploratory or explanatory.

2. Experimental Design: It is an explanatory research that explore the cause-and-effect


relationship among the variables. Research is designed to have controlled
experimental study that infer to causality and can be predictive. An example of
experimental research may be the impact of students’ performance using different
learning styles. Here the students’ performance can be determined for its impact of
learning styles measured through their semester examination results.

3. Comparative Research: This type of research aims to compare the two groups such as
performance of students taught through two different teaching and learning methods,
one using PBL and other using TBL and looking at their performance in continuous
assessment subsequently undertaken. These kinds of research are usually descriptive
in nature.

4. Case Study: This kind of research is exploratory and qualitative and performed on a
case and this case for example can be an institution. A study performed in an
institution or organization however, may not be generalized are usually exploratory

5. research. An example may be students’ performance using standard setting method


in assessment and looking at the range of observed score estimated using standard
error of measurement. The outcome of this research can be generalized to say
students’ performance in other institutions of same country will be the same or
similar.

6. Observational Research: This kind of study design is based on researchers’


observation of subjects identified in research for certain behavior, diseases, effects
and findings in a field or lab setting. Such type of research design can be qualitative or
quantitative. Observational research can be simple observation of things happening
and it can be exploratory, explanatory or descriptive in nature.

7. Action Research: In this type of research design researcher intend to have some action
that may be part of strategy to implement some procedure, teaching learning or
assessment method. This type of research involves both descriptive and explanatory
objectives such as, implementing the open book exam (OBE) during the emergency
remote teaching in Covid-19 pandemic and comparing the same students’
performance with close book exam (CBE). Here researchers may describe the
implantation of OBE and comparing the students’ performance of methods of OBE and
CBE. However, causality of online examination.

8. Mixed Method: This research design involves collection of data using mix methods
such as a combination of above research design. This is a bit tricky like the action
research design as there must be a reason why to combine different designs to collect
data? Instead, a series of research using different methods can be performed to collect
the data. Experts must be consulted to decide on using a mixed method research
design.

A questionnaire-based Research (Survey):

Research is a diligent and thorough inquiry and investigation into a subject. This includes using
all appropriate print and electronic sources, asking the reference librarian for help, and making
use of bibliographies given by other authors. The process of discovering new knowledge or of
synthesis of existing knowledge into a form useful for a stated purpose. Attempt to find out
information in a systematically and scientific manner uses data collection methods that survey
is one of the methods. Questionnaire surveys are the techniques for collecting statistically
sound data about the attributes, attitudes, action, perceptions or opinions of population using a
structured set of questions. Surveys are organized to gain information and insight on various
topics of interest from a predefined group of respondents in a population.

A survey is research method that uses questionnaire or interviews from the participants to
collect the data in a sample about their characteristics, opinion and experiences in order to
generalize the finding to a population that the sample is intended to represent. Questionnaire
are documented to ask the same question to every participants and interview often structured,
where participants are interviewed and their oral responses are documented.

As stated survey is one of the common design in research as observational studies.


Questionnaire can be of following types.

1. Paper surveys that uses the paper and pen to record participants opinions
2. Online surveys that uses computer and internet such as emails of respondents
3. Telephone survey using direct call and conducting the research over telephones
4. Focus group survey using one-to-one interview

Considering the survey items, it can be open ended or close questions. Open-ended items ask
questions that allows respondents to supply answer in whatever way they want. In close-ended
items questions provides several response options that respondent must choose from. Response
options can use a Likert scale, behaviour anchored scale or checklist.

Survey research is most commonly used research design that allows researchers to collect large
amount of information in a relatively short period. Surveys are less expensive compared to
other research methods that allows to collect information on a broad range of things including
opinions, attitudes, behaviours and perceptions as a descriptive type of quantitative research.
However, disadvantages of survey research lies in its reliability and validity. Reliability may
be compromised in survey research depending on a number of factors such as respondents may
not feel encouraged to provide accurate and honest answers or they may not be fully aware of
reasons for any give answer because of knowledge, memory issues or simple boredom. The
strength of survey research lies in its cost effectiveness, reliability, generalizability and
versatility. On the other hand some of the common weakness are inflexibility, lack of potential
depth and extensive validation process that requires knowledge and skills and is labour intense.

Covid pandemic have impacted global community in a number of ways and situation offers
rooms for questionnaire-based survey research to explore such as perception of emergency
remote teaching online as the most hot issue. Covid-19 pandemic has specially responsibility
on teachers apart from teaching and learning and assessing to have research and publishing.
Questionnaire-based survey with little training and skills is one of the most cost effective
method to conduct research. Validation of survey research in terms of validity index, reliability
and factor analysis can allow researchers to collect data that can be analysed to share with
global community in a most easiest way.

Likert Scale in Questionnaire-based Survey:

To measure the opinions of varying degree a 5-point Likert scale from one extreme to another
is the most tool in scientific and academic research. The scale was developed in 1932 by social
psychologist Rensis Likert, who used this scale to identify a person’s perception and opinions.
The scale developed to choose option that best support participants opinion from strongly
agreed to strongly disagreed to collect orderly data in a qualitative research however, assigned
a numerical value assigned to each option makes it a quantitative measure for interpretation
and logical decision.

Among the advantages of using a Likert scale are simple to understand, less time consuming,
gives participants an option to be neutral though, could distort the result so needs to use and
interpret carefully. Most often a Likert scale is developed as 1. Strongly disagree, 2. Disagree
3. Neutral, 4. Agreed and 5. Strongly agreed. Likert scale can be any number but 3-5 point
Likert Scale is considered effective and is recommended by many authors and has been widely
practiced in research. However, to produce best result in survey design the emphasis is on
developing questions as clear as possible and avoiding to use statements. The Likert scale data
can be analysed to use mean and standard deviations to describe the scale.

For calculating the total number of response in each question opined as strongly agreed to
strongly disagreed on a continuum scale, easiest way is to sum up the responses in a question
and divide it by total number of respondents. One of the way to evaluate the calculated score
for each question using the following scale,

1. 1 to 1.80 representing (Strongly disagree)


2. 1.81 to 2.60 representing (do not agree)
3. 2.61 to 3.40 representing (true to some extent)
4. 3.41 to 4.20 representing (agree)
5. 4.21 to 5.00 representing (strongly agreed)
CHAPTER NO: 3

DESIGNING AND DEVELOPING QUESTIONNAIRE ITEMS

Questionnaire is an important tool of survey-based research. There are many tools that can be
used to record people opinion on a given questionnaire however, the questions remains the
most important item to know, what people explicitly feels and think. The first recorded
questionnaire was developed a couple of century back by Adolphe Quetelet, a French
polymath, in 1835, applied to natural science and essentially invented the field of sociology.
The use of questionnaire has yet to be fully employed in research is due to its extraneous work
associated with its development and validation process and the expertise to do it.

There are many advantages and disadvantages to questionnaire utility in survey research.
Despite of stringent criteria to develop and validate questionnaire and yet many pitfalls
associated with it, there is no parallel framework to replace its importance in research [10].
Several guidelines have been provided in literature [11, 12, 13] to minimise the errors,
nevertheless a hands-on training has been found most effective source of developing expertise
to design and validate a questionnaire for research.

By definition a questionnaire is a set of written questions or items designed to collect


standardized information from the respondents about their feelings, thoughts opinions,
experiences and behaviour for the purposes of research. Although it is considered same as
survey, it may be a different research design in the sense that they do not collect mass data
for further analysis. However, the terms are interchangeably used by many researchers.
Questionnaires though provide a comparatively cheap and efficient method of data collection
in quantitative or qualitative research, questionnaire design for many researchers may be a
complex process of developing and validating questionnaire to gather the information.

Before, moving on to put down the guidelines to design and develop a questionnaire, we must
think of what specific information are we trying to collect by administering a questionnaire for
data collection. The best approach to this would be, to create a team of experts in subject area
that we may want to research and begin with a qualitative study of focus group to explore the
key areas for further research. The second thought to use questionnaire should be to find
questionnaire appropriate for research and its standardization, reliability, and validity. A valid
questionnaire measures what it claims to measure however, many would be unable to achieve
that. On reliability aspect a questionnaire may not yield consistent result from sample to
sample, and this may relate to selection of targeted population and their understanding about
purpose of research.

Another, question in relation to developing a new questionnaire would be, do you have a
questionnaire available in literature that can be used to collect data for your research. This may
save your time though; you must ensure that an existing questionnaire is fit to your research
methodology to collect data or if a questionnaire is selected from literature does it require to
do the validation before using it in a different socio-cultural environment. Lastly, you must
decide to present your questions or items in a questionnaire. Is it an open-ended or closed-
ended question? Open-ended questionnaire is generally developed to acquire exploratory
information and closed-ended questionnaire is used to collect quantitative data. Both has its
advantages and disadvantages (See table 1). It is also important to provide an introductory letter
or information sheet for participants to read before consenting to complete the questionnaire.
Often this is not done, and it becomes inconvenient for respondents to appropriately complete
the questionnaire. A checklist should be followed stepwise to develop a questionnaire (See
Figure 1).

In general, questions should be short and to the point having about 10-12 words or less in each
item. However, sometimes short questions can be perceived as abrupt and threatening, and in
such a situation little more descriptive question is proffered. To select a sample to collect data
using a questionnaire or for its validation, it must be scientifically and thoughtfully done and
if necessary, a statistician services should be sought else, an inappropriate sample size may
affect the result. This also applies to pilot project on the validation process of a questionnaire.

There are no general criteria for the required sample size in a validation and reliability study
[14] but as many subject as affordable, around 250-300 subjects are considered a good choice,
but it may not be feasible for every researcher. There are options in literature to have 5-10
subjects per item in a questionnaire required for the validation process in a Principal
Component Analysis. Questionnaire research or its validation though requires relatively little
use of instruments and equipment, it is mandatory to obtain research and ethic committee
approval to either using or developing a questionnaire-based survey research.

Table 1: Advantages and disadvantages of open and closed-ended questions in questionnaire


research.

Advantages of open format questionnaire Advantages of closed format questionnaire

Allows qualitative research for precise Allows quantitative research for data
answer. collection in an easy and quick way.

Allows exploring research question with Ensure all respondents receive same stimuli
individual response precisely. in keeping with standardization.

Enable researchers to generate hypothesis on Easy to record and analyse data for
a research topic descriptive result quantitatively.

Example: What do you like about open- Example: Open-ended exam is easy to pass.
ended examination. Strongly agree, Agree, Not sure, Disagree,
Strongly disagree.

Figure 1: A stepwise checklist to design and develop items and the questionnaire for research.
Steps to Design a Good Questionnaire:

There are no hard-and-fast rules about how to design a questionnaire, but one may follow a
structured format to help develop a questionnaire keeping with good design questionnaire
practice. Here are the six steps, which one can follow to design and develop a good
questionnaire for data collection in a research study.

Step No. 1: Identify the research objective and appropriateness of questionnaire

Main objective and goals of designing questionnaire should be determined as what information we aim
to seek. A well-designed questionnaire should meet the research objectives set to answer the research
question. Often the researchers may neglect important features due to the research team inadequate
preparation. However, a good questionnaire design should overcome these issues by good team work,
in -depth discussion, elaborations and ensuring alignment of objective and items written. Some ideas
can also be sought from the existing questionnaire on the similar topic using literature resourcing.

Step No. 2: Define targeted respondents and their optimal number


The selection of targeted respondents is key to optimise the outcome of research and clearly,
one can’t test everyone and there may be a lot of restrictions to approach and get the desired
information with respect to the targeted audience to respond the questionnaire. Besides, in
questionnaire validation process be it a pilot study or the ultimate administering of
questionnaire for data collection, certain number of respondents are must to determine the
sample size.

Step No. 3: Design questionnaire to seek accurate information

The experts team designing the questionnaire must ensure that respondents understand each
item and answer those without hesitation, disengagement and refusal. Quality questionnaire
is designed and written to encourage respondents to provide accurate, unbiased and
complete information. Questions/items must be phrased in a way that prevents any
misunderstandings or ambiguities. Items writers must avoid using jargon, technical or
scientific language in order the text to be fully understood by anyone completing the
questionnaire.

Step No. 4: Choose question type, open-ended or closed-ended

There are several variety of question types and phrases used. In explorative questionnaires,
usually the open-ended questions are preferred, where participants supply answers rather
than selecting responses from the option list provided. Purpose is to gain an understanding
of the topics associated with a research question. On the other hand, quantitative
questionnaires primarily include closed-ended questions, which have been structured and
predefined by the researcher either in form of multiple choice answers or rating scales such
as the Likert scale. However, one must avoid to produce a questionnaire with a mix of items
having different questions type

Step No. 5: Design question sequence in well-defined components

After having fine-tuned each item individually it is time to consider and improve the overall
flow and layout of the questionnaire. Look for smooth transitions from one question to the
next or one construct to another construct?. Besides, it is also important to sequence the
follow-up questions placed correctly. Also make sure to adhere to skip-rule is implemented if
required. Respondent must know the items not necessary to be respondent versus must
respondent items (often identified by an *) that do not apply to respondents in case.

Step No. 6: Run a pilot study to validate the questionnaire Items

This step is important for evaluation of items to its purposes and to ensure the question is
fully understood by the respondents in terms of language and clarity. Every questionnaire
should run a pilot study through a representative sample preferably from the same
environment and population that ultimately the questionnaire will be administered to before
moving further in validation process.

A pilot study will enable designers to readily identify items with poor language, understanding
and phrasing issues. After the data is collected from a pilot study the initial team of experts
may sit down to discuss the questionnaire and refix the issues arising with experience they
may have from the piloting data. Pilot data should be statistically worked up to ensure the
interest and purpose of questionnaire is fully applied.

CHAPTER NO: 4

CREATING ONLINE SURVEY FORM


You may create a questionnaire-based survey with Likert scale using a Google form. The
reasons why I chose Google form to create a questionnaire-based survey using a Liker scale,
are?

1. Easy to work with an online user friendly survey form.


2. Many ways to create form with multiple choices for respondents .
3. Analysed data available for immediate sharing with respondents.
4. Excel sheet is readily available that can be downloaded for further action

Current chapter 2 will demonstrate, how to create a Google form in a way that makes it easy
for everyone to learn. To create a Google form all you need is to have a free Google account
and follow the steps described below.

Step 1: Once you open up the Google homepage in your Google account, you will find
an icon that looks like a Waffle on the top right or top left corner.

Step 2: Clicking the Waffle sign on google we lands up to a file storage and
synchronization package in Goole called Google Drive.

Step 3: To open Google form we click Google drive and it takes us to home Page that
has a button called “New”

Step 4: As you click “New” it shows a drop down menu with many options including
Google form. For creating a questionnaire we can start with a blank form, leaving aside
the template option.

Step 5: Getting started with it lands up by default on “Question” and “Response”


setting. What we get in response will be towards the end of procedure as we learn
more about Google form. This step will also take you to insert questions. You may also
insert some pictures from your desk top or other resources of multimedia available
for your access.

Step 6: Give survey a title or name and I am using Entrepreneurship Training Impact
(ETI) Inventory, a specially created questionnaire to collect fictitious data to use it for
further demonstration in this micro-credential course. Under this title you may
provide a little bit of description, what this survey is all about

Step 7: Now start adding some questions or items by clicking question 1 and this will
highlight to write question and options to choose by the respondents. Options are
provided below the question by default for participants to respond and select one that
endorses their opinion.
Step 8: Also click multiple choice drop down menu and choose your option from the
list of options. For a Likert scale questionnaire best choice will be a “Linear Scale” from
the list of various other options. In linear scale options, we have numbers as choices,
which we can assign as, “Strongly disagree” No 1 to “Strongly agree as No 4. We have
the option to choose number of choices to be allowed from 1 to as many as 10 shown
in drop down menu in our current example we are reducing it to 4 options. For each
option there is a choice to insert picture as well. Do click “Required” option at the right
lower end, which means respondent has to respond existing step to move to next.

Step 9: On the right hand side in the form there is an options on the top to add on
another question. Another short cut to this is a button below, which says duplicate
and on clicking it drops down to next question.

Step 10: In this step we can type or cut and paste a new question from the
questionnaire file. The options will automatically appear with this question as set in
preceding question and does not need to write it again with every question.

Step 11: After having typed or cut paste the next question we complete the
questionnaire and go to preview by clicking a little “Eye” on the top on the right side.
As we click here, we can see the survey questions in order and ready to administer.
At this point you can use some additional option of background colours or themes or
even can insert a picture imported from desk top or any other online sources.

Step 12: After having built the survey form now we want to save and share the
questionnaire and to do this we have to perform a couple of more steps such as where
to save it or even star (*) it for an easy access in Google drive or can create a link to
share.

Step 13: Next you may decide to send out the survey and to do that we have to choose

the options of send button, which can be sent by email, WhatsApp or a link or URL to
insert on a website. Create the link and save it to share with respondents

Step 14: Create the link by clicking link button and save it to share with respondents.
Here you may go to setting to have some additional options. For example, number of
attempts every time it is opened by the same respondent. In other multiple options
choices you may also check out the shuffle option every time it is reattempted by the
same respondent.

Step 15: You may also can create different sections by clicking the options given at
the bottom on the right side of the form.
Step 16: To add items to new section click add question from the right side of option
list on the top and continue inserting or writing question as in previous exercise done
for section 1. Section can be given a name where title is written.

Step 17: Going back on to home page, we can click on “Response” next to question
and open this to have a readily available result to share with anyone such as the
participant in a workshop or for preparing a report if you may want to.

Step 18: Ultimately we will be interested to down load the excel worksheet to have a
data, which can be converted to text in a Likert scale to produce the raw data for
analysis.

Note:

Keeping with micro-credential principle of chunking in the course into short videos of
around 5 -minutes each and to enhance learning, we have produced a video associated with
validation process of questionnaire based survey. However, another video created to
demonstrate additional features that could not be accommodated in a must watch video due
to time constraint, we have produced an optional to watch video. To equip participants in
this micro-credential course with further steps required to learn in a questionnaire-based
survey using a 4-point Liker scale, you may watch another video, which is optional.

1. Must watch video (Short) Link: https://bit.ly/3p8Fuyj

2. Optional to watch video (Long) Link: https://bit.ly/3i5FWfk

Take-home Exercise:

Design and develop an online Google form survey using the questionnaire format (5-items
and 20 subjects) based on a 4-point Liker scale and create the link to administer it to one
respondent (you) to complete the form using the online link. Ultimately download the Excel
data sheet for further use in this micro-credential course (See Task 1 and 2 of Module 3).

Demonstration to Create Google Form Step by Step:

Step 1:
Step 2:

Step 3:

Step 4:
Step 5:

Step 6:

Step 7:
Step 8:

Step 9:
Step 10:

Step 11:
Step 12:

Step 13:
Step 14:

Step 15:
Step 16:

Step 17:
Step 18:
CHAPTER NO: 5

VALIDATION PROCESS

A questionnaire validation process refers to a questionnaire and it’s scale that has been
developed to be administered among the intended respondents. To complete
the validation processes needs a representative sample, demonstrating adequate reliability and
validity. Validation of a survey inventory (questionnaire) requires the following steps.

Step 1: Establish Face Validity. The survey reviewed by two different experts

Step 2: Run a pilot test with reasonable number of subjects (respondents)

Step 3: Clean the data collected using Microsoft Excel and SPSS

Step 4: Apply factor analysis using Principal Components Analysis (PCA)

Step 5: Check internal consistency using appropriate statistical methods

Step 6: Revise the survey involving experts engaged in setting up of questionnaire

The above steps in validation of survey therefore include pre-testing the questions,
administering the survey, reducing the number of items and reorganising the items into relevant
constructs or factors defined by the factor analysis in terms of PCA.

Data Collection and Cleaning:

After the questionnaire is set and agreed upon among the members of research team (see
appendix), next step is to run a pilot study among the subjects from the same environment that
a larger survey is aimed at. Pilot study basically explores the clarity, language, connotation and
understanding issues of each item by the respondent, who are also asked to provide comments
if any difficulty found in response to each item in questionnaire. A follow through meeting of
research team addresses those issues and fine tune the questionnaire ready to administer for
validation purpose. Once the data is collected using online or hard copy response a work sheet
is generated on Microsoft Excel called the, “Raw Data”. Subsequent steps involved are as
following.

1. Coding the Raw Data with Numeric Values in Excel:

As indicated in the Liker scale see what words have been assigned to each option
from 1-4 (scale used in current study) and prepare to assign numeric values to each
options written in words from strongly agreed to strongly disagreed using the
steps as shown for recoding items with negative connotations, however, positively
written questions is a better option. For coding the raw data to assign numbers
following steps are followed through.
a. Hold the control key and click one column on the top of Excel sheet to highlight
that column and while continuing to press control key, also press command
key and click all the columns from Q1 -Q15.

b. Next press the control key and press letter “F” on key board, representing find
and replace option.

c. In the table find the option of, “Find” and write the word exactly it is written
in the Liker scale options of strongly agree to strongly disagree and press
option of replace below.

d. This opens up option of replace with. Now enter the number you want to code
and then click replace all and this will show all the cells changed to their
numeric number as commanded and now press ok. This will show a popped up
box indicating number of cells changed from text to their respective numeric
numbers. Now repeat the same procedure for text in Likert scale until the
entire raw data is changed to numbers.

2. Recoding the Raw Data with Negative Connotation in Excel:

Read through each items and identify items written with positive and negative
connotation by denoting positively written with (N) and negatively written with
(R). Now change those to a reverse order in Likert scale using the following steps.

e. Hold the control key and click one column at the top to highlight that column
and while continuing to press control key, also press command key and click
all those questions negatively connotated.

f. Next press the central key and press letter “K” on key board, representing find
and replace option.

g. In the table find the option of find and there enter the word from exactly it is
written in the Liker scale and press option replace below.

h. This opens up option of replace with. Here enter the number you want to be
reversed coded then click replace all and this will show a popped up box
indicating exactly the number of cells changed as commanded and now press
ok. This will assign reverse coding to all the items as asked.

3. Cleaning the Data of Unengaged Respondents in Excel:

Cleaning the data from those respondents, who took the survey very casually, are
actually those uncommitted or unengaged respondents and should be removed
from the data. Steps to identify unengaged respondents are as followed.

a. Choose a new column and give it a header of standard deviation (SD).


b. Select the cell below and type, = on key board and then type, “STD” and from
the drop down menu select STDEV.S to open the parenthesis and highlight the
1st respondent entire row (all the items) and click enter.

c. Now fill in the rest of the cells in column (SD) by dragging the 1st cell down and
look for standard deviation values with variations from one respondent to
another respondent. Check out cells with 0 values and remove them from the
data since these are the respondent who choose one option and filled out all
the items with that option without applying their thoughts to logically decide,
which applies the best.

4. Cleaning the Data for Missing Value in SPSS:

Reaching this stage in Microsoft Excel, it is time to transfer the data to SPPS to
identify the missing value, which is a little difficult exercise in Excel to perform
however, it is an easy process to do in SPSS. But before explaining how to find
missing value in the data it will be better to describe steps to import the data file
from Excel to SPSS. Collecting data initially in Excel is more handy since it can
directly download from online survey or can be typed down from the hard copy
paper response by the participants. Subsequently it will be transferred from Excel
to SPSS using the following steps.

a. Identify the location of Excel file from where it will be imported to SPSS.
Prepare the Excel worksheet for import to SPSS by ensuring the rows contains
the variables and column contains the subject. Also look for data not suitable
for SPSS and remove them from the Excel sheet. Be informed that SPSS accepts
number not text and therefore coding of Liker scale in Excel to numeric values
using the method described above is done earlier.

b. In SPSS open file and go to, New and click Data. This opens a new SPSS data
sheet.

c. Now in new SPSS data sheet go to file and from the dropdown menu go to
import file (in new SPSS version) or open data (in old SPSS version) and click
the desired file location as identified from source file of one’s computer.

d. Having identified the Excel file highlight the file source and click open. This will
open worksheet location option to choose and one may choose sheet 1 by
default. Here check off the read variable names and percentage value at 95
and click open.

e. Excel file promptly will be imported to SPSS worksheet for further work to do
the data cleaning, normality of data distribution, frequency output and the
most wanted reliability indices as Cronbatch’s alpha and correlation coefficient
either Pearson’s or Spearman’s correlation coefficient depending on normality
test of distribution whether met with or not respectively.
Identifying Missing Values in SPSS:

In data entry there is a wide range of circumstances that can occur to end up with missing
data due to data entry error or respondents’ negligence in answering all the items in a
variable. However, this is part of research, and we have statistical method available to
manage the missing data and its impact on research outcome. Steps to identify the missing
value and its replacement with series mean can be done in a systematic way as followed.

Step No. 1: Creating new variable to identify missing value:

Go to transform and compute data and take one of the variable such as Q1 to give it a new
name (Q_1) in target variable box. Click in numeric expression box and then click, “All”
and further scroll down the function in special variable box to click, “Missing” appeared in
numeric expression with parenthesis, drag Q1 in parenthesis and click ok to create new
variable (Q_1) in subsequent column.
In this column we can see, where there is a value it is replaced by 00 and missing values are
indicated by 1.00. We can easily identify where missing values are.

Step No. 2: Identifying the numbers of missing value:

Go to descriptive statistics and click descriptive. Move all the variables into variable box.
Under the option, click mean, standard deviation, minimum, maximum and click to continue
and ok. In the outpatient data we can see descriptive statistic table showing number of
missing values in each variable and the valid number list-wise minus the total number.
Check the SPSS data view we can confirm these values missing in data.

Step No. 3: Identifying the percent missing values in frequency statistics

Go to analyse descriptive statistics and click frequencies and move the variable to variable
box. Next click statistics and strike mean median, mode, SD, minimum, maximum and range
to click continue and ok.
It provides statistic table with number 0f missing values and percent missing in the table
towards the end as missing system. If missing values are less than 5 then it is acceptable but
if a big data with multiple missing values we can replace them to have statistically sound
result on analysis.

Step No. 4: Coding the missing values for replacement:

First, we have to code the missing value, for this Go to transform and record into same
variable, Next move over one of the variables to numeric variables box. And click old new
values. Under the old values check system or user- missing.

Under the new value enter a number (missing value) least likely to occur in data set such as
99 and click add to move that value entered in old ànew box, as the missing value and it
will code the same variable not creating new variable. Next move to variable view and go
to missing column that SPSS will treat as valid data.

In the missing column click the three dots of variable to have drop box and hit discrete
missing values and type the missing value in appropriate box and click ok and SPSS will
recognise the data with missing values. If we run frequency statistics again it will show
missing value in variable statistic table with missing number and not missing system. In
SPSS data now one can see the missing values we assign.

Following are the point by point steps in SPSS to identify the missing values.

I. Go to transform in menu bar and click recode into same variable.

II. Transfer all the variables (items) into numeric variable box and click old and new
variables box below.

III. In old value box check off system missing

IV. In new value box enter the missing value (any number unlikely to be a data set
value such as 999 and check off add, click continue and then ok.

V. Go to missing column in variable view and hit the three dots to open the missing
value drop box and check off discrete missing value and click ok.

VI. To allow same missing values to all the responses click missing value and copy
it. Next highlight all the variables below and paste.

Step No. 5: Replacing the missing values with system mean

Go to transform and move down to click, “replace missing value”. In new variable box move
the desired question as the new variable and click ok. In result variable table see the new
variable with number of replaced missing values. Check the SPSS data view to see the
missing value replaced system mean.

Looking for Normality of Data Distribution in SPSS:

Before we embark on SPSS statistical test it is a good idea to go for and check the
distribution of normality since this will be required as an important assumption to
meet with as required for almost all the SPSS statistical methods (see figure 3).
There will be many other assumptions associated with individual statistical test
and those are discussed all along the test as we proceed in this write up. Following
is the steps to perform normality of distribution test in SPSS.

Step 1:

Click Analyse in SPSS drop down menu and go to descriptive statistics and click
explore and this will open explore dialogue box.
Step 2:

In explore dialogue box move the variable to be tested into the dependent list in
explore dialogue box and if you have independent variable move it into factor box
(optional).

Step 3:
On the right side of dialogue box click statistics and leave confidence interval for
mean at 95% by default and click continue.

Step 4:

Click on plot to open explore plots dialogue box and here check off normality plot
with tests and histogram. Now click continue followed by ok. Output table
generated will be, test of normality showing Kolmogorov Smirnov and Shapiro-
Wilk with significance value suggesting to accept or reject the null hypothesis for
data being normally distributed depending on significant (<.05) or unsignificant
(>.05) p value.

Step 5:

For visual impact same can be observed for histogram (see figure 3) and QQ plot
(data points with equal number above and below the line. There is another ways
to establish the normality of distribution and that is to divide skewness/kurtosis
statistics value by standard error value in descriptive table. A resulting value within
-1.96 and +1.96 suggest data is normally distributed. For outlier do check the box
plot and anything without an asterisk is considered ok.

CHAPTER NO: 6
VALIDITY AND RELIABILITY OF THE QUESTIONNAIRE

The content validity is observed by undertaking the four main steps. Of these the first step is
to define the purpose of research in questionnaire we intent to measure. Next, we define
the content domains (constructs) based on relevant standards, skills, tasks, acquired
behaviours, components, factors, etc. that represent any one construct. The reliability of
questionnaire can be determined either as either Cronbatch’s alpha looking at the inter-item
consistency or test-retest reliability depending on research design. Test retest reliability
involves administering the survey with a group of respondents and repeating the survey with
the same group at a later point in time. We then compare the responses at the two timepoints.
Another way to look at reliability is the interitem correlation using either the Pearson’s or
Spearman’s correlation coefficients depending, whether the data is normally distributed or not.

1. Validity of Questionnaire

For validity of Likert scale questionnaire-base survey content validity index (CVI) seeks
experts opinion for basically relevancy of items to the subject area content and the
specific construct created by the research team. CVI generates multiple indices
including I-CVI (Individual-CVI) S-CVI (Scale CVI) with its further two indices called S-
CVI/Ave (Scale CVI Average) and S-CVI/UA (Scale-CVI Universal Average (see figure 2)

Content Validity Index (CVI):

To measure the validity of a scale first thing is to define the construct and develop the
items to measure the construct determined as the content validity. Content validity
index is the degree to which an instrument has appropriate sample items for the
construct being measured [15]. It determines whether items sampled for inclusion on
the tool adequately represent the domain of content addressed by the instrument
[16].

Content validity index is evaluated through individual (I-CVI) and scale related (S-CVI)
after collection of experts response on excel format (see figure 2). Following measure
in a systemic order is carried out as under.

I-CVI (Individual CVI):

I-CVI measure the efficacy of item as the proportion of experts giving an item a
relevance rating of 3 or 4 in a 4-point Liker scale determined as the total agreement
on item divided by the total number of experts (see table 1).

S-CVI (Sample CVI):

S-CVI measure the efficacy of the scale in toto as the content validity of overall sample
determined by an average of I-CVI and this can further be divided into two (see figure
2)
1. S-CVI/Ave (Average):

Average of the I-CVI for all items on the scale in a construct as one of the components
of a questionnaire.

2. S-CVI/UA (Universal Average):

Proportion of items on a scale that achieves a relevance rating of 3 or 4 by all the


experts in a 4-points Likert scale. It’s a conservative approach and is a very useful
index.

Guidelines for Content Validity Index (CVI) Measure:

Most important informative procedure is to compute S-CVI for both of its types, S-CVI/Ave
and S-CVI/UA. Excellent content validity is judged for I-CVI that meets the Lynn’s (1986)
criteria as following. There are no specific cut-off values established for CVI acceptable criteria
however, the values range from 0 to 1, where if I-CVI > 0.79, the item is relevant; if between
0.70 and 0.79, it needs revisions; and if below 0.7, the item is eliminated [17]. A robust criteria
put forward by Lynn’s provides a practical guideline [18] to make logical decision as under.

Lynn’s Criteria:

Excellent I-CVI with 3-5 experts = 1.00

Excellent I-CVI with 6-10 experts = 0.78 and above

Acceptable S-CVI with 6-10 experts = 0.90 and above

If Lynn’s criteria are not met it indicates construct not adequately covered in the initial
round of experts meeting and a second pool of experts’ judgement becomes necessary

Steps to Calculate CVI:

A template for relevancy is prepared and send out to selected experts with rubric (see
figure 2). Ideally 10 or more experts should be selected for structure feedback using a
template. However, 5-10 experts may be a good range to move on. For clarity
simplicity of questionnaire items another template using dichotomous rubric of yes
and no may suffice. Here more complex calculation of relevancy of items for I-CVI, S-
CVI/Ave and S-CVI/UA will be described using the following steps.

1. First thing is to determine the number of agreement among the experts submitted
as a documented form. At least 5-6 experts should be involved however, 10
experts would be ideal. Experts are provided with in-depth description of the
construct to refer to and to make a logical decision on each item. This is calculated
using following function in Excel. In case of a 4-point Likert scale a proportionate
agreement will be 3 and 4 and in case of 5-point Likert scale it will be 4 and 5.
Following function in Excel will determine the item number of agreement by typing
= count if (range of all experts rating, “>=3”) and press enter. A numeric number
will indicate the number of agreement among the expert on that items. Scrolling
down from this number will fill in the remaining items cells on Excel sheet.

2. To calculate I-CVI the formula is, = number of agreement/number of ratters.


3. To calculate the S-CVI/Ave the formula is, = Average I-CVI range of all the items.
4. To calculate the S-CVI/UA, first we need to determine the total agreement among
the expert from the list of agreement (using the example of 6 experts (see table 1)
using the following function.

= Count if (range of agreement, “> = 6). The number achieved using this formula is
divided by the number of ratters (Number of total agreement/ Number of total
ratters)

This way we can determine the relevancy of items to its construct based on
experts’ opinion in terms of, I-CVI, 2. S-CVI/Ave and S-CVI/UA.

Clarity of items can use yes or no criteria and will be sorted out addressing the
comments associated with each items.

2. Reliability of Questionnaire:

Cronbach’s alpha:

Reliability is the extent to which a measurement is consistent if the test is repeated


however, consistency to know about items in questionnaire is about the score across the
items in the construct. Cronbac’s Alpha is more appropriate for nominal interval data
and it evaluate the extent to which different items on questionnaire measure the same
ability or trait [19] (see table 2). If items do not measure the same characteristics of a
construct in a questionnaire, which may not be consistent internally. The coefficient of
reliability with Cronbach’s alpha is 0-1 and an acceptable value [20] is considered any
number >0.7. A negative value is theoretically not possible therefore, reported 0.

Reliability: Item-total correlation:

It refers to positive and strong relationship of items to item and items to total score of
all items in a test (see table 3) and can be interpreted as:

i. Pearson’s correlation coefficient (r)/Spearman’s Correlation Coefficient (rho)

It is the correlation between item-to-item ranges from -1 to 1 and greater the number
effective the item. Pearson’s versus Spearman ‘s Correlation Coefficient is decided
depending on the normality of assumption achieved or not (see figure 3, table 4). If
normality of distribution assumption is met with Pearson’s correlation else, Spearman’s
correlation test is carried out.

ii. Corrected item-total correlation


It is the correlation between each item and a scale overall score that excludes that item,
ranges -1 to +1 and greater the number effective the item.

iii. Cronbach’s Alpha if item deleted

It is the value of overall alpha if the item is not included in the calculation and the value
should be around the overall alpha (see table 2).

Steps to Perform Cronbach’s alpha in SPSS:

Step 1:

In drop down menu in SPSS on the top click Analyse, then scale and the Reliability
Analysis.

Step 2:

Transfer variables (items) q1, q2, q3.....qi and leave the model set as Alpha

Step 3:

In the dialogue box click statistics.

Step 4:

In the box description, select item, scale and scale if item deleted. In the interitem
box, select correlation

Step 5:

Click continue and then ok to generate the output.

To interpret the output, one can follow the rule of George and Mallery (2003, see
table below)

No Coefficient Alpha Interpretation (Remarks)

1 >.9 Excellent

2 >.8 Good

>.7 Acceptable

3 >.6 Questionable
4 >.5 Poor

5 <.5 Unacceptable

Steps to Perform Correlation Coefficient in SPSS:

Step 1:

On the top of SPSS click Analyse and go to correlate and click bivariate.

Step 2:

Move the variables to be tested for correlation to variable box on the right

Step 3:

Check off Pearson or Spearman under the correlation coefficient and click ok.
Correlation coefficients output table will generate for interpretation. A coefficient of .3
and more is considered good correlation between the two variable however, any value
more than .8 indicate redundancy or multicollinearity and one of the two variables
associated with very high correlation should be removed from the items list.

CHAPTER NO: 7

CONSTRUCT VALIDITY PRINCIPAL COMPONENT ANALYSIS

Factor Analysis of Questionnaire:

Factor analysis tells about how the items can be divided off through factor loading to determine
different construct initially thought by the expert. There are two main approaches to factor
analysis based on intent of a researcher, exploratory or confirmatory. Exploratory factor
analysis intent to measure the dimensionality and often used in the early stages of research to
establish the interrelationships among a set of items (variables) in a questionnaire [21].
Whereas, the confirmatory factor analysis is a set of techniques applied to test specific
hypotheses or theories concerning the structure underlying a set of variables [22, 23] Factor
analysis can be principal component analysis (PCA) or standard factor analysis (SFA). In PCA
we work with original variables to produce smaller set of variable with stronger linear
correlation and this provides a practical model. In SFA we use a mathematical model of shared
variance versus total variance and it provides a theoretical model. The two can be summarised
in a way that if we are interested to develop a theory or looking for a theoretical solution and
want it to be uncontaminated with variables, factor analysis is the best. If we are interested in
real world scenario with empirical summary then principal component analysis is the right
choice. In PCA outcome are interrelated factors called components and in factor analysis
outcomes are truly factor as independent variables.

Factory analysis is based on the assumption that all variables (items) correlate to each other to
some extent. Depending on research design variables should be measured at descriptive or
inferential level of statistical test. The sample size for factor analysis is supposed to be big
enough, over 200 but an acceptable range between the subjects and items should be in 10:1
ratio [24] Factor analysis as a principle require big data and using smaller sample size results
cannot be generalized. A big sample over 200 are basically required however, in terms of
subjects and items ratio, 5:1 subjects/items are acceptable. In factor analysis it is important to
know what we are measuring and how we are measuring. For how we are measuring there has
to be in -depth criteria. In factor analysis criteria is provided by, how we explain the variances.
There are three measures to determine our efficiency in factor analysis.

1. Kaiser’s criteria, which uses eigen value and an eigen value of 1.0 is ideal which
determines the amount of total variance explained by the tractor.

2. Scree test, which is the graphic representation of eigen value, shown in the shape of
a curve that changes direction and becomes horizontal.

3. Parallel analysis, which is a quality control check and it compares the size of eigen
value collected from our data with another randomly generated data with eigen value
of same size. SPSS do not provide option for comparative analysis and can be
downloaded as an application (Monte Carlo PCA for parallel analysis) on Microsoft
Windows.

Steps to Perform Factor Analysis:

To start Factor analysis go to analyse in SPSS bar list and next go to dimension reduction and
strike factor to open the dialogue. First move all the items intended for rotation into variable
box. In order to perform rotation and define factors that we are interested in, we have to look
for criteria to proceed with factor analysis.

1. Under descriptive leave the initial solution checked off by default in statistics section.
In correlation section we check off coefficient, determinant and KMO with Bartlett’s
test of sphericity in correlation matrix.
2. Under extraction we select Principles Component analysis (PCA) and check off Scree
plot in extraction section. In analyse section correlation matrix and in display section
unrotated factor solution are left checked off by default.

3. Under the score no option changes are made if we are not interested in saving the
output table.

4. Under the choice of option we check off sorted by size in coefficient display format.
This is an important step to know which item is loaded to which factor. In missing
values exclude cases listwise is left checked off by default.

5. Lastly take a look on rotation and in rotation method select one of two options of
Direct Oblimin in oblique rotation choice or Varimax in orthogonal rotation choice
depending on whether data has the items correlated or uncorrelated. Uncorrelated
method is easy to report as items are independent and easy to interpret.

Output Table in Factor Analysis:

A researcher may start with larger number of items and using item analysis can reduce to lesser
number in refining the questionnaire through dimension reduction technique of factor analysis
so that the constructions become more meaningful. Once we click ok after every option and
method in factor analysis is rightly checked off, it generates the output table and the first table
to read is the correlation matrix.

In PCA items should have a correlation of 0.3 and above. Below 0.3 factor analysis may not
be valid and another way to look at it is the determinant value, which has to be greater than
0.0001. At determinant value of less than 0.0001 items are considered not correlated. Another
area of concern in correlation is when the items are highly correlated called multicollinearity
and a cut off point for multicollinearity is a correlation value of more than 0.8 in a questionnaire
based survey. In case of multicollinearity one of the items should be removed from the
questionnaire. Another way to reconfirm low correlation items as measure of sample adequacy
in factor analysis is to look at the Anti-image matrices. Here one looks at all the diagonal values
(see table 5) and a high diagonal value indicate that item can still be retained in questionnaire
to continue with further steps in PCA. In current example items 3 and 6 have been observed to
have very low correlation (light yellow colour in table 5) but their Anti-imagine value (dark
yellow colour in table 5) have been quite high and can be retained in questionnaire.

More reliable way to look at the inter-item correlation is the KMO and Bartlett’s test table (see
table 4). A KMO value of >0.5 is acceptable for questionnaire to be factored. Similarly in
Bartlett’s test of sphericity we want a significant value of p = <0.001. The other assumption of
data fitness for factor analysis is to look at the descriptive table generated with values of
skewness kurtosis and dividing these values by standard error values. If result yields a score
that fall within the range of -1.96 to +1.96, data is considered normally distributed to meet the
assumption (see table 5). After identifying data fitness using KMO and Bartlett’s test of
sphericity and normality of distribution measured through skewness and kurtosis in descriptive
statistics, next is the first extraction output of communalities. In factor analysis initial extraction
in communalities table is established as 1 and the extracted value of each item against 1
correspond to R2 and it determines, how much an item shares with rest of items to be important
in factor analysis (see table 6)
In extraction section of output table, Scree plot graphically tells us the number of factors
extracted and some of those underlying potential factors (see figure 4). Same output is
elaborated by total variance explained table in terms of factors explained (see table 7). Higher
the number better it is.

Next is the output table is the component matrix both, unrotated and rotated. Other tables
generated in this section may be pattern matrix and structure matrix. Pattern matrix are not
much different from the component matrix however, structured matrix is more about
correlation of factor among each other.

Once the number of components are determined it is time to interpret them and do that
components are rotated determined by loading pattern in component matrix (see table 8) which
tells us about variables clumped together in components or the factors referring to specific
constructs named by the researchers (see table 9, before and after rotation). Factor analysis or
the principle component analysis is all about rotation to achieve different factors or components
and therefore a better understanding of rotation procedure is imperative to make sense of PCA.

Understanding Rotation Method in Factor Analysis:

The process of defining factors in the statistics is referred to rotation. Rotation in factor analysis
is a mathematical procedure that rotates the factor axis in order to produce results that facilitates
interpretation . Using rotation loading pattern becomes clear and easy and a more pronounced.
In factor analysis rotation is associated with score scale and attempt is to reduce multiple items
into clearly defined factors and this is accomplished through factor analysis. Following are the
important steps in rotation methods in PCA.

I. Here we have several methods classified into orthogonal and oblique rotation.
In orthogonal we have Quartimax, Varimax and Equamax. These options are
used when we expect factors to be correlated and this can be checked by
getting started with Direct Oblimin output correlation matrix (see table 4) and
if a correlation of less than 0.32 is achieved we use one of those three,
preferably the Varimax method. Other option is oblique method and it includes
Direct Oblimin and Promax and we use these when factors are correlated.
Using this process we define what rotation method has to be employed.

II. The whole purpose of a rotation is to create a simple structure, which helps in
interpretation, which make sense of factor loading. However, the factor
analysis will identify the factors with certain number of items in it but will not
determine what construct it measure as this is beyond the prereview of the
factor analysis. The suitable names to factors are given by the researchers
based on name reflecting the items under its fold (see table 9).

III. Next we select the rotation method knowing that first three methods are
applied when factors are uncorrelated and last two methods are applied when
factors ae correlated and shows a correlation coefficient of more than .32.

IV. To start, which orthogonal or oblique method to select, we first use Direct
Oblimin method in oblique rotation assuming the factors are correlated. So we
check off the direct Oblimin method and in display box we check off rotated
solution and keep the maximum iteration up to 25 then click continue and
press okay.

V. Using a Direct Oblimin method we directly move down to last component of


correlation matrix in output table and look for correlation between factors
whether greater or less than .32 analysed data (see table 4). Ignoring the
diagonal values, which as a rule is always 1, we focus on values greater than
.32 and having found one oblique method is continued. However for all
practical purpose we may select one of three orthogonal method, most
commonly the Varimax leaving all the setting as it is and click continue and ok.

VI. In output table, we move down to rotation component matrix and look for the
simple structure. By definition a, “simple structure” is an item that has a
significant loading and as many as zero loading as possible. A zero loading is
defined as a factor loading which is greater than negative 0.1 and less than
0.1. There are many definitions of significant loading but a most practical
definition of a significant loading is that a factor with factor loading of 0.3 (see
table 7). However, some researchers may take significant loading value of 0.4
or 0.45.

VII. Now having defined simple structure that has a significant loading and as many
zero loading as possible besides, there may be many other values as well. A
complex variables is defined as one with factor loading of 0.3 on two or more
factors (see table 8). Keeping the definitions of simple structure and complex
variable, we interpret the factor analysis result and ultimately fine tune the
questionnaire ready for research.

CHAPTER NO: 8

QUESTIONNAIRE USABILITY

Questionnaire Factors Utility

It is also part of developing questionnaire to know how to use and analyse factors obtained
from PCA. It depends on the objective of factor analysis and this can be univariate or
multivariate. In univariate objective is to determine factors related to a construct. Univariate
factor analysis does not mean that factor extracted is only one, rather it is about the factor not
interested to establish relationship to other variables otherwise it will become a multivariate
analysis. The objectives of univariate factor analysis therefore are,

1. To determine the factors representing a specific construct in a questionnaire. This


usually will have the data collected on a Leikert scale and this may refer to three
likely situation in exploratory factor analysis (EFA) or confirmatory factor analysis
(CFA) as under,

a) Studies that do not find literature evidences or a theory available to explain


and this becomes an explanatory factor analysis that seeks expert opinion.

b) The studies that requires to use a questionnaire from literature and factor
analysis is performed to know the social-cultural variation and adjustment of
such questionnaire prior to performing PCA. This is important since those
questionnaire may or may not be suitable for certain targeted for population
in a confirmatory factor analysis.

c) Studies in which a questionnaire borrowed from the literature is modified for


its content (items) to determine the variability of the scale inventory for a
targeted population. Chi-squared test will be a good statistics to achieve the
objective.

2. To determine the level of satisfaction obtained from the factor analysis either on
awareness or perceptions of respondent. Here EFA/CFS produces factors based on
correlation or variances explained in terms of percentage of a factors. The mean
score gives the level of each factor in terms of satisfaction, awareness or
perception.

3. To perform bivariate or multivariate analysis on dependent variable established in


factor analysis. This will determine the factors obtained as dependent variable if
it differs on the basis of gender or ethnicity as independent variables requiring a
bivariate or multivariate analysis. The statistical test involved can be independent
t-test and ANOVA depending on the number of independent variable irrespective
of the factors are obtained from the EFA or CFA.

4. To perform predictive analysis using a model for dependent factors. Here the
factors obtained will be independent and dependent variable. However, this will
require a questionnaire design structured with defined construct ideally
demarcated into one dependent variable. Factors obtained can be EFA or CFA.
Here the statistical methods used will be regression model.

Result and Exercise to Perform:

A flow chart developed for validation portrays the entire process of creation, validation and
usability of questionnaire. A fictitious data created to write this paper has a number of table
generated on analysis in Excel and SPSS as Content Validity Index (CVI), Reliability Indices
(Cronbach’s alpha and Correlation Coefficient) and a number of Factor Analysis tables (see
figure 1-4 and tables 1-8).

Analysing the Data for Interpretation:

The fictitious data has been created and Microsoft Excel data sheet developed as the raw data
and is used for analysis and interpretation based on the objective of study, which is the
validation of newly designed questionnaire based survey using 4-point Likert scale. Excel
function used will determine the percentage weighting of each item on a scale from strongly
agreed to strongly disagreed. However, based on objective and the hypothesis a questionnaire
can be designed and developed with the intent to compare mean, perform linear regression and
predictive statistics using t-test, ANOVA and regression methods. ANOVA can further be used
beyond one-way ANOVA to multifactorial ANOVA and ANCOVA depending on research
design developed to have dependent and independent variables and cofactors.

Here we consider analysis and interpretation of a questionnaire developed to measure the


entrepreneurship abilities of participants after attending a training programme. This will use
Microsoft Excel raw data worksheet either directly downloaded from the online survey or
manually developed worksheet from the paper based survey. The analysis will use Excel
functions (formula) for calculation and few steps to tabulate the data on another worksheet and
to create chart in a graphic manner (see table 10 and figure 5). Following are the steps to analyse
percentage value of Likert scale for its validity and reliability. The steps also include how to
develop graph for its eye ball rolling evaluation of result. However, since the advent of Likert
scale in 1932, there have been debates among the users about its best possible usability in term
of reliability and validity of number of points on the scale [25, 26].

Step 1: Calculate the number of subjects or respondents using the following Excel function.

1. Select a cell and type Count: In the next cell in same row type = Counta (scroll the
entire range in respective column as question) press enter. This will yield total number
of responses.

2. Below the cell 1 above type, Count not responded: Type = Countblank (scroll the entire
column range) and press enter.

3. Next to cell 2 below type Total count: Type = Sum (click 1 + click 2 outcome)

Step 2: For this step select cells and type Strongly agree, Agree, Disagree and Strongly disagree
in a column below the total in above step and use following step for each criteria in Likert scale
from Strongly agree to Strongly disagree.

1. In the cell below the Total type Strongly agree followed by Agree, Disagree and
Strongly disagree one after another.

2. Start calculating from first criteria of Strongly agree: Type = Countif (scroll the entire
column range using constant ($), “ Strongly agree”) and enter. This will give the
numeric number of responses as strongly agree.

3. Drag the cell to fill in the rest of three criteria in cells below and later drag these 4 cells
to fill in all the cells in four rows to get the numeric numbers for each question and
each option (criteria.

4. Now to convert the entire data produced in decimals, click percentage (%) option in
the menu bar on top to give percentage weighting of responses for all 4 options in the
Likert scale.
Step 3: Now it is time to tabulate the data in a new worksheet to develop percentage weighting
of each option in each question in their respective construct. Following are the steps to create
new table.

1. Highlight the entire 4 rows in the worksheet and select a cell in new worksheet and
click paste to select paste special followed by values and transformation and this will
show a table with all the data. Selecting another cell before the data table and type
question or item and below that enter Q1-Q5 (in current example) in each construct.

2. Now the table is ready for developing the graph by clicking insert on top menu bar and
next choose bar chart showing option of 2-dimension and click to get the data in a
graphic manner. Use edit option to type construct name on the top of the graph and
readjust the font size as per the requirement.

Summary:

The current eBook described, how to create a questionnaire based survey-research and its
validation in detail, which may help readers and new researchers to understand descriptive type
of research using a survey design. It may help faculty interested to develop their research skills
in questionnaire-based survey research specially during the Covid-19 pandemic to implement
their innovative ideas with a concrete research outcome and sharing of experiences with global
community. Multiple subject areas like perception of online teaching and its challenges,
technology in teaching and learning, open-book exam, clinical education, assessment of
competency, emergency remote learning and assessment and stresses of learning during
Covid-19 pandemic and many more challenging situation encountered during Covid pandemic
can be researched using newly created and validated questionnaires.

FIGURES

Validation of Questionnaire Based Survey Using Likert Scale: Figures


Figure 1: Flow chart of validation process from design to its usability in research involving
reliability, validity and factor analysis.

Figure 2: CVI classification (Polit and Beck, 2006)and ratting form provided for experts’
opinion with description of construct.
Figure 3: Test of normality using Shapiro-Wilk test assuming null hypothesis rejected here and
the bar graph items mean score of three components (constructs)

Figure 4: Scree Plot keeping with Eigen value of 1 indicates 4 components extracted of
questionnaire designed with 3 constructs initially
Figure 5: Result of Likert scale data analysis collected for validation of questionnaire
using Microsoft Excel in a graphic manner.

TABLES
Validation of Questionnaire Based Survey Using Likert Scale: Tables

Table 1: Calculation of Content Validity Index as I-CVI, S-CVI/Ave and S-CVI/UA based on
expert rating of items

Table 2: Reliability coefficient as Cronbach’s alpha and Cronbach’s alpha if item deleted of
each item in cluster
Table 3: Correlation Coefficient as Pearson’s or Spearman’s depending on normality of
distribution in this case is Spearman’s Correlation Coefficient since normality of data
distribution was not achieved.

Table 4: Component Correlation Matrix and KMO and Bartlett’s test for qualifying the
assumptions of factor analysis for intercorrelation component and data adequacy

Table 5: Correlation matrices with Anti-image Covariance and Anti-image Correlation


between the items to determine the measure of sampling adequacy in PCA
Table 6: Descriptive statistics for normality of distribution checked skewness divided by
standard error and looking for values within the range of -1.96 to +1.96 and the Communalities
as initial and extracted values using Principle Component Analysis.

Table 7: Total variance explained as Eigen value, initial extracted sum square and rotated sum
square loading.
Table 8: Component and rotation matrix with extraction method of principal component
analysis using Varimax in Orthogonal rotation.

Table 9: Questionnaire initially created with 3 constructs readjusted after component extraction
and rotation matrix suggesting 4 components (construct) with reshuffling of items.
Table 10: Interpretation of result of Likert scale data analysis collected for validation of
questionnaire
APPENDIX
Validation of Questionnaire Based Survey Using Likert Scale

Appendix Initially developed 15 items Questionnaire clustered into 3 constructs


(components/factors) on a 5-point Likert scale from strongly agreed to strongly
disagreed.
REFRENCES

1. Shahid Hassan. Teaching and Learning Approaches in Clinical Education


during COVID-19 Pandemic: Time to Take-on Assessment as Learning.
International Journal of Science and Research; Volume 9, Issue 11,
November 2020: 1171-1174.

2. Joshi A., Kale S. , Chandel S., Pal D. K. Likert Scale: Explored and Explained.
British Journal of Applied Science & Technology 7(4): 396-403, 2015, Article
no.BJAST.2015.157 ISSN: 2231-0843.

3. Singh YK. Fundamental of Research Methodology and Statistics. New Delhi:


Newage International (P) Ltd. Publisher; 2006.

4. Culbertson JM. What is an attitude? Journal of Cooperative Extension. 1963;


79-84.

5. Schwarz N, Bohner G. The construction of attitudes. In: Tesser A & Schwarz


N, Editors. Blackwell Handbook of Social Psychology. Oxford: Blackwell.
2001;436- 457. 11.

6. Gumucio S, Merica M, Luhmann N, Fauvel G, Zompi S, Ronsse A, Courcaud


A, Bouchon M, Trehin C, Schapman S, Cheminat O. Data collection
quantitative methods, the KAP survey model (knowledge, attitude and
practices). IGC communigraphie: Saint Etienne, France. 2011 Jan:4-7.

7. Taherdoost H. Validity and Reliability of the Research Instrument; How to


Test the Validation of a Questionnaire/Survey in a Research. Int J Acad Res
Management.16;5(3):28-36.

8. Goni, M., Naing, N., Hasan, H., Wan-Arfah, N., Deris, Z., Arifin, W., Hussin, T.,
Abdulrahman, A., Baaba, A. and Arshad, M., 2020. Development and
validation of knowledge, attitude and practice questionnaire for prevention
of respiratory tract infections among Malaysian Hajj pilgrims. BMC Public
Health, 20(1).

9. Connell J, Carlton J, Grundy A, Taylor Buck E, Keetharuth A, Ricketts T et al.


The importance of content and face validity in instrument development:
lessons learnt from service users when developing the Recovering Quality of
Life measure (ReQoL). Qual Life Res. 2018;27(7):1893-1902.
10. Gillham B. Developing a questionnaire (real world research). London:
Continuum, 2000.

11. Oppenheim AN. Questionnaire design, interviewing and attitude


measurement. London: Continuum, 1992.

12. Sapsford R. Survey research. London: Sage, 1999.

13. McColl E, Thomas R. The use and design of questionnaires. London: Royal
College of General Practitioners, 2000.

14. Lance, C.E., Butts, M.M. & Michels, L.C., 2006. The Sources of Four Commonly
Reported Cutoff Criteria: What Did They Really Say? Organizational Research
Methods, 9(2), pp.202–220.

15. Polit D. F., Beck C. T. The content validity index: Are you sure you know what’s
being reported? Critique and recommendations. Research in Nursing and
Health; 29 (5): 489-97.

16. Yusoff M.S.B. ABC of content validation and content validity index
calculation. Education in Medicine Journal. 2019;11(2):49–54.
https://doi.org/10.21315/ eimj2019.11.2.6

17. Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-
Majd, H., and Nikanfar, A. R. (2015). Design and implementation content
validity study: development of an instrument for measuring patient-centered
communication. J. Caring sci. 4:165. doi: 10.15171/jcs.2015.017

18. Lynn, M. R. (1986). Determination and Quantification of Content Validity.


Nursing Research, 35(6), 382–385. http://doi.org/10.1097/00006199-
198611000-00017

19. Tavakol, M., and Dennick, R. (2011). Making sense of Cronbach’s alpha.
International Journal of Medical Education, 2, 53–55. Doi:
10.5116/ijme.4dfb.8dfd

20. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and


applications. Journal of applied psychology, 78(1), 98.

21. Pituch, K. A. and Stevens, J., Applied multivariate statistics for the social
sciences: Analyses with SAS and IBM’s SPSS (6th ed.), Taylor & Francis, New
York, 2016.

22. Hair, J. J., Black, W.C., Babin, B. J., Anderson, R. R., Tatham, R. L., Multivariate
data analysis, Upper Saddle River, New Jersey, 2006.
23. Pallant, J., SPSS survival manual: a step by step guide to data analysis using
SPSS, Open University Press/ Mc Graw-Hill, Maidenhead, 2010.

24. Hair, J.F., Anderson, R.E., Tatham, R.L., and Black, W.C., Multivariate data
analysis (5th ed.), N J: Prentice-Hall, Upper Saddle River, 1998.

25. Colman AM, Norris CE, Preston CC. Comparing rating scales of different
lengths: Equivalence of scores from 5- point and 7-point scales. Psychological
Reports. 1997;80:355-362.

26. Preston CC, Colman AM. Optimal number of response categories in rating
scales: reliability, validity, discriminating power, and respondent
Preferences. Acta Psychologica. 2000;104;1-15.

INDEX
Action Research, 18 Internal Consistency, 15
agree, 11, 20, 27, 38, 54, 73 Interpretation, 47, 53, 63, 73, 74, 75, 78
Alumni, 14 items, 6, 11, 12, 13, 14, 15, 19, 27, 28, 37, 38, 39, 41, 43,
ANCOVA, 53 44, 45, 46, 47, 52, 56, 58, 60, 62, 64, 72, 73, 74, 75,
ANOVA, 52, 53 76, 77, 78
Anti-image Correlation, 60 KMO, 59
Anti-image Covariance, 60 Kolmogorov Smirnov, 42
Assumption, 73 kurtosis, 42
asterisk, 42 Learning activity, 75
Average, 43, 44, 45 Likert scale, 6, 11, 12, 13, 15, 19, 20, 26, 27, 28, 38, 43,
bar graph, 56 44, 45, 53, 54, 57, 63, 64, 73, 74, 75, 76, 77, 78
bibliographies, 18 Linear Scale, 27
box plot, 42 Lynn’s Criteria, 44
Case Study, 18 micro-credential, 6, 7, 26, 28, 74, 76
CFA., 52, 53 Microsoft Excel, 14, 37, 39, 53, 57, 72, 73
Cleaning the Data, 38, 39 Missing Value, 39
close questions, 19 Mixed Method, 18
coding, 15, 37, 38, 39 Module, 78
coefficient, 15, 39, 45, 47, 58, 73, 74, 75, 77 multifactorial ANOVA, 53
Comparative Research, 17 multimedia, 26
construct, 11, 12, 14, 15, 43, 44, 45, 52, 53, 54, 55, 62 multiple choice, 27
content validity, 12, 13, 14, 15, 43, 44, 66, 74, 75, 77 negative connotations, 14, 15, 37, 72, 76
Content validity index, 43, 75 Normality of Data, 41, 73
Content Validity Index, 14, 43, 44, 53, 58 null hypothesis, 42, 56
correlation, 15, 39, 43, 45, 46, 47, 52, 77 numeric values, 15, 37, 39, 72
Covid pandemic, 4, 6, 7, 11, 19, 54 Numeric Values, 37
Covid-19, 6, 7, 11, 18, 19, 54 Observational Research, 18
Covid-19 pandemic, 6, 7, 11, 18, 19, 54 observational studies, 19
Cronbach’s alpha, 14, 15, 16, 45, 46, 53, 58, 74, 77 Online surveys, 19
data cleaning, 14, 39, 74, 76 Open-ended items, 19
Data Collection, 14, 37 Orthogonal, 61
descriptive, 6, 12, 13, 16, 17, 18, 19, 41, 42, 54 Output table, 42
Descriptive Research, 17 Paper surveys, 19
dialogue box, 41, 42, 46 Pearson’s, 15, 39, 43, 45, 46, 59
Distribution, 41, 73 Pilot Study, 14
do not agree, 20 Principal Component Analysis, 12, 14, 15, 73
EFA, 52, 53 Question, 26
Entrepreneurship, 26 Questionnaire, 13, 14, 15, 18, 19, 43, 45, 52, 55, 58, 62,
excel worksheet, 28 64, 74, 75
Experimental Design, 17 raw data, 15, 28, 37, 38, 53, 74
explanatory, 12, 17, 18, 52 recoding, 15, 37, 74, 76
Explanatory Research, 17 Recoding the Raw Data, 38
exploratory, 12, 17, 18, 52, 75 Reformatting, 15
Exploratory Research, 17 Rensis Likert, 19
extraction, 12, 14, 61, 62, 73 Research, 7, 17, 18, 21, 65, 66, 75, 79
face validity, 14 research designs, 13, 17
Face Validity, 37 research methodology, 16
factor analysis, 12, 14, 15, 19, 37, 52, 55, 59, 73, 74, 75, respondents, 6, 15, 17, 18, 19, 20, 26, 27, 37, 38, 43,
77
53, 72, 77
factor reduction, 12
Response, 19, 26, 28
Factors Utility, 52
rotation, 12, 14, 15, 61, 62, 73, 77
Fictitious Data, 72, 73
rotation matrix, 61
Flow chart, 55
Sample CVI, 44
Focus group survey, 19
Scree Plot, 56
Full Case Survey, 14
Shapiro-Wilk, 42, 56
generalizability, 16, 19
significant, 42
George and Mallery, 46
skewness, 42, 60
Google Form, 29 Spearman’s, 15, 39, 43, 45, 46, 59
histogram, 42 SPSS, 14, 37, 39, 40, 41, 46, 47, 53, 66, 67, 72, 73, 74, 76
Individual CVI, 43 SPSS data sheet, 15, 39, 72
inferential statistical, 16, 78 strongly agreed, 11, 19, 20, 37, 53, 64
internal consistency, 15, 37 Strongly disagree, 20, 27, 54, 73
Survey Design, 17 Universal Average, 43
survey items, 15 usability, 6, 16, 53, 55, 74, 75, 78
Take-home Exercise, 28, 72 validated instrument, 6, 11, 16, 17
Task, 28, 72, 73, 76, 77, 78
Validation, 6, 11, 13, 15, 19, 37, 55, 58, 64, 74
Telephone survey, 19
variances, 52
Total variance, 61
Varimax, 61
Transfer, 41, 46, 72
video, 28, 75, 76, 77, 78
transform, 41
VOR Lecture, 75, 76, 77, 78
Uncleaned, 72
Waffle sign, 26
unengaged, 15, 38, 72

TAKE-HOME EXERCISE

Exercise 1: Task using Fictitious Data Provided on Microsoft Excel:


Please follow the links to download fictitious data and ETI inventory (data link 3) and
perform the task as asked. Now refer to steps, how to perform each method provided in the
text along the appropriate site in method section and match it with appropriate table and
figure for better understanding. and follow through to do the take home exercises (task) as
following.

ETII Uncleaned Raw Fictitious Data Link 1 in Microsoft Excel: https://bit.ly/3iypoN9

Task 1: Using the link above download the fictitious data and recode the Liker Scale
response of 50 respondents into their numeric values

ETII Uncleaned Numeric Fictitious Data Link 2 in Microsoft Excel:


https://bit.ly/3ghZ8Fk

Task 2: Using the link above download the fictitious data and clean the data of unengaged
participants using standard deviation statistics in Excel. In doing so do have a look on
participants ID number and reorganize the ID number after deleting the unengaged
participants from the worksheet. Subsequently transfer the file to SPSS worksheet.

ETII Cleaned Numeric Fictitious Data Link 3 in Microsoft Excel and Table of
Inventory: https://bit.ly/35aPj5B

Task 3: Using the link above download the fictitious data and the 10 items
Entrepreneurialship Training (ETI) Inventory. Identify the items written with positive and
negative connotations and labelled those as N (written with positive connotation) and R
(written with negative connotation). Next using the fictitious data 3, recode it to reverse order
in worksheet.

Task 4: Data Transfer from Microsoft Excel to SPSS Data Sheet:

Using the Link 3 of fictitious data in Microsoft Excel, transfer data to SPSS creating a new
data sheet and browse through the variable and data view to write down the label and values
wherever required. Also check the data for measure appropriate for each variable in SPSS
variable view.

ETII Uncleaned Fictitious Data Link 4 in SPSS: https://bit.ly/3iw4Eps

Task 5: Using the link above (Data Link 5) download the fictitious data in SPSS, identify if
there is any missing values and clean the data of those missing values. After identifying the
missing value (you may use number 999) make sure it is mentioned against each question in
variable view of SPSS data sheet.

TAKE-HOME EXERCISE

ETII Cleaned Fictitious Data Link 5 in SPSS: https://bit.ly/3gtqTJZ


Task 6: Assumption Check of Normality of Data Distribution
Using the Link 6 of fictitious data in SPSS, run the appropriate test to check the normality
of data distribution following the appropriate steps and interpret the output table and its
impact on subsequent selection of appropriate statistical test of reliability.

Task 7: Reliability Coefficient Using SPSS:

Using the Link 6 of fictitious data in SPSS perform the reliability tests of coefficient
choosing the appropriate statistical method and interpret the data.

Task 8: Factor Analysis Using Principal Component Analysis (PCA)

Using the Link 6 of fictitious data in SPSS, On SPSS worksheet perform the factor analysis
choosing the right options of statistics, extraction and rotation and determine the factors
extracted and number of variable assigned to those factors. Interpret all relevant output tables
generated as a result of performing the factor analysis.

Exercises 2: Creating Microsoft Excel Data to Perform Task on Excel and SPSS:

Please refer to steps, how to perform each method provided in the text along the appropriate
site in method section and match it with appropriate table and figure for better understanding
and follow through to do the take home exercises (task) as following.

Task 1: Create a fictitious data of 30 participants and 10 variables (questions/items) on a


questionnaire based survey on a Likert scale of Strongly disagree = 1 to Strongly agree = 4

Exercises 3: Data Analysis and Interpretation on Microsoft Excel:

ETII Raw Cleaned Data Link in Microsoft Excel 6: https://bit.ly/35dadRL

Task 5: Download the cleaned data in Microsoft Excel using above link 4 and perform data
analyse and interpretation as percentage weighting of Likert scale options for each variables
(items) and develop the 2-dimension graphic representation.

MICRO CREDENTIAL CPD COURSE


Questionnaire-based Survey: From Validation to Usability

Shahid Hassan MHPE


School of Medicine and IMU Centre for Education at IMU, Malaysia
After attending this micro-credential course, participants will be able to,

1. Identify the types of research design and using hypothetically created data, recode
the Likert scale nominal values to numeric score

2. Apply statistical procedure to of questionnaire-based survey research for recoding


items of negative connotation and data clearing using Microsoft excel and SPSS.

3. Demonstrate calculation of content validity index (CVI) and reliability coefficient


(Cronbach’s alpha) in validation of survey based questionnaire reviewed by experts.

4. Demonstrate appropriate factor analysis in questionnaire based data collected


using Likert scale explaining each and every step associated with it.

5. Analyse questionnaire based data for its usability on a Likert scale based
questionnaire for interpretation.

LO 1: Topic 1, Questionnaires based survey research design using Likert scale

Module 1: Types of research design

Module 2: Questionnaire based survey research using Likert scale

Module 3: Hypothetical questionnaire based survey research data

Module 4: Recording of hypothetically created raw data collected through online survey
using Likert scale

LO 2: Topic 2, Statistical procedure of data cleaning on statistically created data

Module 1: Designing of a questionnaire based survey with appropriate items and creating
the Likert scale

Module 2: Cleaning the data to ensure participants engagement in a Likert based


questionnaire
Module 3: Recording of negatively can notated item

Module 4: Identifying the missing value in data.

LO 3: Topic 3, Content validity and Reliability alpha in questionnaire based survey


using Likert scale.

Module 1: Content validity index both, for individual content validity (I-CVI) index and
scale content validity (S-CVI) index

Module 2: Reliability coefficient for questionnaire-based research

Module 3: Correlation Coefficient as reliability index

LO 4: Topic 4, Principles component analysis as Factor analysis in Questionnaire-


based research

Module 1: Principle component analysis on exploratory factor analysis.

Module 2: Rotation of extracted components in factor analysis.

Module 3: Interpretation of rotated components to redesign factors and associated items.

L O 5: Topic 5, Analysis and Interpretation of Data for Usability of Questionnaire-


based Research

Module 1: Analysis and interpretation of data validation.

Module 2: Descriptive statistics of data analysis and usability

Module 3: Inferential statistics of data analysis and usability.

Learning activity:

L O 1: Topic 1

Module 1: Watch the video (VOR Lecture) on research design.

Module 2: Watch the video (VOR Lecture) on Likert scale as the rubric in questionnaire
based survey.

Module 3: Watch the video (VOR Lecture) about the design of a questionnaire in a survey-
based research created with three constructs (factors) a

1. Participant’s prior knowledge about entrepreneurship for its impact on later achievement
2. Motivation about entrepreneurship if that has some impact on competency acquired

3. Competency developed after attending a programme on entrepreneurship.


Task 1: Create a questionnaire-based survey with five items (questions) using 4-point Liker
scale for 20 participants to respond. At least one or two items of those five should be
negatively connotated questions.

Task 2: Design and develop an online Google form survey using the questionnaire format
based on 4-point Liker scale as developed in task 1 (see Module 3, Task 1) and create the
link to administer it to one respondent (you) to complete the form using the online link to
ultimately download the Excel data sheet for further use in this micro-credential course

Module 4: Watch the video (VOR Lecture) on recoding of a raw date into numeric number
using excel and SPSS.

2. Watch the video on how to transfer data from Microsoft excel to SPSS worksheet.

Task 1: Using a data provided from a questionnaire using a 4-point Likert scale, record the
data into numeric number using Microsoft excel.

Task 2: Same task as task 1 however, using SPSS software.

L O 2: Topic 2

Module 1: Design a questionnaire-based survey as online research to explore students


perceptions about emergency remote teaching during the early days of Corvid pandemic

Module 2: Watch a video (VOR Lecture) about the data cleaning procedures to ensure
participant’s engagement showing their reading and appropriately responding each item in
questionnaire in SPSS using a Likert scale rubric in a questionnaire-based survey.

Task: Using the data provided demonstrate how to ensure participant’s engagement in
survey are included and those not engaged are removed.

Module 3: Watch a video (VOR Lecture) on recording of negatively connotated Items.

Task: Using the data provided demonstrate how to record items with negative connotations.

Module 4: Watch a video (VOR Lecture) on data cleaning to ensure missing data.

Task: Using the data provided demonstrate how to clean the data for missing respondents.

L O 3: Topic 3

Module 1: Watch the video on content validity index of validation process.


Module 2: Watch the video on reliability statistics of validation process using Cronbach’s
alpha.

Module 3: Watch the video on reliability statistics using the correlation coefficient.
Module 1:

Task 1: Determine the 1- CVI, S-CVI/ Ave and S-CVI/ UA of score provided from 6
evaluations on a 15 items based questionnaire using 4 point Likert scale.

Task 2: Using the Lyn’s criteria discuss the outcome of CVI calculated in task 1.

Module 2:
Task 1: Calculate the reliability coefficient of data provided on a questionnaire-based data
and discuss the outcome.

Task2: Using the same data calculate the appropriate standard test of correlation.

L O 4: Topic 4

Module 1:

1. Watch the video (VOR Lecture) on introduction to factor analysis.

2. Watch the video on Principal component analysis.

3. Watch the video on rotation method in PCA

Module 1 Task: Perform the PCA on data provided (download the data using the link)

Module 2 Task: Discuss the items retention or detention from the questionnaire based on
correlation statistic in factor analysis of data provided.

Module 3 Task: Interpret the outcome matrices in task 1 and discuss the likely option of
new questionnaire based on PC result.

LO 5: Topic 5

Module 1: Watch the video on procedure of 1. Analysis, 2. Tabulation, 3. Interpretation and


voice over recorded lecture on usability of questionnaire data.

Module 1 Task: Download the fictitious data using the link below and analyse the data for
tabulation and interpretation using percentage weighting of each items for 4 point Likert
scale.
Module 1 Task 2: Create and compare the new factors in the questionnaire data provided
comparing mean and standard deviation.

Module 1 Task 3:

1. Perform analysis of data not normally distributed.

2. Perform the test of normal distribution on data provided

3. Watch video (VOR Lecture) over recorded lecture on inferential statistical analysis of
opinion based on Likert scale

ABOUT AUTHOR
LinkedIn Profile: https://my.linkedin.com/in/shahid-hassan-29271557

Prof. Dr. Shahid Hassan is currently working in IMU Centre for Education and School of
Medicine in International Medical University at Kuala Lumpur, Malaysia. He has also
worked as Advisor Sindh Social Security Institution, Pakistan in 1998. He joined University
Science Malaysia (USM) in 1998 as lecture in ENT that he later served as Head of
Department from 2002-2004. He completed MHPE from Maastricht University Netherlands
in 2007 and joined Medical Education Department in 2008 in USM.

In 2012, he was appointed as Head of Medical Education Unit in University Putra Malaysia.
2014-2016 he worked as Head of Medical Education Unit in Universiti Sultan Zainal Abidin.
Prof. Shahid has been appointed as Chairperson of various committees related to
curriculum/assessment review and CPC/CPD in four distinguishing universities in Malaysia

Prof. Shahid has been a keen researcher in clinical and social sciences, awarded with
Fundamental Research Grant on professionalism. Honoured with 37 prestigious awards in
research and presentations at national and international levels. These include, World
Intellectual Property Organization (WIPO) Gold Medal for an Innovative Quick Diagnostic
Kit in NPC (EBV Amp Kit) 2004, ITEX (International Invention Technology Exposition)
Award 2005, Salon International Invention Geneva Award 2005, Excellent Scientist Award
2005 and USM Hall of Fame Award 2006 besides number of best paper presentation awards
during 2002 to 2014. He has supervised 15 Master Dissertations and 2 PhD Theses. Shahid
Hassan has published over 150 research papers and two books, including “ Achieving
Excellence in Clinical Education” by Amazon.

Professor Shahid Hassan has been the Overseas Editor of PJORL, and Member Editorial
Board of a number of national and international journals. He has been the Vice President
Pakistan Society of ORL and Head and Neck Surgery and Vice President and EXCO member
of Malaysian Association of Education in Medicine and Health Sciences (MAEMHS).
Currently he is the Lead of Assessment Subgroup of Medical Educators Network (MedNet)
Malaysia.

You might also like