Professional Documents
Culture Documents
Zhou2009 Article DesignOfAQuestionnaireForEvalu
Zhou2009 Article DesignOfAQuestionnaireForEvalu
Zhou2009 Article DesignOfAQuestionnaireForEvalu
DOI 10.1007/s11136-009-9466-2
123
498 Qual Life Res (2009) 18:497–508
that non-medical factors such as changing social roles, of postpartum women in China 0–12 months after delivery
environmental adaptation, interpersonal relationships, (PQOL), consistent with the new needs and changes
psychological disorders, and other social problems after introduced by childbirth. In this paper, we present and
childbirth have become important determinants of the discuss the design procedure of the PQOL.
health of postpartum women. So, any single morbidity or
mortality indicator is inadequate to reflect postpartum
women’s overall health. Methods
To assess disorders with such complex causes, people
have paid increasing attention to ‘‘quality of life’’ (‘‘QOL’’ This study was approved by the ethics committee of Peking
for short), which is defined by WHO as: ‘‘an individual’s University. Participants were informed of the purpose of
perception of his/her position in life in the context of the the study, that participation was voluntary and that infor-
culture and value systems in which he/she lives, and in mation received would be kept confidential.
relation to his/her goals, expectations, standards and con- The methods followed a well-established model used to
cerns’’ [10]. QOL has been widely used as a standard design specific QOL measures, including three phases:
measure of health outcomes in different types of research, (1) item pool development, (2) item pool modification, and
such as QOL surveys in different populations [11], ran- (3) item selection.
domized controlled clinical trials [12], community
intervention programs [13], optimization of health resource Item pool development
distribution [14], and health influence factor studies [15].
QOL measures may be generic or specific [16]. Generic The first phase was to attain content validity, which is the
measures, such as the World Health Organization quality of extent to which a measure reflects the specific intended
life scale (WHOQOL) [17], are designed to be broadly area of content [27]. The following five qualitative methods
applicable across types and severities of diseases, across were used to collect the issues which were important for
different medical treatments or health interventions, and postpartum QOL.
across demographic and cultural subgroups; specific mea-
sures are designed for a particular disease (also named Individual in-depth interview with postpartum women
‘‘disease-specific’’ measures, such as the QOL scales for
cancers [18–20] and those for chronic diseases [21, 22]), In-depth interviews were conducted combined with local
but can also be specific to a particular type of person or an at-home postpartum check-ups by Maternal and Child
age group (also named ‘‘population-specific’’ measures, Health Hospital staff in Laishui County, Hebei Province
such as maternal QOL measures [23]). Compared to gen- and Space Center Hospital staff in Beijing. All women who
eric measures, specific measures are more specific to a met the inclusion criteria (0–12 months after delivery, and
certain group to detect and measure important changes. In willing to take the interview) in these two hospitals’
other words, specific instruments may have higher administrative districts were interviewed. The interview
content validity and higher construct validity, but lower content focused on what the main factors were that affected
generalizability. their QOL after childbirth. With the approval of the par-
Symon et al. designed the first specific QOL scale for ticipants, the interviews were tape recorded.
postpartum women, the mother-generated index (MGI) in
2001 [24]. MGI is an open subjective self-administered Focus group discussion with postpartum women
scale. There are some obvious shortcomings of this kind of
tool [25, 26]: even helped by investigators, participants still In Laishui County, we had discussions with three focus
need to have sufficient cognitive skills to understand the groups: women 0–4 months after delivery, 4–6 months
questions well and write informative answers, and the after delivery, and 6–12 months after delivery. There were
answers might be easily influenced by the environment and 4–6 participants in each group. The discussions included
emotional state of the participants. These shortcomings not only the content of the in-depth interview above, but
have restricted the use of this scale, especially in some also participants’ opinions about other mothers’ lives. With
developing countries, including China. Recently, Hill et al. the approval of the participants, the discussions were tape
[23] developed a new instrument (MAPP-QOL) to measure recorded.
maternal quality of life during the early postpartum period.
However, validity of this instrument to assess postpartum Participant observation
QOL among Chinese women was uncertain.
Our study was conducted from 2004 to 2006 to design a The authors stayed with families with a 0- to 12-month-old
specific, valid, and reliable instrument for measuring QOL child for an entire day to observe a postpartum woman’s
123
Qual Life Res (2009) 18:497–508 499
daily life. During this day with the family, we were able to Item pool modification
get more sensitive information. With the approval of the
families, most of the discussions about mothers’ QOL were This phase included two steps.
tape recorded.
Item pool modification 1
Expert and female health care worker consultation
The aim of this step was to identify potential problems of
A wider, more informal consultation was undertaken with acceptability, relevance, and comprehension, and then to
professionals and female health care workers in Laishui revise the primary item pool. Maternal health experts were
County and Beijing. We sent them an e-mail to introduce invited to review the items already generated, check for
the research, what we wanted to discuss, and guidelines on other issues previously overlooked, modify the question-
how to give their feedback. They were asked to write down naire as necessary, and determine the optimum phrasing for
their opinions on what the important issues of postpartum each question.
women’s lives influenced by childbirth are and which
issues should be included in the questionnaire, and send Item pool modification 2
them back to us by e-mail or mail. We then compiled notes
from all the opinions without changing their original Furthermore, we interviewed women 0–12 months after
meaning. delivery to improve the item pool revised in modification 1.
Participants were asked to identify any questions they
Review of existing literature thought were irrelevant, difficult, confusing, annoying,
upsetting, or repetitive. Moreover, they were asked to read
To supplement the above means of item generation, a lit- the questions carefully, and make any revisions to the
erature review was performed. The following databases phrasing and syntax to be more consistent with their
were searched for any issue related to the effects of speaking style. Based on the analyses, some new items
childbirth on the QOL of the postpartum woman: were generated and added to the already existing ones,
MEDLINE, PubMed, EMBASE, ProQuest, ScienceDirect some items were dropped, and some items were revised to
OnSite, SPRINGER, CNKI. We clipped and pasted all the make the question more easy to understand and more
key sentences or paragraphs into a single WORD file. relevant.
After the tapes and notes were collected, we transcribed By the end of this phase, the final item pool was
the tapes into the WORD file. Each transcript or note was determined, and the authors believed that a high level of
clearly marked with the name of the interviewee, expert or content validity had been achieved and so were confident
health care worker, the date and place and any other rele- that no conceptually new issues would arise and no further
vant details. major refinements would be necessary.
We then reviewed all these transcripts and notes, and
highlighted key phrases about issues related to QOL. We Item selection
analyzed all the key phrases and cut out and pasted
together all the key phrases on a single area, using area Materials
heading.
According to the results of this qualitative data analysis, Data for item selection were generated in two ways: expert
we wrote down as many items as we could. We tried to use interview and a field survey among postpartum women.
the mothers’ own words from the transcript as much as We invited experts to review and score each item, which
possible. The answer options for these items used five- quantified the subjective opinion about the importance of
point Likert scales, containing response categories con- each item. The importance of each item was scored using a
cerned with Intensity [such as ‘‘(1) Not at all, (2) Slightly, 0–100 scale, with 0 indicating the least importance and 100
(3) Moderately, (4) Very, (5) Extremely’’], Frequency indicating the most importance. We then interviewed each
[such as ‘‘(1) Never, (2) Rarely, (3) Sometimes, (4) Often, expert to better understand his or her opinions on why the
(5) Always’’] and Evaluation [such as ‘‘(1) Very dissatis- items were important or not.
fied, (2) Dissatisfied, (3) Neither satisfied nor dissatisfied, In addition, we conducted a field survey using the item
(4) Satisfied, (5) Very satisfied’’]. The PQOL domains were pool as modified in the last step in two fields (Laishui and
scored as summed rating domains and transformed on a 0– Beijing). We took a convenience sample from mothers
100 scale, with 0 indicating the poorest QOL and 100 taking their babies for vaccination or other clinic services.
indicating the best QOL. At the end of this phase, a primary The entry criteria included: mothers had to be 18 years or
item pool was generated. older, at 0–12 months after childbirth, and able to complete
123
500 Qual Life Res (2009) 18:497–508
a self-reporting questionnaire. About 10% of the partici- Item-Removed Cronbach’s Alpha Method
pants were retested after about 2 weeks. All tests were
completed in the clinics by participants themselves. This method chooses items according to the influence on
internal consistency of domains after removing a single
Statistic and selection criteria item. First, we statistic the ‘‘alpha if removed’’ for each
individual item. Then we ranked these ‘‘alpha if removed’’.
Based on the data collected from the experts and the field If ‘‘alpha if removed’’ for one item was lower than that for
survey, items were analyzed for inclusion in the final other items, we concluded that removing this item would
questionnaire by seven independent measures: expert relatively much lower the internal consistency. So, the
scoring method, a score for item importance, factor lower the item-removed Cronbach’s alpha, the stronger the
analysis method—factor load, coefficient of variation recommendation to retain the item.
method—coefficient of variation, item-removed Cronbach’s
alpha method—Cronbach’s alpha of a domain after
Item-Expected Domain Correlation Method
removing an item from it, item-expected domain correla-
tion method—correlation coefficient of an item with its
This method chooses items according to their representa-
expected domain, item-unexpected domain correlation
tiveness. We calculated the correlation coefficient between
method—number of high correlations with unexpected
an item and its expected domain one by one. If the corre-
domains, and test–retest correlation method—a test–retest
lation coefficient was higher than a pre-set 0.4, then the
correlation coefficient. All of these were carried out to
item was considered to be representative and this item was
evaluate the item performance and to rank the items on the
recommended for retention.
basis of reliability and validity. Each analysis indepen-
dently answered the question: ‘‘Should this item be
retained?’’ If at least five of the seven analyses concluded Item-Unexpected Domain Correlation Method
the item should be retained, then the item was retained. All
statistics were done by Statistical Package for Social Sci- This method chooses items according to their indepen-
ences (SPSS 13.0). dence. We calculated the correlation coefficient between an
item and other unexpected domains and counted with how
Expert Scoring Method many unexpected domains the item had a high correlation
(r [ 0.4). If the item had a high correlation with two or
This method chooses items according to their importance. more unexpected domains, respectively, then this item was
As mentioned before, we invited experts to score each item recommended for removal.
for importance using a 0–100 scale. We calculated the
average score of importance for each item, and then ranked Test–Retest Correlation Method
the average scores. The items with an average score
higher than all items’ mean score were recommended for This method chooses items according to their stability. As
retention. mentioned before, we retested the modified item pool in a
small group after about 2 weeks. We examined the corre-
Factor Analysis Method lation between the first test and the retest. If the test–retest
correlation coefficient was higher than the pre-set 0.7, then
Principal components factor analysis with VARIMAX this item was recommended for retention.
rotation was used to choose items according to their rep-
resentativeness. If the load of the item on its expected
domain was higher than the pre-set standard of 0.3, then Results
this item was recommended for retention.
A primary item pool with 167 items was developed
Coefficient of Variation Method
In the item pool development phase, 28 and 24 women in
Coefficient of variation (CV) was used to compare item Laishui and Beijing, respectively, participated in the indi-
variation to avoid the influence of different means vidual in-depth interview. Time after childbirth among
of data
sets in our study. The formula is: CV ¼ S=X 100%; these women ranged from 8 days to 12 months. Another 16
S = standard deviation, X ¼ Mean: If CV of the item was families with a 0- to 12-month-old child accepted the 1-day
higher than the average CV of all items, then this item was visit. Six professionals and 15 female health care workers
recommended for retention. in Laishui and Beijing participated in the consultation and
123
Qual Life Res (2009) 18:497–508 501
provided their opinions on important QOL issues of women In total, after the analysis, 167 items were generated for
0–12 months after delivery. inclusion into the primary item pool.
Important issues obtained from the qualitative methods
were grouped into four categories: child care, physical
function, psychological function and social support. Child A 101-revised-item pool was determined
care refers to the mothers’ feelings about their child,
including his or her physical health, accidents, develop- In the item pool modification phase, ten maternal health
ment, character, and feeding; physical function refers to the experts and 15 women 0–12 months after childbirth from
mothers’ perception of contraception, sleeping, rest, the Maternal and Child Hospital in Laishui took part.
energy, pain, sex life, ability to work, and body image; Experts believed and mothers found most of the 167
psychological function refers to the mothers’ perception of items in the primary item pool easy to understand, but did
confidence in good child care, attitude towards child care, identify some difficult ones. According to their sugges-
role adaptation, negative feelings, and positive feelings; tions, we dropped 75 useless or disturbing items, revised 25
social support refers to the mothers’ social activities, unclear items, and generated nine new items. Thus, a
family relationships, chores/childcare help, house envi- revised item pool with 101 items was obtained. Figure 1
ronment, family economy, and outside environment. shows the structure of the 101-revised-item pool.
Physical health
Accident
Development
Child Care 16 items
Child character
Feeding
Contraception
Energy
Pain
Physical Function 26 items
Sex life
Work ability
Body imagy
Item Pool
Positive feeling
Social activity
Family relationship
House environment
Family economy
Outside environment
123
502 Qual Life Res (2009) 18:497–508
The PQOL with 60 items was formed which indicated that this item was independent, so this item
should be retained. Test–retest correlation: the test–retest
In the item-selection phase, ten experts reviewed the 101 correlation coefficient was 0.905 ([0.7), so this item
items and gave them importance scores. In addition, a total should be retained. All seven screening methods concluded
of 200 postpartum women all 0–12 months after delivery that item CH11 should be retained, which indicated that
(100 in Beijing and 100 in Laishui) were tested with the this item possessed many good traits.
101-item pool, and 20 of them were retested after about The number of items retained by the seven screening
2 weeks. methods independently were: experts scoring, 62 items;
The range of urban women’s age was 19.6–39.3 years, factor analysis, 58 items; coefficient of variation, 52 items;
with a median age of 30.4. The range of rural women’s age item-removed Cronbach’s alpha, 51 items; item-expected
was 19.5–37.5 years, with a median of 25.7. Only 1.0% of domain correlation, 75 items; item-unexpected domain
urban participants had an education lower than primary correlation, 92 items; and test–retest correlation, 90 items.
school, while 10.2% of rural participants did. About 62.0% According to the rule that any item retained by at least five
of urban and 54.5% of rural participants had a Caesarian of the screening methods should be included, 60 items were
section. About 90.0% of the urban participants had only selected.
one child, while 66.2% of the rural ones did. The com- The 60-item self-administered questionnaire grouped
pletion rate of the questionnaire was 99.8%. into four domains (child care, physical function, psycho-
Based on the importance scores and data from the test logical function, and social support) and 20 subject areas.
with the 101-item pool, 60 items were recommended for Compared to the 101-item pool, the areas of sex life, child
retention by at least five of the seven tests. These 60 items care attitude, and child character were deleted in the 60-
were selected. Here is an example of one item that was item PQOL, since there was no item in these areas that was
selected: retained during at least five of the seven selections.
CH11 Do you worry that your child will fall sick?
The streamlined PQOL with 40 items was formed
(1) Never (2) Rarely (3) Sometimes (4) Often
(5) Always
In order to further streamline the survey tool, we selected
Expert Scoring: the average importance score of the two items for each of the 20 subject areas, resulting in a
item was 91.00, which was higher than the mean score of total of 40 items in the streamlined PQOL, the structure of
all items (87.3). So this analysis concluded that this item which is shown in Fig. 2. If more than two items scored the
should be retained. Factor analysis: the results of the factor same in terms of the criteria for retention, we selected the
analysis revealed that the load of this item on child care two items whose meanings were closer to each other to
domain was 0.401, which was higher than the pre-set achieve higher internal consistency. Screening results are
standard of 0.3. So this analysis also concluded that this shown in Tables 1 and 2.
item should be retained. Coefficient of variation: CV of this The flow diagram in Fig. 3 shows the results for each step.
item was 0.3746, which was higher than the average
(0.2642). So this analysis also concluded that this item
should be retained. Item-removed Cronbach’s alpha: the Discussion and conclusions
Cronbach’s alpha of child care domain was 0.6666 after
removing this item, which was lower than when after The design of this 40-item PQOL was based on a standard
removing other items (0.6740, 0.6785, 0.6849, 0.6675, development procedure for a specific QOL tool [28]. The
0.6809, 0.6736, 0.6789, 0.6988, 0.6746, 0.6952, 0.6894, content validity of the PQOL is ensured by using a wide-
0.6848, 0.6964, 0.6945, and 0.6772, respectively), indi- ranging collection of postpartum QOL issues.
cating that removing this item would greatly lower the In this study, we focused on groups of women at dif-
internal consistency (the overall Cronbach’s alpha of child ferent postpartum phases because in our previous study we
care domain was 0.7060). So, this analysis also concluded found that issues related to having had a baby were very
that this item should not be removed. Item-expected similar across different groups of women. The qualitative
domain correlation: the correlation coefficient between this results of the interviews conducted during this study
item and child care domain was 0.546 ([0.4), which showed this again. Although respondents at different
indicated that this item was representative, so this item postpartum stages did have physiological differences, what
should be retained. Item-unexpected domain correlation: they considered as the most important issues for their QOL
the correlation coefficient between this item and physical were similar, including the health of the baby, opportuni-
function, psychological function and social support domain ties for them to rest and support from other family
were 0.099, 0.147, and 0.158, respectively (all \0.4), members. This phenomenon might well be strongly
123
Qual Life Res (2009) 18:497–508 503
Physical health
Accident
Child Care 8 items
Development
Feeding
Contraception
Energy
Work ability
Body image
PQOL
Role adaptation
Psychological
Negative feeling 8 items
Function
Positive feeling
Social activity
Family relationship
Chores/childcare help
Family economy
Outside environment
influenced by the culture and social structures of modern- positive feeling area). Moreover, new additional domains,
day China. Some issues are consistently identified as which were considered crucial in influencing postpartum
important topics throughout the different postpartum stages QOL, were added (child care and social support domain).
of the women’s lives, such as the ‘‘only child’’ phenome- Although these two were ‘‘new’’ domains, the areas
non and the relationship with the husband’s family. While included in them were common issues of concern to both
focusing on women at different points in their first post- researchers and postpartum women. For example, an issue
partum year may have resulted in different mean scores of with breast-feeding has been frequently reported [29]. The
QOL for different groups of women, we believe that this specific domains or areas may help in identifying causes to
does not influence the internal consistency of the tool. an impairment of QOL specifically related to childbirth and
The PQOL contained physical function and psycholog- may allow female health care workers and researchers to
ical function domain, in which some areas (e.g., sleep, design appropriate interventions.
energy, positive and negative feeling) were close to other There are irreconcilable demands in designing the
well-developed QOL scales [17]. However, some areas and questionnaire: we want both maximum validity and reli-
items considered important in other scales were excluded ability and universal acceptance. We hope to generate as
(e.g., appetite), some specific areas and items were added many items as possible to cover all QOL issues to achieve
(e.g., confidence in child care and maternal role adaptation maximum validity and reliability, but the great number of
area were added to psychological function domain, and items would present a major problem—the extreme length
item-‘‘Are you happy being a mother?’’ was added to the of an assessment would preclude its use in many clinical
123
504 Qual Life Res (2009) 18:497–508
123
Qual Life Res (2009) 18:497–508 505
123
506 Qual Life Res (2009) 18:497–508
Table 2 continued
Code Item
and research settings [30]. It is difficult for unsupervised selection is problematic, due to the subjective component
individuals to complete an instrument that requires more of decisions regarding inclusion or exclusion. The strategy
than 10 min [31]. It took about 15 min to complete the of comprehensive selection in this study was designed to
101-item pool, which meant the item pool might be too decrease the uncertainty caused by this subjective nature,
long to be well accepted. Furthermore, the item pool might and to enable the whole questionnaire to be valid and
also contain a large number of items which may be less reliable. The benefit of the comprehensive strategy has
relevant to postpartum QOL. The best option to obtain a been reported in other research [36].
reliable, valid, and acceptable questionnaire is to select
subsets of items.
Researchers usually choose different methods to select Conclusions
the items according to the aim of the research. Some
methods are applied often, such as the distribution of This is the first attempt to design a Chinese QOL mea-
responses [32], item-total correlations [33], the ability to surement for postpartum women (PQOL) through a
discriminate ill and well populations [34] and the results of standard development process. The instrument has an
factor analysis [35]. However, the procedure of item interpretable and multi-area factor structure. Four domains
123
Qual Life Res (2009) 18:497–508 507
123
508 Qual Life Res (2009) 18:497–508
generic quality-of-life measures. Journal of Cardiopulmonary 31. Scientific Advisory Committee of the Medical Outcome Trust.
Rehabilitation, 18(2), 139–144. doi:10.1097/00008483-19980 (2002). Assessing health status and quality-of-life instruments:
3000-00007. Attributes and review criteria. Quality of Life Research, 11, 193–
23. Hill, P., Aldag, J. C., Hekel, B., et al. (2006). Maternal post- 205. doi:10.1023/A:1015291021312.
partum quality of life questionnaire. Journal of Nursing 32. Tennant, R., Hiller, L., Fishwick, R., et al. (2007). The Warwick-
Measurement, 14(3), 205–220. doi:10.1891/jnm-v14i3a005. Edinburgh Mental Well-being Scale (WEMWBS): Development
24. Symon, A. (2002). Postnatal quality of life assessment: intro- and UK validation. Health and Quality of Life Outcomes, 5, 63.
ducing the mother-generated index. Birth (Berkeley, CA.), 29(1), doi:10.1186/1477-7525-5-63.
40–46. doi:10.1046/j.1523-536X.2002.00154.x. 33. Sayers, M. S., & Newton, J. T. (2006). Patients’ expectations of
25. Symon, A., Mackay, A., & Ruta, D. (2003). Postnatal quality of orthodontic treatment: part 1—development of a questionnaire.
life: a pilot study using the mother-generated index. Journal of Journal of Orthodontics, 33, 258–269. doi:10.1179/14653120522
Advanced Nursing, 42(1), 21–29. doi:10.1046/j.1365-2648.2003. 5021753.
02575.x. 34. Kahler, C. W., Strong, D. R., & Read, J. P. (2005). Toward
26. Symon, A., McGreavey, J., & Picken, C. (2003). Postnatal quality efficient and comprehensive measurement of the alcohol prob-
of life assessment: validation of the mother-generated index. lems continuum in college students: the brief young adult alcohol
British Journal of Obstetrics and Gynaecology, 110, 865–868. consequences questionnaire. Alcoholism, Clinical and Experi-
27. Carmines, E. G., & Zeller, R. A. (1991). Reliability and validity mental Research, 29, 1180–1189. doi:10.1097/01.ALC.000017
assessment. Newbury Park: Sage. 1940.95813.A5.
28. DeVellis, R. F. (2003). Scale development: theory and applica- 35. Wan, C.-H., Meng, Q., Luo, J.-H., et al. (2007). Development of
tions (2nd ed.). Thousand Oaks, CA: Sage. the general module of the system of quality of life instruments for
29. Beck, C. T. (1998). A review of research instruments for use cancer patients: item selection and structure of the general
during the postpartum period. The American Journal of Maternal module. Chinese Journal of Cancer, 26(2), 113–117.
Child Nursing, 23(5), 254–261. doi:10.1097/00005721-19980 36. Hao, Y.-T., Sun, X.-F., Fang, J.-Q., et al. (2004). The study of
9000-00008. statistical method used for item selection. Chinese Journal of
30. Fang, J.-Q. (2000). Measurement and application of quality of life Health Statistics, 21(4), 209–211.
(1st ed.). Beijing: Peking University Press.
123