Article

Journal of Mixed Methods Research

1–22
Using Mixed Methods Ó The Author(s) 2019
Article reuse guidelines:
Research as a Tool for sagepub.com/journals-permissions
DOI: 10.1177/1558689819857530
Developing an Indigenous journals.sagepub.com/home/mmr

Cultural Values Instrument in

Malaysia

Mehdi Taghipoorreyneh1 and Ernest C. de Run1

Abstract
The purpose of this article is to illustrate how mixed methods research can be applied as a rig-
orous method for developing a quantitative instrument responsive to an indigenous culture in
Malaysia. In a sequential research design, a three-round Delphi study was conducted to integrate
open-ended and free-listing data, and to generate a list of items. The items were subjected to a
field test to ensure reliability. A correspondence analysis of scale was employed to examine the
validity of the instrument. This research contributes to mixed methods by illustrating the value
of Delphi groups as part of a mixed methods research design and the utility of mixed methods
research in advancing the validity and reliability of an indigenous cultural values instrument.

Keywords
Delphi, expert, indigenous culture, Malaysia, mixed methods research

The substantial integration of qualitative and quantitative findings and data in developing cul-
tural values instruments is seldom seen, although optimizing instrument validity depends on it
(Hitchcock et al., 2005; Onwuegbuzie, Rebecca, & Judith, 2010). This lack of integration may
partly be due to the traditional philosophical assumptions that qualitative or quantitative
approaches cannot be combined and used together. Another obstacle may be the scarcity of
good examples to which researchers can turn when seeking to integrate qualitative and quanti-
tative methods to develop either qualitative or quantitative instruments, but the full list of rea-
sons is probably more complex. Cultural values are conceptualized as shared beliefs that a
group of people use to make sense of their world (Hofstede, 1980). Researchers know how to
enhance the reliability of the instruments that measure values through quantitative techniques.
They are also aware that values are intangible elements of cultures, a latent feature of societies,
a normative system that is independent of individuals and often deeply internalized (Schwartz,
1999). Therefore, these values can be best accessed and understood through qualitative methods
such as interviews and other forms of observation. As such, a successful assessment of cultural

1
University of Malaysia, Sarawak, Malaysia

Corresponding Author:
Mehdi Taghipoorreyneh, Department of Business and Management, Faculty of Economics and Business, University of
Malaysia, FEB, Level 2, Kota Samarahan, Sarawak 94300, Malaysia.
Email: mehdi.taghipoorreyneh@gmail.com
2 Journal of Mixed Methods Research

values must rely on some combination of the two methods to develop a fuller picture and to
benefit from the strength of each method. Using an integrative approach would be far more
beneficial because it allows for producing results that are more valid and reliable.
Thus, the purpose of this article is to illustrate how mixed methods can be applied as a rigor-
ous method for developing cultural instruments to measure individual differences in personal
cultural orientations. The mixed methods can be used at different cultural levels, but in this arti-
cle, it is applied to elaborate individual cultural values differences in a single homogeneous cul-
ture to show the effectiveness of the mixed methods research. A comparison among cultural
groups requires conducting separate studies in each culture to redefine and add cultural values
from the people within these cultures.

Values, Cultural Values, and Culture

Although the terms ‘‘values,’’ ‘‘cultural values,’’ and ‘‘culture’’ are often used interchangeably,
each term is actually a distinct piece of the larger picture. Values are defined as desirable goals
that guide people to select and evaluate actions, policies, and events (Schwartz, 1999). Values
operate at the individual and group levels. At the individual level, values express broad desir-
able goals that guide and affect the way individuals interpret and perceive events and situations
(Schwartz, 1999). At the group level, values are shared conceptions of what is good, right, and
desirable in solution groups (Hofstede, 1980; Schwartz, 1999). Cultural values are the group
values that members of a society share and develop in response to existential challenges
(Hofstede, 1980; Schwartz, 1999). Cultural values therefore play a crucial role in the way that
social institutions function. Culture is defined as all of a group’s guiding values, rituals, heroes,
and symbols taken together as one big whole (Hofstede, 1980).
To measure differences in cultural values, most previous research in cross-cultural psychol-
ogy has focused on the national level values scales (Fontaine & Richardson, 2005; Terpstra-
Tong, Terpstra, & Tee, 2014). However, national level values are conceptually quite different
from cultural values because they prioritize the values that are shared at the national level and
therefore neglect the cultural fragmentation within each nation (Fischer et al., 2005).
Understanding and being able to assess cultural values initially requires a multidimensional
approach to measure the value differences among individuals in a culture (Sharma, 2010) and
then adding the values that are responsive to cultural levels (Cheung, van de Vijver, & Leong,
2011). Mixed methods research can achieve both targets because it captures the voices of key
informants, the people to whom the instrument will be administered (Ingersoll-Dayton, 2011).

Cultural Assessment Approaches

There are two dominant approaches to cultural assessment: culture-centered and personality-
centered approaches (Clark, 1990). Culture-centered approaches are qualitative methods of cul-
tural assessment (Lenartowicz & Roth, 1999) and primarily follow anthropologists’ point of
view that cultures are so diverse and complex that they cannot be measured but merely
described and observed (Haviland, 1990). Such an approach provides descriptive data that
delineate cultural characteristics. Quantitative results are not produced, although the method
does provide useful information for cross-cultural comparisons (Lenartowicz & Roth, 1999).
The personality-centered approach relies on quantitative measures for describing and identi-
fying culture. This approach assesses culture through two basic methods: the use of proxies
(Adler & Graham, 1989) and value inference (Erez & Earley, 1993). Generally, proxies link
cultural groupings to cultural geographic locations such as nationality, country of origin, and
place of birth (Adler & Graham, 1989). A limitation of this method is the absence of ratio and
Taghipoorreyneh and de Run 3

interval measures to compare or rank cultures based on their characteristics. The proxies are
only used to categorize cultures based on their regional affiliation; thus, the culture measure
was nominal (Lenartowicz & Roth, 1999).
The value inference approach assumes a value-based conceptualization of culture
(Lenartowicz & Roth, 1999). Central to this approach is the proposition that culture is a set of
learned characteristics largely shared by members of a particular group (Triandis, 1996).
Examples of such values are collectivism and power distance (Hofstede, 1997), conservatism
and mastery (Schwartz, 1999). This approach provides interval and ratio measure to compare
cultural scores and to test the hypothesized relationships among cultures as an independent vari-
able, along with dependent variables. Attempts to use scales developed by this approach to
compare the cultural values differences across cultures has encountered several problems
including insufficient attention to cultural nuances (Ingersoll-Dayton, 2011), inappropriate defi-
nition and labeling (Ho, 1996) and unsuitable response sets (Hui & Triandis, 1989).
In the evaluation of the above cultural assessment approaches, Lenartowicz and Roth (1999)
indicated that none of these approaches alone can address an inclusive set of criteria relevant to
cultural assessment. Their suggestion was consistent with Clark’s (1990, p.77) recommendation
for future research based on a ‘‘marriage of methodologies.’’ Although the significance of this
marriage has been regularly utilized in mixed methods research (Creswell & Tashakkori, 2007;
Fetters & Freshwater, 2015), far fewer studies have actually been completed using mixed meth-
ods to develop quantitative instruments assessing cultural values (Onwuegbuzie et al., 2010).
The mixed methods approach is particularly relevant to the development of culturally sensi-
tive instruments because it provides a way to incorporate the perceptions of indigenous people
(Ingersoll-Dayton, 2011). Using an integrated approach would be so much more rewarding
because it would allow each method to address aspects of cultural values and would therefore
yield more comprehensive data (Yin, 2006). Researchers have attempted to propose several
systematic ways to optimize the process of integration in mixed methods studies (Fetters &
Molina-Azorin, 2017; Yin, 2006). The framework of five procedures proposed by Yin (2006) is
broadly reflected in this research. Yin (2006) identified the five procedures as research ques-
tions, units of analysis, sampling, instrumentation and data collection, and analytic strategies.

Methods in Extracting Dimensions of Cultural Values

Much of the work on categorizing the values described in the literature is based on dimension
reduction analysis and can be characterized as quantitative. Typically, instrument developers
conceptualize items to develop both theory-based (Inglehart, 2003; Schwartz, 1999) and
nontheory-based (Hofstede, 1980; Rokeach, 1973) instruments to measure shared values
(Triandis, 1996). Principle component analysis and factor analysis have often been used for
extracting value dimensions (Inglehart, 2003; Sharma, 2010). Despite their wide-scale usage,
dimension reduction techniques have been subjected to no small amount of criticism (D. W.
Stewart, 1981).
Most of this criticism focuses on the misapplication of factor analysis as a tool for clustering
purposes (D. W. Stewart, 1981). Because factor analysis provides a set of coordinate axes,
numerous clusters can be defined by the position of the entities on any one dimension and by
various combinations of dimensions (D. W. Stewart, 1981). This is particularly the case in the
values studies in which researchers are interested not in describing a value by factors but in
describing a factor by values. The ultimate determination of dimensions requires analyses
beyond factoring (Pervin, Cervone, & John, 2005).
Another reason for the popularity of the dimension reduction methods can be attributed to
the availability of computer programs that were designed to use reflective indicators to measure
4 Journal of Mixed Methods Research

concepts. Generally, there are two types of indicators: Reflective and formative. Reflective
indicators are defined as observed variables that are manifest of the latent variables. In contrast,
formative indicators are defined as observed variables that define the latent variables (Jarvis,
Mackenzie, Podsakoff, Mick, & Bearden, 2003). The process for evaluating the validity and
reliability of reflective constructs does not apply to formative constructs (Petter, Straub, & Rai,
2007).
The blind use of reflective indicators is a principal cause of the dissatisfaction with the mea-
surement technique. Jarvis et al. (2003) searched the top four journals in marketing for the 24-
year period from 1977 through 2000 to identify all empirical applications of latent variables
structural equation modeling or confirmatory factor analysis. According to them, 29% of the
formative constructs in the top four journals were incorrectly modeled as having reflective mea-
sures. As such, Jarvis et al. (2003) suggested a set of criteria to identify a construct as either for-
mative or reflective. Following their suggestions, the indicators of values dimensions should be
formative because the value dimension is a composite of these indicators. The use of formative
indicators to measure constructs has become popular in recent years because of the development
of some powerful statistical software, such as partial least square. This research provides gui-
dance on how researchers should approach data reduction analysis in mixed methods studies to
extract the value dimensions and to develop formative indicators.

Delphi Technique
The Delphi method is a pragmatic research methods that was first introduced by the RAND
Corporation in the 1950s (Yousuf, 2007). It is a group method that aims to achieve the most reli-
able consensus of opinions from experts by using a series of questionnaires within a controlled
feedback process (Dalkey & Helmer, 1963). The Delphi technique allows experts to address
anonymously in their own time complex topics in their area of expertise (Yousuf, 2007). There
are many varieties of Delphi that range from qualitative to quantitative and mixed methods
Delphi.
We conducted a mixed methods Delphi study because values are abstract ideals (Rokeach,
1973). Much of the difficulties in understanding the concept of ‘‘values’’ originates from the
fact that values are invisible elements of a culture. Values do not exist in isolation in individu-
als’ minds but rather are linked with other beliefs and are embedded in multiple and complex
layers of social interaction (Hofstede, 1980). The complexity of the concept of ‘‘values’’ has
raised a common concern among sociologists regarding the construct validity of instruments that
measure values (Hitchcock et al., 2005). To enhance measurement validity, it is crucial to have
a clear comprehension of what value is being measured and to have a shared understanding of
exactly what values exist in a culture (Ingersoll-Dayton, 2011; Maltseva, 2016). The Delphi
study allows for greater participation and inclusion from experts in indigenous cultures who are
often left out by traditional instrument developers due to the quantitative nature of their research
methods.
Another reason for using the Delphi method relates to the inherent ability of the technique
to provide both qualitative and quantitative tools to analyze data. ‘‘Delphi straddles the divide
between qualitative and quantitative methodologies’’ (Critcher & Gladstone, 1998. p. 443).
There is a preponderance of studies that show the usefulness of integrating qualitative and
quantitative research in a Delphi process. One well-known mixed methods Delphi study is that
of the effectiveness of the enforcement of new international regulations on ship emissions by
Bloor. Sampson, Baker, and Dahlgren (2015). Another mixed methods Delphi study is Tapio,
Paloniemi, Varho, and Vinnari’s (2011) work involving insightful analyses of the strengths,
weaknesses, and pitfalls encountered when combining qualitative and quantitative information
Taghipoorreyneh and de Run 5

in a Delphi process by studying seven disaggregative policy Delphi material conducted in

Finland from 1999 to 2008. In each case, the mixed methods Delphi study has actually vali-
dated researchers’ findings through integration and has enabled the understanding of a phenom-
enon in greater depth. This potential value makes the mixed methods Delphi a suitable tool to
produce a holistic picture and to capture the internal organization of complex abstract entities
such as cultural values.

Correspondence Analysis of Scales

Many values studies have successfully implemented various forms of analysis to identify the
important values dimensions by which people can be differentiated (i.e., smallest space analysis
in Schwartz [1999]; ecological analysis in Hofstede [1980]; factor analysis in Inglehart [2003]).
The analyses in these studies were based on single items. Here, we use the correspondence anal-
ysis of scales, which allows treating multiple values dimensions and demographic characteris-
tics simultaneously to reveal variation within the target culture. The technique was effectively
used by Maltseva (2016) in his analysis to explore the values structure of Swedish values in pro-
social cooperation.
The technique provides an appropriate visual representation to describe cultural values. Such
a visualization of the mutual effect of demographics and cultural values offers a ‘‘big picture’’
of complex dimensional structures, which is advantageous in studies of cultural models
(Maltseva, 2016). It is a particularly strong option for mixed methods researchers because it
solves the issue of simplification that is entailed in the process of data reduction (Neuman,
2011).

Cultural Context
Asia, with its wide spectrum of people from different cultural backgrounds, continues to be an
interesting place for researchers. Among Asian countries, Malaysia has a remarkable cultural
diversity and provides a fertile ground for studying various aspects of culture. The main ethnic
groups within Malaysia are Malays, Chinese, and Indians, with many other ethnic groups repre-
sented in smaller numbers (Terpstra-Tong et al., 2014). Malays speak in Bahasa, practice their
own customs and observe Islam as their religion. Most Chinese speak Mandarin in their daily
life, with their own dialects, such as Cantonese. They are followers of various religions and
practices such as Buddhism and Christianity (Asma, 1992). Indians in Malaysia live mainly in
West Malaysia, and many still follow their customs from their original country of India.
Despite clear differences in symbols, heroes, and rituals, most of the previous studies in this
context did not find significant differences in the cultural values among these ethnic groups.
For example, Fontaine and Richardson (2005) used the Schwartz Value Scale to compare the
cultural values of Chinese, Malay, and Indian ethnic groups in peninsular Malaysia by using a
quantitative approach. They found that at the cultural-level, 82% of the values are shared by all
three ethnic groups without any significance differences at 5%. L. Lim (2001) found no signifi-
cant difference between the Malays and Chinese in Hofstede (1980) work-related values.
Terpstra-Tong et al. (2014) investigated the three ethnic cultures in Malaysia by examining the
individual-level values of managers and professionals. Based on responses to a Schwartz Value
Survey, they found that Chinese and Indians did not differ significantly across all cultural value
dimensions, which is interesting because these two groups have very different cultural heritages.
Only Malays were found by the researchers to be more conservative and less self-transcendent
than Chinese or Indians. These findings are clearly opposed to the presence of distinct cultural
practices and ethnic artifacts in Malaysia. As such, there is a need to examine how Malaysians
6 Journal of Mixed Methods Research

express their values in everyday life and to develop a methodology to describe and measure
them accurately. In this article, the focus is on the Malays because they are the largest ethnic
group in Malaysia. Generally, a Malay, as defined in article 160(2) of the Malaysian constitu-
tion, is a person who professes Islam, habitually speaks the Malay language, and conforms to
Malay customs known as adat.

Method
Overview of the Study Design
An exploratory sequential mixed methods design, our study comprises two phases: a Delphi study
and field test phase. The Delphi study began with a qualitative opening round wherein the experts
were asked to provide responses to open-ended questions concerning the Malay culture. After gen-
erating the data input, the emphasis changed to quantitative analysis and the collection of data to
achieve a consensus of the experts’ opinions on the emerged values and items. At the same time,
the experts were asked to respond to other open-ended questions for clustering purposes.
Once the instrument was developed, it was subjected to a field test to assess the appropriate-
ness of each item and the test reliability of the instrument. The main steps for the mixed meth-
ods Delphi process and field test study procedure are depicted in Figure 1. The necessary steps
for this technique are below.

Phase I Delphi Study

Panel of Experts. The validity of the results of the Delphi technique lies in its ability to draw on
expertise (Yousuf, 2007). In this study, the expertise of the panel members was defined based on
their experience and knowledge of Malay culture. The experts’ level of knowledge was assessed
based on their scientific profile and publications. Subsequently, the experts’ level of experiences
was evaluated in terms of professional accomplishments in multiple domains of professional
functioning relevant to Malay culture. To this end, three of the following four criteria had to be
meet: (a) publication in the area of Malay values in one of the relevant major journals or the pub-
lication of a book related to Malay values; (b) a minimum of a PhD in the various fields related
to Malay culture such as anthropology, history, and so on; (c) the experts needed to be Malay;
and (d) the experts needed to be employed for at least 5 years as a researcher at any cultural insti-
tution or local higher learning institution in an area related to Malay culture. The existence of
published research as a mark of expertise meant that the potential respondents had their expertise
assessed by two or three anonymous referees during the publication process.
As presented in Table 1, a total of seven experts were selected by purposive sampling. A
panel of 5 to 10 members was deemed to be sufficient for an exploratory study with a homoge-
neous population (Martino, 1973). A cover letter, the rounds questionnaire and a study informa-
tion sheet were distributed to the seven participants by e-mail. To increase the survey’s
accessibility, it was produced in Microsoft Word 2013, which could be completed off-line and
returned via e-mail.

Number of Rounds. Three rounds were found to be sufficient to yield consensus among the
panelists. This three-round Delphi process involved a series of questionnaires in which a ques-
tionnaire was subsequently formulated from the respondents’ feedbacks and comments to the
preceding questionnaire. The number of rounds depends on the level of consensus and suffi-
ciency of information (Sobaih, Ritchie, & Jones, 2012).
Taghipoorreyneh and de Run 7

Figure 1. Procedural diagram of using a mixed methods Delphi technique to create an indigenous
cultures measurement instrument.

First Round: Qualitative Data Collection. The purpose of Round 1 was to generate a comprehen-
sive list of the values relating to Malay culture. This round used open-ended questions to glean
as much information as possible (Taylor & Judd, 1989). The experts were asked to (a) describe
the Malays’ ethnic groups, (b) identify specific characteristic of Malays, and (c) indicate up to
10 basic and fundamental values of Malays.
The first two questions provided background information regarding Malay values. To ana-
lyze these two questions, the following steps were taken to ensure proper rigor. First, all data
were transcribed into a Microsoft Word file. Second, from the database, the segments of text hat
related to a single concept were tagged. If responses were repeated, the creation of new themes
emerged.
The purpose of the third question was to generate an initial pool of values that reflected dif-
ferent aspects of Malay culture. The question asked the participants to name basic values
among Malays. This method is referred to as a free-listing technique and has been used
8 Journal of Mixed Methods Research

Table 1. Experts Profile.

Degree University Field of study

PhD Universiti of Malaya Anthropology
PhD Universiti Utra Malaysia Communication
PhD Universiti Putra Malaysia Anthropology
PhD Universiti Malaysia Sarawak History
PhD Universiti Malaysia Sarawak History
PhD Universiti Putra Malaysia Management
PhD Universiti Malaysia Sarawak Anthropology

extensively by researchers (Maltseva, 2016). It was used at the initial stage of the Delphi study
to outline the salient values among Malays. The idea is that the values that appear frequently in
expert responses are probably more important than the values that appear infrequently.
First Round: Results. The information gathered from the panelists showed that culture plays a
very important role either directly or indirectly in the everyday life of Malays. When the panel
members were asked to indicate the unique values of Malays, they were consistent in their
responses that Malays are polite, accommodative, collectivist, and soft-hearted. They stated
that

They are so open and accommodative towards other ethnic groups. (Expert 3, March 5, 2015)

A Malay ethnic group is collective in nature, accommodating, polite, male-authority-oriented with

strong family ties. (Expert 5, March 2, 2015)

Malay ethnic group values modesty and a collectivist culture. (Expert 6, February 12, 2015)

Being friendly and helpful is also important to them.

They have a deep sense of belonging or esprit de corps towards their friends and family. It is not dif-
ficult to win the hearts and minds of the Malays. Malay people like to help others, and they usually
do not like the person who stabs them from behind. (Expert 3, March 5, 2015)

In terms of the unique characteristics of Malays, the responses focused on Malays’ unique-
ness in their communication patterns. Malays prefer other people to communicate with them in
less straightforward and more empathic ways. The common pattern of communication was
echoed by one expert as

Values harmonious relationships, and they are very indirect. (Expert 6, February 12, 2015)

Therefore, the panelists’ responses to the first two questions led us to conclude that Malays
share values such as politeness, an accommodating nature, indirectness, and collectivism in their
societies.
The third question that was asked required the experts to indicate up to 10 values of the
Malay culture. As illustrated in Table 2, the panelists provided a list of values that they per-
ceived to be the most relevant to the Malay culture. To assess the face validity, we looked for
the values that were mentioned by at least two experts to proceed to the next rounds (Bearden,
Netemeyer, & Teel, 1989).
Taghipoorreyneh and de Run 9

Table 2. Values Mentioned by Experts in Round One.

Mentioned by experts
Type of values E1 E2 E3 E4 E5 E6 E7 Raw count suggestion Percentage of suggestion
Politeness X X X X X X 6 13.63
Respect to elder X X X X X 5 11.36
Accommodating X X X X X 5 11.36
Indirectness X X X X 4 9.09
Collectivism X X X X 4 9.09
Patience X X X 3 6.81
Helpfulness X X 2 4.54
Relationship X X 2 4.54
Hierarchy X X 2 4.54
Modesty X X 2 4.54
Faith in God X 1 2.27
Face saving X 1 2.27
Family-oriented X 1 2.27
Loyal X 1 2.27
Sincere X 1 2.27
Generous X 1 2.27
Cautious X 1 2.27
Fair X 1 2.27
Social order X 1 2.27
Kindness X 1 2.27
Cooperative X 1 2.27
Gotong-Royong X 1 2.27

Note. E = expert.

Second Round: Qualitative Data Collection that Led to Categorizing the Value Dimensions. The pur-
pose of the second round was to achieve the experts’ consensus on the values that emerged from
the first round and to categorize the values into distinct dimensions by searching for cross-
themes. Although the first round instrumentation presented open-ended statements to generate a
depth and breadth of input (divergence), the second round initiated the process of convergence.
This round included two sections.
In the first section, the panel members provided data through closed-ended questions with a
Likert-type scale. This scale made it possible to score the list of values from 1 (not important
at all) to 5 (very important). A total score for each value emerged from the statistical analysis.
A descriptive analysis was used to calculate the median of the total score. This median was
considered to be a cutoff point to achieve consensus. The total score was defined as a summa-
tion of the experts’ importance rating of each value. The median is the most accurate statistical
approach to show group views (Martino, 1973). This method is consistent with the average
majority consensus cutoff rate that has been sporadically used in Delphi studies (Heiko, 2012).
Only the responses that received a higher median score than the cutoff point were used in the
next round. In the second section, the experts were asked to define the most repeated values
from their point of view to reduce the overlap across themes. This step led to the creation of a
hierarchy of categories.
Second Round: Results. As presented in Table 3, the values frequently mentioned by the
experts rated a higher score. The findings confirmed our assumptions in the first round. The
median cutoff point for this step was found to be 21. Thus, if the total score for each value was
greater than 21, a consensus is reached.
10 Journal of Mixed Methods Research

Table 3. Importance Rate and Expert Consensus.

Importance rate
Values E1 E2 E3 E6 E7 Total score Consensus
Politeness 5 5 5 5 3 23 Achieved
Respect to elders 5 4 5 5 3 22 Achieved
Accommodating 5 4 4 5 3 21 Achieved
Indirectness 5 3 4 5 5 22 Achieved
Collectivism 5 5 4 4 4 22 Achieved
Tolerance 5 4 4 4 3 20 Not achieved
Helpfulness 4 3 4 5 4 20 Not achieved
Relationship 4 4 4 4 3 19 Not achieved
Hierarchy 4 4 4 5 4 21 Achieved
Modesty 4 4 4 4 3 19 Not achieved
Cultural practice 4 3 4 5 4 20 Not achieved

Note. E = expert.

Table 4. Malay Values and Its Dimensions.

Values Dimensions
Politeness Respect to elders
Indirectness
Being tolerant
Collectivism Group preferences
Majority consensus
Duty
Reciprocation of favor
Accommodating Appreciate differences
Being flexible
Hierarchy Family relationship
Power
Wealth

A summary of the experts’ responses are illustrated in Table 4. The following is a discussion
of the data by themes.
Politeness. The first of the four dimensions of Malay culture revealed by this study is polite-
ness. It is gauged by the way that Malays behave toward one another during interactions. From
the perspective of the experts, politeness emanated from two different levels of social interac-
tion: superior–subordinate (vertical) and peers (horizontal). In superior–subordinate interac-
tions, respect, and tolerance in addressing superiors, particularly elders in the context of society,
were considered to be the most important criteria in the evaluation of politeness. This view was
clearly stated by the experts.

Being polite is important in a hierarchical-based society, and it means respecting others who are
more senior in age. This means having to be tolerant of their ways even if you do not like what they
say and do. (Expert 1, March 18, 2015)

Politeness means being patient in dealing with others. (Expert 7, March 23, 2015)
Politeness is ‘‘Lending them (other people) your ears, elders first.’’ (Expert 6, March 19, 2015)
Taghipoorreyneh and de Run 11

Among peers’ interaction, maintaining harmony and avoiding conflict were the main points
in the assessment of politeness.

Respect and try to ensure not to offend or hurt others’ feelings. As much as possible, he/she tries to
understand other people or other ethnicities and tries to make them happy. (Expert 3, March 20,
2015)

Moreover, being indirect in communication as a way to achieve politeness has been previ-
ously clearly pointed out by Malay researchers (Awang, Maros, & Ibrahim 2012). Therefore,
politeness was categorized based on three values: (a) respect of elders, (b) indirectness, and (c)
being tolerant in dealing with other people.

Collectivism. The second dimension of Malay culture is collectivism. The term presents indi-
viduals’ relationship with group members and the collectivity that dominates in Malay society.
The main belief that underlies the Malay interpretation of collectivism is that people are interre-
lated with one another, and their success depends on their collective effort. The emphasis on
group preferences was well-stated by one expert:

Collectivist means being comfortable working in groups rather than alone. Teamwork is important.
(Expert 1, March 18, 2015)

Other comments suggested that Malays tend to seek a majority consensus and being accepted
by society. Being helpful and fulfilling their duty towards other people is unquestionable.

Collectivism means seeking a majority consensus. (Expert 6, March 19, 2015; Expert 7, March 23,
2015)

Collective values refer to the values that are held together or practiced together by the community.
These values have been practiced for a long time such as helpfulness, Gotong-Royong and concern
for neighbors. (Expert 3, March 20, 2015)

In general, according to the panel members, collectivism was more related to (a) group pre-
ferences, (b) the reciprocation of favor, (c) duty, and (d) seeking a majority consensus.

Accommodation. The third dimension of the Malay culture is an accommodating nature. This
dimension revealed one of the most unique characteristic of the Malay ethnic group. It refers to
the series of actions by and beliefs of the individual and community to accept and to be accepted
by other people and otherness. The basic belief that underlies Malays’ accommodating nature is
that human beings are not the same, and Malays should ‘‘understand’’ (Expert 1, March 18,
2015) and ‘‘appreciate (the) differences’’ (Expert 7, March 23, 2015; Expert 2, March 3, 2015)
of other people. They should ‘‘behave accordingly’’ (Expert 3, March 20, 2015) to live in a
‘‘peaceful and harmonious society’’ (Expert 1, June 12, 2015). We consistently found the fol-
lowing comments in the documents.

Being accommodating is being sensitive to the feelings and nuances of others. (Expert 1, March 18,
2015)

An accommodating nature is the ‘‘willingness to accept others’’ (Expert 7, March 23, 2015).
12 Journal of Mixed Methods Research

The Malays are willing to accept other ethnic groups or others in accordance with their willingness
to compromise. An accommodating attitude causes others to respect them. (Expert 3, March 20,
2015)

Accommodating attitude refers to the willing to accept others without prejudice and bias. (Expert 3,
March 20, 2015)

To achieve this end, Malays need to be flexible and open minded. As mentioned by the experts,

Malays are very open minded. (Expert 2, March 3, 2015)

Accommodating is more about flexibility. (Expert 6, March 19, 2015)

Accordingly, the expert discussions regarding an accommodating nature involved two val-
ues: (a) to appreciate differences and (b) being flexible in dealing with others.

Hierarchy. The fourth dimension of Malays’ culture is hierarchy. This dimension emphasizes
the legitimacy of an unequal distribution of power, wealth and recourses. This value revealed
that Malays’ source of strength comes from their membership in an extended group, their clan
and their position in society.
According to the experts, hierarchy is ‘‘A cultural dimension which states that in some cul-
tures, emphasis is placed on rank, status and other ascribed attributes over equality issues’’
(Expert 1, March 18, 2015).
Hierarchy also ‘‘Refers to the social status in the community, age, family relationships’’
(Expert 3, March 20, 2015) and ‘‘Accept(s) powers’ distance’’ (Expert 6, March 23, 2015).
Therefore, the hierarchy dimension referred to the status-based values in the community
such as (a) power, (b) wealth, and (c) family relationships.

Third Round: Quantitative Data Collection that Led to Achieve a Consensus Toward the Survey Items
Generated From the Qualitative Data Collection. This round comprised two sections to generate
items based on the cultural values that emerged from the previous rounds. The first section was
used to develop a set of items related to Malay values, and the second section was used to ensure
that the developed items were sufficiently viable to present these values.
Each item contains two ways to express motivations for achieving particular goals by point-
ing out the importance of values in the first sentence and by indicating what people want or
would like to do in the second sentence. Thus, the first sentence provides clear value statements,
and the second sentence presents an intention and a feeling. The two sentences were used to
focus more on the basic meaning of each item.
In addition, the respondents were asked to compare themselves with the statements in each
item. The levels of similarity with the items ranged from 1 (not like me at all) to 6 (very much
like me) on a Likert-type scale. This format is more common in everyday life rather than other
alternative formats such as the importance level. People constantly assess other people and com-
pare them with themselves (Schwartz, 2003). After developing these items, two pilot studies
were conducted to improve the clarity and simplicity of the items.
The second section set out to obtain the experts’ agreement on the most relevant items that
present Malay values. In this section, the experts were asked to evaluate the degree of the rele-
vance of the items on the instrument by answering a question (how much the person in the
description is similar to Malays) with 6 (very much like Malay), 5 (like Malays), 4 (somewhat
like Malays), 3 (a little like Malays), 2 (not like Malays), or 1 (not like Malays at all).
Consensus was assessed by the five experts who participated in the previous two rounds.
Taghipoorreyneh and de Run 13

For this section, more than 80% agreement on a 6-point Likert-type scale on the top three
measures (very much like Malay, like Malay, and somewhat like Malay) was considered to be a
consensus. The items with lower ratings were removed from the instrument. The same approach
was used by Putnam, Spiegel, and Bruininks (1995). We decided to be conservative and use
80% as the cutoff point for this final evaluation to identify the instrument that would be accep-
table for use in future studies.

Third Round: Results. Accordingly, 16 items were developed to measure various aspects of
Malay culture. In addition, three items were adapted from the hierarchy dimension of
Schwartz’s (2003) scale because they represented the findings well. After conducting two pilot
studies, seven items were modified to improve clarity, and one item was eliminated because of
a median of less than 3. Finally, a total of 17 items had reached consensus on completion of
Round 3, which is shown in Table 5. This result represents 94% of the total number of the
items. The final instrument is shown in Table 6.

Phase II Field-Test Study: Quantitative Data Collection

The purpose of this field-test study was to ensure that the items developed in the Delphi study
were valid and reliable. To this end, a purposive sample of 352 out of 416 respondents were
qualified to be analyzed. The eligible respondents were Malays with various backgrounds from
across Malaysia. Sixty-six surveys were excluded from the data analysis because of miscoded
and unengaged responses. The respondents were older than 18 years, so that they had a stable
sense of themselves as ethnic members (Phinney, 1989). The instrument was developed first in
English and then translated into Bahasa Melayu. A back-translation was conducted by Malay
graduate students to improve the translation accuracy. Five research assistants were recruited to
directly distribute the questionnaire to the respondents. All items were assessed on a 7-point
Likert-type scale from 1 (not like me at all) to 7 (totally like me). A summary of the respon-
dents’ demographic profile is shown in Table 7.

Comparison Between the Experts and Sample Results. The results of the field test are presented in
Table 8. The findings demonstrate that the pattern of results is similar to the experts’ final eva-
luation results. The items related to politeness received the highest mean score among the
Malay respondents. Similar results were found for the accommodating, collectivism, and hierar-
chy items. As expected, the hierarchy items received the lowest mean score among the respon-
dents. Therefore, the findings from this study lead us to conclude that the items developed in
the Delphi study are reliable in representing Malay values.

Correspondence Analysis. This analysis allows the visualizing and mutual positioning of cultural
values relative to one another with respect to the demographic information in a two-dimensional
space. The output is shown in Figure 2. Based on the median score for each item, the values’
data were divided into a high and low value orientation to be able to attribute and emphasize
the differences that originate from the respondents’ demographic profiles.
The dispersion of demographic characteristics on the graph and their unequal distance from
the high and low value orientations suggest that Malays from different demographic profiles
possess different cultural values. The equidistant position of genders from the high and low
value orientations indicates the absence of strong differences in value orientation between
Malay males and females. However, females have a slightly higher sense of belongingness to
their parents’ culture. In contrast, the wide distribution of the respondents’ geographic profiles
around the graph indicates the importance of this factor in shaping Malays’ value orientations.
14
Table 5. The Experts Final Evaluation Results.

Number of answers Number of opinion Total results

Very much Like Some-what A little Not like Not like Top three
Items like Malay Malay like Malay like Malay Malay Malay at all measure, % Consensus
Proper behavior 2 3 100 Achieved
Tolerant 5 100 Achieved
Indirectness others 2 3 100 Achieved
Indirectness elders 2 2 100 Achieved
Respect elders 3 2 100 Achieved
Sensitive 4 1 100 Achieved
Flexible 1 2 2 100 Achieved
Appreciate differences 4 1 100 Achieved
Majority 1 1 3 100 Achieved
Teamwork 2 1 1 1 80 Achieved
Duty 2 1 1 1 80 Achieved
Advice 1 1 2 1 80 Achieved
Community 1 2 1 1 80 Achieved
Family 2 2 1 100 Achieved
Wealth 2 2 1 80 Achieved
Influential 1 3 1 20 Not achieved
Obedient 2 2 1 100 Achieved
Power distance 3 2 100 Achieved
Taghipoorreyneh and de Run 15

Table 6. Final Malay Value Items.

Items
P1 It is important to her/him to be polite to other people all the time. She/he tries to ensure
not to offend or hurt the feelings of others. (Proper behavior)
P2 It is important to her/him to be tolerant in dealing with her/his elders. She/he believes she
should always listen to those who are senior in age, even if she/he doesn’t like what they
say and do. (Being tolerant)
P3 She/he tries to be indirect when communicating with others. It is important for her/him to
be sensitive about the face value of her/his hearers.(Indirectness others)
P4 It is important for her/him to treat her/his elders with courtesy. She/he tries to express
her/his real feeling indirectly when she/he is not satisfied with them. (Indirectness elders)
P5 It is important to her/him to respect elders. She/he believes she/he should always show
respect to her/his parents and to older people. (Respect to elders)
A1 It is important to her/him to be sensitive to how others might feel when communicating.
She/he makes an effort not to disturb or irritate others. (Being sensitive)
A2 It is important to her/him to be flexible in behaving with others. How she/he behaves with
others depends on who she/he is with, where she/he is, and both. (Being flexible)
A3 It is important to her/him to understand that human beings are not the same. She/he thinks
she/he should always appreciate differences toward others and those from other races.
(Appreciate differences)
C1 It is important to her/him to be socially accepted. She/he wants to avoid doing anything that
people would say is wrong. (Majority consensus)
C2 Teamwork is very important to her/him. She/he is more comfortable working in a group
rather than alone. (Teamwork)
C3 It is important to her/him to fulfill his duty toward her/his group members. She/he believes
she/he should help them, within her/his means, when they are in trouble. (Duty)
C4 It is important for her/him to turn to others close to her/him for help in making decisions.
Before making decision she/he always consults others. (Advice)
C5 It is important for her/him to be concern about her/his community. She/he would like to
participate in practice such as Gotong-Royong to help her/his community when they in
trouble (Community concern)
H1 It is important to her/him to listen to her/him family members. She/he tends to follow their
advice without asking any questions. (Family relationship)
H2 It is important to her/him to be rich. She/he wants to have a lot of money and expensive
things. (Wealth)
H3 It is important to her/him to be obedient. She/he finds it hard to disagree with someone in
higher position than her/him. (Obedient)
H4 It is important to him/her to accept power distance in the society. She/he thinks a person’s
social status reflects her/his place in the society.(Power)

Note. P = politeness; A = accommodating; C = collectivism; H = hierarchy.

Furthermore, Malays’ income level appears to be a more influential factor than gender and geo-
graphic factors in reflecting Malays’ values. Malays with a higher income level clearly indi-
cated a higher value orientation. Finally, the level of value orientation also increased with age.
Overall, the value orientation was somewhat higher for females and for the Malays who were
older and richer, which indicates that the instrument is sufficiently sensitive to reflect the value
differences within different demographic categories in Malay culture.

Discussion
The close distance of the high value orientation from the center of the graph draws attention to
the importance of cultural values. From the results, it appears that politeness is the most
16 Journal of Mixed Methods Research

Table 7. Respondents Demographic Profile.

Variable Item Frequency Percentage

Gender Male 149 42.3
Female 203 57.7
Age, years Younger than 28 104 29.5
29-39 186 52.8
40-50 50 14.2
Older than 51 12 3.4
Place Johor 45 12.8
Kedah 15 4.3
Kelantan 29 8.2
Melaka 93 26.4
Pahang 32 9.1
Penang 4 1.1
Perak 15 4.3
Petaling 1 0.3
Sabah 12 3.4
Sarawak 49 13.9
Selangor 48 13.6
Trengganu 9 2.6
Income level (US$) Below 245 71 20.2
Between 245 and 489 89 25.3
Between 490 and 735 77 21.9
Between 736 and 980 36 10.2
Between 981 and 1,225 28 8.0
Above 1,225 51 14.5

important value dimension among Malays. The findings demonstrate that Malays avoid any
actions and inclinations that lead to conflict with other people because it may violate their
social solidarity. To achieve politeness, Malays need to be indirect in their communication, to
be tolerant in dealing with other people and to respect other people, particularly elders.
At the most abstract level, politeness may indeed be a universal phenomenon (Brown &
Levinson, 1987). However, it would be a grave error to assume that the values attached to
politeness are similar across cultures. Indeed, what counts as polite behavior is different from
one culture to another culture (Brown & Levinson, 1987). The association between politeness
and cultural context is also evident in Malay society. Malays’ politeness ideological notions are
influenced by the concept of budi, which can be translated to wisdom or intellect (K. H. Lim,
2003). In the Malay culture, the achievement of the politeness value types in an individual’s
social world is one of the most significant goals of everyday living such that the concern for
achieving politeness appears to be an even higher value than other important outcomes.
The types of the politeness values identified in this study indicate the dynamic process to
achieve harmony within a society that is both collectivist and hierarchical. Malay politeness
examines the oft-expressed link between harmony and the hierarchical system in which the
maintenance of harmonious relationships is considered to be fundamental to the daily basic reg-
ulation of social interaction. Achieving politeness in both directions will help Malays live in
harmony and preserve the relational and social order. This is more than simply just a term for
‘‘proper behavior,’’ which is often mentioned in universal scales, such as Schwartz’s (2003)
Value Scale, to indicate the politeness level across cultures.
The second dimension that came to the surface was collectivism. It emphasizes Malays’ ten-
dency to seek a majority consensus. This collective value may be influenced by the Confucian
Taghipoorreyneh and de Run 17

Table 8. Results of the Field Test.

Values Items M SD
Politeness Proper behavior 5.78 1.234
Being tolerant 5.98 1.157
Indirectness in dealing with others 5.48 1.343
Indirectness in dealing with elders 5.76 1.281
Respect to elders 6.17 1.103
Accommodation Being sensitive toward others 5.66 1.356
Being flexible 5.80 1.220
Appreciate differences 5.92 1.088
Collectivism Majority consensus 5.30 1.290
Teamwork 5.38 1.429
Duty 5.72 1.202
Advice 5.43 1.334
Community concern 5.47 1.102
Hierarchy Family relationship 5.36 1.289
Wealth 4.78 1.716
Obedient 5.03 1.446
Power distance 5.00 1.532

heritage. In the civilization process, Confucianism spread its impact from mainland China to
neighboring nations over hundreds of years (Chang, 1997). Deliberate attempts to follow and
practice Confucian values are evident in these nations. Indeed, Malaysia has a uniquely amalga-
mated culture deeply steeped in centuries of history and external influences. Therefore, it is logi-
cal to assume that Malay culture has also been influenced by these Confucian values. Values
such as majority consensus, duty, and teamwork can be emphasized as evidence that signifies
the importance of cultural collectivism among Malays.
Cultural collectivism can also be attributed to Malays’ past income resources and climatic
determinism (Inglehart & Welzel, 2005). Collectivism was observed to be stronger in lower
income nations (Van de Vliert, 2011) with greater climatic hardship (Van de Vliert & Postmes,
2012). Therefore, perhaps it should have come as no surprise that this study found Malays to be
strong collectivists.
The third dimension was being accommodating. The emphasis on being accommodating is
manifested by one’s value of appreciating differences and being flexible in dealing with other peo-
ple. Being accommodating is necessary to achieve interpersonal and social harmony. The achieve-
ment of interpersonal and social harmony helps Malays to have a higher level of life satisfaction
(Kwan, Bond, & Singelis, 1997), a higher level of perceived health (S. Stewart et al., 1998), and a
lower level of depression (Chen, Chan, Bond, & Stewart, 2006). Appreciating differences and
being flexible allows them to live peacefully together in a conflict free-manner (Jia, 2008) and in
particular, work together more effectively in a situation where each member has different opinions
concerning the best process to achieve work-related goals (Lun & Bond, 2006).
The fourth dimension was hierarchy. It refers to status-based values in the community used
to get ahead in the social hierarchy. Malays see society as a conglomeration of individuals who
are inherently unequal in rank. They believe a person’s social status reflects his or her position
in society. This fact enables them to admit and accept their position in the social hierarchy and
allows everyone to fulfill his or her prescribed tasks (Lun, 2012). Ultimately, society would
progress without any challenge and disorder. To justify this hierarchical interaction of social life
and to motivate Malays to accept this power distance, they must treat the tangible and abstract
18 Journal of Mixed Methods Research

Figure 2. Correspondence analysis output for Malays cultural values scales and demographic categories.
Note. Cultural values are marked in red, gender in blue, age in green, place in black, and income in purple. Split into H
and L indicates high versus low value orientation.

outcomes of power such as wealth, being obedient, and family relationships as values. This is
perhaps the reason why Malays adhere to hierarchical values in their society.

Limitations
This study does not intend to trace the origin of Malay values. The analysis of the original val-
ues of the traditional Malays is a very difficult task (Hamid, 1991) due to its considerable con-
tact with Hindu, Buddhist, Islamic, and European civilizations. It should also be emphasized
that our goal is not to put our target cultures and their values under a religion umbrella. Religion
as a system of symbols can provide a powerful meaning and transformation mechanism between
values and objective reality and can clothe them with such an aura of faculty that the meanings
seem uniquely realistic to be evaluated by individuals (Geertz, 1973). Therefore, it is not accu-
rate to say that religions form cultural values.

Contribution to Mixed Methods Research

The results illustrate the usefulness of mixed methods research in advancing the validity and
reliability of an indigenous cultural values instrument. It helps researchers understand a wider
range of cultural differences at the individual level, especially in multicultural countries. The
indigenously derived instrument introduced in this article proposed value dimensions such as
‘‘politeness’’ and ‘‘accommodation’’ that did not load on any quantitative or qualitative values
instruments. This study has already shown how these value dimensions differ across demo-
graphic groups in our target culture. Understanding these nuances are important because they
can be usefully employed in a variety of business setting such as the design of the marketing
Taghipoorreyneh and de Run 19

mix, the identification of advertising appeals that are most appropriate for different cultures and
the study of symbolic consumption. Researchers should look at both qualitative and quantitative
methods while simultaneously emphasizing the variations in cultural groups that have hereto-
fore been either ignored or overlooked in purely quantitative or qualitative research.

Utility of Yin’s Framework of Five Procedures to Maximize

Integration
The mixed methods approach allows the development of instruments based on the inclusion of
indigenous participants and consultation with local experts. We followed Yin’s (2006) framework
of five procedures to maximize the integration of the components in our study. The first procedure
in Yin’s framework relates to the research question. In this research, there was a clear rationale for
the use of survey data, list data, and open-ended data in addressing the same substantive question
of what are our target culture’s values. Three further procedures are units of analysis, sampling,
and instrumentation and data collection. Here, the units of analysis were integrated in the sense that
the qualitative open-ended data and quantitative survey data had the same points of reference—
these were experts and people from the same culture. The data collection instruments of all meth-
ods were linked to emphasize the complementary use of each method and to corroborate the find-
ings from prior stages by addressing the same substantive issue of our target culture values. An
example of this is to be found where the findings from the open-ended and list data were integrated
to provide a fuller picture and to produce a comprehensive list of values responsive to the target
culture. The list of value was corroborated through quantitative survey data.
Yin’s (2006) final procedure for achieving integration is the analytic strategies. Here, the
analysis began by observing distinct techniques that were conventionally associated with each
method; these were a descriptive and consensus analyses for the list and open-ended data. The
qualitative inquiry was used to develop the theory-based categorization of values by generating
formative items. It is hoped that the efforts described here might be replicated in the develop-
ment of formative indicators in varied settings. Thereafter, a correspondence analysis of scale
was used for the quantitative survey data. It allowed a more nuanced account and explanation
of the target culture’s values at different layers of demographic groups and the tracing of cumu-
lative cultural development layer after layer.
Finally, the results show the value of Delphi groups as part of a mixed methods research
design in advancing validity and reliability of a quantitative cultural values instrument. The
Delphi literature has several distinct gaps in methodological guidance. Some of the gaps in the
literature for using the qualitative Delphi includes how should researchers approach data reduc-
tion and analysis in qualitative Delphi studies (Brady, 2015)? As demonstrated, this study offers
a solution by describing how qualitative inquiry, which is suited for investigation of cultural
values, can be used to inform instrument development, data reduction, and analysis in Delphi
studies. It is hoped that the efforts described here might be replicated in varied settings.

Acknowledgments
The authors wish to sincerely thank all participants in the Delphi survey for their contribution which have
been vital to achieving results of this study. The authors also wish to thank those who have read and com-
mented on the article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or pub-
lication of this article.
20 Journal of Mixed Methods Research

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or pub-
lication of this article: The study is supported by Zamalah Scholarship, from University of Malaysia,
Sarawak, under UNIMAS/TNC(AA)-10/04-30/01 Jld 2 (176) contract.

ORCID iD
Mehdi Taghipoorreyneh https://orcid.org/0000-0002-3199-0532

References
Adler, N. J., & Graham, J. L. (1989). Cross-cultural interaction: The international comparison fallacy?
Journal of International Business Studies, 20, 515-537.
Asma, A. (1992). The influence of ethnic values on managerial practices in Malaysia. Malaysian
Management Review, 27(1), 3-18.
Awang, S., Maros, M., & Ibrahim, N. (2012). Malay values in intercultural communication. International
Journal of Social Science and Humanity, 2, 201-205.
Bearden, W. O., Netemeyer, R. G., & Teel, J. E. (1989). Measurement of consumer susceptibility to
interpersonal influences. Journal of Consumer Research, 15, 473-481.
Bloor, M., Sampson, H., Baker, S., & Dahlgren, K. (2015). Useful but no Oracle: Reflections on the use of
a Delphi Group in a multimethods policy research study. Qualitative Research, 15, 57-70.
Brady, S. R. (2015). Utilizing and adapting the Delphi method for use in qualitative research. International
Journal of Qualitative Methods, 14(5), 1-6.
Brown, P., & Levinson, S. C. (1978). Universals in language usage: Politeness phenomena. In Questions
and politeness: Strategies in social interaction (pp. 56-311). Cambridge University Press.
Chang, H. C. (1997). Language and words: Communication and the analects of Confucius. Journal of
Language and Social Psychology, 16, 107-131.
Chen, S. X., Chan, W., Bond, M. H., & Stewart, S. M. (2006). The effects of self-efficacy and relationship
harmony on depression across cultures: Applying level-oriented and structure-oriented analyses.
Journal of Cross-Cultural Psychology, 37, 643-658.
Cheung, F. M., van de Vijver, F. J. R., & Leong, F. T. L. (2011). Toward a new approach to the study of
personality in culture. American Psychologist, 66, 593-603.
Clark, T. (1990). International marketing and national character: A review and proposal for an integrative
theory. Journal of Marketing, 54(4), 66-79.
Creswell, J. W., & Tashakkori, A. (2007). Differing perspectives on mixed methods research. Journal of
Mixed Methods Research, 1(4), 303-308.
Critcher, C., & Gladstone, B. (1998). Utilizing the Delphi technique in policy discussion: A case study of a
privatized utility in Britain. Public Administration, 76, 431-449.
Dalkey, N. C., & Helmer, O. (1963). An experimental application of the Delphi method to the user of
experts. Management Science, 9, 458-467.
Erez, M., & Earley, C. P. (1993). Culture, self-identity and work. New York, NY: Oxford University Press.
Fetters, M. D., & Freshwater, D. (2015). Publishing a methodological mixed methods research article.
Journal of Mixed Methods Research, 9(3), 203-213.
Fetters, M. D., & Molina-Azorin, J. F. (2017). The Journal of Mixed Methods Research starts a new
decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods
Research, 11(3), 291-307.
Fischer, R., Ferreira, M. C., Assmar, E. M. L., Redford, P. & Harb, C. (2005). Organizational behaviour
across cultures. Interntional Journal of Cross-Cultural Management, 5, 27-48.
Fontaine, R., & Richardson, S. (2005). Cultural values in Malaysia: Chinese, Malays and Indians
compared. Cross-Cultural Management, 12, 63-77.
Geertz, C. (1973). The interpretation of cultures. New York, NY: Basic Books.
Taghipoorreyneh and de Run 21

Hamid, I. (1991). Masyarakat Melayu dan budaya Melayu [Malay society and Malay culture]. Kuala
Lumpur, Malaysia: Dewan Bahasa dan Pustaka.
Haviland, W. A. (1990). Cultural anthropology (6th ed.). Fort Worth, TX: Holt, Rinehart and Winston.
Heiko, A. V. (2012). Consensus measurement in Delphi studies: Review and implications for future
quality assurance. Technological Forecasting & Social Change, 79, 1525-1536.
Hitchcock, J. H., Nastasi, B. K., Dai, D. Y., Newman, J., Jayasena, A., Bernstein-Moore, R., & . . .Varjas,
K. (2005). Illustrating a mixed-method approach for validating culturally specific constructs. Journal of
School Psychology, 43, 259-278.
Ho, D. Y. F. (1996). Filial piety and its psychological consequences. In M. H. Bond (Ed.), Handbook of
Chinese psychology (pp. 155-165). Hong Kong: Oxford University Press.
Hofstede, G. (1980). Culture’s consequences: International differences in work-related values. Beverly
Hills, CA: Sage.
Hofstede, G. (1997). Cultures and organizations: Software of the mind. New York, NY: McGraw-Hill.
Hui, C. H., & Triandis, H. C. (1989). Effects of culture and response format on extreme response style.
Journal of Cross-Cultural Psychology, 20, 296-309.
Ingersoll-Dayton, B. (2011). The development of culturally-sensitive measures for research on ageing.
Ageing & Society, 31, 355-370.
Inglehart, R. (2003). Human values and social change: Findings from the values surveys. Leiden,
Netherlands: Koninklÿke Brill.
Inglehart, R., & Welzel, C. (2005). Modernization, cultural change, and democracy. New York, NY:
Cambridge University Press.
Jarvis, C. B., Mackenzie, S. B., Podsakoff, P. M., Mick, D. G., & Bearden, W. O. (2003). A critical review
of construct indicators and measurement model misspecification in marketing and consumer research.
Journal of Consumer Research, 30, 199-218.
Jia, W. (2008). Chinese perspective on harmony: An evaluation of the harmony and the peace paradigm.
Chinese Media Research, 4(4), 25-30.
Kwan, V. S., Bond, M. H., & Singelis, T. M. (1997). Pancultural explanation for life satisfaction: Adding
relationship harmony to self-esteem. Journal of Personality and Social Psychology, 73, 1038-1051.
Lenartowicz, T., & Roth, K. (1999). A framework for culture assessment. Journal of International
Business Studies, 30, 781-798.
Lim, K. H. (2003). Budi as the Malay mind: A philosophical study of Malay ways of reasoning and
emotion in peribahasa (Doctoral dissertation). University of Hamburg, Germany.
Lim, L. (2001). Work related values of Malays and Chinese Malaysians. International Journal of Cross-
Cultural Management, 1, 209-226.
Lun, V. M.-C. (2012). Harmonizing conflicting views about harmony in Chinese culture. In X. Huang &
M. H. Bond (Eds.), The handbook of Chinese organizational behavior: Integrating theory, research,
and practice (pp. 467-479). Northampton, MA: Edward Elgar.
Lun, V. M.-C., & Bond, M. H. (2006). Achieving relationship harmony in groups and its consequence for
group performance. Asian Journal of Social Psychology, 9, 195-202.
Maltseva, K. (2016). Using correspondence analysis of scales as part of mixed methods design to access
cultural models in ethnographic fieldwork: Prosocial cooperation in Sweden. Journal of Mixed Methods
Research, 10(1), 82-111.
Martino, J. P. (1973). Technological forecasting for decision making. New York, NY: American Elsevier.
Neuman, W. L. (2011). Social research methods: Qualitative and quantitative approaches (7th ed.).
Boston, MA: Allyn & Bacon.
Onwuegbuzie, A. J., Rebecca, M. B., & Judith, A. N. (2010). Mixed research as a tool for developing
quantitative instruments. Journal of Mixed Methods Research, 4(1), 56-78.
Pervin, L. A., Cervone, D., & John, O. P. (2005). Personality: Theory and research (9th ed.). New York,
NY: John Wiley.
Petter, S., Straub, D. W., & Rai, A. (2007). Specifying formative constructs in information systems
research. MIS Quarterly, 31, 623-656.
Phinney, J. (1989). Stages of ethnic identity development in minority group adolescents. Journal of Early
Adolescence, 9, 34-49.
22 Journal of Mixed Methods Research

Putnam, J. W., Spiegel, A. N., & Bruininks, R. H. (1995). Future directions in education and inclusion of
students with disabilities: A Delphi investigation. Exceptional Children, 61, 553-576.
Rokeach, M. (1973). Beliefs, attitudes and values: A theory of organization and change. San Francisco,
CA: Jossey-Bass.
Schwartz, S. H. (1999). Beyond individualism/collectivism: New cultural dimensions of values. In U. Kim,
H. C. Triandis, C. Kagitcibasi, S.-C. Choi, & G. Yoon (Eds.), Individualism and collectivism: Theory,
method, and applications (pp. 85-119). Thousand Oaks, CA: Sage.
Schwartz, S. H. (2003). A proposal for measuring value orientations across nations. Retrieved from
https://www.europeansocialsurvey.org/docs/methodology/core_ess_questionnaire/
ESS_core_questionnaire_human_values.pdf
Sharma, P. (2010). Measuring personal cultural orientations: Scale development and validation, Academy
of Marketing Science, 38,787-806.
Sobaih, A. E., Ritchie, C., & Jones, E. (2012). Consulting the Oracle? Applications of modified Delphi
technique to qualitative research in the hospitality industry. International Journal of Contemporary
Hospitality Management, 24, 886-906.
Stewart, D. W. (1981). The application and misapplication of factor analysis in marketing research.
Journal of Marketing Research, 18, 51-62.
Stewart, S., Rao, N., Bond, M., McBride-Chang, C., Fielding, R., & Kennard, B. (1998). Chinese
dimensions of parenting: Broadening western predictors and outcomes. International Journal of
Psychology, 33, 345-358.
Tapio, P., Paloniemi, R., Varho, V., & Vinnari, M. (2011). The unholy marriage? Integrating qualitative
and quantitative information in Delphi processes. Technological Forecasting & Social Change, 78,
1616-1628.
Taylor, R. E., & Judd, L. L. (1989). Delphi method applied to tourism. In S. Witt & L. Moutinho (Eds.),
Tourism marketing and management handbook (pp. 95-99). New York, NY: Prentice Hall.
Terpstra-Tong, J. L. Y., Terpstra, R. H., & Tee, D. D. (2014). Convergence and divergence of individual-
level values: A study of Malaysian managers. Asian Journal of Social Psychology, 17, 236-243.
Triandis, H. C. (1996). The psychological measurement of cultural syndromes. American Psychologist, 51,
407-415.
Van de Vliert, E. (2011). Climato-economic origins of variation in ingroup favoritism. Journal of Cross-
Cultural Psychology, 42, 494-515.
Van de Vliert, E., & Postmes, T. (2012). Climato-economic livability predicts societal collectivism and
political autocracy better than parasitic stress does. Behavioral and Brain Sciences, 35, 94-95.
Yin, R. K. (2006). Mixed method research: Are the method genuinely integrated or merely parallel?
Research in the School, 13(1), 41-47.
Yousuf, M. I. (2007). Using experts’ opinions through Delphi technique. Practical Assessment, Research
& Evaluation, 12, 1-8.