This article was downloaded by: [TBTAK EKUAL] On: 1 October 2010 Access details: Access Details: [subscription

number 786636097] Publisher Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 3741 Mortimer Street, London W1T 3JH, UK

Total Quality Management & Business Excellence

Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t713447980

A Methodology to Improve Higher Education Quality using the Quality Function Deployment and Analytic Hierarchy Process

Hendry Raharjoab; Min Xiea; Thong Ngee Gohab; Aarnout C. Brombacherbc a Department of Industrial and Systems Engineering, National University of Singapore, b Centre for Design Technology, National University of Singapore, c Technische Universiteit Eindhoven, Eindhoven, The Netherlands

To cite this Article Raharjo, Hendry , Xie, Min , Goh, Thong Ngee and Brombacher, Aarnout C.(2007) 'A Methodology to

Improve Higher Education Quality using the Quality Function Deployment and Analytic Hierarchy Process', Total Quality Management & Business Excellence, 18: 10, 1097 1115 To link to this Article: DOI: 10.1080/14783360701595078 URL: http://dx.doi.org/10.1080/14783360701595078

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Total Quality Management Vol. 18, No. 10, 1097 1115, December 2007

A Methodology to Improve Higher Education Quality using the Quality Function Deployment and Analytic Hierarchy Process
HENDRY RAHARJO , , MIN XIE, THONG NGEE GOH , & AARNOUT C. BROMBACHER ,
Department of Industrial and Systems Engineering, National University of Singapore; Centre for Design Technology, National University of Singapore; Technische Universiteit Eindhoven, Eindhoven, the Netherlands

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

ABSTRACT In order to formulate an effective strategic plan in a customer-driven education context, it is important to recognize who the customers are and what they want. Using Quality Function Deployment (QFD), this information can be translated into strategies to achieve customer satisfaction. Since the nal strategic plan relies heavily on the way QFD is used, this paper will rst describe the existing problems in its use and then propose a better way to improve it. In this paper, the customers are divided into two major parties, namely, the internal and the external customer. The internal customer comprises of the lecturers and the students, while the external customer is the employers of the graduates. After collecting the Voice of Customer (VOC), the Analytic Hierarchy Process (AHP) technique was employed to generate the priorities of the VOC for each group of customers. Then, the results were used as the input for formulating strategies or Quality Characteristics (QCs) to meet the Demanded Qualities (DQs) using QFD. A simple case study is provided to demonstrate the usefulness of the methodology. A sensitivity analysis was also conducted to anticipate the changes in the DQs that will affect the output of the QFD. This is useful for providing a better strategic planning for the education institution to meet the future needs of its customers. KEY WORDS : Quality of education, quality function deployment, internal customer, external customer, analytic hierarchy process, sensitivity analysis, future needs of customers

Introduction In this era of global education, it is imperative for a higher education institution continuously assure and improve its quality. The word quality in education can be interpreted in

Correspondence Address: Hendry Raharjo, Department of Industrial and Systems Engineering, National University of Singapore, Singapore, 119260. Email: raharjo_h@nus.edu.sg 1478-3363 Print/1478-3371 Online/07/10109719 # 2007 Taylor & Francis DOI: 10.1080/14783360701595078

1098

H. Raharjo et al.

various ways, but in order to provide better education quality, the education institution has to strive for offering a particular competitive advantage to its customers by recognizing, fullling and exceeding their requirements. This is in line with the concept of Total Quality Education (TQE) proposed by Dahlgaard et al. (1995): An education culture characterized by increased customer satisfaction through continuous improvement, in which all employees and students actively participate. A large number of higher education institutions teach Total Quality Management (TQM) to their students; unfortunately only some of them practice what they preach. The implementation of TQM in education required a customer-driven focus (Pitman et al., 1996; Hwarng & Teo, 2000, 2001; Sa & Saraiva, 2001; Sahney et al., 2006). Therefore, it is of great importance to listen to the Voice of the Customer (VOC) (Grifn & Hauser, 1993) to effectively and efciently deliver values to customers. The Quality Function Deployment (QFD) is an alternative way to recognize or listen to the Voice of Customer (VOC) better in manufacturing industries as well as in service industries. The QFD is known as one of the most powerful tools in TQM. It has been applied mostly to industries (Chan & Wu, 2002a), but the underlying concept of a customer-driven process may, to a certain extent, apply to the education eld. The rst crucial step that makes the efforts effective is to identify who the customers are and what they demand from the institution. The customers/stakeholders of the university may involve faculties/lecturers, students, employer of graduates, alumni, administrative staffs, parents, government, local community, and so on (see Reavill, 1997). For the sake of simplicity, we will only focus on three main parties, namely, students (Sirvanci, 1996; Wallace, 1999; Sakthivel & Raju, 2006), lecturers, and employers of graduates (Willis & Taylor, 1999). Focusing on the students is the center pedagogical principle in the constructivism approach. On the other hand, students may also be regarded as the product of higher education (Pitman et al., 1996; Bailey & Bennett, 1996; Bier & Cornesky, 2001). Using a different point of view, the classication of customers may differ from one to another (see Madu et al., 1994; Ermer, 1995; or Duffuaa et al., 2003). In this study, only two major categories are considered, namely, the internal and the external customer. The internal customer consists of the lecturers and the students, while the external customer is the employers of the graduates. The students are considered as customers in terms of the services they receive from the institution, for example, the efciency in modules registration/administration problem, the organizational or non-academic activity, the class punctuality and lecturers attendance frequency, food hygiene/service, and so on. They are not deemed suitable to judge more on the content of the education. The lecturers are regarded as the designer as well as the main player in education, thus the education institution should provide them with the necessary facilities for education purposes. The employers of graduates are considered as the customer because their feedback towards the graduates would provide a valuable measure of the Quality of Performance (QP) (Mergen et al., 2000; Widrick et al., 2002) of the respective higher education. Their feedback may also reect the learning outcome or competency of the graduates. In the following sections, a literature review on the use of the QFD in education will be given and followed with some existing technical and practical problems which motivated the proposed methodology (the second section). Then, in the third section, the basic QFD model, its use and terms will be described. Several reasons to choose the Analytic Hierarchy Process (AHP) in combination with the QFD will also be elaborated. In the fourth section, a methodology to improve higher education quality using the QFD and

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

A Methodology to Improve Higher Education Quality

1099

AHP is proposed using a step-by-step procedure and a owchart. A real-world case study was also used to illustrate the idea (the fth section). A sensitivity analysis was also conducted, based on the case study data, to deal with the dynamics of the education systems as well as to anticipate the future needs of the customers. Lastly, a brief conclusion of the study is provided in the sixth section. QFD in Education and some Problems Since the 1980s, higher education institutions have begun to adopt and apply quality management to the academic domain owing to its success in industry (Grant et al., 2002) and they have also beneted from the application of TQM (Kanji & Tambi, 1999; Owlia & Aspinwall, 1998). QFD, as one of the most powerful TQM tools, has also been used quite extensively in academia. Jaraiedi & Ritz (1994) applied QFD to analyze and improve the quality of the advising and teaching process in an engineering school. Koksal & Egitman (1998) used QFD to improve industrial engineering education quality at the Middle East Technical University. Lam & Zhao (1998) suggested the use of the QFD and the Analytic Hierarchy Process (AHP) to identify appropriate teaching techniques and to evaluate their effectiveness in achieving an education objective. Bier & Cornesky (2001) critically analyzed and constructed a higher education curriculum to meet the needs of the customers and accrediting agency using QFD. Adopting the constructivists point of view, Chen & Chen (2001) introduced a QFDbased approach to evaluate and select the best-t textbook based on the VOC. Kauffmann et al. (2002) also used QFD to select courses and topics that enhance a master of engineering management program effectiveness. They further pointed out the additional benet of QFD in the academic context, that is, to develop collegial consensus by providing an open and measurable decision process. Brackin (2002) wrote the analogy of the use of QFD in the industry with the assessment of engineering education quality by breaking down the assessment items into a set of WHATs and HOWs following the four phases of QFD. Duffuaa et al. (2003) applied QFD for designing a basic statistics course. More recently, Sahney et al. (2004, 2006) used QFD, in combination with SERVQUAL as well as Interpretive Structural Modeling and Path Analysis, to identify a set of minimum design characteristics to meet the needs of the student as an external customer of the educational system. Chen & Yang (2004) explored the possibility to use Internet technology by developing a Web-QFD model. They gave a real-world example of an education system in Taiwan and argued that the Web-QFD may not only provide a more efcient way of using the QFD in terms of cost, time and territory, but also may facilitate a better group decision making process. Aytac & Deniz (2005) used QFD to review and evaluate the curriculum of the Tyre Technology Department at the Kocaeli University Kosekoy Vocational School of Higher Education. It is clear that QFD has been extensively used in improving education quality. However, if one takes a closer look at how QFD was implemented in education, one discovers some problematic areas that need improvement. In this paper, ve major problems will be highlighted. They can be divided into two major categories, namely, the technical problems (the rst, the second, and the third problems) and the practical ones (the fourth and the fth problems). The rst problem is the use of absolute values to assign the degree of customer importance. As pointed out by Chuang (2001), the customers will tend to assign a high degree of importance to most of their requirements, thus resulting in values near the highest possible

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

1100

H. Raharjo et al.

score. These values will have no signicant meaning (Cohen, 1995) and will later produce somewhat arbitrary and inaccurate results for prioritizing technical response. Some examples for using a set of discrete values can be found in Jaraiedi & Ritz (1994), Ermer (1995), Chen & Chen (2001), Kaminski et al. (2004), and Chou (2004). Therefore, relative measurement among the customer requirements is suggested to be a better alternative. The second problem is the technique that is used to obtain the priorities of a groups preference. Some of the studies simply proposed the use of an arithmetic mean or weighted arithmetic mean for obtaining the preference of the customer group, which seems arbitrary and not robust. This case can be found in Bier & Cornesky (2001), Hwarng & Teo (2001), Duffuaa et al. (2003), Kaminski et al. (2004), or Aytac & Deniz (2005). A better approach would be to use a geometric mean that also formed the foundation of a group preference method in the AHP (Ramanathan & Ganesh, 1994; Forman & Peniwati, 1998). The third problem is the difculty in identifying a true relationship between customer requirements and design/technical attributes. It seems quite unrealistic if all customer requirements are related to design/technical attributes so that the QFD correlation matrix will be full blocked. A less severe condition would be that all the technical attributes correlate with a particular customer requirement. It simply shows that the QFD team has difculty in assigning more discriminating relationship values between them. Examples of this case can be found in Duffuaa et al. (2003) or Lam & Zhao (1998), which used a full blocked relationship matrix. The fourth problem is that the exibility in using QFD in education should be enhanced, resting on the assumption that it is not just a plug-and-play decision machine (see Govers, 2001). There are two points to highlight. First, the number of HOQs does not have to be strictly four (Hauser & Clausing, 1988). Based on the necessity of the deployment process, the QFD team may decide how many HOQs to use. An example given by Brackin (2002) to follow the four phases showed the inexibility. Second, the true VOC should come from the proper and right customers. Several researchers in education do not include the students since they may have unnecessary wants and be considered too immature to judge the content of education. On the other hand, Sa & Saraiva (2001) attempted to include kindergarten children as the customers. This approach seems to be overcondent and risky. The fth problem lies in pooling the needs of several different customers into one group. This might possibly lead to a fallacious conclusion since one stakeholder may have a unique need that others may not consider, or even a conicting need with respect to other customers. An example for this case can be found in Koksal & Egitman (1998), which combined three different stakeholders into one. Therefore, in view of these problems, this paper will attempt to ll in the gap by providing a better methodology by using QFD and AHP to improve higher education quality. It is hoped that this will help higher education institutions, in general, improve their quality in the future by providing the best education program for their nation.

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

Using QFD with AHP The Basic QFD Model The Quality Function Deployment (QFD) technique was rst developed in the late 1960s by Yoji Akao and Shigeru Mizuno (Akao & Mazur, 2003). It is generally dened as a

A Methodology to Improve Higher Education Quality

1101

structural framework to translate customer requirements into appropriate technical requirements for product/service development to satisfy customers. There are a lot of good texts on the fundamental understanding of QFD, for example, Cohen (1995), Shillito (1994), ReVelle et al. (1997) or a recent one by Xie et al. (2003). A recent comprehensive literature review on the QFD and how to use it can be found in Chan & Wu (2002b), and a synopsis of recent methodological enhancement on QFD can be found in Kim et al. (2003). The House of Quality (HOQ) is the central component in constructing QFD (Hauser & Clausing, 1988). A typical HOQ chart is shown in Figure 1. The HOQ comprises of several standard main components as follows:

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

. Whats/Demanded Quality (DQ): a list of the customer requirements. . Hows/Quality Characteristic (QC): a list of ways for achieving the Whats, which is called the design/technical attributes/requirements. . Importance Ratings: the relative importance scores of each Demanded Quality which, in this paper, is derived using the AHP method. . Relationship Matrix: a matrix that shows the relationship level between Whats and Hows. In this paper, a ve-level correlation value is used to accommodate better the QFD team in assigning more accurate portrayal of DQ and QC relationships. The ve levels used are: Extreme (9), Strong (7), Moderate (5), Some (3), and Weak (1). . Correlation Matrix: a matrix that shows the relationship between Hows. . Customer Competitive Assessment: a review of competitive products/service characteristics in comparison with ours. . Score: a series of computed numbers for indicating the importance of each How. . Rank: a series of ordinal numbers that denote the rank of the QCs.

Figure 1. Basic model of the House of Quality (HOQ)

1102

H. Raharjo et al.

Throughout this paper, DQ and QC will be used to refer to Demanded Quality/Whats and Quality Characteristic/Hows, respectively. Assume that there are m DQs and n QCs, then the score of each QCs will be computed as in equation (1): ASj
m X i1

Rij IRi , i 1, 2, 3, . . . , m; j 1, 2, 3, . . . , n

(1)

where ASj Absolute Score of QCj Rij the weights assigned to the relationship matrix IRi Importance Rating of DQi The QFD output, which is a set of priorities of the QCs, can be obtained by assigning a sequence of discrete ascending numbers (1, 2, 3, . . ., n) to each QC from the highest score until the lowest one. A relative measure of the absolute score can also be used.
Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

The Use of AHP in QFD In the QFD matrix, the AHP is used to ll the left-hand part of the HOQ, that is, the Importance Ratings (IR) block (refer to Figure 1). There are two main reasons for using the AHP to elicit the degree of importance of the DQs. First, the use of absolute values to identify the degree of importance of DQ in the traditional QFD, for example, 1 to 5, with 5 denoting the most important, may lead to a tendency for the customers to assign values near to the highest possible score, and thus may lead to somewhat arbitrary and inaccurate results for prioritizing the QCs (Cohen, 1995; Chuang, 2001). Second, the use of the traditional QFD may lead to inconsistency in quantifying customers judgment (Akao, 1990, Lu et al., 1994; Armacost et al., 1994). It is therefore necessary to use a more comprehensive and accurate technique to provide an effective framework for determining the priorities of DQs in the QFD. The AHP is known to be a widely used tool to elicit the relative priorities of a set of objects using a pairwise comparison, which is considered as the most accurate way for humans to perfectly compare many criteria or alternatives two at a time. Despite the fact that it has a few shortcomings, for example, the cost and time required to do the pairwise comparison as the items get larger (Wang et al., 1998) and the rank reversal phenomenon (Belton & Gear, 1985; Saaty & Vargas, 1984; Saaty, 1990; Raharjo & Endah, 2006), its exceptional strength in quantifying intangible aspects, relative measurement, and consistency of decision makers outweighs other decision tools. Therefore, it is chosen for the purpose of this study. Generally, the AHP allows the decision maker to structure his problem hierarchically, to make a comparison of the elements with respect to a common property, and nally to synthesize all the judgments according to the hierarchy to obtain overall priorities of the set of objects. Furthermore, the AHP, relying on the fact that human judgment is known to be inconsistent to some extent and cannot be forced to be perfectly consistent in all cases, offers the advantage of doing a systematic check on the consistency of decision makers judgment and admits inconsistency in at a certain level that is less than or equal to 10% (Saaty, 1988). This property also makes the AHP a more exible and realistic approach for decision making as a descriptive theory (Saaty, 2006). For more details of the technical aspects, interested readers may refer to any vastly available AHP literature, for example,

A Methodology to Improve Higher Education Quality

1103

Saaty (1980, 1988). One may also simply use the Expert Choice software for using the AHP to derive priorities from a set of objects. Note that the question posed in the AHP questionnaire does not use a particular reference point, and thus it is purely subjective. In order to obtain a meaningful group preference for each party, it is assumed that each party is a collection of synergistic individuals who act together rather than separate individuals. This is also very reasonable in the context of an education institution with each of the involved parties working towards a common goal. On the basis of this assumption, the Aggregating Individual Judgment (AIJ) approach is the most suitable method, compared with the Aggregating Individual Priorities (AIP) approach (Forman & Peniwati, 1998). The AIJ approach, which uses the geometric mean, can be expressed as follows: " aG ij
n Y k1

#1=n ak ij (2)

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

where: n the number of decision makers, aG the group judgment of the (i,j ) element in the reciprocal matrix. ij Equation (2) assumes that the individuals are of equal importance; otherwise, one may use the weighted geometric mean. In this study, the students, lecturers and employers of graduates are assumed to be of equal importance. The weight of each DQ is calculated through pairwise comparison questionnaires given to every decision maker. Owing to the large number of DQs, the comparisons will be very tedious. Therefore, clustering can be used to reduce the number of comparisons. The DQs are classied into primary DQ group and secondary DQ group using the afnity diagram approach; as an example, the complete students party hierarchy is shown in Figure 2. This afnity diagram is similar to the method of clustering and will later help reduce the number of pairwise comparisons in the AHP. In other words, increasing the level of hierarchy can minimize the workload of using the AHP (Armacost et al., 1994).

Figure 2. An example of students party hierarchy

1104

H. Raharjo et al.

Finally, a reciprocal matrix is constructed to evaluate the consistency level. The consistency level of the judgments can be easily checked using the Expert Choice software; however, if the consistency level goes beyond 10%, a resurvey should be conducted on each of the decision makers. Proposed Methodology In this section, the proposed methodology of using the QFD with AHP will be elaborated. The aim of this methodology is to advance the use of QFD in improving higher education quality. This methodology has overcome the technical as well as the practical problems of using the QFD mentioned previously. Here, the AHP will be used to obtain relative measurement, obtain group preference, and check the inconsistency of decision makers judgments. A method proposed by Nakui (1991) was employed to ensure that no superuous DQs/QCs are being included while still maintaining the signicant relationships between DQ and QC. Each of the customers uses a separate HOQ. Note that the number of HOQs used can be adjusted according to the need of the deployment process. For a clearer description of the idea, a step-by-step approach will be presented. This procedure applies for each customer. A owchart of the step-by-step procedures can be seen in Figure 3. Step 1. Conduct a pilot survey of customer needs. In other words, this is an in-the-eld observation in order to collect the VOC from the true source of information. A variety of methods, such as contextual inquiry, direct observation, focus group, questionnaires, and so on, can be employed. After the survey, the QFD team should sort out and organize the preliminary results. This will provide the QFD team with the big picture of the customers needs. Conduct one-on-one in-depth interviews with the customers. In this step, adopting the Garbage-In-Garbage-Out (GIGO) philosophy, it is very crucial to select some knowledgeable decision makers who are also representative of each of the parties involved. Note that it is important to select the right students to be interviewed in order to avoid unnecessary and self-centered wants. Use an Afnity Diagram to classify or sort out the DQs and construct a hierarchy based on the grouping. The higher the hierarchy, the less the effort to obtain the important values. This hierarchy also serves as the AHP hierarchy. Explore each Demanded Quality hierarchically by a tree diagram and translate it into an appropriate Quality Characteristic. The QC is dened as the strategy/ way to achieve the DQ. One DQ may be related to some QCs, and vice versa. Verify whether the DQs and the respective QCs listed are already valid, otherwise, the QFD team should carry out the interview again. Ask the selected decision makers to make the AHP pairwise comparisons in order to derive the priorities of the DQs. The QFD team may explain to decision makers who are not familiar with the AHP mode of questioning. Obtain the group preference using Aggregating Individual Judgment (AIJ). Then, check whether there is a need to resurvey the decision makers owing to inconsistent judgments. The Expert Choice software can be used to obtain the priorities of DQs as well as to do the inconsistency check.

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

Step 2.

Step 3.

Step 4.

Step 5. Step 6.

Step 7.

A Methodology to Improve Higher Education Quality

1105

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

Figure 3. The proposed methodology using the QFD and AHP

Step 8.

Construct the House of Quality of each customer. The minimum set for constructing the HOQ should exist, such as the DQ, the Importance Rating, the QC, and the QCs score/ranks. Other components (the roof, competitive assessment, etc) might be added as necessary. Microsoft-Excel software would be a good alternative to do the HOQ analysis.

1106 Step 9.

H. Raharjo et al.

Verify the completed HOQ components. Some rules to check the correlation matrix as proposed by Nakui (1991) can be used. For example, if a DQ has no corresponding QC at all, then this DQ should be taken away. Step 10. Compute the QCs scores, and obtain their rankings. The QFD team may evaluate whether there is a need to extend the deployment process by using another HOQ. If there is a need to use another HOQ, a similar process can again be conducted (Step 8). Step 11. Conduct sensitivity analysis to provide a sense of how robust is the decision made by the QFD team if there is a change in the input data. This is also useful to anticipate future needs of customer and variability in the DQs. Step 12. Other downstream analysis, such as gap analysis, SWOT analysis, and so forth, can be added accordingly. A Case Study
Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

The House of Quality of the Customers These real-world case study data were taken from Raharjo & Dewi (2003). The purpose is to illustrate the use of the proposed methodology. The data for the students party were obtained from several representative students from each academic year who were still studying in the university. The students representatives had a minimum GPA of 3.0 out of 4.0. A number of employers of the graduates were interviewed by the QFD team using questionnaires with the help of the graduates themselves, while all the lecturers were interviewed on a one-on-one basis since there was a relatively small number of lecturers in the department. Some samples of the Quality Function Deployment charts that were produced from the process are shown in Figures 4, 5 and 6. The alternative solutions or QCs to each customer parties were derived from their respective Houses of Quality. Note that the roof was not emphasized in this case; however, it can be added if it is deemed necessary (see Wasserman, 1993). Sensitivity Analysis Since the DQs are based on the survey of a certain party at one point in time, it is very likely that the condition will change at another point in time in the near future, see Raharjo et al. (2006) or Kim et al. (2007) for further description of DQs dynamics. Therefore, a sensitivity analysis would be advantageous in dealing with the dynamics of DQs as well as in anticipating the future needs of the customers. Xie et al. (1998) wrote a study on the sensitivity of VOC in the QFD and showed that small changes in the customer requirement weights will not affect the rank in the technical attributes because the relationship matrix is discrete in nature. However, if the variability in the DQ is relatively large, then the prioritization in the QC might still be affected. In this study, the objectives of the sensitivity analysis are to anticipate the future changes of customer interest and to investigate the most sensitive component of the DQ. For the rst purpose, there are two cases to be analyzed for each customer party. The rst case (Case I) is to assign equal weights to each primary Demanded Quality, and the second case (Case II) is when one particular primary Demanded Quality outweighs the rest of the other requirements. An example for employers party importance rating change is shown in Table 1.

A Methodology to Improve Higher Education Quality

1107

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

Figure 4. Trimmed part of HOQ for students party

Figure 5. Trimmed part of HOQ for lecturers party

1108

H. Raharjo et al.

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

Figure 6. Complete HOQ for employers party

As a result of this change of weight in the primary DQ in Case I and Case II, the order of priority of the strategies to be taken has also changed accordingly. This can also be observed by the reversal in the QCs prioritization rank from the QFD chart. As an example, for the employers party, the initial alternative solutions, consecutively from the most important QCs, were to give more team assignment, arrange leadership training, get involved in committee activities, and so on. For Case I, a few of the QCs ranks were reversed, while in Case II the priority reversal occurred more often as shown in Table 2. To investigate the most sensitive component of the DQs, an experimental study was conducted for each of the customers. The most sensitive component is interpreted as the component that causes the largest number of differences in rank due to rank reversals of the QC when the corresponding DQ is changed. Table 3 gives the design of the experiment that will be carried out. There are two factors to be analyzed, namely, the Primary
Table 1. IR Change for employers party sensitivity analysis Academic qualication Case I Case II 0.25 0.05 Leadership skill 0.25 0.85 Interpersonal skill 0.25 0.05 Prob.solving skill 0.25 0.05

A Methodology to Improve Higher Education Quality

Table 2. Alternative solution for employers party Rk. 1 2 3 4 5 6 7
Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

1109

Initial Give more team assignments Leadership training Get involved in committee activities Intensify discussion and presentations Provide ethics and religion courses EQ training Give assignment with time limitation Provide foreign language classes Invite guest lecturers from industries Teach more mostlyused comp.prog Make more reasoning problems Give additional courses

Rk. 1 2 3 4 5 6 7 8 9 10 11 12

Case I Give more team assignments Leadership training Get involved in committee activities Intensify discussion and presentations Give assignment with time limitation Provide foreign language classes Teach more mostlyused comp.prog Invite guest lecturers from industries Provide ethics and religion courses Make more reasoning problems EQ training Give additional courses

Rk. 1 2 3 4 5 6 7 8 9 10 11 12

Case II Give more team assignments Get involved in committee activities Leadership training Intensify discussion and presentations Give assignment with time limitation Provide foreign language classes Teach more mostlyused comp.prog Invite guest lecturers from industries Provide ethics and religion courses Make more reasoning problems EQ training Give additional courses

8 9 10 11 12

DQ (PDQ) and the Importance Rating (IR). The PDQ has p elements of DQ [DQ1, DQ2, DQ3, . . ., DQp] of each customer, and the IR has four elements f2, 3, 4, 5g. A cell of the ith PDQ and the jth IR is dened as a collection of r variables, where r is the number of QCs in the HOQ. For instance, fY1,1,1, Y1,1,2, Y1,1,3, . . . ,Y1,1,rg describes the ranks generated when DQ1 is two-time relatively more important than DQ2, DQ3, . . . , DQp. The response variable of interest is the number of differences between the initial state rank (Xk) and the rank generated from the experiment design cell (Yi,j,k). The initial state was set up under the circumstance that each DQ had equal weight. If the initial state rank is denoted by fX1, X2, X3, . . . , Xrg and the generated rank from one of the experiment cells (i,j) is denoted by fYi, j,1, Yi, j,2, . . . , Yi, j,r g, then the rank difference, Dijk can be expressed as below:   Dijk Xk Yijk  where Dijk the absolute rank difference in the ith PDQ and jth IR for the kth QC. Xk initial rank of the kth QC. Yijk generated rank in the ith PDQ and jth IR of the kth QC. i 1, 2, 3, . . . , p, where p denotes the number of PDQs. j 1, 2, 3, 4 (IR 2-time, 3-time, 4-time, and 5-time, respectively) k 1, 2, 3, . . . , r, where r denotes the number of QCs in the HOQ. (3)

1110

H. Raharjo et al.
Table 3. The design of experiment for Yi,j,k and Di,j,k IR IR 4 times Y1,3,1 Y1,3,2 . . .. Y1,3,r Y2,3,1 Y2,3,2 . . .. Y2,3,r ... Yp,3,1 Yp,3,2 . . .. Yp,3,r 5 times Y1,4,1 Y1,4,2 . . .. Y1,4,r Y2,4,1 Y2,4,2 . . .. Y2,4,r ... Yp,4,1 Yp,4,2 . . .. Yp,4,r PDQ DQ1 2 times D1,1,1 D1,1,2 . . .. D1,1,r D2,1,1 D2,1,2 . . .. D2,1,r ... Dp,1,1 Dp,1,2 . . .. Dp,1,r 3 times D1,2,1 D1,2,2 . . .. D1,2,r D2,2,1 D2,2,2 . . .. D2,2,r ... Dp,2,1 Dp,2,2 . . .. Dp,2,r 4 times D1,3,1 D1,3,2 . . .. D1,3,r D2,3,1 D2,3,2 . . .. D2,3,r ... Dp,3,1 Dp,3,2 . . .. Dp,3,r 5 times D1,4,1 D1,4,2 . . .. D1,4,r D2,4,1 D2,4,2 . . .. D2,4,r ... Dp,4,1 Dp,4,2 . . .. Dp,4,r

PDQ DQ1

2 times Y1,1,1 Y1,1,2 . . .. Y1,1,r Y2,1,1 Y2,1,2 . . .. Y2,1,r ... Yp,1,1 Yp,1,2 . . .. Yp,1,r

3 times Y1,2,1 Y1,2,2 . . .. Y1,2,r Y2,2,1 Y2,2,2 . . .. Y2,2,r ... Yp,2,1 Yp,2,2 . . .. Yp,2,r

DQ2 . . .. . . DQp
Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

DQ2 . . .. . . DQp

To test if there is a signicant difference among the mean of Dijk in each cell, a twosample t-test with unequal variance was employed. The comparison analysis was conducted in pairs to determine the level of sensitivity of each DQ. The higher the mean of Dijk, the more sensitive the corresponding DQ. In support of the inferential analysis that was carried out for each party, a visual inspection method was also used. The boxplots suggest very strongly the conjecture that the means of rank difference are not all the same. As an example, Figure 7 shows one of the students party DQs (Facility) boxplot that reveals the higher the IR level, the higher the mean and variance of Dijk become. This simply implies that the IR level has a signicant effect on the mean of Dijk. Figure 8 also serves as an example of a boxplot for students party DQ at 5-time IR level. The visual investigation of Figure 8 substantiates the fact that the levels of sensitivity of the DQs are not all the same. The DQs of the students party, in the order of the most sensitive one, were facility, lecturer, extracurricular, administration, location,

Figure 7. Boxplot of one students party DQ (facility) by IR level

A Methodology to Improve Higher Education Quality

1111

Figure 8. Boxplot of all students party DQs at 5-time IR level

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

external relationship and curriculum. The mean and standard deviation of each cell in the experiment for the students party are provided in Table 4. A similar procedure to determine the most sensitive DQ was also applied for employers party and lecturers party. Results and Analysis From each of the QFD matrices analysis, the DQs can be sorted out in order of importance so that it can be revealed what matters most for each customer, while the alternative solutions to the department can be derived from reading the score/rank of the Quality Characteristics. For example, based on the level of importance, the attribute that matters most to the employers of the graduates was the Interpersonal Skill of which the subgroups, consecutively from the highest level of importance, were responsibility, honesty, communication skill, personality, and loyalty. While the primary alternative solutions for the employers party, namely, the QCs which have high ranks were to give more team assignment and leadership training, get involved in committee
Table 4. Descriptive statistics for students party Dijk IR Level 2 Students Party PDQ Lecturer Extracurricular Curriculum Location Administration External Relationship Facility Mean 1.81 1.68 0.39 0.77 1.03 0.39 2.65 Stdev 1.97 1.85 1.05 1.50 1.80 0.88 1.85 Mean 3.19 2.68 0.58 0.87 1.71 0.74 4.39 3 Stdev 2.77 2.86 1.29 1.65 2.65 1.55 3.08 Mean 4.06 3.23 0.58 0.97 2.26 0.71 5.23 4 Stdev 3.29 3.64 1.29 1.96 3.14 1.53 3.49 Mean 4.58 3.42 0.65 1.16 2.65 0.71 5.87 5 Stdev 3.69 3.90 1.31 2.45 3.53 1.53 3.83

1112

H. Raharjo et al.

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

activities, and intensify discussion and presentation. For other parties, similar analysis was done accordingly. Based on the gap analysis t-test (Zeithaml et al., 1990) that was conducted to check the signicant distance between the existing and the expected condition of the customer requirements, it was concluded that most of the requirements have a signicant gap, except for some attributes (lecturers working atmosphere, students campus location and extracurricular). The effect of changing the DQ in the sensitivity analysis has provided an insight into the alteration of the QCs priority due to further changes of customers interest. For the students party, the DQs that induced highest volatility in rank, consecutively, were facility, lecturer, extracurricular, administration, location, external relationship, and curriculum. For the lecturers party, the DQs, in order, from the most sensitive, were facility, working environment, curriculum, and bureaucracy. While for the employers party, the DQs that caused highest instability in rank, consecutively, were academic qualication, problem solving skill, interpersonal skill, and leadership skill. The DQs, of which variability may signicantly affect the nal output/decision of the QFD team, may provide the education institution with important information to formulate their strategy. Furthermore, it is also useful for determining which DQs are critical to improve quality in the future. Conclusions This paper has proposed a methodology for improving the quality of higher education institutions using the QFD and AHP. Some points to highlight in improving the use of the QFD in higher education, which have been discussed in this study, are: . It is important to use a relative measurement rather than a set of absolute values for representing the importance values of customer requirements in QFD, and the AHP can be considered as a powerful tool to serve this purpose. . Considerable attention should be paid to obtain a group preference. Using a geometric mean would generally be better compared to using an arithmetic mean in the case where the group acts synergistically towards a common goal. . A careful check should be conducted to identify the true relationship between the DQs and QCs in order to give a useful result. The QFD can be tailored to suit the particular need of the users, for example, in determining how many HOQs to use. In addition, for each customer, this study suggests that there should be one corresponding QFD analysis. For the case study, it can be concluded that endeavors which the higher education institution should take as a main priority were to develop the overall facility, re-evaluate existing curriculum, reduce unnecessary bureaucracy, improve lecturers qualications, and provide more leadership/team training. Furthermore, in order to design effective and efcient strategies, other subsequent/downstream analyses can be added, such as the gap analysis, Strengths-Weaknesses-Opportunity-Threat (SWOT) analysis, optimization with respect to cost constraint (Lai et al., 2007) and so forth. Alternative solutions that are generated from the QFD depend fully on the level of importance of the customer requirements. Any changes in the Demanded Qualitys

A Methodology to Improve Higher Education Quality

1113

level of importance will alter the priority order of the alternative solutions. The HOQ should be adjusted and updated when there are some changes in the existing condition, and it is very possible to add more Quality Characteristics to the HOQ. This is another advantage of using the QFD, that is, it is very useful in generating new or innovative QCs to meet the customer requirements. Finally, the sensitivity analysis may serve as a useful tool to anticipate the changes or variability in the DQ, and to provide the necessary information to better meet the future needs of the customers, and thus it enables the education institution to be alert, proactive, and forward thinking (Kuo, 2006).

Acknowledgement This research is partially supported by funding from Centre for Design Technology, National University of Singapore.

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

References
Akao, Y. (1990) Quality Function Deployment: Integrating Customer Requirements into Product Design (Cambridge, MA: Productivity Press). Akao, Y. & Mazur, G. H. (2003) The leading edge in QFD: past, present and future, International Journal of Quality & Reliability Management, 20(1), pp. 20 35. Armacost, R. L. et al. (1994) An AHP framework for prioritizing customer requirements in QFD: an industrialized housing application, IIE Transactions, 26(4), pp. 7279. Aytac, A. & Deniz, V. (2005) Quality function deployment in education: a curriculum review, Quality & Quantity, 39, pp. 507 514. Bailey, D. & Bennett, J. V. (1996) The realistic model of higher education, Quality Progress, November, pp. 7779. Belton, V. & Gear, T. (1985) The legitimacy of rank reversal a comment, Omega, 13(3), pp. 143 144. Bier, I. D. & Cornesky, R. (2001) Using QFD to construct a higher education curriculum, Quality Progress, April, pp. 64 68. Brackin, P. (2002) Assessing engineering education: an industrial analogy, International Journal of Engineering Education, 18(2), pp. 151156. Chan, L. K. & Wu, M. L. (2002a) Quality function deployment: a literature review, European Journal of Operational Research, 143, pp. 463 497. Chan, L. K. & Wu, M. L. (2002b) Quality function deployment: a comprehensive review on its concepts and methods, Quality Engineering, 15(1), pp. 2335. Chen, J. & Chen, J. C. (2001) QFD-based technical textbook evaluation procedure and a case study, Journal of Industrial Technology, 18(1), pp. 18. Chen, S. H. & Yang, C. C. (2004) Applications of web-QFD and E-Delphi method in the higher education system, Human Systems Management, 23, pp. 245 256. Chou, S. M. (2004) Evaluating the service quality of undergraduate nursing education in Taiwan using quality function deployment, Nurse Education Today, 24, pp. 310318. Chuang, P. T. (2001) Combining the analytic hierarchy process and quality function deployment for a location decision from a requirement perspective, International Journal of Advanced Manufacturing Technology, 18, pp. 842849. Cohen, L. (1995) Quality Function Deployment: How to Make QFD Work for You (Addison-Wesley). Dahlgaard, J. J. et al. (1995) Total quality management and education, Total Quality Management, 6(56), pp. 445 455. Duffuaa, S. O. et al. (2003) Quality function deployment for designing a basic statistics course, International Journal of Quality & Reliability Management, 20(6), pp. 740750. Ermer, D. S. (1995) Using QFD becomes an educational experience for students and faculty, Quality Progress, 28(5), pp. 131 136. Forman, E. & Peniwati, K. (1998) Aggregating individual judgments and priorities with the analytic hierarchy process, European Journal of Operational Research, 108, pp. 165 169.

1114

H. Raharjo et al.

Govers, C. P. M. (2001) QFD not just a tool but a way of quality management, International Journal of Production Economics, 69, pp. 151 159. Grant, D. et al. (2002) Quality management in US Higher education, Total Quality Management, 13(2), pp. 207 215. Grifn, A. & Hauser, J. R. (1993) The voice of the customer, Marketing Science, 12(1), pp. 127. Hauser, J. R. & Clausing, D. (1988) The House of Quality, Harvard Business Review, 66(3), pp. 6373. Hwarng, H. B. & Teo, C. (2000) Applying QFD in higher education, ASQs 54th Annual Quality Congress Proceedings, pp. 255 263. Hwarng, H. B. & Teo, C. (2001) Translating customers voices into operations requirements: a QFD application in higher education, International Journal of Quality & Reliability Management, 18(2), pp. 195 225. Jaraiedi, M. & Ritz, D. (1994) Total quality management applied to engineering education, Quality Assurance in Education, 2(1), 32 40. Kanji, G. K. & Tambi, A. M. B. A. (1999) Total quality management in UK higher education institution, Total Quality Management, 10(1), pp. 129153. Kaminski, P. C. et al. (2004) Evaluating and improving the quality of an engineering specialization program through the QFD methodology, International Journal of Engineering Education, 20(6), pp. 10341041. Kauffmann, P. et al. (2002) Using quality function deployment to select the courses and topics that enhance program effectiveness, Journal of Engineering Education, 91(2), pp. 231 237. Kim, K. J. et al. (2003) A synopsis of recent methodological enhancements on quality function deployment, International Journal of Industrial Engineering, 10(4), pp. 462466. Kim, K. J. et al. (2007) Robust QFD: framework and a case study, Quality and Reliability Engineering International, 23 (1), pp. 3144. Koksal, G. & Egitman, A. (1998) Planning and design of industrial engineering education quality, Computers & Industrial Engineering, 35(34), pp. 639642. Kuo, W. (2006) Assessment for U.S. engineering programs, IEEE Transactions on Reliability, 55(1), pp. 1 6. Lam, K. & Zhao, X. (1998) An application of quality function deployment to improve the quality of teaching, International Journal of Quality & Reliability Management, 15(4), pp. 389413. Lai X. et al. (2007) Optimizing product design using quantitative quality function deployment: a case study, Quality and Reliability Engineering International, 23(1), pp. 4557. Lu, M. et al. (1994) Integrating QFD, AHP, and benchmarking in strategic marketing, Journal of Business & Industrial Marketing, 9(1), pp. 4150. Madu, C. N. et al. (1994) TQM in the university: a quality code of honor, Total Quality Management, 5(6), pp. 375 390. Mergen, E. et al. (2000) Quality management applied to higher education, Total Quality Management, 11(3), pp. 345 352. Nakui, S. (1991) Comprehensive QFD system, Transactions from The Third Symposium on Quality Function Deployment, Novi, Michigan. Owlia, M. S. & Aspinwall, E. M. (1998) Application of quality function deployment for the improvement of quality in an engineering department, European Journal of Engineering Education, 23(1), pp. 105125. Pitman, G. et al. (1996) QFD application in an educational setting, International Journal of Quality & Reliability Management, 13(4), pp. 99108. Raharjo, H. & Dewi, D. R. S. (2003) Application of analytic hierarchy process in quality function deployment for improving quality at industrial engineering department university X. The 7th International Symposium on the Analytic Hierarchy Process, Bali, Indonesia, August 7 9, 2003, Indonesia. Raharjo, H. & Endah, D. (2006) Evaluating relationship of consistency ratio and number of alternatives on rank reversal in the AHP, Quality Engineering, 18(1), pp. 3946. Raharjo, H. et al. (2006) Prioritizing quality characteristics in dynamic quality function deployment, International Journal of Production Research, 44(23), pp. 50055018. Ramanathan, R. & Ganesh, L. S. (1994) Group preference aggregation methods employed in AHP: an evaluation and intrinsic process for deriving members weightages, European Journal of Operational Research, 79, pp. 249 265. Reavill, L. R. P. (1997) Quality assessment and the stakeholder model of higher education, Total Quality Management, 8(23), pp. 246252. ReVelle, J. B. et al. (1997) The QFD Handbook (New York: Wiley). Sa, P. M. E. & Saraiva, P. (2001) The development of an ideal kindergarten through concept engineering/ quality function deployment, Total Quality Management, 12(3), pp. 365372.

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

A Methodology to Improve Higher Education Quality

1115

Downloaded By: [TBTAK EKUAL] At: 11:58 1 October 2010

Shillito, M. L. (1994) Advanced QFD: Linking Technology to Market and Company Needs (New York: Wiley). Saaty, T. L. (1980) The Analytic Hierarchy Process (New York: McGraw-Hill). Saaty, T. L. (1988) Multicriteria Decision Making, The Analytic Hierarchy Process, Planning, Priority, Setting, Resource Allocation (Pittsburgh: RWS Publications). Saaty, T. L. (1990) An exposition of the AHP in reply to the paper Remarks on the Analytic Hierarchy Process, Management Science, 36(3), pp. 259268. Saaty, T. L. (2006) Rank from comparisons and from ratings in the analytic hierarchy/network processes, European Journal of Operation Research, 168, pp. 557570. Saaty, T. L. & Vargas, L. G. (1984) The legitimacy of rank reversal, Omega, 12(5), pp. 513 516. Sahney, S. et al. (2004) A SERVQUAL and QFD approach to total quality education: a student perspective, International Journal of Productivity and Performance Management, 53(2), pp. 143 166. Sahney, S. et al. (2006) An integrated framework for quality in education: application of quality function deployment, interpretive structural modelling and path analysis, Total Quality Management, 17(2), pp. 265 285. Sakthivel, P. B. & Raju, R. (2006) Conceptualizing total quality management in engineering education and developing a TQM educational excellence model, Total Quality Management, 17(7), pp. 913934. Sirvanci, M. (1996) Are students the true customers of higher education, Quality Progress, October, pp. 99102. Wallace, J. (1999) The case for student as customer, Quality Progress, February, pp. 47 51. Wang, H. et al. (1998) A comparative study of the prioritization matrix method and the analytic hierarchy process technique in quality function deployment, Total Quality Management, 9(6), pp. 421430. Wasserman, G.S. (1993) On how to prioritize design requirements during the QFD planning process, IIE Transactions, 25(3), pp. 59 65. Widrick, S. M. et al. (2002) Measuring the dimension of quality in higher education, Total Quality Management, 13(1), pp. 123 131. Willis, T. H. & Taylor, A. J. (1999) Total quality management and higher education: the employers perspective, Total Quality Management, 10(7), pp. 997 1007. Xie, M. et al. (1998). A sensitivity study of the sensitivity of Customer Voice in QFD analysis, International Journal of Industrial Engineering Applications and Practices, 5(1), pp. 301 307. Xie, M. et al. (2003). Advanced QFD Applications (Quality Press, ASQ). Zeithaml, V. A. et al. (1990) Delivering Quality Service: Balancing Customer Perceptions And Expectations (New York: Free Press).