A QUALITY PROCESS APPROACH TO INTERNAL

COMPLAINT SYSTEM ANALYSIS

George L. Mitchell
Manager, QA
BellSouth Telecommunications
Atlanta, GA 30329

SUMMARY

Establishing and maintaining customer satisfaction and loyalty is key to business success in an increasingly
global, competitive marketplace. Process control (behavior) charts can be useful in the service industry to identify
needed process changes. By using response intervals of suppliers/manufactures in taking corrective action for non-
conformities, control charts can be used to examine special causes (extreme values) that contribute to extensive sup-
plier delays. Pareto charts were used in this analysis of a telecommunications network to identify the reasons for
extreme response intervals so suppliers and/or the telecommunications company could take appropriate corrective
action to reduce or eliminate the conditions that lead to extreme values.

KEY WORDS

complaint, response time, statistical process control (SPC)

INTRODUCTION

Local exchange carriers (LECs) in the telecommunications industry install and maintain sophisticated networks
to meet the critical demands of diverse customers. Nonconformities in the telecommunications network need to be
found and reported before the end-user (paying customer) suffers from disrupted communications. Corrective action
requests are widely used in many industries for obtaining corrective supplier/manufacturer action, and they are espe-
cially important in the telecommunications industry where service to customers is expected to be maintained at a reli-
ability level of 0.999994 (Bellcore 1997; 1998). Each LEC or Telco has its own idiosyncratic procedures for issuing
and handling corrective action requests (called “engineering complaints” in the industry). In an attempt to establish
some consistency from LEC to LEC in reporting supplier problems, Bellcore in 1995 developed generic requirements
for engineering complaints (GR-230-CORE). GR-230-CORE allows for a standardization of procedures and a more
accurate analysis of problems between and among suppliers and LECs.
Intuitively we know and studies have shown (Aubrey 1993; Godfrey & Endres 1994; Grieco 1995; Halstead, Droge
& Cooper 1993; Harari 1992; Prasad 1998) that good customer-supplier relations depend on clear and open communi-
cation with immediate feedback and resolution for complaints (in this study, supplier/manufacturer response to an engi-
neering complaint). Supplier response times to an engineering complaint can be measured quantitatively by recording
the days the supplier takes to offer corrective action. This data can be compiled and displayed in a control chart.
Response times have been used with statistical process control (SPC) to monitor various response-time-related activities
(Gilbert, et al. 1990; Johnson & Martin 1996; Lloyd & Steinberg 1996). Corrective action developed from the analysis
of SPC and associated assignable causes of variation leads to more efficient processes. This study used SPC to identify
engineering complaints that were special causes of variation in order to ameliorate the complaint process.

791
792 ASQ’s 54th Annual Quality Congress Proceedings

PURPOSE OF THE INVESTIGATION

A stubborn inconsistency remained between established guidelines for supplier/manufacturer response times and the
actual data despite numerous nonstatistical analyses and the subsequent internal process changes. Previous efforts, while
yielding some improvements, were still far off the targeted complaint-resolution interval. Although the mechanics of pro-
cessing complaints had long been established, no effective method had been tried to statistically analyze these intervals.
In fact, neither suppliers nor the LEC used in the study had systematically monitored response times which may have
contributed to special causes of variation for some engineering complaints. The system required quantitative analysis of
historical data of suppliers’ response times in order to determine the causes of variation and to improve the process.

DATA COLLECTION

Response times were used to measure the time each supplier/manufacturer took from their receipt of a complaint
until providing a resolution acceptable to the LEC. SPC was used to measure “supplier response time.” Historical data
was drawn from the period beginning with 1992 and ending with April 1999. More than 1,400 supplier complaint
responses were used to obtain average response times for control charts.
All supplier/manufacturer responses offering accepted corrective action were used in this study. Supplier
responses were grouped by the month the complaint corrective action was closed. Since there was no consistency as
to the number of complaints that were closed during any month, a variable subgroup size, was used to calculate the
months/subgroups. Complaints closed per month ranged from 4 to 53. The standard deviation was used to measure
the variation within each month since more than 80 percent of the months had 10 or more complaints/observations.
The data is not statistically homogeneous in nature because the data was drawn from many suppliers, providing
various products that may have a simple or complex resolution. But the process is defined and understood by all sup-
pliers, an acceptable resolution is to be provided within 90 days of receipt.

RESULTS

Table 1 shows an example of the raw data. This information is for the most current period from January 1999
through April 1999. The table contains the interval (in days) for the supplier/manufacturer to develop an acceptable
course of action in response to the complaint. These intervals are combined by month and used in the development
of control charts. The first column displays the complaint number. The three alpha characters reference the LEC and
the state where the complaint originated. The first two numbers of the complaint designate the year the complaint
was issued. The last three numbers are sequential for the originating state; for example, the first listed complaint
originated in 1998 from Georgia, and, sequentially, it was the 35th complaint developed in Georgia during 1998.
The second column establishes the parameter for each complaint’s inclusion within a specified month. In January
1999, five complaints were closed while 17 were closed in February. A monthly interval was used in the collection
of the data as a simple convention. The months are color-coded the ease of differentiation. The last column of Table
1 shows supplier/manufacturer response time, in days, for each complaint in column one.
Table 2 summarizes the population characteristics of the first four months in 1999. There is a disparity in March
1999 where the average and standard deviation are much larger than the population characteristics for the other
months. One of the reasons for the higher calculations in March was attributable to complaint BSF95034 where the
supplier/manufacturer took 1,224 days to respond.
Figure 1 is the X-bar chart of monthly unadjusted (raw) data of supplier response times (in days) from 1992
through April 1999. It took, on average, 112 days for a supplier to respond to a complaint with corrective action. The
upper and lower confidence intervals vary in accordance to N (the number of complaints closed during the month).
There were three extreme values that fell outside the upper control limit for a specific month. These months were iden-
tified as January 1992, March 1997, and March 1999.
Figure 2 is a sigma chart. The standard deviation of the raw data was taken of each month to identify the varia-
tion within the month rather than the range due to the month-to-month variability of the number of complaints (aver-
aged more than 16) in each month. Figure 2 identified eight data points that were outside the upper control limits. No
other signals were viewed as requiring investigation.
 ASQ’s 54th Annual Quality Congress Proceedings 793

Table 1. Raw data—closed complaints, January 1999 through April 1999.

COMPLAINT DATE TIME (IN DAYS) FOR COMPLAINT DATE TIME (IN DAYS) FOR
NUMBER CLOSED A SUPPLIER REPONSE NUMBER CLOSED A SUPPLIER REPONSE
BSG98035 1/4/99 33 BSA97078 3/1/99 185
BSG98054 1/20/99 27 BST98002 3/1/99 399
BSN98032 1/20/99 21 BSF98116 3/2/99 110
BSN98028 1/25/99 98 BSG97014 3/2/99 582
BSF98118 1/29/99 69 BSF95034 3/5/99 1224
BSN98033 2/4/99 60 BSG99006 3/11/99 37
BSF99012 2/8/99 13 BSL98015 3/11/99 76
BSA98016 2/10/99 123 BSL98017 3/11/99 132
BSF98101 2/10/99 98 BSN99001 3/12/99 40
BSF98091 2/11/99 142 BSN98029 3/16/99 90
BST98008 2/13/99 265 BSM98005 3/18/99 127
BSG98045 2/17/99 91 BSL99002 3/19/99 29
BSG98048 2/17/99 91 BSF98108 3/26/99 48
BSA98037 2/18/99 69 BSG98002 4/1/99 296
BSA98049 2/19/99 63 BSN98013 4/1/99 300
BSG98043 2/19/99 74 BSF98123 4/2/99 78
BSA98044 2/24/99 8 BSF99009 4/2/99 49
BSF98087 2/24/99 160 BSG98014 4/26/99 351
BSF99017 2/25/99 22 BSF99010 4/27/99 94
BSF99018 2/25/99 25 BSS98020 4/28/99 165
BSF98092 2/26/99 144 BSF98063 4/30/99 265
BSG98037 2/26/99 154

Table 2. Average and standard deviation of supplier response times.

January February March April

Average 49.6 94.23 236.85 199.75
N 5 17 13 8
Standard Deviation 32.87 66.11 337.48 117.34

Three iterations of control chart analyses were made of the data to identify and quantify assignable causes of individ-
ual extreme complaints within months. Extremes were eliminated to establish a stable series with a minimum variance.
Table 3 again lists complaints in the first four months of 1999. This time, however, the table identifies complaints with aster-
isks that were excluded from the calculation of the final control charts. An additional column was added to list the assign-
able cause for each extreme.
Table 4 summarizes the population characteristics of the first four months in 1999 after monthly extremes were elim-
inated. There were no extremes in January, therefore no change in the average and standard deviation. One complaint was
eliminated in February with an assignable cause of a “continuation.” The initial response from the supplier/manufacturer
was unacceptable and was returned for a more acceptable answer. Once this complaint was eliminated form the month,
the average for the month was calculated as 84 days (compared to an average of 94 days with the raw data). The standard
deviation for February was reduced from 66.11 to an adjusted standard deviation of 50.94. March and April experienced
decreases in the average and standard deviation by eliminating 3 values/complaints in each of the months. Each of the six
complaints had assignable causes, which contributed to their extreme values.
794 ASQ’s 54th Annual Quality Congress Proceedings

Figure 1. Supplier response times—unadjusted.

Figure 2. Standard deviation chart—unadjusted.

 ASQ’s 54th Annual Quality Congress Proceedings 795

Table 3. Adjusted Complaints—January 1999 through April 1999.

COMPLAINT DATE TIME (IN DAYS) FOR ASSIGNABLE

NUMBER CLOSED A SUPPLIER RESPONSE CAUSE
BSG98035 1/4/99 33
BSG98054 1/20/99 27
BSN98032 1/20/99 21
BSN98028 1/25/99 98
BSF98118 1/29/99 69
BSN98033 2/4/99 60
BSF99012 2/8/99 13
BSA98016 2/10/99 123
BSF98101 2/10/99 98
BSF98091 2/11/99 142
BST98008 2/13/99 265* Continuation*
BSG98045 2/17/99 91
BSG98048 2/17/99 91
BSA98037 2/18/99 69
BSA98049 2/19/99 63
BSG98043 2/19/99 74
BSA98044 2/24/99 8
BSF98087 2/24/99 160
BSF99017 2/25/99 22
BSF99018 2/25/99 25
BSF98092 2/26/99 144
BSG98037 2/26/99 154
BSA97078 3/1/99 185
BST98002 3/1/99 399* Compensation*
BSF98116 3/2/99 110
BSG97014 3/2/99 582* PCN*
BSF95034 3/5/99 1224* PCN*
BSG99006 3/11/99 37
BSL98015 3/11/99 76
BSL98017 3/11/99 132
BSN99001 3/12/99 40
BSN98029 3/16/99 90
BSM98005 3/18/99 127
BSL99002 3/19/99 29
BSF98108 3/26/99 48
BSG98002 4/1/99 296* Continuation*
BSN98013 4/1/99 300* Compensation*
BSF98123 4/2/99 78
BSF99009 4/2/99 49
BSG98014 4/26/99 351* PCN*
BSF99010 4/27/99 94
BSS98020 4/28/99 165
BSF98063 4/30/99 265
796 ASQ’s 54th Annual Quality Congress Proceedings

Figure 3. Supplier response times—adjusted.

Figure 4. Standard deviation chart—adjusted.

Table 4. Average and standard deviation of adjusted supplier response times.

January February March April

Average 49.6 83.56 87.4 130.2
N 5 16 10 5
Standard Deviation 32.87 50.94 51.08 86.62
 ASQ’s 54th Annual Quality Congress Proceedings 797

Figure 5. Supplier causes of out-of-control conditions.

Figure 3 shows the X-bar chart of supplier/manufacturer response times adjusted for extremes. The grand aver-
age was calculated as 83 days, more than a 25 percent decrease from that of the unadjusted average of 112 days. All
months were in control.
The s-chart (sigma chart) showed similar results. Figure 4 included the standard deviation and monthly variation
for supplier/manufacturer response times. The centerline of the adjusted series in Figure 4 decreased to 77 from 132
in the unadjusted s-bar chart. All months were in control.
From 1992 through April 1999, less than 4 percent of the complaints (52) were eliminated from the analysis in
order to obtain stable control charts. Figure 5 is a Pareto chart of the quantifiable reasons for the 52 extreme com-
plaints that were eliminated from the monthly data in order to bring the system into control. Each factor can be
explained in one of the following ways:
• Product change notice—Developed a PCN to correct a nonconformity to a product or software.
• Continuation—Supplier’s initial response was unacceptable and the complaint was sent to the supplier a
second time.
• Technological change—Product/software redesigns were made without issuing a PCN.
• Compensation—As part of the resolution that may have included approval or compromise.
• Supplier timing error—Supplier failed to process the complaint internally.

CONCLUSION

Special cause (extreme) variation was identified through the analysis of SPC. Pareto charts identified the condi-
tions contributing to the special causes of variation. The results of this analysis and appropriate actions by both indi-
vidual suppliers and the customer (Local Exchange Carrier) can reduce variation and improve the process, thereby
lowering the average response time.
“Continuation,” “compensation,” and “supplier timing error” contributed to more than 50% of the special causes
of variation intervals for complaints. “Continuation,” “compensation,” and “supplier timing error” can be reduced by
supplier and internal customer (LEC) management. Awareness of these potential pitfalls and appropriate corrective
action will reduce or eliminate their occurrences. The internal customer (LEC) could initiate an awareness program
with suppliers about the engineering complaint process.
Proposed corrective action—LEC and supplier procedural expectations include:
798 ASQ’s 54th Annual Quality Congress Proceedings

• Product change notice. When the supplier/manufacturer becomes aware that a PCN is likely, supplier
resources are needed to speed up and manage the approval process. This includes, but is not limited to,
development and testing of the prototype, LEC testing or field trial, and administrative activities.
• Continuation. Each time a complaint is sent to a supplier, the LEC should strongly suggest that the supplier
discuss the proposed resolution with the LEC subject matter expert before the response is officially sent to
the LEC coordinator. Notifying the subject matter expert of the proposed resolution will decrease the possi-
bility that the subject matter expert is “surprised” with the response and therefor less likely to reject it.
• Technological change. The supplier/manufacturer needs to take actions similar to those required in devel-
oping a PCN.
• Compensation. The LEC should notify a supplier when compensation is part of the expected resolution,
advising the supplier/manufacturer that compensation must be addressed. Suppliers should be aware that
when compensation is not attended to in their response, the complaint will be sent back to them as not
having met the conditions for closure.
• Supplier/manufacturer timing error. Usually an oversight the supplier will “forget” a that a complaint
must be managed throughout the organization either because the process has not been documented or
fully understood. The customer (LEC) must therefor remind the supplier that a response is due.
A vigilant awareness program between customer (LEC) and supplier can reduce special causes of variation in the
complaint process. Identification of extreme values and the elimination of these points do not mean that the data series
is in control. There are other checks of control charts, e.g., runs, trend, etc., that were not taken into consideration do
to the non-homogeneous nature of the data. There are many other instances inherent in the data where five categories
displayed in the Pareto charts are inherent but are not identified as extremes. These conditions part of common cause
variation, and any changes made by a supplier/manufacturer to address these conditions will cause a decrease in
response times. Continuing communications between the customer (LEC) and supplier will reduce response times to
complaints, make for a more satisfied customer, and do much to insure a continuing business relationship. As Barlow
and M∅ller (1996) have said, “A complaint is a gift,” and offers a means to improve. Guidelines and process changes
have been proposed that will decrease both assignable and common causes of variation.

REFERENCES

Aubrey, C. F. 1993. “Strengthening the Chain.” TQM 5(5): 51–54.

Barlow, Janelle, and Clause M∅ller. 1996. A Complaint is a Gift. San Francisco: Berrett-Koehler.
Bellcore GR-230-CORE. 1995. “Generic Requirements for Engineering Complaints.” Bellcore 1.
Bellcore GR-512-CORE. 1998. “LSSGR: Reliability,” Bellcore 2 (12).
Bellcore GR-929-CORE. 1997. “Reliability and Quality Measurements for Telecommunications Systems (RQMS).”
Bellcore 3: 3–12.
Gilbert, K. C., J. M. Reeve, and R. A. Wannemacher. 1990. “Improving Information System Efficiency Through
Statistical Process Control.” Journal of Information Systems Management 7(2): 8–14.
Godfrey, A. B., and A. C. Endres. 1994. “The Evolution of Quality Management Within Telecommunications.” IEEE
Communications 3(10): 26-34.
Grieco, P. L., Jr. 1995. “Supplier Partnering.” Executive Excellence 12(10): 14.
Halstead, D., C. Droge, and M. B. Cooper. 1993. “Product Warranties a Post Purchase Service: A Model of Consumer
Satisfaction with Complaint Resolution.” Journal of Services Marketing 7(1): 33–40.
Harari, O. 1992. “Nourishing the Complaint Process.” Management Review 81(2): 41–42.
Johnson, C. C., and M. Martin. 1996. “Effectiveness of a Physician Education-Program in Reducing Consumption of
Hospital Resources in Elective Total Hip-Replacement.” Southern Medical Journal 89(3): 282–289.
Lloyd, S. J., and G. Steinberg. 1997. “The Schema Mapper: An Expert System that Determines the Least Cost
Physical File Structure in a Database Management System.” Journal of Database Management 8(2): 16–22.
Prasad, B. 1998. “Review of QFD and Related Deployment Techniques.” Journal of Manufacturing Systems 17(3):
221–234.