Quality Control Study on Negative
Gynecological Cytology in Mexico
Eduardo Cesar Lazcano-Ponce, M.D., Patricia Alonso de Ruiz, M.D.,
Lizbeth Lopez-Carrillo, M.D., Maria Eugenia Vazquez-Manriquez, M.D.,
and Mauricio Hernandez-Avila, M.D.
An external evaluution of the qudity of diagnosis oJ’gynecologica1
cytology specimens (Papanicolaou or “Pap” tests) in the Cytopa-
thological Laboratory al the Mexico General Hospital was per-
formed, with evaluation of 6,011 Pap tests randomly selectedfrom
31,378 Pap studies previously reported as negative during 1988-
1989. These cests were part of the early detectioir program for
cervical cancer run by the Ministry of Health of Mexico City. A
quality index jor specimens was constructed based on the presence
of endocervical cells, mucus, and squamous metaplastic cells. The
index of falx negatives for the diagnosis of cervical neoplasm in
this sample which correspond to diagnostic error by the examiner
was 0.18% (11). The constructed quality index revealed that
64. I % of specimens were low quality. The correlation between
diagnostic error and low quality was .8Z Results indicate that
laboratorie.r participating in early detection programs for cervical
cancer in developing countries need to implement ( I ) external
evaluation programs for Pap diagnosis; (2) permanent training
programs in registration: and (3) monitoring promdures for col-
lection, Jixotion, and transportation of cytologic materials.
niagn Cytopathol 1994;lO:lO-14. (<) 1994 Wiley-Liw, Inc
Key Words: Quality control for Pap tests; Uterine cervix cancer;
Kappa concordance coefficient; Population cancer detection; Di-
agnostic error by the examiner
Uterine cervical cancer is one of the principal public
health problems in the Republic of Mexico and in much’
of Latin America. When comparing the incidence rates for
uterine cervical cancer found in Latin American countries
with those of other countries, substantive differences ap-
pear. For example, in Cali, Colombia, an incidence rate of
48.2 per 100,000 women has been estimated. In Fortaleza,
Brazil, a rate of 46.5, in Costa Rica a rate of 36.9, in
Received January 29, 1993. Accepted June 30, 1993.
Prom the Mexican Social Security Institute, the Cytopathology Labo-
ratory, Pathology Unit, National University of Mexico, Mexico General
Hospital, the National Institute of Public Health, an.d the National
lnstitutc of Respiratory Diseases
Address reprint requests to Dr. Lizbeth Lopez-Carrillo, Centro de
Investigaciones en Salud Publica, Instituto Nacional do Salud Publica,
Av. IJniversidad 655, Col. Sta. Ma. Ahuacatitlan, Cumnavaca, Mor.,
C1.1’. 62508, Mexico.
10 Diagnostic Cytopathology, Vol 10, No 1
Martinique a rate of 18.0, and in Puerto Rico we find a
rate of 15.6 per 100,OOO women. In Israel the incidence
rate varies between 3.1 and 5.1 per 100,000 in different
ethnic groups. *
Women of Latino origin who live in other countries
continue to have high frequencies. For instance, Latina
women living in the United States have the third highest
frequency of cervical neoplasms reported in that country.
The incidence of uterine cervical cancer among women
with Hispanic surnames who live in the US.is seven times
greater than that of other ethnic groups, with mortality
rates for uterine cervical cancer two times greater than
Anglo-Saxon women (4.8 compared to 2.3/100,000,re-
spectively).’
In Mexico City, the incidence of uterine cervical cancer
is approximately 3,500 cases annually. The number of
deaths due to uterine cervical cancer in 1990 was es-
timated at 400, which represents a crude mortality rate of
7.6/100,000 women. This rate is lower than the national
mortality rate for uterine cervical cancer of 9.5/100,000
women.
A detection program for uterine cervical cancer was
initiated in Mexico in 1942. In 1974 a technical council
was created to normalize the activities of the early detec-
tion program for uterine cervical cancer in populations.
In 1990, the Mexican Social Security Institute per-
formed approximately one million Papanicolaou (Pap)
tests nationwide, which represents coverage of almost
15% of women. In the last 15 yr the mortality trend for
uterine cervical cancer has remained stable in this popula-
’
tion. In contrast, the national early detection of uterine
cervical cancer program in the population without social
security during the same year reached approximately 12%
of women, or 767,115 Pap tests.
One parameter for identifying error indexes’ (false posi-
tives and false negatives) is external evaluation, which
consists of review of all positive cases and 10% of ran-
domly selected negative samples. l o A false negative test
occurs as a result of bad techniques when taking Pap
0 1994 WILEY-LISS, INC

smears, predominantly due to inadequate sampling of the
squamo-columnar area, or as a result of diagnostic error
due to incorrect interpretation by the examiner.
In Mexico and in much of Latin America there is little
information about the quality of the slides used in gyneco-
logical cytology mass detection programs. To generate
this information, we constructed a specimen quality index
for use in quality control program of the Cytopathology
Laboratory of the General Hospital, Ministry of Health,
Pathology Unit in the National University of Mexico,
which participates in the early detection of uterine cervical
cancer program for the metropolitan area of Mexico City.
Methods
A total of 6,011 Pap tests for 1990 from the Ministry of
Health's Mexico City early detection program were evalu-
ated in the Cytopathology Laboratory at the Mexico Gen-
eral Hospital. Tests were selected from 3 1,378Pap tests pre-
viously reported as negative during the 1988-1989 period.
Selection was based on generation of random numbers.
The material was evaluated by four possible observers,
the first two of whom reviewed 6,011 slides separately.
Discordant diagnoses were read by a third observer (503
slides) and, finally, the fourth observer reviewed 35 slides
with discordant results in the three previous readings.
The first reading was performed by second year resi-
dents in pathology who were trained in cytology and by
technicians in cytotechnology training.
The second reading was performed by a fourth year
resident in pathology (with adequate cytodiagnostic train-
ing). The third reading was performed by two cytotechni-
cians with 15 yr experience and a diploma from the Coun-
cil of Cytology Technicians. The fourth observation was
performed by the head of cytodiagnostic services, who has
30 yr experience.
To evaluate the quality of the Pap sampling procedure,
a quality index for gynecological cytology l2 was con-
structed based on the presence of endocervical cells (EC),
mucus (MU), and squamous metaplastic cells (SM), as
described here:
1. High quality, presence of EC, MU, and SM;
2. Good quality, presence of EC and MU;
3. Medium quality, a) presence of EC and MU; b) pres-
ence of MU and SM;
4. Low quality; absence of EC, MU, and SM.
The Pap smear is a diagnostic procedure which is not
error free, the evaluation of which is established through
certain criteria. Due to the impossibility of determining
quantitative measures of diagnostic precision (sensitivity
and specificity) and due to the absence of anatomical-
pathological results (for ethical reasons), Kappa concor-
dance coefficients were used in this study.
GYNECOLOGICAL CYTOLOGY IN MEXICO
Therefore, in this study reproductibility refers to the
extent to which a measurement instrument produces the
same result when applied repeatedly under the same cir-
cumstances. Validity is the extent to which an observa-
tion, technique, or index measures what it is meant to
measure, that is, the lack of a systematic error.
The Kappa correlation coefficient was defined in the
following way:
13 - concordance observed-concordance expected
Kappa -
l-concordance expected
Statistical Analysis
The following variables were described with simple fre-
quencies and central tendency measurements: age, the
cytological diagnosis of the four readings, presence of mi-
croorganisms (bacilli, cocci, mixed flora, and gardnerella),
mucus, endocervical cells, and metaplastic cells.
A table comparing the pre-established quality index and
6,011 cytological diagnoses was constructed. The re-
producibility of the first and second observation was eval-
uated with the Kappa concordance coefficient. The valid-
ity of these readings was also evaluated using the results
of a third reviewer as a standard. All analytical procedures
were performed with SPSS statistical software. l4
Results
This study found that in a random sample of women who
participate in the early detection of uterine cervical cancer
in the metropolitan area of Mexico City and who had a
previous negative Pap smear, 67.5% have a cytological
diagnosis of inflammatory alterations. Those cytologies
without alterations represent 20.6% of the sample, and are
followed in order of importance by the presence of patho-
genic flora in 6.3%, taking as a reference the second diag-
nostic observation (Fig. 1). The index of false negatives
attributed to diagnostic error by the examiner in the re-
port made previous to this study was 0.18%, which corre-
sponds to 11 positive results. For dysplasias, a false nega-
tive index of 0.03% was observeda2The false negative
index was 0.08% for uterine cervical cancer in situi5
0.05% for invasive epidermoid uterine cervical cancer,
and 0.01% for adenocarcinoma' (Fig. 1).
Age was not recorded in 70.1% of the cases, but the
median age in the remainder of the sample was 38.6, with
a minimum age of 18 yr and a maximum reported age of
85. The group from 25 to 34 yr old was most frequent, at
9.2%.
For microbial flora, bacilli predominate with 45.2% of
the cytological diagnoses (Fig. 2). Pathogenic flora were
reported in 7 1.4% of Pap smears. Trichomones vaginalis,
which is included in this diagnosis, was seasonally variable
in relation to the 31,378 Pap tests reported in the 1988-
Diagnostic Cytopathology, Vol 10, No I 11
LAZCANO-PONCE ET AL.

600,
-_

-\
TllchomOneS

5aa!
400

Cnnoida
.2 0O 0O 1

Fig. 3. Distribution by season of trichomones and candida in the Mexico

Pig. 1. Distribution by percentage of cylological diagnosis from the General Hospital during 1988-1989.
sccond observiition.

Fig. 2. Distribution by pcrcentage of microbial flora. Fig. 4. Distribution by perccntage of the quality index

1989 period being highest during the April to June trimes- Table I. Kappa Concordance Coefficients
-
ter (Fig. 3). Cytologicol diagnosis Kappa cocflcicnt n
--
'The distribution by percentage of the constructed yual-
ity index in gynecological cytology reflects that 64.1% of
,.2"
113h
0.40*
.-0.04**
6,Ol 1
503
Pap smears are of low quality, 17.2% are of medium 2:3~ 0.50* 503
quality, 1 1% are good quality, and 7.7% are high quality "Rep,.oci,,cibil,ty between observers I and 2,
(Fig. 4). 'Validity of first observer.
In terms of reproducibility, the concordance between 'Validity of second observer.
* P = 0.001.
the first and second observation was .40. The: Kappa cor- **P = '> 0.05.
relation coefficient observed between the first and third
observations is negative with a coefficient of -0.40, and
the concordance between the second and third observation Discussion
is 0.56 (Table I). From a public health perspective:, quality control and
'The fourth observer evaluated 35 Pap tests, which cor- error index methods are fundamental parts of early cancer
responds to 0.58% of the sample studied, where there was detection programs. The incidence of false negative re-
discordance between the first three diagnostic cytologies. sults depends on the quality of the cytodiagnostic lab-
' ' v ' ~

Finally, a correlation coefficient of .87 exists between oratory, which includes the following elements: trained
the 503 discordant diagnoses from the first two observa- personnel, operational efficiency, and internal and exter-
tions and the fourth quality index, which corresponds to nal evaluation of the Pap diagnoses.
low quality of the specimen obtained. The evaluation of the populational early cancer detec-

tion program should be carried out on different levels:
identification of the population at risk, collection of speci-
mens, processing, cytological diagnosis, reporting, follow-
up, and treatment. Therefore, each of these elements must
fulfill maximum quality requirements. 17,18 If there is a
breakdown of this process, the program is not useful. For
false negative results which are attributed to an error in
the previous diagnostic interpretation, this study found a
prevalence of 0.18% (2 dysplasias, 5 uterine cervical can-
cers in situ, 3 invasive epidemoid uterine cervical cancers,
and 1 adenocarcinoma). These results, obtained in a high
risk population, as is the case of Mexican women, are
greater than the false negative indexes found in industrial-
ized countries whose early cancer detection programs
have included Pap smear quality control monitors for
more than 15 yr and whose false negative index in diagnos-
tic error varies between 0.01 and 0.06%. 19-22
Likewise, through the construction of the quality index
for the cytology specimen, 12i23 our study showed that
64.1% of Pap tests within a probability sample from the
cytodiagnostic laboratory in the Mexico General Hospital
are of bad quality. This is a high proportion compared to
that mentioned by K O S Swho ~ ~ refers to a percentage of
inadequate Pap tests that ranged from 10 to 20% in his
previous experience. The history of false negative reports
attributed to errors in specimen collection vary from 0.04
to 9.3% of samples obtained in populational detection
and between 6 and 55.5% in series of uter-
ine cervical cancer cases. 15,23,25-27 In this study of external
evaluation of cytologies previously reported as negative, a
significant correlation between diagnostic error and low
quality Pap smears exists at .87.
These results show a high percentage of low quality Pap
smears in a probability sample from the early cancer de-
tection program in Mexico City, which causes delay in
diagnosis of uterine cervical cancer, and demonstrates the
need to implement permanent training programs in regis-
tration, sample collection monitoring techniques, fixation,
and transportation of cytological material in the cytodiag-
nostic laboratories. "
Likewise, clinical history for women being examined
can provide important information for making diagnostic
conclusions. Essential clinical data for the interpretation
of a Pap test are age, gynecological, personal, and patho-
logical history, and gynecological signs and symptoms,
among others. 28 In this study, 70.1% of the registration
sheets did not include the woman's age, a basic character-
istic within the context of a diagnostic interpretation, in
addition to its usefulness in the identification of the popu-
lation at risk. Information about the other above-men-
tioned characteristics was not obtained either, due to
deficient use of the registration sheet for cytology in the
early cancer detection program.
In terms of the cytological diagnosis, taking as a refer-
GYNECOLOGICAL CYTOLOGY IN MEXICO
ence the second observer, the greatest prevalence in this
study corresponds to inflammatory alterations at 67.5%.
This percentage is greater than that reported in a series of
7,112 cytologies performed 25 yr ago in the Mexico
Women's Hospital 29 which was 59.2%, and the recently
published prevalence in a series of 1,232 Pap smears in
high risk women in Calcutta, India, 30 which was 15.4%.
These results could indicate an increase in the incidence
of inflammatory cervical lesions in Mexican women, an
improvement in the diagnostic quality of the Pap test,
and/or that women who join the early cancer detection
program are motivated to participate in the program by
symptoms. The latter is suggested by a nested case-control
study in the early cancer detection program in Mexico
City,3' where 43.09% of the 12,741 women studied
showed at least one clinical gynecological symptom.
In relation to the concordance observed, the reproduci-
bility of the reading of 6,011 Pap tests is good between the
first and second observer (Kappa = 0.40). This can be
explained by the concordance obtained for negative diag-
noses and nonspecific alterations. However, a negative
concordance exists (Kappa = -.04) between the first and
third observer in the evaluation of 503 Pap smears. This
may be explained by the inability to identify specific path-
ogenic flora and/or cervical neoplasms by the first ob-
server, which corresponds to personnel in training, with-
out certification or previous experience (Table I). The
correspondence between the reading of the Pap smear by
the second and third observers is satisfactory (Kappa =
0.56), which emphasizes the importance of specialized
training, experience, and previous institutional certifica-
tion. These are necessary to discriminate between the dif-
ferent possibilities in cytology diagnosis.
In spite of its limitations, the Pap test continues to be
the best method for detection of uterine cervical cancer
during the early phases of development of the disease.
Research centered on each of the facets of a population
detection program must be a priority if our goal is im-
provement of diagnostic quality of the Pap test.
The organization of the early cancer detection program
is complex and requires important material and human
resources. Quality control in reading and collection of
cytology samples stands out among organizational char-
acteristics as a factor which reduces the prevalence of false
negative results. 32 In Mexico, as in the majority of other
Latin American countries, the early cancer detection pro-
gram has significant problems in supervision of cytodiag-
nostic laboratories, including deficient Pap test quality
control as shown in this study, as well as lack of certifica-
tion of specialized personnel. In Latin American countries
studies of health services which evaluate the costs and
benefits of early cancer detection programs at a national
level are therefore a priority. When carrying out external
quality control for cytodiagnostic laboratories it is dso
Diagnostic Cytopathology, Vol 10, No I 13
LAZCANO-PONCE ET AL.
important to disseminate results obtained in order t.o im-
plement the corresponding measures needed, 33 and in this
way initiate an organized social response inecessary for
improvement of quality of life of women in developing
countries.
Finally, in most of Latin America, fundamental princi-
pals proposed in health policies indicate the need to con-
tinue mass populational detection programs for uterine
cervical cancer because:j4 1j uterine cervical cancer in
these countries is an important public health problem; 2)
adequate treatment for the disease exists; 3) acceptability
and accessibility of the treatment site is possible; 4) a
recognized latency period or early symptoms of the dis-
case exists; 5 ) the utility of the Pap smear as a test for early
detection has been shown; 6) knowledge of the natural
history of the disease is expanding; 7) a defined policy for
identification of the population at risk is possible; 8) cost-
benefit obtained in health services through early diagnosis
as compared to late detection is substantial; and 9) popula-
tion detection programs should be a continuous effort
instead of a sporadic one.
Faced with this reality, implementation of quality con-
trol for the diagnosis and collection of Pap tests, as well
as the improvement of the organizational design of the
early cancer detection program are needed to realize a
decrease in the morbidity and mortality caused by uterine
cervical cancer in developing countries.
Acknowledgrnent
We would like to thank the second year students specializ-
ing in pathology and the technicians from the 1989 cyto-
technology course in the Cytopathology Laboratory, Pa-
thology Unit, National University of Mexico, Mexico
General Hospital, for their participation in this study. We
would also like to thank the cytotechnicians Lourdes
Murguia Riechers and Clemencia Sosa Cazarin, who par-
ticipated in the reading of 503 Pap tests.
References
1 Compendio de informacih epidcmiolhgica de cancer. 1982- 1988.
Direccion General de Epidemiologia. Mixico. SSA. I99 1.
2. Bosch FX, MuRoz N. Cancer del cue110 utcrino: Evudencia epidemi-
ol6gica actual y nuevas hip6tesis sobre los factores de ricsgo. Revi-
sions Salud Publica in Spanish 1989;1:83-110.
3. Page H, Asire A. Cancer rates and risks. 3rd cd.Washington, D.C.:
Government Printing Ofice, 1985 (USDHHS Publication No.
[NIH] 85-6Y5).
4. Peters RK, Thomas D, Hagan DO, Mack I’M, Henderson BE. Risk
factors for invasive cervical cancer among Latinas and non Latinas
in Los Angeles County. J Natl Cancer Inst 1986;77:941-946.
5. Martin J, Suirez L. Cancer mortality among Mexican Americans
and other Whites in Texas (1969-80). Am J Pub Health 1987;77:
851-853.
h. I.,ogros de la campaiia Nacional contra el Cancer (dtorial). Salud
I’iiblica Mcx 1976:13 (2):5-6.
7. Escandon Romero C, Bcniier Martinez MG, Navarrete Espinoza J,
Vdzquez Martinez JL, Martinez Montaiiez OG, Escobedo de la
I’ciia J. Salud Publica Mex 1992:34:607-614
14 Diognmiie Cyioputhology, Vol 10, NO I
8. Olaiz G. Analisis descriptive en cgncer cervicouterino. Repuhlica
Mexicana en 1990. Registro Nacional de Cancer. Direccion General
de Epidemiologia. 111. Congreso Nacional de Investigacih en Salud
Publica. Cuernavaca, Mor. Mbxico, 28 de enero de 1992.
9. Balfour KB. Quality assurance activities of the College of American
Pathologist. Acta Cytol 1989;33:434-438.
10. Draper GJ. Information requirements for cervical cancer screening
programmes. Screening for cancer of the uterinc cervix. IARC,
1986;76:171-183.
11. Richart R. Evaluation of the true false negative rate in cytology. Am
J Obstet Gynecol 1964;89(6):723-726.
12. National Cancer Institute Workshop: The 1988 Bethesda System for
reporting cervical vaginal cytological diagnoses. JAMA 1989;262:
93 1-934.
13. Kelsey J, Thompson W, Evans A. Methods in obscrvational epide-
miology. New York: Oxford University Press, 1986.
14. Statistical package for diseases social sciences (SPSS/PC) version
4.0.
15. Morcll N, Taylor JR, Snyder RN, %id HK, Saltz A, Willie S.
False-negative cytology rates in patients in whom invasive cervical
cancer subsequently developed. Obstet Gynecol 1982;60:41-45.
16. Love R, Camilli A. The value of screening. Cancer 1981;48(2):489-
494.
17. Canadian Task Force Report: Cervical cancer screening pro-
grammes. Can Med Assoc J . 1976;114:lOO3-l0~3.
18. Alonso dr Ruiz P, M de Larios N, Serrano Espinoza MA, Loren-
zana R. Citologia ginecologica. Monografia No. 5. Mexico City:
Sociedad Mddica del Hospital General de MCxico, 1981.
19. Sedlis A, Walters AT, Balin W, Hontz A, Sciuto L. Evaluation of
two simultaneously obtained cervical cytological smears: A compar-
ison study. Acta Cytol 197418:291-296.
20. Husain OAN, Butler B, Evans DMD, Macgregor JE, Yule R. Qual-
ity control in cervical cytology. J Clin Pathol 1976;27:935-946.
21. Gay JD, Donaldson LD, Goellner JR. False-negative results in cer-
vical cytologic studies. Acta Cytol 1985;29:1043-1046.
22. Van der Graaf Y, Vooijs GP, Gaillard HLJ, Daisy M. Screening
errors in cervical cytology. Acta Cytol 1987;31:434-438.
23. ‘Tuncer M, Graham R,Graham J. Diagnostic efficiency in invasive
cervical cancer. NY J Med 1967;67:2317.
24, Koss L. The Papanicolaou test for ccrvical cancer detection. A
triumph and a tragedy. JAMA 1989;2,61(5):737-744.
25. Richard RM, Vaillant HM. Influence of cell collection techniques
upon cytological diagnosis. Cancer 1965;65:1474-1478.
26. Figge DC, Rennington JL, Schcid Al. Cervical cancer after initial
negative and atypical vaginal cytology. Am J Obstet Gynscol 1970
108:422.
27. Herkowitz RS, Ehrmann RL, Lavizzo-Mourey R, et al. lnvasive
cervical carcinoma in young women. Gynecol Oncol 1979;8:311.
28. Koss L. Cytologic evaluation of the uterine cervix factors influencing
its accuracy. Pathologist 1982:401-407.
29. Garcia G. La deteccion de cancer del cuello uterino en 75,855
mujeres. Gaceta Medica de Mexico 1970;100(2):154-166.
30. Roy A, Dutta K, Majumdar J, Basu S, Chowdhury JR. Cervical
cytology screening in Calcutta and adjoining areas with special refer-
ence to carcinoma of the uterine cervix. Indian J Public Health,
1990;34(2):98- 106.
31. Lazcano-Ponce E, Rojas R, Ldpez Acuiia M, Ldpez Carrillo L,
Hernandez M. Factorrs de riesgo reproductivo y cancer ccrvicouter-
ino en la ciudad dc Mixico. Salud Publica Mex 1993;35:65-73
32. Fetherston W. False-negative cytology in invasive cancer of the
cervix. Clin Obstct Gynecol 1983;26(4):929-937
33. Koss L. Cytology. Accuracy of diagnosis. Cancer 1989;64(1):249-
252.
34. Wilson JMG, Jugner G. Principles and practice of screening for
disease. Public Health No. 34. Geneva: Word Health Organization,
1968.