AJPH RESEARCH

Culturally Responsive Adolescent Pregnancy and

Sexually Transmitted Infection Prevention
Program for Middle School Students in Hawai‘i
Yasuyo Abe, PhD, Linda Toms Barker, MA, Vincent Chan, BA, and Jasmine Eucogco, MPP, MPH

Objectives. To evaluate the effectiveness of Pono Choices, a culturally responsive curriculum. (See Appendix A, available as a
adolescent pregnancy and sexually transmitted infection (STI) prevention program supplement to the online version of this article
targeting middle school youths in Hawai‘i. at http://www.ajph.org, for information on
Methods. We conducted a cluster randomized controlled trial with the school as the unit how the intervention reflects Hawaiian culture.)
of random assignment over 3 semesters between 2012 and 2013. The sample consisted of
Research has suggested that indigenous culture–
based educational strategies can be an effective
36 middle schools and 2203 students. We administered student surveys to collect baseline
alternative to mainstream strategies in addressing
outcomes, student demographic data, and outcomes at 12 months after baseline.
the needs of non-White students.6–8 A study of
Results. We found statistically significant effects for the knowledge assessment, which
high school students in Hawai‘i, 73% of
focused on basic understanding of adolescent pregnancy and STI prevention. The av- whom are non-White, linked their academic
erage percentage of correct responses was 73.6 for the treatment group and 60.4 for the outcomes to the use of culturally responsive
control group (P < .001). We did not find statistically significant effects on behavioral education.9 Another study suggested that
outcomes (initiation of sexual activity or engagement in high-risk sexual behavior) or on Hawaiian students taught with an emphasis on
other nonbehavioral outcomes (attitudes, skills, intentions). native culture had a strengthened sense of
Conclusions. Pono Choices had a statistically significant impact on knowledge of connection to ancestors and an increased desire
adolescent pregnancy and STI prevention among middle school students at 12 months to learn.10 These studies support Pono Choices’
after baseline, though it did not lead to detectable changes in behavioral outcomes focus on Hawaiian culture as a promising
within the 1-year observation period. These results call for an exploration of longer-term approach to engaging youths in the state.
outcomes to assess effects on knowledge retention and behavioral changes. (Am J Public In addition to being culturally responsive,
Pono Choices was designed as a middle school
Health. 2016;106:S110–S116. doi:10.2105/AJPH.2016.303395)
intervention. Research has indicated that sexual
health interventions should start in early ado-
See editorials, p. S5–S31. lescence, particularly for youths from disadvan-
taged or dysfunctional families.11 Research has

H awaiian high school students had the
lowest rate of condom use in the
United States in 2011 (43.9%, compared
with 60.2% nationwide)1 and the 10th
highest adolescent pregnancy rate in 2010
(6.5%, compared with 5.7% nationwide).2
Furthermore, in this racially diverse state,3
female minorities are at disproportionately
high risk for adolescent births (29 per 1000
among Asian/Pacific Islanders aged 15 to 19
years, compared with 22 per 1000 among
their White counterparts in 2012).4 The
state’s chlamydia rate has consistently been
higher than the national average over the
previous decade (2001–2010) and only re-
cently improved.5 Responding to the need
for effective sexual health education cur-
ricula in the state, the University of Hawai9i
at Manoa, with Hawai‘i-based partners,
developed the Pono Choices curriculum, also suggested that the best way to prevent the
which seeks to provide adolescents with the high-risk behaviors that often precede sexual
medically accurate knowledge, attitudes, activity is to work with young adolescents.11,12
The theory of change underpinning this
and skills necessary to reduce the risk of
study predicts that youths exposed to Pono
sexually transmitted infection (STI) and
Choices will improve their knowledge,
unintended pregnancy. Pono Choices was
attitudes, skills, and intentions toward pregnancy
designed exclusively for youths in Hawai‘i, and STI prevention, which will then lead to
emphasizing the values and practices of the host delayed initiation of sex and fewer risky be-
culture to promote pono (right) choices. For haviors in later years. To assess the impact of
example, it uses native language and indigenous Pono Choices, we conducted a cluster ran-
stories and emphasizes the active involvement domized controlled trial. Because the ultimate
of family in discussing key components of the goal of the intervention was to affect youths’
ABOUT THE AUTHORS
Yasuyo Abe, Linda Toms Barker, and Vincent Chan are with IMPAQ International, LLC, Oakland, CA. Jasmine Eucogco is
with athenahealth, Inc., Watertown, MA.
Correspondence should be sent to Yasuyo Abe, IMPAQ International, LLC, 1333 Broadway, Suite 300, Oakland, CA 94612
(e-mail: yabe@impaqint.com). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link.
This article was accepted July 16, 2016.
doi: 10.2105/AJPH.2016.303395

S110 Research Peer Reviewed Abe et al. AJPH Supplement 1, 2016, Vol 106, No. S1

sexual activity, we assessed impacts on 2 be-
havioral outcomes—initiation of sex and en-
gagement in high-risk sexual behaviors—even
though observable changes in these outcomes
were expected to be small among middle
schoolers because of their young age. We also
explored nonbehavioral outcomes on which we
expected the intervention would have more
immediate effects among young adolescents. All
our questions focused on measures of impact
among youths at 12 months after baseline,
namely, what was the impact of Pono Choices on
d initiation of sexual activity?
d engagement in high-risk sexual behaviors?
d knowledge of pregnancy and STI
prevention?
d attitudes toward healthy sexual behaviors?
d skills in managing relationships and
choices?
d intentions about future sexual activity?
METHODS
Middle schools in Hawai‘i implementing
the state sexual health standards for seventh and
eighth grade were eligible to participate in the
study. Participation required that the school
commit to random assignment to either the
intervention or the control group and identify
1 health education teacher who agreed to
cooperate in the study. Schools were recruited
in 2 rounds—in fall 2011 and in spring 2012—
from the islands of O‘ahu, Hawai‘i Maui,
Moloka‘i and Kaua‘i. Overall, 36 schools were
recruited (19 in round 1 and 17 in round 2),
including 21 regular public schools, 14 public
charter schools, and 1 private school.
Round 1 schools recruited during fall 2011
were randomly assigned in November 2011,
before the start of the spring 2012 semester
when the implementation period began.
Round 2 schools were recruited during spring
2012 and randomly assigned in May 2012,
before implementation in the fall 2012 se-
mester. Schools were blocked in terms of island
(O‘ahu, which has a large urban area with 75%
of the state’s population, vs neighbor islands)
and type of school (regular public vs public
charter or private). In addition, timing of
planned implementation of the health cur-
riculum (early vs later in the semester) was used
to block schools in round 1. The timing of
implementation was not used in round 2,
Supplement 1, 2016, Vol 106, No. S1 AJPH
however, because we found that the planned
timing did not necessarily translate to the actual
timing of the health curriculum.
Random assignment was carried out using
computer-programmed protocols based on
randomized sequencing and allocation of
schools. A methodologist on the evaluation
team oversaw the process, witnessed by an-
other senior researcher not associated with
the project. The assignment yielded 18 in-
tervention and 18 control schools. Immedi-
ately after randomization, however, 1 public
charter intervention school and 1 regular
public control school—despite prior com-
mitment to participate—dropped out, de-
clined any further contact, and provided no
additional information. The impact analyses
were based on the data collected from stu-
dents in the remaining 34 schools.
The implementation period for round 1
schools covered 3 semesters (which we refer
to as 3 semester cohorts of students) from
spring 2012 to spring 2013. For round 2
schools, only 2 semester cohorts were
included—fall 2012 and spring 2013. The
schools offered sexual health instruction as
part of a semester-long health class, but not all
schools offered a health class every semester.
All seventh and eighth graders enrolled in
a health class taught by a participating health
education teacher during the implementation
period were eligible to participate in the
study if they had prior parental consent. If
a teacher instructed more than 2 health classes
in a semester, the first 2 classes in which the
sexual health curriculum was taught were
selected for the study. Depending on when the
school was recruited and how often it offered
a sexual health class, classes from as many as 3
semesters per school were included in the
study. The schools were assigned once, and
students were exposed to either Pono Choices
or business-as-usual sexual health instruction
according to their school’s initial assignment.
Parental consent was collected for students
enrolled in all study classes in intervention and
control schools at the start of each semester.
To eliminate potential influence of the as-
signment on parents’ decision, we obtained
their consent before they were informed
about the curriculum used in their child’s
class. The consent form made no reference to
the specific curriculum. When teachers
hosted a parent meeting to explain the cur-
riculum, they did so after the due date for
AJPH RESEARCH
returning the forms. In addition to parental
consent, we obtained student assent at the
time of baseline data collection.
Overall, 2203 students (1383 intervention
students, 820 control students) in the 34
participating schools were enrolled in
study-eligible health classes during the
implementation period. The smaller size of
the control group is attributable to random-
ization, which resulted in intervention
schools, on average, having larger class sizes
and offering more health classes. Of 2203
study-eligible students, we obtained parental
consent for 81%, or 1783 students (1158
intervention students, 625 control students).
We administered the student baseline
survey on the first day of the sexual health
instruction and the follow-up survey at 12
months after baseline at participating schools.
Both surveys included items on students’
sexual behaviors, knowledge, skills, attitudes,
and intentions. The baseline survey also col-
lected information on students’ demographic
backgrounds. The paper-and-pencil surveys
were collected in person in group adminis-
trations. When students could not participate
in group administration, individual surveys
were administered by mail with telephone,
e-mail, and repeat mail follow-ups. Of the
1783 students with parental consent, 1735
(1135 intervention students, 600 control stu-
dents) took the baseline survey and 1548 (997
intervention students, 551 controls) took the
follow-up survey (Figure A, available as
a supplement to the online version of this
article). Table 1 summarizes the characteristics
of the students participating in the study.
In addition to collecting student data, we
conducted classroom observations of randomly
selected Pono Choices lessons (22% of total
lessons delivered) to assess the implementation
of the intervention. These observations were
conducted by trained members of our
evaluation team. We also obtained information
on the sexual health curricula implemented in
the control group from teacher interviews
conducted by the implementation team.
Intervention and Control
Conditions
Pono Choices focused on developing the
knowledge, attitudes, and skills of middle
schoolers. Knowledge development empha-
sized information on reproductive health and
Abe et al. Peer Reviewed Research S111
AJPH RESEARCH
TABLE 1—Student Characteristics at Baseline: Hawai‘i, 2012-2013
Characteristic
Age, y
Seventh grader
Female
Bisexual or homosexual
Student uses non-English language at home
Parent does not speak English
Received mostly As and Bs for grade
b
Native Hawaiian
Asianc
Note. Test of significance takes into account clustering effects. All characteristic indicators are binary, except for age, which is measured in years.
a
Non-missing responses.
b
Includes those who reported to be Native Hawaiian, irrespective of other reported race/ethnicity.
c
Includes those who reported to be Asian, irrespective of other reported race/ethnicity.
*Significant at the .05 level.
**Significant at the .01 level.
pregnancy prevention methods and on the
causes, transmission, and prevention of STIs.
Attitude development addressed beliefs and
opinions about abstinence, STIs, and preg-
nancy. Skills development focused on nego-
tiation and refusal skills, correct condom use,
and improved self-efficacy. All content was
delivered with an emphasis on Hawaiian
cultural values and using Hawaiian language,
stories, and practices. The curriculum included
9.5 hours of content divided into 10 sequential
modules. The basic delivery model assumed
10 continuous sessions (1 module per session)
over part of a semester, but the contents could
be delivered in fewer or more sessions.
Pono Choices was designed as a scripted
curriculum to accommodate varying teacher
knowledge and skill levels. All intervention
group teachers were trained in a 2-day work-
shop to use the instructional manual and pro-
gram kit. In addition, all intervention group
teachers received ongoing support, including
observation and feedback from instructional
coaches and a 1-day refresher training each
semester. Additional information on the in-
tervention is provided in the final study report.13
The control condition was business-
as-usual sexual health instruction in regular
seventh- and eighth-grade health classes. The
state had not adopted a standard health cur-
riculum across schools but required sexual
health instruction that was age appropriate
and medically accurate and provided a list of
S112 Research Peer Reviewed Abe et al.
No. Observationsa Intervention, Mean (SD)
1455 12.27 (0.64)
1473 0.90 (0.30)
1464 0.53 (0.50)
1448 0.03 (0.16)
1484 0.26 (0.44)
1470 0.02 (0.13)
1393 0.72 (0.45)
1467 0.42 (0.49)
1467 0.71 (0.45)
approved curricula. Schools could choose
which and how much of the material they
used and who would teach it.
Outcome Measures
Table 2 describes student outcome mea-
sures constructed from the survey. A psy-
chometric evaluation was conducted to
review items on knowledge, attitudes, and
skills to ensure that the composite scales met
or exceeded the reliability (Cronbach’s alpha)
of 0.50 for internal consistency. Because
the composite scale for intention did not meet
the threshold, the 2 intention items were
analyzed as separate measures.
Impact Estimation Models
For each outcome, the program impact was
estimated as the group difference at 12 months
after baseline. The impact was estimated as
intent-to-treat effects, including all study-
eligible students with data, regardless of the
level of participation. (Appendix B, available as
a supplement to the online version of this ar-
ticle, provides details on the estimation models.)
To account for the effects of clustering
of individuals within each school and to
improve the precision of impact estimates, we
specified a mixed-level model. For the esti-
mation of binary outcomes, we applied the
maximum-likelihood method assuming
the logit distribution for an individual error
Control, Mean (SD) Difference in Group Mean
12.28 (0.60) –0.01
0.86 (0.35) 0.04
0.55 (0.50) –0.02
0.02 (0.14) 0.01
0.39 (0.49) –0.12**
0.02 (0.14) –0.01
0.71 (0.45) 0.01
0.49 (0.50) –0.08
0.74 (0.44) –0.03*
term. For the estimation of continuous out-
comes, we applied the restricted maximum-
likelihood method assuming the normal
distribution for an individual error term.
Besides study design controls (blocking and
semester cohort) and the baseline outcome, the
estimated model included the following
covariates measured at baseline: percentage of
students in the school eligible for free or
reduced-price meal; student age, gender, race
(self-identifying as Asian and as Native Ha-
waiian), and academic performance; and an
indicator of whether the student did not speak
English at home. Covariates were used to
improve impact estimates and were not
assessed for their effects. We chose the cova-
riates from a list of available indicators po-
tentially correlated with the behavioral
outcomes if they empirically explained the
variation in the outcome or if the baseline
equivalence was not established. We applied
the listwise deletion method to handle missing
outcome data and used the dummy variable
adjustment method for missing covariate data
(missing data were replaced by a constant value
and flagged by a dummy indicator).
RESULTS
The implementation data indicate that
Pono Choices was delivered with high ad-
herence to the intended intervention model.
AJPH Supplement 1, 2016, Vol 106, No. S1
 AJPH RESEARCH

TABLE 2—Behavioral and Nonbehavioral Outcome Measures

Outcome Description of Outcome

Initiation of sexual activity
Engagement in high-risk sexual behaviors
Knowledge of pregnancy and STI prevention
Attitudes toward healthy sexual behaviors
Skills in managing relationships and choices
Intention to have sex
Intention to use condom during intercourse
Behavioral outcome
Responded “yes” to the question “Have you ever had sexual intercourse?”
Responded “yes” to both of the following questions:
d In the past 3 months have you had sexual intercourse without a condom?
d In the past 3 months have you had sexual intercourse without an effective method of birth control—including condoms, birth
control pills, the shot (Depo Provera), the patch, the ring (NuvaRing), IUD (Mirena or Paragard), implant (Implanon)?
Nonbehavioral outcomes
Weighted average of the following:
d Percentage of 9 multiple-choice questions that respondents answered correctly regarding pregnancy and STI prevention topics
(excluding knowledge of condom use): (1) effective ways to prevent pregnancy, (2) behaviors associated with high risk of HIV,
(3) definition of viral STIs, (4) signs of STI in another person, (5) process of reproduction, (6) prevention methods for STIs,
(7) refusal skills, (8) transmission mechanism of HIV, and (9) types of birth control.
d Percentage of 9 multiple-choice questions that respondents answered correctly regarding knowledge of condom use: (1)
condom usage, (2) wearing 2 condoms, (3) how far unrolled, (4) no space at tip of condom, (5) unrolling wrong way, (6)
expiration date, (7) removal of condom, (8) lubricants, and (9) wallet as storage.
Average rating of 10 items on 4-point scale (ranging from 1 = not important to 4 = very important) on the importance of healthy
sexual behaviors.
d The 10 items are (1) not having sex until ready, (2) avoiding risky sexual behavior, (3) preventing unwanted pregnancy as an
adolescent, (4) knowing what birth control methods can be used to prevent an unwanted pregnancy, (5) taking personal
responsibility for sexual health, (6) communicating openly about sexual intent with a partner, (7) knowing multiple ways to
prevent STIs and unwanted pregnancy, (8) using condoms to prevent STIs and unwanted pregnancy, (9) using alternative ways
to show affection other than having sex, and (10) understanding changes that happen during puberty.
d The reliability score was a = .86 at baseline and a = .88 at 12 months after baseline.
Average rating of 5 items on a 4-point scale (ranging from 1 = very difficult to 4 = very easy) on difficulty of skills relate to
managing relationship and choices.
d The 5 items are (1) effectively communicating with my partner about my intentions and wishes about sexual activity, (2)
refusing unwanted and/or unprotected sex, (3) identifying if a relationship is healthy or unhealthy, (4) following the steps for
correct condom use, and (5) getting/buying condoms or other birth control.
d The reliability score was a = .66 at baseline and a = .66 at 12 months after baseline.
Response on 4-point scale (ranging from 1 = definite intention to 4 = no intention) to the question “Do you intend to have sexual
intercourse in the next 12 months?”
Response on 4-point scale (ranging from 1 = no intention to 4 = definite intention) to the question “If you were to have sexual
intercourse in the next 12 mo, would you intend to use (or have your partner use) a condom?”

Note. IUD = intrauterine device; STI = sexually transmitted infection.

Teachers completed 98% of planned activities
across all 3 semesters. From attendance
records, we estimated that 94% of students
completed at least 75% of the curriculum.
The classroom observation data that our
evaluation team members collected also
suggest that the quality of the activities
completed was high: The average observer
ratings for quality of delivery and student
engagement were 4.3 and 4.6, re-
spectively, on a scale ranging from 1 to 5,
with 5 being excellent (Table A, available
as a supplement to the online version of
this article).
The contents of the sexual health cur-
ricula in control schools overlapped with
Pono Choices to a varying degree. Each of
4 key topics covered by Pono Choices—
reproductive anatomy, pregnancy pre-
vention, STI prevention, and refusal skills—
was included in the majority (9–12 of 17)
of control group sexual health programs.
However, the fifth key component, the
condom demonstration, was included at only
5 of the 17 control schools. The imple-
mentation of control group curricula was less
extensive and uniform than Pono Choices.
Some control schools provided only 1 or 2
sessions of sexual health instruction; others
provided as many as 10. Only 2 control
schools fully implemented an approved sexual
health curriculum.
Baseline Equivalence of Analytic
Samples
We report impact findings based on 2
analytic samples: 1 used for behavioral out-
come analyses, consisting of students with no
missing behavioral outcome data (955 in-
tervention students, 529 control students),
and 1 used for nonbehavioral outcome

Supplement 1, 2016, Vol 106, No. S1 AJPH Abe et al. Peer Reviewed Research S113
AJPH RESEARCH

analyses, consisting of students with no
missing nonbehavioral outcome data
(819 intervention students, 381 control
students). We evaluated whether these ana-
lytic samples were equivalent across the
groups with respect to baseline outcome
measures and other school and student char-
acteristics (Table B, available as a supplement to
the online version of this article). The
student-level group difference was evaluated
using the same statistical model used to esti-
mate the impacts, except only assignment
group and blocking variables were used as
controls. Outcomes measured at baseline were
equivalent across the groups except for dif-
ferences in attitudes toward healthy sexual
behaviors and intent to use condoms in the
future (Table 3). Additional equivalence tests
(not shown) found that indicators for whether
the student spoke English at home and iden-
tified as Asian were not equivalent. The
nonequivalent indicators were included as
covariates in the impact estimation.
Impact Findings
Table 4 reports the regression-adjusted
estimated effects for behavioral and
nonbehavioral outcomes. The intervention
had no statistically significant effects on be-
havioral outcomes at 12 months after baseline.
The estimated probability of engaging in
high-risk sexual behaviors was 1.5% for the
intervention group and 2.2% for the control
group. The difference—the estimated impact
of Pono Choices—was not statistically signif-
icant (P = .42). Similarly, the estimated prob-
ability of ever having had sex at 12 months after
baseline was 9.6% for both the intervention
group and the control group (for unadjusted
outcomes, see Table C, available as a supple-
ment to the online version of this article).
The intervention had a statistically significant
impact on student knowledge at 12 months after
baseline. On average, the intervention group
students answered 73.6% of knowledge ques-
tions correctly, and the control group answered
60.4% correctly. The difference between the
groups (13.2 percentage points) was statistically
significant (P < .001). By contrast, we did not
find statistically significant impacts on measures
of attitudes, skills, or intentions.
To check the robustness of the results,
we conducted sensitivity analyses, including
the application of an alternative method
to address missing covariates, alternative
covariate specifications, and alternative
estimation methods. In addition, to check
for potential bias resulting from school at-
trition, we conducted impact analyses
based on an alternative sample excluding all
schools that were in the same assignment
blocks as the 2 dropped schools. This alternative
sample of 31 schools was balanced across the
assignment conditions regarding observable
criteria used to block the schools. The analyses
yielded consistent results, showing no statisti-
cally significant effects on any outcomes except
the knowledge measures (Table D, available
as a supplement to the online version of this
article). Sensitivity analyses conducted on the
knowledge measures showed that the result was
robust across alternative technical specifications
and estimation methods as well as with the
31-school alternative sample.
DISCUSSION
The purpose of the study was to evaluate
the effectiveness of Pono Choices, a culturally
responsive adolescent pregnancy and STI
prevention program targeting middle-school
youths in Hawai‘i. The study focused on the

TABLE 3—Summary Statistics for Baseline Outcomes: Hawai‘i, 2012-2013

Baseline Outcome Intervention, Mean (SD) Control, Mean (SD) Difference P for Difference No.
Behavioral outcomes
a
Initiation of sexual activity 0.064 (0.246) 0.071 (0.256) –0.006 .95 1262
Engagement in high-risk sexual behaviorsb 0.007 (0.084) 0.002 (0.047) 0.005 .21 1287
Nonbehavioral outcomes
Knowledge of pregnancy and STI preventionc 62.9 (18.9) 61.4 (20.6) 1.5 .54 1173
d
Attitudes toward healthy sexual behaviors 3.45 (0.49) 3.38 (0.56) 0.06 .028 1172
e
Skills in managing relationships and choices 2.79 (0.56) 2.81 (0.58) –0.02 .86 1157
Intention to have sexf 3.38 (0.81) 3.36 (0.84) 0.03 .75 1136
Intention to use condom during intercourseg 3.60 (0.76) 3.50 (0.89) 0.09 .011 1122

Note. Test of significance takes into account clustering effects. The size of the behavioral analytic sample was 1484, and the size of the nonbehavioral analytic
sample was 1200. Equivalence tests were conducted using nonmissing observations of baseline variables. Assignment group and blocking variables were
included as controls. In the impact estimation, the dummy variable adjustment method was used for missing baseline variables as covariates.
a
A binary indicator (1 = ever had sexual intercourse and 0 = otherwise).
b
A binary indicator (1 = used neither condom nor birth control when having sex and 0 = otherwise).
c
Weighted average of percentage of correct responses to 9 questions regarding pregnancy and STI prevention topics and percentage of correct responses to
9 questions regarding condom use.
d
Average rating of 10 items on 4-point scale (ranging from 1 = not important to 4 = very important) on the importance of healthy sexual behaviors.
e
Average rating of 5 items on a 4-point scale (ranging from 1 = very difficult to 4 = very easy) on difficulty of skills related to managing relationship and choices.
f
Response on 4-point scale (ranging from 1 = definite intention to 4 = no intention) to the question “Do you intend to have sexual intercourse in the next
12 months?”
g
Response on 4-point scale (ranging from 1 = no intention to 4 = definite intention) to the question “If you were to have sexual intercourse in the next
12 months, would you intend to use (or have your partner use) a condom?”

S114 Research Peer Reviewed Abe et al. AJPH Supplement 1, 2016, Vol 106, No. S1
 AJPH RESEARCH

impact at 12 months after baseline to in-
vestigate early responses among youths to the
intervention. Impact analyses found mixed
results across measures, with no impacts on
behavioral measures and most nonbehavioral
measures, but with a sizable and statistically
significant impact on student knowledge.
Given the implementation findings, the
failure to observe impacts on most outcomes
was unlikely to have been driven by poor
implementation of the intervention. Al-
though the failure to detect impacts across
behavioral outcome measures may indicate
that Pono Choices was not effective, it may
also be attributable to the short follow-up
period. Because of the relatively young
average age (12 years) of the students at
baseline, we anticipated that the follow-up
at 12 months after baseline might be too
early to observe changes in sexual behav-
iors, because most students were not yet
sexually active. Impacts on behavioral
outcomes, if any, were expected to be small.
For example, the Centers for Disease
Control and Prevention reported that in
2013, only 10% of youths in Hawai‘i had
ever had sex by eighth grade.14 In our
sample, the unadjusted percentage of
control group students having ever had sex
increased by just 2 percentage points, from
9% at baseline to 11% at 12 months after
baseline, which underscores the challenge
of detecting a program impact that is likely
to be small.
On the other hand, we found a statistically
significant impact on student knowledge, which
covered topics such as effective pregnancy
prevention methods and mechanisms of STI
transmission. Given the constraint on detecting
impacts on behavioral outcomes, observed
impacts on knowledge provide an important
indication of the intervention’s potential effects.
Previous studies have identified or suggested
that knowledge is a protective or mediating
factor leading to behavior changes.15–23
Thus our finding of a positive impact on
knowledge, which was considered to be
a proximal outcome, is not inconsistent with
finding no impact on more distal behavioral
outcomes.
Failure to detect effects on nonbehavioral
outcomes other than knowledge may be
partly attributable to measurement limita-
tions. These nonbehavioral measures—the
average of 4-point scale ratings—may not
have been adequate to capture the group
difference. The unadjusted control group
means of the average ratings on attitudes, skills,
and intentions at baseline ranged from 3.0 to
3.7, with 4.0 being the highest, leaving little
room for the rating to improve.
The study has several important limitations,
including that its findings cannot be general-
ized because the schools were purposively
selected on the basis of their willingness to
participate and administrative support for the
study. Another limitation arises from the de-
sign. As with most studies in school settings,
randomization for this study was done before
we could identify study-eligible students. To
make individual-level inferences, we assumed
that enrollment in study schools was in-
dependent of assignment and that parents
consented without knowledge of assignment
status. These assumptions, however, are not
verifiable. The study also had attrition at both
school and student levels, which could lead to
bias in the impact estimates. Although sensitivity

TABLE 4—Estimated Impacts of Pono Choices on Behavioral and Nonbehavioral Measures at 12 Months After Baseline: Hawai‘i, 2013-2014

Outcome Intervention Group, Mean Control Group, Mean Difference (SE) P for Difference
Behavioral outcomes (n = 1484)
Engagement in high-risk sexual behaviorsa 0.015 0.022 –0.007 (0.0091) .42
b
Initiation of sexual activity 0.096 0.096 0.000 (0.0167) .99
Nonbehavioral outcomes (n = 1200)
Knowledge of pregnancy and STI preventionc 73.6 60.4 13.2 (2.07) < .001
Attitudes toward healthy sexual behaviorsd 3.61 3.61 0.00 (0.037) .99
e
Skills in managing relationships and choices 3.00 2.97 0.03 (0.039) .42
f
Intention to have sex 3.30 3.25 0.05 (0.091) .56
Intention to use condom during intercourseg 3.71 3.73 –0.02 (0.043) .56

Note. STI = sexually transmitted infection. The knowledge measure was statistically significant after adjusting for multiple comparisons. The measure remained
statistically significant using either the Benjamini and Hochberg method or the Bonferroni method to adjust for 5 comparisons under the nonbehavioral
outcome domain. The measure also remained statistically significant using either method to adjust for 7 comparisons across behavioral and nonbehavioral
outcome domains.
a
A binary indicator (1 = ever had sexual intercourse and 0 = otherwise).
b
A binary indicator (1 = used neither condom nor birth control when having sex and 0 = otherwise).
c
Weighted average of percentage of correct responses to 9 questions regarding pregnancy and STI prevention topics and percentage of correct responses to
9 questions regarding condom use.
d
Average rating of 10 items on 4-point scale (ranging from 1 = not important to 4 = very important) on the importance of healthy sexual behaviors.
e
Average rating of 5 items on a 4-point scale (ranging from 1 = very difficult to 4 = very easy) on difficulty of skills relate to managing relationship and
choices.
f
Response on 4-point scale (ranging from 1 = definite intention to 4 = no intention) to the question “Do you intend to have sexual intercourse in the next
12 months?”
g
Response on 4-point scale (ranging from 1 = no intention to 4 = definite intention) to the question “If you were to have sexual intercourse in the next
12 months, would you intend to use (or have your partner use) a condom?”

Supplement 1, 2016, Vol 106, No. S1 AJPH Abe et al. Peer Reviewed Research S115
AJPH RESEARCH
analyses based on the alternative sample ex-
cluding the assignment blocks with the dropped
schools confirmed that findings were consistent
and we controlled for observable characteristics
that differed at baseline, it is still possible that
potential bias resulting from overall and differ-
ential attrition across the conditions may remain.
To summarize, Pono Choices, a culturally
responsive intervention targeting early ado-
lescents, did not lead to detectable behavioral
changes at 12 months after baseline. The
program, however, did have a statistically
significant impact on knowledge regarding
adolescent pregnancy and STI prevention.
Because of the curriculum’s cultural focus, we
conducted additional exploratory analyses for
the subsample of Native Hawaiian students:
Additional subgroup analyses suggested that
the effects for the Native Hawaiian subsample
tended to be larger, albeit not statistically
significant except for the knowledge measure
(Table E, available as a supplement to the online
version of this article). Further analysis also
found that the difference in knowledge results
between Native Hawaiian and other students
was not statistically significant. One implication
worth exploring further is the potential for
culturally responsive interventions to bridge the
gap in student outcomes between the targeted
groups and the broader population.
These early results indicate that further
research could reveal whether and how
a culturally responsive curriculum might ef-
fectively support a population known to be at
higher risk for adolescent pregnancy and STIs.
Further research could also show whether
knowledge gained with Pono Choices was
retained and led to eventual differences in
behaviors. The results also underscored the
challenge of studying an intervention targeting
early adolescents. Although early intervention
was hypothesized to have an impact in later
years, a multiyear study with a long follow-up
period is not always feasible. Future research
might combine proximal outcome estimation
(e.g., effects on knowledge) on a younger
sample with behavioral analyses of an older
sample (e.g., effects of knowledge on behav-
ioral outcomes) to approximate the impact on
ultimate behavioral outcomes.
CONTRIBUTORS
Y. Abe and L. Toms Barker drafted, reviewed, and revised
the submitted article. Y. Abe served as the lead meth-
odologist for the study, led the outcome evaluation,
developed the impact analysis plan, led outcome data
S116 Research Peer Reviewed Abe et al.
processing and analyses, and oversaw the analysis team.
L. Toms Barker served as the overall project director of
the study and as the lead researcher for the implementation
study, developed the implementation analysis plan, led
the analyses, and provided the oversight of field data
collection activities. V. Chan contributed to the data
management and analyses; provided oversight for
processing baseline and follow-up data; assisted with
creating analysis indicators, the impact estimation, and
writing up the results; and contributed to the discussion
section. J. Eucogco contributed to the data analyses
and writing and to the background literature review and
the interpretation of the results and assisted with the
impact estimation and drafting of the results.
ACKNOWLEDGMENTS
This article is based on a study funded by a Teen
Pregnancy Prevention Research and Demonstration
Program (Tier 2) grant (TP2AH000017) from the Office
on Adolescent Health (OAH), US Department of Health
and Human Services (HHS). The study was conducted
by IMPAQ International, LLC, an independent evaluator
contractedbythegrantee,theUniversityofHawai9i at Manoa.
The full report from the study on which this article is based is
located on OAH’s Web site at: http://www.hhs.gov/ash/
oah/oah-initiatives/evaluation/grantee-led-evaluation/
grantees-2010-2014.html. The study was registered with
ClinicalTrials.gov (trial no. NCT02612324).
We thank the University of Hawai9i team—especially
Kelly Roberts, Tammy Tom, and Holly Manaseri—for
their support of the evaluation and their sharing of
key implementation data. We are also grateful to
Futoshi Yumoto, a former IMPAQ staff member, for
his contribution to psychometric analyses; Colleen
McLelland for her contribution in leading field
data collection and providing project assistance;
and Ward Research staff for invaluable data collection
support. Furthermore, we thank the principals,
teachers, and students who participated in the data
collection for their cooperation with
and contributions to this study.
HUMAN PARTICIPANT PROTECTION
The study was approved by the University of Hawai9i at
Manoa institutional review board. Informed consent was
obtained from students’ parents.
REFERENCES
1. Eaton DK, Kann L, Kinchen S, et al. Youth Risk
Behavior Surveillance—United States, 2011. MMWR
Surveill Summ. 2012;61(SS04):1–162.
2. Kost K, Henshaw S. US Teenage Pregnancies, Births and
Abortions, 2010: National and State Trends by Age, Race and
Ethnicity. New York, NY: Guttmacher Institute; 2014.
Available at: http://www.guttmacher.org/pubs/
USTPtrends10.pdf. Accessed July 10, 2015.
3. US Census Bureau. 2010 Census. Available at: http://
factfinder.census.gov. Accessed October 24, 2015.
4. Ventura SJ, Hamilton BE, Mathews TJ. National and
state patterns of teen births in the United States, 1940–
2013. Natl Vital Stat Rep. 2014;63(4):1–34.
5. Centers for Disease Control and Prevention. 2011
sexually transmitted disease surveillance, chlamydia pro-
files. Available at: http://www.cdc.gov/std/
chlamydia2011. Accessed July 27, 2016.
6. Castagno AE, Brayboy BMJ. Culturally responsive
schooling for indigenous youth: a review of the literature.
Rev Educ Res. 2008;78(4):941–993.
7. Cornelius C. Iroquois Corn in a Culture-Based Curriculum.
Albany: State University of New York Press; 1998.
8. Kana9iaupuni S. A Brief Overview of Culture-Based
Education. Honolulu, HI: Kamehameha Schools Research
& Evaluation Division; 2007.
9. Kana9iaupuni S, Ledward B, Jensen U. Culture-Based Ed-
ucation and Its Relationship to Student Outcomes. Honolulu, HI:
Kamehameha Schools Research & Evaluation Division; 2007.
10. Kaiwi MK, Kahumoku W. Makawalu: Standards,
Curriculum, and Assessment for Literature Through an In-
digenous Perspective. Vol. 3. Honolulu, HI: Kamehameha
Schools; 2006.
11. Moore KA, Sugland BW. Next Steps and Best Bets:
Approaches to Preventing Adolescent Childbearing. New York,
NY: Child Trends; 2001. Available at: http://eric.ed.
gov/?id=ED414555. Accessed August 27, 2015.
12. Hawkins JD, Catalano RF, Kosterman R, Abbott R,
Hill KG. Preventing adolescent health-risk behaviors
by strengthening protection during childhood. Arch
Pediatr Adolesc Med. 1999;153(3):226–234.
13. Abe Y, Toms Barker L, Chan V, Eucogco J. The Final
Impact Evaluation Report: Evaluation of the Pono Choices
Program—A Culturally-Responsive Teen Pregnancy and
Sexually Transmitted Infection Prevention Program for Middle
School Youth in Hawai9i. IMPAQ International. Available
at: http://www.hhs.gov/ash/oah/oah-initiatives/
evaluation/grantee-led-evaluation/grantees-2010-2014.
html. Accessed September 30, 2016.
14. Centers for Disease Control and Prevention. Youth
online. Available at: https://nccd.cdc.gov/youthonline/
App/Default.aspx. Accessed August 27, 2015.
15. Kirby D, Lepore G. Sexual Risk and Protective Factors:
Factors Affecting Teen Sexual Behavior, Pregnancy, Childbearing,
and Sexually Transmitted Disease: Which Are Important? Which
Can You Change? Washington, DC: National Campaign
to Prevent Teen and Unplanned Pregnancy; 2007.
16. Kirby D. Emerging Answers 2007: Research Findings on
Programs to Reduce Teen Pregnancy and Sexually Transmitted
Diseases. Washington, DC: National Campaign to Pre-
vent Teen and Unplanned Pregnancy; 2007.
17. Kirby D, Short L, Collin J, et al. School-based pro-
grams to reduce sexual risk behaviors: a review of ef-
fectiveness. Public Health Rep. 1994;109(3):339–360.
18. Terzian M, Andrews KM, Moore KA. Preventing
Multiple Risky Behaviors: An Updated Framework for Policy
and Practice. Washington, DC: Child Trends; 2011.
19. Fisher JD, Fisher WA, Williams SS, Malloy TE.
Empirical tests of an information-motivation-behavioral skills
model of AIDS-preventive behavior with gay men and het-
erosexual university students. Health Psychol. 1994;13(3):238–250.
20. Espada JP, Morales A, Guillén-Riquelme A, Ballester
R, Orgilés M. Predicting condom use in adolescents:
a test of three socio-cognitive models using a structural
equation modeling approach. BMC Public Health. 2016;16:35.
21. Bazargan M, Stein JA, Bazargan-Hejazi S, Hindman
DW. Using the information-motivation behavioral
model to predict sexual behavior among underserved
minority youth. J Sch Health. 2010;80(6):287–295.
22. Mongkuo MY, Lucas N, Taylor A. The effects of
motivation and knowledge on HIV prevention behav-
ior among historically Black college students: an appli-
cation of the information-motivation-behavioral skills
model. Greener J Med Sci. 2012;2(2):38–44.
23. Tyler J, Lichtenstein C. Risk, protective, AOD
knowledge, attitude, and AOD behavior: factors associ-
ated with characteristics of high-risk youth. Eval Program
Plann. 1997;20(1):27–45.
AJPH Supplement 1, 2016, Vol 106, No. S1