Universiti Teknologi Malaysia Regional Conference in Engineering

Education 2016
RHEd/APCETE/REES 2016

A Methodology on Assessing Program Educational

Objectives for Undergraduate Engineering
Programs 
A. Y. Chua1, A. W. C. Oreta2, R. R. Seva3
1
Mechanical Engineering Department, De La Salle University, Manila, Philippines
2
Civil Engineering Department, De La Salle University, Manila, Philippines
3
Industrial Engineering Department, De La Salle University, Manila, Philippines
(alvin.chua@dlsu.edu.ph, andres.oreta@dlsu.edu.ph, rosemary.seva@dlsu.edu.ph

*
Corresponding author: alvin.chua@dlsu.edu.ph Tel: 632-5244611 loc 503

Abstract: In implementing Outcomes-Based Education in engineering programs, it is important to identify appropriate

Student Outcomes (SOs) and Program Educational Objectives (PEOs). PEOs are expected graduate attributes three to five
years after graduation. This paper describes a methodology for developing PEOs of different engineering programs as well
as the design of an Alumni tracer survey form as assessment tool that matches the PEOs adapted. As a first step in the
methodology, there is a need to identify PEOs that are aligned to professional specializations. These objectives will be
critical in choosing the correct educational strategies to be implemented within the program. Second, the questions in the
Alumni Tracer Study (ATS) were developed to measure the attainment of the PEOs. The questions targeted the evidences to
attain the PEOs. The ATS was initially administered to some alumni to determine the effectiveness of the proposed survey
form. Results from a mock study showed that there is a need to improve some of the questions to effectively assess PEOs of
the engineering programs.

Keywords: Engineering Education, Outcomes-Based Education, Program Educational Objectives

1. INTRODUCTION process, involving program constituencies, for the

The implementation of outcome-based education (OBE)
requires the identification of Program Educational
Objectives (PEOs) that are broad statements describing
what the graduates are prepared to achieve within three to
five years after graduation. The process of checking the
attainment of PEOs is usually done at the end of each
academic year (Kanmani, 2014). Attainment of PEOs
measures the success of a program, thus, it is critical that
appropriate assessment be conducted. Moreover, results
of PEO assessment are an important part of the
continuous quality improvement (CQI) process that OBE
espouses. Accreditation bodies in engineering such as the
Accreditation Board for Engineering and Technology
(ABET) focus on assessment and evaluation process to
ensure that educational objectives and outcomes are met
(Christoforoua & Yigita, 2008). Recently, ABET does not
anymore include in its Criteria the assessment of the
attainment of the PEO’s. However, ABET
(www.abet.org) explicitly states that “there must be a
documented, systematically utilized, and effective
periodic review of these program educational objectives
that ensures they remain consistent with the institutional
mission, the program’s constituents’ needs, and these
criteria.” Hence, a methodology for PEO’s assessment
and review is significant for CQI. CQI is a conscious
effort on the part of an educational institution to improve
its program offerings in a regular and systematic way in
order to satisfy stakeholders (Tshai, Ho, Yap, & Ng,
2014).
Since PEOs are defined as achievement after graduation,
assessment is done a few years after graduation and
separate from program outcomes (Leonard & Nault,
2004). Data for PEO assessment may be gathered from
surveys of graduates that have finished after three to five
years, employers, and parents. Actual meetings with
employers can also be done to facilitate discussion and
exchange of ideas in a timely manner (Christoforoua &
Yigita, 2008). Since it will take years of waiting for
PEO’s to be assessed, Leonard and Nault (2004)

1
 Regional Conference in Engineering Education 2016
RHEd/APCETE/REES 2016

proposed an approach that allows stakeholders to provide process used in this study. The methodology is described
their perception of PEO achievement through scanning. as follows:
The barriers for achieving the PEO are discussed further
using focus group discussion. This approach, however, (a) First, the constituents (faculty, student, alumni,
does not preclude the gathering of evidences at some parents, employers) of the program meet to formulate
future time regarding the performance of graduates in the Program Educational Objectives of the Program.
relation to the PEO. The only advantage of this approach After a series of consulting with the different
is the timeliness of feedback to effect program change. constituents and stakeholders of a program, the
Surveys usually take time and do not allow for PEO’s are formulated. Among the considerations in
discussions. The answers obtained are static and are the design of the PEO’s are the institution’s mission
limited to the topics included in the survey. and vision, trends in the profession, relevance of the
program to the society. If there are different
Surveys, however, are useful in getting standardized engineering programs under a college/school, it is
responses. It had been used in gathering comments from suggested that the PEOs similar to each program be
final year students about the extent of learning such as the considered to create a common assessment tool.
one by Noor, Rejab, Sani, and Sulaiman (2008). The
study focused on the quantity of time and quality of
activities spent to achieve the program objectives (POs)
among mechanical engineering students and the
relationship of these two variables on the improvement of
skills of students. Results of the study suggest that quality
of activities has greater correlation on POs. Scott and
Yates (2002) also used surveys to determine capabilities
that are important for successful practice of engineering
independent of POs. Outcome of the study showed that
emotional intelligence is a crucial determinant for
success.

Although much effort had been given to the assessment

process it had been difficult to incorporate the results in
effecting program change and improvement due to
misalignment in data gathering (Soundarajan, 2004).
Assessment is not tied to a particular course so it is
difficult to determine the improvement that needs to be
incorporated in specific courses. This also happens when
PEO’s are evaluated after three to five years after
graduation. Information gathered cannot be trickled down
to the course level so tangible improvements are difficult
to attain. Figure 1 Methodology on PEO Design & Assessment

PEO assessment is evolving as educational institutions try
their best to continuously improve their program through
feedback from employers and graduates. The challenge is
the process of evaluating whether the PEOs had been
achieved or if they are realistic. PEOs formulated are
usually wishes of stakeholders. They are usually lofty and
may not be achievable within 3 to 5 years after
graduation. Thus, it is possible that the first round of
evaluation of PEOs show unsatisfactory performance.
The authors are not aware of any study done on the
process of reassessing PEOs to incorporate the results of
initial evaluation. Such a study is important because it
either paves the way for more relevant PEOs or better
pedagogical strategies to achieve the PEOs initially
formulated. This study aims to illustrate a method of PEO
reassessment.
2. DESIGN AND ASSESSMENT OF PEO’S
This section describes a methodology of designing,
assessing and evaluating the PEOs for continuous quality
improvement. Figure 1 shows the PEO assessment
(b) The next step will be the identification of
evidences/indicators to detail the important aspect of
the PEO that need to be achieved. The indicators are
listed as a guide in the formulation of questions for a
survey that will be used in the assessment. The
indicators must be measurable so that the
achievement of the PEOs can be verified.
(c) There are various methods of assessment of PEOs
like the exit interviews, employer survey, advisory
survey and the alumni graduate survey. In this study,
an Alumni Tracer Study (ATS) is explored to
determine its effectiveness in getting information on
the achievement of the PEOs. There are challenges in
an ATS such as the difficulty in reaching out to
respondents. The contact details of the alumni
usually change after five years. It is therefore
suggested that an on-line survey be done to reach as
many graduates as possible. The survey could be
deployed using Facebook, Yahoo groups, e-mail or
any other social networking facilities. The creation of
an Alumni society under each program is also
recommended to access the graduates and help in the
assessment process.

2
 Regional Conference in Engineering Education 2016
RHEd/APCETE/REES 2016

(d) Finally, the result of the survey will be evaluated to
gain understanding on the achievement of the PEO’s
and their relevance to the program. The next steps to
be done to improve the program and achieve the
objectives are then recommended. In the assessment
of the PEO’s, the time frame and target performance
for the graduates for each PEO must be assessed. The
time frame and the target need not be the same for
each PEO as shown in an example in the PEOs for
University of North
Texas(https://engineering.unt.edu/technology/about-
us/accreditation).
alumni in civil engineering, mechanical engineering and
industrial engineering. After two weeks, a total of 44
responses were gathered.
The online survey has a total of nine slides since the free
version allows a maximum of ten questions only. The
first slide (Figure 2) collects basic information about the
year of graduation, if the alumnus is a licensed engineer,
job information and post-graduate training and degrees of
the alumnus. The data on highest degree earned and
certificate programs completed are assessment on PEO2
on Lifelong Learning.

The methodology presented in Figure 1 was implemented
at the De La Salle University Gokongwei College of
Engineering (DLSU-GCOE) After several meetings with
the stakeholders of the programs, the college of
engineering at De La Salle University was able to
develop a list of PEOs. For DLSU-GCOE, it was decided
that three PEO’s will be formulated for each program
focusing on the three aspects:
1) Leadership in the Profession
2) Lifelong Learning
3) Social Responsibility
Table 1 shows the PEOs with the listed indicators for the
Civil Engineering Program of DLSU as example.
The data on the first and present job or position in a
company addresses PEO1 on Leadership in the
Profession. The information aims to determine the
advancement of the alumnus from his first job to the
present. Table 2 shows that the data provides useful
information about their first and present jobs and the type
of work in the company where they are employed. From
the sample data, it can be observed that those who
graduated for more than ten years have advanced from an
entry-level position such as estimator, cadet engineer or
junior structural engineer to managerial positions like
president or manager of a company.

Table 1. PEO for the Civil Engineering Program of

DLSU

Figure 2. Personal Data

The next section describes the assessment and evaluation
process done to the PEOs developed using the Alumni
Tracer Study as an initial assessment tool. An evaluation Table 2. Sample Responses on the First and Present
of the results was done to verify the achievement or non- Position in the Company
achievement of the PEOs together with recommendations
to improve the PEO assessment process.
Degree Year First Present Type of Work
Grad Position Position
uated
3. PILOT TESTING OF THE ALUMNI TRACER BSME 1994 Assistant Manager Sales (Aircon)
STUDY Manager
BSME 2007 Cadet Production Manufacturing
Engineer Planner
To initially assess the achievement of the PEOs, an online BSCE 2005 Structural President Construction
survey with Monkey Survey as the platform was Engineer
designed. This online survey was aimed to test the BSCE 2009 Structural Plant and Healthcare
questions designed to assess the achievement of the Design Facility
Engineer Manager
PEOs. The online survey was implemented to determine BSIE 2012 Senior Assistant Logistics
what kind of data will be gathered and how the alumni Assistant Manager
will respond to the survey questions. The link to the BSIE 2011 Manageme Roll-out Production
survey was posted in social media and emails to some nt Trainee Manager Management

3
 Regional Conference in Engineering Education 2016
RHEd/APCETE/REES 2016

guest speaker in conventions or conferences related to

his/her field of expertise. Q4 also addresses PEO3 on
Question 2 (Q2) aims to determine whether fresh Service to the Profession. In the Philippines, Continuing
graduates from De La Salle University are attractive to Professional Development (CPD) seminars are conducted
the industry. Figure 3 shows the response on what the to update and upgrade the professional engineers and
alumni did immediately after graduation. In the being a lecturer in these seminars is a form of service to
Philippines, a professional licensure examination is the professional organization.
required for civil engineering, mechanical engineering,
chemical engineering and electronics & communication
engineering. It is expected that majority of the graduates
of these courses reviewed for the board exams. However,
graduates of industrial engineering and manufacturing
engineering have no licensure examinations. Hence, these
graduates can work immediately as engineers. Being an
entrepreneur after graduation is another path that some
alumni pursue especially those who have a family
business. Hence, it was included as an option in Q2.

Figure 4. Q3 Summary of Responses

Figure 3. Q2 Summary of Responses

Q3 in Figure 4 addresses the PEO1 as demonstrated in the

output or accomplishments as an engineer. The indicators
under Q3 are the positive responses on publishing a
technical paper or article, presenting a technical paper in
conferences or seminars, signing engineering designs,
proposing projects and producing engineering designs or
products.

Q4 in Figure 5 and Q5 in Figure 6 also addresses PEO1. Figure 5. Q4 Summary of Responses

These two questions aim to quantify the contribution of
the alumni thru the conduct of seminars and as being a

4
 Regional Conference in Engineering Education 2016
RHEd/APCETE/REES 2016

Figure 8. Q7 Responses on Recognition

Figure 6. Q5 Summary of Responses

Q6 in Figure 7 also addresses PEO1 by determining if the

graduate has been recognized in his/her field of expertise.
Being given a recognition by a professional organization
in your field indicates that you’re a leader and authority
in our profession.

Figure 7. Q6 Responses on Recognition

Q7 addresses PEO3 on Service to the Profession through

active participation in the professional organizations. Q7
asks the respondents to indicate the professional
organizations where they are active members. In the
Philippines, each engineering profession has an
accredited professional organization like PSME for
Mechanical Engineering, PICE for Civil Engineering and
Figure 9. Q8 Summary of responses
PICHE for Chemical Engineering. Some alumni also join
international professional organizations. Sample
responses in Figure 8 show that some graduates join more
than one professional organization but majority are not
actively involved in professional organizations.

Q8 in Figure 9 and Q9 in Figure 10 aims to assess PEO3

on the engagement of the graduates in community service
such as volunteers in medical missions, disaster related
activities, government services and membership in socio-
civic organizations.

5
 Regional Conference in Engineering Education 2016
RHEd/APCETE/REES 2016
Figure 10. Q9 Summary of responses
Based on the results of the on-line pilot testing, PEO 1
which is about leadership was achieved by majority of the
respondents through the advancement of job position in a
company. The indicators like conduct of seminars and the
awards needs to be evaluated if truly achievable for a 3-5
year engineering graduate. Looking at PEO 2 related to
life-long learning, it showed that only a small portion of
the respondents was able to take up graduate school or
other certification programs. The survey needs to place
more question on the seminars/trainings undergone by the
alumni after graduation as this also indicates life-long
learning. Finally, PEO 3 about social responsibility was
achieved by only a few graduates. Only about 33% of
respondents were able to join socio-civic activities and
only 6% are members of socio-civic organizations. This
may indicate the priorities of newly graduated engineers
which will be to serve his family and improve on his
career. There is a need to review PEO 3 if this could be
achieved by an alumnus three to five year after
graduation as this objective may only be achievable after
several years of working in the field.
3. CONCLUSIONS
The paper was able to represent a methodology of
assessing PEOs using an Alumni Tracer Study (ATS). It
shows that the ATS using an online survey can be an
effective tool to gather information that can be used to
assess the attainment of PEOs especially when the right
indicators/evidences are identified. The initial results
showed that there is still a need to change, add or refine
the questions in the different PEO’s. Although the pilot
test was simply conducted to check the type of responses
that can be gathered using the survey questions, the
number of responses for each PEO shows an initial
observation that some PEO’s may not be achievable in
three to five years. A practical time frame must be set to
check the attainment of the PEO’s. Moreover, the target
performance for each PEO has to be evaluated after
conducting a survey with a significant number of
respondents. The paper also demonstrated the power of
technology in getting responses for the survey through
different modalities.
In the future, a full on-line graduate and employer survey
will be implemented to further validate the achievement
of PEOs considering the time frame and targets per PEO.
With better assessment tools, more reliable data can be
produced to assess and evaluate the achievement of the
PEOs in the engineering program.
ACKNOWLEDGMENT
The authors would like to acknowledge the help of all the
alumni in Civil Engineering, Industrial Engineering and
Mechanical Engineering who answered the on-line
survey.
REFERENCES
[1] Christoforoua, A. P., & Yigita, A. S. (2008).
Improving teaching and learning in engineering
education through a continuous assessment process.
European Journal of Engineering Education, 33(1),
105-116.
[2] Kanmani, B. (2014). Program educational
objectives: A graphical representation. Paper
presented at the 2014 International Conference on
Interactive Collaborative Learning (ICL).
[3] Leonard, M. S., & Nault, E. W. (2004). An
Integrated Approach to Evaluation of Program
Educational Objectives and Assessment of Program
Outcomes Using ABET Criteria for Accreditation of
Engineering Programs. Paper presented at the
Proceedings of the 2004 American Society for
Engineering Education Annual Conference &
Exposition.
[4] Noor, M. M., Rejab, M. R. M., Sani, M. S. M., &
Sulaiman, A. S. (2008). Assessment of the
Mechanical Engineering Programs by Exit Surveys
at University Malaysia Pahang. Paper presented at
the 4th International Conference on University
Learning and Teaching Shah Alam Malaysia.
[5] Scott, G., & Yates, K. W. (2002). Using successful
graduates to improve the quality of undergraduate
engineering programmes. European Journal of
Engineering Education, 27(4), 363-378.
[6] Soundarajan, N. (2004). Program assessment and
program improvement: closing the loop. Assessment
& Evaluation in Higher Education, 29(5), 597-610.
[7] Tshai, K. Y., Ho, J. H., Yap, E. H., & Ng, H. K.
(2014). Outcome-based Education – The Assessment
of Programme Educational Objectives for an
Engineering Undergraduate Degree. Engineering
Education: a Journal of the Higher Education
Academy, 9(1), 74-85.