Running head: CITIZEN DEVELOPMENT PROGRAMME 1

Component B: Group Project – Review of a Citizen Development Programme – We C.A.R.E.

EDID 6504 – Programme Evaluation and Course Assessment Methods

Randall James, Avalon Kelly, Shenelle Mohammed, Alana St. Cyr

UWI ID – 98 717 367

The University of the West Indies, Open Campus

Course Facilitator and Coordinator – Dr. Camille Dickson-Deane

CITIZEN DEVELOPMENT PROGRAMME 2

Table of Contents

Introduction ......................................................................................................................................1

Reflection .........................................................................................................................................4

New Citizens Development Programme ..........................................................................................6

Needs to Be Addressed ................................................................................................................7

Main Target Group .......................................................................................................................7

Objectives .....................................................................................................................................8

Anticipated Outcomes ................................................................................................................10

Consideration and Comparison of Evaluation Models ..................................................................11

Goal-Oriented Model of Evaluation ...........................................................................................11

Transactional Model of Evaluation ............................................................................................13

Comparison Between the Models ..............................................................................................14

Selection of Evaluation Model.......................................................................................................15

Assurance of Validation of Evaluation ..........................................................................................17

Outcome Evaluation.......................................................................................................................18

The Purpose of the Outcome Evaluation....................................................................................19

Essential Steps to Complete the Outcome Evaluation ...............................................................21

Logic Framework ...........................................................................................................................25

Conclusion .....................................................................................................................................33

References ......................................................................................................................................34
CITIZEN DEVELOPMENT PROGRAMME 3

Introduction
The Government of the Republic of Trinidad and Tobago has offered the Community

Education (Skills Training) Programme since its inception in the 1960s (Community

Development, 2018). This programme falls under the portfolio of the Community Development

Division of Social Action Programmes. According to the Government’s website of Community

Development, Culture and the Arts (2018), the Skills Training Programme was created “to

promote and encourage continuing learning opportunities for the disadvantaged”. This is done

via the honing of the natural creativity, talents and skills of householders, through the

development of marketable skills, to help citizens create goods and services from home, help

reduce household expenditure, and educate citizens of ways to make wiser financial decisions

with limited resources. Another clear objective of this programme is to help generate

employment either for self (self-employment) or in opportunities to work for existing

organisations. The programme is open to all but the main target is persons who are unskilled,

unemployed or underemployed.

The Education (Skills Training) Programme was borne out of a need from the 1960s seen

in the rise of low-income families, in poverty and unemployment, of students being unable to

complete their education due to limited finances and resources, and a subsequent increase in ‘at

risk’ areas and vulnerable citizens in different areas of the nation (Community Education (Skills

Training) Programme, 2018). As a result, the programme has always maintained focus on the

human and the family.

There have been benefits to the country both socially and economically through

achievement of its objectives over time. According to the Government’s Community

Development website (2018), there has been a reduction in the number of unskilled persons in
CITIZEN DEVELOPMENT PROGRAMME 4

affected vulnerable communities and there has been an increase in opportunities for the

disadvantaged to procure employment. Where self-employment has been created, there has been

an upsurge in local goods and services being offered to locals and visitors alike. This results in

increased revenue for the country. This lends itself to sustainability, which is also evident in the

survival of the educational programme over the decades, being offered between 60 and 75 hours

per course, depending on the level of difficulty. Participants receive a certificate of participation

on successful completion of the course, which also helps motivate individuals to continue

succeeding, as well as encourage new participants to join the programme to learn and/or develop

new skills (Zheng, Rosson, Shih & Carroll, 2015).

Reflection

Sheila Stuart, a social affairs officer employed at the sub-office for the Economic

Commission for Latin America and the Caribbean in Port of Spain, has purported that crime is a

recurring cycle of poverty (Trinidad Express, 2012). She continued that “poor children are more

likely to become poor adults”, suggesting that these youth may be vulnerable to leading lives of

crime which impact negatively on society and on the economy in the short and long-term.

Whereas the previously outlined Education (Skills Training) Programme caters to

developing marketable skills and eventual employment opportunities in the adult, there is a need

to focus on the young men of these households and communities. This is an group that is not

addressed by that programme. The effects of poverty are multiple and cyclical, and with a steady

increase in the rate of crime among adolescents in Trinidad and Tobago, there exists a great need

to target those who are vulnerable to engaging in this lifestyle for their own development and for

the subsequent benefit of the wider society. Furthermore, Professor Rhoda Reddock, vice

principal of the University of the West Indies, educator and activist, has purported that this
CITIZEN DEVELOPMENT PROGRAMME 5

growing issue of youth involved in crime is very much one of gender, as statistics have revealed

a much higher rate of young men being involved in crime than that of women (Trinidad and

Tobago Newsday, 2018). Dr. Reddock, at a panel discussion on Promotion of Gender Equity,

made claims that this problem is attributed to lack of appropriate socialisation and education

among communities of a lower socio-economic and socially depressed status. Additionally,

there is a perpetuation of inaccurate beliefs about gender roles. Dr. Reddock observed that,

based on the statistics of violent crime and male youth involvement, it would appear that there is

an attempt to imitate and encapsulate a false idealogy of masculinity.

One of the main factors contributing to this high and increasing level of crime among the

male youth is that of school drop-outs (Reddock, 2018). As a result, while girls continue in their

studies and achieve levels of success, the boys view their role as a physical one, either being

outside playing or being initiated in criminal and violent activity. Dr. Reddock noted that, while

the primary role is that of the parent(s) in raising the child to be upstanding, respectful citizens to

all, that it does take a village to raise the child (Trinidad and Tobago Newsday, 2018). She

pointed out the responsibility of the Government, as well as that of the community, in playing a

positive and constructive role in socialising and educating the male youth effectively, ensuring

their ability to be self-sufficient and to understand their true purpose and role in society, which is

not in a world of crime and violence.

Seepersad (2016) has proposed that in order to reduce the likelihood of violence in youth,

that approaches should be preventative, as opposed to reactive. This is seen as not only the more

appropriate intervention, but the more cost effective as well. Seepersad has also stated that youth

delinquency leads to criminal activity. Lall (2007) conducted a study among the male youth in
CITIZEN DEVELOPMENT PROGRAMME 6

Trinidad and Tobago due to an observation of an increase in delinquency and illegal activity

among this group, both at home and abroad. His research confirmed these observations.

These facts point to the indisputable correlation between poverty and crime. Also, it

shows that the illegal/criminal activity has a negative impact on the society and economy,

perpetuating unemployment as criminal minds are nurtured. This could, in turn, compromise the

upward movement of the country in various facets: economically, socially and morally. This

data combined shows a need to focus on the male youth, on developing a sense of purpose and

social conscience, while developing employable trade skills.

New Citizen Development Programme – We C.A.R.E.

In the same vein of the training programme previously outlined, ‘We C.A.R.E.’ is a new

citizen development programme aimed at young men, between the ages of sixteen and eighteen,

who come from low-income households and socially-depressed areas. This age range was

selected due to the research brought forth by Seepersad (2016) and Lall (2007), both of whom

referenced illegal criminal activity among the male youth. It would follow, then, that this is a

crucial social gap and issue to be addressed as vigorously and as soon as is possible.

The name and acronym, ‘We C.A.R.E.’, translated, means: We (the adults, guardians,

leaders, members of society) care about: ‘we Community’, ‘we Adolescents and children’, ‘we

Responsibility’, and ‘we Economy and Environment’. The name, which plays on local parlance

to effect familiarity, understanding, affection and camaraderie, reflects the goal of the

organisation which is to (re-) engender community spirit, unity and strength, national pride, with

a strong focus on personal and communal responsibility in caring for our young who are our

future. It intends to emphasise our stance as a nation that we do care about our young and, by
CITIZEN DEVELOPMENT PROGRAMME 7

extension, about all citizens, and in improving the social, economical culture of the country,

making it a safe place for all to enjoy and thrive.

The approach intended is a preventative one, with a recognition of the possibility for

reactive interventions with youth already engaged in crime. The organisation’s silent, yet

driving, motto revolves around the saying, ‘It takes a village to raise a child’. If raised well, this

child then grows to care for and respect others, the environment, the economy and the

community as he has been taught.

Needs To Be Addressed

The needs have been itemised as follows: social and educational (skills training) well-

being of young men.

Main Target Group

The focus of this initiative is on male youth between the ages of sixteen and eighteen who

come from underprivileged households, at-risk and socially depressed communities of low socio-

economic standing in select areas of the country. These youth may be enrolled at and attending

school or may have dropped out of school. This target group may live with parents or guardians

that might be unemployed. These children may also be exposed to various forms of violence or

abuse, and may be vulnerable/exposed to, involved in or are considering a life of crime. This age

group also represents a crucial developmental period in a young man’s life where he is

vulnerable to being shaped by his immediate environment (Andrews & Bonta, 2010).

The intention of this programme is to administer preventative measures among the male

youth of this age range before they even entertain or engage in illegal activity (Seepersad, 2016).

Registration is free and could be voluntary, however, principals and teachers are encouraged to
CITIZEN DEVELOPMENT PROGRAMME 8

make recommendations for children who may fall in the categories identified and who could

benefit from this programme. School counsellors will also make recommendations. For those

who may not be attending school, enrollment will be encouraged by advertising in the

communities and appealing to parents and guardians to register their sons and male charges.

Programme coordinators may also recruit personally by approaching the youth directly to enroll.

Objectives

The overall goals of the ‘We C.A.R.E.’ programme are listed below, , the structure

informed by Bloom’s taxonomy:

Terminal goal: By the end of the ‘We C.A.R.E.’ programme, the participants, young males

between the ages of sixteen and eighteen, will be able to demonstrate improved cognitive,

affective and psychomotor skills, social and moral behaviour with 100% accuracy, within

appropriate contexts.

Enabling Objectives

1. Apply the education/training they need for success in obtaining employment.

2. Demonstrate the ability to complete a career and technical education sequence that

matches career interests

3. Exhibit appropriate social skills in execution of task.

4. Apply marketable and employable skills within context of the trade

(mechanics, carpentry or plumbing), in specified time frame, with 100%

accuracy.
CITIZEN DEVELOPMENT PROGRAMME 9

This programme, designed to offer bi-weekly workshops, will be implemented throughout

the schools and communities in the select areas over the course of three months. Competent and

well-respected counsellors would be among other highly trained professionals in the necessary

fields conducting sessions. Among other activities, they would be aiding in guiding and mentoring

participants in identifying old, undesirable attitudes and behaviours, and developing new attitudes,

behaviours and conflict resolution skills. These professionals also guide participants in developing

goals for future ambition and employability. Other qualified professionals will conduct sessions

on development of employable trade skills in mechanics, carpentry or plumbing. The participant

will select the trade he wants to pursue. The maximum number of participants would be fifteen.

On successful completion of the programme, participants earn a certificate of achievement.

Anticipated Outcomes

The ‘We C.A.R.E.’ citizen developmental initiative was devised to address a dire need

and gap in the society, in raising our young men to lead a life away from crime. Among a

number of reasons, a main reason is to reduce the alarming increasing rate in crime carried out

by our youth. This initiative, then, intends to be aggressive; therefore, expectations are high with

respect to the successful achievement of its objectives over time (Hargreaves, 2005). As

Hargreaves (2005) has informed, educational and socio-cultural changes take years to take effect,

so with this recognition, there is the anticipation to have patience, hope and determination, while

maintaining the appropriate intensity of focus and drive in the realisation of the programme’s

goals.

Indeed, it is anticipated that the participants will benefit from the carefully developed

areas of this programme as they will be educated, and their social needs will be attended to.
CITIZEN DEVELOPMENT PROGRAMME 10

Participants’ natural talents and creative energy will also be enhanced in an effort to develop,

over time, marketable skills which will aid in future employment possibilities.

Ongoing assessment and evaluation are expectations of the programme which will help

determine value of the programme and provide a transparent, authentic account of what works

and what does not. This will offer ongoing opportunities to improve the design and delivery of

the programme as is deemed necessary from the results of the evaluations. Sustainability of the

initiative is also anticipated as this programme will be ongoing over the years, making a positive

influence on a number of boys and young men. Additionally, there will be counsellors and

educators working in partnership to follow up on programme graduates, in order to evaluate their

progress and intervene when and where required.

All in all, this programme is expected to be successful in effectively developing,

according to the objectives outlined, the social and employable skills in the participants, thereby

contributing to the positive development of the nation and anticipated reduction of crime.

Consideration of Evaluation Model for the ‘We C.A.R.E.’ Programme

In order to evaluate the effectiveness of ‘We C.A.R.E.’ and any programme, there must

be some form of evaluation which would attempt to analyse various spheres of the programme,

including, its features, the various activities and strategies employed, as well as the overall

results (Patton, 1987; 2002). This will help identify the strengths and weaknesses of the

programme and how it could be improved moving forward.

The model or models selected must also be suited to evaluate the programme or there may be a

risk in measuring irrelevant variables and procuring inaccurate outcomes which could be

misleading (Charters & Jones, 1973; Rossi, 1972). The Goal-Oriented Model has been selected
CITIZEN DEVELOPMENT PROGRAMME 11

as those constructs are more potentially suited to explore the effective evaluation of ‘We

C.A.R.E.’.

Goal-Oriented Model of Evaluation

This model focuses on the degree of the realisation of the programme’s objectives and

overall goals, as this is viewed as the purpose of any programme (Herman, 1987). According to

Herman, the goals and objectives act as the measures against which the programme has achieved

or is achieving success through its procedural goals. Before adopting this model, the evaluator

must determine that the objectives are clear and must establish a connection between the

objectives and the goals. The evaluator must consider closely how each activity or strategy

could be paired with a desired outcome or objective which must be measurable.

Justification of the Goal-Oriented Model

Regarding the evaluation of this programme, the goal-oriented evaluation model would

prove most valuable as the established goals and objectives of the programme are used as the

criteria for measuring success, and the bias of the evaluator is significantly reduced (Marsh,

1978). This lends itself to a more objective and reliable approach to evaluation. The goal-

oriented approach is suitable for a variety of contexts as its design allows for identification of

applicable and useful instruments that are unique to the programme.

The flexibility of the goal-oriented model is also advantageous in programmes such as

‘We C.A.R.E.’, which seeks to develop citizens, in its ability to assess procedural goals

throughout the development of the programme (Marsh, 1978; King, 2009). According to Shaw

and Gaynor (1982), the goal-oriented model does not only evaluate goals achieved at the end of a

programme but is capable of assessing as such during its course, as goals may be designed to

adapt to various phases of the programme. Therefore, evaluation of procedural goals also seeks
CITIZEN DEVELOPMENT PROGRAMME 12

to make sure that the programme is being executed as intended. This is referred to as differential

evaluation where the level of effectiveness differs as the programme progresses along its course

(Marsh, 1978). This process ensures, especially, that the planners of the programme are specific

in the setting of goals and objectives which results in a greater likelihood that they will be

achieved. Additionally, evaluation of procedural goals helps to inform how the programme

could be improved. This proves beneficial to the overall programme in its planning,

development and execution and more effective evaluation of the outcome (Marsh, 1978; King,

2009).

The goal-oriented approach is also considered most valuable for outcome evaluation as it

enforces the specification of programme goals which, in turn, heightens the likelihood that such

goals will be realised. The by-product is that this is facilitated while making clear the purpose

and function of the programme. The clear specification of goals encourages general agreement

among groups that may exist with reference to the purpose of the programme. Another by-

product of the goal-oriented approach allows the programme to evaluate tasks with respect to the

expectations established for the programme. This is significantly beneficial to the programme’s

evaluation as there is the occurrence of “target tropism” (Davis, 1974) which acts as a self-

fulfilling prophecy. This informs that the programme is more likely to achieve its goals based on

the expectations determined for itself, thereby making any amendments deemed necessary to

realise the goals.

Additionally, this goal-oriented approach to evaluation is suitable for personnel of the

programme who may lack the skills, experience and/or qualification to undertake an evaluation

task. This approach includes straightforward procedures which could be developed and utilised

in the functioning of the programme. Therefore, the skills needed to identify and construct goals
CITIZEN DEVELOPMENT PROGRAMME 13

in researchable, measurable terms could be developed through instruction and practise. Once the

skill has been learnt, programme staff are able to apply it to the planning and development of

activities, as well as to the overall programme evaluation.

It must be mentioned, however, that there is a notable limitation to the use of this model,

though only if the evaluator disregards the internal and informal goals of the organisation, as

well as the internal dynamics that could have an influence on achieving the set goals (Marsh,

1978). Being informed of these limitations, however, counteracts such a limitation as it gives the

evaluator the opportunity to supplement the approach with appropriate measurement activities

and be in greater control of the evaluation process.

Assurance of Validity of Evaluation of ‘We C.A.R.E.’

To ensure the programme’s success and sustainability in the long term, there must be

assurance that the evaluation methods selected reflect reliable results (Field, 2013). Murphy

and Dingwall (2003) have further supported that, “following procedures alone is not

sufficient to produce trustworthy results”, however, this in no way disregards the use and

importance of using procedures in effecting valid evaluation outcomes. Therefore, in

assessing the validity of the evaluation of the ‘We C.A.R.E.’ programme, what is being

considered is how closely that which is being measured actually relates to or matches with

objectives believed to be measured (Roberts, Priest, Traynor, 2006). As a result, the

following are considerations of achieving trustworthy, valid evaluation outcomes of the

programme (Roberts et al., 2006):

1. Ensure that the programme is evaluated by experts in the field of instructional design.

These experts must be qualified and it is recommended that they possess significant

experience in the field of instructional design and evaluation, particularly of citizen

CITIZEN DEVELOPMENT PROGRAMME 14

development programmes such as ‘We C.A.R.E.’. Evaluation by these experts provides

authenticity and highest quality to the programme and evaluation. These individuals will

have the expertise to apply the mixed methods of evaluation effectively to suit the goals

of the programme and will have the capacity to make appropriate recommendations based

on the results.

2. Make certain that the evaluation methods concur appropriately with respective objectives

and goals. This ensures relevance of the strategies employed, purpose of the programme

and successful achievement of goals.

3. Use collaborative evaluations to facilitate significant interaction with stakeholders or

participants as this will heighten its validity (Brandon, 1998).

Adhering to these recommendations, it is assured that the ‘We C.A.R.E.’ programme would

be evaluated efficiently, meeting all criteria. Therefore, relevant recommendations would be

made, leading to improvement, longevity and future investment of the programme.

Outcome Evaluation

Programmes are created with the intention of bringing about positive changes in the lives,

attitudes, behaviours and skills of specific groups of people. Such is the goal of the ‘We

C.A.R.E.’ programme. In order to assess whether the goals of the programme have been

achieved, and to what extent, an evaluation must be conducted of its original intentions and

eventual outcomes. To procure these results, an outcome evaluation will be done.

Outcomes are the result of a programme intended for the participants involved (Weiss,

1998). The outcome evaluation of the ‘We C.A.R.E.’ programme, therefore, will serve to

establish whether the objectives of a programme have been realised by the participants through
CITIZEN DEVELOPMENT PROGRAMME 15

its activities, as well as assess whether there have been any unexpected outcomes (Royse, Thyer

& Padgett, 2015).

The Purpose of the Outcome Evaluation

The purpose of the outcome evaluation would be to assess whether the set objectives of

this initiative programme have been achieved through its activities, and to what extent. The ‘We

C.A.R.E.’ planners also seek to do an outcome evaluation for other key benefits and purposes

such as, informed decision making and funding resources, accountability (for ethical purposes),

and outcomes. Each will be detailed below.

Informed decision making and funding. If there is no outcome evaluation (or if poorly

conducted), the ‘We C.A.R.E.’ programme could be implemented for years without anyone

knowing with certainly the true extent of its success. Results of the evaluation, then, help

programme developers make improved decisions in future planning, whether areas need to be

improved, and in the implementation of resources, for example, funding. The outcome

evaluation will aid in more efficient realisation of the goals of the programme, as it will also

support its sustainability. Additionally, investors would be interested in understanding how

resources, such as their funding, have been utilised towards the achievement of the objectives of

the programme.

Accountability. A programme such as ‘We C.A.R.E.’ is projected on a national level

and relies on the keen interest and support of various parties, both from the private and public

sector. Support is necessary via financial investment of these parties. Therefore, it stands to

reason that, as alluded to in the previous section, there is great need to assess how these funds

have been used for the efficient and effective accomplishment of the programme. This lends
CITIZEN DEVELOPMENT PROGRAMME 16

itself to accountability which, when the evaluation is done and is conducted accurately, it

eradicates any doubt of ethical concerns of the use of funds and other resources.

Outcomes. Evaluation of the outcomes or results will help inform how ‘We C.A.R.E.’

would need to improve in the future. This involves which activities may need to be strengthened

or revised for intervention with participants or assessing how the funding and resources could be

more efficiently used to effect maximum desired results. The evaluation will also assess whether

there has been an achievement in intended outcomes, that is, to what extent has there been a

positive change in social behaviour and employable skills of the participants. Likewise, the

evaluation will also assess whether there have been any unintended or unexpected outcomes.

Evaluation Design and Justification

Selecting the most appropriate evaluation design for ‘We C.A.R.E.’ is critical so as to

assess the outcomes as efficiently as possible, as well as to identify and account for unintended

and unexpected outcomes and the factors which may have contributed to their occurrence and

overall achievement of the programme’s objectives.

Considering the objectives and structure of the programme, the quasi-experimental

design has been selected as the most appropriate for outcome evaluation. Morra-Imas and Rist

(2009) have informed that in this design, participants to be evaluated are intentionally, not

randomly, selected. Additionally, this design affords the evaluators sufficient information to

make sound connections and conclusions. The quasi-experimental design deemed best suited for

the outcome evaluation is the interrupted time series design (with one group).

The interrupted time series design, focusing on this treatment group comprising the

participants, allows the evaluators to collect data of effectiveness of activities at intervals

CITIZEN DEVELOPMENT PROGRAMME 17

throughout the duration of the programme, including before or at commencement of the

programme, and at the programme’s end (Reichardt & Mark, 2014). There are justifiable

reasons why this design is deemed beneficial to the nature of ‘We C.A.R.E.’. The various

intervals of observation are likely to inform the eventual outcomes. Reichardt and Mark (2014)

have explained that observations made could be assessed to predict possible trends if ‘We

C.A.R.E.’ did not exist. Analysis of observations made after completion of the programme

would reveal the trend. A comparison, then, between the two trends, will ascertain the change

brought about in the participants, effected by the programme. Furthermore, the risk of

maturation and changes in statistical representation of the participants compromising effective

outcome evaluation would be considerably reduced, due to the frequency of observations

throughout the programme.

Selection of Evaluation Team

The credibility of the outcome evaluation relies on the expertise of the evaluators, as well

as on the level of transparency of the entire evaluation process. Once a team of evaluators is

formed, this team must analyse the (historical, social, psychological and demographical) makeup

of the participants, as well as make considerations of the time frame and cost involved in the

planning outcome evaluation process. There are a number of drawbacks that could be

encountered conducting the outcome evaluation and these must be considered. Therefore,

meticulous investigation, consideration, planning and communication are needed beforehand in

order to prepare and avoid or minimise problems. Considering the gravity of this, the norms and

standards of the ‘We C.A.R.E.’ programme organisation have been used to guide the selection of

the most appropriate evaluation team for the programme according to the following criteria.
CITIZEN DEVELOPMENT PROGRAMME 18

Independence

It is preferred that evaluators do not have a vested interest in the programme. They must

not have been involved in the formation of design or policies of the programme, nor must there

be any expectation to do so in a foreseeable future. Therefore, evaluators must be independent to

the programme, recruited externally. This helps prevent influence of bias of the evaluator,

encouraging a more objective approach. Evaluators are free to express their opinion as they

desire and to conduct the evaluation with no risk to their professional career.

Qualifications and Competence

A significant criteria that is considered in the selection of evaluators is the level of

competence acquired to conduct the evaluation process. Evaluators must possess the knowledge,

skills set and professional experience necessary to conduct an evaluation successfully. An

evaluator must not undertake any evaluation activity if he does not possess the required

professional skills. Therefore, the educational background of the evaluator is of import where

this individual must have earned the relevant training and qualifications, preferably of a

postgraduate degree or equivalent or higher, that is specific to the discipline of evaluation,

project management and (advanced) statistical research. All in all, evaluators must be selected

based on competence, and via a transparent process.

Composition of Team

The number of members of the team depends on the size of the evaluation. The

evaluation seeks to assess a group of sixteen participants, per programme cycle, therefore, a total

of four to five evaluators would be employed. One of these will be the principal evaluator

(national or international) who will oversee all evaluation tasks and manage the evaluation team.
CITIZEN DEVELOPMENT PROGRAMME 19

Due to the programme objectives lending to the multi-faceted nature of the programme, the team

of evaluators needs to reflect different types of experience and expertise that are specific to the

skills being sought to evaluate.

The evaluation team also aims to be gender balanced, with members that reside in

different parts of Trinidad. Team members also aim to become familiarised with the

characteristics of the programme participants, as well as their social and cultural values. While

maintaining objectivity, this facilitates better understanding of participant practices and beliefs

throughout the evaluation process. Consideration could also made to employ a team, firm or

individuals from a concerned country. This is beneficial in that there may be increased assurance

that local factors are taken into play when conducting evaluations, as well as to enhance support

for developing the ability to conduct evaluations locally and in developing countries.

Essential Steps To Complete The Outcome Evaluation

Deciding on and stating the purpose of the evaluation

Evaluators must know from the clients (for example, the planners or managers of the

programme, as well as other funding agencies) the reasons for carrying out the evaluation.

Evaluators also need to know how the results of the evaluation will inform further decision

making regarding the running of ‘We C.A.R.E.’. Knowing the evaluation’s purpose will also aid

evaluators in creating informed, directed and thoughtful questions for their assessments.

Obtaining understanding of the programme

The title alone of the programme is insufficient in informing evaluators of the nature of the

programme and what it intends to accomplish. As a team, evaluators need to examine the theory

and beliefs behind the programme’s purpose and activities. In addition, examination of the
CITIZEN DEVELOPMENT PROGRAMME 20

programme’s logic model will be useful in providing this understanding, as evaluators will then

have a more informed idea of the principles that support the programme. After analysing

research to clarify and strengthen the logic model, outcome evaluation questions could be

formulated to help the programme to realise objectives, as outlined below:

 To what extent was there programme fidelity (adherence to the intended programme

logic)?

 To what extent could changes in participants’ behaviour, knowledge, skills and attitudes

be attributed to the programme’s activities?

 To what extent did the ‘We C.A.R.E.’ programme meet the expectations of stakeholders?

Choosing design and creating the outcome evaluation

This involves selecting the outcome evaluation design or designs best suited to the

programme. For the ‘We C.A.R.E.’ programme, quasi-experimental designs and the interrupted

time were seen as most appropriate. Decision making for the design of the outcome evaluation

could also be aided by questions formed from a design matrix (Newcomer, Hatry, & Wholey,

2010. p. 23). However, before data could be collected from the instruments and sources of data,

their reliability must be tested. Therefore, they must be piloted first to collect sample

information.

Performing outcome evaluations

The data gathered will be analysed. Efficient and appropriate instruments and methods

must be employed. Those instruments suited for the ‘We C.A.R.E.’ programme are: interview

schedules – individual and focus group interviews or discussions, interviews with stakeholders,

analysis of reflective journals written individually by participants, observation analysis schedules

CITIZEN DEVELOPMENT PROGRAMME 21

(structured and unstructured), surveys, questionnaires, observational checklists and rating scales.

This collection of data will also be conducted intermittently, before the start of the programme

(preliminary analysis), at specific intervals throughout (intervention), then at the end (post

analysis). The initial period of observation and analysis will be conducted over the three months,

the duration of the programme, where data will be gathered every one to two weeks or monthly,

depending on the evaluation. Data will be analysed (almost) simultaneously with collection or

soon after.

Organising and presentating preliminary draft of evaluation report

The preliminary draft of the report should be structured according to emerging themes

and not the evaluation questions (Morra-Imas & Rist, 2009). The reason is that a number of the

original evaluation questions may not be of the same significance throughout the programme’s

duration. Also, clients of the programme need to be informed first of its results before they

could be published. This is to afford clients the opportunity to make any necessary amendments

to or provision of factual details. The final presentation of the findings will also include

limitations, unintended and unexpected outcomes. There will be discussion surrounding these

and all findings.

Disseminating evaluation outcomes

The clients that requested the evaluation must be the first recipients of the disseminated

findings. Also of great import, as recipients, would be those parties who have the influence to

effect the changes recommended within the evaluation report. Finally, these findings need to be

distributed to those persons who will be influenced by its bearing or lack thereof.

Logic Framework
CITIZEN DEVELOPMENT PROGRAMME 22

A logic framework, otherwise known as a logic frame or log frame, was created for its

usefulness in the overall improvement of the planning, funding, operating and evaluation of the

‘We C.A.R.E.’ programme (Jackson, 2013). This instrument was designed in a table form

displaying information for easy identification and understanding, information which further

serves to clarify the overall goals of the programme, indicators, verification and potential

assumptions. As simple as it appears, the log frame helps to bring structure to ‘We C.A.R.E.’ in

that it aids in planning and managing the programme in an easily identifiable sequence of

activities. The following logic framework (Table 1) has been designed for ‘We C.A.R.E.’.

Table 1

Logic Framework of the ‘We C.A.R.E.’ Programme

Project Indicators Verification Assumptions

Summary

Overall Participants, Number of Comparison of Participants

Goals young males participants results among attend and are

between 16 and that demonstrate as intervention faithful to the

18 years of age, such during 90% to observational programme, are

demonstrate 100% of all activities made at receptive and

improved activities by (mid to the start, during and open to change

cognitive, long-term) and at the end (after) the and participate

affective and exhibit appropriate programme; wholly.

psychomotor behaviour by the

CITIZEN DEVELOPMENT PROGRAMME 23

skills and social end of the Observational 2. Participants

behaviour, with programme instruments would learn; behaviour

100% accuracy, academic year. include checklists and attitudes

within using different improve and

appropriate time headings. skills are learned.

frames Other assessments

and contexts. include role play, 3. Participants

surveys, attend all

questionnaires, sessions.

focus group

discussions, one- 4. Persons

on-one interviews conducting

with participants, sessions in the

members of programmeme

immediate family, are highly trained

environs, workplace professionals in

and schools, the field,

reflections through equipped to help

journaling, practical participants

assessments achieve the set

objectives.
CITIZEN DEVELOPMENT PROGRAMME 24

5. Participants

continue with

learned

behaviour,

attitudes and

skills within their

own societies.

Outcome Participants apply Number of Comparative results 1. Participants

3 marketable and participants from quantitative attend all training

employable skills applying skills with and practical sessions.

within context of 100% accuracy assessments of

the trade skills, including, 2. Trainers are

(mechanics, written and oral adequately

carpentry or tests on subject, knowledgeable

plumbing) from start and skilled in the

according to throughout the field and in

level, in specified programme; teaching and

time frame, with observation of guide

100% accuracy. application of participants

practical, efficiently.

psychomotor skills

from mid-term to
CITIZEN DEVELOPMENT PROGRAMME 25

end of programme,

using checklist

observational

instrument

Output 3 Participants Percentage of Ongoing and 1. Participants

identify and accurate comparative are literate.

explain key explanations question-answer

terminologies, assessments, both 2. Participants

definitions and oral and written attend all training

processes with sessions.

relation to

respective 3. Participants

employable trade. are interested in

learning the

theoretical

content of the

trade.

4. Trainers are

adequately

knowledgeable
CITIZEN DEVELOPMENT PROGRAMME 26

and skilled in the

field and in

teaching and

guide

participants

efficiently.

Output Participants apply Degree of practical Observations of 1. Resources are

3.1 psychomotor application – practical assessment available for

skills to demonstrated with displaying practical

respective 100% accuracy psychomotor skills, assessments

employable trade, conducted from

according to level mid-term 2. Participants

throughout are interested in

programmeme to its learning the

end practical skills of

the trade

Checklist

observational

instrument is

utilised
CITIZEN DEVELOPMENT PROGRAMME 27

Activities Specific to the With 100% 1. Comparative 1. Participants

3 trade learnt by the accuracy: assessment of are literate

participant, question-answer

participants: Mechanics: sessions – written 2. Participants

· Number of tires and oral, conducted operate

1. List and changed in twenty throughout efficiently under

explain key minutes programme, from examination

terminology and · Engines assessed beginning to end; conditions and

definitions – and changed within time restrictions

orally, via thirty minutes 2. Comparative

individual observational 3. All materials

presentation in Carpentry assessments of and resources

sessions; written, · Dining table activities using necessary for

responding to built within fifty checklist practical

questions on minutes instrument. assessment are

assessment · Wardrobe readily available.

constructed

2. Individually within ninety 4. Participants

apply skills and minutes understand the

demonstrate task use of all the

required – Plumbing tools provided.

CITIZEN DEVELOPMENT PROGRAMME 28

Mechanics – · Number of pipes 5. The activity is

change a tire, assessed repaired in appropriate for

change an engine; thirty minutes the ability, level

Carpentry – · Installation of (age) of the

construct a study waste disposal participant.

desk or a dining system within thirty

table, construct a minutes

wardrobe; Assessment and

Plumbing – repair repair of waste

a broken pipe, disposal system

install and repair within twenty

a waste disposal minutes

system

Conclusion

This paper has served to outline the processes in reviewing a citizen development

programme, with specific reference to young males, between the ages of sixteen and eighteen, in

Trinidad. It is the view of the author that the newly developed ‘We C.A.R.E.’ programme will

provide effective and much needed attention to the crime situation, through a series of social and

trade skills training workshops, while providing emotional and psychological support and care to

its participants. The duration of the ‘We C.A.R.E.’ programme is one three months and

participants and programme will be evaluated keenly. The young men that successfully
CITIZEN DEVELOPMENT PROGRAMME 29

participate in and graduate from ‘We C.A.R.E.’, will be able, in turn, to care for others by

making a significant positive social and economic impact on their immediate communities, on

Trinidad and Tobago and beyond.

CITIZEN DEVELOPMENT PROGRAMME 30

References

Amodeo, L. B., & Wooden, S. (1984). Transactional evaluation: A way to involve student

teachers in change. Education, 104(3).

Andrews, D. A., & Bonta, J. (2010). The psychology of criminal conduct. Routledge.

Brandon, P. R. (1998). Stakeholder participation for the purpose of helping ensure evaluation

validity: Bridging the gap between collaborative and non-collaborative

evaluations. American journal of evaluation, 19(3), 325-337.

Charters Jr, W. W., & Jones, J. E. (1973). On the risk of appraising non-events in program

evaluation. Educational Researcher, 2(11), 5-7.

Community Development. (2018). Retrieved from Ministry of Community Development, Culture

and The Arts: http://www.cdca.gov.tt/community-development/community-education-

programme/

Community Education (Skills Training) Programme. (2018). Retrieved from TTConnect -

Government At Your Service:

https://www.ttconnect.gov.tt/gortt/portal/ttconnect/!ut/p/a1/04_Sj9CPykssy0xPLMnMz0

vMAfGjzOK9A40MTD0tjQ38Aw0sDYyCPA1dDUy9jd3NjYEKIlEUBLm7ARW4mhp

6eIcZGxgYEKffAAdwJKg_ODVPP1w_Cq8yL1MMBZjOBCvA446C3NCICs9MRwCZ

JcB2/dl5/d5/L2dBISEvZ0FBIS9nQSEh/?WCM_GLOBAL_CONTE

Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics. (4th Ed.). SAGE Publications

Inc.

Guba, E. G., & Lincoln, Y. S. (1981). Effective evaluation: Improving the usefulness of
CITIZEN DEVELOPMENT PROGRAMME 31

evaluation results through responsive and naturalistic approaches. Jossey-Bass.

Hargreaves, A. (2005). Educational change takes ages: Life, career and generational factors in

teachers’ emotional responses to educational change. Teaching and teacher

Education, 21(8), 967-983.

Herman, J. L. (1987). Program evaluation kit. Volume 2. Thousand Oaks, CA: Sage Lall, Vidya.

2007. Bullying, Victimization and Delinquency in Primary Schools in Trinidad and

Tobago: Some Preliminary Results. Caribbean Journal of Criminology and Social

Psychology 12(1-2): 155-76.

House, E. R. (1983). Assumptions underlying evaluation models. In Evaluation models (pp. 45-

64). Springer, Dordrecht.

Jackson, E. T. (2013). Interrogating the theory of change: evaluating impact investing where it

matters most. Journal of Sustainable Finance & Investment, 3(2), 95-110.

King, G. (2009). A relational goal-oriented model of optimal service delivery to children and

families. Physical & Occupational Therapy in Pediatrics, 29(4), 384-408.

Lall, V. (2007). Bullying, victimization and delinquency in primary schools in Trinidad and

Tobago: some preliminary results. Caribbean Journal of Criminology and Social

Psychology, 12(1-2), 155-76.

CITIZEN DEVELOPMENT PROGRAMME 32

Marsh, J. (1978). The goal-oriented approach to evaluation: Critique and case study from drug

abuse treatment. Evaluation and Program Planning, 1(1), 41-49.

Melrose, M. (1996). Encouraging transactional and critical models of curriculum evaluation-

different approaches: Theory and practice in higher education. A paper presented at the

HERDSA Conference. July, 1996.

Mertens, D. M. (2014). Research and evaluation in education and psychology: Integrating

diversity with quantitative, qualitative, and mixed methods. Sage publications.

Morra-Imas, L. G., & Rist, R. C. (2009). The road to results: Designing and

conducting effective development evaluations. World Bank Publications.

Murphy, E. A., & Dingwall, R. (2003). Qualitative methods and health policy research.

Transaction Publishers.

Patton, M.Q. (1987). Qualitative Research Evaluation Methods. Thousand Oaks, CA: Sage

Publishers.

Patton, M. Q. (2002). Two decades of developments in qualitative inquiry: A personal,

experiential perspective. Qualitative social work, 1(3), 261-283.

Reddock, R. (2018, March 7). Trinidad and Tobago Newsday.

CITIZEN DEVELOPMENT PROGRAMME 33

Reichardt, C. S. & Mark, M. M. (2004). Quasi-experimentation. In J. S. Wholey, H. P. Hatry, &

K. E. Newcomer. (Eds.). Handbook of Practical Program Evaluation. (pp. 126-149). San

Francisco, CA: Jossey-Bass.

Roberts, P., Priest, H., & Traynor, M. (2006). Reliability and validity in research. Nursing

Standard (through 2013), 20(44), 41.

Rossi, P. H. (1972). Testing for success and failure in social action. Evaluating social programs,

11-65.

Royse, D., Thyer, B. A., & Padgett, D. K. (2015). Program evaluation: An introduction to an

evidence-based approach. Cengage Learning.

Seepersad, R. (2016). Crime and Violence in Trinidad and Tobago - IDB Series on Crime and

Violence in the Caribbean. Inter-American Development Bank - Country Department

Group.

Shaw, R. E., & Gaynor, J. (1982). Goal-oriented evaluation for program planning: A case

study. Evaluation and Program Planning, 5(2), 141-148.

The Impact of Poverty on Crime. (2012, August 4). Trinidad Express.

Weiss, C. H. (1998). Evaluation: Methods for studying programs and policies. 2nd Ed. Upper

Saddle River, NJ: Prentice Hall.

Youth in Crime Rate High. (2018, March 7). Trinidad and Tobago Newsday.

Zheng, S., Rosson, M. B., Shih, P. C., & Carroll, J. M. (2015, February). Understanding student
CITIZEN DEVELOPMENT PROGRAMME 34

motivation, behaviors and perceptions in MOOCs. In Proceedings of the 18th ACM

conference on computer supported cooperative work & social computing (pp. 1882-

1895). ACM.