Symposium: Social Justice and Lifelong Learning in

Europe

Title of Paper: The Challenges of conducting surveys

with adults who have experienced ‘educational
disadvantage’
Authors: Jane Carrigan, Imelda Byrne and Valerie McLoughlin

Institution: Educational Disadvantage Centre

St. Patrick’s College, Drumcondra, Dublin 9, Ireland

I feel this questionnaire is very statistical and it may not entirely get to the true
feelings, reasons and particular issues, why some people have chosen to
take up adult education. Boxes are easy to fill out especially for research
purposes, but sometimes I find it difficult to really express what you would
wish to say in a box because it does not always leave room for elaboration…I
find emotionally, psychologically and identity wise I may just be tired of being
boxed. Just my own opinion

Survey Participant No. 603

Introduction

This paper is a discussion of the many challenges that faced researchers in

Ireland as they undertook fieldwork as part of Lifelong Learning 2010
‘Towards a Lifelong Learning Society in Europe’. The project is financed by
the European Commission under the Sixth Framework Programme and seeks
to examine the contribution of the education system to lifelong learning.
LLL2010 began in September 2005 and is scheduled to be completed in
August 2010. LLL2010 aims and objectives are to assess and compare the
effectiveness of adult education policies and practices in different EU member
states and to note the implications for the creation of a European knowledge
society. The consortium consists of research teams from thirteen different
countries; Estonia, Belgium, Scotland, England, Ireland, Bulgaria, Russia,
Czech Republic, Norway, Russia, Austria, Slovenia and Lithuania.

The project aims to disseminate its findings to policy makers, service users
and practitioners. However, it also aims to involve relevant actors in the
process. As such Subproject 3 consisted of a survey of 1,000 adult learners
who are currently studying in the formal education system. Subproject 3,was

1
lead by the Higher Institute for Labour Studies (HIVA) in Belgium and the
Institute for International and Social Studies (IISS) in Estonia.

Through administering a survey, the project set out to document the

experiences and opinions of adult learners who have returned to education
following a break of two or more years. Students from across the educational
spectrum, from those attending literacy training up to and including those
undertaking state examinations and degrees, were sampled. The sample was
stratified by level of education, using the International Standard Classification
of Education (ISCED) levels. ISCED levels were developed by UNESCO in
the 1970s and are used to compile internationally comparable statistics on
education.

This paper concentrates on the challenges posed by undertaking survey work

with those attending courses on ISCED levels 1, 2 and 3. Adults traditionally
undertaking these courses would have received limited formal education.
ISCED level one, for example, refers to first stage basic education or primary
education. The second level refers to lower secondary education or second
stage of basic education, while the third ISCED levels refers to upper
secondary education (ISCED, 1997). Entrants to ISCED level three in the
school system would typically be aged 15 –16 years. Adult learners from
these levels, who participated in the survey, had experienced educational
disadvantage.

The aim of this paper, however, is not just to highlight the many real
challenges faced by researchers when working with vulnerable
students/people but to describe how those challenges were faced and to offer
recommendations for further research.

Background

The Lisbon Agenda, set out by the European Council in March 2000, aimed to
create by 2010 an EU that would be the most competitive knowledge based
economy in the world. Access to life-long learning was to play a vital role in
this process and a host of documents detailing the EU’s commitment to life-
long learning have followed (2000, 2001, 2002, 2003). The presence of these
policy documents testifies to the popularity of life-long learning among policy
makers.

Both the Irish government’s White and Green papers on Adult Education
(2000, 1998) inform government policy on the role of adult education and
were influenced by the EU commitment to the concept of life-long learning.
Share et al (2007) note that while adult education in Ireland was once one of
the most neglected sectors in education provision it is now one of the most
rapidly rising. The targets of the White Paper on Adult Education (Learning
for Life, 2000) for the National Adult Literacy Programme include ‘a
continuous increase in the numbers of clients reached’ and ‘prioritisation of
those with lowest literacy levels’. Maunsell et al (2008) note that the policy

2
objectives for lifelong learning in Ireland underline core concerns that lifelong
learning ‘should embrace personal, cultural and social goals as well as
economic ones and be seen as promoting collective as well as personal
advancement’.

The past decade has witnessed unprecedented economic growth in Ireland.

The Quarterly National Household Survey (2006) notes that an increase in
educational attainment has also occurred; almost a quarter of those aged 15-
64 had a third level qualification, which compares to just a fifth three years
previously, and almost 40% of all 25-34 year olds declared having a third level
qualification in 2005 compared to a figure of just over 34% in 2002. However
despite that growth, figures from the latest census in Ireland (Census, 2006)
indicate that while the number of people educated to primary level only
continues to fall, it nonetheless still stands at 15.2% of people aged 15 and
over. Maunsell et al (2008) highlight the fact that Ireland has the second
lowest level of literacy among the 22 countries surveyed as part of the
International Adult Literacy Survey (IALS). This survey (IALS, 1997), which
was conducted in Ireland in 1995, revealed that nearly one quarter of the
population in Ireland has literacy levels rated at the lowest level.

Challenges Faced

The focus of subproject 3 was the participation of adults in lifelong learning

and the lead research teams identified two general research questions; what
is the role of the formal education system in stimulating participation in lifelong
learning and what is the role of the formal education system in reducing
inequality in participation? A national report from Ireland is currently being
compiled; and it is hoped, following detailed analysis, that answers to the
research questions will be provided. This paper, however, offers a reflection of
conducting surveys with adults who have experienced educational
disadvantage.

Questionnaires from a sample size of 1,000 learners were to be completed.

Essentially the questionnaires consisted of four groups of respondents;
ISCED 1 & 2, ISCED 3, ISCED 4 and ISCED 5 & 6 with a target of 250
students from each of these groups. The lead teams with responsibility for
the survey acknowledged that it was not possible to use just one type of
survey method because of the diversity of the participants and they therefore
recommended different methods for different groups. Numerous academics
(De Vaus, 1996; Parry et al 2001) have also suggested that face-to-face
interview methods achieve better response rates than either postal
questionnaires or telephone methods. With this in mind a decision was made
by the Educational Disadvantage Centre to concentrate research efforts in
obtaining the maximum number of participants from ISCED levels 1 and 2,
using face-to face interview methods. Two researchers [JC and IB] were
assigned to doing structured interviews with learners from ISCED 1 and 2
level courses.

3
Table 1 outlines the actual numbers from each ISCED level and the method
used. As can be seen from this table, despite intense effort on behalf of the
research team, over 62% (n=620) of our sample consists of learners from
courses at ISCED level 4 and higher.

ISCED Target Actual Method

Level Number Number
1 +2 250 178 Face-to Face interviews
3 250 202 Written Questionnaire
4 250 341 Written Questionnaire and online survey
5+6 250 279 Online survey
Total 1,000 1,000
Table 1

Most basic education classes lasted no more than two hours and the average
length of time for a face-to-face interview was thirty-four minutes. This meant
in practice that researchers [JC and IB] could usually interview just four
people per session. Very few education classes took place in the afternoon
and so researchers in all three levels concentrated on morning and evening
classes. ISCED 3 participants did not have face-to-face interviews but
completed the survey in a group setting with one member of the research
team [McL] on hand to facilitate. Classes for these groups were also
generally small. Before surveys were commenced in either one-to-one or
group format, the researcher would first address a group of possible
participants and explain the research project and encourage questions.
Possible participants were also provided with a written explanation of the
survey.

It was noted that while the researchers involved in face-to-face surveys with
individuals were able to establish a rapport with the learner and work through
the questionnaire in a sensitive manner, the researcher working with ISCED 3
learners (McL) faced a more difficult task. In a group setting it was clear that
the survey was a challenge to some of the students. This disparity in ability in
ISCED 3 courses meant some learners were able to complete the survey in
approximately 20 minutes while a few learners in the same group were still
working through the questions after 50 minutes. While the researcher was
conscious of this and sensitive to learners’ difficulties, it had the very negative
effect of highlighting the difference in ability between various students in the
same group and embarrassing those who struggled. Unlike the face-to-face
interviews, in a group interview it was not possible to hear the learners’
personal stories, which often detailed experiences of disadvantage. According
to the ISCED 3 researcher [McL], many older students, in particular female
ones, still bore a fierce resentment to what they saw as being cheated out of
an education when they were young due to poverty and the attitude that
educating girls was unnecessary. This was also reflected in the experiences of

4
the ISCED 1 and 2 researchers [JC and IB] although it was felt that face-to-
face interviews with individuals allowed more time for learners’ previous
experiences of education to be heard and acknowledged.

The biggest challenge faced by the researchers, and indeed participants, was
the length and difficulty of the questionnaire. One college of further education
refused access as the principal felt that attempting this survey would have a
negative effect on learners and he did not want to “set [my] students up to
fail”. The survey consisted of over fifty questions including a number of Likert
scales in which learners were asked the degree to which they agreed or
disagreed with various statements. While one-to-one interviews with
researchers allowed for statements to be made clear, it was felt that some
ISCED 3 participants, even with a trained and experienced researcher
facilitating, were sometimes reluctant to ask for assistance and seemed to be
answering questions incorrectly. Face-to face interviews, although more
resource intensive, did allow for a more comprehensive completion of the
survey. Absenteeism was also a feature with ISCED level three learners. The
researcher found that the number of learners anticipated in a class often did
not respond to the actual numbers who were there at the appointed time.
This meant that the researcher often had to return to the same site again and
repeat the exercise.

All the researchers agreed that the cooperation of school principals, adult
education organisers and tutors was crucial to recruitment and paramount to
our success in reaching our target of 1,000 surveys of adult learners. The
overwhelming majority of institutions, staff and tutors recognised the
importance of the survey and were more than accommodating to the research
team. Parry et al (2001) note that despite a shared understanding among the
research community that the recruitment of respondents in areas of
deprivation presents particular difficulties, the issue has been largely
neglected in the literature. The researchers noted that although many of the
adult education organisations that facilitated the research were in areas that
had experienced deprivation, the response rate among learners was high.
Goyder (1987) notes that there is little evidence that members of lower socio-
economic groups have a hostile attitude towards surveying in general. In fact
of the few who chose not to partake, two reasons dominated. A number of
students chose not to participate through self-disqualification i.e. they felt that
the survey did not apply to them. Parry et al (2001) cite Goyder et al (1999)
who hypothesize that this may be the reason for non-response among
members of lower socio-economic groups. Others chose not to participate as
they did not want to miss out on their class and for a number of learners they
received only two hours tuition per week. Other students were studying
towards the Leaving Certificate and time was precious to them and their
tutors. There was often a sense of resentment at being asked by their tutors
to give up what could amount to nearly half a class to fill in what they
perceived to be yet another survey that meant little to them. One ISCED 3
participant summed up this element of survey fatigue and frustration:

Boxes are easy to fill out especially for research purposes, but sometimes I
find it difficult to really express what you would wish to say in a box because it

5
does not always leave room for elaboration…I find emotionally,
psychologically and identity wise I may just be tired of being boxed. Just my
own opinion
(Survey Participant No. 603)

This comment serves as a reminder of our ethical responsibilities as

researchers.

Profile of Learners

The typical profile of participants from ISCED level 1-3 was female, aged
between 28-37, Irish, not in full time employment and earning less than the
average monthly income.

The overwhelming majority of participants in the survey of adult learners were

female (70.5%, n=705) as compared to male participants (28.8%, n=288).
This figure however was not unexpected. Downes et al (2006) acknowledge
that adult education in lifelong learning has been driven by a thriving
community-based women’s education sector. The ESRI Annual School
Leavers’ Survey (2006) has also noted that females are still more likely than
males to go on to further and higher education as a greater proportion of
females than males complete the Leaving Certificate, although the gap is
lessening. When only ISCED level 1-3 learners (n=379) are taken into
account, the gap between males and female participation is evident; males
make up 23.6% (n=90) while female participation is at 76% (n=289).
Participants undertaking courses in ISCED level 1 and 2 numbered 178. Of
the total number of males who participated in the study, 22.4% (n=40) were
completing courses in ISCED level 1 & 2 whereas almost 78% of female
participants (n=138) were doing those courses. Our results supported the
literature in this area which suggests that men are underrepresented in
literacy and basic education programmes (Downes et al, 2006; Learning for
Life, 2000; Owens, 2000; OECD report Education at a Glance, 2000).

Participants were also asked what the highest level of education they had
completed. Of the 352 learners from ISCED levels 1-3, 6.2% (n=22)
responded that they had no formal education or below primary education.
32.6% (n=115) reported that they had completed primary level only. The mode
year of birth of ISCED 1-2 participants is between 1950-1959. For ISCED 3
participants however the mode year of birth is between 1980-1989. Indeed
when the ISCED levels are combined the mode year of birth is between 1980
and 1989. This indicates that the participants in general have a relatively
young profile, which is of concern when compared with the demands of a
knowledge society. The International Adult Literacy Survey (2000) in Ireland
did demonstrate substantially lower levels of literacy in older age groups but
also showed poor levels of literacy among the 16-25 age group. This
indicated that mass access to educational opportunities had not eradicated
the problem.

Participants were almost evenly split between married and unmarried with just
over 10% being widowed or divorced. 197 respondents answered the

6
question on net monthly income, of which, over 60% (n=120) earned below
the average net monthly income in Ireland. Almost 90% of participants in
ISCED 1-3 courses were Irish. A number of Irish participants (n=15) further
identified themselves, as Irish Travellers and they were all undertaking
courses at ISCED level 1-2. The Department of Education (2006)
acknowledges that historically Travellers were often marginalised from the
educational system. Up until the 1990s Travellers were often educated
through segregated provision. The White Paper on Adult Education (2000)
notes that access to mainstream adult education programmes is one of the
key features of the development of an equality strategy within the adult
education sector. Downes et al (2007) highlights the fact that issues such as
health, childcare and housing standards can impact of educational attainment
and notes that the Report of the High Level Group on Traveller Issues (2006)
connects the important issues of poverty and ethnicity.

Conclusions and suggestions for future research

The Educational Disadvantage Centre’s mission is to promote equality in
education and to contribute to best practice in national and international policy
regarding educational disadvantage and equality in education. The data
gathered from this survey will provide invaluable information nationally and
participation in LLL2010 will allow for comparative analysis. Parry et al (2001)
argue that ‘one of the purposes of research that targets disadvantaged
communities is to give a voice to those who are marginalized or socially
excluded’ and we hope, through our research efforts, that the experience of
learners, particularly those who have experienced educational disadvantage
will inform policy. With this in mind we would like to make a number of
suggestions for future research:

• Surveys should be written so as to be accessible to those with literacy

difficulties.
• Personal information should be sought at the beginning of any survey.
This has two benefits. Firstly participants may gain confidence in
answering these common factual questions and be encouraged to
continue. Secondly, if participants do give up, the research team will be
able to use the demographics to build up a profile, which may be used
to inform future survey design.
• Research with groups who have been marginalised involves additional
time and resources.
• Questionnaires should not be too long – in practice we found that it
was vital to have supportive tutors as the initial reaction of learners
when seeing the length of the survey was one of trepidation. The
tutors were able to endorse what the trained researcher was saying;
however the length of the survey discouraged some non-respondents
and there were also a relatively high number of incompletes.
• An awareness of the sensitivities of adult learners needs to be
recognised and acknowledged. Many adult learners undertaking
courses in ISCED levels 1-3 had had negative experiences of
education previously.

7
 • A qualitative aspect of the study may have alleviated the pressure
some learners experienced in trying to put their experiences into neat
boxes in the questionnaire. Furthermore there is a need in undertaking
such research for people to be given the opportunity to speak of any
negative experiences they may have had with the education system in
the past and how that may have impacted on their current learning.

Finally, conducting survey work with adult learners, many of whom had
experienced disadvantage, was a very rewarding experience for all the
researchers involved. It did involve however much time and patience and
resources. Despite these obstacles, in the interests of social inclusion and
indeed social injustice, it is imperative that research with groups who are
vulnerable or marginalised continues to be carried out.

8
Bibliography

Central Statistics Office. Principal Statistics Census 2006

De Vaus, D.A. (1996) Surveys in Social Research New South Wales, Allen &
Unwin).

Department of Education and Science (1998). Adult Education in an Era of Lifelong

Learning: Green Paper on Adult Education. Dublin: Government Publications.

Department of Education and Science (2000). Learning for life: White Paper on adult
education. Dublin: Government Publications.

Department of Education and Science (2000). Learning for life: White Paper on adult
education. Dublin: Government Publications.

Department of Education and Science (2006) Recommendations for a Traveller

Education Strategy 2006-2010, Joint Working Group on Traveller Education, Dublin:
DES

Department of Justice, Equality and Law Reform (2006) Report of the High Level
Group on Traveller Issues, Dublin: DJELR

Downes, P. & Gilligan, A.G. (2007) eds. Beyond Educational Disadvantage Dublin:
IPA

Downes, P., Maunsell, C., McLoughlin, V. & Taljunaite, M. (2006) ‘Lifelong learning
in Ireland and Lithuania: Some examples of Irish policy and practice for Lithuania to
consider? Filosofija.Sociologija (ed. Lithuanian Academy of Sciences), No. 4 pp. 29-
36.

European Commission (2000). A Memorandum on Lifelong Learning. Brussels.

European Commission (2001). Making a European Area of Lifelong Learning a

Reality. Brussels.

European Commission (2002). European Report on Quality Indicators of Lifelong

Learning. Brussels.

European Commission (2003). Implementing lifelong strategies in Europe: progress

repsort on the follow-up to the Council resolution of 2002 on lifelong learning.
Brussels.

Gorby, S., McCoy,S. and Watson,D. (2006) 2004 Annual School Leavers’ Survey of
2002/2003 Leavers. Dublin: ESRI and Department of Education & Science

Goyder. J. (1987) The Silent Majority. Cambridge University Press

Maunsell, C., Downes, P. & McLoughlin, V. (2008) National Report on Lifelong

Learning in Ireland. LLL2010: Sub-project 1: Towards a Lifelong Learning Society in

9
Europe - the Contribution of the Education System. Educational Disadvantage Centre.
Dublin.

Morgan, M., Hickey, B., & Kellaghan, T. (1997). International adult literacy survey:
Results for Ireland (A report to the Minister for Education). Dublin: Government
Publications.

OECD. (1997). Literacy skills for the knowledge society: Further results from the
International Adult Literacy Survey. Paris: Author

Organisation for Economic Co-operation and Development (OECD) (2000)

Education at a Glance. Paris: OECD

Owens, T. (2000) Men on the Move: A Study of Barriers to Male Participation in

Education and Training Initiatives, Dublin: AONTAS

Parry, O.; Bancroft, A.; Gnich, W. and Amos,A. (2001) ‘Nobody home? Issues of
respondent recruitment in areas of deprivation’. Critical Public Health, Vol. 11, 4
pp305-317

Share, P., Tovey, H. and Corcoran, M. (2007) A Sociology of Ireland (3rd Edition). Gill
& MacMillan

UNESCO (1997) International Standard Classification of Education (ISCED 1997).

Paris

10