Online Learning for Marginalized Communities: Know your Target Group,

Experiment, and Evaluate and Learn

Francisco J. Proenza, Universitat Pompeu Fabra

Keynote Address delivered at Stellar-ICel Conference

Kota Kinabalu, 27 May 2015

The single most important challenge of the Twenty First Century is poverty and
inequality in the midst of plenty.

We must learn to harness the unprecedented technological change we are

experiencing, so that its economic benefits are widely shared, and enable us to
drastically reduce, even abolish poverty, and redress income inequality.

It might seem odd to bring up this issue in Malaysia, a country with a remarkable
record achieving inclusive growth. But we are living in a digital world on steroids. As
innovation, artificial intelligence and robotization advance, the replacement of
middle-skill tasks will accelerate everywhere. Never before has the world seen such a
dramatic expansion in technology and disruption in labor markets.

Young people are aware how important it is for them to learn and use ICTs in this
new world. Let me cite Alejandro, an Argentinian low-income youth we interviewed.
Alejandro lives in an impoverished neighborhood of Buenos Aires where many at risk
youngsters live. He said:

“Knowing how to use a PC is always useful. For Whatever job they ask if you
have computer skills, just in case. Even for a street sweeping job they ask for
computer skills. I don’t know...maybe they have in mind a computerized
machine, a computerized broom.”
(cited in Proenza 2015)

How can online learning help Alejandro and other marginal learners?

Three elements are essential. First, you have to know your target group and its
context. Second, you must experiment. Third, you must evaluate your experiments,
to learn and improve implementation and impact.

I. Know Your Target Group and its Context

We need to identify the target group, understand the conditions under which it
functions, and identify what kinds of learning would engage and help them.

Where should we focus?

People with home connections tend to be better off than users of public access ICT.

Here I will focus on marginalized learners who use public access venues (i.e.
cybercafés and telecentres, including in the latter category libraries equipped with
computers connected to the Internet).

Not all public access users are marginal learners. Many are transitory users.

  1
It is in rural areas where you find the greatest access limitations. The rural/urban
access divide is commonplace in developing countries everywhere.

-In China, urban Internet penetration is 63 percent compared to 28 percent in

rural communities.

-In India urban penetration is 32 percent to 7 percent in rural areas.

- Even in Malaysia, a middle-income country that has made significant strides,

rural Internet penetration is 61 percent compared to about 71 in urban areas.

Not only is the access challenge more pronounced in rural areas, incomes are
generally lower. Also, it is in rural areas that traditionally disenfranchised ethnic
minorities usually live.

Once we decide to focus on rural areas, we are not talking about cybercafés but
about government sponsored telecentres, rural telecentres. These are centres that
require subsidies, certainly to get them started and frequently to keep them running.

Why? Because the sustainability and impact of rural telecentres is hindered by:
i. high connectivity costs, ii. high equipment maintenance costs, iii. digital illiteracy,
and iv. the political allure of rural telecentres

Let me go over these briefly.

High connectivity costs

The more remote places (e.g. Bario) you have to service through VSAT technology,
and this can be quite costly (e.g. $250/day in Sri Lanka (Proenza 2008); about
$600/month in e-Bario in 2006-20071).

Fortunately, we can lower connection costs through innovation; e.g. $15/month in

SARI project, run by n-logue in India (Best and Kumar 2008).

High equipment maintenance costs

Breakdowns become a major problem when spare parts and expert technicians are
distant. To make matters worse, telecentre managers and staff are often ignorant of
what is needed to repair equipment. What would require a simple fix in an urban
area often becomes an ordeal in a rural community.

I was once in Jequitinhonha, one of the most remote rural areas of Brazil, visiting a
primary school that had received 5 computers a year prior to my visit. The children
were not allowed to use the computers for fear they might break them. Only two or
three teachers used the computers occasionally. A printer had been broken for
several months because they had not been able to get the technician to visit them,
in part because of the administrative paper work involved.

                                                                                                               
1  Alvin Yeo, interview, Singapore, May 2015. The figure of $600 for e-Bario is inclusive of
maintenance costs; i.e. the contract provided for visits to the area by maintenance staff to
make repairs.  

  2
In another instance, this time in small town not too far from Yogyakarta, Indonesia,
we talked to an Internet service provider who had generously equipped the local
school with computers connected to the Internet for free. The school administrator
kept them locked up, for fear of breaking them.

Digital illiteracy

ICTs cannot be explained. They must be experienced. Furthermore, if few peers are
online, networking benefits are negligible. If only a few of your friends use the
technology your ability to communicate is limited.

In 2005 I visited the head of the agricultural extension agency in Indonesia. This
official was in charge of a staff of over 30,000 people. His office was huge, but there
was no computer anywhere in sight. His business card had no email address.

I asked him whether he felt computers and the Internet could be useful for his work.
His response was no

A few days later in a small town near Yogyakarta we met a farmers’ cooperative
made up of young leaders. I asked them if they had used computers and the
Internet. Their response: Why do you ask such a silly question? Of course we use
these tools. All of us use them every day, about 2-3 hours a day.

Next we visited a telecentre sponsored by UNDP located in an Islamic school. We

met with three leaders entrusted with the facility and for promoting the centre. All
three praised the initiative as a harbinger of progress. Shortly afterwards, the wife of
the village leader, in an aside, told the women in our World Bank team, in
confidence, that they really did not know how to use the computers or how it could
be used for the benefit of the community.

I have found, again and again, that digital illiteracy is the most common and
pernicious obstacle to rural telecentre sustainability and impact. If people do not
know how to use computers and the Internet they will not use the telecentre. Many
initiatives have failed because only a few people used the facilities; e.g. in India:
Drishtee, n-logue, Gyandoot, eChoupal (Proenza 2015).

Connectivity costs can be addressed through technological innovations. Training of

local people can reduce high maintenance costs. But if the community does not use
the facilities then there is no market and public subsidization cannot be justified.

This is why it is essential, as a first step, to ensure that rural marginal learners
receive digital literacy and get a chance to practice what they learn.

The allure of rural telecentres

Cybercafés are predominantly urban. On the other hand, most rural public access
venues are government-sponsored telecentres. Whereas cybercafés obey the laws of
the market, telecentres have to fulfill political and bureaucratic aspirations. If they do
not, funding may be cut.

Rural people love telecentres, even if they do not use them or understand what they
can be used for. Politicians are keenly aware of this. They love to inaugurate and

  3
even support telecentres, especially just before elections. Unfortunately, they do not
always keep track to what happens afterwards.

In Muy Muy, a small town in Nicaragua, Central America, the World Bank sponsored
project known as Agrarian Information System (SIA) installed a telecentre. The
centre received guidance and support from a steering committee composed of 20
members. In 2004 I attended a meetings of this steering committee. All of them
were very appreciative of the centre, and saw it as a harbinger of modernization and
progress for the municipality. Nevertheless, only 8 of these had used the centre’s
telephone and none had used the computer or the Internet

Once I went on a working tour of recently installed e-library centres in an Asian

country. Many centres there are located in religious centres. We stopped at one site
and, afterwards, shortly after we had started our journey to the next centre, we
stopped. We had arrived. Since these are rural areas I had assumed that it would
take a while before we arrived at the next centre but no. We arrived immediately
because in fact, this other centre was a short distance away, perhaps 200-400 m.

From a technical standpoint two centres so close to each other makes no sense,
since they would serve the same pool of potential users. So I asked my counterpart:
why are these two centres located so close to each other?

His response was telling: because both of the two leaders of these centres wanted an
e-library in their own premises. To avoid conflict, we just gave each of them a
centre.

Here in Malaysia an excellent study by Gnaniah, Yeo and other UNIMAS colleagues
(2004) compared the implementation of e-Bario with that of e-Bedian. The study
illustrates how digital illiteracy combines with the political allure of telecentres to
frustrate implementation and impact.

According to these authors, both projects tried to follow a similar implementation

protocol that had digital literacy training as an important element. e-Bario came first
and was the model they were trying to replicate in e-Bedian. In Bario the community
was properly trained and the centre is being used daily.

In contrast, when the implementation program was barely starting in Bedian, the
Prime Minister visited the site, and short cuts were taken to make sure the telecentre
facilities were ready for his visit. In practice, this meant that the digital literacy
training was postponed.

The researchers noted their disappointment this way:

“Today, after nearly 10 months of setting up the telecentre, the telecentre

has yet to have a manager to manage, and a technician to maintain it like in
Bario. The community have yet to fully receive ICT awareness and literacy
training as not many want to be committed to this. The problem identified is
that they do not see, understand nor appreciate the importance of ICT in their
daily lives.”

You need to know your target group and its context!

  4
II. Experiment

Let’s speculate a bit by examining some proposals that in principle make sense.

Distance support to digital illiteracy training

We know how to teach digital literacy face to face, but face-to-face training is
expensive.

Consider a typical digital-literacy training program. I use Chile’s digital national

literacy campaign to illustrate. Their training program was used in Chile’s network of
over 300 libraries, in part funded by a 10 million dollar grant by the Bill and Melinda
Gates Foundation. Many other institutions implemented similar training. In all, Chile’s
national digital literacy campaign trained 121,000 people in the period 2002-2004 at
an average cost of $284/trainee.

The program comprised 3 modules: a basic module with 5 sessions; a second 5-

session module covering Word Processing and Internet; and a 3rd 4-sesson module
covering email. These 14 sessions would be taught, two one-hour sessions every day
day, over 7 consecutive days. A regional team of 17 trainers would visit libraries and
train local trainers.

Can we reduce the attendant training costs by relying in part on video instruction?

Instead of running through all three modules perhaps we can add a session at the
end of the Basic Module, that teaches users how to find, download, and watch videos
using their computer or their mobile.

Module 1: Basic
1. Getting to know the computer
2. The mouse
3. Operating System Windows XP
4. The Keyboard
5. Word processing basics
Extra: Finding, downloading & watching
videos on computer and on mobile

Modules 2 and 3 would still be taught, but at the learner’s own pace, with assistance
as needed from the telecentre operator.

The use of video is revolutionizing the way we teach and learn; and the potential for
using video for the benefit of marginalized learners is huge.

Sebastian Thrun, the inventor of the driveless car, and Google Glass is also the
Stanford teacher who, together with Peter Norvig, taught the first MOOC and
founded UDACITY one of the prominent MOOC academies. In an interview with

  5
Thomas Friedman of the New York Times, Thrun describes how he was teaching his
AI class to 160,000 online students and about 200 students enrolled in Stanford.

All of a sudden he realized one evening that about 170 of his 200 Stanford students
had stopped going to class. He asked them why:

“You guys are paying $50,000 to see me the famous professor. Why are you
not coming to class.”

They responded:

“You know what,... we prefer you on tape. If we don’t get something we can
rewind you.”
(Friedman 2014)

They liked Thrün more on video because they could learn at their own pace.

Can self-paced video training support face-to-face digital literacy training (i.e.
blended learning) of large masses of people? The time appears ripe for
experimentation with such a program.

There is already some material online, but I suspect localized video instruction that is
suited to the specific needs of your target group is needed; e.g. instruction in the
local language, and using the same or very similar equipment.

Distance learning needs of young adults

Telecentre users are a mixed group, dominated by young people between the ages of
16 and 30. Youngsters who have not completed their formal education and have little
job experience will have different needs than adult learners. The former need a
diploma or certificate they can use to get a job.

This will require experimentation, with content and course development and with
new forms of certification that are reliable and recognized by industry.

Distance learning needs of mature adults

Mature rural public access users who already have a job are more likely self-paced
learners wanting to hone their skills to get a raise or a better job. Online learning
programs can help self-learners improve specific skills of broad importance, such as,
for example: math skills, writing skills, English language skills, computer coding; as
well as courses of practical every day value like

“how to do online banking,” or

“how to search for health information online,”
“how to search for employment opportunities,”
“how to search for and enroll in online courses,

We do not have to start from scratch. There are materials (e.g. in

www.khanacademy.org) that mature learners could use but that is presently
available in English but could be localized into Malay or other local languages.

  6
 Online learning for telecentre staff

Since we are not talking about sophisticated self-learners, some complementary

help, orientation and structure may be required.

Much of this help and structure can be provided by telecentre operators. Further, we
can strengthen telecentre user experience by training operators and telecentre staff.
Some of this training may be done online on:

- how to train digital literacy;

- telecentre management skills;
- English language skills to make online search more powerful;
- basic computer and equipment repair skills.

Distance English language training

The Internet enables users to learn new things at their own pace. This is true
regardless of gender or age. It is not just the ability to tap on existing content and
online media, it is also the ability to communicate with others and find information to
solve ordinary problems (e.g. health advice, how to fix things, how to cook a meal,
how to solve computer problems) by participating in forums.

Internet content is predominantly in English: 55% in mid 2014, compared to 0.3% in

Malay. Russian is the second most important language, accounting for only 6 percent
of all content (http://w3techs.com/technologies/overview/content_language/all).

If we focus on the primary language of Internet users, English is again dominant with
28%, followed by Chinese, 23%, Spanish, 8%, Arabic, 5%, Japanese, 4% and
Portuguese, 4% (http://www.internetworldstats.com/stats7.htm).

There is plenty of room for Internet penetration to expand in China, but it will be a
while before Chinese is widely adopted elsewhere as a second language.

English is also the most widely spoken foreign language in Europe. In 19 of 25

European countries surveyed in 2012, English was the most widely known language
apart from the mother tongue. It is spoken by over 50 percent of the population in
Estonia, Greece, Finland, Germany, Slovenia, Austria, The Netherlands, Sweden,
Denmark, Malta, Cyprus, and Luxembourgh (TNS Opinion & Social 2012).

English is a compulsory subject in most Japanese schools. Singapore has 4 officially

recognized languages, but English is spoken by an estimated 70% of the population.
(www.wikipedia.org)

In countries like Malaysia, the Philippines, and India with large English speaking
populations, the development of English language skills among young telecentre
users could have potentially high economic and social payoff.

These ideas are of course tentative. Before proceeding with implementation, one
would want to: i. take a close look at specific conditions affecting the behavior and
the aspirations of the target group of marginal learners; ii. take into account local
policy considerations; and start testing them on a small pilot scale.

  7
III. Evaluate

It is remarkable how poorly documented telecentre experiences have been. There

are many documents but many are advocacy white papers.

Some research looks into specific aspects, but there are very few reliable
independent evaluations; and even fewer randomized experiments.

Some experiences have been documented well.

- The failed experience of the Sari project in India has also been well
documented.

- The successful e-Bario telecentre experience in Sarawak is also well

documented and there is an excellent paper done in 2004 comparing this
experience with the e-Bedian less successful initiative.

It is no coincidence that university researchers have played an important role in

these initiatives.

There is, however, an inherent conflict between the need to evaluate online learning
experiments and the political allure of rural telecentres.

Politicians and government agencies love to inaugurate telecentres, but tend to

forget them soon afterwards. The problem is compounded by the real possibility that
political opponents may use evaluations to their advantage and to embarrass
government.

Consider digital literacy training.

We like to think that once digital literacy training has been imparted trainees go on
to use computers and the Internet with increasing levels of sophistication. Our
perception is reinforced by our own experience and predominantly urban
surroundings.

In practice, the opportunities to learn and use ICT in rural areas are limited.

And not all digital-literacy training is equally effective.

In 2002-2004 a digital literacy program was implemented in Malappuram, Kerala,

India.

- Kerala’s program (US$20/trainee) was an awareness campaign in which

participants did not touch the computers.

- At around the same time, Chile’s national digital literacy campaign

($208/trainee) focused on teaching computer and Internet use, giving
trainees a chance to practice their new skills in libraries and telecentres.

- Despite its higher cost, Chile’s program appears to have been much more
effective (Proenza 2015).

  8
The outsourcing of both implementation and evaluation to independent agents (e.g.
private enterprise, universities) may help reduce the political sting of evaluations.

Beyond formal evaluations we need to engage marginal online learners in evaluation.

Every country should have a national publicly funded portal dedicated to making
educational materials available to its citizens; with some parts catering to the
specific needs of marginal users. These portals should offer users the opportunity to
comment on and rate the quality and value of the materials.

Some evaluations will be negative. Don’t take it personally. Use the information
gathered to learn and improve training materials.

Some of the most powerful portals derive their power precisely in their ability to
gather user information about the services they provide: amazon, airBnB, eBay.

We must treat e-learners with respect. One way is to make online training content
accountable by engaging e-learners in the evaluation of materials.

A very powerful example of stakeholder engagement in educational evaluation is

www.ratemyprofessors.com. Over 13 million students primarily of US universities
have rated their professors. Five categories are rated using 1-5 likert scale (Bergin et
al. 2012).

Something similar could be done to enable users to evaluate online educational

materials, based on simple criteria like:

- USEFULNESS: does the material address a topic I am interested in.

- CLARITY: is the material clearly presented, easy to follow.

- USER ENGAGEMENT: is the material engaging, attractive, absorbing (as

opposed to boring or dull).

In addition, allow users to comment. Ask them to recommend how to improve the
online training materials.

Sebastian Thrun, the Stanford professor we talked about earlier, has what he calls
the Universal Law of Innovation: Build It, Break it, Improve It.

Thrun’s law of innovation can be brought to bear to the task of providing online
learning for marginalized learners:

If you experiment you are bound to fail, more often than you would hope for
or want to.

But if you don’t experiment you will fail to learn new ways.

And if you don’t evaluate your experiments, you will never learn or you may
learn the wrong lessons.

  9
Let us go over the 10 key points we have made:

1. In most countries, the majority of marginal learners live in rural areas. This
is bound to be your primary target group.

2. For online learning to reach rural marginalized learners, digital literacy

training is an indispensable first step.

3. The design of a video online training program for novice users could help
enhance the computer and Internet skills and facilitate practice by large
numbers of marginalized learners.

4. There is a need to develop formal coursework and forms of accreditation

for online training to serve young rural telecentre users who are about to
enter the workforce.

5. We must identify topics that interest large numbers of marginal self-

learners living in rural areas. This will help us develop, test and perfect a
suitable online learning program.

6. Train telecentre staff, as a gateway to marginal learners. Make them digital

literacy teachers, give them English language skills to make their online
search more powerful, teach them basic computer and equipment repair
skills.

7. The development of programs of online instruction designed to improve the

English language skills of Internet users (beginning with telecentre operators)
could make a significant impact on the ability of rural marginal learners to
learn online and to access valuable content and network.

8. The outsourcing of implementation and evaluation to independent

contractors (e.g. university researchers, private specialists) who respond to a
broadly represented board of stakeholders may help minimize potential
conflicts of interest in the evaluation of online learning experiments.

9. Innovation in online learning for marginalized learners will involve trial and
error. It is this trial and error and continuous evaluation that will let you
innovate and improve your interventions.

10. Make the process of distance learning accountable by engaging users,

including marginal learners, in the evaluation of online learning content.

I once visited MIT’s media lab and at the time they had a wonderful motto: “We build
tools not for people to use them, but to enable people to participate in the creative
process that digital tools make possible.”

I have prepared projects for international agencies for 35 years. I have seen
development fashions come and go. It is tempting to follow the latest fashion, and in
the process forget what is important and make costly mistakes. The fast pace of
change in the new digital world poses significant challenges, but also presents
remarkable opportunities. We need to seize these opportunities and run with them.

  10
We will be able to do so if we know our target group and its context, if we
experiment intelligently, and if we subject our experiments to evaluations - formal
and informal - that engage marginal learners and help us learn and improve our
interventions.

In the final analysis our aim to empower the rural poor. The best definition of a
telecentre I have heard was given years ago by a housemaid in Southern Chile. This
was a poor woman who worked as a domestic housekeeper. When asked what a
telecentre meant to her, she replied:

“It is a place where I forget my pots and pans, and realize my dream of
writing poetry.”
(Garrido, Morales and Villarroel 2002)

We want online learning to give traditionally marginalized people a voice, a set of

tools to innovate, to collaborate with others to solve common problems, to improve
their well being, to become full participants in modern society, to realize their
dreams.

  11
 Sources Cited

Bergin, Adam, Kevan Sharp, Todd A. Gatlin, Adrian Villalta-Cerdas, Austin Gower,
and Santiago Sandi-Urena. 2012. “Use of RateMyProfessors.com Data as a
Supplemental Tool for the Assessment of General Chemistry Instruction.” Journal of
Chemical Education. 90:289-295. http://pubs.acs.org/doi/abs/10.1021/ed300277n

Best, Michael, and Rajendra Kumar. 2008. Sustainability Failures of Rural

Telecenters: Challenges from the Sustainable Access in Rural India (SARI) Project.
Information Technologies and International Development 4 (4): 31–45.

Friedman, Thomas L. 2014. “Democratizing Education.” Thomas Friedman interview

of Sebastian Thrün. New York Times Next New World Conference.
http://www.nytfriedmanforum.com/videos/democratizing-education.aspx

Garrido, Rodrigo, Manuel Morales y Alejandra Villarroel. 2002. “Aportando a la

Disminucion de la Brecha Digital en Comunidades de Bajo Desarrollo Humano de la
Araucanía-El Rol de la Red de Operadores de Telecentros Comunitarios.” Working
Paper.
http://www.virtualeduca.info/encuentros/encuentros/valencia2002/actas2002/actas0
2/513.pdf

Ghaniah, Jayapragas, Alvin Yeo, Peter Songan, Hushairi Zen, and Khairuddin Ab
Hamid. 2004. “A Comparison on the Implementation Approaches for the e-Bario and
e-Bedian Projects.” in Work with Computing Systems, edited by M.G. Helander, A.W.
Yeo. Kuala Lumpur: Damai Sciences.

Proenza, Francisco J. 2008. “Towards High Impact Sustainable Telecenters in Sri

Lanka.” Consultancy report prepared for the World Bank, December 7.

Proenza, Francisco J. (ed.). 2015. Public Access ICT across Cultures: Diversifying
Participation in the Network Society. The MIT Press.
http://mitpress.mit.edu/books/public-access-ict-across-cultures

TNS Opinion & Social. 2012. “Europeans and their Languages.” Special
Eurobarometer No. 386. Survey conducted for the European Commission.
http://ec.europa.eu/public_opinion/archives/ebs/ebs_386_en.pdf

  12