Professional Documents
Culture Documents
Evaluating The Process of Triple Bottom Line Repor
Evaluating The Process of Triple Bottom Line Repor
net/publication/238399986
CITATIONS READS
40 3,965
3 authors:
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Evaluation of the Wimmera Catchment Management Authority's Regional Delivery Program View project
All content following this page was uploaded by Michael Mitchell on 28 May 2014.
This is the author's version of the work. It is posted here by permission of Taylor & Francis
for personal use, not for redistribution. The definitive version was published in Local
Environment, Volume 13 Issue 2, March 2008. doi:10.1080/13549830701581937
(http://dx.doi.org/10.1080/13549830701581937)
ABSTRACT This paper presents an evaluation framework that aims to improve the
outcomes of triple bottom line (TBL) reporting. Currently, most assessments of TBL reporting
focus on report content. Our view is that effective evaluation of TBL reporting should go
beyond reports as the outputs of a process and focus on what is being learned through the
process, the capacity- and relationship-building that might eventuate, and how this impacts
on the development of sustainability. To enable more effective reflection of TBL reporting
processes, we developed a set of evaluation criteria focussed on the process of reporting.
These criteria were identified from case studies of social, environmental and/or sustainability
reporting processes documented in the literature. A large number of potential criteria were
synthesised into three desired outcomes of TBL reporting and ten prompts for discussion and
reflection during evaluation. This evaluation framework should assist those dedicated to
challenging assumptions that inhibit progress towards a sustainable future for society.
Introduction
The triple bottom line (TBL) is a simple and increasingly popular way to organise thoughts
and action on sustainability. The term was created by Elkington to emphasise that
sustainability has three dimensions: “economic prosperity, environmental quality and – the
element which business had preferred to overlook – social justice” (Elkington, 1998, p. 70). It
emphasises that the pursuit of sustainable development is more than just reconciling potential
conflicts between economic growth and ecological sustainability; there is a social dimension
to sustainability.
1
any harm resulting from their activities and to create economic, social and environmental
value” (SustainAbility Ltd., n.d.). In practice, however, the TBL is generally associated with
its narrower definition “as a framework for measuring and reporting corporate performance
against economic, social and environmental parameters” (ibid.). This reporting application of
the TBL concept builds on Elkington’s premise that it is possible “to measure progress
against the triple bottom line” and that measurement is necessary because “what you can’t
measure you are likely to find hard to manage” (Elkington, 1998, p. 72). However, the
reduction of the TBL to a “decision algorithm” mechanism has been criticised because “there
needs to be a focus on things that count, not the things that can be counted” (Vanclay, 2003,
p. 74).
Organisations have started using the TBL as a format for their public reporting systems 2 .
Shell was one of the first companies to do so, hiring SustainAbility to develop the tools to
help it account for its progress against the TBL (Shell International, 1998). A central task for
organisations that adopt TBL reporting 3 is the identification and presentation of indicators
that track the organisation’s progress on issues related to sustainability (Global Reporting
Initiative, 2006). A whole industry and science is now evolving to assist organisations in this
task. By incorporating this task with the organisation’s regular reporting mechanisms, the
potential exists for TBL reporting to evolve into an ongoing and iterative cycle through which
organisations and their stakeholders can learn, improve their relationships and take
progressive action to enhance sustainability outcomes.
In spite of this unrelenting tendency for organisations to use reporting for public
impressions management, we believe it should be possible to look beyond what is presented
in reports, and evaluate the impact on those involved in the process of reporting. The critical
role of learning and public participation in the development of sustainability is widely
recognised (e.g. Scott & Gough, 2003; Meppem & Gill, 1998; Buckingham-Hatfield &
Percy, 1999). Yet most evaluation of social and environmental reporting has focused on
report content, exacerbating a preoccupation with how reports are to be presented and which
performance indicators to make public. Such a preoccupation is likely to deflect attention
away from the reflection and learning required to enhance sustainability, and the use of
indicators to nurture greater public engagement. We therefore argue that, to increase the
2
potential for TBL reporting to lead to change, it might be more effective to focus evaluation
on the ongoing and cyclical process of reporting: the capacity- and relationship-building that
can occur, and how this in turn might impact on TBL outcomes.
One approach to theorising the link between reporting process and outcomes has been to
see reporting as a learning process through which hidden assumptions that govern prevailing
discourses within an organisation and among its external stakeholders can be revealed and
confronted (Thomson & Bebbington, 2005). The inability to uncover hidden assumptions can
be one of the main constraints on change, especially when these assumptions are an inherent
part of corporate responses to such a complex and conflict-ridden concept as sustainability
(Milne et al., 2006). Thomson and Bebbington (2005) advocate a collaborative educative
process where an organisation and its stakeholders work together to reveal the hidden
assumptions that constrain efforts to progress towards sustainability. This approach is
consistent with organisational learning theory (Argyris & Schön, 1996) and provides a useful
framework to analyse the link between reporting and learning (Gond & Herrbach, 2006;
Berthoin Antal & Sobczak, 2004).
Another approach was used to examine how organisations changed after they adopted
environmental accounting initiatives (Larrinaga-González & Bebbington, 2001; Larrinaga-
González et al., 2001; Ball, 2005). Looking for evidence of change, researchers found that,
for the most part, such initiatives did not result in substantive organisational change, but
rather “institutional appropriation”, meaning that “the radical intent behind environmental
accounting [was] lost, or swallowed up by corporate hegemony, with the result that the issues
which environmental accounting [sought] to address [remained] unaddressed” (Larrinaga-
González & Bebbington, 2001, p. 270). However, Ball’s (2007) study suggested that the
potential for organisational change may be strengthened when internal stakeholders align
their aspirations with those of external social movements committed to environmentalism
and/or social justice.
3
Methods
The primary methodological tasks were to collate and synthesise the conclusions of
researchers such as those introduced above who had studied and/or engaged in social and/or
environmental reporting activities by organisations and to reformulate them into a set of
evaluation criteria. Through this process we were able to identify the key attributes of the
reporting processes that contributed to successful as well as adverse outcomes, and then
reduce the large number of attributes into a manageable set for practitioners to employ. Four
steps were involved:
1. Literature search
2. Selection of case studies for detailed review
3. Structured reviews
4. Synthesis of criteria – desired outcomes and prompts for reflection
Literature search
As a starting point, word searches of electronic databases were used (e.g. “triple bottom line”,
“sustainability” and/or “reporting”) to identify examples of published research of case studies
related to social, environmental and/or sustainability reporting. Clearly, this did not provide
all the relevant published research available due to the wide array of terms used to refer to
this kind of reporting. Many were found through subsequent searches of literature cited in the
initial case studies identified. The number of studies gradually increased to over 150 through
this constant snowballing process. However, very few of these publications involved any in-
depth analysis of a reporting process based on case study field engagement.
In selecting case studies for in-depth review, we gave priority to those that were: (1) studies
of reporting processes in which the researchers were actively involved and used, for example,
participant observation and/or action research methods; (2) analyses of reporting processes,
not just descriptions; and (3) based on an in-depth analysis of one or two organisations as
opposed to general surveys of organisations. The 21 case studies selected for in-depth review
covered a range of organisational contexts: ten in the private sector and five in the public
sector, including four from local government. The six other contexts were a non-
governmental organisation, two community-based case studies and three multi-organisational
contexts involving natural resource management planning or monitoring.
Structured reviews
In-depth reviews were structured around four topics: (1) background details; (2) reporting
processes, including the ways in which stakeholders were engaged and through which
sustainability indicators were developed; (3) outcomes of these processes; and (4)
conclusions leading to potential evaluation criteria. This structure enabled us to identify
potential evaluation criteria embedded in the author’s analysis which were not necessarily
highlighted in their conclusions. Our case study reviews were e-mailed to authors to give
them the opportunity to verify the evaluation criteria we had identified from their work and to
seek advice of additional case studies to include.
4
Synthesis of criteria – desired outcomes and prompts for reflection
We identified almost 200 potential evaluation criteria from the case studies. The process of
synthesising this large set of criteria was undertaken in stages, allowing us to iteratively
refine our synthesis and develop an evaluation framework as the research proceeded. The first
attempt at synthesis was undertaken after nine reviews had been completed. The criteria
identified from these reviews were grouped according to common themes. Twenty themes
were identified at this first attempt, and we then progressively reduced this number to a more
useful list of ten (Table 1). A database was then created to match each of the ten synthesised
criteria with their original sourced criteria. This was used to check that the ten criteria
provided maximum coverage of all the original sourced criteria and to minimise overlap
between the ten synthesised criteria. As the thematic analysis progressed, a logical structure
became apparent in that the criteria were grouped around three questions that would facilitate
practical reflection of reporting processes:
Drawing on the literature reviewed, we identified three desired outcomes of TBL reporting
that corresponded to these questions (Table 2):
a. The process is collectively ‘owned’ by those with responsibility for the organisation, the
staff they employ and other external stakeholders.
b. The organisation and stakeholders consider all sustainability issues across the TBL and
identify any unsustainable practices that may require action to remedy.
c. The process is one of ongoing dialogue and reflection resulting in actions that make a
difference, potentially challenging business-as-usual.
The final step was to restate the ten criteria as questions to be employed by practitioners
reviewing a TBL reporting process.
A term capturing many of the criteria related to stakeholder involvement was “ownership”, a
term specifically used by Bell and Morse (2004, p. 13); Gahin et al. (2003, p. 664); and Potts
(2004, p. 23). Not only should managers feel committed to the goals of the reporting process,
but, more importantly, they need to empower their internal and external stakeholders to have
influence so that the outcomes of the process are shaped and owned by all (O’Dwyer, 2005;
Thomson & Bebbington, 2005).
5
Evaluation criterion 1: all stakeholders involved
To evaluate progress towards the first desired outcome, it is necessary to constantly review
whether all stakeholders who should be involved are involved – an issue that came up in
many of the case studies (e.g. Adams & McNicholas, 2007; Bell & Morse, 2004). Making
decisions about who should be involved is context-specific; these decisions need to be made
by the organisation together with its stakeholders (Adams, 2004).
The need for stakeholder input into the reporting process was consistently highlighted.
Empowering stakeholders to be heard extends from facilitating their input into decisions
about what sustainability indicators are used to track performance (Bell & Morse, 2004;
Eckerberg & Mineur, 2003), through to decisions about what the organisation should do to
improve its sustainability performance (Adams & McNicholas, 2007; O’Dwyer, 2005). The
experience in one case study suggests that some stakeholders may need to become better
informed about sustainability issues so that their focus is not solely directed at the financial
bottom line (Ogden & Clarke, 2005). However, as Thomson and Bebbington (2005)
emphasise, everybody involved in TBL reporting needs to develop a deeper awareness about
sustainability, and this is best achieved through dialogue that reveals and critically examines
taken-for-granted assumptions about current societal arrangements.
Organisations need to consider the extent that the TBL reporting process enables those
involved to discuss and reflect on the organisation’s broader impacts. Some researchers
emphasised that this means specifically thinking about socio-economic issues (e.g. Potts,
6
2006), often requiring organisations to focus on their contributions to a sustainable society, as
opposed to merely sustaining their business (Milne et al., 2004). Researchers also emphasised
how important it is to move beyond reporting on issues already covered by government
regulations (Larrinaga-González & Bebbington, 2001; Veleva et al., 2003).
An oft-quoted goal of sustainable development is to seek to improve the quality of life for all
people throughout the world today without jeopardising the ability for future generations to
do the same given the world’s finite resources (World Commission on Environment and
Development, 1987). It follows that an inherent aspect of sustainability is distributive justice,
an issue that can be difficult to confront, and thus warrants specific emphasis (Milne et al.,
2004; Ogden & Clarke, 2005; Eckerberg & Mineur, 2003).
Many of the case studies highlight the benefits of developing sustainability indicators in
discussion with stakeholders and by reflecting on what the sustainability issues are for those
in the local context (e.g. Eckerberg & Mineur, 2003). Focusing on the local context does not,
of course, mean that global issues are ignored. Indeed, global issues are frequently those of
greatest concern to those in the local context. The emphasis is on developing and presenting
indicators that track progress towards sustainability goals meaningful to all involved so that
practical actions can be proposed and undertaken (Bell & Morse, 2004; Brown & Gray, 2003;
Potts, 2006; Rogers, 2005). Sustainability indicator sets designed for universal application
(e.g. Global Reporting Initiative, 2006) should only be used as a guide, rather than in a way
that makes those who use them become “passive consumers of someone else’s sustainability
indicators” (Bell & Morse, 2004, p. 13).
Criteria that related to the process to be followed can be drawn together under two themes:
(1) that the process evolves into an ongoing and iterative cycle of learning (Bell & Morse,
2004) with sufficient time for reflection (Ball, 2005); and (2) that it leads to practical actions
that will often challenge business-as-usual operations (Milne et al., 2004; Tregidga & Milne,
2006).
Organisations need to avoid the tendency to use reporting for public impressions
management, but instead use it as a tool to engage stakeholders in deciding what actions to
take to enhance sustainability (Livesey & Kearins, 2002; Milne et al., 2004; Thomson &
Bebbington, 2005). To meet the challenge of how to integrate environmental, social and
economic aspects into decision-making, the focus for discussion and reflection should be on
finding solutions to problems (Gahin et al., 2003; Thomson & Bebbington, 2005). As part of
the problem-based focus, participants need to explore the trade-offs that might result from
7
actions undertaken or, indeed, what trade-offs are implicit if no action is taken (Livesey &
Kearins, 2002; Tregidga & Milne, 2006).
Organisations should not think that the publication of a TBL report is the end point, but rather
see it as the start of a further process of dialogue and reflection that will lead to future actions
(Bell & Morse, 2004; O’Dwyer, 2005). If TBL reporting is to be an ongoing cycle of learning
and action, there must be time for reflection and discussion to regularly re-evaluate
sustainability-related objectives so that action addresses the most important issues not just
those that are the most expedient (Ball, 2005; Gahin et al., 2003).
The final criterion was identified from our review of those case studies that challenged the
boundaries drawn around organisational spheres of influence, and those that criticised the
way in which organisations are able to pass on their external impacts to others (Ball, 2005).
Some aspects of society are clearly unsustainable but are beyond the scope of one
organisation to influence, even if these aspects are critical to the organisation’s future or that
of its stakeholders. There are clear benefits from organisations working together to find ways
to ensure we all operate “within the carrying capacity of supporting ecosystems” (Veleva et
al., 2003, p. 112).
Concluding comments
The evaluation framework provides core criteria for evaluating TBL reporting processes. Our
intention is that these criteria will form the basis for an evaluation process that is both
formative (learning to improve) and participatory. While a discussion on methods for
applying the framework is beyond the scope of this paper, a couple of points need to be
highlighted.
8
First, the framework is intended to be used to evaluate reporting processes that use the
TBL as the core organising concept. We have purposefully built on experiences from a much
broader range of reporting, including social audits, environmental reporting as well as more
traditional styles of annual reports, so that we can build on this wealth of experience.
However, the move towards TBL reporting is a trend that warrants specific evaluation, not
only because of the number of organisations taking it up, but also because of the view that it
is a crucial means to assist the private sector take action on sustainability issues (Elkington,
1998).
Second, this evaluation framework can be used in a broad range of corporate, public
sector and other organisational contexts, as our criteria have been identified as being relevant
across all these contexts. We have intentionally made use of case study analyses critical of
reporting, believing that it is crucial to incorporate criteria from authors sceptical about the
potential for TBL reporting to lead to change.
Our intention is that this evaluation framework will benefit those involved in the
reporting process: those with responsibility for organisational decisions and strategies, the
staff they employ and the organisation’s external stakeholders. Providing an opportunity for
all involved to reflect on the outcomes of TBL reporting processes might increase the
potential for change. Ultimately, our hope is that this evaluation framework will provide a
way forward for those within organisations and among social movements dedicated to
challenging the assumptions that inhibit progress towards a sustainable future for society.
1
The research upon which this paper is based has been financed by the Cooperative Research Centre
for Irrigation Futures whose assistance is gratefully acknowledged.
2
While TBL reporting is a trend mostly adopted in the corporate sector, it has also been taken up in
the public and not-for profit sectors as well, especially by local governments. The TBL concept has
also been used in natural resource management planning, and to analyse national and regional
trends. However, the focus of this paper is on its use by organisations.
3
Reports by organisations that follow a TBL format are often termed ‘sustainability reports’.
However, Gray and Milne (2002) argue that accounting for a “global concept” like sustainability
would require “a detailed and complex analysis of the organisation’s interactions with ecological
9
systems, resources, habitats and societies, and [an interpretation of] this in the light of all other
organisations’ past and present impacts on those same systems” (ibid., p. 5). A TBL report, on the
other hand, is a comparatively more manageable task and a mere approximation of this (Gray &
Milne, 2004). We therefore refer to TBL reporting rather than sustainability reporting.
Reference List
Adams, C. (2004) The ethical, social and environmental reporting-performance portrayal gap,
Accounting, Auditing and Accountability Journal, 17 (5), pp. 731-757.
Adams, C. & McNicholas, P. (2007) Making a difference: Sustainability reporting, accountability and
organisational change, Accounting, Auditing and Accountability Journal, 20 (3), pp. 382-402.
Argyris, C. & Schön, D. (1996) Organizational learning II: Theory, method, and practice (Reading,
MA, Addison-Wesley).
Ball, A. (2005) Environmental accounting and change in UK local government, Accounting, Auditing
and Accountability Journal, 18 (3), pp. 346-373.
Bell, S. & Morse, S. (2004) Experiences with sustainability indicators and stakeholder participation:
A case study relating to a ‘Blue Plan’ project in Malta, Sustainable Development, 12 (1), pp. 1-14.
Berthoin Antal, A. & Sobczak, A. (2004) Beyond CSR: Organisational learning for global
responsibility, Journal of General Management, 30 (2), p. 77-98.
Brown, A. & Gray, I. (2003) Should, can and how might researchers help to integrate social,
economic and environmental considerations into the planning process?, in: B. Pritchard, A.
Curtis, J. Spriggs & R. Le Heron (Eds) Social dimensions of the triple bottom line in rural
Australia (Canberra, Bureau of Rural Sciences).
Deegan, C., Rankin, M. & Tobin, J. (2002) An examination of the corporate social and environmental
disclosures of BHP from 1983-1997: A test of legitimacy theory, Accounting, Auditing and
Accountability Journal, 15 (3), pp. 312-343.
Dowling, J. & Pfeffer, J. (1975) Organizational legitimacy: Social values and organizational
behaviour, Pacific Sociological Review, 18 (1), pp. 122-136.
Eckerberg, K. & Mineur, E. (2003) The use of local sustainability indicators: Case studies in two
Swedish municipalities, Local Environment, 8 (6), pp. 591-614.
Elkington, J. (1998) Cannibals with forks: The triple bottom line of 21st century business (Gabriola
Island, BC Canada, New Society Publishers).
10
Gahin, R., Veleva, V. & Hart, M. (2003) Do indicators help create sustainable communities?, Local
Environment, 8 (6), pp. 661-666.
Global Reporting Initiative (2006) G3 Sustainability Reporting Guidelines 2006 (Amsterdam: Global
Reporting Initiative), <http://www.globalreporting.org>.
Gond, J. & Herrbach, O. (2006) Social reporting as an organisational learning tool? A theoretical
framework, Journal of Business Ethics, 65 (4), pp. 359-371.
Gray, R. & Milne, M. (2004) Reporting on the triple bottom line: Mirages, methods and myths, in: A.
Henriques & J. Richardson (Eds) The triple bottom line, does it all add up? Assessing the
sustainability of business and CSR (London, Earthscan).
Livesey, S. & Kearins, K. (2002) Transparent and caring corporations? A study of sustainability
reports by The Body Shop and Royal Dutch/Shell, Organization and Environment, 15 (3), pp.
233-258.
Meppem, T. & Gill, R. (1998) Planning for sustainability as a learning concept, Ecological
Economics, 26 (2), pp. 121-137.
Milne, M., Kearins, K. & Walton, S. (2006) Creating adventures in Wonderland: The journey
metaphor and environmental sustainability, Organization, 13 (6), 801-839..
Milne, M., Tregidga, H. & Walton, S. (2004) Playing with magic lanterns: The New Zealand Business
Council for Sustainable Development and corporate triple bottom line reporting, paper for the
15th International Conference on Social and Environmental Accounting Research (3rd
Australasian CSEAR conference), <http://www.accg.mq.edu.au/CSEAR/pdf/Milne.pdf>
(Accessed 21 June 2005).
Neu, D., Warsame, H. & Pedwell, K. (1998) Managing public impressions: Environmental disclosures
in annual reports, Accounting, Organizations and Society, 23 (3), pp. 265-282.
O’Dwyer, B. (2005) The construction of a social account: A case study in an overseas aid agency,
Accounting, Organizations and Society, 30 (3), pp. 279-296.
Ogden, S. & Clarke, J. (2005) Customer disclosures, impression management and the construction of
legitimacy: Corporate reports in the UK privatised water industry, Accounting, Auditing and
Accountability Journal, 18 (3), pp. 313-345.
11
Potts, T. (2004) Triple bottom line reporting: A tool for measuring, communicating and facilitating
change in local communities, in: H. Cheney, E. Katz, & F. Solomon (Eds) Sustainability and
social science: Round table proceedings (Melbourne: Institute for Sustainable Futures and CSIRO
Minerals), <http://www.minerals.csiro.au/sd/pubs/Potts_Final.pdf> (Accessed 4 May 2005).
Potts, T. (2006) A framework for the analysis of sustainability indicator systems in fisheries, Ocean &
Coastal Management, 49 (5-6), pp. 259-280.
Rogers, M. (2005) Social sustainability and the art of engagement: The Small Towns: Big Picture
experience, Local Environment, 10 (2), pp. 109-124.
Scott, W. & Gough, S. (2003) Sustainable development and learning: Framing the issues (London,
Routledge).
Shell International (1998) The Shell Report 1998: Profits and principles - does there have to be a
choice?, <http://www.shell.com/shellreport/> (Accessed 30 September 2006).
Thomson, I. & Bebbington, J. (2005) Social and environmental reporting in the UK: A pedagogic
evaluation, Critical Perspectives on Accounting, 16 (5), pp. 507-533.
Vanclay, F. (2003) Experiences from the field of social impact assessment: Where do TBL, EIA and
SIA fit in relation to each other?, in: B. Pritchard, A. Curtis, J. Spriggs & R. Le Heron (Eds)
Social dimensions of the triple bottom line in rural Australia (Canberra, Bureau of Rural
Sciences).
Veleva, V., Hart, M., Greiner, T. & Crumbley, C. (2003) Indicators for measuring sustainability: A
case study of the pharmaceutical industry, Benchmarking: An International Journal, 10 (2), pp.
107-119.
World Commission on Environment and Development (1987) Our common future (The Brundtland
Report) (Oxford, Oxford University Press).
Zadek, S. (2001) The civil corporation: The new economy of corporate citizenship (Sterling, Va,
Earthscan).
12
Table 1: Synthesised criteria for evaluating TBL reporting and their sources
Case study references
Corporate reporting (10) Public sector reporting (5) Other (6)
Bebbington, 2001
Milne et al., 2004
O’Dwyer, 2005
Rogers, 2005
Adams, 2004
Milne, 2006
Larrinaga &
Thomson &
Potts, 2004
Potts, 2006
Tregidga &
Ball, 2005
Ball, 2007
Evaluation
criteria for
2002
2007
2003
TBL reporting:
1. all stakeholders
involved X X X X X
2. stakeholders
have a voice X X X X X X X X X X X X X X X X X X X X
3. leaders
committed X X X X X X X X X X X X X
4. impacts across
the TBL X X X X X X X X X X X
5. social and equity
issues included X X X X X X X X
6. locally relevant
indicators X X X X X X X X X X X
7. solutions to
problems X X X X X X X X X X X X X X X
8. time for
reflection X X X X X X X X X X X
9. organisational
changes X X X X X X X X X X X X X X X X
10. collaboration
with others X X X X X X X
13
View publication stats
Table 2: Framework for evaluating processes of triple bottom line (TBL) reporting
Things to think about Desired outcome
6. To what extent is the process of identifying TBL issues and indicators focussed on identifying unsustainable practices in the
local context; to what extent are the resulting indicators tested for their completeness with other relevant TBL indicator sets,
and presented so that they can be easily understood by all interested parties?
14