Professional Documents
Culture Documents
Bélanger Et Al., 2015
Bélanger Et Al., 2015
Valérie Bélanger, Anne Vanasse, Diane Parent, Guy Allard & Doris Pellerin
To cite this article: Valérie Bélanger, Anne Vanasse, Diane Parent, Guy Allard & Doris Pellerin
(2015) DELTA: An Integrated Indicator-Based Self-Assessment Tool for the Evaluation of Dairy
Farms Sustainability in Quebec, Canada, Agroecology and Sustainable Food Systems, 39:9,
1022-1046, DOI: 10.1080/21683565.2015.1069775
1022
DELTA: Self-Assessment Tool for Dairy Farm Sustainability 1023
1. INTRODUCTION
Regional or national
level
Public Composite indicator,
indice or single index
Farm to local and
regional level
Policy-maker
Indicator
Field or farm level Farmers
FIGURE 1 Relationships between indicators, users and the level of analysis (adapted from
Braat 1991).
2. METHODOLOGY
2.1. Premises to the Development of DELTA, A Self-Assessment Tool
for Dairy Farms
Methodological aspects that are inherent to the indicators must be taken into
consideration for the construction of a sustainability integrated self-assessment
tool for use by farmers. For Binder, Feola, and Steinberger (2010), it is
necessary to emphasize what such assessment is meant to achieve. Then,
several premises were considered in order to develop the farm-level
1026 V. Bélanger et al.
self-assessment tool named DELTA for dairy farms in Quebec. The premise of
each are discussed herein.
the temporal scale, sustainability includes a notion of time. How long a farm
should be sustainable? It was necessary to define the duration of our
assessment to bring more sense in the process of indicator selection. For
us, a sustainable farm should, at least, still be in activity over the next two
generations. Such a context was described to the experts that participated in
the selection of indicators through the use of participatory processes.
● Target the end-user group: farmers. The identification of end users and
the definition of practical objectives for the indicators were identified as an
essential step by several authors (Mitchell, May, and McDonald 1995; Gir-
ardin, Bockstaller, and van der Werf 1999). As mentioned in the first section
of this article, different end-user groups require different types of indicators
(Figure 1). Here, as this tool is developed for self-assessment, farmers are
first in attempting to use it.
● Use of participatory processes. Since sustainability could be seen as a
social construct, the participation of different actors is essential through a
mix of bottom-up and top-down approaches. If only bottom-up
approaches are used, it can lead to an unequal representation of the
three dimensions of sustainability (Ripoll-Bosch et al. 2012). As such,
several authors agree that the participation and consultation of farmers is
a key element in building and developing indicators (King et al. 2000;
Rigby et al. 2001). For this project, different sets of experts were chosen
for each dimension of sustainability during the selection of indicators.
Expert groups were made of farmers, researchers, and advisors or stake-
holders from government.
Those premises and boundaries are necessary for choosing the most
preferred indicators and their characteristics in a specific context. The
complete methodology to construct and select an indicator set using a
participatory approach is presented in Bélanger et al. (2012). In this
article, the use of participatory processes is fully detailed as it is a central
point to the construction of our tool. A total of 13 indicators were chosen
for the agri-environmental dimension, 8 indicators for the techno-eco-
nomic dimension, (Larochelle 2011) and 20 indicators for the social
dimension. In each dimension of sustainability, indicators are grouped
into components to facilitate comprehension and presentation. Each
dimension is of equal significance, but within it, relative weights have
been given to components and, within the components, to individual
indicators. The choice of reference values for indicators is also explained.
Social issues are central to explain farm sustainability (Ripoll-Bosch et al.
2012). Social aspects have been put aside in many attempts to assess
sustainability (Binder, Feola, and Steinberger 2010). Adding to that, as
mentioned by Gomez-Limòn and Sanchez-Fernandez (2010), the sustain-
ability concept needs to be understood as a social construct, which
changes as a function of society and, thus, needs to be specifically
formulated for any given set of geographical and temporal conditions.
1028 V. Bélanger et al.
The selection of social indicators in our tool and the explanations con-
cerning their construction are forthcoming.
possible explanations and the variability that those indicators can provide in
terms of results. The choice of the four farms was based on their results from
the section on results consistency in the end-user validation. For Flyvbjerg
(2006), one possible strategy for the selection of cases is a selection based on
information-oriented selection. This strategy maximizes the utility of informa-
tion from a small sample. Cases are selected on the basis of expectations
about their information content. Two farms were selected in each of both
regions. Similar methodology using a case study investigation approach has
been employed by Ripoll-Bosch et al. (2012) to achieve a comparative eva-
luation of sustainability in four different sheep farming systems.
A total of 98% of farmers (20% agreed and 78% strongly agreed) agreed
that it was easy to collect the main data on their farms (Table 1). These data
included their agri-environmental fertilization plan and their financial state-
ments. These results corresponded to our objective to develop indicators with
data easily accessible on the farm. It must be noted that in Quebec the agri-
environmental fertilization plan is compulsory for each farm, so it was rela-
tively easy to collect data for the agri-environmental dimension. While the
social data collection was not part of this question, a pretest was used for the
social questionnaire that confirmed its easiness to fill out while not time-
consuming.
The choice of presenting the results through a visual integration using the
radar diagrams seemed to be appreciated by the farmers while 38% agreed
and 56% strongly agreed (Table 1). This farmer easiness to understand radar
diagrams could come from their acquaintance with such tools that are also
used by many dairy farmer groups in Quebec for their management practices.
TABLE 1 Results from the end-user validation
1030
Scores consistency Agri-environmental scores 17 2.8 0.90 53 18 71
Techno-economic scores 17 2.7 0.92 47 18 65
Social results 17 2.8 0.64 59 12 71
Tool utility Presentation of scores using 17 3.4 0.49 65 35 100
tables
Presentation of the regional 17 3.5 0.51 53 47 100
average
Presentation of the leading 17 3.3 0.69 47 41 88
group average
Meeting with advisor to 17 2.7 0.69 47 12 59
interpret results
Follow-up every year or two 16 2.6 0.70 50 6 56
years using an online tool
How the tool can enhance 17 2.9 0.56 71 12 83
the farmer practices
*1 = not important, not satisfied or not at all and 4 = very important, very satisfied or totally.
DELTA: Self-Assessment Tool for Dairy Farm Sustainability 1031
It was then logical to keep this form of diagram to present the data. These
results provide evidence for visual representation rather than aggregated
measures of sustainability.
The last constituent analyzed from the first section of the questionnaire
was the overall importance for farmers regarding the three sustainability
dimensions. Farmers gave more importance to the techno-economic dimen-
sion, followed by the agri-environmental dimension, and to a lower degree
the social dimension. This is different than what Ripoll-Bosch et al. (2012)
reported at the local and the farm level: higher importance was given to the
economic and social dimensions compared to the environmental dimension.
The techno-economic dimension was rated very important by 59% of farmers.
Combining the ranks important and very important, 100% was reached for the
techno-economic dimension, 94% for the agri-environment dimension and
83% for the social dimension. The agri-environmental dimension rated in
second place by the farmers could be explained by the implementation in
2002 of the Agricultural Operations Regulation (REA) in Quebec aiming,
among other things, to strike a better balance between fertilizer applications
and plant requirements. It also aims at ensuring adequate management of
livestock manure while reducing the environmental impact of its use as
fertilizer. The introduction of this regulation was perceived by farmers as an
increased consideration for the environment in agricultural activity. Contex-
tual events explaining farmer reactions have also been considered by Meul,
Nevens, and Reheul (2009). They mentioned that sustainability had a negative
connotation with farmers because this concept was first introduced when
agricultural activity was considered to be a major source of environmental
problems in the Flanders region.
Within the social dimension, some indicators have been mentioned once
or twice by different farmers and because there are many indicators in the
social dimension, it was difficult to identify precise ones. However, this could
underline the fact that for many farmers, it was the first time that they looked
upon the social aspects of their farm sustainability, as was reported by some
farmers in the questionnaire.
Through this end-user validation, we found out that the tool was correctly
designed and that our set of indicators fairly represents the dairy farms reality.
TABLE 2 Scores for each component of the farm case studies with main characteristics for
each farm
Region* 1 or 2 1 1 2 2
Number of cows Nb 63 43 48 51
Land area Ha 157.7 226.4 127.5 135.4
Importance devoted to agri- 1 to 4 3 4 3 3
environmental dimension**
Importance devoted to techno- 1 to 4 4 4 3 3
economic dimension**
Importance devoted to social 1 to 4 3 3 3 2
dimension**
Agri-environment 100 65.2 78.1 90.1 95.6
Soil quality 20 17.9 16.8 18.4 20.0
Cropping practices 30 14.9 19.1 30.0 30.0
Fertilization management 30 19.5 22.3 28.7 26.6
Farmland management 20 13.0 20.0 13.0 19.0
Techno-economic 100 92.0 95.3 60.4 66.1
Technical management 20 20.0 15.3 11.2 16.5
Economic viability 25 20.8 25.0 23.0 11.0
Expense control 30 26.2 30.0 14.2 16.7
Labor efficiency 15 15.0 15.0 1.9 12.4
Forage self-sufficiency 10 10.0 10.0 10.0 9.5
Social 100 74.0 65.0 56.0 67.0
Quality of life 25 13.0 19.0 15.0 16.0
Social integration 15 9.0 9.0 8.0 11.0
Farm succession 30 25.0 14.0 17.0 23.0
Entrepreneurship 30 27.0 23.0 16.0 17.0
*1 = Monteregie region and 2 = Bas-St-Laurent region.
**1 = not important and 4 = very important.
DELTA: Self-Assessment Tool for Dairy Farm Sustainability 1035
farms A and B, a larger gap was observed in cropping practices and fertiliza-
tion management. This could be explained by the context of the regions.
Monteregie region (1) has a longer growing season than Bas St-Laurent region
(2) and is situated near the urban center of Montreal. With the longer growing
season than farms C and D, farms A and B have the possibility to grow corn
and by extension, grow less perennial forage species. This difference could
explain the lower scores obtained for cropping practices and fertilization
management components. For the first three indicators in the cropping prac-
tice components, scores were lower for farms A and B (Table 3). If farms A
and B are growing corn on a large area, it carries at the same time smaller
areas for perennial crops that experts have classified as essential on a dairy
farm for sustainability. Also, corn production limits the choice for different soil
tillage practices under our climatic conditions where corn harvesting is done
in late autumn making it impossible to seed green manures into those fields.
In the fertilization management component, the manure management indica-
tor can be understood in the same way since late harvest does not allow
spreading manure in those fields without environmental risks. Another obser-
vation was the possible synergy existing between the techno-economic
dimension and the component of entrepreneurship in the social dimension.
Both farms A and B had the highest score for this component. It seemed to be
profitable to have some characteristics of an entrepreneur such as a good
education, human resource management on the farm and entrepreneurial
abilities.
The reverse relation was observed for farms C and D in region 2, the Bas-
St-Laurent region, East of Quebec city and where fewer crops are possible due
to climatic conditions. For farms C and D, lower scores were obtained for the
techno-economic dimension instead of the agri-environmental dimension.
The larger gap was found in the expense control component. Despite that
the scores for this component were similar in farms C and D (Table 2), large
differences were observed in the individual indicators for this component
(Table 3) and it is in accord with Ripoll-Bosch et al. (2012) who mentioned
that similar average scores for sustainability attributes can also hide large
differences within individual indicators. Farm C had a low score for the
operational expense on gross income and this, in turn, could be related to
higher purchases of concentrates as expected from a low milk from forage
indicator. For farm D, machinery expenses per hectoliter was one indicator
with a low score and could probably be linked with the low security margin
indicator as the security margin is calculated using all expenses and that the
machinery expenses are part of it.
For the social dimension, results of the four farms were different. Even if
it was not the goal of these case studies, comparisons helped to understand
the tool and how it could be useful for farmers to see their farm through the
scores. The social integration component score was similar for the four farms.
There was also room for improvement in the quality of life component for the
1036 V. Bélanger et al.
Soil quality Organic matter content 12.0 10.4 9.4 11.0 12.0
Phosphorus soil saturation 8.0 7.4 7.3 7.4 8.0
................................................................................................................................................................
Cropping practices Perennial crops 16.0 9.0 9.8 16.0 16.0
Annual crops: soil tillage practices 4.0 0.7 2.2 4.0 4.0
Annual crops: green manure 3.0 1.1 0.1 3.0 3.0
Annual crops: crop rotations 4.0 3.0 4.0 4.0 4.0
Annual crops: integrated pest 3.0 1.0 3.0 3.0 3.0
management
................................................................................................................................................................
Fertilization Manure storage structure 3.0 0.0 3.0 3.0 2.0
management Manure management 9.0 3.4 5.3 8.6 7.6
Phosphorus balance 9.0 7.8 6.7 8.1 9.0
Nitrogen balance 9.0 8.3 7.3 9.0 7.9
................................................................................................................................................................
Farmland Watercourse protection 6.0 6.0 6.0 6.0 6.0
management Land drainage 4.0 4.0 4.0 4.0 4.0
Windbreaks 4.0 0.0 4.0 0.0 3.0
Field slope 3.0 3.0 3.0 3.0 3.0
On-farm woodlot 3.0 0.0 3.0 0.0 3.0
Quality of life Work and workload 2.0 0.0 2.0 0.0 0.0
Holidays 2.0 0.0 1.0 0.0 2.0
Satisfaction 7.0 3.0 4.0 5.0 5.0
Social support 4.0 3.0 3.0 3.0 4.0
Health and stress 5.0 2.0 5.0 2.0 2.0
Social and professional 5.0 5.0 4.0 5.0 3.0
relationships
................................................................................................................................................................
Social integration Contribution in local services 2.0 0.0 0.0 1.0 0.0
Agricultural neighborhood 4.0 4.0 4.0 2.0 4.0
Cohabitation 6.0 4.0 4.0 4.0 6.0
Social contribution 1.0 0.0 1.0 1.0 1.0
Regional presence of agriculture 2.0 1.0 0.0 0.0 0.0
(Continued )
DELTA: Self-Assessment Tool for Dairy Farm Sustainability 1037
TABLE 3 (Continued)
four farms (Table 2). When we looked at the individual indicators level, only
one farm received points for the work and workload indicator. For the social
support indicator and the social and professional relationships indicator, all
farms obtained an acceptable score.
From the examination of these case studies, another important point was
revealed for the farm succession component. Farms A and D had higher overall
general scores (Table 2) for the farm succession component. At the indicator
level (Table 3), we observed that those higher scores were linked to a better
score for the preparation for retirement indicator. For the four indicators in this
component (Table 3), there were three of them that must have both generations
on a farm to have the full score for the indicator. Calculations of these three
indicators at the moment cannot correctly evaluate this consideration equally
between farms with both generation and farms without having already a
succession plan. Although we expect an older farmer who does not know if
he will have someone to take over the farm scores low, we cannot penalize a
young farmer starting up for not having a lot of preparation for his retirement.
So calculations of these three indicators must be reviewed. As another example,
one variable in the calculation of the farm succession integration indicator is the
percentage of assets of the farm that is transmitted today. It would be better to
calculate the percentage of the farm assets owned by each generation: the
parents, the current generation, the children and others if applicable. It is
noticeable as well that such information would not be available if aggregated
measures were used leaving out the possibility for improvement of the tool. As
Ikerd (2006) mentioned, better indicators and measures of sustainable farming
should at least improve the process of exploration.
These case studies brought the notion of individual results versus aggre-
gated results in regard of farmers’ needs to better understand the sustainability of
their farm. The four case studies and their scores by components and by
1038 V. Bélanger et al.
Forage self-sufficiency
Economic viability (25)
(10)
Labor efficiency (15) Expense control (30)
b)
Production per cow
(8)
100
Forage self- 80 Milk from forages
sufficiency (10) (12)
60
40
20
Milk per worker (15) 0 Security margin (15)
Machinery expenses
Debt per hL (10)
per hL (10)
Operational expense
on gross income (20)
FIGURE 2 Radar diagrams presenting the results of the Farm A and the results of the leading
group for a) the component scores of the three dimensions of sustainability and b) the indicator
scores for the techno-economic dimension. Numbers in parentheses in the diagrams are the
maximum possible scores for the component (a) or for the indicator (b).
having the same low scores as of farm A. It can be a general tendency and
farmers have to work on these components. Again in Figure 2a, farmer A can
see good scores in the techno-economic dimension within the components:
technical management, expense control, labor efficiency and forage self-
sufficiency but had a lower score than the leading group in only one compo-
nent: economic viability. In Figure 2b, the radar presents the results from farm
1040 V. Bélanger et al.
A but for all individual indicators of the techno-economic dimension. Then the
economic viability component is viewed through the two indicators compris-
ing it: debt per hectoliter and operational expense on gross income. Farmer A
can see which one had the lowest score and in this case, it was the indicator
debt per hectolitre (Figure 2b). Even if radar diagrams do not reflect all
interrelations between the subsystems, they allow for the identification of
hotspots at the farm level (von wirén-Lehr 2001).
It is also possible to see that maximum score has been reached for some
components (Figure 2a) and for some indicators (Figure 2b). Achieving max-
imum score can be very attractive for farmers but it could also raise some
questions. For example, in Figure 2a, the maximum score for technical man-
agement was attained but the score for its cropping practices was lower than
the one for the leading group. Since cropping practices had a higher relative
weight than technical management, this could mean that farmer A was putting
more effort on the dairy herd than in the fields. Such examples should be
viewed as eye opening for farmers.
Another positive point of a visual integration is the shape of the area under
the curve. With a rapid overview, if the curve is shaped like a star on the radar, it
means that the farm is not in equilibrium. In agricultural management, to get
good economic results, it is better to be average in all subsystems than to be
high in one and low elsewhere (Levallois and Pellerin 2008). Maximizing one
component can come at the expense of the resilience for the overall system,
which in turn reduces overall sustainability (National Research Council 2010).
The resilience is defined by the capacity of a system to buffer shocks and
stresses (Pretty 2008). Häni (2006) mentioned that an optimal farm sustainabil-
ity situation will not be achieved by one individual maximum indicator but
more by a balance between the scores for all indicators. If we look back at
Figure 2a, farm A had a radar diagram with an acceptable shape (not a sharp star
shape) and moreover, the shape followed the one for the leading group mean-
ing that farm A may be on the right path to achieve more sustainable practices.
But it also enabled farm A to identify which components have a lower score and
the need to start working on them to improve farm sustainability. As Ikerd
(2006) mentioned, no objective quantifiable means exist for ensuring balance
and harmony among the economic, environmental and social dimensions of
sustainable farming and the three dimensions are inherently interrelated;
increasing one while the two others decrease is not good.
friendliness have been central throughout the development of our tool. The
visual integration leaves freedom to farmers to view and understand the tradeoffs
between subsystems and decide what could be done to attain a more sustainable
farm. Without aggregating indicators into one single index, choices are clearer
since they have all the results in hand. Reflections can be made to set the main
priority between the three dimensions of sustainability. According to the results
from the end-user validation, the techno-economic dimension was the most
important for dairy farmers. It is also reported in the literature that economic
viability is often a precondition for several aspects of the social dimension as well
(Van Cauwerbergh, et al. 2007). In our view, in agreement with Bennett, Pang-
born, and Bywater (2010), the priority will refer to the internal farm context.
Farmers perceive sustainability as a division between internal factors they can
control and external factors that are difficult to control. As one farmer mentioned
during his interview, some choices can be made to enhance life quality without
being optimal economically. This demonstrates the intrinsic tradeoffs that occur
on each farm when decision-making has to be done to enhance farm resilience.
Sustainability in agricultural systems incorporates concepts of both resilience and
persistence (Pretty 2008). Bruges and Smith (2008) mentioned that the transition
towards a more sustainable agriculture is seen as a function of learning about
economic, social, and ecological complexities and uncertainties.
A sustainable farm
Liveable
Community
FIGURE 3 New schematization of the sustainable farm (adapted from Landais 1998; Parent
2001).
and classify indicators and suggest to farmers the most important challenges to face
cannot replace the knowledge of the farmers of their own farm as well as their
awareness of what they will be able to do to recover from a difficult situation. By
the observation of their own scores in tables and radar diagrams, farmers will know
and set their priorities to enhance their farms. Some researchers work on resilience
to understand the dynamics of a farming system and to manage farm sustainability
(Darnhofer, Fairweather, and Moller 2010). This work must continue to better
integrate the concept of resilience in the sustainability assessment at the farm level.
4. CONCLUSION
This article outlines the construction and the application of DELTA, an integrated
indicator-based self-assessment tool for the evaluation of dairy farm sustainabil-
ity in Quebec, Canada. While several premises were presented in order to justify
our methodological decisions made for DELTA, the end-user validation results
enabled us to conclude that our indicators were consistent with the dairy farm
reality. Moreover, it confirmed that the presentation of all individual indicator
scores was necessary for the farmer’s understanding. The visual integration
using the radar diagrams was also part of this understanding. According to our
results from the end-user validation, the techno-economic dimension was the
most important for dairy farmers. Reflections have to continue on the main
priority among the three dimensions of sustainability in order to better define
the place of the social dimension. The presentation of four case studies facili-
tated the identification of major tradeoffs occurring in farm management to
further enhance farm resilience. However, the work on the link between resi-
lience and sustainability is still challenging for the operationalization of resilience
DELTA: Self-Assessment Tool for Dairy Farm Sustainability 1043
ACKNOWLEDGMENTS
The authors especially thank the farmers who decided to participate in the
follow up of this research project by filling out the questionnaire used for the
validation of the tool. This work has not been published elsewhere and has
not been submitted for publication elsewhere.
FUNDING
REFERENCES