Journal of Cleaner Production 294 (2021) 126322

Contents lists available at ScienceDirect

Journal of Cleaner Production

journal homepage: www.elsevier.com/locate/jclepro

Education for Sustainable Development in Spanish engineering

degrees. Case study

nchez-Carracedo a, *, Ba
Fermín Sa
rbara Sureda a, Francisco Manuel Moreno-Pino b,
Daniel Romero-Portillo c
a
University Research Institute for Sustainability Science and Technology, Universitat Polit
ecnica de Catalunya, UPC-BarcelonaTech, Barcelona, Spain
b
Universidad de Ca
diz, Ca

diz, Spain
c
Universidad Pablo de Olavide, Sevilla, Spain

a r t i c l e i n f o a b s t r a c t

Article history: The purpose of this work is to present a methodology for analyzing the perception of engineering stu-
Received 28 July 2020 dents about the sustainability learning they have achieved during their university studies. This study has
Received in revised form been carried out in the context of the Spanish university system. Specifically, the opinion of 3364 stu-
26 January 2021
dents from nine engineering degrees taught at three universities is analyzed. In the framework of the
Accepted 7 February 2021
Available online 11 February 2021
EDINSOST project, a questionnaire of 34 questions related to four sustainability competencies has been
designed and validated. To measure the learning declared by the students, composite indicators have
Handling editor: Dr Sandra Caeiro been constructed and validated using factor analysis. The results show that students consider that they
have achieved only two thirds of the sustainability competencies they should have on completion of their
Keywords: studies (66.3%). In the nine degrees, the worst results are obtained for the competency “application of
Sustainability learning ethical principles related to the values of sustainability”, in which the fourth-year students declare
Sustainability competencies having achieved, on average, only 53% of the expected learning outcomes. These results clearly show that
Education for sustainable development engineering degrees should devote more time and effort to include Education for Sustainable Devel-
Sustainability in engineering degrees
opment in the curriculum. The methodology proposed in this work can be easily replicated in other
Student’s perceptions
contexts and in other countries.
© 2021 Elsevier Ltd. All rights reserved.

1. Introduction by the overexploitation of forests had a huge impact on the mining

Global civilization has important challenges to face in order to
secure the future of our planet (Ros et al., 2019). We live on a finite
planet, but we do not take into account its limits in our activity as a
species (Lade et al., 2020). Year after year, humans consume more
than the planet is capable of regenerating. According to the latest
data published by the research organization Global Footprint
Network, in 2016 we were consuming the equivalent of 1.69 Earth
planets (Global Footprint Network, 2019). In order to survive as a
species, we need to consider sustainability in our activities.
The concept of sustainability was coined by Hans Carl von Car-
lowitz in the 18th century. Carlowitz was a manager of mining
activity in the German city of Freiberg. The shortage of wood caused
* Corresponding author. University Research Institute for Sustainability Science
cnica de Catalunya, UPC-BarcelonaTech. C. Jordi
and Technology. Universitat Polite
Girona, 31, 08034, Barcelona, Spain.

nchez-Carracedo).
E-mail address: fermin@ac.upc.edu (F. Sa
industry for silver extraction and on the metal industry of the time.
Carlowitz proposed using the forest sustainably, felling only trees
that could regenerate in a timely manner through planned refor-
estation (von Carlowitz, 1713).
Almost three centuries later, the Brundtland report defined
sustainable development as “that which meets the needs of the
present without compromising the needs of future generations”.
While there are many definitions and controversies about the
meaning of the concept of sustainable development and its
implementation, a widely accepted framework to promote it is to
achieve the sustainable development goals (SDGs) (Zamora-Polo
and Sa
nchez-Martín, 2019). Although as recognized by Caiado
et al. (2018), achieving the SDGs will not be an easy undertaking,
important tasks and challenges must be addressed.
For the implementation of the SDGs to be effective, education is
considered a key synergistic element (Lim et al., 2018; Giangrande
et al., 2019; Hallinger and Chatpinyakoop, 2019; Aleixo et al., 2020).
Indeed, Goal 4.7 of Sustainable Development Goal 4 addresses

https://doi.org/10.1016/j.jclepro.2021.126322
0959-6526/© 2021 Elsevier Ltd. All rights reserved.

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa
Education for Sustainable Development (ESD), and recognizes it as
an essential factor for the achievement of sustainability (UN, 2015).
In this paper, the terms sustainability and ESD will be used inter-
changeably as synonyms, although in some contexts some differ-
ences can be found between both concepts.
Promoting sustainable development at all educational levels is
essential (Leicht et al., 2018), since we are educating future agents
of change and transformation of society (Mulder et al., 2012) that
will make the immediate future sustainable or not (Svanstro €m
et al., 2012). In this sense, the role of higher education is key to
promoting the transition towards a more sustainable society
(Stephens and Graham, 2010) and to deliver graduates to foster this
transition (Stough et al., 2018). It is essential to create awareness in
higher education about the social and moral responsibilities asso-
ciated with professional practice (Sibbel, 2009). It is therefore
essential that sustainability is integrated into Higher Education
Institutions (HEI) policies and strategies (Farinha et al., 2018),
although as recognized by various researchers (Aleixo et al., 2018a),
this does not always happen. Avila
et al. (2017) have analyzed the
existing barriers to the introduction of ESD in 172 universities
across all continents. They emphasize that a change of thinking is
necessary, since ESD should not only be part of teaching, research
and campus operations, but should also be embedded in the re-
lationships with external partners. Fissi et al. (2021) highlight that
including green issues (green building, waste management, etc.) in
all the main dimensions of university activities is important to
promote ESD.
Many researchers have studied methodologies to introduce
sustainable development into the curriculum of universities
(Mateus et al., 2020), and stress that promoting cross-disciplinary
research is imperative in order to address the great challenges
associated with ESD (Hensley, 2020). Some works have analyzed
different models of introduction of ESD into higher education
(Priyadarshini and Abhilash, 2020). Thurer et al. (2018) have
analyzed, from 247 articles, the integration of sustainability in the
curriculum of the engineering area. They conclude that the
assessment of the results of the integration of sustainability in the
curricula, in practice, is done quite superficially. Other authors have
analyzed the key concepts to convey (Boni et al., 2016), and other
research emphasizes the importance for academics to develop
collaborative approaches and redesign their disciplines by taking
into account the multicultural vision of sustainability (Filho et al.,
2018). The introduction of ESD should take into account the ho-
listic and transdisciplinary vision of the concept (environmental,
economic, social/cultural and institutional/educational/political)
(Aleixo et al., 2016), although in their research conducted in Por-
tuguese HEIs, Aleixo et al. (2018b) highlight that the implementa-
tion of ESD is more advanced in some dimensions than in others. As
some authors acknowledge, ESD practice must be extended to be
more systemic and cross-disciplinary (Stevenson et al., 2017).
Other investigations use a bibliometric approach. This is the case
of Tejedor et al. (2019), which have analyzed the keywords related
to sustainability in engineering education in 171 papers. They
conclude that, in recent years, the number of keywords related to
institutional and university policy aspects to incorporate or apply
sustainability in higher education has decreased, while the key-
words related to the professional development of faculty members,
and the implementation and use of learning strategies has
increased.
Research has also been done about the competencies that
should be included in the curriculum of universities for the intro-
duction of sustainable development, and whether or not these
competencies should be transversal, in addition to analyzing their
degree of integration (Trencher et al., 2018; Lambrechts et al., 2013).
Wiek et al. (2011) have identified five competencies related to
2
Journal of Cleaner Production 294 (2021) 126322
sustainable development: systems-thinking, anticipatory, norma-
tive, and strategic and interpersonal competency. Other authors
have investigated how they can be operationalized at different
levels (Wiek et al., 2015). The UN adds other competencies to be
considered: critical thinking, self-awareness and integrated
problem-solving, (UNESCO, 2017). In addition to these compe-
tencies, other authors identify three other clusters of competencies:
diverse modes of thinking, methodological plurality, and compe-
tencies for autonomy (Salovaara et al., 2019). It should also be noted
the research by Lozano et al. (2017), which proposes a framework to
relate competencies and pedagogical approaches in order to pro-
vide a more holistic and systemic approach to ESD.
Spanish universities have incorporated ESD in accordance with
the guidelines of the document “Guidelines for the introduction of
Sustainability in the Curriculum” approved by the Executive Com-
mittee of the CRUE (Conference of Rectors of Spanish Universities)
Working Group on Environmental Quality and Development Sus-
tainable in 2005, updated in 2011 (CRUE, 2012). These guidelines
were approved before the UNESCO SDG (UN, 2015), although they
are related to the competencies in ESD approved by the United
Nations Economic Commission for Europe (UNECE, 2012).
The CRUE proposes four transversal competencies in sustain-
ability that must be integrated into all university degrees in Spain
(CRUE, 2012). Based on these competencies, the EDINSOST project
(Segal
nchez-Carracedo, 2019) defines sustainability maps
as and Sa
for different degrees. These sustainability maps contain the sus-
tainability learning outcomes that a graduate must have declared
on completion of their studies. Sa
nchez-Carracedo et al. (2019)
analyze the presence of ESD in the curricula of a set of engineer-
ing degrees in the Spanish university system using these maps.
The objective of this work is to analyze the level of sustainability
competencies achieved by graduates of nine engineering degrees in
Spain according to the Engineering sustainability map of the
EDINSOST project.
2. Material and methods
2.1. Objectives and research questions
To achieve the main objective, a comparative approach is
adopted. Specifically, this paper explores the level of uniformity of
improvements in sustainability competencies between domain
levels, universities and degrees. Likewise, the paper has two goals:
to investigate the existence of patterns beyond the general trend,
and to analyze if there is evidence that confirms the existence of
university policies for the acquisition of competencies related to the
SDGs.
This study is based on the following research questions:
C Q1: How much do engineering students in the Spanish uni-
versity system improve their sustainability competencies
during their studies at the university?
C Q2: Is the improvement homogeneous in all the domain
levels of the analyzed competencies?
C Q3: Is the improvement homogeneous in all the degrees
analyzed?
C Q4: Is the improvement homogeneous in all the universities
analyzed?
2.2. Instruments
To answer the four research questions, the EDINSOST project
engineering sustainability map (Sa
nchez-Carracedo et al, 2018b,
2019), which can be found in the Supplementary Material, is used

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa
Table 1

nchez-Carracedo et al., 2019 for a complete description).
Competencies and CUs of the Education Sustainability Map (see Sa
C1. Critical contextualization of knowledge
C2. Sustainable use of resources
C3. Community processes
C4. Ethics
as a starting point. The sustainability map is a competencies map
(S

anchez-Carracedo et al., 2018a) that contains the sustainability
competencies defined by the CRUE (2012). Each competency is
defined by one or more Competency Units (CU), and for each CU a
set of learning outcomes is defined. Learning outcomes are classi-
fied using the simplified Miller pyramid as taxonomy (Miller, 1990;

nchez-Carracedo et al, 2018b, 2019), which has three domain
Sa
levels: Know, Know how and Demonstrate þ Do. Table 1 shows the
competencies and CUs of the engineering sustainability map.
Based on the learning outcomes of the Engineering Sustain-
ability Map, the EDINSOST project has defined and validated a
questionnaire of 34 questions. The questionnaire can be found in
the supplementary material.
2.3. Methodology
The questions in the questionnaire are statements that are
answered according to a 4-point Likert scale: “Strongly disagree”,
“Disagree”, “Agree” and “Strongly Agree”. To perform the statistical
treatment of the responses, a numerical value was assigned to each
of the responses: value 0 for “Strongly disagree”, value 1 for
“Disagree”, value 2 for “Agree” and value 3 for “Strongly agree”. The
Engineering Sustainability Map has 24 cells, and each cell contains
one or more learning outcomes. Since the questionnaire contains
34 questions, some of the questions refer to the same cell on the
map. The supplementary material presents a table with the rela-
tionship between the questions and the cells of the Sustainability
Map.
To validate the questionnaire, a group of 15 experts and a control
group consisting of 20 final year students were formed. Both the
group of experts and the control group assessed the relevance and
clarity of each question, also indicating whether they considered
that any question should be eliminated or added. The description of
the validation process can be found in Sa
nchez-Carracedo et al.
(2018c).
The questionnaire was developed using the Google Forms tool.
The data collection was carried out during the second semester of
2018. First- and fourth-year students from various degrees taught
at various universities answered the form. Sample data is presented
in Section 2.4. The data analysis was performed with aggregated
data, since the study is not longitudinal nor is the number of re-
sponses uniform in all degrees and universities.
To analyze the information collected in the questionnaires, two
composite indicators were created. These tools are useful for
maximizing information and reducing complexity in a valid, reli-
able and precise way (de Vaus, 2002). A composite indicator is a
statistical technique that transforms individual variables, in this
case Likert items, to generate an aggregate measure that facilitates
the transmission of information about the phenomenon studied.
Variables have been defined to represent the different aspects to be
analyzed (Competencies, CUs and domain levels), provided that
these aspects are made up of more than one Likert item. The
relevance of generating methodologically adequate indicators lies
in avoiding simplistic aggregations and providing transparency in
3
Journal of Cleaner Production 294 (2021) 126322
1.1 Historical perspective
1.2 Creativity and innovation
2.1 Considers sustainability as an engineer
2.2 Considers the environmental impact as an engineer
2.3 Considers the social impact as an engineer
2.4 Management of an engineering project
3.1 Collaborative work related to sustainability
4.1 Behaves according to the deontological principles
their construction that benefits the indicators, especially in terms of
credibility and replicability (Rosen, 1991 in Nardo et al., 2008).
The construction of the first composite indicator consists of
three stages: First, the preparation of the information. The median
was imputed to the missing values, and the variables were stan-
dardized, since a significant proportion of items had skewed dis-
tributions. Furthermore, a one-dimensionality and reliability
analysis was performed to confirm that all the questions referring
to the same competency, CU or domain level formed a single un-
derlying concept (de Vaus, 2002). Using PCA, it was possible to
evaluate the likelihood of reducing the number of analysis di-
mensions while losing the minimum amount of information.
Although PCA is regularly practiced for ratio scale variables, recent
applications to ordinal scale variables exist that justify its applica-
tion in this work (Vermunt and Magidson, 2005 in Nardo et al.,
2008). In the PCA, Kaiser (1960) was applied by selecting only
those components with eigenvalues greater than 1. It was also
ensured that these components explain a minimum of 10% of the
variance, and that the sum of all the selected components explains
a minimum 60% of the cumulated explained variance. All PCAs
confirmed the ability to summarize in the foreseen dimension
(Competency, CU, domain level).
Next, once the appropriate treatment for Likert items has been
defined, the composite indicators are constructed. The factorial
scores of the PCA components are then obtained, provided that the
KMO test and the Bartlett sphericity test confirm the quality of the
results (de Vaus, 2002). Finally, to provide semantic meaning and
facilitate the interpretation of the data (de Vaus, 2002), the com-
posite indicators, which are normalized variables with mean 0 and
standard deviation 1, are transformed to the original unit of mea-
surement 0e3. This indicator is called absolute learning in Section 3
and Equation (1) is used to calculate it:
CI  minCI
*3; (1)
maxCI  minCI
CI being the composite indicator.
A second composite indicator was constructed from the first to
measure the increase in student learning in sustainability compe-
tencies. This indicator is the ratio resulting from the relationship
between the indicator of acquisition of sustainability competencies
of fourth-year students and first-year students, based on the
learning they could achieve about the learning declared by first-
year students.1 Since the data has been obtained through a cross-
sectional study, and not through a data study panel, the learning
percentage indicator will be a university/degree-aggregated
1
First-year students declare having a certain level of sustainability when they
enter university, greater than 0. If the level they declare to have is X, measured
between 0 and 3, the percentage of improvement they have is ((3/x)- 1) * 100.

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa Journal of Cleaner Production 294 (2021) 126322

indicator. This indicator is called “relative learning” in Section 3,
and the formula to calculate it is presented in Equation (2).
AL4  AL1
; (2)
3  AL1
3.1. - Research question 1
Figs. 1 and 2 allow us to answer the first research question: How
much do engineering students in the Spanish university system
improve their sustainability competencies during their studies at
the university?

3.1.1. Analysis by competencies

AL4 being the 4th course university/degree-aggregated learning,
Fig. 1 shows the learning declared by the students in each
and AL1 the 1st course university/degree-aggregated learning.
competency and on average. The figure shows that the learning
declared in the four competencies is different. On average, first-
year students report learning close to 50% (1.49 out of 3) when
2.4. Sample they enter university, and fourth-year students 66% (1.99 out of 3).
This implies an average relative learning of 33%, as observed in
This work analyzes four engineering degrees from three uni- Fig. 1 (b).
cnica de Catalunya - BarcelonaTech
versities: The Universidad Polite Clearly, C4 -ethics is the competency in which students declare
(UPC), the Universidad Polite
cnica de Madrid (UPM) and the Uni- less learning both in absolute terms (1.59 out of 3) and in relative
versity of Seville (US). The degrees analyzed are the Bachelor De- terms (24%). This result is worrying. The formation of a civic ethic,
gree in Informatics Engineering (BDIE), the Bachelor Degree in both in personal and ethical conduct, must be reinforced in engi-
Mechanics Engineering (BDME), Bachelor Degree in Chemical En- neering. Strengthening ethical values such as global perception is
gineering (BDCHE), and the Bachelor Degree in Industrial Tech- essential, as it reinforces other values such as solidarity and re-
nologies Engineering (BDITE). Not all degrees are taught at all sponsibility (Morin, 2001), promoting human development and the
universities, so the final number of degrees analyzed is nine instead common good. In addition, we consider that the teaching of a civic
of twelve. Table 2 presents the number of first- and fourth-year ethic must be transversal and, therefore, permeable in different
students who have answered the questionnaire, classified by uni- courses and subjects, from the most basic of the first course to the
versity and degree. In Table 3, the data is broken down to identify most specialized of the final courses.
how many students from each degree and course have answered Fig. 1 also shows that C3-participation in community processes
the questionnaire at each university. is the competency with the highest relative learning (39%), fol-
lowed by C2-sustainable use of resources and C1-critical contex-
tualization of knowledge (35% and 32%, respectively). However,
3. - Results and discussion there are hardly any relevant differences in the declared absolute
learning in these three competencies (z2 out of 3). The absolute
To answer the four research questions, this section shows competency learning declared by the fourth-year students follows
several figures that contain two grouped bar graphs labeled as (a) the gradation: C4 <C2 <C1 <C3.
and (b). The figures labeled as (a) represent the learning declared by
fourth-year students and the increase in learning declared by these 3.1.2. Analysis by competency units
students with respect to that declared by first-year students, both Fig. 2 shows the learning declared by the students in each CU.
calculated by using Equation (1). The figure range varies from 0 to 3, Since C3 and C4 have only one CU, the data shown in this figure is
according to the numerical transformation described in the meth- identical to that shown in Fig. 1 for these two competencies and
odology. The figures labeled (b) represent, in percentage, the rela- does not need to be analyzed. However, C1 and C2 have more than
tive learning declared by fourth-grade students, calculated one CU.
according Equation (2); that is, the percentage of learning that The comparative analysis of Figs. 1 and 2 for these UC shows
these students achieve with respect to what they could have that:
declared. The different figures shown in this section represent the
students’ declared learning in terms of competencies, CUs and C The absolute learning of the CUs in C1 is similar, but differ-
domain levels. ences are observed in relative learning. Relative learning for
CU1.1 - historical perspective is 34%, compared to 29% for
Table 2 CU1.2 - creativity and innovation. Learning, both absolute
Number of first- and fourth-year students classified by university and degree. and relative, follows the gradation: CU1.1> CU1.2.
C The absolute and relative learning of C2 is different in the
University Bachelor Degree
four CUs. CU2.2-environmental impact is the CU where stu-
UPC UPM US BDIE BDME BDCHE BDITE Overall
dents declare less learning, both in absolute terms (1.91 out
1 278 564 315 603 123 88 343 1157 of 3) and in relative terms (29%).
4 238 227 60 310 19 35 161 525 C CU2.4 - management of engineering projects is the CU of the
Overall 516 791 375 913 142 123 504 1682
C2 with the highest relative learning (44%), despite the fact

Table 3
Number of students from each degree and course who have answered the questionnaire at each university.

University UPC UPM US

Bachelor Degree BDIE BDME BDIE BDCHE BDITE BDIE BDME BDCHE BDITE

1 219 59 338 44 182 46 64 44 161
4 227 11 51 25 151 32 8 10 10
Overall 446 70 389 69 333 78 72 54 171

4

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa
Fig. 1. Learning declared by students in each competency and on average.
that the CU2.3 - social impact presents slightly higher ab-
solute learning.
The results clearly show that engineering curricula focus on
working on the economic aspects of projects, while neglecting
environmental aspects. Teachers often talk about development and
sustainable development, blurring the line between development
and sustainability (Mayer, 1998). Thus, a classic prejudice among
engineering students is, for example, that “the best technology will
win the market” (Mulder et al., 2012). However, it is also important
for an engineer to learn about the use and conservation of resources
(CU2.2). In other words, learning about the implementation of
clean and economically viable production processes that contribute
to development but, at the same time, mitigate the impact of the
human footprint.
On the other hand, engineers must understand and be aware of
the complexities of the environment in which they carry out their
work. They must be able to recognize the interrelation, circum-
stances, risks, coincidences, dangers of all phenomena and,
consequently, accept to live with uncertainty (Mayer, 1998). Living
with the unpredictable favors critical thinking and responsible and
creative decision-making in the face of possible alternative sce-
narios. This leads us to the opinion that a greater learning in CU1.1
would enable better results to be obtained in the learning of CU1.2.
In engineering studies, training students in creativity and
5
Journal of Cleaner Production 294 (2021) 126322
Fig. 2. Learning declared by students in each competency unit.
innovation is essential, since it involves the imagination and pro-
motes the development of original ideas (Hensley, 2020).
3.2. - Research question 2
To answer the second research question, “Is the improvement
homogeneous in all the domain levels of the analyzed compe-
tencies?” Fig. 3 is presented.
Fig. 3 shows the learning declared by the students at each
domain level (L1-know, L2-know how and L3-demonstrate þ do) of
each competency and the mean for each level. Different learning is
observed in the three domain levels:
C Students report greater absolute learning at the highest
levels of the taxonomy (L3) except for C2-sustainable use of
resources. In C2, learning is higher at the intermediate level
of taxonomy (L2-know how). C3 is the most balanced com-
petency, and students achieve very similar learning at all
three domain levels.
C Regarding relative learning, the L1 and L3 levels of C3-
participation in participatory processes stand out. C4-ethics
does not exceed 25% in any of the three levels, and the
same is true for level L1 of C1-contextualization of
knowledge.
C Although the four competencies behave differently when
their domain levels are considered, on average both absolute
and relative learning are greater at the highest levels of the
taxonomy in increasing order (L1 <L2 <L3). In particular, at
levels L2 and L3, first-year students report absolute learning

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa Journal of Cleaner Production 294 (2021) 126322

3.3. Research question 3

Figs. 4e6 allow us to answer the third research question: Is the

improvement homogeneous in all the degrees analyzed?

3.3.1. Analysis by competencies and degrees

Fig. 4 shows the learning declared by students in each compe-
tency organized by degrees. The comparison of Figs. 1 and 4 in-
dicates that the BDIE is the only degree in which the learning of
fourth-year students behaves as the mean: C4 <C2 <C1 <C3. In
the BDME, the relative learning declared for C4-ethics stands out
(44%) compared to the average relative value for this competency
(24%, see Fig. 1 (b)). As already happens in the mean, C4 is the
competency that obtains the worst learning results in absolute
learning in every degree.
The students of the four degrees analyzed declare a different
absolute learning pattern in the four competencies, but the
declared values are similar for C1, C2 and C3 (z2 over 3). However,
in relative learning there is a lot of variability. BDME students
report good results in all four competitions, BDIE students excel in
C3, and BDITE students in C2 and C3 (in all cases, learning is above
39%).

3.3.2. Analysis by competency units and degrees

Fig. 5 shows the declared learning by students in each CU
organized by degrees. Since C3 and C4 only have one CU, the data

Fig. 3. Learning declared by students at each domain level of each competency and on
average.

of close to 50% (1.47 and 1.49 out of 3, respectively), and

fourth-year students around 66% (1.96 and 2.00 out of 3
respectively). This implies an average relative learning that
ranges between 31% (L1) and 34% (L3). In other words,
fourth-year students declare that they have learned in the
three domain levels approximately a third of what they could
have learned.

These results can be explained by the development of more

application-oriented tasks in engineering education (Tejedor et al.,
2018). The concern to find solutions to problems is inherited from a
learning that has often been compartmentalized, divided up and
atomized, in a school that focused its attention on the application of
heuristics (L3) rather than on deepening knowledge (L1) and un-
derstanding (L2) of problems. However, it is at the levels of
knowledge (L1) and understanding (L2) where it is necessary to
adopt a more critical and reflective approach about the reasons that
exist (or not) to make certain decisions when addressing a problem
or task. The continuous taking of sustainable decisions and the
application of such decisions (L3) requires prior knowledge and
understanding (L1, L2) of the problems. This makes us reflect on the
need not only to increase learning in the different domain levels
(L1, L2, L3), but also to seek its balance in each of the four compe-
tencies analyzed.
Fig. 4. Learning declared by students in each competency, according to degrees.

6

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa Journal of Cleaner Production 294 (2021) 126322

Fig. 5. Learning declared by students in each competency unit, according to degrees.

shown in this figure is identical to that shown in Fig. 4 and does not
need to be analyzed.
Relative learning in CUs (Fig. 5 (b)) has a lot of variability in the
Fig. 6. Learning declared by students in each competency for each degree and in each
four degrees. In general, BDME and BDITE students report higher university.
relative learning compared to BDIE and BDCHE students. However,
the absolute learning declared in the four degrees is similar, as
already observed in Fig. 4. differ from those presented in Fig. 2. This is the case of the BDCHE
A more exhaustive analysis of Figs. 4 and 5 highlights the and BDITE students, who state that they possess less learning in
following aspects: knowledge of sustainable engineering projects considering, holis-
tically, environmental, economic and social aspects (28% and 39%,
C In general, there are differences between the relative respectively). On the other hand, BDME students declare more
learning of CU1.1-historical perspective and CU1.2-creativity learning (48%) in aspects related to accessibility, ergonomics, safety
and innovation, except for the BDIE. The absolute learning and, ultimately, how to contribute to improving the common good
declared by the students for CU1.1, however, is greater than of society.
that declared for CU1.2, except for the BDIE, in which the
opposite occurs.
3.3.3. Analysis by competencies and degrees broken down
C The pattern for absolute and relative learning for C2’s CUs is
according to university
different in all four degrees. CU2.2-considers environmental
Fig. 6 shows the learning declared by the students in each
impact, is the CU with the least relative learning in the BDIE
competency for each degree at each university. The analysis of the
and BDME (10% and 43%, respectively). CU2.4-management
data presented in this figure indicates that students of the same
of engineering projects, is the CU with the highest relative
degree perceive a different relative learning in the four compe-
learning achieved in three of the four degrees studied (BDIE,
tencies according to the university where they study. In BDIE and
BDCHE and BDITE). In the BDME, the four CUs reach very
BDITE, the low relative learning declared in the US with respect to
similar values.
that declared in the other universities stands out. The BDCHE is the
degree which presents more differences between universities: at
These results are consistent with the analysis made for Fig. 2.
UPM, relative learning is higher in C2 (39%) and C3 (36%), while in
However, depending on the degree analyzed, some of these results
the US it is higher in C1 (31%) and C4 (25%).
7

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa Journal of Cleaner Production 294 (2021) 126322

In absolute learning, however, the differences between the four

competencies are not so evident, and the BDME is the degree in
which students declare more uniform learning in the two univer-
sities where it is taught.
Clearly, the same degree develops sustainability competencies
differently depending on the university. We can therefore conclude
that these competencies are not considered within the core com-
petencies of the degree, and that each university develops them
according to local criteria. Thus, in response to the research ques-
tion, learning in sustainability of the four competencies is different
depending on the degree and, for the same degree, it is different
depending on the university in which it is taught. This is probably
due to the fact that, unlike in other countries, in Spain there is no
clear state or regional strategy that supports the application and
development of ESD policies in higher education.

3.4. - Research question 4

Figs. 7e9 enable us to answer the fourth research question: Is

the improvement homogeneous in all the universities analyzed?

3.4.1. Analysis according to competencies and universities

Fig. 7 shows the learning declared by the students in each
competency according to universities. In the three universities
studied, the competency in which the students declare the least
absolute and relative learning is C4-ethics, while the greatest ab-
solute and relative learning is in C3-community processes. Both
absolute and relative learning present similar patterns in the three
universities. The absolute learning declared by the students for C2-

Fig. 8. Learning declared by students in each competency unit, according to

universities.

sustainable use of resources is slightly lower than that declared for

C1 and C3, while for C4 it is clearly lower than that of the other
three competencies, obtaining its highest value in the UPC (35%).
Regarding relative learning, the lowest value is declared for C4,
while for the other competencies C3> C2> C1. The University of
Seville stands out in a negative way, since it does not exceed 28% of
relative learning in any of the four competencies.

3.4.2. Analysis according to competency units and universities

Fig. 8 shows the declared learning by students in each CU for
each university. Since C3 and C4 only have one CU, the data shown
in this figure is identical to that shown in Fig. 7 and does not need to
be compared.
Regarding C1 - contextualization of knowledge, the two CUs
present very similar values in absolute learning in the three uni-
versities. However, differences are observed in relative learning.
The UPC and the US achieve better results in CU1.1 - historical
perspective, while the UPM achieves slightly better results in CU1.2
- creativity and innovation.
Regarding C2-sustainable use of resources, at UPC students
report considerably less learning in CU2.2-considers environmental
impact, compared to other CUs. In the UPM students declare that
Fig. 7. Learning declared by students in each competency, according to universities. they learn less in the CU2.2-consider environmental impact, and in
8

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa
Fig. 9. Declared learning by students in each competency for each university and in
each degree.
the US the least learning is declared for the CU2.1-consider sus-
tainability as an engineer and CU2.2-consider environmental
impact.
Fourth-year students at the UPC and UPM report more learning
about how to maximize the positive impact of their professional
activities on society than US students. On the other hand, students
from the three universities declare more relative learning in aspects
related to the management and economic viability of an engi-
neering project, their planning techniques and process optimiza-
tion. This allows us to affirm that the analyzed universities focus
their learning related to resource management on project man-
agement, without taking into account their sustainability.
3.4.3. Analysis by competencies and universities broken down by
degrees
Fig. 9 shows the learning declared by the students in each
competency for each university organized by degree. Relative
learning is different in the four competencies between students
from the same university according to the degree they study. This
seems to suggest that, within the same university, there is no
9
Journal of Cleaner Production 294 (2021) 126322
common policy to develop ESD in all degrees. It seems that each
degree develops its own policy, either because it is defined by its
management teams or because that policy does not exist and de-
pends on what teachers do in their subjects.
Regarding absolute learning, the UPC obtains similar values in
its two degrees, except in C4-ethics of the BDIE. The behavior of the
BDIE is consistent with the rest of the analysis carried out in this
paper for the rest of the degrees. The BDME, on the other hand,
presents a learning in C4 similar to that of the rest of the compe-
tencies, being the only degree of the nine studied in which this
happens. In addition, it presents the highest learning values both
absolute and relative for C4 in comparison with the rest of the
degrees. This is undoubtedly due to the fact that those responsible
for the degree, or the professors in charge of working at ESD, are
especially aware of ethical issues and have incorporated this com-
petency into the curriculum.
The three degrees at UPM also behave differently in both ab-
solute and relative learning, so we assume that there is no uni-
versity policy on ESD either. In the BDIE the learning of C3-
community processes stands out, probably due to the knowledge
that computer engineers have of collaborative work tools (since the
same happens in computer engineers from other universities).
BDITE seems to be the degree that achieves the highest relative
learning in all competencies. This may be due to the fact that the
first-year students of this degree are those who declare a lower
level in the four competencies when they begin their studies, so
they have a greater margin for improvement. This fact is striking,
since an expected result was that first-year students from all de-
grees and universities present similar results. The fact that the
average access marks are different in the different degrees probably
has an influence in the fact that the first-year students declare a
different learning depending on the degree in which they have
enrolled. The number of students who have answered the survey at
the BDITE of UPM is very high, so this does not seem to be a factor
that could account for the difference.
Finally, at the US, the BDIE is the degree that presents a worse
relative learning, despite the fact that in absolute learning there are
no significant differences with the rest of degrees, except perhaps
with regard to C4-ethics, in which there is hardly any improvement
in learning. The other three degrees present very similar values
regarding absolute learning. In relative learning, however, the re-
sults reported by BDME students stand out, as was the case at the
UPC. Perhaps the very nature of Mechanical Engineering studies
encourages students to develop the four sustainability compe-
tencies. However, while this could be reasonable for C1, C2 and C3,
it is curious to observe that it also occurs in C4-ethics. There does
not seem to be any compelling reason for mechanical engineering
students to declare themselves more competent in ethical issues
than the other engineering students at their university, but this is
the case in the two mechanical engineering degrees we have
studied.
These results highlight aspects, in part, expected. Unlike what
happens in other neighboring countries such as Sweden or Wales,
in Spain no centralized system exists to implement ESD in HEI.
While universities enjoy some institutional autonomy (Farinha
et al., 2018), as In Portugal, in the Spanish context CRUE (2012)
proposes a series of “recommendations” to integrate sustainabil-
ity into university curricula. Consequently, this work can be
considered as part of a local action that seeks to “measure” the
degree of implementation of a more global “recommended
agenda”, such as the integration of the four CRUE sustainability
competencies throughout the Spanish university system. From this
perspective, the case study presented could successfully connect a
bottom-up approach with professional bodies in a kind of mixed
implementation.

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa
3.5. - Limitations of this work
This work presents some limitations that must be considered
when interpreting the results. First, only 4 degrees taught at 3
universities have been studied. Second, the student sample is large,
but a larger sample, including more degrees and more universities,
is needed to draw definitive conclusions beyond a case study. Third,
this study is not longitudinal, but of the repeated cross-sectional
type, in which the samples from each course do not include the
same subjects nor are they collected at different times (Bryman,
2016). In other words, the first- and fourth-year students who
have answered the questions are not the same, nor have three ac-
ademic years elapsed between the two surveys, which were con-
ducted in the same semester. The observed improvement should
therefore be interpreted as an overall improvement, although we
do not know if this improvement occurs on the surveyed students.
Even if it were a longitudinal study, we could not rule out that other
factors may have influenced learning. What we are able to say is
that the results presented reflect the increase in learning on
average, but not the average increase in learning, which would
require the surveys to have been answered by the same students
when they took the first and fourth grades. Fourth, the students
were not randomly selected, but responded voluntarily to the
survey. Finally, the survey measures the perception of students
regarding their own sustainability competencies, not their actual
knowledge.
4. - Conclusions and future work
This work analyzes the perception that the students of nine
engineering degrees taught in three universities have about their
sustainability competencies. This perception is measured using two
composite indicators: absolute learning and relative learning. Ab-
solute learning measures students’ perception of learning at a given
stage of their studies. Relative learning measures the percentage of
learning that students claim to have achieved at the end of their
studies compared to what they should have at a given moment. The
study presented in this paper compares the first- and fourth-grade
students’ perception of four sustainability competencies:
 C1. Critical contextualization of knowledge by establishing in-
terrelations with social, economic, environmental, local, and/or
global problems.
 C2. Sustainable use of resources and prevention of negative
impacts on the natural and social environment.
 C3. Participation in community processes that promote
sustainability
 C4. Application of ethical principles related to the values of
sustainability in personal and professional behaviour.
The most important conclusion of this work is that, of the four
sustainability competencies analyzed, the “application of ethical
principles related to the values of sustainability” is the least
developed in the nine degrees studied, according to the students’
perception. However, strengthening ethical values such as global
perception is essential, as it reinforces other values such as soli-
darity and responsibility (Morin, 2001), promoting human devel-
opment and the common good.
On average, students stated that they already know half of what
they should know when they enter university in the four compe-
tencies, while in the fourth year they achieve only two thirds of the
learning they should have achieved. Nevertheless, mechanical en-
gineering students seem to feel more prepared in these compe-
tencies than students from other degrees.
In addition to the poor results obtained in terms of ethical
10
Journal of Cleaner Production 294 (2021) 126322
principles, it is surprising that students state that they feel less
competent in the “sustainable use of resources” (a competency
directly related to engineering studies) than in the “critical con-
textualization of knowledge” or in “participation in community
processes”. It is important for an engineer to learn about the use
and conservation of resources; in other words, learning about the
implementation of clean and economically viable production pro-
cesses that contribute to development but, at the same time,
mitigate the impact of the human footprint.
Engineers must be aware of the complexities of the environ-
ment in which they carry out their work. They must be able to
recognize the interrelation, circumstances, risks, coincidences and
dangers of all phenomena (Mayer, 1998). However, the results
presented in this paper show that engineering curricula seem to
focus on working on the economic aspects of projects, while
neglecting environmental aspects.
The continuous taking of sustainable decisions and the appli-
cation of such decisions (Level L3 of the learning taxonomy) re-
quires prior knowledge and understanding (Levels L1, L2) of the
problems. However, the results presented in this paper show that
students report greater absolute learning at the highest levels of the
taxonomy (L3) except for “sustainable use of resources”. Although
the four competencies behave differently when their domain levels
are considered, on average both absolute and relative learning are
greater at the highest levels of the taxonomy. These results can be
explained by the development of more application-oriented tasks
in engineering education (Tejedor et al., 2018). However, engi-
neering degrees should not neglect the development of the lower
levels of the taxonomy, since it is crucial that students have a solid
theoretical foundation of the methodologies they apply.
No pattern between degrees or universities is identified in this
study. The same degree develops sustainability competencies
differently depending on the university. Relative learning is
different in the four competencies between students from the same
university according to the degree they study. In other words, there
are no degrees or universities in which students declare to have
clearly achieved a higher level of sustainability competencies on
completion of their studies. The results suggest that the analyzed
universities have no policy for the inclusion of ESD in their degrees.
This policy could exist independently in each degree, or it may not
exist at all. This situation is coherent in the Spanish context, where
there is no centralized system to implement ESD, although the
universities do enjoy a certain institutional autonomy. Thus, the
results may depend directly on the motivation of the professors of
each degree to include ESD in their subjects.
It is possible that the students’ access grade influences the re-
sults of this work. Perhaps the students of degrees that have a lower
access grade declare a lower level of sustainability competencies at
the beginning of their studies, and therefore have a greater capacity
for improvement. However, the students of the US, in general,
declare to have a higher level of sustainability competencies when
they begin their university studies than students of other univer-
sities. This fact is probably due to the Kruger-Dunnin effect (Kruger
and Dunnin, 1999): “an individual with fewer competencies and
less knowledge has an illusory feeling of superiority, considering
himself/herself to be more intelligent than another better prepared
individual”. Analyzing whether or not that this is a factor of influ-
ence forms part of the authors’ future work.
Taking into account the results presented in this work, engi-
neering degrees should improve the way they develop ESD in order
to enable students to achieve the third part, which they state they
are not currently achieving. It is curious that some published
studies indicate that the study guides for engineering subjects
declare that professional ethics is the competency that is most
developed among the four competencies of the CRUE (Sa
nchez-

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa
Carracedo et al., 2019). This contradiction implies that it is neces-
sary to carry out future work to compare the results of this work
with the results that would be expected from the subject learning
guides. This analysis could provide a valid methodology to help
identify weaknesses in the teaching and learning processes of the
subjects in the degrees in order to correct them, and that such
processes can be effectively implemented in regard to ESD, thereby
favouring full coherence between theory and practice.
If the HEI pursue the training of critical and reflective pro-
fessionals who not only possess the appropriate knowledge in their
discipline, but also the confidence that their actions as engineers
can contribute to the construction of more sustainable societies
through the exercise of their profession, it is necessary to analyze
the perception that engineering students have regarding their de-
gree of acquisition of sustainability competencies, as well as to
identify difficulties and interferences in their development.
The tasks identified in the last two paragraphs are part of the
future work of the authors within the EDINSOST2-ODS project. The
results of the work presented in this paper belong to the EDINSOST
project, which was an analysis project. EDINSOST2-ODS is an
intervention project that focuses on integrating ODS in a set of
seven degrees from eight universities. The degrees sample includes
three levels of incidence. First, a degree related to one of the three
dimensions of sustainability (environmental, social and economic)
will be studied: the Degree in Business Administration and Man-
agement. Second, because of its long-term multiplier effect, two
undergraduate degrees in the sciences of education (Pedagogy and
Primary Education) will be analyzed, since their graduates are the
future teachers of the new generations of citizens. Finally, covering
the short-term effect, work will be done on technological degrees
(informatics engineering, telecommunications engineering, indus-
trial technology engineering and design engineering), for its great
impact on the challenges of society.
The EDINSOST2-ODS project is aligned with European strategy
Horizon 2020. The transdisciplinarity in the approach to address
the challenge on which the project is based arises from the need to
form interdisciplinary teams, capable of providing value in what is
the implementation of actions in educational institutions that seek
programs with greater implications of social responsibility, the
development of values oriented to commitment and social change.
The results of EDINSOST2-ODS will allow:
 Improve ESD in the degrees studied, increasing student learning
and knowledge of teachers. As previously commented, EDIN-
SOST aimed to analyze the situation in ESD in Spain. EDINSOST2-
ODS will implement improvements in some degree curricula
and will evaluate the cost of implementing these improvements
in other degrees after the project is finished.
 Train some teachers of the degrees studied so that they incor-
porate the SDGs in their teaching.
 Measure how students’ learning evolves throughout the 4 years
of the bachelor degrees and the project to verify the effective-
ness of the work developed in the project.
Funding
This work was supported by the Spanish Ministerio de Econo-
mía y Competitividad, from study design to submission, under
grant number EDU2015-65574-R; by the Spanish Ministerio de

n y Universidades, the Spanish Agencia Estatal de
Ciencia, Innovacio

n (AEI), and the Fondo Europeo de Desarrollo Regional
Investigacio
(FEDER), from study design to submission, under grant number
RTI2018-094982-B-I00.
11
Journal of Cleaner Production 294 (2021) 126322
CRediT authorship contribution statement
Fermín Sa
nchez-Carracedo: Conceptualization, Data curation,
Funding acquisition, Investigation, Methodology, Project adminis-
tration, Resources, Supervision, Validation, Visualization, Writing -
original draft, Writing - review & editing. Ba
rbara Sureda:
Conceptualization, Investigation, Methodology, Validation, Visual-
ization, Writing - original draft. Francisco Manuel Moreno-Pino:
Conceptualization, Investigation, Methodology, Validation, Visual-
ization, Writing - original draft. Daniel Romero-Portillo: Concep-
tualization, Data curation, Formal analysis, Methodology, Software,
Visualization, Writing - original draft.
Declaration of competing interest
None.
Acknowledgements
This work was supported by the Spanish Ministerio de Econo-
mía y Competitividad, from study design to submission, under
grant number EDU2015-65574-R; by the Spanish Ministerio de
Ciencia, Innovacio
n y Universidades, the Spanish Agencia Estatal de

n (AEI), and the Fondo Europeo de Desarrollo Regional
Investigacio
(FEDER), from study design to submission, under grant number
RTI2018-094982-B-I00.
Appendix A. Supplementary data
Supplementary data to this article can be found online at
https://doi.org/10.1016/j.jclepro.2021.126322.
References
Aleixo, A.M, Azeiteiro, U.M., Leal, S., 2016. Toward sustainability through higher
education: sustainable development incorporation into Portuguese higher ed-
ucation institutions, in Davim, J.P., Leal Filho, W. (Eds.), Challenges in Higher
Education for Sustainability. Springer, Berlin, pp. 159e187.
Aleixo, A.M., Leal, S., Azeiteiro, U.M., 2018a. Conceptualization of sustainable higher
education institutions, roles, barriers, and challenges for sustainability: an
exploratory study in Portugal. J. Clean. Prod. 172, 1664e1673. https://doi.org/
10.1016/2Fj.jclepro.2016.11.010.
Aleixo, A.M., Azeiteiro, U.M., Leal, S., 2018b. The implementation of sustainability
practices in Portuguese higher education institutions. Int. J. Sustain. High Educ.
19, 146e178. https://doi.org/10.1108/IJSHE-02-2017-0016.
Aleixo, A.M., Azeiteiro, U.M., Leal, S., 2020. Are the sustainable development goals
being implemented in the Portuguese higher education formative offer? Int. J.
Sustain. High Educ. 21, 336e352. https://doi.org/10.1108/IJSHE-04-2019-0150.

€
Avila, L.V., Leal Filho, W., Brandli, L.L., Macgregor, C.J., Molthan-Hill, P., Ozuyar, P.G.,
Moreira, R.M., 2017. Barriers to innovation and sustainability at universities
around the world. J. Clean. Prod. 164, 1268e1278. https://doi.org/10.1016/
j.jclepro.2017.07.025.
Boni, A., Lopez-Fogues, A., Walker, M., 2016. Higher education and the post-2015
agenda: a contribution from the human development approach. J. Global
Ethics 12, 17e28. https://doi.org/10.1080/17449626.2016.1148757.
Bryman, A., 2016. Social Research Methods, fifth ed. Oxford university press, United
Kingdom.
Caiado, R., Filho, W., Osvaldo, Q., Quelhas, D., Nascimento, D., Avila,
L., 2018.
A literature-based review on potentials and constraints in the implementation
of the sustainable development goals. J. Clean. Prod. 198, 1276e1288. https://
doi.org/10.1016/j.jclepro.2018.07.102.
CRUE, 2012. Directrices para la sostenibilizacio
n curricular. https://www.crue.org/
wp-content/uploads/2020/02/Directrices_Ingles_Sostenibilidad_Crue2012.pdf.
accessed 10 March 2020.
de Vaus, D., 2002. Surveys in Social Science, sixth ed. Allen & Unwin, Australia.
Farinha, C.S., Azeiteiro, U., Caeiro, S.S., 2018. Education for sustainable development
in Portuguese universities: the key actors’ opinions. ? Int. J. Sustain. High Educ.
19, 912e941. https://doi.org/10.1108/IJSHE-09-2017-0168.
Filho, W.L., Raath, S., Lazzarini, B., Vargas, V.R., de Souza, L., Anholon, R.,
Quelhas, O.L.G., Haddad, R., Klavins, M., Orlovic, V.L., 2018. The role of trans-
formation in learning and education for sustainability. J. Clean. Prod. 199,
286e295. https://doi.org/10.1016/j.jclepro.2018.07.017.
Fissi, S., Romolini, A., Gori, E., Contri, M., 2021. The path toward a sustainable green

nchez-Carracedo, B. Sureda, F.M. Moreno-Pino et al.
F. Sa
university: the case of the University of Florence. J. Clean. Prod. 279 https://
doi.org/10.1016/j.jclepro.2020.123655.
Giangrande, N., White, R.M., East, M., Jackson, R., Clarke, T., Saloff Coste, M., Penha-
Lopes, G., 2019. A competency framework to assess and activate education for
sustainable development: addressing the UN sustainable development goals 4.7
challenge. Sustainability 11, 2832. https://doi.org/10.3390/su11102832.
Global Footprint Network, 2019. National Footprint and Biocapacity Accounts 2019.
http://data.footprintnetwork.org/#/countryTrends?cn¼5001&amp;type¼earth.
accessed 10 March 2020.
Hallinger, P., Chatpinyakoop, C., 2019. A Bibliometric Review of Research on Higher
Education for Sustainable Development, vol. 11, p. 2401. https://doi.org/
10.3390/su11082401, 1998e2018. Sustainability.
Hensley, N., 2020. Educating for sustainable development: cultivating creativity
through mindfulness. J. Clean. Prod. 243 https://doi.org/10.1016/
j.jclepro.2019.118542.
Kaiser, H.F., 1960. The application of electronic computers to factor Analysis. Educ.
Psychol. Meas. 20, 141e151. https://doi.org/10.1177/001316446002000116.
Kruger, J., Dunning, D., 1999. Unskilled and unaware of it: how difficulties in
recognizing one’s own incompetence lead to inflated self-assessments. J. Pers.
Soc. Psychol. 77, 1121e1134. https://doi.org/10.1037/0022-3514.77.6.1121.
Lade, S.J., Steffen, W., De Vries, W., Carpenter, S.R., Donges, J.F., Gerten, D., Hoff, H.,
Newbold, T., Richardson, K., Rockstro €m, J., 2020. Human impacts on planetary
boundaries amplified by Earth system interactions. Nat. Sustain. 3, 119e128.
https://doi.org/10.1038/s41893-019-0454-4.
Lambrechts, W., Mula, I., Ceulemans, K., Molderez, I., Gaeremynck, V., 2013. The
integration of competences for sustainable development in higher education:
an analysis of bachelor programs in management. J. Clean. Prod. 48, 65e73.
https://doi.org/10.1016/j.jclepro.2011.12.034.
Leicht, A., Heiss, J., Byun, W.J., 2018. Issues and Trends in Education for Sustainable
Development. UNESCO Publishing. https://unesdoc.unesco.org/ark:/48223/
pf0000261445. accessed 19 September 2020.
Lim, M.M., Jørgensen, P.S., Wyborn, C.A., 2018. Reframing the sustainable develop-
ment goals to achieve sustainable development in the Anthropocene-a systems
approach. Ecol. Soc. 23 https://doi.org/10.5751/ES-10182-230322.
Lozano, R., Merrill, M.Y., Sammalisto, K., Ceulemans, K., Lozano, F.J., 2017. Con-
necting competences and pedagogical approaches for sustainable development
in higher education: a literature review and framework proposal. Sustainability
9, 1889. https://doi.org/10.3390/su9101889.
Mateus, D., Pinho, H., Nogueira, I., Rosa, M., Cartaxo, M., Nunes, V., 2020. Partici-
pation of students in the project Valorbio: a case study to accelerate the
implementation of sustainability principles in the curriculum. Int. J. Sustain.
High Educ. 2, 244e263. https://doi.org/10.1108/IJSHE-09-2019-0254.
Mayer, M., 1998. Educacio
n ambiental: de la accio
n a la investigacio

n. Ensen ~ anza de
las ciencias. Rev. Invest. Exp. Didact. 16, 217e232. https://www.raco.cat/index.
php/Ensenanza/article/view/21530/21364. accessed 10 March 2020.
Morin, E., 2001. Los siete saberes necesarios para la educacio
n del futuro. http://
www.ideassonline.org/public/pdf/
LosSieteSaberesNecesariosParaLaEdudelFuturo.pdf. accessed 10 March 2020.
Miller, G.E., 1990. The assessment of clinical skills/competence/performance. Acad.
Med. 65, 63e67. https://doi.org/10.1097/00001888-199009000-00045.
Mulder, K.F., Segala s, J., Ferrer-Balas, D., 2012. How to educate engineers for/in
sustainable development: ten years of discussion, remaining challenges. Int. J.
Sustain. High Educ. 13, 211e218. https://doi.org/10.1108/14676371211242535.
Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., 2008. Handbook on Constructing
Composite Indicators: Methodology and User Guide. OECD Publishing, Paris.
Priyadarshini, P., Abhilash, P.C., 2020. From piecemeal to holistic: introducing sus-
tainability science in Indian Universities to attain UN-Sustainable Development
Goals. J. Clean. Prod. 247 https://doi.org/10.1016/j.jclepro.2019.119133.
Ros, F.V., Magro, S., Fore
s, I.M., 2019. Anthropocene, the challenge for" Homo sa-
piens" to set its own limits. Cuadernos Invest. Geogr. 45, 33e59. https://doi.org/
10.18172/cig.3681.
Salovaara, J.J., Soini, K., Pietika €inen, J., 2019. Sustainability science in education:
analysis of master’s programmes’ curricula. Sustain. Sci. 15, 901e915. https://
doi.org/10.1007/s11625-019-00745-1.

nchez-Carracedo, F., Soler, A., Martín, C., Lo
Sa
pez, D., Ageno, A., Cabre
, J., Garcia, J.,
Aranda, J., Gibert, K., 2018a. Competency maps: an effective model to integrate
professional competencies across a STEM curriculum. J. Sci. Educ. Technol. 27,
448e468. https://doi.org/10.1007/s10956-018-9735-3.
12
Journal of Cleaner Production 294 (2021) 126322

nchez-Carracedo, F., Segal
Sa
pez, D., Cabre
as, J., Vidal, E., Martín, C., Climent, J., Lo
, J.,
2018b. Improving engineering educators’ sustainability competencies by using
competency maps: the EDINSOST project. Int. J. Eng. Educ. 34, 1527e1537.
Sa


nchez-Carracedo, F., Alvarez, M.J., Barro
n, A., Caballero, D., Lo
pez, E., Mun~ oz, J.M.,
Lugo-Mun ~ oz, M., Sureda, B., Vidal, E., Vidal, S., 2018c. Elaboracio
n de un cues-
tionario para evaluar el nivel de sostenibilidad de los estudiantes de grados en
ingeniería TIC in XXIV Jornadas de Ensen ~ anza Universitaria de la Inform
atica,
JENUI 2018. Actas de las Jenui, Espan ~ a, pp. 141e148.

nchez-Carracedo, F., Moreno-Pino, F.-M., Sureda, B., Antúnez, M., Gutie
Sa
rrez, I.,
2019. A methodology to analyze the presence of sustainability in engineering
curricula. Case of study: ten Spanish engineering degree curricula. Sustain-
ability 11 (17), 4553. https://doi.org/10.3390/su11174553.
Segalas, J., S

anchez-Carracedo, F., 2019. The EDINSOST Project: Improving Sustain-
ability Education in Spanish Higher Education in Proceedings of the 19th
ERSCP- European Roundtable for Sustainable Consumption and Production,
pp. 217e240. Barcelona.
Sibbel, A., 2009. Pathways towards sustainability through higher education. Int. J.
Sustain. High Educ. 10, 68e82. https://doi.org/10.1108/14676370910925262.
Stephens, J.C., Graham, A.C., 2010. Toward an empirical research agenda for sus-
tainability in higher education: exploring the transition management frame-
work. J. Clean. Prod. 18, 611e618. https://doi.org/10.1016/j.jclepro.2009.07.009.
Stevenson, R.B., Lasen, M., Ferreira, J.A., Davis, J., 2017. Approaches to embedding
sustainability in teacher education: a synthesis of the literature. Teach. Teach.
Educ. 63, 405e417. https://doi.org/10.1016/j.tate.2017.01.013.
Stough, T., Ceulemans, K., Lambrechts, W., Cappuyns, V., 2018. Assessing sustain-
ability in higher education curricula: a critical reflection on validity issues.
J. Clean. Prod. 172, 4456e4466. https://doi.org/10.1016/j.jclepro.2017.02.017.
Svanstro€m, M., Gro €ndahl, F., Mulder, K.F., Segal as, J., Ferrer-Balas, D., 2012. How to
educate engineers for/in sustainable development. Int. J. Sustain. High Educ. 13,
211e218. https://doi.org/10.1108/14676371211242535.
Tejedor, G., Segalas, J., Rosas-Casals, M., 2018. Transdisciplinarity in higher educa-
tion for sustainability: how discourses are approached in engineering educa-
tion. J. Clean. Prod. 175, 29e37. https://doi.org/10.1016/j.jclepro.2017.11.085.
Tejedor, G., Rosas-Casals, M., Se galas, J., 2019. Patterns and trends in engineering
education in sustainability. Int. J. Sustain. High Educ. 20, 360e377. https://
doi.org/10.1108/IJSHE-07-2018-0131.
Thurer, M., Tomasevic, I., Stevenson, M., Qu, T., Huisingh, D., 2018. A systematic
review of the literature on integrating sustainability into engineering curricula.
J. Clean. Prod. 181, 608e617. https://doi.org/10.1016/j.jclepro.2017.12.130.
Trencher, G., Vincent, S., Bahr, K., Kudo, S., Markham, K., Yamanaka, Y., 2018. Eval-
uating core competencies development in sustainability and environmental
master’s programs: an empirical analysis. J. Clean. Prod. 181, 829e841. https://
doi.org/10.1016/j.jclepro.2018.01.164.
UN, 2015. Transforming Our World: the 2030 Agenda for Sustainable Development.
Resolution Adopted by the General Assembly on 25 September 2015. http://
www.un.org/ga/search/view_doc.asp?symbol¼A/RES/70/1&amp;Lang¼E.
accessed 10 March 2020.
UNECE, 2012. Learning for the Future. Competences in Education for Sustainable
Development nece.org/fileadmin/DAM/env/esd/ESD_Publications/Compe-
tences_Publication.pdf (accessed 10 March 2020).
UNESCO, 2017. Education for Sustainable Development Goals: Learning Objectives.
https://unesdoc.unesco.org/ark:/48223/pf0000247444. Accessed 20 July 2020.
von Carlowitz, H.C., 1713. Sylvicultura oeconomica. Hausswirthliche Nachricht und
Naturma ۧige Anweisung zur Wilden Baum-Zucht. https://www.forstbuch.de/
wp-content/uploads/2018/04/Carlowitz_1713_Einleitung.pdf. accessed 10
March 2020.
Wiek, A., Withycombe, L., Redman, C.L., 2011. Key competencies in sustainability: a
reference framework for academic program development. Sustain. Sci. 6,
203e218. https://doi.org/10.1007/s11625-011-0132-6.
Wiek, A., Bernstein, M., Foley, R., Cohen, M., Forrest, N., Kuzdas, C., Kay, B., With-
ycombe Keeler, L., 2015. Operationalising competencies in higher education for
sustainable development. In: Barth, M., Michelsen, G., Rieckmann, M.,
Thomas, I. (Eds.), Routledge Handbook of Higher Education for Sustainable
Development. Taylor and Francis Group, London, pp. 241e260.
Zamora-Polo, F., Sa
nchez-Martín, J., 2019. Teaching for a better world. Sustainability
and Sustainable Development Goals in the construction of a change-maker
university. Sustainability 11, 4224. https://doi.org/10.3390/su11154224.