See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/313587085

Model for contractor performance evaluation in construction industry

Conference Paper · October 2016

DOI: 10.1109/SMC.2016.7844636

CITATIONS READS

4 1,043

3 authors:

Maria Creuza Borges De Araújo Luciana Hazin Alencar

Universidade Federal de Campina Grande (UFCG) Federal University of Pernambuco
17 PUBLICATIONS   106 CITATIONS    31 PUBLICATIONS   464 CITATIONS   

SEE PROFILE SEE PROFILE

Caroline Maria de Miranda Mota

Federal University of Pernambuco
56 PUBLICATIONS   748 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Studies on Crime and Violence in Pernambuco View project

Modelo multicritério para avaliação e gestão de projetos em desenvolvimento nas organizações View project

All content following this page was uploaded by Maria Creuza Borges De Araújo on 16 March 2020.

The user has requested enhancement of the downloaded file.

 2016 IEEE International Conference on Systems, Man, and Cybernetics • SMC 2016 | October 9-12, 2016 • Budapest, Hungary
Model for Contractor Performance Evaluation in
Construction Industry
Maria Creuza Borges de Araújo
Department of Exact, Technological and Human Sciences
Universidade Federal Rural do Semi-árido
Angicos, Brazil
creuza.borges@ufersa.edu.br
Abstract— The success or failure of a construction project
depends largely on the contractor’s performance since they are
responsible for important activities in the building process. In
this way, it is necessary to monitor the performance of the
contractors who already working with the organization to control
failures during the development of work. Due the complexity and
necessity of considering various criteria in the decision making; it
is a multicriteria problem. Additionally, contractor evaluation
method should classify them according to performance levels to
help the client to decide whether to hire the contractor for future
projects. In this context, the model proposed uses ELECTRE
TRI method to classify the construction contractors according to
their performance. Subsequently, a literature review was made to
identify the most used criteria to assess contractors in the
construction industry. Then, these criteria were used in a
numerical simulation of the proposed model.
Keywords— Contractor Performance Evaluation, Construction
Industry, ELECTRE TRI
I. INTRODUCTION
Performance evaluation of contractors who works with the
company is an important activity in the construction industry
since they influence the success of projects in this sector. It is
critical because suppliers have a great impact on the
operational and strategic performance of organizations [1]. In
this way, [2] affirms that contractor performance should be
monitored and controlled on a regular basis so that any
undesirable shortfalls or failures could be identified and
rectified without further losses or delays. Therefore, the
contractor evaluation model, items, criteria and policy are the
most important issues for the employer in implementing a
construction project [3].
Due the necessity of considering various factors to an
adequate contractor evaluation, this is considered a
multicriteria problem ([1], [4], [5], [6]). The choice of criteria
depends on the type of industry, client's needs, stakeholder's
preferences and so on. [5] considered the ratio of start on time,
ratio of finish on time, ratio of the accuracy of payments, right
first time and health and safety inspections to evaluate
contractors in highway projects. To assess the performance of
contractors in complex product projects, [7] consider the
following criteria: quality, costs, delivery ability, cooperation
ability, competition ability and service ability. Quality
standards ([2], [8], [9], [10]), personnel ([2], [9], [11]), health
and safety ([8], [11], [12], [13], [14]) and cooperation relations
978-1-5090-1897-0/16/$31.00 ©2016 IEEE
Luciana Hazin Alencar, Caroline M. Miranda Mota
Department of Management Engineering,
Universidade Federal de Pernambuco
Recife, Brazil
alencarlh@gmail.com; carol3m@gmail.com
([2], [7], [10], [12]) are very used in project suppliers
evaluation.
Moreover, it is necessary choice an adequate method of the
contractor's performance assessment. In this way, the literature
presents the use of different methods to the evaluation of
contractors in construction industry: Stochastic Quality
Function Deployment [9], Analytic Network Process [13],
Gray Relevancy Model [15], Multi-Attribute Analysis [2],
Optimum performance method [16] and Weighted average
[13]. It was observed that the methods used in papers cited
results in an ordering of contractors. However, decision-
making techniques to help evaluate current suppliers should
aim to classify performance of individual suppliers against
desired levels of performance so as to devise suitable action
plans to increase supplier's performance and capabilities [5].
In this way, this paper proposes a contractor evaluation
model to construction industry which uses the ELECTRE TRI
method to classifying the contractors according to their levels
of performance. This model can help the companies to create
action plans to increase contractor's performance during the
project like as decide whether to hire the contractor for future
projects. Moreover, a numerical simulation of the method is
present. The criteria used in this simulation were identified in a
literature review about the criteria most used in contractor
evaluation in construction industry.
II. ELECTRE TRI
ELECTRE TRI is a well-known sorting method based on
an outranking relation [17]. Therefore, its application results in
the assignment of alternatives in predefined ordered classes
which are defined by limit profiles.
Given a set of alternatives A = {a1, a2, …,ai), these
alternatives should be assigned to predefined classes {C1, …,
Ch+1, …, Cp+1) according to a set of criteria K = {k1,…,
ki,…,km}. C1 is the worst class and Cp+1is the best class. The
categories are defined according to the limit profiles {b0, …,bh,
…, bp), where b0 is the lower limit profile and bp is the upper
limit profile. Each class Ch is delimited by its lower limit (bh)
and upper limit (bh+1).ELECTRE TRI method can be
summarized as follows.
Firstly, the partial concordance index cj(a,b), concordance
index c(a,b) and partial discordance index dj(a,b) should be
SMC_2016 002631
 2016 IEEE International Conference on Systems, Man, and Cybernetics • SMC 2016 | October 9-12, 2016 • Budapest, Hungary

calculated as shown in Equations 1, 2 and 3. The concordance compared with other approaches. Moreover, the authors cited
index considers the criteria in which ‘a outranks b’. that another advantage is that the classification of an action
(solution) is independent of the classification of other actions.
Furthermore, this method is non-compensatory and can
0, ( )− ( )≥ ( ) consider heterogeneous criteria.
1, ( )− ( )≤ ( ) III. PROPOSED MODEL
( , ) = (1)
( ) ( ) ( )
ℎ The proposed model aims to classify the contractors that
( ) ( )
working with the construction firm and, based on the class to
which the contractor is assigned, decided about hire it or not in
∑ future projects. The model is shown in Figure 1.
( , ) = (2)
∑
Decision Maker

0, ( ) − ( ) ≤ ( )
1, ( ) − ( ) > ( ) Identification of contractors
( , ) = (3)
( ) ( ) ( )
, ℎ
( ) ( )

Determination of evaluation
criteria
Where,
pj (bh) = preference threshold
Application of ELECTRE TRI
qj (bh) = indifference threshold
vj = veto threshold
Subsequently, the credibility index σ (a,bh)∈[0,1] must be Contractor’s classification
measured (Equation 4).
( , )
σ (a, b ) = c (a, b ) . ∏ ∈ )
(4)
( , Good Moderate Bad
Where, K j ∈ K: d (a, b) > (a, b )
Given the credibility index, a credibility level, denoted by Hire the Development Do not hire
λ, should be defined, representing the minimum value of contractor in program to hire the contractor
σ(a,bh) in order to validate or not the outranking relation [18]. future projects contractor in in future
Subsequently, the values of σ(a,bh), σ(bh,a)and λ determines the future projects
preferences relations between ‘a’ and bh:
σ (a,bh) ≥ λ and σ (bh,a) ≥ λ → ‘a’ is indifferent to bh. Fig. 1. Proposed Model
σ (a,bh) ≥ λ and σ (bh,a) < λ → ‘a’ is preferable to bh.
Initially, the firm should determine a decision maker to the
σ (a,bh) < λ and σ (bh, a)≥ λ → bh is preferable to ‘a’. evaluation. This person should know the needs and preferences
σ (a,bh)< λ and σ (bh, a)< λ → ‘a’ is incomparable to bh. of organization to conduct the process in an appropriate
manner.
Two procedures can be used to the assignment of
alternatives in classes: After, the decision maker will identify the contractors that
work with the company. Due the cost and time dispended in
Optimistic assignment: ‘a’ is compared successively to the process, only strategic ones should be evaluated.
b0,…,bh until bh outranks a. ‘a’ is assigned to the lowest Following, it is necessary to identify the evaluation criteria
category Ch for which the upper profile bh is preferred to a. which will be used in the model. For the criteria choice, it is
Pessimistic assignment: ‘a’ is compared successively to bp, necessary to observe the factors that influence the decision
bp-1, … until ‘a’ outranks bh where h ≤ p. The alternative 'a’ is making according to the needs of company.
then assigned to the highest category Ch if a is preferable to bh. After, ELECTRE TRI will be applying. The use of this
According to [19] one advantage of this method is that the method is related to several questions. Firstly, ELECTRE TRI
comparisons are done between an action 'a’ and a reference can achieve the aim of model, which is assigning the
profile b, in opposition to other methods in which the contractors to predefined ordered classes. Secondly, the
comparisons are made between all the actions, so that the method could be used with heterogeneous criteria. Thirdly,
number of comparisons in this method is inferior when

SMC_2016 002632
 2016 IEEE International Conference on Systems, Man, and Cybernetics • SMC 2016 | October 9-12, 2016 • Budapest, Hungary

ELECTRE TRI offers to the decision maker the flexibility to disregarding ‘other’ category, were used as the criteria of
define the optimistic and pessimist point of view. simulation, as shown in Table I:
Moreover, in this approach, the classification of one TABLE I.
contractor is independent of the classification of others. SUPPLIER EVALUATION CRITERIA
Furthermore, ELECTRE TRI presents non-compensatory
Category Author
evaluation. As said by [20], this kind of method avoids that
contractors with higher performance in some criteria and very Relations (29) [2], [7], [9], [11], [12], [13], [14]
low performance in others obtain good results. This is Management (28) [2], [7], [9], [11], [12], [13], [14], [16]
important because several companies prefer contractors with
[2], [7], [9], [11], [12], [13], [14],
well-balanced performance. Finally, as comparisons are made Quality (22)
[16], [22]
between the alternatives and the reference profiles and not
Personnel (12) [2], [7], [13]
between all the alternatives, the number of comparisons is
inferior in relation to others methods. Time (11) [2], [7], [9], [11], [12], [13], [16], [22]
The model results in the classification of contractors in Financial (10) [2], [13], [14]
three predefined classes:
Good contractors: contractors that are adequate to the The criteria, codification, and weights are exposed in Table
company’s need and have positive results in the evaluation. In II.
this way, the business should contract them in future projects.
TABLE II
Moderate contractors: they are capable of attending firms CRITERIA, CODIFICATION AND WEIGHTS
expectative, but need to improve some problems in the Code Criterion Weight
performance. The company needs to inform them about the
problems and only accept new contracts after the improvement C1 Relations 0.17
of contractor performance. C2 Management 0.15
Bad contractors: they have poor results in the performance C3 Quality 0.18
evaluation and the construction company should not contract
them in future projects. C4 Personnel 0.16

According to [4] performance evaluation must be done C5 Time 0.17

periodically to maintain the performance levels that company’s
desire.
IV. NUMERICAL APPLICATION
It was made a simulation of the proposed model in the
construction industry. Simulation is an interactive process
which has the advantage the fact of reveals important
information and new perceptions about the problem and
permits improvements in the model before its application in
real situations.
With this purpose, it was structured a typical problem in
which the performance of contractors that who already
working with the company in a project were evaluate. As a
result, the contractors are assigned to predefined categories
which determine if the firm should hire them to future projects
or not.
The LAMSADE's MCDA software [21] was used in the
application.
A. Application
Initially, it was defined the decision maker to apply the
model. In this case, ten contractors that work with the firm
were evaluated: Co1, Co2, Co3, Co4, Co5, Co6, Co7, Co8, Co9,
Co10.
Following, it was made a Literature Review to identify the
criteria most used for construction industries to evaluate
contractors in projects. As a result, the criteria found are
divided into categories. The six categories more cited,
C6 Financial 0.17
The criteria can be described as follows:
a. Relations: possibility of construct a mutual and
trustworthy dependency between client and supplier, through
information exchange and accorded achievements. The
alternative evaluation occurs in a subjective way according
with the following preference levels: Very good, good,
regular, bad and very bad.
b. Management: considers the general management
conditions of the contractor, like as management systems and
control, management knowledge, work management and
integration management. The alternative evaluation occurs in a
subjective way according with the following preference levels:
high, medium, low and very low.
c. Quality: considers with the product/service is according
with the quality specifications. For quality measurement, it was
considered the percentage of items or services according to the
specifications in relation with the total number of items
received.
d. Personnel: considers the quality of construction
personnel, observing questions like as potential, adequacy of
personnel to the type of work, specialization, amount and
experience of personnel. The alternative evaluation occurs in a
subjective way according with the following preference levels:
high, medium, low and very low.

SMC_2016 002633
 2016 IEEE International Conference on Systems, Man, and Cybernetics • SMC 2016 | October 9-12, 2016 • Budapest, Hungary

e. Time: is related with the delivery of product/service at TABLE V

PARAMETERS
the agreed period. The time criterion was measured considering
the percentage of deliveries made at the right time. Criteria
Parameters
f. Financial: It is related with the general economic C1 C2 C3 C4 C5 C6
conditions of the contractor, like as profits, interest rates, Indifference
0.25 1 5 1 5 0.25
Threshold
market position, among others. The alternative evaluation Preference
occurs in a subjective way according with the following 0.50 2 10 2 10 0.50
Threshold
preference levels: Very good, good, regular, bad and very bad. Veto
0.75 2 12 2 10 0.50
Threshold
For the criteria which were qualitative and assess from
opinions of the decision maker, the verbal scales were Moreover, the value of credibility level (λ) in this
converted in numerical scales, like as show in Table III. simulation was 0.7. It was observed the optimistic and
pessimistic views in the application of the method.
TABLE III
SCALES OF JUDGMENT B. Results
Scale of Judgment for Scale of Judgment for As a result, both optimistic and pessimistic views of
Relation and Financial Management and Personnel ELECTRE TRI assigned contractors in three classes, as shown
Verbal Numerical Verbal Numerical in Table VI.
Very Good 1.00 High 3.00
TABLE VI
Good 0.75 Medium 2.00 ASSIGNMENT OF CONTRACTORS IN CLASSES

Regular 0.50 Low 1.00 Contractors

Class
Bad 0.25 Very Low 0.00
Good Co1, Co3, Co5, Co9
Very Bad 0.00
Moderate Co4, Co7, Co8, Co10

Bad Co2, Co6

In order to assign contractors into classes, the limit profiles
were determined. These profiles values represent what is
expected of contractors of certain categories. Table IV In accordance with the results of the application, the firm
represents the limit profiles considered in this simulation. should hire the contractors Co1, Co3, Co5, Co9 in future projects
because they are adequate to the needs of the client,
TABLE IV considering the predefined criteria. The contractors Co4, Co7,
LIMIT PROFILES
Co8, Co10 were evaluated as medium and need improvements
Limit Criteria to contract in future projects. Finally, contractors Co2, Co6
Class Profile should not be hired in future works due the fact that not
C1 C2 C3 C4 C5 C6
Good L3 1 3 100
achieve an acceptable performance level regarding the criteria
3 100 1
chooses.
Moderate L2 0.75 2 90 3 85 0.75
It is important that the company give feedback to the
Bad L1 0.25 1 75 1 75 0.25 contractors, exposing the class they were assigned and the
L0 0.00 0 0 0 0 0.00
implications of this classification.
The initial credibility level used in the simulation was 0.7.
The preference, indifference and veto thresholds used in the To verify the sensibility of the method to variations in this
method are defined. To asset the preference thresholds the level, it was simulated the results with credibility levels of 0.6,
analyst considers the lower difference between the functions of 0.8 and 0.9. It was observed that the categorization was the
alternatives compatible with the preference of one alternative same to credibility levels of 0.6 and 0.7. In the case of
under the other in each criterion. To define the indifference credibility levels of 0.8 and 0.9, the results are the same. In this
threshold it was consider the bigger difference between two case, the criteria in Class ‘Good' had not changed, while Co2
functions that preserve the indifference of two alternatives in was assigned in ‘Moderate' class at the optimistic view and
each criterion. Finally, the veto threshold it was asset continues in ‘Bad' class at a pessimistic view. Therefore, it was
considering the lower difference between the functions of two observed that this variation did not modify the results in a
alternatives incompatible with the outranking of one alternative significant way.
under the other in each criterion.
V. FINAL CONSIDERATIONS
These thresholds are employed to minimize imperfections
in alternative’s evaluation. Table V presents these parameters. Contractor's evaluation is a strategic question in the
construction industry since the contractors are essential to the
achievement of results in this sector. This assessment is so
important because even if the firm has a structured selection
process and choice adequate contractor but not evaluate the

SMC_2016 002634
 2016 IEEE International Conference on Systems, Man, and Cybernetics • SMC 2016 | October 9-12, 2016 • Budapest, Hungary
performance of them after the contract, it is possible that the
performance level of the contractor decreases over the time and
the company will not recognize this due the lack of assessment.
Moreover, it is essential to employ structured models to
contractor evaluation. In the application of these models, it is
important to use criteria adequate to the needs and preferences
of decision makers, like as methods that are in accordance with
the type of criteria, problematic and information available. Due
the need of considers various factors in the assessment of
contractors, this is considered a multicriteria decision model.
In this context, it was proposed a multicriteria model for the
evaluation of contractors that already work with contractor
firms. It is observed that in this type of problem it is not
interesting only order the contractors according to the
performance level, but assigning them in different categories
that help the decision maker to decide about contract them or
not in future projects considering their performance during the
project duration. In this way, ELECTRE TRI was employed in
order to assign contractors in different classes that help firms in
future business.
Moreover, ELECRE TRI could be used with qualitative
and quantitative criteria, which are employed in big part of
problems. This method has a non-compensatory logic, such
that high performances in some criteria do not compensate very
low performance in others and normally contractors with
equilibrate performances are assigned to better classes.
Additionally, the assessment of one contractor does not depend
on the others, since the comparisons are between alternatives
and limit profiles. Because of this, the number of comparisons
is inferior to other methods and less effort is necessary to use
the method. Furthermore, this type of comparison ensures
stability for the entry of new contractors. Finally, the decision
maker has the flexibility of choice between a pessimistic or
optimistic point of view according to his preferences.
A numerical application was made to demonstrate the
method. In the simulation, it was employed the criteria
identified in a systematic review that found the factors most
used in contractors evaluation at construction projects. It was
perceived that this model can be used in the evaluation of
suppliers in others sectors, adjusting the criteria to each
situation. The model can help the firms in the improvement of
contractor’s management and to choose adequate partners in
future projects.
ACKNOWLEDGMENT
The authors would like to thank CNPq (Conselho Nacional
de Desenvolvimento Científico e Tecnológico) for their
financial support.
REFERENCES
[1] F. Cebi and I. Otay, “A two-stage fuzzy approach for supplier evaluation
and order allocation problem with quantity discounts and lead time,”
Information Sciences, vol. 339, pp. 143–157, 2016.
[2] S. T. Ng, E. Palaneeswaran and M.M. Kumaraswamy, “A dynamic e-
Reporting system for contractor’s performance appraisal”, Advances in
Engineering Software, vol. 33, pp. 339-349.
View publication stats
[3] M.C.B Araújo and L.H. Alencar, “Integrated model for supplier
selection and performance evaluation,” South African Journal of
Industrial Engineering, vol. 26, pp. 41-55, August 2015.
[4] [4] L. Osiro, F.R. Lima-Junior and L.C.R. Carpinetti, “A fuzzy logic
approach to supplier evaluation for development,” International Journal
of Production Economics, vol.153, pp. 95–112, 2014.
[5] T. Lam and K. Gale, “Highway maintenance: impact of framework
agreements on contractor performance,” Engineering, Construction and
Architectural Management, vol. 21, pp. 336 – 347, 2014.
[6] A.T. de Almeida, R.J.P. Ferreira, C.A.V. Cavalcante. A review of the
use of multicriteria and multi-objective models in maintenance and
reliability IMA Journal of Management Mathematics (2015) 23 pages.
doi:10.1093/imaman/dpv010
[7] D. Lee, T. Lim and D. Arditi, “Automated stochastic quality function
deployment system for measuring the quality performance of
design/build contractors”, Automation in Construction, vol. 18, pp. 348-
356, 2009.
[8] C. Mi, Z. Ma and Z. Ding, "A linguistic evaluation model considering
gray information and its application on complex product supplier
performance," Journal of Grey System, vol. 25, pp. 34-43.
[9] H. Xiao and D. Proverbs, “Factors influencing contractor performance:
an international investigation,” Engineering, Construction and
Architectural Management, vol. 10, pp. 322 – 332, 2003.
[10] A. Ebrahimi, M. Alimohammadlou and S. Mohammadi, S,
“Identification and prioritization of effective factors in assessment and
ranking of contractors using fuzzy multi-criteria techniques,” Decision
Science Letters, vol. 5, pp. 95-108, 2015.
[11] D.C.A. Butcher and M.J. Sheehan, “Excellent contractor performance in
the UK construction industry,” Engineering, Construction and
Architectural Management, vol. 17, pp. 35 – 45, 2010.
[12] S. Maturana, L.F. Alarcón, P. Gazmuri, and M. Vrsalovic, “On-Site
Subcontractor Evaluation Method Based on Lean Principles and
Partnering Practices,” Journal of Management in Engineering, vol. 23,
pp. 67-74, 2007.
[13] P. Rashvand, M.Z.A. Majid and J.K. Pinto, “Contractor management
performance evaluation model at prequalification stage”, Expert
Systems with Applications, vol. 42, pp. 5087-5101, 2015.
[14] W. Zhang, Y. Yang, and W. Liu, “Contractor Evaluation Model For
Large Project Based On Grey Relevancy,” Applied Mechanics and
Materials, pp. 357-360, vol. 2384-2387, 2013.
[15] W. Zhang, Y. Yang, and W. Liu, “Contractor Evaluation Model For
Large Project Based On Grey Relevancy,” Applied Mechanics and
Materials, pp. 357-360, vol. 2384-2387, 2013.
[16] R. Soetanto, D.G. Proverbs, and G.D. Holt, “Achieving quality
construction projects based on harmonious working relationships -
Clients’ and architects’ perceptions of contractor performance,”
International Journal of Quality & Reliability Management, vol. 18, pp.
528-548, 2001.
[17] J. Zheng, S. A. M. Takougang, “Learning criteria weights of an
optimistic ELECTRE TRI sorting rule,” Computers & Operations
Research, vol. 49, pp. 28–40, 2014.
[18] D. V. S. Pereira and C. M. M. Mota, “Human Development Index Based
on ELECTRE TRI-C Multicriteria Method: An Application in the City
of Recife,” Social Indicators Research, vol. 125, pp. 19-45, 2016.
[19] E. Oliveira and C. H. Antunes, “An Evolutionary Algorithm based on an
outranking relation for sorting problems,” IEEE International
Conference on Systems Man and Cybernetic, pp. 2732-2739, 2010.
[20] M.C.B. Araújo, L.H.Alencar, C.M.M.Mota, “Contractor Selection in
Construction Industry: a Multicriteria Model”, IEEE International
Conference on Industrial Engineering and Engineering Management, pp.
519 – 523, 2015.
[21] Université Paris Dauphine. LAMSAD SOFTWARE. Available at
http://www.lamsade.dauphine.fr/spip.php?rubrique67.
[22] H. Xiao and D. Proverbs, “The performance of contractors in Japan, the
UK and the USA,”International Journal of Quality & Reliability
Management, vol. 19, pp. 672 – 687, 2002.
SMC_2016 002635