Optimization of decision system for Turnkey project procurements

A Tsung-Shi Liu & B Ting-Ya Hsieh

Graduate Institute of Construction Engineering and Management, National Central University, Taiwan

Abstract
The research attempts to optimize the decision system for Turnkey project procurements. The idea is that if the paper-based drawings and information can be linked to a BIM model, the MAT evaluation of proposals is visualized. To establish the BIM-based MAT decision system, the key evaluation criteria need to be identified in advance. Next, develop the function of information comparison among the bidders proposals directly by BIM. Then an appropriate algorithm is adopted to conduct dynamic simulations to detect the difference among proposals. According to the difference, the panel can provide a score for each evaluation item by an objective standard, and finally decide the best tender. The results of this research will effectively improve MAT evaluation problems and advance efficiency and quality of Turnkey project procurements.
Keywords: Turnkey project, Most advantageous tender (MAT), Evaluation criteria, Decision system, Building information model (BIM).

Introduction

The number of public construction projects delivered by the Turnkey method is few in Taiwan. There are two main reasons why the method cannot be successfully promoted. One of them is about problems of the Most Advatangeous Tender (MAT) evaluation. Due to adoption of MAT, the agency has to establish evaluation rules and invite specialists to form a panel and evaluate tenders, but people often question the equity and transparecny of the evaluation processes. Besides, the magnitude (or total price) of a Turnkey project is in average larger than that of a project delivered by the traditional method. Appropriately evaluating the bidders proposals and selecting the contractor becomes the key to promoting the Turnkey method. As to evaluation of tenders, the invited specialists need provide a score or ranking for each evaluation item. However, based on conceptual drawings and information in the bidders proposal, presentation, and small-scale demonstration model at most, the evaluation panel has to make a decision in a short time and without thorough discussions. The specialists do feel much responsibility and pressure because the processes are too hasty and not serious enough. These problems cause the bottleneck of promoting MAT. The research attempts to optimize the decision system for Turnkey project procurements. If the paper-based drawings and information can be linked to a BIM model, the MAT evaluation of proposals is visualized. Then the owner and the panel can discuss the bidders proposal in details and at ease. The proposal can even be adjusted on the spot according to the panels suggestions.

Through thorough discussions, a decision on which proposal is the best for the owners requirements is made. To establish the BIM-based MAT decision system, the key evaluation criteria need to be identified in advance. Next, develop the function of information comparison among the bidders proposals directly by BIM. Then an appropriate algorithm is adopted to conduct dynamic simulations to detect the difference among proposals. According to the difference, the panel can provide a score for each evaluation item by an objective standard, and finally decide the best tender. The results of this research may contribute to improve evaluation and decision problems in Turnkey projects, and to advance efficiency and quality of public procurements. The scope of this research is limited on public buildings, and mainly focused on the evaluation criteria such as technology, quality, function, and management. The methodology of this research includes literature review and interviews.

2
2.1

Literature review
Turnkey project procurements and MAT evaluation

2.1.1 Turnkey project procurements According to FIDIC, there are two standard forms of contract on a turnkey basis: one is Design-Build (DB) and the other is EPC. The Turnkey method has the following advantages compared to the traditional delivery method: a. reducing interfaces between design and construction; b. lowering disputes between the contractor and the owner; c. shortening project duration; d. reducing risks; e. simplifying management. Therefore, a variety of countries including Taiwan make great efforts to promote the Turnkey method for public procurements. The Government Procurement Law promulgated in 1999 specifies that An entity may, according to the needs of efficiency and quality, conduct the procurement on a turnkey basis. Furthermore, the definition of Turnkey by the Law includes detailed design and construction, and may extend to basic design, testing, training, maintenance or operation for a specified period, etc. 2.1.2 Evaluation of most advantageous tender (MAT) The most advantageous tender (MAT) is an awarding approach different from the lowest price tender (LPT). Price is not the only award criteria in MAT so it can avoid bidders using low price strategy to win the tender and then supply low quality services. The Law includes the awarding approach and allows government entities to conduct MAT as long as tenders for construction work are with different qualities. The most important task of MAT is evaluation of tenders. As to the task, the Law specifies that the evaluation criteria shall be used to determine the most advantageous tender by comprehensively evaluating the technology, quality, function, commercial terms, or price of the tenders with ranking or score. The evaluation items and sub-items for selecting the MAT, according to the Law, may be chosen from the following. Herein commercial terms, past achievements in contract performance and financial plan are omitted because of limited research scope. a. technology: such as professional manpower or expertise, technical feasibility, construction methods, degree of environmental protection, landscape preservation, ecological conservation, completeness of plan, degree of understanding of the subject procurement, etc.; b. quality: such as quality control capability, safety, stability, reliability, aesthetic attractiveness, comfort of use, durability, service life, etc.; c. function: such as handiness, expandability, compatibility, adaptability, special effects, etc.;

d. management: such as work interface handling, schedule management, site management, safety and health management, project management capability, subcontracting plan, etc.; e. price: such as the completeness and reasonableness of total tender price and its composition, cost control measures, subsequent operating costs, maintenance costs, cost effectiveness, etc.; f. any other matters related to the function or benefit of procurement. Regarding scores or weights allotted for evaluation items, the procurement entity can decide by itself or refer to the example shown in Table 1 published by the Public Construction Committee (PCC).
Table 1. MAT evaluation items and scores for Turnkey projects (PCC, 2011). Evaluation items Scores (100 in total) 10 1 Past achievements in contract performance 5 2 Quality control plan 5 3 Site, safety and health management plan 5 4 Proposed schedule and its appropriateness 25 5 Function and quality of proposed materials and equipments 6 7 8
Entire design ideas (e.g., functionality, aesthetics, convenience, economy, creation, etc.) Professional expertise and capability of project participants Completeness and rationality of estimated price 25 5 20

2.2

Status quo of Turnkey in Taiwan and MAT evaluation problems

2.2.1 Status quo of Turnkey adopted in public procurements According to the statistics published by PCC in 2011, the number of Turnkey projects is about 0.5% of that of all procurements every year. The ratio is relatively low. There are many reasons why Turnkey cannot be fully promoted since 1999, but one of them are critical. For adopting MAT the agency has to establish evaluation rules and invite specialists to form a panel. However, people often question the equity and transparecny of evaluation processes. Besides, the price (or magnitude) of a Turnkey project is in average much more (or bigger) than that of a traditionally delivered project. It means that if the Turnkey contractor cannot be determined properly the risk of failure will be extremely high instead. Therefore, to establish a better decision system for evaluating bidders proposals and selecting the contractor becomes the key to promoting the Turnkey method. 2.2.2 Current MAT evaluation problems Firstly, the evaluation processes are based on conceptual drawings and information in the bidders proposal and presentation, and small-scale demonstration model at most. The evaluation panel has to make a decision in a short time and without thorough discussions. In other words, the panel can neither study such things as price vs. quality, constructability and LEED contents. in detail, nor compare the bidders proposal to the others. These cannot easily be done at once in the traditional paper-based proposal. Therefore, most invited specialists do feel much responsibility and pressure because the processes are so hasty and not serious enough. Also people often suspect the procurement agency has room to manipulate the processes. Secondly, most studies on MAT in Taiwan only focus on how to better determine evaluation criteria and allot respective weight or score. Few studies put emphasis on how to give a score with objective evaluation standard or quantified difference level. As to this, two suggestions are as follows: a. how to judge the bidders advantage and weakness in a fair and reasonable way is also a key to success. Thus it is necessary to analyze the scoring mechanism and principles, especially for those hard quantified or judged factors. (Chen, 2002) b. Except for determining evaluation items, sub-items and respective scores and weights, the objective standard and trade-off index of scoring level or basis should be included. Then the reviewers subjective minds can be reduced so as not to affect the evaluation results. (Liu, 2007)

3
3.1

Assisting MAT evaluation by BIM

Functions and applications of BIM

The Building Information Modeling (BIM) technique was developed to aid planning and design activities, but with the evolution of IT technique its functionality is being expanded. Currently BIM has the following functions and applications (illustrated in Figure 1). Not only can BIM make evaluation of tenders visualized, but also can effectively assist to improve most evaluation problems in Turnkey project procurements.

Figure 1. Functions and applications of BIM (Lott, 2009)

a. 3D demonstration: BIM is an object-oriented technique which allows complicated curved design and lets viewers visualize types and construction methods of an architecture from any angle. b. Elements with information: Every BIM element carries geometric information, and can save kinds of physical information. Hence, it is easy to access the information during evaluation processes. c. Auto-quantity take-off: With geometric information in the BIM, quantity can be auto-calculated conveniently. Once the design has changes, the number of quantity will be adjusted by itself. d. Clash detection: With suitable tools, clash or low constructability among objects in the BIM can be auto-detected and shown by a different color. It is helpful to check design faults. e. 3D simulations: Using BIM can easily explore more alternatives by value engineering. Also, the virtual construction planning can help to investigate constructability. f. Construction management: BIM can link both time and cost information, which means 4D and 5D functions. Thus it is easy to check the rationality of scheduling and cost structure or update them. g. Life-cycle considerations: With suitable software, analysis of building sustainability (6D) such as energy-saving, carbon-reduction, or LEED can be conducted. h. Operations and maintenance management: With proper tools, inspecting facility management, asset management, etc. in advance is convenient.

3.2

Modeling a BIM and its capability maturity

3.2.1 Modeling a BIM A BIM master model is likely to consist of a variety of models. The models that may be part of the BIM are as follows. Hence, the procurement entity should prescribe what models need to deliver in tendering documents for Turnkey project bidders. a. Site model (contextland, buildings, landscape, etc.) b. Architectural model (walls, floors, roof, etc.) c. Structural model (structural systems)

d. MEP models (mechanical, electrical, plumbing) e. FP model (fire protection) f. Specialty models (equipments, finishes, temporary constructionscaffolding, formwork, etc.) Besides, BIM can be developed into different types according to the level of development (LOD). The procurement entity should prescribe the required LOD in tendering documents. For example, a. Schematic or design model: This type has a lower LOD. It is generally established by the architect and includes components created by structural and mechanical engineers. Based on this model, the design ability of the contractor can be identified. b. Construction model: This type has a higher LOD. The model is assembled in most cases by the general contractor according to construction documents prepared by the architect. The construction ability of the contractor can be observed from this model. 3.2.2 BIM capability maturity evaluation The capability maturity of the BIM should be evaluated by the procurement entity before MAT evaluations. The task can be done according to Capability Maturity Model (CMM) designed by National Institute of Building Science (NIBS) in the U.S. The CMM matrix is shown in Table 2. It has 11 evaluation items and the credit score of every item is between 1 and 10. The evaluation team has to decide a weight of importance for each area of interest in advance, and then gives a score by the CMM matrix. Next, calculate the credit sum and finally determine the BIM maturity level. An interactive evaluation example is illustrated in Table 3.
Table 2. BIM Capability Maturity Model (NBIMS, 2006).
Maturity Level 1 2 3 E F G Business Timeliness/ Delivery Process Response Method Separate Most Single Point Processes Response Info Access No Not Manually reIA Expanded Few Bus Most Single Point Data Processes Response Info Access w/ Set Collect Info Manually re- Limited IA Enhanced Add Two Roles Aware of CM Some Bus Data Calls Not Network Data Construction/ Partially And Root Processes In BIM But Access w/ Set Supply Supported Cause Collect Info Most Other Basic IA Analysis Data Is . . . . . . . . . . . . . . . . . . . . . Full Supports Internal and Business All BP Real Time Netcentric Knowledge External External Roles processes are Collect & Access w/ SOA Role Efforts routinely Maintain In Live Feeds Based Management Supported sustained by Real Time CAC CM using RCA and Feedback loops A Data Richness Basic Core Data B C D Life-cycle Roles Or Change Views Disciplines Management No Complete No Single Role No CM Project Phase Fully Capability Supported Planning & Only One Role Aware of CM Design Supported H I J Graphical Spatial Information Information Capability Accuracy Primarily TextNot No Ground No Technical Spatially Truth Graphics Located 2D NonBasic Initial Intelligent As Spatial Ground Designed Location Truth NCS 2D Non- Spatially Limited Intelligent As Located Ground Truth Designed - Int Spaces K Interoperability /IFC Support No Interoperability
Forced Interoperability Limited Interoperability

. . .

10

. . . . . . . . . . . . nD - Time & Integrated Computed All Info Uses Cost into GIS w/ Ground Truth IFCs For Interoperability Full Info w/Full Flow Metrics

Table 3. The interactive Capability Maturity Model and an evaluation example (NIBS, 2007).
Area of Interest Data Richness Life-cycle Views ITIL Maturity Assessment Roles or Disciplines Business Process Delivery Method Timeliness/Response Graphical Information Spatial Capability Information Accuracy Interoperability/IFC Support Weighted Importance 84% 84% 90% 90% 91% 91% 92% 93% 94% 95% 96% Choose your perceived maturity level Credit Completely Authoritative Information Includes Operations & Warranty Limited Control Operations & Sustainment Supported All BP Collect & Maintain Info Real-time Access w/Live Feeds Web Enabled Services Secure 4D - Add Time Integrated into a complete GIS Computed Ground Truth w/Full Metrics Full Info Transfers Between COTS Credit Sum Maturity Level 6.7 5.9 4.5 7.2 7.3 9.1 7.4 8.4 8.5 9.5 5.8 80.1 Gold

4
4.1

Establishment of BIM-based MAT decision system for Turnkey projects

Determination of MAT evaluation criteria

Based on literature review, appropriate MAT evaluation items, sub-items and detailed contents for Turnkey projects may be referred to Table 4. The table is suitable for building projects and only shows three evaluation items: technology (function and quality included), management and price.
Table 4. Evaluation criteria of MAT for Turnkey projects (Chen, 2002).
Level 1 Evaluation items Level 2 Sub-items (sequenced by importance) Construction technology and capability Operation management technology Use of technical resources Level 3 Detailed contents

Technical capability

Management capability

Price

Applied construction methods and its feasibility Proposed solutions for issues in owners requirements Provided function and performance after completion Prominence of proposed operation plan Efforts to survey site and its surrounding conditions Quality and technical specifications of chosen materials Design contents Appropriateness of proposal and completeness of design specifications Considerations of green building and energy-saving Application of value Ideas of value engineering engineering Analysis of functions and performance of materials and equipments Contract management Proposed schedule and its appropriateness Establishment of procedures for quality inspection and testing Schedule control Completeness and rationality of design and construction scheduling plan Work interface handling of Turnkey Disaster disposal and safety Emergency response plan & health management Safety and health management plan Construction management Feasibility and completeness of construction plan Completeness of quality control plan Rationality of proposed price Rationality of estimated price and quantity Completeness of cost structure

4.2

Development of information comparison table by using BIM

To detect the difference among proposals more conveniently, the research proposes a comparison table into which BIM can automatically export information. The concept is illustrated in Figure 2. The aforementioned information comparison table should have the same items as those listed in Table 4. The information needed for evaluation can be divided into two groups. One group is about quantity which can be exported from BIM, and has two subgroups as follows. Embedded information in BIM: e.g. spatial and physical information, etc. External information linked to BIM: e.g. time, cost, etc. The other group is about quality. Such information needs BIM with proper tools to visualize. Visible information by BIM: e.g. landscape, construction sequences, constructability, etc. Hidden information identified with tools: e.g. LEED, O&M management, etc.
Sub-item Detailed item
Total floor area Design Bldg. coverage ratio Contents Quality of materials Duration Proposed schedule Quantity take-off Price Estimated cost

Bidder A

Figure 2. Concept of info comparison table by BIM (Model: Benjamin D. Hall Research Building at U. of WAS).

4.3

Creation of scoring matrix for MAT evaluation

The evaluation of MAT, except for determining evaluation criteria and respective weights, depends heavily on reviewers scoring. Therefore, it is necessary to have an objective standard of scoring level. Then the reviewers subjective minds can be reduced so as not to affect the evaluation results. However, no study on scoring standards for Turnkey projects can be found. The research proposes that the BIM Capability Maturity Model Matrix (shown in Table 2) be a good reference to create the scoring matrix for MAT evaluation. According to the perceived difference, the reviewer can provide a score for each evaluation item by this scoring standard, and finally decide the best tender.

4.4

Development of simulation model for MAT decision

Currently the decision method for MAT is traditional and single-wayed. That is, the bidder who gets the highest score or the lowest ranking wins award of tender. The research proposes the other way, which depends on the past experience to analyze whether the highest scorer is the best bidder. This task needs a MAT decision simulation model to implement. The idea of developing the simulation model is to utilize the Artificial Neuron Network (ANN) algorithm. Prior to doing so, similar Turnkey cases in the past need to be collected. The goal of this research is to collect about 50 public building Turnkey projects since 1999. And the information needed to collect includes project background, evaluation items and weights, number of reviewers, scoring component and total score of MAT, award conditions and final contractors performance, etc. Then the collected information is normalized and serves as input data of developing an ANN model. Secondly, it is necessary to define success and failure of contract performance. Then the level of success should be assigned such as best (e.g. 1 point), common (0.5), worst (0). With the definition and success level, output data can be obtained. Next, all data are randomly divided into two groups: one is for training and the other is for testing. The first group is used to train a stable ANN network, and the other to test reliability of this network. After that, the simulation model is formed.

4.5

Development of interactive scoring interface for MAT evaluation

Currently, reviewers have to write scores and comments down in the evaluation sheet given by the procurement entity. Then these sheets are collected and analyzed by manpower, which needs lots of effort. The research proposes a web-based interactive scoring interface illustrated in Figure 3.
Reviewer I Evaluation scores in all
Evaluation items 1 2 9 others Total score 4 80 3 72 technology quality Bidder A 18 9 B 16 8 1 2 9 others 5% Total score 4 80

Reviewer I Evaluation scores for Bidder A

Evaluation items technology quality weights 20% 10% Comments Good tech. Good qual. Scores 18 9

MAT evaluation scoring matrix

Sub-item: 1.1 Constr. tech. capability Evaluation Sub-items Constr. tech. capability Comments Terrible capability Bad capability Good capability Excellent capab. Scores 1 2 9 10

Reviewer I

Sub-item scores for Bidder A

Comments Good capability Good design Scores 9 9

Evaluation item: 1.technology Evaluation Sub-items Constr. tech. capability Design contents

1.1

1.1 1.2

Total score

90

Figure 3. The web-based interactive scoring interface for MAT evaluation.

With the interactive interface and iPad, scores can be collected easily. Then these scores can be analyzed by the proposed ANN simulation model to find the best tender, which is shown in Figure 4.
The Owner Total score comparison in all
Reviewer I Bidder A B 80 72 II 78 75 Total Score 441 430

The Owner Total core comparison by items

Evaluation items Bidder 1.technology A B 2.quality Reviewer II I 18 9 17 8 scores 91 43

Figure 4. Concept of proposed ANN simulation model for MAT decision.

Conclusions

Through the research, the optimum decision system for Turnkey project procurements can be established on the platform of both BIM and iPad. The visualized evaluation system is illustrated in Figure 5. By such a system, the owner and the evaluation panel can discuss the bidders proposal in details. The proposal can even be adjusted on the spot according to the panels suggestions. This paper presents the steps of establishing the BIM-based MAT decision system prototype. Firstly, the key evaluation criteria need to be identified in advance. Next, develop the function of information comparison among the bidders proposals directly by BIM. Then an appropriate algorithm is adopted to conduct dynamic simulations to detect the difference among proposals. According to the difference, the panel can provide a score for each evaluation item by an objective standard, and finally decide the best tender. The results of this research will effectively improve MAT evaluation problems and advance efficiency and quality of Turnkey project procurements. The research is still ongoing. After all research works are done, tests will be proceeded to verify effectiveness of the proposed BIM-based MAT decision system.

Figure 5. Visualized MAT evaluation method: iPad+ BIM+ANN network.

References
CHEN, S. J., 2002. Study on most advantageous tender evaluation criteria for public design-build project procurements. Thesis of Institute of Civil Engineering at National Taiwan University. LIU, M.M., 2007. A study on the advantageous tender evaluation system at government procurement law. Thesis of Institute of Public Affairs Management at National Sun Yat-sen University. LOTT, F. R., 2009. Negotiating BIM Scope-Answering the Big Questions. In: 2009 BIM Strategy Webinar II. National Institute of Building Sciences (NIBS), 2007. National BIM Standard Version 1 - Part 1: Overview, Principles, and Methodologies, Available online: www.wbdg.org/bim/nbims.php