Professional Documents
Culture Documents
Procurement Using Best Value Method
Procurement Using Best Value Method
Abstract: Best-value (BV) procurement is increasingly used in conventional design-bid-build (DBB) delivery, which raises questions
regarding the influence of qualifications-based evaluation criteria when selecting construction contractors. The objectives of this study
are to (1) examine BV procurement outcomes that can be expected from DBB projects, (2) determine which evaluation criteria have the
Downloaded from ascelibrary.org by 27.34.104.212 on 10/18/18. Copyright ASCE. For personal use only; all rights reserved.
greatest dispersion among competing contractors, and (3) identify whether a relationship exists between cost- and qualifications-based
proposal submissions. A sample of 162 public institutional BV-procured DBB projects was collected that include evaluation results of
684 contractor proposals. Both descriptive and inferential statistics were used, including coefficient of variation and correlation analy-
sis of evaluation criteria. The results show that BV-selected contractors offer substantial qualifications benefits in comparison with
lowest- and average-bidder alternatives. Certain qualifications-based criteria, such as technical proposals, contractor interview scores,
and safety proposals were found to have a greater dispersion among competing bidders than cost proposals. Contributions of this study
to the body of knowledge include a sizable empirical data set of BV-procured DBB projects in the vertical sector and a unit of measure
that considers dispersion among competing contractor proposals. In current construction industry practice, it is recommended that
project owners consider BV for complex DBB projects and prioritize certain qualifications-based criteria that are well-suited for differ-
entiating between competing contractor proposals. DOI: 10.1061/(ASCE)CO.1943-7862.0001550. © 2018 American Society of Civil
Engineers.
Author keywords: Best-value procurement; Design-bid-build; Bidding cost; Qualifications-based evaluation criteria; Coefficient of
variation; Kruskal-Wallis H test.
BV Procurement in Construction
Literature Review BV procurement theoretically achieves a balance between LB and
QBS methodologies by considering both cost and qualifications
A number of studies have discussed different procurement proc-
criteria. Previous research has demonstrated various benefits of BV
esses used in the construction industry. The typical procurement
procurement in the construction industry compared with the tradi-
procedures are LB, QBS, and BV.
tional LB system. For example, Perrenoud et al. (2017) found that
contractors who received more favorable qualifications-based
LB Procurement in Construction evaluation scores tended to achieve better performance in project
LB traditionally provides several benefits to construction owners. quality, professionalism, risk communication, and overall customer
The first benefit is in terms of short-term monetary savings because satisfaction. BV has also been shown to achieve a reduction in cost
the owner is guaranteed to achieve the lowest cost option at the time growth, schedule growth, disputes, and claims (Abdelrahman et al.
of bidding. The second benefit relates to simplicity of the procure- 2008).
ment selection process (Yu and Wang 2012). In highway construc- Researchers have found that BV procurement brings benefits to
tion projects, LB is often supported by federal, state, and municipal both construction owners and contractors. For example, Gransberg
legislation due to the transparency it fosters in the evaluation pro- and Shane (2015) identified BV as a procurement method which
cess as the lowest bidder is readily discernable (Gransberg and can achieve greater consistency in long-term project performance.
Ellicott 1997; Tran et al. 2016). Abdelrahman et al. (2008) reported that BV has been found to
Despite its advantages and widespread use, LB procurement has achieve positive project performance due to its emphasis on
several drawbacks. For example, LB excludes quality considera- value-added services and qualifications-based criteria. Ahmed et al.
tions during the evaluation process, such as a contractor’s technical (2012) emphasized the need for BV in contractor selection for high-
proposal, past performance, and other qualifications-based criteria way projects rather than conventional LB due to the ability to con-
(Ahmed et al. 2012). Researchers (e.g., El Wardani et al. 2006; sider the contractor’s responsibility for maintenance across the
Richey 2012) have also found that LB has been linked to incon- project’s operational life span. According to Sullivan and Guo
sistent performance during the construction phase. Through analyz- (2009), BV procurement can benefit contractors by providing im-
ing 70 vertical construction projects, El Wardani et al. (2006) found proved cash flow and increased profitability because it can lead to a
that LB-procured projects suffered 9 and 5.6% greater cost and more performance-based project environment.
schedule growth, respectively, when compared with similar proj- It is important that many professional organizations in the de-
ects procured via BV methods. Richey (2012) noted a new library sign and construction industry support the use of BV for procuring
project in Palo Alto, California, that accrued $1.7 million in change construction services and have lobbied for its continued growth.
orders and a one-year delay. The LB-selected contractor was The AIA promotes the potential of qualifications-based evaluation
awarded with an initial bid that was $8 million below the estimated criteria in providing better contractor selection outcomes in terms
cost of $32 million. In this project, a dispute arose when the owner of cost savings and value engineering (Sandquist 2007). The
alleged the contractor had bid as low as possible and then submitted Construction Management Association of America (CMAA) has
an unreasonable number of requests for information and change promoted the use of qualifications-based criteria as an essential as-
orders. Conversely, the contractor argued that the design and spec- pect of procuring construction management services which poten-
ifications were not explicitly defined, which caused substantial cost tially increases project performance (McKeon 2016). A joint
growth and delays (Richey 2012). Such disputes are not uncommon publication between the AGC and the NASFA advocated for
in traditional LB systems (Sandquist 2007). and defined recommended best practices for owners to use in
BV procurement (AGC and NASFA 2008). Additionally, both
the Design-Build Institute of America (DBIA) and AGC identified
QBS in Construction qualifications-based criteria as important in achieving lower final
QBS is defined as a procurement system which completely concen- project costs regardless of project delivery method (DBIA 2012;
trates on the evaluation of qualifications-based criteria and does not AGC 2009).
include consideration of cost proposals. The use of QBS in the con-
struction sector is rare and generally confined to APDM projects.
BV Procurement in DBB Projects
Previous research has indicated that owners tend to favor noncost
criteria over time when they become more experienced in using Although LB is still the predominant method for procuring con-
APDMs (Gransberg and Shane 2015). Traditionally, architecture tractors in DBB projects, BV is becoming increasingly common.
Variation
Mann-Whitney U
Step 1 St ep 2 St ep 3 Step 4
of request for proposal (RFP), evaluation matrices/scores, and bid- laboratories, and office spaces. The majority of projects were ren-
ding costs. Step 3 performed descriptive analysis to identify selec- ovations, and inspection revealed no substantial difference between
tion outcomes of BV-procured DBB projects and inferential procurement procedures or contractor proposal responses for new
statistics analysis to investigate the COV among evaluation scores construction and renovation projects, likely due to the similarity in
for competing bidders as well as the relationship between contrac- owner organizations, facilities, and construction scopes. Further-
tor bid costs and their respective evaluation scores based on more, all projects in the data sample used virtually identical BV
qualifications-based criteria. Step 4 was the collation of key find- procurement procedures, including virtually identical evaluation
ings. The following sections present data collection and data analy- criteria, similar weighting schemes, and consistent evaluation scor-
sis in detail. ing procedures in the owners’ RFPs.
The BV procurement processes in the data sample considered
similar evaluation criteria centered on both qualifications (technical
Data Collection proposal, past performance, contractor interview scores, and safety)
Initially, more than 177 BV-procured DBB construction projects and nonqualifications (cost and schedule) items. Table 1 provides
were collected from both the horizontal and vertical sectors. Project descriptive statistics related to the evaluation weights for each
records were collected for analysis rather than via a survey meth- evaluation criterion as established in the owner’s RFP. The evalu-
odology. The following project records were collected for each ation weights shown are normalized to a 100% scale to show the
project in the data set: the project RFP, the owner’s evaluation ma- relative importance the owners placed on each criterion. For exam-
trix and associated evaluation scores of all competing contractors, ple, the cost criterion examined from 162 projects was found to
and bid costs from all competing contractors. The final data set was have a median of 25% of the total evaluation weight, whereas
limited to projects in the North American vertical sector. All proj- the remaining 75% was distributed in the schedule and other
ects were procured by public owners in the institutional sector, in- qualifications-based criteria. The evaluation criteria were analyzed
cluding state, city, and municipal agencies as well as elementary, in isolation to understand the ranking of selected contractors, the
secondary, and postsecondary school systems. The final data set differential in evaluation scores between selected contractors and
consisted of 162 projects that met the above criteria. their competitors, the COV in evaluation scores among competing
In the data sample, the projects were filtered to ensure similar bidders, and the relationships between bidding costs and
construction scopes, facility types, and BV procurement proce- qualifications-based criteria evaluations.
dures. All projects were vertical construction in institutional facili- The owner’s evaluation matrix from each project in the data
ties. The scopes were largely consistent in terms of functional sample (n ¼ 162) was reviewed. Each project’s evaluation matrix
use of the buildings, which mainly consisted of classrooms, contained the owner evaluation committee’s scoring results for
in 88% of cases, which indicates that BV-selected contractors in nificant differences in COV values. An assessment of the pairwise
DBB projects commonly demonstrate strong qualifications in both results provided in Table 9 identifies three groupings of evaluation
their written proposals and their project team interviews. criteria with different COV values that are statistically significant.
The first group of four evaluation criteria has the largest COV val-
Differential in Evaluation Scores between Selected, Lowest, ues, including technical proposal (x̄ ¼ 27.3%, med ¼ 21.0%),
and Average Bidders schedule (x̄ ¼ 26.0%, med ¼ 23.0%), contractor interview
In comparison with LB procurement, BV achieves substantial ben- (x̄ ¼ 24.8%, med ¼ 21.0%), and safety (x̄ ¼ 22.1%, med ¼
efits associated with contractor qualifications and schedule propos- 19.0%). The second group consists solely of cost proposals, which
als. As shown in Table 7, BV-selected contractors achieved higher have moderate COV values (x̄ ¼ 12.6%, med ¼ 10.0%) statisti-
evaluation scores in all of the qualifications-based criteria when cally significantly different from all other evaluation criteria.
compared with the lowest-bid contractors. For example, there were The third group, consisting solely of past performance, shows
substantial increases in the quality of technical proposals and con- the lowest COV values (x̄ ¼ 8.9%, med ¼ 4.0%).
tractor interviews with a 21.2 and 15.1% differential, respectively. The COV results lead to several discussion points. First, this
One can observe from Table 7 that marginal improvements were study found relatively higher COV values for technical proposals,
also observed in BV-selected contractors’ safety proposals (3.3%) contractor interviews, and safety proposals than for cost proposals.
and past performance (2.4%). BV-selected contractors also pro- These results therefore indicate that substantial dispersion exists in
posed shorter construction schedule durations on average (6.4% the qualifications between competing contractors, even in DBB
shorter duration than the lowest-bid contractors). projects where contractors are proposing on a largely static scope
Although BV procurement resulted in the selection of higher bid represented by a complete set of contract documents. These results
costs than traditional LB procurement, BV-selected contractors still support the AGC’s (2017) position that contractors are not a com-
represented lower costs than the average bidder. BV-selected con- modity; rather, construction is a professional service where pro-
tractors submitted bid costs that were 6.9% more expensive than the curement emphasizes evaluation of qualifications-based criteria.
lowest-bid contractors, but in the broader context they were still A second discussion point is that there is relatively low
2.4% less expensive than the average bid cost. Table 7 shows that dispersion among the bid costs of competing contractors in
BV-selected contractors offered substantial benefits compared with DBB projects. Analysis of all contractor bids revealed the COV
the average bidder in several qualifications-based criteria, such as between competing bids to be fairly low in comparison with other
technical proposal (17.9%), contractor interview (14.2%), and evaluation criteria on a per-project basis (x̄ ¼ 12.6%, med ¼
schedule proposal (7.5%). Marginal gains compared with the aver- 10.0%). DBB project owners can therefore anticipate competing
age bidder were found in safety proposal (2.6%) and past perfor- contractor bids to be fairly consistent,. likely because contractors
mance (2.4%). are bidding on a complete set of contract documents, which means
that the project scope is fully defined and contractor estimates are
based on nearly identical project parameters, material quantities,
COV among Competing Bidders for Each Evaluation
and specifications. This finding further supports Yu et al.’s (2013)
Criterion
call for the appropriate use of multicriteria BV procurement for
Table 8 summarizes the descriptive statistics results for COVamong construction services rather than single-criteria methods such
competing bidders for each evaluation criterion. as LB.
Relationship between Qualifications-Based Evaluation study clearly indicate the ability of BV procurement to provide con-
Criteria and Bid Costs struction owners with a selection outcome that balances cost- and
qualifications-based criteria.
This study found that, in BV-procured DBB projects contractor
qualifications have no direct association with bid cost. All contrac-
tor bids from the data sample were normalized against the low-bid
Conclusion, Contributions, and Limitations
alternative and the average bid on a per-project basis. As shown in
Table 10, neither measure of contractor bid cost was found to have a As BV procurement expands across the construction industry,
statistically significant correlation with the qualifications-based cri- construction owners are gradually coming to view it as an appro-
teria of technical proposals, contractor interviews, or safety propos- priate option for DBB projects. However, few owners have direct
als. Contractor bid cost compared with the lowest-bid alternative experience in applying BV procurement in the DBB delivery sys-
did have a statistically significant and inversely proportional cor- tem. For this reason, this research aimed to add to the existing
relation with past performance; however, the correlation coefficient literature, which lacks extensive empirical data sets on this in-
(rs ¼ −0.109) was so weak that it represented no association of creasingly relevant topic, by exploring BV procurement imple-
practical significance to the industry (Zou et al. 2003). Therefore, mentation. Toward this end, the authors quantitatively analyzed
the research hypothesis that greater evaluation scores for each project’s RFP and evaluation matrix, which included all bid-
qualifications-based evaluation criteria correspond with greater ding and evaluation scoring data for every participating contrac-
bid costs was rejected. tor’s BV proposal (n ¼ 684), via both descriptive and inferential
This study found no evidence that, when owners determine that statistics.
a contractor has demonstrated greater qualifications than compet- This study found that BV procurement achieves a balance be-
itors, that contractor’s corresponding bid cost is higher than com- tween LB and QBS procurement in that it results in selection of the
peting bids. This result was somewhat surprising given the best-qualified contractor in 52% of cases and the lowest-bid con-
conventional wisdom that firms with greater qualifications may tractor in slightly less than half. In the analysis, BV-selected con-
command higher fees (Yu et al. 2013). A potential explanation tractors ranked as the top-qualified and lowest-cost bidders nearly a
is that qualified contractors may leverage their experience and tech- quarter of the time, ranked in the top two for qualifications and cost
nical expertise to execute the construction phase more efficiently 67% of the time, and ranked in the top three 88% of the time.
than their competitors, which can lead to overall cost savings BV-selected contractors also had substantially greater qualifica-
and in turn result in competitive bid costs. This is supported by tions when compared with the lowest- and average-bid alternatives.
the fact that 67% of BV-selected contractors ranked among the Owner evaluation scores for qualifications-based criteria were
top two bidders in both cost and qualifications criteria and 88.3% found to have statistically significant greater COV values than cost
were ranked among the top three. In this manner, the results of this proposals, indicating that a range of contractor qualifications is
Table 10. Spearman’s correlation of all evaluation criteria versus lowest bid and average bid
Code Variable (1) (2) (3) (4) (5) (6) (7)
(1) Cost versus lowest bid 1 — — — — — —
(2) Cost versus average bid 0.797a 1 — — — — —
(3) Technical proposal −0.057 0.013 1 — — — —
(4) Past performance −0.109a −0.057 0.034 1 — — —
(5) Contractor interview −0.091 −0.030 0.131b 0.169a 1 — —
(6) Schedule −0.030 0.026 −0.125a −0.007 0.209b 1 —
(7) Safety 0.053 0.035 0.024 −0.019 0.050 −0.054 1
a
Significant at the 0.01 level (2-tailed).
b
Significant at the 0.05 level (2-tailed).