Engineering, Construction and Architectural Management 1995 2 | 3,167-178

Improved business results through benchmarking*

EDWARD W. MERROW* & WILLIAM H. CROKER *Independent Project Analysis Inc., 1150 Sunset Hills Road, Suite 300, Reston, VA 22090, USA and Chevron Research and Technology Company, Projects and Engineering Technology, 100 Chevron Way, PO Box 1627, Richmond, CA 94802-0627, USA Invited paper; discussion open until March 1996

INTRODUCTION

Chevron has successfully used benchmarking to improve the performance of our project management system. For many years, we at Chevron believed that we were managing our capital projects very well and that our performance was probably significantly above average for our industry. This perception was based solely on our internal comparison of projects. We routinely compared each project with the previous one we had done and, if we met the cost, schedule and start-up requirements, we thought we had performed our responsibilities with excellence. Even if we did not meet all the requirements, we attributed the deviations to special causes that were unique to the project circumstances and were not caused by our project management system. But when we analysed these assumptions carefully, we realized that completing a project within the budget and the schedule was not very significant unless we compared our results with our best competitors. Internal comparisons are based solely on how well our system performed compared to targets determined by the same system. A comparison was needed to tell us how our system was performing in relation to the best competitors in the industry. This revelation, coupled with our desire to contribute to our corporate goal of becoming 'Better than the Best', inspired us to benchmark the performance of our project management system against the best in the industry. In mid-1990 we conducted our first benchmarking studies. Before we describe Chevron's experience with benchmarking, let us understand whether there is room for improvement in our industry and exactly what is benchmarking. The paper will then cover the results of the Chevron benchmarking, the changes to our project management system, our progress to date, and the requirements to achieve a step change in system performance.

*Presented at the European Construction Institute Conference, Lisbon, Portugal, 18 November 1994 by William H. Croker.

1995 Blackwell Science Ltd

167

168

Merrow. E.W. & Croker. W.H.

IS THERE MUCH ROOM FOR IMPROVEMENT? According to the Commerce Department, the United States process industries spent just over 5100 billion on capital projects in the United States in 1993. Based on an analysis of data from top process industry companies, Independent Project Analysis Inc. (IPA) - a major project management benchmarking company in the United States and in Europe - concluded that at least $20 billion of that was wasted on rework, changes of direction, inefficiencies, and failure to follow best engineering and construction practices. And this does not include the cases where either the wrong project was built or the project should not have been built at all! In addition to the cost savings, IPA found that at least a 20% improvement in cycle time is available and, as measured by operability, an even larger improvement in plant quality can be obtained. The data indicate that, of the major projects performed by the process industries, less than one in five meet their primary business objective in cost (less than 10% over-run), schedule (less than 3 months' slip), and operability (greater than 85% of nameplate in the second 6 months) (Fig. 1). Benchmarking is not the sole solution but it can be an important part in an overall solution to this grim record.

WHAT IS BENCHMARKING? Other than being another tiresome 'new management' buzzword, what is 'benchmarking' and why should it be of interest? The term 'benchmarking' can include a number of different activities built around a single simple powerful idea: 'I can improve by the careful study of how others do business'. At its best, benchmarking is a continuing discovery process that opens the organization to new and even radical ideas, and can play a pivotal role in improving effectiveness.
1995 Blackwell Science Ltd. Engineering, Construction and Architectural Management 2 | 3, 167-178

Improved business results through benchmarking

169

Benchmarking is not a one-shot bullet that will improve performance; it is not industrial espionage; and it should not be what it so often becomes, i.e. industrial tourism. A number of activities fall into the general category of benchmarking. There is internal and external benchmarking; with competitors and non-competitors; oneon-one or data-based; domestic or global; qualitative or quantitative. The benchmarking that is described here is Quantitative Competitive Benchmarking, i.e. the benchmarking of one's performance against competitors using quantitatively developed and derived measures of project excellence and effectiveness. Quantitative Competitive Benchmarking is a logical first step in benchmarking that is often followed by other benchmarking activities as key performance gaps are identified. Quantitative Competitive Benchmarking uses quantitative databases drawn from a large number of firms in the same or allied industries. The goal is to obtain a rigorous numerical comparison of business results on an 'apples and apples' basis. Quantitative Competitive Benchmarking answers that most basic of questions: 'How good are we in the things that are important to our business?' Quantitative Competitive Benchmarking has been developed in many areas of interest to the owner and contractor communities, such as capital project system effectiveness, operating costs, maintenance effectiveness, plant quality and reliability, and construction management. Quantitative Competitive Benchmarking needs to be understood in the context of an overall quality improvement effort. In the 'Ten Steps to Quality' first articulated by Xerox (Fig. 2), the first four steps are covered by an initial

1995 Blackwell Science Ltd. Engineering, Construction and Architectural Management 2 | 3, 167-178

170

Merrow, E.W. & Croker, W.H.

benchmarking study. The most important and difficult step is the very first - what exactly should be benchmarked? WHAT SHOULD BE BENCHMARKED? In Quantitative Competitive Benchmarking, the determinants of what should be benchmarked are derived from analysis of historical data. The databases must serve to link project system performance back to work practices. How this process works is illustrated as follows. What determines our level of cost predictability in capital projects? Empirical analyses (and common sense) suggest that the most important thing to examine is the relationship between estimating accuracy (as measured by the ratio of the bare estimate to the actual cost on a constant dollar, constant scope basis) and the level of definition achieved when the estimate was made. As they say in real estate, 'Location, location, location'. In projects, it is 'Scope, scope, scope'. Figure 3 shows this relationship. Despite its scatter, it is statistically strong but is not strong enough for forecasting purposes. The second relationship is estimating accuracy vs. the use of new technology. Most project professionals know that new technology adds to estimating difficulties because new technology often causes late, unpleasant surprises. The numbers support that view, as shown in Fig. 4, but again it is a relationship that only a statistician could love. Now comes the power of analysis - when we combine just those two factors (Fig. 5), things begin to behave when we combine, and the other factors that drive cost predictability we get the relationship shown in Fig. 6, which provides quite a reliable relationship.

1995 Blackwell Science Ltd, Engineering, Construction and Architectural Management 2 | 3, 167-178

Improved business results through benchmarking

171

Why, therefore, is analysis important? Because it provides us with the drivers of key project results in terms of work practices (such as front-end loading) and project characteristics (such as new technology). Those drivers are what need to be benchmarked. Analysis links outcomes to inputs such as definition, technology, project management practices, and the use of value-adding practices by the owner and contractor. It also enables one to focus on the critical work processes. If the work processes do not change, nothing happens. This is one area in which benchmarking differs from other types of studies: reorganization is not viewed as a particularly useful response to performance shortfalls. Only after work processes have been improved does it make sense to inquire about whether the form of

1995 Blackwell Science Ltd. Engineering, Construction and Architectural Management 2 | 3, 167-178

172

Merrow, E.W. & Croker. W.H.

organization tends to support or undermine the good work processes. In case after case, capital project systems have reorganized only to have nothing change. WHAT HAPPENS AFTER A BENCHMARKING STUDY? The products of a good benchmarking study will be: A set of numerical performance metrics that can be adopted and monitored An analysis of company strengths and weaknesses A comparison against demonstrated best practices A solid foundation for continuous improvement.

RESULTS OF THE CHEVRON BENCHMARKING

As the result of the situation discussed earlier Chevron decided, in mid-1990, to conduct a number of benchmarking studies to assess the performance of our project management system and compare it to the best elsewhere. Our process (Fig. 7) uses benchmarking results to identify performance gaps which would then enable us to develop plans to close the gaps and become competitive in the marketplace. The final step in our process is to establish metrics for continuous improvement using the original benchmarking data as the baseline. When we started the benchmarking studies, we were very optimistic about the results but, to our surprise, the results were sobering. Our first study, which was conducted by A.T. Kearney, was a qualitative assessment of our system performance. This study identified several areas where we had significant opportunities for improvement. Our initial reaction to the Kearney Report was that of denial.
1995 Blackwell Science Ltd, Engineering, Construction and Architectural Management 2 | 3, 167-178

Improved business results through benchmarking

173

We, as engineers, always demand data. One of the Chevron managers had a sign on his desk that read 'In God we trust. Everyone else bring data'. Since Kearney's study was qualitative, we looked for another consultant who could help us quantify our performance and selected Independent Project Analysis, Inc. (IPA). At that time, IPA had over 200 oil and chemical industry projects in their database and had the capability of quantitatively comparing our projects with most major oil and chemical projects in North America. We asked them to analyse 26 of our completed refinery and chemical projects and compare them to the industry. The results provide us with a loud wake-up call. The analysis convinced us that our projects were poorly defined at the time we funded them. As the projects moved through engineering, we made numerous changes which resulted in major cost over-runs and delays. Our project costs were significantly higher than those of our average competitors. We took longer than the best in the industry, both in start-up and total cycle time. And, finally, new technology was almost nonexistent in our projects. The 'Not Invented Here' syndrome was not only alive but was thriving at Chevron. In summary, every project was executed in accordance with the skills, wisdom, and desires of the project management teams. Even though every team did its best, the results were totally predictable and, on average, well below those of our competition. With this kind of shock treatment you could hardly sweep the findings under the rug! We shared the results widely within Chevron with our contractor partners. It took a while to convince ourselves that the poor performance of our projects was not due to a lack of competence on the part of our project professionals. It was due primarily to the fact that we lacked a consistent process and discipline in execution
1995 Blackwell Science Ltd, Engineering, Construction and Architectural Management 2 | 3, 167-178

Merrow, E.W. & Croker, W.H.

174

and did not provide state-of-the-art tools to our project professionals. This helped us get over the denial and move on to accepting the challenge of becoming 'Better than the Best'.
C H A N G E S TO THE C H E V R O N PROJECT M A N A G E M E N T S Y S T E M

We shared the results of our top corporate and operating companies' management and sought their guidance in developing improvement plans. A Steering Committee sponsored by our Vice Chairman with representatives from senior management of our major operating companies was chartered. Under the guidance of this steering committee we funded four quality improvement teams which had representation at senior levels from all of our major operating companies that had significant capital programmes (Fig. 8). These quality improvement teams worked intensively over a period of 3 months and with the help of our consultants, both internal and external, developed some remarkable recommendations, most of which were approved by the Steering Committee. The teams recommended that a consistent project management process be implemented throughout the Corporation to provide a system with predictable performance. The supporting technology companies were charged with the task of strengthening their ties with the operating companies and developing a centre of excellence in project management. Corporate policies were revised to eliminate barriers and facilitate good investment decisions. Roles and responsibilities were revised to enhance accountability and, finally, metrics were established to benchmark the process on an ongoing basis. In other words, our project management process was totally re-engineered. With these and other recommendations in hand, we created a vision to systematically improve our work processes, measure progress, and achieve system performance that exceeds the performance of the best. The target date was the end of 1995, only 4 short years away at that time. This goal was widely communicated throughout Chevron with strong endorsement from top management. This

1995 Blackwell Science Ltd, Engineering, Construction and Architectural Management 2 | 3, 167-178

Improved business results through benchmarking

175

process is not a quick fix. Major efforts are required over an extended period of time to achieve excellence. The vision was supported by a new detailed project management process and training. During this period we have also developed many tools that have not only helped us improve the front-end loading of our projects, but have also added significant value to the projects. We identified champions throughout the organization who formed a network to drive the system towards excellence. PROGRESS TO DATE You might wonder where we stand in implementation and what the results look like. Figure 9 is an overview of our progress. It is important to note that the best are also continuously improving which means that benchmarking must also continue, over time, to determine relative performance. Chevron has made significant progress in the last2years. As can be seen in Fig. 10, our front-end loading index, a measure of project definition at the time of funding (which used to be worse than the industry average) has now improved to 'Better than the Best' projects in IPA's database. We still have significant opportunity to drive this to the best practical level. Benchmarking has indicated to us that our remaining gap in front-end loading is execution planning. Consistent application of the process and tools has driven a totally unpredictable system to a highly predictable state. The front-end loading index of our projects in 1992 and 1993 are clustered into a very narrow band. The previous projects had unpredictable results (Fig. 11). The predictability of the system provides a basis for continuous improvement and the addition of value-improving practices to further improve our projects.

1995 Blackwell Science Ltd, Engineering, Construction and Architectural Management 2 | 3, 167-178

176

Merrow, E.W. & Croker, W.H.

1995 Blackwell Science Ltd, Engineering, Construction and Architectural Management 2 | 3, 167-178

Improved business results through benchmarking

177

What has all this meant to the bottom line? IPA estimates that the savings during 1992 were approximately $115 million from a group of projects totalling approximately S700 million in appropriated funds. The savings in 1993, with projects totalling $1.2 billion, are nearly $295 million. Fifty-five million dollars of this is directly attributable to savings from performing value engineering studies. Also shown in Fig. 12 are actual savings by the year of expenditure. As you can see, these estimates show that we have saved a total of $153 million to date, with the balance to be saved over the remaining life of the projects. Even if the savings are only half of those estimated, we believe that we are on the right track. We are continuing to develop new value improving practices, fine tune our process, and apply it to all projects in Chevron. We believe that our goal of becoming 'Better than the Best' by the end of 1995, although difficult, is definitely attainable. We are determined to continue to move towards our vision.

STEPS NECESSARY TO ACHIEVE A STEP CHANGE IN SYSTEM PERFORMANCE Let us now summarise what it takes to make a step change in the performance of a system: Benchmark and understand the results Share the results widely by communicating, communicating, communicating. Get over the 'denial phase' as quickly as possible Seek the endorsement and commitment of top management
1995 Blackwell Science Ltd. Engineering, Construction and Architectural Management 2 | 3, 167-178

178

Merrow, E.W. & Croker. W.H.

Develop and communicate a clear vision of where you want to go and make sure that all stakeholders not only understand it but commit to doing their part in achieving it Support the vision with well defined plans, tools, and processes which are championed by key managers and professionals in the system And, finally, ensure consistent implementation, measurement, and continuous improvement A step change in performance in a short period of time, therefore, is attainable if a quality process is followed.

1995 Blackwell Science Ltd, Engineering, Construction and Architectural Management 2 | 3, 167-178