USING THE BALANCED SCORECARD TO IMPROVE PROJECT MANAGEMENT PRACTICE

JAMES NORRIE (Ryerson University Toronto, Canada) DEREK H.T. WALKER (RMIT University, Melbourne Australia) 1. INTRODUCTION

There have been numerous reported cases of projects, particularly IT projects, being delivered substantially over time and well above budget. Indeed one of the more notorious UK IT projects, Taurus, was abandoned after expending 500 million pounds. Lack of leadership and project definition were cited in the Taurus case as a serious causal factor (Drummond 1998). Many commercial research firms like the Standish Group (www.standishgroup.com/chaos.html) report high failure rates of IT-related projects indicating a pattern of investments that fail to yield an improvement in business productivity or profit, often referred to as the information or productivity paradox. As a profession, project managers must be concerned about this. A review of the literature reveals projects often being perceived as failing due to poor leadership and enfeebled articulation of the project vision. This class of failure can be attributed as a failure of alignment between strategic objectives and project goals or the lack of a management framework able to effectively plan and control strategic project outcomes. Therefore, any tool or technique that can help clarify a project vision by reducing project outcomes to a more concrete, identifiable subset of measurable goals and objectives is an invaluable leadership tool. In this paper we have identified the challenge of providing project leadership in situations where a clear intersection business strategy and project goals may be missing. This can occur because the project vision is ill defined at the outset or where the business environment is too dynamic to permit continual articulation and updating of the intersections between business strategy and a particular projects goals. A possible gap in project management practices arises at the point where these intersections are not well understood and internalized among all project resources. It may also occur when this clarity is realized only by a few key stakeholders or uniquely by the project leader. Dysfunctional management structures that over-rely on hierarchy and functional domains may also give rise to this same disconnect between strategy and project goals. Current best practice implicitly assumes that project teams will have a clear vision, originating through a process led by the executive sponsor and project leader. Negatively impairing this assumption may create communication and decision-making challenges that may be new to many project managers. As a result, this ambiguity can create severe leadership challenges. In response, project managers may tend to try and create an illusion of tangible progress through relying even more heavily upon on-time and on-budget measures this tactic will not directly address either the strategy ambiguity or create appropriate project goals and this may leave the project divorced from critical executive oversight and leadership will with regard to on-strategy or on-quality project management outcomes. A major premise of this paper is that, in a project setting, the on-strategy aspects of project management are clearly the responsibility of the project sponsor and project leaders/managers in tandem requiring that they wrestle with this issue and create a process to resolve any ambiguities whereas the traditional on budget, ontime and on-quality aspects of project management (the commonly stated triple constraint) rest more clearly as the direct responsibility of the project manager and project team. The vexing problem of only providing project leaders with project-based tools and process to resolve gaps that really exist at the level of business strategy and goals is why we initially sought alternative methodologies that could resolve this. Using the Balanced Scorecard as a basic framework, we propose an emergent methodology with a series of four steps that help engage the project team to identify specific deliverables and metrics which act as connectors to the business strategy. The outcome of these efforts is to provide a blueprint for the project team to follow in daily decision-making even in the absence of or in the face of constantly changing business strategy. The intent is to strengthen the potential for leaders to deliver successful project outcomes in less than ideal business environments by extending their indirect influence on a projects daily operational decision-making. In turn, we believe this can help make the project management profession a part of the solution to the productivity paradox rather than a contributor to it by improving the strategic outcomes of major IT projects.

2.

PROJECT LEADERSHIP, VISION AND THE BSC

As is the case elsewhere in business, within project management we must distinguish between management of a project (the day-to-day operations of a project plan in pursuit of an agreed set of outcomes on-time and within budget) and project leadership (the higher pursuit of purposeful action by the team to follow its leaders while moving in concert with business strategy and within acceptable norms and values of the organization to deliver a project result which is on-strategy and of acceptable quality). There is general agreement within the academic literature about the differences between leadership and management see for example (Zaleznik 1977; Kotter 1990). There is an extensive body of literature in this domain and limited utility in exploring it in any detail in this paper. It is interesting to note that there appears to be general agreement in the academic literature and theory about the need for leadership to exert itself most when a business context is vague or dynamic, but there is not a similar volume of citations that appear to indicate how to accomplish this specifically in a project management setting. Project management by definition is about implementing a change program (Turner and Cochrane 1993; Briner et al. 1996; Cleland 1999) either in the form of system changes such as in IT projects or developing a building project, new automotive product, aeroplane or weapon system for example. This creates a dilemma for project managers who, faced with this set of circumstances, do not appear to have been given very many tools to address it. The project manager acts in many ways as both leader and manager, though the role of the project sponsor may, depending on the project and personalities of the project sponsor and project manager, influence the extent of this overlap (Morris 1994; Briner et al. 1996; Cleland 1999). In particular relation to this paper, the primary problem is how to lead a project team when the project context or business strategy is either ill-defined or constantly changing. This can create a situation where there is not a clear connection between the project outcomes and the business strategy, or where team members do not necessarily agree on the interpretation of the strategy in relation to their objectives. This problem has been explored in the past (Turner and Cochrane 1993; Bennis and Nanus 1997; Bennis et al. 2001). There are both general team-based solutions within the existing literature (Katzenbach and Smith 1993; Robbins and Finlay 1997; Yukl 1998) and specific project-based solutions on how to remedy this instance (Briner et al. 1996; Thite 1999). We may deduce that if the connection is ambiguous or understood by only a few key stakeholders, rather than being more broadly accepted by those than can influence project outcomes, there is a risk the project may fail to deliver. Many researchers (particularly Senge,1990), have stressed in their work that a narrowly held vision is insufficient in most leadership contexts to create purposeful coordinated action among all followers. This problem has been studied and documented sufficiently for us to agree with this conclusion. Intuitively it would seem that for a project with a long time horizon, the most likely cause of a lack of strategic connections may be rapidly changing industry or business circumstances. This would most likely occur in a mega project setting, for instance the implementation of enterprise resource planning (ERP) or customer relationship management (CRM) technology, where the time horizon for such a significant IT project may run into years and impact upon all core operations of a company. It is critical especially in these types of projects, where changed strategies can radically influence the projects goals and objectives, that the project be directly connected and remain constantly connected to the companys emerging and changing business strategy. Regardless of the originating reasons for the problems identified above, any gaps between vision and strategy can create a potential problem in terms of daily operational management of a project under ambiguous conditions. While the authors do not intend to minimize the efforts of previous authors to address this problem, in general, it seems that the solutions almost always focus on a desire to present methods directed at the behaviour of the team leader or addressing aspects of the problem resulting from the companys culture or values. There is often a heavy emphasis on motivational theory as the underpinnings of how leaders influence team members. While useful, these do not seem (in the authors experience) to address the core problem of insufficiency or instability of strategy to properly develop and express a project vision that is connected to the business strategy. As a result, we strive to set out a succinct methodology that involves deploying a performance management framework particularly attuned to the project management context to assist leaders facing this ambiguity and its possible consequences. 3. A BALANCED SCORECARD (BSC) APPROACH TO PROJECT MANAGEMENT

We now turn our attention to specific methods which follow-on from the point at which the strategic choices for the business have been made to the question of how to operationalize that strategy through a series of initiatives

or projects planned and executed by the business. In our view, the Balanced Scorecard is one of the few consistent methodologies that appears to have much rigour or acceptance attached to it. The BSC has been extensively developed, tested and has demonstrated its value within corporate settings as an ideal tool to extend the strategic plan of the business into operational plans and strategic measures that allow the organization to conclude if it is executing on strategy regardless of the ambiguity or their understanding or agreement with the underlying strategy that led to the definition of these measures. In fact, as a leader, one of the major benefits of this tool is its ability to take a very complex, often ill-defined business strategy and reduce it to a level of specificity in measurement that actually enlightens various stakeholders and members of the team as to their specific expected contribution in executing a complex business strategy. However, while proven at the corporate level, to the best of our knowledge, there have been only a few published papers (Stewart and Mohamed 2001; Stewart 2001) with regard to applying this same methodology in a project context or to extend the benefits of this methodology to the management of major IT projects. Yet there are numerous researchers and authors whose work suggests that project outcomes should be linked directly in measurement terms to broader corporate strategy and measures. We suggest that the project management profession must direct its attention more thoroughly to this issue of alignment and delve into the realm of performance management so as to enable project leaders to deliver on-strategy project results. To address this need, we propose that a project Balanced Scorecard (PBSC) might provide the means to apply appropriate performance measurement and leadership techniques that already exist in other parts of the corporation, to improve the strategic results of project teams. By developing and implementing this tool in a real-time, real-life situation and comparing the results to similar projects that did not use the PBSC in their execution, it would be possible to determine if the intended impact of this tool can actually be achieved in practice. A case study was undertaken to accomplish this goal. For the purposes of this study, the original work of (Kaplan and Norton 1992), on strategic measurement methodology is newly extended into a project management context. We intended to respect the intent, but not necessarily the specific methodology or technology, of Kaplan and Nortons original work (see below). We developed, tested and applied a strategic measurement system based on the BSC methodology specifically to projects. Our intent was to create an approach that was of immediate value to project managers and senior executives of large corporations faced with the increasingly complex issue of how to ensure a timely understanding of business strategy among, disparate and dispersed global project teams who have varying degrees of ability and interest in understanding the companys overall strategy. Using a PBSC can help address a project vision gap by making strategy easier to understand and apply in a practical rather than theoretical form for project team members. The assumption thereafter is that this helps the company improve their competitive position by ensuring that project managers focus on the on-strategy and on quality aspects of project management with the same level of effort as they do on budget and on time metrics. By making this exercise easier to do and more obvious to them at a project operational level, this objective is more easily achieved. It is clear that projects require a task management focus to be successfully completed. And that there is an appropriate emphasis on process as a method of tracking and reporting tasks, usually in the form of a project plan based on a work breakdown structure. An understanding of the project management process, its phases and the appropriate methods to manage deliverables has clearly been the emphasis of the early evolution of our discipline. However, at the risk of our integrity as a profession, it is important that we do not over-emphasize the management aspects without due regard for the essentials of leadership in a project management context. We believe there is a pressing need within our field for academics and professionals alike to acknowledge that the management of a project is the easier of these two things to accomplish. Task and process management is relatively easy to learn and can be applied routinely. Leadership of a project is a different matter altogether. In their landmark work Project Leadership (Briner et al. 1996, p67), the author emphasizes the role of sustainer as a key aspect of being a successful project sponsor. They stress the need for project managers to orient themselves towards alignment and away from enforcementan elusive concept of trying to create congruence among the team and the project goals using a variety of activities and sources of power to influence others to act in accordance with the leaders desired outcomes rather than relying on a traditional command and control management orientation. We submit that the use of a Project Balanced Scorecard might be an alternate but powerful tool that can accomplish this shift from a command and control to a collaborative method.

The results from our study suggest that using a PBSC provides an indirect form of influence on daily decisionmaking within project teams. It demands that project team members link their own actions and decisions with the overall intended strategy of the project (which, in turn, is an extension of the corporate strategy if the BSC methodology has been consistently applied). When properly executed, this methodology can extend the virtual presence of the leader and inject it into every critical project event and decision. This permits the project leader some level of confidence that influence at the operational level will be felt as a result of going through the exercises associated with the PBSC and this is one of the primary benefits cited by executives and project managers interviewed about the use of this tool in a project setting. The essential four-step sequence used to build a BSC originally designated by Kaplan and Norton is: strategy review; strategy mapping; developing measures, targets and initiatives; and implementation (Kaplan and Norton 1998a, 1998b). This involves mapping cause and effect relationships within the enterprise in order to attach meaningful measures and metrics to the execution of activities likely to generate desired results. Intuitively, when we want to translate this existing methodology into a project context, several important changes must be undertaken: 1. 2. 3. In a project context, the role of the BSC must change from measuring the overall achievement of strategic objectives to measuring the specific results of the project and comparing them to the intended impact of the project on the execution of business strategy. Instead of purely mapping business strategy, we are interested in both mapping the intersection of the project strategy and business strategy and more closely aligning them as a result of this review and using the Project BSC as a tool for leadership to assist with this alignment issue. We must modify the approach to measure project-related deliverables and objectives rather than business initiatives and appropriately settle on project-based measurements for these. 4. THE BALANCED SCORE CARD PILOT STUDY

Research for this paper was initiated while one of the authors was providing consulting services to a global telecommunications firm in the area of strategic project management. The firm, a large global telecommunications company, has worldwide operations and a particularly large presence in the United States and Europe. It specializes primarily in services to business customers in 70 countries throughout the world and had revenues exceeding $300M USD at the time. This firm was selected both because of its representative nature and because of a unique opportunity to initiate and follow two projects similar in scope and budget size that would afford the opportunity to test for the impact of the BSC used in a project management setting. Each of two candidate projects (code names Blue and Pip) met the criteria specified earlier in terms of being vital to the companys business strategy. The project teams were virtually identical in size with between 40 and 45 full-time personnel assigned to them at any one time, in addition to temporary teams of consultants and contractors from various vendors required to support key project deliverables. Both projects had estimated timelines at the outset of 9 to 12 months from initiation to completion with multi-million dollar budgets. During this period, both project teams were closely monitored, and consultation took place with the respective global project managers and the appropriate executive sponsors. One project (Blue) was managed using the firms existing project management methodology and according to existing company practices. The ultimate ontime and on-budget results for this project are summarily noted in TABLE 1. For the second project (Pip), the concept of the Project BSC was introduced to the project team during initial training workshops that preceded the final project planning stages. These sessions were held on the companys premises in March and April 2000. This involved extensive initial training on the concepts and practices of the traditional BSC methodology (2 days) as well as specifically extending and integrating it with the teams help into the companys existing project management framework (3 days). Actual measured outcomes in the three key result areas for Project PIP (the group using the BSC) were better than for the Project Blue control group. While it is not possible to categorically state that this is all due to the use of this methodology, there is strong evidence the BSC Scorecard had a positive impact on planned project outcomes. This ensured a high degree of ownership and involvement from the team for the use of this tool in the execution of the project. Table 1 provides a summary comparison of the results in terms of on-time and on-budget delivery for this project. Each project team was also surveyed with a particular interest on measuring any apparent differences between the teams that were attributable to the use of the Project Balanced Scorecard. A copy of the questionnaire used

is contained in Appendix B. This method closely follows the case study research method proposed by (Yin 1994) to add a substantial amount of qualitative insight to the quantitative results. The survey was piloted for general comprehensibility and interpretation first with a group of experienced consultants who had worked with the personnel of the client firm on other projects but who were not involved in these projects. In addition, they were asked to validate the use of language so as to ensure that employees of the candidate firm would interpret the questionnaire appropriately. Surveys were administered early in January, 2001. Information was also collected as to the controllable factors that may also have influenced project performance outside of the introduction of the Project BSC. This will allow future researchers to identify specifically the best practices that could be further refined to prove an impact from using this new tool on various elements of traditional project management practices in use in most companies. A series of final interviews to gather additional anecdotal evidence about the qualitative impact of using the PBSC were conducted in January 2001 both in person and over the phone with members of the executive team of the client firm directly involved in this research. The available sample of approximately 40 individuals was limited in size to respondents who had worked on the two projects of which 34 returned surveys. While the return rate is excellent, the sample size is not statistically representative of the total population of respondents and so is a limiting factor that could not be addressed by any specific alternate research design. As a result, the reliability of the sample measured using Cronbachs Alpha is a low .63, a function of the limited variability in the responses to individual items from such a small sample. This also meant limiting statistical tests to those relevant for small sample sizes. The research was designed as exploratory and action based in nature to conduct immediately useful research on project management practices in a corporate setting and the authors acknowledge this potential design constraint. A variety of statistical techniques were applied to the survey data in order to determine any meaningful variations in the scores between the two groups (the experiment design being one project using a BSC and the other not). In addition, individual results were compared to respondents answers in terms of job title and length of time measured in months they had served on their project. Surprisingly, the survey results do not indicate any meaningful variations based on demographics or project tenure, but they do provide several important insights in other areas. Results presented here are necessarily limited due to the scope of this paper; however, we present the most relevant results that support our original proposition presented at the beginning of this paper. TABLE 1 - ACTUAL PROJECT RESULTS Specific Key Result On-Budget Delivery On-Time Delivery Completed Deliverables Project Blue (nonScorecard using Group) 112 % ($9.6M actual vs. $8.6M budget) +122% (11 months actual versus 9 planned) 94% (36 of 38 deliverables were 100% complete) Project PIP (Scorecard using Group) 98% ($12.1M actual vs. $12.3M budget) +110% (11 months actual versus 10 planned) 95% (18 of 19 deliverables were 100% complete) Actual Difference 14% 12% 1% %

It is also worth noting that while both projects ran longer than initially planned, both groups reported high selfreport scores for on-time delivery in the questionnaire. In our view, this is explained by the tendency of this company to set very aggressive timelines that project teams often do not feel are realistic or reasonable. Delivery times, as a consequence, even close to these very aggressive targets can be perceived as a reasonable approximation of acceptable on-time performance by the project teams. We considered it important to provide a more coherent explanation of these results rather than merely conclude that the PBSC facilitates better deliverables. TABLE 2 provides a list of critical performance indicators that recorded highly significant differences in the control group vs. the experimental project team. This may signify that the PBSC positively impacts such things as the project vision development process, project performance communication, and more effective monitoring and project control. Thus its use likely contributes to both project leadership and management effectiveness.

TABLE 2 - PROJECT FACTORS POSITIVELY IMPACTED BY THE TEAM USING THE BALANCED SCORECARD TOOL FACTORS MENTIONNED as POSITIVELY IMPACTED 1. Clearer Project Vision 2. Clearer Goals & Objectives 3. More Clearly Communicates Project Status 4. Assuring an Appropriate Project Team Structure 5. Improved Status Reporting Administration 6. Improved Project Tracking & Control 7. Higher Executive/Sponsor Involvement 8. Helped Implement Effective Risk Management Practices 5. CONCLUSIONS

One specific implication of our pilot study results is the suggestion that moving beyond established project performance measures, by using a PBSC framework, provides significant performance improvement potential even for traditional deliverables such as time and cost. We hypothesize this is because the BSC facilitates a wider perspective of project management success measures and facilitates the monitoring of strategic leading rather than lagging performance indicators. Because project teams are required to identify, monitor and report a broader range of more strategic measures to assess project goals and objectives, it may be more likely a strategic outcome will result. Results from TABLE 2 suggest that a BSC framework provides project management teams with early insight into risk areas requiring more attention. One of the primary purposes of the PBSC is that it can be used as a communications vehicle for the articulation of vision, goals and objectives. The scorecard itself is less significant than the value of its use since the process of developing and using this tool increases the effectiveness of communication between the project sponsor, the project management team members, and other project stakeholders. It provides each party with a common communication channel to articulate project performance measures and results. We have argued that this can create a more powerfully committed team with a deep sense of purpose and vision (project leadership). We also argued that using the PBSC in the way we have indicated has the benefit of making complex strategy more understandable at the operational level in terms of specific operating targets for the project (project management). While we recognise that the actual business strategy and resulting project vision may themselves be weak and tenuous, our research results nonetheless indicate that the process of using the PBSC can help motivate project team followers to perform more productively in executing strategically. We therefore suggest that the process of building a PBSC brings added depth to any strategic projects business case and to managements ability to provide oversight for the totality of project benefits from a number of perspectives. Undertaking the process itself, as it is with the BSC in a corporate setting, adds value to participants and promotes reliable communication and more effective decision-making because of its ability to make a strong connection between business strategy and project vision through an understanding of measures and metrics that can guide individual and collective action and outcomes. 6. REFERENCES

1. Bennis, W. On Becomming a Leader. San Francisco, Peresus Publishing, 1989. 2. Bennis, W. and Nanus, B. Leaders Strategies for Taking Charge. New York, Harper Business, 1997. 3. Bennis, W., Spevietzer, G. and Cummings, T. G. 2001. The Future of Leadership. San Fancisco, JosseyBass. 4. Briner, W., Hastings, C. and Geddes, M. Project Leadership. Aldershot, UK, Gower, 1996. 5. Cleland, D. I. Project Management Strategic Design and Implementation. Singapore, McGraw-Hill, Singapore, 1999. 6. Collins, J. and Porras, J. I. Building Your Company's Vision. Harvard Business Review. 74 (5), 1996. 7. Drummond, H. Riding a Tiger: Some Lessons of Taurus. Management Decision. 36 (3), 1998, . 141-146. 8. Greiner, L. E. and Schein, V. Power and Organisation Development. New York, Addison-Wesley Management, 1988. 9. Hersey, P., Blanchard, K. and Johnson, D. E. Management of Organizational Behaviour. London, Prentice Hall International, 1996.

10. Kaplan, R. S. and Norton, D. P. The Balanced Scorecard - Measures that Drive Performance. Harvard Business Review. 80 (1), 1992, . 171-179. 11. Kaplan, R. S. and Norton, D. P. Putting the Balanced Scorecard to Work. Harvard Business Review on Measuring Corporate Performance. Boston, MA, Harvard Business School Publishing, 1998a, . 147-181. 12. Kaplan, R. S. and Norton, D. P. Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review on Measuring Corporate Performance. Boston, MA, Harvard Business School Publishing, 1998b, . 183-211. 13. Katzenbach, J. R. and Smith, D. K. The Wisdom of Teams - Creating the High-Performance Organization. Boston, Harvard Business School Press, 1993. 14. Kotter, J. P. What Leaders Really Do. Harvard Business Review. 68 (3) 1990. 15. Kotter, J. P. Why Transformation Efforts Fail. Harvard Business Review. 73 (2), 1995, . 59-67. 16. Kotter, J. P. John P. Kotter on what leaders really do. Boston, Harvard Business School Press, 1999. 17. Loosemore, M. 1999. Responsibility, Power and Construction Conflict. Construction Management and Economics. 17 (6): 699-709. 18. Morris, P. W. G. The Management of Projects A New Model. London, Thomas Telford, 1994. 19. Pinto, J. K. Power & Politics in Project Management. Sylva, N.C., Project Management Institute, 1998. 20. Robbins, H. and Finlay, M. Why Teams Dont Work - What Went Wrong and How to Make it Right. London, Orion Publishing Group Ltd, 1997. 21. Senge, P. M. The Fifth Discipline - The Art & Practice of the Learning Organization. Sydney, Australia, Random House, 1990. 22. Stewart, R. A. and Mohamed, S. Utilizing the Balanced Scorecard for IT/IS, 2001. 23. Performance Evaluation in Construction. Journal of Construction Innovation. 1 (2): 147-163. 24. Stewart, W. E. (2001). Balanced Scorecard for Projects. Project Management Journal. 32 (1): 38-53. 25. Thite, M. (1999). Leadership Styles in Information Technology Projects. International Journal of Project Management. 18 (2): 235-241. 26. Turner, J. R. and Cochrane, R. A. The Goals and Methods Matrix: Coping with Projects With Ill-defined Goals and/or Methods of Achieving Them. International Journal of Project Management. 11 (2), 1993, . 93 102. 27. Yin, R. Case Study Research. Thousand Oaks, California, Sage, 1994. 28. Yukl. Leadership in Organisations. Sydney, Prentice-Hall, 1998. 29. Zaleznik, A. Managers and Leaders Are They Different. Harvard Business Review. 55 (3), 1977.