Consistently Doing the Right Projects and Doing Them Right - What Metrics Do You Need?

Consistently Doing the Right Projects and Doing Them Right – What Metrics Do You Need? T. J. Cooke-Davies Human Systems International Limited, United Kingdom. University of Technology, Sydney University College, London Abstract Few topics are more central to the art and science of managing projects than “project success”. It would seem to be self-evident that every person involved in the management of a project will be striving to make it successful. In the world of the twenty-first century “success”, like its close relative “winning”, seems to be an unquestioned “good”. So surely there can be nothing too difficult about measuring project success? Unfortunately, behind this rather obvious-sounding question, there lies a seething mass of complex assumptions and inter-related concepts that have led one author almost despairingly to ask, “Measuring success – can it really be done, and if carried out, what purpose does it serve?” Difficulties abound for many reasons: the different viewpoints, interests and expectations of groups of stakeholders involved in any given project; the subjective nature of perceptions of “success”; the tendency of perceptions to evolve over extended periods of time; the difficulty of assessing complex phenomena using simple metrics – the list is a lengthy one. On closer examination, project “success” turns out to be a rather slippery subject! And yet the need remains. Every project is undertaken to accomplish some purpose, and it is both natural and right to seek to assess the extent to which that purpose has been achieved. Equally, if the art and science of project management is to advance, then practices that lead to success are to be encouraged over those that lead to failure. Indeed, these two aspects of the need to understand project success each lead to a different aspect of the topic that will be covered more fully in this paper. Success criteria are the measures against which the success or failure of a project will be judged, and success factors are those inputs to the management system that lead directly to the success of the project. Each is important, but the two should not be confused. Projects in organizations come in all shapes and sizes, and yet the performance of each of them must be monitored, and decisions must be taken both inside and outside of the project in the light of changing circumstances. New research among thirty leading organizations in Europe, USA and Australia suggests that there is much dissatisfaction with the metrics they use to monitor the health of projects at corporate, portfolio and project levels. Organizations taking part are drawn from varied industries and markets, including Aerospace and Defence, Pharmaceutical R&D, Financial Services, Automobiles, Government, Electronics, Information Systems, Retail, Energy and Transport. The presentation will review both the “State of Knowledge” about success criteria and success factors, and the “State of the Art” in terms of hierarchies of metrics in order to answer such questions as, “What metrics does an organization need?”, and, “How can it know which are the right projects, and whether they are being done right?” Introduction: Corporate strategy is implemented through projects, so if organizations want to improve the execution of their strategy, they need to improve the success of their projects. To do this, they need to improve their corporate project management capability, of which metrics are an essential component. This paper will provide a research-based answer to the question, “What are the characteristics of a suite of project management metrics that is capable of supporting an organization’s efforts to improve project success?” But before answering that question, there is some ground to be cleared in preparation by reviewing first the connection between corporate strategy and project success, and then by examining the role that project management metrics currently play and some commonly-encountered problems. Strategy, projects and project success. In today’s turbulent times, it seems as if every organization holds on to some form of “strategic intent” as a compass to guide it through the rough waters. This may be a collective intention arrived at through a deliberate process of analysis and dialogue that is then formally written down in an “or ganizational strategy” document or it may represent the emergent characteristics of the combined actions of the whole organization expressed as a purpose or a series of goals. But just as an individual person seems to need a meaning in order to make sense of his or her life (Frankl, 1959) , so an organization seems to need a strategic intent in order to survive in a turbulent business environment. (de Geus, 1997) The strategic intent, of course, varies from organization to organization, but it will always encompass two kinds of goals: improvement of the current products or services and the processes and technologies for delivering them; and innovation and introduction of new products or services, processes or technologies. (Anbari, 2003). Regardless of industry or market sector, each of these two kinds of goals requires projects to be established, executed and completed so, in a very real sense, projects can be considered as an integral part of “strategy in action”, and the successful implementation of every strategy can be said to depend to a greater or lesser extent on the successful completion of projects. Unfortunately, at this point in the argument, it all becomes rather more problematical! Behind the discussion of project success lies a seething mass of complex assumptions and inter-related concepts that have led one author almost despairingly to ask, “Measuring success – can it really be done, and if carried out, what purpose does it serve?” (De Wit, 1988). Difficulties abound for many reasons: the different viewpoints, interests and expectations of groups of stakeholders involved in any given project; the subjective nature of perceptions of “success”; the tendency of perceptions to evolve over extended periods of time; the difficulty of assessing complex phenomena using simple metrics – the list is a lengthy one. On closer examination, project “success” turns out to be a rather slippery subject! And just as an exact definition of what constitutes project “success” eludes us, so the results achieved by projects continue to disappoint those associated with their sponsorship, execution or implementation (Morris and Hough, 1987; O'Connor and Reinsborough, 1992; KPMG, 1997; Cooke-Davies, 2001). It has been argued elsewhere that one way through the maze of discussions about project success is to consider the answers to three separate questions: “What factors lead to project management success?” “What factors lead to a successful project?”, and “What factors lead to consistently successful projects?” (CookeDavies, 2002 p185). But the focus of that article is success factors, those inputs to the management system that lead directly to the success of the project. These should not be confused with success criteria, which are the measures against which the success or failure of a project will be judged. Since this paper proposes answers to a question about “metrics” (the science of measuring), the focus will be rather more on success criteria than on success factors, but it is necessary to spell out the distinctive characteristics of the criteria that are normally used to assess the success of projects at each of the three levels. Level 1. Project management success – was the project done right? This is the measure of success that has dominated the practitioner-oriented literature on project management. In the folklore of the project manager it is about managing time, cost and quality. In reality, project objectives are rarely this simple. There will often be a business case to be born in mind or a gross profit to be made; there may be health, safety and environmental objectives to be accomplished; if the project is a technical one, or a “platform” new product development, there could be scientific or technical goals to reach. Nevertheless, the principle is simple: once the objectives of a project have been clearly defined and the constraints spelled out, then the project manager and her or his team can use their best endeavours to deliver the project so that it meets the objectives within the constraints. If anything changes, which is likely given the inherent uncertainty that is involved in any new endeavour, then techniques such as project risk management and project change control can be called into play as appropriate. As a guided missile seeks its target, adjusting its trajectory as appropriate along the way, so the project team seeks to achieve the project objectives. Is this then an appropriate level at which to measure the success of a project? There are three different kinds of arguments that suggest that it is. Firstly, modern project management has developed from a base of managing relatively “discrete” projects, each with its own organization, and each established to accomplish specific purposes (Morris, 1994). The kind of success criteria that are broadly used as measures of “project management success” have not only been those most commonly applied in the history of project management (Project Management Institute, 2000), but they also allow the project team as a coherent organizational unit to be accountable for its own performance and the practice of aligning accountability with authority is one of the well-attested principles of good management practice. Secondly, the underlying concept behind measuring success at this level is based on the well-understood principles of first-order cybernetics (Schwaninger, 1997) in much the same way that a thermostat or a guided missile operate. This is clearly appropriate for projects in which both the goals and the methods of achieving them are relatively clear at the outset (Turner and Cochrane, 1993). Thirdly, the capture of data about the extent to which projects within the same enterprise are successful in terms of project management success enables the enterprise to compare and contrast the practices that are generally associated with successful projects with those associated with unsuccessful ones. This in turn provides the enterprise with valuable information about which project management practices are in need of improvement within project teams. These are convincing arguments that support the case for continuing to measure project management success for many projects in many organizations. It is far from being the whole story, however, and for the second level of success it is necessary to turn to the second of De Wit’s levels – what he calls “project success” (1988). Level 2. Project success – was the right project done? This level of project success is perhaps the one that is of most interest to the “owner” or “sponsor” of the project. It is about “value for money” in its broadest sense. The assumption is that the project will be successful only if it is successfully operated, and delivers the benefits that were envisaged by the people and organizations (i.e. the stakeholders) that agreed to undertake the project in the first place. An analysis of six recent project management “bodies of knowledge” identified sixty core elements that are central to the way a project manager thinks about his or her work (Cooke-Davies, 2001). When these are clustered into eleven topic areas, and related to each other through a “systemigram”, it becomes clear that “anticipated benefits” become the touchstone not only for formal “stage gate” reviews of projects, but also for the continuous “informal assessment” of the likely success of projects carried out by owners, sponsors or senior management, and influencing decisions about priorities and resource allocation. Comparison of the eleven topic areas with previously published research about project success reveals a silence about “benefits,” perhaps because little has been written about benefits management or benefits realization until recently, and perhaps because the subject of “benefits” has been subsumed in the general discussion about “project purpose” or “project goals.” Nevertheless, there are three reasons why this is an appropriate level at which to measure the success of a project separately from the first level that was discussed, project management success. The strategic environment for projects. Practices on individual projects. Corporate Strategy and Goals. Processes and decisions t o t ranslat e st rat egy int o programmes and proj ect s, and resource t hem. Pr oj ect Pr oj ect t o deliver Pr oj ect t o deliver Pr oj ect t o deliver t o deliver Operat ions management t o derive benef it f rom t he pr oduct s. Pr oduct . Pr oduct . Pr oduct . Pr oduct . Exploit Exploit Pr oduct f or Exploit Pr oduct f or Exploit Pr oduct f or Pr oduct f or Benef it . Benef it . Benef it . Benef it . Figure 1: The involvement of both project management and operations management in the achievement of “project success” Firstly, as Figure 1 shows, benefits are not delivered or realized by the project manager and project team, they require the actions of operations management. This calls for a close co-operation between the project team on the one hand and the “sponsor,” “customer” and/or “user(s)” on the other. Thus the discussion of project success involves dialogue with a wider cross-section of the organization than is appropriate or necessary for proje ct management success. Secondly, delivering project success is necessarily more difficult than delivering project management success, because it inevitably involves “second order control” (both goals and methods liable to change) and thus brings into play an additional set of corporate processes to those that are involved in delivering project management success. And thirdly, the extent of project success is unlikely to become clear during the life of the project itself, whether success is measured quantitatively in terms of financial benefits or qualitatively in some less tangible form. For these three reasons, project success is itself a viable level at which to establish success criteria. It is not being suggested here that project success is somehow a “better” level at which to establish success criteria. Both project success and project management success are important to any project. If a project achieves project success without project management success, there is the inevitable conclusion that even greater benefits could have been realized. On the other hand, if project management success is achieved without project success, then the owner or sponsor has failed to obtain the benefits that the project was designed to provide. And that brings us to the third level of success. Level 3. Consistent project success – Were the right projects done right, time after time? As the focus moves from project management success, through project success to consistent project success, a completely new set of criteria come into play, as adjudged by different groups of stakeholders. Projects are the means by which all organizations accomplish business change, as well as the means by which some organizations deliver profits to their shareholders. The consistency with which projects accomplish both project success and project management success is thus a matter of interest to every organization that is competing in markets for scarce resources, such as customers or finance. At this level, a discussion of the criteria by which consistent project success is achieved is one that embraces the whole organization, and that will inevitably be influenced by its chosen market and strategic intent. For operations-driven organizations (such as financial services companies, or mass manufacturers) consistent project success in such areas as effective overall IT expenditure and new product development can lead to competitive advantage. For project-based organizations (such as engineering contractors, defense suppliers or turnkey IT systems providers), consistent project success can lead to profitable expansion. In either case, as the proportion of total work that is carried out in the form of projects increases, so consistent project success assumes an increasing strategic significance. In recent years there has been a growing interest in project portfolio management for new product development (e.g. Cooper et al., 2001) , specifically for R&D (e.g. Matheson and Matheson, 1998) or generally for project spend in organizations (e.g. Artto et al., 2001). But many organizations, particularly in traditional project-based industries, do not adopt a portfolio approach. For such organizations, as for all others, the effective and efficient use of scarce resources (particularly, but not only, people and money) remain of paramount importance. Thomas and Jugdev (2002) in their award-winning article on project management maturity models emphasize that long-range competitive advantage is enjoyed by those organizations that make the best use of their strategic assets (i.e. resources) which adds further weight to the claim that productivity of key resources is an important criterion at this third level. Project metrics: their role and some commonly encountered problems. One practical implication of this discussion of three different levels of success criteria is that an organization would do well to monitor its performance using a “suite” of project metrics that incorporates all three levels of success, if it is serious about understanding and improving its success in the field of projects. As figure 2 shows, each of the different levels of success is of interest to different levels of the corporate hierarchy, and each is visible after different amounts of time have elapsed relative to the project duration. No significant studies have as yet been published about the nature and extent of project metrics that are in general use, although Atkinson (1999) and Lim and Mohamed (1999) each argue that the need for multi-layered project success criteria is intimately linked to the need for more comprehensive metrics. Unpublished research within the Human Systems Network, however, supports this view and further indicates that few organizations are happy with the metrics that they use. Two phases of this research will inform the conclusions reached in this paper. Metrics lie at the heart of management information systems, and it is no different for projects. Accepted wisdom about measurement can be summed up in the phrase, “If you want it, measure it. If you can’t measure it, forget it.” (Belasco and Stayer, 1993, p216). The clear implication is that the primary function of information systems and metrics is to assist in control. The current fashion for “dashboards” makes this explicit. “While driving a car, it makes good sense to pay close attention to what you are doing, looking at the road ahead and checking who is on either side of you and who is visible in the rearview mirror. A good driver will glance at the dashboard of the car as they proceed. This dashboard holds the essential information needed to drive the car, the speedometer, the oil pressure gauge, the gas gauge, and a few others.” (Eckes, 2001, p21). Level of Hierarchy Required to give Commitment. Consistent Project Success. (Were the right projects done right, time after time?) Project Success – Was the right project done? Project Management Success – Was the project done right? Time elapsed before measurement is possible. Figure 2: Measurement of project success. Organizational commitment and time elapsed. The primary uses to which organizations put project management metrics are to establish a baseline for control purposes and to predict future costs (Egberding and Cooke-Davies, 2001). The least often used purposes are to assess corporate capability and to identify opportunities for improvement. These findings are entirely consistent with Eckes’ analogy of metric s being used the way that a driver of a car uses the dashboard. When looking at metrics in a little more detail, it is clear that organizations keep many different types: financials, time, quality, technical performance, resources used and available, project benefits, human factors and composites (dashboards, traffic lights etc.) (Egberding and Cooke-Davies, 2002). Indeed, there is much “good practic e” around where metrics are concerned, although only one organization (a U.S.A.-based defense contractor) was able to lay claim to “best practice” in each of the four areas that will be described below. Alongside these pockets of good practice, however, there is also considerable dissatisfaction with the metrics, especially from the financial community who tend to distrust the accuracy and objectivity of many of metrics produced from within the project management function. Composite metrics (such as “traffic light” indicators), while much liked by senior management, are also distrusted for their subjectivity and openness to bias. Where metrics are kept of “benefits management” they evoke almost universal dissatisfaction. There appears to me much room for improvement. Both the good practices and the causes of dissatisfaction are drawn upon in the section of this paper that describes the characteristics of a “best practice” suite of project management metrics, but before coming to that there is one more fundamental observation to be made. In an interesting paper delivered here in Australia, Thomas and Tjåder (2000) pointed out that “control” and “learning” assume two very different paradigms. As we have already seen, control is a feature of mechanical systems that are actively steered by an external “driver”. Learning, on the other hand is a feature of higher organisms and is, moreover, an activity that is of necessity self-directed from within the organism. These two paradigms coexist uneasily together, and all of the evidence available to me suggests that in the thinking of management it is the “control” agenda that dominates. Organizations are even less satisfied with their ability to transfer lessons learned from one project to others than they are with their suites of project management metrics! A discussion of paradigms (indeed of the underlying theories of how human beings work together to accomplish jointly-held objectives) is not the focus of this paper, and merits a much fuller discussion elsewhere. It is sufficient to remark at this point that the topic of control versus learning is incorporated into the third of the four “best practice” characteristics (a culture that encourages effective decision-making) that are described below. So, having considered the link between project success and organizational strategy, and reviewed the role that project management metrics play in project success, it is now possible to turn to the question that this paper set out to answer, “What are the characteristics of a suite of project management metrics that is capable of supporting an organization’s efforts to improve project success?” Characteristics of “best practice” in project management metrics. The characteristics are grouped together into four clusters. These do not represent four different sets of actions to be taken, but rather four different aspects of a well-rounded set of organizational “vital signs” that indicate the health of the project management efforts throughout the organization. Cluster I: Explicit improvement targets for project success that relate to organizational objectives. Arising from the primary and secondary research that informed our earlier discussion of the three levels of project success, three separate characteristics can be identified in this cluster of characteristics. 1 Improvement targets are set for each of the three levels: project management success, project success and consistent project success. These targets should be explicitly related to current annual business goals so that the benefits of improvement are directly linked to organizational strategy. Table 1: Three levels of project success summarized. 2 Ownership is clearly established for each of the targets at each level, distinguishing clearly between governance (sponsorship etc.) and management of the project. (Turner and Keegan, 1999). 3 Decision-making follows the principle of “subsidiarity”, with decisions be ing taken at the lowest level at which the decision-maker is in possession of all sufficient and necessary information. Aside from published assertions about the importance of “authority” and “empowerment”, evidence for this is provided by proprietary quantitative evidence showing the correlation between team empowerment and cycle times in pharmaceutical research and development and by research into the maturity of project management in different industries (Cooke-Davies and Arzymanow, 2002). A detailed review of the literature on project success criteria and success factors allows the material for this cluster of characteristics to be summarised in Table 1. The information has been derived from a detailed analysis of 44 published items of research into project management success, most particularly the excellent tabular summaries in Morris and Hough (1987), Belassi and Tukel (1996) and Crawford (2001, Appendices C and D) Cluster II: A comprehensive hierarchy of metrics that provides suitable information for the right people to make the right decisions. Aside from improvement targets for each of the three levels of project success, people at different organizational levels and in different job functions each require different sets of metrics to inform their decision-making. The second cluster of characteristics describes three different attributes of a well-formed hierarchy of metrics. 4 Different organizational levels are concerned with different kinds of decisions, and so need different levels of “roll-up” At a minimum, three levels can be distinguished, corresponding to the three levels described in characteristics 1 to 3: the top level of the discrete business unit or enterprise providing governance to the total portfolio of projects and programs within the business unit; the level of project sponsor or governance team responsible for the business benefits of a program or project; and the level of the project manager and project management team. 5 The assessment of multiple projects is qualitatively different from the assessment of a single project many times over, and so additional analysis is required in addition to the “roll-up” described in #4. At the top level, the metrics should cover more ground than simply the sum of the metrics from individual projects and programs within the portfolio. Metrics such as productivity, attrition, and process maturity are also relevant at this level. 6 Different job functions are interested in different aspects of project success. In addition to different hierarchical levels (see #4 above), a number of different job functions focus on different aspects. All job functions are interested in project costs, but finance managers are especially interested in resource utilization and financial benefits; project support offices are interested in human factors metrics (e.g. competencies, external certification) and quality metrics; business unit managers are interested in portfolio metrics; and technical functional heads are interested in specific metrics indicating the technical performance of the product being produced by the project. An effective suite of metrics provides each job function with those metrics that allow them to do their job effectively. Cluster III: A culture that encourages metrics to be used to improve the quality of decision-making in the organization. Metrics are of value only to the extent that they encourage people to make high-quality decisions about projects, so as to improve the specific aspects of project success for which they have responsibility and authority. This cluster of characteristics is, in many ways, the most important, since it indicates the health of the human heart of an organization. 7 Communications lies at the heart of complex organizational responses, and metrics play an important role in shaping conversations. This is the primary means of reconciling the conflicting agendas of control and learning. Decisions are made in conversation, and a well-crafted suite of metrics promotes dialogue that both encourages the right decisions to improve success, and also that results in the individuals involved in the dialogue learning something of benefit to the organisation. It is important that metrics are valid and are used to indicate “what is”, rather than to achieve specific outcomes desired by some parties at the expense of others. 8 Studies have shown that trust matters to project success (Construction Industry Institute, 1993) and conversations triggered by metrics can either foster or hinder the creation of trust. Thus metrics should be valued by and visible to all those who contribute to them. 9 The flow of metrics around an organization, coupled with different levels of confidence in the metrics, means that decision-making can be opaque and project management’s input undervalued. This can be especially true in two situations: financial data emanating from the project management function that is at variance with financial data emanating from the financial function, and estimates (particularly of either time or benefits) that are repeatedly shown to be optimistic, and can thus continually be challenged. 10 If the control function of metrics is overemphasised at the expense of learning, and coupled with a tendency to “micromanagement” on the part of governance bodies, there is likely to be a tendency to distort the metrics that are provided upwards, and thus to fall foul of characteristic #7 (valid data). Cluster IV: An infrastructure of processes and systems that collects necessary and useful metrics as efficiently as possible, and that is continually improved. This fourth and final cluster of characteristics encompasses the processes and systems that are used as a basis to provide the whole suite of metrics. 11 Metrics are only as reliable as the processes that produce them. Thus if the process is distrusted, so will the metrics be. For example, “traffic light” reporting (red, amber, green status) is often liked by senior management, because it allows the compact visual presentation of extensive amounts of data in a way that is easily absorbed, but it is distrusted elsewhere because of the variability of the judgements that influence the chosen colour. One area that is particularly prone to this difficulty is project planning – both for resource utilization and for forecasts of the date at which specific milestones will be achieved. The quality of time and resource forecasts is highly dependent on the quality of the planning process, and the availability of accurate data from past projects on which estimates can be based. 12 Multiple data flows lead to additional work reconciling information used as the basis of different metrics. This can be either hindered or helped by ERP systems. The accuracy of project financial data appears to be improved by the introduction of a well-planned ERP system, but individual project teams will need both careful training and ongoing support if the data available to projects for financial decisions is to match the rolled-up data being provided through financial management. 13 “Best Practice” separates the capture of data (done once) from the production of reports (done as automatically as possible). This calls for established disciplines, agreed definitions and wellthought-out IT/IS. Ideally, once a keystroke has been captured, it does not need to be re-entered into any system, but can be stored centrally and the accessed in as many different ways as is necessary for the production of the entire hierarchy of metrics, not only for projects, but for operational functions such as payroll and departmental financial and productivity statements. This raises, of course, questions over the audit and storage of input data. 14 Most organizations have considerable scope for reducing cost and effort in gathering and reporting metrics. The search for greater efficiency, accuracy and immediacy forms a continuous activity in the best run project-based businesses. Concluding comments: the role of metrics in developing project management capability. An assessment of the suite of project management metrics that an organization uses from the point of view of the four clusters of characteristics described in this paper performs two functions at the same time. Firstly it provides a window into the quality of the measurement system that is used by different people to inform the decisions that they make about projects, and thus to point to areas that can be improved. Secondly, and perhaps more importantly, these four clusters of characteristics highlight areas of strength and weakness in an organization’s overall project management capability by indicating the extent to which vital signs can be monitored and changes responded to. For both of these reasons every organization that wishes to improve its execution of corporate strategy would do well to devote some energy to assessing its suite of project management metrics. References Anbari, F. T. (2003). Strategic implementation of six sigma and project management. PMI Global Congress 2003 Europe. Philadelphia, PA: Project Management Institute. Artto, Karlos A., Martinsuo, Miia, and Aalto, Taru. (2001) Project Portfolio Management. Strategic management through projects. Helsinki, Finland: Project Management Association Finland. Atkinson, Roger. (1999). Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. International Journal of Project Management 17, no. 6: 337-42. Belasco, James A. and Stayer, Ralph C. (1993) Flight of the Buffalo New York: Warner. Belassi, Walid, and Oya Icmeli Tukel. (1996). A new framework for determining critical success/failure factors in projects. International Journal of Project Management 14, no. 3: 141-51. Construction Industry Institute. (1993). Cost-trust relationship. U.S.A.: Construction Industry Institute. Cooke-Davies, T. J. (2001) Towards improved project management practice: Uncovering the evidence for effective practices through empirical research. USA: Dissertation.com. ———. (2002). The "Real" Success Factors on Projects. International Journal of Project Management 20, no. 3: 18590. Cooke-Davies, T. J., and A. Arzymanow. (2002). The Maturity of Project Management in Different Industries. Proceedings of IRNOP V. Fifth International Conference of the International Network of Organizing by Projects. Rotterdam: Erasmus University. Cooper, Robert G., Edgett, Scott J., and Kleinschmidt, Elko J. (2001) Portfolio Management for New Products. 2nd Edition ed. Cambridge, MA: Perseus. Crawford, Lynn. (2001). "Project Management Competence: The value of standards." Henley Management College. de Geus, Arie. (1997) The living company: habits for survival in a turbulent business environment. Boston, Mass: Harvard Business School Press. De Wit, Anton. (1988). Measurement of project success. International Journal of Project Management 6, no. 3: 164-70. Eckes, George. (2001) The Six Sigma Revolution. How General Electric and others turned process into profits. New York, NY: John Wiley & Sons. Egberding, M. and Cooke -Davies, T. J. (2002) GTN Metrics Survey: preliminary report on Findings Unpublished Work Egberding, M. and Cooke -Davies, T. J. (2001) A quick summary of metrics pilot survey and other relevant background Unpublished Work Frankl, Viktor E. (1959) Man's search for meaning. New York: Washington Square Press. KPMG. (1997). Profit-Focused Software Package Implementation, KPMG, London. Lim, C. S., and M. Zain Mohamed. (1999). Criteria of project success: an exploratory re-examination. International Journal of Project Management 17, no. 4: 243-48. Matheson, David and Matheson, Jim. (1998) The smart organization. Creating value through strategic R&D Boston: Harvard Business School Press. Morris, Peter W. G. (1994) The management of projects London: Thomas Telford. Morris, Peter W. G. and Hough, George H. (1987) The Anatomy of Major Projects. A study of the reality of project management London: Wiley. O'Connor, Michael M, and Laurie Reinsborough. (1992). Quality projects in the 1990s: a review of past projects and future trends. International Journal of Project Management 10, no. 2: 107-14. Project Management Institute. (2000). A Guide to the Project Management Body of Knowledge. 2000 ed ed. Philadelphia: Project Management Institute. Schwaninger, Markus. (1997) Status and Tendencies of Management Research: a Systems Oriented Perspectives. In Multimethodology. The theory and practice of combining management science methodologies. Chichester: John Wiley and Sons. Thomas, Janice, and Kam Jugdev. (2002). Project Management Maturity Models: The Silver Bullets of Competitive Advantage? Project Management Journal 33, no. 4: 4-14. Thomas, Janice, and Jimmy Tjåder. (2000). On Learning and Control - Competing Paradigms or Co-existing Requirements for Managing Projects in Ambiguous Situations? IRNOP IV. Sydney: University of Technology in Sydney. Turner, J. Rodney, and Robert A. Cochrane. (1993). Goals -and-methods matrix: coping with projects with ill defined goals and/or methods of achieving them. International Journal of Project Management 11, no. 2: 93 to 102. Turner, J. Rodney, and Anne Keegan. (1999). The Versatile Project-based Organization: Governance and Operational Control. European Management Journal 17, no. 3: 296-309.