Program Analysis

Abstract—Forensic analysis of a suspect program is a daily challenge encounters forensic analysts and law-enforcement. It requires determining the behavior of a suspect program found in a computer system subject to investigation and attempting to reconstruct actions that have been invoked in the system. In this research paper, a forensic analysis approach for suspect programs in an executable binary form is introduced. The proposed approach aims to reconstruct high level forensic actions and approximate action arguments from low level machine instructions; That is, reconstructed actions will assist in forensic inferences of evidence and traces caused by an action invocation in a system subject to forensics investigation.

The area of tool support for software inspection has been under active research since the early 1990’s. Although numerous implementations exist and development is still taking place, no tool has achieved a break-through. The main reason is that one tool usually demonstrates only one new idea, neglecting other features. A different approach must be taken, and software inspection tools should be seen as integral parts of the development environment. This paper categorises the existing tools into four generations based on the transition from traditional meeting support to asynchronous distributed inspections implemented with web technologies. Based on the analysis of 16 tools and our experiences, we summarize the most important features and add two new aspects to be notified when implementing the next generation of inspection tools for use in modern software development, flexibility and integration. The major focus is on comprehension, and we have taken the first steps at achieving this.

Formal methods and testing are two important approaches that assist in the development of high-quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing.

Debugging large-scale parallel applications is challenging. In most HPC applications, parallel tasks progress in a coordinated fashion, and thus a fault in one task can quickly propagate to other tasks, making it difficult to debug. Finding the least-progressed tasks can significantly reduce the effort to identify the task where the fault originated. However, existing approaches for detecting them suffer low accuracy and large overheads; either they use imprecise static analysis or are unable to infer progress dependence inside loops. We present a loop-aware progress-dependence analysis tool, Prodometer, which determines relative progress among parallel tasks via dynamic analysis. Our fault-injection experiments suggest that its accuracy and precision are over 90% for most cases and that it scales well up to 16,384 MPI tasks. Further, our case study shows that it significantly helped diagnosing a perplexing error in MPI, which only manifested at large scale.

Binary code search has received much attention recently due to its impactful applications, e.g., plagiarism detection, malware detection and software vulnerability auditing. However, developing an effective binary code search tool is challenging due to the gigantic syntax and structural differences in binaries resulted from different compilers, architectures and OSs. In this paper, we propose BINGO— a scalable and robust binary search engine supporting various ar-chitectures and OSs. The key contribution is a selective inlining technique to capture the complete function semantics by inlining relevant library and user-defined functions. In addition, architecture and OS neutral function filtering is proposed to dramatically reduce the irrelevant target functions. Besides, we introduce length variant partial traces to model binary functions in a program structure agnostic fashion. The experimental results show that BINGO can find semantic similar functions across architecture and OS boundaries, even with the presence of program structure distortion, in a scalable manner. Using BINGO, we also discovered a zero-day vulnerability in Adobe PDF Reader, a COTS binary.

This article presents results from the first statistically significant study of cost escalation in transportation infrastructure projects. Based on a sample of 258 transportation infrastructure projects worth US$90 billion and representing different project types, geographical regions, and historical periods, it is found with overwhelming statistical significance that the cost estimates used to decide whether such projects should be built are highly and systematically misleading. Underestimation cannot be explained by error and is best explained by strategic misrepresentation, that is, lying. The policy implications are clear: legislators, administrators, investors, media representatives, and members of the public who value honest numbers should not trust cost estimates and cost-benefit analyses produced by project promoters and their analysts.

Megaproject Planning and Management: Essential Readings contains the seminal articles from the growing body of research on megaproject planning and management along with an original introduction by the editor, Bent Flyvbjerg. The leading and most cited authority in the field, Flyvbjerg has used crowdsourcing and 25 years of experience to cherry-pick from several hundred articles and books the writings that define the field. This volume will be an indispensable source for those wishing to speak with authority about how megaprojects are prepared, delivered, and fought over. The target audience is students, academics, practitioners, and media pundits alike, as well as communities affected by megaprojects.

Back cover text: Megaprojects and Risk provides the first detailed examination of the phenomenon of megaprojects. It is a fascinating account of how the promoters of multi-billion dollar megaprojects systematically and self-servingly misinform parliaments, the public and the media in order to get projects approved and built. It shows, in unusual depth, how the formula for approval is an unhealthy cocktail of underestimated costs, overestimated revenues, undervalued environmental impacts and overvalued economic development effects. This results in projects that are extremely risky, but where the risk is concealed from MPs, taxpayers and investors. The authors not only explore the problems but also suggest practical solutions drawing on theory, experience and hard, scientific evidence from the several hundred projects in twenty nations and five continents that illustrate the book. Accessibly written, it will be the standard reference for students, scholars, planners, economists, auditors, politicians and interested citizens for many years to come.

The article first describes characteristics of major infrastructure projects. Second, it documents a much neglected topic in economics: that ex ante estimates of costs and benefits are often very different from actual ex post costs and benefits. For large infrastructure projects the consequences are cost overruns, benefit shortfalls, and the systematic underestimation of risks. Third, implications for cost–benefit analysis are described, including that such analysis is not to be trusted for major infrastructure projects. Fourth, the article uncovers the causes of this state of affairs in terms of perverse incentives that encourage promoters to underestimate costs and overestimate benefits in the business cases for their projects. But the projects that are made to look best on paper are the projects that amass the highest cost overruns and benefit shortfalls in reality. The article depicts this situation as ‘survival of the unfittest’. Fifth, the article sets out to explain how the problem may be solved, with a view to arriving at more efficient and more democratic projects, and avoiding the scandals that often accompany major infrastructure investments. Finally, the article identifies current trends in major infrastructure development. It is argued that a rapid increase in stimulus spending, combined with more investments in emerging economies, combined with more spending on information technology is catapulting infrastructure investment from the frying pan into the fire.

As megaprojects have become ubiquitous, their real benefits and costs have come under increased scrutiny. We interviewed Bent Flyvbjerg, who has extensively studied megaproject development. Flyvbjerg has found systematic problems in the development process: by intentionally misrepresenting information and deliberately disregarding risks, proponents instigate projects that result in fewer benefits and higher costs than promised.

The Supplementary Green Book Guidance on Optimism Bias (HM Treasury 2003) with reference to the Review of Large Public Procurement in the UK (Mott MacDonald 2002) notes that there is a demonstrated, systematic, tendency for project appraisers to be overly optimistic and that to redress this tendency appraisers should make explicit, empirically based adjustments to the estimates of a project’s costs, benefits, and duration. HM Treasury recommends that these adjustments be based on data from past projects or similar projects elsewhere, and adjusted for the unique characteristics of the project in hand. In the absence of a more specific evidence base, HM Treasury has encouraged departments to collect data to inform future estimates of optimism, and in the meantime use the best available data. In response to this, the Department for Transport (henceforth DfT), has contracted Bent Flyvbjerg in association with COWI to undertake the consultancy assignment "Procedures for dealing with Optimism Bias in Transport Planning". The present Guidance Document is the result of this assignment.

"Over budget, over time, over and over again" appears to be an appropriate slogan for large, complex infrastructure projects. This article explains why cost, benefits, and time forecasts for such projects are systematically over-optimistic in the planning phase. The underlying reasons for forecasting errors are grouped into three categories: delusions or honest mistakes; deceptions or strategic manipulation of information or processes; or bad luck. Delusion and deception have each been addressed in the management literature before, but here they are jointly considered for the first time. They are specifically applied to infrastructure problems in a manner that allows both academics and practitioners to understand and implement the suggested corrective procedures. The article provides a framework for analyzing the relative explanatory power of delusion and deception. It also suggests a simplified framework for analyzing the complex principal-agent relationships that are involved in the approval and construction of large infrastructure projects, which can be used to improve forecasts. Finally, the article illustrates reference class forecasting, an outside view de-biasing technique that has proven successful in overcoming both delusion and deception in private and public investment decisions.

This article presents results from the first statistically significant study of traffic forecasts in transportation infrastructure projects. The sample used is the largest of its kind, covering 210 projects in 14 nations worth U.S.$59 billion. The study shows with very high statistical significance that forecasters generally do a poor job of estimating the demand for transportation infrastructure projects. For 9 out of 10 rail projects, passenger forecasts are overestimated; the average overestimation is 106%. For half of all road projects, the difference between actual and forecasted traffic is more than ±20%. The result is substantial financial risks, which are typically ignored or downplayed by planners and decision makers to the detriment of social and economic welfare. Our data also show that forecasts have not become more accurate over the 30-year period studied, despite claims to the contrary by forecasters. The causes of inaccuracy in forecasts are different for rail and road projects, with political causes playing a larger role for rail than for road. The cure is transparency, accountability, and new forecasting methods. The challenge is to change the governance structures for forecasting and project development. Our article shows how planners may help achieve this.

Do different types of megaprojects have different cost overruns? This apparently simple question is at the heart of research at the University of Oxford aimed at understanding the characteristics of megaprojects, particularly in terms of how they are established, run and concluded. In this study, we set out to investigate cost overruns in the Olympic Games. To do so, we examined the costs of the Games over half a century, including both summer and winter Olympics. We looked at the evolution of final reported costs and compared these to the costs established in the Games bids, submitted to the International Olympic Committee (IOC) up to seven years before the Games occurred. In so doing we established the largest dataset of its kind, and documented for the first time in a consistent fashion the costs and cost overruns for the Olympic Games, from 1960 to 2012. We discovered that the Games stand out in two distinct ways compared to other megaprojects: (1) The Games overrun with 100 per cent consistency. No other type of megaproject is this consistent regarding cost overrun. Other project types are typically on budget from time to time, but not the Olympics. (2) With an average cost overrun in real terms of 179 per cent – and 324 per cent in nominal terms – overruns in the Games have historically been significantly larger than for other types of megaprojects, including infrastructure, construction, ICT, and dams. The data thus show that for a city and nation to decide to host the Olympic Games is to take on one of the most financially risky type of megaproject that exists, something that many cities and nations have learned to their peril. For the London 2012 Games, we find that: (1) With sports-related real costs currently estimated at USD14.8 billion, London is on track to become the most costly Olympics ever. (2) With a projected cost overrun of 101 per cent in real terms, overrun for London is below the historical average for the Games, but not significantly so. (3) The London cost overrun is, however, significantly higher than overruns for recent Games since 1999. London therefore is reversing a positive trend of falling cost overruns for the Games.

As megaprojects have become ubiquitous, their real benefits and costs have come under increased scrutiny. We interviewed Bent Flyvbjerg, who has extensively studied megaproject development. Flyvbjerg has found systematic problems in the development process: by intentionally misrepresenting information and deliberately disregarding risks, proponents instigate projects that result in fewer benefits and higher costs than promised.

Back cover text: This book aims to enlarge the understanding of decision-making on mega-projects and suggest recommendations for a more effective, efficient and democratic approach. Authors from different scientific disciplines address various aspects of the decision-making process, such as management characteristics and cost-benefit analysis, planning and innovation and competition and institutions. The subject matter is highly diverse, but certain questions remain at the forefront. For example, how do we deal with protracted preparation processes, how do we tackle risks and uncertainties, and how can we best divide the risks and responsibilities among the private and public players throughout the different phases of the project? - Presenting a state-of-the-art overview, based on experiences and visions of authors from Europe and North America, this unique book will be of interest to practitioners of large-scale project management, politicians, public officials and private organisations involved in mega-project decision-making. It will also appeal to researchers, consultants and students dealing with substantial engineering projects, complex systems, project management and transport infrastructure.

Coupling measures are considered important due to their inherent relationships with external software quality attributes. The ongoing research on object oriented software quality has been focusing on devising dynamic metrics for software quality assessment because of their ability to capture the runtime behavior of software. A number of dynamic coupling metrics have been proposed in recent times to measure software quality. This paper is an attempt to fill in the gap between existing work and researchers by providing them essential knowledge about the dynamic coupling metrics, methodologies and techniques proposed in this area. This paper presents a survey of existing research related to dynamic coupling metrics for object oriented software systems. Metrics have been categorized according to some of their major characteristics such as dynamic analysis techniques, validation of metrics, relation with static metrics etc. The capability of these metrics in quantifying external software quality attributes is also studied. The survey found that very few dynamic coupling metrics have been proposed and most of the proposed metrics lack empirical validation limiting their applicability in software quality assessment.

The purpose of this study was to evaluate the effect of an integrated, laptop-based approach on preservice teachers’ computer attitudes, ability and use. Pre-post program analysis revealed significant differences in behavioural attitudes and perceived control (self-efficacy), but not in affective and cognitive attitudes. In addition, there was a significant improvement in all ten computer ability areas (operating systems, communication, World Wide Web, word processing, spreadsheets, database, graphics, multimedia, web page design, and programming). Finally, preservice teachers used laptop computers significantly more in a university setting than in their field placements.

This paper focuses on problems and their causes and cures in policy and planning for large infrastructure projects. First, it identifies as the main problem in major infrastructure development pervasive misinformation about the costs, benefits, and risks involved. A consequence of misinformation is massive cost overruns, benefit shortfalls, and waste. Second, the paper explores the causes of misinformation and finds that political-economic explanations best account for the available evidence: planners and promoters deliberately misrepresent costs, benefits, and risks in order to increase the likelihood that it is their projects, and not the competition's, that gain approval and funding. This results in the "survival of the unfittest," where often it is not the best projects that are built, but the most misrepresented ones. Finally, the paper presents measures for reforming policy and planning for large infrastructure projects, with a focus on better planning methods and changed governance structures, the latter being more important.

Despite an array of policies at the federal and state level aimed at regulating stormwater discharges, engineered solutions enforced by local governments often fall short of meeting water quality standards. Although the implications of land use planning and development regulations are important for stormwater management, they are often overlooked as critical initial steps to improving water quality. This study explores the role of ‘form-based’ regulations as tools for achieving urban planning and water quality objectives. Form-based codes are a new generation of development codes aimed at regulating urban development based on urban form and density, rather than land use. We present an exploratory case study of the feasibility of form-based codes in the Jordan Lake Watershed in North Carolina, a rapidly growing region where fragmented local governments face stringent nutrient reduction standards under new state regulations. Through program analysis and interviews, we explore the viability of form-based codes for reducing development impacts on Jordan Lake’s water quality. We consider the legal feasibility of code enforcement, regional and local barriers and opportunities, and implementation given existing regulatory frameworks. Our findings suggest that high quality information and data modeling are foundational to gaining support for a consensus agreement on the sources and degree of water quality impairment. Furthermore, implementing form-based solutions for water quality is greatly aided by (1) experienced regional planning bodies that have regulatory authority, and (2) local governments whose staff are experienced in implementing complex development ordinances, reviewing architectural renderings, and communicating development requirements with the public and developers.

Risk, including economic risk, is increasingly a concern for public policy and management. The possibility of dealing effectively with risk is hampered, however, by lack of a sound empirical basis for risk assessment and management. This article demonstrates the general point for cost and demand risks in urban rail projects. The article presents empirical evidence that allow valid economic risk assessment and management of urban rail projects, including benchmarking of individual or groups of projects. Benchmarking of the Copenhagen Metro is presented as a case in point. The approach developed is proposed as a model for other types of policies and projects in order to improve economic and financial risk assessment and management in policy and planning.