Code-based metrics such as coupling and cohesion are used to measure a system’s structural complexity. But dealing with large systems—those consisting of several millions of lines — at the code level faces many problems. An alternative approach is to concentrate on the system’s building blocks such as programs or modules as the unit of examination. We present an approach that uses information in a release history of a system to uncover logical dependencies and change patterns among modules. We have developed the approach by working with 20 releases of a large Telecommunications Switching System. We use release information such as version numbers of programs, modules, and subsystems together with change reports to discover common change behavior (i.e. change patterns) of modules. Our approach identifies logical coupling among modules in such a way that potential structural shortcomings can be identified and further examined, pointing to restructuring or reengineering opportunities. 1

The dearth of published empirical data on major industrial systems has been one of the reasons that software engineering has failed to establish a proper scientific basis. In this paper we hope to provide a small contribution to the body of empirical knowledge. We describe a number of results from a quantitative study of faults and failures in two releases of a major commercial system. We tested a range of basic software engineering hypotheses relating to: the Pareto principle of distribution of faults and failures; the use of early fault data to predict later fault and failure data; metrics for fault prediction; and benchmarking fault data. For example, we found strong evidence that a small number of modules contain most of the faults discovered in pre-release testing, and that a very small number of modules contain most of the faults discovered in operation. However, in neither case is this explained by the size or complexity of the modules. We found no evidence to support previous claims relating module size to fault density, nor did we find evidence that popular complexity metrics are good predictors of either fault-prone or failure-prone modules. We confirmed that the number of faults discovered in pre-release testing is an order of magnitude greater than the number discovered in 12 months of operational use. We also discovered fairly

Large software systems evolve slowly but constantly. In this paper we examine the structure of several releases of a telecommunication switching system (TSS) based on information stored in a database of product releases. We tracked the historical evolution of the TSS structure and related the adaptations made (e.g. addition of new features, etc.) to the structure of the system. Such a systematic examination can uncover potential shortcomings in the structure of the system and identify modules or subsystems that should be subject to restructuring or reengineering. Further, we have identified additional information that would be useful for such investigations but is currently lacking in the database.

This paper uses the case study and its results to qualitatively compare our approach against current ad hoc practices used to improve existing measurement frameworks

Measurement definition, collection, and analysis is an essential component of high quality software engineering practice, and is thus an essential component of the software engineering curriculum. However, providing students with practical experience with measurement in a classroom setting can be so time-consuming and intrusive that it’s counter-productive—teaching students that software measurement is “impractical ” for many software development contexts. In this research, we designed and evaluated a very low-overhead approach to measurement collection and analysis using the Hackystat system with special features for classroom use. We deployed this system in two software engineering classes at the University of Hawaii during Fall, 2003, and collected quantitative and qualitative data to evaluate the effectiveness of the approach. Results indicate that the approach represents substantial progress toward practical, automated metrics collection and analysis, though issues relating to the complexity of installation and privacy of user data remain. 1.

this report is to review and summarize the empirical evidence thus far on the costs and benefits of SPI. The intention is that this review would be utilized to support the business case for initiating and continuing SPI programs, to aid in the selection amongst the alternative improvement paradigms, to make more accurate estimates of the costs and benefits of such efforts, and to help set and manage the expectations of technical staff and management. The need for such a review is supported by the results of two recent surveys that were conducted by the SEI. The first survey was administered to individuals at the National SEPG Conference in 1993 and at an SPI tutorial during the Software Engineering Symposium in 1993 [25]. The respondents represented organizations that had mature SPI programs. More than seventy percent stated that they need information on the benefits of SPI (by choosing the "very high" or "high" response category in terms of characterizing their needs), which was also ranked as the highest need of the respondents. This indicates a need for consolidation of the evidence on the benefits of SPI. The second survey solicited information from organizations that had conducted software process assessments between 1992 and 1993 [26]. The results indicate that 77% of the respondents "Strongly Agree" or "Agree" that SPI has taken longer than expected and 68% stated that SPI has cost more than expected. This indicates a need for information to help estimate the costs of SPI and to set and manage expectations from SPI. Two general paradigms to SPI have emerged, as described by Card [10]. The first is the analytic paradigm. This is characterized as relying on "quantitative evidence to determine where improvements are needed and whether an improvement initiative has b...

Software cost and quality estimation has become an increasingly important field due to the increasingly pervasive role of software in today's world. In spite of the existence of about a dozen software estimation models, the field continues to remain nottoo -well-understood, causing growing concerns in the software-engineering community. In this dissertation, the existing techniques that are used for building software estimation models are discussed with a focus on the empirical calibration of the models. It is noted that traditional calibration approaches (especially the popular multipleregression approach) can have serious difficulties when used on software engineering data that is usually scarce, incomplete, and imprecisely collected. To alleviate these problems, a composite technique for building software models based on a mix of data and expert judgement is discussed. This technique is based on the well understood and widely accepted Bayes' theorem that has been successfully applie...

Abstract—Understanding how to implement software process improvement (SPI) successfully is arguably the most challenging issue facing the SPI field today. The SPI literature contains many case studies of successful companies and descriptions of their SPI programs. However, the research efforts to date are limited and inconclusive and without adequate theoretical and psychometric justification. This paper extends and integrates models from prior research by performing an empirical investigation of the key factors for success in SPI. A quantitative survey of 120 software organizations was designed to test the conceptual model and hypotheses of the study. The results indicate that success depends critically on six organizational factors, which explained more than 50 percent of the variance in the outcome variable. The main contribution of the paper is to increase the understanding of the influence of organizational issues by empirically showing that they are at least as important as technology for succeeding with SPI and, thus, to provide researchers and practitioners with important new insights regarding the critical factors of success in SPI.

The most successful measurement programs are ones in which researcher, practitioner, and customer work hand in hand to meet goals and solve problems. But such collaboration is rare. The authors explore the gaps between these groups and point toward ways to bridge them. In any scientific field, measurement generates quantitative descriptions of key processes and products, enabling us to understand behavior and result. This enhanced understanding lets us select better techniques and tools to control and improve our processes, products, and resources. Because engineering involves the analysis of measurements, software engineering cannot become a true engineering discipline unless we build a solid foundation of measurement-based theories. One obstacle to building this base is the gap between measurement research and measurement practice. This status report describes the state of research, the state of the art, and the state of practice of software measurement. It reflects discussion at the Second International Software Metrics Symposium, which we organized. The aim of the symposium is to encourage researchers and practitioners to share their views, problems, and needs, and to work together to define future activities that will address common goals. Discussion at the symposium revealed that participants had different and sometimes conflicting motivations.

Reuse metrics must accurately reflect the amount of effort saved. We must have realistic measures to ensure the credibility of the value we place on reuse and to separate reuse benefits from those of other technologies also competing for limited investment dollars. This paper defines a reuse metrics and Return On Investment (ROI) model at IBM that distinguishes reuse savings and benefits from those already gained through accepted software engineering techniques. Used in conjunction with a planned reuse program, the potential of reuse serves as a powerful motivator. We derive three reuse metrics from readily available software data elements and use these metrics in a return on investment model that establishes a sound business justification for reuse. 1 1.0 Overview Management traditionally uses software metrics to quantify the software development process. With an emerging technology, however, metrics must extend beyond their traditional role; applying the reuse metrics must also encou...