Objective: Our objective is to describe how software engineering might benefit from an evidence-based approach and to identify the potential difficulties associated with the approach. Method: We compared the organisation and technical infrastructure supporting evidence-based medicine (EBM) with the situation in software engineering. We considered the impact that factors peculiar to software engineering (i.e. the skill factor and the lifecycle factor) would have on our ability to practice evidence-based software engineering (EBSE). Results: EBSE promises a number of benefits by encouraging integration of research results with a view to supporting the needs of many different stakeholder groups. However, we do not currently have the infrastructure needed for widespread adoption of EBSE. The skill factor means software engineering experiments are vulnerable to subject and experimenter bias. The lifecycle factor means it is difficult to determine how technologies will behave once deployed. Conclusions: Software engineering would benefit from adopting what it can of the evidence approach provided that it deals with the specific problems that arise from the nature of software engineering. 1.

There is a growing interest in empirical study in software engineering, both for validating mature technologies and for guiding improvements of less-mature technologies. This paper introduces an empirical methodology, based on experiences garnered over more than two decades of work by the Empirical Software Engineering Group at the University of Maryland and related organizations, for taking a newly proposed improvement to development processes from the conceptual phase through transfer to industry. The methodology presents a series of questions that should be addressed, as well as the types of studies that best address those questions. The methodology is illustrated by a specific research program on inspection processes for ObjectOriented designs. Specific examples of the studies that were performed and how the methodology impacted the development of the inspection process are also described. Categories and Subject Descriptors K.6.3 [Management of Computing and Information Systems]: Software Management --Software process. General Terms Measurement, Design, Experimentation, Verification. Keywords Empirical studies, OO design inspections, software process, experimental process, software quality 1 CHALLENGES OF SOFTWARE TECHNOLOGY

Abstract—Understanding how to implement software process improvement (SPI) successfully is arguably the most challenging issue facing the SPI field today. The SPI literature contains many case studies of successful companies and descriptions of their SPI programs. However, the research efforts to date are limited and inconclusive and without adequate theoretical and psychometric justification. This paper extends and integrates models from prior research by performing an empirical investigation of the key factors for success in SPI. A quantitative survey of 120 software organizations was designed to test the conceptual model and hypotheses of the study. The results indicate that success depends critically on six organizational factors, which explained more than 50 percent of the variance in the outcome variable. The main contribution of the paper is to increase the understanding of the influence of organizational issues by empirically showing that they are at least as important as technology for succeeding with SPI and, thus, to provide researchers and practitioners with important new insights regarding the critical factors of success in SPI.

Research on software reuse libraries has extensively dealt with representation and retrieval issues of software artifacts. While representation in terms of metadata is a key issue, most systems neglect the possibilities of leveraging knowledge about the corresponding problem domain. In this paper, we present KOntoR -- an ontology-enabled approach to software reuse. We show how background knowledge provided in the form of ontologies can increase the value of reuse libraries. This is achieved by integrating explicit and implicit metadata semantically, thus providing means for deriving new facts. Additionally, we give three examples that show how software library users can benefit from formalized knowledge, e.g. about software licenses or programming technologies.

This paper describes the development of an experience factory in an Australian organization. Information structures were well developed and used in the daily work of the organization. This included the use of network technology as well as the personal interaction between department members. Highly motivated personnel drove improvement via new techniques, knowledge, and tools. A special focus existed to simplify work tasks through tool support. Daily work and problem solving was strongly based on personnel interaction and access to knowledge bases (documentation, mail lists, etc.). The goal of the project was to package personnel experience and best practices and provide an effective framework for access and integration. The system was decommissioned shortly after the completion of the project. The reasons for this are discussed.

Relying on volunteer labor, open source projects like the Apache web server create commercial quality software. Why developers contribute freely without direct remuneration has been widely debated. We offer empirical evidence that such participation can be explained by existing theories in labor economics. Analyzing panel data covering a four-year period, we find that increases in human capital, measured as project contribution, do not lead to increased wages. In contrast, credentials earned through a merit-based ranking system are associated with significantly increased wages. Our results suggest that status within an open source meritocracy operates as a credible signal of productive capacity. * We thank the open source programmers who have contributed to this study. We also thank the participants of the

Abstract—Experience management refers to the capture, structuring, analysis, synthesis, and reuse of an organization’s experience in the form of documents, plans, templates, processes, data, etc. The problem of managing experience effectively is not unique to software development, but the field of software engineering has had a high-level approach to this problem for some time. The Experience Factory is an organizational infrastructure whose goal is to produce, store, and reuse experiences gained in a software development organization [6], [7], [8]. This paper describes The Q-Labs Experience Management System (Q-Labs EMS), which is based on the Experience Factory concept and was developed for use in a multinational software engineering consultancy [31]. A critical aspect of the Q-Labs EMS project is its emphasis on empirical evaluation as a major driver of its development and evolution. The initial prototype requirements were grounded in the organizational needs and vision of Q-Labs, as were the goals and evaluation criteria later used to evaluate the prototype. However, the Q-Labs EMS architecture, data model, and user interface were designed to evolve, based on evolving user needs. This paper describes this approach, including the evaluation that was conducted of the initial prototype and its implications for the further development of systems to support software experience management. Index Terms—Experience management, knowledge management, experience reuse, user interface evaluation, empirical study. 1

Abstract not found

Ownership is a key aspect of large-scale software development. We examine the relationship between different ownership measures and software failures in two large software projects: Windows Vista and Windows 7. We find that in all cases, measures of ownership such as the number of low-expertise developers, and the proportion of ownership for the top owner have a relationship with both pre-release faults and post-release failures. We also empirically identify reasons that low-expertise developers make changes to components and show that the removal of low-expertise contributions dramatically decreases the performance of contribution based defect prediction. Finally we provide recommendations for source code change policies and utilization of resources such as code inspections based on our results.

Management of architecture knowledge is vital for improving an organization’s architectural capabilities. Despite the recognition of the importance of capturing and reusing architecture knowledge, there is no suitable support mechanism. We propose a conceptual framework for providing appropriate guidance and tool support for making tacit or informally described architecture knowledge explicit. This framework identifies different approaches to capturing implicit architecture knowledge. We discuss different usages of the captured knowledge to improve the effectiveness of architecting process. The report also presents a brief description of a prototype of a web-based architecture knowledge management tool to support the storage and retrieval of the captured knowledge. The report concludes with open issues that we plan to address in order to successfully transfer this support mechanism for capturing and using architecture knowledge to the industry. 1.