Software development is often unpredictable and imprecise for many reasons – new technologies emerge, the needs of customers change, and we are constantly building larger and more complex software systems. Unlike traditional engineering disciplines, there are no blueprints for software. Best practices and lessons learned on how to best build software vary from person to person, organization to organization, and project to project. These “rules” for building better software are often based on personal experience or anecdotes and, while useful, are rarely backed by scientific evidence or apply only in a very limited number of scenarios. For decades, researchers have attempted to derive rules for controlling the cost and quality of software, but with mixed success at best.

The NSF Transfer Learning in Software Engineering project sought to address one of the major reasons that scientific predictability of software quality and effort has eluded us: that it is difficult to identify relevant data from past software projects from which to draw rules for engineering the software system at hand. To tackle this problem, we engaged in a number of activities.

We developed the XTREE algorithm that evaluates the impact of proposed changes to software structure (i.e., refactoring) on software quality. For example, will reducing the complexity of a particular piece of code actual reduce the number of bugs in the system? These decisions are evaluated using data from the project’s history of changes. The XTREE algorithm enables software project managers to make decisions on how to allocate effort based on past performance, rather than guesswork and anecdote.

We developed the LACE2 privacy algorithm, which enables private companies and other organizations to contribute highly-detailed software development data (for example, the size of source code files and the number of defects in them) without revealing the origin of the data. By growing the amount of data available, it is more likely we can identify a past project from which to transfer rules to the current project.

We applied a tool from psychology, the repertory grid, to extract lessons learned from professional software engineers and compare them in a scientifically valid way. This technique shows promise for documenting, comparing, and quantifying lessons learned rather than relying on simple anecdotes.

We applied natural language processing techniques to pool together data from multiple NASA missions on how software bugs occur in operation. These techniques show promise for generating new sources of lessons learned from unstructured data (i.e., reports, emails) that can be used to make quantitative decisions.

Finally, we examined whether one of the most commonly-held laws of software development is, in fact, a law in the scientific sense. In a survey of several dozen software engineers, the assertion that “the longer a problem remains in the system, the more expensive it is to fix” was the most commonly-believed “law” of software engineering. Yet, in a study of several hundred software projects that employed the Team Software Process, we find that this assertion was not true.

In total, this research produced five publications authored by Fraunhofer staff and supported the research effort of three undergraduate students. This work brought together software engineering data and lessons learned from commercial companies, government organizations, and open source repositories.

This research made significant headway to addressing the problem of transferring rules for developing better software quantitatively from project to project. There is still much work to be done to tame the complexities of software development introduced by variations in individuals, organizations, and their processes.

Last Modified: 06/30/2017
Modified by: Lucas Layman