An empirical model of software managers' information needs for software engineering technology selection

The current situation regarding technology selection in software engineering can be compared to a patient who buys a drug he has heard about, but for which no package insert is available and for which existing evidence about its appropriateness in the current situation (disease) is ignored because it is not easily accessible. Most people will agree that the availability of appropriate information can be a major contribution to informed and successful decision-making. The introduction of a new software engineering technology is a critical decision. The use of limited information, especially with respect to a technology's inherent benefits and risks, might dramatically influence the success of this decision. Empirical software engineering tries to provide evidence about a technology's benefits. However, there seems to be a lack of recognition of this work in industry. When reporting results from experiments, empirical software engineering researchers do not provide information that is relevant for software managers. Information that would support the application of empirical research results in software engineering decision-making is often neglected. Thus, it is no wonder that these results are not widely used in decision-making in industry. We propose characterizing and formalizing software managers' information needs so that information relevant for the decision-making process is recognized by empirical software engineering research and can thus be made available. In order to find out what software managers need to know to help them judge the appropriateness and impact of a software technology, we started from a literature-based information needs model and empirically investigated the information needs of software and senior managers. By merging software managers' information needs with those of senior management, we arrived, by induction, at a model that characterizes the information needed by managers in the decision-making process, especially when selecting a software engineering technology. We have used the information needs model for two purposes. First, we built a repository, which was integrated with the respective processes into a framework for technology selection. The framework allows easy, goal- and problem-oriented access to evidence collected from experiments. Second, we analyzed how the information needs model can be used to provide relevant information when reporting results from experiments. For this purpose, we proposed extending existing reporting guidelines with appropriate sections for the relevant information. The effectiveness of the information needs model has been evaluated in an empirical study. Software managers who received an experiment report that followed our information needs model could judge a technology's appropriateness significantly better than those who read a report about the same experiment that did not explicitly address their information needs. We conclude from our research that results from experiments can be considered as a relevant source of information for decision-making when selecting software engineering technologies if certain kinds of information are provided. Our research has shown that especially information regarding the technology, the context in which it is supposed to work, and most importantly, the impact of the technology on development costs and schedule as well as on product quality is relevant for decision makers when selecting software engineering technologies.