An Algorithmic Approach to Missing Data Problem in Modeling Human Aspects in Software Development

Calikli, Gul and Bener, Ayse (2013). An Algorithmic Approach to Missing Data Problem in Modeling Human Aspects in Software Development. In: PROMISE '13: 9th International Conference on Predictive Models in Software Engineering, ACM, New York, USA, article no. 10.

DOI: https://doi.org/10.1145/2499393.2499398

Abstract

Background: In our previous research, we built defect prediction models by using confirmation bias metrics. Due to confirmation bias developers tend to perform unit tests to make their programs run rather than breaking their code. This, in turn, leads to an increase in defect density. The performance of prediction model that is built using confirmation bias was as good as the models that were built with static code or churn metrics.

Aims: Collection of confirmation bias metrics may result in partially "missing data" due to developers' tight schedules, evaluation apprehension and lack of motivation as well as staff turnover. In this paper, we employ Expectation-Maximization (EM) algorithm to impute missing confirmation bias data.

Method: We used four datasets from two large-scale companies. For each dataset, we generated all possible missing data configurations and then employed Roweis' EM algorithm to impute missing data. We built defect prediction models using the imputed data. We compared the performances of our proposed models with the ones that used complete data.

Results: In all datasets, when missing data percentage is less than or equal to 50% on average, our proposed model that used imputed data yielded performance results that are comparable with the performance results of the models that used complete data.

Conclusions: We may encounter the "missing data" problem in building defect prediction models. Our results in this study showed that instead of discarding missing or noisy data, in our case confirmation bias metrics, we can use effective techniques such as EM based imputation to overcome this problem.

Viewing alternatives

Metrics

Item Actions

Export

You can export this page using these formats Digital Object Identifier - DOI

About

• Item ORO ID
• 45347
• Item Type
• Conference or Workshop Item
• ISBN
• 1-4503-2016-3, 978-1-4503-2016-0
• Project Funding Details

• Funded Project Name	Project ID	Funding Body
  Discovery Grant	402003-2012	NSERC (Natural Sciences and Engineering Research Council of Canada)

• Keywords
• Algorithms; Human Factors; Measurement
• Academic Unit or School
• Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
  Faculty of Science, Technology, Engineering and Mathematics (STEM)
• Copyright Holders
• © 2013 ACM
• Depositing User
• Gul Calikli