“Works for Me! Characterizing Non-Reproducible Bug Reports”, Mona Erfani Joorabchi, Mehdi Mirzaaghaei, and Ali Mesbah.
In Proceedings of the Working Conference on Mining Software Repositories (MSR), 62–71, 2014
[PDF]  [Slides]


Bug repository systems have become an integral component of software development activities. Ideally, each bug report should help developers to find and fix a software fault. However, there is a subset of reported bugs that is not (easily) reproducible, on which developers spend considerable amounts of time and effort. We present an empirical analysis of non- reproducible bug reports to characterize their rate, nature, and root causes. We mine one industrial and five open-source bug repositories, resulting in 32K non-reproducible bug reports. We (1) compare properties of non-reproducible reports with their counterparts such as active time and number of authors, (2) investigate their life-cycle patterns, and (3) examine 120 Fixed non-reproducible reports. In addition, we qualitatively classify a set of randomly selected non-reproducible bug reports (1,643) into six common categories. Our results show that, on average, non-reproducible bug reports pertain to 17% of all bug reports, remain active three months longer than their counterparts, can be mainly (45%) classified as “Interbug Dependencies”, and 66% of Fixed non-reproducible reports were indeed reproduced and fixed.


  author = {Erfani Joorabchi, Mona and Mirzaaghaei, Mehdi and Mesbah, Ali},
  title = {Works For Me! {C}haracterizing Non-Reproducible Bug Reports},
  booktitle = {Proceedings of the Working Conference on Mining Software Repositories (MSR)},
  publisher = {ACM},
  pages = {62--71},
  year = {2014},
  url = {http://salt.ece.ubc.ca/publications/docs/mona-msr14.pdf},
  slides = {40727387}