Bellwethers: A Baseline Method For Transfer Learning

In IEEE Transactions on Software Engineering

Recommended citation: Krishna, R. & Menzies, T. "Bellwethers: A Baseline Method For Transfer Learning". In IEEE Transactions on Software Engineering, 2018.


Software analytics builds quality prediction models for software projects. Experience shows that (a) the more projects studied, the more varied are the conclusions; and (b) project managers lose faith in the results of software analytics if those results keep changing To reduce this conclusion instability, we propose “bellwethers”: given N projects from a community the bellwether is the project whose data yields best predictions on all others. The bellwethers offer a way to tame conclusion stability because conclusions about a community are stable as long as this bellwether continues as the best oracle. Bellwethers are also simple to discover (just wrap a for-loop around standard data miners). When compared to other transfer learning methods (TCA+, transfer Naive Bayes, value cognitive boosting), using just the bellwether data to construct a simple transfer learner yields comparable predictions. Further, bellwethers appear in many SE tasks such as defect prediction, effort estimation, bad smell detection and predicting issue lifetime. We hence recommend using bellwethers as a baseline method for transfer learning against which future work should be compared.


  author    = {Rahul Krishna and
               Tim Menzies},
  title     = {Bellwethers: A Baseline Method For Transfer Learning},
  journal   = {CoRR},
  volume    = {abs/1703.06218},
  year      = {2017},
  url       = {},
  archivePrefix = {arXiv},
  eprint    = {1703.06218},
  timestamp = {Wed, 07 Jun 2017 14:42:11 +0200},
  biburl    = {},
  bibsource = {dblp computer science bibliography,}