2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE)

Ensembles Based Combined Learning for Improved Software Fault Prediction: A

Comparative Study

Chubato Wondaferaw Yohannese, Tianrui Li, Macmillan Simfukwe, Faisal Khurshid

School of Information Science and Technology
Southwest Jiaotong University
Chengdu 611756, China
freewwin@yahoo.com, trli@swjtu.edu.cn, macsims85@gmail.com, faisalnit@gmail.com

Abstract—Software Fault Prediction (SFP) research has made
enormous endeavor to accurately predict fault proneness of
software modules to maximize precious software test resources,
reduce maintenance cost, help to deliver software products on
time and satisfy customer, which ultimately contribute to
produce quality software products. In this regard, Machine
Learning (ML) has been successfully applied to solve
classification problems for SFP. Moreover, from ML, it has
been observed that Ensemble Learning Algorithms (ELA) are
known to improve the performance of single learning
algorithms. However, neither of ELA alone handles the
challenges created by redundant and irrelevant features and
class imbalance problem in software defect datasets. Therefore,
the objective of this paper is to independently examine and
compare prominent ELA and improves their performance
combined with Feature Selection (FS) and Data Balancing (DB)
techniques to identify more efficient ELA that better predict
the fault proneness of software modules. Accordingly, a new
framework that efficiently handles those challenges in a
combined form is proposed. The experimental results confirm
that the proposed framework has exhibited the robustness of
combined techniques. Particularly the framework has high
performance when using combined bagging ELA with DB on
selected features. Therefore, as shown in this study, ensemble
techniques used for SFP must be carefully examined and
combined with both FS and DB in order to obtain robust
performance.
Keywords-Software Fault Prediction, Ensemble Learning
Algorithms, Feature Selection, Data Balancing.
I. INTRODUCTION
The growing demands of quality software in different
industry have been igniting the Software Fault Prediction
(SFP) research area; thereby the quality can be cautiously
inspected and undertaken before releasing the software. SFP
is targeted to inspect and detect faulty proneness of software
modules and help to focus more on those specific modules
predicted as faulty so as to manage resources efficiently and
reduce the number of faults occurring during operations. In
this regard, statistical and Machine Learning (ML)
techniques have been employed for SFP in most studies [2-
14]. On the other hand, from ML techniques, Ensemble
Learning Algorithms (ELA) have been demonstrated to be
useful in different areas of research [7, 15-18], where all of
them have confirmed that ELA can effectively solve
classification problems with better performance when
compared with an individual classifier. As illustrated in the
literatures [6, 14, 32, 35], is more like, to make wise
decisions, people may counsel many experts in the area and
take consideration of their opinions rather than only depend
on their own decisions. In fault prediction, a predictive
model generated by ML can be considered as an expert.
Therefore, a good approach to make decisions more
accurately is to combine the output of different predictive
models. So that, all can improve or at least equal to the
predictive performance over an individual models [6, 14, 32,
35]. Therefore, in this study, we develop a new framework to
compare eminent ELA, namely, bagging [16, 30, 32, 35] and
AdaBoost.M1 [16, 31, 32, 35] with J48 Decision Tree (DT)
as a base classifier. In addition to that, we use McCabe and
Halstead Static Code Metrics [19, 22] datasets for
experimental analysis.
In ML, the ELA is known to improve the predictive
performance of individual classifiers, but neither of these
ensemble techniques alone solves the data skewness (class
imbalance) problem [16, 26] and the existence of redundant
and irrelevant features, specifically in defect datasets [26].
Thus, to deal with these issues, the ensemble based
combined framework has to be designed specifically.
Therefore, in this study, we combine ELA with Feature
Selection (FS) [9, 12-14, 20, 21] and Data Balancing (DB)
[11, 20, 23-25] techniques. FS is carried out by removing
less important and redundant features, so that only important
features are left for training the predictive models and the
performance of ELA could be improved. Moreover, as
software defect datasets are composed of Not Fault Prone
(NFP) instances with only a small percentage of Fault Prone
(FP) instances, DB is carried out to resolve this skewed
nature of defect datasets, so that building SFP models on
balanced data could improve ELA performance.
Therefore, this paper aims to independently examine and
compare ELA and realize their performance improvement
when combined with FS and DB to identify efficient
techniques that better perform for SFP. Hence, the main
contribution of this study is the empirical analysis of
multiple ELA in combination with FS and DB. Interestingly,
the proposed framework has exhibited the robustness of
combined techniques. Particularly, it has high performance
when combining ensemble techniques with DB on selected
features, which constitutes a primary contribution credited to
this study.

978-1-5386-1829-5/17/$31.00 ©2017 IEEE

 The remainder of this paper is organized as follows:
related works presented in Section II . Section III discusses
details of the proposed framework and the algorithms.
Section IV presents the experimental design. Section V
reports our results and discussion together with a comparison
of ELA when combined with FS as well as both FS and DB
techniques. Section VI discusses threats to validity with
regard to our experimental setups. Section VIII presents final
conclusions based on obtained results and future works.
II. RELATED WORK
This section focuses on the studies that have tried to
address the problems in SFP using ELA.
These studies typically build SFP models to help
software engineers to focus development activities on FP
modules, make better use of test resources and produce
quality software products [9, 26, 28, 29]. Khoshgoftaar et al.
[18] investigated FS, Under-sampling with AdaBoost
algorithm and their main drive was to make a comparison
between repetitive sample FS and individual FS. And they
confirmed that using boosting is effective in improving
classification performance. On the other hand, Shanthini et al.
[15] investigated bagging ensemble with Support Vector
Machine (SVM) as a base learner for SFP. They showed that
the proposed ensemble of SVM is superior to individual
approach for SFP. As noticed, both of these studies don’t
consider employing different ensemble approaches so as to
identify the techniques that are more efficient for SFP.
As discussed earlier, the ELA is known to increase the
performance of individual classifiers, but neither of these
ensemble techniques alone solves the existence of irrelevant
features and class imbalance problem of defect datasets. To
deal with these issues, we have realized that the performance
of ELA can be increased by keeping the quality of software
defect datasets, which can be done by applying either FS
and/or resolving class imbalance problem [26]. For instance,
Shivaji et al. [12] investigated multiple FS using NB and
SVM classifiers. Wang et al. [13] made a comprehensive
empirical study to assess ensembles of feature ranking
Figure 1. Ensembles Based Combined Framework For Improved SFP
methods. They demonstrated that ensembles of few rankers
are effective and even better than ensembles of many or all
rankers. As noticed, these studies don’t compare different
ensemble techniques and perform DB to resolve class
imbalance issues. Therefore, to the best of our knowledge, no
study has made an attempt to make a combined comparison
of ELA with FS and DB (in our case, Information Gain (IG)
and Synthetic Minority Over-sampling Techniques
(SMOTE)) concepts together for SFP.
Having this gap in mind, which has not been addressed
by many studies, we design a new framework that follows
two strategies step by step. Details of our new framework for
improved SFP based on combined ensembles are presented
in Section III.
III. ENSEMBLE BASED COMBINED FRAMEWORK FOR
IMPROVING SOFTWARE FAULT PREDICTION
An ensemble based combined framework for improved
SFP is shown in Figure 1. We follow two strategies to attain
the objective of the study.
In strategy one, we perform FS using IG evaluation
methods based on Ranker search methods for all datasets
used in this study to identify useful features for ensemble
learning. The input of the framework for this strategy is top
ranked features from all datasets. Then we build the SFP
models on the selected features using two prominent ELA
(Bagging and AdaBoost.M1 with J48 DT as the base
classifier). Experiments have carried out by running a 10-
fold cross-validation and each experiment repeated 10 times.
Then the results are captured using AUC, and Accuracy
Performance Evaluation (PE) criteria. As shown in Figure 1,
this strategy serves as to compare combined ELA. Also we
use the efficiently performing techniques for further
comparable reference in the subsequent performance
experiments.
In strategy two, we resolve the data skewness problem
using SMOTE algorithm on the selected features of the
defect datasets. To get reasonably balanced data for
ensemble classification, we make the target ration for NFP
and FP module as recommended to be 65% and 35%, This will be used to compare with the previous performance
respectively by Khoshgoftaar et al. [5]. As shown in Figure 1, experiment.
this strategy serves as a demonstration of performance Finally, we make comparison between the selected
improvement by combining both IG and SMOTE with ELA, combined ELA from both strategy one and strategy two
and as a final point to realize the efficient ensemble classifier. category based on AUC PE criteria to realize performance
improvement and more efficiently performing combined
Overall Algorithm : Experiment al P rocedure
ELA for SFP.
Dat aset s m {D 1, D 2, ..., D n }; / *defect dat aset s*/
FS m IG; / *redunda nt and ir relevant fea t ure removing met hod */
DB m SMOTE; / *Dat a Balancing met hod*/ IV. EXPERIMENTAL DESIGN
ELA m {BaG, AdB}; / *Ensemble Learning Algorit hms*/
R m 10; / *t he number of repeat ing experiment */ During the experiments, in this study, performance
N m 10; / *t he numb er of folds*/
for each D  Dat aset s do evaluation is carried out by running a 10-fold cross-
Pi m C i ,D / D / *probabilit y of arbit rary t uple blongs t o class C i * /
m
validation [7]. First, we rank the attributes using the IG FS
Info(D ) m ¦ pi log2 ( pi ) / * entropy of D * / techniques for each dataset independently. After ranking the
iv 1
InfoA (D ) m ¦ Dj / D u Info(D j ) / * expect ed informat ion required t o
attributes, following the recommendation from literatures
j 1 [13], we select the top [log2 n] attributes (n is the total
classify a t uple based on t he part it ioning by at t ribut e A * /
InfoGain (A ) m Info(D )  InfoA (D ) / *informat ion gained by A * / number of independent features). Then the class attribute is
for each A  D do / * attiribute or feature selection * / included to yield final datasets.
rankOfA m InfoGain (A ) / * t he highest r a nked at t ribut e
A is t he one wit h t he largest informat ion gain */ The performances of each ensemble techniques are
rankOfA '[ ] m {rankOfA 1, rankOfA 2, .... rankOfA n }; evaluated against each other for each dataset. Employed
endfor
for I = 0 t o lengt h-1 do / * sort A based on I nfoG ain(A ) */ algorithms are performed using WEKA version 3.8 [27]. The
highR ank = I; final comparison is based on using the AUC measure.
for J = I + 1 t o lengt h-1 do
if rankOfA '[J ] > rankOfA '[highR ank] Since the proposed ensemble based combined framework
h ighR ank = J ;
endif is implemented step by step following two strategies, and
endfor evaluated using eight publicly available software defect
Temp = rankOfA '[I]; rankOfA '[I] = rankOfA '[highR ank ];
rankOfA '[highR ank ] = Temp; datasets. Moreover, the experimental procedures have
endfor presented in the following pseudo-code to summarize overall
selectedFeatures m top ª¬log2n º¼ + class A; / * select ing t op ra nked feat ures*/
for each t imes  [1, R] do / *R t imes N-fold cross-validat ion*/ combined approach. The selected eight datasets are the latest
D' m selectedFeatures; and cleaned versions from publicly accessible PROMISE
foldDat a m generat e N folds from D';
for each fold m [1, N] do repository of NASA software projects [1]. These datasets are
t est Dat a m fold Dat a[fold]; popular in SFP studies and have been used by many studies
t rainingDat a m D' - t est Dat a;
for each combFSELA m ELA do / *evaluat e combined [29]. Table I summarizes some main characteristics of the
ELA wit h FS*/ selected datasets. All the datasets consist of McCabe and
fault P redict or m combFSELA(t r ain ingDat a);
V m evaluat e combined ELA on t est Dat a; / * obtain Halstead Static Code Metrics. Note that data imbalance is
t ot al vot e received by each class (FP and NFP )*/ consistently observed in all datasets. The value of using
predict orP erformance m choose t he class t hat receives
t he highest t ot al V as t he final classificat ion; static code metrics to build SFP models has been empirically
endfor illustrated by Menzies et al. [10].
endfor
endfor
S mino  D'; / * S mino is a subset of # F P i ns t an ces */ V. ANALYSIS AND DISCUSSIONS
S majo  D'; / * S majo is a subset of # NFP ins t an ces */
Pmajo m recommended/ required percent age of S majo ; This section presents the experimental analysis and
x i  S mino ; / * is the # FP ins t an ce underconsi deration
it s one of the k - nearest neighbors for x i :x i^  S mino */ discussions based on the objective of the study. Performance
k m 5; / *set 5 as k-nearest neighbors for each example x i  S mino * / comparison in terms of AUC and Accuracy among
G  [0, 1]; / * is a uniformal dist ribut ion random var iable */
100 prominent ELA with FS technique (in our case IG) and data
rat e = (( S majo * )  (S majo  S mino )) / S mino * 100; / * det ermine rat e imbalance problem solution (in our case SMOTE) presented
Pmajo
t o synt het ically creat e inst an ces from S mino * / in the following figures and tables. The results are based on
for x i  S mino
x i^ m one of randomly choosed # FP from k - near est neighbors; the performance of ELA combined with IG FS and ELA
x newMino m x i  (x i^  x i ) u G ; / * new # FP ins t an ce creation */ combined with both IG and SMOTE.
endfor
for each t imes  [1, R] do / *R t imes N-fold cross-validat ion*/
balancedData m D'  x newMino ; / * balanced data on s elected features */ TABLE I. A DESCRIPTION OF DATASETS.
foldDat a m generat e N folds from balancedData ;
for each fold m [1, N] do Dataset #Attr. #Ins. #NFP #FP %NFP %FP
t est Dat a m fold Dat a[fold]; ar1 29 121 112 9 92.56% 7.44%
t rainingDat a m balancedData - t est Dat a;
for each combDBFSELA m ELA do / *evaluat e combined ELA ar4 29 107 87 20 81.31% 18.69%
wit h DB and FS */ JM1' 21 9593 7834 1759 81.66% 18.34%
fault P redict or m combDBFSELA(t rainingDat a); KC2 21 522 415 107 79.50% 20.50%
V m evaluat e combined ELA on t est Dat a; / * obt ain t ot al vot e MC1'' 38 1988 1942 46 97.69% 2.31%
received by each class (FP and NFP )*/ MW1' 37 264 237 27 89.77% 10.23%
predict orP erformance m choose t he class t hat receives t he
highest t ot al V as t he fin al classificat ion; PC3' 37 1125 985 140 87.56% 12.44%
endfor PC4'' 37 1287 1110 177 86.25% 13.75%
endfor
endfor
endfor
 (a) (b)
Figure 2. Comparison of ELA Combined with IG FS using Accuracy, and AUC

TABLE II. CLASSIFICATION RESULTS OF ELA COMBINED WITH IG A. Comparison: ELA Performance Combined with IG
IGDTBagging IGDTAdaBoost.M1
Performance comparison of bagging and AdaBoost.M1
Dataset
Accuracy AUC Accuracy AUC using IG FS are given in Figure 2 (a) and (b) and Table II. In
JM1' 81.766 0.72 80.568 0.696 terms of both indexes used in this study AdaBoost.M1
MC1" 97.712 0.8 98.305 0.821 appears to perform low and bagging demonstrates the
MW1' 88.799 0.678 86.373 0.669 highest values.
PC3' 86.587 0.803 85.024 0.777
PC4" 88.874 0.908 87.887 0.894
Thus, the result reflects the better performance of
ar1 90.404 0.755 87.929 0.744 bagging over AdaBoost.M1. Except that out of eight datasets,
ar4 84.545 0.833 80.709 0.794 in MC1” and KC2, AdaBoost.M1 shows better performance
KC2 81.934 0.833 82.718 0.801 in accuracy as well as in AUC measure using MC1” dataset.
Average 87.58 0.791 86.19 0.775
However, considering the average performance of all
TABLE III. CLASSIFICATION RESULTS OF ELA COMBINED WITH BOTH
datasets, bagging still outperforms AdaBoost.M1.
IG AND SMOTE
B. Comparison: ELA Performance Combined with IG and
Dataset
SMOTEIGDTBagging SMOTEIGDTAdaBoost.M1 SMOTE
Accuracy AUC Accuracy AUC
JM1' 80.773 0.855 78.926 0.835
In Figure 3 (a) and (b) and Table III, the performance
MC1" 96.596 0.988 97.303 0.992 comparison of bagging and AdaBoost.M1 combined with
MW1' 84.966 0.916 85.595 0.907 both IG and SMOTE are given. In terms of both indexes,
PC3' 83.266 0.905 83.834 0.911 AdaBoost.M1 appears to perform low and bagging
PC4" 90.158 0.962 90.439 0.963
ar1 82.092 0.901 81.931 0.896
demonstrates the highest values. However, for MC1"(97.303,
ar4 77.401 0.854 76.923 0.846 0.992), MW1' (85.595), PC3' (83.834, 0.911), and PC4"
KC2 80.39 0.871 79.078 0.836 (90.439, 0.963) datasets, AdaBoost.M1 outperforms bagging
Average 84.46 0.907 84.25 0.898 ensemble learning in both accuracy and AUC (except MW1')
measures. Nevertheless, considering the average

(a) (b)
Figure 3. Comparison of ELA Combined with both IG and SMOTE using Accuracy and AUC
performance of all datasets, bagging still outperforms
AdaBoost.M1. Thus, the results reflect the better
performance of combined bagging but closely followed by
combined AdaBoost.M1 on software defect datasets. On the
other hand, based on this performance results, we can say
that, after resolving class imbalance problem with some
datasets, AdaBoost.M1 competitively shows good
performance, which clearly needs further investigation with
more datasets from another software metrics.
C. Comparison: IGDTBagging with SMOTEIGDTBagging
As expected, selecting useful features and resolving class
imbalance problem has been proved to be useful and
improve ELA performance. In this regard, based on our
proposed framework, Sections V (A) and (B) experimental
results show achieved performance improvements. And the
more efficiently performed ELA in average is found to be
combined bagging in both strategy one and two. Therefore,
this section points out the performance improvement
achieved through combined bagging ELA when combined
with IG as well as combined with both IG and SMOTE using
AUC PE. Accordingly, as shown in Figure 4, the
performance of combined bagging algorithm gives better
results in all datasets when combining with both IG and
SMOTE than combining only with IG. These affirms the
contribution of combined preprocessing as removing
irrelevant and redundant features as well as resolving class
imbalance problem and its power to improve the
performance of ELA.
Figure 4. Comparison of Bagging ELA Combined with IG and both IG
and SMOTE using AUC
VI. THREAT TO VALIDITY
There are threats that may have an effect on our
experimental results. The proposed prediction models were
created without changing the parameter setting except that
DT algorithm is used with ensembles techniques, which was
not default in both cases. Thus, investigations were not made
by changing the default parameters setting to see how the
variation affects the model performance. In addition to that,
as many software metrics are defined in literature, different
software metrics might be better indicator to defectiveness of
modules. However, we used static code software metrics
which were available in selected datasets. Conclusions were
drown based on the important features selected using IG. In
terms of the total instances and number of classes, the
datasets may not be good representatives. However, this
practice is common among the fault prediction research area.
VII. CONCLUSION AND FUTURE WORKS
This study made empirical evaluation of the capability of
ELA in predicting FP software modules and compared their
performance combined with FS and both FS and DB
techniques using eight NASA software defect datasets. Our
objective of using FS and DB was that, by combining those
filtering techniques with ELA, we would be able to prune
non-relevant features and balance classes, and then learn an
ELA that performs better than from learning on the whole
feature set and in imbalanced classes. Accordingly, the
experimental results reveal our combined technique assures
the performance improvement. Thus, dealing with the
challenges of SFP mentioned in this study, our proposed
framework confirms remarkable classification performance
and lays the pathway to software quality assurance.
As the future work, we plan to explore more ELA
including vote and stacking, and more data preprocessing
techniques with more defect datasets which consist of
different software metrics; and to realize how the proposed
framework helps to identify the more efficient combined
ensemble techniques and improve its classification
performance to accurately predict FP software modules.
ACKNOWLEDGEMENT
This work is supported by the Fundamental Research
Funds for the Central Universities (No. 2682015QM02).
REFERENCES
[1] T. Menzies, R. Krishna, and D. Pryor. (2016). The Promise
Repository of Empirical Software Engineering Data. Available:
http://openscience.us/repo
[2] E. Arisholm, L. C. Briand, and E. B. Johannessen, "A systematic and
comprehensive investigation of methods to build and evaluate fault
prediction models," Journal of Systems and Software, vol. 83, pp. 2–
17, 2010.
[3] K. O. Elish and M. O. Elish, "Predicting defect-prone software
modules using support vector machines," Journal of Systems and
Software, vol. 81, pp. 649–660, 2008.
[4] I. Gondra, "Applying machine learning to software fault-proneness
prediction," Journal of Systems and Software, vol. 81, pp. 186–195,
2008.
[5] T. M. Khoshgoftaar, C. Seiffert, J. V. Hulse, A. Napolitano, and A.
Folleco, "Learning with limited minority class data," in the Sixth
International Conference on Machine Learning and Applications,
Cincinnati, OH, 2007.
[6] J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and
Techniques: Morgan Kaufmann Publishers Inc., 2011.
[7] R. Kohavi, "A study of cross-validation and bootstrap for accuracy
estimation and model selection," in the International Joint Conference
on Artificial Intelligence, 1995.
[8] I. H. Laradji, M. Alshayeb, and L. Ghouti, "Software defect
prediction using ensemble learning on selected features," Information
and Software Technology, vol. 58, pp. 388–402, 2015.
[9] R. Malhotra, "A systematic review of machine learning techniques for
software fault prediction," Applied Soft Computing, vol. 27, pp. 504-
518, 2015.
[10] T. Menzies, J. Greenwald, and A. Frank, "Data mining static code
attributes to learn defect predictors," IEEE Transactions on Software
Engineering, vol. 33, pp. 2–13, 2007.
[11] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, [33] F. Provost and T. Fawcett, "Robust classification for imprecise
"SMOTE: Synthetic minority over-sampling technique," Journal of environments," Machine Learning, vol. 42 pp. 203–231, 2001.
Artificial Intelligence Research, vol. 16, pp. 321-357, 2002. [34] C. Catal, "Performance evaluation metrics for software fault
[12] S. Shivaji, E. J. Whitehead, R. Akella, and S. Kim, "Reducing prediction studies," Acta Polytechnica Hungarica, vol. 9, pp. 193–206,
features to improve code change-based bug prediction," IEEE 2012.
Transactions on Software Engineering, vol. 39, pp. 552–569, 2013. [35] R. Polikar, "Ensemble based systems in decision making," IEEE
[13] H. Wang, T. M. Khoshgoftaar, and A. Napolitano, "A comparative Circuits and Systems Magazine, vol. 6, pp. 21-45, 2006.
study of ensemble feature selection techniques for software defect
prediction," in the Ninth International Conference on Machine
Learning and Applications, IEEE, Washington, DC, 2010.
[14] E. Frank, M. A. Hall, and I. H. Witten, The WEKA Workbench.
Online Appendix for "Data Mining: Practical Machine Learning
Tools and Techniques," 4th ed.: Morgan Kaufmann, 2016.
[15] A. Shanthini and R. M. Chandrasekaran, "Analyzing the effect of
bagged ensemble approach for software fault prediction in class level
and package level metrics," in the IEEE International Conference on
Information Communication and Embedded Systems (ICICES), India,
2014.
[16] Mikel Galar, Alberto Fernandez, Edurne Barrenechea, Humberto
Bustince, and F. Herrera, "A Review on Ensembles for the Class
Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based
Approaches," IEEE Transactions on Systems, Man, and Cybernetics,
Part C (Applications and Reviews), vol. 42, pp. 463 - 484, 2012.
[17] [17] S.K. Mathanker, P.R. Weckler, T.J. Bowser, N. Wang, and N. O.
Maness, "AdaBoost classifiers for pecan defect classification,"
Computers and Electronics in Agriculture, vol. 77, pp. 60–68, 2011.
[18] Taghi M. Khoshgoftaar, Kehan Gao, and A. Napolitano, "Improving
software quality estimation by combining feature selection strategies
with sampled ensemble learning," in the IEEE 15th International
Conference on Information Reuse and Integration (IRI), San
Francisco, California, USA, 2014.
[19] D. Radjenovic, M. Hericko, R. Torkar, and A. Zivkovic, "Software
fault prediction metrics: A systematic literature review," Journal of
Information and Software Technology, vol. 55, pp. 1397–1418, 2013.
[20] H. Liu, H. Motoda, and L. Yu, "A selective sampling approach to
active feature selection," Artificial Intelligence, vol. 159, pp. 49–74,
2004.
[21] S. Liu, X. Chen, W. Liu, J. Chen, Q. Gu, and D. Chen, "FECAR: A
feature selection framework for software defect prediction," in the
38th Annual International Computers, Software and Applications
Conference, Vasteras, 2014.
[22] T. J. McCabe, "A complexity measure,," IEEE Transactions on
Software Engineering, vol. SE-2, pp. 308–320, 1976.
[23] V. García, J. S. Sánchez, and R. A. Mollineda, "On the effectiveness
of preprocessing methods when dealing with different levels of class
imbalance," Knowledge-Based Systems, vol. 25, pp. 13-21, 2012.
[24] H. He and E. A. Garcia, "Learning from Imbalanced Data," IEEE
Transactions on Knowledge and Data Engineering, vol. 21, pp. 1263-
1284, 2009
[25] P. Sarakit, T. Theeramunkong, and C. Haruechaiyasak, "Improving
emotion classification in imbalanced YouTube dataset using SMOTE
algorithm," in the 2nd International Conference on Advanced
Informatics: Concepts, Theory and Applications, Chonburi, 2015.
[26] W. Y. Chubato and T. Li, "A Combined-Learning Based Framework
for Improved Software Fault Prediction," International Journal of
Computational Intelligence Systems, vol. 10, pp. 647–662, 2017.
[27] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H.
Witten, "The WEKA data mining software: an update; SIGKDD
Explorations," Retrieved 01 Sep. 2017.
[28] C. Catal, "Software fault prediction: A literature review and current
trends," Expert Systems with Applications, vol. 38, pp. 4626–4636,
2011.
[29] T. Hall, S. Beecham, D. Bowes, D. Gray, and S. Counsell, "A
systematic literature review on fault prediction performance in
software engineering," IEEE Transactions on Software Engineering,
vol. 38, pp. 1276–1304, 2012.
[30] L. Breiman, "Bagging predictors," Machine Learning, vol. 24, pp.
123-140, 1996.
[31] Yoav Freund and R. E. Schapire, "Experiments with a new boosting
algorithm," in Thirteenth International Conference on Machine
Learning, San Francisco, 1996, pp. 148-156.
[32] Polikar R., “Ensemble Learning,” Scholarpedia, 2009.