Fraud Detection Based-on Data Mining
on Indonesian e-Procurement System (SPSE)
Hasan Asy'ari Arief, G.A. Putri Saptawati, Yudistira Dwi Wardhana Asnar
School of Electrical Engineering and Informatics
Bandung Institute of Technology
Bandung, Indonesia
hasan.asyari@indonesiandeveloper.com, putri@informatika.org, yudis@informatika.org
Abstract - This paper focuses on detection of potential fraud
that occurs in the procurement process via the Indonesian E­
Procurement System (SPSE). Potential frauds in procurement
take very diverse forms such as corruption, collusion and tender
fixation and more importantly, they are found in various stages
ranging from the budgeting to the utilization stages.
By analyzing the data, we show that there are several techniques
that may work effectively to serve the goal of detecting these
frauds. Furthermore, we also obtain that SPSE data contain a
huge number of data-related issues such as inconsistencies in
hierarchical structure among official agencies and company
names among institutions as well as missing data points. In
addition to that, the size of data to deal with is gigantic (about
515,069 projects), which renders the fraud detection mechanism
has been a non-trivial problem.
In this paper, we implement a fraud detection mechanism using
data mining techniques based on supervised learning. The use of
supervised learning depends on the availability of data labeling
(fraud and non-fraud) which are extracted using string matching
and manual extraction procedure from several data sources
including court rulings, Komisi Pemberantasan Korupsi (KPK)
publication and public comment. Test results show that Naive
Bayes algorithm with 14 attributes obtained from dimension
reduction produces the best performance with promising result
compared to the other fraud detection techniques available to
date. Besides, the sensitivity analysis deployed in the dimension
reduction process has not only significantly reduced
dimension of the problem but also has improved the performance
of the fraud detection technique.
Keywords - fraud detection, e-procurement, data labeling,
data mining, dimensionality reduction
I.INTRODUCTION
Electronic Procurement System (SPSE) is a system that
manages almost all procurement projects undertaken by the
Government of the Republic of Indonesia. Until the beginning
of 2016, the total budget that has been processed using SPSE
reached IDR 870 trillion from 731 agencies and 34
provinces [1]. The procurement processes that occur in the
SPSE are divided into two major selection phases i.e.
preparation stage and implementation stage. At the stage of the
selection preparations, committing officer called PPK submits a
procurement plan to Pokja ULP - a unit assigned to handle
technical procurement processes. Pokja ULP then creates and
determines procurement document. During the implementation
978-1-5090-5671-2/16/$31.00 ©2016 IEEE
phase, this unit creates procurement package, announces and
opens the procurement process, processes the selections
procedure and announces the winner. After the announcement,
it deals with the complaints and then finally closes the
procurement process with contract signing [2].
According to Journal Senarai LKPP, the potential for fraud
in the procurement process may occur during the planning
stages (such as budget markup and packaging work) to the
utilization stage (quantity and quality of procured goods or
services that do not correspond with the requirements or not
used at all) [3]. Potential frauds in the procurement system take
a very diverse forms starting from collusion, corruption and
tender fixation [4]. Various studies have been conducted to
detect potential frauds in the procurement system, such as:
fraud detection techniques based on statistical analysis using
bidding price [5] [6] and reserve price ratio [4] [7], fraud
detection techniques based on proximity and similarity using
spatial data and document of the payment transactions [8] [5]
and fraud detection technique using data mining approach [9]
[J 0] [11]. Based on our research, there are two fraud detection
techniques that have been used to detect fraudulent in SPSE,
i.e: fraud detection techniques using the history of user activity
[J 2] and potential risk analysis method from Opentender.net
[13].
One of the issues in conducting fraud detection in SPSE is
the limited data types provided by the SPSE itself This has
the
substantially makes many existing fraud detection techniques
cannot be deployed to its full potential. The other problems that
have been encountered from SPSE data are such as: (1)
existing missing value in SPSE data, (2) inconsistent
subdivision names among government institutions, (3)
inconsistent names among the same companies across SPSE
provider, (4) unavailability of labeled data and (5) occurrence
of skewed distribution of labeled data. Therefore, to detect
potential fraud on SPSE based on data mining, an extensive
number of data processes preparation have to be performed.
We summarize our implementation for fraud detection
process in six steps as follows: (1) selecting of the appropriate
supervised algorithm based on data mining, (2) determining a
suitable performance measurement techniques, (3) defining and
searching the baseline of fraud detection performance, (4)
enriching attributes and building the model, (5) selecting the
most influential attribute (using dimensional reduction
techniques) and (6) analyzing the results.
 Our analysis concludes that data mmmg technique
conducted in this research is very promising on detecting
fraudulent data in SPSE. The implemented model can detect 52
out of 54 projects with salvage value reaches IDR 273 billion.
Moreover, from a variety of supervised algorithms which were
tested, it was found that Naive Bayes followed by sensitivity
analysis to reduce the dimension shows the highest
performance result.
The rest of this paper is organized as follows. Section 2
discusses about related works for fraud detection that has been
done for SPSE data. Section 3 discusses the steps we take to
prepare the data, and followed by Section 4 with extensive data
analysis discussion. Section 5 elaborates the six steps of data
mining implementation for discovering potential fraud on
SPSE that we have briefly mentioned earlier. Section 6 will
highlight concluding remarks on our study and finally section 7
provides insights regarding some possible future works.
II.RELATED WORKS
There are two known approaches for fraud detection on
Indonesian e-Procurement Data: potential risk analysis from
Openteder.net [13], and user behaviour analysis from BPKP
including committing institutions, bidding projects, bidding
winners, bidding participants and bidding schedules. These
data are then pre-processed, resulted in 515,609 projects until
April 2016 with 640 engaging institutions from a list of
155,328 institutions in the database.
2) Data Extraction (based on existing fraud detection
technique)
The next step is to add additional attributes obtained from
all other existing fraud detection techniques that are already
available from SPSE data. These attributes are summarized in
Table I.
3) Data Extraction Based on Bidding Schedules
This step is performed by rotating or pivoting the schedules
of bidding stages, followed by analyzing the impact of each
stage.
TABLE L ATTRlBUTES FROM V ARlOUS FRAUD DETECTION TECHNIQUES
Code Corresponding Techniqnes Attribntes
Reserve price ratio analysis [4]
al
[7]

[12]. Potential risk analysis's technique uses a matrix of criteria skewness_population

a2 Bid price analysis [5] [6] std�riceJasio
that determines the value of the risk of a project. This technique
is used in opentender.net based on five criteria analysis: the
experience_count
value of the contract, the number of bidders, the reserve price a3 Project Cost Analysis [14] p_distance
ratio, the schedule of project and the number of repeated
winners [14]. The five criteria are then grouped according to nominal_QUARTER
the project risk value for each criterion. Following that, the risk nominaljumlah_peserta_
values of all criteria are then accumulated. The project with the menawar
highest accumulated value is the one that has the greatest nominal_nilai_proyek
Potential Risk Analysis from
a4 nominalJatio_hps_dan_ha
potential of containing fraud. Opentender.net [13]
rga
The other known approach is fraud detection based on user nominal win count
behaviour. Arumsari from BPKP wrote [I2] there are six opentender
additional steps to examine the existence of fraud on
procurement project, those steps are (1) examination of
approval procedure on an auction, (2) validation of the time­ 4) Additional Statistical Data Extractionfrom SPSE
frame validity for the use of user id, (3) verification of ULP This step is performed to obtain new attributes based on
access control, (4) investigation for possible collusion between statistical analysis on SPSE data as well as the changes they
owner/ULP with service provider, (5) investigation for possible undergo. From this step, we obtain 18 new attributes.
collusion between service providers, and (6) inquisition of the
possibility of fraud existence for the availability of SPSE From all these processes, we finally obtain 52 attributes that
application. enable us to conduct our analysis. Furthermore, we also
highlight several issues regarding the data and summarize them
IILDATA PREPARATION as follows.
Data preparation is used to prepare SPSE data in order to be • Bidding participant data include many inconsistent
ready for mining purposes. This process is divided into three values mostly because they are extracted from various
stages: (1) data extraction, (2) data cleansing and (3) data sources (not one central data source), even if they refer
labeling. to the same information.
A. Data Extraction • Nomenclatures used in each institution differ from one
SPSE data extraction is carried out through four stages i.e. another.
original SPSE data extraction, data extraction based on existing
fraud detection techniques, data extraction based on bidding • Bidding projects include a huge number of missing
schedules and additional statistical data extraction from SPSE. values.

1) Original SPSE Data Extraction • Bidding schedules and stages may differ for each
This step is carried out by downloading all data and project.
attributes that are freely accessible through SPSE official
website. From this step, we have obtained five types of data
B. Data Cleansing TABLE IL DATA ANALYSIS RESULTS FOR COMPATIBILITY BETWEEN
THE EXISTING OF FRAUD DETECTION TECHNIQUES AND THE AVAILlBILlTY OF
Data cleansing is conducted to complete the missing values SPSE DATA
in our data.
Fraud Detection Data Data
1) Completing Missing Value Techni9ne Re9nirement Availabili�
To deal with the missing values, we synchronize the data Bid price analysis • The bid price of all • The bid price of
participants of the all participants
with other equivalent attributes. For example for missing [5) [6} auction of the auction
values in budgeting year, we use the year when the project
started while for missing values in institutions and funding Reserve price ratio • Win bid price • Win bid price
2 • Reserve price • Reserve price
source, we use the name of institutions carrying out the analysis [4] [7]
projects. With this process, we are able to complete the missing
• Company's address • Company's
values, which in tum enable us to implement fraud detection address
• Project location
techniques. • The entire value • Project location
Project Cost companies in the • The entire value
2) Hierarchy Extraction 3 procurement system companies in the
Hierarchy extraction is conducted by matching the agency Analysis [l4]
• Company workers procurement
and units names with all ministries and government agencies data system
based on lexical similarity. Assuming that all corresponding • The workload of the
agencies and units are operating under one or more appropriate project company
government units, all values sharing lexical similarity are then • All bidders data with • All bidders data
Pattern Rotation
4 consistent naming but inconsistent
mapped under the same institution. As a result, all agencies and
Analysis [18] naming
units found in the data are placed in a government-based­
hierarchy which suits our technique.
• The address of • The address of
C. Data Labeling company participants company
and
This step is implemented to label the e-Procurement data as auctions organizers
fraud or non-fraud using information extraction from court Fraud Detection by • Home address of
5 participants and
decision data, [(PK publication data and public comment data. Similarity [8]
auctions organizers
We scan all those data sources to find any information whether
• payments Document
a particular project contains fraud or not. In order to make sure for participants
the fraud projects are correctly labeled, the labeling process • Document validation
required the following data: project name, agencies to which for auctions organizers
the project is offered, companies winning the project and the • The value of the • The value of the
fiscal year of the project or otherwise will not be processed. contract contract
Information extraction for the data sources above is performed • The number of bidders • The number of
Potential Risk • The reserve price ratio bidders
using string matching based on several criteria and manual
6 Analysis/rom • Scheduling of project • The reserve price
extraction technique.
• The number of ratio
Opentender.net [13] Scheduling of
The results of this information extraction process repeated winners •

project
discovered 202 fraud projects and 19,653 non-fraud projects.
• The number of
Non-fraud labeling is performed by eliminating all projects repeated winners
having any indication as fraudulent. This guarantees that all Data mining techniques
successfully labeled data not mislabeled and clear from any
information noise. • All bidders data with • All bidders data
Association Rule
consistent naming but inconsistent
[9] naming
IV.DATA ANALYSIS

Data analysis is implemented to understand data • Data labeling • Win price and
requirement for the existing fraud detection techniques and its • Win price and reserve reserve price
compatibility with our available data. Table II consists the price ratio ratio
Cluster Analysis"
summary of data compatibility from each existing fraud 2 • Skewness distribution • Skewness
detection technique and this has enabled us to conclude that the [4] [9] from bid prices distribution from
• Median value from bid bid prices
existing techniques suit the characteristic of the SPSE data are
prices • Median value
bid price analysis [5J [6J, reserve price ratio analysis [4] [7] from bid Erices
and potential risk analysis from Opentender.net [13]. Supervised • Data labeling
Furthermore, the data mining techniques that could potentially
3 Learning [/0] [/6]
be used is supervised learning [IO] [I6] [17].
[/7]

3. In this paper cluster analysis was not implemented because the attributes that must be analyzed
are very diverse, hence it needs equalization for each attribute before the process can be carried out In
addition to this diversity, there are a large enough number of attributes that complicate the
implementation of this particular analysis. Please note that cluster analysis is only deployed on similar
attributes. Moreover, the enormous efforts needed to define cluster results as the occurrence of
fraudulent activities makes cluster analysis not practical to implement
 V.FRAUD DETECTION ON SPSE
We then implement fraud detection techniques using data
mining supervised learning. This discussion is arranged into
several steps.
Step 1: The selection of supervised algorithms that suit the
procurement data is analyzed in this section. From the results
of analysis, we notice that several effective algorithms to detect
fraud using supervised learning [10] can be deployed to
analyze SPSE data, i.e. Naive Bayes, Bayesian networks,
decision tree and neural network.
Step 2: In this step we analyze the proper performance
measurement techniques for supervised learning in fraud
detection. Our research has concluded that there are two proper
parameters to determine the performance of fraud
detection techniques: SAR (Squared Error, Accuracy ROC
Curve) and misclassification cost called Cost Model. SAR is
suitable for justifying the overall condition of supplied data
mining model [19] while cost model is suitable because it is
capable of measuring the saving value and misclassification
cost from implementation of fraud detection technique on
supervised model [17].
TABLE IlL COST MATRIX FOR FRAUD DETECTION IN E-PROCUREMENT
SYSTEM
NonFrand (Trne) Frand (Trne)
Prediction(NonFrand) 0 - (Saving Values)
Saving Values -
Prediction(Fraud) - (Investigation Cost)
( Investigation Cost )
Cost models are formed from cost matrix by measuring
misclassification of a detection process [17]. Cost matrix used
in this paper is shown in Table III. Misclassification costs are
calculated based on total investigation cost and the salvage
value of the bidding price that occurs in a project. Cost matrix
in Table III consists of four conditions established based on the
predicted results and the saving from each quadrant. Cost
modeling. All attributes are then used to build the model which
then followed by performance testing. The result of
performance testing is exhibited in Table IV. Table IV shows
that the average performance of all these attributes is 82.88%
with saving value of 75.73% or IDR 263 billion. The
performance increases significantly compared to the
performance of the test results using a combination of groups
attributes in Step 3.
TABLE IY. TESTS RESULTS USING ALL AV ALAIBLE ATTRlBUTES
Attributes Algorithm SAR(%) COST (%) Avg. (%)
all" Naive Bayes 90.0 75.73 82.88
2 all" Bayes Network 90.3 74.68 82.53
3 all" Decision Tree 70.1 -23.66 23.25
nTests are performed using all available attributes
Step 5: Dimensional reduction is implemented to select and
reduce the attributes that can provide the best performance.
From the available literature, we note that there are several
analytical techniques for dimensional reduction that can be
carried out such as Correlation Analysis (CA), Principal
Component Analysis (PCA) and Weighted Principal
Component Analysis (WPCA) [20]. We combine these three
techniques with sensitivity analysis (SA) to analyze how much
an attribute changing affects the misclassification cost. The
baseline used for SA is the best performance that is generated
from the previous step, which is 82.88%. The SA steps are
implemented by comparing the performance of the model
generated with and without each attribute to the baseline value.
If the result is smaller than one percent from the baseline, then
the attribute is removed because it is considered insignificant.
These steps are repeated for all 52 attributes and yields to 14
significant attributes for our fraud detection model. The test
result of dimensional reduction is depicted in Table V.
TABLE Y. TEST RESULTS FROM DIMENSIONAL REDUCTION PROCESS

matrix is formulated under the assumption that the procurement No RD" l;h Algorithm SAR(%) COST(%) Avg
is only investigated if there are fraud indications from the
WPCA 11 Naive Bayes 85.99 67.67 76.83
procurement data, not to all projects. This assumption
eliminates the cost of unnecessary investigations otherwise the 2 WPCA 11 Bayes Network 83.77 61.19 72A8
investigation costs would be enormous. WPCA 11 Decision Tree 54.63 -114A7 -29.92
3
Step 3: In this step we group attributes based on the fraud 4 PCA 11 Naive Bayes 55.20 -46.29 4A5
detection techniques. From these data it is known that there are
5 PCA 11 Bayes Network 53.17 -54.97 -0.90
four groups of attributes that can be used. All of these groups
are then merged to build the data mining model using 6 PCA 11 Decision Tree 30AO -117.64 -43.62
supervised algorithms from step 1. The goal is to find a group 7 CA 29 Naive Bayes 65.88 -109.30 -21.71
of attributes or combination of groups that have the best
performance in fraud detection using supervised algorithms. 8 CA 29 Bayes Network 58.18 -26.85 15.67

After building and validating the model based on all these 9 CA 29 Decision Tree 27.80 -96.74 -34A7
groups of attributes, as well as their combination, we found that
10 SA 14 Naive Bayes 89.79 78.68 84.24
group (a4) from Potential Risk Analysis from Opentender. net
[13] gives the best performance of 80.19% with saving 11 SA 14 Bayes Network 88.30 68.98 78.64
percentage of 78.70%. 12 SA 14 Decision Tree 82.94 70.07 76.51

Step Having this finding, we implement attribute

4: a Dimensional reduction technique
bThe total attributes from dimensional reduction implementation
enrichment for both statistical calculations and scheduled
auction to improve the performance. From this enrichment
process, we obtained 52 attributes that can be used for the
 Through this experiment, we obtain the most influential between each attributes. Thus in this algorithm, the relationship
attributes generated through sensitivity analysis (SA) combined among attributes are ignored and the analysis focused on the
with Naive Bayes (NB) algorithm. The result of this technique relationship of each attribute to the label class. We obtain
reaches its best performance at 84.24% with saving value of several new insights from this and summarize them as follows.
78.68% or IDR 273 billion. In addition to improving
performance, the SA also reduces attributes from 52 attributes
• Procurements that use national budget have bigger
to 14 attributes. potential to be more fraudulent than the ones using state
budget.
TABLE VI. LISTS OF 14 ATTRIBUTES USING SENSITIVITY ANALYSIS • Procurements with higher price values have potential to
No Attribute Name Description be more fraudulent than the ones with lower price
values.
admin and teknis Contains the number of bidders who
meet the technical and administrative • Procurements with category of "pengadaan barang" are
requirements
more likely to be fraud than other categories.
2 is_apbd Indicates a project sourced trom national • Procurements with a longer period of auctions have
budget or state budget
greater probabilities to be fraud. Furthermore,
procurements with more auction phases are potentially
3 jumlah_peserta_menawar Contains the number of participants that
bid in the auction more fraudulent than procurements with less auction
phases.
4 lama_prosesJelang Contains information on how long the
auction process is underway
• Procurements with more changes on scheduling phase
have the potential to be more fraudulent than the ones
5 Ipse_nama_kota Contains name of the city which hold the with fewer changes on scheduling phase.
auction
• The use of statistical ratio of bid prices and win bid
6 nilai�royek Contains information on the won price price are still quite effective to be used in detecting
of the auction fraudulent data in SPSE.

7 nominal_nilai_proyek Contains ranking / grouping of the range • The analysis of bidders and lists of terms to bid may
of winning price also be used to analyze fraudulent in SPSE.

8 pivot_23_WEEKDAY Contains weekday information of the VI.CONCLUSION

"post-announcement " phase
9 proyek_kategori Contains information category of project
10 proyek_metode_dokumen Contains information of documents used
in the method of collecting the required
documents for the auction
II rasio tawar Contains information of the avarage
ratio from bid prices
12 ratio_hps_dan_harga Contains ratio information between win
bid price and reserve price
13 selisih_harga_terendah Contains information of difference
between win price and lowest bid price
This study has pointed out several important remarks in the
field of data mining, especially in the area of fraud detection
techniques. Our finding has shown that supervised learning is
appropriate to be deployed for fraud detection purpose in SPSE
data. It is also important to note that data labeling is
compulsory for the success of this technique implementation.
Our findings also highlight that naive bayes yields the
highest performance compared with neural network algorithms,
decision tree and Bayesian network and the performance is
substantially enhanced by dimensional reduction using four
methods including Principal Component Analysis (PCA),
Weighted Principal Component Analysis (WPCA), Correlation
Analysis (CA) and Sensitivity Analysis ( SA).

14 total�erubahanjadwal Contains the total change of each of the
stages that exist in an auction process
Step 6: In this step, we analyze the probability distribution
of the model producing the best performance. The purpose of
this analysis is to find out how the value of each attributes
affects the process of fraud detection. Furthermore, we are
interested to discover new knowledge on how to find
fraudulent data in SPSE.
The model to analyze is the one with established 16
attributes resulted from dimensional reduction. This model is
built using naive Bayes algorithm. Naive Bayes algorithm
works with the assumption of conditional independence
Last but not least, we realize that key attributes such as
project values, winning price statistics, bid price and reserve
price are still playing significant role on detecting fraudulent
data, especially in SPSE. It is also interesting to note discover
that procurements funded through national budget have bigger
potential to be more fraudulent than the ones using state
budget.
VII.FuTURE WORKS
The process of data labeling in SPSE which had been
conducted in this study is still dominated by manual labeling
process. This makes our limited resources are not sufficient to
carry out the manual labeling process for all data. We are aware
that named entity recognition techniques can be developed to
perform automate this process and encourage the
implementation of this automatic data labeling. We believe this
will significantly improve the data mining model and produce a
better performance.
REFERENCES
[I] LKPP. (2016, May) Smart Report LPSE V.2. [Online]. http/lreport­
lpse.lkpp. go.id/v2/beranda
[2] LKPP, "Peraturan Kepala Lembaga Kebijakan Pengadaan Barang/Jasa
Pemerintah," Lembaga Kebijakan Pengadaan Barang/Jasa
Pemerintah,Perka No. 1 Tahun 2015,2015.
[18] Kevin Lang and Robert W. Rosenthal, "The Contractors' Game," The
RAND Journal of Economics, pp. 329-338,1991.
[19] Rich Caruana and Alexandru Niculescu-Mizil, "Data Mining in Metric
Space: An Empirical Analysis of Supervised Learning Performance
Criteria," ,New York,2004.
[20] Amruta D. Pawar, Prakash N. Kalavadekar, and Swapnali N. Tambe, "A
Survey on Outlier Detection Techniques for Credit Card Fraud
Detection," IOSR Journal of Computer Engineering, vol. 16, no. 2,
pp. 44-48,2014.

[3] LKPP, "Senarai Pengadaan Barang 1 Jasa Pemerintah," Jurnal LKPP

Senarai,vol. I, 20II.

[4] Sidhartha S. Padhi and Pratap K.J Mohapatra, "Detection of collusion in

government procurement auctions," Journal of Purchasing &
Supply Management,pp. 207-221,20II.

[5] Yuliya Bolotova,John M. Connot, and Douglas I Miller, "The impact of

collusion on price behavior: Empirical results from two recent
cases," International Journal of Industrial Organization, pp. 1290-
1307,2008.

[6] Jeannette Brosig and J. Philipp Reib, "Entry decisions and bidding
behavior in sequential first-price procurement auctions: An
experimental study," Games and Economic Behavior, pp. 50-74,
2007.

[7] Rieko Ishii, "Favor exchange in collusion: Empirical study of repeated

procurement auctions in Japan," International Journal of Industrial
Organization,pp. 137-144,2009.

[8] Stefan Ruping, Natalja Punko, Bjorn Gunter, and Henrik Grosskreutz,
"Procurement Fraud Discovery using Similarity Measure
Learning," TRANSACTIONS ON CASE-BASED REASONING,
2008.

[9] Ghedini Celia Ralha and Vinicius Sannento Carlos Silva, "A multi-agent
data mining system for cartel detection in Brazilian," Expert
Systems with Applications, pp. 11642-11656,2012.

[10] Rekha Bhowmik, "Data Mining Techniques in Fraud Detection," Journal

of Digital Forensics,Security and Law,vol. 3,pp. 35-54,2008.

[11] Philip K. Chan, Wei Fan, Andreas Prodromidis, and Salvatore J Stolfo,
"Distributed Data Mining in Credit Card Fraud Detection," IEEE
Intelligent Systems' Special Issue on Data Mining,pp. 67-74,1999.

[12] Arumsari, Totok P., lswahyudi, Mucharor , and Akib P.. (2014) AUDIT
ATAS PELAKSANAAN LELANG SECARA ELEKTRONIK
DALAM PENGADAAN BARANG DAN JASA PEMERINTAH.
[Online].
http/lwww.bpkp.go.idlinvestigasi/berita/read/13521101AUD1T­
ATAS-PELAKSANAAN-LELANG-SECARA-ELEKTRONIK­
DALAM-PENGADAAN-BARANG-DAN-JASA-
PEMERINTAH.bpkp

[13] ICW. (2014) Metode Analisa Data OpenTender. [Online].

http://www.opentender.netlcontent/metode-analisa-data

[14] Patrick Bajari and Lixin Ye, "Deciding Between Competition and
Collusion," The review of economics and statistics, vol. 85, pp.
971-989,2003.

[15] Rosa M. Abrantes-Metz, Luke M. Froeb, John F. Geweke, and

Christopher T Taylor, "A variance screen for collusion,"
International Journal of Industrial Organization, pp. 467 - 486,
2006.

[16] Sam Maes, Karl Tuyls, Bram Vanshoenwinkel, and Bernard Manderick,
"Credit Card Fraud Dete tion Using Bayesian and Neural
Networks," , 1993.

[17] Clifton Phua, Damminda Alahakoon, and Vincent Lee, "Minority Report
in Fraud Detection: Classification of Skewed Data," ACM
SlGKDD Explorations Newsletter,vol. 6,no. 1,pp. 50-59 ,2004.