Expert SystemsWith Applications, Vol. 9, No. 4, pp.

503-512, 1995
Copyright © 1995 Elsevier Science Ltd
Pergamon Printed in the USA. All fights reserved
0957-4174/95 $9.50 + .IX)
0957-4174(95)00020-8

Effectiveness of Neural Network Types for Prediction of

Business Failure

J, EFR~M BORITZ AND DUANE B KENNEDY

School of Accountancy, University of Waterloo, Waterloo, Ontario, Canada N2L 3G 1

Abstract--The study examines the effectiveness of different neural networks in predicting bankruptcy
filing. Two approaches for training neural networks, Back-Propagation and Optimal Estimation Theory,
are considered. Within the back-propagation training method, four different models (Back-Propagation,
Functional Link Back-Propagation With Sines, Pruned Back-Propagation, and Cumulative Predictive
Back-Propagation) are tested. The neural networks are compared against traditional bankruptcy
prediction techniques such as discriminant analysis, logit, and probit. The results show that the level of
Type I and Type H errors varies greatly across techniques. The Optimal Estimation Theory neural
network has the lowest level of Type 1 error and the highest level of Type H error while the traditional
statistical techniques have the reverse relationship (i.e., high Type I error and low Type H error). The
back-propagation neural networks have intermediate levels of Type I and Type H error. We demonstrate
that the performance of the neural networks tested is sensitive to the choice of variables selected and that
the networks cannot be relied upon to "sift through" variables and focus on the most important variables
(network performance based on the combined set of Ohlson and Altman data was frequently worse than
their performance with one of the subsets). It is also important to note that the results are quite sensitive
to sampling error. The significant variations across replications for some of the models indicate the
sensitivity of the models to variations in the data.

1. I N T R O D U C T I O N of neural networks to prediction of business failure are

constrained by the choices of input variables, models and
THE ABILrrV TO accurately predict business failure is a
model structure. Like other techniques, neural network
concern to anyone who relies on a particular business for
approaches can be significantly influenced by such
income, either employment or investment, goods, or
choices. For example, in a previous study (Boritz,
services. In the past several years, a number of artificial
Kennedy, & Albuquerque, 1995) we have demonstrated
neural network models have been developed to predict
the sensitivity of neural network performance (i.e.,
corporate bankruptcy. The studies using neural networks
predictive accuracy) to the choice of input variables and
for bankruptcy prediction have shown both promise
sampling proportions in the training sample.
(Salchenberger, Sinar, & Lash, 1992; Tam & Kiang,
The purpose of this study is to extend our previous
1992; Fanning & Cogger, 1994) and disappointment
work by examining the effect of neural network type on
(Bell, Ribar, & Verchio, 1990).
bankruptcy prediction accuracy. Two approaches for
Most studies in this area have focussed on the
training neural networks, Back-Propagation and Optimal
effectiveness of back propagation networks relative to
Estimation Theory, are considered. Within the back-
benchmarks represented by conventional statistical tech-
propagation training method, four different models
niques such as discriminant analysis, logit and probit.
(Back-Propagation, Functional Link Back-Propagation
However, these studies have not reported the sensitivity
With Sines, Pruned Back-Propagation, and Cumulative
of model accuracy to variations in the variables selected,
Predictive Back-propagation) are tested. The neural
sampling methods, network type, and network archi-
networks are compared against traditional prediction
tecture. Thus, we have a limited understanding about the
techniques such as discriminant analysis, logit, and
degree to which the results of studies of the application
probit which serve as benchmarks for evaluating the
relative effectiveness of the neural network approach.
Requests for reprints should be sent to Duane Kennedy, School of Three sets of variables are considered in order to
Accountancy, Faculty of Arts, University of Waterloo, Waterloo, determine if the results are sensitive to the choice of
Ontario N2L 3G1, Canada. variables. The analysis is restricted to a single neural

503
504
network architecture consisting of one layer of hidden
nodes with the number of hidden nodes set equal to the
number of input variables.
The remainder of this paper is divided into five
sections. The next section contains a general description
of neural networks and then describes the networks
examined in this paper. The following section reviews
previous research on the prediction of bankruptcy filing.
The fourth section describes the research design. The
fifth section presents results of the analysis and the final
section contains concluding comments.
2. N E U R A L N E T W O R K S
Artificial neural networks consist of nodes and weights
for the degree of influence which one node exerts on
other nodes. The nodes are arranged in a pattern with a
series of input nodes forming the base layer, one or more
output nodes at the top layer, and one or more hidden
layers of nodes between the input and output layers; The
input values for a node can be either excitatory, if the
weight is positive, or inhibitory, if the weight is negative.
The output of a node is equal to the weighted sum of its
input values transformed by a transfer function. Transfer
functions can be linear or non-linear. A commonly used
function is the sigmoid function 1/(1 +e-Z), which has
upper and lower limits such that input value changes
outside of these limits do not affect the output of the
node.
Neural network architectures can be either feedback
or feedforward. In a feedforward network, inputs are
only received from previous layers. Hence the informa-
tion always flows in a forward direction, from the input
nodes, through the hidden layers of nodes, to the output
nodes. The back-propagation paradigm, a common
training method for neural networks, is a feedforward
network. In a feedback network, nodes can receive inputs
from nodes in any other layer in the network (including
the same layer); the direction of information flow is not
restricted.
Neural network operation involves two phases, learn-
ing and testing. In the learning phase, the network is
taught to solve certain problems or identify specific
patterns based on given information. Two types of
learning are available for network operation, supervized
and unsupervized learning. In unsupervized learning, the
network is presented with a set of facts, however no
response is known by the network. The network
processes the input data and formulates an algorithm to
classify the presented input. This type of learning can be
viewed as a multivariate technique to explore the
underlying structure of the data. In the supervized
learning mode, the network is presented with a set of
facts along with the correct response. The network uses
the information and correct response to formulate
connection weights for each node in the network. Once
the network can correctly classify an acceptable level of
J. E. Boritz and D. B. Kennedy
the presented facts, it can be used to classify new data.
During the testing phase, the neural network's usefulness
is measured by its ability to correctly classify previously
unseen data.
One o f the most common training methods for neural
networks is the Back-Propagation paradigm. Back-
propagation networks learn by modifying the input
connection weights until the difference (error) between
the output and target is minimized. These networks use
the generalized data rule for propagating errors through
the network. This rule places responsibility for the error
on every node in the network. The weights are first
modified for the output node. Then, the error is
propagated backward to all nodes pointing to the output
units in the preceding layer. These nodes, in turn,
propagate the error backward to nodes that point to them,
and so on, until the input level nodes are reached (Jones
& Hoskins, 1987). The standard back-propagation
method for training neural networks is examined in this
study as well as three variations of the back-propagation
method (which are briefly described in the following
subsections). In addition, we examine a neural network
based on Optimal Estimation Theory. In all cases, the
networks are used to find a relationship between the
input variables and a dichotomous output variable which
has values of 1 for bankrupt companies and 0 for non-
bankrupt companies.
2.1. Functional Link Back-Propagation With Sines
This network uses the standard back-propagation algo-
rithm to adjust weights; however, it introduces additional
nodes in the input layer to help increase the learning rate.
The input variables are transformed by two functions.
Pair-wise interaction terms are added between all input
variables and then the sine function is used to function-
ally expand the data. 1 The overall effect is to map the
input data items into a larger pattern space, thereby
enhancing the representation and increasing the predic-
tion accuracy. This results in a large number of nodes in
the Functional Link Back-Propagation With Sines net-
works; hence, learning by this network takes a great deal
of time. The Functional Link Back-Propagation With
Sines networks typically took approximately twice the
time to train compared to the other back-propagation
networks.
In theory, higher order interaction terms such as second or third
order can be constructed. NeuralWorks Professional H, the software
used to construct the back-propagationneural networks in this study,
automatically constructs the pair-wise interaction terms. The software
also allows for multiple levels of sine functionsfor each input variable;
for example, three levels would consist of the sin(~rXi), sin(2~rXi), and
sin(3~Xi) functions where Xi are the input variables. Pilot testing by
Boucher (1990) found that the predictive accuracy was higher with one
layer of sine transformations compared to the accuracy of a model that
contained only the pair-wise interaction terms.
Neural Networks and Business Failure Prediction
2.2. Pruned Back-Propagation
This network disables all connections that have little
impact on the output of the network (i.e., low weight
values). It is based on a concept introduced by David
Rummelhart of Stanford University, where both the
network complexity and error are minimized as the
network learns. Rummelhart observed that generalization
ability declines with increased complexity of neural
networks. Too many nodes produces a neural network
that memorizes the input data and lacks the ability to
generalize (often referred to as overfitting). As a result,
increases in the network's ability to classify companies
in the learning set are linked with a decline in its ability
to classify companies in the testing set (Salchenberger et
al., 1992).
2.3. Predictive Cumulative Back-Propagation
This is a standard back-propagation network with the
input and output layers using a linear transfer function,
and the hidden layer using a sigrnoid transfer function.
This network uses the cumulative generalized delta rule
to speed up neural network training. The cumulative
generalized delta rule updates the weights after two data
items are presented to the network instead of after each
item. The weights are accumulated in another variable
until both data items are presented, at which time the
actual node weights are updated. This approach can
decrease learning time up to 50%.
2.4. Optimal Estimation Theory Neural Network
Shepanski (1988) introduced the Optimal Estimation
Theory paradigm as an alternative approach to training a
feedforward neural network. It generates a set of least
squares estimators for the interconnection weights allow-
ing the optimal set of weights to be determined in a
single pass rather than requiring the training data set be
presented to the network many times as in back-
propagation. One advantage of the Optimal Estimation
Theory approach is significant reductions in the time
required to train a neural network compared to the back-
propagation paradigm. A limitation of the basic Optimal
Estimation Theory method is the necessity of having the
same number of neurons in the hidden and input layers.
To overcome this limitation, the model can be modified
by using a nonlinear transformation of the input variables
(Pao, 1989). The enhanced variables are constructed by
finding the k least correlated pairs of variables, taking the
product of one variable (in the pair) by the cosine of the
other variable, and using these products as additional
inputs to the neural network. Multiple values for k are
considered in our analysis.
3. PREVIOUS RESEARCH
Bankruptcy prediction, through classification of known
cases and generalization to other cases, has been a
505
subject of study for almost 30 years. The accurate
prediction of bankruptcy is important to investors,
creditors and auditors. An accurate prediction model can
help investors and creditors avoid heavy losses stemming
from surprise bankruptcies and could help alert auditors
about potential going concern problems (Boritz, 1991).
Beaver (1966, 1968) was one of the first researchers
to study the prediction of bankruptcy. He analyzed
several financial statement ratios, one-by-one, to evalu-
ate their predictive ability, then developed cut-off scores
for each ratio to classify companies as failed and non-
failed. For example, if a company had a debt-to-equity
ratio of 1.0 and the cut-off score for debt-to-equity is 2.0,
then the company would be classified as healthy.
Although this method has high predictive power, limita-
tions of this process include the requirement to apply the
classification technique using one ratio at a time and the
difficulty of resolving conflicts when one ratio classifies
the company as healthy while another predicts distress.
Beaver's work was followed by Altman's (1968,
1983) discriminant analysis-based model and Ohlson's
(1980) logit-based model. Zavgren (1983) and Jones
(1987) review relevant bankruptcy studies and analyze
several key techniques, the data collection process, the
relative merits of the statistical approaches to prediction
and consider alternative ways of evaluating the validity
of a prediction.
This study focuses on two major models, representing
different classes of statistical techniques; Altman's
(1968) model based on multiple discriminant analysis
and Ohlson's (1980) model based on the use of logistic
regression. We rely on these models in our choice of
input variables for training and testing the various neural
networks being studied. We also use these models and
their underlying statistical approaches for developing
benchmarks against which we compare the performance
of the neural networks.
3.1. Altman's Model
Altman (1968, 1983) introduced a class of models based
on the discriminant analysis technique to produce a
dichotomous classification of falling/non-failing finns.
This technique involves the application of discriminant
analysis to a representative "learning" set of data to
identify the discriminant function which best classifies
the companies into known categories (e.g., bankrupt and
non-bankrupt). This function is then used to classify
other data. Altman's variables and discriminant function
are shown in Panel A of Table 1. The model's prediction
is based on the value obtained for the Z score. For Z
values greater than 2.675, the company is classified as
"healthy," while the company is classified as bankrupt
for lower Z values.
Using the frequency of Type I and Type II errors as a
basis for evaluation, Airman's model typically has good
predictive power for one year before bankruptcy but the
506 J. E. Boritz and D. B. Kennedy

TABLE 1
Definition of Variables

Panel A: Altman

Variables:
WCTA = working capital/total assets
RETA = retained earnings/total assets
EBITTA = earnings before interest and taxes/total assets
MEDEBT = market value equity/book value of total debt
SALETA = sales/total assets
Model:
Z-- 0.210 WCTA + 0.014RETA + 0.033EBITTA + 0.006MEDEBT
+ 0.999SALETA

Panel B: Ohlson

Variables:
SIZE = log (total assets/GNP price-level index)
TLTA = total liabilities/total assets
WCTA = working capital/total assets
CLCA = current liabilities/current assets
OENEG = 1 if total liabilities>total assets
NITA = net income/total assets
FUTL = funds provided by operations/total liabilities
INTWO = 1 if net income negative for last two years
CHIN = change in net income = (NIt-NIt_l)/(INItl + INIt-ll)
NYSE,AMSE = dummy variables for exchange listing
9 Variable Model:
y = - 1 . 3 2 - 0.407SIZE + 6 . 0 3 T L T A - 1.43WCTA + 0.0757CLCA
- 2 . 3 7 N I T A - 1.83FUTL + 0.2851NTWO- 1.72OEN E G - 0 . 5 2 1 CHIN
11 Variable Model:
y = - 2 . 6 3 - 0 . 2 6 7 S I Z E + 5 . 6 3 T L T A - 1.43WCTA + 0.0585CLCA
- 2 . 3 5 N I T A - 1.99FUTL + 0.3071NTWO- 1.56OENEG
-0.5092CHIN-0.854NYSE-0.0513AMSE

model's predictive ability decreases substantially for
predictions two and three years prior to bankruptcy.
According to Altman (1983), the model's weakness in
predicting bankruptcy in an early stage is caused by a
dramatic deteriorating trend in ratios as bankruptcy
approaches and this deterioration is most serious
between the third and the second years prior to
bankruptcy.
3.2. Ohlson's Model
Ohlson (1980) introduced the logistic regression tech-
nique to estimate the probability of failing/non-failing
for a company. This approach potentially can give a
decision maker a better sense of the distribution of
financial risk than a model based on discriminant
analysis. The objective is to determine the conditional
probability of an observation belonging to a class, given
values of the independent variables for the observation.
Logit, and the related probit technique, make use of a
cumulative probability function to constrain the pre-
dicted value of the dependent variable within the interval
(0, 1). The two models are similar, except that the logit
model uses the cumulative logistic function and probit
uses the cumulative normal distribution.
Ohlson's variables and models are shown in Panel B
of Table 1. Two models are shown, the primary model
using nine variables and a second model which is
extended to eleven variables by adding dummy variables
for exchange listing. The cumulative logistic function is
used to transform the y value into a bankruptcy
probability. For probabilities greater than 0.50, the
company is classified as "healthy," while the company is
classified as bankrupt for lower probabilities.
Boritz et al., 1995 tested the effect of various
proportions of bankrupt firms in the learning and testing
populations on the predictive ability of conventional
techniques and neural networks; samples of bankrupt
firms ranging from 1% to 50% were created for both
learning and testing populations. Boritz et al. also
showed that when the techniques were trained on a
sample containing 1% bankrupt firms (which approx-
imates the population proportion), all techniques except
quadratic discriminant analysis predicted that substan-
tially all observations would be non-bankrupt. As the
bankruptcy proportion increased in the training sample,
the techniques became more capable o f differentiating
between bankrupt and non-bankrupt.
The back-propagation neural network continued to
predict that all observations were non-bankrupt until the
proportion of bankrupt firms in the training sample
exceeded 20%, while the other techniques began to
Neural Networks and Business Failure Prediction
differentiate at lower training sample proportions. The
proportion of bankrupt firms in the testing sample had
little impact on the rates o f Type I or Type II errors.
4. RESEARCH DESIGN
This study examines four versions of the Back-Propaga-
tion algorithm (standard Back-Propagation, Functional
Link Back-Propagation With Sines, Pruned Back-propa-
gation, and Cumulative Predictive Back-Propagation),
the Optimal Estimation Theory approach for training
neural networks, and several conventional techniques
such as discriminant analysis, logit and probit. Three sets
o f variables (Altman, Ohlson, and combined A l t m a n and
Ohlson) are considered to enhance the generalizability of
the results.
4.1. Data Selection
The sample o f bankrupt companies used for the present
study was obtained from Boritz et al., 1995 which was
based on the data set developed by K e n n e d y and Shaw
(1991). The sample consisted of 171 companies which
filed for bankruptcy between 1971 and 1984 inclusive.
K e n n e d y and Shaw (1991) determined the exact date of
bankrupty filing for all companies in the data set. This
ensured that the sample contained the last set o f financial
statement data available prior to the bankruptcy filing}
The non-bankrupt companies used in the present
study also were obtained from Boritz et al., 1995. The
sample was collected from the Compustat H Database
and consisted of 6,153 non-bankrupt companies selected
from the same time period as the bankrupt companies.
These companies were selected by randomly assigning a
" f a i l i n g " year for each company in order to obtain a
similar distribution over time for both the non-bankrupt
and bankrupt c o m p a n i e s )
F o r both the bankrupt and non-bankrupt f i r m s , data
were collected to allow calculation of the variables used
by Airman (1968) and Ohlson (1980). These variables
are listed in Table 1. We present results for the primary
Ohlson model which uses 9 variables rather than the 11
variable version. For each set of variables and data set
(bankrupt and non-bankrupt), a learning subset and
testing subset, consisting of two-thirds and one-third of
the data, respectively, were randomly selected. The result
was two learning (testing) subsets, one consisting of 115
(56) bankrupt companies and the other with 4,100
(2,053) non-bankrupt companies. Random samples con-
taining 115 (56) companies were selected from the
learning (testing) subsets of non-bankrupt companies.
These samples were combined with the subsets of
bankrupt companies to create a learning (testing) sample
2For a complete description of the data collection procedure for the
bankrupt companies, see Kennedy and Shaw (1991), p. 100.
3 For a complete description of the data collection procedure for the
non-bankrupt companies, see Boritz et al., 1995.
507
of 230 (112) companies containing fifty percent bankrupt
firms, The learning data set is used to calculate
coefficients/network weights and then the test sample is
used to measure predictive accuracy o f the models. This
random data set selection, estimation, and testing process
is replicated five times to reduce the impact o f random
variation in data set composition. 4
4.2. Computational Procedure
The SAS statistics package (SAS Institute Inc., 1989)
was used to construct the discriminant analysis, logit,
and probit models. The Optimal Estimation Theory
neural network was constructed using a specialized
Fortran program. The back-propagation neural networks
were constructed using NeuralWorks Professional H. 5
This software package requires scaling of all input data.
Scaling involves mapping each variable to a range with
minimum and m a x i m u m values of 0.0 and 1.0, respec-
tively. F o r example, if the variable W C T A has minimum
and m a x i m u m values o f - 0 . 8 0 and 0.60, respectively;
then the values - 0 . 8 0 and 0.60 are transformed to
become 0.0 and 1.0, respectively, and all other values are
m a p p e d linearly between 0.0 and 1.0. As a result, the
minimum and m a x i m u m values must be determined for
each variable. The m i n i m u m and m a x i m u m values were
calculated for the entire data set (i.e., including all
bankrupt and non-bankrupt firms) to ensure that values
for data in the testing sample would be within the range
used b y the learning sample. ~
The discriminant analysis, logit, and probit techniques
and the Optimal Estimation Theory neural network
calculate parameter values directly from the data set. The
back-propagation neural network uses a learning process
4The number of firms in the example represents the theoretical
number for each replication. The actual number of companies in each
replication may differ from this value due to the random selection
process. For computational simplicity, the two random splits into
learning/testing samples and the 50% proportion of bankrupt finns
were actually conducted simultaneously.
The discriminant analysis, logit, and probit models were con-
structed using version 6.07 of SAS on a DEC 5000/200 computer with
Ultrix operating system. The Optimal Estimation Theory model also
was estimated on the DEC computer. The back-propagationwas carded
out on a 486DX2 66mHZ DOS-based microcomputer with 16
megabytes of memory.
6Examination of the data disclosed a small number of outlier
observations. For example, for the variable Current Liabilities/Current
Assets there was one data value in excess of 6,000 while the 99th
percentile for the variable was 5.4. Winsorization is a process where
extreme values are replaced by less extreme values which has the effect
of reducing the impact of the outlier while retaining the observation.A
total of six data values involving two Ohlson variables (Current
Liabilities/Current Assets and Funds From Operations/Total Debt)
were winsorized to reduce the effect of outliers on the range of
variables. Similarly, a total of seven data values involving two Altman
variables (Market Value of Eqnity/Book Value of Total Debt and
Retained Earnings/Total Assets) were winsorized. The winsorized
values were used for all techniques~
508 J. E. Boritz and D. B. Kennedy

TABLE 2
Type I and Type II Errors and Misclassification Costs for Ohlson's Variables

Cost ratios

Type I error Type II error 0.05 1 20

Back-propagation NN 27.30 17.15 17.63 22.23 26.82

Functional link back-propagation with sines NN 28.73 17.90 18.42 23.32 28.21
Pruned back-propagation NN 21.45 17.25 17.45 19.35 21.25
Predictive cumulative back-propagation NN 20.82 14.99 15.27 17.91 20.54
Optimal estimation theory NN (0 enhanced variables) 6.91 47.27 45.35 27.09 8.83
Optimal estimation theory NN (10 enhanced variables) 6.51 37.80 36.31 22.16 8.00
Optimal estimation theory NN (25 enhanced variables) 11.06 33.98 32.89 22.52 12.15
Lineardiscriminant analysis 28.05 13.60 14.29 20.83 27~36
Quadratic discriminant analysis 52.46 7.07 9.23 29.77 50.30
Non-parametric discriminant analysis 23.32 14.17 14.61 18.75 22.88
Logit 28.75 13.82 14.53 21.29 28.04
Probit 29.16 13.82 14.55 21.49 28.43
Altman model 34.37 28.45 28.73 31.41 34.09
Ohlson model (9 variable version) 8.91 38.65 37.23 23.78 10.33
Ohlson model (11 variable version) 34.26 17.33 18.14 25.80 33.45

The results shown represent averages for 5 replications.

The first two numeric columns represent the Type I and Type II errors (expressed as percentages). Type I error is defined
as a prediction of non-bankrupt when the company is actually bankrupt. Type II error is defined as a prediction of bankrupt
when the company is actually non-bankrupt.
The remaining numeric columns present the misclassification costs for the cost ratio shown at the top of the column.
Misclassification cost is calculated as (Type I error rate times the percentage of bankrupt firms times the cost ratio plus Type
II error rate times the percentage of non-bankrupt firms) divided by (percentage of bankrupt firms times cost ratio plus
percentage of non-bankrupt firms).
The Altman and Ohlson models represent benchmarks and are based on the coefficients reported in Altman (1968) and
Ohlson (1980). The models are shown in Table 1. The Ohison models use the coefficients for one year prior to bankruptcy
and 9 and 11 explanatory variables (i.e., omitting or including the dummy variables for stock exchange).

requiring that each observation be presented multiple

Actually Actually
times to the network. Each observation in the learning set
bankrupt non-bankrupt
was presented 5,000 times to all four back-propagation
networks. One hidden layer consisting of 5, (9, 13)
Predicted Correct Type II error
processing elements was used for all four neural
bankrupt
networks for the analysis based on Altman (Ohlson,
Predicted Type I error Correct
combined) variables. 7 The Functional Link Back-Propa-
non-bankrupt
gation With Sines network used one level of sine
transformations. Three different values are tested for the
number of enhanced variables in the Optimal Estimation
The prediction accuracies for all techniques based on
Theory neural network; 0, 5, and 10 for the Altman
Ohlson's variables are summarized in Table 2. 8 Table 3
variables, 0, 10, and 25 for the Ohlson variables, and 0,
summarizes the results for the Altman variables. Table 4
10, 20 for the combined variables.
summarizes the results for the combined set of both the
Ohlson and Altman variables.
Determination of which technique is best is a function
5. R E S U L T S
of the relative costs of Type I and Type II errors. In
The neural networks in this study use financial statement
variables to classify companies as either bankrupt or
non-bankrupt. The following table describes the resulting s The results in Tables 2, 3, and 4 are based on trainingand testing
Type I and Type II errors. samples containing50% bankrupt finns. The analysis was replicated
usinga testingsamplecontaining1% bankruptfirms and the Type I and
Type II errors were similarto errors reported in the tables.The analysis
was also replicated using a training sample containing one percent
7This architecture was chosen because the Optimal Estimation bankrupt finns. Similar to Boritz et al., 1995, all techniques except
Theory network is restricted to havingan equal number of nodes in the quadratic discriminant analysis produced models that predicted that
input and hidden layers. The numberof nodes in the hidden layer will substantiallyall new observationswould be non-bankrupt(i.e., Type I
be varied in future work on this project. errors approaching 100% and Type II errors approaching0%).
Neural Networks and Business Failure Prediction 509

TABLE 3
Type I and Type II Errors and Misclassification Costs for Altman's V a r i a b l e s

Cost ratios

Type I error Type II error 0.05 1 20

Back-propagation NN 40.60 13.46 14.75 27.03 39,3t

Functional link back-propagation with sines NN 29.27 19.96 20.40 24.62 28.83
Pruned back-propagation NN 39,56 17.49 18.54 28.53 38.51
Predictive cumulative back-propagation NN 33.54 20.03 20.67 26.79 32.90
Optimal estimation theory NN (0 enhanced variables) 10.85 73.35 70.37 42.10 13.83
Optimal estimation theory NN (5 enhanced variables) 10.86 55.41 53,29 33.14 12,98
Optimal estimation theory NN (10 enhanced variables) 11.50 55.21 53.13 33.36 13.58
Linear discriminant analysis 40.91 15.50 16.71 28.21 39.70
Quadratic discriminant analysis 4.01 70.16 67.01 37.09 7.16
Non-parametricdiscriminant analysis 30,66 22.35 22.75 26.51 30.26
Logit 22.90 19.97 20.11 21.44 22.76
Probit 25.38 21.08 21,28 23.23 25.18
Altman model 34.37 28.45 28.73 31.41 34.09
Ohlson model (9 variable version) 8.91 38.65 37.23 23.78 10.33
Ohlson model (11 variable version) 34.26 17.33 18.14 25.80 33.45

The results shown represent averages for 5 replications.

The first two numeric columns represent the Type I and Type II errors (expressed as percentages). Type I error is defined
as a prediction of non-bankrupt when the company is actually bankrupt. Type II error is defined as a prediction of bankrupt
when the company is actually non-bankrupt.
The remaining numeric columns present the misclassification costs for the cost ratio shown at the top of the column.
Misclassification cost is calculated as (Type I error rate times the percentage of bankrupt firms times the cost ratio plus Type
II error rate times the percentage of non-bankrupt firms) divided by (percentage of bankrupt firms times cost ratio plus
percentage of non-bankrupt firms).
The Altman and Ohlson models represent benchmarks and are based on the coefficients reported in Altman (1968) and
Ohlson (1980). The models are shown in Table 1. The Ohlson models use the coefficients for one year prior to bankruptcy
and 9 and 11 explanatory variables (i.e., omitting or including the dummy variables for stock exchange).

general, Type I and Type II errors are unlikely to be
equally costly. Unfortunately, there is no generally
accepted basis for trading off Type I and Type II errors in
bankruptcy prediction models. Different parties have
different perceptions of the relative costs of Type I and
Type II errors. For example, investors may be willing to
tolerate a higher rate of Type II error in order to have a
low rate of Type I errors. In contrast, management may
prefer a lower rate of Type II errors to avoid the "self-
fulfilling prophecies" when false bankruptcy signals
actually result in bankruptcy filings. We calculate
misclassification costs under three assumptions regard-
ing the ratio of the cost of the Type I and Type II errors.
For example, a cost ratio of 20 (0.05) indicates that a
Type I (II) error is considered to be twenty times more
costly than a Type II (I) error and the cost ratio of one
implies that the errors are equally costly. Misclassifica-
tion cost is calculated as:
(Type I error*%bankrupt firms*cost ratio)
+ (Type II error*%non-bankrupt firms)
(%bankrupt finns*cost ratio) + %non-bankrupt firms
For example in Table 2, the misclassification cost of
17.63 for the Back-Propagation Neural Network with
cost ratio of 0.05 is calculated as
[(27.30"0.5"0.05) + (17.15*0.5)]/[(0.5*0.05) + 0.5].
In Tables 2, 3, and 4, the results labeled "Airman"
and "Ohlson 9(11)" represent performance benchmarks.
These results are calculated using the variables and
coefficients exactly as reported by Altman (1968) and
Ohlson (1980) (see Table 1) but applied to our test
samples.
With reference to Table 2, the Optimal Estimation
Theory networks have the lowest level of Type I error,
followed by the back-propagation networks and the
traditional statistical techniques. The Predictive Cumu-
lative and Pruned Back-Propagation neural networks and
non-parametric discriminant analysis have similar levels
of Type I error while the standard Back-Propagation and
Functional Link Back-Propagation With Sines neural
networks have similar Type I errors compared to linear
discrimlnant analysis, logit and probit. Type I error is
high for quadratic discriminant analysis. However, this
relationship is reversed for Type II error, where the
traditional techniques have the lowest level, followed by
the back-propagation networks. The Optimal Estimation
Theory networks have much higher levels of Type II
errors.
The Ohlson 9 variable benchmark has low Type I
error and high Type II error which is comparable to the
Optimal Estimation Theory networks in both cases. The
Altman and Ohlson 11 variable benchmarks have
510 J. E. Boritz and D. B. Kennedy

TABLE 4
Type I and Type II Errors and Misclassification Costs for the Combined Set of Altman and Ohlson Variables

Cost ratios

Type I error Type II error 0.05 1 20

Back-propagation NN 25.73 15.97 16.43 20.85 25.27

Functional link back-propagation with sines NN 25.52 16.68 17.10 21.10 25.10
Pruned back-propagation NN 38.02 11.i8 12.46 24.60 36.74
Predictive cumulative back-propagation NN 25.23 14.88 15.37 20.06 24.74
Optimal estimation theory NN (0 enhanced variables) 28.59 73.35 71.22 50.97 30.72
Optimal estimation theory NN (10 enhanced variables) 26.52 73.61 71.37 50.07 28.76
Optimal estimation theory NN (20 enhanced variables) 36.20 67.08 65.61 51.64 37.67
Lineardiscriminant analysis 27.49 12.66 13.37 20.08 26.78
Quadraticdiscriminant analysis 32.68 24.60 24.98 28.64 32.30
Non-parametric discriminant analysis 25.90 13.13 13.74 19.52 25.29
Logit 24.74 16.10 16.51 20.42 24.33
Probit 25.10 16.10 16.53 20.60 24.67
Altman model 34.37 28.45 28.73 31.41 34.09
Ohlson model (9variable version) 8.91 38.65 37.23 23.78 10.33
Ohlson model (11 variable version) 34.26 17.33 18.14 25.80 33.45

The results shown represent averagesfor 5 replications.

The first two numeric columns representthe Type I and Type II errors (expressed as percentages).Type I error is defined
as a prediction of non-bankrupt when the company is actually bankrupt. Type II error is defined as a prediction of bankrupt
when the company is actually non-bankrupt.
The remaining numeric columns present the misclassificationcosts for the cost ratio shown at the top of the column.
Misclassificationcost is calculated as (Type I error rate times the percentage of bankrupt firms times the cost ratio plus Type
II error rate times the percentage of non-bankrupt firms) divided by (percentage of bankrupt firms times cost ratio plus
percentage of non-bankrupt firms).
The Altman and Ohlson models represent benchmarks and are based on the coefficients reported in Altman (1968) and
Ohlson (1980). The models are shown in Table 1. The Ohlson models use the coefficients for one year prior to bankruptcy
and 9 and 11 explanatory variables (i.e., omitting or including the dummy variables for stock exchange).

relatively high Type I error (only quadratic discriminant
analysis is higher) while the Type II error is moderate for
Ohlson 11 variables relative to the other techniques and
higher for the Altman variables.
When the cost ratio is one, the Predictive Cumulative
Back-Propagation Neural Network and non-parametric
discriminant analysis have the lowest misclassification
costs. As the cost ratio increases, the Optimal Estimation
Theory networks have the lowest misclassification cost,
while linear and non-parametric discriminant analysis,
logit, and probit are favoured for the lowest cost ratios.
Table 3 summarizes the results for the Altman
variables. The results differ from those based on the
Ohlson variables reported in Table 2. Quadratic dis-
criminant analysis has the lowest Type I error (4%),
followed by the Optimal Estimation Theory networks
(approx. 11%), logit and probit (23-25%), and then the
back-propagation networks and the other two dis-
criminant analysis techniques (30-41%). The relative
results for Type II error are almost exactly reversed.
When the cost ratio is one, logit and probit have the
lowest misclassification costs. As the cost ratio increases
(decreases), quadratic discriminant analysis and the
Optimal Estimation Theory neural networks have the
lowest (highest) costs.
Table 4 summarizes the results for the combined set of
Altman's and Ohlson's variables. The Type I error is
similar for many of the techniques (24.7-28.6%) except
quadratic discriminant analysis, Optimal Estimation
Theory neural network with 20 enhanced variables, and
Pruned Back-Propagation neural network. The Type II
errors are similar for many techniques (11.2-16.7%)
except quadratic discriminant analysis and the Optimal
Estimation Theory neural networks.
A comparison of Tables 2 and 3 indicates that,
generally speaking, all models based on Ohlson's
variables are superior to those same models based on
Altman's variables. That is, in most instances (20 out of
24 cells) the models based on Ohlson's variables have
both Type I and Type II error lower in Table 2 than in
Table 3. There are only four exceptions to this general
pattern: Back-Propagation Type II error, Quadratic
Discriminant Analysis Type I error, Logit Type I error
and Probit Type I error. Also, when the cost ratio is 1, the
models based on Ohlson's variables are superior in the
sense of having a lower overall misclassification cost in
all 12 of the ceils in that column. When the cost ratios are
0.05 and 20, the models based on Ohlson's variables are
superior in 11 and 9 of the 12 cells in their respective
cohunns.
Comparison of Type I error and Type 1_Ierror in Tables
2 and 4 shows that 12 of the 24 cells in Table 4 (which
is based on the combined Altman and Ohlson variables)
have superior performance compared to that reported in
Neural Networks and Business Failure Prediction 511

TABLE 5
Coefficients of Variation for Type I and Type II Error Statistics Reported in Tables 2, 3, and 4

Ohlson Altman Combined

Type I Type II Type I Type II Type I Type II

error error error error error error

Back-propagation N N 52.49 48.95 38.15 50.12 28.17 58.39

Functional link back-propagation with sines NN 39.87 72.79 51:27 77.26 54.33 40.70
Pruned back-propagation NN 18.48 45.83 38.30 87.24 38.19 32.08
Predictive cumulative back-propagation NN 44.34 27.00 48.56 89.93 37.53 27.40
Optimal estimation theory NN 55.87 16.94 25.37 16.40 107.38 37.89
(0 enhanced variables)
Optimal estimation theory NN - - 49.80 13.89 - -
(5 enhanced variables)
Optimal estimation theory NN 59.75 26.63 48.15 2.60 110.50 32.05
(10 enhanced variables)
Optimal estimation theory NN . . . . 92.90 40.72
(20 enhanced variables)
Optimal estimation theory NN 67.15 29.96 . . . .
(25 enhanced variables)
Linear discriminant analysis 25.35 25,50 26.55 35.49 25.33 39.96
Quadratic discriminant analysis 39.96 121.32 104.00 21.27 88.43 105.11
Non-parametric discriminant analysis 22.75 23.43 33.59 12.06 17.95 30.93
Logit 17.16 46.34 24.69 32.41 19.48 30.14
Probit 17.84 46.34 38.61 44.14 18.36 30.14
AItman model 13.46 27.14 13.46 27.14 13,46 27.14
Ohlson model (9 variable version) 4.85 13.32 4;85 13.32 4.85 13.32
Ohlson model (11 variable version) 8.93 16.08 8.93 16.08 8.93 16.08

The results shown are based on 5 replications.

Type I error is defined as a prediction of non-bankrupt when the company is actually bankrupt. Type II error is defined as a prediction
of bankrupt when the company is actually non-bankrupt.
The Altman and Ohlson models represent benchmarks and are based on the coefficients reported in Altman (1968) and Ohlson
(1980). The models are shown in Table 1. The Ohlson models use the coefficients for one year prior to bankruptcy and 9 and 11
explanatory variables (i.e., omitting or including the dummy variables for stock exchange).

corresponding cells in Table 2 which is based solely on
Ohlson's variables. Some interesting additional observa-
tions about Table 4 are:
• the Type II error for all of the Back-Propagation
networks is lower in Table 4 compared with Table 2,
• the Type I errors for the Back-Propagation neural
network and the Functional Link Back-Propagation
with Sines neural network are lower, but interestingly,
the Type I errors for the Pruned Back-Propagation
neural network and the Predictive Cumulative Back-
Propagation neural network are significantly higher in
Table 4 compared with Table 2,
• all three versions of the Optimal Estimation Theory
neural networks are significantly inferior with the
combined set of variables relative to the results using
Ohlson's variables alone. This underlines the necessity
of selecting appropriate variables to apply in a neural
network setting and not simply leaving it to the neural
network itself to sort out those variables.
We examined the sensitivity of results across the five
replications by considering the coefficients of variation
for the various models. As can be seen from an
inspection of Table 5, there is a fair degree of variation in
the results for the back-propagation models, and sig-
nificant variation in Type I errors for the OET models
based on Ohlson and Combined variables and the
quadratic discriminant analysis models.
6. CONCLUSIONS
This study found that there are large differences in Type
I error, Type II error, and misclassification costs among
the neural networks tested. The Optimal Estimation
Theory neural networks had the lowest rate of Type I
error but the highest rate of Type II error. The Back-
Propagation neural networks had high Type I error but
lower rates of Type II error.
It is difficult to make blanket statements about the
performance of the various techniques because there is a
good deal of variation in their relative strengths and
weaknesses according to which set of variables is used to
fit the model, and whether Type I errors or Type II errors
are of greatest interest. While the neural network models'
performance is in line with that of the more conventional
techniques such as discriminant analysis and logit/probit,
it is noteworthy that their performance is not a dramatic
improvement over those conventional techniques. The
latter have the advantage of being fairly well understood
512
and easy to apply using readily available statistical
packages such as SAS. In contrast, the neural network
models are not as well understood, are not as easy to
apply, and do not appear to provide a significant
advantage in terms of dramatic reductions in mis-
classification costs.
This observation is not intended to be a sweeping
rejection of neural network models. For, while we have
varied a number of important factors in this study, we did
not attempt to develop optimal neural network archi-
tectures for addressing the prediction of business failure
with the various networks. We are in the process of
conducting a study wherein we will, in fact, vary the
architectures of these networks.
In summary, while our study demonstrates that the
various types of neural networks perform reasonably
well in predicting business failure, we find that their
performance is not in any systematic way superior to the
more conventional techniques that have been applied to
this problem such as discriminant analysis and logit/
probit. Comparing results across the three sets of
variables shows that the relative performance of neural
networks and traditional statistical techniques is affected
by the choice of variables in the learning sample. We
demonstrate that the performance of the neural networks
tested is sensitive to the choice of variables selected and
that the networks cannot be relied upon to "sift through"
variables and focus on the most important variables
(network performance based on the combined set of
Ohlson and Altman variables was frequently worse than
their performance with one of the subsets). It is also
important to note that the results are quite sensitive to
sampling error. The significant variations across replica-
tions for some of the models indicate the sensitivity of
the models to variations in the data. Researchers
reporting on the results of applying neural networks
should be replicating their models several times to obtain
a reliable measure of model performance. Our results
suggest the need to exercise caution in interpreting results
of studies where such replication was not performed.
There is potential for extending the research reported
here as neural networks are affected by many parameters.
Examples include using different neural networks,
changing the number of hidden processing elements, and
changing the set of explanatory variables.
Acknowledgements--We thank the Editor, anonymous reviewers, and
participants at the Third Annual Research Workshop on AI/ES in
Accounting, Auditing and Tax for their useful comments. We gratefully
acknowledge the contribution of Jeff Boucher and Shafin Kanji in pilot
testing some aspects of this research, computing assistance provided by
Nick Favron, and financial support provided by the Social Sciences and
Humanities Research Council of Canada.
J. E. Boritz and D. B. Kennedy
REFERENCES
Altman, E. (1968). Financial ratios, discriminant analysis and the
prediction of corporate bankruptcy. The Journal of Finance, 23(4),
(September), 589-609.
Altman, E. (1983). Corporate financial distress--A complete guide to
predicting, avoiding and dealing with bankruptcy, New York: John
Wiley & Sons.
Beaver, W. (1966). Financial ratios as predictors of failure. Empirical
Research in Accounting: Selected Studies 1966, Journal of
Accounting Research, 4, (Supplement), 71-111.
Beaver, W. (1968). Market prices, financial ratios, and the prediction of
failure. Journal of Accounting Research, 6(2), (Autumn),
179-192.
Bell, T. B., Ribar, G. S., & Verchio, J. (1990). Neural nets vs. logistic
regression: A comparison of each model's ability to predict
commercial bank failures. Proceedings of the 1990 Deloitte
Touche~University of Kansas Symposium on Auditing Problems,
29-53.
Boucher, J. (1990). Artificial neural systems and the prediction of
corporate failure. Master's Project, School of Accountancy, Uni-
versity of Waterloo.
Boritz, J. E. (1991). The going concern assumption. Toronto: Canadian
Institute of Chartered Accountants.
Boritz, J. E., Kennedy, D. B., & Albuquerque, A. (1995). Predicting
corporate failure using a neural network approach. International
Journal of Intelligent Systems in Accounting, Finance and Manage-
ment, 4(2), (June), 95-111.
Compustat Services, Inc., (1992). Compustat 11 Database, Englewood,
CO: Standard & Poor's Compustat Services Inc.
Fanning, K. M., & Cogger, K. O. (1994). A comparative analysis of
artificial neural networks using financial distress prediction.
International Journal of Intelligent Systems in Accounting, Finance
and Management, 3(4), (December), 241-252.
Jones, E L. (1987). Current techniques in bankruptcy prediction.
Journal of Accounting Literature, 6, 131-164.
Jones, W., & Hoskins, J. (1987). Back propagation, a generalized delta
rule. Byte, (October), 155-162.
Kennedy, D. B. (1992). Classification techniques in accounting
research: empirical evidence of comparative performance. Contem-
porary Accounting Research, 8(2), (Spring), 419-442.
Kennedy, D. B., & Shaw, W. H. (1991). Factors influencing auditor
opinion on bankrupt firms. Contemporary Accounting Research,
8(1), (Fall), 97-114.
Ohlson, J. (1980). Financial ratios and the probabilistic prediction of
bankruptcy. Journal of Accounting Research, 18(1), (Spring),
109-131.
Pao, Y. H. (1989). Adaptive pattern recognition and neural networks.
Reading, MA: Addison Wesley.
Salchenberger, L. M., Cinar, E. M., & Lash, N. A. (1992). Neural
networks: A new tool for predicting thrift failures. Decision
Sciences, 23(4), (July/August), 899-916.
SAS Institute Inc. (1989). SAS/STAT User's Guide, Version 6, 4th ed.,
Vol. 1 and 2, Cary, NC: SAS Institute Inc.
Shepanski, J. E (1988). Fast learning in artificial neural systems:
Multilayer perceptron training using optimal estimation. Proceed-
ings of the IEEE 2nd International Conference in Neural Nets, Vol.
4, San Diego, July 1988.
Tam, K. Y., & Kiang, M. Y. (1992). Managerial applications of neural
networks: The case of bank failure predictions. Management
Science, 38(7), (July), 926-947.
Zavgren, C. V. (1983). The prediction of corporate failure: The state of
the art. Journal of Accounting Literature, 2, (Spring), 1-38.