The European Journal of Applied Economics - Vol. 16 #2

Vol.
16 Nº 2
OCTOBER 2019
journal.singidunum.ac.rs
Predicting the type of auditor opinion: The role of technology as an absorptive Ownership concentration and firm The rising government expenditure
Statistics, Machine learning, capacity in economic growth in emerging performance: An empirical analysis in Nigeria: Any influence on growth?
or a combination of the two? p. 1-58 economies: A new approach p. 59-78 in Oman p. 79-94 p. 95-108
What can we expect in the future of academic

Forecasting model of Vietnamese Causes of failure of the phillips curve: Do large firms benefit more from
research? The most common research
consumers’ purchase decision of does tranquillity of economic R&D investment?
problems analysed in the top journals in
domestic apparel p. 109-121 environment matter? p. 139-154 p. 155-173
the field of entrepreneurship p. 122-138
Vol. 16 No. 2
w w w. j ou r na l.sing idunum.a c .rs

Vol. 16 No. 2
Publisher:
Singidunum University
E d it o r ia l B o a r d
Professor Milovan Stanišić, Singidunum University mstanisic@singidunum.ac.rs
Emeritus Professor Slobodan Unković, Singidunum University unkovic@singidunum.ac.rs
Professor Francesco Frangialli, UNWTO frangialli@gmail.com
Professor Gunther Friedl, Technische Universität München, München gunther.friedl@wi.tu-muenchen.de
Professor Karl Ennsfellner, IMC University of Applied Sciences, Krems karl.ennsfellner@fh-krems.ac.at
Professor Gyorgy Komaromi, International Business School, Budapest gyorgy@komaromi.net
Professor Vasile Dinu, University of Economic Studies, Bucharest dinu_cbz@yahoo.com
Professor Ada Mirela Tomescu, University of Oradea, Oradea ada.mirela.tomescu@gmail.com
Professor Radojko Lukić, University of Belgrade rlukic@ekof.bg.ac.rs
Professor Alexandar Angelus, Lincoln University angelus@lincolnuca.edu
Professor Nemanja Stanišić, Singidunum University nstanisic@singidunum.ac.rs
Professor Verka Jovanović, Singidunum University vjovanovic@singidunum.ac.rs
Professor Milan Milosavljević, Singidunum University mmilosavljevic@singidunum.ac.rs
Professor Olivera Nikolić, Singidunum University onikolic@singidunum.ac.rs
Professor Goranka Knežević, Singidunum University gknezevic@singidunum.ac.rs
Professor Mladen Veinović, Singidunum University mveinovic@singidunum.ac.rs
Professor Jovan Popesku, Singidunum University jpopesku@singidunum.ac.rs
Professor Zoran Jeremić, Singidunum University zjeremic@singidunum.ac.rs
Professor Vesselin Blagoev, Varna University of Management blagoev@vum.bg
Professor Michael Minkov, Varna University of Management minkov@iuc.bg
Associate Professor Christine Juen, Austrian Agency for International Mobility and Cooperation in Education, Science and Research, Wien chrisine.juen@oead.at
Associate Professor Anders Steene, Södertörn University, Stockholm/Hudinge anders.steene@sh.se
Associate Professor Ing. Miriam Jankalová, University of Zilina, Prague miriam.jankalova@fpedas.uniza.sk
Associate Professor Bálint Molnár,Corvinus University of Budapest, Budapest molnarba@inf.elte.hu
Associate Professor Vesna Spasić, Singidunum University vspasic@singidunum.ac.rs
Associate Professor Michael Bukohwo Esiefarienrhe, University of Agriculture, Dept. of Maths/Statistics, Markurdi esiefabukohwo@gmail.com
Associate Professor Goh Yen Nee, Graduate School of Business, Universiti Sains Malaysia yngoh@usm.my
Research Associate Professor Aleksandar Lebl, Research and Development Institute for Telecommunications and Electronics, Belgrade lebl@iritel.com
Roberto Micera, PhD, Researcher, National Research Council (CNR) Italy r.micera@iriss.cnr.it
Assistant Professor Patrick Ulrich, University of Bamberg patrick.ulrich@uni-bamberg.de
Assistant Professor Jerzy Ładysz, Wrocław University of Economics, Poland jerzy.ladysz@ue.wroc.pl
Assistant Professor Konstadinos Kutsikos, University of the Aegean, Chios kutsikos@aegean.gr
Assistant Professor Theodoros Stavrinoudis, University of Aegean, Chios tsta@aegean.gr
Assistant Professor Marcin Staniewski, University of Finance and Management, Warsaw staniewski@vizja.pl
Assistant Professor Gresi Sanje, İstanbul Bilgi Üniversitesi, Istanbul gresi.sanje@bilgi.edu.tr
Assistant Professor Michał Biernacki, Wrocław University of Economics, Poland michal.biernacki@ue.wroc.pl
Assistant Professor Piotr Luty, Wrocław University of Economics, Poland piotr.luty@ue.wroc.pl
Assistant Professor Blazenka Hadrovic Zekic, Faculty of Economics in Osijek, Croatia hadrovic@efos.hr
E d it o r ia l O f f ice
Editor in Chief: Professor Nemanja Stanišić, Singidunum University nstanisic@singidunum.ac.rs
Managing Editor: Associate Professor Gordana Dobrijević, Singidunum University gdobrijevic@singidunum.ac.rs
Technical Editor: Aleksandra Stojanović, Singidunum University astojanovic@singidunum.ac.rs
English Language Editor: Marijana Prodanović, Singidunum University mprodanovic@singidunum.ac.rs
Prepress: Miloš Višnjić Design: Aleksandar Mihajlović
ISSN: 2406-2588
The European Journal of Applied Economics is published twice a year.
Contact us:
The European Journal of Applied Economics Access to full text articles:
32 Danijelova Street, 11010 Belgrade, Serbia The European Journal of Applied Economics (www.journal.singidunum.ac.rs),
Phone No. +381 11 3094046, +381 11 3093284 SCindeks (https://scindeks.ceon.rs/journaldetails.aspx?issn=2406-2588&lang=en).
Fax. +381 11 3093294
E-mail: journal@singidunum.ac.rs Copyright © 2019
Web: www.journal.singidunum.ac.rs Singidunum University, Belgrade
Printed by: Caligraph, Belgrade All rights reserved.
This is an open access journal which means that all content is freely available without charge to the user or his/her institution. Users are allowed to read, download, copy, distribute, print, search, or link to
the full texts of the articles in this journal without asking for prior permission from the publisher or the author. This is pursuant to the BOAI definition of open access.
CONTENTS
PREDICTING THE TYPE OF AUDITOR OPINION: STATISTICS,

1 - 58 MACHINE LEARNING, OR A COMBINATION OF THE TWO?
Nemanja Stanišić, Tijana Radojević*, Nenad Stanić
THE ROLE OF TECHNOLOGY AS AN ABSORPTIVE CAPACITY IN

59-78 ECONOMIC GROWTH IN EMERGING ECONOMIES: A NEW APPROACH
Richard Angelous Kotey*, Joshua Yindenaba Abor
OWNERSHIP CONCENTRATION AND FIRM PERFORMANCE:

79-94 AN EMPIRICAL ANALYSIS IN OMAN
Mawih Kareem Al Ani*, Asma Mohammed Al Kathiri
THE RISING GOVERNMENT EXPENDITURE IN NIGERIA:

95-108 ANY INFLUENCE ON GROWTH?
Segun Subair Awode
FORECASTING MODEL OF VIETNAMESE CONSUMERS’ PURCHASE

109-121 DECISION OF DOMESTIC APPAREL
Dung Tien Luu
WHAT CAN WE EXPECT IN THE FUTURE OF ACADEMIC

RESEARCH?THE MOST COMMON RESEARCH PROBLEMS
122-138 ANALYSED IN THE TOP JOURNALS IN THE FIELD OF
ENTREPRENEURSHIP
Irena Đalić
CAUSES OF FAILURE OF THE PHILLIPS CURVE: DOES

139-154 TRANQUILLITY OF ECONOMIC ENVIRONMENT MATTER?
Yhlas Sovbetov*, Muhittin Kaplan
DO LARGE FIRMS BENEFIT MORE FROM R&D INVESTMENT?

155-173
Oyakhilome Ibhagui
III
IV
Original paper/Originalni naučni rad
PREDICTING THE TYPE OF AUDITOR OPINION: STATISTICS, MACHINE

LEARNING, OR A COMBINATION OF THE TWO?
Nemanja Stanišić, Tijana Radojević*, Nenad Stanić
Singidunum University,
Belgrade, Serbia
Abstract: Article info:

The goal of this study is to overcome the identified methodological limi-
Received: January 9, 2019
tations of prior studies aimed at predicting the type of auditor opinion
Correction: March 5, 2019
and draw definite conclusions on the relative predictive performance
Accepted: April 11, 2019
of different predictive methods for this particular task. Predictive per-
formance of twelve candidate models from the realms of statistics and
machine learning is assessed separately for the two common real-life
scenarios: a) when prior information on the client (i.e. types of audit
opinion received in the past) is available and can be used in prediction,
and b) when such information is not available (e.g. new companies).
The results show that, in the first scenario, several methods from both
realms achieve comparable predictive performance of around 0.89, as
measured by the Area under the curve (AUC). In the second scenario,
however, machine learning algorithms, particularly tree-based ones,
such as random forest, perform significantly better, achieving the AUC
of up to 0.79. Finally, we develop and assess the predictive performance
of two hybrid models aimed at combining the strong points of both
statistical (i.e. interpretability of results) and machine learning (i.e.
handling a large number of predictors and improved accuracy) ap-
proaches. The complete procedure is demonstrated in a reproducible Keywords:
manner, using the largest empirical data set ever used in this stream of auditor opinion,
research, comprising 13,561 pairs of annual financial statements and financial reports,
the corresponding audit reports. The procedures described in this study generalized linear mixed models,
allow audit and finance professionals around the globe to develop and random forest,
test predictive models that will aid their procedures of audit planning guided regularized random forest,
and risk assessment. ensembles.
1
*E-mail: tradojevic@singidunum.ac.rs
EJAE 2019  16 (2)  1-58
STANIŠIĆ, N., RADOJEVIĆ,T., STANIĆ, N.  PREDICTING THE TYPE OF AUDITOR OPINION: STATISTICS, MACHINE LEARNING, OR A COMBINATION OF
THE TWO?
INTRODUCTION
According to the international auditing standards, auditors are required to consider a number of
factors related to the risk of material misstatements in their clients’ financial reports, to provide a basis
for designing and performing audit procedures (ASB GAAS Section 315, 2013; IAASB ISA 315, 2013).
Namely, they are required to “discuss the susceptibility of the entity’s financial statements to material
misstatement” (ASB GAAS Section 315, 2013, para. 5; IAASB ISA 315, 2013, para. 10) and “identify
and assess the risks of material misstatement at: (a) the financial statement level and (b) the assertion
level for classes of transactions, account balances, and disclosures” (ASB GAAS Section 315, 2013, para.
26; IAASB ISA 315, 2013, para. 35). In light of this, developing and employing models that are, given
the data obtained from financial statements, being audited and other available company information,
predictive of the forthcoming type of auditor opinion, and thus provide timely and reliable assessments
of the risk of detectable material misstatements, are of great practical importance to auditors (Bell and
Tabor, 1991). By using such models, auditors can screen their client portfolio and direct attention
to those clients who have high estimated probability of receiving a qualified audit opinion, and thus
not only save time and money (Gaganis, Pasiouras, Spathis et al., 2007), but also reduce the potential
litigation cost and reputation risk.
As a result, over the past decades, numerous models for predicting the type of audit opinion have
been proposed in the literature (Kirkos et al., 2007). Earlier studies have mainly relied on the use of
classical statistical modeling techniques, mainly probit and logistic regression models (Dopuch et al.,
1987; Francis and Krishnan, 1999; Krishnan and Krishnan, 1996), while more recent ones have shown
an increased preference for modern machine learning techniques, such as decision trees and artificial
neural networks (Fernández-Gámez et al., 2016; Pourheydari et al., 2012; Saif et al., 2013; Yasar et al.,
2015).1 Although the abundance of empirical research, conducted on the topic, might be expected to
provide a solid basis for practitioners to identify a predictive technique that is best suited for predicting
audit qualifications, we argue that this is not possible for two main reasons.
The first reason is that prior studies have not fully employed the respective strong points of the two
sets of techniques. Specifically, the studies demonstrating the use of statistical techniques have not
taken the advantage of their capacity to model individual effects of clients and auditor firms, essentially
ignoring the valuable information that is readily available in prior audit reports. On the other hand, the
studies demonstrating the use of machine learning techniques have considered only a limited pool of
theoretically-driven predictors, failing to take advantage of their capacity to handle a large number of
predictors that can be generated based on the data available from the complete set of financial reports.
Therefore, the predictive performance achieved and reported in prior studies can be deemed subop-
timal; consequently, any comparison based on these results would be indicative rather than definite.
The second reason is that most studies — with the exception of the study by Doumpos and col-
leagues (2005) — have not differentiated between the cases where prior information on the client
and the auditor (i.e. audit reports from prior periods) is available and cases where such information
is not available. This difference is crucial, since the highest achievable predictive performance differs
substantially between the two scenarios. By not differentiating between the two scenarios, researchers
1 Even though the distinction between statistical and machine learning methods is not a clear-cut one (statistics is the basis
for many modern machine learning methods), we use this terminological demarcation in this study mainly to highlight
the fact that statistical methods are better at modeling the systematic effects suggested by the researcher, and machine
learning methods are more capable of handling a large number of predictors, which is relevant to the aims of our study.
This distinction also highlights the temporal trend in the usage of predictive methods observed in prior literature.
2
EJAE 2019  16 (2)  1-58
THE TWO?
have been not only reported an aggregated accuracy across the two scenarios, but, more importantly,
failed to examine a likely possibility that, to achieve optimal results, the two scenarios require two dif-
ferent predictive techniques.
Considering the above-mentioned, the main motivation for conducting this study is to build several
candidate models from the realms of both statistics and machine learning, using a single empirical data
set, in a way that allows us to draw more valid conclusions on their relative predictive performance than
is possible based on the existing body of research. To do so, we first demonstrate how to employ the
respective strong points of the two sets of techniques more effectively. Specifically, we: a) demonstrate
how client- and auditor-specific individual effects can be modeled using random effects within the lo-
gistic regression framework and b) design a feature-generation procedure aimed at making better use
of the data available in the complete set of financial reports. Those two methodological amendments are
demonstrated to meaningfully improve the predictive performance of statistical and machine learning
techniques, respectively. Next, we demonstrate that, when their potentials are more fully employed, each
set of techniques is better suited to one real-life scenario. While mixed-effects logistic regression, which
is a statistical tool, should be preferred for the interpretability of its output, when prior information on
the client is available, machine learning techniques, such as random forest [RF] and guided regular-
ized random forest [GRRF], should be preferred for their superior predictive performance in scenarios
where no prior information is available (e.g. new companies). Finally, motivated by this finding, we
build two hybrid models that combine the feature selection capability and flexibility of machine learn-
ing algorithms with the improved interpretability and the ability to account for the systematic effects
present in panel data sets of regression models. The first one is a stacked ensemble that uses predictions
from seven different machine learning algorithms as predictors and mixed-effects logistic regression
as a meta-learner. This model outperformed each of the individual regular predictive models in both
scenarios. The second hybrid model involves the inclusion of a compact set of classification rules selected
by the GRRF algorithm as dummy variables in mixed-effects logistic regression. This model is shown
to be particularly useful in scenarios where prior information is available, providing the best trade-off
between interpretability and accuracy.
The complete model development and validation procedure is demonstrated in a reproducible man-
ner using a large empirical data set on audit opinions and financial reports. As such, the framework
presented herein can be used by audit and finance professionals around the globe to develop and test
the predictive models that will aid the process of audit planning and risk assessments.
LITERATURE REVIEW
The line of research that this study belongs to is that aimed at forming expectations of the type of
auditor opinion.2 The main challenges are identifying factors, associated with receiving qualified or
modified opinion (including qualified opinion, adverse opinion, and disclaimer of opinion), as op-
posed to unqualified audit opinion, and developing predictive models that, with the highest achievable
accuracy, estimate the probability of class membership at the level of the individual financial report.
2 Lines of research which, although somewhat related, should be clearly demarcated from the one under con-
sideration in this study, are those devoted to: detection of earnings management (DeAngelo, 1986; Dechow
et al., 1995; DeFond & Jiambalvo, 1994; Healy, 1985; Jones, 1991); detection of fraudulent reporting (Beneish,
1999; Glancy and Yadav, 2011; Humpherys et al., 2011; Ngai et al., 2011; Perols, 2011; Perols et al., 2017; Zhou
and Kapoor, 2011); evaluation of earnings quality (Dechow et al., 2010) or prediction of a particular type of
auditor qualification, such as going-concern qualification (Bell and Tabor, 1991; Maggina and Tsaklanganos,
2011; Monroe and Teh, 2009; Mutchler and Hopwood, 1997; Yeh et al., 2014).
3
EJAE 2019  16 (2)  1-58
THE TWO?
In Table 1, we present prior studies aimed at modeling qualified audit opinions in chronological
order.
Sample Factors Associated with Qualified/

Paper Method
Characteristics Modified Audit Opinions
◆ Being listed on stock exchange for less than
five years
◆ Low stock returns of the stocks in the
275 qualified and 441 industry
Dopuch et al. unqualified opinions Probit ◆ Increase in variability of stock returns
(1987) from 1969–1980 (US regression
◆ Decrease in value of beta coefficient
public companies)
◆ Net income negative in current year
◆ Increase in financial leverage
◆ Decrease in ratio of receivables to total assets
◆ Listed over-the-cou nter
24 qualified and 49 ◆ Small companies
Kinney and unqualified opinions T-test and ◆ Low profitability
McDaniel (1989) from 1976–1985 (US regression ◆ High financial leverage
public companies) ◆ Low growth
◆ Negative stock returns
◆ Low share of receivables in total assets
◆ High financial leverage
◆ Small size (assets)
◆ Negative net income in current year
◆ High volatility (SD) of stock returns
163 qualified and
◆ Low stock returns
1,674 unqualified One-stage and
Krishnan et al. ◆ Higher levels of outsider ownership
audit opinions from two-stage probit
(1996) ◆ Small size (assets) relative to auditor client
1986–1988 (US pub- model
lic companies) portfolio
◆ Low growth (in assets)
◆ Being listed on stock exchange for longer
time
◆ Probability of litigation as measured by the
variable constructed by Stice (1991)
8 qualified and 103
unqualified audit ◆ Low profitability
opinions from Kruskal-Wallis ◆ Low growth
Laitinen and
1992–1994 (public test and logistic ◆ High leverage
Laitinen (1998)
companies listed regression ◆ High credit risk
on Helsinki stock ◆ Small companies
exchange)
◆ High leverage
◆ Net income negative in current year
◆ Small companies
284 modified and
◆ High stock price volatility
Francis and 2,324 unmodified
Probit model ◆ Low stock returns
Krishnan (1999) from 1986–1987 (US
companies) ◆ Being listed on stock exchange longer
◆ High accruals as measured by the difference
between net income and cash flow from op-
erations
4
EJAE 2019  16 (2)  1-58
THE TWO?
173 qualified and

173 matched-pair High discretionary accruals as measured by:
Contingency-table ◆ The modified Jones model
Bartov et al. unqualified opinions
test and logistic
(2000) from the period ◆ The cross-sectional Jones model
regression
1980–1997 (US com- ◆ The cross-sectional modified Jones model
panies)
Multicriteria
50 qualified and 50 decision aid clas- ◆ High share of receivables in sales
matched-pair un- sification method ◆ Low profitability (net profit/total assets)
Spathis et al.
qualified audit opin- (UTADIS), discri- ◆ Low liquidity (working capital/total assets)
(2003)
ions from 1997–1999 minant analysis, ◆ Low asset turnover (sales/total assets)
(Greek companies) and logistic ◆ High credit risk (Z score)
regression
859 qualified and ◆ Poor credit rating

5,189 unqualified ◆ Low liquidity
Doumpos et al. Support vector
audit opinions from ◆ Low profitability
(2005) machines (SVMs)
1998–2003 (UK and ◆ Low fixed assets turnover
Irish companies) ◆ Low growth
162 qualified and ◆ Low profitability as measured by profit mar-
23 unqualified audit Ordinary least gin to total assets
Caramanis and
opinions from 2001 squares (OLS)
Spathis (2006) ◆ Low liquidity as measured by current assets
(Greek public regression
companies) divided by current liabilities
114 qualified and ◆ Large size (assets)

114 unqualified Discriminant ◆ Low profitability
Gaganis and
audit opinions from analysis and logis-
Pasiouras (2006) ◆ Low growth
2003–2004 (UK and tic regression
Irish companies) ◆ A non-Big N auditor
264 qualified and

Probabilistic ◆ Low profitability
3,069 unqualified
neural network ◆ Poor credit rating
Gaganis, audit opinions from
(PNN), artificial
Pasiouras, and 1997–2004 (public ◆ Extreme values (both high and low) of days
neural network
Doumpos (2007) companies listed on payable outstanding
(ANN), and
the London stock ◆ Small companies
logistic regression
exchange)
980 qualified and K-nearest neigh- ◆ High credit risk

Gaganis, 4,296 unqualified bors (k-NN), ◆ High liquidity (current ratio)
Pasiouras, Spa- audit opinions from discriminant ◆ High leverage
this, et al. (2007) 1998–2003 (UK analysis, and ◆ Low growth
companies) logistic regression ◆ Low profitability
225 qualified and C4.5 decision tree, ◆ High credit risk (Z score)
225 unqualified multilayer percep- ◆ Low profitability
Kirkos et al.
audit opinions from tron (MLP), and
(2007) ◆ High leverage
1995–2004 (UK and Bayesian belief
Irish companies) network ◆ Low revenue
347 qualified and Multilayer percep- ◆ Low liquidity as measured by working capital
671 unqualified tron (MLP), ra- ◆ Low profitability as measured by net income
audit opinions from dial basis function per employee, EBIT margin, and pre-tax
Pourheydari et al. profit margin
2001–2007 (public (RBF), probabilis-
(2012)
companies listed tic neural network ◆ Higher values of days sales outstanding
on Tehran stock (PNN), and logistic ◆ High credit risk
exchange) regression ◆ Low growth
5
EJAE 2019  16 (2)  1-58
THE TWO?
708 qualified and

310 unqualified ◆ Low profitability (net profit per employee)
audit opinions from Support vector ◆ Small size (number of employees, sales)
Saif et al. (2012) 2001–2007 (public machine (SVM) ◆ Low growth
companies listed and decision trees ◆ High cash flow from investing activities rela-
on Tehran stock tive to revenue
exchange)
708 qualified and
310 unqualified
audit opinions from Multilayer percep- ◆ Low inventory turnover
Saif et al. (2013) 2001–2007 (public tron (MLP) and
companies listed decision trees ◆ High liquidity (current ratio)
on Tehran stock
exchange)
55 qualified and 55
unqualified audit ◆ Low liquidity
Discriminant
opinions from ◆ High leverage
Yasar et al. analysis, logistic
2010–2013 (public
(2015) regression, and ◆ Low productivity of operations
companies listed
C5.0 decision tree ◆ Low profitability
on Istanbul stock
exchange)
12 qualified and 293 ◆ High leverage
unqualified audit ◆ Low liquidity
Zdolšek et al.
opinions from 2009 Logistic regression
(2015) ◆ Low efficiency
(Slovenian compa-
nies) ◆ Low profitability
78 qualified and ◆ Small size

Probabilistic neu- ◆ Low liquidity
Fernández- 369 unqualified
ral network (PNN)
Gámez et al. audit opinions from ◆ Low profitability
and multilayer
(2016) 2008–2010 (Spanish
perceptron (MLP) ◆ Low solvency (EBITDA/total liabilities)
public companies) ◆ Low productivity
Table 1. Review of Studies Aimed at Modeling Qualified Audit Opinions
Characteristics of clients, both financial and non-financial, are the most important group of predic-
tors used in prior studies, which is expected, given that the main reasons for issuing a modified audit
opinion are materially misstated financial reports and substantial uncertainties related to the client’s
operations.3 Characteristics of audit firms are somewhat less frequently considered as predictors, with
the auditor’s membership of Big N being the most researched effect. Most studies have found that the
auditor’s membership of Big N is not predictive of qualifications (Kirkos et al., 2007; Krishnan and
Krishnan, 1996; Mutchler and Hopwood, 1997; Ruiz-Barbadillo et al., 2004).4
The main general conclusion, arising from the results of prior research, is that companies with poor
financial conditions (i.e. high leverage, low liquidity, high credit risk, small size, low efficiency, low
growth) get qualified audit opinions more frequently. There are several potential reasons for this. The
first is that poor financial prospects on their own, when inadequately disclosed as a going concern in
3 The potential reasons for issuing a modified audit opinion, in order of decreasing frequency, are: a) materially misstated
financial reports (reports do not conform to the international standards of financial reporting) or inadequate disclosure;
b) substantial uncertainties exist about the entity’s ability to continue operations (going concern) or some other important
aspect of client’s operations; c) auditor’s inability to obtain enough evidence to verify the information presented in financial
reports (scope limitation); and d) auditor’s lack of independence.
4 One study (Gaganis and Pasiouras, 2006) reports that the auditor’s membership of Big N decreases the chances of qualified
opinion.
6
EJAE 2019  16 (2)  1-58
THE TWO?
the financial statements, can be a reason for expressing a qualified or adverse opinion (IAASB ISA 570,
2013, para. 20). Secondly, as noted by Beneish (1999), companies facing poor prospects have greater
incentives for earnings manipulation. Empirical findings do not contradict this hypothesis, as more
earnings management cases are found to be directed towards overstatement than understatement of
financial result (Jones et al., 2008, p. 506; Kinney and McDaniel, 1989, p. 84). Thirdly, companies with
poor financial conditions may also have low-quality human resources in their accounting departments,
resulting in more errors. Finally, the auditors of poorly performing firms are more likely to decide that
contingencies of a given magnitude are material (Dopuch et al., 1987, p. 437), conceivably because of a
greater risk of stockholder lawsuits (Kinney and McDaniel, 1989, p. 72) and relatively low importance
that such clients have in their portfolio (Krishnan et al., 1996).
Regarding the modeling techniques being used, a shift is observable from the classic statistical tools,
such as probit and logistic regression, to more contemporary machine learning tools, such as artificial
neural networks, decision trees and k-nearest neighbors. It is acknowledged in the literature that the
predictive performance of specific classifiers varies widely across different types of classification task,
depending mostly on the characteristics of the data, such as size, dimensionality, and structure. From
the results of the prior research, it is difficult to conclude which set of tools is more effective for the task
of predicting the type of audit opinion. The key factors that hinder the comparability of the reported
classification accuracies across the studies are the following:
◆ Different outcome variables are used. Researchers use both qualified (Dopuch et al., 1987;
Krishnan and Krishnan, 1996) and modified (Francis and Krishnan, 1999) types of audit opinion
as the target category. Both categories are legitimate choices, but the classification accuracies
between the two cannot be directly compared.
◆ Different predictors are used. Most researchers use only financial metrics, but some use market
data (Dopuch et al., 1987; Francis and Krishnan, 1999; Krishnan and Krishnan, 1996), non-
financial data, or latent constructs (Fernández-Gámez et al., 2016) as well. Even when nominally
using the very same predictors, the calculations are likely to differ due to the differences in
national reporting frameworks.
◆ Samples from different countries and periods and of different sizes are used. The difficulty of
prediction is expected to vary across periods and countries. Also, the accuracies reported in the
studies that have used small samples may have been overestimated due to overfitting.
◆ Different model validation methods are employed. Results obtained using bootstrap replications
(Spathis et al., 2003; Zdolšek et al., 2015), k-fold cross-validation (Kirkos et al., 2007), and differ-
ently constructed hold-out samples (Gaganis, Pasiouras, and Doumpos, 2007; Pourheydari et al.,
2012) are not directly comparable. Some studies even report classification accuracy in training
set, which is known to be overoptimistic.
◆ Different measures of predictive performance are reported. Studies report both average (Doumpos
et al., 2005; Gaganis, Pasiouras, Spathis et al., 2007) and overall classification accuracy (Gaganis,
Pasiouras, and Doumpos, 2007; Kirkos et al., 2007; Pourheydari et al., 2012; Spathis et al., 2003;
Yasar et al., 2015), which are not directly comparable. Furthermore, as the proportions of types
of audit opinion vary across countries and periods, achieving a high overall predictive accuracy
is easier in countries and periods where one type of opinion is dominant — such as in a study
done by Zdolšek et al. (2015), where a sample from Slovenia with only a 3.93 percent share of
qualified opinions in all opinions is used, or in a study by Caramanis and Spathis (2006), where
7
EJAE 2019  16 (2)  1-58
THE TWO?
a sample from Greece with 87.57 percent of qualified opinions in all opinions is used — than
in those where the two types are more evenly distributed, such as in the study by Dopuch et al.
(1987), where the share of qualified opinions was 38.41 percent. No studies report the Kappa
metric, which corrects for the differences in proportions of types of audit opinion.
When we try to narrow down the focus to studies that have reported comparative predictive perfor-
mance of statistical and machine learning techniques on the same data sets, using the same outcome
variable and the same measure of accuracy, the following two studies appear to be the most relevant:
1. Gaganis, Pasiouras, and Doumpos (2007, p. 122) reported overall accuracies of 84.35 percent,
80.55 percent, and 86.14 percent achieved by probabilistic neural network (PNN), artificial neural
network (ANN) and logistic regression, respectively.
2. Pourheydari et al. (2012, p. 11086) reported overall accuracies of 87.75 percent, 84.81 percent,
78.44 percent, and 77.60 per cent achieved by multilayer perceptron (MLP), probabilistic neural
network (PNN), radial basis function (RBF) neural networks, and logistic regression, respectively.5
The results of the two studies are conflicting: logistic regression achieves higher accuracy in the first
study, while artificial neural networks are shown to be more accurate in the second. It should be noted,
however, that the confidence intervals for the classification accuracies have not been reported, meaning
that the possibility that the observed within-study differences are insignificant has not been ruled out.
Most importantly, even if the findings of the prior studies were consistent, we argue that the conclu-
sions drawn from such studies would not be definite, because the respective strengths of the two sets
of tools have not been fully exploited.
Specifically, the studies relying on statistical methods have missed the opportunity to model the
systematic effects of individual auditors and clients. Given that the differences in auditors’ propensities
to issue (e.g. auditor’s reporting conservatism) and clients’ propensities to receive (e.g. client’s pro-
pensity to manage earnings or the quality of its accounting department) a modified audit opinion are
expected to exist and persist over time, the failure to model the individual effects (i.e. to use the prior
information when available) has likely resulted in suboptimal predictive performance of the statistical
models. At the root of this suboptimal modeling choice there may be the unfortunate circumstance
that most studies — with the exception of a study by Doumpos et al. (2005) — have not differentiated
between cases where prior information on client and auditor (i.e. audit and/or financial reports from
prior periods) is available from cases where such information is not available.6 Differentiating between
these two real-life scenarios is extremely important for the task of prediction, from both a theoretical
and practical perspective. The inclusion of the systematic effects is not only expected to improve the
predictive performance of the statistical models, but also to yield unbiased estimates of the coefficients
with the explanatory variables.
5 Yasar et al. (2015) reported overall accuracies of 87.3 percent, 92.7 percent, and 98.2 percent, achieved by dis-
criminant analysis, logistic regression, and C5.0 decision tree, respectively. However, this study is disregarded,
since the reported accuracies are measured in different samples (in the training sample for the former two,
and in a test sample, comprising only 26 observations, for the third one). Also, Spathis et al. (2003, p. 278)
reported overall accuracies of 78.83 percent, 74.34 percent, and 74.70 percent based on 200 bootstrap replica-
tions, achieved by the multicriteria decision aid classification method (UTADIS), linear discriminant analysis,
and logistic regression, respectively. This study is omitted, however, since UTADIS is neither a statistical nor
machine learning technique.
6 A transparent and informative model validation technique was conducted by Doumpos et al. (2005, p. 211),
who reported overall accuracies of 84.58 percent, 84.84 percent, and 84.84 percent achieved by support vector
machines with linear, RBF, and quadratic kernel, respectively.
8
EJAE 2019  16 (2)  1-58
THE TWO?
On the other hand, the studies relying on machine learning techniques have failed to fully employ
their feature extraction capabilities. These studies have typically relied on the traditional theory-driven
metrics, developed for the task of financial analysis, instead of using all the available information from
the complete set of financial reports to boost the predictive performance (machine learning techniques
are expected to make relatively better use of the data available in the complete set of financial state-
ments, since they are designed to be capable of handling more predictors and modeling more complex
relationships) and extract novel, data-driven predictors that might be more relevant for the task of
predicting auditor opinions.
Finally, no study has made an attempt to combine the respective strong points of the two approaches
to construct a hybrid model that would make the best use of the data available while preserving the
desired level of parsimony and interpretability of the model.
Based on the review of prior research in the field, we recognize the need for a study which will: 1) de-
velop models from the realms of statistics and machine learning that fully exploit their respective strong
points, which are modeling of the systematic effects on one side, and flexibility and feature extraction
capabilities on the other; 2) consider the possibility of developing hybrid models that would integrate
the best of both worlds; 3) test the predictive performance of the models, while differentiating between
the setting where prior information on the client is available and the setting where prior information
is not available; 4) report confidence intervals for the predictive accuracies, in addition to their point
estimates; and 5) do everything in a reproducible manner, and without relying on the assumption that
long time series of historical reports or market data are available, to allow researchers and professionals
around the globe to build models and expert systems for the assessment of the probability of material
misstatements in financial reports that will suit their needs.
MATERIALS AND METHODS
Data
The data from 13,561 complete sets of annual financial statements for 4,701 companies are combined
with the data from the corresponding audit reports, forming an unbalanced panel data set. The client
companies included in the sample represent a supermajority of medium- and large-sized companies
registered in the Republic of Serbia. The information on the auditor firm name and the type of audit
opinion is hand-collected from the audit reports issued by 64 audit firms (Big 4 plus 60 other audit
firms), which, again, represents a supermajority of all the auditor firms registered in this country. To
the best of our knowledge, this is the largest data set used in the literature devoted to predicting the
type of audit opinion.
In the total sample of audit opinions (13,561), the following frequencies of the four main types of
audit opinions are observed: adverse opinion (71), disclaimer of opinion (644), qualified opinion (3,706),
and unqualified opinion (9,140). We present the absolute and relative frequencies of the specific types
of audit opinion by period in Table 2.
9
EJAE 2019  16 (2)  1-58
THE TWO?
Absolute Frequencies by Period Relative Frequencies by Period

2010 2011 2012 2013 2010 2011 2012 2013
Adverse opinion 9 15 22 25 0.27% 0.45% 0.60% 0.78%
Disclaimer of opinion 127 140 200 177 3.76% 4.24% 5.44% 5.52%
Qualified opinion 910 900 1,011 885 26.94% 27.26% 27.49% 27.62%
Unqualified opinion 2,332 2,246 2,445 2,117 69.03% 68.04% 66.48% 66.07%
Total 3,378 3,301 3,678 3,204 100.00% 100.00% 100.00% 100.00%
Table 2. Observed Frequencies for Each Type of Audit Opinion by Period
Since adverse opinions and disclaimers of opinion are relatively rare, we combine them with quali-
fied opinions to establish a ‘modified opinion’ category. The absolute and relative frequencies of the
modified and unmodified opinions are shown in the following Table 3.
Absolute Frequencies by Period Relative Frequencies by Period

2010 2011 2012 2013 2010 2011 2012 2013
Modified opinion 1,046 1,055 1,233 1,087 30.97% 31.96% 33.52% 33.93%
Unmodified opinion 2,332 2,246 2,445 2,117 69.03% 68.04% 66.48% 66.07%
Total 3,378 3,301 3,678 3,204 100.00% 100.00% 100.00% 100.00%
Table 3. Frequencies of Modified and Unmodified Audit Opinions by Period as Observed in the Sample
The observed frequency of the two main types of audit opinions by industry classification is pre-
sented in Appendix A.
In the complete data set, there are 9,140 (67.4%) unmodified and 4,421 (32.6%) modified audit
opinions, indicating no significant class imbalance.
The dependent variable in all the predictive models is the dichotomous variable that indicates the
type of audit report received by company i in period t. A value of 0 indicates an unmodified (unqualified)
audit opinion while a value of 1 indicates a modified (qualified, disclaimer, or adverse) audit opinion.
Given the above-mentioned definition of the dependent variable, we model the probability that
either: a) the auditor issues a disclaimer of opinion (i.e. withdraws) for a valid reason, which occurs in
around five percent of all cases in the sample; or b) the financial statements do not meet one or more
of the requirements for issuing an unqualified opinion, and hence contain either non-pervasive (for
a qualified opinion) or pervasive (for an adverse opinion) material misstatements, which occurs in
around 28 percent of all cases in the sample.7
7 The national Law on Auditing requires all auditors to apply the International Standards on Auditing, the International
Standard on Quality Control, and the related standards published by the International Auditing and Assurance Standards
Board of the International Federation of Accountants. Accordingly, an unqualified/unmodified audit opinion has the same
meaning as in all previous studies, and implies that the financial statements:
a) Have been prepared in accordance with IFRS;
b) Comply with relevant statutory requirements and regulations;
c) Provide adequate disclosure of all material matters, and
d) Provide adequate disclosure of any changes in the accounting principles or in the method of their application and the
effects thereof.
If any of the listed conditions is not met, a modified opinion is issued.
10
EJAE 2019  16 (2)  1-58
THE TWO?
All computations and modeling are conducted within the R software environment (R Core Team, 2017).
The Predictors
Initially, we consider a comprehensive set of theory-driven metrics, derived from prior studies.
In addition to that, to make a better use of quantitative data obtainable from clients’ complete sets of
financial statements, we design and conduct a feature generation procedure, and subsequently use the
feature extraction capabilities of machine learning algorithms to identify features (predictors) highly
relevant for the task of predicting auditor opinions.
Theory-Driven Predictors
Among the theory-driven predictors, there are 31 financial variables that, based on the previous
research, are expected to be associated with qualified audit opinion. The potentially relevant predictors
are classified into eight groups: 1) size (Francis and Krishnan, 1999; Kinney and McDaniel, 1989; Kirkos
et al., 2007); 2) profitability (Caramanis and Spathis, 2006; Dopuch et al., 1987; Francis and Krishnan,
1999; Kinney and McDaniel, 1989; Kirkos et al., 2007; Pourheydari et al., 2012); 3) liquidity (Caramanis
and Spathis, 2006; Pourheydari et al., 2012); 4) leverage (Dopuch et al., 1987; Francis and Krishnan,
1999; Kinney and McDaniel, 1989; Kirkos et al., 2007); 5) cash conversion cycles (Gaganis, Pasiouras,
and Doumpos, 2007; Pourheydari et al., 2012); 6) credit risk (Gaganis, Pasiouras, and Doumpos, 2007;
Kirkos et al., 2007; Pourheydari et al., 2012); 7) earnings quality/accruals (Bartov et al., 2000; Francis
& Krishnan, 1999), and 8) other (Gaganis, Pasiouras, and Doumpos, 2007).8 Details on the calculation
of the theory-driven predictors are presented in Table 4.
Predictor Group Predictor Name Calculation item ids (as assigned in the layout of chart of accounts) in superscript
Ln total assets Ln (operating assets022)
Size
Ln revenue Ln (operating revenue201)
Net result Net profit229 - Net loss230
EBIT Profit before tax223 - Loss before tax224 + Interest expenses667
EBITDA EBIT + Depreciation661
Operating result Operating profit213 - Operating loss214
Return on assets EBIT / Operating assets022
Return on equity Net result / Capital101
Profitability EBIT × (1 - Corporate tax rate [15%]) / (Capital101 + Long-term
Return on invested
liabilities113 + Short-term financial liabilities117 - Excess cash [any
capital
cash over 3% of operating revenue])
Operating result / (Operating assets022 - Long-term financial invest-
Return on capital in-
ments009 - Short-term financial investments018 - Excess cash [any
vested in core business
cash over 3% of operating revenue])
Operating margin Operating result / Operating revenue201
Net margin Net result / Operating revenue201
8 The group named ‘other’ includes an average net monthly salary (a proxy for employees’ quality and motiva-
tion level) and the percentage of foreign ownership, both of which have been calculated based on the data
presented in the financial statements.
11
EJAE 2019  16 (2)  1-58
THE TWO?
Net working capital Current assets012 - Current liabilities116

Net working capital to
Net working capital / Operating assets022
Liquidity assets
(Receivables016 + Receivables from overpaid corporate income tax017
Quick ratio
+ Cash and cash equivalents019) / Current liabilities116
Equity to total liabili- Capital101 / (Long-term provisions and liabilities111 + Deferred tax
ties liabilities123)
Leverage
(Long-term loans114 + Other long-term liabilities115 + Short-term
Debt ratio
financial liabilities117) / Operating assets022
Accounts receivables (sales)639 / Sales revenue202 × 365 / (1 + VAT
Days sales outstanding
rate [18%])
Days payable Operating liabilities (end of the year)640 / Operating liabilities (credit
outstanding turnover without opening balance)643 × 365
Cash
Conversion (Raw materials616 + Work in process617 + Finished goods618 + Mer-
Days sales of
chandise inventory619) / Operating liabilities (credit turnover with-
inventory
out opening balance)643 × 365 / (1 + VAT rate [18%])
Days sales outstanding - Days payable outstanding + Days sales
Cash conversion cycle
of inventory
Z’ = 0.717 × Net working capital to assets + 0.847 × (Retained
Z score for private profits108 / Operating assets022) + 3.107 × Return on assets + 0.420
companies × Equity to total liabilities + 0.998 × (Operating revenue201 / Op-
erating assets022)
Z score for non-man- Z’’ = 6.56 × Net working capital to assets + 3.26 × (Retained prof-
ufacturers and emerg- its108 / Operating assets022) + 6.72 × Return on assets + 1.05 × Equity
Credit Risk ing markets to total liabilities
- 4.336 - 4.513 × (Net income / Operating assets022) + 5.679 × ((Long-
Zmijewski score term provisions and liabilities111 + Deferred tax liabilities123) / Op-
erating assets022) + 0.004 × (Current assets012 / Current liabilities116)
- 6.307 × (Net result / Operating assets022) + 4.068 × ((Long-term
Shumway score provisions and liabilities111 + Deferred tax liabilities123) / Operating
assets022) - 0.158 × (Current assets012 / Current liabilities116)
Cash balance at the end of the period343 - Cash balance at the begin-
Free cash flow to
ning of the period340 + Dividends paid333 + Buy-up treasury shares
equity (FCFE)
and stakes330 - Increase in the capital stock326
Free cash flow to firm FCFE - Long-term and short-term borrowings (net inflows)327 +
Earnings (FCFF) (Interest paid308 × (1 - corporate tax rate [15%]))
Quality FCFE minus net result
(FCFE - Net result) / Operating revenue201
to revenue
Cash flow from opera- ((Net cash inflow from operating activities311 - Net cash outflow
tions minus operating from operating activities312) - (Operating result)) / Operating rev-
result to revenue enue201
(Share capital - % held by foreign investors624 + Limited liabil-
Percentage of foreign ity capital - % held by foreign investors626 + General and limited
ownership partnership capital - % held by foreign investors628) / Total paid
Other in capital633
Liabilities for net salaries and wages (credit turnover excluding
Average net monthly
opening balance)644 / Average number of employees based on end-
salary in EUR
of-the-month data605 / 12 × 1000
Table 4. Theory-Driven Predictors
12
EJAE 2019  16 (2)  1-58
THE TWO?
Descriptive statistics for the theory-driven predictors are presented in Appendix B (Table B1).
Data-Driven Predictors
In the second stage, we generate a set of data-driven predictors. One new predictor variable is calcu-
lated from each pair of 391 items available from the complete set of financial reports (i.e. balance sheet,
income statement, cash flow statement, statement of changes in equity, and statistical annex) by division.
After discarding redundant variables (reciprocals) and constants (variables divided by themselves),
the procedure yields 76,245 predictors which, when combined with the original 391 items, add up to a
total of 76,636 predictors. Since the calculation of features often involves a division, where the value of
the denominator (a numeric item from financial statements) is 0, the resulting values include positive
and negative infinite values. This problem is common in calculating theory-driven financial metrics,
but is even more pronounced in the case of data-driven features. The positive and the negative infinite
values are replaced with the maximum and minimum observed values within the matching predic-
tor. Furthermore, all the predictors with little or no variation (less than 95 percent unique values) are
discarded. After applying the described procedure, 1,515 data-driven predictors remain in the data set.
The combined data set, which includes both theory- and data-driven predictors, comprises a total
of 1,546 potentially relevant predictors.
Predictive Models
We build twelve predictive models. All the predictors are preprocessed using standardization
(centering and scaling). To facilitate a fair comparison of predictive performance, all the models that
need tuning are tuned in a consistent way, using 10-fold cross-validation and Area under the receiver
operating characteristic curve (AUROC) as the performance metric. The complete preprocessing,
model fitting and tuning, as well as comparative analysis of predictive performance are carried using
R’s ‘caret’ package (Kuhn, 2017).9 The information on the predictive models used, including software
implementation which is used and the parameters that need to be tuned, are presented in Table 5.
Method Software Implementation Tuning Parameters

trials (the number of boosting iterations)
model (logical: should the tree be decomposed
R’s package ‘C50’
C5.0 into a rule-based model?)
(Kuhn and Ross, 2017)
winnow (logical: should feature selection be
used?)
R’s package ‘randomForest’ (Liaw mtry (number of variables randomly sampled as
Random Forest
and Wiener, 2002) candidates at each split)
mtry (number of variables randomly sampled as
Regularized Ran- R’s package ‘RRF’ (Deng, 2013; candidates at each split)
dom Forest Deng and Runger, 2012, 2013) coefReg (the coefficient(s) of regularization)
coefImp (importance coefficient)
9 Only the training data set is used for model tuning. The data are preprocessed separately for tuning and evaluation. Each
model is tuned for the two training sets separately.
13
EJAE 2019  16 (2)  1-58
THE TWO?
n.trees (number of trees used in the prediction)

interaction.depth (the maximum depth of vari-
able interactions)
Stochastic Gradient R’s package ‘gbm’
Boosting (Ridgeway, 2017) shrinkage (a shrinkage parameter applied to each
tree in the expansion)
n.minobsinnode (minimum number of observa-
tions in the trees terminal nodes)
nrounds (the max number of iterations)
max_depth (maximum depth of a tree)
eta (the learning rate)
gamma (minimum loss reduction required to
Extreme Gradient R’s package ‘xgboost’ make a further partition on a leaf node of the tree)
Boosting (Chen et al., 2017) colsample_bytree (subsample ratio of columns
when constructing each tree)
min_child_weight (minimum sum of instance
weight [hessian] needed in a child)
subsample (subsample ratio of the training instance)
K-Nearest R’s package ‘class’ (Venables and
k (number of neighbors considered)
Neighbors Ripley, 2002)
Multilayer R’s package ‘RSNNS’ (Bergmeir
size (number of units in the hidden layer(s))
Perceptron and Benitez, 2012)
Support Vector
Machine with R’s package ‘kernlab’ (Karatzo- sigma (inverse kernel width)
Radial Basis glou et al., 2004) C (cost of constraints violation)
Function Kernel
Linear Discriminant R’s package ‘MASS’ (Venables
None
Analysis and Ripley, 2002)
R’s package ‘stats’ (R Core Team,
Logistic Regression None
2017)
R’s package ‘stats’ (R Core Team,
Probit Regression None
2017)
Mixed-Effects R’s package ‘brms’ (Bürkner,
None
Logistic Regression 2017)
Table 5. Information on the Predictive Models
Seven of the twelve listed models – namely, C5.0, k-nearest neighbors, multilayer perceptron, sup-
port vector machine, linear discriminant analysis, logistic regression, and probit regression — have
been used for the task of predicting the type of audit opinion in prior studies, while the remaining five
— random forest, regularized random forest, stochastic gradient boosting, extreme gradient boosting,
and mixed-effects logistic regression — are new to this stream of literature.
We briefly explain the rationale for using mixed-effects logistic regression as a predictive model
in this study. Firstly, using a mixed-effects logistic regression framework with the company- and
auditor-specific effects included as random effects allows the prior information on client’s propensity
to receive, and auditor’s propensity to issue, a modified opinion to be effectively accounted for when
available, which is expected to result in a significant improvement in classification accuracy, especially
in the out-of-time sample which comprises clients represented in the training sample. It also allows
the model to provide predictions for instances with unseen (new) companies and auditors, which
14
EJAE 2019  16 (2)  1-58
THE TWO?
would be problematic were logistic regression with fixed effects used instead. Mixed-effects logistic
regression also demonstrated higher accuracy than fixed-effects logistic regression (the results of the
fixed-effects models are discussed in the results section), indicating that regularization of the individual
effects, conducted within the random-effect framework, improves model performance. The output of a
preliminary variance components model (a model with no predictors included) indicates a significant
variability in both company- and auditor-specific effects, justifying the described modeling choice.
The dichotomous dependent variable is modeled using Bernoulli distribution and the logit link
function:
1
Modified ijt ~ Bernoulli ( ), where
1 + exp (- xijt )
k
xijt = α + ui + u j + ∑ Predictornit × β n + ei jt
n =1
In the equation above, α is the population-level intercept, ui and uj are company- and auditor-
specific varying intercepts respectively, β n is the regression coefficient with nth predictor, and ei jt is the
error term. The model has been fitted within the Bayesian framework, with priors on all fixed effects
set to a Gaussian N(0,1) distribution.
Model Validation Procedure
To assess the classification accuracy of the described predictive models, we employ a method similar
to that described by Doumpos et al. (2005, p. 210). The combined data set is first split into two parts:
a) the first set, comprising two thirds (66.66 percent) of the companies, randomly selected; and b) the
second set, comprising the remaining third (33.33 percent) of the companies in the sample. Next, the
first set is split into a training set, including the observations from the period 2010–2012, and an out-of-
time test set (OOT), which includes only the observations from the year 2013. Lastly, the observations
from the second set from the year 2013 are used to form the out-of-sample and out-of-time test set
(OOS and OOT). The described sub-setting procedure is illustrated in Figure 1.
Figure 1. Sub-Setting the Data

15
EJAE 2019  16 (2)  1-58
THE TWO?
Using model validation terminology, both test sets are out-of-time, meaning that they comprise
only the observations from later periods than those represented in the training set. The purpose of the
out-of-time (or temporal) validation is to emulate the typical situation that occurs in practice. That
is to say, classification models are trained on historical data yet need to be applied in a new reporting
period, which may be different from the observed periods in multiples respects, such as typical levels
of financial metrics (economic cycles), firms’ propensities for earnings manipulation, new accounting
or auditing standards and regulations, new techniques and patterns of earnings manipulation, etc. In
contrast to that, using the whole data set to train the classification models and employing a standard
cross-validation procedure to assess their accuracy would result in overly optimistic estimates of accu-
racy, as the classification models would be allowed to learn from the information presented in financial
and audit reports from the current reporting period, and those reports are typically not available at the
beginning of a new audit season.
While being out-of-time, the first test set comprises the same companies that are represented in the
training set. As having historical data on the client (auditor reports and financial metrics from previous
periods) is a significant advantage that can be efficiently used by some models to achieve improved
classification accuracy, that test set is employed for assessing the expected model classification accuracy
in the setting when auditor reports and financial metrics from previous periods for the client under
consideration are available.
On the other hand, in addition to being out-of-time, the second test set is also out-of-sample, in
the sense that it exclusively comprises the observations from companies that are not represented in
the training set. Consequently, this test set is used to assess the expected classification accuracy of the
models in the upcoming periods in the setting when no prior information on the client is available
(e.g. a newly established company).
Both scenarios described above occur in real life, and since classification accuracies are expected
to differ significantly between the two, it is valuable to have a realistic assessment of the expected clas-
sification accuracy for both.
RESULTS
Comparative Analysis of the Predictive Performance of the Individual Models
Table 6 summarizes the comparative performance of all the twelve predictive models in both test
sets. Because of the differential baseline (no information) rates in the two test samples (0.6704 vs.
0.6413), in addition to classification accuracy at 50% cut-off, for each predictive model, we report the
values of Cohen’s Kappa (a chance-corrected measure of proportion agreement; for details, see Cohen,
1960) with the corresponding 95% confidence intervals at 50% cut-off, and the AUROC metric that
summarizes overall model performance over all possible cut-offs. For performance comparison, we
primarily rely on AUROC.
16
With Theory-Driven Predictors With Theory- and Data-Driven Predictors
Out-of-time Out-of-sample and out-of-time Out-of-time Out-of-sample and out-of-time
(No information rate: 0.6704) (No information rate: 0.6413) (No information rate: 0.6704) (No information rate: 0.6413)
Accuracy at 50% Accuracy at 50% Accuracy at 50% Accuracy at 50%
Kappa cut-off / Kappa cut-off / Kappa cut-off / Kappa cut-off /
Model
(95%CI) Area under the (95%CI) Area under the (95%CI) Area under the (95%CI) Area under the
curve curve curve curve
0.4150 0.7691 / 0.3824 0.7455 / 0.4718 0.7831 / 0.4043 0.7465 /
C5.0
(0.3697–0.4602) 0.7910 (0.3189–0.4459) 0.7733 (0.4293–0.5143) 0.8172 (0.3429–0.4657) 0.7702
0.4405 0.7784 / 0.3619 0.7390 / 0.4894 0.7962 / 0.4052 0.7549 /
Random Forest
(0.3960– 0.4849) 0.8124 (0.2975–0.4264) 0.7754 (0.4467–0.5322) 0.8526 (0.3425–0.4679) 0.7869
Regularized Random 0.4377 0.7784 / 0.3476 0.7305 / 0.5020 0.7957 / 0.4109 0.7502 /
Forest (0.3930–0.4824) 0.8130 (0.2831–0.4121) 0.7704 (0.4603–0.5436) 0.8436 (0.3496–0.4722) 0.7907
Gradient Boosting 0.3988 0.7644 / 0.3540 0.7371 / 0.4457 0.7714 / 0.3909 0.7380 /
Machine (0.3529–0.4446) 0.7903 (0.2890–0.4190) 0.7737 (0.4026–0.4889) 0.8052 (0.3295–0.4523) 0.7800
Extreme Gradient Boost- 0.3920 0.7620 / 0.3540 0.7371 / 0.5022 0.7901 / 0.3650 0.7183 /
ing (0.3460–0.4382) 0.7895 (0.2890–0.4190) 0.7708 (0.4613–0.5431) 0.8215 (0.3041–0.4259) 0.7393
0.3893 0.7466 / 0.2825 0.6948 / 0.3716 0.7508 / 0.2090 0.6714 /
K-Nearest Neighbors
(0.3402–0.4297) 0.7627 (0.2174–0.3475) 0.6931 (0.3253–0.4178) 0.7867 (0.1411–0.2769) 0.6673
0.3515 0.7499 / 0.2808 0.7136 / 0.3365 0.7489 / 0.2744 0.7080 /
Multilayer Perceptron
(0.3039–0.3991) 0.7885 (0.2127–0.3490) 0.7574 (0.2879–0.3850) 0.7808 (0.2066–0.3423) 0.7485
0.2987 0.7396 / 0.2379 0.6995 /
Support Vector Machine
(0.2487–0.3488) 0.7474 (0.1680–0.3077) 0.7269
Linear Discriminant 0.2914 0.7354 / 0.2615 0.7108 /
Analysis (0.2413–0.3414) 0.7717 (0.1919–0.3310) 0.7642
0.3166 0.7443 / 0.2855 0.7183 /
Logistic Regression Models did not converge because of the large number of predictors
(0.2672–0.3660) 0.7759 (0.2170–0.3540) 0.7657
0.3011 0.7396 / 0.2759 0.7155 /
Probit Regression
(0.2512–0.3511) 0.7759 (0.2070–0.3449) 0.7696
Mixed-Effects Logistic 0.5779 0.8233 / 0.3053 0.7221 /
Regression (0.5393–0.6165) 0.8818 (0.2381–0.3726) 0.7597
THE TWO?
EJAE 2019  16 (2)  1-58
Table 6. Comparative Predictive Performance of the Twelve Predictive Models
17
EJAE 2019  16 (2)  1-58
THE TWO?
The first thing to be noted is that classification performance indeed differs meaningfully between
the two scenarios. Regardless of the technique being used, it is easier to achieve better performance in
the out-of-time than in the out-of-sample and out-of-time test set. Even the machine learning tech-
niques, which do not explicitly account for the systematic effects present in the data, achieve better
performance in the out-of-time test set than in the out-of-sample and out-of-time test set, which can
primarily be attributed to overfitting.
Another key insight is that the comparative performance of classifiers depends heavily on the avail-
ability of prior information. When prior information is available, mixed-effects logistic regression ef-
fectively accounts for it, providing the most accurate predictions. In the absence of prior information,
the machine learning algorithms — particularly the tree-based ones such as random forest, regularized
random forest, gradient boosting machine, and C5.0 — provide more accurate predictions by making
relatively better use of predictors.
We proceed to examine the significance of the improvements produced by the addition of data-
driven predictors. While statistical techniques, which are not designed to handle such a large number
of predictors, report problems with model convergence, the machine learning algorithms, particularly
the tree-based ones, seem to use the abundance of data-driven predictors effectively to achieve better
classification performance. To compare the performance of the machine learning models using theory-
driven predictors with the corresponding models using both theory- and data-driven predictors, we use
one-tailed bootstrap tests.10 According to the bootstrap test, the addition of the data-driven predictors
gives significant improvement in predictive performance for all the machine learning models except for
multilayer perceptron where out-of-time prediction is concerned. The finding that the use of data-driven
predictors improves the performance of machine learning algorithms for out-of-time prediction indicates
that data-driven predictors can add value in the process of classification. Predictor importance metrics,
obtained from the random forest algorithm, confirm the relevance of data-driven predictors. The following
lists the ten most important predictors as per the mean-decrease-in-Gini (or Gini importance) criterion:
1. Cash inflow from operating activities / Long-term provisions and liabilities.
2. Liabilities for contributions on salaries and wages paid by the employee (credit turnover without
opening balance) / Salaries, salary compensations, and other benefits to employees.
3. Total cash outflow / Current liabilities.
4. Total cash outflow / Long-term provisions and liabilities.
5. Total cash inflow / Long-term provisions and liabilities.
6. Operating revenue / Current liabilities.
7. Net result.
8. Total cash inflow / Current liabilities.
9. Sales and prepayments / Long-term provisions and liabilities.
10. Cash outflow from operating activities / Long-term provisions and liabilities.
The fact that only one theory-driven predictor is among the top ten most important predictors (i.e.
net result, ranking seventh) confirms our belief that the theory-driven predictors commonly used in the
literature, such as current ratio, quick ratio, conversion cycles, and debt ratio, are not the most relevant
predictors of auditor qualifications and that the data provided in the full set of financial statements can
be better used to achieve improved predictive performance.
10 Comparison using the DeLong test (DeLong et al., 1988) of AUCs for nested models developed and validated on the same
data is problematic and often leads to overly conservative results (Demler et al., 2012).
18
EJAE 2019  16 (2)  1-58
THE TWO?
On the other hand, for out-of-sample and out-of-time predictions, using data-driven predictors
improves performance for only the regularized random forest model, worsens the performance of the
extreme gradient boosting model, and does not seem to affect the performance of the remaining models.
To conclude, where individual predictive models are concerned, mixed-effects logistic regression is
the most effective technique for making out-of-time predictions, whereas other methods — primarily
the tree-based machine learning algorithms such as random forest, regularized random forest, gradient
boosting machine, and C5.0 — achieve the most accurate out-of-sample and out-of-time predictions.11
While the addition of data-driven predictors is shown to be advantageous for some models for out-
of-time prediction, for the best performing models listed above the addition of data-driven predictors
is either unfeasible (mixed-effects logistic regression) or does not result in a statistically significant
improvement of the predictive performance (machine learning algorithms). Therefore, these models
should be trained using theory-driven predictors only.
Hybrid Models
In view of the finding that machine learning and statistical techniques have their own advantages
with regard to predicting the type of audit opinion, we proceeded to specify two hybrid models aimed
at combining the respective strengths of the two approaches within a single predictive model.
Stacked Ensemble
The first hybrid model is a stacked ensemble, aimed at achieving the highest possible predictive
performance. According to the standard procedure for building ensemble learners, a meta-learner is
fitted using the out-of-bag predicted class probabilities estimated by seven different machine learning
algorithms and consequently used to generate predictions in the two test sets. Nevertheless, besides
using a regular logistic regression as a meta-learner, we used a mixed-effects logistic regression, which
is expected to better optimize the values of the parameters in view of the panel structure of the data.
The modified version of the stacked ensemble is of the specification:
xijt =α + ui + u j + β1 × C.50 probit + β 2 × RF probit + β 3 × RRF probit + β 4 × GBM probit

+ β 5 × XGBOOST probit + β 6 × KNN probit + β 7 × MLP probit + eijt
The comparative predictive performance of stacked ensembles using regular logistic regression and
mixed-effects logistic regression is presented in Table 7. The complete output of the hybrid models is
presented in Appendix D.
11 This claim is supported by the results of the principal component analysis presented in Appendix E (see Table E3).
19
EJAE 2019  16 (2)  1-58
THE TWO?

Out-of-Sample and Out-of-Sample and
Out-of-Time Out-of-Time
(No information (No information
(No information (No information
rate: 0.6704) rate: 0.6704)
rate: 0.6413) rate: 0.6413)
Accuracy Accuracy Accuracy Accuracy
at 50% at 50% at 50% at 50%
Kappa cut-off / Kappa cut-off / Kappa cut-off / Kappa cut-off /
Model
(95%CI) Area (95%CI) Area (95%CI) Area (95%CI) Area
under the under the under the under the
curve curve curve curve
Stacked
Ensemble 0.4556 0.3725 0.5538 0.3867
0.7821 / 0.7390 / 0.8144 / 0.7371 /
with Logistic (0.4119– (0.3091– (0.5142– (0.3251–
0.8207 0.7692 0.8628 0.7772
Regression as 0.4993) 0.4359) 0.5934) 0.4485)
Meta-Learner
Stacked
Ensemble 0.6060 0.3910 0.6134 0.4340
with Mixed- 0.8350 / 0.7455 / 0.8340 / 0.7540 /
Effects Logistic (0.5684– 0.8872 (0.3284– 0.7858 (0.5767– 0.8925 (0.3744– 0.7975
Regression as 0.6435) 0.4536) 0.6502) 0.4934)
Meta-Learner
Table 7. Performance of Stacked Ensembles
The results indicate that the stacked ensemble that uses the mixed-effects logistic regression model
as the meta-learner may achieve classification performance better than any individual classifier in both
test samples (the difference is statistically tested later in this study). This supports the view that in order
to achieve the best predictive performance, strong points of both machine learning and statistical tools
need to be fully employed in a single predictive model. For this model, the use of data-driven predictors
appears to be advantageous, as it leads to slight improvements in classification performance, significant
at α = 0.1, for both out-of-time prediction and out-of-sample and out-of-time prediction (one-tailed
bootstrap test reported p-values of 0.06126 and 0.08471, respectively).
Combining Feature-Selection Capabilities of Tree-Based Models with Mixed-Effects

Logistic Regression
Despite achieving a high predictive performance, it can be argued that the models presented so
far lack interpretability. Since some researchers and practitioners interested in predicting the type of
audit opinion may be willing to trade some of the accuracy for improved interpretability, our second
hybrid modeling strategy was aimed at optimizing the interpretability/accuracy trade-off rather than
maximizing predictive accuracy. To achieve a good interpretability/accuracy trade-off, we have com-
bined the feature extraction capabilities of the tree-based classification algorithms with the capability
of accounting for the systematic (auditor- and company-specific) effects inherent in mixed-effects
logistic regression. Firstly, in consideration of the typological redundancy in the composition of the
most important data-driven predictors, as indicated earlier by random forest’s predictor importance
metric, we have employed the guided regularized random forest (GRRF) algorithm developed by Deng
and Runger (2012) to extract a non-redundant set of classification rules. In the GRRF algorithm, the
20
EJAE 2019  16 (2)  1-58
THE TWO?
relative importance of each predictor is initially assessed using the ordinary random forest algorithm.
The variable importance metric is subsequently used to guide the feature selection process (hence the
term ‘guided’) in the regularized random forest algorithm (Deng and Runger, 2012). The product of
this procedure is a compact and non-redundant set of highly interpretable classification rules, presented
in Table 8. R’s RRF package (Deng, 2013) has been used to build RF and GRRF classifiers; the inTrees
(Deng, 2014) package has been used for the extraction of the classification rules.
Rule Modified Rule

Rule Specification
No. Opinion Interpretation
1 Total cash inflow / Long-term provisions and liabilities > 1.2687 No Solvency
Liabilities for contributions on salaries and wages paid by the em-
ployee (credit turnover without opening balance) / Salaries, salary
2 No Liquidity
compensations, and other benefits to employees (cash outflows) ≤
0.1558
3 Operating revenue / Current liabilities > 2.0425 No Liquidity
4 Total cash inflow / Long-term provisions and liabilities ≤ 0.4124 Yes Solvency
Table 8. The Data-Driven GRRF Classification Rules
Descriptive statistics for the variables used in the GRRF rules are presented in Appendix B (Table B2).
These four classification rules can be further combined with other predictive techniques without
any loss of interpretability. To optimize the weights assigned to the classification rules with regard to
the panel structure of the data, we included the rules in the form of the following dummy variables
into a mixed-effects logistic regression:
 
 
 Total cash in flow it 
GRRF rule1it = 0 if > 1.2687 
 Long-term provision and liabilitiesit 
 1, otherwise 
 Liabilities for contributions on salaries and wages paid by the employee 

 
 (credit turnover without opening balance)it 
GRRF rule2it = 0 if ≤ 0.1558
 Salaries, salary compensations and other benefits to employees (cash outflows)it 
 0, otherwise 
 Operating revenueit 
0if > 2.0425
GRRF rule3it =  Current liabilitiesit 
 
 1, otherwise 
 Total cash inflow it 

1 if ≤ 0.4124 
GRRF rule4it =  Long-term provisions and liabilitiesit 
 
 0, otherwise 
21
EJAE 2019  16 (2)  1-58
THE TWO?
This resulted in a model of the following specification:
xijt =α + ui + u j + β1 × GRRF rule1it + β 2 × GRRF rule2it + β 3 × GRRF rule3it + β 4 × GRRF rule4it + eijt
Since the dummy variables are coded in such a way that the rule-based classification outcomes in-
dicating an increased risk of a modified opinion are assigned the code 1, and the outcomes indicating
a lower risk of a modified opinion are assigned the code 0, all the regression coefficients are positive,
and their exponentiated values indicate an increase in the odds of receiving a modified audit opinion
associated with the unfavorable outcomes of the corresponding classification rules. The regression
coefficients are presented in Table 9.
Model Parameters Estimate Std. error z value Pr(>|z|)

Intercept -2.8638 0.2437 -11.752 < 2e-16 ***
Mixed-effects Logistic Rule 1 1.2357 0.1718 7.191 6.45e-13 ***

Regression with Outcomes on
Rule 2 1.1177 0.1350 8.279 < 2e-16 ***
the GRRF Classification Rules
Included as Predictors Rule 3 1.2894 0.1518 8.492 < 2e-16 ***
Rule 4 1.3942 0.2169 6.428 1.29e-10 ***
Table 9. Summary of M-ELR with Outcomes on the GRRF Rules Included as Dummy-Coded Predictors
The results of this model demonstrate that the client’s poor solvency and poor liquidity are the main
indicators of an increased risk of receiving a modified opinion.
Poor solvency of the client, as indicated by a low value (less than or equal to 1.27) of the ratio of total
cash inflow to long-term provisions and liabilities, increases the odds of receiving a modified opinion
3.44 times. A further drop in the value of this ratio (to less than or equal to 0.41) implies an additional
four-fold (4.03) increase in the odds of receiving a modified opinion.
The ratio of yearly credit turnover of the account liabilities for contributions on salaries and wages
paid by the employee to total yearly cash outflows for salaries, salary compensations, and other benefits
to employees is identified as highly indicative of the client’s liquidity. Specifically, when the value of this
ratio rises to more than 0.16 — which is roughly the proportion of the said contributions in the total
salary costs — this indicates that salaries are being calculated but not regularly paid out to employees,
leading to a three-fold increase in the odds of the client receiving a modified opinion. Another sign
of the client’s impaired liquidity is the ratio of operating revenue to current liabilities dropping below
2.04, which is associated with a 3.63-fold increase in the odds. Predictive performance of this model
is presented in Table 10.
22
EJAE 2019  16 (2)  1-58
THE TWO?
Out-of-Sample and
(No information rate: 0.6704) (No information rate:
0.6413)
Accuracy at Accuracy at
Kappa 50% cut-off / Kappa 50% cut-off /
Model
(95%CI) Area under the (95%CI) Area under the
curve curve
Mixed-Effects Logistic Regression 0.5961 0.3092

0.8275 / 0.7108 /
with Data-Driven Predictors (0.5586– (0.2442–
0.8702 0.7126
Selected by GRRF Algorithm 0.6336) 0.3743)
Table 10. Performance of Mixed-Effects Logistic Regression with Predictors Defined by GRRF Algorithm
The results indicate that where out-of-time predictions are concerned, a good trade-off can be
achieved, since mixed-effects logistic regression, which used only four data-driven predictors defined
by the GRRF algorithm, was only slightly less precise (0.8702 vs. 0.8818) than the mixed-effects logis-
tic regression that used 31 theory-driven covariates. Even trading the added precision of the stacked
ensemble with mixed-effects logistic regression as a meta-learner (0.8702 vs. 0.8925), for the parsi-
mony and the unmatched interpretability of the second hybrid model, may be justified in some cases.
Another noteworthy advantage of the hybrid model is that it converges rapidly without preprocessing
(standardizing) the predictors.
In contrast, achieving a good model interpretability while maintaining good performance is much
more difficult where out-of-sample and out-of-time predictions are concerned, because in this case
models cannot rely on the use of prior information but can achieve good performance only by effectively
modeling the complex relationships present in the data. Consequently, the added interpretability of
the second hybrid model does not compensate for the drop in predictive performance in the out-of-
sample and out-of-time test set.
Summary of the Relative Performance of the Predictive Models
To formally test the differences in the observed performance of the predictive models presented
in this study, we perform pairwise comparisons of the AUROC values using the method described by
DeLong et al. (1988) for all models developed using the same predictors and tested on the same test
sets (results shown in Appendix C). As a brief summary of the relative performance of the predictive
models, we list the best few performing models (according to the value of the AUROC metric, based
on the DeLong test for differences in AUROC) in Table 11.
23
EJAE 2019  16 (2)  1-58
THE TWO?

Out-of-Sample and Out-of-Sample and
1. Stacked Ensemble
with Mixed-Ef-
fects Logistic Re-
gression as Meta-
Learner
Models for which 1. Stacked Ensemble
the predictive 2. Random Forest with Mixed-Effects
1. Stacked Ensemble
performance is with Mixed-Ef- 3. Gradient Boosting Logistic Regression
not statistically fects Logistic Re- Machine 1. Stacked Ensemble as Meta-Learner
significantly lower gression as Meta- 4. C5.0 with Mixed-Effects 2. Regularized Random
(α = 0.05) than that Learner Logistic Regression Forest
5. Extreme Gradient
of any other model as Meta-Learner
2. Mixed-Effects Lo- Boosting 3. Random Forest
with the same pre-
dictors and for the gistic Regression 6. Regularized Ran- 4. Gradient Boosting
same test sample dom Forest Machine
7. Probit Regression
8. Logistic Regression
9. Linear Discrimi-
nant Analysis
Table 11. The Best Performing Models According to the AUROC Statistic by Type of Predictors Used and by Test Set
The results clearly indicate that the stacked ensemble with mixed-effects logistic regression as
meta-learner is the predictive model with the best overall performance in both test sets, regardless of
the predictors used. As stated earlier, for this model, the use of data-driven predictors leads to slight
improvements in classification performance, significant at α = 0.1, for both out-of-time prediction and
out-of-sample and out-of-time prediction.
Alternative Modeling Strategies Considered in This Study
Alternative modeling options that have been considered in our study are:
a) Modeling the systematic effects of clients and auditors as fixed rather than random, in a logistic
regression model;12
b) Inclusion of dummy variables for identification of auditor firms, aimed at helping methods other
than mixed-effects logistic regression to model the related systematic effects;
c) Use of principal components extracted from the complete set of theory- and data-driven covariates
as predictors in the models, and
d) Informing methods other than mixed-effects logistic regression on prior audit opinions for the client
using lagged variables (applicable only for OOT prediction).
The results of the three above-listed modeling options are shown in Appendix E. The first and the
third options did not result in an improved performance as measured by the AUROC metric. The
second option improved the predictive performance of certain machine learning methods (namely,
12 A model with fixed effects for both companies and auditors was fitted and used for OOT prediction, whereas a model with
fixed effect only for auditors was fitted and used for OOS and OOT prediction). To make the prediction viable, we had to
remove the auditors that were not present in the training sample from the test samples, which resulted in a reduced sample
size (a reduction from 2,139 to 1,951 observations in OOT and from 1,065 to 976 observations in OOS and OOT).
24
EJAE 2019  16 (2)  1-58
THE TWO?
C5.0, random forest and regularized random forest) for OOT prediction, but not to a significant degree;
anyhow, this also seems to lead to severe overfitting problems in other methods (gradient boosting
machine, K-nearest neighbors, and multilayer perceptron); moreover, it renders prediction unfeasible
for the remaining methods (extreme gradient boosting, support vector machine, and linear discrimi-
nant analysis) when the auditor identification dummies from the training set that have become part of
the classification rules are missing in the test set. For these reasons, none of the first three alternative
modeling options seems to be advantageous for achieving an improved classification performance.
The fourth modeling option has been shown to significantly improve the predictive performance
of the models other than mixed-effects logistic regression but has not been included in our primary
analysis since it is known to introduce bias in models. As the circumstance that the predictive models,
other than MELR, has not provided information on prior audit opinions, is an important aspect of
our study that needs to be carefully considered when interpreting the results presented in our primary
analysis, we will briefly explain why this was the case, how this circumstance affected the compara-
tive performance of the predictive models presented so far in the study, and how this methodological
limitation can be overcome in future studies.
First, it should be noted that the first 11 algorithms listed in Table 5 were trained using theory- and
data-driven variables, as described and listed in sections 3.2.1. and 3.2.2; on the other hand, however,
the MELR models (as implied by the model specification presented in section 3.3) were trained using
the same sets of variables along with a set of dummy variables identifying auditors and companies. The
main reason for the use of different sets of predictors is the fact that the original data set is unbalanced.
Namely, when the data set is unbalanced, MELR incorporates the prior information by modeling the
client-specific systematic effects based on the identifier variables. Informing algorithms other than
MELR of prior opinions is, nonetheless, not trivial; technically, this can be achieved via the inclusion
of dummy identifiers for auditors and companies among the predictors. This method does not result
in an improved classification performance since the algorithms other than MELR are either incapable
of taking advantage of a large number of predictors that carry information pertaining to a tiny subset
of the sample (i.e. tree-based algorithms may either omit the dummy identifiers from the rules and lose
valuable information, or include them in the rules in an unregularized manner, resulting in overfitting)
in the case of machine learning algorithms, or are incapable of handling such a large number of predic-
tors in general, in the case of statistical tools. One alternative method for informing the algorithms on
the prior audit opinions is the inclusion of two lagged variables (i.e. Modified_t-1 and Modified_t-2)
among the predictors. Due to the unbalanced structure of the dataset, however, the inclusion of the
lagged variables results in numerous missing observations for these variables.13 Moreover, the observa-
tions on the lagged variables are not missing completely at random (MCAR), but are rather missing at
random (MAR), meaning that the propensity for an observation to be missing is related to the observed
data (e.g. the propensity of the client not having prior opinions available for both preceding years is
related to the size of the client, as measured by their revenue and total assets, because the client’s size is
an important criterion for determining whether the external audit is obligatory for the client). The MAR
pattern of missingness has implications for the modeling options. Namely, whilst the MELR analysis
is robust and fully functional under the MAR assumption of missingness, providing unbiased results
13 Specifically, without considering the lagged variables, there are 6,950 complete observations (4,743 unmodified and 2,207
modified opinions) in the training set. After adding the first lagged variable (Modified_t-1), 4,077 complete observations
(2,896 unmodified, 1,181 modified and 2,873 missing values) are left in the training set. After adding the second lagged
variable (Modified_t-2), 1,779 observations (1,311 unmodified, 468 modified and 5,171 missing values) remain in the
training set. The size of the OOT test set reduces as well – from 6,248 to 3,124 observations.
25
EJAE 2019  16 (2)  1-58
THE TWO?
(Baayen et al., 2008; Gibbons et al., 2010, p. 1) due to the implicit imputation that internally takes place
in these types of models (Ashbeck and Bell, 2016, p. 7), the remaining algorithms considered in our
research require the use of specific procedures for dealing with missing data before becoming functional.
The methods commonly proposed in the literature are complete-case analysis, the missing-indicator
method, and the missing data imputation method. Even though these methods make the algorithms
functional, most of them, when applied to MAR data, result in the decreased efficiency of the analysis
and/or biased predictive models. Complete-case analysis decreases the sample size considerably, thus
reducing the efficiency of the analysis, and has been shown to introduce bias when data is not MCAR
(see e.g. Pedersen et al., 2017, p. 159; Schafer and Graham, 2002, pp. 155–162). Furthermore, the re-
sulting model is not applicable for predictions in cases where prior data are incomplete (where either
Modified_t-1 or Modified_t-2 are missing). Similarly, the missing-indicator and single imputation
methods are both expected to introduce bias in the presence of MAR patterns (see e.g. Pedersen et al.,
2017, p. 159; Schafer and Graham, 2002, pp. 155–162).
Table 12 shows the predictive accuracies of the models for OOT prediction when lagged variables
are added to the set of predictors and the three aforementioned methods for dealing with missing data
are employed.
26
Out-of-Time Out-of-Time Out-of-Time Out-of-Time
(No information rate: (No information rate: (No information rate: (No information rate:
(No information rate: (No information rate:
0.6704) 0.6704) 0.6704) 0.6704)
0.6665) 0.6665)
Missing-indicator Single KNN imputa- Missing-indicator Single KNN imputa-
Complete case analysis Complete case analysis
method tion method tion
Accuracy Accuracy Accuracy Accuracy Accuracy Accuracy
at 50% at 50% at 50% at 50% at 50% at 50%
Kappa cut-off / Kappa cut-off / Kappa cut-off / Kappa cut-off / Kappa cut-off / Kappa cut-off /
Model
(95%CI) Area (95%CI) Area (95%CI) Area (95%CI) Area (95%CI) Area (95%CI) Area
under the under the under the under the under the under the
curve curve curve curve curve curve
0.6734 0.6681 0.6604 0.6836 0.6711 0.6720
0.8590 / 0.8569 / 0.8541 / 0.8633 / 0.8565 / 0.8574 /
C5.0 (0.6370- (0.6337- (0.6256- (0.6476 - (0.6370 - (0.6379 -
0.8894 0.8848 0.8908 0.8848 0.8899 0.8942
0.7099) 0.7026) 0.6952) 0.7195) 0.7051) 0.7061)
0.6502 0.6715 0.6546 0.6805 0.5376 0.5814
0.8511 / 0.8574 / 0.8532 / 0.8617 / 0.8139 / 0.8266 /
Random Forest (0.6124 - (0.6374- (0.6193- (0.6445 - (0.4966 - (0.5427-
0.9009 0.8884 0.8953 0.8884 0.8782 0.8914
0.6880) 0.7057) 0.6899) 0.7166) 0.5786) 0.6201)
0.6859 0.6650 0.6684 0.6715 0.6595 0.6606
Regularized Random 0.8638 / 0.8551 / 0.8560 / 0.8574 / 0.8527 / 0.8518 /
Forest (0.6502 - (0.6306- (0.6341- (0.6351 - (0.6248 - (0.6261-
0.8941 0.8869 0.8916 0.8909 0.8934 0.8944
0.7217) 0.6995) 0.7027) 0.7079) 0.6942) 0.6951)
0.6829 0.6660 0.6655 0.6873 0.6582 0.6798
Gradient Boosting 0.8622 / 0.8560 / 0.8551 / 0.8644 / 0.8513 / 0.8593 /
Machine (0.6470 - (0.6315- (0.6311- (0.6516 - (0.6235 - (0.6463-
0.8958 0.8923 0.8977 0.8910 0.8929 0.8945
0.7188) 0.7005) 0.7000) 0.7230) 0.6928) 0.7134)
0.6844 0.6553 0.6592 0.6786 0.6520 0.6679
Extreme Gradient 0.8638 / 0.8518 / 0.8537 / 0.8606 / 0.8499 / 0.8560 /
Boosting (0.6485 - (0.6203- (0.6243- (0.6425 - (0.6169 - (0.6336-
0.8873 0.8909 0.8947 0.8872 0.8941 0.8955
0.7203) 0.6903) 0.6941) 0.7147) 0.6871) 0.7023)
0.6747 0.6364 0.6437 0.4015 0.3385 0.3958
0.8585 / 0.8429 / 0.8457 / 0.7638 / 0.7429 / 0.7644 /
K-Nearest Neighbors (0.6385 - (0.6007- (0.6083- (0.3529 - (0.2908 - (0.3497-
0.8824 0.8626 0.8728 0.8038 0.7845 0.8127
0.7110) 0.6721) 0.6791) 0.4502) 0.3861) 0.4419)
0.6787 0.6600 0.6297 0.5832 0.2915 0.5480
0.8590 / 0.8527 / 0.8424 / 0.8181 / 0.7373 / 0.8121 /
Multilayer Perceptron (0.6429 - (0.6254- (0.5935- (0.5432 - (0.2412 - (0.5082-
0.8909 0.8844 0.8912 0.8546 0.7923 0.8718
0.7146) 0.6947) 0.6660) 0.6231) 0.3418) 0.5878)
THE TWO?
EJAE 2019  16 (2)  1-58
27
28
0.6307 0.6081 0.6529
0.8441 / 0.8350 / 0.8509 /
Support Vector Machine (0.5919 - (0.5707- (0.6178-
THE TWO?
0.8707 0.8656 0.8640

0.6696) 0.6454) 0.6880)
0.6886 0.2945 0.6727
Linear Discriminant 0.8649 / 0.7396 / 0.8579 /
Analysis (0.6530 - (0.2441- (0.6387-
0.8941 0.8252 0.8974
0.7243) 0.3449) 0.7068)
EJAE 2019  16 (2)  1-58
0.6879 0.3115 0.6737

0.8649 / 0.7452 / 0.8588 /
Logistic Regression (0.6522 - (0.2616- (0.6396- Models did not converge because of the large number of predictors
0.8908 0.8232 0.8974
0.7236) 0.3614) 0.7078)
0.6903 0.3056 0.6713
0.8660 / 0.7438 / 0.8579 /
Probit Regression (0.6548 - (0.2555- (0.6371-
0.8931 0.8257 0.8977
0.7259) 0.3558) 0.7055)
0.6527 0.6493 0.6209
Mixed-Effects Logistic 0.8521 / 0.8518 / 0.8424 /
Regression (0.6150 - (0.6137 - (0.5838 -
0.8858 0.8931 0.8940
0.6904) 0.6850) 0.6581)
Table 12. Comparative Predictive Performance of the Twelve Predictive Models for OOT Prediction when Using Lagged Variables to Inform Models on Prior Audit Opinions
Along with the Methods for Dealing with Missing Observations
EJAE 2019  16 (2)  1-58
THE TWO?
The results suggest that the classification accuracies for OOT prediction of most of the algorithms
presented in this study, when ignoring (or being willing to accept for the sake of improved classifica-
tion accuracy) the likely adverse consequences of the methods used for dealing with missing data, are,
in a statistical sense, comparable.
The multiple imputation method that considers the systematic effects present in the data is, argu-
ably, the only methodologically sound approach to dealing with missing data in datasets such as the
one analyzed in our study, and it can and should be employed to improve the predictive performance
of all the models presented in this study, including MELR. As this modeling procedure was not vi-
able within the software used in the study, we identify our failing to employ this procedure as a major
methodological limitation of the study and suggest that future research seriously consider this option.
CONCLUSIONS
Models for predicting the type of audit opinion have numerous applications in the field of finance.
Both statistical and machine learning approaches have been used for this task in prior research, but
for numerous reasons pointed out in this study, no definite conclusion can be drawn on their relative
predictive performance based on the results reported therein. Moreover, no previous study has con-
sidered the option of combining the two approaches. To address these issues, we have conducted this
study, making several important methodological contributions to this line of research.
Firstly, we have used a hand-collected data set comprising 13,561 pairs of annual financial statements
and the corresponding audit reports, which is, to the best of our knowledge, the largest empirical data
set ever used in this stream of research.
Secondly, we have exploited more fully the capability of machine learning algorithms to handle
a large number of predictors. This option was largely overlooked in prior research, where predictor
variables were either selected solely based on theory, or selected among a limited pool of conceivably
relevant financial metrics by employing a statistical procedure (e.g. statistical significance test or back-
ward stepwise elimination). To make better use of machine learning algorithms, we have considered
using all numeric items presented in financial statements, and the relations between them, as potentially
relevant predictors. We have presented an innovative method for feature generation, which has gener-
ated a total of 76,636 predictors. That is by far the largest number of potential predictors considered in
a single study. The use of data-driven predictors has been shown to improve predictive performance;
in addition to that, it has resulted in gaining insights into new types of financial metrics that are highly
relevant for predicting the type of audit opinion. These metrics combine the data from cash flow
statements and income statements with the data from the balance sheet, which is innovative from the
perspective of classical financial analysis. It should be noted that, since the relevance of these metrics
is suggested by the empirical evidence, there is no supporting theory or prior literature available. The
GRRF algorithm will identify different ratios as having discriminatory power in different countries and
different periods. Based on the authors’ knowledge of the local economy, the metrics identified in this
study, along with their corresponding thresholds, can be considered to be highly relevant as indicators
of solvency and liquidity.
Thirdly, we have fully exploited the capacity of statistical techniques to account for the systematic
effects present in the data, which has resulted in significant improvements in predictive performance
over the statistical models used in prior studies.
29
EJAE 2019  16 (2)  1-58
THE TWO?
Fourthly, we have compared the predictive performance of the models in two common and equally
important real-life settings: when prior information on the client is available and when prior informa-
tion on the client is not available. This aspect of the research design, largely neglected in prior research,
has allowed us to assess the differential predictive performance of the models in those two settings,
which has proved to be crucial for understanding the potential for their use. The results have shown
that machine learning techniques — primarily the tree-based machine learning algorithms such as RF,
RRF, GBM, and C5.0 — are the most appropriate predictive models to use when prior information on
the client is not available. On the other hand, for the clients on which prior information is obtainable, a
properly specified mixed-effects logistic regression has been shown to yield the predictive performance
equal to that yielded by the tree-based machine learning algorithms but with improved interpretabil-
ity and without introducing bias into the estimated probabilities. Importantly, these two approaches
respectively have outperformed by a wide margin the machine learning and statistical classifiers used
in prior research.
Finally, to explore the possibility of combining the relative strengths of the two regular approaches,
we have specified two innovative hybrid models. The stacked ensemble using mixed-effects logistic
regression as a meta-learner has yielded predictive performance better than any individual classifier
in both test samples, making it the most effective compound modeling strategy for predicting the type
of audit opinion. The second hybrid modeling strategy has integrated the classification rules obtained
from the GRRF algorithm into a structured panel model, resulting in a parsimonious, interpretable,
and computationally easy to fit model that has achieved an excellent interpretability/accuracy trade-off
in case of out-of-time predictions.
The main conceptual takeaway from this study is that for the development of effective models for
prediction of the type of auditor opinion, the strong points of both statistical and machine learning
approaches should be used to the full extent and combined when possible.
Regarding the practical applicability of this study, the procedure described herein can be thought of as
a framework for developing and testing models for predicting auditor opinions globally. The procedure
itself is described in enough detail to allow complete reproducibility. As regards the availability of data
needed to employ this framework, both audit opinions, as the outcome variable, and financial reports,
from which most predictors are extracted, are directly observable, easily obtainable, and have relatively
consistent form globally. Therefore, the proposed framework can be applied in any country. Moreover,
by taking advantage of the random-effects capability of mixed-effects logistic regression (specifically, by
the inclusion of random intercepts and random effects by countries), the framework can appropriately
handle empirical data coming from multiple countries. The models are highly modular in the sense
that additional predictors can be seamlessly incorporated, based on their availability. Importantly, the
models are applicable for predicting the type of audit opinion for both existing and new companies:
owing to the random-effect capability, prior information on the client is used when available but is not
necessary to get a risk assessment that is significantly better than the naive classification.
All the predictive models developed in this study provide probabilistic classifications, which can be
of great value during the stage of audit planning. Specifically, auditing firms can use different cut-offs
based on their specific misclassification costs or choose cut-off points based on the preferred expected
level of predictive accuracy within specific types of audit opinion (see Appendix F for observed pre-
dictive accuracies of the models by specific type of audit opinion at different cut-off points). Related
to this, we recommend future research to follow the methodological framework presented in this
study and examine more extreme modified opinions (i.e. disclaimer of opinion and adverse opinion)
30
EJAE 2019  16 (2)  1-58
THE TWO?
separately, as Appendix F shows that predictive accuracies vary across different types of modified opin-
ions. Additionally, the predictive output from the Bayesian mixed-effects logistic regression, such as that
specified in this study, is particularly rich in information and can be used in creative ways to form a basis
for decision-making: the risk assessments can be aggregated at the level of the audit firm, and various
probabilistic statements regarding risk exposure can be made to help form the price of the audit, etc.
At this point, it needs to be emphasized that the predictive models presented in this study assess
the risk of material misstatements due to events that are repeatedly detected through a regular external
audit and, as such, are primarily useful for audit planning and resource allocation decisions. The risk of
material misstatements, due to events that are difficult to detect through a regular audit, as examined
by Dechow et al. (2011), Perols (2011), and Perols et al. (2017) using proxies other than audit opinions,
is relatively more difficult to assess reliably, even with greater resource allocation, and is better consid-
ered to be a part of making client portfolio decisions aimed at reducing litigation and reputation risk.
It is also important to note that the economic significance of the incremental performance improve-
ments described in the results section (and, thus, the optimal choice of a specific predictive model)
depends heavily on the size of the auditing firm. Large auditors (and particularly members of the Big
4) should be expected to use the best performing model, which is stacked ensemble with mixed-effects
logistic regression as meta-learner for both OOT predictions and OOS and OOT predictions. They
may also consider the use of data-driven predictors, as this seems to give a slight improvement (sta-
tistically significantly at α = 0.1) in the predictive performance of the stacked ensemble. The rationale
for using this more involved modeling strategy is that, for these auditors, the benefits gained from a
more optimal resource allocation (even a one percent improvement) should outweigh the (for them)
immaterial marginal financial costs of developing the most sophisticated model. On the other hand,
for small local auditing firms with more constrained financial and human resources, a properly speci-
fied mixed-effects logistic regression that uses only theory-driven predictors may suffice for both OOT
predictions and OOS and OOT predictions.
Once the audit season is completed, the audit regulators can compare the predictions produced
by the models to the actual opinions issued at the level of individual audit firms, and effectively direct
their supervisory inspections toward those auditors with the highest estimated misclassification er-
rors. Furthermore, the estimated probability of receiving a modified opinion correlates strongly with
the probability of there being material misstatements in the corresponding financial reports (as stated
earlier, non-pervasive and pervasive material misstatements are the leading reasons for a modified
opinion being issued; based on our sample, they are present in around 85 percent of cases in which a
modified opinion is issued) and, as such, is highly indicative of the credibility of the financial reports,
which is of interest to all major stakeholders. Accordingly, the predicted probability obtained from the
models can be used by financial institutions as a valuable input (risk metric) for their decision-making
processes—for instance, by banks when making decisions on loan approvals, or by credit rating agen-
cies when assigning credit ratings.
A potential limitation of this study is that the relative performance of the models might vary de-
pending on the choices made in the outlier treatment procedure described in the methodology section.
Therefore, we recommend that future research experiments with different outlier treatment procedures
and reports their effects on the relative performance of the models. Another potential limitation of our
study is data imbalance. Given the relatively high frequency of qualified audit opinions in Serbia (32.6%)
compared to elsewhere in Europe, e.g. 16% in Spain, 2% in Germany, 0.5% in Austria, 3.7% in Switzer-
land, 1.3% in France, and 0.5% in the United Kingdom (Blandón and Bosch, 2013; Gassen and Skaife,
31
EJAE 2019  16 (2)  1-58
THE TWO?
2009), and other parts of the world, e.g. 13% in the US and 11% in China (Abad et al., 2017; Dhaliwal
et al., 2014), the results of the study may only be generalized to a smaller number of countries with
unusually high frequencies of qualified audit opinions. In light of this, future studies should consider
using different techniques for dealing with class imbalance. We also encourage researchers building
on the framework proposed herein to include additional non-financial predictors, such as employee
turnover, board structure, etc., when available, as these may be expected to marginally improve the
predictive performance of the models.
REFERENCES
Abad, D., Sánchez-Ballesta, J. P., & Yagüe, J. (2017). Audit opinions and information asymmetry in the stock
market. Accounting & Finance, 57(2), 565–595. https://doi.org/10.1111/acfi.12175
ASB GAAS Section 315. (2013). GAAS section 315, Understanding the Entity and Its Environment and Assessing
the Risks of Material Misstatement. Retrieved from https://www.aicpa.org/Research/Standards/AuditAt-
test/DownloadableDocuments/AU-C-00315.pdf
Ashbeck, E. L., & Bell, M. L. (2016). Single time point comparisons in longitudinal randomized controlled trials:
power and bias in the presence of missing data. BMC Medical Research Methodology, 16(1), 43. https://
doi.org/10.1186/s12874-016-0144-0
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for sub-
jects and items. Journal of Memory and Language, 59(4), 390–412. https://doi.org/10.1016/J.JML.2007.12.005
Bartov, E., Gul, F. A., & Tsui, J. S. L. (2000). Discretionary-accruals models and audit qualifications. Journal of
Accounting and Economics, 30(3), 421–452. https://doi.org/10.1016/S0165-4101(01)00015-5
Bell, T. B., & Tabor, R. H. (1991). Empirical Analysis of Audit Uncertainty Qualifications. Journal of Accounting
Research, 29(2), 350. https://doi.org/10.2307/2491053
Beneish, M. D. (1999). The Detection of Earnings Manipulation. Financial Analysts Journal, 55(5), 24–36. https://
doi.org/10.2469/faj.v55.n5.2296
Bergmeir, C., & Benitez, J. M. (2012). Neural Networks in R Using the Stuttgart Neural Network Simulator:
RSNNS. Journal of Statistical Software, 46(7), 1–26. Retrieved from http://www.jstatsoft.org/v46/i07/
Blandón, J. G., & Bosch, J. M. A. (2013). Audit firm tenure and qualified opinions: New evidence from Spain.
Revista de Contabilidad, 16(2), 118–125. https://doi.org/10.1016/j.rcsar.2013.02.001
Bürkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical
Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01
Caramanis, C., & Spathis, C. (2006). Auditee and audit firm characteristics as determinants of audit qualifications.
Managerial Auditing Journal, 21(9), 905–920. https://doi.org/10.1108/02686900610705000
Chen, T., He, T., Benesty, M., Khotilovich, V., & Tang, Y. (2017). xgboost: Extreme Gradient Boosting. Retrieved
from https://cran.r-project.org/package=xgboost
Cohen, J. (1960). A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement,
20(1), 37–46. https://doi.org/10.1177/001316446002000104
DeAngelo, L. E. (1986). Accounting numbers as market valuation substitutes: A study of management buyouts
of public stockholders. Accounting Review, 61(3), 400–420. Retrieved from /han/GoogleScholar/www.
jstor.org/stable/10.2307/247149
Dechow, P., Ge, W., & Schrand, C. (2010). Understanding earnings quality: A review of the proxies, their de-
terminants and their consequences. Journal of Accounting and Economics, 50(2–3), 344–401. https://doi.
org/10.1016/j.jacceco.2010.09.001
Dechow, P. M., Ge, W., Larson, C. R., & Sloan, R. G. (2011). Predicting Material Accounting Misstatements.
Contemporary Accounting Research, 28(1), 17–82. https://doi.org/10.1111/j.1911-3846.2010.01041.x
32
EJAE 2019  16 (2)  1-58
THE TWO?
Dechow, P. M., Sloan, R. G., & Sweeney, A. P. (1995). Detecting Earnings Management. The Accounting Review,
70(2), 193–225. https://doi.org/10.2307/248303
DeFond, M. L., & Jiambalvo, J. (1994). Debt covenant violation and manipulation of accruals. Journal of Account-
ing and Economics, 17(1–2), 145–176. https://doi.org/10.1016/0165-4101(94)90008-6
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the Areas under Two or More Cor-
related Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics, 44(3), 837.
https://doi.org/10.2307/2531595
Demler, O. V, Pencina, M. J., & D’Agostino, R. B. (2012). Misuse of DeLong test to compare AUCs for nested
models. Statistics in Medicine, 31(23), 2577–2587. https://doi.org/10.1002/sim.5328
Deng, H. (2013). Guided Random Forest in the RRF Package. ArXiv, 1–2. Retrieved from http://arxiv.org/
abs/1306.0237
Deng, H. (2014). Package ‘ inTrees .’ Retrieved from https://cran.r-project.org/package=inTrees
Deng, H., & Runger, G. (2012). Feature selection via regularized trees. In 2012 Annual International Joint Confer-
ence on Neural Networks, IJCNN 2012. Brisbane, Australia. https://doi.org/10.1109/IJCNN.2012.6252640
Deng, H., & Runger, G. (2013). Gene selection with guided regularized random forest. Pattern Recognition,
46(12), 3483–3489.
Dhaliwal, D. S., Liu, Q., Xie, H., & Zhang, J. (2014). Negative Press Coverage, Litigation Risk, and Audit Opinions
in China. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2381696
Dopuch, N., Holthausen, R., & Leftwich, R. (1987). Predicting audit qualifications with financial and market
variables. The Accounting Review, 62(3), 431–454.
Doumpos, M., Gaganis, C., & Pasiouras, F. (2005). Explaining qualifications in audit reports using a support vec-
tor machine methodology. Intelligent Systems in Accounting, Finance and Management, 13(4), 197–215.
https://doi.org/10.1002/isaf.268
Fernández-Gámez, M. A., García-Lagos, F., & Sánchez-Serrano, J. R. (2016). Integrating corporate governance
and financial variables for the identification of qualified audit opinions with neural networks. Neural
Computing and Applications, 27(5), 1427–1444. https://doi.org/10.1007/s00521-015-1944-6
Francis, J. R., & Krishnan, J. J. (1999). Accounting Accruals and Auditor Reporting Conservatism. Contemporary
Accounting Research, 16(1), 135–165. https://doi.org/10.1111/j.1911-3846.1999.tb00577.x
Gaganis, C., & Pasiouras, F. (2006). Auditing models for the detection of qualified audit opinions in the UK
public services sector. International Journal of Accounting, Auditing and Performance Evaluation, 3(4),
471. https://doi.org/10.1504/IJAAPE.2006.011207
Gaganis, C., Pasiouras, F., & Doumpos, M. (2007). Probabilistic neural networks for the identification of qualified
audit opinions. Expert Systems with Applications, 32(1), 114–124. https://doi.org/10.1016/j.eswa.2005.11.003
Gaganis, C., Pasiouras, F., Spathis, C., & Zopounidis, C. (2007). A comparison of nearest neighbours, discrimi-
nant and logit models for auditing decisions. Intelligent Systems in Accounting, Finance and Management,
15(1–2), 23–40. https://doi.org/10.1002/isaf.283
Gassen, J., & Skaife, H. A. (2009). Can Audit Reforms Affect the Information Role of Audits? Evidence from the
German Market. Contemporary Accounting Research, 26(3), 867–898. https://doi.org/10.1506/car.26.3.10
Gibbons, R. D., Hedeker, D., & DuToit, S. (2010). Advances in Analysis of Longitudinal Data. Annual Review of
Clinical Psychology, 6(1), 79–107. https://doi.org/10.1146/annurev.clinpsy.032408.153550
Glancy, F. H., & Yadav, S. B. (2011). A computational model for financial reporting fraud detection. Decision
Support Systems, 50(3), 595–601. https://doi.org/10.1016/j.dss.2010.08.010
Healy, P. M. (1985). The effect of bonus schemes on accounting decisions. Journal of Accounting and Economics,
7(1), 85–107.
Humpherys, S. L., Moffitt, K. C., Burns, M. B., Burgoon, J. K., & Felix, W. F. (2011). Identification of fraudulent
financial statements using linguistic credibility analysis. Decision Support Systems, 50(3), 585–594. https://
doi.org/10.1016/j.dss.2010.08.009
33
EJAE 2019  16 (2)  1-58
THE TWO?
IAASB ISA 315. (2013). ISA 315, Identifying and Assessing the Risks of Material Misstatement through Under-
standing the Entity and Its Environment. Retrieved from https://www.iaasb.org/system/files/meetings/
files/20130415-IAASB-Agenda_Item_5-D_Disclosures - ISA 315 %28Revised%29 for reference ONLY.pdf
IAASB ISA 570. (2013). ISA 570, Going Concern. Retrieved from http://www.ifac.org/system/files/downloads/
a031-2010-iaasb-handbook-isa-570.pdf
Jones, J. J. (1991). Earnings Management During Import Relief Investigations. Journal of Accounting Research,
29(2), 193–228. https://doi.org/10.2307/2491047
Jones, K. L., Krishnan, G. V., & Melendrez, K. D. (2008). Do Models of Discretionary Accruals Detect Actual
Cases of Fraudulent and Restated Earnings? An Empirical Analysis. Contemporary Accounting Research,
25(2), 499–531. https://doi.org/10.1506/car.25.2.8
Karatzoglou, A., Smola, A., Hornik, K., & Zeileis, A. (2004). kernlab - An S4 Package for Kernel Methods in R.
Journal of Statistical Software, 11(9), 1–20. Retrieved from http://www.jstatsoft.org/v11/i09/
Kinney, W. R., & McDaniel, L. S. (1989). Characteristics of firms correcting previously reported quarterly earn-
ings. Journal of Accounting and Economics, 11(1), 71–93. https://doi.org/10.1016/0165-4101(89)90014-1
Kirkos, E., Spathis, C., Nanopoulos, A., & Manolopoulos, Y. (2007). Identifying Qualified Auditors’ Opinions: A
Data Mining Approach. Journal of Emerging Technologies in Accounting, 4, 183–197.
Krishnan, J., & Krishnan, J. (1996). The Role of Economic Trade-Offs in the Audit Opinion Decision: An Empirical Anal-
ysis. Journal of Accounting, Auditing & Finance, 11(4), 565–586. https://doi.org/10.1177/0148558X9601100403
Krishnan, J., Krishnan, J., & Stephens, R. G. (1996). The Simultaneous Relation Between Auditor Switching and
Audit Opinion: An Empirical Analysis. Accounting & Business Research (Wolters Kluwer UK), 26(3),
224–236.
Kuhn, M. (2017). caret: Classification and Regression Training. Retrieved from https://cran.r-project.org/
package=caret
Kuhn, M., & Ross, Q. (2017). C50: C5.0 Decision Trees and Rule-Based Models. Retrieved from https://cran.r-
project.org/package=C50
Laitinen, E. K., & Laitinen, T. (1998). Qualified audit reports in Finland: evidence from large companies. European
Accounting Review, 7(4), 639–653. https://doi.org/10.1080/096381898336231
Liaw, A., & Wiener, M. (2002). Classification and Regression by randomForest. R News, 2(3), 18–22. Retrieved
from http://cran.r-project.org/doc/Rnews/
Maggina, A., & Tsaklanganos, A. A. (2011). Predicting audit opinions evidence from the athens stock exchange.
Journal of Applied Business Research, 27(4), 53–68.
Monroe, G. S., & Teh, S. T. (2009). Predicting uncertainty audit qualifications in Australia using publicly available
information. Accounting & Finance, 33(2), 79–106. https://doi.org/10.1111/j.1467-629X.1993.tb00200.x
Mutchler, J. F., & Hopwood, W. (1997). The Influence of Contrary Information and Mitigating Factors on Audit
Opinion Decisions on Bankrupt Companies. Journal of Accounting Research, 35(2), 295–310. https://doi.
org/10.2307/2491367
Ngai, E. W. T., Hu, Y., Wong, Y. H., Chen, Y., & Sun, X. (2011). The application of data mining techniques in
financial fraud detection: A classification framework and an academic review of literature. Decision Sup-
port Systems, 50(3), 559–569. https://doi.org/10.1016/j.dss.2010.08.006
Pedersen, A. B., Mikkelsen, E. M., Cronin-Fenton, D., Kristensen, N. R., Pham, T. M., Pedersen, L., & Petersen, I.
(2017). Missing data and multiple imputation in clinical epidemiological research. Clinical Epidemiology,
9, 157–166. https://doi.org/10.2147/CLEP.S129785
Perols, J. (2011). Financial Statement Fraud Detection: An Analysis of Statistical and Machine Learning Algo-
rithms. AUDITING: A Journal of Practice & Theory, 30(2), 19–50. https://doi.org/10.2308/ajpt-50009
Perols, J. L., Bowen, R. M., Zimmermann, C., & Samba, B. (2017). Finding needles in a haystack: Using data ana-
lytics to improve fraud prediction. In Accounting Review (Vol. 92, pp. 221–245). https://doi.org/10.2308/
accr-51562
34
EJAE 2019  16 (2)  1-58
THE TWO?
Pourheydari, O., Nezamabadi-Pour, H., & Aazami, Z. (2012). Identifying qualified audit opinions by artificial
neural networks. African Journal of Business Management, 6(44), 11077–11087. https://doi.org/10.5897/
AJBM12.855
R Core Team. (2017). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation
for Statistical Computing. Retrieved from http://www.r-project.org
Ridgeway, G. (2017). gbm: Generalized Boosted Regression Models. Retrieved from https://cran.r-project.org/
package=gbm
Ruiz-Barbadillo, E., Gómez-Aguilar, N., De Fuentes-Barberá, C., & García-Benau, M. A. (2004). Audit quality
and the going-concern decision-making process: Spanish evidence. European Accounting Review, 13(4),
597–620. https://doi.org/10.1080/0963818042000216820
Saif, S. M., Sarikhani, M., & Ebrahimi, F. (2012). Finding rules for audit opinions prediction through data mining
methods. European Online Journal of Natural and Social Sciences, 1(2), 28–36.
Saif, S. M., Sarikhani, M., & Ebrahimi, F. (2013). An Expert System with Neural Network and Decision Tree for
Predicting Audit Opinions. IAES International Journal of Artificial Intelligence (IJ-AI), 2(4), 151–158.
Retrieved from http://iaesjournal.com/online/index.php/IJAI/article/view/3950
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2),
147–177. https://doi.org/10.1037/1082-989X.7.2.147
Spathis, C., Doumpos, M., & Zopounidis, C. (2003). Using client performance measures to identify pre-engagement
factors associated with qualified audit reports in Greece. The International Journal of Accounting, 38(3),
267–284. https://doi.org/10.1016/S0020-7063(03)00047-5
Stice, J. D. (1991). Using Financial and Market Information to Identify Pre-Engagement Factors Associated with
Lawsuits against Auditors. The Accounting Review, 66(3), 516–533. https://doi.org/DOI:
Venables, W. N., & Ripley, B. D. (2002). Modern Applied Statistics with S (4th ed.). New York: Springer. Retrieved
from http://www.stats.ox.ac.uk/pub/MASS4
Yasar, A., Yakut, E., & Gutnu, M. M. (2015). Predicting Qualified Audit Opinions Using Financial Ratios : Evi-
dence from the Istanbul Stock Exchange. International Journal of Business and Social Science, 6(8), 57–67.
Yeh, C.-C., Chi, D.-J., & Lin, Y.-R. (2014). Going-concern prediction using hybrid random forests and rough set
approach. Information Sciences, 254, 98–110. https://doi.org/10.1016/j.ins.2013.07.011
Zdolšek, D., Jagrič, T., & Odar, M. (2015). Identification of auditor’s report qualifications: An empirical analysis
for Slovenia. Economic Research-Ekonomska Istrazivanja , 28(1), 994–1005. https://doi.org/10.1080/133
1677X.2015.1101960
Zhou, W., & Kapoor, G. (2011). Detecting evolutionary financial statement fraud. Decision Support Systems,
50(3), 570–575. https://doi.org/10.1016/j.dss.2010.08.007
35
EJAE 2019  16 (2)  1-58
THE TWO?
APPENDIX A: OBSERVED FREQUENCY OF SPECIFIC TYPES OF AUDIT OPINION BY

INDUSTRY CLASSIFICATION
Unmodi- % Unmodi-
ISIC Section Modified Total Division Modified Total
fied modified fied
01 496 347 843
A: Agriculture,
forestry and 522 364 886 41.08% 02 12 4 16
fishing
03 14 13 27
05 3 8 11
06 4 0 4
B: Mining and
46 51 97 52.58% 07 12 15 27
quarrying
08 27 22 49
09 0 6 6
10 641 325 966
11 73 59 132
12 16 8 24
13 59 39 98
14 104 62 166
15 52 30 82
16 72 44 116
17 85 23 108
18 77 30 107
19 20 11 31
20 153 59 212
C: Manufac- 21 28 10 38
2634 1473 4107 35.87%
turing 22 150 64 214
23 146 108 254
24 66 41 107
25 286 177 463
26 117 33 150
27 119 39 158
28 130 83 213
29 58 105 163
30 19 23 42
31 93 56 149
32 37 24 61
33 33 20 53
36
EJAE 2019  16 (2)  1-58
THE TWO?
D: Electricity,
gas, steam, and
131 79 210 37.62% 35 131 79 210
air conditioning
supply
E: Water supply; 36 171 106 277
sewerage, waste
management, 358 189 547 34.55% 38 187 82 269
and remediation
activities 39 0 1 1
41 275 136 411

F: Construction 773 410 1183 34.66% 42 249 165 414
43 249 109 358
G - Wholesale 45 242 67 309
and retail trade;
repair of motor 2776 805 3581 22.48% 46 2048 519 2567
vehicles and
motorcycles 47 486 219 705
49 271 142 413

50 10 3 13
H: Transporta-
400 187 587 31.86% 51 8 4 12
tion and storage
52 97 35 132
53 14 3 17
I: Accommoda- 55 118 103 221
tion and food 165 157 322 48.76%
service activities 56 47 54 101
58 76 48 124
59 10 7 17
J: Information 60 13 18 31
and 257 109 366 29.78%
communication 61 65 19 84
62 74 13 87
63 19 4 23
64 47 52 99
K: Financial and
insurance 53 53 106 50.00% 65 3 1 4
activities
66 3 0 3
L: Real estate
120 78 198 39.39% 68 120 78 198
activities
69 36 6 42
70 119 51 170
M: Professional, 71 138 55 193

scientific, and
424 183 607 30.15% 72 44 25 69
technical
activities 73 74 17 91
74 7 1 8
75 6 28 34
37
EJAE 2019  16 (2)  1-58
THE TWO?
77 27 3 30
78 10 0 10
N: Administra- 79 5 14 19
tive and support 163 51 214 23.83%
service activities 80 50 9 59
81 42 19 61
82 29 6 35
O: Public ad-
ministration and
defense; 0 4 4 100.00% 84 0 4 4
compulsory
social security
P: Education 8 15 23 65.22% 85 8 15 23
Q: Human health
and social work 14 12 26 46.15% 88 14 12 26
activities
90 1 0 1
R: Arts, 91 12 1 13
entertainment, 54 26 80 32.50%
and recreation 92 24 12 36
93 17 13 30
94 1 2 3
S: Other service
27 27 54 50.00% 95 12 16 28
activities
96 14 9 23
N/A 215 148 363 40.77% NA 215 148 363
Total 9140 4421 13561 32.60% 9140 4421 13561
Table A1. Types of Audit Opinion by ISIC (International Standard Industrial Classification of All Economic
Activities) Sections and Divisions
38
APPENDIX B: DESCRIPTIVE STATISTICS FOR THEORY-DRIVEN AND THE MOST IMPORTANT DATA-DRIVEN PREDICTORS
2010 2011 2012 2013

(n=3378) (n=3301) (n=3678) (n=3204)
Pctl Pctl Pctl Pctl Pctl Pctl Pctl Pctl
Predictor Median Median Median Median
(25) (75) (25) (75) (25) (75) (25) (75)
LN total assets (in 000 EUR) 7.54 8.25 9.12 7.68 8.35 9.22 7.56 8.26 9.14 7.67 8.38 9.25
LN total revenue (in 000 EUR) 7.45 8.22 8.98 7.67 8.39 9.18 7.63 8.33 9.12 7.70 8.37 9.15
Net result (in 000 EUR) 0.00 37.91 285.79 0.95 78.47 421.44 -0.34 63.85 386.23 -0.50 73.30 370.80
EBIT (in 000 EUR) 1.30 94.12 412.23 5.75 148.93 556.23 1.98 127.98 493.30 3.38 136.42 489.80
EBITDA (in 000 EUR) 34.82 210.45 596.39 47.91 281.05 761.20 37.78 225.60 692.17 38.76 249.12 709.55
Operating result (in 000 EUR) -11.28 106.85 469.53 -22.70 152.30 563.81 -14.14 167.20 582.75 -27.45 141.44 512.74
Return on assets 0.00 0.03 0.08 0.00 0.04 0.10 0.00 0.04 0.10 0.00 0.04 0.09
Return on equity 0.00 0.04 0.19 0.00 0.06 0.21 0.00 0.06 0.22 0.00 0.10 0.20
Return on invested capital 0.00 0.04 0.12 0.00 0.05 0.14 0.00 0.05 0.14 0.00 0.05 0.12
Return on capital invested in core business -0.01 0.04 0.12 -0.01 0.04 0.12 -0.01 0.05 0.13 -0.01 0.04 0.11
Operating margin -0.01 0.03 0.09 -0.02 0.03 0.09 -0.01 0.04 0.10 -0.02 0.03 0.08
Net margin 0.00 0.01 0.05 0.00 0.02 0.07 0.00 0.02 0.06 0.00 0.02 0.10
Net working capital (in 000 EUR) -283.40 213.73 1068.37 -326.72 237.06 1288.93 -284.54 283.85 1303.24 -343.80 261.51 1418.04
Net working capital to assets -0.10 0.07 0.25 -0.10 0.07 0.27 -0.10 0.09 0.31 -0.11 0.09 0.30
Quick ratio 0.30 0.59 1.03 0.30 0.62 1.06 0.30 0.62 1.10 0.30 0.63 1.14
Equity to total liabilities 0.20 0.68 1.75 0.22 0.67 1.82 0.22 0.69 1.85 0.26 0.77 2.19
Debt ratio 0.04 0.19 0.40 0.04 0.19 0.39 0.04 0.20 0.39 0.03 0.17 0.37
Days sales outstanding (in days) 32.74 60.55 111.08 31.56 58.90 106.91 29.13 57.69 99.32 29.82 58.24 98.89
THE TWO?
EJAE 2019  16 (2)  1-58
39
40
Days payable outstanding (in days) 41.85 76.03 150.03 39.70 72.87 147.20 34.64 67.82 134.73 35.77 68.12 137.40
THE TWO?
Days sales of inventory (in days) 11.24 31.28 73.88 9.79 31.28 71.66 9.67 31.28 70.38 10.71 31.28 72.15
Cash conversion cycle (in days) -12.68 26.36 78.56 -13.54 26.36 77.10 -10.61 26.36 77.38 -12.04 26.36 82.55
Z score for private companies 0.96 1.91 3.18 1.03 2.11 3.39 1.02 2.19 3.54 1.06 2.22 3.60
EJAE 2019  16 (2)  1-58
Z score for non-manufacturers and emerging

0.15 2.21 5.02 0.18 2.38 5.35 0.26 2.55 5.70 0.16 2.63 6.09
markets
Zmijewski score -2.54 -1.11 0.34 -2.60 -1.15 0.28 -2.69 -1.13 0.29 -2.80 -1.28 0.18
Shumway score 0.81 2.02 3.19 0.76 1.99 3.12 0.67 1.98 3.14 0.62 1.86 3.05
FCFE (in 000 EUR) -13.33 3.12 103.83 -19.33 3.32 112.17 -11.64 7.18 135.24 -20.15 4.01 126.02
FCFF (in 000 EUR) -379.74 -7.98 80.71 -432.36 -7.19 83.71 -385.37 -6.05 76.54 -275.84 -1.52 107.29
FCFE minus net result to revenue -0.05 -0.01 0.03 -0.06 -0.01 0.03 -0.05 -0.01 0.04 -0.05 -0.01 0.04
CFO minus operating result to revenue -0.07 0.00 0.08 -0.07 0.00 0.09 -0.07 -0.01 0.08 -0.05 0.01 0.10
Percentage foreign 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average net monthly salary (in EUR) 212.59 289.83 417.61 228.44 307.99 457.56 226.81 305.69 450.43 239.22 316.46 468.89
Table B1. Descriptive Statistics for Theory-Driven Predictors for Each Period
2010 2011 2012 2013

(n=3378) (n=3301) (n=3678) (n=3204)
Pctl Pctl Pctl Pctl Pctl Pctl Pctl Pctl
Predictor Median Median Median Median
(25) (75) (25) (75) (25) (75) (25) (75)
Total cash inflow/Long-term provisions and
liabilities 1.11 2.27 4.29 1.15 2.40 4.58 1.21 2.62 4.86 1.21 2.71 5.10
(used in Rules 1 and 4)
Liabilities for contributions on salaries and
wages paid by the employee (credit turnover
without opening balance)/Salaries, salary
compensations, and other benefits to 0.13 0.14 0.15 0.13 0.14 0.15 0.13 0.14 0.15 0.13 0.15 0.16
employees (cash outflows)
(used in Rule 2)
Operating revenue/Current liabilities
1.16 2.48 4.50 1.26 2.59 4.83 1.29 2.87 5.28 1.26 2.86 5.22
(used in Rule 3)
Table B2. Descriptive Statistics for Data-Driven Predictors for Each Period
THE TWO?
EJAE 2019  16 (2)  1-58
41
42
APPENDIX C: PAIRWISE COMPARISONS OF AUCS FOR TWO CORRELATED ROCS USING THE METHOD DESCRIBED BY DELONG ET
AL. (1988)
THE TWO?
Ensemble
XG- Ensemble
C5.0 RF RRF GBM KNN MLP SVM LDA Logit Probit MELR with
BOOST with LR
M-ELR
EJAE 2019  16 (2)  1-58
Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z=
Z = 2.2387,
-3.3425, -3.3712, 0.14935, 0.31003, 2.6966, 0.42464, 4.6567, 2.7463, 2.2194, -8.4778, -4.4371, -10.099,
C5.0 p-value =
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value < p-value = p-value <
0.02517
0.0008304 0.0007484 0.8813 0.7565 0.007005 0.6711 3.213e-06 0.006028 0.02646 2.2e-16 9.117e-06 2.2e-16
Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z=
Z = 4.5405,
-0.29492, 3.8212, 3.8516, 4.9989, 3.3958, 7.6883, 5.0897, 4.5097, -7.0304, -2.5421, -9.0038,
RF p-value =
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value <
5.612e-06
0.7681 0.0001328 0.0001173 5.764e-07 0.0006844 1.491e-14 3.587e-07 6.493e-06 2.059e-12 0.01102 2.2e-16
Z= Z= Z= Z= Z= Z= Z= Z= Z= Z=
Z = 4.5195,
3.7505, 3.7541, 5.0421, 3.3882, 7.6257, 5.0429, 4.4945, -7.0327, -2.3152, -9.071,
RRF p-value =
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value <
6.198e-06
0.0001765 0.000174 4.606e-07 0.0007036 2.427e-14 4.584e-07 6.972e-06 2.026e-12 0.0206 2.2e-16
Z= Z= Z= Z= Z= Z= Z= Z= Z=
Z = 2.2564,
0.23321, 2.5976, 0.33168, 4.7937, 2.8115, 2.2578, -8.5642, -4.7892, -10.156,
GBM p-value =
p-value = p-value = p-value = p-value = p-value = p-value = p-value < p-value = p-value <
0.02404
0.8156 0.009387 0.7401 1.637e-06 0.004931 0.02396 2.2e-16 1.674e-06 2.2e-16
Z= Z= Z= Z= Z= Z= Z=
Z = 2.524, Z = 1.9438,
XG- 0.15898, 4.6294, 2.4943, 1.9431, -8.7525, -4.8749, -10.439,
p-value = p-value =
BOOST p-value = p-value = p-value = p-value = p-value < p-value = p-value <
0.0116 0.05192
0.8737 3.668e-06 0.01262 0.05201 2.2e-16 1.089e-06 2.2e-16
Z= Z= Z= Z= Z= Z= Z=
Z = -1.21,
-2.4921, 1.4145, -0.81809, -1.1995, -11.426, -7.967, -13.229,
KNN p-value =
p-value = p-value = p-value = p-value = p-value < p-value = p-value <
0.2263
0.0127 0.1572 0.4133 0.2303 2.2e-16 1.625e-15 2.2e-16
Z= Z= Z= Z= Z= Z=
Z = 2.3609,
4.5186, 3.0031, 2.3609, -8.8482, -4.4291, -10.092,
MLP p-value =
p-value = p-value = p-value = p-value < p-value = p-value <
0.01823
6.224e-06 0.002673 0.01823 2.2e-16 9.463e-06 2.2e-16
Z= Z= Z= Z= Z= Z=
-2.3576, -2.7092, -2.7246, -11.379, -8.7168, -13.083,
SVM
p-value = p-value = p-value = p-value < p-value < p-value <
0.01839 0.006744 0.006438 2.2e-16 2.2e-16 2.2e-16
Z= Z= Z= Z= Z=
-1.6235, -2.0698, -10.32, -5.9935, -10.987,
LDA
p-value = p-value = p-value < p-value = p-value <
0.1045 0.03847 2.2e-16 2.054e-09 2.2e-16
Z= Z= Z= Z=
-0.0075588, -9.9685, -5.417, -10.605,
Logit
p-value = p-value < p-value = p-value <
0.994 2.2e-16 6.061e-08 2.2e-16
Z= Z= Z=
-10.013, -5.4682, -10.657,
Probit
p-value < p-value = p-value <
2.2e-16 4.546e-08 2.2e-16
Z= Z=
6.6183, -1.4504,
M-ELR
p-value = p-value =
3.634e-11 0.1469
Z=
Ensemble -8.8592,
with LR p-value <
2.2e-16
Ensemble
with
M-ELR
Table C1. DeLong’s Test for Two Correlated ROC Curves: Performance of the Models with Theory-Driven Predictors in Out-of-Time Test Set
THE TWO?
EJAE 2019  16 (2)  1-58
43
44
Ensemble
XG- Ensemble
C5.0 RF RRF GBM KNN MLP SVM LDA Logit Probit M-ELR with
BOOST with LR
THE TWO?
M-ELR
Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z=
-0.23407, 0.30975, -0.04991, 0.35356, 5.2109, 1.7658, 3.1823, 0.89294, 0.76484, 0.38196, 0.96835, 0.43162, -1.2132,
C5.0
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value =
EJAE 2019  16 (2)  1-58
0.8149 0.7567 0.9602 0.7237 1.879e-07 0.07743 0.001461 0.3719 0.4444 0.7025 0.3329 0.666 0.225
Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z= Z=
1.6042, 0.22176, 0.51947, 5.4498, 1.5754, 3.7847, 0.90421, 0.78369, 0.47308, 1.0218, 1.1984, -1.0844,
RF
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value =
0.1087 0.8245 0.6034 5.042e-08 0.1152 0.0001539 0.3659 0.4332 0.6362 0.3069 0.2308 0.2782
Z= Z= Z= Z= Z= Z= Z= Z= Z= Z=
Z = 1.094,
-0.38375, -0.038732, 5.0532, 3.2781, 0.48308, 0.36842, 0.067355, 0.68285, 0.22603, -1.5682,
RRF p-value =
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value =
0.2739
0.7012 0.9691 4.345e-07 0.001045 0.629 0.7126 0.9463 0.4947 0.8212 0.1168
Z= Z= Z= Z= Z= Z= Z= Z= Z= Z=
0.52466, 5.3015, 1.7079, 3.2272, 0.90247, 0.78179, 0.40181, 1.0034, 0.51685, -1.2589,
GBM
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value =
0.5998 1.148e-07 0.08766 0.00125 0.3668 0.4343 0.6878 0.3157 0.6053 0.2081
Z= Z= Z= Z= Z= Z= Z= Z= Z=
XG- 5.1073, 1.3662, 2.9564, 0.58871, 0.47284, 0.11287, 0.7716, 0.16931, -1.5329,
BOOST p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value =
3.268e-07 0.1719 0.003112 0.5561 0.6363 0.9101 0.4404 0.8656 0.1253
Z= Z= Z= Z= Z= Z= Z= Z=
-4.1339, -2.1151, -4.3968, -4.4818, -4.7312, -3.8458, -6.8889, -6.7025,
KNN
p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value =
3.566e-05 0.03442 1.098e-05 7.4e-06 2.232e-06 0.0001201 5.622e-12 2.049e-11
Z= Z= Z= Z= Z= Z= Z=
1.9942, -0.80987, -0.99259, -1.4511, -0.17594, -1.0098, -2.4073,
MLP
p-value = p-value = p-value = p-value = p-value = p-value = p-value =
0.04613 0.418 0.3209 0.1467 0.8603 0.3126 0.01607
Z= Z= Z= Z= Z= Z=
-2.2615, -2.3381, -2.569, -1.7819, -3.2888, -4.0461,
SVM
p-value = p-value = p-value = p-value = p-value = p-value =
0.02373 0.01938 0.0102 0.07477 0.001006 5.207e-05
Z= Z= Z= Z= Z=
-0.40288, -1.8989, 0.41787, -0.39769, -1.6947,
LDA
p-value = p-value = p-value = p-value = p-value =
0.687 0.05758 0.676 0.6909 0.09013
Z= Z= Z= Z=
-1.8592, 0.55909, -0.27995, -1.5984,
Logit
p-value = p-value = p-value = p-value =
0.063 0.5761 0.7795 0.11
Z= Z= Z=
0.93227, 0.02592, -1.2921,
Probit
p-value = p-value = p-value =
0.3512 0.9793 0.1963
Z= Z=
-0.6207, -2.593,
M-ELR
p-value = p-value =
0.5348 0.009514
Z=
Ensemble -1.8364,
with LR p-value =
0.0663
Ensemble
with
M-ELR
Table C2. DeLong’s Test for Two Correlated ROC Curves: Performance of the Models with Theory-Driven Predictors in Out-of-Sample and Out-of-Time Test Set
THE TWO?
EJAE 2019  16 (2)  1-58
45
46
Ensemble Ensemble M-ELR with
C5.0 RF RRF GBM XGBOOST KNN MLP
with LR with M-ELR GRRF rules
THE TWO?
Z = -5.645, Z = -4.1546, Z = 1.988, Z = -0.61689, Z = 2.8133, Z = 3.7931, Z = -7.3954, Z = -9.5817, Z = -5.4511,

C5.0 p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value < p-value =
1.651e-08 3.259e-05 0.04681 0.5373 0.004903 0.0001488 1.41e-13 2.2e-16 5.005e-08
Z = 3.1876, Z = 8.1076, Z = 4.6142, Z = 6.9548, Z = 8.8324, Z = -3.8005, Z = -6.9936, Z = -2.1434,
EJAE 2019  16 (2)  1-58
RF p-value = p-value = p-value = p-value = p-value < p-value = p-value = p-value =

0.001435 5.164e-16 3.947e-06 3.53e-12 2.2e-16 0.0001444 2.68e-12 0.03208
Z = 6.4879, Z = 3.2197, Z = 5.7204, Z = 7.5411, Z = -5.6945, Z = -7.8825, Z = -3.0667,
RRF p-value = p-value = p-value = p-value = p-value = p-value = p-value =
8.705e-11 0.001283 1.063e-08 4.662e-14 1.237e-08 3.209e-15 0.002164
Z = -2.5429, Z = 1.6643, Z = 3.1048, Z = -8.3756, Z = -10.398, Z = -6.5032,
GBM p-value = p-value = p-value = p-value < p-value < p-value =
0.01099 0.09605 0.001904 2.2e-16 2.2e-16 7.864e-11
Z = 3.2303, Z = 4.4052, Z = -7.1575, Z = -9.5975, Z = -5.1898,
XGBOOST p-value = p-value = p-value = p-value < p-value =
0.001237 1.057e-05 8.215e-13 2.2e-16 2.105e-07
Z = 0.53676, Z = -8.9634, Z = -11.597, Z = -8.0493,
KNN p-value = p-value < p-value < p-value =
0.5914 2.2e-16 2.2e-16 8.325e-16
Z = -9.5238, Z = -11.308, Z = -8.0783,
MLP p-value < p-value < p-value =
2.2e-16 2.2e-16 6.567e-16
Z = -5.9764, Z = -0.94275,
Ensemble
p-value = p-value =
with LR
2.281e-09 0.3458
Z = 4.7514,
Ensemble
p-value =
with M-ELR
2.02e-06
M-ELR with
GRRF rules
Table C3. DeLong’s Test for Two Correlated ROC Curves: Performance of the Models with Theory- and Data-Driven Predictors in Out-of-Time Test Set
Ensemble with Ensemble with M-ELR with
C5.0 RF RRF GBM XGBOOST KNN MLP
LR M-ELR GRRF rules
Z = -1.8282, Z = -2.2387, Z = -1.1285, Z = 2.6972, Z = 6.0165, Z = 1.7117, Z = -0.84645, Z = -2.6246, Z = 3.4589,
C5.0 p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value = p-value =
0.06751 0.02517 0.2591 0.006993 1.782e-09 0.08696 0.3973 0.008674 0.0005424
Z = -0.86954, Z = 0.86774, Z = 3.8082, Z = 7.3415, Z = 3.3479, Z = 1.938, Z = 4.9311,
Z = -1.379, p-
RF p-value = p-value = p-value = p-value = p-value = p-value = p-value =
value = 0.1679
0.3846 0.3855 0.00014 2.112e-13 0.0008143 0.05262 8.175e-07
Z = 1.4141, Z = 4.1863, Z = 7.4955, Z = 3.4971, Z = 2.3871, Z = -0.81864, Z = 5.0811,
RRF p-value = p-value = p-value = p-value = p-value = p-value = p-value =
0.1573 2.835e-05 6.605e-14 0.0004703 0.01698 0.413 3.753e-07
Z = 4.0254, Z = 6.6888, Z = 2.5698, Z = -1.8444, Z = 4.3705,
Z = 0.29825, p-
GBM p-value = p-value = p-value = p-value = p-value =
value = 0.7655
5.687e-05 2.249e-11 0.01018 0.06512 1.24e-05
Z= Z=
Z = 4.0604, Z= Z = 1.5197,
XG- -3.7499, -4.8282,
p-value = -0.64293, p- p-value =
BOOST p-value = p-value =
4.9e-05 value = 0.5203 0.1286
0.0001769 1.378e-06
Z= Z=
Z= Z=
-4.8933, -7.6332,
KNN -8.3037, p- -2.3398, p-
p-value = p-value =
value < 2.2e-16 value = 0.0193
9.916e-07 2.29e-14
Z=
Z= Z = 2.2405,
-4.0402,
MLP -2.4675, p-val- p-value =
p-value =
ue = 0.01361 0.02506
5.341e-05
Z=
Z = 4.1161,
Ensemble -2.6308,
p-value =
with LR p-value =
3.853e-05
0.008518
Z = 7.0902,
Ensemble
p-value =
with M-ELR
1.34e-12
M-ELR
with GRRF
rules
Table C4. DeLong’s Test for Two Correlated ROC Curves: Performance of the Models with Theory- and Data-Driven Predictors in Out-of-Sample and Out-of-Time Test Set
THE TWO?
EJAE 2019  16 (2)  1-58
47
EJAE 2019  16 (2)  1-58
THE TWO?
APPENDIX D: COMPLETE OUTPUT OF THE HYBRID MODELS
Model Parameters Estimate Std. error z value Pr(>|z|)

Intercept -2.78959 0.07024 -39.717 < 2e-16 ***
C5.0 0.03290 0.31839 0.103 0.91770
RF 2.01953 0.62237 3.245 0.00117 **
Stacked ensemble with logistic RRF 2.38909 0.57824 4.132 3.6e-05 ***
regression as meta learner
(theory-driven predictors) GBM -0.55581 0.51326 -1.083 0.27885
XGBOOST 0.49897 0.43499 1.147 0.25135
KNN 1.59955 0.13773 11.614 < 2e-16 ***
MLP -0.26654 0.26603 -1.002 0.31638
Intercept -3.27834 0.08045 -40.748 < 2e-16 ***
C5.0 0.88376 0.30336 2.913 0.00358 **
RF 4.30689 0.56498 7.623 2.48e-14 ***
Stacked ensemble with logistic
regression as meta learner RRF 2.04153 0.51107 3.995 6.48e-05 ***
(theory- and data-driven GBM -2.39261 0.32336 -7.399 1.37e-13 ***
predictors)
XGBOOST 1.57105 0.20341 7.724 1.13e-14 ***
KNN 0.82217 0.14607 5.629 1.82e-08 ***
MLP -0.21596 0.15481 1.395 0.16301
Intercept -4.2019 0.2391 -17.575 < 2e-16 ***
C5.0 1.1939 0.5073 2.354 0.01859 *
RF 0.8296 0.9624 0.862 0.38869
Stacked ensemble with mixed- RRF 2.3983 0.8948 2.680 0.00736 **
effects logistic regression as meta
learner (theory-driven predictors) GBM 1.3937 0.8220 1.695 0.08999 .
XGBOOST 0.9356 0.6751 1.386 0.16574
KNN 1.3238 0.2241 5.906 3.5e-09 ***
MLP 0.5433 0.4292 1.266 0.20558
Intercept -4.099045 0.203777 -20.115 < 2e-16 ***
C5.0 1.129696 0.406793 2.777 0.005485 **
RF 3.984554 0.762891 5.223 1.76e-07 ***
Stacked ensemble with mixed- 0.000236
effects logistic regression as meta RRF 2.536672 0.689888 3.677
***
learner (theory- and data-driven
predictors) GBM -0.491317 0.475486 -1.033 0.301466
XGBOOST 1.152543 0.280092 4.115 3.87e-05 ***
KNN 0.351922 0.205382 1.714 0.086620 .
MLP -0.007802 0.212315 -0.037 0.970686
48
EJAE 2019  16 (2)  1-58
THE TWO?
Intercept -2.8638 0.2437 -11.752 < 2e-16 ***

Rule 1 1.2357 0.1718 7.191 6.45e-13 ***
Mixed-effects logistic regression
with GRRF classification rules as Rule 2 1.1177 0.1350 8.279 < 2e-16 ***
predictors
Rule 3 1.2894 0.1518 8.492 < 2e-16 ***
Rule 4 1.3942 0.2169 6.428 1.29e-10 ***
Significance codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Table D1. The Complete Output of the Hybrid Models
APPENDIX E: ALTERNATIVE MODELING STRATEGIES
With Theory-Driven Predictors

Out-of-Time Out-of-Sample and Out-of-Time
(No information rate: 0.6961) (No information rate: 0.6629)
Test sample size reduced from 2,139 Test sample size reduced from 1,065
obs. to 1,951 obs. because of auditors obs. to 976 obs. because of auditors who
who were not in the training sample were not in the training sample
Accuracy at 50% Accuracy at 50%
Kappa cut-off / Kappa cut-off /
Model
curve curve
Fixed effects logistic 0.5138 0.3242

0.7929 / 0.7586 0.7336 / 0.7537
regression (0.4716 -–0.5561) (0.2538–0.3945)
Table E1. Predictive Performance of the Fixed-Effects Logistic Regressions
With Theory-Driven Predictors and Dummy Variables

for Specific Auditor Firms
Model
curve curve
0.4423 0.3716
C5.0 0.7756 / 0.8026 0.7296 / 0.7517
(0.3984–0.4863) (0.3096–0.4336)
0.4436 0.3582
Random Forest 0.7821 / 0.8309 0.7371 / 0.7669
(0.3990–0.4883) (0.2936–0.4227)
0.4559 0.3388
Regularized Random Forest 0.7863 / 0.8298 0.7286 / 0.7702
(0.4117–0.5002) (0.2738–0.4039)
0.0850 0.0159
Gradient Boosting Machine 0.6396 / 0.5281 0.6000 / 0.4885
(0.0334 - 0.1367) (-0.0565–0.0882)
Extreme Gradient Boosting Unable to make predictions due to missing predictors
-0.0339 -0.0494
K-Nearest Neighbors 0.6064 / 0.5090 0.5869 / 0.4744
(-0.088– 0.0205) (-0.1245–0.0258)
49
EJAE 2019  16 (2)  1-58
THE TWO?
-0.1253 -0.1436
Multilayer Perceptron (-0.1772– 0.5423 / 0.5975 (-0.2157– 0.5239 / 0.6371
-0.0734) -0.0716)
Support Vector Machine Unable to make predictions due to missing predictors
Linear Discriminant Analysis Unable to make predictions due to missing predictors
0.3380 -0.1795
Logistic Regression 0.7461 / 0.7566 0.6582 / 0.6136
(0.2899–0.3861) (0.1111–0.2479)
0.3382 0.1774
Probit Regression 0.7471 / 0.7541 0.6582 / 0.6174
(0.2900–0.3864) (0.1088–0.2459)
Table E2. Machine Learning Methods with Theory-Driven Predictors and Dummy Variables Identifying Auditor Firms
With 129 Principal Components Derived from Theory-

and Data-Driven Predictors
Model
curve curve
0.3709 0.3220
C5.0 0.7574 / 0.7830 0.7268 / 0.7512
(0.3238–0.4180) (0.2556–0.3884)
0.3704 0.3053
Random Forest 0.7602 / 0.8139 0.7230 / 0.7697
(0.3228–0.4179) (0.2380–0.3728)
0.4438 0.3538
Regularized Random Forest 0.7798 / 0.8239 0.7324 / 0.7607
(0.3994–0.4881) (0.2896–0.4180)
0.4116 0.3448
Gradient Boosting Machine 0.7719 / 0.7888 0.7333 / 0.7748
(0.4116–0.4575) (0.2795–0.4100)
0.3956 0.3335
Extreme Gradient Boosting 0.7606 / 0.7929 0.7211 / 0.7433
(0.3499–0.4413) (0.2691–0.3978)
0.3854 0.2466
K-Nearest Neighbors 0.7546 / 0.7778 0.6836 / 0.6817
(0.3398–0.4311) (0.1801–0.3131)
0.3880 0.3292
Multilayer Perceptron 0.7578 / 0.7686 0.7211 / 0.7539
(0.3421–0.4339) (0.2644–0.3940)
0.3222 0.2470
Support Vector Machine 0.7433 / 0.7655 0.6995 / 0.7324
(0.2733–0.3711) (0.1780–0.3160)
Linear Discriminant Analy- 0.3130 0.2817
0.7382 / 0.7637 0.7164 / 0.7594
sis (0.2641–0.3618) (0.2131–0.3503)
0.3307 0.2978
Logistic Regression 0.7447 / 0.7703 0.7202 / 0.7483
(0.2822–0.3791) (0.2302–0.3655)
0.3165 0.2855
Probit Regression 0.7405 / 0.7690 0.7164 / 0.7507
(0.2676–0.3654) (0.2173–0.3537)
Mixed-Effects Logistic 0.5839 0.2840
0.8252 / 0.8766 0.7052 / 0.7085
Regression (0.5456–0.6222) (0.2175–0.3505)
Table E3. Performance of the Models Using 129 Principal Components Driven from Theory- and Data-Driven
50 Predictors as Predictors
EJAE 2019  16 (2)  1-58
THE TWO?
APPENDIX F: PREDICTIVE ACCURACY BY SPECIFIC TYPE OF AUDIT OPINION AND BY

CUT-OFF POINT
Cut-off
Type of
Model Audit 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90
Opinion
Unqualified 33.33% 54.60% 71.13% 84.80% 93.10% 95.68% 97.84% 99.16% 100.00%
Qualified 91.38% 82.59% 66.21% 49.83% 37.59% 27.41% 18.10% 8.97% 1.21%
C5.0
Disclaimer 100.00% 98.17% 95.41% 90.83% 79.82% 65.14% 45.87% 23.85% 5.50%
Adverse 87.50% 81.25% 75.00% 62.50% 31.25% 31.25% 6.25% 6.25% 0.00%
Unqualified 20.85% 48.05% 70.29% 86.12% 93.51% 97.56% 98.95% 99.37% 99.72%
Qualified 96.72% 85.86% 72.59% 55.00% 40.34% 22.59% 11.55% 5.69% 1.21%
RF
Disclaimer 100.00% 99.08% 97.25% 90.83% 75.23% 58.72% 36.70% 22.02% 5.50%
Adverse 93.75% 87.50% 75.00% 56.25% 50.00% 37.50% 6.25% 0.00% 0.00%
Unqualified 21.13% 48.68% 71.13% 85.29% 93.93% 97.42% 98.81% 99.44% 99.79%
Qualified 96.21% 86.21% 73.45% 54.48% 38.97% 23.45% 12.41% 5.86% 1.21%
RRF
Disclaimer 100.00% 99.08% 94.50% 88.07% 76.15% 60.55% 36.70% 22.94% 6.42%
Adverse 93.75% 87.50% 75.00% 56.25% 56.25% 37.50% 12.50% 0.00% 0.00%
Unqualified 11.72% 51.88% 75.45% 86.75% 93.38% 97.14% 98.81% 99.51% 99.93%
Qualified 97.59% 82.24% 63.62% 48.45% 34.31% 24.14% 12.76% 4.66% 0.17%
GBM
Disclaimer 100.00% 98.17% 94.50% 91.74% 82.57% 66.06% 39.45% 20.18% 0.92%
Adverse 93.75% 81.25% 75.00% 56.25% 43.75% 37.50% 18.75% 6.25% 0.00%
Unqualified 20.01% 53.14% 73.85% 86.19% 93.31% 96.09% 98.74% 99.16% 99.86%
XG- Qualified 95.86% 81.38% 64.48% 48.62% 34.83% 22.93% 13.79% 7.76% 1.21%
BOOST Disclaimer 100.00% 98.17% 95.41% 90.83% 75.23% 60.55% 46.79% 23.85% 3.67%
Adverse 87.50% 75.00% 75.00% 62.50% 50.00% 37.50% 18.75% 6.25% 6.25%
Unqualified 40.73% 40.73% 71.27% 71.34% 87.66% 87.73% 96.93% 96.93% 99.44%
Qualified 87.07% 87.07% 68.45% 68.45% 45.17% 45.17% 21.03% 21.03% 7.41%
KNN
Disclaimer 96.33% 96.33% 85.32% 85.32% 64.22% 64.22% 42.20% 42.20% 15.60%
Adverse 81.25% 81.25% 62.50% 62.50% 50.00% 50.00% 25.00% 25.00% 6.25%
Unqualified 16.53% 57.32% 76.78% 88.42% 93.65% 96.58% 98.33% 99.37% 99.86%
Qualified 97.07% 79.83% 61.55% 44.48% 30.34% 21.55% 13.62% 5.69% 2.59%
MLP
Disclaimer 100.00% 98.17% 97.25% 88.07% 73.39% 54.13% 37.61% 14.68% 2.75%
Adverse 87.50% 81.25% 68.75% 43.75% 31.25% 25.00% 18.75% 18.75% 0.00%
51
EJAE 2019  16 (2)  1-58
THE TWO?
Unqualified 0.00% 6.76% 83.05% 90.45% 96.03% 97.28% 98.19% 99.16% 100.00%
Qualified 100.00% 98.62% 47.24% 34.66% 24.31% 17.93% 11.38% 6.03% 0.00%
SVM
Disclaimer 100.00% 99.08% 87.16% 69.72% 53.21% 44.04% 33.94% 17.43% 0.92%
Adverse 100.00% 93.75% 62.50% 56.25% 37.50% 31.25% 12.50% 6.25% 0.00%
Unqualified 7.88% 40.93% 77.20% 90.93% 95.26% 97.42% 98.68% 99.09% 99.79%
Qualified 99.14% 84.31% 57.59% 36.21% 23.97% 15.52% 9.83% 5.34% 1.72%
LDA
Disclaimer 100.00% 99.08% 96.33% 74.31% 55.96% 40.37% 24.77% 13.76% 7.34%
Adverse 100.00% 81.25% 81.25% 50.00% 43.75% 18.75% 12.50% 0.00% 0.00%
Unqualified 10.60% 40.17% 72.52% 89.33% 95.75% 97.42% 98.47% 99.23% 99.65%
Qualified 97.76% 86.03% 64.48% 41.55% 24.83% 16.55% 10.17% 5.17% 1.90%
Logit
Disclaimer 100.00% 98.17% 95.41% 86.24% 62.39% 45.87% 28.44% 14.68% 7.34%
Adverse 100.00% 81.25% 75.00% 62.50% 43.75% 31.25% 6.25% 0.00% 0.00%
Unqualified 10.32% 38.28% 71.90% 89.40% 95.75% 97.42% 98.54% 99.23% 99.65%
Qualified 97.41% 87.41% 65.69% 40.00% 23.62% 15.00% 9.31% 4.66% 1.90%
Probit
Disclaimer 100.00% 99.08% 96.33% 83.49% 59.63% 41.28% 23.85% 12.84% 8.26%
Adverse 100.00% 81.25% 75.00% 56.25% 43.75% 25.00% 6.25% 0.00% 0.00%
Unqualified 47.28% 56.47% 67.10% 71.72% 77.02% 77.67% 78.26% 78.52% 78.87%
Qualified 72.63% 68.36% 62.59% 54.66% 40.95% 38.66% 36.47% 33.58% 29.87%
M-LER
Disclaimer 75.69% 71.56% 65.60% 58.94% 49.77% 48.62% 46.33% 42.66% 37.61%
Adverse 68.75% 59.38% 53.13% 48.44% 35.94% 32.81% 31.25% 31.25% 29.69%
Unqualified 25.31% 60.95% 78.17% 88.21% 92.89% 96.37% 98.26% 99.09% 99.72%
Stacked Qualified 96.03% 81.72% 65.17% 55.00% 42.76% 31.72% 19.83% 11.55% 2.59%
model
with logit Disclaimer 100.00% 96.33% 93.58% 87.16% 78.90% 65.14% 51.38% 31.19% 11.93%
Adverse 93.75% 81.25% 75.00% 62.50% 43.75% 31.25% 25.00% 12.50% 0.00%
Unqualified 67.64% 79.57% 85.50% 89.82% 92.75% 94.77% 95.75% 97.77% 99.44%
Stacked
model Qualified 88.62% 80.69% 73.45% 67.41% 61.38% 53.45% 45.00% 36.72% 21.21%
with M- Disclaimer 96.33% 92.66% 90.83% 88.07% 85.32% 82.57% 77.06% 67.89% 46.79%
LER
Adverse 68.75% 56.25% 56.25% 50.00% 43.75% 37.50% 31.25% 31.25% 25.00%
Unqualified 62.27% 76.29% 83.47% 87.59% 90.38% 93.03% 94.56% 96.72% 98.54%
M-LER
with Qualified 87.93% 81.55% 75.52% 70.34% 64.14% 57.07% 50.00% 38.79% 24.66%
GRRF Disclaimer 94.50% 92.66% 91.74% 88.07% 86.24% 80.73% 76.15% 66.06% 53.21%
dummies
Adverse 75.00% 62.50% 50.00% 50.00% 50.00% 43.75% 37.50% 31.25% 12.50%
Table F1. Predictive Accuracy by Specific Type of Audit Opinion: Models with Theory-Driven Predictors, OOT
Predictions
52
EJAE 2019  16 (2)  1-58
THE TWO?
Cut-off
Type of
Model Audit 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90
Opinion
Unqualified 31.04% 53.00% 70.86% 84.63% 93.41% 96.63% 97.95% 98.98% 99.71%
Qualified 88.52% 79.67% 64.92% 46.23% 32.79% 21.31% 12.13% 5.25% 0.33%
C5.0
Disclaimer 98.53% 98.53% 91.18% 79.41% 70.59% 61.76% 47.06% 26.47% 7.35%
Adverse 100.00% 100.00% 88.89% 88.89% 88.89% 88.89% 66.67% 33.33% 11.11%
Unqualified 17.13% 43.92% 67.06% 84.04% 93.70% 97.22% 98.54% 99.71% 100.00%
Qualified 96.07% 85.25% 70.16% 48.20% 30.82% 16.72% 7.54% 3.61% 0.98%
RF
Disclaimer 100.00% 98.53% 89.71% 79.41% 66.18% 52.94% 29.41% 19.12% 7.35%
Adverse 100.00% 100.00% 100.00% 88.89% 88.89% 66.67% 44.44% 44.44% 0.00%
Unqualified 18.59% 43.34% 68.08% 83.75% 91.95% 96.93% 98.39% 99.85% 100.00%
Qualified 95.41% 85.25% 66.89% 48.85% 30.82% 17.05% 7.21% 4.59% 0.66%
RRF
Disclaimer 100.00% 100.00% 89.71% 82.35% 70.59% 54.41% 29.41% 22.06% 7.35%
Adverse 100.00% 100.00% 100.00% 88.89% 88.89% 66.67% 44.44% 33.33% 0.00%
Unqualified 12.45% 51.83% 75.40% 87.41% 94.14% 97.51% 98.54% 98.98% 100.00%
Qualified 96.72% 79.02% 60.98% 44.26% 29.51% 21.31% 8.85% 2.62% 0.00%
GBM
Disclaimer 100.00% 95.59% 92.65% 80.88% 64.71% 52.94% 45.59% 25.00% 0.00%
Adverse 100.00% 88.89% 88.89% 88.89% 88.89% 77.78% 44.44% 33.33% 0.00%
Unqualified 19.47% 52.56% 74.38% 87.85% 94.14% 96.78% 98.24% 99.27% 99.85%
XG- Qualified 93.77% 77.70% 61.31% 43.93% 27.87% 16.72% 8.85% 4.59% 2.30%
Adverse 100.00% 88.89% 88.89% 88.89% 88.89% 77.78% 55.56% 44.44% 22.22%
Unqualified 37.63% 37.92% 67.94% 67.94% 85.80% 85.80% 95.31% 95.31% 98.83%
Qualified 81.97% 81.97% 56.72% 56.72% 36.07% 36.07% 15.74% 15.74% 2.62%
KNN
Disclaimer 95.59% 95.59% 79.41% 79.41% 58.82% 58.82% 32.35% 32.35% 14.71%
Adverse 100.00% 100.00% 77.78% 77.78% 44.44% 44.44% 33.33% 33.33% 33.33%
Unqualified 15.67% 57.98% 76.57% 90.48% 94.73% 97.22% 98.54% 99.27% 99.85%
Qualified 95.74% 72.46% 55.74% 38.36% 22.30% 15.08% 8.85% 3.93% 0.66%
MLP
Disclaimer 98.53% 92.65% 86.76% 69.12% 55.88% 47.06% 39.71% 14.71% 5.88%
Adverse 100.00% 100.00% 100.00% 88.89% 77.78% 55.56% 44.44% 22.22% 0.00%
Unqualified 0.00% 5.12% 81.70% 88.58% 94.73% 96.19% 97.66% 98.83% 100.00%
Qualified 100.00% 97.05% 43.93% 33.11% 19.34% 13.77% 8.85% 4.92% 0.00%
SVM
Disclaimer 100.00% 98.53% 79.41% 67.65% 50.00% 36.76% 27.94% 19.12% 1.47%
Adverse 100.00% 100.00% 88.89% 88.89% 55.56% 44.44% 33.33% 22.22% 0.00%
53
EJAE 2019  16 (2)  1-58
THE TWO?
Unqualified 7.76% 41.73% 79.80% 92.83% 96.34% 98.10% 98.39% 98.98% 99.85%
Qualified 98.03% 84.59% 57.38% 32.13% 19.34% 11.15% 5.57% 1.97% 0.66%
LDA
Disclaimer 98.53% 95.59% 88.24% 63.24% 50.00% 38.24% 32.35% 20.59% 4.41%
Adverse 100.00% 100.00% 88.89% 66.67% 66.67% 44.44% 22.22% 0.00% 0.00%
Unqualified 12.74% 41.14% 74.23% 90.78% 96.19% 97.95% 98.68% 99.12% 99.85%
Qualified 96.72% 84.59% 60.98% 36.39% 20.98% 11.15% 5.25% 3.28% 0.98%
Logit
Disclaimer 98.53% 94.12% 86.76% 67.65% 55.88% 38.24% 32.35% 20.59% 7.35%
Adverse 100.00% 100.00% 88.89% 77.78% 66.67% 44.44% 33.33% 0.00% 0.00%
Unqualified 12.15% 39.24% 74.08% 91.07% 96.34% 97.95% 98.54% 99.12% 99.85%
Qualified 96.39% 85.25% 62.30% 36.07% 20.00% 10.82% 5.25% 2.95% 0.98%
Probit
Disclaimer 98.53% 95.59% 92.65% 69.12% 54.41% 39.71% 32.35% 16.18% 5.88%
Adverse 100.00% 100.00% 100.00% 66.67% 66.67% 44.44% 11.11% 0.00% 0.00%
Unqualified 29.80% 38.36% 47.29% 58.49% 74.38% 75.15% 75.40% 75.77% 75.77%
Qualified 73.61% 70.08% 64.18% 55.74% 31.56% 27.79% 26.15% 25.49% 25.08%
M-LER
Disclaimer 74.26% 72.79% 69.12% 61.40% 41.18% 36.40% 34.56% 31.25% 27.21%
Adverse 75.00% 75.00% 72.22% 66.67% 38.89% 30.56% 25.00% 25.00% 25.00%
Unqualified 19.91% 55.34% 75.26% 86.82% 91.80% 95.02% 97.66% 99.27% 99.85%
Stacked
model Qualified 94.43% 76.72% 60.00% 45.57% 34.75% 25.25% 15.08% 5.25% 0.66%
with Disclaimer 100.00% 95.59% 85.29% 77.94% 69.12% 58.82% 50.00% 30.88% 8.82%
logit
Adverse 100.00% 100.00% 88.89% 88.89% 77.78% 66.67% 55.56% 44.44% 22.22%
Unqualified 52.86% 73.21% 83.02% 89.02% 91.80% 93.41% 95.46% 96.93% 98.98%
Stacked
model Qualified 80.66% 66.56% 55.08% 46.89% 36.07% 28.85% 23.28% 13.44% 6.23%
LER
Adverse 100.00% 88.89% 88.89% 88.89% 88.89% 77.78% 66.67% 55.56% 22.22%
Unqualified 44.95% 64.71% 78.18% 84.33% 88.87% 90.63% 93.56% 97.22% 99.27%
M-LER
with Qualified 75.74% 60.98% 49.18% 40.33% 32.79% 28.52% 22.62% 13.44% 6.89%
GRRF
dum- Disclaimer 91.18% 82.35% 79.41% 75.00% 63.24% 63.24% 55.88% 39.71% 26.47%
mies
Adverse 100.00% 100.00% 88.89% 77.78% 77.78% 66.67% 66.67% 33.33% 22.22%
Table F2. Predictive Accuracy by Specific Type of Audit Opinion: Models with Theory-Driven Predictors, OOS and
OOT Predictions
54
EJAE 2019  16 (2)  1-58
THE TWO?
Cut-off
Type of
Model Audit 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90
Opinion
Unqualified 23.15% 45.33% 65.90% 81.38% 90.66% 95.40% 98.12% 99.09% 99.72%
Qualified 96.90% 89.31% 75.52% 61.72% 47.41% 31.03% 18.45% 8.62% 3.45%
C5.0
Disclaimer 99.08% 97.25% 97.25% 94.50% 84.40% 72.48% 55.96% 36.70% 10.09%
Adverse 87.50% 81.25% 75.00% 50.00% 50.00% 37.50% 18.75% 12.50% 6.25%
Unqualified 15.48% 41.35% 68.69% 85.01% 94.14% 98.12% 98.81% 99.65% 100.00%
Qualified 98.62% 93.79% 81.38% 64.31% 43.62% 26.03% 14.14% 5.69% 0.86%
RF
Disclaimer 100.00% 98.17% 96.33% 95.41% 85.32% 72.48% 39.45% 17.43% 2.75%
Adverse 93.75% 81.25% 81.25% 68.75% 43.75% 37.50% 25.00% 0.00% 0.00%
Unqualified 11.02% 36.19% 59.55% 79.92% 91.70% 97.00% 98.61% 99.51% 99.86%
Qualified 99.14% 94.66% 85.52% 70.00% 48.97% 29.66% 17.24% 7.07% 1.72%
RRF
Disclaimer 99.08% 99.08% 96.33% 95.41% 88.07% 70.64% 44.95% 21.10% 8.26%
Adverse 100.00% 81.25% 81.25% 68.75% 43.75% 31.25% 18.75% 0.00% 0.00%
Unqualified 13.74% 44.28% 68.34% 82.57% 89.40% 94.70% 96.86% 98.26% 99.72%
Qualified 97.93% 88.45% 74.14% 58.10% 45.17% 33.28% 23.79% 11.55% 2.41%
GBM
Disclaimer 100.00% 98.17% 96.33% 93.58% 89.91% 82.57% 66.97% 44.95% 12.84%
Adverse 81.25% 81.25% 81.25% 68.75% 50.00% 43.75% 37.50% 18.75% 6.25%
Unqualified 35.70% 57.25% 70.50% 81.03% 88.77% 92.54% 95.19% 97.42% 98.81%
XG- Qualified 93.79% 83.62% 73.97% 64.14% 53.79% 42.07% 33.28% 21.38% 12.59%
Adverse 81.25% 81.25% 81.25% 75.00% 68.75% 56.25% 43.75% 25.00% 12.50%
Unqualified 45.05% 45.26% 75.17% 75.17% 91.35% 91.49% 97.91% 97.91% 99.58%
Qualified 90.00% 90.00% 69.31% 69.14% 39.31% 39.31% 17.07% 17.07% 4.66%
KNN
Disclaimer 90.83% 90.83% 85.32% 85.32% 56.88% 56.88% 33.03% 33.03% 13.76%
Adverse 87.50% 87.50% 68.75% 68.75% 37.50% 37.50% 31.25% 31.25% 6.25%
Unqualified 53.00% 79.29% 82.01% 84.45% 95.19% 96.58% 96.79% 97.49% 99.37%
Qualified 80.69% 55.69% 51.55% 47.93% 26.72% 22.59% 21.03% 16.38% 6.90%
MLP
Disclaimer 98.17% 91.74% 91.74% 90.83% 70.64% 63.30% 61.47% 51.38% 24.77%
Adverse 87.50% 75.00% 56.25% 50.00% 31.25% 18.75% 18.75% 12.50% 0.00%
Unqualified 32.71% 62.06% 78.52% 86.19% 91.84% 95.12% 97.14% 98.40% 99.51%
model
Adverse 81.25% 81.25% 75.00% 62.50% 62.50% 43.75% 37.50% 25.00% 0.00%
55
EJAE 2019  16 (2)  1-58
THE TWO?
Unqualified 56.97% 75.24% 82.36% 87.59% 90.38% 93.93% 96.03% 98.05% 99.23%
Stacked
model Qualified 92.76% 86.21% 78.97% 72.41% 65.86% 57.76% 48.79% 37.93% 22.24%
LER
Adverse 81.25% 68.75% 62.50% 56.25% 50.00% 43.75% 31.25% 25.00% 18.75%
Unqualified 62.27% 76.29% 83.47% 87.59% 90.38% 93.03% 94.56% 96.72% 98.54%
M-LER
with Qualified 87.93% 81.55% 75.52% 70.34% 64.14% 57.07% 50.00% 38.79% 24.66%
GRRF Disclaimer 94.50% 92.66% 91.74% 88.07% 86.24% 80.73% 76.15% 66.06% 53.21%
dummies
Adverse 75.00% 62.50% 50.00% 50.00% 50.00% 43.75% 37.50% 31.25% 12.50%
Table F3. Predictive Accuracy by Specific Type of Audit Opinion: Models with Theory- and Data Driven Predic-
tors, OOT Predictions
Cut-off
Type of
Model Audit 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90
Opinion
Unqualified 17.42% 43.63% 61.79% 78.48% 89.75% 94.73% 97.22% 98.98% 99.71%
Qualified 94.75% 84.59% 72.13% 56.72% 41.31% 24.92% 14.43% 7.54% 2.30%
C5.0
Disclaimer 100.00% 95.59% 92.65% 85.29% 73.53% 63.24% 51.47% 32.35% 10.29%
Adverse 100.00% 100.00% 100.00% 77.78% 66.67% 55.56% 55.56% 33.33% 0.00%
Unqualified 10.25% 33.24% 59.59% 79.36% 94.14% 97.66% 98.98% 99.85% 100.00%
Qualified 97.38% 89.18% 76.39% 57.38% 35.08% 18.69% 7.21% 3.28% 0.33%
RF
Disclaimer 100.00% 100.00% 95.59% 86.76% 70.59% 52.94% 32.35% 13.24% 2.94%
Adverse 100.00% 100.00% 100.00% 88.89% 66.67% 55.56% 44.44% 33.33% 0.00%
Unqualified 8.49% 29.87% 53.59% 72.18% 90.48% 97.22% 98.24% 99.56% 100.00%
Qualified 98.69% 91.48% 81.31% 64.92% 40.33% 21.97% 12.13% 4.26% 1.31%
RRF
Disclaimer 100.00% 100.00% 97.06% 91.18% 75.00% 52.94% 32.35% 19.12% 5.88%
Adverse 100.00% 100.00% 100.00% 88.89% 77.78% 66.67% 44.44% 44.44% 11.11%
Unqualified 14.06% 43.19% 66.47% 79.50% 87.85% 94.44% 96.78% 98.83% 99.27%
Qualified 97.70% 82.30% 72.13% 58.03% 40.98% 30.82% 20.98% 9.84% 1.97%
GBM
Disclaimer 100.00% 100.00% 94.12% 88.24% 79.41% 72.06% 57.35% 35.29% 11.76%
Adverse 100.00% 100.00% 88.89% 88.89% 77.78% 77.78% 66.67% 44.44% 11.11%
Unqualified 30.60% 52.56% 65.74% 75.99% 82.58% 90.04% 93.70% 97.07% 99.27%
XG- Qualified 88.20% 73.44% 60.33% 51.80% 44.92% 36.72% 27.87% 19.67% 9.18%
Adverse 100.00% 88.89% 77.78% 77.78% 77.78% 77.78% 66.67% 66.67% 55.56%
56
EJAE 2019  16 (2)  1-58
THE TWO?
Unqualified 36.16% 36.16% 65.74% 65.74% 86.68% 86.68% 96.34% 96.34% 99.27%
Qualified 82.62% 82.62% 56.72% 56.39% 29.84% 29.84% 12.79% 12.79% 3.93%
KNN
Disclaimer 94.12% 94.12% 67.65% 67.65% 41.18% 41.18% 20.59% 20.59% 10.29%
Adverse 88.89% 88.89% 66.67% 66.67% 44.44% 44.44% 22.22% 22.22% 11.11%
Unqualified 50.37% 77.16% 79.94% 81.84% 93.12% 95.75% 96.78% 97.80% 98.98%
Qualified 78.03% 55.74% 53.11% 52.79% 25.90% 21.64% 19.34% 14.10% 3.28%
MLP
Disclaimer 97.06% 79.41% 77.94% 76.47% 50.00% 44.12% 44.12% 39.71% 20.59%
Adverse 100.00% 66.67% 66.67% 66.67% 55.56% 55.56% 55.56% 55.56% 44.44%
Unqualified 23.57% 51.54% 67.79% 79.80% 88.14% 94.29% 97.51% 98.24% 99.71%
model
Adverse 100.00% 100.00% 88.89% 77.78% 77.78% 77.78% 55.56% 44.44% 33.33%
Unqualified 39.82% 62.96% 75.11% 81.70% 87.85% 92.09% 95.90% 97.36% 98.83%
Stacked
model Qualified 88.20% 77.38% 63.61% 55.08% 46.56% 38.36% 28.20% 20.00% 8.85%
LER
Adverse 100.00% 100.00% 100.00% 77.78% 77.78% 66.67% 55.56% 55.56% 22.22%
Unqualified 44.95% 64.71% 78.18% 84.33% 88.87% 90.63% 93.56% 97.22% 99.27%
M-LER
with Qualified 75.74% 60.98% 49.18% 40.33% 32.79% 28.52% 22.62% 13.44% 6.89%
GRRF Disclaimer 91.18% 82.35% 79.41% 75.00% 63.24% 63.24% 55.88% 39.71% 26.47%
dummies
Adverse 100.00% 100.00% 88.89% 77.78% 77.78% 66.67% 66.67% 33.33% 22.22%
Table F4. Predictive Accuracy by Specific Type of Audit Opinion: Models with Theory- and Data Driven Predictors,
OOS and OOT Predictions
57
EJAE 2019  16 (2)  1-58
THE TWO?
PREDVIĐANJE VRSTE REVIZORSKOG MIŠLJENJA: STATISTIKA,

MAŠINSKO UČENJE ILI KOMBINACIJA NAVEDENIH?
Rezime:
Cilj ovog istraživanja je prevazilaženje metodoloških ograničenja uočenih
u prethodnim istraživanjima u oblasti predviđanja vrste revizorskog
mišljenja i izvlačenje pouzdanih zaključaka o uporedivim prediktivnim
performansama različitih metoda koje se koriste u te svrhe. Prediktivne
performanse dvanaest modela iz oblasti statistike i mašinskog učenja
su ocenjene u dva različita praktična scenarija: a) kada su prethodne
informacije (vrste revizorskih mišljenja) o klijentu dostupne i mogu se
koristiti za predikciju i b) kada su prethodne informacije nedostupne
(npr. novoosnovana društva). Rezultati pokazuju da, u prvom scenariju,
nekoliko metoda iz obe grupe prediktivnih metoda ostvaruju upore-
dive performanse u vrednosti od 0,89, mereno površinom ispod krive
(eng. Area under the curve). U drugom scenariju, međutim, algoritmi
mašinskog učenja, posebno oni zasnovani na drveću odlučivanja, kao
što je random forest, ostvaruju značajno bolje rezultate od statističkih
metoda, i to u vrednosti od 0,79. Razvili smo i ocenili performanse dva
hibridna modela, koji za cilj imaju da iskoriste prednosti statističkih
metoda (interpretabilnost rezultata) i metoda mašinskog učenja (obrada
velikog broja objašnjavajućih varijabli i veća preciznost). Celokupna
procedura je prikazana na reproducibilan način, uz korišćenje najvećeg
empirijskog skupa podataka korišćenog u dosadašnjim istraživanjima
Ključne reči:
ovog tipa, koji obuhvata 13.561 par godišnjih finansijskih izveštaja i
mišljenje revizora, finansijski
korespondirajućih revizorskih izveštaja. Procedure opisane u ovom
izveštaji, generalizovani linearni
članku omogućavaju revizorskim kućama i finansijskim službenicima
mešoviti modeli, random forest
širom sveta da razviju i testiraju prediktivne modele koji podržavaju
(drveće odlučivanja), statistički
procedure revizorskog planiranja i ocenu rizika ispravnosti podataka
paket GRRF (Guided Regularized
u finansijskim izveštajima.
Random Forest), skupovi
58
THE ROLE OF TECHNOLOGY AS AN ABSORPTIVE CAPACITY IN

ECONOMIC GROWTH IN EMERGING ECONOMIES: A NEW APPROACH
Richard Angelous Kotey*, Joshua Yindenaba Abor
Department of Finance
University of Ghana Business School

Studies have shown that the effects of Foreign Direct Investment (FDI)
on economic growth have not always been direct, especially in developing
regions; certain characteristics must exist in the economy for the effects
of FDI to be well absorbed. Therefore, this study sought to assess the
economic impact of FDI on economic growth in Sub-Saharan African
(SSA) countries, factoring in technology as an absorptive capacity.
Because of the scarcity of data on a viable proxy for technology in the
African context, we measure technology in a novel approach, using
annual number of published innovation-related papers as a proxy for
technological presence. Data from forty-three Sub-Saharan countries
over a 19-year period (from 1990 to 2008) was analyzed. Using a Fixed
Effects (FE) regression model, the study found that FDI had a negative
and significant effect on GDP, which is our proxy for economic growth.
However, when FDI is interacted with technology, the relationship turns
positive and significant. This implies that countries with technological
presence are more able to absorb from FDI than those with little technol-
ogy. Furthermore, the study found that countries with high technology
were able to absorb more from FDI than those with low technology.
Keywords:
foreign direct investment,
technology, innovation, absorptive
capacity
Jel classification:
C23, O47, F23, O11
59
*E-mail: rakotey@st.ug.edu.gh
EJAE 2019  16 (2)  59-78
KOTEY R. A ., ABOR J. Y.  THE ROLE OF TECHNOLOGY AS AN ABSORPTIVE CAPACITY ON ECONOMIC GROWTH IN EMERGING ECONOMIES: A NEW
APPROACH
INTRODUCTION
Foreign Direct Investment (FDI) has been an important mechanism for universal growth and
development through trade and economic interaction since the emergence of globalization in the last
three or so decades. This is especially so for developing countries because of their peculiar challenges,
the difficulty in assessing the international capital market, weak domestic markets, and low levels of
income and savings. Therefore, developing nations tend to look beyond their borders for investment
that will generate enough growth for them. FDI and official loans from multinational institutions (such
as the IMF and World Bank) have been viable means of accessing such capital investments (Aseidu,
2002). In the case of FDIs, the main drivers at the national level -that is, the policymakers- have im-
plemented reforms to promote trade among countries which foster economic advancement. This has
led to the progressive breakdown of international barriers (Twarowska & Kakol, 2013), and improved
shared prosperity among countries with competitive advantage and efficiency. Not only has this made
investment opportunities available for both the private and public sector (Sinani & Meyer, 2004), but
also a means through which technology innovations can be shared (IMF, 1991; Meyer, 2001).
Multinational Corporations (MNCs) play a major role in global innovation (Kotey, 2019). Accord-
ing to a UNCTAD report in 2005, one half of the world's total expenditure in R&D comes from MNCs.
This figure increases to more than two-thirds when considering the cost of R&D in the business sector
alone (UNCTAD, 2005). It is, therefore, no surprise that about 80 percent of the world's technologies are
owned by MNCs (Dunning, 1992). Much of the research MNCs undertake is usually done in developed
economies; little or no research is done in developing economies (UNCTAD, 2005; UNCTAD, 2010;
Kotey, 2019). This may account for the widening technology gap between the developing and developed
economies. However, through interaction between MNCs and local firms in developing economies,
particularly through direct investments, knowledge and technology may trickle down to the local firms,
particularly through mechanisms like imitation, competition, and backward and forward linkages.
Table 1 R&D Expenditure of Selected MNCs and Other Countries

R&D Expenditure (in R&D Expenditure in
MNC Countries
million $) 2009 (in million $)
Toyota 9403 USA *(2013) 473400
Microsoft 8437 China *(2015) 409000
VW 8043 Germany 11799.80
Pfizer 7507 United Kingdom 8731.63
Novartis 7163 Japan 4185.27
Nokia 6942 Canada 3639.43
Johnson & Johnson 6764 Sweden 3251.97
Samsung Electronics 6265 Brazil 343.55
General Motors 5875 Ethiopia 5.48
Honda Motors 5857 South Africa *(2012) 4.80
Daimler 5785 Botswana 1.95
Intel 5473 Rwanda 0.24
Sony 5172 Côte d'Ivoire *(2010) 0.14
IBM 4787 Benin 0.04
Takeda Pharmaceutical 4712 Togo 0.02
Author’s own computation. R&D expenditure of selected MNCs and countries in 2009. Source UNCTAD 2010
60
EJAE 2019  16 (2)  59-78
KOTEY R. A ., ABOR J. Y. THE ROLE OF TECHNOLOGY AS AN ABSORPTIVE CAPACITY ON ECONOMIC GROWTH IN EMERGING ECONOMIES: A NEW
APPROACH
Technology is an important tool for firms to remain competitive both on the micro and macro levels.
Technology improves the quality of outputs, reduces the production and processing time, and reduces
the cost of production (UNCTAD, 2010). It is key for a country to have relevant and contemporary tech-
nology. However, because of the cost of creating modern technology, developing economies tend to be
handicapped in this aspect, as they mostly cannot afford it. Still, through trade globalization, technology
can be transferred through contagion. This occurs through the rub-off effect as companies interact in
the global market. Through this effect, knowledge is able to trickle down from the developed economies,
which usually have more technology, to developing economies, which usually have less technology.
The economic impact of FDI on growth has been well researched in academia (Alfaro, Chan-
da, Kalemli-Ozcan & Sayek, 2004; Borensztein, De Gregorio & Lee, 1998; Carkovic & Levine, 2005;
Chakraborty & Nunnenkamp, 2008; Li & Liu, 2005, Kotey 2019; Kombui & Kotey, 2019). Though some
studies have been done on technology as a spillover effect on FDI (Blomstrom & Kokko, 1996; Bwalya,
2006; Dutse, 2012; Ghali & Rezgui, 2008; Marin & Bell, 2006; Sinani & Meyer, 2004), few studies have
adopted technology as an absorptive capacity1 in an FDI-economic growth relationship (See Figure
1). A typical study that has looked into absorptive capacity is Agbloyor, Gyeke-Dako, Kuipo, & Abor
(2016), who looked at the relationship between FDI and growth when institutions are factored in, and
found that countries with strong institutions have higher economic growth through FDI than countries
with weak institutions. Although studies on technology have shown that FDI is a major way technology
reaches developing economies, not many studies have looked at its effect as an absorptive capacity on
economic growth (Liu, 2008; UNCTAD, 2010). There is, therefore, a need for a critical look at FDI on
economic growth from the lens of technology as an absorptive capacity. Also, there is still very little
evidence on the impact of technology as an absorptive capacity on economic growth factoring in FDI
in an African context (Figure 2). The reason is partly due to little data being available on technology;
there are very few variables to measure technology in African countries, mainly because most of the
countries do not have data on R&D, patents, etc. This paper is unique in that it examines the impact
of FDI on growth in Sub-Saharan African (SSA) countries using technology as an absorptive capacity.
It also adopts a new approach to measure technology by using the presence of published technology-
enhancing research in the host country as a proxy for technological presence.
Figure 1 Published Articles over Selected Time Frame
Author’s own computation. The authors counted all published articles on FDI effects on growth and productivity, as well as
FDI on technology from 1998 to 2016 within the science direct database. The graph shows that there have been few studies
on FDI and Technology over the years among the three categories.
1 Absorptive capacity, as defined by (Rehman, 2016), is the capability of host economies to absorb or internalize external
traits from FDI spillovers. That said, if a host form has a high absorptive capacity, it would be able to benefit or absorb
more from FDI, and vice versa.
61
EJAE 2019  16 (2)  59-78
APPROACH
Figure 2 FDI Studies done in Selected Economic Regions around the World
Author’s own computation. Data from Science direct.
LITERATURE REVIEW
Studies on FDI and economic growth broadly fall into two categories; studies that have found a
direct relationship between FDI and economic growth, and studies that have found that the relationship
between FDI and growth is not so direct, thus absorptive capacities form part of the relationship. For
example, in some developing economies, it was more challenging for researchers to find a consistent
relationship between FDI and economic growth. Instead, they tested the relationship on the presence
of absorptive capacities (Clark, Highfill, Campino, & Rehman, 2011; Kotey, 2019).
Some studies that found FDI’s effect strengthened with the presence of an absorptive capacity
include Agbloyor, Gyeke-Dako, Kuipo, & Abor (2016); Borensztein, De Gregorio, & Lee (1998); Li
& Liu (2005); Ramirez (2006); Rehman (2016); Sinani & Meyer (2004). Li and Liu (2005) researched
84 countries (from developed and developing economies) from 1970 to 1999, examining the causal
relationship between FDI and economic growth. They found that the relationship between economic
growth and FDI is strong and positive when interacted with human capital. Borensztein et al. (1998)
also found a similar result; human capital positively affects FDI and growth. Other studies have shown
a link between FDI and technology. Ramirez (2006) conducted a study in Mexico using a 1960 to 2001
time-series data. He found out that FDI increased labour productivity. Additionally, Sinani and Mayor
(2004) studied the spillover effect from technology from FDI, sampling domestic firms in Estonia from
the period from 1994 to 1999. Their study revealed that the magnitude of technology spillover depended
on the characteristics of both the FDI inflow and the local firm; the magnitude of foreign presence and
the firm size affected the spillover effect. Chakraborty and Nunnenkamp (2008) also did a similar study
in India. They found that trade liberalization magnified FDI’s effect on domestic companies.
62
EJAE 2019  16 (2)  59-78
APPROACH
Theoretical Framework on technology as an absorptive capacity
There are theories that have been used to explain FDI, such as the product life cycle theory, ex-
change rate theory, dunning’s eclectic paradigm, among others. But these theories explain only FDI.
In order to find a meaningful theory to explain technology as an absorptive capacity to FDI, we delve
into theories in the area of industrialization. Specifically, the main theories that underpin the study are
the dependency and modernization theory.
Dependency Theory
Dependency theory first emerged in the early 1970s. Dependency theory thrives on the following
assumptions: that the world is a capitalist economy, that foreign investments always move from de-
veloped economies to developing ones, and that developed nations extract resources from developing
nations (Apter, 1987; Larrain, 2013; Scott, 1995; So, 1990). Dependency theorists believe that FDI does
not lead to economic growth, at least not in the long run. They believe that, through foreign investment,
developed countries deprive developing countries of the natural resources they need to develop, making
them dependent on the foreign firms or states for economic growth. Thus, they become monopolists,
causing unfair completion in the local markets (Adams, 2009). Sylwester (2005) also stated that de-
pendency theorists believe FDI has a crowding-out effect that affects domestic investment by raising
the costs of investments and also causing market distortions that are detrimental to economic growth
and development. We assume that foreign investors coming into SSA countries out-compete the local
firms due to their better innovations (Matunhu, 2011), thereby creating an opportunity technology to
be shared or transferred.
Modernization Theory
Modernization theory first appeared in the 1950s and 1960s, and has evolved over time (up to the
late 1990s). It has no single proponent, but has been attributed to American social scientists from the
early 1950s (Preston, 2012). As it evolved, its definition has expanded (Fourie, 2012; Lehmann, 2010).
The modernization theory generally explains how a country modernizes or changes from its traditional
way of life to a modern one (Apter, 1987; Scott, 1995; So, 1990). The study adopts the Economic version
of the Modernization Theory. This theory explains how technology and social innovations are able to
spur growth. The study is particularly supported by the Diffusion of Innovations Theory, which essen-
tially explains why innovation spreads, and measures the rate at which it is able to spread. On the other
hand, the Economic Modernization Theory specifically stipulates that FDI is necessary for economic
progress in a country. This study agrees with the modernization theory, in that developing economies
cannot obtain the technology needed for economic growth without the presence of FDI. FDI provides
the necessary resources the local economies require to swiftly achieve economic development. The
model for the study also supports this thinking.
Based on these underlying frameworks, we hypothesize that the presence of technology affects the
ability of a country to absorb more of the effects of FDI.
63
EJAE 2019  16 (2)  59-78
APPROACH
METHODOLOGY
A quantitative research approach was adopted for the study. An unbalanced annual panel data of 43
Sub-Saharan African countries from 1990 to 2008 was used for the study. We employed a fixed effects
(FE) model estimation after conducting preliminary tests; we conducted a Hausman test to determine
the model appropriateness, and tested for heteroscedasticity, endogeneity, simultaneity, and reverse
causality issues (see Appendix 3). The large number of countries and multiple years give a higher de-
gree for freedom and credibility to the findings of the study (Brooks, 2008; Baltagi, 2001; Jensen 2012).
For reciprocity, the data (DOI:10.17632/tfjwys9s5p.1) used for the study has been uploaded on the
Mendeley data repository, which can be retrieved from https://data.mendeley.com/datasets/tfjwys9s5p/1.
The main statistical software used to analyze the data is STATA.
Regression Model
The regression model takes the form;+
yit =α + βχ it + µi + ε it (1)
Where Y is the dependent variable for country i at time t and X is a set of explanatory
variables for country i at time t . μi is the country-specific fixed effect, which is time-
invariant. a is the constant term. ß represents the coefficients to be estimated for the
independent variables, and εit is the error term or idiosyncratic noise.
The model for the study is specified as:
GDPit = α + β1 FDI it + β 2TECH it + β 3 ( FDI ∗ TECH )it + ∑ µj i = 1 β j X it + µi + ε it (2)
Here are the interpretations:

a denotes the constant. It is the expected value of the dependent variable when all independent
variables are set to zero.
FDIit denotes the Foreign Direct Investment (FDI) net inflows into country i at time t . It refers
to direct investment equity flows entering into a country. It can also be said to be the net inflows from
foreign investment enough to acquire at least 10 percent voting rights in a domestic firm. It is the sum
total of equity capital, reinvestment of earnings, and other capital.
ß1 is the coefficient of FDI to be estimated in the regression model. We expect a relationship other
than positive because of our sample set. Studies done in developing economies usually show a negative
relationship.
TECHu denotes the Technology variable for each country and each year. We proxy technology by
the number of annual publications in scientific and technical journal articles, which is an indication of
the presence of innovation or technology in the local country. This refers to the sum of scientific and
64
EJAE 2019  16 (2)  59-78
APPROACH
engineering articles or publications in physics, biology, chemistry, mathematics, clinical medicine,

biomedical research, engineering and technology, and earth and space sciences fields published within
the year. We assume that this type of technology or innovation is growth-enhancing. We also impose
an assumption that technological presence is reflected in the number of local applied science papers
published.
ß2 denotes the coefficient for TECH variable. We expect a positive relationship indicating the posi-
tive relationship between economic growth and technology.
(TECH* FDI) denotes the interaction between FDI inflows and Technology. We refer subsequently
to this variable as our interaction term.
ß3 represents the coefficient of the interaction term. We expect the interaction to be positive. A
positive coefficient suggests that countries with the presence of innovation or technology are better
able to absorb FDI. Thus, the countries are able to benefit from FDI in terms of achieving higher levels
of economic growth.
denotes the control variables. We include a set of information conditions to be sure we are
capturing the effect of technology and FDI indicators on economic growth. Following Adams (2009)
and Agbloyor et al. (2016), our control variables include; government expenditure, political stability,
labour force, and trade openness.
Table 2 Model Variables, Interpretation and Sources

Variable Meaning and interpretation Source
Annual FDI inflows coming into the country. The vari- International Monetary
lnFDI able is log-transformed to reduce variation and make it Fund(IMF), Balance of Payments
normally distributed. database
Technology present in the host country. Measured by
National Science Foundation, Sci-
Tech the number of annual publications in applied science
ence and Engineering Indicators.
journals.
Absorptive capacity: derived by FDI*TECH. The vari-
lninteraction able is log-transformed to reduce variation and make it World Bank data
normally distributed.
Annual GDP in constant US$. The variable is log-
World Bank national accounts,
lnGDP transformed to reduce variation and make it normally
OECD national accounts
distributed.
Annual government expenditure in constant US$. The
lnExpcu variable is log-transformed to reduce variation and World Bank national accounts
make it normally distributed.
Political stability. Measures the level of political free-
Pstab World Governance Indicators
dom (legal and political risk) in the country.
World Bank national accounts
Country openness to the external world. Measured by
Open data, OECD national accounts
the sum of imports and exports multiplied by GDP
data
Labour force. Defined as people 15 years and older who
International Labour Organization
Labper meet the International Labour Organization definition
database
of the economically active population.
65
EJAE 2019  16 (2)  59-78
APPROACH
Justification of Technology Variable
The common proxy for measuring technology (see Appendix 1) in literature has been intellectual
property rights (patents, copyrights, industrial design, and utility model, etc.), and R&D factors, such
as R&D expenditure, employees, etc. (see Branstetter, Fisman , & Foley, 2006; João , 2010; Liu, 2008;
Neven & Siotis, 1996). Considering such data is non-existent for Sub-Saharan Africa, this study uses a
new approach for measuring technology from FDI.
We use the annual sum of scientific and technical journal publications by each country as a proxy.
In our estimation, journal publications represent new knowledge or technology that has been found
or created. We expect a positive relationship with economic growth (see Appendix 2).
To use this variable as a proxy for technology, we make some key assumptions;
1. Applied science publications in the respective countries represent technology creation or tech-
nological presence.
2. This type of technology is economic growth-enhancing.
3. The relationship between technology and the number of applied science publications is positive.
Descriptive Statistics
The table below presents the descriptive statistics:
Table 3 Descriptive Statistics
Variable Observations Mean Std. Dev. Min Max
lnGDP 808 9.521283 0.581875 8.003491 11.47627
lnFDI 803 7.608242 1.006778 2 9.994977
Tech 785 156.7213 598.5485 0.2 6137.3
lninteraction 776 8.962239 1.533451 2.897627 13.78295
lnExpcu 728 8.679416 0.594942 7.176319 10.7284
Open 778 73.12675 49.07174 0 531.7374
Labper 789 0.108678 0.083699 0.001791 0.419002
Pstab 430 -0.481628 0.905858 -2.994749 1.19232
Because the SSA countries in the model are from both developed, under-developed and developing
economies, there is a wide variation in their GDP values.
Observing the log-transformed values of FDI and GDP, GDP has a higher mean than FDI, as ex-
pected. But, the FDI variable shows a higher variation than that of GDP, which shows that FDI inflows
in the dataset vary much widely as compared to the GDP values.
66
EJAE 2019  16 (2)  59-78
APPROACH
The mean value of the technology variable of 156.72 indicates that, on average, 157 publications are
published annually by the SSA countries in our dataset, and this varies by about 599 publications. The
mean of the log-transformed government expenditure is similar to the mean of the interaction term,
but its standard deviation is lower, indicating a lower variation.
Trade openness accounts for how open the SSA countries are to the external world. On average, the
level of trade openness is 7313%, with a standard deviation of 4910%.
On average, about 11% of the population is over 15 years and economically active in the sampled
data. The standard deviation of 8%, which shows how little variation is within the mean.
The political stability data only starts from 1996, hence the lower number of observations of 430.
On average, the level of political stability is -0.48, which is not surprising, generally, because the level
of political stability is very low among SSA countries.
Correlation and Covariance
The strength and direction of correlation among the individual variables are demonstrated in the
correlation table.
Table 4 Correlation Table

lnGDP lnFDI Tech lninteraction lnExp Open labper pstab
lnGDP 1.0000
lnFDI 0.6971 1.0000
Tech 0.5976 0.3340 1.0000
lninterac- 0.8438 0.8861 0.5162 1.0000

tion
lnExpcu 0.9457 0.6386 0.5983 0.8048 1.0000
Open -0.0468 0.2823 -0.0779 0.0731 -0.0283 1.0000
Labper -0.4219 -0.4879 -0.2352 -0.4604 -0.4248 -0.3828 1.0000
Pstab -0.0245 0.0750 -0.0035 0.1114 0.0652 0.1746 -0.4419 1.0000
As can be seen from the table above, there is a stronger correlation between FDI and GDP, Tech
and GDP, GDP and government expenditure, and the interaction term and GDP than expected, with
correlations coefficients above 0.50 in all cases. The rest of the variables have a weak correlation with
the dependent variable. FDI and the interaction term are strongly correlated, with a correlation coef-
ficient of 0.89, and this is expected, since the interaction term is the product of FDI and technology.
The interaction term is also strongly correlated with Technology, with a coefficient of 0.512. Moreover,
government expenditure seems to be strongly correlated with GDP and the interaction term. The rest
of the correlation coefficients are lower than 0.50.
67
EJAE 2019  16 (2)  59-78
APPROACH
RESEARCH FINDINGS
Scatterplot matrix
We present a scatterplot matrix to examine the pictorial relationship between GDP (in logged val-
ues), technology, the interaction term, and their log-transformed values.
In the scatterplot of the dependent variable and the interaction term, we observe a positive relation-
ship. Further examinations show that the interaction term is high when GDP is high enough or reaches
a certain point (or threshold). In the region where GDP is much lower (from 0 to 10), the interaction
term remains leveled and close to zero. This may mean that, at low GDP levels, the technology is very
low, and its absorptive capacity tends to be low also. But when GDP is high enough, the technology is
high enough to absorb more from FDI.
We observe a positive relationship between the dependent variable and the log of the interaction
term. However, there are no data points from point 0 to 5 on the logged interaction’s axis. Above point
5, we see the great number of data points clustered and from point 12 to 15, we see much less clustering
and data points. This may suggest there are more countries with low technology compared to those
with high technology. That may also mean countries with low GDP’s may have low FDI absorption
compared with countries with high GDP’s.
The relationship between technology and the interaction term is generally a positive one, albeit
some data points show that high technology occurs even at low levels of technology. We thus see a bi-
directional relationship in the scatterplot; one set shows that technology or technological presence is
increasing when the level of the interaction term is also increasing; the other shows that technology is
not much affected by the interaction term (indicating they are not affected by technology). The increas-
ing positive relationship shows a positive relationship between technology and FDI.
Figure 3 Scatterplot Matrix
Source: Authors’ own computation
68
EJAE 2019  16 (2)  59-78
APPROACH
When we look at the relationship between the logged values of the technology and interaction term,
we observe a similar positive relationship between them where the data points begin at some point and
are clustered in the region closer to the origin.
Regression analysis
We estimate our regression model using fixed effects. We regress GDP on FDI, Technology, the in-
teraction term, and the control variables. We also include an OLS regression as our first model for the
sake of comparison. We present our fixed effects (FE) regression results in the second column labeled M2
(meaning model 2), and another FE regression result in model 3 (M3). In M1, we include all the variables
in the regression and run an OLS regression. In M2, we include all our variables and run an FE regression.
In M3, we remove political stability and run an FE regression. We include M3 because the number of
observations reduces to about half when political stability is included in the model (because the data starts
from 1996), so we remove it to observe whether a change in the sample observation could affect the results.
When we run on OLS, the technology variable is significant, at 10%, and the interaction term is
significant, at 1%. The R-squared of 93% also suggests a high predictive power of the model. The coef-
ficient of technology and its standard errors are positive, and close to zero. However, the coefficient
for the interaction term is 0.07, and significant at 5%, showing that GDP increases by 0.07% when
the interaction term is 1%. As explained, the number of observations is 350. As can be observed, the
coefficient for the interaction term is higher than that of FDI. This may support the assertion that
technological presence may have an increased effect of FDI on economic growth. We then proceed to
our main regression results (M2 and M3) to see if the situation is the same or not.
R-squared for the M2 is 88%, signaling that 88% of the variation in GDP is caused by the independent
variables in our linear model. The adjusted R square is 87%, which is also high. Although this is a cross-
country study, it is not surprising to have such high R squares; as some studies on economic growth and
FDI in Sub-Saharan Africa have had similar high R squares (Adams, 2009; Abor, 2010). The number of
observation is 350, and the number of groups is 38. The table below presents the results of the regression.
Our independent variables are significant in M2 and M3 (at 1% and 5% significance levels), with
the exception of political stability, which is insignificant.
69
EJAE 2019  16 (2)  59-78
APPROACH
Table 5 Regression table (Fixed effects)

(M1) (M2) (M3)
VARIABLES lnGDP lnGDP lnGDP
lnFDIcu 0.0315 -0.104*** -0.0938***

(0.0223) (0.0203) (0.0154)
Tech 2.43e-05* 5.28e-05*** 3.41e-05**
(1.37e-05) (1.51e-05) (1.37e-05)
lninteraction 0.0708*** 0.138*** 0.117***
(0.0168) (0.0184) (0.0134)
lnExpcu 0.692*** 0.384*** 0.521***
(0.0254) (0.0303) (0.0249)
Open -0.000717*** -0.00212*** -0.00104***
(0.000205) (0.000151) (0.000147)
labper -0.583*** -2.081*** -1.733***
(0.157) (0.184) (0.132)
pstab -0.0778*** 0.00116
(0.0107) (0.0108)
Constant 2.737*** 6.162*** 4.954***
(0.197) (0.267) (0.220)
Observations 372 372 677

R-squared 0.928 0.884 0.842
Adjusted R-squared 0.9271 0.868 0.831
Number of countries 39 39
Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1
In M2, we observe that lnFDI is significant, at a 1% alpha level, but has a negative effect on GDP,
with a coefficient of 0.104. This implies that GDP reduces by 0.104% when FDI inflows are increased
by 1%. In M3, the coefficient of FDI is much smaller, with a negative coefficient of 0.094, signaling that
GDP reduces by 0.094% when FDI is increased by 1%. This is congruent with studies done on economic
growth in Africa; FDI tends to be negatively related to GDP. Asiedu (2002) explained that FDIs that
come to Africa are extractive in nature, and usually do not seek to satisfy or serve the local market.
Therefore, they do not contribute as much towards economic growth as expected, aside from the taxes
they pay to the local government and the low-level professionals they employ locally.
The technology variable is significant, at 99% and 95% confidence interval in M2 and M3, respectively.
However, the coefficients are positive, and close to 0. This means the technology present in the sample
countries has a positive relationship with economic growth; however, its effect on economic growth is
very minimal. This is in congruence with the observations from the scatter plot matrix.
When we interact FDI with technology, its effect is more pronounced on the dependent variable. We
observe from the coefficients in M2 and M3, respectively, that GDP increases by 0.138% and 0.117% when
the interaction term increases by 1%, all being significant at a 1% level of significance. The coefficients
show that FDI is more absorbed when technology is present in the host country, as observed by com-
70
EJAE 2019  16 (2)  59-78
APPROACH
paring the coefficients of FDI and the interaction term. This may also mean that countries with higher
technology are able to transform the negative impact of FDI on economic growth into a positive one.
Government expenditure also has a positive relationship on economic growth, and is very signifi-
cant, at a 1% level of significance. The coefficients show that GDP increases by 0.384% and 0.521% in
M2 and M3, respectively, when government expenditure increases by 1%. This shows that, for SSA
countries, government spending leads to higher economic growths, as government expenditures are
generally aimed towards infrastructural and economy-wide growth.
Trade openness is also significant at a 99% confidence interval, although the effect to GDP is nega-
tive. The coefficients show that GDP reduces by 0.2%, and 0.1% M2 and M3 respectively, when trade
openness increases by 1%. We observed from our sample countries that imports generally exceed export
levels, whilst exports are usually unprocessed raw materials and imports finished value-added products.
This means trade openness harms, rather than benefits, economic growth.
This is also supported by the stylized fact that FDI that comes into SSA countries are usually in the
extractive industry. Because a higher chunk of the FDI inflows coming into SSA countries goes into the
extractive sectors (e.g. mining, oil exploration, etc.), the expected growth effect from these investments
are not substantially beneficial to the local markets, since a higher portion of revenues are hauled into
external markets.
Labour force is also significant, at a 1% level in all three models. However, the coefficients are negative
in each case. The interpretation is that, although the sample countries do have a high labour force, this does
not automatically translate into high economic growth. The authors reason that a high quantum of the la-
bour force is mostly unskilled and unspecialized; therefore, they are not able to significantly increase output.
As expected, the political stability variable is positively correlated with GDP. However, the coef-
ficient is insignificant, at a 5% alpha level.
CONCLUSION
There is evidence that technology increases, not only productively, but has a ripple effect on the
economic growth of SSA countries. Technology mostly comes into developing economies through
MNC engagements in the form of FDI. Since the level of technology present in the country also af-
fects the country's ability to absorb FDI, SSA countries are not benefiting much from FDI due to low
levels of the technology present in such economies. Hence, it is important for SSA countries to adopt
technology-enhancing strategies in order to absorb more from FDI inflows.
RECOMMENDATIONS
Technology has a positive relationship with growth. Therefore, for an increase in economic growth,
higher levels of technology are needed to absorb more from FDI inflows. Therefore, local governments
in SSA countries must put measures into place that create and absorb higher levels of technology. In-
dustrialization must be encouraged, businesses must be given the necessary impetus, and there should
be an investment in young entrepreneurs. This will help increase the level of technology, and that will
have a ripple effect on growth.
71
EJAE 2019  16 (2)  59-78
APPROACH
In addition, not all FDI is beneficial to host countries’ growth. This is why local governments must
collaborate with MNCs that can help the economy. The extractive kind of FDI must be minimized.
Finally, local firms must be encouraged to collaborate with MNCs. Through collaborations, MNCs
transfer – directly or indirectly- knowledge and technology to local firms. This could increase techno-
logical presence in the country.
REFERENCES
Adams, S. (2009). Foreign Direct Investment, domestic investment, and economic growth in Sub Saharan Africa.
Journal of Policy Modeling, 31, 939–949.
Agbloyor, E. K., Gyeke-Dako, A., Kuipo, R., & Abor, J. (2016). Foreign Direct Investment and Economic Growth
in SSA: The Role of Institutions. Thunderbird International Business Review, 58, 479–497.
Alfaro, L., Chanda, A., Kalemli-Ozcan, S., & Sayek, S. (2004). FDI and economic growth: the role of local financial
markets. Journal of international economics, 64(1), 89-112.
Apter, D. E. (1987). Rethinking development: modernization, dependency, and postmodern politics (No. 330.9
A6). Beverly Hills, CA: Sage.
Aseidu, E. (2002). On the Determinants of Foreign Direct Investment to Developing Countries: is Africa Differ-
ent? World Development, 30(1), 107-119.
Blomstrom, M., & Kokko, A. (1996). Multinational corporations and spillovers. Journal of Economic Surveys,
12(2), 1-31.
Borensztein, E., De Gregorio, J., & Lee, J. (1998). How does foreign direct investment affect economic growth?
Journal of International Economics, 45 (1), 115-135.
Bwalya , S. M. (2006). Foreign direct investment and technology spillovers: Evidence from panel data analysis of
manufacturing firms in Zambia. Journal of Development Economics, 81, 514–526.
Carkovic, M., & Levine, R. (2005). Does foreign direct investment accelerate economic growth?. Does foreign
direct investment promote development, 195.
Chakraborty, C., & Nunnenkamp, P. (2008). Economic reforms, FDI and economic growth in India: a sector
level analysis. World Development, 36(7), 1192-1212.
Dunning, J. (1992). Multinational Enterprises and Global Economy.
Dutse, A. Y. (2012). Technological Capabilities and FDI-related Spillover: Evidence from Manufacturing Industries
in Nigeria. American International Journal of Contemporary Research, 2(8), 201-211.
Fourie, E. (2012). A future for the theory of multiple modernities: Insights from the new modernization theory.
Social Science Information, 51(1), 52-69.
Ghali, s., & Rezgui, S. (2008). FDI contribution to Technical Efficiency in the Tunisian Manufacturing Sector.
Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1260755 on 3/5/2011
IMF. (1991). A study of the Soviet economy. World Bank, OECD and EBRD. Paris: OECD.
Jordaan, J. (2008). Intra & inter industry externalities from foreign direct investment in the Mexican manufactur-
ing sector: new evidence from Mexican regions. World Development, 36(12), 2838-2854.
Kombui, D. N. & Kotey, R. A. (2019). Foreign Direct Investment in an Emerging Economy: Exploring the De-
terminants and Causal Linkages. Academic Journal of Economic Studies, 5(1), 51-62.
Kotey, R. (2019). Foreign Direct Investment and Spillover Effects in Africa: An Empirical Review. International
Journal of Science and Management Studies (IJSMS).
Larrain, J. (2013). Theories of development: Capitalism, colonialism and dependency. John Wiley & Sons.
Lehmann, D. (2010). Development theory: four critical studies. Routledge.
72
EJAE 2019  16 (2)  59-78
APPROACH
Li, X., & Liu, X. (2005). Foreign direct investment and economic growth: an increasingly endogenous relation-
ship. World Development, 33(3), 393-407.
Liu, Z. (2008). Foreign direct investment and technology spillovers: Theory and evidence. Journal of Development
Economics 85 (2008), 176–193.
Marin, A., & Bell, M. (2006). Technology Spillovers from Foreign Direct Investment (FDI): an Exploration of the Active
Role of MNC Subsidiaries in the Case of Argentina in the 1990s. Journal of Development Studies, 42(4), 678-697.
Matunhu, J. (2011). A critique of modernization and dependency theories in Africa: Critical assessment.
Preston, P. (2012). Theories of development. Routledge.
Ramirez, M. (2006). Is foreign direct investment beneficial for Mexico? An empirical analysis, 1960-2001. World
Development, 34(5), 802-817.
Rehman, N. U. (2016). FDI and economic growth: empirical evidence from Pakistan. Journal of Economic and
Administrative Sciences, 32(1), 63-76. DOI:10.1108/JEAS-12-2014-0035
Scott, C. V. (1995). Gender and development: rethinking modernization and dependency theory. Lynne Rienner
Publishers Inc.
Sinani, E., & Meyer, K. E. (2004). Spillovers of technology transfer from FDI: the case of Estonia. Journal of
Comparative Economics(32), 445-466.
So, A. Y. (1990). Social change and development: Modernization, dependency and world-system theories (No.
178). Sage.
Sylwester, K. (2005). Foreign direct investment, growth and income inequality in less developed countries. Inter-
national Review of Applied Economics, 19(3), 289-300. DOI:10.1080/02692170500119748
Twarowska, K., & Kakol, M. (2013). International business strategy- reasons and forms of expansion into foreign
markets. Active citizenship by knowledge management and innovation. Zader Croatia.
UNCTAD. (1992). World Investment Report. United Nations, New York.
UNCTAD. (2005). Economic development in Africa; Rethinking the Role of Foreign Direct Investment (Vol.
Sales No. E.05.II.D.12 I). New York and Geneva: United Nations Publications. DOI:UNCTAD//GDS/
AFRICA/2005/1
UNCTAD. (2010). Foreign direct investment, the transfer and diffusion of technology, and sustainable develop-
ment. United Nations Conference on Trade and Development, (p. TD/B/C.II/EM.2/2). Geneva.
73
EJAE 2019  16 (2)  59-78
APPROACH
APPENDIX
Appendix 1
The table below shows a list of variables used to measure technology in literature;
Frequently Used Measures for Technology found in Literature

PROXY STUDY MERIT DEMERIT
Elenkov & Manev (2009),
New products/product in- Measures actual imple-
Sasidharan, (2006), Hale Not all products succeed
novations mentation
and Long (2006)
In some situations, Patents
Measures technological
Patents/patent applica- Makri & Scandura (2010), are not useful. patenting
progress and the impor-
tions Jung et al. (2008), is not harmonized across
tance of patents
countries
Measures technological
Ideas might not become
Invention disclosures Axtell et al. (2000) progress and the number
products
of ideas generated
Marin and Bell (2006; Difficult to measure, the
Measures improvements
Innovations in processes 2008), West et al. (2003), challenge of innovators
in processes and methods
Kinoshita (2000) dilemma.
Bwalya (2006), Czarnitzki
Ratio of new product sales Measures customers re- Sales output is affected by
& Kraft (2004), Javorcik &
to total sales sponse to innovation many variables
Spatareanu (2008)
Ratio of new product sales Gumusluoglu & Ilsev Show how R&D had im- Sales output is affected by
to cost of R&D (2009), UNCTAD (2010) pacted sales (efficiency) many variables
UNCTAD (2010), García-
Data very available (Most
Morales et al. (2008), Ne- Does not measure the ef-
Cost of R&D used measure of technol-
ven & Siotis (1996), Lan- ficiency
ogy)
cheros (2016)
Data very available (Most
Number of employees into UNCTAD (2010), García- Does not measure the ef-
used measure of technol-
R&D Morales et al. (2008) ficiency
ogy)
Measures innovation and
Entry into New markets Elenkov & Manev (2009) Some acquisitions fail
efficiency
Liu (2008), Dutse (2012), Easy to measure/ Quan- Productivity is the result
Productivity and efficiency
Chuang and Lin (1999), tifiable of many variables
UNCTAD (1999), Gorg
May be affected by more
Knowledge creation and and Greenaway (2001), Relatively closer measure
than one variable and hard
skills (labour) Pradhan (2006), Seghir, of technology
to disentangle.
(2012), Bwalya (2006)
Payments of Royalties and Quantifiable and easy to May be a weak form of
UNCTAD (2010)
License fees compute measure in some cases.
74
EJAE 2019  16 (2)  59-78
APPROACH
Appendix 2
The table below presents a view of SSA countries and their annual publications.
Appendix 3 Pre-Estimation Tests
Breusch-Pagan / Cook-Weisberg test for heteroskedasticity
The table below presents our test results after we perform the Breusch-Pagan / Cook-Weisberg test
for heteroskedasticity.
Test for Heteroscedasticity

Breusch-Pagan / Cook-Weisberg test for heteroskedasticity
Ho: Constant variance
Variables: fitted values of GDP
chi2(1) = 0.08
Prob> chi2 = 0.7743
Our P-value (of 0.7743), from the test is insignificant, indicating that the variation from the regres-
sion line is constant among our data points and therefore, the normal standard errors are not biased.
Also, we observe from our scatter plot that there is no heteroscedasticity; data that exhibits heterosce-
dasticity are usually cone-shaped.
75
EJAE 2019  16 (2)  59-78
APPROACH
HAUSMAN TEST
The table below presents the Hausman test results;
Hausman test results
--- Coefficients ----

(b) (B) (b-B) sqrt(diag(V_b-V_B))
FE RE Difference S.E.
FDI -0.0841837 -0.0754175 -0.0087662 0.0047944
Tech 0.0000305 0.0000275 2.97E-06 3.17E-06
lninteraction 0.1090934 0.1070176 0.0020758 0.004901
lnExpcu 0.5883393 0.6519714 -0.0636321 0.0115638
Open -0.0972443 -0.1144304 0.0171861 0.007321
Labper -1.683713 -1.251441 -0.4322716 0.0618394
chi2(6) = (b-B)'[(V_b-V_B)^(-1)](b-B)
37.66
Prob>chi2
The chi-square for the test is 37.66 and the P-value is 0.000 which is significant so we reject the null
hypothesis and conclude FE model is appropriate for the regression. Thus, there is some correlation
between the independent variable and error term hence random effects estimation is not appropriate
for the model.
76
EJAE 2019  16 (2)  59-78
APPROACH
Test for endogeneity
To test for possible endogeneity, simultaneity and reverse causality issues, we run an IV (instrumental
variable) regression. We employed a Two-Stage Least Squares. The table below is the regression output.
Table 9 Test for Endogeneity using 2stage least squares

(1) (2)
Variables lnGDP lnGDP
lnFDI -0.136 -0.104

(0.0975) (0.0790)
lninteraction 0.176*** 0.193***
(0.0614) (0.0707)
lnExpcu 0.659*** 0.598***
(0.0378) (0.0687)
Open -0.000104 -0.000279
(0.000423) (0.000336)
Labper -0.750*** -0.626***
(0.190) (0.167)
Pstab -0.0953*** -0.0922***
(0.0147) (0.0135)
Constant 3.326*** 3.459***
(0.460) (0.531)
Observations 372 372

R-squared 0.917 0.918
Standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
From the above table, we observe a very high R-squared (of about 92%) for both models. When the
coefficients for the interaction term and FDI are observed, we see the results are similar to the main
FE regression model the study employed. Therefore, when we control of endogeneity, the regression
results do not change that much. That means endogeneity may not be a problem in the main regression.
77
EJAE 2019  16 (2)  59-78
APPROACH
ULOGA TEHNOLOGIJE KAO APSORPCIONOG KAPACITETA U

PRIVREDNOM NAPRETKU ZEMALJA U RAZVOJU: NOVI PRISTUP
Rezime:
Istraživanja su pokazala da efekti direktnih stranih investicija na eko-
nomski rast nisu uvek bili neposredni, posebno ne u oblastima koje se
razvijaju; potrebno je da u samoj privredi već postoje određene oso-
benosti, a kako bi se navedeni efekti pravilno absorbovali. Otuda, ovo
istraživanje je imalo za cilj da proceni ekonomski uticaj direktnih stranih
investicija na ekonomski rast zemalja Supsaharske Afrike, uzimajući
u obzir tehnologiju kao apsorpcioni kapacitet. Zbog nedostatka poda-
taka o održivom merilu, a kada je u pitanju tehnologija iz perspektive
Afrike, promeravamo tehnologiju uz upotrebu inovativnog pristupa,
koristeći godišnji broj objavljenih radova u vezi sa inovacijama, kao
merilo tehnološkog prisustva. Analizirani su podaci iz četrdeset tri
supsaharske zemlje, koji ilustruju period od 19 godina (od 1990. do
2008). Upotrebom regresionog modela fiksnih efekata (Fixed Effects),
istraživanje je pokazalo da direktne strane investicije imaju negativan i
značajan uticaj na bruto domaći proizvod (BDP), što je naš parametar
za ekonomski razvoj. Ipak, kada se investicije udruže sa tehnologijom,
ovaj odnos postaje značajan i pozitivan. Navedeno pokazuje da zemlje u Ključne reči:
kojima je tehnologija razvijenija lakše prihvataju i koriste strane direktne direktne strane investicije,
investicije, u odnosu na zemlje u kojima je tehnologija slabije razvijena. tehnologija, inovacije,
apsorpcioni kapacitet
78
OWNERSHIP CONCENTRATION AND FIRM PERFORMANCE:

AN EMPIRICAL ANALYSIS IN OMAN
Mawih Kareem Al Ani*, Asma Mohammed Al Kathiri
Assistant Dean, Associate Professor of Accounting

College of Commerce and Business Administration
Dhofar University, Oman

This study investigates the effect of ownership concentrations on firm
performance. A sample of 115 Omani companies in three sectors (i.e.
Correction: February 19, 2019
financial, industrial and service) was selected for the study. The sample
Accepted: June 08, 2019
companies selected were listed in the Muscat Securities Market for a
period of five years (2011–2015). Four types of ownership were ana-
lysed, namely, the ownership of Omani investors, the Gulf Cooperation
Council (GCC) country investors, Arab non-GCC investors, and foreign
investors. Firm performance was measured by return on assets (ROA),
return on equity (ROE), and market fair value (MFV) of share. Panel
regression was used to study the effect of ownership concentrations
on firm performance. Results reveal a positive and significant effect
of GCC countries and foreign investors on ROA only in the industrial
sector. Moreover, a positive and significant effect of Omani and GCC
countries investors was found on MFV in the service sector. Finally,
results found no effect of ownership concentrations on ROE in all sectors.
Keywords:
ownership structure, return on
assets, return on equity, market fair
value, Oman
79
*E-mail: mawih@du.edu.om
EJAE 2019  16 (2)  79-94
M. K. AL ANI, A. M. AL KATHIRI  OWNERSHIP CONCENTRATION AND FIRM PERFORMANCE: AN EMPIRICAL ANALYSIS IN OMAN
INTRODUCTION
Every capital market has its approach to building ownership of its listed companies. The vision of the
capital market controls the type of ownership structure that a company decides to adopt. “The ownership
structure is defined by the distribution of equity with regard to votes and capital, as well as the identity of
the equity owners” (Owiredu, Oppong & Churchull, 2014). Ownership concentration refers to the percent-
age of shares held by an owner relative to the total shareholding of the firm. It is contrary to ownership
identity, which refers to the actual names of major shareholders (Owiredu et al., 2014).
Ownership concentrations are one of the frequently debated issues in the published studies. The effect
of different types of shareholders on firm performance was also studied. Moreover, many types of owner-
ship concentrations underwent individual and collective research. Ownership concentrations, such as a
large shareholder, and foreign, institutional, family, block-holder, managerial, employee, government,
retail, and domestic ownerships were studied in the earlier literature. Previous studies have examined the
effect of ownership concentration on firm performance. However, conclusive evidence to demonstrate
the effect of ownership concentration on firm performance remains lacking (Rajput & Bharti, 2015).
In Oman, a different structure of the ownership of companies exists as listed in the Muscat Securi-
ties Market (MSM). This structure is defined on the basis of one general criterion, namely, ownership
identity of the investor, regardless of nationality (i.e. Omani or non-Omani). The term “Omani investor”
refers to all Omani individuals or institutions that can buy or sell shares in MSM. However, non-Omani
investors are grouped into three clusters, namely: 1) investors from the Gulf Cooperation Council (GCC)
countries, which have nearly similar characteristics to Omani investors; 2) Arab investors who are ex-
cluded from GCC countries, and 3) foreign investors. According to the foreign capital investment law
in Oman No. 102, 1994 (and its amendments), a foreign investor is defined as a legal or natural person
who owns a percentage of shares of companies inside Oman. GCC investors are individual investors or
institutions that operate in other GCC countries outside Oman, whereas Arab non-GCC investors are
individual investors or institutions that operate in Arab non-GCC countries. In summary, four types or
identities of shareholders in Omani companies are listed in MSM, namely, Omani, GCC, Arab non-GCC
and foreign investors.
As observed, the ownership or shareholding structure of Omani-listed companies in MSM is non-
uniform. In several companies, the Omani investor has a high percentage of shares that can reach up to
99%. In other companies, it is the foreign investor who has a high percentage of shares. Moreover, a dif-
ferent ownership or shareholding structure exists across the three sectors in MSM. Thus, predicting the
style of this structure is extremely difficult. For example, in the industrial sector, GCC investors have a
high level of ownership in certain companies, but the opposite is true in the service sector. Therefore, the
following question is raised: Will the differences in ownership percentage of each shareholder possibly
affect the financial and market performance of the listed companies?
This study aims to investigate the effect of ownership percentage of each shareholder’s type or identity
on the firm performance of Omani companies listed in the MSM for a period of five years (2011–2015). The
ownership percentage of each concentration is the independent variable, which has four sub-independent
variables, namely, ownership percentage of Omani investors, ownership percentage of GCC investors,
ownership percentage of Arab non-GCC investors, and ownership percentage of foreign investors. Con-
versely, out of the three dependent variables, two are proxies of financial performance, namely, return on
asset (ROA) and return on equity (ROE). The third variable is market performance, which is measured
by the market fair value (MFV) of the share at the closing price.
80
EJAE 2019  16 (2)  79-94
The current area of research remains scarce, as only a few researchers have focused on this problem.
In 2016, Al-Matari and Al-Arussi studied a sample of 81 companies from only the industrial and service
sectors for a period of three years (2012–2014). They focused only on financial performance in terms of
profitability. A research gap has arisen owing to the lack of empirical study concerning this problem.
The current study aims to fill this research gap, and be among the pioneer studies that investigate the
effect of ownership concentration on firm performance in the Omani context. Moreover, the present
study covers all companies listed in three sectors, namely, industry, services and finance, in MSM.
The remainder of the paper is structured as follows: Section Two provides a literature review, Section
Three discusses the research method and models, Section Four presents the findings for each model,
while Section Five concludes.
LITERATURE REVIEW
A significant amount of empirical evidence exists on the impact and association of ownership
concentration and firm performance. One of the most important findings from the previous literature
is that ownership structure has had important implications for corporate governance, protection of
minority shareholders’ interest, and other variables (Kuznetsov, Kapelyushnikov & Dyomina, 2010).
Empirical evidence from prior studies was derived from different types of ownership concentration
and various measures of firm performance. Earlier researchers studied one or more ownership con-
centrations, and the majority used ROE, ROA, and Tobin’s Q as dependent variables. In other words,
they investigated the effect of ownership concentration on financial performance and firm value. Con-
versely, a few studies looked into the effects of ownership concentration on stock market performance.
However, the results are mixed. Most studies concluded with a positive effect, whereas the opposite is
true for other studies. Several researchers obtained mixed or no results. Therefore, the past literature
can be classified into four clusters.
Literature Review—Positive Effect
In this cluster, previous studies observed a positive effect of ownership concentrations on firm
performance. (e.g. Srithanpong, 2012; Alimehmeti & Paletta, 2012; Isik & Soykan, 2013; Fauzi &
Locke, 2012; Rajput & Bharti, 2015; Amran & Ahmed, 2013; Yasser & Al Mamun, 2017). These stud-
ies examined the effect of many types of ownership concentrations on financial performance, firm
value, and market-based performance. For example, Srithanpong (2012) studied the effect of foreign
ownership on performance. Rajput and Bharti (2015) investigated four types of ownership concentra-
tion models, namely, foreign institution, family, government, and retail ownerships. Alimehmeti and
Paletta (2012) and Isik and Soykan (2013) researched the impact of only one model, namely, large
shareholders, whereas Amran and Ahmed (2013) evaluated the impact of two models, namely, mana-
gerial and family ownerships. Fauzi and Locke (2012) assessed the effect of ownership structure on
firm performance in 79 listed companies in New Zealand. Results of these studies showed a significant
positive impact of all types of ownership concentrations on financial performance in terms of ROA
and ROE, and firm value in terms of Tobin’s Q and market-based performance. The positive results
confirm that the ownership concentrations being analysed in these studies might be less dispersed and
more concentrated, which consequently increase financial performance, firm value, and market-based
81
EJAE 2019  16 (2)  79-94
performance. For example, Alimehmeti and Paletta (2012: 45) showed a positive association between
ownership concentration and firm value. This result is ‘confirming the agency perspective that higher
concentration increases shareholder power and control aligning managers, shareholders’ interests, and
consequently increasing firm value.’
Literature Review—Negative Effect
Another cluster of research provides evidence of the negative effect of ownership concentrations
on firm performance. Kuznetsov et al. (2010), Al-Saidi and Al-Shammari (2014), Wang and Shailer
(2015) and Shahab-u-Din and Javid (2011) are examples of such studies.
The negative impact has many explanations. One is that concentrated ownership might largely
influence firm performance. For example, Al-Saidi and Al-Shammari (2014) pointed out that large
shareholders significantly influence managers, who act only in the interest of large shareholders. In
other words, extra attention is given to these shareholders, whereas less attention is given to other
ownership types. Din and Javid (2011) provided another explanation for this negative relation. Their
research was related to the control of the concentrated ownership of the company in which the said
ownership will make decisions that serve their interests but influence other ownership concentrations.
Literature Review—Mixed Results
Many empirical studies concluded, with mixed results the association between ownership concentra-
tions and firm performance. Results showed positive, negative, or no effects. Srivastava (2011), Chen
(2012), Khan and Nouman (2017), Ahmed and Abdel Hadi (2017), Ongore (2011), Pathirawasam and
Wickremasinghe (2012), Vintilă, Gherghina, and Nedelescu (2014) Khamis, Hamdan, and Elali (2015)
and Zakaria, Purhanudin, and Palanimally (2014) are examples of these types of evidence.
Results of the previous literature are dependent on the power held by the type of ownership con-
centration. For example, Ahmed and Abdel Hadi (2017) found that the government ownership con-
centration positively affects performance, because the government is fully authorised to enact laws and
regulations. Furthermore, it significantly supports the business environment. Moreover, a negative
association is observed between ROE and insider ownership, because this class of ownership concen-
tration can damage the profitability of the firm instead of improving it. Khan and Nouman (2017)
found a negative relationship between managerial ownership and performance, because managers
favour family relations over performance, which is damaging for the organisation. In the same study,
the relationship between block-holder ownership concentration and performance was significantly
positive because this class of ownership positively reduces agency problems.
82
EJAE 2019  16 (2)  79-94
Literature Review—No Association
The final group of evidence pertains to empirical studies that found no relationships or impacts of
ownership concentrations on firm performance. Few studies, such as Qin, Mishra and Smyth (2012) and
Abdul Rahman and Md Reja (2015), concluded this result for certain reasons. For example, Abdul Rahman
and Md Reja (2015) concluded that the model used in several studies does not explain the changes in firm
performance owing to other factors, such as research sample, variables, period of study, or statistical data.
Accordingly, ownership concentration is a multidimensional concept that can be measured using
various measures, such as managerial, foreign, family, and large ownerships. In Oman, ownership
concentration exerts a relatively substantial role. Furthermore, knowledge of the leaders in the listed
companies and how they influence firm performance is very important. Most of these measures are
covered by the abovementioned review of the existing literature. However, the ownership concentrations
(Omani and non-Omani investors) employed by MSM in Oman have not undergone optimal analy-
ses. This observation creates a gap requiring analysis. Moreover, no consensus exists on the empirical
evidence of ownership concentration and firm performance. The available results are mixed, which
require further research of the concept. The present study intends to fill this research gap.
In alignment with most of the literature reviews, we present the hypotheses of the study.
H1: Ownership concentration of Omani listed firms has a positive effect on ROA.
H2: Ownership concentration of Omani listed firms has a positive effect on ROE.
H3: Ownership concentration of Omani listed firms has a positive effect on MFV.
METHODOLOGY
Population, Sample and Data
The population for this study consists of 118 companies listed in the industrial, financial, and service
sectors of the MSM during 2011–2015. This population is divided into three sectors, namely, financial
(35), industrial (45), and services (38) sectors. The sample of the study covered (115) companies with
a total of 575 final observations. The study excluded three listed companies from the service sector,
because of unavailable data for all variables within the study period. The firm-level panel data for the
study were primarily obtained from the MSM database for 2011–2015. All data are available in the link
(Ani, Mawih Kareem, 2019; ‘ownership concentration in Oman’, Mendeley Data, https://data.mendeley.
com/datasets/6gmd83wznm/3 . The following table shows the size of the population and the sample:
Table 1 Population and Sample

Sector Service sector Industrial sector Financial sector Total
Size of population 38 45 35 118
Sample size 35 45 35 115
Sample/population
92.1% 100% 100% 97.45%
ratio
Companies excluded 3 0 0 2.55%
83
EJAE 2019  16 (2)  79-94
Variables
This study aims to empirically investigate the effect of ownership concentrations on firm performance
of Omani companies listed in MSM during 2011–2015. The independent variables are the percentage
of ownership concentrations:
-Percentage of Omani Ownership Concentration (OOC)
-Percentage of Arab Gulf Ownership Concentration (GCCOC)
-Percentage of Arab non-GCC Ownership Concentration (ANGCCOC)
-Percentage of Foreign Ownership Concentration (FOC)
The dependent variables are as follows: financial performance, which is measured by ROA and
ROE; and market-based performance, which is measured by MFV of share on the closing date at the
end of the year.
The study employed three general measures of dependent variables to avoid any problems that be-
long to the firm or sector. In addition, the study used three general measure variables, which have also
been utilised in most studies. A commonly used measure of firm performance is ROA, which provides
a picture of the effectivity of firm management in terms of generating profit using available assets. An-
other good measure of firm performance is ROE, which is a measure of the effectivity of shareholders’
funds being used by the management of the firm. ROA and ROE have been employed as measures for
financial performance in several studies (Unsal, Ugurlu & Sakinc, 2009; Srivastava, 2011; Rajput and
Bharti, 2015). Apart from ROA and ROE, researchers have also used another measure, namely, MFV,
for market-based performance (e.g. Kumar, 2004; Srivastava, 2011). This measure reflects the movement
of share price, which can be easily understood by investors. Table 2 provides the definition of variables.
Table 2 Definition of Variables

Variable Definition
MFV Closing price at the end of the year
ROA Net income after tax÷ total assets
ROE Net income after tax ÷ total equity
Percentage of ownership Number of shares owned by the total shares of the company
The study used ROA to measure the firm’s ability to generate profit from the total assets and ROE
to reflect the returns of shareholder equity. ROA and ROE were calculated in accordance with the
abovementioned formula in Table 2. The means of ROA, ROE, and MFV were calculated for each
firm, ownership concentration, sector and year. Lastly, data distributions within each sector and year
were assessed.
84
EJAE 2019  16 (2)  79-94
Model Specification
In this study, panel regression was used to recognise the impact of independent and dependent
variables. Afterwards, an equation was formulated to evaluate the influence of independent variables
on dependent variables. Data were organised in the form of a balanced panel. The panel data models
(multiple regression analysis) were used for data analysis using IBM Statistical Package for the Social
Sciences (SPSS 22). The general form of the models used was
where
FP = firm performance
Α = constant term
β = parameters are coefficients for estimation
e = error
i = firm
t = time.
Three univariate regression models were drawn from the cited general form to study the impact of
ownership concentrations on firm performance as follows:
The three univariate regression models were applied in the three sectors in MSM, namely, the fi-
nance, industrial, and service sectors.
85
EJAE 2019  16 (2)  79-94
RESULTS AND DISCUSSION
Descriptive Statistics
Table 3 shows the descriptive statistics for all variables.
Table 3 Descriptive Statistics

N Minimum Maximum Mean Std. Deviation
Variables
Statistic Statistic Statistic Statistic Statistic
OOC 115 0.01 1.00 0.7782 0.24751
GCCOC 115 0.00 0.99 0.1421 0.21830
ANGCCOC 115 0.00 0.65 0.0125 0.14573
FOC 115 0.00 0.17 0.070 0.01927
ROE 115 −1.39 1.74 0.1021 0.51014
ROA 115 −0.02 1.52 0.1357 0.21231
MFV 115 0.00 4.74 0.8512 1.08973
Table 3 shows that Omani investors have the majority of ownership in the sample at approximately
78%, followed by Arab investors from GCC countries at 14%, and foreign investors at approximately
7%. Arab investors from non-GCC countries own the least ownership at approximately 1%. The mean
of ROE is 10%. However, this finding does not imply that the listed firms can increase the wealth of
investors because the minimum of ROE is negative. In other words, several companies are unable to
generate profit for their shareholders. The mean of ROA is 13.5%, which does not imply that the listed
firms can generate profit from their assets. The minimum of ROA is negative, which indicates that certain
companies are unable to generate profit from their assets. Finally, the mean of MFV is positive, which
denotes that investors have a positive perspective about the listed firm in terms of market performance.
Correlation matrix and Multicollinearity
The study has four independent variables of ownership concentration, namely, OOC, GCCOC,
ANGCCOC, and FOC. The three dependent variables are ROA, ROE and MFV. Table 4 summarises
the results of correlation and multicollinearity:
Table 4 Correlation Matrix and Multicollinearity

Sector Variables OOC GCCOC ANGCCOC FOC ROA ROE MFV
OOC 1
GCCOC 0.203 1
ANGCCOC −0.112 0.356 1
Finance
FOC 0.040 0.124 0.237 1

ROA −0.214 0.427* 0.018 −0.251 1
ROE −0.028 0.059 0.077 −0.166 0.378 1
MFV −0.246 0.232 −0.056 −0.076 0.414* 0.147 1
86
EJAE 2019  16 (2)  79-94
OOC 1
GCCOC −0.044 1
ANGCCOC 0.159 0.441* 1
Finance
FOC 0.222 0.212 −0.246 1

ROA −0.201 −0.025 −0.021 0.411** 1
ROE −0.082 0.132 0.052 −0.080 0.215 1
MFV −0.186 0.148 −0.013 0.106 0.358* 0.421* 1
OOC 1
GCCOC −0.036 1
ANGCCOC 0.115 −0.156 1
Finance
FOC 0.351* 0.241 0.103 1

ROA −0.082 0.110 −0.028 −0.039 1
ROE −0.100 0.193 0.229 −0.143 .256 1
MFV 0.365* 0.422* −0.367 0.340 0.125 0.016 1
*Correlation is significant at the 0.05 level (2-tailed).
**Correlation is significant at the 0.01 level (2-tailed).
Table 4 indicates that multicollinearity is not a problem as the correlations within all independent
and dependent variables are relatively low, whereas the majority of the variables are non-significant.
The remainder is significant, but less than 0.80. Therefore, we infer that all models are dependable.
Table 4 shows a number of significant associations among dependent (ROA, ROE and MFV) and
independent (OOC, GCCOC, ANGCCOC and FOC) variables. For example, ROA has a significant
positive association with GCCOC (0.447) at 0.05 in the finance sector, whereas ROE and MFV have
no relationship with all ownership concentration indicators (independent variables). In the industrial
sector, ROA is also positively correlated with FOC (0.411) at 0.01, whereas ROE and MFV have no
relationships with all independent variables. In the service sector, MFV has a positive relationship with
OOC and GCCOC at 0.05. However, a weak and statistically non-significant correlation is observed
among ROA, ROE and MFV and all independent variables.
87
EJAE 2019  16 (2)  79-94
Findings of Model 1 (ROA)
Table 5 shows the regression results of model 1. The table provides a snapshot of the regression
results on an aggregate basis. In this model, the dependent variable is ROA.
Table 5 Regression Results of Model 1 (ROA)

Finance Sector Industrial Sector Service Sector
I.V(s)* Coeffi- Coeffi- Coeffi-
Model T-Value Sig T-Value Sig. T-Value Sig
cients cients cients
Constant 2.568 4.688 0.000 1.154 2.354 .023 0.093 6.064 0.000
OOC −0.112 −0.306 0.762 −0.031 −0.486 0.629 −0.133 −1.249 0.222
GCCOC 0.486 2.056 0.049 0.086 0.195 0.847 0.102 0.551 0.586
ROA
ANGCCOC −0.109 −0.371 0.713 0.103 0.142 0.887 −0.107 −0.151 0.881
FOC −0.148 −1.290 0.207 1.414 2.995 0.004 0.061 −0.092 0.928
R-Squared 0.103 0.170 0.013
F-Value 2.184 3.008 0.125
Sig. 0.012 0.040 0.945
* I.V(s): Independent variables
Table 5 shows that GCCOC has a significant positive impact on ROA in the finance sector at 0.05.
OCC, ANGCCOC, and FOC have a non-significant relationship with ROA. In the industrial sector,
FOC has a significant positive impact on ROA at 0.05. OCC, GCCOC, and ANGCCOC have a non-
significant relationship with ROA. In the service sector, no significant impact is observed among all
types of ownership concentrations and ROA.
Findings of Model 2 (ROE)
Table 6 shows the regression results of model 2. In this model, the dependent variable is ROE.
Table 6 Regression Results of Model 2 (ROE)

I.V(s) Coeffi- Coeffi- Coeffi-
Constant 1.153 2.564 0.016 0.085 0.884 0.381 0.093 4.039 0.000
OOC 0.012 0.694 0.493 −0.001 −0.642 0.524 0.101 0.743 0.464
GCCOC 0.051 0.537 0.595 0.091 0.794 0.431 0.121 0.961 0.345
ROE
ANGCCOC −0.017 −0.020 0.984 0.001 0.209 0.836 0.153 1.242 0.224
FOC −0.017 −0.020 0.984 0.001 0.209 0.836 0.153 1.242 0.224
R-Square 0.096 0.044 0.001
F-value 0.099 0.334 1.011
Sig. 0.960 0.801 0.403
88
EJAE 2019  16 (2)  79-94
Table 6 reveals no significant impact on all types of ownership concentrations and ROE because
the Sig. value of all variables is non-significant at 0.05 and 0.01.
Findings of Model 3 (MFV)
Table 7 shows the regression results of model 3 with MFV as the dependent variable.
Table 7 Regression Results of Model 3 (MFV)

I.V(s) Coeffi- Coeffi- Coeffi-
Constant 1.552 2.083 0.047 1.635 3.436 0.001 1.489 1.899 0.068
OOC −0.106 −0.807 0.427 −0.256 −1.282 0.207 0.486 0.627 0.038
GCCOC 0.112 0.823 0.417 0.211 1.116 0.271 1.210 2.461 0.020
MFV ANGC-
−0.085 −0.541 0.593 −0.003 −0.128 0.899 −0.115 −0.988 0.332
COC
FOC −0.006 −0.456 0.652 0.114 0.841 0.405 0.017 0.134 0.895
R-Square 0.059 0.027 0.122
F-value 0.425 0.586 2.436
Sig. 0.736 0.627 0.036
In the service sector, OOC and GCCOC have a significant positive impact on MFV at 0.05. FOC
and ANGCCOC have a non-significant relationship with MFV. In the financial and industrial sectors,
all types of ownership concentrations have no impact on MFV.
Result Discussion
Model 1 (Dependent Variable—ROA)
In the finance sector, results of this model imply that a high concentration of ownership for only
GCC investors increases financial performance in terms of ROA, whereas that of other investors has
no influence. R2 is only 10.3%, which implies that independent variables included in the regression
equation explain only 10.3% of changes in ROA, which is extremely low. The underlying reason for this
result is because only one independent variable (GCCOC) is considered in the regression equation of
this model. In the finance sector, most of the shareholders are from GCC countries, and their experience
is reflected in achieving the profitability in this sector. This result is consistent with those of previous
studies (e.g. Ahmed & Abdel Hadi, 2017; Isik & Soykan, 2013; Fauzi & Locke, 2012; Rajput & Bharti,
2015). These studies suggested a positive impact of ownership concentration on ROA.
In the industrial sector, FOC has a significant positive impact on ROA at 0.05. OCC, GCCOC and
ANGCCOC have a non-significant relationship with ROA. These results imply that high concentra-
tions of ownership for foreign investors increase financial performance in terms of ROA, whereas other
concentrations of ownership have no influence on ROA. R2 is only 17.0%, which implies that inde-
89
EJAE 2019  16 (2)  79-94
pendent variables included in the regression equation explain only 17.0% of changes in ROA, which is
extremely low. The possible explanation is that three of the independent variables are non-significant
in this model and sector. According to these results, H1 is supported in the two sectors that confirm
the positive impact of ownership concentration on ROA. In the industrial sector, foreign investors are
the main investors because they bring technology, experience, skills and prestige together with capital
to countries in which they invest. In addition, foreign investors plan to increase profit and avoid losses
in this type of industry. This finding is consistent with that of Srithanpong (2014). That is, foreign
ownership can improve firm performance. This result contradicts that of Khan and Nouman (2017),
who suggested that foreign investors did not enhance firm performance.
No significant impact is found among all the types of ownership concentrations and ROA in the
service sector,. This result indicates that an increase or decrease in ownership concentrations has no
effect on ROA. Thus, H1 is not supported in this sector. This result is consistent with that of Khamis et
al. (2015), who suggested that many ownership concentrations had no impact on firm performance,
owing to the power of the investor and its role in the market.
Model 2 (Dependent Variable—ROE)
This result indicates that an increase or decrease in ownership concentrations for all sectors has
no effect on ROE. In this model, H2 is not supported. Suggestively, ownership concentrations have
no impact on ROE. Two potential reasons underlie this result. Firstly, investors in MSM are unable
to monitor managers, and prevent the tendencies to accumulate wealth for their benefit. Secondly,
despite the importance of ROE in assessing the performance of a company, investors in MSM seem
to use their experience or other measures instead of ROE to distinguish between companies that are
profit creators and profit burners.
The result of this model is inconsistent with those of previous studies, such as Srivastava (2011), Abbas,
Naqvi and Mirza (2013) and Rajput and Bharti (2015). This inconsistency in results might be partially because
an individual investor in Oman has less ability to maximise ROE compared with ROA. Institutional investors
are prevalent in Oman and concerned with ROA as it appears in the MSM report (MSM companies Guide,
2017). However, this notion is consistent with that of Abdul Rahman and Md Reja (2015), who assumed
that if investors have insufficient ownership percentage then it will not affect performance.
Model 3 (Dependent Variable—MFV)
The results of this model imply that a high concentration of ownership for Omani and GCC inves-
tors increases market performance in terms of MFV, whereas other concentrations of ownership have
no influence on MFV. The value of R2 is 0.122, which indicates that a 12.2% change in the independ-
ent variable is due to the dependent variable. This variation is not much, but is significant. The reason
behind this result is that the three variables within the model did not yield any significant results., An
increase or decrease in ownership concentrations has no effect on MFV in the finance and industrial
sectors. In other words, no significant impact of any of the types of ownership concentrations and MFV
was observed. Based on the cited results, H1 is supported in the service sector.
90
EJAE 2019  16 (2)  79-94
However, we cannot confirm the same result in the two other sectors. Moreover, this result is consist-
ent with that of Kumar (2004) and Srivastava (2011), who assumed that the capital market in most of
emerging countries is small, and not active globally.
The present study has several limitations. Firstly, the study used the ownership concentration clas-
sified by MSM only. Future studies may consider other classifications aside from MSM. Secondly, the
current study focused on financial and market performance of firms only. However, non-financial
goals can be of equal importance for managers and investors. Therefore, future studies should consider
financial and non-financial goals, and assess them in firms with different ownership concentrations.
Different criteria for firm value, such as economic value added and Tobin’s Q, can be used in the
analysis in future studies, and the results can be subjected to further discussion. Thirdly, the period
of study is limited to five years (2011–2015). This time series may be unstable because of its short
duration. Future studies may require longer and different time series. Fourthly, the present study was
conducted in the MSM, which is considered a small sample in an emerging market. Further studies
may be conducted on entire GCC markets, which have several similarities in terms of laws, regulations,
and the nature of economies.
CONCLUSION
The present study investigated the impact of ownership concentration on the performance of all
firms listed in three sectors at MSM from 2011 to 2015. The findings are in line with the second stream
of literature review, which reveals mixed results. Results show that GCCOC and FOC in the finance
and industrial sectors, respectively, have significant and positive impacts on firm performance in terms
of ROA. Moreover, a significant and positive impact of OOC and GCCOC on firm performance in
terms of MFV is found only in the service sector. This finding indicates that MFV and ROA increase
in the case of these ownership structures in a firm because these investors control the unfavourable
activities of the top management and make decisions that favour other minor shareholders. The study
has a unique finding: none of the types of ownership concentrations have had any impact on ROE in
all three of the sectors. The reason behind this result is that investors can maximise their ROA, but not
ROE. ROA in Oman is used extensively to evaluate the profitability of companies, because institutional
investors prefer its use.
This study has certain implications. Firstly, decision-makers in MSM should encourage companies
to disclose further information about other ownership structures, such as family and managerial con-
centrations. Secondly, the positive and significant relationship between foreign ownership and ROA
in the industrial sector appears to have gained universal acceptance. In this respect, MSM has formed
a mechanism to monitor this scenario. Thirdly, the results will contribute to the improvement of MSM
by improving the corporate governance model in this market.
91
EJAE 2019  16 (2)  79-94
REFERENCES
Abbas, A. H., Naqvi, A. & Mirza, H. H. (2013). Impact of Large Ownership on Firm Performance: A Case of non-
Financial Listed Companies of Pakistan. World Applied Sciences Journal, 21 (8), 1141-1152, DOI: 10.5829/
idosi.wasj.2013.21.8.1916.
Abdul Rahman, A. N. A., & Md Reja, B. A. F. (2015). Ownership Structure and Bank Performance. Journal of Eco-
nomics, Business and Management. 3(5), 483-488.
Ahmad, H. & Javid, A. (2010). The Ownership Structure and Dividend Payout Policy in Pakistan (Evidence from
Karachi stock Exchange 100 Index). International Journal of Business Management and Economic Research,
1(1), 58-69.
Ahmed, N., & Hadi, O. A. (2017). Impact of Ownership Structure on Firm Performance in the MENA Region: An
Empirical Study. Accounting and Finance Research, 6 (3), 105-115.
Alimehmeti, G., & Paletta, A. (2012). Ownership Concentration and Effects over Firm Performance: Evidences from
Italy. European Scientific Journal, 8(22), 39-49.
Al-Matari, E. M., & Al arussi, A. S. (2016). The Effect of the Ownership Structure Characteristics on Firm Perfor-
mance in Oman: Empirical Study. Corporate Ownership and Control Journal, 13(2), 93-100.
Al-Saidi, M., & Al-Shammari, B. (2014). The Relationship between a Firm’s Value and Ownership Structure in
Kuwait: Simultaneous Analyses Approach. International Business Research, 7(5), 32-48.
Amran, N. A., & Ahmed, A. C. (2013). Effects of Ownership Structure on Malaysian Companies Performance. Asian
Journal of Accounting and Governance, 4, 51–60.
Chen, L. (2012). The Effect of Ownership Structure on Firm Performance Evidence from Non-financial Listed Com-
panies in Scandinavia. Unpublished master thesis, Aarhus School of Business. Aarhus University. Denmark.
Din, S.D., & Javid, A. Y. (2011). Impact of Family Ownership concentration on the Firm's Performance: Evidence
from Pakistani Capital Market. Online at https://mpra.ub.uni-muenchen.de/37566/ MPRA Paper No. 37566,
posted 29. 10:28 UTC.
Fauzi F. and Locke, S. (2012). Board Structure, Ownership Structure and Firm Performance: A case of New Zealand
Listed-Firms. Asian Academy of Management Journal of Accounting and Finance, 8 (2), 43–67.
Isik, O., & Soykan, M. E. (2013). Large Shareholders and Firm Performance: Evidence from Turkey. European
Scientific Journal, 9(25), 23-37.
Khamis, R., Hamdan, A. M. & Elali, W. (2015). The Relationship between Ownership Structure Dimensions and
Corporate Performance: Evidence from Bahrain. Australasian Accounting, Business and Finance Journal,
9(4), 38-56.
Khan, F. U., & Nouman, M. (2017). Does Ownership Structure Affect Firms Performance? Empirical Evidence from
Pakistan. Pakistani Business Review, 19 (1),1-23.
Kumar, J. (2004). Does Ownership Structure Influence Firm Value? Evidence from India. The Journal of Entrepre-
neurial Finance and Business Ventures, 9(2),61 –93.
Kuznetsov, A., Kapelyushnikov, R. & Dyomina, N. (2010). The Impact of Concentrated Ownership on Firm Perfor-
mance in an Emerging Markets: Evidence from Russia. https://www.researchgate.net/publication/229051266.
Ministry of Commerce and Industry. (1994). The Foreign Capital Investment. Oman Government Publishing Services.
Ongore, V.O. (2011). The relationship between ownership structure and firm performance: An empirical analysis
of listed companies in Kenya. African Journal of Business Management, 5 (6), 2120-2128, DOI: 10.5897/
AJBM10.074.
Owiredu, A., Oppong, M. & Churchull. R.Q. (2014). Effects of Ownership Structure on the Performance of Listed
Companies on the Ghana Stock Exchange. Archives of Business Research, 2(4), 70-86, DOI: 10.14738/
abr.24.346.
92
EJAE 2019  16 (2)  79-94
Pathirawasam, C., & Wickremasinghe, G. (2012). Ownership Concentration and Financial Performance: The
Case of Srilankan Listed Companies. Corporate Ownership and Control, 9(4), 170-177.
Qin, Z., Mishra, V. & Smyth, R. (2012). An Empirical Examination of Endogenous Ownership in Chinese Private
Enterprises. Working paper 38-12, Monash University.
Rajput, N., & Bharti, M. (2015). Shareholder Types, Corporate Governance and Firm Performance: An Anecdote
from Indian Corporate Sector. Asian Journal of Finance and Accounting, 7(1), 45-63.
Srithanpong, T. (2012). Foreign Ownership and Firm Performance in the Thai Construction Industry. Interna-
tional Proceedings of Economics Development and Research, 55 (14), 72-76. DOI: 10.7763/IPEDR.
Srivastava, A. (2011). Ownership Structure and Corporate Performance: Evidence from India. International
Journal of Humanities and Social Science, 1(1), 23-29.
Unsal, A., Ugurlu, E. & Sakinc, I. (2009). Ownership Identity and Firm Performance in Manufacturing Compa-
nies in Turkey: A Multinational Logit Model Approach. International Journal of Economics and Finance,
1(2), 23-34.
Vintilă, G., C. Gherghina, Ş. & Nedelescu, M. (2014). The Effects of Ownership Concentration and Origin on
Listed Firms value: Empirical Evidence from Romania. Romanian Journal of Economic Forecasting, XVII
(3), 51-71.
Wang, K., & Shailer, Gr. (2015). Ownership Concentration AND Firm Performance Emerging Markets: A Meta
–Analysis. Journal of Economic Surveys, 29 (2), 199-229.
Yasser, Q. R., & Al Mamun. A. (2017). The Impact of Ownership Concentration on Firm Performance: Evidence from
an Emerging Market. Emerging Economy Studies, 3 (1), 34-53, https://doi.org/10.1177/2394901517696647.
Zakaria, Z., Purhanudin, N. & Palanimally, Y. R. (2014). Ownership Structure and Firm Performance: Evidence
from Malaysian Trading and Services Sector. European Journal of Business and Social Sciences, 3 (2), 32-43.
93
EJAE 2019  16 (2)  79-94
KONCENTRACIJA VLASNIŠTVA I UČINAK KOMPANIJE: EMPIRIJSKA

ANALIZA U OMANU
Rezime:
Ovaj radi se bavi uticajem koncentracije vlasništva na učinak firme.
Uzorak od 115 kompanija iz Omana, koje su deo tri sektora (odnosno,
finansijskog, privrednog i sektora usluga) odabran je za potrebe ovog
istraživanja. Navedene kompanije bile su deo tržišta hartija od vrednosti
Muskata (Oman) u periodu od pet godina (2011–2015). Analizirane su
četiri vrste vlasništva, naime, vlasništvo omanskih investitora, investitora
iz zemlje – pripadnica Saveta za zalivsku saradnju, arapski investitori,
koji ne pripadaju navedenom Savetu, kao i strani investitori. Učinak
firme meren je prinosom na poslovnu imovinu (ROA), prinosom na
kapital (ROE), kao i fer tržišnom vrednošću akcija. Panel regresija
je upotrebljena kako bi se utvrdili efekti koncentracije vlasništva na
učinak firme. Rezultati otkrivaju pozitivan i značajan uticaj, a kada su
u pitanju investitori iz zemalja pripadnica Saveta za zalivsku saradnju,
odnosno, strani investitori, na prinos na poslovnu imovinu, I to samo u
sektoru privrede. Štaviše, pozitivan i značaja uticaj investitora iz Omana
i zemalja članica Saveta za zalivsku saradnju na fer tržišnu vrednost, Ključne reči:
primećen je u sektoru usluga. Naposletku, rezultati nisu ukazali na vlasnička struktura, prinos na
uticaj koncentracije vlasništva na prinos na kapital – bez obzira na to poslovnu imovinu, prinos na
o kojem je sektoru reč. kapital, fer tržišna vrednost, Oman
94
THE RISING GOVERNMENT EXPENDITURE IN NIGERIA: ANY INFLUENCE

ON GROWTH?
Segun Subair Awode
Nigerian Institute of Social and Economic Research (NISER)

Nigeria

This study attempts to examine whether government expenditure in
Nigeria has had any influence on growth in the economy. The study
Correction: February 08, 2019
focuses primarily on capital and recurrent types of government expendi-
ture, and these were regressed against the real gross domestic product.
Secondary time series data ranging from 1981 to 2016 obtained from
the CBN Statistical Bulletin were used. Having established that the
series were co-integrated in the long run through the Cointegration
technique of Johansen, the study then used the error correction and
Granger causality techniques to achieve its objectives. Results indicated
that recurrent expenditure exerts a significant positive influence on real
GDP, while the influence of capital expenditure on real GDP turned
out negative. The Granger causality test revealed that both capital and
recurrent expenditures Granger cause real GDP. The study, therefore,
advocates for a strong monitoring and evaluation system of the way in
which government funds, especially those intended for capital projects,
are being used, so as to bring about a meaningful influece on the economy.
Keywords:
capital expenditure, economic
growth, error correction mecha-
nism, Granger causality, gross
domestic product
95
*E-mail: awodesegun@gmail.com
EJAE 2019  16 (2)  95-108
AWODE S. S.  THE RISING GOVERNMENT EXPENDITURE IN NIGERIA: ANY OUTCOME ON GROWTH?
INTRODUCTION
There is no denying the fact that governments in every nation, developed or developing, are charged
with carrying out some statutory roles in the country, some of which include (but not limited to) pro-
tecting the citizenry against domestic and external attacks, catering to the welfare needs, and providing
social services for its people. To say that carrying out these (and other equally important) roles requires
a huge amount of spending by the government is no exaggeration.
Government expenditure in Nigeria, as noted by Iheanacho (2016), has been rising rapidly, with
such recurring increase evidenced in almost all the sectors of the economy. A cursory breakdown of
the 2019 appropriation bill of Nigeria shows that the Federal Government seeks to spend USD 24.41
billion, which is about 2.5% higher than the 2018 estimates, whose proposed size of USD 23.8 billion
was 16% higher than the 2017 estimates. A huge chunk of this (2019 appropriation bill) was proposed
to be spent on recurrent expenditure, which stood at USD 11.18 billion; capital expenditure at USD 5.6
billion; and debt servicing at USD 5.92 billion, on its own representing about a quarter of the total esti-
mate. The capital expenditures alone stand at 23% of the total budget, while its Siamese twin (recurrent
expenditures) stands at 45.75%.
Given that the government’s major concern, as noted earlier, is primarily the welfare and living
conditions of the masses, which statutorily determines the government’s spending pattern. A colossal
amount of concern keeps lingering over the skyrocketing and superfluous government spending in
Nigeria, yet the masses keep languishing in abject poverty (Iheanacho, 2016; Oyinlola & Akinnibosun,
2013). It becomes even more worrisome to discover that many past administrations have spent so
much on capital and recurrent expenditures, yet the results remain infrastructural gaps and impover-
ishments (Oteng-Abayie, 2011; Adekunle, 2007). Bitter enough, recurrent expenditures now outstrip
capital expenditures with almost double its size (Njoku et al. 2014). One big question that comes to
mind is “where exactly is the money going if not to the betterment of people’s lives and engineering
growth of the economy?”
Empirical findings of the effects of federal government expenditures on growth in the Nigerian
context have been mixed and inconsistent. In relation to the issues already raised, extant studies, such
as Njoku et al. (2014), Udoffia and Godson (2016), Akonji et al. (2013) and many others reported
that recurrent expenditure impacts growth positively in Nigeria, while the findings of studies, such as
Aigheyisi (2013) and Ayinde et al. (2015) suggested otherwise. In fact, by confirming the inconsisten-
cies among extant studies, Ayinde et al. (2015) suggested a further re-evaluation and reassessment of
the direction of causal impact between recurrent expenditure and economic growth.
Further important empirically misunderstood evidence about the growing volume of government spend-
ing and economic growth in Nigeria is the argument on Wagner’s Law, which argued that long-run tenden-
cies exist for public expenditure to grow relatively to the growth of the economy, and that, as the economy
develops over time, the activities and functions of the government would increase. Many extant studies,
such as Akonji et al. (2013) supported this while some others like Awode and Akpa (2018) and Olayungbo
and Olayemi (2018) refute the claim, but by considering the poverty level of the people, such increases in
government spending, which are potentially informed by the catastrophic occurrences mentioned in the
background, may not suggest the true growth in economy. Therefore, to provide empirical evidence for
solving these observed research problems, the need to embark on this current study remains pertinent. It
is against these backdrops that the study draws its motivation to investigate empirically how the Nigerian
economy (in terms of real GDP) has fared following several periods of huge budgetary allocations.
96
EJAE 2019  16 (2)  95-108
Having introduced the focus of the paper in this section, the literature was extensively reviewed in
the second section; methodological issues were dealt with in section three, section four presents the
results and their discussions, while section five concludes the paper.
LITERATURE REVIEW
Many studies have been carried out in the past, using several approaches and methodologies, in
the bid to address empirically how public spending impacts the growth of the Nigerian economy. One
such study is Aigheyisi (2013) whose work was conducted to investigate the impact of government
expenditure on Nigeria’s economy from 1980 to 2011. The study used a multiple regression analysis
and employed a disaggregated data on capital and recurrent expenditures. The study’s findings indi-
cate that the impacts of the variables on GDP were statistically insignificant. However, they became
significant following a one-period lag, with the impact of recurrent expenditure negative, while that of
capital expenditure was positive. Similar to this, Akonji et al. (2013) also did a study to investigate the
linkages between the different components of government expenditure and real gross domestic product
for Nigeria. The techniques employed both the Granger causality and Error Correction techniques. The
result showed that the link between capital expenditure and real GDP is in consonance with Wagner’s
law, while that of total recurrent expenditure and real GDP proved bi-causal. However, the causation
from recurrent expenditure to real GDP is stronger.
Iheanacho (2016) also carried out a study to empirically determine the quantum of contributory
impact government expenditure has on growth in Nigeria using a disaggregated data approach for the
period between 1986 and 2014. The study made a distinct contribution by controlling for influence
revenue from non-oil sources. Findings revealed negative and positive relationship between recurrent
expenditure and economic growth in the short and long run, respectively, in Nigeria. However, the
results indicated a long-run negative relationship between capital expenditure and growth in Nigeria.
Similarly, Ayinde et al., (2015) did a study to unravel the relationship among public expenditure, rev-
enue, and economic growth in Nigeria from 1981 to 2011 using the Co-integration, Error Correction
Mechanism and Combined Estimators Analysis Approach. Findings showed evidence of a long run
relationship among the variables, while it was further revealed that the effects of capital expenditure,
oil revenue, federation account, and federal retained revenue on growth were positive in Nigeria.
Njoku et al. (2014) also did a study to determine the effect of public expenditures on economic
growth in Nigeria from 1961 to 2013 by adopting a quantitative research methodology. Findings
showed that capital expenditure on administration, recurrent expenditure on social and community
services, as well as recurrent expenditure on economic services are growth-enhancing; while economic
expenditure, capital transfers, recurrent expenditure on administration, and recurrent expenditure on
transfers retract the growth of the economy in Nigeria.
Oyinlola and Akinnibosun (2013) also contributed to the empirics on the public expenditure-growth
nexus in Nigeria from 1970 to 2009 by using the Gregory-Hansen structural break cointegration to
analyze a disaggregated public expenditure, and the result affirms the existence of Wagner’s law, while
also revealing a structural break in 1993, accounting for the political upheaval of the annulled general
election in the country then.
97
EJAE 2019  16 (2)  95-108
Udoffia and Godson (2016) did a study to determine the impact of government expenditure on Eco-
nomic Growth in Nigeria from 1981 to 2014. Using time series data sourced from the CBN Statistical
Bulletin, findings showed that both capital and recurrent expenditures of the federal government have a
positive effect on economic growth in Nigeria. Still in the effort to address how government spending af-
fected economic growth in Nigeria, Chude and Chude (2013) also did a study to investigate the effects of
public expenditure on economic growth in Nigeria over a period stretching from 1977 through 2012, using
disaggregated data and employing the Error Correction Model (ECM) as the technique of analysis. Find-
ings from the study indicate that, in the long, expenditure on education influences economic growth reacts
positively to education expenditure in Nigeria.
Slightly different from the foregoing nature of studies, Ayuba (2013) concentrated on the social expend-
iture-growth nexus in Nigeria between 1990 and 2009. The study applies VECM-Based Causality and the
findings suggested that economic growth causes changes in health expenditure, and that there is a unidirec-
tional causality running from economic growth to health expenditure, thereby supporting Wagner’s Law.
The results further revealed that the economic growth Granger causes both education and aggregate social
expenditures in Nigeria.
Kairo et al. (2017) examined the impact of government spending on human capital development us-
ing secondary set of data from 1990 to 2014 and employing the Autoregressive Distributed Lag (ARDL)
Model Approach as the technique of analysis. Findings showed that a long run relationship exists between
expenditure and human development index. The results further demonstrated that government expenditure
has remained positive, but to a very large extent insignificant to human capital development in Nigeria.
Elsewhere, Adil, Ganaie and Kamaiah (2017) carried out an empirical investigation into Wagners’s
hypothesis in India from 1970 to 2013 employing the Autoregressive Distributed Lag (ARDL) Model. The
study showed a long-run cointegration existed between economic growth and government expenditure, but
evidence for Wagner’s law was not present.
Chow, Cotsomitis and Kwan (2002) conducted a study in the UK to show that relying on a bivariate
Wagner model (economic growth regressed on government expenditure) was not adequate to understanding
the long-run relationship between the variables using time series data from 1948 to 1997. The study intro-
duced a third variable- money supply-, and showed that a long-run equilibrium existed between govern-
ment spending and economic growth. The conducted Granger causality test indicated a one-way direction
of causal effect from both income and money supply to government expenditure.
Chang, Liu and Caudill (2004) re-examined the existence Wagner’s law using its five different versions
across three emerging countries of Asia and seven other industrialized economies. The study employed the
Johansen cointegration test and error correction mechanism (ECM), and results showed a one-way causation
that runs from economic growth to government expenditure in South Korea, Taiwan, Japan, the United King-
dom, and the United States, thus validating Wagner’s law in these countries. The other countries- Australia,
Canada, New Zealand, South Africa, and Thailand posted no evidence of causation effect from economic
growth to public spending. The study also confirmed a long-run cointegration existing between the variables.
Abbasov and Aliyev (2018) tested Wagner’s and Keynes’ law in nine former Soviet Union countries
from the first quarter of 2000 to the third quarter of 2017. The study adopted the Autoregressive Distributed
Lag (ARDL) Model of estimation. Findings from the study validated Wagner’s law for Latvia, Lithuania,
Uzbekistan, Georgia, Kyrgyzstan, and Ukraine, while the Keynesian hypothesis was validated for Estonia,
Uzbekistan, Azerbaijan, Kyrgyzstan, and Moldova in the long run. Unlike Chow, Cotsomitis and Kwan
(2002) and Chang, Liu and Caudill (2004), where a unidirectional relationship was established from income
98
EJAE 2019  16 (2)  95-108
to public expenditure, this study found a short-run bi-causal relationship in all of the countries studied
except Lithuania and Kyrgyzstan.
Antonis, Constantinos and Persefoni (2013) compared Wagner’s and Keynesian law in pre-World
War II Greece with data from 1833 to 1938. In addition to the empirical analysis using the Autore-
gressive Distributed Lag (ARDL) estimation method, the study also conducted a test to determine the
presence or otherwise of structural breaks in the series. Findings from the study showed that economic
growth reacts positively to public spending, thus confirming Wagnerćs law in Greece. On the other
hand, the Keynesian hypothesis was valid for the entire sample period (1833-1938), but not so for the
sub-sample period of 1881-1938.
Olayungbo and Olayemi (2018) conducted a study whose major focus was to unravel the dynamic
relationship existing among non-oil revenue, government spending, and economic growth in Nigeria using
secondary data ranging from 1981 to 2015. The study employed the error correction, impulse response
function as well as granger causality techniques to achieve its stated objective. Results indicate that gov-
ernment spending impacts the economy negatively while the effect of non-oil revenue on the economy
was positive both in short and long run. However, the granger causality test revealed a unidirectional
causality running from government spending to both non-oil revenue and economic growth in Nigeria.
Awode and Akpa (2018) conducted a study to test the existence or otherwise of Wagner’s law in
Nigeria in the short and long run. The study applied the autoregressive distributed lag technique and
controlled for structural breaks within the period 1981-2016. The study noted a negative and insignificant
effect relationship between government expenditure and economic growth, thus refuting the claim of
Wagner’s law in Nigeria. The effect of oil export earnings was controlled for, and it was found to influence
government spending positively both in the short run and long run. The study, therefore, noted the need
for the economy to be more diversified into more labour-intensive sectors to increase output per worker,
and to ensure that government expenditure is based more on tax earnings than on oil export earnings.
Babatunde (2018) carried out a study to determine how the government’s infrastructural spending
relates to the growth of the Nigerian economy using data series from both primary and secondary sources
from 1980 to 2016. The primary data for the study was obtained using statistical random sampling
for the sample selection, and analyzed using descriptive statistics. The secondary data were analyzed
by employing vector error correction mechanism. Results of the study indicate that public spending
on the transport, communication, education, and health sectors impacts economic growth directly in
Nigeria. However, the government’s agricultural and natural resources spending indirectly impact the
economy. An element of fiscal illusion was further revealed in the sense that agriculture and natural
resources sector in Nigeria receive more funding from the private sector than from the government.
The present study, therefore, attempts to both examine the growth effect of rising government ex-
penditure in Nigeria by disaggregating government expenditure broadly into its capital and recurrent
categories as well as establish whether any causal relationship exists between government expenditure
and economic growth in Nigeria.
99
EJAE 2019  16 (2)  95-108
METHODOLOGY
Model Specification
The model used in this study was adapted from Akonji et al., (2013) and Udoffia and Godson’s
(2016) studies in an attempt to replicate their efforts to test Wagner’s Law and test the true empirical
state of the Keynesian theory on connectivity between the variables employed in the study.
To begin with, the model first considers the simple purport of endogenous growth model, which
specifies that, for a nation to achieve a growth in its state of economy, it must invest in human capital
relative to its stock of labour. By this notion, we have:
RGDP = f ( AK a Lb ) (1)
Where
RGDP represents the Real Gross Domestic Product;
A represents the total factor productivity, which incorporates the export trade (ET);
K is the human capital; and
L is labour.
By relating with Wagner’s law, which specifies that an economic growth can be achieved through
government spending, now, given that;
Total Government Spending = f (Government Expenditure) (2)
In equation (2), the components of government spending will be limited to capital and recurrent
expenditures only.
By combining equations (1) and (2), we have,
RGDP= f(capital expenditue, recurrent expenditure) (3)
In mathematical terms, equation (3) above can be linearized as thus;
βu + β1CE + β 2 RE
RGDP = (4)
Where
βu represents the intercept (constant term),
β1 and β2 represent the parameters of the model, and
CE represents Capital Expenditures (in billions),
RE represents total Recurrent Expenditure (in billions)
In log and econometrics forms, equation (4) becomes;
(5)
βu + β1CE + β 2in RE + e
inRGDP =
Where
In represents the natural logarithm properties,
e represents the Residual or the Error Term
100
EJAE 2019  16 (2)  95-108
Based on economic theory – specifically Wagner’s Law, a direct positive relationship is expected
between government expenditures (capital and recurrent) and real GDP. It is assumed that more gov-
ernment spending will bring about an increase in real GDP.
Time series data from 1981 to 2016 were used for this study. Specifically, the data used was on:
capital and recurrent components of federal government expenditures, and the data on real gross
domestic product (RGDP), which proxies economic growth. These datasets were obtained from the
CBN Statistical Bulletin, and analyzed using the statistical package E-views 9.
Results and Discussion
Table 1: Descriptive Statistics

LOG(RGDP) LOG(CE) LOG(RE)
Mean 24.96960 4.789211 5.372348
Median 24.53515 5.542390 5.645415
Maximum 27.06539 7.049946 8.320768
Minimum 23.48328 1.411011 1.558313
Std. Dev. 1.140242 1.963357 2.320157
Skewness 0.673019 -0.519424 -0.303626
Kurtosis 2.014776 1.702326 1.694984
Jarque-Bera 4.173729 4.144743 3.107734

Probability 0.124076 0.125887 0.211429
Sum 898.9055 172.4116 193.4045

Sum Sq. Dev. 45.50532 134.9169 188.4095
Observations 36 36 36
Source: Author, 2019
From Table 1, which shows the result of the descriptive statistical properties of the variables employed
in the study, it was revealed that all the variables were leptokurtic in nature, since their Kurtosis values
were greater than 3. However, while real GDP skewed positively, implying that real GDP exhibited
more of increasing values over the period under review, both capital and recurrent expenditures showed
negative skewness, meaning that both capital and recurrent expenditures exhibited more of decreasing
values over the period under review. Furthermore, the Jarque-Bera statistic, which is a formal test of
normality, reveals that all the variables were normally distributed, since the p-values associated with
their Jarquw-Bera statistic were not statistically significant at 5% level.
101
EJAE 2019  16 (2)  95-108
Table 2: Unit Root Test Result

Augmented Dickey Fuller (ADF) Test
Variables Level 1st Difference Status
LOG(RGDP) 1.4213 -5.2116 I(1)
LOG(CE) 1.6216 -5.0620 I(1)
LOG(RE) 3.2341 -2.2171 I(1)
Critical Values
1% -2.6326 -2.6347
5% -1.9507 -1.9510
10% -1.6111 -1.6109
The Augmented Dickey Fuller (ADF) test was employed to assess the unit root properties of the vari-
ables and the emanated results, as shown in Table 2, and indicated that all the variables, which were not
stationary at levels, became stationary after first differencing which means that the order of integration
of all the variables is order one. This implies that an element of possible long-run relationship exists
among the variables. This, therefore, requires further investigation of long-run co-movement among
the variables. In carrying this out, the Johansen co-integration technique comes in handy.
Table 3: Johansen Co-integration Result

Hypothesized Trace 0.05 Max-Eigen 0.05
No. of CE(s) Statistic Critical Value Statistic Critical Value
r≤0 40.71352* 29.79707 24.08978* 21.13162

r≤1 16.62375* 15.49471 12.40533 14.26460
r≤2 4.218411* 3.841466 4.218411* 3.841466
* denotes rejection of the hypothesis at the 0.05 level

**MacKinnon-Haug-Michelis (1999) p-values
The result of the Johansen co-integration has been presented in Table 3 and this result revealed
that a long-run co-movement exists among the variables. This is due to the fact that the Trace statistic
revealed three cointegrating equations, while the Max-Eigen statistic showed two, indicating that the
variables employed in this study are cointegrated in the long run. By putting this into better perspec-
tive, the result provided evidence of a convergence relationship among the variables in the long run.
102
EJAE 2019  16 (2)  95-108
Table 4: Error Correction Mechanism (ECM) Results

Dependent Variable: LOG(RGDP)
Method: Least Squares
Variable Coefficient Std. Error t-Statistic Prob.
LOG(CE) -0.353330 0.112256 -3.147534 0.0036
LOG(RE) 0.721135 0.095457 7.554587 0.0000
C 22.75126 0.145057 156.8433 0.0000
ECM(-1) -0.811908 0.083859 -9.681768 0.0000
R-squared 0.934265 F-statistic 146.8636

Adjusted R-squared 0.927904 Prob(F-statistic) 0.000000
Durbin-Watson stat 1.837085
The results presented in Table 4 refer to the estimates of the impact of federal government expenditures
on economic growth in Nigeria over the 1981-2016 period using the Error Correction Mechanism. The
results indicated that capital expenditure of the government contributes negatively to the growth of the
economy. Specifically, a 1% increase in capital expenditure led to 35% reductions in the growth of the
economy. The implication of this is that, despite the huge yearly budgetary allocations for capital expen-
ditures, the statutory role of the government in ensuring a growing economy has not been achieved. This
may be due to either ineffective usage and mismanagement of such budgeted funds, or partial or total
embezzlement and looting of the budgeted funds by government officials, or both. Moreover, by the nature
of capital expenditures, it may take several years for them to start to yield positive impacts on the economy.
However, the result showed that recurrent expenditure wields a positive and significant impact on the
growth of the economy. A 1% increase in recurrent expenditure increases the growth of the economy by
approximately 72%. The implication of this is that recurrent expenditure is growth-enhancing in Nigeria.
The results also indicate that the error correction coefficient is negative and significant, which im-
plies that a long-run co-movement exists among the variables. Specifically, a short-run shock to the
system has the potential of returning the series to about 81% of its equilibrium level in the preceding
year. This is a very high speed of adjustment!
The model has a good fit, as about 93% of variations in real GDP are explained by both capital and
recurrent expenditures. This is buttressed by the F-statistic, which is statistically significant at 1% level.
Finally, the model does not suffer a serial correlation problem.
Table 5: Granger Causality Result

Null Hypothesis F-Statistic Prob. Granger Causality
LOG(CE) does not Granger Cause LOG(RGDP) 11.6447 0.0018 Unidirectional causality
LOG(RGDP) does not Granger Cause LOG(CE) 2.15887 0.1515 CE → RGDP
LOG(RE) does not Granger Cause LOG(RGDP) 14.1150 0.0007

Unidirectional causality
LOG(RGDP) does not Granger Cause LOG(RE) 1.16712 0.2881
RE → RGDP
LOG(RE) does not Granger Cause LOG(CE) 0.73768 0.3968

No Causality
LOG(CE) does not Granger Cause LOG(RE) 2.77104 0.1057
103
EJAE 2019  16 (2)  95-108
The Granger Causality technique was employed to determine the direction of causation (if any)
existing between government expenditure and economic growth in Nigeria, the result of which has been
presented in Table 5. The results reveal a unidirectional causal link running from both capital and recurrent
expenditures to real GDP, implying that both capital and recurrent expenditures Granger cause economic
growth in Nigeria, thereby lending empirical support to the Keynesian theory. This is in line with the
findings of Olayungbo and Olayemi (2018), but in stark contrast with that of Awode and Akpa (2018).
CONCLUSION
Empirical findings from this study revealed that government expenditure has the potential to bring
about growth in the economy. This is evidenced in the Granger causality test result, that showed that
a unidirectional causality running from both capital and recurrent expenditures to real GDP exists in
Nigeria. This means that real GDP can better be predicted on account of capital expenditure, recurrent
expenditure, and real GDP than by using the history of real GDP alone. Findings further revealed that,
while recurrent expenditure influences growth positively in Nigeria, capital expenditure, on the other
hand, contributes negatively to the growth of the Nigerian economy.
The reason for the negative impact of capital expenditure on economic growth cannot be far-fetched.
It should be noted that, by the very nature of capital expenditures, it may take up to several years before
they result in positive outcomes. Nevertheless, the result perfectly explains the corrupt nature of the
country whereby the capital expenditure has turned into an avenue for mismanagement, embezzlement
and fungibility of government funds.
In lieu of the foregoing, this study therefore, strongly recommends the need on the part of the
government to ensure that funds intended for capital projects are being used for their real course and
not just on some white elephant projects that create avenues for mismanagement of funds. This can
be achieved by putting an effective monitoring and evaluation (M&E) system in place in the way and
manner in which government funds are being used. This will serve as a way of not only bringing sanity
into the business of governance, but also a way of ensuring prudence in public sector spending so that
rising government expenditures can have a positive outcome on the general well-being and by exten-
sion, propel growth in the economy.
104
EJAE 2019  16 (2)  95-108
REFERENCES
Abbasov, J. A. & Aliyev, K. (2018). Testing Wagner’s Law and Keeynesian Hypothesis in Selected Post-Soviet Countries,
Acta Universitatis Agricuturae et Silviculturae Mendelianae Brunensis, 66(5), pp. 1227-1237
Adekunle, Z. C. (2007). Unraveling the Mystery between Public Expenditure and Growth:
Empirical Evidence from Nigeria. International Journal of Economics, 4(1): 21-31.
Adil, M. H., Ganaie, A. A. & Kamaiah, B. (2017). Wagner’s Hypothesis: An Empirical Verification, IIM Kozhikode
Society & Management Review, 6(1), pp. 1–12
Aigheyisi, O. S. (2013). The Relative Impacts of Federal Capital and Recurrent Expenditures on Nigeria ’ s Economy
(1980-2011). American Journal of Economics, 3(5), 210–221.
Akonji, D. R., Olateju, A. O. & Wakili, A. M. (2013). Nexus between Public Expenditure and Economic Growth
by testing Wagner’s law Time Series: Evidence from Nigeria. International Journal of Development and
Sustainability, 2(4), 2383–2395
Antonis, A., Constantinos, K & Persefoni, T. (2013). Wagner’s Law versus Keynesian Hypothesis: Evidence from
pre-WWII Greece, Panoeconomicus, 4, pp. 457-472
Awode, S. S. & Akpa, E. O. (2018). Testing Wagner’s Law in Nigeria in the Short and Long Run. Acta Univer-
sitatis Danubius. Oeconomica, 14(7)
Ayinde, K., Kuranga, J. & Adewale, F. L. (2015). Modeling Nigerian Government Expenditure, Revenue and
Economic Growth: Co-integration, Error Correction Mechanism and Combined Estimators Analysis
Approach. Asian Economic and Fianancial Review, 5(6), 858–867.
Babatunde, S. A. (2018). Government Spending on Infrastructure and Economic Growth in Nigeria. Economic
Research-Ekonomska Istrazivanja, 31(1), 997-1014.
Chang, T., Liu, W & Caudill, S. B. (2004). A Re-examination of Wagner’s Law for Ten Countries bBased on Cointegra-
tion and Error-correction Modelling Techniques, Applied Financial Economics, 14 (8), pp. 577-589
Chow, Y., Cotsomitis, J. A. & Kwan, A. C. C. (2002). Multivariate Cointegration and Causality Tests of Wagner
Hypothesis: Evidence form the UK, Applied Economics, 34 (13), pp. 1671-1677
Chude, N. P. & Chude, D. I. (2013). Impact of Government Expenditure on Economic Growth in Nigeria. Inter-
national Journal of Business and Management Review, 1(4), 64–71.
Iheanacho, E. (2016). The Contribution of Government Expenditure on Economic Growth of Nigeria Disaggre-
gated Approach. International Journal of Economics & Management Sciences, 5(5), 1–8.
Kairo, C. I., Mang, N. J., Okeke, A. & Aondo, D. C. (2017). Government Expenditure and Human Capital De-
velopment in Nigeria: An Auto-Regressive Distributed Lagged Model Approach (ARDL). International
Journal of Advanced Studies in Economics and Public Sector Management, 5(1), 143–158.
Njoku, C. O., Ugwu, K. E. & Chigbu, E. E. (2014). Government Public Expenditures: Effect on Economic Growth
(The Case of Nigeria, 1961-2013). International Journal of Research in Management, Science & Technol-
ogy, 2(1), 16–29.
Olayungbo, D. O. & Olayemi, O. F. (2018). Dynamic Relationship among Non-oil Revenue, Government Spend-
ing and Economic Growth in an Oil Producing Country: Evidence from Nigeria. Future Business Journal
4, 246-260.
Oteng-Abayie, E. F. (2011). Government Expenditure and Economic Growth in Five ECOWAS Countries: A
Panel Econometric Estimation, Journal of Economic Theory, 2(3): 11-14
Oyinlola, M. A. & Akinnibosun, O. (2013). Public Expenditure and Economic Growth Nexus: Further Evidence
from Nigeria. Journal of Economics and International Finance, 5(4), 146–154.
Sala-I-Martin, X. and Bairo, R. S. (2012). Public Finance in Model in Economic Growth. The Review of Economic
Studies, http://www.jseir,org. 59(4): 645 – 661.
105
EJAE 2019  16 (2)  95-108
Udoffia, D. T. & Godson, J. R. (2016). The Impact of Federal Government Expenditure on Economic Growth in
Nigeria (1981-2014). Greener Journal of Social Sciences, 6(4), 092–105
Wagner, A. (1958). The Nature of Fiscal Policy. In R. A. Musgrave, & A. T. Peacock, Classics in the Theory of
Public Finance (pp. 1-8). London: Macmillan.
106
EJAE 2019  16 (2)  95-108
Year RGDP CE (N’Billion) RE (N’Billion)

1981 6.11E+10 6.57 4.85
1982 5.14E+10 6.42 5.51
1983 3.55E+10 4.89 4.75
1984 2.85E+10 4.10 5.83
1985 2.89E+10 5.46 7.58
1986 2.07E+10 8.53 7.70
1987 2.41E+10 6.37 15.65
1988 2.33E+10 8.34 19.41
1989 2.42E+10 15.03 25.99
1990 3.08E+10 24.05 36.22
1991 2.74E+10 28.34 38.24
1992 2.93E+10 39.76 53.03
1993 1.58E+10 54.50 136.73
1994 1.81E+10 70.92 89.97
1995 2.85E+10 121.14 127.63
1996 3.50E+10 212.93 124.49
1997 3.58E+10 269.65 158.56
1998 3.20E+10 309.02 178.10
1999 3.59E+10 498.03 449.66
2000 4.64E+10 239.45 461.60
2001 4.41E+10 438.70 579.30
2002 5.91E+10 321.38 696.80
2003 6.77E+10 241.69 984.30
2004 8.78E+10 351.25 1,110.64
2005 1.12E+11 519.47 1,321.23
2006 1.45E+11 552.39 1,390.10
2007 1.66E+11 759.28 1,589.27
2008 2.08E+11 960.89 2,117.36
2009 1.69+E11 1,152.80 2,127.97
2010 3.69E+11 883.87 3,109.44
2011 4.12E+11 918.55 3,314.51
2012 4.61E+11 874.70 3,325.16
2013 5.15E+11 1,108.39 3,214.95
2014 5.68E+11 783.12 3,426.94
2015 4.81E+11 818.35 3,831.98
2016 4.87E+11 613.25 4,108.31
Source: CBN Statistical Bulletin
107
EJAE 2019  16 (2)  95-108
UVEĆANA POTROŠNJA VLADE U NIGERIJI: POSTOJI LI POVEZANOST SA

NAPRETKOM?
Rezime:
Ovaj rad nastoji da utvrdi da li su i kakav uticaj troškovi vlade u Ni-
geriji imali na rast privrede u toj zemlji, primarno se usmeravajući na
kapitalne i stalne trškove vlade, koji su uticali na stvarni bruto domaći
proizvod (BDP). Sekundarni podaci u vidu vremenskih serija – iz peri-
oda 1981 – 2016, preuzeti iz Statističkog biltena CBS (Centralnog biroa
za statistiku), upotrebljeni su za ove namene. Nakon što je utvrđeno
da su navedene serije, u dužem intervalu, bile u kointegraciji sa pripa-
dajućom tehnikom Johansena, u istraživanju je upotrebljena ispravka
grešaka (error correction), kao i Grejndžerov test uzročnosti (Granger
causality test), kako bi se postigli postavljeni ciljevi. Rezultati ukazuju
na to da stalni troškovi imaju značajan pozitivan uticaj na stvarni BDP,
dok je uticaj kapitalnih troškova na BDP negativan. Grejndžerov test
uzročnosti otkrio je da, kako stalni, tako i kapitalni troškovi utiču na
stvarni BDP. Otuda, ovaj rad se zalaže za sistem temeljnijeg praćenja i Ključne reči:
procene načina na koji se koriste sredstva vlade, posebno ona namenjena kapitalni troškovi, ekonomski
kapitalnim projektima, kako bi uticaj na samu privredu bio pozitivan. razvoj, mehanizam ispravke
grešaka, Grejndžerova uzročnost,
bruto domaći proizvod
108
FORECASTING MODEL OF VIETNAMESE CONSUMERS’ PURCHASE

DECISION OF DOMESTIC APPAREL
Dung Tien Luu*
Lac Hong University, Viet Nam

Ho Phi Dinh, Phan Thiet University, Viet Nam

The study of the determinants of consumer purchase decision of domestic
Received: February 20, 2019
goods is necessary in the context of Vietnams’ international integration
in order to support domestic firms to improve their competitiveness.
This study aims to analyze the factors influencing Vietnamese con-
sumers’ purchase decision of domestic apparel based on the Binary
Logistic regression model. This study uses survey data gathered from
240 consumers in Vietnam in 2019. The research results showed that
i) Perceived price, ii) Perceived quality, iii) Consumer ethnocentrism,
and iv) Demographic variables (age; the family with a child; the level of
education; income; and sex) have significant influences on the consumer
purchase decision of domestic apparel in Vietnam.
Keywords:
consumer ethnocentrism, domestic
apparel, perceived pric, perceived
quality, Vietnam.
109
*E-mail: dunglt@lhu.edu.vn
EJAE 2019  16 (2)  109-121
TIEN LUU D.  FORECASTING MODEL OF VIETNAMESE CONSUMERS’ PURCHASE DECISION OF DOMESTIC APPAREL
INTRODUCTION
Vietnam's economy continues its deep international integration process with the continued signing
of the Comprehensive and Progressive Partnership Trans-Pacific Partnership (CPTPP), the world's
largest free-trade area with a combined market of 600 million people, and together with 16 signed
linkage mechanism (1990-2016) promises to continue to increase growth opportunities for many
economic sectors. At the same time, competitive pressure on the domestic market will also increase
strongly in Vietnamese firms. The relaxation of trade policies has provided consumers with more for-
eign manufactured product choices than ever before. Consequently, their attitudes toward products
originating from foreign countries have been of interest to domestic business and consumer behavior
researchers for decades (Lu Wang & Xiong Chen, 2004). Therefore, it is important to understand both
the theoretical and practical aspects of the factors affecting the behavior of the Vietnamese consumer
purchase decision.
Consumer behavior and consumer purchase behavior of domestic goods have received significant
attention from researchers (Knight, 1999; Watson & Wright, 2000; Vida & Damjan, 2001; Lu Wang &
Xiong Chen, 2004; Nguyen, Nguyen & Barrett, 2008; Dmitrovic, Vida, & Reardon, 2009; Renko, Crnjak
Karanović, & Matić, 2012; Ha-Brookshire, 2012; Purwanto, 2014; He & Wang, 2015; Asshidin, Abidin,
& Borhan, 2016; Mansi & Pandey, 2016; Kacprzak & Pawłowska, 2017; Puška et al., 2018). Previous
studies used a microeconomic theory approach, the behavioral economics theory to explain consumers'
domestic purchasing behavior and test for multiple product groups. However, there is no full research
model in all cases, and many studies, rather than the use of dependent variables, are the real decisions
of consumers made using other variables such as a willingness to buy or intention to purchase. Thus,
the predictability power of these research models is not really significant.
The present study aims to determine factors affecting Vietnamese consumers’ purchase decision of
domestic apparel based on the binary logistic regression model. The structure of the article consists of five
parts: i) Introduction, ii) Literature review, iii) Methodology, iv) Results and discussions, and v) Conclusions.
LITERATURE REVIEW
Kotler and Armstrong (2009) argue that consumers' decision-making processes were influenced
by the factors including cultural, social, personal, and psychological. Consumer behavior patterns are
used to describe the relationship between the three factors, including the stimulus, black box conscious-
ness, and responses responding to consumer stimuli that affect purchasing decisions of consumers. The
Theory of Reasoned Action (TRA) developed by Fishbein and Ajzen (1975) shows that the intention of
an individual's behavior is influenced by two factors, including behavioral attitude, and subjective norm.
These two factors directly affect the behavioral intent and then affect the actual behavior of an individual.
The Theory of Planned Behavior (TPB) states that the intention of an individual is influenced by three
factors: behavioral attitudes, subjective norms, and perceived behavior control. Perceived Behavioral
Control (PBC) refers to the ease or difficulty of performing a behavior and whether or not the behavior
is controlled (Ajzen, 1991). This extension involves explanations for when people intend to perform an
activity that is impeded by their lack of confidence or lack of the right to conduct behavior (Ajzen, 1991).
110
EJAE 2019  16 (2)  109-121
There are many previous kinds of research that have identified the factors that explain consumers’
purchase behavior towards domestic goods. Key elements proposed in the previous studies include
consumer ethnocentrism, perceived price, perceived quality, and demographic variables.
Consumers ethnocentrism
Ethnocentrism refers to the tendency of individuals to see their cultural group as proving the norms
for acceptable behaviors and preferences. Highly ethnocentric individuals are intolerant and judgmental
with respect to cultures different from their own (Luque-Martinez, Ibanez-Zapata, & Barrio-Garcia,
2000). The tendency of consumers to be ethnocentric represents their beliefs about the appropriateness
and moral legitimacy of purchasing foreign products (Shimp & Sharma, 1987). Ethnocentric consumers
prefer domestic goods because they believe that products from their own country are the best (Klein,
Ettenson, & Morris, 1998). Moreover, a concern for morality leads consumers to purchase domestic
products, even though the quality is poorer than that of imports (Wall & Heslop, 1986). Consumer
ethnocentrism may play a significant role when people believe that their personal or national well-
being is under threat from imports (Sharma, Shimp, & Shin, 1995; Shimp & Sharma, 1987). The more
importance a consumer places on whether or not a product is made in his/her home country, the higher
his/her ethnocentric tendency (Huddleston, Good, & Stoel, 2001). Sharma, Shimp, and Shin (1995)
show the effect of consumer preference leads to a preference for domestic goods, low prices for foreign
goods and unwillingness to buy foreign goods while always favoring domestic goods. According to
Netemeyer, Durvasula, and Lichtenstein (1991), the level of consumer preference indicates the degree
of influence of buyers' beliefs, attitudes and behavior toward domestic products. In a study that exam-
ined the relationship between consumer ethnocentrism and evaluations of foreign-sourced products.
Lantz and Loeb (1996) found that highly ethnocentric consumers have more favorable attitudes toward
products from culturally similar countries. The higher the purchasing power, the higher the domestic
buying behavior (Shimp & Sharma, 1987; Watson & Wright, 2000; Lu Wang & Xiong Chen, 2004;
Nguyen, Nguyen, & Barrett, 2008; Zafer Erdogan & Cevahir Uzkurt, 2010). In addition, individuals
with high levels of consumer ethnocentrism would have more favorable attitudes toward products from
culturally similar countries in comparison to products from culturally dissimilar countries, consumers'
ethnocentric tendencies play a significant role in predicting purchase intentions towards domestically
produced goods (Vida & Damjan, 2001; Dmitrovic, Vida, & Reardon, 2009; Renko, Crnjak Karanović,
& Matić, 2012; Purwanto, 2014; He & Wang, 2015).
H1 - Consumers' ethnocentrism has a positive impact on purchase decision of domestic apparel in
Vietnam.
Perceived price
The price represents an extrinsic sign and provides one of the most important forms of informa-
tion available to consumers when making a purchase decision (Jin & Sternquist, 2003). Diehl, Kornish,
and Lynch (2003) argue that consumers are price sensitive because they tend to look for lower-priced
products and they want satisfaction through price comparisons between different products. Teas and
Agarwal (2000) argue that the price offered was positively related to the perception of product quality
and sacrificed by consumers. Zeithaml (1988) distinguished prices into two categories: objective price
111
EJAE 2019  16 (2)  109-121
is the actual price of the product or service and price is perceived by the consumer. Price perception is
a comparison between the actual objective price and the price at which consumers refer to, reflecting
the buyer's subjective feelings or feelings about the price objective. On the consumer side, they often
do not remember or know the actual price of the product, but instead feel the price suitability based
on the perceived usefulness of the merchandise. This approach refers to the suitability of the price
of the product or service that the user perceives. When consumers feel positive about price stability,
and compare it to the usefulness they receive, they will shape their buying intentions and behavior
(Ha-Brookshire, 2012; Winit & Gregory, 2009; Prasad, 2014). Higher price is associated with higher
likelihood of purchase (Sternquist, Byun, & Jin, 2004).
H2 - The perceived price of domestic apparel has a positive influence on consumer purchase deci-
sion in Vietnam.
Perceived quality
Consumers’ intention to purchase domestic products will be influenced by perceived quality. Per-
ceived quality is a cognitive response to a product which influences product purchase (Kumar, Lee,
& Kim, 2009). In literature, quality perception is treated as a multi-dimensional concept, including
appearance, color and design, durability, fashion, functionality, prestige, reliability, technical advance-
ment, value for money, and workmanship (Darling & Wood, 1990; Klein, Ettenson, & Morris, 1998;
Scott, Powers, & Swan, 1997; Watson & Wright, 2000). Thelen, Ford, and Honeycutt (2006) concluded
that product characteristics may influence product preference for domestic versus imported more than
consumer ethnocentrism levels. Consumers of the developing countries will go for non-local products
because they are generally deemed to be of high quality (Khattak, Saeed, & Shah, 2011). One of the
ways in which consumers form perceptions about a brand is based upon the quality (Dooley & Fryx-
ell, 1999). A positive perception of quality is the source of the consumer's purchase decision (Aaker,
1991). Quality perception influences consumers' buying intentions (Dodds, Monroe, & Grewal, 1991;
Asshidin, Abidin, & Borhan, 2016).
H3 - The perceived quality of domestic apparel has a positive influence on consumer purchase deci-
sion in Vietnam.
Demographic variables
Demographic factors play an important role in the purchasing process. Age, level of education,
occupation, number of household members, sex, marital status, and income are key demographic
variables that influence consumer behavior (Iqbal Ghafoor & Shahbaz, 2013; Alooma & Lawan, 2013;
Mazloumi et al., 2013; Sharma & Kaur, 2015; Mansi & Pandey, 2016; Kacprzak & Pawłowska, 2017;
Puška et al., 2018).
H4 - There is a significant relationship between demographic variables and consumer purchase decision
of domestic apparel in Vietnam.
The logit model was estimated to explain and predict consumer’s purchase decision of domestic products.
The logit model was chosen for this study because of its mathematical simplicity and because of its asymp-
totic characteristics that constrain the predicted probabilities to a range between zero and one (Maddala,
1983). The maximum likelihood (ML) estimation procedure was used to obtain the model parameters. The
112
EJAE 2019  16 (2)  109-121
model, selected to analyze the dependence of purchase decision (Y) on the main factors, was specified as:
Y = β1 + β2CONSUMER ETHNOCENTRISM + β3PERCEIVED QUALITY + β4 PERCEIVED
PRICE + β5AGE + β6FAMILY WITH CHILD + β7LEVEL OF EDUCATION + β8INCOME + β9SEX
+ µ.
In which:
- Y: Purchasing decision.
- β1 to β9: Regression coefficients of the independent variables (Consumer ethnocentrism, Perceived
quality, Perceived price, Age, Family with child, Level of education, Income, and Sex, respectively.)
- µ: Residual (other factors not in the model).
METHODOLOGY
Binary Logistic Model was used to estimate parameters in the model. There are 14 items to measure
Perceived price, Perceived quality, and Consumer ethnocentrism in the model (see Table 2). All items
are measured by 5-point Likert scales, which were 5 – strongly agree, 4 – agree, 3 – not sure, 2 – disa-
gree and 1 – strongly disagree. We adopted previously validated measures with additional testing of
reliability for indicator scale in this study.
The logit model is generally specified as follows:
p =1
Ln [ =] β 0 + β1 X 1 + β 2 X 2 + … + β k X i (1)
p=0
Where:
Ln: Ln is logarithm to the base of the mathematical constant e (e = 2.714).
P (Y=1) = P0: Probability of consumer purchase decision of domestic apparel.
P (Y = 0) = 1- P0: Probability of consumers’ non-purchasing decision of domestic apparel.
Xi: Independent variables.
β is a scalar parameter to be estimated in equation (1).
Odds ratio ( O0 ) .
P0 P ( Puchaser )
=
O0 =
1 − P 0 P ( Non _Purchaser)
Replace ( O0 ) into the function (1):
LnO0= β 0 + β1 X 1 + β 2 X 2 + … + β k X i ( 2 )
The logarithm of the Odds ratio is a linear function with independent variables Xi (Cox, 1970).
Agresti (2007) indicated the prediction function of the binary logit model:
113
EJAE 2019  16 (2)  109-121
e( LnOdds )
E (Y / Xi ) =
1 + e( LnOdds )
E (Y/Xi): Probability of Y = 1, when X equals Xi
LnOdds = β0 + β1X1 + β2X2 + … + βkXi (2)
According to Hair, Black, Babin, and Anderson (2010), the sample size used in the exploratory fac-
tor analysis was determined by the minimum (min = 50) and the number of variables included in the
model. The proportion of samples compared to an analytical variable was 5/1 or 10/1, which included
14 observations and therefore the sample size is at least 14*5 = 70 observations. Green (1991) suggested
that in the regression model the minimum sample size was determined by the empirical formula 50 +
8*independent variables in the model. This study has 08 independent variables so the sample size is at
least 50 + 8*8 = 114 observations. Statistical analysis of the data obtained in this study was performed
using the SPSS 22.0 software tool.
The sample for the study was drawn conveniently from 240 consumers in Ho Chi Minh city which
represent different geographic, cultural, and commercial backgrounds of Vietnam. The stratified
sampling plan was followed based on the population distribution in the districts of the city in order
to ensure the representation of the research sample. Personal interviews were conducted at the large
supermarkets by university students recruited from the city. The raw data set was publicly available at
https://data.mendeley.com/datasets/dwj7hg3dt2/1
The research required that we examine a product category in which a domestic alternative was
available. Apparel products were chosen as the domestic product category in this study.
The result of the survey showed that 121 cases (50.4 percent) purchased domestic apparel regularly
while 119 cases (49.6 percent) did not. Demographic variables were described in Table 1.
Demographic Variables Frequency Percentage

1. Male 126 52.5%
Sex
0. Female 114 47.5%
1. Yes 150 62.5%
Family with child
0. Others 90 37.5%
1. High school or higher 158 65.8%
Level of education
0. Others 82 34.2%
1. 500 USD or higher 67 27.9%
Income/month:
0. Less than 500 USD 173 72.1%
1. 0%
27 11.3%
2. Less than 5%
45 18.8%
3. 6-10%
Willing to Pay for domestic prod- 62 25.8%
ucts on imported products: 4. 11-15%
53 22.1%
5. 16-20%
19 7.9%
6. Less than 20% and more
34 14.2%
114
EJAE 2019  16 (2)  109-121
1. None 54 22.5%
2. Less than 30 USD/ month 71 29.6%
Consumption of domestic products: 2. 30-50 USD/ month
3. More than 50 USD/ month 52 21.7%
63 26.3%
1. regular buyers (buying domestic 121 50.4%
apparel products sometimes, fre-
Purchasing decision (Y) quently or always)
0. irregular buyers (never or rarely
buy domestic apparel products) 119 49.6%
Table 1. Description of demographic variables
Reliability and Validity
The reliability and validity of indicators in the model are tested by the system of criteria. As can be
seen from Table 2, the lowest Cronbach’s alpha value is 0.712, exceeding the cut value of 0.70 recommended
(Hair, Black, Babin, & Anderson, 2010). Convergent validity was estimated by factor loading. The value
of Kaiser-Mayer-Olkin (KMO) was 0.833 (between 0.5 and 1.0) which means that data is significant for
conducting a factor analysis. All loadings of variables are higher than the 0.50 (see Table 2). According to
Hair, Black, Babin, and Anderson (2010), loadings ± 0.50 or greater are considered practically significant.
Code Items
Perceived price; Cronbach’s alpha = 0.712 Factor Loadings
The price of domestic apparel is more acceptable than foreign manufac-
PP1 0.626
tured products.
PP2 Compared to the quality, the price of domestic apparel is cheaper. 0.744
The amount of money to buy domestic apparel is perfectly suited to me
PP3 0.793
personally.
The amount of money to buy domestic apparel compared with foreign manu-
PP4 0.549
factured products is reasonable.
Perceived quality; Cronbach’s alpha = 0.777
PQ1 Seam strength is not inferior to foreign manufactured apparel. 0.732
PQ2 The fabric is not inferior to foreign manufactured apparel. 0.715
PQ3 Brand prestige is not inferior to foreign manufactured apparel. 0.741
PQ4 Production techniques are not inferior to foreign manufactured apparel. 0.725
Consumers ethnocentrism; Cronbach’s alpha = 0.821
CE1 Foreign manufactured apparel products are a bad behavior of Vietnamese. 0.648
CE2 Vietnamese had better buy goods made in Vietnam. 0.654
Buying foreign manufactured apparel is contributing to the loss of job among
CE3 0.806
Vietnamese workers.
CE4 Buying foreign manufactured apparel will help other countries get rich. 0.756
CE5 Foreign manufactured apparel cause harm to domestic firms. 0.768
People should only buy foreign manufactured apparel when it cannot be
CE6 0.546
produced domestically.
Table 2. Results of factor analysis and reliability tests.

115
EJAE 2019  16 (2)  109-121
Hypothesis Testing
Table 3 shows maximum likelihood estimates of the binary logistic regression model, all of three esti-
mated coefficients have expected signs and significance at the 10%, 5% level or higher level. The Nagel-
kerke R2 is 0.330, an upper bound R2 for binary-choice models. The likelihood ratio test is significant
at the 1-percent level, indicating the model has significant explanatory power.
Change in Prob-
ability for signifi-
Beta Wald Sig. Exp (B)
cant Coefficients
(P0 = 10%)
Consumer ethnocentrism 0.716 6.686 .010 2.047 18.52
Perceived quality 0.673 7.129 .008 1.960 17.88
Perceived price 0.791 10.794 .001 2.205 19.68
Age -0.073 5.771 .016 0.930 9.37
Family with child 0.802 6.311 .012 2.229 19.85
Level of education 1.513 18.941 .000 4.540 33.53
Income -0.906 6.877 .009 0.404 4.30
Sex -0.567 3.345 .067 0.567 5.93
Constant -6.217 13.428 .000 .002 -
Omnibus test
Chi-Square 68.290
Significance 0.000
Nagelkerke R Square 0.330
Correct prediction 75.8%
Table 3. Maximum Likelihood Estimates, Goodness-of-Fit Measures, and Change in Probability for significant
Coefficients
The probabilities presented in Table 3 show the effects of changes in the individual explanatory variables
on the likelihood of consumer purchase decision, assuming that all other explanatory variables are set to
zero. The likelihood of the consumer purchase decision increases by 18.52 percent if consumers get one
more unit in their ethnocentrism. The likelihood of the consumer purchase decision increases by 17.88
percent if consumers have one more unit in their perceived quality of domestic apparel. If consumers
have one more unit in their perceived price of domestic apparel, there is a 19.68 percent increase in the
likelihood of purchasing decision.
Among the demographic variables, the coefficients related to the families with a child, with a high school
education or above, younger consumer, female consumer, and lower income have impacts on consumer
purchase decision and are significant at the 10%, 5%, and 1% level. Probabilistically, those families with
a child were 19.85% more likely to buy domestic apparel regularly than others. Those with a high school
education or higher were 33.53% more likely to buy domestic products regularly than were those with
less than high school education. Similarly, those who are younger consumers, female consumers, and
lower income consumers were 9.37%, 5.93%, and 4.30% more likely to buy domestic apparel, respectively.
Facing the context of Vietnam's extensive international integration, competition between domestic and
foreign manufactured goods is fierce. Consumer purchase decisions for domestic products play an im-
116
EJAE 2019  16 (2)  109-121
portant role in the competitiveness of domestic enterprises. This study suggests that the research model
is quite adequate when assessing the impact of behavioral, psychological, and demographic factors on
consumer purchase decision of domestic product. The research results contribute significantly to the cur-
rent theoretical framework in clarifying consumer purchase behavior for domestic goods in one of the
emerging markets. The research results are similar to the results of previous studies cited in the literature
review of this article.
Based on the research results, the forecasting model of domestic apparel purchase by theVietnamese
consumer could be shown as Table 4:
Order Variables β (beta) Xi

Min Max
1 Consumer ethnocentrism 0.716 1 5
2 Perceived quality 0.673 1 5
3 Perceived price 0.791 1 5
4 Age -0.073 16 50
5 Family with child 0.802 0 1
6 Level of education 1.513 0 1
7 Income -0.906 0 1
8 Sex -0.567 0 1
9 Constant -6.217 - -
LogOdds -5.205 1.875
e LogOdds 0.006 6.502
1 + e LogOdds 1.006 7.502
P (Y / Xi ) 0.006 86.67
Table 4. Forecasting model of Vietnamese consumer purchase behavior of domestic apparel products
CONCLUSIONS
The study uses the Binary logistic econometric model to explain the factors affecting the Vietnam-
ese consumer purchase decision of domestic apparel based on the data obtained from 240 consumers
in Vietnam. The results show that consumer purchased decision was significant influences by i) the
perception of price, ii) the perception of quality, iii) consumer ethnocentrism, and iv) demographic
variables (age; families with a child; the level of education; income; and sex). This study provides valu-
able implications for business in the domestic market in the context of Vietnam's international integra-
tion. Domestic firms must recognize the significant relationship between consumers’ perceived price,
perceived quality of domestic products, consumer ethnocentrism, and their demographic variables
with their purchase decision. Positive perceptions of quality and price are important for the long-term
success of a brand. Therefore, it is recommended that domestic firms employ aggressive strategies to
improve consumer perception of their brands in terms of quality and price appeal among those who
have a higher interest in domestic products.
117
EJAE 2019  16 (2)  109-121
This study has its limitations which can be resolved in future studies. Firstly, this study only conducted
a product survey in one area of Vietnam, and therefore it is possible that the sample is not representative.
In future studies, a wide range of products should be examined, and in more areas too, such as urban
and rural ones. In addition, a comparison should be made with as many countries as possible so as to
have more multidimensional assessments of consumers' domestic purchases; Secondly, this study uses
the Binary Logistic model in accordance with the nature of the dependent variable. However, consum-
ers’ buying behavior also involves buying intentions, willingness to pay, satisfaction and loyalty after
purchase. Therefore, further research should consider including these variables in the research model
which are based on other quantitative models such as the structural equation model.
REFERENCES
Aaker, D. A (1991). Managing Brand Equity: Capitalizing on the Value of a Brand Name. New York: The Free Press.
Agresti A. (2007). An introduction to Categorical Data Analysis. New York: A John Wiley & Sons Publication.
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50
(2), 179-211.
Alooma, A.G., & Lawan, L.A. (2013). Effects of Consumer Demographic Variables on Clothes Buying Behaviour
in Borno State, Nigeria. International Journal of Basic and Applied Science, 1(4), 791-799.
Asshidin, N. H. N., Abidin, N., & Borhan, H. B. (2016). Perceived quality and emotional value that influence
consumer's purchase intention towards American and local products. Procedia Economics and Finance,
35, 639-643.
Cox, D. R. (1970). Analysis of Binary Data. London: Chapman and Hall.
Darling, J.R. & Wood, V.R. (1990). A longitudinal study comparing perceptions of the US and Japanese consumer
products in a third/neutral country: Finland 1975 to 1985. Journal of International Business Studies, 21(3),
427-50.
Diehl, K., Kornish, L. J., & Lynch Jr, J. G. (2003). Smart agents: When lower search costs for quality information
increase price sensitivity. Journal of Consumer Research, 30 (1), 56-71.
Dmitrovic, T., Vida, I., & Reardon, J. (2009). Purchase behavior in favor of domestic products in the West Balkans.
International Business Review, 18(5), 523-535.
Dodds, W.B., Monroe, K.B., & Grewal, D. (1991). Effects of price, brand, and store information on buyers’ product
evaluations. Journal of Marketing Research, 28, 307-319.
Dooley, R.S. & Fryxell, G.F. (1999). Attaining decision quality and commitment from the dissent: the moderating
effects of loyalty and competence in strategic decision-making teams. Academy of Management Journal,
42 (4), 389-402.
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research.
Reading, Mass: Addison-Wesley.
Green, S. B. (1991). How many subjects does it take to do a regression analysis? Multivariate Behavioral Research,
26, 499‐510.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th Ed. Vol. 5). New
York, Upper Saddle River: Pearson Prentice Hall.
Ha-Brookshire, J. E. (2012). Country of parts, country of manufacturing, and country of origin: consumer pur-
chase preferences and the impact of perceived prices. Clothing and Textiles Research Journal, 30(1), 19-34.
He, J., & Wang, C. L. (2015). Cultural identity and consumer ethnocentrism impacts on preference and purchase of
domestic versus import brands: An empirical study in China. Journal of Business Research, 68(6), 1225-1233.
118
EJAE 2019  16 (2)  109-121
Huddleston, P., Good, L.K. & Stoel, L. (2001). Consumer ethnocentrism, product necessity and Polish consum-
ers’ perceptions of quality. International Journal of Retail & Distribution Management, 29 (5), 236-46.
Iqbal, H.K., Ghafoor M.M., & Shahbaz, S. (2013). Impact of Demographic Factors on Store Selection: An Insight
in Pakistani Society. Journal of Marketing Management, 1(1), 34-45.
Jin, B., & Sternquist, B. (2003). The influence of the retail environment on price perceptions: An exploratory study
of US and Korean students. International Marketing Review, 20(6), 643-660.
Kacprzak, A., & Pawłowska, A. (2017). Work and shopping overflow: Consequences and differentiation among
selected psychological and demographic characteristics. European Management Journal, 35(6), 755-765.
Khattak, M. N., Saeed, M. M., & Shah, T. A. (2011). Consumers’ Attitudes Towards Non-Local Products: An
Empirical Evidence from Pakistan”. Interdisciplinary Journal of Contemporary Research in Business, 3(2),
2039-2048.
Klein, J.G., Ettenson, R. & Morris, M.D. (1998). The animosity model of foreign product purchase: an empirical
test in the people’s republic of China. Journal of Marketing, 62, 89-100.
Knight, G. A. (1999). Consumer preferences for foreign and domestic products. Journal of Consumer Marketing,
16 (2), 151-162.
Kumar, A., Lee, H. J., & Kim, Y. K. (2009). Indian consumers' purchase intention toward the United States versus
the local brand. Journal of business research, 62(5), 521-527.
Lantz, G. & Loeb, S. (1996). Country of origin and ethnocentrism: an analysis of Canadian and American prefer-
ences using social identity theory. Advances in Consumer Research, 23, 374-388.
Luque-Martinez, T., Ibanez-Zapata, J. A., & Barrio-Garcia, S. (2000). Consumer ethnocentrism measurement-
An assessment of the reliability and validity of the CETSCALE in Spain. European Journal of Marketing,
34(11/12), 1353-1374.
Lu Wang, C., & Xiong Chen, Z. (2004). Consumer ethnocentrism and willingness to buy domestic products in a
developing country setting: testing moderating effects. Journal of Consumer Marketing, 21 (6), 391-400.
Maddala, G. S. (1983). Limited-dependent and qualitative variables in econometrics (No. 3). Cambridge University
Press.
Mansi, M., & Pandey, R. (2016). Impact of demographic characteristics of procurement professionals on sustain-
able procurement practices: Evidence from Australia. Journal of Purchasing and Supply Management,
22(1), 31-40.
Mazloumi, S.S.S., Efteghar, A., Ghalandari, A., Saifi, B., & Aghandeh, I. (2013). Evaluating the effect of demo-
graphic differences on consumers’ purchasing behavior (Case Study: Tetra Pak Consumers). International
Research Journal of Applied and Basic Sciences, 4(7), 1866-1867.
Nguyen, Tho D, Nguyen Trang T.M, & Nigel J. Barrett. (2008). Consumer ethnocentrism, cultural sensitivity,
and intention to purchase local products evidence from Vietnam. Journal of Consumer Behavior, 88-100.
Netemeyer, R. G., Durvasula, S., & Lichtenstein, D. R. (1991). A cross-national assessment of the reliability and
validity of the CETSCALE. Journal of marketing research, 28 (3), 320-327.
Kotler, P. & Armstrong, G. (2009). Principles of Marketing. Hanoi: Labor-Social Publication.
Prasad, G. H. S. (2014). Factors Influencing Buying Behavior of a Selected Apparel Retailer’s Customers. Annual
Research Journal of Symbiosis Centre for Management Studies, Pune, 1, 41-55.
Purwanto, E. (2014). The effect of consumer ethnocentrism on perceived domestic product quality and purchase
intentions among young consumers in Jakarta, Indonesia. International Journal of Asian Social Science,
4(9), 1003-1012.
Puška, A., Stojanović, I., Šadić, S., & Bečić, H. (2018). The influence of demographic characteristics of consum-
ers on decisions to purchase technical products. The European Journal of Applied Economics, 15(2), 1-16.
Renko, N., Crnjak Karanović, B., & Matić, M. (2012). Influence of consumer ethnocentrism on purchase inten-
tions: Case of Croatia. Ekonomska misao i praksa, (2), 529-544.
119
EJAE 2019  16 (2)  109-121
Scott, W.H., Powers, T.L. & Swan, J.E. (1997). Modeling industrial buyer complaints: implications for satisfying
and saving consumers. Journal of Marketing Theory and Practice, 5 (4), 12-22.
Sharma, S., Shimp, T. & Shin, J. (1995). Consumer ethnocentrism: a test of antecedents and moderators. Journal
of the Academy of Marketing Science, 23 (1), 26-37.
Shimp, T. A., & Sharma, S. (1987). Consumer ethnocentrism: Construction and validation of the CETSCALE.
Journal of marketing research, 280-289.
Sternquist, B., Byun, S. E., & Jin, B. (2004). The dimensionality of price perceptions: a cross-cultural comparison of
Asian consumers. The International Review of Retail, Distribution and Consumer Research, 14(1), 83-100.
Teas, R. K., & Agarwal, S. (2000). The effects of extrinsic product cues on consumers’ perceptions of quality,
sacrifice, and value. Journal of the Academy of Marketing Science, 28(2), 278-290.
Thelen, S., Ford, J. B., & Honeycutt Jr, E. D. (2006). Assessing Russian consumers' imported versus domestic
product bias. Thunderbird International Business Review, 48(5), 687-704.
Vida, I., & Damjan, J. (2001). The Role of Consumer Characteristics and Attitudes in Purchase Behavior of Do-
mestic versus Foreign Made Products: The Case of Slovenia. Journal of East-West Business, 6(3), 111-133.
Wall, M. & Heslop, sL.A. (1989). Consumer attitudes towards the quality of domestic and imported apparel and
footwear. Journal of Consumer Studies and Home Economics, 13, 337-58.
Watson, J. J., & Wright, K. (2000). Consumer ethnocentrism and attitudes toward domestic and foreign products.
European Journal of Marketing, 34 (9/10), 1149-1166.
Winit, W., & Gregory, G. (2009). Do Consumers Really Pay More for Their Own Country Brand? The Impact of
Price and Ethnocentrism on Local Brand Preference. In Proceedings of ANZMAC Sustainable Management
and Marketing Conference, (http://www. Duplication.net.au/ANZMAC09/papers/ANZMAC2009-534. pdf).
Zafer Erdogan, B., & Uzkurt, C. (2010). Effects of the ethnocentric tendency on consumers' perception of prod-
uct attitudes of foreign and domestic products. Cross-Cultural Management: An International Journal,
17 (4), 393-406.
Zeithaml, V. A. (1988). Consumer perceptions of price, quality, and value: a means-end model and synthesis of
evidence. Journal of Marketing, 52 (3), 2-22.
120
EJAE 2019  16 (2)  109-121
MODEL PREDVIĐANJA DONOŠENJA ODLUKA KUPACA U VIJETNAMU

PRILIKOM KUPOVINE ODEĆE DOMAĆE PROIZVODNJE
Rezime:
Kako bi se podržale firme koje pripadaju domaćem tržištu, a teže
postizanju konkurentnosti, u svetlu integracije Vijentama na među-
narodnom planu, neophodna je analiza pokazatelja odluka kupaca
prilikom kupovine domaćih proizvoda. Ovo istraživanje ima za cilj
da utvrdi činioce koji utiču na odluke kupaca u Vijetnamu, u vezi sa
kupovinom odeće domaće proizvodnje – a sama analiza zasnovana je
na modelu binarne logističke regresije. U istraživanju su upotrebljeni
podaci koji ilustruju 240 kupaca u Vijetnamu, iz 2019. godine. Rezultati
pokazuju da: i) utisak o ceni i) utisak o kvalitetu, iii) etnocentrizam Ključne reči:
kupca kao i iv) demografske varijable (starosna dob; deca u porodici; etnocentrizam kupca, odeća
stepen obrazovanja; prihod i pol) značajno utiču na odluke kupaca u domaće proizvodnje, utisak o ceni,
Vijetnamu, u vezi sa kupovinom odeće domaće proizvodnje. kvalitetu, Vijetnam
121
Review Paper/ Pregledni naučni rad.
WHAT CAN WE EXPECT IN THE FUTURE OF ACADEMIC RESEARCH?

MOST COMMON RESEARCH PROBLEMS ANALYSED IN THE TOP
JOURNALS IN THE FIELD OF ENTREPRENEURSHIP
Irena Đalić1 *
1 Faculty of Transport and Traffic Engineering, University of East Sarajevo,

In the modern era of academic research, it is extremely difficult to define
Received: April 02, 2019
an entirely or largely unexplored research problem. Every researcher
Correction: May 08, 2019
is faced with this question in their research. It takes a lot of work and
Accepted: May 15, 2019
effort to primarily find a topic that is relevant and that will be attractive
for study and research. The focus of this paper is research problems
related to the field of entrepreneurship. This paper deals with trending
research problems over the course of the last five years in the field of
entrepreneurship. The survey was conducted with the aim of identifying
the least explored areas of entrepreneurship in order to predict future
research topics. The methods used to achieve this scientific goal include:
description, classification and explanation. By analyzing the sample of
393 papers from the five most cited journals in the field of entrepre-
neurship, the author came to the conclusion that in the past five years
the majority of papers dealt primarily with the topic of innovation and
advanced technology, and hardly touched upon the topic of women's
entrepreneurship. This paper should help future researchers to select
the topics or fields of research in the domain of entrepreneurship.
Keywords:
entrepreneurship, journals,
indexed, citations
122
*E-mail: i.naric@yahoo.com
EJAE 2019  16 (2)  122-138
ĐALIĆ. I WHAT CAN WE EXPECT IN THE FUTURE OF ACADEMIC RESEARCH? MOST COMMON RESEARCH PROBLEMS ANALYSED IN THE TOP
INTRODUCTION
What initiated the publication of research results in scientific journals is the need of scientists
and researchers to communicate with each other and gain an insight into current studies. The age
of scientific journals dates from1665, when the French Journal des sçavans and the English journal
Philosophical Transactions of the Royal Society began to periodically publish the results of scientific
studies (Kronick, 1976). After the appearance of these journals, there was an increase in the number
of journals in all areas of scientific studies. A significant number of journals are specialized in one
particular area of scientific research, although there are newspapers which publish articles in various
fields of research. Papers published in journals deal with the latest studies in a particular scientific field.
Papers published in scientific journals are mainly intended for immediate scientific community and
may be incomprehensible to those who are not sufficiently familiar with the particular area of research
that the journal deals with.
There are several types of papers published in scientific journals, although the exact terminology and
definitions vary depending on the scientific field they focus on, as well as from the scientific journals
themselves. The papers published in journals according to the Regulations on publishing scientific
publications in the Republic of Srpska (RS Official Gazette, 2017, no. 77/17), can be categorized as:
original scientific papers, review papers, short or preliminary communications, critical reviews, informa-
tive annexes, books, instruments, computer software, cases, scientific event reports, and the like. The
format of scientific papers can vary greatly from journal to journal. Despite that, the rules of writing
journal papers are mainly determined by IMRAD methodology, recommended by the International
Committee of Medical Journal Editors (ICMJE). There is a profusion of articles on the subject of what
IMRAD methodology is and how to use it (Radek, 2016; Vuckovic, 2014; Papakostidis & Giannoudis,
2018; Malicki, 2016; Nair & Nair, 2014; Bertin & Atanassova, 2014; Krausman et al. 2016).
This paper focused on the analyses of journals in the field of entrepreneurship that currently rank
among top five with regard to the number of citations. A total amount of 393 papers from 5 journals
(Research Policy, Entrepreneurship Theory and Practice, Journal of Business Venturing, Small Business
Economics, Journal of Product Innovation Management) published from 2013 until the beginning of
2018 were analyzed, providing that the papers from the beginning of 2018 were taken into account as
well.
According to the conducted research it will be possible to determine which topic is most frequently
and most commonly explored, and which one is investigated (explored) to an insufficient/inadequate
degree. These results could show what will happen in the current research trends in the field of entre-
preneurship in the upcoming years. The main goal of this paper is to draw attention to the unexplored
areas in the field of entrepreneurship and provide guidance for future researchers regarding the prob-
lems that need to be solved.
In the first part of the paper, the author will review the relevant literature to confirm the relevance
of our research. The second part relates to the indexing of journals and citation database. In this sec-
tion, our goal is to explain what general indexing and citation is and how citation database is formed.
The results of the research are presented in the third part of the paper. In this section, the author will
describe the problems faced during the research, along with research findings. In the end, the author
summarizes these findings and provides the list of relevant literature used in this research.
123
EJAE 2019  16 (2)  122-138
LITERATURE REVIEW
There are numerous studies dealing with the analysis of journals in all research fields. This is not
an unfamiliar topic. In this way, the question of which topics are extensively and which ones poorly
explored in a given field of study is answered. Such a study is the research by Ritzberger (2008) in
which he analyzed and ranked the various journals in the field of economics. There are many authors
who have tried to classify and explain the ranking of journals in the field of economy (Kalaitzidakis,
Mamuneas & Stengos, 2003; Lubrano, Bauwens, Kirman & Protopopescu, 2003; Kodrzycki & Yu, 2006;
Mingers & Harzing, 2007; Bornmann, Butz & Wohlrab, 2018). The fact that many authors have dealt
with the explanation of different methods of ranking journals in the field of economy proves that this
job is not easy (Liner & Amin, 2004; Koczy & Strobel, 2007; Wohlrab, 2016; Subochev, Aleskerov &
Pislyakov, 2018).
We have already noted that the focus of this paper is to analyze research problems in the field of
entrepreneurship. Several authors have dealt with this theme so far (Rey-Martí, Ribeiro-Soriano &
Palacios-Marqués, 2016; Alvarez-Garcia, Maldonado-Erazo, del Río & Sarang-Lalangue, 2018; López-
Fernández Serrano Bedia & Pérez Pérez, 2016; Xu, Chen, Fung & Chan, 2018; Ferreira, Reis & Miranda,
2015; McDonald, Gan, Fraser, Oke & Anderson, 2015). In our paper, the author focused on the problems
that are most studied and on the ones that are least studied in the field of entrepreneurship. In this
way, the author wants to give a scientific contribution and advise future scientists what to study in the
near future; and that is what this paper aims to achieve. This is supported by the fact that, through this
research related to the top five journals in the field of entrepreneurship, which are the highest ranked
in the last five years (2013-2018), not a single paper that deals with the analysis of journals in the field
of entrepreneurship turned up. The part of the paper associated with the literature review will include
three areas that are most and two areas that are least explored.
The vast majority of the authors who have fallen within the scope of the research have studied the
character of innovation and advanced technologies and their influence on the growth and development
of enterprises and the economy (Werfel & Jaffe, 2013; Hashi & Stojčić, 2013; Thomä & Bizer, 2013; Mus-
teen & Ahsan, 2013; Dachs & Peters, 2014; Hung & Tu, 2014; Dai, Maksimov, Gilbert & Fernhaber, 2014;
Sahut & Peris-Ortiz, 2014; Audretsch, Coad & Segarra, 2014; Slater, Mohr & Sengupta 2014; Venturini,
2015; Block, Fisch, Hahn & Sandner, 2015; Sarooghi, Libaers & Burkemper, 2015; Parrilli & Heras, 2016;
Visnjic, Wiengarten & Neely, 2016; Walsh, Lee & Nagaoka, 2016; Dorner, Fryges & Schopen, 2017;
Nambisan, 2017; Belderbos, Jacob & Lokshin, 2018; De Massis, Audretsch, Uhlaner & Kammerlander,
2018). All these authors have dealt with the character of innovation and advanced technologies and their
influence on the growth and development of enterprises and the economy. This research has shown that
innovation is crucial for the development of enterprises, especially for the development of enterprises
which were engaged in some form of advanced technologies ever since their establishment. In order for
such companies to survive in the turbulent business environment, they have to turn to innovation and
continuous improvement of their operations (Colombel, Krafft & Vivarelli, 2016).
In addition, there are authors who have studied theoretical entrepreneurship and the importance
of introducing the subject of entrepreneurship at universities in order to educate young people in this
area (McCloskey, 2013; Leyden, 2014; Zhang & Cueto, 2017; Abreu & Grinevich, 2013; Freitas, Marques
& e Silva, 2013; Jung, 2014; Mowery & Ziedonis, 2015; Muscio, Quaglione & Ramaciotti, 2016; Walter,
Parboteeah & Walter, 2013; McCaffrey, 2014; Walter & Block, 2016; Braunerhjelm, Ding & Thulin,
2018; Guerrero, Urbano, Fayolle, Klofsten & Mian, 2016).
124
EJAE 2019  16 (2)  122-138
The next field considering the number of published papers is the field that relates to knowledge, or the
sources of knowledge (Huber, 2013; Autant-Bernard, Fadairo & Massard, 2013; Olmos-Peñuela, 2014;
Agarwal & Shah, 2014; Batabyal & Beladi, 2015; Roper, Love & Bonner, 2017; Marvel, 2013; Musteen,
Datta & Butts, 2014; Frederiksen, Wennberg & Balachandra, 2016; Zahra, 2015). These authors strived
to highlight the importance of knowledge both in the development of enterprises and starting a busi-
ness. In order for entrepreneurs to realize their ideas, they must first gain certain knowledge and skills.
One of the least explored areas is family business. Several authors have dealt with this kind of en-
trepreneurship. (Jaskiewicz, Block, Combs & Miller, 2017; Parker, 2016; Le Breton–Miller & Miller,
2015; Memili, Fang, Chrisman & De Massis, 2015; Cruz, Larraza–Kintana, Garcés–Galdeano & Berrone,
2014; Schmid, Achleitner, Ampenberger & Kaserer, 2014; Wilson, Wright & Scholes, 2013; Brannon,
Wiklund & Haynie, 2013).
The lowest number of papers in the survey (a total of 393 papers), with only two exploring this
field being recorded in the five journals, refers to female entrepreneurship (Hunt, Garant, Herman &
Munroe, 2013; Eddleston, Ladge, Mitteness & Balachandra, 2016).
It is impossible to cover the entire sample of 393 papers that were involved in the research. Therefore,
in this section, only a few papers relating to certain fields of the research are presented. The classifica-
tion of papers in various areas, as well as the fact that the topics discussed in these journals are now
trending, will be discussed in the part of this paper related to the research results.
INDEXING OF JOURNALS AND CITATION DATABASE
Until the end of the nineties of the 20th century, the collection of references and data on citations
was conducted by hand and came down to text reading and data input in the appropriate fields in the
database. Today this work is automated (Bergmark, 2000; Besagni, 2004; Cronin & Sugimoto, 2015).
Computer programs are being used that, based on optical character recognition of texts in digitized
print or electronic versions of scientific papers, identify relevant bibliometric data (title, author's name,
affiliation author, abstract, keywords, references, title of the parent publication, and when it comes to
newspapers – their volume, number , pagination, place of publication, publisher standard numbers -
ISBN, ISSN, dOI, etc., information about the funder of research), and then, using complex algorithms,
these computer programs analyze and classify data in the fields within the database (JACS, 2010). It is
very important that authors and journals strictly adhere to standards when quoting references. Thus
processed data serve as the basis for bibliometric analysis. Citation analysis was first conducted by
Garfield (2006; 1965). According to him, citation analysis is examination of incidence, patterns and
graphs of citations in articles and books. The number of citations a certain paper has shown how many
times individual scientists cited a paper in their papers, thus showing its use value which has its own
qualities. This dimension of quality is usually described by expressions such as echo, the significance,
the importance of scientific paper. These terms, however, say nothing about the character of the paper,
the reasons why the paper is important, useful, or has an echo. According to Plomp (1994), the indi-
cator of scientific success of a particular author's paper is that paper being cited at least 25 times. So
during the 1960s bibliographic database were first formed because of the need to allow users to easily
monitor, search and access the most relevant literature. An indexed bibliographic database monitors
and handles a large number of carefully selected publications, most of which consist of papers from
scientific journals.
125
EJAE 2019  16 (2)  122-138
A separate entity within the bibliographic databases makes reference database. Citing is a common
practice in scientific communication, so authors end their papers with a list of references. Citation
databases are secondary sources of scientific and technical literature which, with bibliographic descrip-
tion of the document (paper, book, etc.), consists of a list of references that the author/authors of the
document referenced. The primary task of citation indexes is to serve as a source of relevant scientific
literature, because the system of inclusion of journals and/or documents and publications has an inbuilt
mechanism of selectivity. The role of the citation indexes as a tool of bibliometrics and scientometrics
is primarily to assess the quality of scientific productivity (of articles, journals, etc.) in the creation of
a scientific policy, and to easily track the most reviewed and most important publications in the areas
of science (Meho & Yang, 2007). The most famous three citation databases are Web of Science, Scopus
and Google Scholar:
1. Bibliographic database Current Contents (CC) and citation databases Science Citation Index
(SCI), Social Sciences Citation Index (SSCI) and Arts and Humanities Citation Index (AHCI) are the
most selective worldwide databases. They were made by the Institute for Scientific Information (ISI)
from Philadelphia by the year 2004 (that is why they are called "ISI" databases), after which they were
acquired by Thomson Corporation (Thomson Reuters). As of 1997, these citation databases merged into
a single database, Web of Science (WoS). Web of Science is the oldest citation database (includes paper
s from 1900 to the present), which includes the world's scientific literature which is a subject to very
strict selection and quality control. This service contains multiple databases available on the Internet:
Science Citation Index Expanded, Social Sciences Citation Index, Arts and Humanities Citation Index,
Conference Proceedings Citation Index - Science Conference Proceedings Citation Index-Social Science
& Humanities. Since 2005, the Web of Science has been merged with the Book Citation Index (includes
monographs by major publishers), and since 2012 with the Data Citation Index (primary data) (more
about this in: Falagas, Pitsouni, Malietzis & Pappas 2008; Erfanmanesh, Didegah & Omidvar, 2017).
2. The index database Scopus was launched by the multinational publishing company Elsevier. This
service includes approximately 22,000 journals, mostly from the field of medicine, natural and social
sciences. It is possible to find citations in the literature published after 1996. A feature of this database
is that it covers Biomedical Science much better than all the others, but it is not entirely systematic
and consistent (more on this in: Bosman, Mourik, Rasch, Sieverts & Verhoeff, 2006; de Moya-Anegón,
2007; Khiste & Paithankar, 2017).
3. Google Scholar is an electronic resources browser, which can search the entire academic litera-
ture available on the Internet, without any selection. Consequently, many versions of the same reference
may be found on different sites. However, older literature is poorly represented. Furthermore, Google
Scholar does not provide a list of publishers whose data it collects, the list of journals, information
about the time period or scientific disciplines it covers (more on this in: Jacso, 2005; Bar-Ilan, 2008).
The same journal can be represented in a number of relevant databases. The analysis of citations in
ISI citation databases obtains numerical indicators. The most popular among them is called the Impact
Factor (IF) (Jemec, 2001). The IF is a measure that includes the annual average number of citations of
recently published papers from that journal (Garfield, 1999). It is often used as an expression of the
relative importance of the journal in a given field. Journals with higher IF are usually considered more
important than those with lower value factors. The current year journals’ IF is calculated by dividing
the number of citations received in the current year for papers published in the previous two years by
the number of papers published in the same two-year period. The impact factor is calculated by the
following formula (Sombatsompop, Markpin & Premkamolnetr, 2004; de & Rijcke Rushforth, 2015):
126
EJAE 2019  16 (2)  122-138
Citations y −1 + Citations y − 2
IFy = , y = the current year
Publications y −1 + Publications y − 2
When using the number of citations as an indicator of the paper quality, there are several reasons
why it is necessary to be cautious. It cannot be said that no one reads the papers that are not quoted or
that these papers have no scientific value, although one's scientific contribution is often reflected in high
citation rate (Oosthuizen & Fenton, 2014). In addition, the very act of citation does not imply recognition
in a positive way, but it may also be motivated by the need for correction, criticism or denial of ideas and
papers of others (Verma, 2015). Although ISI emphasizes the multidisciplinary and international character
of its databases, and the quality of the journal that it deals with as well, there was, throughout its history,
a lot of criticism when it comes to the representation of national journals and certain disciplines (Elliott,
2014). The reason for the criticism is its focus on newspapers in English and on developed countries at the
expense of small countries, developing countries and non-English-speaking countries (Callaway, 2016).
RESULTS AND LIMITATIONS OF THE RESEARCH
During the research, which lasted from early April until the end of May 2018, the author was able
to form a random sample of 393 papers from the field of entrepreneurship. The total number of paper
published in these five journals is much larger, but the author, in this period of research and, in view of
the limitations, managed to collect 393 papers. The author analyzed the papers from the five journals
that are currently considered as most cited in the field of entrepreneurship. During the research, the
author used the method of description, classification and explanation, owing to which some useful
results were obtained which will be discussed in this section. Obtained data and research results will be
shown in the tables for the sake of clarification. When sorting and analyzing data, statistical methods
to calculate the mean, median and mode motion were used, along with the methods whose purpose
was to find the maximum and minimum values.
LImitations of the research
The author has decided to choose this topic because it is trending (popular, recent, current) and so
as to, for the sake of future studies, give a clearer picture of what topics have and haven’t already been
explored. In this way, both us and future researchers who deal with entrepreneurship problems will be
able to select the topics that will allow them to actually contribute to science.
In addition to being faced with minor technical problems during the research, the author also expe-
rienced a bigger problem. Namely, since the author is not subscribed to any journal indexed database, it
was very difficult to obtain relevant journals and papers. Despite the promises, the Ministry of Science and
Technology did not provide funds that would allow the scientific community of the Republic of Srpska to
subscribe to any of the afore-stated databases. This method of support would greatly help researchers to
come up with relevant literature which is extremely useful in research of any field of science as it would,
first and foremost, shorten the time necessary to conduct studies.
After a long period of researching and collecting the relevant literature from available sources, the
author was able to form a sample of 393 papers and thus, more or less, successfully overcome this problem.
127
EJAE 2019  16 (2)  122-138
Results of the research
The author has already mentioned at the beginning that the subject of analysis were papers from
the five journals that have been among the top quoted ones in the last five years. They are:
1. Research Policy,
2. Entrepreneurship Theory and Practice,
3. Journal of Business Venturing,
4. Small Business Economics,
5. Journal of Product Innovation Management.
In Table 1 the author will show the participation of the journal according to the number of papers
in the total sample.
Table 1 - Journals’ participation in the sample.
No. Name No. of papers Share

1. Research Policy 123 31.30%
2.t Entrepreneurship Theory and Practice 91 23.15%
3. Journal of Business Venturing 76 19.34%
4. Small Business Economics 49 12.47%
5 Journal of Product Innovation Management 54 13.74%
TOTAL 393 100.00%
Source: The author
Table 1 shows that the majority of papers is collected from the journal Research Policy (IF: 4.661)
which also ranks first as the most cited journal in the field of entrepreneurship (123 papers or 31.30%
of the total sample). The second positioned source of papers is the journal Entrepreneurship Theory
and Practice (IF: 4.916), which is in the second place regarding the number of citations with 91 papers
or 23.15%. The next one, in terms of the number of papers in the sample, is the third-ranked journal
- Journal of Business Venturing (IF: 6.000) with 76 papers or 19.34%. The fourth place in the sample
is taken by the fifth-ranked Journal of Product Innovation Management (IF: 4.305) with 54 papers or
13.74%. Whereas the fifth place in the sample holds the journal Small Business Economics (IF: 2.852)
with 49 papers or 12:47%. According to these results, we can see that the journal which ranks first in
the number of citations was the most accessible to the author, because she was able to collect 123 pa-
per samples from it. The least accessible to the author was the journal Small Business Economics from
which the author obtained 49 papers.
128
EJAE 2019  16 (2)  122-138
The author classified all 393 papers according to the studied areas and divided them into 8 groups:
1. Sources of knowledge for entrepreneurs and small and medium enterprises,
2. Research management,
3. Theoretical and educational contribution to entrepreneurship,
4. Family business,
5. Financing SMEs,
6. Innovation and advanced technology,
7. Women’s entrepreneurship, and
8. State’s support in the development of entrepreneurship and the SME sector.
The main goal of this research is to try and obtain the answer to the question: what is the most commonly
studied section in the field of entrepreneurship, that is, what section is the most commonly studied
and which one the least explored in the last five years? By obtaining an answer to this question we can
make a prediction on what section of entrepreneurship will be the most researched in the future. Table
2 shows the sample papers’ share in the sections listed below.
Table 2 - The sample papers’ share in the research sections.

No. Section in the field of entrepreneurship No. of papers Share
1 Innovation and advanced technology 114 29.01%
Theoretical and educational contribution to entre-
2 104 26.46%
preneurship
3 Sources of knowledge 56 14.25%
State’s support in the development of entrepreneur-
4 52 13.23%
ship and the SME sector
5 Financing SMEs 35 8.91%
6 Research management 18 4.58%
7 Family business 12 3.05%
8 Woman’s entrepreneurship 2 0.51%
TOTAL 393 100.00%
Source: The author
Table 2 shows the period from 2013 to 2018, provided that the author took papers from the beginning
of 2018 into account papers which were published by the time this study was conducted.
Table 2 clearly shows that the majority of papers are the ones published in the field of innovation and
advanced technology, a total of 114 or 29.01% of the sample. The next sector, considering the number
of papers, is the sector that relates to the theoretical and educational contribution to entrepreneur-
ship. Most of these papers discuss the importance of introducing entrepreneurship as a subject at
universities and schools. There were 104 papers dealing with this theme or 26.46% of the total sample.
According to the number of published papers, the next one is the sector of sources of knowledge with
56 papers or 14,25% of the total sample. In this sector, the authors mostly dealt with the importance
of knowledge acquisition and skills in the field of entrepreneurship and how to provide this knowl-
edge. A large number of authors dealt with state policies concerning the support in the development
of entrepreneurship and the SME sector. These authors pointed out the importance and contribution
129
EJAE 2019  16 (2)  122-138
of government support in the development of entrepreneurship. Yet the state is the one that needs to
deal with all issues related to the development of its economy. The total sample has 52 papers related
to this field or 13.23%. Then there are papers related to the field of SMEs funding. The authors have
studied different sources of financing small businesses, especially businesses that deal with advanced
technologies, and risky jobs. These are mostly newer forms of financing such as angel investors, venture
capital funds, mezzanine financing, crowdfunding and others. A number of 35 papers covers this field
or 8.91%. The last three places are held by the management studies sector with 18 papers or 4.58%, the
family business sector with 12 papers or 3.5% and the women's entrepreneurship sector with only two
papers in the total sample or 0.51%.
The author will specify in Table 3 how great the sample journals’ participation in the different sectors
of entrepreneurship is.
Journal Entrepre- Journal of
Research
Research
Policy
neurship
Theory and
Journal of
Business
Small Busi-
ness Eco-
Product
Innovation
∑
Venturing nomics
sector Practice Management
Innovation and 55 20 7 22 10 114

advanced tech-
nology 48.25% 17.54% 6.14% 19.30% 8.77% 100%
Theoretical and 28 39 11 6 104
20
educational con-
tribution to en-
26.92% 37.50% 10.58% 5.77% 100%
trepreneurship 19.23%
5 13 21 14 3 56
Sources of knowl-
edge
8.93% 23.21% 37.50% 25.00% 5.36% 100%
State’s support in 23 18 1 - 10 52
the development
of entrepreneur-
44.23% 34.62% 1.92% - 19.23% 100%
ship and the SME
sector
8 4 5 - 18 35
Financing SMEs
22.86% 11.43% 14.28% - 51.43% 100%
10 - 3 5 - 18
Research manage-
ment
55.55% - 16.67% 27.78% - 100%
1 7 - 2 2 12
Family business
8.33% 58.33% - 16.67% 16.67% 100%
1 1 - - - 2
Women’s entre-
preneurship
50.00% 50.00% - - - 100%
∑ 123 91 76 54 49 393
Source: The author
130
EJAE 2019  16 (2)  122-138
Table 3 shows how great the participation of a particular journal is within each sector of entrepreneur-
ship, of course, all in the context of the sample of 393 papers. In the sector of innovation and advanced
technology, the dominant journal is Research Policy with 48.25% of the articles, while the lowest par-
ticipation has the Journal of Business Venturing with only 6.14%. As for the sector of theoretical and
educational contributions to entrepreneurship, the largest share of 37.50% has the Journal of Business
Venturing, while the lowest participation in the same field of entrepreneurship has the Journal of
Product Innovation Management with 5.77%. Within the sector relating to the sources of knowledge,
the largest share has the Journal of Business Venturing with 37.50%, while the lowest share has the
Journal of Product Innovation Management with 5.36% of papers in this research field. In the sector
of state support, the journal with the biggest share is Research Policy with 44.23% of the articles, while
the journal Small Business Economics has no papers in this field. In the sector of financing SMEs, the
largest share has the Journal of Product Innovation Management with 54.43% of the articles, while the
journal Small Business Economics has no involvement in this field. In the sector of research manage-
ment, the largest share has the journal Research Policy with 55.55% of papers and journals, while the
journals Entrepreneurship Theory and Practice and the Journal of Product Innovation Management
have not published any papers in this field of entrepreneurship. In the sector of family business, the
largest share has the journal Entrepreneurship Theory and Practice with 58.33% of the articles, while
the lowest share has the Journal of Business Venturing that has not published a paper in this scientific
field. According to this research, women's entrepreneurship is the least studied in the field of entre-
preneurship. There are only two papers, one from the journal Research Policy and the other one from
the journal Entrepreneurship Theory and Practice.
According to this research we can see clearly that the most frequently studied sector in the context of
entrepreneurship is the one that refers to innovation and advanced technology. In the total sample
of 393 papers from the five currently most cited journals in the field of entrepreneurship, there is 114
or 29.01% of papers dealing with this topic. Therefore, this sector is quite explored when it comes to
entrepreneurship, but considering the fact that this field is subject to constant change, it will be vastly
researched in the future and many papers concerning this topic will be published.
On the other hand, there are only two papers covering the topic of women's entrepreneurship in our
sample. Therefore, this area is still on the sidelines of entrepreneurship research. Women's entrepreneur-
ship as a research topic is quite attractive and there should be a lot of research done considering that
the role of women both in entrepreneurship and in the economy in general is always a popular topic.
The position of women in the economy is an issue that is still insufficiently explored, and which should
be studied furthermore since authors constantly strive to promote and maintain gender equality in all
fields of life, including social and economic, and also in the field of entrepreneurship. Furthermore,
according to the results of research (8.91% of the sample), it can be concluded that the papers in the
field of funding risky entrepreneurial ventures from alternative or new sources of financing are insuf-
ficiently covered topics that deserve more research attention.
131
EJAE 2019  16 (2)  122-138
CONCLUSIONS
After the survey, the author formed a sample of 393 papers from five currently most cited journals.
The collected papers were published from 2013 until the beginning of 2018. Due to limited access to
relevant databases of indexed journals, our sample contains 393 papers that the author has been able
to collect during the survey. The papers were selected randomly, so that the author could not influence
the results of the research.
According to the research results, it is concluded that the journal Research Policy, which also ranks
first in the citation, was the most approachable. That is, the author managed to collect 123 papers from
this journal which makes up 31.30% of the sample. The majority of papers that the author collected
from this journal were about innovation and advanced technologies (55 papers), and the least presented
papers dealt with the topics of family business and women's entrepreneurship (1 paper per each field)
while the ratio, in these fields, is the same in the total sample. Most of the papers relate to the field of
innovation and advanced technology and the field of women’s entrepreneurship has the fewest papers.
The worst approach was in the case of the Journal of Product Innovation Management, from which
the author was able to get 49 papers, whereas only 5 more paper (54) was the author able to take over
from the journal Small Business Economics.
According to the results of the survey, the most frequently studied entrepreneurship area is the one that
refers to innovation and advanced technologies. The majority of the papers concerned this topic (114 of
393). Therefore, this area occupies 29.01% of the total sample. The author believe that this field will continue
to be attractive because there are always interesting topics for researchers in this field of entrepreneurship.
The aim of this research was to find an area that is not explored sufficiently enough in the context
of entrepreneurship. The author believes that this goal was achieved.
According to the survey the author came to the conclusion that female entrepreneurship and
financing of high-risk entrepreneurial ventures are the topics that are insufficiently explored in en-
trepreneurship in general. In the total sample of 393 papers there are only two papers dealing with
female entrepreneurship. They make up only 0.51% of the sample. Female entrepreneurship is a very
popular topic these days when women are more and more encouraged to get involved in all economic
trends and in entrepreneurship as well. The author thinks that there is a lot that needs to be done to
empower women's entrepreneurship, particularly in transition economies where the role of women in
the economy is still marginalized.
The author leaves future researchers with so many open and unanswered questions, such as: the
role of women in transition economies, ways to overcome the traditional problems in these economies
when it comes to women, the future and the perspective of women's entrepreneurship, ways to acquire
certain knowledge and skills to start a business and many other questions. The answers to these questions
should be applied not only in theory but in practice too. Such researches should help women to dare
to start a business, because there are plenty of ideas but there is not enough courage to realize them.
132
EJAE 2019  16 (2)  122-138
REFERENCES
Abreu, M., & Grinevich, V. (2013). The nature of academic entrepreneurship in the UK: Widening the focus on
entrepreneurial activities. Research Policy, 42(2), 408-422.
Agarwal, R., & Shah, S. K. (2014). Knowledge sources of entrepreneurship: Firm formation by academic, user
and employee innovators. Research Policy, 43(7), 1109-1133.
Audretsch, D. B., Coad, A., & Segarra, A. (2014). Firm growth and innovation. Small business economics, 43(4),
743-749.
Autant-Bernard, C., Fadairo, M., & Massard, N. (2013). Knowledge diffusion and innovation policies within
the European regions: Challenges based on recent empirical evidence. Research Policy, 42(1), 196-210.
Bar-Ilan, J. (2008). Which h-index?—A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2),
257-271.
Batabyal, A. A., & Beladi, H. (2015). Knowledge goods, ordinary goods, and the effects of trade between leading
and lagging regions. Research Policy, 44(8), 1537-1542.
Belderbos, R., Jacob, J., & Lokshin, B. (2018). Corporate venture capital (CVC) investments and technological
performance: Geographic diversity and the interplay with technology alliances. Journal of Business Ven-
turing, 33(1), 20-34.
Bergmark, D. (2000). Automatic extraction of reference linking information from onlinedocuments. Cornell Uni-
versity.
Bertin, M., & Atanassova, I. (2014). A study of lexical distribution in citation contexts through the IMRaD stand-
ard. PloS Negl. Trop. Dis, 1(200,920), 83-402.
Besagni, D. (2004, January). Citation recognition for scientific publications in digital libraries. In null (p. 244). IEEE.
Block, J. H., Fisch, C. O., Hahn, A., & Sandner, P. G. (2015). Why do SMEs file trademarks? Insights from firms
in innovative industries. Research Policy, 44(10), 1915-1930.
Bornmann, L., Butz, A., & Wohlrabe, K. (2018). What are the top five journals in economics? A new meta-ranking.
Applied Economics, 50(6), 659-675.
Bosman, J., Mourik, I. V., Rasch, M., Sieverts, E., & Verhoeff, H. (2006). Scopus reviewed and compared: The
coverage and functionality of the citation database Scopus, including comparisons with Web of Science
and Google Scholar.
Brannon, D. L., Wiklund, J., & Haynie, J. M. (2013). The varying effects of family relationships in entrepreneurial
teams. Entrepreneurship Theory and Practice, 37(1), 107-132.
Braunerhjelm, P., Ding, D., & Thulin, P. (2018). The knowledge spillover theory of intrapreneurship. Small busi-
ness economics, 51(1), 1-30.
Callaway, E. (2016). Publishing elite turns against impact factor. Nature, 535(7611), 210-211.
Cronin, B., & Sugimoto, C. R. (Eds.). (2015). Scholarly metrics under the microscope: from citation analysis to
academic auditing (pp. 933-940). Association for Information Science and Technology by Information
Today, Incorporated.
Cruz, C., Larraza–Kintana, M., Garcés–Galdeano, L., & Berrone, P. (2014). Are family firms really more socially
responsible?. Entrepreneurship Theory and Practice, 38(6), 1295-1316.
Dai, L., Maksimov, V., Gilbert, B. A., & Fernhaber, S. A. (2014). Entrepreneurial orientation and international
scope: The differential roles of innovativeness, proactiveness, and risk-taking. Journal of Business Ventur-
ing, 29(4), 511-524.
De Massis, A., Audretsch, D., Uhlaner, L., & Kammerlander, N. (2018). Innovation with Limited Resources:
Management Lessons from the G erman M ittelstand. Journal of Product Innovation Management, 35(1),
125-146.
133
EJAE 2019  16 (2)  122-138
de Moya-Anegón, F., Chinchilla-Rodríguez, Z., Vargas-Quesada, B., Corera-Álvarez, E., Muñoz-Fernández, F.

J., González-Molina, A., & Herrero-Solana, V. (2007). Coverage analysis of Scopus: A journal metric ap-
proach. Scientometrics, 73(1), 53-78.
Dorner, M., Fryges, H., & Schopen, K. (2017). Wages in high-tech start-ups–Do academic spin-offs pay a wage
premium?. Research policy, 46(1), 1-18.
Eddleston, K. A., Ladge, J. J., Mitteness, C., & Balachandra, L. (2016). Do you see what I see? Signaling effects
of gender and firm characteristics on financing entrepreneurial ventures. Entrepreneurship Theory and
Practice, 40(3), 489-514.
Elliott, D. B. (2014). The impact factor: a useful indicator of journal quality or fatally flawed?. Ophthalmic and
Physiological Optics, 34(1), 4-7.
Erfanmanesh, M. A., Didegah, F., & Omidvar, S. (2017). Research productivity and impact of Library and Infor-
mation Science in the Web of Science. Malaysian Journal of Library & Information Science, 15(3), 85-95.
Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, web of sci-
ence, and Google scholar: strengths and weaknesses. The FASEB journal, 22(2), 338-342.
Ferreira, M. P., Reis, N. R., & Miranda, R. (2015). Thirty years of entrepreneurship research published in top jour-
nals: analysis of citations, co-citations and themes. Journal of Global Entrepreneurship Research, 5(1), 17.
Frederiksen, L., Wennberg, K., & Balachandran, C. (2016). Mobility and Entrepreneurship: Evaluating the Scope
of Knowledge–Based Theories of Entrepreneurship. Entrepreneurship Theory and Practice, 40(2), 359-380.
Freitas, I. M. B., Marques, R. A., & e Silva, E. M. D. P. (2013). University–industry collaboration and innovation
in emergent and mature industries in new industrialized countries. Research Policy, 42(2), 443-453.
Garfield, E. (1965, December). Can citation indexing be automated. In Statistical association methods for mecha-
nized documentation, symposium proceedings (Vol. 269, pp. 189-192). Washington, DC: National Bureau
of Standards, Miscellaneous Publication 269.
Garfield, E. (1999). Journal impact factor: a brief review. Canadian Medical Association Journal, 161(8), 979-980.
Garfield, E. (2006). Citation indexes for science. A new dimension in documentation through association of ideas.
International journal of epidemiology, 35(5), 1123-1127.
Guerrero, M., Urbano, D., Fayolle, A., Klofsten, M., & Mian, S. (2016). Entrepreneurial universities: emerging
models in the new social and economic landscape. Small Business Economics, 47(3), 551-563.
Hashi, I., & Stojčić, N. (2013). The impact of innovation activities on firm performance using a multi-stage model:
Evidence from the Community Innovation Survey 4. Research Policy, 42(2), 353-366.
Huber, F. (2013). Knowledge-sourcing of R&D workers in different job positions: Contextualising external per-
sonal knowledge networks. Research Policy, 42(1), 167-179.
Hung, S. C., & Tu, M. F. (2014). Is small actually big? The chaos of technological change. Research Policy, 43(7),
1227-1238.
Hunt, J., Garant, J. P., Herman, H., & Munroe, D. J. (2013). Why are women underrepresented amongst paten-
tees?. Research Policy, 42(4), 831-843.
Jacso, P. (2005). As we may search—comparison of major features of the Web of Science, Scopus, and Google
Scholar citation-based and citation-enhanced databases. Current science, 89(9), 1537-1547.
Jacso, P. (2010). The impact of Eugene Garfield through the prism of Web of Science.
Jaskiewicz, P., Block, J. H., Combs, J. G., & Miller, D. (2017). The effects of founder and family ownership on hired
CEOs’ incentives and firm performance. Entrepreneurship Theory and Practice, 41(1), 73-103.
Jemec, G. B. (2001). Impact factor to assess academic output. The Lancet, 358(9290), 1373.
Jung, H. J. (2014). The impacts of science and technology policy interventions on university research: Evidence
from the US National Nanotechnology Initiative. Research policy, 43(1), 74-91.
Kalaitzidakis, P., Mamuneas, T. P., & Stengos, T. (2003). Rankings of academic journals and institutions in eco-
nomics. Journal of the european economic association, 1(6), 1346-1366.
134
EJAE 2019  16 (2)  122-138
Khiste, G. P., & Paithankar, R. R. (2017). Analysis of Bibliometric term in Scopus. International Journal of Library
Science and Information Management (IJLSIM), 3(3), 81-88.
Kóczy, L. A., & Strobel, M. (2007). The ranking of economics journals by a tournament method. Mimeographed,
University of Maastricht.
Kodrzycki, Y. K., & Yu, P. (2006). New approaches to ranking economics journals. The BE Journal of Economic
Analysis & Policy, 5(1).
Krausman, P. R., Cox, A. S., & Knipps, A. C. (2016). Abstract abstracts and other abstractions. The Journal of
Wildlife Management, 80(6), 955-956.
Kronick, D. A. (1976). A history of scientific and technical periodicals: the origins and development of the sci-
entific and technological press, 1665-1790.
Le Breton–Miller, I., & Miller, D. (2015). Article Commentary: The Arts and Family Business: Linking Family
Business Resources and Performance to Industry Characteristics. Entrepreneurship Theory and Practice,
39(6), 1349-1370.
Leyden, D. P., Link, A. N., & Siegel, D. S. (2014). A theoretical analysis of the role of social networks in entrepre-
neurship. Research Policy, 43(7), 1157-1163.
Liner, G. H., & Amin, M. (2004). Methods of ranking economics journals. Atlantic Economic Journal, 32(2),
140-149.
López‐Fernández, M. C., Serrano‐Bedia, A. M., & Pérez‐Pérez, M. (2016). Entrepreneurship and family firm re-
search: A bibliometric analysis of an emerging field. Journal of Small Business Management, 54(2), 622-639.
Lubrano, M., Bauwens, L., Kirman, A., & Protopopescu, C. (2003). Ranking economics departments in Europe:
a statistical approach. Journal of the European Economic Association, 1(6), 1367-1401.
Malički, M. (2016). POUZDANOST ZNANSTVENIH PUBLIKACIJA U BIOMEDICINI (Doctoral dissertation,
University of Split. School of Medicine.).
Marvel, M. R. (2013). Human Capital and Search–Based Discovery: A Study of High–Tech Entrepreneurship.
Entrepreneurship theory and practice, 37(2), 403-419.
McCaffrey, M. (2014). On the theory of entrepreneurial incentives and alertness. Entrepreneurship Theory and
Practice, 38(4), 891-911.
McCloskey, D. N. (2013). Tunzelmann, Schumpeter, and the hockey stick. Research Policy, 42(10), 1706-1715.
McDonald, S., Gan, B. C., Fraser, S. S., Oke, A., & Anderson, A. R. (2015). A review of research methods in en-
trepreneurship 1985-2013. International Journal of Entrepreneurial Behavior & Research, 21(3), 291-315.
Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of
Science versus Scopus and Google Scholar. Journal of the american society for information science and
technology, 58(13), 2105-2125.
Memili, E., Fang, H., Chrisman, J. J., & De Massis, A. (2015). The impact of small-and medium-sized family firms
on economic growth. Small Business Economics, 45(4), 771-785.
Mingers, J., & Harzing, A. W. (2007). Ranking journals in business and management: a statistical analysis of the
Harzing data set. European Journal of Information Systems, 16(4), 303-316.
Mowery, D. C., & Ziedonis, A. A. (2015). Markets versus spillovers in outflows of university research. Research
Policy, 44(1), 50-66.
Muscio, A., Quaglione, D., & Ramaciotti, L. (2016). The effects of university rules on spinoff creation: The case
of academia in Italy. Research Policy, 45(7), 1386-1396.
Musteen, M., & Ahsan, M. (2013). Beyond cost: the role of intellectual capital in offshoring and innovation in
young firms. Entrepreneurship Theory and Practice, 37(2), 421-434.
Musteen, M., Datta, D. K., & Butts, M. M. (2014). Do International Networks and Foreign Market Knowledge
Facilitate SME Internationalization? Evidence From the C zech R epublic. Entrepreneurship theory and
practice, 38(4), 749-774.
135
EJAE 2019  16 (2)  122-138
Nair, P. R., & Nair, V. D. (2014). Organization of a Research Paper: The IMRAD Format. In Scientific Writing
and Communication in Agriculture and Natural Resources (pp. 13-25). Springer, Cham.
Nambisan, S. (2017). Digital entrepreneurship: Toward a digital technology perspective of entrepreneurship.
Entrepreneurship Theory and Practice, 41(6), 1029-1055.
Olmos-Peñuela, J., Castro-Martínez, E., & D’Este, P. (2014). Knowledge transfer activities in social sciences and
humanities: Explaining the interactions of research groups with non-academic agents. Research Policy,
43(4), 696-706.
Oosthuizen, J. C., & Fenton, J. E. (2014). Alternatives to the impact factor. the surgeon, 12(5), 239-243.
Papakostidis, C., & Giannoudis, P. V. (2018). Impact Factor and Altmetrics: What Is the Future?. In Medical
Writing and Research Methodology for the Orthopaedic Surgeon (pp. 71-79). Springer, Cham.
Parker, S. C. (2016). Family firms and the “willing successor” problem. Entrepreneurship Theory and Practice,
40(6), 1241-1259.
Parrilli, M. D., & Heras, H. A. (2016). STI and DUI innovation modes: Scientific-technological and context-specific
nuances. Research Policy, 45(4), 747-756.
Plomp, R. (1994). The highly cited papers of professors as an indicator of a research group's scientific performance.
Scientometrics, 29(3), 377-393.
Pravilnik o publikovanju naučnih publikacija u Republici Srpskoj. (2017). Službeni glasnik RS, br. 77/17. 14-23
Radek, I. (2016). I. MEĐUNARODNA KONFERENCIJA UDRUGE NASTAVNIKA JEZIKA STRUKE NA
VISOKOŠKOLSKIM USTANOVAMA" OD TEORIJE DO PRAKSE U JEZIKU STRUKE”. Strani jezici:
časopis za primijenjenu lingvistiku, 44(1), 70-73.
Rey-Martí, A., Ribeiro-Soriano, D., & Palacios-Marqués, D. (2016). A bibliometric analysis of social entrepre-
neurship. Journal of Business Research, 69(5), 1651-1655.
Ritzberger, K. (2008). A ranking of journals in economics and related fields. German Economic Review, 9(4),
402-430.
Roper, S., Love, J. H., & Bonner, K. (2017). Firms’ knowledge search and local knowledge externalities in innova-
tion performance. Research Policy, 46(1), 43-56.
Rushforth, A., & de Rijcke, S. (2015). Accounting for impact? The journal impact factor and the making of bio-
medical research in the Netherlands. Minerva, 53(2), 117-139.
Sahut, J. M., & Peris-Ortiz, M. (2014). Small business, innovation, and entrepreneurship. Small Business Econom-
ics, 42(4), 663-668.
Sarooghi, H., Libaers, D., & Burkemper, A. (2015). Examining the relationship between creativity and innova-
tion: A meta-analysis of organizational, cultural, and environmental factors. Journal of business venturing,
30(5), 714-731.
Schmid, T., Achleitner, A. K., Ampenberger, M., & Kaserer, C. (2014). Family firms and R&D
behavior–New evidence from a large-scale survey. Research Policy, 43(1), 233-244.
Slater, S. F., Mohr, J. J., & Sengupta, S. (2014). Radical product innovation capability: Literature
review, synthesis, and illustrative research propositions. Journal of Product Innovation Management,
31(3), 552-566.
Sombatsompop, N., Markpin, T., & Premkamolnetr, N. (2004). A modified method for calculating
the Impact Factors of journals in ISI Journal Citation Reports: Polymer Science Category in 1997–2001.
Scientometrics, 60(2), 217-235.
Subochev, A., Aleskerov, F., & Pislyakov, V. (2018). Ranking journals using social choice theory
methods: A novel approach in bibliometrics. Journal of Informetrics, 12(2), 416-429.
Thomä, J., & Bizer, K. (2013). To protect or not to protect? Modes of appropriability in the small
enterprise sector. Research Policy, 42(1), 35-49.
136
EJAE 2019  16 (2)  122-138
Venturini, F. (2015). The modern drivers of productivity. Research Policy, 44(2), 357-369.
Verma, I. M. (2015). Impact, not impact factor.
Visnjic, I., Wiengarten, F., & Neely, A. (2016). Only the brave: Product innovation, service business model in-
novation, and their impact on performance. Journal of Product Innovation Management, 33(1), 36-52.
Vučković-Dekić, L. (2014). Kako se piše naučno/stručni rad. Биомедицинска истраживања, 5(1).
Walsh, J. P., Lee, Y. N., & Nagaoka, S. (2016). Openness and innovation in the US: Collaboration form, idea
generation and implementation. Research Policy, 45(8), 1660-1671.
Walter, S. G., & Block, J. H. (2016). Outcomes of entrepreneurship education: An institutional perspective. Journal
of Business Venturing, 31(2), 216-233.
Walter, S. G., Parboteeah, K. P., & Walter, A. (2013). University Departments and Self–Employment Intentions
of Business Students: A Cross–Level Analysis. Entrepreneurship Theory and Practice, 37(2), 175-200.
Werfel, S. H., & Jaffe, A. B. (2013). Induced innovation and technology trajectory: Evidence from smoking ces-
sation products. Research Policy, 42(1), 15-22.
Wilson, N., Wright, M., & Scholes, L. (2013). Family business survival and the role of boards. Entrepreneurship
Theory and Practice, 37(6), 1369-1389.
Wohlrabe, K. (2016). Taking the temperature: A meta-ranking of economics journals.
Xu, N., Chen, Y., Fung, A., & Chan, K. C. (2018). Contributing forces in entrepreneurship research: A global
citation analysis. Journal of Small Business Management, 56(1), 179-201.
Zahra, S. A. (2015). Corporate entrepreneurship as knowledge creation and conversion: The role of entrepreneurial
hubs. Small Business Economics, 44(4), 727-735.
Zhang, S. X., & Cueto, J. (2017). The study of bias in entrepreneurship. Entrepreneurship Theory and Practice,
41(3), 419-454.
137
EJAE 2019  16 (2)  122-138
ŠTA MOŽEMO OČEKIVATI U BUDUĆNOSTI AKADEMSKOG

ISTRAŽIVANJA? NAJČEŠĆI ISTRAŽIVAČKI PROBLEMI ANALIZIRANI U
OKVIRU NAJISTAKNUTIJIH ČASOPISA IZ OBLASTI PREDUZETNIŠTVA
Rezime:
U modernoj eri akademskog istraživanja, izuzetno je teško definisati
celokupno ili nedovoljno ispitan istraživački problem. Svaki istraživač
se u svom istraživačkom radu suočava sa ovim pitanjem. Potrebno je
mnogo rada i truda da bi se pronašla tema koja je relevantna i koja
će biti privlačna za izučavanje i istraživanje. U središtu ovog rada su
istraživački problemi iz oblasti preduzetništva. Rad se bavi istraživačkim
problemima koji su se najčešće javljali u oblasti preduzetništva tokom
proteklih pet godina. Sprovedena je anketa, sa ciljem da se identifikuju
najmanje ispitane oblasti preduzetništva, kako bi se mogle predvideti
buduće teme za istraživanje. Metode koje su se primenjivale, da bi se
ovaj akademski cilj postigao, uključuju: opis, klasifikaciju i objašnjenje.
Analizom uzorka, koji obuhvata 393 rada iz pet najuticajnijih časopisa
koji se bave temom preduzetništva, autor je došao do zaključka da se,
tokom proteklih pet godina, većina radova prvenstveno bavila inova-
cijama i naprednom tehnologijom, te da se jedva dotakla teme ženskog Ključne reči:
preduzetništva. Ovaj rad bi trebalo da pomogne budućim istraživačima preduzetništvo, referentni časopisi,
da odaberu teme ili oblasti istraživanja u domenu preduzetništva. citati
138
CAUSES OF FAILURE OF THE PHILLIPS CURVE: DOES TRANQUILLITY OF

ECONOMIC ENVIRONMENT MATTER? 1
Yhlas Sovbetov, Muhittin Kaplan
Istanbul University, Turkey

Although empirical literature regarding the Phillips curve is sizeable
Received: May 06, 2019
enough, there is still no wide consensus on its validity and stability.
Correction: June 12, 2019
The literature shows that the Phillips relationship is fragile and varies
Accepted: June 18, 2019
across countries and time periods; a statistical relationship that appears
strong during one decade (country) may be weak the next (other). This
variability might have some grounds for idiosyncrasy of a country
and its economic environment. To address it, this paper scrutinizes
the Phillips relationship over 41 countries over the period 1980-2016,
paying attention to how inflation dynamics behave during tranquil and
recessionary periods. As a result, the paper confirms the variability of
the Phillips relationship across countries, as well as time periods. It
documents that the relationship holds in the majority of developed
countries, while it fails to hold in emerging and frontier economies
during tranquil periods. On the other hand, the relationship totally
collapses during recessionary periods, even in developed markets.
This shows that tranquillity of economic environment is significantly
important for the Phillip trade-off to work smoothly. Moreover, both
backward- and forward-looking fractions of inflation remarkably increase
during recessionary periods as a result of the Phillips coefficient loses
its significance within the model. This indicates that markets become
more inflation-sensitive during these periods.
Keywords:
The Phillips Curve, inflation,
recession, tranquility
1 This research article was generated from PhD dissertations submitted by the corresponding author to Institute of Social
Sciences, Istanbul University
139
*E-mail: yhlas.sovbetov@ogr.iu.edu.tr
EJAE 2019  16 (2)  139-154
SOVBETOV, T., KAPLAN, M.  CAUSES OF FAILURE OF THE PHILLIPS CURVE: DOES TRANQUILLITY OF ECONOMIC ENVIRONMENT MATTER?
INTRODUCTION
The empirical study of Alban William Phillips (1958) on change in wage of inflation and fluctua-
tions in unemployment has greatly influenced the course of macroeconomics. He discovered a strong
negative trade-off relationship between unemployment and inflation in the UK over the 1861-1957
period, which is today known as the “Phillips Curve”. Despite some early criticisms of the basic tenets
of the Phillips Curve (Samuelson and Solow, 1960; Phelps, 1967; Friedman, 1968; Lucas, 1976), the
hypothesis remains one of the most important foundations for macroeconomics. However, after 2008
Great Recession, many studies have challenged the validity of the Phillips curve (Ball and Mazumder,
2011; Russel and Banerjee, 2008; Paul, 2009; Bernanke, 2010; Karan Singh et al, 2011; Hall, 2011; Daly
et al., 2012; Ojapinwa and Esan, 2013; Nub, 2013; Wimanda et al, 2013; Simionescu, 2014; Coibion and
Gorodnichenko, 2015; Friedrich, 2016; IMF, 2013; Doser et al., 2017, Sovbetov and Kaplan, 2019), when
the unemployment rate rapidly scaled up, but inflation did not decline as much as the curve predicted
it would2 . They also underline the variability of the Phillips relationship across countries.
Russel and Banerjee (2008), Paul (2009), Fendel et al. (2011), Ojapinwa and Esan (2013), Nub (2013),
Simionescu (2014), Coibion and Gorodnichenko (2015), and Murphy (2018) investigate nonlinearities
in the Phillips Curve caused by external factors. This is quite important as any significant changes in be-
haviour of inflation and unemployment during recessionary periods might also be one of the reasons for
the failure of the relationship. However, the mentioned studies do not focus on behaviour of the Phillips
Curve during recessionary periods. To our best knowledge, the extant literature lacks empirical research
that examines behavioural changes of the Phillips relationship during recessionary and tranquil periods.
To fill this gap, this research examines the Phillips curve with an up-to-date data over the 1980-2016
period, focusing on tranquil and recessionary periods separately. In order to addresses the variability
of the Phillips relationship, this examination has been carried out across 41 different countries from
developed, emerging, and frontier markets. For robustness, the research also considers both backward-
and forward-looking versions of expectation-augmented Phillips model (EAPC). The study mainly
pursues answers for two questions: (1) is the Phillips relationship still valid? (2) Is there any significant
change in the Phillips relationship during recessionary and tranquil periods?
The rest of the paper is structured in the following order: the second section briefly reviews the for-
mulation of the Phillips curve and covers the main causes behind its failure during the Great Recession
in 2008. The third section describes the data and specifies the methodology for this study. The fourth
section presents the findings and interprets them thoroughly, while the final section provides conclusions.
LITERATURE REVIEW
The Great Recession in 2008 has rekindled interest in the Phillips curve with a particular focus on causes of
failure of the empirical relationship between inflation and unemployment, and on “missing disinflation”. Bernanke
(2010) argues that the main causes of the absence of disinflation are well-anchored expectations of households,
which were imposed by highly credible Central banks for a long time. His “anchored expectations” hypothesis basi-
cally states that the credibility of modern central banks has convinced people for a long-run stability in inflation.
However, this hypothesis would work only for countries where these two conditions are valid: (1) The country’s
central bank is highly credible; (2) the impact of external shocks is ineffective to households’ budgets. If one of these
two conditions fails, then the “anchored expectations” hypothesis might generate more significant disinflations.
2 This phenomenon is often referred to as “missing disinflation”.
140
EJAE 2019  16 (2)  139-154
Ball and Mazumder (2011, 2015) also support Bernanke’s hypothesis. They attempt to explain the
failure of the Phillips Curve in the U.S. throughout 2007-2010 by assuming that inflation expectations are
fully anchored at the Federal Reserve’s target, and that the level of short-term unemployment captures
the labour-market slack. However, Coibion and Gorodnichenko (2015) argue that the “anchored expecta-
tions” hypothesis was not case, as oil prices were very undulant during 2007-2009 periods. According to
West Texas Intermediate (WTI or NYMEX) data, the crude oil prices per barrel was 72 USD in January
2007, when it skyrocketed to historically high record levels of 161 USD within just a year, in June 2008.
During the following six months, it follows a decreasing trend until the prices drop to minimum level
of 49 USD in January 2009. Afterwards, it once again peaks in June 2009, at 81 USD level. Coibion and
Gorodnichenko (2015) argue that the households’ expectations are scaled up during the Great Reces-
sion in response to sharp increases in oil prices. The households are more responsive to oil prices when
they form their future inflation expectations when comparing to professional analysts. The survey-based
measures of their expectations reflect changes in the price of their own consumption bundles rather than
the overall inflation in the economy. Since gasoline remains as a large portion of the consumption of their
income, they adjust their position according to the oil price changes. Therefore, the increased household
expectations hindered the downward adjustment of prices, and caused divergence between future infla-
tion expectations of the households and professional analysts. The authors believe it was the key reason
behind the absence of disinflation during this period, and they advise analysts to use a better proxy for
expected inflation, referring to the Michigan Survey of Consumers (MSC) dataset.
Doser et al (2017) also find that consumer expectations of inflation played an important role during
the recent missing disinflation, however, they urge that nonlinearities in the Phillips curve are another
reason behind its failure during the Great Recession. They argue that the higher unemployment that
emerged due to recessions might not lead to a sharp decrease in inflation. They add that the original
1970’s Phillips curve was a convex curve, not a linear relationship. Thus, when unemployment is already
high, a further increase in unemployment leads to a smaller disinflation when compared to the case when
unemployment was at its historical average.
On the other hand, Russell and Banerjee (2008) study the NAIRU Phillips curve under non-stationarity
conditions in the series. They argue that the non-stationarity of inflation having a time-varying mean
might be one of the key reasons behind the failure of the Phillips curve. Moreover, a positive relationship
might also emerge between inflation and the unemployment rate in the long-run. Ojapinwa and Esan
(2013) and Simionescu (2014) further observe a weak positive relationship between inflation and unem-
ployment in Nigeria and Romania, respectively. These findings clearly show that the Phillips trade-off
might not always behave as in theory.
Del Boca et al (2010) study the Phillips relationship in Italy and find that the trade-off breaks down
during periods of high inflation and unstable macroeconomic environment. They believe that the com-
parative disadvantage of the Italian economy to withstand adverse supply-side shocks and the poor
quality of monetary policy are key reasons of this failure. Nub (2013) explores the Phillips trade-off in
Germany with an updated data over the period from 1970 to 2012, and fails to detect a significant negative
short-run trade-off. He argues that it happens due to influences of some other factors on the behaviour
of inflation and employment. For instance, European Monetary Union (EMU) policies might be one of
the reasons behind these influences. Although the EMU eliminates inflation bias due to countries' policy
credibility problems (Clerc, Dellas, and Loisel, 2010), it might cause a strong form of nominal rigidities.
Nub (2013) believes that negative spillovers coming from other members of the EU might also cause a
shock to Germany through the EMU.
141
EJAE 2019  16 (2)  139-154
Paul (2009) documents the failure of the Phillips trade-off in India. He addresses the liberalization-
policy of the early 1990s and supply shocks, such as droughts and oil prices as the main reasons behind
this failure. Although he argues that the trade-off might work in the short-run once these shocks are
taken into consideration in the model, he believes that this relationship is often evasive or absent in
less-developed economies. Sovbetov and Kaplan (2019) also point out that in less-developed and crisis-
prone markets the Phillips curve might not work smoothly due to a lack of well-established and freely
operating structure of macroeconomic foundations and motivations, as well as a lack of economic
environment tranquillity.
In respect of the above-mentioned studies, one can infer that shocks in expectations might cause
failure of the Phillips relationship. And sharp changes in expectations tend to occur during an unstable
economic environment (recessionary or post-recessionary recovery periods). Therefore, the tranquillity
of economic environment should matter in terms of stability and validity of the Phillips relationship. In
this regard, this research aims to contribute to the aforementioned field of the Phillips curve literature
by examining the relationship during recessionary and tranquil periods separately over 41 different
countries from developed, emerging, and frontier markets.
DATA AND METHODOLOGY
Following Sovbetov and Kaplan (2019), the base equation of this research is formed as below. This
odel is specified in order to empirically examine the validity of the backward- and forward-looking
Neo-Classical Phillips Curve (NCPC) over Q1:1980-Q1:2016 across 41 countries (Appendix-A) during
tranquil and recessionary economic periods.
πt =
β 0 + β1π te + β 2 (ut − u * ) + β3π te DUMMYµ + β 4 (ut − u * ) DUMMYµ + ε t (1)
where DUMMY is a dummy variable that gets a value of 1 during recessionary periods and zero
during other periods; π t and π t are proxied by the first differences of the logarithm CPI and expected
e
CPI inflation over one year respectively; Ut, and U* are proxied by unemployment rate and NAIRU
respectively in logarithmic form. The β2 and β2+β4 indicate the Phillips coefficients during tranquil and
recessionary periods, respectively. Similarly, β1 and β1+β3 show fractions of πe in the current inflation
during tranquil and recessionary periods respectively. Note that if π t equates to π t −1 then the model
e
converts to NCPC with backward-looking specification. And if it equates to the expected inflation
Et(πt+1), then the model takes the shape of NCPC with forward-looking specification. In addition, the
study uses manual calculations of standard errors β2+β4 and β1+β3 with formulas of
SE=
β 2− β 4 SEβ2 2 + SEβ2 4 and SE=
β 1− β 3 SEβ21 + SEβ2 3 3
in order to find out the
significance of the Phillips coefficients during recessionary periods.
Data for these variables are obtained from Thomson Reuters Eikon Datastream, and a fixed constant
term is added to all series to handle negative values during the transformation into the logarithmic
form, which only shifts β0 up, leaving other variables unaffected.
3 Cov(β2+β4) and Cov(β1+β3) is approximately zero.

142
EJAE 2019  16 (2)  139-154
The expected inflation data also retrieved from the Thomson Reuters Eikon Datastream, where the
data is formed by their own forecasting survey methodology. The complete raw data used in this study
is made available publicly via the following link: https://data.mendeley.com/datasets/8v2mpt7dtp/1,
making the results of this study reproducible.
The definitions of recessionary and tranquil periods is quite important in this study. With very basic
approach, this study refers all non-growing economic periods as "recessionary periods," while remaining
(growing economic periods) as "tranquil periods." Because various aspects of the economy are disrupted
during economic recessions, the study aims to capture all their influences over the Phillips relationship
through changed expectations. Alternatively, the study could also just focus on currency crashes that
might also fairly proxy the shocks during inflation dynamics. It might, however, be an insufficient ap-
proach in some cases in which shocks in inflation fail to visibly influence expectations.
Table 1 provides an overview for the main features of the dataset. The first and second columns of
the table present current and expected inflation of CPI derived from Thomson Eikon Datastream. The
third column is derived by HP filtering 4 the current inflation (first column) with lambda 1600. Com-
paring Thomson's forecasted expected inflation, the HP filter derives more accurate estimation. Thus,
the table presents NAIRU figures in the fifth column that are obtained using HP-filtering methodology.
Table 1. Overview of Mean and Standard Deviation of Variables

Current Inflation Expected Inflation HP InflationTrend UnemploymetRate NAIRU UnemploymentGap
Mean StdDev. Mean StdDev. Mean StdDev. Mean StdDev. Mean StdDev. Mean StdDev.
AG 0.0115 0.0151 0.0197 0.0253 0.0176 0.0283 0.1175 0.0447 0.1169 0.0378 0.0006 0.0191
AU 0.0027 0.0023 0.0031 0.0008 0.0028 0.0005 0.0645 0.0171 0.0644 0.0160 0.0001 0.0043
BD 0.0019 0.0018 0.0022 0.0010 0.0019 0.0009 0.0910 0.0167 0.0905 0.0151 0.0005 0.0056
BG 0.0021 0.0020 0.0022 0.0009 0.0021 0.0005 0.0824 0.0089 0.0817 0.0052 0.0007 0.0054
BR 0.0476 0.1053 0.0867 0.1978 0.0464 0.0808 0.1145 0.0298 0.1147 0.0267 -0.0002 0.0106
CH 0.0045 0.0072 0.0058 0.0055 0.0044 0.0042 0.0368 0.0059 0.0368 0.0057 0.0000 0.0011
CL 0.0056 0.0052 0.0061 0.0043 0.0057 0.0040 0.0866 0.0196 0.0865 0.0157 0.0001 0.0100
CN 0.0019 0.0018 0.0023 0.0008 0.0020 0.0005 0.0780 0.0136 0.0779 0.0116 0.0001 0.0051
CZ 0.0054 0.0076 0.0068 0.0076 0.0054 0.0045 0.0545 0.0187 0.0545 0.0168 0.0000 0.0061
DK 0.0020 0.0019 0.0024 0.0007 0.0020 0.0006 0.0662 0.0261 0.0657 0.0224 0.0005 0.0076
ES 0.0029 0.0024 0.0032 0.0016 0.0029 0.0015 0.1754 0.0588 0.1736 0.0523 0.0017 0.0142
FN 0.0017 0.0020 0.0022 0.0010 0.0018 0.0007 0.1000 0.0294 0.0983 0.0247 0.0016 0.0095
FR 0.0016 0.0017 0.0019 0.0007 0.0016 0.0007 0.0899 0.0100 0.0895 0.0078 0.0004 0.0047
GR 0.0048 0.0079 0.0053 0.0049 0.0048 0.0046 0.1466 0.0603 0.1458 0.0556 0.0007 0.0155
HN 0.0108 0.0116 0.0114 0.0092 0.0110 0.0075 0.0773 0.0179 0.0773 0.0152 0.0001 0.0052
ID 0.0101 0.0134 0.0109 0.0121 0.0101 0.0042 0.0668 0.0191 0.0667 0.0174 0.0001 0.0065
IN 0.0077 0.0069 0.0078 0.0025 0.0077 0.0021 0.0824 0.0210 0.0823 0.0209 0.0000 0.0005
IR 0.0022 0.0029 0.0027 0.0016 0.0022 0.0014 0.0902 0.0417 0.0897 0.0366 0.0005 0.0105
IT 0.0026 0.0019 0.0030 0.0016 0.0027 0.0014 0.0957 0.0188 0.0954 0.0170 0.0003 0.0050
JP 0.0003 0.0025 0.0006 0.0010 0.0003 0.0007 0.0414 0.0080 0.0413 0.0070 0.0001 0.0028
4 HP is a technique used to derive long-run levels of variables. The λ is a smoothing parameter that is set by using the Ravn
and Uhliq (2002) frequency rule: the number of periods per year divided by 4, raised to the power of x, and multiplied
by 1600. Hodrick and Prescott (1997) recommend the value 2 for x, whereas Ravn and Uhliq (2002) suggest using 4 for
x. Following Hodrick and Prescott (1997), we derive λ=1600 for our dataset.
143
EJAE 2019  16 (2)  139-154
KO 0.0037 0.0034 0.0049 0.0022 0.0038 0.0016 0.0359 0.0121 0.0359 0.0062 0.0000 0.0087
MX 0.0095 0.0103 0.0105 0.0096 0.0096 0.0067 0.0395 0.0106 0.0392 0.0076 0.0003 0.0061
MY 0.0030 0.0029 0.0042 0.0015 0.0029 0.0007 0.0325 0.0043 0.0324 0.0022 0.0001 0.0033
NL 0.0022 0.0021 0.0024 0.0008 0.0022 0.0006 0.0627 0.0165 0.0620 0.0123 0.0007 0.0074
NW 0.0022 0.0022 0.0024 0.0007 0.0022 0.0003 0.0388 0.0088 0.0389 0.0068 0.0000 0.0039
OE 0.0021 0.0026 0.0023 0.0008 0.0021 0.0007 0.0475 0.0062 0.0475 0.0045 0.0001 0.0036
PH 0.0058 0.0043 0.0069 0.0030 0.0060 0.0023 0.0889 0.0184 0.0890 0.0161 -0.0001 0.0068
PO 0.0088 0.0112 0.0118 0.0147 0.0091 0.0111 0.1410 0.0311 0.1406 0.0216 0.0004 0.0145
PT 0.0032 0.0028 0.0036 0.0027 0.0032 0.0022 0.0831 0.0359 0.0825 0.0337 0.0007 0.0095
RM 0.0337 0.0456 0.0343 0.0449 0.0300 0.0297 0.0736 0.0243 0.0725 0.0193 0.0011 0.0131
RS 0.0481 0.0941 0.0761 0.1456 0.0449 0.0577 0.0777 0.0219 0.0777 0.0187 0.0001 0.0080
SA 0.0072 0.0043 0.0081 0.0029 0.0072 0.0021 0.2334 0.0324 0.2337 0.0278 -0.0004 0.0109
SD 0.0015 0.0028 0.0022 0.0015 0.0016 0.0014 0.0825 0.0191 0.0812 0.0145 0.0013 0.0085
SP 0.0019 0.0028 0.0026 0.0012 0.0019 0.0013 0.0238 0.0074 0.0236 0.0059 0.0002 0.0043
SW 0.0009 0.0026 0.0014 0.0013 0.0010 0.0012 0.0342 0.0090 0.0335 0.0049 0.0006 0.0056
TH 0.0033 0.0042 0.0044 0.0023 0.0033 0.0016 0.0181 0.0113 0.0181 0.0085 0.0000 0.0063
TK 0.0321 0.0292 0.0380 0.0319 0.0322 0.0251 0.0850 0.0190 0.0848 0.0158 0.0002 0.0091
TW 0.0016 0.0041 0.0024 0.0014 0.0016 0.0013 0.0372 0.0119 0.0373 0.0103 0.0000 0.0047
UK 0.0024 0.0029 0.0031 0.0019 0.0024 0.0012 0.0440 0.0204 0.0435 0.0177 0.0005 0.0049
US 0.0025 0.0021 0.0028 0.0007 0.0025 0.0007 0.0601 0.0161 0.0602 0.0122 -0.0001 0.0072
VE 0.0354 0.0265 0.0408 0.0279 0.0358 0.0199 0.1059 0.0335 0.1063 0.0278 -0.0004 0.0143
144
EJAE 2019  16 (2)  139-154
The estimation of the results of the Eq.1 model and the manual computation of coefficients and
standard errors of Phillips coefficients during recessionary periods are displayed at Table 1, where su-
perscripts "N" and "R" indicate estimations for tranquil (normal) and recessionary periods, respectively.
Table 1. Backward/Forward-Looking NCPC Model during Tranquil/ Recessionary Periods

Backward-Looking PC Model Forward-Looking PC Model
Market Countries Obs
Class.
N
πt-1 N
UGAP R
πt-1 R
UGAP C N
Et(πt+1) N
UGAP R
Et(πt+1) R
UGAP C
D Australia 0.5645*** -0.0874* 0.4435*** -0.1620 0.0017*** 0.6755*** -0.0900* 0.7845*** -0.0344 0.0015*** 145
(0.0739) (0.0486) (0.1757) (0.1819) (0.0004) (0.1147) (0.0476) (0.2233) (0.1456) (0.0006)
D Austria 0.4559*** -0.0829* 0.4582*** -0.3037** 0.0012*** 0.5608*** -0.0607** 0.6664*** -0.2564*** 0.0009*** 145
(0.0942) (0.0427) (0.1388) (0.1341) (0.0002) (0.0845) (0.0285) (0.1323) (0.0982) (0.0002)
D Belgium 0.5603*** 0.0184 0.5982*** 0.1200 0.0011*** 0.7012*** -0.0363* 0.7783*** -0.0134 0.0005** 145
(0.0927) (0.0363) (0.1790) (0.0793) (0.0002) (0.0808) (0.0197) (0.0926) (0.0697) (0.0002)
D Canada 0.5248*** -0.0883** 0.5856*** -0.1611 0.0013*** 0.7120*** -0.0672** 0.8435*** -0.0490 0.0002 145
(0.1034) (0.0389) (0.1758) (0.1082) (0.0003) (0.0660) (0.0328) (0.1354) (0.1521) (0.0003)
D Denmark 0.5514*** -0.0342 0.6855*** 0.0196 0.0014*** 0.5598*** -0.0219* 0.7644*** -0.0324 0.0004 145
(0.0811) (0.0247) (0.1688) (0.0472) (0.0003) (0.1166) (0.0113) (0.1783) (0.0416) (0.0004)
D Finland 0.6161*** -0.0154 0.7213*** -0.0588 0.0010*** 0.5473*** -0.0641*** 0.7787*** -0.0648 0.0003 145
(0.0678) (0.0341) (0.1417) (0.0574) (0.0003) (0.0620) (0.0207) (0.1770) (0.0427) (0.0003)
D France 0.5113*** -0.0614* 0.5886*** 0.0760 0.0003* 0.5955*** -0.0444** 0.8046*** 0.0331 -0.0001 145
(0.0488) (0.0366) (0.0913) (0.1682) (0.0001) (0.0355) (0.0214) (0.1194) (0.2915) (0.0002)
D Germany 0.4294** -0.0921* 0.4958** -0.1189 0.0026*** 0.5853*** -0.0341** 0.7944*** -0.0308 0.0006* 145
(0.1832) (0.0516) (0.2037) (0.0768) (0.0006) (0.1047) (0.0178) (0.1535) (0.0409) (0.0004)
D Ireland 0.4929*** -0.0455* 0.5540** -0.0276 0.0007** 0.5154*** -0.0479* 0.6461** -0.0642 0.0008 145
(0.1368) (0.0261) (0.2225) (0.0556) (0.0004) (0.1150) (0.0246) (0.2170) (0.0638) (0.0006)
D Italy 0.5896*** 0.0283* 0.7061*** -0.0476** 0.0004** 0.9769*** -0.0069 0.9924*** -0.0657* 0.0004 145
(0.0496) (0.0145) (0.0599) (0.0220) (0.0002) (0.0606) (0.0256) (0.0913) (0.0369) (0.0003)
D Japan 0.2183** -0.2154* 0.3865** -0.5047** 0.0009** 0.5259*** -0.2260** 1.0723*** -0.4751** 0.0006* 145
(0.1053) (0.1274) (0.1671) (0.2302) (0.0004) (0.1433) (0.1161) (0.4230) (0.2375) (0.0003)
D Netherlands 0.5081*** -0.0349* 0.5503*** -0.0533 0.0010*** 0.5107*** -0.0253* 0.6894*** -0.0033 0.0011*** 145
(0.1009) (0.0206) (0.1319) (0.0382) (0.0002) (0.1738) (0.0149) (0.2071) (0.0482) (0.0004)
D Norway 0.4815*** -0.0993* 0.5603*** -0.4354* 0.0022*** 0.5182*** -0.0264* 0.7156*** -0.3042* 0.0010** 145
(0.0802) (0.0570) (0.1569) (0.2500) (0.0004) (0.0733) (0.0155) (0.1940) (0.1782) (0.0005)
D Portugal 0.5397*** -0.0549* 0.6637*** -0.0748 0.0026*** 0.5512*** -0.0388* 0.6513*** -0.0377 0.0020*** 144
(0.0723) (0.0324) (0.1525) (0.0667) (0.0006) (0.0757) (0.0208) (0.1967) (0.0647) (0.0008)
D Singapore 0.5658*** 0.0212 0.2786 -0.0504 0.0010*** 0.6629*** -0.0033 0.1974 -0.0194 0.0006 145
(0.1048) (0.0252) (0.1860) (0.0540) (0.0003) (0.1117) (0.0234) (0.2682) (0.0434) (0.0004)
D South Korea 0.4615*** -0.0689** 0.5193*** 0.5234 0.0017*** 0.5371*** -0.1028** 0.9613*** 0.1518 0.0025*** 145
(0.0597) (0.0314) (0.0886) (0.6053) (0.0004) (0.1305) (0.0529) (0.2141) (0.381) (0.0007)
D Spain 0.5410*** -0.0345* 0.6498*** -0.0125 0.0021*** 0.6508*** -0.0434* 0.7359*** -0.0096 0.0023*** 145
(0.1072) (0.0193) (0.1734) (0.0419) (0.0007) (0.1097) (0.0257) (0.1794) (0.0493) (0.0007)
D Sweden 0.5611*** -0.0563** 0.6200*** -0.0179 0.0010*** 0.6756*** -0.0610** 0.9789*** -0.0515 0.0006 145
(0.0776) (0.0275) (0.1299) (0.1120) (0.0003) (0.1138) (0.0291) (0.2319) (0.1242) (0.0004)
D Switzerland 0.6162*** -0.0753* 0.5732*** -0.0472 0.0006*** 0.5943*** -0.1453** 0.7924*** 0.0151 0.0005 145
(0.0639) (0.0447) (0.1268) (0.2333) (0.0002) (0.0913) (0.0698) (0.1576) (0.2776) (0.0003)
D United 0.4635*** -0.0470** 0.5018*** -0.1163 0.0004 0.6308*** -0.1885** 0.7848*** -0.0204 0.0016*** 145
Kingdom (0.0490) (0.0235) (0.1514) (0.1295) (0.0003) (0.0680) (0.0822) (0.1744) (0.1275) (0.0005)
D United 0.4019*** -0.0921*** 0.4078*** -0.0770 0.0018*** 0.7431*** -0.0746** 0.8020*** -0.0952 0.0007 145
States (0.1174) (0.0291) (0.1253) (0.0637) (0.0004) (0.1378) (0.0297) (0.1989) (0.0731) (0.0005)
Notes: Numbers in the table are coefficient estimates with HAC standard errors in parentheses. The *, **, and *** denote si-
gnificance at 10%, 5%, and 1% levels respectively. The market classification is in S&P standards, and the superscripts "N"
and "R" indicate estimations for normal (tranquil) and recessionary periods respective.
145
EJAE 2019  16 (2)  139-154
Table 1. continued
Backward-Looking PC Model Forward-Looking PC Model
Countries Obs
Market N
πt-1 N
UGAP R
πt-1 R
UGAP C N
Et(πt+1) N
UGAP R
Et(πt+1) R
UGAP C
Class.
E Brazil 0.6328*** 0.2578 0.7273*** -0.2465 0.0052*** 0.4545*** 0.6443 0.5691*** 0.0413 0.0130** 106
(0.1395) (0.3374) (0.2099) (0.6209) (0.0019) (0.0938) (0.5081) (0.1420) (1.1816) (0.0052)
E Chile 0.7207*** -0.0889** 0.7888*** -0.0273 0.0020*** 0.9057*** -0.0396 0.9235*** -0.0797 0.0005 121
(0.0955) (0.0391) (0.1630) (0.1157) (0.0006) (0.0484) (0.0280) (0.1086) (0.1388) (0.0005)
E China 0.6191*** -0.0248* 0.6318*** -0.0110 0.0011*** 0.5816*** -0.0452* 0.7683*** 0.0189 0.0004 145
(0.0624) (0.0144) (0.1178) (0.0607) (0.0003) (0.1094) (0.0260) (0.1497) (0.0958) (0.0005)
E Czech 0.5375*** 0.0503 0.6318** -0.2434 0.0015** 0.5118*** -0.0404 0.7479** -0.1972 -0.0004 93
(0.2952) (0.1047) (0.3166) (0.2155) (0.0007) (0.2952) (0.0839) (0.3762) (0.1752) (0.0009)
E Greece 0.9730*** 0.0002 0.8208*** -0.0186 0.0008*** 0.9324*** -0.0104 0.2145 -0.0226 0.0036*** 145
(0.0301) (0.0259) (0.0721) (0.0424) (0.0003) (0.1401) (0.0526) (0.1760) (0.1198) (0.0013)
E Hungary 0.4646* -0.2417 0.6549** 0.0656 0.0052* 0.5314*** -0.4268 0.6037*** 0.1812 0.0018 101
(0.2752) (0.2759) (0.3236) (0.4856) (0.0028) (0.1082) (0.3061) (0.1207) (0.4789) (0.0019)
E India 0.5483*** -0.0390 0.5807*** -0.0797 0.0055*** 0.4984*** -0.0896** 0.6857*** -0.0136 0.0068*** 145
(0.1116) (0.0249) (0.1350) (0.0488) (0.0008) (0.1071) (0.0444) (0.1477) (0.1004) (0.0008)
E Indonesia 0.5892*** -0.0284 0.6152*** 0.1966 0.0060*** 0.5592*** -0.0013 0.6401*** -0.0192 0.0063*** 145
(0.0474) (0.0732) (0.0932) (0.7892) (0.0007) (0.0551) (0.0631) (0.1434) (0.9998) (0.0009)
E Malaysia 0.5217*** -0.0391** -0.0311 -0.0479 0.0011* 0.5452*** -0.0626** 0.0743 -0.0495 0.0007 125
(0.1521) (0.0182) (0.4490) (0.0815) (0.0006) (0.1602) (0.0263) (0.3487) (0.0835) (0.0005)
E Mexico 0.6102*** 0.2117 0.7227*** 0.7032 0.0064*** 0.7361*** 0.1043 0.9072*** 0.3224 0.0043** 145
(0.0894) (0.2023) (0.1217) (0.5722) (0.0021) (0.0852) (0.1543) (0.1192) (0.6259) (0.0018)
E Philippines 0.6119*** -0.0570** 0.6645*** -0.2853** 0.0030*** 0.4902*** -0.0516** 0.7646*** -0.2211** 0.0043*** 145
(0.0392) (0.0257) (0.2410) (0.1173) (0.0005) (0.0750) (0.0256) (0.1833) (0.1025) (0.0008)
E Poland 0.6309** -0.2459 0.7186** 0.1151 0.0065* 0.4408*** -0.1187 0.6725*** -1.5115 0.0023 109
(0.2698) (0.2949) (0.3555) (0.4710) (0.0035) (0.0967) (0.1273) (0.1188) (1.3253) (0.0028)
E Russia 0.5994** 0.2381 0.6143** -2.2745 0.0246* 0.4576** 0.1887 0.6822** -2.7840 0.0225** 100
(0.2596) (0.4535) (0.2697) (2.8064) (0.0145) (0.2128) (0.4116) (0.3338) (3.5023) (0.0106)
E SouthAftica 0.5832*** -0.0270 0.6988*** 0.0934 0.0027*** 0.5118*** -0.0350 0.9690*** 0.0988* 0.0017* 145
(0.0549) (0.0254) (0.0869) (0.0625) (0.0005) (0.0942) (0.0251) (0.1094) (0.0538) (0.0009)
E Taiwan 0.5311** -0.2679*** 0.5895** -0.2745 0.0020*** 0.5265** -0.2569*** 0.6322*** -0.1971 0.0015*** 145
(0.2633) (0.0649) (0.2446) (0.1709) (0.0004) (0.1946) (0.0858) (0.2383) (0.2506) (0.0005)
E Thailand 0.4898*** -0.0464 0.5730*** -0.0638 0.0017*** 0.5716*** -0.0837 0.7290*** -0.0924 0.0014** 144
(0.1302) (0.0358) (0.1682) (0.0809) (0.0004) (0.1475) (0.0525) (0.1950) (0.0938) (0.0005)
F Turkey 0.6434*** 0.0346 0.7029*** -0.0884 0.0077*** 0.6954*** -0.0712 0.8248*** -0.0219 0.0056*** 145
(0.0915) (0.1650) (0.1482) (0.3256) (0.0024) (0.0368) (0.0882) (0.0571) (0.2039) (0.0018)
F Argentina 0.6617*** -0.3075 1.0092*** 0.4933 0.0187** 0.6110*** 0.0104 0.6912*** 0.0944 -0.0011 129
(0.1031) (0.2209) (0.2871) (1.2137) (0.0088) (0.0461) (0.1737) (0.0590) (0.2598) (0.0015)
F Romania 0.1873 0.0304 0.4118 -1.0172 0.0250** 0.4559*** 0.0402 0.5862** 0.8197 0.0027 102
(0.2207) (0.2335) (0.2714) (0.9186) (0.0107) (0.1008) (0.1106) (0.2366) (0.6136) (0.0039)
F Venezuela 0.6938*** 0.0253 0.8090*** -0.3190* 0.0109*** 0.4020*** 0.0740 0.5337*** 0.0695 0.0221*** 112
(0.1100) (0.0541) (0.1287) (0.1713) (0.0029) (0.1053) (0.0819) (0.1249) (0.2545) (0.0047)
Notes: Numbers in the table are coefficient estimates with HAC standard errors in parentheses. The *, **, and *** denote si-
gnificance at 10%, 5%, and 1% levels respectively. The market classification is in S&P standards, and the superscripts "N"
and "R" indicate estimations for normal (tranquil) and recessionary periods respective.
Table 1 derives several plausible results. In the left hand side of the table, estimates of the backward-looking
model show that the Phillips relation (NUGAP) works in the majority of developed countries during normal
(tranquil) economic periods, but its significance remains limited at 10% level. Apparently, the coefficient
gains statistically more significance in Canada, South Korea, Sweden, United Kingdom, and the United
States in individual cases, while it completely fails in a few developed countries, such as Belgium, Denmark,
Finland, Italy, and Singapore. It remains unclear whether the Phillips trade-off derives positive significant
results in Italy. Del Boca et al (2010) also underlined the failure of the Phillips relationship in their study. To
generalize, when the outliers are excluded, the average backward-looking Phillips coefficient (NUGAP) appears
around -0.07 for the developed market sample during normal (tranquil) periods.
The backward-looking NCPC, even the forward-looking NCPC in the right-hand side of the Table 1,
fails to work in the majority of emerging and frontier markets during both tranquil and recessionary periods.
This supports the findings Paul (2009) and Sovbetov and Kaplan (2019), who observe that the rela-
146
EJAE 2019  16 (2)  139-154
tionship is often evasive or absent in less-developed and crisis-prone markets due to a lack of smoothly
operating macroeconomic foundations and the tranquillity of the economic environment. It is worth noting
that the majority of the sample of emerging markets is comprised of Latin American and Asian countries
that have experienced many sovereign debt crises and currency crashes during 1980-1999. 5 In addition,
NCPC also fails to work in Greece, Romania, and Turkey due to undulant economic conditions. These
two countries have experienced about 35-40 quarters of recessions just during 1980-1990 (Appendix-A).
Moreover, the backward-looking fraction of inflation (Nπt-1) appears statistically significant at 1%
level in the majority of the sample during normal economic periods, with exception of Romania, where
lagged inflation ambiguously fails to be significant. Its average magnitude in developed, emerging, and
frontier markets is about 50.73%, 60.63%, and 67.78%, respectively. This, once again, shows that the
most developed countries are less backward-looking compared to emerging and frontier ones.
On the other hand, the table plainly shows that the Phillips relation (RUGAP) collapses, and the backward-
looking fraction of inflation (Rπt-1) remarkably increases in magnitude within whole sample countries
without any loss in significance during recessionary periods. The average coefficient of past inflation scales
up from 50.73% to 56.34% in developed markets, from 60.63% to 67.10% in emerging markets, and from
67.78% to 90.91% in frontier markets. This indicates that markets become more inflation-sensitive during
recessionary periods, as the backward-looking coefficient gains weight and significance.
On the right hand side of the table, results of forward-looking model show that the Phillips relation
( UGAP) works in the majority of developed countries during normal economic periods, with better
N
significance levels compared to backward-looking cases. It is clear that the Phillips coefficient (NUGAP)
gains remarkable significance especially in Austria, Belgium, Denmark, Finland, France, Germany,
Japan, Switzerland, and United States. When insignificant results are excluded, the average Phillips
coefficient (NUGAP) appears the same as it was in the backward-looking case, -0.07, for the developed
market sample during normal (tranquil) periods. In the cases of emerging and frontier countries,
forward-looking model generates alike results as backward-looking one. The forward-looking NCPC
seems not to work in these samples.
Moreover, the forward-looking fraction of inflation [NEt(πt+1)] appears statistically significant at 1% level in
the majority of the samples during normal economic periods, without any exceptions. Its average magnitude
in developed, emerging, and frontier markets is about 62.05%, 58.53%, and 48.96% respectively. This indicates
that developed countries are more forward-looking than emerging and frontier ones. Besides, Phillip relation
(RUGAP) fails to be valid throughout the whole sample during recessionary periods, and a forward-looking frac-
tion of inflation [NEt(πt+1)] considerably increases in magnitude within all sample countries without any loss in
significance, even in emerging and frontier markets. The average coefficient of expected inflation rises from
62.05%to 80.09% in developed markets; from 58.53% to 74.13% in emerging markets; and from 48.96% to
60.37% in frontier markets. This indicates that the dominance of the expected inflation in the forward-looking
model increases in recessionary periods comparing to tranquil (normal) periods in all the sampled countries,
regardless of their market classification. In other words, countries become more inflation-sensitive during
recessionary periods, as forward-looking coefficient gains weight and significance.
Notice that both backward- and forward-looking models estimates overall increase in weight and significance
of inflation factor (past inflation in backward-looking case and expected future inflation in forward-looking case)
during recessionary periods. Apparently, this picks up due to two issues. First, the Phillips coefficient loses its
significance during recession, thus, current inflation becomes more sensitive to past or expected future inflations.
5 During 1980-1990 periods (80 quarters) Argentina has experienced 39 quarters, Brazil 35 quarters, Mexico 20 quarters,
and Venezuela 33 quarters of recessions.
147
EJAE 2019  16 (2)  139-154
Second, neither models incorporate past inflation and expected future inflation variables simultaneously
in a hybrid form. Thus, the study cannot clearly conclude whether markets become more backward- or
forward-looking during recessionary periods. The findings only show that inflation becomes more sensi-
tive to its past or expected future values during recessionary periods and the Phillips relation demises.
In addition, the sample of emerging and frontier markets are predominantly comprised of inflation-
prone fragile countries that have experienced many non-growth periods since the beginning of the
analysis period (1980). Developed markets, however, have relatively fewer recessionary periods. This
also might have some impact on limited increases in past inflation coefficients of backward-looking
model in developed markets during recessionary periods, while the coefficient increases remarkably
in emerging and frontier markets.
CONCLUSION
This study examines the behaviour of NCPC during tranquil and recessionary periods and documents
several findings. Based on the results of this research, first of all, the study finds that both backward- and
forward-looking NCPC models work in the majority of developed markets during tranquil periods. However,
the significance of the backward-looking model is much weaker compared to the forward-looking model.
Second, both backward- and forward-looking NCPC models fail to work in the majority of emerging
and frontier markets, even in tranquil periods. This is because they are predominantly comprised of
inflation-prone fragile countries that have experienced many recessionary periods since the beginning
of the analysis period. This supports the findings Paul (2009) and Sovbetov and Kaplan (2019), who
conclude that the relationship is often evasive or absent in less developed and crisis-prone countries
due to a lack of well-established and smoothly operating macroeconomic foundations.
Third, both backward- and forward-looking NCPC models completely collapse, deriving statistically
insignificant Phillips coefficient during recessionary periods in the whole sample. This shows that the
tranquillity of economic environment significantly matters for the Phillips trade-off to work smoothly.
Fourth, the study documents that developed countries tend to be more forward-looking (less
backward-looking) comparing to emerging and frontier ones during tranquil periods.
Fifth, during recessionary periods both backward- and forward-looking fractions of inflation re-
markably increase in magnitude within whole sample countries without any loss in significance. This
indicates that markets become more inflation-sensitive during recessionary periods. Apparently, this
picks up for two reasons. First, the Phillips coefficient loses its significance during recessions, thus,
current inflation becomes more sensitive to past or expected future inflation. Second, neither models
incorporate past inflation and expected future inflation variables simultaneously in a hybrid form.
Thus, the study cannot clearly conclude whether markets become more backward- or forward-looking
during recessionary periods or not. This should be considered by future researches in related fields.
148
EJAE 2019  16 (2)  139-154
REFERENCES
Ball, L., and Mazumder, S. (2011). Inflation Dynamics and the Great Recession.IMF Working Paper No: 11/121.
Ball, L., and Mazumder, S. (2015). A Phillips Curve with Anchored Expectations and Short-Term Unemploy-
ment. IMF Working Paper No: 15/39.
Bernanke, B. (2010). The Economic Outlook and Monetary Policy. Speech delivered at the Federal Reserve Bank
of Kansas City Economic Symposium, Jackson Hole, Wyoming, 27 August.
Clerc, L., Dellas, H., and Loisel, O. (2010). To be or not to be in monetary union: A synthesis. Banque de France,
Working Paper Series No: 30.
Coibion, O., and Gorodnichenko, Y. (2015).Is the Phillips Curve Alive and Well after All?Inflation Expectations
and the Missing Disinflation. American Economic Journal: Macroeconomics 7 (1): 197–232.
Daly, M., Hobijn, B., and Lucking, B. (2012). Why Has Wage Growth Stayed Strong? Federal Reserve Bank of San
Francisco Economic Letter, 2012–10.
Del Boca, A., Fratianni, M., Spinelli, F., and Trecroci, C. (2010).The Phillips curve and the Italian lira, 1861–1998.
North American Journal of Economics and Finance, 21, 182–197.
Doser, A., Nunes, R., Rao, N., and Sheremirov, V. (2017).Inflation Expectations and Nonlinearities in the Phillips
Curve. Federal Reserve Bank of Boston, WP No.17-11.
Fendel, R., Lis, E.M., and Rulke, J.C. (2011). Do Professional Forecasters Believe in the Phillips Curve? Evidence
from the G7 Countries.Journal of Forecasting, 30, 268-287.
Friedman, M. (1968).The Role of Monetary Policy.American Economic Review, 58(1), 1–17.
Friedrich, C. (2016). Global inflation dynamics in the post-crisis period: What explains the puzzles? Economics
Letters, 142(2), 31–34.
Hall, R.E. (2011). The long slump. American Economic Review, 101(2), 431–69.
Hodrick, R.J., and Prescott, E.C. (1997). Postwar U.S. Business Cycles: an Empirical Investigation. Journal of
Money Credit and Banking, 29(1), 1–16.
International Monetary Fund (IMF). (2013). The Dog That Didn’t Bark: Has Inflation Been Muzzled or Was It
Just Sleeping? In World Economic Outlook, 79–96. Washington, DC, April.
Karan Singh, B., Kanakaraj, A., and Sridevi, T.O. (2011).Revisiting the empirical existence of the Phillips curve
for India.Journal of Asian Economics, 22, 247–258.
Lucas, R. (1976). Econometric Policy Evaluation: A Critique. In The Phillips Curve and Labor Markets. Edited
by Brunner, K. and Meltzer, A. Carnegie-Rochester Conference Series on Public Policy 1. New York:
American Elsevier, 19-46.
Murphy, A. (2018). The Death of the Phillips Curve? Federal Reserve Bank of Dallas, Research Department,
Working Paper 1801, https://doi.org/10.24149/wp1801
Nub, P. (2013).An Empirical Analysis of the Phillips Curve: A Time Series Exploration of Germany.Thesis.Lin-
naeus University.
Ojapinwa, T.V., and Esan, F. (2013). Does Philips Relations Really Exist in Nigeria? Empirical Evidence.Interna-
tional Journal of Economics and Finance, 5(9), 123-133.
Paul, B.P. (2009). In search of the Phillips curve for India.Journal of Asian Economics, 20(4), 479-488.
Phelps, E. (1967). Phillips Curves, Expectations of Inflation and Optimal Unemployment over Time. Economica,
34(135), 254-281.
Phillips, A.W. (1958). The Relationship between Unemployment and the Rate of Change of Money Wage in the
United Kingdom 1861-1957. Economica, 25(100), 283-299.
Ravn, M.O., and Uhlig, H. (2002). Notes on Adjusting the Hodrick-Prescott Filter for the Frequency of Observa-
tions. The Review of Economics and Statistics, 84(2), 371-380.
149
EJAE 2019  16 (2)  139-154
Russel, B., and Banerjee, A. (2008).The Long-run Phillips Curve and Non-stationary Inflation.Journal of Macro-
economics, 30(4), 1792-1815.
Samuelson, P.A., and Solow, R.M. (1960).Analytical Aspects of Anti-inflation Policy.American Economic Review,
50(2), 177-194.
Simionescu, M. (2014).Testing the Existence and Stability of Phillips Curve in Romania.Montenegrin Journal of
Economics, 10(1), 67-74.
Sovbetov, Y., and Kaplan, M. (2019).Empirical Examination of Stability of Neo-Classical Phillips Curve since
1980. Theoretical and Applied Economics, (forthcoming).
Wimanda, R.E., Turner, P.M., and Hall, M.J.B. (2013).The shape of the Phillips curve: The case of Indonesia.
Applied Economics, 45, 4114–4121.
150
EJAE 2019  16 (2)  139-154
APPENDIX
Table A1. Country Codes and Number of Recessions Different Time Periods
1980-1990 1990-2000 1980-2016 1990-2016 2000-2016
Country Name Code
40 quarters 40 quarters 145 quarters 105 quarters 65 quarters
Argentina AG 22 17 55 33 16
Australia AU 9 3 15 6 3
Germany BD 13 13 42 29 16
Belgium BG 6 6 23 17 11
Brazil BR 19 16 51 32 16
Canada CH 8 3 11 3 0
Chile CL 10 8 32 22 14
China CN 11 4 23 12 8
Czech Republic CZ - 13 24 24 11
Denmark DK 15 11 50 35 24
Spain ES 9 6 32 23 17
Finland FN 5 15 43 38 23
France FR 2 5 22 20 15
Greece GR 21 14 75 54 40
Hungary HN - 17 27 27 10
Indonesia ID 9 7 17 8 1
India IN 10 9 24 14 5
Ireland IR 15 11 47 32 21
Italy IT 7 13 47 40 27
Japan JP 6 16 46 40 24
South Korea KO 4 4 11 7 3
Mexico MX 16 4 31 15 11
Malaysia MY 3 3 13 10 7
Netherlands NL 11 3 31 20 17
Norway NW 12 12 44 32 20
Austria OE 10 2 32 22 20
Philippines PH 11 7 21 10 3
Poland PO - 7 15 15 8
Portugal PT 4 7 37 33 26
Romania RM 19 23 56 37 14
Russia RS - 26 37 37 11
South Africa SA 11 12 28 17 5
Sweden SD 9 11 32 23 12
Singapore SP 3 5 24 21 16
Switzerland SW 6 13 31 25 12
Thailand TH 10 8 29 19 11
Turkey TK 19 18 47 28 10
Taiwan TW 6 4 30 24 20
United Kingdom UK 5 6 18 13 7
United States US 6 2 18 12 10
Venezuela VE 20 13 53 33 20
Notes: Numbers in the table show the quarter numbers with negative GDP growth (recession). The "-" denote missing data.
151
EJAE 2019  16 (2)  139-154
Table A2. Results of Unit Root Tests for Series of backward- and forward-looking EAPC
ADF (intercept) PP (intercept)
CPI EI U_U' CPI EI U_U'
AG 0.0791 0.0508 0.0001 0.0000 0.0000 0.0000
(L:2|N:126) (L:2|N:126) (L:0|N:144) (B:7|N:128) (B:7|N:128) (B:2|N:144)
AU 0.0008 0.0000 0.0010 0.0000 0.0000 0.0423
(L:1|N:143) (L:0|N:144) (L:2|N:142) (B:8|N:144) (B:7|N:144) (B:6|N:144)
BD 0.1003 0.0355 0.0419 0.0000 0.0000 0.0008
(L:3|N:141) (L:3|N:141) (L:4|N:140) (B:10|N:144) (B:10|N:144) (B:9|N:144)
BG 0.0000 0.0350 0.4779 0.0000 0.0001 0.0718
(L:0|N:144) (L:1|N:143) (L:3|N:141) (B:9|N:144) (B:9|N:144) (B:3|N:144)
BR 0.2191 0.115 0.0000 0.0993 0.2073 0.0000
(L:2|N:103) (L:3|N:103) (L:8|N:136) (B:1|N:105) (B:6|N:106) (B:8|N:144)
CH 0.0045 0.0040 0.0000 0.0000 0.0000 0.0172
(L:4|N:140) (L:4|N:140) (L:1|N:143) (B:10|N:144) (B:10|N:144) (B:5|N:144)
CL 0.7175 0.7916 0.0002 0.0000 0.0001 0.0007
(L:7|N:137) (L:7|N:137) (L:1|N:119) (B:8|N:144) (B:7|N:144) (B:4|N:120)
CN 0.0173 0.0106 0.0431 0.0000 0.0084 0.0241
(L:3|N:141) (L:2|N:142) (L:0|N:144) (B:8|N:144) (B:4|N:144) (B:4|N:144)
CZ 0.0677 0.0002 0.0081 0.0000 0.0000 0.0509
(L:3|N:96) (L:0|N:100) (L:5|N:87) (B:3|N:99) (B:17|N:100) (B:3|N:92)
DK 0.0361 0.0273 0.5460 0.0000 0.0000 0.2476
(L:4|N:140) (L:3|N:141) (L:1|N:143) (B:10|N:144) (B:9|N:144) (B:7|N:144)
ES 0.3683 0.0393 0.6306 0.0000 0.0003 0.3053
(L:7|N:137) (L:7|N:137) (L:1|N:143) (B:8|N:144) (B:10|N:144) (B:7|N:144)
FN 0.0154 0.0120 0.0001 0.0000 0.0001 0.0282
(L:4|N:140) (L:4|N:140) (L:4|N:140) (B:10|N:144) (B:10|N:144) (B:9|N:144)
FR 0.0308 0.0239 0.0159 0.0055 0.0298 0.0371
(L:11|N:133) (L:0|N:144) (L:1|N:143) (B:9|N:144) (B:12|N:144) (B:4|N:144)
GR 0.3312 0.5571 0.0000 0.0000 0.0000 0.036
(L:4|N:140) (L:4|N:140) (L:8|N:136) (B:10|N:144) (B:11|N:144) (B:7|N:144)
HN 0.3649 0.4055 0.0098 0.0000 0.0000 0.0714
(L:3|N:141) (L:6|N:138) (L:1|N:99) (B:10|N:144) (B:11|N:144) (B:0|N:100)
ID 0.0000 0.0000 0.0000 0.0000 0.0018 0.0015
(L:0|N:144) (L:1|N:143) (L:4|N:140) (B:1|N:144) (B:9|N:144) (B:2|N:144)
IN 0.0031 0.0009 0.0000 0.0000 0.0000 0.0000
(L:3|N:141) (L:4|N:140) (L:4|N:140) (B:9|N:144) (B:10|N:144) (B:10|N:144)
IR 0.0067 0.0011 0.1113 0.0000 0.0000 0.0652
(L:4|N:140) (L:4|N:140) (L:2|N:142) (B:7|N:144) (B:3|N:144) (B:8|N:144)
IT 0.0061 0.0001 0.0024 0.0673 0.0881 0.0027
(L:8|N:136) (L:9|N:135) (L:0|N:144) (B:9|N:144) (B:10|N:144) (B:6|N:144)
JP 0.0001 0.0086 0.6762 0.0000 0.0000 0.3394
(L:1|N:143) (L:2|N:142) (L:0|N:144) (B:9|N:144) (B:8|N:144) (B:6|N:144)
KO 0.0001 0.0149 0.1671 0.0000 0.0000 0.0440
(L:3|N:141) (L:2|N:142) (L:2|N:142) (B:8|N:144) (B:9|N:144) (B:5|N:144)
Notes: The numbers in the table are rejection probabilities of the null hypotheses of ADF and PP tests including intercept.
Probabilities below 0.10 denote rejection of these null hypotheses, thus, confirm stationarity of the CPI (inflation), EI
(expected inflation), and U_U' (unemployment gap) series of related countries. The lag and observation parameters are
presented in the parentheses where "L", "B", and "N" denote lag length, Newey-West bandwidth using Bartlett kernel, and
observation number respectively. The lag length is determined by Schwarz Information Criterion (SIC) under maximum
lag length specification of 13. Initials are given in table A1.
152
EJAE 2019  16 (2)  139-154
Table A2. (continues)

ADF (intercept) PP (intercept)
CPI EI U_U' CPI EI U_U'
MX 0.0415 0.3655 0.0414 0.0672 0.0901 0.0567
(L:0|N:144) (L:9|N:135) (L:4|N:140) (B:7|N:144) (B:6|N:144) (B:6|N:144)
MY 0.0000 0.0027 0.0006 0.0000 0.0000 0.001
(L:0|N:144) (L:1|N:143) (L:0|N:124) (B:4|N:144) (B:8|N:144) (B:4|N:124)
NL 0.0274 0.0040 0.1580 0.0000 0.0000 0.0557
(L:3|N:141) (L:4|N:140) (L:12|N:132) (B:10|N:144) (B:10|N:144) (B:5|N:144)
NW 0.0602 0.0000 0.2180 0.0000 0.0000 0.0848
(L:3|N:141) (L:3|N:141) (L:0|N:144) (B:9|N:144) (B:10|N:144) (B:5|N:144)
OE 0.0059 0.0101 0.0842 0.0000 0.0000 0.1008
(L:4|N:140) (L:3|N:141) (L:0|N:144) (B:9|N:144) (B:9|N:144) (B:1|N:144)
PH 0.0002 0.0005 0.0001 0.0000 0.0000 0.0000
(L:2|N:142) (L:2|N:142) (L:4|N:140) (B:7|N:144) (B:7|N:144) (B:9|N:144)
PO 0.0964 0.2312 0.0066 0.0001 0.0076 0.0768
(L:9|N:108) (L:6|N:112) (L:2|N:106) (B:3|N:117) (B:3|N:118) (B:6|N:108)
PT 0.5028 0.3723 0.1767 0.0000 0.0005 0.0000
(L:7|N:137) (L:7|N:137) (L:1|N:143) (B:10|N:144) (B:10|N:144) (B:9|N:144)
RM 0.0000 0.0000 0.0001 0.0000 0.0000 0.0022
(L:0|N:101) (L:0|N:102) (L:4|N:108) (B:8|N:101) (B:8|N:102) (B:7|N:112)
RS 0.1050 0.1158 0.0004 0.0002 0.0410 0.0006
(L:2|N:97) (L:1|N:99) (L:4|N:100) (B:3|N:99) (B:2|N:100) (B:7|N:104)
SA 0.0364 0.2099 0.0000 0.0000 0.0007 0.0000
(L:2|N:142) (L:2|N:142) (L:5|N:139) (B:7|N:144) (B:9|N:144) (B:10|N:144)
SD 0.1198 0.0463 0.1913 0.0000 0.0002 0.1991
(L:3|N:141) (L:3|N:141) (L:1|N:143) (B:9|N:144) (B:9|N:144) (B:7|N:144)
SP 0.0000 0.0035 0.0000 0.0000 0.0000 0.0159
(L:0|N:144) (L:3|N:141) (L:1|N:143) (B:4|N:144) (B:3|N:144) (B:10|N:144)
SW 0.0496 0.1493 0.0000 0.0000 0.0000 0.0228
(L:4|N:140) (L:3|N:141) (L:1|N:143) (B:10|N:144) (B:9|N:144) (B:7|N:144)
TH 0.0000 0.0003 0.0000 0.0000 0.0000 0.0002
(L:0|N:144) (L:1|N:143) (L:4|N:140) (B:7|N:144) (B:6|N:144) (B:1|N:144)
TK 0.5735 0.6090 0.0000 0.0000 0.0099 0.0000
(L:3|N:141) (L:2|N:142) (L:8|N:136) (B:9|N:144) (B:7|N:144) (B:10|N:144)
TW 0.0000 0.0042 0.1369 0.0000 0.0000 0.1121
(L:3|N:141) (L:6|N:138) (L:5|N:139) (B:7|N:144) (B:6|N:144) (B:9|N:144)
UK 0.0117 0.0089 0.0413 0.0000 0.0000 0.0064
(L:4|N:140) (L:4|N:140) (L:2|N:142) (B:10|N:144) (B:9|N:144) (B:8|N:144)
US 0.0002 0.0000 0.0295 0.0000 0.0000 0.0022
(L:2|N:142) (L:4|N:140) (L:5|N:139) (B:7|N:144) (B:3|N:144) (B:6|N:144)
VE 0.0074 0.4144 0.0000 0.0095 0.2463 0.0000
(L:0|N:143) (L:0|N:111) (L:4|N:140) (B:8|N:143) (B:2|N:111) (B:9|N:144)
153
EJAE 2019  16 (2)  139-154
UZROCI NEUSPEHA FILIPSOVE KRIVE: DA LI JE BITNO DA EKONOMSKO

OKRUŽENJE NE OSCILIRA?
Rezime:
Iako je empirijska literatura koja se odnosi na Filipsovu krivu (the
Phillips curve) značajnog obima, i dalje ne postoji konsenzus o va-
lidnosti i stabilnosti iste. U literaturi se navodi da je Filipsov odnos
nestalan i da je drugačiji od zemlje do zemlje i u različitim vremenskim
periodima; statistički odnos koji se u toku jedne decenije (u nekoj
zemlji) čini jakim, može biti slab u narednoj/nekoj drugoj. Razlozi za
ovu nestalnosti mogu biti osnova za osobenosti neke zemlje i njenog
ekonomskog okruženja. Kako bismo se pozabavili ovom temom, u
radu smo detaljno istražili Filipsov odnos u 41 zemlji tokom perioda
1980-2016, obraćajući pažnju na dinamiku inflacije tokom perioda bez
većih oscilacija i recesije. Kao rezultat, u radu je zaključeno da Filipsov
odnos varira, u zavisnosti od zemlje i vremenskog perioda. Dokazano je
da je taj odnos važeći za većinu razvijenih zemalja, dok nije primenjiv
u zemljama u razvoju i nerazvijenim zemljama, tokom perioda bez
većih oscilacija. S druge strane, odnos je nepostojeći tokom perioda
recesije, čak i na razvijenim tržištima. Ovo dokazuje da je period bez
većih oscilacija u ekonomskom okruženju od izuzetnog značaja, kako
bi Filipsov balans funkcionisao bez problema. Štaviše, frakcije – očeki-
vane inflacije i inflacije u prethodnom periodu, značajno se uvećavaju
tokom perioda recesije kao rezultat toga što Filipsov koeficijent gubi Ključne reči:
na značaju u okviru modela. Ovo ukazuje na činjenicu da su tržišta Filipsova kriva, inflacija, recesija,
osetljivija na inflaciju tokom ovih perioda. stabilnost
154
DO LARGE FIRMS BENEFIT MORE FROM R&D INVESTMENT?
Oyakhilome Ibhagui
The African Institute for Mathematical Sciences (AIMS)

River Place, Arlington Blvd, Virginia

We examine the importance of firm size in the relationship between
Received: May 18, 2019
research & development (R&D) and firm performance. Our empiri-
Correction: June 11, 2019
cal analysis, based on data drawn from Nasdaq-listed companies for
Accepted: July 24, 2019
the period 2002 to 2017, shows that R&D can have effects of varying
magnitudes on firm performance, depending on firm size. When R&D
weakens firm performance, the negative effects are more pronounced
for small-sized firms, but when the impact of R&D is positive, leading
to an improvement in firm performance from increased R&D, large-
sized firms tend to reap most of the benefits. Accordingly, we show
that firm size matters in understanding the scale of the impact of R&D
on firm performance.
Keywords:
research and development (R&D),
firm performance,
firm size
155
E-mail: wallace@aims.ac.za
EJAE 2019  16 (2)  155-173
IBHAGUI O.  DO LARGE FIRMS BENEFIT MORE FROM R&D INVESTMENT?
INTRODUCTION
The aim of this study is to examine how the relationship between R&D and firm performance might
be dependent on firm size. We use data on publicly listed (Nasdaq-listed) firms from 2002 to 2017.
In today’s business world, firms are constantly on the quest to maintain a competitive edge over their
competitors. Most firms are faced with a tough and competitive business environment. To survive and
gain a competitive edge, firms have continuously striven to develop innovative products; otherwise,
they may face the risk of bankruptcy. This underscores why firms devote huge resources to research
and development (R&D) spending. It can, therefore, be inferred that how well a firm performs should
be linked to its investment in R&D. Erickson and Jacobson (1992) notes that R&D expenditures enable
firms to earn high profits and prevent imitation by rivals. Wang (2011) also postulates that firms that
invest more in R&D earn more profits than firms which do not. R&D expenditure is also expected to
help modernize the production process, thereby making products more appealing to buyers at home
and abroad (Salim and Bloch, 2009). In the last few decades, the intensity of R&D has increased many
folds (Pandit, Wasley, and Zach, 2011). Consequently, R&D has emerged as a key factor in promoting
a firm’s competitive advantage internationally.
A sizable number of empirical studies have been devoted to uncovering the likely impact of R&D
on firm performance. The results from these studies are so far, however, ambiguous. While some
studies reported a positive impact (see Johnson and Pazderka, 1993; Long and Ravenscraft, 1993; Lee
and Shim, 1995; Monte and Papagni, 2003; Connolly and Hirschey, 2005; Ho et al., 2006; Ghaffar and
Khan, 2014), others have found a negative relationship (see Gou et al., 2004; Lin and Chen, 2005; Lin
et al., 2006; Artz et al., 2010; Pandit et al., 2011; Donelson and Resutek, 2012). One likely explanation
for this ambiguity might be the failure to account for the contingent role that firm size plays in the
R&D–firm performance nexus. Given the plausible impact of firm size on firm performance, together
with the fact that the impact of R&D on firm performance is still shrouded in debates and controver-
sies, it becomes empirically imperative to ascertain whether accounting for firm size will help to better
explain the rather unsettled relationship between R&D and firm performance. This idea forms the
bedrock upon which our empirical analysis rests. To this end, our study seeks to determine whether
certain threshold levels of firm size exist which can help explain the conflicting relationship between
R&D and firm performance. In other words, do large firms benefit much more when R&D enhances
firm performance? Conversely, in instances where R&D shrinks firm performance, do large firms bear
the greatest brunt or is it their smaller counterparts that bear the brunt? These are new questions in the
literature, which we provide answers to in this paper.
In the literature, most of the empirical studies conducted have primarily employed correlation and
multiple regression analyses (see Morbey, 1989; Morbey and Reithner, 1990; Bae and Kim, 2003; Con-
nolly and Hirschey, 2005; Huang and Liu, 2005). These methods assume a linear relationship between
R&D and firm performance. This means that R&D expenditure is expected to continually enhance or
worsen firm performance across the board. Huang and Liu (2005) note that these assertions are not
rational. Even though increases in R&D investments may generate profits, it would also result in ris-
ing R&D costs (Shy, 1995), and potentially worsen profits and near-term firm performances before
the benefits of R&D spending begin to kick in. This suggests that the relationship between R&D and
firm performance may not be globally linear. As such, the potentially different impact of R&D on firm
performance cannot be empirically modelled using the standard multiple linear regression
156
EJAE 2019  16 (2)  155-173
To this end, we employ the nonlinear, threshold regression model à la Hansen (1999) to model the
links between R&D and firm performance. This technique is very appropriate when possible nonlineari-
ties between variables are of interest. We, therefore, draw on this framework to ascertain whether the
R&D-firm performance nexus is contingent on firm size, i.e. a nonlinear relationship. In other words, we
seek to uncover whether size confers an advantage on firms and, more importantly, whether large-sized
firms are better positioned to reap the positive benefits of rising R&D activities over small-sized firms.
We find that the relationship between R&D and firm performance changes for different levels of
firm size. When the relationship is negative, the negative impact of R&D on firm performance is most
severe for small-sized firms. On the other hand, when positive, the beneficial impact of R&D on firm
performance is most significant for large-sized firms. Our results suggest that large firms are in the best
position to reap the beneficial impacts of R&D whenever they occur.
To the best of our knowledge, no study has explored the contingent role that firm size plays in the
impact of R&D on firm performance using threshold models. The closest empirical study to our work
is Ibhagui and Olokoyo (2018). Their study, however, differs in that their focus is on the role of firm
size in the relationship between leverage and firm performance. Knott and Vieregger (2018) is another
related study. They develop a model linking R&D to the market value of firms. With this, they find a
positive relation between R&D and market value across their full sample of firms. They also show that
market value increases in R&D only for firms with R&D spending below the optimal R&D level. Our
paper differs from this study in that, rather than investigating the effects of different ranges of R&D
on firm performance, we instead examine how R&D influences firm performance when we account
for differences in firm size in a nonlinear threshold modelling framework. While our result also yields
a positive link between R&D and Tobin’s Q (our measure of firm market performance), the major
highlight of our paper is the finding that large firms benefit more from any positive impact of R&D on
firm market performance than small firms.
Lastly, it is important to mention that our paper is different from standard studies in innovation
economics, which examine the relationship between firm size and R&D intensity, and find R&D in-
tensity to first increase and then decrease with firm size. Instead of focusing on firm size and R&D
intensity links, which would be a rehashing of well-studied themes in the literature, in this paper we
instead focus on how firm size influences the impact of R&D on firm performance using Nasdaq-listed
companies. While the innovation economics literature documents the nonlinearity between firm size
and R&D intensity, we show, instead, that the nonlinearity is between R&D and firm performance,
with firm size acting as the threshold variable or nonlinear switcher.
The rest of the paper is organized as follows: In Section 2, we present the literature review. Section
3 presents the empirical methodology. The empirical result is reported in section 4, while section 5
concludes the paper with suggestions for future studies.
157
EJAE 2019  16 (2)  155-173
LITERATURE REVIEW
The empirical literature is replete with a sizable number of empirical studies on the relationship
between R&D and firm performance but, so far, the available evidence and results are mixed, and
largely inconclusive. R&D expenditure is generally considered as investments that have the potential
to bring in returns in the future (Gartrell, 1990; Chauvin and Hirschey, 1993; Martınez-Zarzoso and
Suarez-Burguet, 2000). Furthermore, such investments are expected to help firms maintain a competitive
edge in their line of business. Despite the preponderance of empirical studies, how R&D investments
impact firm performance is still subject to divergent views. For instance, while some empirical studies
find a positive impact (see Johnson and Pazderka, 1993; Long and Ravenscraft, 1993; Lee and Shim,
1995; Monte and Papagni, 2003; Connolly and Hirschey, 2005; Ho et al., 2006; Sharma, 2012), others
reported contrary results (see Gou et al., 2004; Lin and Chen, 2005; Lin et al., 2006).
In the empirical study by Bae and Kim (2003) based on cross-sectional data of the U.S., Germany
and Japan, a positive link is reported between R&D investments and a firm’s market value. A related
study by Monte and Papagni (2003) based on panel regressions also found similar results, in that R&D
intensity was reported to have a significantly positive influence on a firm’s productivity. This is also in
line with results reported by Ho et al. (2005). Bhagwat et al. (2001) also examined the subject matter for
the case of pharmaceutical companies. Results revealed that for each 1% increase in R&D, earnings per
share will increase by one-quarter percent. On the contrary, Gou et al. (2004) find that R&D intensity has
a negative impact on a firm’s profitability, while Lin and Chen (2005), in their study based on the OLS
technique, report a negative correlation between R&D and firm performance measures. Czarnitzki and
Kraft (2006) estimated the impact of R&D spending on a firm’s financial stress and credit ratings. This
was with a view to comparing the performance of firms from Western and Eastern Germany. Results
revealed that a firm’s innovative activities had a positive impact on firm value in Western Germany,
while a negative impact is reported for Eastern Germany. Gagic (2016) finds that innovativeness is
linked to the performance of services business, such as restaurants business.
Lewin and Chew (2005) also note that increasing R&D spending does not necessarily guarantee
higher profits. While the empirical contribution by Pauwels et al. (2004) report that the introduction of
new products boosts a firm’s financial performance and value in the long term. This result is contrary
to the empirical findings by Artz et al. (2010) in that R&D spending was found to be positively related
to patents, while a negative relationship exists between patents and firm growth, as well as between
patents and return on assets. Similar results were also reported by Sher and Yang (2005), Lin et al.
(2006) and Pandit et al. (2011).
In the study by Ghaffar and Khan (2014), earnings per share of firms, return on equity, and return on
assets were used as firm performance measures. Empirical results report a positive correlation between
R&D and each measure of firm performance. This result is contrary to those reported by Donelson and
Resutek (2012), in that R&D expenditure was found to be negatively related to profits. In a related study by
Yu (2017), results revealed that the effect of the first, second, and third lag of R&D expenditure on profits
is positive. Kumbhakar, Ortega-Argiles, Potters, Vivarelli and Voigt (2010), Ayam (2012) and Gui-long
et al. (2017) also found similar results.
One major feature of these studies is that they are based on linear empirical techniques, thereby ignor-
ing the possibility of threshold effects or nonlinearities in the R&D–firm performance nexus. Our study
postulates that the vague understanding that exists in the R&D –firm performance nexus might be because
the literature has largely ignored the role contingent factors play in the R&D –firm performance nexus.
158
EJAE 2019  16 (2)  155-173
Aside from R&D, factors such as human resources, marketing, and financial leverage might also have
an impact on firm performance (Morbey and Reithner, 1990; Erickson and Jacobson, 1992; Chauvin
and Hirschey, 1993; Boer, 1994) while financial distress can impact firm management, Radjen (2015).
Evidently, ever-rising R&D spending might not result in ever-rising profits. This makes it imperative
to ascertain the threshold level of R&D expenditure. In the literature, a good number of studies have
considered contingent factors that might explain the R&D–firm performance nexus. Those tested are
labour productivity (see Morbey and Reithner, 1990), marketing intensity (see Tassey, 1983; Connolly
and Hirschey, 1984; Erickson and Jacobson, 1992; Chauvin and Hirschey, 1993; Gou et al., 2004; Ho
et al., 2005; Lin et al., 2006), debt structure (see Baysinger and Hoskisson, 1989; Long and Ravenscraft,
1993), firm size (see Chauvin and Hirschey, 1993; Ito and Pucik, 1993; Sterlacchini, 1999; Gou et al.,
2004), export activity (see Ito and Pucik, 1993), and diversification (see Gomez-Mejia, 1992).
From the survey of the literature, it is evident that existing empirical studies majorly employed
linear estimation techniques. Moreover, there is no empirical study, to the best of our knowledge, on
the contingent role that firm size plays in the relationship between R&D and firm performance. This
paper, therefore, seeks to address this gap in the literature by exploring the role firm size plays in the
relationship between R&D and firm performance. We employ Hansen’s (1999) threshold regression
model. This approach will enable us to uncover the optimal level of firm size at which R&D improves
or impedes firm performance. Unlike the Hansen (1999) model, the standard linear approach is highly
restrictive, as it assumes that the impact of R&D on firm performance remains the same, irrespective
of the size of a firm. In reality, this might not be true, as differences in firm sizes may alter the relation-
ship between R&D and firm performance. Our study enriches the literature, as well as providing fresh
insights into the R&D-firm performance nexus.
EMPIRICAL METHODOLOGY
Here, we present a description of the empirical specification, the data, summary statistics, and the main
empirical results. The complete raw data used in this study is made publicly available via the following
link: https://data.mendeley.com/datasets/n28bk9fpsf/3 , making the results of this study reproducible.
Empirical Specification
As with studies examining the relations between economic variables, it is quite possible that the
impact of R&D on firm performance may vary with firm characteristics, such as size. In other words,
how firm performance responds to additional investment in R&D may depend on the size of the firm.
It could be the case that firms need to attain a certain level of size before the beneficial effects of R&D
on their performance begin to manifest. This is a purely empirical, rather than theoretical, question
that requires a flexible empirical specification to accommodate this possibility. We address this issue in
this section. We examine whether the relationship between R&D and firm performance is contingent
on firm size. Our threshold variable is, thus, firm size, while our analytical framework for the empiri-
cal specification is based on the Hansen (1999) panel threshold regression model. We seek to establish
how firm size influences the relations between R&D and firm performance. To achieve this objective,
we specify the following panel threshold regression model:
159
EJAE 2019  16 (2)  155-173
FPERit =µi + β15 RDit I ( Dit ≤ d1 ) + β 25 RDit I ( Dit > d1 ) + ϕ 5 controlsit + ε it5(1)
where i=1,…n signify individual firms, t=2002,…..,2017 represents time period, FPER represents
firm performance, µi is the time invariant firm specific fixed effect, I (.) is the indicator function,
while ε it is the error term, D is the threshold variable (firm size), and d1 is its estimated value, the
s
threshold value.
The empirical method used in the study is based on the Hansen (1999) threshold regression model
implemented in Stata using the “xthreg” command. Full, detailed step-by-step information on the
procedure used in the empirical analysis is provided by Wang (2015) (The Stata Journal).
Data
Our raw data samples comprise firms listed on Nasdaq during the period of 2002 to 2017. In total,
we choose 476 companies with 7,616 observations. The main explanatory variable is R&D intensity
measured as total R&D expense/net sales of listed firms. Hall and Bagchi-Sen (2007) and Ehie and Olibe
(2010) note that R&D intensity is superior to absolute R&D investment amount in that the latter fails
to differentiate R&D investment of dissimilar scales enterprises.
The dependent variable in this study, firm performance, is proxied using three different measures,
Tobin’s Q, return on equity (ROE), and return on assets (ROA). Our study considers Tobin's Q as
a proxy for market value, while the ROA and ROE measures are accounting indicators. Apart from
R&D, firm performance is also affected by a variety of internal and external variables. Therefore, we
also considered two additional control variables which are marketing intensity and capital structure.
Marketing-oriented companies devote a lot of resources to marketing campaigns, in addition to R&D
investments (Connolly and Hirschey, 2005). This is the main reason we included marketing intensity
(total sales cost divided by the operating revenue) as one of our explanatory variables. A reasonable
capital structure and appropriate debt ratio can also improve company performance and reduce financ-
ing costs. In contrast, excessive financial leverage may amplify a firm’s operating risk. We, therefore, use
debt-to-equity ratio to control for the impact of capital structure. Our threshold variable is firm size.
Results and Discussions
Before examining the threshold relationship, it is expedient to investigate the descriptive statistics of
the variables. This shows the characteristics of the variables. Table 2 presents the descriptive statistics
using concepts like mean, maximum, minimum and the standard deviation. The standard deviation, a
measure of dispersion of the variables from the mean, show that the actual deviations from the mean
ROA, ROE, NIG, R&D, Size, LEV, MI and OC are minimal. Hence, the variables are stable. Moreover,
the mean and standard deviations of the variables are within the minimum and maximum values.
Hence, they display a high level of consistency. Overall, each variable has observation of 7,616, hence,
our panel is fully balanced.
160
EJAE 2019  16 (2)  155-173
Table 2
2002-2017
Variable Observations Mean Min Max SD
Tobin's Q 7616 1.78 -0.49 9.55 1.82
ROE 7616 0.02 -3.07 2.09 0.59
ROA 7616 0.003 -1.31 0.31 0.24
NIG 7616 -0.05 -9.62 10.51 2.71
R&D 7616 0.71 0 39.39 3.75
Size 7616 5.65 1.71 9.88 1.92
Lev 7616 0.13 0 0.86 0.18
MI 7616 0.01 0 0.15 0.03
OC 7616 0.94 0.01 3.01 0.66
The interpretation further proceeds with the panel unit root test and the threshold analysis. The
analytical framework for this study follows Hansen’s (1999) panel threshold model. This is employed
to uncover how different firm sizes might alter the relationship between R&D and firm performance.
Before we embark on our analysis, we first ascertain whether our panel data is stationary. For this
purpose, we utilize two tests, the Levin–Lin–Chu ADF (Levin et al., 2002) and the IPS ADF (Im et al.,
2003) tests. In Table 1, we report the results of the panel unit root test. From the reported results, it is
evident that our variables are stationary, that is, that the variables are all I(0) variables. Having obtained
this result, we proceed with our empirical analysis.
Table 1: Panel Unit Root Test Results

Levin-Lin Chu (LLC) lm, Pesaran and Shin (IPS)
Variables Statistic p-value Statistic p-value
Tobin's Q -47.89*** 0.00 -32.75*** 0.00
ROA -82.92*** 0.00 -24.32*** 0.00
ROE -50.43*** 0.00 -62.00*** 0.00
Firm size -130.00*** 0.00 -11.46*** 0.00
R&D intensity -34.65*** 0.00 -4.20*** 0.00
Marketing intensity -33.27*** 0.00 -12.10*** 0.00
Capital structure -160.00*** 0.00 -76.18*** 0.00
Note: *** denotes significance at 1% or below.
In our empirical analysis, the bootstrap method is employed to obtain F-Statistics approximations,
after which we then estimate the p-values. The results of the single threshold and double threshold tests
are presented in Table 2. We repeat the bootstrap procedure several times for each panel threshold tests.
The results obtained reveal that the p-values of the three proxies of firm performance, ROA, ROE, and
Tobin’s Q, are all significant for the single threshold model, while only the p-values of two proxies of
firm performance, ROE and Tobin’s Q, are significant for the double threshold model. With these results,
we conclude that firm size has a significant double threshold on the relationship between R&D and
firm performance for the ROE and Tobin’s Q measures of firm performance, while the ROA measure
reports a single threshold. For the Tobin’s Q, ROA, and ROE and measure of firm performance, the
threshold estimates are 7.21, 7.15, and 2.17 respectively.
161
EJAE 2019  16 (2)  155-173
In Table 3, we present the estimated coefficients on the regressors for each proxy of firm performance.
When Tobin’s Q is the proxy for firm performance, we observe that the coefficient of R&D is positive
when the threshold variable - firm size, - is less than its estimated threshold value. Likewise, when the
threshold variable - firm size - falls between its low and high threshold values, and when the threshold
variable is above its high threshold value. We, however, report that the coefficient is insignificant when
the threshold variable lies between its low and high threshold values. When ROA is the dependent
variable, results show that the coefficient of R&D is negative, while when the threshold variable is
less than its estimated threshold value and the same is true when it is above the estimated threshold
value. The result is similar when ROE is the dependent variable. Except in the case where Tobin’s Q is
the dependent variable, all other proxies for firm performance reveal that the impact of R&D on firm
performance is negative, irrespective of whether the threshold variable is large, small, or between the
estimated threshold values. Since Tobin’s Q measures a firm’s market-based performance, while ROE
and ROA measure operations-based performance based on a firm’s accounting or book performance
alone, we conclude that R&D has a positive threshold effect on firm market performance, and a nega-
tive threshold effect on firm book or accounting performance.
What has clearly emerged from our empirical findings is that when R&D improves firm perfor-
mance (as in the case of firm market performance), the beneficial impact is larger and most significant
for large-sized firms. In other words, large-sized firms reap more from increased R&D expenditure
in instances where R&D improves firm performance. This is explainable since larger firms have the
financial capacity to attract, recruit, and maintain top-notch researchers domestically and from all
around the world. In response, these researchers formulate and make breakthroughs, which lead to
policies that ensure that firms reap maximally from the results of R&D through increases in sales and
profitability, hence their performance. Conversely, when R&D reduces firm performance, the nega-
tive effect is lower for large-sized firms. The results also show that small-sized firms (firm size below
the estimated threshold) benefit less when R&D improves performance, and they are most negatively
affected when R&D shrinks firm performance. That is, the impact is more severe for small-sized firms
when R&D weakens firm performance. This is clear from the reported coefficient values, as the negative
impact is higher for small-sized firms. What this result suggests is that firms need to attain a higher
firm-size level to reap the benefits or mitigate any demerits stemming from the acceleration of R&D
expenditure. In other words, the bigger a firm becomes, the more likely it benefits from any positive
R&D effect. Lastly, one reason the response of Tobin’s Q (firm market performance) to R&D being
positive is that markets often respond positively to positive news, such as a commitment to R&D. This
positive response improves companies’ equities and market value, boosting firm market performance.
On the other hand, ROE and ROA, which are measures of firm accounting or book performance, may
first shrink on increased resources committed to R&D in the current financial year before they sub-
sequently begin to reflect gains from R&D. In other words, high R&D may be a leading indicator for
improved firm accounting or book performance.
162
EJAE 2019  16 (2)  155-173
Table 2: Test of Threshold Effects between R&D and Firm Performance – Threshold Variable is Firm Size
Tobin's Q ROA ROE
Estimated threshold
2.17 3.84 3.36
value
Single threshold effect F-stat 243.42*** 26.19* 503.70***
p-value 0.00 0.06 0.00
Estimated threshold
2.17 3.84 3.36
value I
Estimated threshold
Double threshold effect 7.21 7.25 2.17
value II
F-stat 234.15*** 10.7 125.08***
p-value 0.00 0.38 0.00
Double thresh- Singlethreshold Double threshold

Final Comments
old points point points
Note: F-statistics and p-values come from repeated bootstrap procedures, ***, **, and * represent significance at 1%, 5% and
10% levels, respectively
Table 3: Estimated Coefficients of the Effect of R&D on Firm Performance at the Threshold Points
Coefficients Estimated coefficient t-stat Robust se R2

Tobin's Q β0 0.40*** 5.50 0.73 0.73
β1 0.016 0.28 0.06 0.06
β2 1.60*** 3.63 0.44 0.44
ROA β0 -0.10*** -8.65 0.01 0.01
β1 -0.07*** -8.32 0.01 0.01
ROE β0 -0.12*** -10.82 0.01 0.01
β1 -0.08*** -7.97 0.01 0.01
β2 -0.03*** -4.16 0.03 0.03
Note: In the case of 2 threshold points, β 0 represent the coefficient of R&D when the threshold variable, firm size, is less that
its smaller threshold value; β1 is the coefficient when the threshold variable, which falls between its smaller and larger
threshold values, while β2 is the coefficient when the threshold variable is above its larger threshold value. In the case
of one threshold point, β0 is the R&D coefficient when firm size, the threshold variable, is below the estimated threshold
value, while β1 is the coefficient when the threshold variable is greater than the threshold variable.
163
EJAE 2019  16 (2)  155-173
This paper also captures the likely impact of the control variables - marketing intensity, firm size,
and capital structure - on firm performance. The results are presented in Table 4 and show that firm
size has a significant negative impact on firm performance. This result may be due to the period and
firms selected, and is akin to the outcome often obtained when firm size is specified linearly. Another
plausible explanation for this counterintuitive negative effect of firm size is that when firms become
large, they naturally come under the control of different sorts of managers, some of whom pursue
their self-interests at the expense of business-related or firm-wide interests. As such, core performance
maximization that enhances firm overall performance may be indirectly replaced with inefficiencies
and manager-friendly, but performance-destructive, policies, thus dampening firm performance, even
as firm grows. Furthermore, the negative result also implies that the failure of big firms to increase
capital structure ratio, having increased their size, may lead to of sub-optimal financial management. In
addition, it possibly highlights the nonlinearities in the effect of firm size on firm performance wherein
firm size positively affects performance for some firm size ranges and negative for others. This kind
of outcome has been well explored in Pervan and Višić (2012). Meanwhile, marketing intensity and
capital structure have an insignificant negative impact when Tobin’s Q serves as the proxy for firm
performance. Furthermore, when ROA serves as the dependent variable, capital structure and firm size
had significant negative impacts, while marketing intensity reported an insignificant negative impact.
For the ROE proxy for firm performance, we report that firm size had a significant positive impact on
firm performance, while capital structure and marketing intensity reported a significant negative impact.
Table 4: Impacts of Other Covariates (Control Variables) on Firm Performance

Firm Performance Measures
Tobin's Q ROA ROE
Capital structure -0.01 -0.08** -0.23***
(-0.02) (-2.05) (-15.05)
Control Variables Firm size -0.31*** -0.13*** 0.03***
(-10.94) (-14.65) -10.46
Marketing intensity -1.23 -0.28 -0.88***
(-0.88) (-0.79) (-4.76)
F-stat 27.09 57.67 70.71

p-val 0.00 0.00 0.00
R2 0.21 0.05 0.25

No of observations 7,616 7,616 7,616
Note: The impact of control variables on firm performance is reported, alongside the t-statistic in (). ***, **, and * signify
significance at 1%, 5%, and 10% levels.
Our main takeaway consists of two parts. First, when R&D improves firm performance, the beneficial
impact is most significant when the firm size is large. Second, when R&D is negatively associated with firm
performance, the effect is most severe for small-sized firms. In other words, small-sized firms are hit the
hit when R&D weakens firm performance. Thus, large-sized firms benefit more from increases in R&D.
From the foregoing, any view that large firms do not benefit more from increases in R&D appears to be
unjustified. According to the results, small-sized firms, in fact, have more to worry about, as they benefit
less when R&D enhances performance, and usually bear the brunt when R&D worsens firm performance.
164
EJAE 2019  16 (2)  155-173
Our results are quite instructive, especially for business managers and policymakers, as they pro-
pose that discussions on firm size should be a board room standard when decisions on R&D are being
made. Our paper enriches the literature, in addition to providing fresh insights and perspectives that
will be of immense benefit to policymakers, researchers, and business managers. To our knowledge,
no previous study has considered the contingent role that firm size plays in the link between R&D and
the performance of publicly-listed NASDAQ firms.
CONCLUSION
In this study, we have attempted to answer an important economic decision question: does firm
size matter in the relationship between R&D and firm performance? This is with a view to uncovering
how firm size might provide more insights into the relationship between R&D and firm performance.
Our empirical analysis employs data from firms listed on Nasdaq during the period of 2002 to 2017.
In total, we have 476 companies with 7,616 observations. Three proxies of firm performance are em-
ployed: Tobin’s Q, ROE, and ROA. Our explanatory variable of interest is R&D intensity and firm size
(the threshold variable); we also included control variables, such as marketing intensity and capital
structure. For the specification of our empirical analysis, we have employed Hansen’s (1999) threshold
regression model.
From the empirical findings, firm size matters in the relationship between R&D and firm perfor-
mance. In specific terms, we find that in instances where R&D worsens firm performance, the negative
impact is most evident in small-sized firms. Large-sized firms, on the other hand, benefit more when
R&D improves firm performance. When firm performance is proxied with Tobin’s Q, the gains from
R&D improve as firm size becomes larger. On the contrary, when ROA and ROE are proxies of firm
performance, the negative impact of R&D becomes worse as firm size becomes smaller.
We recommend that future studies control for other plausible determinants of firm performance.
It is also important to extend this analysis to firms listed on the exchanges of other countries, sub-
regions, and economic blocs. Future studies should also include a wider array of plausible thresholds
and controls in the threshold model and investigate the potential lead-lag relations between R&D and
firm performance, especially for the accounting firm performance measures, such as ROA and ROE.
At this juncture, it is necessary to emphasize that, as the investigation performed in this paper relates
to Nasdaq-listed companies alone, future studies could perform further similar investigations on firms
listed on other exchanges.
165
EJAE 2019  16 (2)  155-173
REFERENCES
Ayam, R. T. (2012). Research and development and firm performance: Investigating the need for research and
development expenditure as a factor of enhancing the performance of firms.
Adegbite, S. A. (2012). An evaluation of technology innovation on the performance of indigenous textile weaving
firms in southwestern Nigeria. Journal of Business and Management Volume 1, Issue 1, 01-14.
Adeyeye, A. D., Jegede, O. O. and Akinwale, Y. O. (2013). The impact of technology innovation and R&D on
firms’ performance: an empirical analysis of Nigeria’s service sector, International Journal of Technological
Learning, Innovation and Development, Vol. 6, No. 4, 374–395.
Artz, K. W., Norman, P. M., Hatfield, D. E. and Cardinal, L. B. (2010). A longitudinal study of the impact of
R&D, patents, and product innovation on firm performance. Journal of Product Innovation Management,
27(5), 725-740.
Bae, S. C. and Kim, D. (2003). The effect of R&D investment on market value of firm: evidence from the U.S.,
Germany, and Japan. Multinational Business Review, 11, 51–75.
Baysinger, B. D. and Hoskisson, R. E. (1989). Diversification strategy and R&D intensity in large multi-product
firms, Academy of Management Journal, 32, 310–12.
Bhagwat, Y., DeBruine, M. and Gondhalekar, V. (2001). R&D Leverage – A Measure to Evaluate the Impact
of R&D on Earnings of Pharmaceutical Companies, Journal of Research in Pharmaceutical Economics,
11(3-4), 55 – 68.
Boer, F. P. (1994). Linking R&D to growth and shareholder value, Research Technology Management, 37, 16–22.
Chauvin, K. W. and Hirschey, M. (1993). Advertising, R&D expenditures and market value of the firm, Financial
Management, 22, 128–40.
Chu, H., Sher, P. J. and Chiu, Y. (2010). R&D intensity, firm performance and the identification of the threshold:
fresh evidence from the panel threshold regression model, Applied Economics, 42:3, 389-401.
Connolly, R. A. and Hirschey, M. (2005). Firm size and the effect of R&D on Tobin’s q, R&D Management, 35, 217–23.
Czarnitzki, D. and Kraft, K. (2012). Spillovers of innovation activities and their profitability. Oxford Economic
Papers, 64(2), 302-322.
Donelson, D. C. and Resutek, R. J. (2012). The effect of R&D on future returns and earnings Forecasts, Springer
Science+Business Media, LLC.
Ehie, I. and Olibe, K. (2010). The effect of R&D investment on firm value: An examination of US manufacturing
and service industries. International Journal of Production Economics, 128(1), 127-135.
Erickson, G. and Jacobson, R. (1992). Gaining comparative advantage through discretionary experience: the
returns to R&D and advertising, Management Science, 38, 1264–79.
Gartrell, K. D. (1990). Innovation, industry specialization and stockholder wealth, California Management Re-
view, 32, 87–101.
Gagic, Snježana (2017). Restaurant Innovativeness: A Case Study of Vojvodina. In European Journal of Applied
Economics, 13, 2, 57-69.
Ghaffar, A. and Khan, W. A. (2014). Impact of research and development on firm performance International
Journal of Accounting and Financial Reporting, Vol. 4, No. 1.
Gomez-Mejia, L. R. (1992). Structure and process of diversification, compensation on strategy, and firm perfor-
mance, Strategic Management Journal, 13, 381–97.
Gou, B., Wang, Q. Z. and Shou, Y. Y. (2004). Firm size, R&D, and performance: An empirical analysis on soft-
ware industry in China, in International Engineering Management Conference 2004, IEEE, Singapore.
Gui-long, Z., Yi, Z., Kai-hua, C. and Jiang, Y. (2017). The impact of R&D intensity on firm performance in an
emerging market: Evidence from China’s electronics manufacturing firms, Asian Journal of Technology
Innovation, DOI: 10.1080/19761597.2017.1302492
166
EJAE 2019  16 (2)  155-173
Hall, L. A. and Bagchi-Sen, S. (2007). An analysis of firm level innovation strategies in the U. S. biotechnology
industry. Technovation, 27(1-2):4-14.
Hansen, B. E. (1999). Threshold effects in non-dynamic panels: estimation, testing, and inference. Journal of
Econometrics, 93, 345–368.
Ho, H. K., Tjahjapranta, M. and Yap, C. M. (2006). Size, leverage, concentration, and R&D investment in gen-
erating growth opportunities, Journal of Business, 79, 851–76.
Huang, C. J. and Liu, C. J. (2005). Exploration for the relationship between innovation, IT and performance,
Journal of Intellectual Capital, 6, 237–52.
Ibhagui, Q. W. and Olokoyo, F. O. (2018). Leverage and firm performance: New evidence on the role of firm size.
North American Journal of Economics and Finance (2018), https://doi.org/10.1016/j.najef.2018.02.002
Im, K. S., Pesaran, M. H. and Shin, Y. (2003). Testing for unit roots in heterogeneous panels, Journal of Econo-
metrics, 115, 53–74.
Ito, K. and Pucik, V. (1993). R&D spending, domestic competition, and export performance of Japanese manu-
facturing firms, Strategic Management Journal, 14, 61–75.
Johnson, I. D. and Pazderka, B. (1993). Firm value and investment in R&D, Managerial and Decision Economics,
14, 15–24.
Kumbhakar, S. C., OrtegaArgilés, R., Potters, L., Vivarelli, N. and Voigt, P. (2010). How corporate R&D affects
firms' productivity: Empirical evidence and toehold for targeted policy measures. Contributed paper for
the 2nd Conference on corporate R&D (CONCORD - 2010).
Knott, A. M., and Vieregger, C., (2018). A Theory of R&D When Firms Differ. Available at SSRN: https://ssrn.
com/abstract=2382885 or http://dx.doi.org/10.2139/ssrn.2382885.
Lee, J. and Shim, E. (1995). Moderating effects of R&D on corporate growth in U.S. and Japanese hi-tech indus-
tries: an empirical study, The Journal of High Technology Management Research, 6, 179–91.
Lee, J. and Shim, E. (1995). Moderating effects of R&D on corporate growth in U.S. and Japanese hi-tech indus-
tries: an empirical study, The Journal of High Technology Management Research, 6, 179–91.
Levin, A., Lin, C. F. and Chu, C. S. (2002). Unit root tests in panel data: asymptotic and finite-sample properties,
Journal of Econometrics, 108, 1–24.
Lewin, T. and Chew, E. (2005). High R&D, greater profits. Automotive News Europe, 10(23), 1 & 26.
Lin, B. W. and Chen, J. S. (2005). Corporate technology portfolios and R&D performance measures: A study of
technology intensive firms, R&D Management, 35, 157–70.
Lin, B., Lee, Y. and Hung, S. (2006). R&D intensity and commercialization orientation effects on financial per-
formance, Journal of Business Research, 59, 679–85.
Long, W. F. and Ravenscraft, D. J. (1993). LBO’s debt and R&D intensity, Strategic Management Journal, 14, 119–35.
Martınez-Zarzoso, I. and Suarez-Burguet, C. (2000). The determinants of trade performance: influence of R&D
on export flows, Applied Economics, 32, 1939–46.
Monte, A. D. and Papagni, E. (2003). R&D and the growth of firms: Empirical analysis of a panel of Italian firms,
Research policy, 32, 1003–14.
Morbey, G. K. (1989). R&D expenditures and profit growth, Research Technology Management, 32, 20–23.
Morbey, G. K. and Reithner, R. M. (1990). How R&D affects sales growth, productivity and profitability, Research
Technology Management, 33, 11–15.
Pandit, S., Wasley, C. E. and Zach, T. (2011). The effect of research and development (R&D) inputs and outputs
on the relation between the uncertainty of future operating performance and R&D expenditures. Journal
of Accounting, Auditing and Finance, 26(1), 121-144.
Pauwels, K., Silva-Risso, J., Srinivasan, S. and Hanssens, D. M. (2004). New products, sales promotions, and firm
value: The case of the automobile industry. Journal of Marketing, 68(4), 142-156. http://dx.doi.org/10.1509/
jmkg.68.4.142.42724
167
EJAE 2019  16 (2)  155-173
Pervan M. and Visic J. (2012), Influence of firm size on its business success. Croatian Operational Research Review
(CRORR), Vol. 3, 2012
Pitelis, C. N. (2009). Edith Penrose’s, “The theory of the growth of the firm’ fifty years later. Available at http://
ssrn.com/abstract=1477885
Radjen, D. (2015). The Analysis of the Effects of Financial Distress on the Top Management in the Republic of
Serbia. The European Journal of Applied Economics, 12(1), 19–25. https://doi.org/10.5937/ejae12-7977.
Salim, R. A. and Bloch, H. (2009). Business expenditures on R&D and trade performances in Australia: is there
a link? Applied Economics, 41(3), 351-361.
Sharma, C. (2012). R&D and firm performance: Evidence from the Indian pharmaceutical industry. Journal of
the Asia Pacific Economy, Vol. 17, No. 2, May 2012, 332–342.
Sher, P. J. and Yang, P. Y. (2005). The effects of innovative capabilities and R&D clustering on firm performance:
The evidence of Taiwan's semiconductor industry. Technovation, 25(1), 33-43. http://dx.doi.org/10.1016/
S0166-4972 (03)00068-3
Shy, O. (1995). Industrial organization: Theory and applications. The MIT Press.
Sterlacchini, A. (1999). Do innovative activities matter to small firms in non-R&D-intensive industries? An ap-
plication to export performance, Research Policy, 28, 819–32.
Tassey, G. (1983). Competitive strategies and performance in technology-based industries, Journal of Economic
Business, 36, 21–40.
Wang, C. H. (2011). Clarifying the effects of R&D on performance: Evidence from the high technology industries.
Asia Pacific Management Review, Vol. 16, No. 1, 51-64.
Wang, Q., (2015). Fixed-Effect Panel Threshold Model Using Stata, Stata Journal, 15(1), 121–134.
Yu, Z. (2017). R&D Expenditure and Firm Performance in China. Major Paper presented to the Department
of Economics of the University of Ottawa in partial fulfillment of the requirements of the M.A. Degree.
168
EJAE 2019  16 (2)  155-173
APPENDIX 1
The Threshold Model
This model is nonlinear in that it captures instances where the relationship between variables might
be different at certain sections of the data. The model also allows us to split the data sample into two
regimes, D < d1 and D > d 1 for all values of R&D, where D is the threshold variable (firm size) and
d1 is its estimated value, the threshold value. Our threshold variable is D ∈ V where V is a vector
of all regressors. It is this threshold variable that divides the data samples into different regimes. d1 ,
on the other hand, is the threshold values associated with D . In this study, we adopt firm size as the
threshold variable, since we are interested in how R&D weakens firm performance for varying levels
of firm size.
In this paper, our regressors of interest are return on asset (ROA), return on equity (ROE), and
Tobin’s Q. Taking a cue from Hansen (1999), we formulate a model where the regressors, control, and
threshold variables are exogenous. The panel threshold regression model is specified as:
yit = β1' xit I ( pit ≤ γ ) + β 2' xit ( pit > γ ) + ν it (1.1)
where=
ν it µi + eit
We draw the observed data samples from a panel ( yit , pit , xit ;1 ≤ i ≤ n,1 ≤ t ≤ T ) .i and t represent
firm and time, while xit is a set of regressors. The threshold variable is pit ,which can be a member of
xit ,while refers to the unobserved time invariant fixed effects.
The equation specified above can be re-written as
yit µi + β1' xit I ( pit ≤ γ ) + β 2' xit ( pit > γ ) + eit
=
(1.2)
where yit is a real-valued scalar variable, xit is an m × 1 vector of regressors, pit is a scalar
threshold variable with Dim( yit ) = Dim( pit ) , while the unobserved threshold value is γ .
The vectors of slope parameters associated with the different regimes are β1' and β 2' where
A = { pit | ( pit ≤ γ )} and B = { pit | ( pit > γ )} (1.3)
while I (.) is the indicator function defined for an arbitrary element d in a set A ∪ B
The equation specified above gives rise to two possibilities. These possibilities depended on whether
d ∈ { pit | ( pit ≤ γ )} or d ∈ { pit | ( pit > γ )} ,which in turn yields the two regimes specified below:
µi + β1' xit + eit pit ≤ γ

yit =  (1.4)
µi + β1' xit + eit pit > γ
169
EJAE 2019  16 (2)  155-173
This equation can also be re-written such that both regimes are now expressed in a compact man-
ner. In this specification, the regressors and thresholds are represented in a column vector, while the
slope parameters are set in a row vector. This can be expressed as:
 xit I ( pit ≤ γ ) 
yit µi + ( β1' , β 2' )
=  + eit (1.5)
 it
x I ( p it > γ ) 
=yit µi + β ' xit (γ ) + eit (1.6)
 xit I ( pit ≤ γ ) 
We divide β ' = ( β1' , β 2' ) and xit =   into two regimes where the threshold variable
 xit I ( pit > γ ) 
is at most its threshold value, and when the threshold variable is above its threshold value. In
what follows, we estimate the slopes of β1' and β 2' . We reiterate that the error component has
been divided into two parts where eit is assumed to be an independent and identically distrib-
uted (iid) with constant and finite variance. This assumption requires that the threshold variable
and regressors eliminate endogenous variables, which may correlate with the error term. Hence,
eit is a martingale difference sequence {eit , Ft } on the probability space (Ω, F , P ) for each i
since E (=
eit ) 0 < ∞ and E (eit | Ft −1 ) = 0 where Ft −1 is a natural filtration at time t. Likewise,
E (eit | pit ) = E (eit | xit ) = 0 and ( xit , pit ) are measurable with respect to Ft −1 where Ft −1 is the
sigma field generated by N = {x (i-j)t , p( io − j ) t , e( i −1− j ) t : j ≥ 0}
Estimating the Model
As a first step in estimating the model specified above, we eliminate firm specific effects, . This is
done using within transformation in which contemporaneous observations are subtracted from the
within group average for each variable. The transformation of equation (1.1) yields:
=yit⊥ β t⊥ xit (γ ) + eit⊥ (1.7)
 1 T
 ⊥  i T

=
where yit⊥  yit −

∑ yit=
T t =1 
, eit  eit −
 T
∑e
t =1
it  and β ' = ( β1' , β 2' )

(1.8)
 1 T 
 ( xit I ( pit ≤ γ ) − ∑ (xit I ( pit ≤ γ ) 
⊥  T t =1 
and xit =  T  (1.9
 ( xit I ( pit > γ ) − ∑ (xit I ( pit > γ )
1
 T t =1 
In the equation specified below, we denote the errors and stacked data connected with firm i , with
one-time period deleted as in Hansen (1999):
170
EJAE 2019  16 (2)  155-173
 yi⊥2   xi⊥2 (γ ) '   ei⊥2 

     
 ⋅   ⋅   ⋅ 
  ⊥   ⊥   (2.0)
yi =  ⋅ , xi (γ ) =  ⋅
⊥
, ei =  ⋅ 
 ⋅   ⋅   ⋅ 
 ⊥   ⊥ '
  ⊥ 
 yiT   xiT (γ )   eiT 
Also, we denote data stacked over all firms as Y ⊥ , X ⊥ and e ⊥ where:
 y2⊥   x ⊥ (γ ) '   ei⊥2 
   2   
 ⋅   ⋅   ⋅ 
     
 ⋅   ⋅   ⋅ 
 ⋅   ⋅   ⋅ 
     
Y ⊥ =  yt⊥ , x ⊥ (γ ) =  xt⊥ (γ ) ' , e ⊥ =  eit⊥  (2.1)
 ⋅   ⋅   ⋅ 
     
 ⋅   ⋅   ⋅ 
 ⋅   ⋅   
     ⋅ 
 ⊥  xn⊥ (γ ) '   en⊥ 
 yn     
=
This can be re-specified as: Y ⊥ X ⊥ (γ ) β + e ⊥ (2.2)
The assumptions guiding the original equation are reflected in the transformed equation. We can
therefore estimate β using least squares for any which in turn yields:
βˆ (γ ) = ( X ⊥ (γ )' X ⊥ (γ ) −1
X ⊥ (γ )' Y ⊥ ) (2.3)
From this estimated equation, we can obtain the vector of regression residuals from the threshold
dependent slope parameter. This is specified as:
⊥
eˆ= (γ ) Y ⊥ − X ⊥ (γ ) β
= (γ ) Y ⊥ − X ⊥ (γ )( X ⊥ (γ )' X ⊥ (γ ) −1
X ⊥ (γ )' Y ⊥ ) (2.4)
Subsequently, we use the regression residual to compute the sum of errors. Following Hansen (1999),
the threshold value γ, which determines the sample split is estimated by least squares. Thus, we find
γˆ γ that minimizes the concentrated sum of squared errors, such that the least squares estimator of
γˆ =γ arg min S1 (γ )1 . We have the parameter estimate as βˆ = βˆ (γˆ ) after obtaining γˆ . The slope
parameters estimated at the different regimes partitioned by γˆ is βˆ (γˆ ) . Thus, β̂1 and β̂ 2 represents
the vector of slopes associated with the regimes I ( pit ≤ γˆ ) and I ( p it > γˆ ) . Furthermore, we partition
the data sample into regimes after obtaining the estimate γˆ of the threshold value γ. As a final step, we
estimate the final slope parameters associated with the regimes, which in turn yields β1 = β1 (γˆ ) for

I ( pit ≤ γˆ ) and βˆ2 (γ ) for I ( pit > γˆ ) .
171
EJAE 2019  16 (2)  155-173
APPENDIX 2
I. Correlation Matrix
Table I: Cross Correlation Matrix

Tobin's Q ROE ROA R&D Size Lev MI
Tobin's Q 1
ROE -0.034*** 1
ROA -0.192*** 0.463*** 1
R&D 0.151*** -0.186*** -0.453*** 1
Size -0.106*** 0.238*** 0.420*** 0.144*** 1
Lev 0.055*** 0.033*** -0.090*** 0.038*** 0.220*** 1
MI 0.023** 0.020* 0.017 0.046*** 0.097*** 0.023** 1
II .Definition of Variables
Table II. Variable Definition

Variable Type Variable Name Variable Explanation
Dependent Variable Tobin’s Q Market value over book value
ROE Return on common equity
ROA Return on total assets
Independent Variable R&D R&D expenditure/Sales turnover
Control Variable Size Log (Turnover)
Lev Capital structure: Debt/Equity
MI Marketing expenditure/Sales turnover
172
EJAE 2019  16 (2)  155-173
DA LI VELIKE FIRME OSETE ZNAČAJNIJI BOLJITAK OD INVESTICIJA U

VEZI SA ISTRAŽIVANJEM I RAZVOJEM?
Rezime:
U radu smo analizirali važnost veličine firme u odnosu u kojem se
nalaze istraživanje i razvoj, s jedne i učinak firme, s druge strane. Naša
empirijska analiza, zasnovana na podacima dobijenim iz kompanija
koje su deo popisa Nasdaq, u periodu 2002-2017. godina, pokazuju da
istraživanje i razvoj mogu da imaju uticaj na razlike u učinku firme,
u zavisnosti od same njene veličine. Kada ulaganje u istraživanje i
razvoj oslabi učinak firme, negativni efekti su više primetni u malim
firmama, ali kada ulaganje u istraživanje i razvoj ima pozitivan efekat,
koji dovodi do napretka u samom učinku firme, upravo iz navedenog
razloga, velike firme najviše osećaju sav boljitak ovog procesa. U vezi Ključne reči:
sa tim, potvrdili smo da veličina firme jeste važna kada je u pitanju
istraživanje i razvoj, učinak firme,
razumevanje stepena uticaja istraživanja i razvoja na sam učinak firme.
veličina firme
173
CIP - Каталогизација у публикацији
Народна библиотека Србије, Београд
33
The EUROPEAN Journal of Applied

Economics / editor-in-chief Nemanja Stanišić. -
Vol. 12, No. 2 (2015)- . - Belgrade : Singidunum
University, 2015- (Belgrade : Caligraph). - 28
cm
Polugodišnje. - Је наставак: Singidunum

Journal of Applied Sciences = ISSN 2217-8090
ISSN 2406-2588 = The European Journal of
Applied Economics
COBISS.SR-ID 214758924
Vol. 16 Nº 2
Vol. 16 Nº 2
OCTOBER 2019
Predicting the type of auditor opinion: The role of technology as an absorptive Ownership concentration and firm The rising government expenditure
Statistics, Machine learning, capacity in economic growth in emerging performance: An empirical analysis in Nigeria: Any influence on growth?
or a combination of the two? p. 1-58 economies: A new approach p. 59-78 in Oman p. 79-94 p. 95-108
What can we expect in the future of academic

Forecasting model of Vietnamese Causes of failure of the phillips curve: Do large firms benefit more from
research? The most common research
consumers’ purchase decision of does tranquillity of economic R&D investment?
problems analysed in the top journals in
2019
domestic apparel p. 109-121 environment matter? p. 139-154 p. 155-173

the field of entrepreneurship p. 122-138

The European Journal of Applied Economics - Vol. 16 #2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The European Journal of Applied Economics - Vol. 16 #2

Uploaded by

Copyright:

Available Formats

Vol.

What can we expect in the future of academic

w w w. j ou r na l.sing idunum.a c .rs

PREDICTING THE TYPE OF AUDITOR OPINION: STATISTICS,

THE ROLE OF TECHNOLOGY AS AN ABSORPTIVE CAPACITY IN

OWNERSHIP CONCENTRATION AND FIRM PERFORMANCE:

THE RISING GOVERNMENT EXPENDITURE IN NIGERIA:

FORECASTING MODEL OF VIETNAMESE CONSUMERS’ PURCHASE

WHAT CAN WE EXPECT IN THE FUTURE OF ACADEMIC

CAUSES OF FAILURE OF THE PHILLIPS CURVE: DOES

DO LARGE FIRMS BENEFIT MORE FROM R&D INVESTMENT?

PREDICTING THE TYPE OF AUDITOR OPINION: STATISTICS, MACHINE

Nemanja Stanišić, Tijana Radojević*, Nenad Stanić

Abstract: Article info:

Sample Factors Associated with Qualified/

173 qualified and

859 qualified and ◆ Poor credit rating

114 qualified and ◆ Large size (assets)

264 qualified and

980 qualified and K-nearest neigh- ◆ High credit risk

708 qualified and

78 qualified and ◆ Small size

MATERIALS AND METHODS

Absolute Frequencies by Period Relative Frequencies by Period

Absolute Frequencies by Period Relative Frequencies by Period

Net working capital Current assets012 - Current liabilities116

Method Software Implementation Tuning Parameters

n.trees (number of trees used in the prediction)

Model Validation Procedure

Figure 1. Sub-Setting the Data

Comparative Analysis of the Predictive Performance of the Individual Models

Table 6. Comparative Predictive Performance of the Twelve Predictive Models

xijt =α + ui + u j + β1 × C.50 probit + β 2 × RF probit + β 3 × RRF probit + β 4 × GBM probit

With Theory-Driven Predictors With Theory- and Data-Driven Predictors

Combining Feature-Selection Capabilities of Tree-Based Models with Mixed-Effects

Rule Modified Rule

 Liabilities for contributions on salaries and wages paid by the employee 

 Total cash inflow it 

This resulted in a model of the following specification:

Model Parameters Estimate Std. error z value Pr(>|z|)

Mixed-effects Logistic Rule 1 1.2357 0.1718 7.191 6.45e-13 ***

Mixed-Effects Logistic Regression 0.5961 0.3092

Summary of the Relative Performance of the Predictive Models

With Theory-Driven Predictors With Theory- and Data-Driven Predictors

Alternative Modeling Strategies Considered in This Study

0.8707 0.8656 0.8640

0.6879 0.3115 0.6737

APPENDIX A: OBSERVED FREQUENCY OF SPECIFIC TYPES OF AUDIT OPINION BY

41 275 136 411

49 271 142 413

M: Professional, 71 138 55 193

2010 2011 2012 2013

Z score for non-manufacturers and emerging

Z = -5.645, Z = -4.1546, Z = 1.988, Z = -0.61689, Z = 2.8133, Z = 3.7931, Z = -7.3954, Z = -9.5817, Z = -5.4511,

RF p-value = p-value = p-value = p-value = p-value < p-value = p-value = p-value =

APPENDIX D: COMPLETE OUTPUT OF THE HYBRID MODELS

Model Parameters Estimate Std. error z value Pr(>|z|)

Intercept -2.8638 0.2437 -11.752 < 2e-16 ***

APPENDIX E: ALTERNATIVE MODELING STRATEGIES

With Theory-Driven Predictors

Fixed effects logistic 0.5138 0.3242

With Theory-Driven Predictors and Dummy Variables

With 129 Principal Components Derived from Theory-

lnFDIcu 0.0315 -0.104* -0.0938*