HIGHER SCHOOL OF ECONOMICS

ADVANCED ECONOMETRICS 2019-2020

REPORT III
Topic: Ranking and evaluation of banks in Russia 2017-2018

Team 5
Cai qitan⋅ Mohammed Yunnuss⋅ Pham Thu Huong⋅ Sigde Shakti⋅ Siglovoy Evgeny⋅ Vu Thi Phuong

Moscow, 2020
CONTENT
I. DATA PREPARATION
1. Theoretical Background and Hypotheses
2. Data
II. BINARY CHOICE MODEL
III. ORDERED RESPONSE MODEL

The contribution of each student in a team: 1/6

I. DATA PREPARATION
1. Theoretical Background and Hypotheses
The ratings reflect the Bank’s high return on equity, capital adequacy and strong liquidity position.
Every operating bank wants to achieve the highest rating. It reflects the potential, prospects, prestige and
development of a bank that must be achieved. In the this report, we are provided with Total Assets, Bank’s
Capital, Credit Portfolio and Net Income to to evaluate the rating of the bank. The assets are the financial
instruments that either the bank is holding (its reserves) or those instruments where other parties owe
money to the bank. Bank capital is a storage of cash and safe assets that banks hold as a buffer. Net
income, also called net profit, is a calculation that measures the amount of total revenues that exceed total
expenses. And a credit portfolio is an investment portfolio comprised of debts, like home and car loans.
Since the experts consider possible upgrading of the ratings, provided that the Bank continues to execute its
strategy of balanced organic growth.
In our report, we try to test following hypothesis: Bank rating is positively related to assets, bank
capital, net income.
2. Data
The dataset contains 135 bank financial factors sample for rank, total asset, bank capital, net income
and credit portfolio during 2017 to 2018. 33 of these banks have A; 99 banks have B and only 3 banks have
C. more than 50 % of the banks in the dataset are located in Moscow or Moscow region. And 8% are in
Saint Petersburg and Leningradskaya region. The other regions have less than 4% taken individually.
To investigate the data, we have generated variable Ranking that takes value of “1” if the rating is A
and higher. In other instances, the variable takes “0” if the bank has a lower ranking value. The procedure
allows us to set a binary choice model. And we also divided all the number by 10000 for normalization.
We suggest that the rating of 2018 year has an association with the amount of assets, income, bank capital.
Our expectations are that the amount of assets, income and bank’s capital are positively associated with
rating. Moreover, in order to study and compare more clearly the influence of factors on bank ranking, we
have added variables on assets growth, bank capital growth, income growth and credit portfolio growth to
estimate and choose the best model.
By looking on the scatterplot of the amount of assets, we
may suspect up to two outliers. These are the largest
banks in the sample: VTB and Azprombank. Our
solution to these outliers is dropping them. We also
notice that it doesn’t make sense that if bank capital is
negative, therefore we drop those banks which contain
negative capital.
II. BINARY CHOICE MODEL
1. Results of linear probability models by ordinary least squares (Table 1)
Ranking is a dependent variable.
Table 2.1:

We believe that the best model to choose is LPM1. Why this is not really an appropriate model:
- If we use model LPM1 (or any other model from table 1) to forecast if default-like event happens,
we cannot be sure that forecasted probability will be between 0 and 1. And if probability is higher than
100% or negative, it does not make sense. And in case of LPM1 there are predicted values that are higher
than 1.
- Distribution of the error is supposed to be normal to apply OLS. However, for binary models’
errors can be 1 or 0, as Ranking can be 1 or 0. Therefore, errors are not normally distributed. It means that
results of F-test and t-test are not informative.
2. Coefficients in the linear probability models are interpreted as change in probability that Ranking equals
1 (if everything else constant). Therefore, for LPM1: if bank capital in 2018 increases by 1 unit, probability
of Ranking being 1 increases by 1,184*10-4. If net income of a bank in 2018 increases by 1 unit, probability
of Ranking being 1 decrease by 4,933*10-5. If credit portfolio of a bank in 2018 increases by 1 unit,
probability of Ranking being 1 decrease by 2,825*10-5. If assets in 2018 increases by 1 unit, probability of
Ranking being 1 increases by 6,970*10-6.
3. Marginal effects equal estimated coefficients. Marginal effect = estimated coefficient*change in
independent variable, or if logarithms are used: estimated coefficient*(ln (new independent variable)-
ln(initial independent variable)).
4. Logit&Probit models’ estimation results (in the Table2. 2 and Table 2.3)
Table 2.2: Table 2.3:
5.
Table 2.4.
The Logit and Probit models are not very
different: all coefficients have the same signs;
they correspondently (Logit1&Pribit1; ...;
Logit5&Probit5) predict correctly almost the
same amount of values at 50% cutoff.
According to the criteria, the Probit model is
the best model. We choose Probit for further
research.

6. Difference between the linear probability model and Logit (or Probit) model:
 Coefficients in these 2 models have different interpretations. In Linear model they are marginal
effects, but in Probit model, marginal effects need to be calculated.
 Probit model never gives values for probabilities more than 1 and/or lower than 0, whereas Linear
model does.
 In Linear model, the independent variable can be correlated with each other. On the contrary, in
Probit model, the variable must not be correlated with each other.
 In Probit model the standard normal density function is used, whereas in the other model it is a
linear combination of constant and independent variables.
  Linear model tends to overestimate/underestimate probabilities with extreme values of
independent variable. Probabilities are not smoothed.
7. Test our model against the model with only intercept.
Loglikelihood value for unrestricted model: lnL = -31,592
Loglikelihood value for constrained model: lnL(0) = -71,949
pseudo-R2= 1-lnL/lnL(0) = 0,561
H0: the explanatory variables are all jointly equal to 0.
Likelihood ratio test statistic: LR = 2*(-31,592+71,949) = 80,714
Chi2(5) = 24,72 (at 5% level of significance). Thus, the null hypothesis is rejected.
Wald test shows the same result: P-value = 0,0002. We can conclude that this model is significant.
8. Test the significance of the chosen explanatory variables
P-value
const*** 0,000
NA_BC18 0,163
NA_NI18 0,073
NA_CP18 0,847
A18 0,325
moscow 0,586
9. Plot the ROC curve. Assess the quality of our model in terms of specificity and sensitivity
Area under ROC curve = 0.9390
If cutoff=50%:
Sensitivity=70,97%
Specificity=98,02%

10. Also, correctly classified criteria can be informative in terms of comparing models. At 50% cutoff for
model Probit1 it equals 91,67% (sensitivity=70,97%; specificity=98,2%); for model Probit1mod: 90,91%
(sensitivity=70,97%; specificity=97,03%). Therefore, Probit1 predicts better what banks will get rating
higher than A, and Probit1mod is better at classifying banks with lower rating.
Other criteria:
Probit1 Probit1mod
AIC 75,18 79,25
BIC 92,48 70,65
LR 80,71 79,30
 According to LR test Probit1mod is not nested in Probit1.
11. Modification of the model
According to pseudo R2 non-modified model
is better, which is expected.
However, according to AIC and BIC, the
modified model is better. Also, joint
significance test confirms that these two
variables are not needed:
chi2 = 0,8439 vs chi2(3, at 5%) = 0,34.

12. We believe that in 2018 net income (NA_NI18) is the most influential variable among the rest of
variables in this analysis. Its coefficient is positive and the largest, therefore, NA_NI18 has a positive
impact on bank’s rating. If averaged bank capital increases, probability that bank will get a rating that is
higher than A increases (if another factors constant).
13. Null hypothesis: the effect of NA_NI18 is not quadratic. P-value=0,002. The null hypothesis is
rejected. The effect is quadratic.
14. Is effect of NA_NI18 the same for two groups: banks of Moscow and Moscow region and all the rest?
According to LR test the effect is different.
15. Calculate
Marginal effects of a bank with average characteristics: NA_BC18: 2,87*10-4 ; NA_NI18: 6,44*10-4
Average marginal effects: AV_BC: 1,124*10-4 ; AV_NI: 3,913*10-4
16. To conclude, the most influential factor in forming rating is net income. On average, the higher net
income the more credible bank. Assets, bank capital, credit portfolio and net income are strongly correlated
and on practice they are interconnected, therefore, it should be considered during analysis. Based on two
periods it is hard to summarize something about dynamic. However, bearing in mind that 2018 data have
negative effect, we can say that if, for example, credit portfolio grows in more extent it will give greater
probability of being ranked as A and higher.

III. ORDERED RESPONSE MODEL

Ranking Banks. As we could see from the data, all banks are grouped into three categories: A, B and C.
Due to this, we use the variable ranking_o to generate an ordered variable in the following way: 0 - if
bank is estimated as “C”, 1 – if bank is estimated as “B” and 2 – if estimated as “A”.
1. Estimate the model. We constructed 4 models: probit and logit with all variables and with only absolute
non-dummy variables of 2018. It shows in Table 3.1
Table 3.1

We chose the probit2 model, as it has the lowest value of the Schwartz criterion. So, as we could
see from the model, it is significant, and the following coefficients are valuable: net income 2018 and bank
capital in 2018. As we could see, the probability of a bank to be C-ranked is when the density function is
less or equal than -1,66, the probability that bank is ranked as B is the difference of density function
between the points of 1,92 and -1,66. And the probability of a bank to be A-ranked is the difference
between 1 and other probabilities.
2. The difference between logit and probit models. As we could see from the table above, there is not
much difference between these models. But, because of the reasons, described above, we have chosen
probit model.
3. Testing the significance of each coefficient. Table 3.2
In the table 3.2 shows that no variables are
significant on the 5% level and two – on the 10%
level. We tested the significance of other variables,
and their p-value exceeded the 10% level, so they are
not significant.
4. Testing our model against the model with only intercept. We would test our chosen model against the
model with the only intercept.
Table 3.3

As we could see in the tables 3.3, the model with only intercept is worse, as its information criteria
are bigger than in the model, that we chose. So, we chose the probit2 model is the best model.
5. Modifying the model. LM-test. After testing our model against the model with only intercept, we will
modify our probit model by leaving only that regressors, that are significant on the 10% level. We also use
the Lagrange multiplier test to see, whether these models are significantly different or not.
Table 3.4

In the table 3.4 shows that the model probit4 is better than probit2 by the information criteria and by
the fact that one coefficient in the model probit4 is now significant on the 5% level. However, from the
Lagrange multiplier test we could see, that the models are pretty the same, that could be explained by the
fact, that we chose only significant coefficients.
6. Choosing the most significant variable. As for the variable, we choose the one that is significant on the
5% level in the modified model (probit4): bank capital in 2018. The table 3.4 shows us that the increase of
this variable by one increases the probability of bank to be A-ranked on the 0,00096.
7. Calculating marginal effects. We calculated the average marginal effects and the marginal effect of the
average bank.
Table 3.5

From this table we could see, that the marginal effects of this variable on the average bank are
negative in case of B and C ranking and positive in case of A and all effects are insignificant. Also, the
average marginal effects are insignificant and negative, except the average marginal effect in case of A-
ranking banks, so the bank capital plays significant and positive role in the evaluation of good banks.
8. Making a conclusion of variables’ influence. And in conclusion we should say, what is the influence
of different variables on the probability of bank to increase its rank. From the table 3.1 we could see, that
bank capital, bank assets, net income and growth of bank capital and net income positively influence in the
probability of bank to be A-ranked, whether other variables: credit portfolio, growth of net income and
growth of credit portfolio influences negatively an such probability. And only two coefficients are
significant: bank capital and assets, so we could say, that on the basis of our model, the rating agencies
prefer these variables to others in our model, while evaluating banks.
.