K.V. Mardia, H.R. Southworth, C.C.

Taylor,
On bias in maximum likelihood estimators,
Journal of Statistical Planning and Inference,
Volume 76, Issues 1–2,
1999,
Pages 31-39,
ISSN 0378-3758,
https://doi.org/10.1016/S0378-3758(98)00176-1.
(https://www.sciencedirect.com/science/article/pii/S0378375898001761)
Abstract: It is well known that maximum likelihood estimators are often biased, and
it is of use to estimate the expected bias so that we can reduce the mean square
errors of our parameter estimates. Expressions for estimating the bias in maximum
likelihood estimates have been given by Cox and Hinkley (1974), (Theoretical
Statistics, Chapman & Hall, London). In this paper, a new simple expression is
derived for estimating the first-order bias in maximum likelihood estimates. This
is then applied to certain specific models; namely, estimating the bias in the
concentration parameter in the von Mises–Fisher distribution – for which there are
some ad hoc results in the literature (Best and Fisher, 1981, Comm. Statist. Ser.
B. Simulation Comput. 31, 493–502) – and for the scale parameter in the Cauchy
distribution. In both problems, the first-order bias is found to be linear in the
parameter and the sample size. Simulations are used to verify our theoretical
results and to give some indication of how large the parameter and the sample size
need to be for our estimates of bias to hold well.
Keywords: Fisher; von Mises; Cauchy distribution; Score function