LOS | on OPEN Q ACCESS Freely available online Obtaining Reliable Likelihood Ratio Tests from Simulated Likelihood Functions Laura Morch Andersen” Deparment of Food and Resouce Economics, Univesity of Copenhagen, Feerisbeg, Copenhagen, Denk Abstract ‘Mixed models: Modes llowing for continuous heterogeneity by assuming that value of one or more parameters follow a specified distribution have become increasingly popular. This is known as ‘mixing’ parameters, and itis standard practice by researchers - and the default option in many statistical programs - to base test statistics for mixed models an simulations using ‘asymmetric draws (eg. Halton draws}. Problem 1: Inconsistent LR tests due to asymmetric draws: This paper shows that when the estimated likelihood functions depend on standard deviations of mixed parameters this practice is very likely to cause misleading test results for the number of ‘draws usually used today. The paper illustrates that increasing the number of draws is a very inefficient solution strategy requiring very large numbers of draws to ensure against misleading test statistics, The main conclusion ofthis paper i that the ‘problem can be solved completely by using fully antithetic draws, and that using one dimensionally antithetic draws is not enough to solve the problem. Problem 2: Maintaining the correct dimensions when reducing the mixing distribution: second point ofthe paper s that even when fully antithetic draws are used, models reducing the dimension ofthe mixing distribution must replicate the relevant dimensions ofthe quasi-random draws in the simulation of the restricted likelihood. gain this isnot standard in research or statistical programs. The paper therefore recommends using fully antithetic draws replicating the relevant dimensions of the 4quasi-random draws inthe simulation of the restricted likelihood and that this should become the default option in statistical programs. JEL classification: C15; C25. Cation: Andersen LM (201) Obtaining ble Le thood Rao Tet rom Sd Lksshood Functions. Fos ONE 0) ¢ 106126, 0410.37 ourspane ator obo Rapa, Univers of Est dont. Nay Received November 5, 201% Accepted August Copyright: © 2014 Lau Morch Andersen. Tiss an open aces ante dsrbuted under the tems of the Creative Commons Aiuto cen, which Funding The research was unde by The Danch Soci Sclece Research Cone FS, The funder had no role in sty des, dt colton an ana, felson to pul, preparation of the manwscre Competing Interests: The author hes dered that no competing interests. * ema etoxwck Introduction Models allowing for continuous heterogenciey have been developing rapidly: shanks to advances in computational speed and underssanding of simulation methods for approxima integrals, (ee eg. [1] 2k Bh HI; [5] o€ (6), One way of introducing continuous heterogeneity is to assume that vale of ‘one oF more parameters follow a specified distribution. -nown as ‘mixing’ parameters, and the result of the estimation is the moments characterising the mixing disribucion rather than single valle of the mixed parameter. Caleulating the likelihood of 4 mixed model means that a conventional Hkelihoodd must be integrated over all possible values of the mixed parameters ‘weighted by the mixing density. Often, this integral does not have 4 close form and the integral is therefore approximated by ether Monte Carlo of Quasi-Monte Carlo integration, ‘This means that values of the mised parameter are drawn either randomly or quasirandomly from the underlying distibution and used 10 calculate the numerical integral which is then used as an proximation Many models are estimated by simulated maximum Hkelthood, and restrictions casi tested using Likelihood Ratio tests, As an ‘example, the Mixed! MuldNomial Logit (MMNL, []) nove a LOS ONE | wowplosoneorg standard way for researchers to introduce continuous heterogene ity into diserewe models, and Mixed MoliiNomial Logic is nos available in many different software packages [8]. compares the accuracy of Mixed MuliNomial Logit estimation in SAS, NLOGIT-LIMDEP and at wsersriten acld-in module for Stata They find it strange that both SAS and the Stata module allows the estimated standard deviations ofthe mixing distributions to be native, and that both packages ace the ser to reverse the sign in these cases, There i however nothing strange about the nogative values of the estimated standard deviations of the mising liributions. I'the mixed distribution depe ds on the vulie of variance, its common to maximize the likelihood value over the standard deviation which isthe square root of the vasianee. This relationship between the satiance and the standard deviation reas that the likelihood function will be symmetric around zero for the standard deviation, and the simulated likelihood must therefore also be symmetric around zero in this dimension, ‘The statistical software Biogeme ako sometimes ‘report negative standard deviations, and the results of estimations using Biogeme vary depending om the sgn of the staring valle for the standard deviations. AS will he ilstated inthe fellowing, this is problematic becanse standard deviations with identical absolut October 2014 | Volume 9 | sue 10 | 106136values should lead to the same likelihood value independent of the sign, and therefore also to the same result when maximizing the Tiklihood This paper lustrates that if the eonvensional Hkelihood is the simulated mixed likelihoods will always. be symmetric, but i dhe conventional likelihood is asymmetric this ‘snot tie unless the draws for the Quasi-Monte Cao integration are also symmetsic around zero, Antthetic draws. have. been smiggested a3 a variance reduction technique for Monte Carlo integration of asymmetric funetions, especially for Bayesian inference (see ea [9] oF (6). Hovsever, the technique i not generally sed in mixed models which ‘tho use Monte Carlo integration, and the simulated mised likelihoods are therefore usually not symmetric. As the number of draws increases the degree of symmetry will increase, but as it will abo be usted in the following, the degree of symmetry is wot always sulicent within the range of draws usually applied. One example of an asymmetric conventional likelihood is the likelihood of a logit ‘model, which will he used to ilstrate the problem in this paper. ‘Antithetic Halon draws have alo been suggested as an st for faster computation of Quasi Monte Carlo integrals, for mote precise point estimates within a reasonable time frame (recently in (10). In the present paper we focus on the ily of Likethood Ratio tess of mixed models instead of precision of point estimates or estimation speed. If Mixed MuliNomial Logit movels are estimated without antthetc dra, Likelihood Ratio esis may be compromised, whiel again may lead to fake conclusions. This paper illustrates why’ asymmestic trae are likely to lead to fbe Likelihood Ratio tess and why ntithetie draws solve this problem. The parpone of thi paper is support the idea of using filly antiheti' draws for simulated likelihoods, and not least to warn researchers who use standard ‘estimation procedures for e.g, Mixed! MultiNomial Lagi, that th Likelihood Ratio tests may be invalidated by the asymmetric nature of the draws Using data simulated under a Mixed MulkiNomial Logit model specification, we strate how asymmetry of the simulated Tkelihood fiction causes the Hkelihood to depend on the signs ind thatthe problem of inconsistent Likelihood Ratio test caused by the asymmetry of the (quasi-random draus is completely remaved when one uses fully antihetc draws insead of conventional asymmetric draus. Nowe that this solution only solves the problem of ‘lse” local maxima (here the parameters of the statistical model are uniquely ‘leniiied, but dhe maxim Ticelihond function varies) wl ‘eased by the asyrnmetry; it does NOT solve the problem of "re local maxim (where both Fikelihood value and parameter values vary) which ean occur as tesult ofa non-linear tility anton insufficient data. Wee also show thatthe same mechanisms appear in a real data set with invalidating implications for Likelihood of the estimated oles factorization Ratio tests, Having solved the problem of inconsistent Likelihood Ratio tests caused by asymmetric draws, the paper tums to another problem which could sil invalidate Likelihood Ratio tests, even ‘when antithetic draws are used. When restricting the numberof «mensions ofthe mixing distribution fram n to n-1 most standard procecares simply estimate the restricted model using the n-I fist et of quisé-random draws, imespective of which of the m original slimensions is restricted. As iustrated in this paper, keeping track of which dimension ik restricted, and removing exactly this dimension when estimating the resticted model, may lead 10 heer Likelihood Ratio tests The problem and solution presented in this paper not only apples to Mixed! MuliNomial Logit models, but also to other LOS ONE | wowplosoneorg ‘Obtaining Reliable UR Tests for Simulated Log Uelthoods ‘models estimated by maximum simulated likelihood! and the paper therefore provides a valuable contrition tothe ongoing srugsle to improve simulation methods, Haltom draws are used in this raper, but the properties ofthe antithetic draws car be generalized to other types of draws. The structure ofthe paper is ‘The above section introduced the problems associated ssith Likelihood Ratio tests performed on Simulated loglikelihood values, if these are simulated using asymmetric draws. The fllowing theory section consists of a subsection which ouilines the standard way of estimating and testing within panel Mixed MuliNomial Logit models, a subsection which explains why asymmetric draws used in Quasi Monte Carlo integration of mixed Helihoods may invalidate Likelihood Ratio tests, followed by wo subsections which first ilastrate the problem on simulated data by comparing rests in the true epini using conventional Halton dra ad then usin real data, The method! section introduces sntithetie Halton dws, The results section also contains two separate subsections, one which presents the encouraging results of using filly antthetc of draws and anokher which introduces dhe second problera which concerns tests reducing the dimension of the mixing distribution. The final section concludes on both problems, Theory [As mentioned above this section consist of Four subsections, ‘which intraduces the problem of inconsistent Hkelihood ration teste due to asymmetric dra, 1. Estimation and testing in Panel Mixed Logit Models In-a conventional logit (11) i is assumed that all individuals have the same usliy function, but in a Mixed MaltiNonial Logi (MMNI oF MXL) of Random Puramever Logit (RPL) model ((7), itis assumed that (part of) the individual tiiy is drawn from a tssibation, ‘Thi means chat che invita ality s know tthe individual, bt only the cistibution is assumed to be observable to the econometrician, The mixed likelihood fsnction i then the likelihood Rinetion of the conventional multinomial logit mol imegrated over all posible values of f whieh in a panel mixed logit becomes ([12): LO) heo-,([p where L(p) is the Bikeihood of dhe mise lot given the mixing stitution off given by 8, Lf) i the keibood for incvidal is the mumber of individuals, p(yf.x) is the likelihood of a conventional logit model given ancl /(f0) is the density of f given 0. The likelihood funetion is maximized over 0 Calculating the likelihood funesion in equation (1) is very ‘cumbersome, especially if follows a multivariate distribution, but the problem ean be reduced signiiantly by: simslating” the likelihood using either random draves (Monte-Carlo integration) or ‘quasi-random draws (Quasi Monte Carlo integration), In 2006, (13) suggested! an algoridun using random draws, but with an ‘execution time which was competitive with existing tools using ‘Quas-Monte Carlo for mixed logit models, bu in general Quas- Monte Carlo integration is found to he more eiient (see eg [19] Tor asymptotic properties of Quasi-Monte Carlo integration). This is often done sing quasérandom Halkon sequences which were first presented by [14] and [15]. The efficiency of Halton sequences compared to random draves is discussed in detail in both [16] and [17]. Both find that Halkon sequences greatly improve accuracy, allowing for far fewer draws and faster Wx BIO.) ny October 2014 | Volume 9 | sue 10 | 106136