Thursday, January 10, 2013
Efficiency of LAD vs OLS
# The following post follows class notes from Panel Data Analysis II by Jeffrey Wooldridge (though all errors are mine)
# When deciding on the best method to estimate a coefficient, LAD and OLS are two commonly considered.
# Often times these estimators are theoretically estimating the same coefficient.
# That is, in the linear relationship y=xB + u with u~symmetrically distributed and independent of x with E(u|x)=0 and med(u|x)=0, both OLS and LAD are consistent estimators of B.
# However, in terms of efficiency, the two estimators individual efficiency is variable.
# Assuming u is independent of x, the asymptotic variance of the LAD estimator (n^.5 * (B[LAD]-B))= is 1/(4*f(0)^2)*E(x'x)^-1
# with f(0) being the pdf of the error distribution at the quantile being estimated (median 0).
# While the asymptotic variance of OLS is (n^.5 * (B[OLS]-B))=sigma2*E(x'x)^-1
# with sigma2 being the variance of the error distribution.
# Now we can evaluate the differences in asymptotic variances given difference distributions of errors.
# For instance, assume the underlying distribution is standard normal
# Yields 1.57. This indicates that LAD is 57% less efficient than OLS when the errors are distributed normally.
# On the other hand. If we had a distribution of errors v = exp(|u|)*(2-|u|)*sign(u) with u~normally.
N = 1000000
u = rnorm(N)
v = exp(abs(u))*(2-abs(u))*sign(u)
# We can estimate the variance of the errors by sampling.
# Is estimated around 12.7
# To calculate the density at the median 0 we can sum across the pdf values of u in which v = 0.
# Which is u=2 and u=-2. Because the normal is symmetric:
# Which is about .108
# So avar(lad) = (X'X)^-1 *
# Which is about 2.315*(X'X)^-1
# As opposed to the OLS which is around 12.7*(X'X)^-1
# Thus the LAD estimator is about 6 times more effective than the OLS estimator when the error term is distributed in this manner.
# A leading example of when the LAD estimator is preferred to the OLS estimator is when the errors are distributed double exponential f(u)=1/2k * exp(-|u|/k)
# The variance(u) = 2*k^2 making the avar = 2*k^2*(x'x)^-1 while the density at 0 is 1/2k which makes the avar = k^2*(x'x)^-1 making LAD twice as efficient as OLS. Of course despite this special cases, we know that most data tends to look more normal than fat tailed making OLS preferable to LAD.