The Delta method to estimate standard errors from a non-linear transformation

* Stata do file

* The Delta method can be used to estimate the standard errors after a regression estimation.

* Imagine you have some parameter G = (3*b0-b1)*b2^2 = 3*b0*b2^2-b1*b2^2

* Where y = bo + b1*x1 + b2*x2 + u

* The delta method can be used to estimate the standard errors of G.

* The delta method states that var_hat(G)=(dG/db) var(b) (dG/db)

* dG/db is a gradient vector:

* dG/db = [dG/db0, dG/db1, dG/db2]
* dG/db = [b2^2, -b2^2, 2*(b0-b1)*b2]

* var_hat(G) = (3*b2^2)^2 * se(b0)^-2 + (-b2^2)^2 * se(b1)^-2 + (2*(b0-b1)*b2)^2 * se(b2)^-2

[There is an error in the code because I failed to include a covariance term for the coefficients.  Please see the more recent update on the method.]

clear
set obs 1000

gen x1 = rnormal()
gen x2 = rnormal() * 4

global b0 = 1
global b1 = 1.5
global b2 = .3

local true_G = (3*${b0}-${b1})*${b2}^2

di `true_G'

gen y = ${b0} + ${b1}*x1 + ${b2}*x2 + rnormal()*8

reg y x1 x2

* G = (3*b0-b1)*b2^2 = 3*b0*b2^2  - b1*b2^2
local Ghat = (3*_b[_cons]-_b[x1])*_b[x2]^2

di "Ghat = `Ghat' is our estimate (true = `true_G')"

* Let's see if we can't use the delta method to derive a standard error.
local var_hatG = (3*_b[x2]^2)^2 * _se[_cons]^2 + (-_b[x2]^2)^2 * _se[x1]^2 + (2*(_b[_cons]-_b[x1])*_b[x2])^2 * _se[x2]^2

di "Standard error estimate is " `var_hatG'^.5

* Alternatively, let us attempt to bootstrap our standard errors.

cap program drop deltaOLS
program define deltaOLS, rclass
  reg y x1 x2
  return scalar Ghat = (3*_b[_cons]-_b[x1])*_b[x2]^2
end

bs Ghat=r(Ghat), rep(500): deltaOLS
* The bootstrap standard errors are similar to that of the delta method's standard errors.

cap program drop deltaMonteCarlo
program define deltaMonteCarlo, rclass
  clear
  set obs 1000

  gen x1 = rnormal()
  gen x2 = rnormal() * 4

  gen y = ${b0} + ${b1}*x1 + ${b2}*x2 + rnormal()*8

  reg y x1 x2

  return scalar Ghat = (3*_b[_cons]-_b[x1])*_b[x2]^2
end

simulate Ghat=r(Ghat), reps(500): deltaMonteCarlo
sum
* We can see that our estimates of the standard error via the Delta method and bootstap is quite close to the monte carlo estimates on observed standard errors from 500 replications.