Monday, December 9, 2013

To Cluster or Not to Cluster - That is the Question

Several very prominent econometricians have fallen on either side of this
question, typically they are concerned that the number of clusters is
small and the size in each cluster is large.  One camp argues that the
only reasonable way of deriving consistent standard errors is through
clustering while the alternative camp argues that clustering does not
have good small sample properties.

In this post I will explore how clustering works given these different
conditions.

cap program drop cluster_se

program define cluster_se, rclass
syntax [anything], NGrp(numlist >0 integer) NInd(numlist >0 integer)
  
  clear
  set obs `ngrp'

  * In order to examine clustering we will structure the error
  * in one of the most basic forms possible.  There is a different
  * mean unobservable for each group and a different mean x
  * for each group.  There is no correlation between x and v.
  gen x=rnormal()
  gen v=rnormal()
  gen id=_n

  * Duplicate the data by number of individuals in each group
  expand `nind'

  * u is a function of both the group mean error and the individual
  * observation
  gen u=v+rnormal()

  * In order to test if our standard error are working properly
  * let's look at the rate of false rejections given that there
  * is no true relationship between x and y.
  gen y=0*x+u
  
  * Basic regression
  reg y x
  return scalar noclus_se=_se[x]
  
  * Grab the p values
  test x=0
  return scalar noclus_r=`=r(p)<.05'

  * Now cluster across our groups
  reg y x, cluster(id)
  return scalar clus_se=_se[x]
  
  test x=0
  return scalar clus_r=`=r(p)<.05'

end

* First let's see how our new command works
cluster_se , ng(100) ni(100)
return list

simulate clus_r = r(clus_r) clus_se = r(clus_se)  /// 
       noclus_r = r(noclus_r) noclus_se = r(noclus_se),  /// 
    rep(100): cluster_se , ng(100) ni(100)

sum
* We can see by clustering we get the correct rejection rate
* while by failing to cluster we reject at ~74% of the time.
* Notice also the standard deviation of the standard error
* is much larger for clustered standard errors.

* Now let's see what happens if we decrease the number of
* groups.

simulate clus_r = r(clus_r) clus_se = r(clus_se)  /// 
       noclus_r = r(noclus_r) noclus_se = r(noclus_se),  /// 
    rep(100): cluster_se , ng(10) ni(100)

sum
* We can see now our clustered standard errors are 
* overrejecting though they are still much better that
* the non-clustered.

* Let's try something even more extreme.
simulate clus_r = r(clus_r) clus_se = r(clus_se)  /// 
       noclus_r = r(noclus_r) noclus_se = r(noclus_se),  /// 
    rep(100): cluster_se , ng(3) ni(200)

sum
* Now the clustered standard errors are working even worse
* though they are performing still much better than the
* non-clustered standard error.

simulate clus_r = r(clus_r) clus_se = r(clus_se)  /// 
       noclus_r = r(noclus_r) noclus_se = r(noclus_se),  /// 
    rep(100): cluster_se , ng(3) ni(20)

sum
* Clustering is still giving us better rejection rates though
* neither method is working particularly well at this point.

* From this simple simulation I would be inclined to argue
* that it is always reccomended to cluster when possible.

Formatted By Econometrics by Simulation

No comments:

Post a Comment