You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
hi @wxwx1993 and @Naeemkh,
post-matching when running erf parametric i think we need to change code so that we adjust for counter using GEE, otherwise we are underestimating the standard error (this came up with michelle q and falco, for SGI project).
I think Danielle and Falco understand this statistical issue better than me, but my impression is that it centers around how we understand the counter_weight variable in the data structure. Since counter_weight records how many times an observation should be duplicated, it may need to be accounted for explicitly in a statistical outcome model, such as through a "clustering" structure in a Generalized Estimating Equation (GEE).
In the SGI project, we expanded out the data (I forget what this is called) using counter_weight to repeat each row a counter_weight number of times. Then, a GEE model was run with row_index as the "cluster ID".
But let's start simply. that is 1) if using a GLM in R, and set weight = counter_weight, I thought the GLM wouldn't treat two identical units independently, rather, they will use a weighted LS and corresponding weighted covariance matrix to calculate the variance. I do not think in this case, GEE will make difference. 2) If using a GLM in R on unweighted data (with many duplicated rows. and GLM will wrongly treat them independently), then a GEE is necessary.
The paper that @wxwx1993 linked looks cool; it found that (for binary treatments), "We have shown that sandwich standard errors are not generally valid for the post-matching least squares estimator. In this section, we will demonstrate that, when matching is done without replacement, clustered standard errors (Liang and Zeger 1986; Arellano 1987) can be employed to obtain valid estimates of the standard deviation of post-matching regression coefficients. ... In this [later] section, we show that a clustered version of the nonparametric [block] bootstrap (Efron 1979) is also valid."
@wxwx1993 For our SGI project, we did get different results using 1) weighted GLM using counter_weight and 2) GEE on the dataset containing duplicate rows. (Falco and Danielle were more involved in this step than I was, so they may be able to expound.)
Activity
Naeemkh commentedon Feb 22, 2023
Please add more details to this issue. Thanks.
Naeemkh commentedon Apr 10, 2023
@daniellebraun, @m-qin, @fbargaglistoffi, please add more details to this issue.
m-qin commentedon Apr 10, 2023
I think Danielle and Falco understand this statistical issue better than me, but my impression is that it centers around how we understand the
counter_weightvariable in the data structure. Sincecounter_weightrecords how many times an observation should be duplicated, it may need to be accounted for explicitly in a statistical outcome model, such as through a "clustering" structure in a Generalized Estimating Equation (GEE).In the SGI project, we expanded out the data (I forget what this is called) using
counter_weightto repeat each row acounter_weightnumber of times. Then, a GEE model was run withrow_indexas the "cluster ID".wxwx1993 commentedon Apr 10, 2023
The variances estimates of the matching estimator are of interest. Importantly, a paper under binary treatment (https://www.tandfonline.com/doi/full/10.1080/01621459.2020.1840383) may illustrate some points raised here.
But let's start simply. that is 1) if using a GLM in R, and set weight = counter_weight, I thought the GLM wouldn't treat two identical units independently, rather, they will use a weighted LS and corresponding weighted covariance matrix to calculate the variance. I do not think in this case, GEE will make difference. 2) If using a GLM in R on unweighted data (with many duplicated rows. and GLM will wrongly treat them independently), then a GEE is necessary.
wxwx1993 commentedon Apr 10, 2023
If someone wants to write a theoretical paper on variance estimation of matching estimators, I am up for it.
m-qin commentedon Apr 10, 2023
The paper that @wxwx1993 linked looks cool; it found that (for binary treatments), "We have shown that sandwich standard errors are not generally valid for the post-matching least squares estimator. In this section, we will demonstrate that, when matching is done without replacement, clustered standard errors (Liang and Zeger 1986; Arellano 1987) can be employed to obtain valid estimates of the standard deviation of post-matching regression coefficients. ... In this [later] section, we show that a clustered version of the nonparametric [block] bootstrap (Efron 1979) is also valid."
m-qin commentedon Apr 10, 2023
@Naeemkh maybe this thread should be under Discussion rather than Issue, haha. Seems like the methodology is still being developed
Naeemkh commentedon Apr 10, 2023
Thanks, @wxwx1993 and @m-qin. I will move it under discussions.
m-qin commentedon Apr 10, 2023
@wxwx1993 For our SGI project, we did get different results using 1) weighted GLM using
counter_weightand 2) GEE on the dataset containing duplicate rows. (Falco and Danielle were more involved in this step than I was, so they may be able to expound.)