Generating variable that indicates the observations that are the Nearest-Neoighbors

[FitzGerald Blindman]

Hello Everyone,

I want to estimate how the magnitude of the Child Penalty (according to Kleven et al. 2019: https://www.aeaweb.org/articles?id=10.1257/app.20180010) varies for parents to children with disabilities compared to the general population. Since parents to children with disabilities are different in their background characteristics compared to the general population, I used some kind of natural experiment, but my results are not very good for now.

So I thought to use matching to create "synthetic" control group for the parents for children with disabilities, with nearest-neighbor method. The trouble is that in this case I need to estimate the child penalty for every group separately, so I can't use the package

teffects psmatch

to regress, but I need commands that will generate new variable that will indicate me what observations are "neighbors" in my case.

For example, suppose the the variable called "treat" is binary indicator for whether the person has child with disabilities, so I regress:

logit treat mean_income_pre_birth age_at_first_birth number_of_children years_of_education, robust
predict propensity_score

Now I have the propensity score to being treated of the observations in my sample, but how can I create new variable that will indicate me what are the observations which gets 0 in the variable "treat" but would included in the regression if I would use something like

teffects psmatch (mean_income_after_birth) (treat mean_income_pre_birth age_at_first_birth number_of_children years_of_education, logit)

.

I hope it would be understood, because I'm really desperate for help.

Thank you already,

Fitzgerald.

[Felix Bittmann]

1. Why matching? What are your arguments for using this method? What about a standard regression design? Weighting?
2. I dont understand what exactly prevents you from using a standard method, like teffects nnmatch or kmatch? kmatch also offers a by option.
3. To find raw matches, you can use nnmatch as such:

sysuse nlsw88, clear
drop if missing(union)
teffects nnmatch (wage south hours age collgrad) (union), nn(1) gen(res)
list res1 in 1/10

[FitzGerald Blindman]

1. Why matching? What are your arguments for using this method? What about a standard regression design? Weighting?
2. I dont understand what exactly prevents you from using a standard method, like teffects nnmatch or kmatch? kmatch also offers a by option.
3. To find raw matches, you can use nnmatch as such:

sysuse nlsw88, clear
drop if missing(union)
teffects nnmatch (wage south hours age collgrad) (union), nn(1) gen(res)
list res1 in 1/10

Hello, thank you first of all.

1. I do it for the professor I work for him, he wants to create synthetic control group by nearest-neighbor method, I don't know what are hos arguments for using this methods.
2. I don't know what are the standard method, I thought that what are the forum for.
3. Can you please explain me exatcly what the next code does? I see it creates 44 new variables but I don't really understand what are their values mean.

Thank again,

Fitz

[Felix Bittmann]

I am not really sure why Stata creates up to 44 matches as I only specify 1 match, but res1 will contain each matched case, based on their order in the dataset. For example, case 1 (treated, union = 1) gets matched to case 1642 (untreated, union = 0). I understand that you just want these raw matches as specified in res1.

With standard methods I mean just to rely on an available command instead of computing raw matches and doing the rest manually. Have a look at kmatch, as it allows the direct computation of effects (such as ATE or ATT) by groups:

ssc install kmatch, replace
kmatch ps treatment control1 control2 (outcome), by(groupvar) comsup

[FitzGerald Blindman]

I am not really sure why Stata creates up to 44 matches as I only specify 1 match, but res1 will contain each matched case, based on their order in the dataset. For example, case 1 (treated, union = 1) gets matched to case 1642 (untreated, union = 0). I understand that you just want these raw matches as specified in res1.

With standard methods I mean just to rely on an available command instead of computing raw matches and doing the rest manually. Have a look at kmatch, as it allows the direct computation of effects (such as ATE or ATT) by groups:

ssc install kmatch, replace
kmatch ps treatment control1 control2 (outcome), by(groupvar) comsup

Thank you

So the first method (teffects nnmatch) just give me the number of the observation that matched, but it's not vary useful because I won't take observation after observation and mark them as the match group, right?

I don't use the standard method since what I want is identify the groups and than estimate the child penalty as Kleven et al. (2019) separately for every group.

I'm not really sure how to use kmatch since I don't have any outcome variable in this stage, I just need to understand hoe Stata creates variable that gives the value 1 for every observation that matched to the observations that gets the value 1 for the variable "treatment".

Thanks again,

Fitz

[Felix Bittmann]

I am still not sure what you really need to compute. In your posts you repeatedly state that you want to estimate the child penalty, so there is a (financial) outcome variable?

Regarding the nnmatch, this specification finds a match for every person in the dataset, regardless of how good the fit is. To adjust the degree of similarity, you can use the caliper option, which discards bad matches.

In any case, the 1:1 matching as currently specified is rather outdated and I would rather not use it. If you want to create a synthetic control group, reweighting is a convenient and robust approach. This also works without any outcome variable, it just makes two samples similar to each other. You can also achieve this in kmatch:

kmatch eb treatment control1 control2, wgen(w) comsup

The new variable w is a pweight that you can use in any analysis. You can also easily test that the control variables are now "matched":

reg treatment conf1
reg treatment conf1 [pweight=w]

In the second regression model, conf1 no longer predicts group assignment. I would argue, per definition, this has created a synthetic control group.

Now, if you problem are the various groups, you can estimate the command separately by group (just use the if qualifier).

[FitzGerald Blindman]

I am still not sure what you really need to compute. In your posts you repeatedly state that you want to estimate the child penalty, so there is a (financial) outcome variable?

Regarding the nnmatch, this specification finds a match for every person in the dataset, regardless of how good the fit is. To adjust the degree of similarity, you can use the caliper option, which discards bad matches.

In any case, the 1:1 matching as currently specified is rather outdated and I would rather not use it. If you want to create a synthetic control group, reweighting is a convenient and robust approach. This also works without any outcome variable, it just makes two samples similar to each other. You can also achieve this in kmatch:

kmatch eb treatment control1 control2, wgen(w) comsup

The new variable w is a pweight that you can use in any analysis. You can also easily test that the control variables are now "matched":

reg treatment conf1
reg treatment conf1 [pweight=w]

In the second regression model, conf1 no longer predicts group assignment. I would argue, per definition, this has created a synthetic control group.

Now, if you problem are the various groups, you can estimate the command separately by group (just use the if qualifier).

Hi,

The Child Penalty need to be estimated for each group separately, but leave it for now (if you are curious just read about the methodology in Kleven et al. 2019).

Just suppose I have dummy variable "treatment", and I want to know what are the observations that are the nearest-neighbor for every treated observation according to the propensity score estimated by

logit treatment var1 var2 var3 var4, robust

.
Is it so complicated in Stata to generate new variable (let's call him "matched") that will give the value 1 for every observation that matched to the treatment groups according to the propensity score?

I'm really thankful for you keep trying to help me since I'm really fell clueless.

Fitz

[Felix Bittmann]

I am not sure what you mean with:

...that will give the value 1 for every observation that matched to the treatment groups according to the propensity score?

Since every observation has a match (regardless of how good this is). I am not sure if there is a misunderstanding: matching occurs on the individual (case) level but does not automatically created matched samples (e.g. two groups that are, on average, similar to each other).

Lets look at an example. We have a binary treatment group (union). We have a list of controls we want to match. Manually, we do it like so:

. sysuse nlsw88, clear
(NLSW, 1988 extract)

. drop if missing(union)
(368 observations deleted)

. gen id = _n

. logit union age hours ttl_exp

Iteration 0:  Log likelihood = -1046.3424  
Iteration 1:  Log likelihood = -1041.2122  
Iteration 2:  Log likelihood = -1041.1954  
Iteration 3:  Log likelihood = -1041.1954  

Logistic regression                                     Number of obs =  1,877
                                                        LR chi2(3)    =  10.29
                                                        Prob > chi2   = 0.0162
Log likelihood = -1041.1954                             Pseudo R2     = 0.0049

------------------------------------------------------------------------------
       union | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         age |    .005397   .0179123     0.30   0.763    -.0297105    .0405044
       hours |   .0123674   .0057563     2.15   0.032     .0010853    .0236495
     ttl_exp |   .0211837   .0122403     1.73   0.084    -.0028068    .0451742
       _cons |  -2.078382   .7341779    -2.83   0.005    -3.517344   -.6394197
------------------------------------------------------------------------------

. predict pr
(option pr assumed; Pr(union))
(1 missing value generated)

. sort pr

. list pr id union age hours ttl_exp in 1/20

     +-----------------------------------------------------+
     |       pr     id      union   age   hours    ttl_exp |
     |-----------------------------------------------------|
  1. | .1425388    690   Nonunion    35       7   .4038461 |
  2. | .1451887    876   Nonunion    36       3        3.5 |
  3. | .1470532    731   Nonunion    37       7   1.615385 |
  4. | .1480791   1083   Nonunion    35       1   6.012821 |
  5. | .1495079    720      Union    35       2   5.961538 |
     |-----------------------------------------------------|
  6. | .1508445   1519   Nonunion    39       4   4.269231 |
  7. | .1544073   1471   Nonunion    35       2   7.756411 |
  8. | .1547262    404   Nonunion    36       5   5.865385 |
  9. | .1559999   1069   Nonunion    35       8   4.826923 |
 10. | .1563334   1506      Union    37      10   3.269231 |
     |-----------------------------------------------------|
 11. | .1570136    579   Nonunion    35       2   8.692307 |
 12. | .1580351    411   Nonunion    38       8   4.788462 |
 13. | .1582624    980   Nonunion    34       2   9.391025 |
 14. | .1601641   1812   Nonunion    36      17   .7948718 |
 15. |  .160888    721   Nonunion    42       5   6.525641 |
     |-----------------------------------------------------|
 16. | .1616411    453   Nonunion    42       5   6.788462 |
 17. | .1622283    746   Nonunion    44      10   3.564103 |
 18. | .1624757    142   Nonunion    40      15       1.75 |
 19. | .1627927   1183   Nonunion    38      10   5.288462 |
 20. | .1635198   1532   Nonunion    35      16   2.801282 |
     +-----------------------------------------------------+

As you see, observation 5 is treated, observations 4 and 6 are untreated. Their PS is extremely similar to the treated observation, so we know these two cases are very similar with respect to the controls. We can verify that the absolute difference in PS between cases 5 and 6 is the smallest, so these two should be matched (note that this match is bidirectional, 5 to 6 and 6 to 5). Of course, we do not want to do this manually for all cases, so we can use this command (ignore the outcome var wage here, it does not influence our matching):

. teffects psmatch (wage) (union age hours ttl_exp), nn(1) gen(res)

Treatment-effects estimation                   Number of obs      =      1,877
Estimator      : propensity-score matching     Matches: requested =          1
Outcome model  : matching                                     min =          1
Treatment model: logit                                        max =          2
--------------------------------------------------------------------------------------
                     |              AI robust
                wage | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
---------------------+----------------------------------------------------------------
ATE                  |
               union |
(Union vs Nonunion)  |   1.306107   .2451396     5.33   0.000     .8256427    1.786572
--------------------------------------------------------------------------------------

. sort pr

. list pr id union age hours ttl_exp res1 in 1/20

     +------------------------------------------------------------+
     |       pr     id      union   age   hours    ttl_exp   res1 |
     |------------------------------------------------------------|
  1. | .1425388    690   Nonunion    35       7   .4038461      5 |
  2. | .1451887    876   Nonunion    36       3        3.5      5 |
  3. | .1470532    731   Nonunion    37       7   1.615385      5 |
  4. | .1480791   1083   Nonunion    35       1   6.012821      5 |
  5. | .1495079    720      Union    35       2   5.961538      6 |
     |------------------------------------------------------------|
  6. | .1508445   1519   Nonunion    39       4   4.269231      5 |
  7. | .1544073   1471   Nonunion    35       2   7.756411     10 |
  8. | .1547262    404   Nonunion    36       5   5.865385     10 |
  9. | .1559999   1069   Nonunion    35       8   4.826923     10 |
 10. | .1563334   1506      Union    37      10   3.269231      9 |
     |------------------------------------------------------------|
 11. | .1570136    579   Nonunion    35       2   8.692307     10 |
 12. | .1580351    411   Nonunion    38       8   4.788462     10 |
 13. | .1582624    980   Nonunion    34       2   9.391025     10 |
 14. | .1601641   1812   Nonunion    36      17   .7948718     10 |
 15. |  .160888    721   Nonunion    42       5   6.525641     10 |
     |------------------------------------------------------------|
 16. | .1616411    453   Nonunion    42       5   6.788462     10 |
 17. | .1622283    746   Nonunion    44      10   3.564103     10 |
 18. | .1624757    142   Nonunion    40      15       1.75     29 |
 19. | .1627927   1183   Nonunion    38      10   5.288462     29 |
 20. | .1635198   1532   Nonunion    35      16   2.801282     29 |
     +------------------------------------------------------------

And indeed, we see that Stata has matched observation 5 (with id 720) to observation 6. Maybe this is only the first step in the algorithm used in the article you cite, but then we require more information on the concrete further steps. I did not find this directly (and as this is an economic paper, I am probably not familiar with it at all).
.

[FitzGerald Blindman]

Thank you Felix.

Ignore the paper I cited, it's nor relevant for my question.

Now what I want is that observation 6 (the nonunion observation that matched to the union observation 5) will receive the value 1 in new variable (let's call it "match", and as well as observation 9 that matched to observation 10, and so on, so I will be able to command:

keep if treat == 1 | match ==1

And I'll have sample of treatment group and "control" group created by the nearest-neighbor matching method.

Is it possible in Stata?

Thank you again,

Fitzgerlad

[Felix Bittmann]

I think, for this to work you need 1:1 matching without replacement. This can be done in kmatch:

kmatch ps union age hours ttl_exp, nn(1) wor idgenerate(match) idvar(idcode) comsup
drop if missing(match1)
count if union == 0
count if union == 1

[FitzGerald Blindman]

I think, for this to work you need 1:1 matching without replacement. This can be done in kmatch:

kmatch ps union age hours ttl_exp, nn(1) wor idgenerate(match) idvar(idcode) comsup
drop if missing(match1)
count if union == 0
count if union == 1

Hey Felix,

It isn't have to 1:1 matching, it's enough for me just to know what observations matched so I can later use them as two separate samples.

Fitz