Problem with "osample" from "teffects nnmatch"

[Juan Hernandez]

Dear Statalist,

I am sending this question again, because I didn't get an answer last time I wrote: My team and I are running a propensity score matching analysis on agricultural data in order to assess the impact of credit in several farmer outcomes. For this, we are using the command teffects nnmatch, that allows to perform the matching exactly on the observable covariates, which is useful for categorical variables for example.

Anyways, we are now getting an error after running the command, which is similar to the one of this thread:http://www.statalist.org/forums/foru...match-question

We first run a line of code that looks like the following:

teffects nnmatch (dep_variable x_covariate) (treatment)

which gives the following error message:

no exact matches for observation 23472; use option osample() to identify all observations with deficient matches

We then include the option osample:

teffects nnmatch (dep_variable x_covariate) (treatment) , osample(newvar)

.

After Stata identifies the observations for which an exact match cant be found (they have a 1 in newvar). We do the following:

teffects nnmatch (dep_variable x_covariate) (treatment) if newvar == 0

Unfortunately, we end up with an error message similar to when we first ran the command, that is, the Stata claims it didnt find exact matches for a certain observation, different than the one at the beginning. Does anybody know a solution for this problem?


Best regards,


Juan Hernández

[William Lisowski]

This is a guess, but maybe it will help you see a solution, or else will prompt someone else to explain why I'm foolish and in the process provide a more plausible solution. The warning is that all I know about teffects nmatch is what I have just read in the documentation.

You are doing nearest neighbor matching, apparently requiring the default of one match per observation. (I say "apparently" because you do not tell us the exact command you used.) However, for purposes of this explanation, I am going to pretend you are requiring two matches per observation.

Suppose we have two groups A and B with observations A1, A2, A3, ... and B1, B2, B3, ... .

Suppose A1 matches to B1 and B2, B1 matches to A1 and A2, and B2 only matches to A1. Then it seems possible to me that initially B2 will be identified as having deficient matches, and when you eliminate B2 in the second step, then A1 will now have deficient matches. And if you now eliminate A1, then in the third step B1 will now have deficient matches.

I do not see a solution for this problem, other than repeatedly building up the list of exclusions until no more observations are excluded, or loosening the matching criteria, which you do not divulge to us.

With that said, let me add the following. Section 12.1 of the Statalist FAQ linked to from the top of the page is pertinent to the presentation of your problem.

12.1 What to say about your commands and your problem

Say exactly what you typed and exactly what Stata typed (or did) in response. N.B. exactly!

I understand you wanted to simplify what you saw as unnecessary detail, but the problem is, what the questioner sees as unnecessary detail may be what the respondent sees as a potentially important condition.

[Juan Hernandez]

Thank you for your reply William,

Sorry for not showing the exact code I am using I am desperately looking for a solution. I checked the help file and it says that the default option is to use only 1 match per observation.
If it helps at this point, this is the exact code I am running (I use loops because I have differente outcome and treatment variables) and xdisc and xcont are globals which contain discrete (dummy or categorical) and continous variables as well

foreach var in 1 2 3 4 5 7 8{ 
foreach var2 of global outcome{
teffects nnmatch (`var2') (D_Cr`var'), atet ematch($xdisc) osample(SRT)
teffects nnmatch (`var2') (D_Cr`var') if SRT == 0, atet ematch($xdisc) 
drop SRT
}
}

Best regards,


Juan

[William Lisowski]

I think that looking at your data is your next step to further insight. Perhaps the brief documentation of the osample option has led to a misunderstanding of how it works. if I understand the documentation correctly, your choice of the atet option and the default vce(iid) option (at least, I think that's what the default option is) requires matches only for the treatment group; perhaps SRT is not set to 0 for the control group as you assume it is. (That is, perhaps you should be using SRT!=1 rather than SRT==0.)

Suppose your code fails when `var2' is Y and `var' is 1, and suppose the variable ID identifies your observations. I suggest you run the following.

global xdisc (whatever you currently have xdisc set to)
local var 1
local var2 Y
teffects nnmatch (`var2') (D_Cr`var'), atet ematch($xdisc) osample(SRT)
tab D_Cr`var' SRT, missing
teffects nnmatch (`var2') (D_Cr`var') if SRT == 0, atet ematch($xdisc) osample(SRT2)
tab D_Cr`var' SRT2, missing
tab SRT SRT2, missing

Perhaps examining these results will lead to further insight. Also, looking at examples where SRT==0 & SRT2!=0 may inform you further.

Again, if you report back with the result, please follow the guidance in the FAQ and copy and paste the commands and results from the Results window into a CODE block so we can see everything Stata tells us. The more you help us understand what is happening, without abridging the results, the more chance there is we can help. Showing us a loop is not as helpful as taking the time to create a simple example, like the one above, where everything is obvious and transparent to the reader.