How to obtain prediction for user-specified values of covariate after nonparametric estimation using npregress?

[John Park]

I have estimated $E(Y|z,x1,x2)$ by typing npregress kernel y z x1 x2, noderivatives

Now I want to estimate $E(Y|z=1,x1,x2)-E(Y|z=0,x1,x2)$ using the fitted values, where x1 and x2 still run over all sample observations but z are fixed at user-specified values. How could I achieve this?

Thanks!

[Clyde Schechter]

I will assume that z is a dichotomous 0/1 variable. (If it isn't, post back.)

But I am unclear what you mean by "x1 and x2 still run over all sample observations." If you mean that you want this at several combinations of x1 and x2 that jointly provide reasonably good coverage of the observed values of x1 and x2, then:

margins r.z, at(x1 = (list_of_interesting_values_of_x1) x2 = (list_of_interesting_values_of_x2))

If, on the other hand, you mean that you want the expectations to be calculated over the observed joint distribution of x1 and x2, then it's just:

margins r.z

[John Park]

I will assume that z is a dichotomous 0/1 variable. (If it isn't, post back.)

But I am unclear what you mean by "x1 and x2 still run over all sample observations." If you mean that you want this at several combinations of x1 and x2 that jointly provide reasonably good coverage of the observed values of x1 and x2, then:

margins r.z, at(x1 = (list_of_interesting_values_of_x1) x2 = (list_of_interesting_values_of_x2))

If, on the other hand, you mean that you want the expectations to be calculated over the observed joint distribution of x1 and x2, then it's just:

margins r.z

Hi Clyde, thanks a lot! This is very helpful.
Sorry for the confusions due to my insufficient description.
You are right, z is a dichotomous 0/1 variable.
By "x1 and x2 still run over all sample observations", I meant that I want to estimate [E(Y|z=1,x1_1,x2_1)-E(Y|z=0,x1_1,x2_1),...,E(Y|z=1,x1_n,x2_n)-E(Y|z=0,x1_n,x2_n)], where x1_i,x2_i comes from the i-th observation of my data {(Y_1,z_1,x1_1,x2_1),...,(Y_n,z_n,x1_n,x2_n)}. You are right that I want "the expectations to be calculated over the observed joint distribution of x1 and x2", which could be done using your second line of code.

One additional question, although "margins r.z" could give me the marginal effect of z, I guess I also need E(Y|z=0,x1,x2) and E(Y|z=1,x1,x2) separately (over the observed joint distribution of x1 and x2). How could I achieve this?

[Clyde Schechter]

I also need E(Y|z=0,x1,x2) and E(Y|z=1,x1,x2) separately (over the observed joint distribution of x1 and x2). How could I achieve this?

margins z

By the way, be careful about terminology. E(Y|whatever) is not a marginal effect. It is a predictive margin. A marginal effect, with respect to some variable, say x, is d/dx (E(Y|whatever)).

[John Park]

margins z

By the way, be careful about terminology. E(Y|whatever) is not a marginal effect. It is a predictive margin. A marginal effect, with respect to some variable, say x, is d/dx (E(Y|whatever)).

Hi Clyde, thank you so much for quick response, which is really helpful! Also thanks for clarifying the terminology. You are right that E(Y|whatever) is not a marginal effect. I meant "margins r.z" gives the marginal effect of the dichotomous variable z. I also agree that a marginal effect with respect to some continuous variable, say x, is d/dx (E(Y|whatever)).

After running the codes, I got an additional question. After typing 'margins z', the program gives me two numbers, which looks like E(Y|z=0,x1,x2) and E(Y|z=1,x1,x2) averaged over (x1,x2):


margins `z'

Predictive margins Number of obs = 1,000

Expression: Mean function, predict()

------------------------------------------------------------------------------
| Margin
-------------+----------------------------------------------------------------
__00000B |
0 | .4837176
1 | .6437066
------------------------------------------------------------------------------
Note: You may compute standard errors using vce(bootstrap) or reps().




After typing 'margins r.z', the program gives me the following single number,which looks like E(Y|z=1,x1,x2) -E (Y|z=0,x1,x2) averaged over (x1,x2):

margins r.`z'

Contrasts of predictive margins

Expression: Mean function, predict()

------------------------------------------------
| df chi2 P>chi2
-------------+----------------------------------
__00000B | (not testable)
------------------------------------------------

--------------------------------------------------------------
| Contrast
-------------+------------------------------------------------
__00000B |
(1 vs 0) | .1599889
--------------------------------------------------------------
Note: You may compute standard errors using vce(bootstrap) or reps().



How to get the entire predicted values over the observed distribution of x1 and x2 instead of the mean predicted values?
That is, how could I get a n by 1 vector E1=[E(Y|z=1,x1_1,x2_1),...,E(Y|z=1,x1_n,x2_n)]', a n by 1 vector E0=[E(Y|z=0,x1_1,x2_1),...,E(Y|z=0,x1_n,x2_n)]' and another n by 1 vector E10=[E(Y|z=1,x1_1,x2_1)-E(Y|z=0,x1_1,x2_1),...,E(Y|z=1,x1_n,x2_n)-E(Y|z=0,x1_n,x2_n)].

I need E1, E0 and E10 as variables to do some subeqeunt regressions. For example, in the next stage I need to do regress T, x, E1.

Thanks again!

[Clyde Schechter]

margins z, at(x1 = x1_1 x2 = x2_1) ... at(x1 = x1_n x2 = x2_n)
margins r.z, at(x1 = x1_1 x2 = x2_1) ... at(x1 = x1_n x2 = x2_n)

is the basic code for calculating them. In order to get the results stored, immediately after each of the -margins- commands, save the matrix r(table) in a real matrix. The the first row of that matrix will be the vector you want. Pay attention to how those columns are named so you can refer to them properly for subsequent computations.

[John Park]

margins z, at(x1 = x1_1 x2 = x2_1) ... at(x1 = x1_n x2 = x2_n)
margins r.z, at(x1 = x1_1 x2 = x2_1) ... at(x1 = x1_n x2 = x2_n)

is the basic code for calculating them. In order to get the results stored, immediately after each of the -margins- commands, save the matrix r(table) in a real matrix. The the first row of that matrix will be the vector you want. Pay attention to how those columns are named so you can refer to them properly for subsequent computations.

Thank you very much, Clyde! This is awesome! Really appreciate it.
One last question, is there a simpler or more compact way of writing these two lines of codes?
I have one thousand observations (x1_1,x2_1)...(x1_1000,x2_1000), which means that I will need to type at() one thousand times after "margins z" or "margins r.z".

Thanks again!

[Clyde Schechter]

Where do these 1000 pairs of values for x1 and x2 come from? Are they selected values of interest, or are they the values of x1 and x2 in observations 1 through 1,000 of the data set? If they are the observations in the data set, then you can do something like this:

npregress kernel y i.z x1 x2, noderivatives

gen E1 = .
gen E0 = .
forvalues i = 1/`=_N' {
    local x1i = x1[`i']
    local x2i = x2[`i']
    quietly {
        margins z, at (x1 = `x1i' x2 = `x2i')
        matrix M = r(table)
        replace E0 = M["b", "0.z"] in `i'
        replace E1 = M["b", "1.z"] in `i'
    }
}
gen Ediff = E1 - E0

With 1,000 observations this will take a few minutes to run, and you won't see intermediate output along the way. So be patient and resist the temptation to abort the calculation. If patience is difficult, remove the -quietly- so you will see intermediate output being created. But then you will have all that output cluttering up your log.

If the x1_# x2_# pairs are not the first 1,000 observations in your data set, then it boils down to creating an x1_list and x2_list local macro that contains them, in the correct order. Then instead of looping from 1/`=_N' you would loop from 1/100 with x1i and x2i being set to the i'th word of those local macros. Just how you would create those local macros depends on where these numbers currently are or how you will create them. Extracting the i'th word would be done with -local x1i :word `i' of `x1_list'- (and the analogous code for x2_list).

[John Park]

Where do these 1000 pairs of values for x1 and x2 come from? Are they selected values of interest, or are they the values of x1 and x2 in observations 1 through 1,000 of the data set? If they are the observations in the data set, then you can do something like this:

npregress kernel y i.z x1 x2, noderivatives

gen E1 = .
gen E0 = .
forvalues i = 1/`=_N' {
local x1i = x1[`i']
local x2i = x2[`i']
quietly {
margins z, at (x1 = `x1i' x2 = `x2i')
matrix M = r(table)
replace E0 = M["b", "0.z"] in `i'
replace E1 = M["b", "1.z"] in `i'
}
}
gen Ediff = E1 - E0

With 1,000 observations this will take a few minutes to run, and you won't see intermediate output along the way. So be patient and resist the temptation to abort the calculation. If patience is difficult, remove the -quietly- so you will see intermediate output being created. But then you will have all that output cluttering up your log.

If the x1_# x2_# pairs are not the first 1,000 observations in your data set, then it boils down to creating an x1_list and x2_list local macro that contains them, in the correct order. Then instead of looping from 1/`=_N' you would loop from 1/100 with x1i and x2i being set to the i'th word of those local macros. Just how you would create those local macros depends on where these numbers currently are or how you will create them. Extracting the i'th word would be done with -local x1i :word `i' of `x1_list'- (and the analogous code for x2_list).

Hi Clyde,
Thank you very much! This works and completely solved my problem! Really appreciate all your guidance.