Can a marginal effect be -1.37?

[Meng Yu]

After running a xtlogit model with an interaction term between a score (continuous variable) and respondents' area of residence (categorical), I calculated the marginal effects of the score on their mental health (dependent variable) by their area of residence. I then got the result of -1.37 for one area of residence. My understanding is marginal effect is probability and bounded by -1 and 1. So is this a mistake?
Thank you.

[Joseph Coveney]

I'm guessing that you did something like the following.

.ÿ
.ÿversionÿ17.0

.ÿ
.ÿclearÿ*

.ÿ
.ÿsetÿseedÿ`=strreverse("1624391")'

.ÿ
.ÿquietlyÿsetÿobsÿ250

.ÿgenerateÿintÿpidÿ=ÿ_n

.ÿgenerateÿdoubleÿpid_uÿ=ÿrnormal()

.ÿgenerateÿbyteÿlocÿ=ÿruniformint(1,ÿ3)

.ÿ
.ÿquietlyÿexpandÿ10

.ÿgenerateÿdoubleÿscoÿ=ÿruniform(-0.5,ÿ0.5)

.ÿ
.ÿgenerateÿdoubleÿxbuÿ=ÿ0ÿ+ÿ0ÿ*ÿlocÿ+ÿ0ÿ*ÿscoÿ+ÿpid_u

.ÿgenerateÿbyteÿmehÿ=ÿrbinomial(1,ÿinvlogit(xbu))

.ÿ
.ÿ*
.ÿ*ÿBeginÿhere
.ÿ*
.ÿxtlogitÿmehÿi.loc##c.sco,ÿi(pid)ÿreÿnolog

Random-effectsÿlogisticÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿÿ=ÿÿ2,500
Groupÿvariable:ÿpidÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿgroupsÿ=ÿÿÿÿ250

Randomÿeffectsÿu_iÿ~ÿGaussianÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿObsÿperÿgroup:
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿminÿ=ÿÿÿÿÿ10
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿavgÿ=ÿÿÿ10.0
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmaxÿ=ÿÿÿÿÿ10

Integrationÿmethod:ÿmvaghermiteÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿIntegrationÿpts.ÿ=ÿÿÿÿÿ12

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿWaldÿchi2(5)ÿÿÿÿÿ=ÿÿÿ3.37
Logÿlikelihoodÿ=ÿ-1625.2036ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿÿÿÿ=ÿ0.6436

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿmehÿ|ÿCoefficientÿÿStd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿlocÿ|
ÿÿÿÿÿÿÿÿÿÿ2ÿÿ|ÿÿ-.1871807ÿÿÿÿ.200976ÿÿÿÿ-0.93ÿÿÿ0.352ÿÿÿÿ-.5810863ÿÿÿÿÿ.206725
ÿÿÿÿÿÿÿÿÿÿ3ÿÿ|ÿÿ-.0324857ÿÿÿ.2024435ÿÿÿÿ-0.16ÿÿÿ0.873ÿÿÿÿ-.4292676ÿÿÿÿ.3642962
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿscoÿ|ÿÿÿ.3543603ÿÿÿ.2967663ÿÿÿÿÿ1.19ÿÿÿ0.232ÿÿÿÿÿ-.227291ÿÿÿÿ.9360115
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿloc#c.scoÿ|
ÿÿÿÿÿÿÿÿÿÿ2ÿÿ|ÿÿ-.5418914ÿÿÿ.3988608ÿÿÿÿ-1.36ÿÿÿ0.174ÿÿÿÿ-1.323644ÿÿÿÿ.2398613
ÿÿÿÿÿÿÿÿÿÿ3ÿÿ|ÿÿ-.1853702ÿÿÿ.4014892ÿÿÿÿ-0.46ÿÿÿ0.644ÿÿÿÿ-.9722746ÿÿÿÿ.6015341
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿ_consÿ|ÿÿÿÿ.099914ÿÿÿÿ.149571ÿÿÿÿÿ0.67ÿÿÿ0.504ÿÿÿÿ-.1932398ÿÿÿÿ.3930678
-------------+----------------------------------------------------------------
ÿÿÿÿ/lnsig2uÿ|ÿÿÿ.1077718ÿÿÿ.1535406ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-.1931623ÿÿÿÿ.4087058
-------------+----------------------------------------------------------------
ÿÿÿÿÿsigma_uÿ|ÿÿÿ1.055364ÿÿÿ.0810206ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ.9079362ÿÿÿÿ1.226731
ÿÿÿÿÿÿÿÿÿrhoÿ|ÿÿÿ.2529244ÿÿÿ.0290121ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ.2003658ÿÿÿÿ.3138585
------------------------------------------------------------------------------
LRÿtestÿofÿrho=0:ÿchibar2(01)ÿ=ÿ210.96ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>=ÿchibar2ÿ=ÿ0.000

.ÿ
.ÿmarginsÿloc,ÿdydx(sco)ÿ//ÿ<=ÿthis?

AverageÿmarginalÿeffectsÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿ2,500
ModelÿVCE:ÿOIM

Expression:ÿPr(meh=1),ÿpredict(pr)
dy/dxÿwrt:ÿÿsco

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿÿÿÿDelta-method
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿdy/dxÿÿÿstd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
scoÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿlocÿ|
ÿÿÿÿÿÿÿÿÿÿ1ÿÿ|ÿÿÿ.0717823ÿÿÿ.0599302ÿÿÿÿÿ1.20ÿÿÿ0.231ÿÿÿÿ-.0456788ÿÿÿÿ.1892434
ÿÿÿÿÿÿÿÿÿÿ2ÿÿ|ÿÿ-.0380424ÿÿÿ.0539689ÿÿÿÿ-0.70ÿÿÿ0.481ÿÿÿÿ-.1438195ÿÿÿÿ.0677347
ÿÿÿÿÿÿÿÿÿÿ3ÿÿ|ÿÿÿ.0342992ÿÿÿ.0548615ÿÿÿÿÿ0.63ÿÿÿ0.532ÿÿÿÿ-.0732274ÿÿÿÿ.1418258
------------------------------------------------------------------------------

.ÿ
.ÿexit

endÿofÿdo-file


.

If so, then it is not probability, but rather rate of change in probability.

[Joseph Coveney]

Left off in midthought, sorry. The scale of the rate of change shown reflects the scale chosen for the scores. If you multiply or divide your score values by a constant, say, 10, then you will change the magnitude of what's shown by -margins-.

[Meng Yu]

I used the command

margins, dydx (score) at (area=(0 1 2 3))

. I don’t understand what you meant by rate of change in probabilities. How do I interpret the result then? Thank you.

[Richard Williams]

What was the preceding xtlogit command and what version of Stata are you using? Showing your commands and output would help. When you just give a brief summary of the results, we have to make guesses (as Joe did) about what you did, and those guesses may or may not be right. (Incidentally, I am impressed that Joseph can simulate data for an xtlogit command!)

Besides what Joe mentions, the default options for xtlogit post-estimation commands have changed over time. Back in Stata 13, the default margins option was xb. Now it is pr. If you are using Stata 13, negative XB values can easily occur.

[Meng Yu]

I used Stata 16. The preceding xtlogit command is something like this

xtlogit mental_health c.official_language##i.province i.sex i.age...

I used bootstrap weights and longitudinal weight in the command too.

Here are the results:

Delta-method
dy/dx Std. Err. z P>z [95% Conf. Interval]

English
_at
1 0.3621627 0.1504266 2.41 0.016 0.067332 0.6569934
2 0.1792645 1.218689 0.15 0.883 -2.209321 2.56785
3 -1.360954 0.2609176 -5.22 0 -1.872343 -0.8495646
4 0.9206823 0.2661714 3.46 0.001 0.398996 1.442369
5 -0.3010329 0.2247107 -1.34 0.18 -0.7414578 0.139392

Can I interpret the result for province 3 as "a one unit change in the score was associated with a rate of change of 1.36 in the probability of not having mental health problems?

[Richard Williams]

Using code tags for the output would help greatly too. If you don't use code tags, every space after the first gets deleted, and columns do not line up correctly. Also just show the entire margins output, so we can see what the ats correspond to. You are showing things in bits and pieces and that makes things harder to follow.

In any event, marginal effects are not probabilities. In the case of a continuous variable, the marginal effect is the instantaneous rate of change, and change can be either positive or negative, i.e. increases in X can either increase or decrease the likelihood that Y = 1. For more, see

https://www3.nd.edu/~rwilliam/xsoc73994/Margins02.pdf

[Meng Yu]

 
 margins, dydx (English) at (province=(0 1 2 3 4))













Average marginal effects
Number
of
obs
=
22760 (rounded)

Model VCE    : Bstrap *
















Expression   : Linear prediction,
predict()







dy/dx w.r.t. : English
















1._at        : province        =
0
















2._at        : province        =
1
















3._at        : province        =
2
















4._at        : province        =
3
















5._at        : province        =
4



















Delta-method






dy/dx
Std. Err.
z
P>z
[95% Conf.
Interval]










English









_at








1
0.362163
0.150427
2.41
0.016
0.067332
0.656993


2
0.179265
1.218689
0.15
0.883
-2.20932
2.56785


3
-1.36095
0.260918
-5.22
0
-1.87234
-0.84956


4
0.920682
0.266171
3.46
0.001
0.398996
1.442369


5
-0.30103
0.224711
-1.34
0.18
-0.74146
0.139392

Thank you for the handout. I haven't got a chance to read it but will. I used to think the marginal effect of a categorical variable is probability and interpreted my result as "certain percentage points of likelihood."

[Richard Williams]

These aren't the results or margins command you presented earlier. Note too that it says Expression : Linear prediction, predict() , not pr. Again, just showing everything all at once instead of in pieces (and inconsistent pieces) would make it a lot easier to help you. See the Statalist FAQ for tips on asking questions effectively.

In any event, I think the main thing is for you to be clear on what marginal effects are, and how they are different for categorical and continuous vars. Probabilities shouldn't go negative, but marginal effects (for continuous and categorical variables) certainly can.

Also, the help for bootstrap has sections on using bootstrap with weights and complex data structures, like panel data. It doesn't seem super-hard to do it correctly but it does require a little work.

[Meng Yu]

 
margins, dydx (English) at (province=(0 1 2 3 4))














Average marginal effects
Number
of
obs
=
22760 (rounded)

Model VCE    : Bstrap *
















Expression   : Linear prediction,
predict()







dy/dx w.r.t. : English
















1._at        : province        =
0
















2._at        : province        =
1
















3._at        : province        =
2
















4._at        : province        =
3
















5._at        : province        =
4



















Delta-method







dy/dx
Std. Err.
z
P>z
[95% Conf.
Interval]










English









_at








1
0.3621627
0.1504266
2.41
0.016
0.067332
0.6569934


2
0.1792645
1.218689
0.15
0.883
-2.209321
2.56785


3
-1.360954
0.2609176
-5.22
0
-1.872343
-0.8495646


4
0.9206823
0.2661714
3.46
0.001
0.398996
1.442369


5
-0.3010329
0.2247107
-1.34
0.18
-0.7414578
0.139392

I copied from Excel and it rounded number according to cell size. All three results I have posted are the same results. These results were released from a data centre. I do not have the results directly from Stata.

[Meng Yu]

 
. margins, dydx (religious) at (sex=(0 1))







Warning: cannot perform check for estimable functions.














Average marginal effects                        Number
of
obs
=
21,156




Model VCE    : Bstrap *

















Expression   : Linear prediction, predict()







dy/dx w.r.t. : 1.religious
















1._at        : sex             =           0
















2._at        : sex             =           1


















Delta-method






dy/dx
Std. Err.
z
P>z
[95% Conf.
Interval]











1.religious








_at








1
 0.0244205
0.086667
0.28
0.778
-0.14544
0.194284


2
0.1224152
0.081307
1.51
0.132
-0.03694
0.281773

This is the result from another study of mine when two categorical variables interacted in a xtlogit model. I interpreted it as for women, being religious was 12 percentage points more likely to have mental health problems than not being religious (the result was not statistically significant).

[Richard Williams]

Ah. Yes, I've been tripped up by Excel changing spaces to zero too.

If your variables are both categorical, I wonder if you would be happier with

margins sex#religious

Then you'd see the actual predicted probabilities for each of the 4 sex/religious combinations.

[Meng Yu]

Thank you. I can give it a try, but I wonder what would the reference group be for each combination? Also, what does my command produce if they are not probabilities?

When rate of change in probability is negative, does it mean the relationship between X and Y is negative?

[Richard Williams]

You might want to do some readings on adjusted predictions and marginal effects. There are several handouts at

https://www3.nd.edu/~rwilliam/stats3/index.html

[Meng Yu]

I have read your 2011 slides a few times. https://www3.nd.edu/~rwilliam/stats3/Margins01.pdf
On page 40

But, when we estimate marginal effects for different ages, we see that the effect of black differs greatly by age. It is less than 1 percentage point for 20 year olds and almost 9 percentage points for those aged 70.

I wonder what the percentage point refers to here.