Inconsistent estimation sample levels using eydx

[Gustav Egede Hansen]

Hi everybody

I am encountering a strange problem. I am using - margins, eydx - but it is not working. It provides the error "inconsistent estimation sample levels 0 and 1 of factor treatment". However, if I use - margins, dydx - with exactly the same specifications, then the problem does not arise. Does anyone have knowledge on this issue?

I have also done the same with a different outcome and here eydx works fine.

I cannot provide data as it is confidential.

Best,
Gustav

[William Lisowski]

Here is someone else who reported the same error.

https://www.statalist.org/forums/forum/general-stata-discussion/general/1446490-eprobit-margins-inconsistent-estimation-sample-levels-0-and-1-of-factor

Since you don't tell us what estimation command you are running I cannot tell if this experience is relevant to yours.

[Gustav Egede Hansen]

One model I am running is:

qui reg salary i.time##i.treatment##i.gender
margins treatment, at(time=(99 101) gender =(0))
margins, eydx(treatment) at(time=(99 101) gender =(0)) post
margins, coeflegend
di (exp(_b[1.treatment:2._at])-1)*100

On many outcomes it is running fine, but then when I change to another outcome (e.g. benefits) or gender = 1, then I receive the error "inconsistent estimation sample levels 0 and 1 of factor treatment". Boh salary and benefits have zeros, but I don't think that is the problem?

[Gustav Egede Hansen]

I also encountered this error after running a similar code:
"could not calculate numerial derivatives -- discontinouous region with missing values"

[Gustav Egede Hansen]

Hi again William,

Here is some coding and some example data. In the data, men and women experience the same decline of 5000 but the eydx provides the two different percentage decreases, respectively 12.5% for men and 16.6% for women. It is the relative impact of a treatment that I am interested in.

I think the above error originates from having negative or 0's in my outcome variable. It is not the case in the example dataset but my original dataset.

* men
reg salary i.time##i.treatment##i.gender
margins treatment, at(time=(97 98 99 101 102 103) gender =(0))
marginsplot            

qui xtreg salary i.time##i.treatment##i.gender
margins treatment, at(time=(99 101) gender =(0))
margins, eydx(treatment) at(time=(99 101) gender =(0)) post
margins, coeflegend
di (exp(_b[1.treatment:2._at])-1)*100

* women
reg salary i.time##i.treatment##i.gender
margins treatment, at(time=(97 98 99 101 102 103) gender =(1))
marginsplot    

qui reg salary i.time##i.treatment##i.gender
margins treatment, at(time=(99 101) gender =(1))
margins, eydx(treatment) at(time=(99 101) gender =(1)) post
margins, coeflegend
di (exp(_b[1.treatment:2._at])-1)*100

* Example generated by -dataex-. For more info, type help dataex
clear
input float(id time treatment salary gender)
 1  97 1 30000 1
 1  98 1 30000 1
 1  99 1 30000 1
 1 100 1 30000 1
 1 101 1 25000 1
 1 102 1 25000 1
 1 103 1 25000 1
 2  97 1 30000 1
 2  98 1 30000 1
 2  99 1 30000 1
 2 100 1 30000 1
 2 101 1 25000 1
 2 102 1 25000 1
 2 103 1 25000 1
 5  97 1 40000 0
 5  98 1 40000 0
 5  99 1 40000 0
 5 100 1 40000 0
 5 101 1 35000 0
 5 102 1 35000 0
 5 103 1 35000 0
 6  97 0 30000 1
 6  98 0 30000 1
 6  99 0 30000 1
 6 100 0 30000 1
 6 101 0 30000 1
 6 102 0 30000 1
 6 103 0 30000 1
 7  97 0 30000 1
 7  98 0 30000 1
 7  99 0 30000 1
 7 100 0 30000 1
 7 101 0 30000 1
 7 102 0 30000 1
 7 103 0 30000 1
 8  97 0 40000 0
 8  98 0 40000 0
 8  99 0 40000 0
 8 100 0 40000 0
 8 101 0 40000 0
 8 102 0 40000 0
 8 103 0 40000 0
 9  97 0 40000 0
 9  98 0 40000 0
 9  99 0 40000 0
 9 100 0 40000 0
 9 101 0 40000 0
 9 102 0 40000 0
 9 103 0 40000 0
12  97 1 40000 0
12  98 1 40000 0
12  99 1 40000 0
12 100 1 40000 0
12 101 1 35000 0
12 102 1 35000 0
12 103 1 35000 0
13  97 0 40000 0
13  98 0 40000 0
13  99 0 40000 0
13 100 0 40000 0
13 101 0 40000 0
13 102 0 40000 0
13 103 0 40000 0
14  97 0 40000 0
14  98 0 40000 0
14  99 0 40000 0
14 100 0 40000 0
14 101 0 40000 0
14 102 0 40000 0
14 103 0 40000 0
15  97 0 30000 1
15  98 0 30000 1
15  99 0 30000 1
15 100 0 30000 1
15 101 0 30000 1
15 102 0 30000 1
15 103 0 30000 1
16  97 0 30000 1
16  98 0 30000 1
16  99 0 30000 1
16 100 0 30000 1
16 101 0 30000 1
16 102 0 30000 1
16 103 0 30000 1
17  97 1 40000 0
17  98 1 40000 0
17  99 1 40000 0
17 100 1 40000 0
17 101 1 35000 0
17 102 1 35000 0
17 103 1 35000 0
18  97 1 40000 0
18  98 1 40000 0
18  99 1 40000 0
18 100 1 40000 0
18 101 1 35000 0
18 102 1 35000 0
18 103 1 35000 0
19  97 1 30000 1
19  98 1 30000 1
end

[William Lisowski]

Regarding post #3, it is possible that because you are running your regression quietly, you are not seeing output from the regression that would explain the error message from margins. You should retry one of the outcomes that failed removing the qui before your reg.

Regarding post #5, I note that you are running xtreg for men but reg for women. And again, I would remove the qui from the regression commands and examine the regression output. I expect you will find that the coefficient estimates are not the same in the two populations, so there is no reason to expect the margins to be the same.

[Gustav Egede Hansen]

I have tried without - qui - in both - reg - and - margins - and there is nothing suspicion to see. (and the use of both xtreg and reg was just a typo)

Regarding your comment that the coefficients might not be the same. The margins are the same at 5000, i.e. they experience the same decrease in salary after treatment. Now, I want to look at the percentage decrease, because their starting points prior to treatment are different. And, it is when running - eydx - that I run into problems. Do you know if eydx are able to handle 0's and negative values? As eydx performs a log transformation, it could here the problem lies. However, when I change one of the values to a 0, eydx seems to handle that perfectly (contrary to a regular log transformation):

sort id time
replace salary = 0 if id == 1 & time == 99
qui reg salary i.time##i.treatment##i.gender
margins treatment, at(time=(99 101) gender =(1))        
margins, eydx(treatment) at(time=(99 101) gender =(1)) post
margins, coeflegend
lincom _b[1.treatment:2._at]-_b[1.treatment:1bn._at]
di (exp(r(estimate))-1)*100

* log salary
gen ln1_salary = ln(salary)
reg ln1_salary i.time##i.treatment##i.gender  
margins treatment, at(time=(99 101) gender =(1)) post
margins,coeflegend
lincom (_b[1bn._at#0bn.treatment]-_b[1bn._at#1.treatment])-(_b[2._at#0bn.treatment]-_b[2._at#1.treatment])
di (exp(r(estimate))-1)*100

So from my own experiment I get that eydx are able to handle 0's.

However, the error relates to the factor variable "inconsistent estimation sample levels 0 and 1 of factor treatment". So the problem might relate to something entirely different.


The prior I listed data was pruned a bit, so here are some new data:

* Example generated by -dataex-. For more info, type help dataex
clear
input float(id time treatment salary gender)
 1  97 1 30000 1
 1  98 1 30000 1
 1  99 1     0 1
 1 100 1 30000 1
 1 101 1 25000 1
 1 102 1 25000 1
 1 103 1 25000 1
 2  97 1 30000 1
 2  98 1 30000 1
 2  99 1 30000 1
 2 100 1 30000 1
 2 101 1 25000 1
 2 102 1 25000 1
 2 103 1 25000 1
 5  97 1 40000 0
 5  98 1 40000 0
 5  99 1 40000 0
 5 100 1 40000 0
 5 101 1 35000 0
 5 102 1 35000 0
 5 103 1 35000 0
 6  97 0 30000 1
 6  98 0 30000 1
 6  99 0 30000 1
 6 100 0 30000 1
 6 101 0 30000 1
 6 102 0 30000 1
 6 103 0 30000 1
 7  97 0 30000 1
 7  98 0 30000 1
 7  99 0 30000 1
 7 100 0 30000 1
 7 101 0 30000 1
 7 102 0 30000 1
 7 103 0 30000 1
 8  97 0 40000 0
 8  98 0 40000 0
 8  99 0 40000 0
 8 100 0 40000 0
 8 101 0 40000 0
 8 102 0 40000 0
 8 103 0 40000 0
 9  97 0 40000 0
 9  98 0 40000 0
 9  99 0 40000 0
 9 100 0 40000 0
 9 101 0 40000 0
 9 102 0 40000 0
 9 103 0 40000 0
12  97 1 40000 0
12  98 1 40000 0
12  99 1 40000 0
12 100 1 40000 0
12 101 1 35000 0
12 102 1 35000 0
12 103 1 35000 0
13  97 0 40000 0
13  98 0 40000 0
13  99 0 40000 0
13 100 0 40000 0
13 101 0 40000 0
13 102 0 40000 0
13 103 0 40000 0
14  97 0 40000 0
14  98 0 40000 0
14  99 0 40000 0
14 100 0 40000 0
14 101 0 40000 0
14 102 0 40000 0
14 103 0 40000 0
15  97 0 30000 1
15  98 0 30000 1
15  99 0 30000 1
15 100 0 30000 1
15 101 0 30000 1
15 102 0 30000 1
15 103 0 30000 1
16  97 0 30000 1
16  98 0 30000 1
16  99 0 30000 1
16 100 0 30000 1
16 101 0 30000 1
16 102 0 30000 1
16 103 0 30000 1
17  97 1 40000 0
17  98 1 40000 0
17  99 1 40000 0
17 100 1 40000 0
17 101 1 35000 0
17 102 1 35000 0
17 103 1 35000 0
18  97 1 40000 0
18  98 1 40000 0
18  99 1 40000 0
18 100 1 40000 0
18 101 1 35000 0
18 102 1 35000 0
18 103 1 35000 0
19  97 1 30000 1
19  98 1 30000 1
19  99 1 30000 1
19 100 1 30000 1
19 101 1 25000 1
19 102 1 25000 1
19 103 1 25000 1
20  97 1 30000 1
20  98 1 30000 1
20  99 1 30000 1
20 100 1 30000 1
20 101 1 25000 1
20 102 1 25000 1
20 103 1 25000 1
end

[Gustav Egede Hansen]

Hi again,

I think I have "solved" the issue. I followed some advide from this thread (https://www.stata.com/statalist/arch.../msg00919.html), which said that some models might be too complex and requires a lot of information to run. I dropped some control variables and then my models were running. This also explain why it is only some moderations that will not run (e.g. a small age category), or when I used benefits as an outcome (a more rare outcome than salary).